Speeding up Philter document redaction with a GPU

March 20, 2021
|In blog
|By mtnfog

In a lot of cases using Philter on a CPU will provide sufficient performance. However, in deployments where performance has higher importance using Philter with GPU can provide a 10x or more improvement in performance.

In this post we will show how Philter running on AWS on a m5.large EC2 instance averaged about 1 second to filter a document. When the same set of documents were filtered using Philter on a p3.2xlarge EC2 instance the average time per document fell to around 0.1 seconds. That’s a significant difference!

Things to Know

As of Philter 1.10.0 the only filter that can use the GPU is the named person’s filter. If you aren’t using this filter then you will not see any performance benefit from running with a GPU. This will likely change in the future.

Philter with a GPU on AWS EC2

The AWS EC2 p3.2xlarge instance type has a single Tesla V100 GPU. You do need to install the NVIDIA CUDA drivers onto the EC2 instance. Contact us or refer to Philter’s User’s Guide for the installation scripts. No Philter configuration changes are needed to use the GPU because when a GPU is present Philter will automatically use it. In summary, the steps are:

  1. Launch an instance of Philter on any EC2 instance type.
  2. Stop the Philter instance.
  3. Change the instance type to p3.2xlarge and start the instance.
  4. Install the NVIDIA CUDA drivers. (Contact us or see Philter’s User’s Guide for the installation scripts.)
  5. Reboot the instance.

Philter is ready to serve API requests!

Monitoring the Performance

You can monitor the performance of Philter using the Prometheus monitor. Once enabled, Philter’s metrics will be available for scraping at http://philter:9100/metrics. You will want to look at the philter_ner_entity_time_ms_seconds_sum and philter_ner_entity_time_ms_seconds_count metrics.

The first metric is the total number milliseconds spent applying the NER (named person’s) filter. The second metric is the total number of times the filter was applied. Dividing those two numbers will give us the average time spent each time applying the filter. The screenshot below shows an example Grafana configuration for those metrics.