Yahoo! Vespa and Entity Annotations

Some interesting news this week is that Yahoo! has open-sourced their software that drives many of their content recommendation systems. The software, called Vespa, is available at vespa.ai.

Annotations on words and phrases in the text can be provided as text is ingested into Vespa. This process is described in the Vespa Annotations API documentation. But in order to make these annotations you need something that can identify persons, places, and things in the text! Idyl E3 Entity Extraction Engine is perfect for this and here’s how:

You probably have a pipeline in which text is gathered from some source and eventually pushed to your search application, in this case we’re using Vespa. All that is needed is to modify your pipeline to first send the text to Idyl E3 to get the entities. Once a response is received from Idyl E3 the text along with its annotations can be sent on to Vespa. It really is that easy. You can customize the types of entities to extract through the entity models installed in Idyl E3. So you could annotate persons, places, and things like buildings, schools, and airports.

To recap, in case you have not yet read about Vespa it is worth a few minutes to read about. Its ability to ingest text with annotations makes a natural fit for Idyl E3. You can certainly use Idyl E3 to annotate text for Vespa now and we’re going to make some improvements to make working with Vespa even easier.