Idyl E3 Frequently Asked Questions
These FAQs are applicable to Idyl E3 3.0.0.
Introduction and Getting Started
- Where did you get the name Idyl E3?
- What can I do with Idyl E3?
- How can I get started using Idyl E3?
- How do I extract entities from my text using Idyl E3?
- What can I do with the extracted entities?
- How much does Idyl E3 cost?
- Can I get help integrating Idyl E3 into my current systems and processes?
- How do I manage Idyl E3?
- What happens if an extracted entity can belong to more than one entity type?
- What are the types of models?
- How are models made?
- How many entity models can I use with Idyl E3?
- What do I get when I purchase a model?
- What is different in a newer version of a model?
- How do I install a model into my instance of Idyl E3?
- Can I create my own models?
- On how many instances of Idyl E3 can I install an entity model?
- Can I get a discount if I purchase multiple entity models?
- Can I get a discount for licenses that are used in an AWS auto-scaling environment where not all licenses will be utilized at the same time?
Scaling Idyl E3
Introduction and Getting Started
Where did you get the name Idyl E3?
“Idyl”, an archaic form of idyll, is a short text that describes a picturesque scene. “E3” stands for Entity Extraction Engine. Since we are dealing with text a literary term seemed appropriate, and what better term than one that describes something happy and peaceful!
What can I do with Idyl E3?
With Idyl E3 you can extract named-entities from natural language text. This means that given text in a supported language, Idyl E3 can extract things like persons, places, buildings, cities, and more. The types of entities that can be extracted are customizable through entity models that can be downloaded from our website.
With this capability you can build solutions for text and document analysis and learn more about and improve your understanding of large text archives.
How can I get started using Idyl E3?
From the Idyl E3 home page select your deployment environment and follow the provided deployment steps. Idyl E3 is currently available as a download and as a pre-configured image for Amazon Web Services through the AWS Marketplace.
How do I extract entities from my text using Idyl E3?
Text can be submitted to Idyl E3 through a couple of ways:
- By submitting the text to Idyl E3’s API (using one of our client SDKs, a tool such as cURL, or your own implementation).
- By utilizing a plugin that consumes text from an external stream or source, such as a Kafka topic or Kinesis stream.
What can I do with the extracted entities?
Anything you want! When you receive the entities back you can do anything you want with them. Some customers use the entities to drive external business processes, others simply store the entities for future querying and references, and some use the extracted entities to generate statistics on their text. The entity publisher plugins allow you to push the extracted entities to an external source such as a Kinesis Firehose stream or to EntityDB where the entities can be stored, queries, and processed by a rules engine.
How much does Idyl E3 cost?
Idyl E3 is open-source software available under the Apache License, version 2.0. Idyl E3 is available at no cost.
Can I get help integrating Idyl E3 into my current systems and processes?
Yes! We will be glad to help. Please contact us for development and integration support.
How do I manage Idyl E3?
Idyl E3’s configuration is controlled by its properties file. All configuration changes are done through this file. After modifying any of the values you must restart Idyl E3 for the changes to take affect. See the user documentation for details on each of the available properties.
What happens if an extracted entity can belong to more than one entity type?
In some sentences it is possible for an entity to be belong to more than one type of entity. For example, in the sentence “Susan went to visit Virginia”, Virginia could be identified as a place entity or as a person entity. If this happens the entity having the greatest confidence will be returned.
If it is important in your use that you receive both entities back then use separate instances of Idyl E3 where one instance has one entity model (person) and the other instance has the second entity model (places).
What are the types of models?
There are three types of models used by Idyl E3:
- Sentence Models
- Token Models
- Entity Models
Sentence models take input text and identify the individual sentences in the text. The token models take each sentence and identify the tokens (words) in each sentence. Lastly, the entity models take the tokens and determine which (if any) are named entities. These entities are then returned as the output of the entity extraction. Each model is the output of machine learning algorithms that enables Idyl E3 to identify sentences, tokens, and entities from natural language text.
How are the models made?
To build our models we require a large sample of training text in the language for the model being created. We parse and annotate the text to get it in the required format for training. A percentage of the training data is kept separate for purposes of evaluating our created model. When complete, a model is created that can be used with Idyl E3.
How many entity models can I use with Idyl E3?
As a general rule, each entity model requires 4 GB of RAM so we recommend using no more than one entity model per 4 GB of system memory. This leaves enough memory for the operating system itself and Idyl E3.
What do I get when I purchase a pre-made model?
You get a zip file that contains the model itself and a manifest file that describes the model. The manifest file is how Idyl E3 recognizes the type of the model. You also get 12 months of updates to the entity model. Any updated versions of the model that we release in the 12 months following your purchase you will be able to download for free.
What is different in a newer version of a model?
We have two primary goals when releasing new versions of entity models. The first goal is to improve the performance and accuracy of the entity model. The second goal is to continually adapt the model to recognize new words and phrases in use. New words are added to the dictionary each year and the entity models need to be updated. We also try to diversify our training data across multiple sources of text in order for the model to be applied as broadly as possible.
How do I install a model into my instance of Idyl E3?
Copy the model’s .bin file and .manifest file to Idyl E3’s models directory and restart Idyl E3. The model will automatically be found and loaded.
Can I create my own models?
Yes, the tools required to create your own models are included with Idyl E3. Refer to the user documentation for the detailed steps.
On how many instances of Idyl E3 can I install an entity model?
When you purchase an entity model you are given a selection that determines the number of Idyl E3 instances on which you can install the entity model. For custom site license quotes for a quantity not listed on the purchase page please contact us.
Can I get a discount if I purchase multiple entity models?
Yes, please contact us and let us know the entity models you need and the quantity of each and we will provide a discounted custom quote.
Can I get a discount for licenses that are used in an AWS auto-scaling environment where not all licenses will be utilized at the same time?
Yes, please contact us and us know the entity models you need and your expected average and peak usage of the auto-scaled EC2 instances. For example, let us know if your average number of Idyl E3 instances in-service at one time is 5 and your auto-scaling group’s maximum instance count is 20.)
Scaling Idyl E3
How can I scale Idyl E3 for increased performance?
Idyl E3 can be deployed behind a load balancer (such as an Elastic Load Balancer when on AWS or an Apache reverse proxy). Entity extraction requests sent to the load balancer will be distributed across the instances of Idyl E3 running behind the load balancer.
AWS auto-scaling can also be used to increase (and decrease) the number of Idyl E3 instances. In addition to the standard EC2 performance metrics, the scaling of Idyl E3 instances can be triggered by a Cloud Watch metric containing entity extraction times or by the size of an SQS queue that contains entity extraction requests.
When configuring AWS auto-scaling or configuring Idyl E3 behind a load balancer please keep in mind the entity model licensing that you have purchased. Be sure your auto-scaling group’s maximum number of instances is less than or equal to the purchased entity model license quantities. License discounts are available for AWS EC2 auto-scaled instances – contact us for details and price quotes.
Refer to the Idyl E3 User’s Guide for instructions on how to cluster the Idyl E3 dashboard sessions.