Deep Learning Entity Models in Idyl E3 2.5.0

Idyl E3As we mentioned in an earlier post, Idyl E3 2.5.0 will include the ability to create and user deep learning neural network entity models. As we get closer to the release of Idyl E3 2.5.0 we wanted to introduce the new capability and compare it with the current entity models.

In Idyl E3 2.4.0 you can create entity models through a perceptron algorithm. This algorithm requires as input annotated training text and a list of features. Feature selection can be a difficult task. Too many features can result in over-fitting the model such that it performs well on the input text but does not generalize well to other text. Feature selection is a crucial part of producing a useful, quality model.

Idyl E3 2.5.0’s ability to create and use deep learning models still requires annotated input text but does not require a list of features. The features are discovered automatically during the execution of the neural network algorithm through the use of word vectors, produced by applications like Word2vec or GloVe. Using a tool like this, generate a file of vectors from your training text to provide to Idyl E3 during model training. In summary, manual feature selection is not required for deep learning models.

While word vectors really helps with deep learning model training, training a deep learning model can still be a challenging task. A neural network has many hyperparameters that tune the underlying algorithms. Small changes to these hyperparameters can have a dramatic effect on the generated model. Hyperparameter optimization is an active area of academic and industry research. Tools and best practices exist to help with hyperparameter selection and we will provide some useful resources to help in the near future.

Idyl E3 2.5.0 and newer versions will continue to support using and creating maximum entropy based models so you can choose which type of model you want to create and use.


Leave a Reply