Korean named entities recognition method based on maximum entropy model and neural network model

Abstract

The invention belongs to the technical field of named entities recognition, and discloses a Korean named entities recognition method based on a maximum entropy model and a neural network model. The method comprises the steps that a prefix tree dictionary is built, when any one combined noun template or any one proper noun template is matched in an input sentence, the combined noun template or the proper noun template are recognized into a target word; the target word is obtained in a target word selection module, the target word is searched in an entity dictionary, and when only one subclass is matched, the subclass serves as a tag of the target word; the maximum entropy model is adopted, and various linguistics information is utilized; a feedforward neural network model is constructed; adjacency words form an entity tag through a template selection rule. All data used in the method is extracted in a training corpus with tags and a field-independent entity dictionary, the data is very easily migrated to other application fields, and the performance cannot be reduced obviously.

Description

technical field [0001] The invention belongs to the technical field of named entity recognition, in particular to a Korean named entity recognition method based on maximum entropy and a neural network model. Background technique [0002] Named Entity Recognition (NER) is a basic task in the field of natural language processing. The named entity of its research subject generally includes 3 major categories (entity category, time category and number category) and 7 subcategories (person name, place name, organization name, time, date, currency and percentage). Time and digital entities can be identified through finite state machines, which is relatively simple. However, entity classes such as person names, place names, and organization names are open, and new named entities are constantly generated, and there are many ambiguities, which are difficult to solve by using the location method. Accurate labeling of named entity types often involves analysis at the semantic level, ...

AI Technical Summary

Problems solved by technology

[0004] To sum up, the problems existing in the existing technology are: the current named entity recognition process is cumbersome, expensive and poor in portability, the model calculation process is complicated, the generalization ability is poor, and it cannot handle unregistered words and other problems.

Images