Bi-LSTM-based named entity identification method

Abstract

The invention relates to a Bi-LSTM-based named entity identification method. The method comprises the steps that 1, a training corpus for named entity identification is tagged to form a tagged corpus;2, words and characters in the tagged corpus are converted into vectors; 3, a Bi-LSTM-based named entity identification model is built through the vectors of the words and the characters, and parameters of the Bi-LSTM-based named entity identification model are trained; and 4, named entity identification prediction is conducted on to-be-predicted data through the trained named entity identification model. According to the method, by adopting the vectors based on the words and the characters, features of the characters and the words can be obtained simultaneously, and meanwhile the unknown word problem can be avoided; and in addition, compared with a traditional pure CRF model algorithm, by adopting a bidirectional long short-term memory (Bi-LSTM) neural network, the method has the advantage that more character and word features can be absorbed, and therefore the entity identification precision can be improved.

Description

technical field [0001] The invention belongs to the field of information technology, and in particular relates to a Bi-LSTM-based named entity recognition method. Background technique [0002] Named Entity Recognition (NER for short) refers to the recognition of entities with specific meanings in a text, mainly including names of people, places, institutions, and proper nouns. [0003] The practical scenarios of the named entity recognition method include: [0004] Scenario 1: Event detection. Place, time, and person are several basic components of time. When constructing an event summary, relevant persons, places, units, etc. can be highlighted. In the event search system, related people, time, and places can be used as index keywords. The relationship between several components of an event describes the event in more detail at the semantic level. [0005] Scenario 2: Information retrieval. Named entities can be used to enhance and improve the effect of the retrieval s...

AI Technical Summary

Problems solved by technology

[0010] Dictionary-based named entity recognition relies heavily on the dictionary database and cannot identify unregistered words

HMM (Hidden Markov) and CRF (Conditional Random Field) methods based on word frequency statistics can only relate to the semantics of the previous word of the current word, and the recognition accuracy is not high enough, especially the recognition rate of unregistered words is low

The method based on the artificial neural network model has the problem of gradient disappearance during training, and in actual applications, the number of network layers is small, and the final named entity recognition results have no obvious advantages

Images