Tour field named entity recognition method based on condition random field

Abstract

The invention relates to a method for identifying named entities in the field of traveling, belongs to the artificial intelligence field. Based on language material collection, labeling and text pretreatment conducted manually, the invention provides a method for identifying named entities in the field of traveling based on stack condition random field model. The method comprises the following two steps: in the low-grade condition random field, word used as dividing granularity is combined with characteristic dictionaries such as a tourist attraction common word table, a tourist attraction common suffix table, a place name common word table, and the like, so as to realize simple traveling name entity identification through establishing an effective characteristic molding board; the identification result is transmitted to the high-grade model, in the high-grade model, phrase used as dividing granularity is combined with a molding board with a difficult characteristic module, so as to realize the identification of nesting scenic spots, local products and delicacies, and locations. In open tests, compared with a monolayer model, the F value of the stack condition random field model increases by 8 percent; and compared with an HMM (Hidden Markov Model) model, the correct rate of the stack condition random field model increases by 8 percent, the recall rate thereof increases by 22 percent, and the F value thereof increases by 15 percent.

Description

technical field [0001] The invention relates to a named entity recognition method in the field of tourism based on a conditional random field, which belongs to the field of artificial intelligence. Background technique [0002] At present, in the field of named entity recognition, some rule methods and statistical methods are mainly used. In the rule-based method, due to the variety of rules, the cost of summarizing a unified rule that can identify all types is too high, and the rules cannot cover all fields, so it is basically infeasible. The Institute of Computing Technology, Chinese Academy of Sciences proposed a statistical method for character labeling and recognition entities based on Hidden Markov Model. However, since the hidden Markov model is a generative model, it has some inherent defects and deficiencies: First, in order to ensure the correctness of the derivation, strict independence assumptions need to be made. In fact, most sequence data cannot be represent...

AI Technical Summary

Problems solved by technology

However, since the Hidden Markov Model is a generative model, it has some inherent defects and deficiencies: First, in order to ensure the correctness of the derivation, strict independence assumptions need to be made

Annotation bias problem can adversely affect the results of natural language processing

Images