Method and device for establishing named entity labeling resource library, storage medium and computer device

Abstract

The invention relates to a method and device for establishing a named entity labeling resource library, a storage medium and a computer device. Calculation is performed by using a resource library of the round of iterations formed by a small number of seed banks and unlabeled texts in an unlabeled text set, seed banks of a next round of iterations are generated by calculating the average utility value of each named entity in the unlabeled texts, then the generated seed banks and other unlabeled texts form a resource library of the next round of iterations, and a next round of seed banks is calculated, so that calculation continues till all the unlabeled texts are calculated, new named entities are found, and the named entity labeling resource library is generated. The method is simple in calculation, obtained results are high in confidence degree, and the method is suitable for large-scale text processing. Text data is non-structured data, effectiveness evaluation on the non-structured data is generally difficult, so that the quantitative evaluation on text named entities can be achieved by adopting the method.

Description

technical field [0001] The invention relates to the technical field of information processing, in particular to a method, device, storage medium and computer equipment for constructing a named entity tagging resource library. Background technique [0002] A named entity refers to a person's name, an organization's name, a place name, and all other entities identified by a name. In a broad sense, a named entity also includes numbers, dates, currencies, addresses, etc. Named Entity Recognition (NER) is one of the basic technologies of natural language processing, which plays an important role in improving the performance of many natural language processing application systems. At present, NER mainly uses statistical models as processing techniques, such as Hidden Markov Model (HMM), Conditional Random Field Model (Conditional Random Field, CRF) and other statistical models. These statistical models require a large number of annotation resources. As a training set, a manually ...

AI Technical Summary

Problems solved by technology

The resources in these manually labeled resource libraries are very limited, not enough to adapt to large-scale application scenarios such as machine translation, and with the development of society, new named entities are constantly being born, such as organization names, movie names, product names , book titles, etc., so the manually labeled resource library is far from meeting the needs of named entity recognition

Images