Recognition method and system of named entities in microblog messages

Abstract

The invention provides a recognition method of named entities in microblog messages. The recognition method includes that a few named entities are specified as seeds; a certain number of microblog messages from the original microblog message set to be processed are automatically marked as a training data set; and then the training data set is utilized to train a named entity identifier and the trained named entity recognizer is utilized to recognize the named entities in the microblog messages. According to the recognition method of the named entities in the microblog messages, only a few existing seed entities need to be specified to enable a high quality training set to be automatically marked; the labor costs are significantly reduced for the microblog messages which are texts capable of being updated rapidly; and an iterative mode is utilized to generate high quality marked data step by step and each time first N newly named entities which can most reflect the appearing law of the named entities in real microblog data are selected to add into a seed bank, so that finally generated marked data can well cover the integral microblog message set.

Description

technical field [0001] The invention relates to network data processing and analysis, in particular to a method for automatically identifying named entities in microblog message texts. Background technique [0002] Microblog is a newly emerging form of information publishing and dissemination on the Internet. Because of its convenience, shortness and speed of publishing information, Weibo has quickly attracted the attention of Internet users. At present, there are hundreds of millions of microblog users in China. On large-scale microblog platforms such as Sina, Tencent, Sohu and Netease, users generate a large number of microblog message texts every day. There are close to 100 million blog messages. On the Weibo platform, every Internet user is a "self-media", that is, users can spread what they see and hear by posting Weibo messages, as well as express their views, needs and interests. These messages are aggregated to form a massive message collection, and such a massive ...

AI Technical Summary

Problems solved by technology

The current research on named entity recognition is usually oriented to the recognition of standardized texts, such as scientific and technological documents, news reports, etc., while Weibo message texts have their own characteristics, such as arbitrary language forms, irregular grammar, and scattered sentence forms. Existing named entity recognition methods cannot accurately analyze the named entities appearing in the text of Weibo messages

Moreover, the existing named entity recognition method (also known as supervised named entity recognition method) needs to manually mark a certain amount of data set as training data to train the named entity recognition model. Manual standard training corpus is time-consuming. Laborious work, and it is difficult to label large-scale training data

In addition, due to the continuous increase of Weibo messages and changes in Weibo content over time, this method of manually labeling the training set is not only time-consuming and laborious, but also cannot reflect the characteristics of Weibo data in a timely and accurate manner, so the recognition efficiency is low.

Currently, there is no method for named entity recognition for Weibo message text

Images