Low-resource language entity extraction method based on bilingual word vectors

Abstract

The invention provides a low-resource language entity extraction method based on bilingual word vectors. Semantic features of languages are considered in a low-resource language entity extraction task, and the problem of unsupervised learning of low-resource language entity extraction is solved. The method comprises the following three stages: step (1), constructing bilingual word vectors based ona comparable corpus; (2) constructing a source language entity extraction model; and (3) constructing a target language entity extraction model. Compared with the prior art, the reinforcement learning and the bilingual word vector are introduced into the low-resource language entity extraction task for the first time, and the problem that the low-resource language lacks an entity extraction annotation corpus is solved. Through the bilingual word vectors, word meaning characteristics of the cross-language text are effectively represented, and the problems that low-resource language semantic information is insufficient and semantic information cannot be directly transferred among languages in the model transferring process are solved. Meanwhile, the reinforcement learning thought is adopted, and unsupervised learning of the low-resource language entity extraction task is achieved.

Description

technical field [0001] The invention relates to the field of information extraction in the fields of artificial intelligence and natural language processing, and in particular to a multilingual-based entity extraction method. Background technique [0002] Entity extraction aims to mine user-focused knowledge from unstructured text in cyberspace, such as names of people, places, institutions, etc., or entities with domain characteristics. For languages ​​such as English, which exist widely in the network and have many researchers, there are already a large number of labeled entity extraction training corpora, and models based on machine learning and deep learning can be used for supervised entity extraction. In the cyberspace, there are also a large number of low-resource languages, such as Chinese and Japanese, which have fewer annotation corpora, and the manual annotation of training data consumes a lot of manpower and time. Therefore, the traditional supervised entity extr...

AI Technical Summary

Problems solved by technology

[0005] The present invention provides a low-resource language entity extraction method based on bilingual word vectors, which considers the semantic features of the language in the low-resource language entity extraction task, and solves the unsupervised learning problem of low-resource language entity extraction

Images