Chinese named entity identification data enhancement algorithm based on sequence generative adversarial network

Abstract

The invention provides a method for selecting positive sample data in source domain data to extend training data of a target domain by fusing semantic difference and label difference of sentences in the source domain and the target domain, so as to achieve the purpose of enhancing named entity recognition performance of the target domain. Based on a conventional Bi-LSTM + CRF model, in order to fuse semantic differences and label differences of sentences in a source domain and a target domain, semantic difference and label difference are introduced through state representation and reward setting in reinforcement learning; therefore, the trained decision network can select sentences having positive influence on the named entity recognition performance of the target domain in the data of thesource domain, expand the training data of the target domain, solve the problem of insufficient training data of the target domain, and improve the named entity recognition performance of the targetdomain at the same time.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a method of using a sequence generation confrontation network to enhance data and improve the performance of Chinese named entity recognition. Background technique [0002] In recent years, deep learning has made great progress in image, speech and natural language processing. As an emerging technology of machine learning algorithms, deep learning is motivated by the establishment of a neural network that simulates the human brain for analysis and learning. In the field of images, people use deep neural networks to realize target detection in images, such as combining convolutional neural networks with candidate windows to detect pedestrians in images; in the field of speech, deep learning is used for speech synthesis and recognition provide us with an intelligent voice system; in the field of natural language processing, deep learning is applied to various life scenarios, ...

AI Technical Summary

Problems solved by technology

[0015] 1. Although modifying the structure of the deep model can enhance the semantic representation of the text, it does not solve the problem of lacking a large amount of labeled data

[0016] 2. The introduction of external resources requires a lot of manpower and time to collect external resources, and it is necessary to design effective rules to add external resources to the model

Images