Efficient lifelong relationship extraction method and system based on dynamic regularization

Abstract

The invention provides an efficient lifelong relationship extraction method and system based on dynamic regularization. The method comprises the following steps: receiving a plurality of data sets, employing a neural model to sequentially train training samples in the data sets, and enabling each data set to correspond to one task; establishing memory blocks for the trained data set to store memory data, accessing the memory data of all the memory blocks during training of the new data set, and defining a memory data loss function, a feature loss function and an EWC loss function in order to overcome catastrophic forgetting; establishing a training loss difference model during a continuous training task period, and calculating regularization factors of a feature loss function, a memory data loss function and an EWC loss function respectively; and obtaining an optimal relationship extraction parameter according to the feature loss function and the regularization factor thereof, the memory data loss function and the regularization factor thereof, and the EWC loss function and the regularization factor thereof. According to the method, the accuracy of lifelong relationship extractionis higher, and extra parameters are not introduced.

Description

technical field [0001] The invention belongs to the field of computer natural language processing, and in particular relates to an efficient lifelong relationship extraction method and system based on dynamic regularization. Background technique [0002] Relation extraction aims to identify the relationship facts of paired entities in text, which can be applied to many natural language processing fields, such as knowledge base construction and question answering systems. Compared with traditional methods that focus on manually designed features, today's CNN-based (convolution neural network) or RNN (recurrent neural network) neural network methods have achieved impressive progress in relation extraction, but most neural models use methods that assume a set of predetermined relations, but this assumption does not Always applicable in practical relation extraction scenarios. [0003] Existing research on lifelong learning is dedicated to overcoming the phenomenon of catastrop...

AI Technical Summary

Problems solved by technology

[0003] Existing research on lifelong learning is dedicated to overcoming the phenomenon of catastrophic forgetting, dealing with lifelong learning of neural network models (also known as continuous learning). However, this heuristic method requires storing all previous training data as well as new data to train a completely new model, which is expensive and time-consuming

[0004] Therefore, the goal of lifelong learning in relation extraction is to make the model perform well on a series of tasks, thus avoiding revisiting all previous data at each stage, but most existing methods are designed for a fixed set of relations. designed, they cannot adapt the trained model to the newly added relationship without catastrophically forgetting the previously learned knowledge, the phenomenon of catastrophic forgetting, which refers to the significant performance drop of the model when switching from the old task to the new task

To alleviate the forgetting problem, it has been proposed to use regularization terms to prevent drastic changes in parameter values ​​while still being able to find good solutions for new tasks, or to augment models with episodic memory modules, which have been used in simple image classification datasets. obtained considerable performance gains on , but it turns out that they perform poorly in natural language processing scenarios

In fact, only limited literature discusses lifelong learning on natural language processing tasks such as relation extraction; to fill the gap in this field, Wang, H., 2019 proposed a method to overcome the forgetting problem of relation extraction models , they introduced an explicit alignment model to alleviate the distortion of the sentence embedding space when the model learns new data, and achieved the best performance, but although this method can work effectively, it is very dependent on the use of the alignment model , which introduces more parameters into an already over-parameterized relation extraction model, which leads to an increase in the number of supervisory signals, memory and computational resources required for training

Images