Natural language processing model training method and device

Abstract

The invention discloses a natural language processing model training method and device. The method is applied to a plurality of natural language processing models with different task targets. A plurality of natural language processing models share a part of hidden layers, and the method comprises the following steps: fusing label information of training samples corresponding to all tasks and representing the fused label information as a multi-label matrix, calculating a hit probability according to a task target vector output by the model and the multi-label matrix, and calculating a single task loss value according to the hit probability; calculating a multi-task loss value according to each single-task loss value; and finally, adjusting parameters of each model according to the multi-task loss value. In the method, a plurality of natural language processing models are combined for learning; shallow feature representations of a plurality of tasks are shared by sharing part of hidden layers, gradients are propagated back at the same time to help the hidden layers to escape from a local minimum value, fusion of a plurality of labels and vector representations of the labels help increase the expression generalization ability and the representation ability of the labels, and then the accuracy and the generalization of each model can be improved.

Description

technical field [0001] The present application relates to the technical field of natural language processing, in particular to a training method and device for a natural language processing model. Background technique [0002] Natural language processing is an important direction in the field of computer science and artificial intelligence. It studies how to make computers understand the meaning of natural language texts and express given intentions and thoughts through natural language texts. The former is called Natural language understanding, which is called natural language generation. [0003] Whether it is natural language understanding or natural language generation, there are many tasks, which can be roughly divided into lexical analysis, sentence analysis, semantic analysis, information extraction, high-level tasks, etc. according to the task type. Among them, since all natural languages ​​have lexical and syntactic features, there is no need to limit the field of ...

AI Technical Summary

Problems solved by technology

However, the data sets of different task models are very different, so the task model can only achieve better results on this data set, and for data other than this data set, the task model needs to suffer a certain generalization loss

In addition, because the training only focuses on the aforementioned single task goal, and there is inherent implicit commonality between different texts, it will cause a certain amount of information loss

Images