Language modeling system structure searching method for translation tasks

Abstract

The invention discloses a language modeling system structure search method for translation tasks, and the method comprises the following steps: obtaining and processing training data through the Internet, and modeling and training a network structure representation space; carrying out normalization operation on structure parameter values of meta-structure topology and operation in the training process; optimizing structure parameters and model parameters of the used model, and adjusting and optimizing a network structure and target parameters; further obtaining a discretized final structure according to the weight difference of different topologies and operations obtained after tuning, wherein the search result comprises the topological structure of the meta-structure and the operation used between the nodes; and circularly unfolding the searched meta-structures by using a connection mode between the meta-structures to obtain an integral model, performing parameter tuning on the model again by using training data, and finally training until convergence. The method greatly improves the possibility that the optimal solution of the model structure falls into the representation space of the search structure, thereby improving the effectiveness of the network structure search method.

Description

technical field [0001] The invention relates to a language modeling technology, in particular to a language modeling system structure search method for translation tasks. Background technique [0002] As with many deep learning-based systems, one of the core issues of neural network-based natural language processing tasks is to design the structure of the neural network. Especially for the difficult natural language processing task of translation, the network structure of neural machine translation is often very complex, and the design of the network structure requires a lot of skills and engineering experience. Although researchers continue to propose new network structures to improve model performance, there is still no complete solution for how to explore the network structure more scientifically. In the traditional method, it is necessary to find a network structure with better performance by constantly trying new networks. There are two problems in this method: one is ...

AI Technical Summary

Problems solved by technology

[0006] In view of the inconsistency of training and decoding in the structure search task oriented to natural language processing tasks in the prior art, which leads to the problem that the performance of the decoded structure is not optimal, the technical problem to be solved by the present invention is to provide a translation task Language modeling system structure search method, to increase the consistency of the search process and decoding

Images