Machine translation model optimization method based on Transform model

Abstract

The invention discloses a machine translation model optimization method based on a Transform model, and the method comprises the steps: enabling Transform individuals to have different structures and parameters through designing variable length codes and candidate blocks, and providing a plurality of candidate models for the learning of word vectors; then designing a crossover and mutation strategy to enable Transform individuals to perform information exchange, and enabling excellent structures or parameters for processing word vectors to be inherited to the next generation; then designing an environment selection strategy to generate next generation Transform individuals, eliminating models with relatively poor word vector learning effects, and reserving models with relatively excellent word vector learning effects; and then finding a transformer model with an optimal word vector learning effect through continuous iterative evolution search for finally solving a machine translation task so that the transformer model can better learn word vector expression in the machine translation task, and the precision of the machine translation task is improved.

Description

technical field [0001] The invention relates to the field of evolutionary computing, in particular to a method for optimizing a machine translation model based on a Transformer model. Background technique [0002] Transformer is a sequence-to-sequence proposed by Google in 2017 to solve machine translation tasks. Before Transformer was proposed, machine translation models can be divided into two categories: models based on feedback neural networks or convolution-based sequence regression Model. Most of the models based on the feedback neural network are composed of RNN or LSTM structure. The input of each layer in the model depends on the output state of the previous layer. The process requires a lot of time and computing resources; the convolution-based network model is composed of a multi-layer convolutional neural network, and the number of convolution operations in this model will increase rapidly when calculating the relationship of long-distance information, such as i...

AI Technical Summary

Problems solved by technology

Although the Transformer model has achieved good results in machine translation, there are still several problems: 1. The arrangement mode of the MHA layer and the FFN layer in the Transformer model with different network layers is fixed, and existing studies have shown that Transformer's different layer arrangement modes have better performance than the base Transformer model on other natural language processing tasks

3. The number of layers and hyperparameters of the Transformer model are set by experts combined with domain knowledge. If non-professionals want to use the Transformer model to solve machine translation tasks, it is difficult to independently design a model that meets expectations

Images