Neural machine translation decoding acceleration method based on non-autoregression

Abstract

The invention discloses a neural machine translation decoding acceleration method based on non-autoregression. The neural machine translation decoding acceleration method comprises the steps of: constructing an autoregression neural machine translation model through employing a Transformer model based on an autoattention mechanism; constructing training parallel corpora, generating a machine translation word list, and training the two models from left to right and from right to left until convergence; constructing a non-autoregression machine translation model; acquiring codec attention and hidden layer states of a left-to-right autoregression translation model and a right-to-left autoregression translation model; calculating the difference between the output and the output corresponding to the autoregressive model, and taking the difference as additional loss for model training; extracting source language sentence information, and predicting a corresponding target language sentence bya decoder; and calculating the loss of predicted distribution and real data distribution, decoding translation results with different lengths, and further acquiring an optimal translation result. According to the neural machine translation decoding acceleration method, knowledge in the regression model is fully utilized, and 8.6 times of speed increase can be obtained under the condition of low performance loss.

Description

technical field [0001] The invention relates to a neural machine translation inference acceleration method, in particular to a non-autoregressive neural machine translation decoding acceleration method. Background technique [0002] Machine translation is the technique of translating one natural language into another. Machine translation is a branch of natural language processing, one of the ultimate goals of artificial intelligence, and has important scientific research value. At the same time, with the rapid development of Internet technology, machine translation technology has played an increasingly important role in people's daily life and work. [0003] From the rule-based method in the 1970s to the instance-based method in the 1980s, the statistical method in the 1990s, and the neural network-based method today, machine translation technology has achieved good results after years of development. more widely used in daily life. [0004] Although the traditional rule-...

AI Technical Summary

Problems solved by technology

[0010] Aiming at the problems of neural machine translation model error propagation during inference and the decoding speed in actual use is difficult to meet the real-time response requirements, the technical problem to be solved by the present invention is to provide a non-autoregressive neural machine translation decoding acceleration method, which can make Large-scale neural machine translation models can have higher response speed and better practical application, while the translation quality of the model can still be guaranteed

Images