Neural machine translation method by introducing source language block information to encode

Abstract

The present invention relates to a neural machine translation method for introducing source language block information to encode. The method comprises: inputting bilingual sentence-level parallel data, and carrying out word segmentation on the source language and the target language respectively to obtain bilingual parallel sentence pairs after being subject to word segmentation; encoding the source sentence in the bilingual parallel sentence pairs after being subject to word segmentation according to the time sequence, obtaining the state of each time sequence on the hidden layer of the lastlayer, and segmenting the input source sentence by blocks; according to the state of each time sequence of the source sentence and the segmentation information of the source sentence, obtaining the block encoding information of the source sentence; combing the time sequence encoding information with the block encoding information to obtain final source sentence memory information; and by dynamically querying the source sentence memory information, using attention mechanism to generate a context vector at each moment through a decoder network, and extracting feature vectors for word prediction.According to the method provided by the present invention, block segmentation is automatically carried out on the source sentence without the need of any pre-divided sentence to participate in the training, and the method can capture the latest and the best block segmentation manner of the source sentence.

Description

technical field [0001] The invention relates to the field of machine translation, in particular to a neural machine translation method that introduces coding of source language chunk information. Background technique [0002] Neural machine translation technology usually uses a neural network-based encoder-decoder framework to model the entire translation process end-to-end, and this method has achieved the best translation performance in many different languages. Among them, the encoder network is responsible for encoding the input source sentence into a fixed-dimensional vector with memory information, and the decoder will generate the corresponding translation result through the encoding vector obtained by the encoder. For the encoder, the input source sentence is usually regarded as a sequence of words appearing in order after word segmentation. When the encoder reads the source sentence, it can construct corresponding memory information for the source sentence. During ...

AI Technical Summary

Problems solved by technology

However, due to the imperfection of these external tools, it is possible to introduce new errors when obtaining block information, and these errors will continue to spread in subsequent work, resulting in error accumulation

Images