Translation method based on multi-modal machine translation model

Abstract

The invention provides a translation method based on a multi-modal machine translation model, which comprises the following steps: obtaining a source end sentence and a corresponding translation image, and preprocessing the source end sentence and the translation image to obtain the processed source end sentence, the global feature of the translation image and the local feature of the translationimage; establishing a multi-modal machine translation model, and training the multi-modal machine translation model according to the multi-modal machine translation model, the multi-modal machine translation model comprising an encoder and a decoder, and the decoder comprising a context-guided capsule network; translating the processed to-be-translated source end sentence and the corresponding translation image based on a trained multi-modal machine translation model to generate a target end sentence corresponding to the to-be-translated source end sentence. Therefore, the context is introduced into the decoder of the multi-modal machine translation model to guide the capsule network to translate, and introduction of a large number of parameters can be avoided while rich multi-modal representation is dynamically generated, so that the performance of multi-modal machine translation is effectively improved.

Description

technical field [0001] The present invention relates to the technical field of machine translation, in particular to a translation method based on a multimodal machine translation model, a computer-readable storage medium and a computer device. Background technique [0002] In related technologies, the existing multimodal machine translation methods usually use image features to be translated as global information, and use attention mechanisms to dynamically extract image context features to learn multimodal joint representations, but image features are used as global information And the method of learning multi-modal joint representation cannot capture the characteristics of dynamic generation in the translation process; and the single-attention mechanism cannot handle complex multi-modal translation tasks, and the multi-attention mechanism faces the problem of too many parameters. As a result, overfitting problems arise, which in turn greatly reduces the translation perfor...

AI Technical Summary

Problems solved by technology

[0002] In related technologies, the existing multimodal machine translation methods usually use image features to be translated as global information, and use attention mechanisms to dynamically extract image context features to learn multimodal joint representations, but image features are used as global information And the method of learning multi-modal joint representation cannot capture the characteristics of dynamic generation in the translation process; and the single-attention mechanism cannot handle complex multi-modal translation tasks, and the multi-attention mechanism faces the problem of too many parameters. As a result, overfitting problems occur, which in turn greatly reduces the translation performance of multimodal machines

Images