Multi-modal machine translation method based on variational reasoning and multi-task learning

Abstract

The invention discloses a multi-modal machine translation method based on variational reasoning and multi-task learning. Before the method has translation capability, multi-task modeling is performedon multi-modal information such as images, texts and the like by using a variational reasoning theory, and then a variational multi-modal machine translation model is obtained by training under the condition of giving a sufficient training set, so that the machine translation capability is obtained. And finally, predicting a plurality of translated texts through bundle search and maximum likelihood. The innovation point of the invention lies in that a model capable of integrating multi-modal information such as images into machine translation, namely variational multi-modal machine translation, is created and used. According to the variational model, a set of feature extraction neural network framework confusing image and text semantics is constructed, a modeling process and a self-learning updating process are derived at the same time, a detailed derivation algorithm is given, and an application method is given in an instructive mode.

Description

technical field [0001] The present invention relates to the field of computer technology and machine translation technology, in particular to a multi-modal machine translation method based on variational reasoning and multi-task learning. Background technique [0002] The background technology involves four major blocks: variational inference and variational encoder-decoder framework, information bottleneck theory, multi-task learning, and neural machine translation. [0003] 1) Variational Inference and Variational Encoder-Decoder (Variational Inference and VariationalEncoder-Decoder) [0004] Variational inference is a common approximate inference technique. Variational encoder-decoder is an important application in the field of machine translation. The variational encoder-decoder is generalized from the variational auto encoder (Variational Auto Encoder), which maps the input data x to different output data y. Different from the general encoder-decoder framework, the v...

AI Technical Summary

Problems solved by technology

[0035] How to balance multiple loss functions of multi-task learning is a problem, and there is no effective method with a complete theoretical basis.

Images