BERT embedded speech translation model training method, BERT embedded speech translation model training system, speech translation method and speech translation equipment

Abstract

The invention belongs to the technical field of speech translation, and relates to a BERT embedded speech translation model training method and system and a speech translation method and device.The training method comprises the following steps: collecting model training data; pre-training a BERT model by utilizing a source language in the training data, taking the pre-trained BERT model as a machine translation model coding layer, training a machine translation model by utilizing paired source language and target language texts, and obtaining a plurality of machine translation models by setting the number of decoding layers in the machine translation model; training a speech recognition model by using the speech translation data of the source language pairs; and taking the trained speech recognition model coding layer as a speech translation model coding layer initialization parameter, weighting the outputs of the plurality of machine translation models in an entropy weighting mode to train a speech translation model, and completing speech translation model training in combination with a model loss function. The recognition performance of the speech translation model is improved, and then the speech translation efficiency and quality are improved.

Description

technical field [0001] The invention belongs to the technical field of speech translation, and in particular relates to a BERT embedded speech translation model training method and system based on entropy weighted knowledge distillation, a speech translation method and equipment. Background technique [0002] Speech translation is an end-to-end model that converts speech in one language into text in another language, that is, skips the step of converting the source language into text in the traditional model, and directly converts the speech in the source language into the target Language text is a hot research topic at present. In terms of model selection, the transformer model proposed by Google has become the main choice due to the effectiveness of its self-attention mechanism modeling, the efficiency of parallel processing, and the simplicity of the model structure, which has significantly improved the recognition rate. But one of the main problems is the amount of data...

AI Technical Summary

Problems solved by technology

For the neural network, even a network with relatively poor effect may produce good results for a certain input, it is obviously impossible to use the uncertainty of the neural network to guide the training with a single teacher model, which will affect the efficiency and quality of speech translation.

Images