Data-enhanced machine translation method based on similar word and synonym replacement

Abstract

The invention belongs to the technical field of processing or transformation of natural languages, and discloses a data-enhanced machine translation method based on similar word and synonym replacement. The characteristics that word vectors are finally clustered well are utilized to obtain a similar word table and a synonym table with high quality; the similar word table and the synonym table areconstructed using the word vectors obtained in the training process of a large language, and similar words and synonyms in a scarce small language are replaced; a parallel corpus of the small languageis expanded, and a neural network machine translation model of the small language is trained by the adoption of an encoding-decoding structure and a neural network of an attention mechanism. Trainingdata is expanded, parameters of a neural network translation model can be well studied in enough data, and the problem of unregistered words in the neural machine translation can be alleviated, so that the translation quality of the translation model is improved. When the translation quality of the entire network on a development set is no longer significantly improved, the network parameters have been well studied.

Description

technical field [0001] The invention belongs to the technical field of natural language processing or conversion, and in particular relates to a data-enhanced machine translation method based on similar words and synonyms replacement. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: With the improvement of computer computing power and the application of big data, deep learning has been further applied, and Neural Machine Translation based on deep learning has attracted more and more attention. As a research hotspot of artificial intelligence, machine translation has very important scientific research value and practical value. In the NMT field, one of the most commonly used translation models is the encoder-decoder model with an attention-based mechanism. The main idea is to encode the sentence to be translated (collectively referred to as "source sentence" hereinafter) into a vector representation through an...

AI Technical Summary

Problems solved by technology

[0004] (1) Large-scale, high-quality bilingual parallel corpus is difficult to obtain, and the cost of constructing high-quality bilingual parallel corpus by human translation is relatively high

[0005] (2) Lack of large-scale, high-quality bilingual parallel corpus, resulting in insufficient training data and poor performance of the neural network machine translation model in small languages, making it difficult to build a high-performance machine translation system

However, as a small language, both translation talents and translation systems are very scarce.

Images