Construction method and system of incremental-translation-oriented structured language model

Abstract

The invention discloses a construction method and a construction system of an incremental-translation-oriented structured language model. The method comprises the following steps: step 1, performing dependency grammar analysis on incrementally generated translation segments to obtain dependency tree segment assembly; step 2, extracting a discriminant feature instance on the dependency tree segment assembly, and calculating a feature score of the discriminant feature instance by a discriminant dependency grammar model; step 3, performing pruning on the dependency tree segment assembly according to the feature score, taking a maximal value of the feature score as the score of the structured language model, reserving the segment having the highest score in the structured language model, and acquiring the optimized dependency tree segment assembly; and step 4, splicing the next translation segment onto the dependency tree segment assembly through a shift-specification operation, repeating the step 1, the step 2 and the step 3 until finishing the translation, and generating the complete dependency tree. According to the construction method and the construction system of the incremental-translation-oriented structured language model, the grammar information and the long-distance dependency information can be merged into the language model, the effective optimization algorithm is proposed for dynamic calculation of the structured language model in a decoding process, and the translation quality is improved.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular, the invention relates to a structured language model oriented to an incremental translation model. Background technique [0002] Statistical language model, as a statistical model to calculate the probability of natural language generation, plays a vital role in many problems of natural language processing. In the problem of machine translation, for newly generated translation fragments, we use the language model to calculate its generation probability, and keep the translation with higher probability, so as to achieve the effect of improving the translation quality. The n-gram language model, also known as the n-1 order Markov model, is the most widely used model in statistical language models. It is based on the limited history assumption: the probability of the nth word is only related to the previous n-1 words . This assumption greatly reduces the complexi...

AI Technical Summary

Problems solved by technology

However, it uses a more complex probability model and needs to be calculated in a complete syntax tree

It cannot meet the needs of incrementally generating translations and simultaneously performing language model calculations in machine translation problems

Images