Linguistic model training method and system based on distributed neural networks

Abstract

The invention discloses linguistic model training method and system based on distributed neural networks. The method comprises the following steps: splitting a large vocabulary into a plurality of small vocabularies; corresponding each small vocabulary to a neural network linguistic model, each neural network linguistic model having the same number of input dimensions and being subjected to the first training independently; merging output vectors of each neural network linguistic model and performing the second training; obtaining a normalized neural network linguistic model. The system comprises an input module, a first training module, a second training model and an output model. According to the method, a plurality of neural networks are applied to training and learning different vocabularies, in this way, learning ability of the neural networks is fully used, learning and training time of the large vocabularies is greatly reduced; besides, outputs of the large vocabularies are normalized to realize normalization and sharing of the plurality of neural networks, so that NNLM can learn information as much as possible, and the accuracy of relevant application services, such as large-scale voice identification and machine translation, is improved.

Description

Technical field [0001] The invention relates to a language model, and in particular to a language model training method and system based on distributed neural networks. Background technique [0002] Language models play a very important role in natural language processing, especially in large-scale speech recognition and machine translation. The current mainstream language model is a probability-based statistical language model, especially a statistical model based on n-grams. With the rise of neural networks, more and more people are using neural networks to generate statistical language models. [0003] Statistical language models are widely used in various natural language processing problems, such as language recognition, word segmentation, machine translation, part-of-speech tagging, etc. Simply put, a statistical language model is a model used to calculate the probability of a sentence, that is, [0004] p(w 1 ,w 2 ,…,w k ) [0005] It is known that a sentence (...

AI Technical Summary

Problems solved by technology

[0023] The technical problem to be solved by the present invention is to provide a language model training method and system based on a distributed neural network, so that it can simultaneously solve the problem of learning a large vocabulary neural network language model and normalizing between multiple neural networks , so as to overcome the shortcomings of the existing neural network language model learning methods

Images