General topic-embedding-model joint-training method

Abstract

The invention discloses a general topic-embedding-model joint-training method. The method comprises the following steps: step 1, preprocessing an input original document corpus to obtain target text;step 2, constructing a vocabulary table for the target text; step 3, initializing network structures, initializing parameter matrices of models, and constructing a negative sampling table; and step 4,carrying out joint modeling on the topic embedding models, and training the models in a manner of multiple iterations, wherein each iteration process is divided into the following three steps: step 1, using an expectation-maximization (EM) algorithm to train a topic model part; step 2, using a stochastic-gradient-descent algorithm to train an embedding model part; and step 3, using a complete-gradient-descent algorithm to train a regularization term part. The invention can provide a general manner for jointly training the topic model and the embedding model, and problems such as the problemsthat existing model combination manners are too dependent on unique models, generality is insufficient, and two models are difficult to improve at the same time are solved.

Description

technical field [0001] The invention relates to the fields of artificial intelligence, neural network and natural language processing, in particular to a joint training method for a general topic embedding model. Background technique [0002] One of the core tasks of natural language processing is to understand the semantics of text. Many downstream tasks such as text classification, part-of-speech tagging and machine translation use it as an initialization step to improve model performance. "Look at its companion and know its meaning" This is a famous saying of J.R. Firth, a famous British linguist. In other words, if two words appear in similar contexts, they are similar. This is the famous distribution hypothesis, which drives the development of many text models. [0003] Topic model and embedding model are the two main types of models. The origin of the two models is different. The topic model is derived from Bayesian statistics. The probabilistic latent semantic analys...

AI Technical Summary

Problems solved by technology

[0005] Purpose of the invention: In view of the problems and deficiencies in the above-mentioned prior art, the purpose of the invention is to provide a joint training method for general topic embedding models, which solves the problem that the existing model combination method relies too much on specific models and lacks versatility, making it difficult to simultaneously improve Two models etc.

Images