Kalman filter word vector learning method based on Diesel process

Abstract

A Kalman filter word vector learning method based on Diesel process, the method includes: training and preprocessing the corpus, generating an LDS language model system, initializing the system parameters, assuming that the process noise satisfies a normal distribution, defining the aggregation class theta t =(μ t ,∑ t ), μ t For the frequency of word t in the corpus, calculate θ t The prior distribution of Dirichlet, the posterior distribution is calculated by Kalman filter derivation and Gibbs sampling estimation, the candidate clusters are extracted by MCMC sampling algorithm, the selection probability of the candidate clusters is calculated, and the candidate with the highest probability value is selected Choose the cluster as θ t , calculate the estimated value of the minimum mean square error of the clustering, substitute the calculation result into the LDS language model, train the model through the EM algorithm, make the model parameters stable, input the preprocessed corpus into the trained LDS language model, and pass Carl The Mann filter updates the formula in one step to compute the implicit vector representation.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular, to a method for learning word vectors of Kalman filter based on Diesel process. Background technique [0002] In natural language processing (NLP) related tasks, in order to hand over natural language to algorithms in machine learning, it is usually necessary to mathematicize the language first, because machines only recognize mathematical symbols. Vectors are things that people abstract from the natural world and hand them over to machines for processing. Word vectors are a way to mathematicize words in language. [0003] One of the simplest word vector representations is One-hot Representation, which is to use a very long vector to represent a word. The length of the vector is the size of the dictionary. The vector component has only one 1, and the others are all 0 and 1 positions. corresponds to the position of the word in the dictionary. However, this kind of word v...

AI Technical Summary

Problems solved by technology

However, this kind of word vector representation has two disadvantages: (1) it is easily troubled by the curse of dimensionality, especially when it is used in some algorithms of Deep Learning; (2) it cannot describe the relationship between words well. similarity

However, the existing technology has not applied the Diesel process to the word vector representation of natural language processing

Images