Low-rank decomposition based delicate topic mining method

Abstract

The invention discloses a low-rank decomposition based delicate topic mining method. The delicate topic mining method comprises the following steps: conducting word dividing and stopword removal processing on an original corpus text; generating a topic matrix on the basis of a word frequency matrix obtained through pre-processing; decomposing the original corpus text into topic background and keywords by the topic matrix. According to the delicate topic mining method, a delicate model for expressing text contents without introducing a new implicit variable is brought forward; the model adopts an LDA (Latent Dirichlet Allocation) model as the basis to extract topic distribution of a text collection, and introduces in an improvement method of principal component analysis, namely the robustness principal component analysis method, in combination with the characteristics of text topics constituted by different aspects, in order to decompose each topic into a low-rank part and a rarefaction part; the low-rank part represents common words under the topic, and the rarefaction part is the delicate descriptions in different angles under the topic, so that the purpose of delicately expressing a text is realized, and the problems that the conventional topic model can only mine the topic background of the text, and cannot delicately describe emphasis points of the text are effectively solved.

Description

technical field [0001] The invention belongs to the technical field of text processing and mining, and in particular relates to a fine topic mining method based on low-rank decomposition. Background technique [0002] Mining hidden topics in text collections is one of the important research contents in the field of text mining. Topic models represented by Latent Dirichlet Allocation (LDA) have been widely used in recent years. These models transform the high-dimensional sparse word frequency matrix representation into a low-dimensional semantic space representation, that is, topic space representation, thereby playing a role in dimensionality reduction. This has wide applications in applications such as text modeling, text classification and information extraction. [0003] The reality corpus can be divided into economics, politics, entertainment, health and other topics according to the content. However, in practical applications, each topic needs to be further divided i...

AI Technical Summary

Problems solved by technology

[0005] The present invention provides a fine topic mining method based on low-rank decomposition, which aims to solve the problem that the traditional topic model can only mine the topic background of the text, but cannot describe the emphasis of the text in detail

Images