A method and system for analyzing potential topic phrases in text data

Abstract

The invention discloses a method and system for analyzing potential subject phrases of text data. The method includes: collecting a text data set, and performing word segmentation on the text data set to obtain the expression form of words in the text data set; extracting words according to the text data set The effective phrases formed after the words are matched, and the mixed expression form of the words and phrase sets that are not matched into effective phrases are obtained; the word vector model is obtained after the word vector training is performed on the text data set of the mixed expression form; the DR‑Phrase LDA is constructed and Solve various parameters; train DR‑Phrase LDA, and output potential topic phrases of text data according to the training results. The present invention adopts the phrase topic model based on the word vector, and this model can reasonably improve the statistical information of the phrase in the model training with the help of Chinese linguistic laws in the training of the probability topic model, and specifically uses the method of the word vector to measure the relationship between the phrase component words, Quantitatively reflects the semantic relationship of words in the whole text and phrase parts, making the model more accurate.

Description

technical field [0001] The invention relates to the field of text data mining and analysis, in particular to a method and system for analyzing potential topic phrases of text data. Background technique [0002] With the development of information technology, a large amount of electronic texts have been accumulated in various fields, resulting in information overload. In order to help people quickly retrieve, find and effectively use this information, text semantic and structural analysis has become one of the current research hotspots. Among them, the analysis of potential subject information from text data is one of the key technologies for advanced application systems such as information retrieval, recommendation systems, and automatic summarization. Existing common methods use traditional "bag of words"-based probabilistic topic models such as LDA and PLDA for text topic analysis. The topic results analyzed by these methods are presented in the form of keywords, while h...

AI Technical Summary

Problems solved by technology

[0004] Purpose of the invention: In order to overcome the deficiencies of the prior art, the present invention provides a method for analyzing potential topic phrases of text data, which overcomes the readability and consistency of the topic results obtained from the traditional topic model training based on the "bag of words" and poor visualization; and solve the problem that similar methods cannot obtain effective topic phrase results due to the lack of statistical information of phrases; the present invention also provides a system for analyzing potential topic phrases in text data

Images