A Topic Detection Method Based on Document Content and Interrelationships

Abstract

The invention relates to a topic detection method based on document content and mutual relation. The method comprises the steps of preprocessing an obtained document to obtain a co-occurrence matrix and a two-two relation matrix of document features, constructing a target function on this basis, conducting iterative computation on a document representative degree matrix, a document subordination degree matrix, a word representative degree matrix and a word subordination degree matrix, and outputting the word representative degree matrix, wherein each row corresponds to one topic, and by using the word with the largest value in each row as a topic word for describing the topic, the topic word used for describing the topic is obtained. According to the topic detection method based on the document content and mutual relation, while document clustering and word clustering are conducted, joint clustering is more effective than respective clustering, a more comprehensive model is obtained when the relation of the document content and the relation among the documents are simultaneously considered compared with mere consideration of one information thereof, and through the introduction of the subordination degree and the representative degree, the method is not only applicable to clustering problems but also applicable to topic modeling problems.

Description

technical field [0001] The invention belongs to the technical field of digital computing equipment or data processing equipment or data processing methods especially suitable for specific functions, and in particular relates to a subject detection method based on document content and interrelationships. Background technique [0002] In many natural language processing and analysis problems, it is necessary to automatically detect the semantic topics of text content from massive Internet data through topic modeling methods, and at the same time group and classify documents. [0003] The current topic modeling method is represented by LDA or pLDA, which considers the topic model as a hidden variable, and solves the topic model based on hidden Dirichlet distribution. Most existing probability distribution-based topic modeling methods only consider document content. [0004] However, in many practical applications, there are often interrelated relationships between documents, s...

AI Technical Summary

Problems solved by technology

Most existing probability distribution-based topic modeling methods only consider document content

Images