Microblog-oriented dynamic topic detection and evolution tracking method

Abstract

The invention provides a microblog-oriented dynamic topic detecting and evolution tracking method and belongs to the technical field of intelligent information processing. The method includes the steps of 1, establishing a distributed crawler to acquire microblog data; 2, pre-processing the microblog data; 3, performing Chinese word segmentation to remove stop words, and acquiring a word set VOC; 4, subjecting the microblog data to LDA (latent Dirichlet allocation) clustering in each time interval so as to extract latent topics; 5, screening out microblog hot topics in each time interval; 6, subjecting the hot topics of a global time to hierarchical clustering to acquire inter-topic aggregation and differentiation relations; 7, visualizing a topic evolution process according to the inter-topic aggregation and differentiation relations. The method has the advantages such that topic word distribution of an event in different times and a fine-grained topic of a same topic in different times are mined under low time complexity, efficiency is high, and robustness is high; the method has greater practical value.

Description

technical field [0001] The invention belongs to the technical field of intelligent information processing, and in particular relates to a microblog-oriented dynamic topic detection and evolution tracking method. Background technique [0002] With the explosive growth of text information on the Internet, it is becoming more and more difficult for people to obtain interesting topics or event information in a timely manner from massive text information. Topic detecting and tracking (TDT) technology aims to organize language and text information flow according to events, and develop a series of core technologies that can meet the needs of the above users. Topic tracking is one of the subtasks of TDT. Topic tracking can help people effectively gather and organize scattered information, and understand all the information about a topic as a whole. As a new research direction of TDT, the topic evolution analysis aims to discover the relationship between various events in the topic ...

AI Technical Summary

Problems solved by technology

[0008] Aiming at the defect that the existing theme detection system cannot analyze and calculate the evolution trend of the theme, the present invention calculates the similarity relationship between themes in different time periods in real time through hierarchical clustering, thereby analyzing the evolution trend of the theme over time, and drawing the theme Evolution trend graph, specifically proposes a dynamic topic detection and evolution tracking method for Weibo

Images