A Time Sensitive and Adaptive Subtopic Online Detection Method and System

Abstract

The invention relates to a time-sensitive and self-adaptive subtopic online detection method and system. The method includes: 1) performing vectorized representation on each document in the document stream; 2) incrementally clustering the documents, and adjusting the central weight of the subtopic according to the weight of the document decaying with time; 3) when clustering When the number of generated subtopics or the weight ratio of a certain subtopic meet the threshold condition, or when the subtopic meets the long tail detection condition, merge subtopics or delete meaningless subtopics; 4) According to the weight of each new subtopic Given its intrinsic document distribution, it generates summaries for new subtopics and outputs representations. The system includes a document representation module, an incremental clustering module, a new subtopic discovery module, and a summary generation module. In the present invention, the weight of historical documents decays with time, and the number and content of subtopics are dynamically updated based on threshold judgment and long tail detection, which can effectively improve the efficiency of subtopic detection.

Description

technical field [0001] The invention belongs to the field of information technology, and specifically relates to a time-sensitive and self-adaptive subtopic online detection method and system, which can be applied to emergency event detection, subtopic analysis, public opinion analysis, social media data mining and other fields. Background technique [0002] Weibo is short for Microblog. By registering a Weibo account, users can follow friends, celebrities, institutions, etc., so that different users can establish network relationships. The news flow of Weibo is full of various things, but different social entities pay attention to completely different content. For example, product companies pay attention to the real-time word-of-mouth of related products on the Internet, and famous people pay attention to their public opinion image among netizens. with impact. Therefore, online subtopic detection for specific target entities based on social networks has attracted great at...

AI Technical Summary

Problems solved by technology

[0009] The above system has the following problems: First, it does not distinguish between the weight of historical documents and the weight of latest documents

Second: Unable to adjust the content and quantity of subtopics adaptively

Third: The system based on burst detection can only get sudden events, but cannot detect popular events (subtopics), that is, events that the public cares about for a long time (hot subtopics) cannot be detected

Images