Streaming data event text topic and detection system

Abstract

The invention discloses a streaming data event text topic and a detection system, intermediate process redundancy can be eliminated, and the detection time can be reduced. According to the technical scheme, the system is characterized in that a topic detection module constructs a topic and event detection algorithm model, and text data is crawled from all large network media and social platforms in real time through the crawler technology; a topic tracking module gives a topic abstract and keyword information according to the text abstract information provided by a text abstract module; an association detection module detects texts in all directions, divides event affiliation of the texts, sets a time window with a determined length, sends a detected topic clustering result to the topic tracking module to obtain a clustering result with smaller granularity; and an event recognition module carries out event recognition in a hierarchical clustering mode, gives a designed topic abstract and event extraction algorithm and sends the topic abstract and keyword information to an event extraction module, and key information of the topic and event is analyzed, and a large number of topic sets are obtained.

Description

technical field [0001] The invention belongs to the technical field of topic detection and tracking, event detection and extraction, and in particular relates to a topic and event detection system for streaming text data. Background technique [0002] In recent years, with the rapid development of the Internet and the Internet of Things, a large amount of information is generated on the Internet every day, and the large amount of information on the Internet is exploding. It is becoming more and more difficult for people to quickly and accurately retrieve high-quality information from the Internet. Useful news information, and these massive data will appear in many applications, and a large part of this data exists in the form of streaming data. Streaming data is characterized by fast, large volume, disorder, and requires fast response. All kinds of emergencies occur frequently at home and abroad. News websites and social platforms are the most direct and fastest way for peo...

AI Technical Summary

Problems solved by technology

However, the evolution of spam content upgrades is too fast, complex character reorganization, special symbols, etc. make audit methods helpless

Traditional retrieval methods are based on the user's deep understanding of their own query needs. That is to say, users need to accurately express their query needs into query expressions when retrieving news information. This is the conversion of query needs into query expressions. Any deviation from will seriously affect the search results

In this type of retrieval needs, users cannot precisely define their own needs and can only describe them abstractly. This type of retrieval needs is difficult to satisfy by today's keyword-based search engines.

Images