Detection and filter method of network community garbage information based on topic consensus coverage rate

Abstract

The invention provides a detection and filter method of network community garbage information based on a topic consensus coverage rate, belongs to the research category of data quality, relates to the technical field of feature research of user behaviors, evaluation of network information quality, feature value extraction of text content, building and optimization of a text classification model, and the like. Mainly aiming at the situation that an effective automatic detection and filter mechanism for the network community garbage information does not exist, a garbage information detection model is built, a topic convergence restricted relationship is constructed according to main topic content and normal replay content, a feature value of the topic consensus coverage rate is provided and applied to a text classifier, and accordingly automatic detection and filter of the network community garbage information are achieved. The method can be widely applied to the problems of screening of various contents in network community quality management, automatically judge and clean irrelevant advertisements, invalid contents, even malicious opinions, and improve the network community information quality to a certain degree.

Description

technical field [0001] The invention belongs to the research field of data quality, in particular to a method for detecting and filtering spam information in network communities based on topic consensus coverage. Background technique [0002] Topic Consensus: According to an important conclusion in the research field of user behavior characteristics, the content posted by ordinary users who communicate on the same topic in the online community is related to the topic, while users with bad intentions will publish some irrelevant topics. content. J.M.Reagle pointed out in his monograph "Good Faith Collaboration --The Culture of Wikipedia" that the reason why users who post different opinions in the online community can successfully communicate on the same topic is because they have basic knowledge of the topic. Consensus, and believe that the other party that communicates with oneself will also release the reply content according to this basic consensus [1]. And Jim Giles, t...

AI Technical Summary

Problems solved by technology

However, the above detection methods do not take into account the similarity of posting content under the same topic. Therefore, for those spam messages that are similar to normal posting content and appear relatively hidden, the detection accuracy and recall rate of these methods are not high.

Images