Method for establishing garbage text library, filtering method and system

Abstract

An embodiment of the invention discloses a junk text library establishing method and belongs to the technical field of establishment of computer text libraries, wherein the method comprises: S100, acquiring at least one pre-collected junk text sample from text; S200, detecting whether long characteristic words are present in each junk text sample or not; if yes, recording the long characteristic words into a long characteristic word set; S300, classifying the junk text samples corresponding to the long characteristic word set based on Bayes classifier to obtain junk text and non-junk text; S400, comparing the number of new junk text with a preset convergence threshold, executing step 500 if the number of new junk text is less than the convergence threshold, and executing step 600 otherwise; S500, finishing the establishment of the junk text library, and ending the process; S600, acquiring new junk sample files from the text, and returning to execute the step S200 to step S500. According to the embodiment of the invention, the method allows the junk text library to be established just with few text samples collected, time and labor are saved, and the precision is greater.

Description

technical field [0001] The invention relates to the technical field of computer text database establishment, in particular to a method for establishing a garbage text database, a method for filtering garbage text and a system for establishing a garbage text database. Background technique [0002] With the popularization of the Internet industry and the continuous in-depth development of e-commerce applications, people interact more and more frequently on the Internet. However, as the amount of information continues to increase, unnecessary spam information also increases, causing users to Receiving unwanted junk information when obtaining information, thus making wrong judgments or choices. [0003] In the prior art, some online games or forums have provided a detection function similar to spam comments, and its usual processing steps are: 1. Segment the text entered by the user; 2. Perform keyword matching on the word segmentation results; 3. If the keyword is matched, the...

AI Technical Summary

Problems solved by technology

[0004] 1. Existing technical solutions require a large amount of spam texts to be compared with normal texts to determine the spam feature words when extracting advertising keywords, and the statistics are time-consuming and laborious;

[0005] 2. The relevant garbage keywords after the keywords are included lack the function of further learning;

Images