Method for carrying out harmful content recognition on network text and short message service

Abstract

The invention belongs to the technical field of text processing, in particular to a method for carrying out harmful content recognition on network text and short message service, which comprises the following steps of: inputting a text to be detected, determining a text coding format, carrying out format conversion on the text, comparing the text with a short string word bank, comparing the text with a long string word bank, carrying out copy detection on a result, and displaying a final result. The method can be used for the detection and the filtration on harmful, violent and reactionary texts in the internet, inhibits the spreading of the harmful content, and protects physical and psychological health of youngsters.

Description

technical field [0001] The invention belongs to the technical field of word processing, and in particular relates to a method for decoding, analyzing and filtering (copy detection) of text content. Background technique [0002] With the increasing use of the Internet, all kinds of bad information (pornography, reactionary, violence, etc.) on the Internet are increasingly interfering with the normal order of the Internet. Due to the lack of effective monitoring of information release by traditional media, a large amount of information that should have been strictly regulated has flooded. How to effectively control the dissemination of these information and ensure the security of network content has become one of the main contents of research on bad text retrieval and monitoring. Most of the existing network filtering systems are based on the filtering of URL addresses. This technology simply directly shields the network users from the bad content website URL address database...

AI Technical Summary

Problems solved by technology

Most of the existing network filtering systems are based on the filtering of URL addresses. This technology simply directly shields the network users from the bad content website URL address database (commonly known as "blacklist") set in the firewall of the network operator. Websites, there are few commercial systems on the market at home and abroad that directly target network text content filtering

Although the URL-based blocking technology is simple and efficient, it has serious limitations: because network operators cannot update the blacklist in time, this will cause many new pornographic websites to slip through the net; at the same time, not all webpages under some domain names The content is unhealthy, which in turn will lead to some normal content being blocked innocently

For example, "Falun Gong" is a reactionary word, but if it is an article against Falun Gong, if it is blocked rashly, it will misunderstand the meaning of the author

Images