Method and system for filtering sensitive web page based on multiple classifier amalgamation

Abstract

The invention discloses a system and a method for filtering sensitive webpage, which is based on multi-classifier fusion. The processing object is a webpage, and the processing result is whether the webpage contains sensitive content, which may be pornography, reaction, violence and other unhealthy Internet contents harmful to society. The system comprises a data stream obtaining and preprocessing unit, an image and text stream filtering unit and an information fusion unit of image filter and text filter, by the cooperation of multiple classifiers, the system acquires source code of a webpage by using the URL of the webpage, a text and an image are separated at preprocessing stage to obtain text information and effective image information; an input webpage is divided into three modes by decision tree algorithm; the webpage is recognized by using a consecutive text classifier, a discrete sensitive text classifier and an image classifier, the output result recognized by the classifiers is fused and calculated, then a judge factor is given, and the final result is returned to a browser.

Description

technical field [0001] The invention relates to the technical field of information filtering, in particular to a method for identifying webpages containing sensitive information. Background technique [0002] Because Internet sensitive information has caused great harm to Internet users, especially young people, it has attracted extensive attention from researchers and the industry. [0003] There are currently many sensitive information filtering methods, including black and white lists, IP filtering, keyword matching and other filtering methods. Generally speaking, on the one hand, these filtering technologies adopt a very mechanical method, which can achieve 100% filtering efficiency for some sensitive web pages, and the response time is also very short, but the update cycle of filtering parameters can only follow the actual sensitive web pages The appearance of the website changes, and it cannot cope with the rapid changes of the actual sensitive website. On the other ...

AI Technical Summary

Problems solved by technology

[0006] Similar to the mechanical filtering method, the above methods do not make good use of web features and cannot achieve satisfactory results at present. For example, text-based sensitive webpage recognition cannot identify normal webpages related to sensitive topics. Based on Sensitive web page recognition of images has a high false positive rate

The existing fusion algorithm is only fused by AND or operation, which cannot fundamentally improve the recognition rate

Images