Hyper text transfer protocol (HTTP)-analysis-based uniform resource locator (URL) automatically classifying and grading system and method

Abstract

The invention discloses a hyper text transfer protocol (HTTP)-analysis-based uniform resource locator (URL) automatically classifying and grading system and an HTTP-analysis-based URL automatically classifying and grading method. The system comprises an HTTP request / response message preprocessor, an HTTP request message resolving server, an HTTP response message resolving server, a switch, an HTTP storage and correlation server and an HTTP grading and classifying server, wherein the HTTP request / response message preprocessor is used for distributing an HTTP request / response message image data flow; the HTTP request message resolving server is used for recombining the HTTP request message image data flow; the HTTP response message resolving server is used for recombining the HTTP response message image data flow; the HTTP storage and correlation server is used for storing message information; and the HTTP grading and classifying server is used for automatically classifying and grading URLs. The method comprises the following steps of: preprocessing an HTTP message image data flow to acquire the HTTP request / response message image data flow, and distributing the HTTP request / response message image data flow; recombining the request / response message image data flow, and storing the message information which is extracted from recombined data; correlating the stored information; and classifying and grading the URLs according to correlated information. By the system and the method, the URLs can be efficiently classified and graded.

Description

technical field [0001] The present invention relates to the technical field of network behavior monitoring and network behavior management, and more specifically, to a system and method for automatically classifying and grading URLs based on HTTP analysis. Background technique [0002] According to the statistical report of Internet tracking agency Netcraft on October 9, 2011, the total number of global websites is about 500 million, of which only 150 million are actually active. According to data analysis: At present, the total number of websites in the world is huge, and "spam websites" exceed 50%, and they are in a growing trend. The Internet environment needs to be cleaned and purified. According to the "New Media Blue Book" released by the Chinese Academy of Social Sciences in 2011, the total number of Internet sites in China reached 1.91 million in 2010, and the number of web pages reached 60 billion. [0003] As we all know, web pages and all other resources on the I...

AI Technical Summary

Problems solved by technology

The disadvantages of this are: (1) In the HTTP / 1.1 standard, a TCP stream can contain multiple HTTP Requests and Responses, which increases the complexity of judging HTTP parsing and extraction

(2) The HTTP Response message may carry audio and video data, resulting in a long duration of the TCP stream, which increases the time and space overhead of the system

Images