Web page pattern recognition method and visual structure learning method based on deep learning

Abstract

The invention discloses a deep learning-based webpage mode recognition method. The method comprises the following steps: taking a Text source code of a hypertext markup language of a webpage with unformatted text as an algorithm for input; carrying out work segmentation on the source code; designing a stack denoising automatic encoder as a characteristic learning algorithm of the webpage; initiating the input of SDAE by adopting a neural network language model; classifying text feature vectors learned through the SDAE by adopting a classification algorithm; and outputting the classification result. The invention furthermore discloses a visual structure learning method, through which the structure of the Text source code in an HTML format is learned by using a machine learning method and then the visual structure of the webpage is learned. According to the deep learning-based webpage mode recognition method and the visual structure learning method, the artificial language is processed by using a natural language processing method, and characteristic learning is carried out on the Text source code in the HTML format by using a deep learning method and a neural network language model; and through the deep learning-based webpage mode recognition method and the visual structure learning method, the webpage modes of various websites such as blogs, forums and information can be precisely recognized.

Description

technical field [0001] The present invention relates to the technical field of the Internet, in particular to a web page pattern recognition method and a visual structure learning method based on deep learning. Background technique [0002] Web page recognition is a key step in data acquisition systems and search engines. At present, the following methods are mostly adopted for web page identification: manually setting identification rules to identify web pages or identifying web pages based on other non-deep learning machine learning methods. [0003] However, due to the highly abstract nature of natural language, it is difficult for machines to achieve ideal recognition results for plain text pattern recognition. Contents of the invention [0004] The technical problem to be solved by the present invention is to provide a web page pattern recognition method and a visual structure learning method based on deep learning. [0005] The technical scheme that the present inv...

AI Technical Summary

Problems solved by technology

[0003] However, due to the highly abstract nature of natural language, it is difficult for machines to achieve ideal recognition results for plain text pattern recognition