Webpage denoising system and method based on maximum similarity matching

Abstract

The invention discloses a webpage denoising system and method based on maximum similarity matching, belonging to the technical field of the internet. The system comprises a webpage acquisition module, a pretreatment module, a webpage DOM (document object model) generation feature tree module, a feature tree maximum similarity matching module and an aggregation evaluation module, wherein the webpage acquisition module is connected with the pretreatment module and transmits webpage code data; the pretreatment module is connected with the webpage acquisition module and transmits a pretreated target webpage; the pretreatment module is connected with the webpage DOM generation feature tree module and transmits pretreated webpage data; the webpage DOM generation feature tree module is connectedwith the feature tree maximum similarity matching module and transmits feature tree data; the feature tree maximum similarity matching module is connected with the aggregation evaluation module and transmits a webpage content block candidate set; and finally, the aggregation evaluation module outputs the webpage content block. The invention can be better suitable for majority of content-type websites.

Description

technical field [0001] The present invention relates to a system and method in the field of Internet technology, in particular to a web page denoising system and a denoising method based on LCS (Longest Common Subsequence, longest common subsequence) feature tree maximum similarity matching. Background technique [0002] With the continuous development of Internet technology, Internet information has shown explosive growth. How to find core subject information from massive webpage information has become a trend in the field of Web research today. A web page generally contains some content blocks, but in addition to these content blocks, it often contains navigation bars, copyright information, announcement messages and various forms of advertisements, which exist for commercial purposes or for the convenience of users, which have nothing to do with the theme The information may be referred to as a web page noise block. How to reduce the noise in web pages is of great signi...

AI Technical Summary

Problems solved by technology

The method is based on machine learning, which is too complex to be efficient

Images