A Webpage Text Extraction Method Based on Logical Link Blocks

Abstract

The invention discloses a webpage text extraction method based on a logic link block, and the method only depends on the current webpage in the process of extracting a webpage template and a text, anddoes not need heuristic rule support, so that the method has good universality; the extraction process of the webpage template does not need manual intervention, and the automation degree is high; the analysis process is simple, and label analysis does not need to be carried out on the webpage, so that the analysis speed is high, the anti-interference performance is high, and the method can better adapt to a Web page with nonstandard design; the method also has a better extraction effect on pages with very short text contents; and finally, the template extracted by the method is simple in form and easy to use. Therefore, it is determined that the method has potential application value in the aspect of Web page text extraction, can be used for text extraction of various news, blogs or webpages with similar structures, and also has wide application prospects in other Web information processing and mining fields with low requirements for link block fine granularity.

Description

technical field [0001] The invention belongs to the field of computer technology, and relates to a web page text extraction method, in particular to an extraction method (Content Extraction based on Logical Link Blocks, CELLB). Background technique [0002] Web page information extraction refers to extracting specific information content from web pages according to specific analysis and application requirements. These specific information contents not only include some superficial content directly from web pages, but also deep content formed based on specific analysis of web pages. Web page text extraction is one of the main directions of this research. It has a long research history and many related methods. Document 1 summarizes webpage extraction methods into five categories based on Wrapper, template, machine learning, visual layout features, and HTML features. Among them, Wrapper and template methods are generally considered to be less versatile, and generally require...

AI Technical Summary

Problems solved by technology

[0048] In view of the fact that current methods based on HTML tag trees have high requirements for HTML normativeness, and methods based on text density and other information cannot handle the problems such as short webpage texts, the present invention proposes a news, blog and other information based on logical link blocks. Extraction method of webpage text extraction template (Content Extraction based on Logical LinkBlocks, CELLB)

Images