Automatic BBS (bulletin board system) page acquisition method

Abstract

The invention discloses an automatic BBS (bulletin board system) page acquisition method. The method comprises the following steps: 1, acquiring all element information of a BBS page; step 2, cross-contrasting node elements in a system library; step 3, comparing the node quantity if the node name is identical; step 4, determining the node name and the node quantity are respectively identical, and cross-contrasting two node identifiers as a current floor node; step 5, recording XPath of the floor node, segmenting the floors of a post, extracting the XPath of the floor content, and acquiring the universal information. A majority of BBS sites can be processed by automatically analyzing an html structure, the acquisition development efficiency can be effectively improved, the acquisition system structure is optimized, the acquisition flow is simplified, and a novel high-efficient solution is provided for the large-scale acquisition system.

Description

technical field [0001] The invention relates to the technical field of Internet computer BBS processing, in particular to a method for automatically collecting BBS pages. Background technique [0002] With the advancement of science and technology, Internet information has entered an era of explosion and diversity, and the Internet has become a huge information base. Internet information collection can save you a lot of resources in information collection, resource integration, capital utilization, and human investment. Widely used in industry portal information collection, competitor intelligence data collection, website content system construction, vertical search, public opinion monitoring, scientific research and other fields. [0003] Conventional BBS (Bulletin Board System) runs service software on the computer, allowing user terminals to connect through the network, upload and download data, and exchange information with other users. The page parsing template of each...

AI Technical Summary

Problems solved by technology

[0006] 1. It is necessary to manually configure the BBS page parsing template and formulate the XPath of the corresponding information

[0007] 2. It is difficult to formulate uniform rules for the massive information capture of the website

Generally, the parsing template is configured separately for each site, and the workload is heavy;

[0008] 3. The follow-up has brought a lot of rule maintenance work, and the problem of real-time update of the rules after the site revision;

[0009] 4. If the revision of the BBS site cannot be discovered in time, then the data collection of these BBS sites will be abnormal

[0010] The above technical defects are particularly prominent for large-scale acquisition systems, and new technical methods are urgently needed to replace manual maintenance work

Images