A Distributed Internet Data Rapid Acquisition System and Acquisition Method
A collection method and technology of a collection system, applied in the directions of network data indexing, network data retrieval, network data query, etc., can solve the problems of multiple data sources, low real-time performance, and large amount of data collection, and achieve high scalability and real-time Highly flexible and scalable effects
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment Construction
[0017] The implementation of the present invention will be described in detail below with examples, so as to fully understand and implement the implementation process of how the present invention uses technical means to solve technical problems and achieve technical effects.
[0018] A distributed Internet data fast acquisition system of the present invention, such as figure 1 As shown, there are five layers including torrent website setting node, hyperlink collection layer, real-time queue, webpage download and analysis layer, webpage data storage layer,
[0019] Among them, the torrent website setting node is used to set various parameters of the storage data source and extraction rules, etc., and is a single node; the torrent website setting node uses a relational database.
[0020] Among them, the hyperlink collection layer is used to request the hyperlink list webpage of the data source and extract the hyperlink of the target webpage; the hyperlink collection layer is com...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com