Network data collection method
A collection method and network data technology, which is applied in the field of big data, can solve problems such as time-consuming, messy content, and complicated network information capture methods, and achieve the effect of improving the collection and capture speed and storage speed
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Examples
Embodiment
[0029] A network data collection method, comprising:
[0030] Step 1, crawling webpage content from the Internet through a web crawler, and extracting required attribute content;
[0031] Step 2, provide the crawler with the URL that needs to grab the data network through the URL queue;
[0032] Step 3, process the content captured by the crawler through the data processing module;
[0033] Step 4, store the URL information of the website that needs to grab data, the data extracted from the webpage by the crawler, and the data processed by the DP through the data storage module.
[0034] Step 1 includes:
[0035] Step 11, write the URL information of the website that needs to grab data into the URL queue;
[0036] Step 12, the crawler obtains the site URL information of the website that needs to grab data from the URL queue;
[0037] Step 13, the crawler crawls the corresponding web page content from the Internet, and extracts the content value of a specific attribute;
...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com