Data acquisition method and device and computer readable storage medium

A data collection and data technology, applied in the field of big data, can solve the problems of low collection accuracy, low efficiency, slow timeliness, etc., and achieve the effect of efficient data collection and efficiency improvement

Pending Publication Date: 2020-09-18
PING AN BANK CO LTD
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a data collection method, device, electronic equipment and computer-readable storage medium, the main purpose of which is to solve the problems of low efficiency, slow timeliness, low collection precision and incomplete data collection in the data collection method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data acquisition method and device and computer readable storage medium
  • Data acquisition method and device and computer readable storage medium
  • Data acquisition method and device and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0052] The invention provides a data collection method. refer to figure 1 As shown, it is a schematic flowchart of a data collection method provided by an embodiment of the present invention. The method may be performed by a device, and the device may be implemented by software and / or hardware.

[0053] In this embodiment, the data collection method includes:

[0054] S1. Obtain a target collection website, and divide the target collection website into different structural levels according to the website structure.

[0055] In the embodiment of the present invention, the target collection website includes any website on the Internet, such as CNKI, Baidu Library, etc.

[0056] In order to realize data collection on the target collection website more accurately and efficiently, the embodiment of the present inventio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a big data technology, and discloses a data acquisition method, which comprises the following steps of: acquiring a target acquisition website, and dividing the target acquisition website into different structural levels according to a website structure; generating data acquisition rule items corresponding to the different structural levels according to the data acquisition attributes in the different structural levels; obtaining the underlying data frameworks of the different structural levels; constructing a rule template with the same structure level as the target collection website according to the underlying data framework, and adding the data acquisition rule item to a corresponding level of the rule template, and acquiring data of the corresponding structural level of the target acquisition website by utilizing the data acquisition rule item of each structural level in the rule template to obtain a target data set. The invention further provides a data acquisition device, electronic equipment and a computer readable storage medium. The problems that a data generation method is complex and occupies computing resources can be solved.

Description

technical field [0001] The present invention relates to the field of big data technology, in particular to a data collection method, device, electronic equipment and computer-readable storage medium. Background technique [0002] Data collection is the cornerstone of the field of big data and artificial intelligence. For example, website developers often collect the data of the website comprehensively and completely. The collection process consumes a lot of labor costs and takes up a lot of computing resources. [0003] At present, most of the industry uses data collection tools such as Octopus and Jisouke for data collection, but these data collection tools often use a fixed set of data collection rules. However, the data on the website includes different data types, and the data collection rules adopted are also different. Therefore, adopting a fixed set of data collection rules cannot guarantee that different types of data can be completely and accurately collected during...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/951G06F16/95
CPCG06F16/95G06F16/951
Inventor 颜超
Owner PING AN BANK CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap