Data import method and device, computer equipment and storage medium

A data import and data technology, applied in the computer field, can solve problems such as error-prone configuration files and low data import efficiency

Pending Publication Date: 2020-08-25
CHINA PING AN LIFE INSURANCE CO LTD
View PDF0 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present invention provides a data import method, device, computer equipment and storage medium to solve the problem that it is easy to make mistakes when writing configuration files, and the data import efficiency is low in the case of a large amount of data import

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data import method and device, computer equipment and storage medium
  • Data import method and device, computer equipment and storage medium
  • Data import method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0029] The data import method provided by this application can be applied in such as figure 1 In the shown application environment, the application environment includes a server and a client, wherein the server and the application client are connected through a network, the network may be a wired network or a wireless network, and the client specifically includes but is not limited to various Various personal computers, notebook computers, smart phones...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data importing method and device, computer equipment and a storage medium. The method comprises the steps of obtaining to-be-imported data information; when the current timereaches the execution time, obtaining to-be-imported data from the corresponding data table according to the data source identifier, storing the to-be-imported data in a distributed file system, and obtaining data structure information from the corresponding data table; packaging the data structure information according to the data type to obtain configuration file information; and sending the configuration file to the open-source distributed data storage system cluster to indicate the open-source distributed data storage system cluster to extract the to-be-imported data from the distributed file system according to the configuration file, thereby realizing data import. According to the technical scheme, import of different data formats can be supported, the problem of data inclination occurring in the data intake process is automatically solved, various errors caused by manual writing of configuration files are avoided, and the production efficiency is greatly improved in the whole process.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data import method, device, computer equipment and storage medium. Background technique [0002] Druid is an open source data connection pool in Java. It is an open source, distributed, open source distributed data storage system that supports real-time analysis of massive data. Druid is commonly used in business intelligence / OLAP (Online Analytical Processing, Online Analytical Processing) applications to analyze large amounts of real-time and historical data. It is designed to help enterprises quickly process ultra-large-scale data, and enable fast query and analysis. [0003] At present, the official access method for data stored in Druid is: the user writes a matching configuration file in advance according to the data structure and table structure of the external data source. This configuration file is called Ingestion Spec in Druid ( Data ingestion specificati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/18G06F16/17G06F16/182G06F16/25
CPCG06F16/1734G06F16/1815G06F16/182G06F16/258
Inventor 陈赟
Owner CHINA PING AN LIFE INSURANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap