Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

ETL data lineage query system and query method

A technology for querying systems and lineages, applied in the field of data management, which can solve problems such as not easy to modify, unable to effectively provide ETL process data modification, and ETL process not easy to improve

Inactive Publication Date: 2015-09-16
GUANGDONG KINGPOINT DATA SCI & TECH CO LTD
View PDF4 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The current ETL process supervision can only supervise the output fields and output preview of an intermediate process alone, and cannot effectively give the data modification status of the entire ETL process
This will make it very difficult to modify after the ETL process is created, because once one of the links is modified, the links after this link must be checked and corrected separately, and the ETL process is not easy to improve

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • ETL data lineage query system and query method
  • ETL data lineage query system and query method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] In order to facilitate those skilled in the art to understand the technical solutions and beneficial effects of the present invention, specific implementation methods are described below in conjunction with the accompanying drawings.

[0032] See figure 1 as shown, figure 1 It is a structural diagram of the ETL data lineage query system of the present invention. An ETL data lineage query system includes an operation module 1 and a data lineage management module 2 . The operation module 1 divides and executes data processing tasks, and the data lineage management module 2 can manage the data and data lineage involved in system operation, and form a data lineage table and a data lineage subsidiary table.

[0033] The running module 1 includes an operation division unit 11 and a script running unit 12 .

[0034] The operation division unit 11 divides the ETL task in the ETL task script file input into the operation division unit 11 into operations and outputs a task scr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to an ETL data lineage query system and query method. The ETL data lineage query system is characterized by comprising an operation module and a data lineage management module, the operation module is capable of running task scripts, dividing tasks, generating a task script file containing operation information and transmitting a task division file containing the operation information to the data lineage management module; the data lineage management module is capable of receiving a user configuration file, collecting a source data file, the task script file containing the operation information and storing data lineage information.

Description

technical field [0001] The invention relates to the field of data management, in particular to an ETL data lineage query system and a query method. technical background [0002] With the expansion of the business scope of the enterprise, the enterprise needs to integrate and analyze the data of different businesses in various regions. It can be considered that business intelligence is the process of collecting, managing and analyzing business information, the purpose of which is to enable decision makers at all levels of the enterprise to obtain knowledge or insight, and prompt them to make decisions that are more beneficial to the enterprise. Business intelligence generally consists of data warehouse, online analytical processing, data mining, data backup and recovery and other components. A data warehouse is a strategic collection of all types of data that support decision-making processes at all levels of an enterprise. It is a single data store created for analytical r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/33
Inventor 许飞月李青海简宋全侯大勇邹立斌
Owner GUANGDONG KINGPOINT DATA SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com