Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A field-level pedigree analysis method and device based on etl

An analysis method and field technology, applied in the field of ETL, can solve problems such as heavy workload, reduce workload, improve work efficiency, and avoid multiple analyses.

Active Publication Date: 2019-06-04
GUANGDONG KINGPOINT DATA SCI & TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Existing technologies require a large amount of work for lineage analysis at the field level and require multiple analyzes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A field-level pedigree analysis method and device based on etl
  • A field-level pedigree analysis method and device based on etl
  • A field-level pedigree analysis method and device based on etl

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0037] Such as figure 1 As shown, it is a flow chart of an ETL-based field-level lineage analysis method provided by the present invention. This ETL-based field-level lineage analysis method includes the following steps:

[0038] Step S1, perform division operation on the content of the ETL task script file, and record the division operation result in the pedigree analysis subsidiary table.

[0039] Specifically, the content of the ETL task script file is analyzed and the operation is divided, each SQL statement is regarded as an operation, and the operation is divided according to the content of the ETL task script file according to this rule. For each step of the ETL, Each complete SQL language is numbered from small to large in logical order, named according to "step name + number" to form the division operation result, and recorded in the lineage analysis subsidiary table.

[0040] Step S2, according to the needs of the user, locate a certain field in a certain step for l...

Embodiment 2

[0056] Such as figure 2 As shown, it is a functional block diagram of an ETL-based field-level lineage analysis device provided by the present invention, and a kind of ETL-based field-level lineage analysis device includes:

[0057] The division operation module 1 is used to perform division operation on the content of the ETL task script file, and record the division operation result in the pedigree analysis subsidiary table.

[0058] Specifically, the content of the ETL task script file is analyzed and the operation is divided, each SQL statement is regarded as an operation, and the operation is divided according to the content of the ETL task script file according to this rule. For each step of the ETL, Each complete SQL language is numbered from small to large in logical order, named according to "step name + number" to form the division operation result, and recorded in the lineage analysis subsidiary table.

[0059] The positioning module 2 is used to locate a certain ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a blood-relationship analysis method and a blood-relationship analysis device of field level on the basis of ETL (Extract Transform Loading). The device comprises a dividing operation module, a positioning module, a positioning and searching module, a first retrieval module, a second retrieval module, a third retrieval module and a recording module. Compared with the prior art, the blood-relationship analysis method and device provided by the invention have the advantages that the displaying of the change process of the fields on the level of the fields is realized, an iterative method is adopted to carry out blood-relationship analysis on involved relevant fields in parallel by establishing a blood-relationship table and a blood-relationship subsidiary table, so that the working efficiency is improved; and blood-relationship analysis graphics of all the fields of all the steps are recorded by adopting the blood-relationship table, and when the blood-relationship of the fields is involved again later, retrieval analysis again is not needed, so that multiple analyses are avoided and the workload is reduced.

Description

technical field [0001] The invention relates to the technical field of ETL in the process of building a data warehouse, in particular to an ETL-based field-level lineage analysis method and device. Background technique [0002] As the business of the enterprise continues to expand, the amount of data is also increasing. In order to achieve better and efficient management, the enterprise needs to integrate and analyze the data of different businesses in various regions. At present, business intelligence technology is used by more enterprises to assist the company to make better decisions, reduce risks, and improve performance. Business intelligence technology generally consists of data warehouse, online analytical processing, data mining, data backup and recovery, etc. The construction of a data warehouse requires ETL processing of data, which is the most important step in ensuring data quality. [0003] ETL, including data extraction (Extract), data transformation (Transfo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/25G06F16/21
CPCG06F16/21G06F16/254
Inventor 李青海简宋全侯大勇邹立斌
Owner GUANGDONG KINGPOINT DATA SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com