Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Partition data processing method and application method for slowly changing dimension table

A technology of data processing and partitioning algorithm, applied in the field of data processing and ETL algorithm in data warehouse, to achieve stable efficiency, improve query efficiency and basic efficiency

Pending Publication Date: 2022-07-22
KUNMING UNIV OF SCI & TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, as various types of data requirements are continuously moved to the "online", the query efficiency of the conventional slow-changing dimension tables can no longer meet the increasing needs of various end users

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Partition data processing method and application method for slowly changing dimension table
  • Partition data processing method and application method for slowly changing dimension table
  • Partition data processing method and application method for slowly changing dimension table

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0036] Example 1: as Figure 1-3 As shown, the steps of the storage processing method for the slowly changing dimension table data partition with the table name goal_table_hs in the data warehouse:

[0037] 1) Create a database table partitioned by the natural month time range, and specify the start timestamp of the slowly changing dimension table as the partition key;

[0038] 2) Prepare two database tables with a similar table structure to the target slowly changing dimension table for temporary data storage in the partitioning algorithm;

[0039] 3) Use the slowly changing dimension partitioning algorithm to realize the partition processing when the data is put into the warehouse;

[0040] 4) Periodically perform partition maintenance on partitions with partitioned slowly changing dimension tables.

[0041] Further, described step 1) comprises the following work:

[0042] Create a database table goal_table_hs with time range partitions, and execute the following data def...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a partition data processing method of a slow change dimension table and a using method, and belongs to the technical field of ETL data processing in a data warehouse. The method comprises the following steps: firstly, establishing a partition database table in a database according to a time range of a natural month, and appointing a start timestamp of a slow change dimension table as a partition key; when new data in the data warehouse needs to be warehoused and the processing time node is the beginning of a month, performing closed-chain processing on open-chain total data in the slow-change dimension table before access by using a partition slow-change dimension table algorithm, then calculating new warehousing data, and performing open-chain processing on the newly accessed data; when the partition slow change dimension table processed by the method is used, the start timestamp is placed between the beginning of the month where the target date is queried and the target date, and at the moment, the data to be queried is locked in the same partition, so that the query efficiency of the slow change dimension table is greatly improved. According to the method, the query efficiency and the query stability can be improved while the characteristics of the slow change dimension table are reserved.

Description

technical field [0001] The invention relates to a partition data processing method and a using method of a slowly changing dimension table, belonging to the field of data processing, in particular to the field of ETL algorithms in data warehouses. Background technique [0002] Data warehouse technology was first proposed by Bill Inmon in the mid-1980s. Data warehouse is not a ready-made product, but a dynamic process of data application. [0003] The ETL process of data is the first step in building a data warehouse, the basic work of the data warehouse, and the preparation for subsequent operations of the data warehouse. ETL is the English acronym for the three steps Extract, Transform, and Load during data scheduling, which is the abbreviation of Extraction, Transformation, and Loading, which is the process of sequentially extracting, transforming, and loading business data scattered in various systems into the data warehouse. . [0004] Slowly changing dimension tables...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25G06F16/28G06F16/22
CPCG06F16/254G06F16/283G06F16/2282
Inventor 徐丽娜赵冲
Owner KUNMING UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com