Data management method and device, storage medium and system

Abstract

The invention discloses a data management method, which is executed by a metadata system, and comprises the following steps: responding to an offline data task output result table of a data warehouse,executing a data quality check task on the result table to obtain a check result; judging whether the verification result is verification failure or not; in response to the fact that the verificationresult is verification failure, determining a task node with verification failure, and judging whether the task node with verification failure is configured with a first alarm mode or not; wherein the first alarm mode is used for indicating whether the task node is a key task node; and in response to the fact that the task node which fails in verification is configured with a first alarm mode, sending an interrupt signal to the data warehouse so as to enable the data warehouse to interrupt execution of the offline data task. According to the data management method provided by the invention, the data management efficiency can be improved, the avalanche effect of downstream tasks is prevented, and the problem of large-area data is avoided. Meanwhile, the invention further provides a data management device, a storage medium and a system.

Description

Technical field [0001] The invention relates to the technical field of data warehouses, in particular to data management methods, devices, storage media and systems. Background technique [0002] Data warehouse is an indispensable part of companies with massive amounts of data. It is created for further mining of data resources and for decision-making needs. In this process, a series of operations such as data collection, cleaning, and integration are bound to be involved. Then in the data-oriented process, along with data development, data management is bound to be inseparable. [0003] In the process of implementing the present invention, the inventor found that the prior art has the following defects: When performing data management, warehouse developers often need to spend a lot of energy to pay attention to some other systems for checking data quality or additional development data. Quality inspection tasks; moreover, this method of data management is actually a hindsight met...

AI Technical Summary

Problems solved by technology

[0003] In the process of implementing the present invention, the inventor found that the existing technology has the following defects: when performing data management, warehouse developers often need to spend a lot of energy to pay attention to some other systems for checking data quality or additional development data Quality inspection tasks; moreover, this method of data management is actually a method of hindsight. When there is a problem with data quality, it cannot perceive the data quality problem at the first time, which will easily lead to an avalanche effect and large-scale data problem

Images