Multi-source heterogeneous data quality detection method based on petri network

Abstract

The invention discloses a petri network-based multi-source heterogeneous data quality detection method, which can use a scheduling plan module to configure data quality jobs and then use a message feedback mechanism of a petri network to form a table and chart form quality analysis report for each time of quality jobs and finally accumulate data quality detection problems. A data quality problem knowledge base is formed, sustainable development opinions and suggestions are provided for subsequent data quality detection, it is guaranteed that all data sources are independent of one another, SQLforms are unified, and the problem that multi-source heterogeneous data needs a specific query method for different data sources is solved. A graphical and draggable quality task and job management method is provided for a user, a quality detection scheduling plan can be configured, a data quality analysis result is generated for the user to check, and a data quality knowledge base is establishedto improve the capability of solving quality problems. Effective support is provided for system data quality improvement, so that the data value is improved.

Description

technical field [0001] The invention belongs to the technical fields of big data, data governance and data analysis, and in particular relates to a petri-net-based multi-source heterogeneous data quality detection method. Background technique [0002] The information age has now transitioned to the era of big data, artificial intelligence, and the Internet of Things. In the fields of big data analysis, data warehouse construction, machine learning, neural networks, and Internet of Things data transmission, data is the cornerstone of all of these. Data quality is crucial in the use of data. Only with quality-assured data can algorithms get closer to the correct answer. [0003] In the context of big data, data governance has the problems of huge data volume, various data sources, and various data representation methods. With the introduction of the concept of Data Lake, big data governance does not require prior structured processing of data, and data is stored in relational...

AI Technical Summary

Problems solved by technology

[0007] Aiming at the defects and problems existing in existing equipment, the present invention provides a multi-source heterogeneous data quality detection method based on petri net, which effectively solves the problem that the existing data quality detection often only targets a single data source and a single data format. , lack of system-wide considerations, interpretable results, and cannot be used in the current big data environment

Images