Data filling method and data filling system based on quality control

Abstract

The invention discloses a data filling method and a data filling system based on quality control. The data filling method includes the steps of determining missing data according to existing data of a database; establishing a data dependency relationship of the database, and determining a dependency confidence level of the data dependency relationship; determining deducible data and at least one group of non-deducible data in the missing data according to the existing data and the data dependency relationship; determining one group of to-be-retrieved data from the groups of non-deducible data according to a preset rule; deducing the deducible data according to the existing data and the data dependency relationship, and calculating a deduction confidence level according to the dependency confidence level; when the deduction confidence level is higher than a preset threshold value, filling the deducible data, retrieving the to-be-retrieved data from external resources, and calculating a retrieving confidence level according to the dependency confidence level; when the retrieving confidence level is higher than a preset threshold value, filling the to-be-retrieved data. The data filling method and the data filling system based on quality control have the advantages that by means of executing deducing and retrieving alternately, high filling accuracy can be guaranteed at small cost, and the confidence level is enabled to be high due to the fact that the dependency confidence level of the data dependency relationship is taken into consideration.

Description

Technical field [0001] This application relates to the technical field of database processing, and in particular to a data filling method and system based on quality control. Background technique [0002] Generally, there are often some vacancies in the data sources of various databases, some are caused by the lack of original data, and some are caused by operational errors. The vacancy information in these databases will cause incomplete data, which is a common problem in various databases. The data filling technology is proposed to estimate, predict, or retrieve the vacancy information in the data source through some technical means. [0003] The existing data filling methods for string data can generally be divided into two categories: data filling methods based on inference and data filling methods based on retrieval. [0004] The data filling method based on inference mainly combines some given data quality rules (such as Functional Dependencies) to infer the vacancy informatio...

AI Technical Summary

Problems solved by technology

[0006] However, the main disadvantage of the inference-based data filling method is reflected in the filling of the only vacant information, that is, if the information corresponding to the vacant information does not appear in the complete part of the data set, then it is impossible to infer and fill accurately. This vacancy information leads to a low accuracy rate of data filling; while the retrieval-based data filling method can accurately fill vacancy information and improve the accuracy rate of data filling, but it needs a large number of external resources when retrieving vacancy information. The retrieval query, which will generate a large number of retrieval query operations, correspondingly will cause a large system overhead

[0007] Moreover, none of the above methods considers the reliability of the data dependencies in the data set, leading to the quality control of the filled data, which will lead to low credibility of the filled data

Images