Data reliability evaluation method based on improved Apriori algorithm and Bayesian network reasoning

Abstract

The invention discloses a data reliability evaluation method based on an improved Apriori algorithm and Bayesian network reasoning, and belongs to the data processing field. The evaluation method comprises the steps: adopting a clustering and local linear embedding learning algorithm; performing data encoding on an input data matrix, establishing a directed acyclic graph of a Bayesian network through a domain background according to a correlation relationship of data of each dimension, and obtaining a conditional probability table through an improved Apriori algorithm to obtain local and global reliability values of multi-dimensional data. According to the invention, the algorithm evaluates the reliability of the data from the relationship between the data structure and the data, does notneed to determine the reliability index and the data distribution condition, and reduces the subjectivity of the previous data reliability evaluation. The algorithm has universality, is suitable for discrete data values, and is also suitable for the reliability of interval numbers. The algorithm is high in accuracy, the association relationship between the same dimension and different dimensions of the high-dimensional data can be mined, and the local reliability and the global reliability of each piece of data are obtained.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data reliability evaluation method based on improved Apriori algorithm and Bayesian network reasoning. Background technique [0002] With the advent of the era of big data, data mining algorithms are widely used in various fields, making data the most valuable raw material for many organizations. Many organizations sell data, and others provide services and solutions for mining data. Indeed, there is increasing reliance on secondary data sources such as estimates and forecasts, which may have different characteristics that affect overall reliability. At this point, more traditional approaches to reliability become less useful, because the metadata needs to contain certain information that is hidden behind the scenes by the system's data representation. [0003] Reliability originated in the field of industrial engineering quality control and was originally defined as the abilit...

AI Technical Summary

Problems solved by technology

However, whether it is the expert scoring table, the reliability of the data transmission process or the reliability evaluation based on the source information of the data, the reliability indicators formulated are subject to a certain degree, and a relatively objective data reliability is required at this time. Combining the above two reliability evaluation methods together to formulate a relatively complete data reliability evaluation system

Images