Parallel-PSO-based maximum fault-tolerant frequent item set mining method

Abstract

The invention discloses a parallel-PSO-based maximum fault-tolerant frequent item set mining method. The parallel-PSO-based maximum fault-tolerant frequent item set mining method includes the steps: converting the target transaction database into a corresponding binary matrix; if the converted binary matrix is a sparse matrix, setting a minimum item support degree to delete the item which does notmeet the threshold value; initializing a particle swarm according to the matrix dimension; adding a Gaussian disturbance item into a speed updating formula of the particle swarm algorithm to preventthe population from falling into local optimum; according to the concept of the fault-tolerant block, designing a fitness function of a particle swarm algorithm, converting particles in a population into an RDD data set in a Spark platform, achieving a termination condition after iteration, and obtaining a maximum fault-tolerant block, wherein when the support degree of the fault-tolerant block islarger than a minimum support degree threshold value, an item set corresponding to the fault-tolerant block is a maximum fault-tolerant item set. The parallel-PSO-based maximum fault-tolerant frequent item set mining method improves the operation efficiency of the algorithm, guarantees that the efficiency of the conditional algorithm with different fault tolerance degrees in the same target transaction database is unchanged, and meanwhile has high performance for the sparse target transaction database.

Description

technical field [0001] The invention relates to the technical field of frequent item set mining, in particular to a maximum fault-tolerant frequent item set mining method based on parallel PSO. Background technique [0002] The problem of frequent itemset mining is to find all frequent itemsets, that is, the set of items that reach a given minimum threshold. This problem is the basis of data mining and knowledge discovery, such as association rule mining, subspace clustering, etc. As an important achievement in the direction of data mining, frequent itemsets reflect the mining of data in a noise-free environment. In many engineering research fields, the mining of frequent itemsets becomes more difficult due to the presence of noise in the data. Noise in these data can be due to various reasons such as measurement errors, missing values, and certain anomalies. The previous method of eliminating noise is often to divide the larger pattern into smaller patterns. However, th...

AI Technical Summary

Problems solved by technology

[0004] The present invention aims at the fact that most current fault-tolerant frequent itemset mining algorithms are not efficient when dealing with large-scale target transaction databases and sparse target transaction databases, and the algorithm cannot maintain the same efficiency under the condition of different fault-tolerant degrees under the same target transaction database Problem, provide a maximum fault-tolerant frequent itemsets mining method based on parallel PSO

Images