Differential distinguishability k prototype clustering method based on MapReduce

Abstract

The invention discloses a differential distinguishability k prototype clustering method based on MapReduce. The differential distinguishability k prototype clustering method comprises the following steps: step 1, preprocessing an input data set D; step 2, carrying out task setting of a MapReduce framework; 3, determining a local center point set Q in each Map task; 4, determining the clustering number k according to the local center point set Q; step 5, setting parameters of a differential distinguishability implementation mechanism; step 6, dividing each data record of the data set D1 into corresponding clusters; 7, calculating a new round of clustering center point; step 8, comparing the central points of the two rounds of clustering; and 9, dividing a data set D1 according to the finalclustering center point. According to the method, a simple parameterization method is provided for practitioners of big data mining; the data processing efficiency is improved, and the safety and effectiveness of the data can be ensured.

Description

technical field [0001] The invention relates to a differential discriminability k-prototype clustering method based on MapReduce, which belongs to the technical field of network space security. Background technique [0002] Data mining is an efficient and in-depth data analysis technology under the background of big data. It has absorbed a large number of technologies in many application fields such as machine learning, database, and statistics, and has quickly become a research hotspot in all walks of life. As an important direction of data mining, cluster analysis is widely used in various scenarios. Clustering can design different algorithms according to the characteristics of data sets and specific analysis tasks. According to the type of processing objects, clustering algorithms can be divided into three categories: numerical data clustering algorithms, categorical data clustering algorithms, and hybrid data clustering algorithms. Data Clustering Algorithms. Most clus...

AI Technical Summary

Problems solved by technology

However, there are two problems in the above privacy protection method: (1) it has background-related dependence, that is, it assumes a certain attack model or background knowledge of the attacker; (2) it lacks a strict theoretical basis, which proves that the level of privacy protection is very high difficulty

[0005] In summary, in big data analysis, hybrid data clustering methods are likely to cause privacy leaks, and traditional data processing models cannot meet the needs of big data computing.

Images