Method and system for clustering optimization based on Canopy algorithm

Abstract

The invention provides a method for clustering optimization based on a Canopy algorithm. The method comprises the following steps of step (101) carrying out packet processing on all original data based on the Canopy algorithm in order to obtain N Canopy sets and a center of each Canopy set; and step (102) taking a number N of the Canopy sets as a divided number k constructed by a K-means clustering algorithm; taking the center of each Canopy set as a cluster center of a divided kth cluster; and adopting the K-means clustering algorithm to carry out clustering optimization processing on all original data based on the determined cluster number k and cluster center, and outputting a clustering optimization result. According to the method and the system, the Canopy clustering algorithm is used for preprocessing of the K-means clustering algorithm and is used for finding the proper k value and cluster center. The running time of the whole cluster is greatly reduced, the computational efficiency of the algorithm is improved, and the fault tolerance of the algorithm is increased.

Description

Technical field [0001] The invention relates to a clustering method, belonging to the field of data mining, and in particular to a clustering optimization method and system based on Canopy algorithm. Background technique [0002] With the rapid development of computer technology, the amount of data has shown an exponential growth. How to find hidden, previously unknown and potentially valuable information from a large amount of data has become a problem of increasing concern to people. This is why data mining produce. Cluster analysis is a very important part of it. Clustering is a process of dividing a group of data into each class, so as to minimize the distance within the class and maximize the distance between classes, that is, the data in the same class are as similar as possible, and the data in different classes are as similar as possible different. [0003] K-means clustering is a typical distance-based exclusive division method: given a data set of n objects, it can con...

AI Technical Summary

Problems solved by technology

But the disadvantage is: K-means clustering requires the user to give the number of clusters in advance. The selection of k is generally based on some empirical values ​​and multiple experimental results. For different data sets, the value of k has no reference

In addition, K-means is sensitive to "noise" and outlier data, and a small amount of such data can greatly affect the average

Images