Data flow clustering method based on density and extension network

Abstract

The invention relates to a data flow clustering method based on density and an extension network. A Spark parallel computing platform is used for analyzing and improving a traditional data flow clustering algorithm, the data flow clustering algorithm based on density and the extension grid is provided, so that defects of the method for manually setting a clustering parameter are improved, and clusters in any shape can be acquired. The algorithm comprises the basic steps as follows: 1, local density of each sampling point and distances with the other sampling points are used for determining the quantity of cluster centers in a grid, the cluster centers are automatically determined, and influence on a clustering result due to improper selection of an initial centroid is avoided; 2, data points outside the grid are clustered by expanding the network, so that clusters in the grid are expanded, and clustering accuracy is ensured; 3, adjacent density estimation and grid boundary are introduced for combining the grids, so that memory consumption is saved; and 4, an attenuation factor is used for updating the grid density in real time, and reflecting an evolution process of a space dataflow.

Description

technical field [0001] The invention belongs to the technical field of data mining, and in particular relates to a data stream clustering method based on density and extended grid. Background technique [0002] With the rapid development of hardware technology, network communication technology, various sensing devices, and various information technologies, in social networks, sensor networks, e-commerce, network monitoring, meteorological environment monitoring, and financial retail enterprises and other application fields, emerging A large amount of real-time dynamic data is called streaming data. [0003] Different from traditional data types, streaming data has the characteristics of large data volume, infinite or unknown length, fast dynamic change, unstable rate, and high cost of accessing historical data. It is very difficult to use traditional data mining algorithms to process streaming data, so clustering algorithms based on data streams emerge as the times require....

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to solve the defects of traditional data stream clustering algorithm, such as manually setting clustering parameters, improper selection of initial centroid, large memory consumption, and low clustering efficiency, by improving the density grid clustering algorithm based on data stream , to ensure the accuracy of clustering results, and has the advantages of good clustering effect and high mining efficiency

Images