Pruning mode-based DBSCAN block segmentation optimization method

Abstract

The invention discloses a pruning mode-based DBSCAN block segmentation optimization method. The method involves a data pruning block segmentation algorithm, a block data dynamic clustering algorithm,a data block combination node calculation method and a data block effective combination algorithm. The method comprises the main steps of calculating a dimension block quantity, traversing all dimensions from a dimension to perform block segmentation on data, and performing pruning operation on data blocks not meeting a continuous block segmentation requirement; dynamically calculating DBSCAN important parameters according to block data density, and dynamically realizing a block clustering process; judging whether two data blocks come from a same father data block or not through formula calculation, and performing a block combination process by taking the judgment as a standard; and performing combination algorithm implementation on data blocks needed to be combined, and solving a clustering process of associating two clusters with low association together due to linear connection by utilizing the combination process. Through the method, the clustering effect and efficiency of DBSCAN are improved; the DBSCAN is applicable to more diversified scenes; and the application of the DBSCAN to conventional fields and big data analysis is promoted.

Description

technical field [0001] The present invention relates to DBSCAN algorithm optimization technology, in particular to a pruning-based parallel accelerated DBSCAN algorithm implementation method, so that DBSCAN can be applied to a wider range of scenarios and promote the application of DBSCAN in traditional fields and big data analysis. Background technique [0002] In the information society, with the continuous development of information technology such as cloud computing and artificial intelligence, and the widespread application of big data technology, using Spark to solve data clustering problems and dig out the rich information contained in massive data has become an increasingly An important way to pursue personalized service. The DBSCAN algorithm is widely used in data mining. Using a density-based method, the high-density area is divided into clusters, and clusters of arbitrary shapes can be found while eliminating the influence of noise. However, the parallel implemen...

AI Technical Summary

Problems solved by technology

[0003] The main purpose of the present invention is to overcome the low correlation between the data that solves the traditional DBSCAN algorithm of prior art and needs O(N 2 ) time complexity implementation algorithm, and it cannot be applied to the same data set with different cluster density requirements and the situation where two clusters with low correlation are associated together due to linear connection. A DBSCAN block optimization based on pruning method is proposed method

Images