Mass spatial data density clustering method based on elastic distribution dataset

Abstract

The invention relates to a mass spatial data density clustering method based on an elastic distribution dataset. According to the method, first, automatic meshing and data distribution are performed according to distribution of data in space based on the design ideology of "RDD partition--intra-partition parallel computing--local result merging" targeting quick mining of an aggregation characteristic base of large-scale spatial data, so that data volumes in meshes are relatively balanced, and the purpose of balancing arithmetic node loads is achieved; second, a local density definition suitable for parallel computing is proposed, a computing mode of a cluster center is improved, and the defect that a cluster center object needs to be judged by drawing a decision graph through an original algorithm is overcome; and last, quick clustering processing of the large-scale spatial data is realized through clustering, merging and other optimization strategies in the meshes and between the meshes. Through the method, quick clustering of the large-scale spatial data can be effectively realized, and the method has high precision and better system processing performance compared with traditional density clustering methods.

Description

technical field [0001] The invention relates to a mobile device, in particular to a mass spatial data density clustering method based on an elastic distribution data set. Background technique [0002] Cluster analysis plays an important role in spatial data mining. Spatial clustering analysis divides spatial data into several clusters according to their aggregation characteristics, so that the data in the same cluster have greater similarity, while the data in different clusters have greater differences. According to different guiding ideologies, clustering algorithms can be divided into partition-based clustering, hierarchical-based clustering, density-based clustering, grid-based clustering, and specific model-based clustering. The classic partitioning algorithm k-means and its improved algorithm k-medoids, k-means++ determine the cluster center and classify the data through multiple iterations. The algorithm is simple to implement, but it is sensitive to noise and handle...

AI Technical Summary

Problems solved by technology

Due to the pursuit of high throughput, the parallel clustering algorithm based on the Hadoop-MapReduce framework needs to read and write the disk multiple times to access the intermediate results, resulting in a large I / O overhead and high latency of the algorithm, which cannot be used for real-time clustering

Images