Data clustering method and system, computer equipment and storage medium

Abstract

The invention discloses a data clustering method and system, computer equipment and a storage medium, and belongs to the field of data processing. The method comprises the steps of obtaining a first similarity matrix of original data, performing connected domain analysis on the first similarity matrix, obtaining a first clustering result in combination with distribution statistical analysis of similarity, and obtaining a relationship of potential to-be-merged categories in the first clustering result at the same time; according to the relationship of the potential to-be-merged categories, merging the to-be-merged categories in the first clustering result to obtain a second clustering result; and performing strong connected component analysis on the second clustering result, and performing category internal noise data cleaning on the second clustering result according to an analysis result to obtain a third clustering result. According to the technical scheme provided by the invention, the high-accuracy data set subjected to clustering processing can be quickly obtained from the original data and is used for model training.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data clustering method, system, computer equipment and storage medium. Background technique [0002] In recent years, deep learning has made impressive progress in various fields. In the field of artificial intelligence, training or tuning a model using deep learning requires the collection and processing of large amounts of data. Especially for specific scene tasks, in the absence of public data sets, often only the corresponding data sets can be obtained by collecting, preprocessing, and manual error correction. This process often consumes a lot of time and resources, which greatly limits the actual scenarios of deep learning applications. [0003] In a feasible existing technology, obtain the similarity matrix of the original data, select K data as the initial grouping, and then use the K-split block clustering algorithm to cluster the remaining data into the selected K In ...

AI Technical Summary

Problems solved by technology

However, when using the K-split block algorithm, it is difficult to determine the appropriate K value. In addition, when the K most irrelevant data are taken out from the data set, there may be "noise" data obtained, which will increase the number of K-split algorithm splits. leading to a decrease in clustering speed and efficiency

Images