High-dimensional data visualization method based on probability multi-level graph structure

Abstract

The invention relates to a high-dimensional data visualization method based on a probability multi-level graph structure, and belongs to the technical field of data visualization and dimension reduction. The high-dimensional data visualization method comprises the steps of: (1) giving a high-dimensional data set, wherein the data set comprises n data points, and the dimension of each data point isD; 2) calculating k neighbors of each data point, constructing a nearest neighbor graph structure G0, and constructing a probability multi-level graph structure based on the graph structure G0 to obtain a probability multi-level graph structure set; 3) laying out probability multi-level graphs layer by layer based on the probability multi-level graph structure set to obtain data low-dimensional representation, wherein the dimension of each data point is two-dimensional or three-dimensional; and 4) constructing a scatter view based on the low-dimensional data for data mining and analysis. According to the high-dimensional data visualization method, an optimization calculation process is accelerated by utilizing a hierarchical graph structure, and a visualization effect is optimized by introducing probability-based sampling.

Description

technical field [0001] The invention relates to the technical field of data visualization and dimensionality reduction, in particular to a high-dimensional data visualization method based on a probabilistic multi-level graph structure. Background technique [0002] High-dimensional data visualization is an important task in data analysis, and plays a vital role in deep learning, life science and network analysis. Dimensionality reduction algorithms learn complex information in data, transform high-dimensional data into low-dimensional data, and analyze the distribution of data. [0003] Over the past few decades, a large number of visualization methods for high-dimensional data have been proposed. The t-SNE algorithm is one of the most successful dimensionality reduction algorithms. The invention patent application document with the publication number CN110458187A discloses a malicious code family clustering method and system, wherein the method includes using the T-SNE alg...

AI Technical Summary

Problems solved by technology

[0004] The t-SNE algorithm is often used to visualize high-dimensional data with an inherently nonlinear structure. However, t-SNE is difficult to deal with growing large-scale data due to the quadratic relationship between the computational complexity and the number of data points.

Although BH-SNE, LargeVis and other algorithms obtain smaller algorithm complexity by constructing the nearest neighbor network and negative sampling technology, these methods still face two main problems when applied to large-scale data: 1) The visualization effect is often not enough to impress Satisfactory; 2) The optimization process is still time-consuming

Images