Random forest visualized data analysis method based on largeVis

Abstract

The invention relates to a random forest visualized data analysis method based on LargeVis. The random forest visualized data analysis method comprises the steps of preprocessing a training dataset; extracting important characteristics of the training dataset through a random forest; adopting LargeVis to reduce the dimension; based on the random forest of LargeVis, conducting visualized processing. By means of the random forest visualized data analysis method based on LargeVis, aiming at high-dimension data, through the characteristic importance trained by the random forest, new sub-high-dimension data is formed, and then through the data subjected to LargeVis dimension reduction, the sub-high-dimension data is sent into the random forest to be predicted and analyzed to form visualization,the classifying precision can be improved, the visualization time can also be prolonged, and meanwhile, the random forest visualized data analysis method adapts to different pieces of data.

Description

technical field [0001] The present invention relates to pattern recognition, machine learning, big data analysis, especially a random forest visualization data analysis method based on LargeVis. Background technique [0002] In the era of big data, the dimensionality of data features is getting higher and higher, and it is particularly important to analyze data through a method of dimensionality reduction. At the same time, how to visualize high-dimensional data is also the focus of research in the current environment. At present, the most classic dimensionality reduction method is PCA (Principal Component Analysis). It not only reduces the dimensionality of high-dimensional data, but more importantly, removes noise through dimensionality reduction and discovers patterns in the data. PCA replaces the original n features with a smaller number of m features. The new features are linear combinations of the old features. These linear combinations maximize the sample variance and...

AI Technical Summary

Problems solved by technology

[0003] At present, the t-SNE algorithm in manifold learning has been widely used, but there are the following shortcomings: when dealing with large-scale high-dimensional data, the efficiency of t-SNE is significantly reduced (including the improved algorithm); the parameters in t-SNE It is sensitive to different data sets. After adjusting the parameters on one data set and getting a good visualization effect, it is found that it cannot be applied to another data set, and it takes a lot of time to find suitable parameters, which is important for the entire classification. The limitation of the model is very huge; pure original high-dimensional data directly enters the model training and classification through dimensionality reduction, with low accuracy and long training time

In addition, the current methods for data dimensionality reduction generally use the original data for dimensionality reduction and use existing models for classification, but this may have problems such as low accuracy and unexplainable data for dimensionality reduction

Images