Text increment dimension reduction method based on tensor decomposition

Abstract

The invention relates to a text increment dimension reduction method based on tensor decomposition. The method comprises the following steps: dividing text data into a plurality of subsets, and constructing a text feature graph cluster for each subset; expressing the second-order tensors as second-order tensors, adding feature dimensions to the second-order tensors to form third-order tensors, decomposing the third-order tensors, and obtaining which feature words and feature word relations the text features of which are subjected to dimensionality reduction according to a decomposed relation matrix, thereby realizing the target of incremental text dimensionality reduction. Compared with the prior art, the method has the advantages of high-efficiency dimension reduction, simplicity, accuracy, suitability for mass data and the like.

Description

technical field [0001] The invention relates to the fields of machine learning and natural language information processing, in particular to a text incremental dimensionality reduction method based on tensor decomposition. Background technique [0002] With the development of information technologies such as the Internet, the Internet of Things, and cloud computing, data resources in cyberspace are growing and accumulating at an unprecedented rate, and the world has entered the era of networked big data. In addition to the massive quantitative characteristics of data volume, big data also has complex characteristics such as discretization, diversification, and unstructured data attributes, which leads to the outbreak of data "dimension disaster", and the result will seriously affect data analysis. and decision support accuracy and efficiency. In order to make better use of the data, it is necessary to reduce the dimensionality of the data. Data dimensionality reduction is ...

AI Technical Summary

Problems solved by technology

Various existing data dimensionality reduction methods, such as principal component analysis, linear discriminant analysis, latent semantic analysis, etc., are mostly based on statistical theory, and are quite effective in dimensionality reduction of structured data, but ignore the semantics contained in the data , which often lead to serious deviations in dimensionality reduction results and low accuracy

Failure to study the problem of semantic preservation in dimensionality reduction will lead to dimensionality reduction results of semantic loss, which will be a fatal blow to big data applications

Images