A Text Incremental Dimensionality Reduction Method Based on Tensor Decomposition

Abstract

The invention relates to a text incremental dimensionality reduction method based on tensor decomposition, by dividing text data into multiple subsets and constructing a text feature map cluster for each subset, expressing it as a second-order tensor, and then combining multiple A second-order tensor plus a feature dimension forms a third-order tensor and decomposes the third-order tensor. According to the decomposed relationship matrix, it can be obtained which feature words and feature word relationships the dimensionality-reduced text features are composed of. Achieving the goal of incremental text dimensionality reduction. Compared with the prior art, the present invention has the advantages of efficient dimensionality reduction, simplicity and precision, applicable to a large amount of data, and the like.

Description

technical field [0001] The invention relates to the fields of machine learning and natural language information processing, in particular to a text incremental dimensionality reduction method based on tensor decomposition. Background technique [0002] With the development of information technologies such as the Internet, the Internet of Things, and cloud computing, data resources in cyberspace are growing and accumulating at an unprecedented rate, and the world has entered the era of networked big data. In addition to the massive quantitative characteristics of data volume, big data also has complex characteristics such as discretization, diversification, and unstructured data attributes, which leads to the outbreak of data "dimension disaster", and the result will seriously affect data analysis. and decision support accuracy and efficiency. In order to make better use of the data, it is necessary to reduce the dimensionality of the data. Data dimensionality reduction is ...

AI Technical Summary

Problems solved by technology

Various existing data dimensionality reduction methods, such as principal component analysis, linear discriminant analysis, latent semantic analysis, etc., are mostly based on statistical theory, and are quite effective in dimensionality reduction of structured data, but ignore the semantics contained in the data , which often lead to serious deviations in dimensionality reduction results and low accuracy

Failure to study the problem of semantic preservation in dimensionality reduction will lead to dimensionality reduction results of semantic loss, which will be a fatal blow to big data applications

Images