Unsupervised learning method based on neural cluster

Abstract

The invention relates to an unsupervised learning method based on a neural cluster. An asymptotic formula of mutual information is used, a neural cluster network model based on a mutual information maximization principle and a learning theory are developed and applied to unsupervised representation learning, a novel unsupervised learning objective function based on mutual information maximizationis developed, and a method for efficiently and robustly solving and optimizing a target function is provided, so that the problems of difficulty in training, low convergence rate and the like in unsupervised representation learning in the fields of image processing and the like are solved, and the performance, efficiency and robustness of the unsupervised representation learning in the fields of image processing and the like can be greatly improved. The method not only can process conditions of a complete basis and an undercomplete basis in signal representation, but also can process conditions of an overcomplete basis, which are difficult to achieve by other methods at present. According to the method, the learning and training efficiency and performance in the fields of image recognitionand the like can be remarkably improved while the calculation complexity is not obviously increased, and the method has an important research significance and wide engineering practical values.

Description

technical field [0001] The invention relates to the technical field of machine learning and signal processing, in particular to an unsupervised learning method based on neural clusters. Background technique [0002] How to discover the unknown structure in the data is the key to machine learning. It is very important to learn a good feature representation from the observation data, because this clearer feature description can help reveal the underlying structure of the data. Especially in recent years, representation learning has received a lot of attention. One class of algorithms for unsupervised representation learning is based on probabilistic models, such as maximum likelihood estimation, maximum a posteriori probability estimation, and related methods. Another class of algorithms is based on reconstruction errors or generation criteria. The objective function usually contains squared errors with additional constraints. Sometimes reconstruction errors or generation cri...

AI Technical Summary

Problems solved by technology

However, since the mutual information is notoriously difficult to calculate, directly using the mutual information formula to define the objective function will encounter great obstacles in its calculation and optimization in most cases

Images