Multi-modal deep leaning classification method based on semi supervision

Abstract

While deep learning is used for classification, multi-modal information with rich samples and classification contribution variability of each modality are considered, and the problem of insufficient samples is solved by using a semi supervision method. Data of different modalities of a hyperspectral image is sent into a deep neural network, the semi supervision method is used and a large number ofunlabeled samples are utilized, and the deep neural network based on self-encoding is used for feature learning. All labeled and unlabeled data are sent into the self-encoding deep neural network tocarry out learning, similar networks are designed for different modalities, a respective initialization parameter is obtained through self-encoding reconstruction, and hidden attributive classification of labeled samples is obtained through a clustering method. For the unlabeled data, a deep characteristic is calculated through a multi-target deep network, then a similar marked sample is searchedbased on a clustering label, and finally, labels of the unlabeled samples are predicted according to the label information of the labeled samples.

Description

technical field [0001] The present invention relates to a semi-supervised method based on semi-supervised multi-modal deep learning to solve the problem of insufficient samples while considering the rich multi-modal information of samples and the differences in the classification contributions of each modality while using deep learning to classify. Learn how to classify. Background technique [0002] Hyperspectral remote sensing images have multiple modal information such as texture, spatial correlation, and spectrum. The fusion of so many aspects of feature information enables hyperspectral remote sensing to detect more ground object information, which greatly improves human understanding of the objective world. cognitive ability. [0003] At present, more researchers directly use deep models to fuse part of the modal information, and fewer researchers construct deep network architectures for the differences in the classification contribution of modalities. In addition, t...

AI Technical Summary

Problems solved by technology

[0003] At present, more researchers directly use deep models to fuse part of the modal information, and fewer researchers construct deep network architectures based on the differences in modal classification contributions.

In addition, the classification performance of images mainly depends on a large number of training samples, but in reality there are often insufficient labeled samples

Images