Asymmetric GM multi-modal fusion saliency detection method and system based on CWAM

Abstract

The invention discloses an asymmetric GM multi-modal fusion saliency detection method and system based on CWAM, and the method comprises the steps: collecting image data for preprocessing, and forminga sample data set; constructing a convolutional neural network model based on a deep learning strategy, and inputting the sample data set for training to obtain a saliency detection graph; forming aset by the trained saliency detection images, and calculating a loss function value between the set and the corresponding real human eye annotation image set to obtain an optimal weight vector and anoptimal bias term; and inputting a to-be-detected image into the trained convolutional neural network model, and performing prediction judgment by using the optimal weight vector and the optimal biasterm to obtain a saliency detection image of the image. According to the method, multi-scale and multi-level rich image information of the depth map and the RGB image can be effectively utilized, andthe problem of dissolution when high-level features are transmitted to a low level is effectively solved; and after a channel attention module is added, the expression of the salient region is enhanced.

Description

technical field [0001] The invention relates to the technical field of visual saliency detection of deep learning, in particular to a CWAM-based asymmetric GM multimodal fusion saliency detection method and system. Background technique [0002] When looking for objects of interest in images, humans can automatically capture semantic information between objects and their contexts, pay high attention to salient objects, and selectively suppress unimportant factors. This precise mechanism of visual attention has been explained in various biological logic models. The goal of saliency detection is to automatically detect the most informative and attractive parts of an image. In many image applications, such as image quality assessment, semantic segmentation, image recognition, etc., identifying salient objects can not only reduce the computational cost, but also improve the performance of saliency models. Early saliency detection methods used manual features, that is, mainly fo...

AI Technical Summary

Problems solved by technology

Most of the existing saliency detection methods have adopted the method of deep learning, using the method of combining the convolution layer and the pooling layer to extract image features, but the image features obtained by simply using the convolution operation and the pooling operation are not representative. characteristics, especially the pooling operation will lose the feature information of the image, which will lead to poor effect of the obtained saliency prediction map and low prediction accuracy

Images