A Graph-Based Saliency Fusion Method for 3D Gaze Prediction

Abstract

The invention discloses a saliency fusion method for 3D gaze point prediction based on a graph, which includes generation of a saliency map and fusion based on a graph. The generation of the saliency map includes obtaining a saliency map of each frame of an original picture from an original video sequence; The graph-based fusion includes: taking the minimum saliency smoothing constraint between each superpixel in the original picture and its adjacent superpixels, and the minimum saliency difference between the original picture and its adjacent original pictures as the goal , combined with the saliency map to construct the energy function of the original image; solve the energy function in the original image to obtain the target saliency map. The present invention considers the saliency smoothing constraints between superpixels and their adjacent superpixels, as well as the saliency difference between the original picture and its adjacent original pictures, so that the saliency fusion method of the present invention can be used in the process of multimodal feature fusion The different modal features in the predictive saliency are better.

Description

technical field [0001] The invention belongs to the fields of image processing and computer vision, and more particularly relates to a saliency fusion method for 3D gaze point prediction based on graphs. Background technique [0002] In the field of visual attention, there are already quite a few models for 2D visual attention, which can be roughly divided into two categories: human gaze point prediction models and salient object detection models. The former computes salient intensity maps at the pixel scale, and the latter aims to detect and segment salient objects or regions in a scene. There are quite a lot of visual attention models for human eye gaze prediction, but the research on gaze prediction models for 3D videos has just begun in recent years. In a nutshell, the framework of most 3D gaze prediction models is extended from 2D gaze prediction models. The framework mainly includes two steps. The first step is to extract a series of feature maps from the original co...

AI Technical Summary

Problems solved by technology

[0010] In view of the above defects or improvement needs of the prior art, the present invention provides a saliency fusion method based on graph-based 3D gaze point prediction, thereby solving the problem of different modal feature predictions in the process of multi-modal feature fusion in the prior art Significantly inconsistent or even contradictory technical issues

Images