Scene graph generation method based on context graph attention mechanism

Abstract

The invention relates to a visual relation detection technology in the field of computer vision, and discloses a scene graph generation method based on a context graph attention mechanism, which improves the accuracy of scene graph generation by fully mining external knowledge and context information of a target. The method comprises the following steps: fusing an external knowledge vector, a spatial feature and a visual feature of a target through a context to obtain a fused feature vector; initializing a graph attention network according to the adjacent matrix of the target in combination with the fused feature vector; calculating a frequency coefficient of a target relationship by using statistical information in the sample data set, and calculating a graph attention coefficient by using a target context feature; obtaining the final vector representation of the targets through information iteration of the graph attention network, calculating the relation between the targets, performing gradient descent updating through the relation between the targets and the loss function of the targets, and therefore, a visual relation detection model is generated; and for a to-be-detected image, generating a scene graph according to the visual relation detection model.

Description

technical field [0001] The invention relates to a visual relationship detection technology in the field of computer vision, in particular to a method for generating a scene graph based on a context graph attention mechanism. Background technique [0002] Scene graph is a structured representation of image content, which is a graph structure with image objects as vertices and relationships between objects as connecting edges. The scene graph not only encodes the semantic and spatial information of each object in the scene, but also represents the relationship between each pair of objects, as an abstract representation of objects and their pairwise relationship, containing higher-level scene understanding knowledge. Although some achievements have been made in target detection using deep learning technology, it is still a challenging task to deduce the structured representation of images from visual data, so the research on related technologies of scene graph generation has gr...

AI Technical Summary

Problems solved by technology

[0006] KERN models the relationship detection by combining statistical information between targets, but the model only initializes the graph structure through the probability information in the statistical information, but does not fully exploit the semantic information in these statistical information, so it cannot be better alleviated. The problem of unbalanced sample distribution in the data set

Images