Fine-grained action detection method of convolutional neural network based on multistage condition influence

Abstract

The invention discloses a fine-grained action detection method of a convolutional neural network based on multistage condition influence. The method comprises the steps of: establishing a convolutional neural network influenced by multistage conditions; fusing the explicit knowledge added in the visual scene with the multi-level visual features; enabling the multi-level conditional influence convolutional neural network MLCNet to take a conditional influence multi-branch convolutional neural network structure as a main trunk, generating multi-level visual features, encoding additional spatialsemantic information of human body structure and object context information as a condition, dynamically influencing feature extraction of a CNN through affine transformation and an attention mechanism, and finally fusing and modulating multi-mode features to distinguish various interactive actions; and carrying out model training on the convolutional neural network influenced by the multi-level condition, and outputting a fine-grained action detection result by the obtained model. According to the method, the proposed method is evaluated on the basis of two most common references, namely HICO-DET and V-COCO, and experimental results show that the method is superior to the existing method.

Description

technical field [0001] The invention belongs to the image information retrieval in the field of computer technology, relates to the detection of the relationship between objects in the image, and is used for locating and classifying the person-object pair in the image and the interaction relationship between them. It is a volume based on multi-level condition influence A fine-grained action detection method based on product neural network. Background technique [0002] Fine-grained action detection aims to locate and classify person-object pairs in images and the relationship between them, which can be used in many multimedia applications such as image captioning and retrieval. In some contexts, action recognition and Human Centered Visual Relationship Detection (HCVRD) are considered similar to fine-grained action detection, but they have substantial differences. Action recognition focuses on classifying individual actions in images or video clips without considering inter...

AI Technical Summary

Problems solved by technology

[0021] The problem to be solved by the present invention is: capture the high-level semantic information of a single entity from a complex visual scene, locate and classify the person-object pairs in the image and the relationship between them

Images