First-visual-angle interactive action recognition method based on global and local network fusion

Abstract

The invention provides a first-visual-angle interactive action recognition method based on global and local network fusion. The method comprises: a video is sampled to obtain different actions to obtain images and thus an action sample is formed; dimension unification processing is carried out on an action segment obtained by sampling, data enhancement is carried out, a 3D convolutional network based on a global image as an input is trained, and global spatio-temporal features of the action are learned to obtain a network classification model; a local area with significance action occurrence in the action segment is located by using a sparse optical flow; after dimension unification processing of local areas of different actions, hyperparameters of the network are adjusted, a 3D convolutional network based on a global image as an input is trained, local significant action features are learned to obtain a network classification model; and action samples are obtained by multiple samplingon the same video, statistics and ranking of the prediction numbers provided by the global and local models are carried out based on a voting method, and the type with the largest prediction numbersis used as an identified action tag.

Description

technical field [0001] The present invention relates to an interactive action recognition and image processing technology, in particular to a first-view interactive action recognition method based on global and local network fusion. Background technique [0002] In recent years, with the development of portable devices, the popularization of head-mounted cameras has produced more and more first-person videos, which brings the need to analyze human behavior from the first perspective. The first-person view video brings a new perspective to capture social interaction and object interaction, but the long-term action and unstructured shooting scene brought by the normally-on mode of the head camera makes the action of the first-view video Parsing becomes challenging. The interactive action in the first perspective includes two types, one is the self-motion from the observer, and the other is the action from the interactor. Often, the interaction affects the observer, so there ...

AI Technical Summary

Problems solved by technology

This makes the traditional single classifier method of action recognition unable to obtain high-precision recognition effect, so it is necessary to finely analyze the action features based on the combination of global and local methods to obtain efficient representation

Images