Video action recognition method and system based on hybrid convolution multi-level feature fusion model

Abstract

The invention relates to a video action recognition method and system based on a multi-level feature fusion model of hybrid convolution, which belongs to the field of computer vision technology, and adopts two-dimensional convolution and separable three-dimensional convolution to construct a hybrid convolution module; along the time dimension Perform a channel shift operation on each input feature, construct a time shift module, promote the information flow between adjacent frames, and compensate for the defect of two-dimensional convolution operation in capturing dynamic features; derived from different convolution layers of the backbone network Multi-level complementary features are modulated spatially and temporally, so that the features at each level have consistent semantic information in the spatial dimension and variable visual rhythm clues in the temporal dimension; by constructing bottom-up features Flow and top-down feature flow, so that each feature complements and complements each other, and the feature flow is processed in parallel to achieve multi-level feature fusion; model training is performed using a two-stage training strategy.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and relates to a video action recognition method and system based on a hybrid convolution multi-level feature fusion model. Background technique [0002] The rapid development of the field of artificial intelligence research has prompted human-computer interaction technology to gradually penetrate into people's daily life, and the research on human action recognition derived from it has received extensive attention. In video-based action recognition tasks, traditional methods mainly rely on specific feature design, which has severe domain limitations. In order to overcome the above defects and obtain a more general feature representation, Convolutional Neural Networks (CNN) based on biological visual perception mechanisms have been widely used in the field of action recognition. [0003] The performance of the model on human action recognition is closely related to its ability to represe...

AI Technical Summary

Problems solved by technology

Although pre-defining visual rhythm changes at the input level can significantly improve the model recognition effect, the model complexity increases significantly due to the parameter training involving multiple network branches

Images