Video group behavior recognition method based on cascade Transformer

Abstract

The invention relates to the field of computer vision and deep learning, in particular to a video group behavior recognition method based on cascade Transformer, which comprises the following steps: firstly, acquiring and generating a video data set, extracting three-dimensional spatial-temporal features from the video data set through a three-dimensional backbone network, and selecting a key frame image spatial feature map; preprocessing the key frame image spatial feature map, sending the preprocessed key frame image spatial feature map into a human body target detection Transformer, and outputting a human body target box in the key frame image; then, mapping a sub-feature map corresponding to the screened human body target box on the key frame image feature map, calculating query / key / value in combination with a key frame image surrounding frame feature map, inputting the query / key / value into a group behavior recognition Transfomer, and outputting a group level spatial-temporal coding feature map; and finally, classifying group behaviors through a multi-layer perceptron. The method has the effect of effectively improving the group behavior recognition accuracy.

Description

technical field [0001] The invention relates to the fields of computer vision and deep learning, in particular to a video group behavior recognition method based on cascade Transformer. Background technique [0002] Nowadays, surveillance video has been widely used in social public places and plays an extremely important role in maintaining social public safety. Effective identification of abnormal behaviors and events in surveillance video can better play the role of surveillance video. Group behavior is the most common human behavior in videos. Group behavior recognition can effectively prevent dangerous events by automatically identifying group behavior in videos, and has a wide range of application values. [0003] In natural scenes, video group behavior recognition mainly faces two major challenges. One is that the scene is relatively complex, mainly manifested in the large scale change of the human body, background lighting, mutual occlusion between groups, etc., whi...

AI Technical Summary

Problems solved by technology

Its disadvantage is that when the graph convolutional network extracts the spatial features of a single-frame group, it does not highlight the discriminative individual features in the group, and only performs simple weighted fusion in the video time feature dimension, which cannot extract video timing features well.

Images