Human behavior recognition method integrating space-time dual-network flow and attention mechanism

Abstract

The invention discloses a human behavior recognition method integrating the space-time dual-network flow and an attention mechanism. The method includes the steps of extracting moving optical flow features and generating an optical flow feature image; constructing independent time flow and spatial flow networks to generate two segments of high-level semantic feature sequences with a significant structural property; decoding the high-level semantic feature sequence of the time flow, outputting a time flow visual feature descriptor, outputting an attention saliency feature sequence, and meanwhile outputting a spatial flow visual feature descriptor and the label probability distribution of each frame of a video window; calculating an attention confidence scoring coefficient per frame time dimension, weighting the label probability distribution of each frame of the video window of the spatial flow, and selecting a key frame of the video window; and using a softmax classifier decision to recognize the human behavior action category of the video window. Compared with the prior art, the method of the invention can effectively focus on the key frame of the appearance image in the originalvideo, and at the same time, can select and obtain the spatial saliency region features of the key frame with high recognition accuracy.

Description

technical field [0001] The invention relates to an early human behavior recognition method, in particular to a human behavior recognition method that integrates a spatio-temporal dual network flow and an attention mechanism, and belongs to the technical field of computer vision behavior recognition. Background technique [0002] Human behavior recognition in video sequences is a research topic involving computer vision, pattern recognition and artificial intelligence. It has been a hot research topic because of its wide application value in commercial, medical and military fields. However, due to the diversity and non-rigidity of human behavior and the inherent complexity of video images, it is still a challenging task to propose a robust and real-time accurate method. [0003] At present, most of the research starts from the three aspects of moving target detection, action feature extraction and action feature understanding. The general human action recognition method basic...

AI Technical Summary

Problems solved by technology

However, the simple introduction of the attention mechanism fails to effectively identify the significant parts of the behavior and classify the human behavior in the most distinctive area.

Images