Human skeleton behavior recognition method and device based on deep reinforcement learning

Abstract

The invention discloses a human skeleton behavior recognition method and device based on deep reinforcement learning. The method comprises: uniform sampling is carried out on each video segment in a training set to obtain a video with a fixed frame number, thereby training a graphic convolutional neural network; after parameter fixation of the graphic convolutional neural network, an extraction frame network is trained by using the graphic convolutional neural network to obtain a representative frame meeting a preset condition; the graphic convolutional neural network is updated by using the representative frame meeting the preset condition; a target video is obtained and uniform sampling is carried out on the target video, so that a frame obtained by sampling is sent to the extraction frame network to obtain a key frame; and the key frame is sent to the updated graphic convolutional neural network to obtain a final type of the behavior. Therefore, the discriminability of the selectedframe is enhanced; redundant information is removed; the recognition performance is improved; and the calculation amount at the test phase is reduced. Besides, with full utilization of the topologicalrelationship of the human skeletons, the performance of the behavior recognition is improved.

Description

technical field [0001] The invention relates to the technical fields of computer vision, reinforcement learning and deep learning, in particular to a human skeleton behavior recognition method and device based on deep reinforcement learning. Background technique [0002] Action recognition aims to distinguish the action category in a given video, and is an important research direction in computer vision. Behavior recognition has a wide range of applications, such as video surveillance, human-robot interaction, etc. Compared with the traditional color video, the skeleton-based video contains the 3D position of the key bones of the human body, and it has higher robustness to the transformation of the viewing angle, the scale of the human body and the speed of motion. Moreover, with the development of depth sensors (such as Kinect) and the maturity of human pose estimation algorithms, data based on human skeletons is increasing day by day. Therefore, the research on behavior ...

AI Technical Summary

Problems solved by technology

However, most CNN-based methods consider all frames to be equally important, thus ignoring the most critical frames in the video

Images