Human body behavior recognition system based on graph convolutional neural network

Abstract

The invention discloses a human body behavior recognition system based on a graph convolutional neural network, and relates to the technical field of human body behavior recognition. The method comprises the following steps: 1, constructing an undirected space-time skeleton diagram for human joint data collected by a depth sensor, and taking the undirected space-time skeleton diagram as an input signal of space-time diagram convolution; 2, inputting the constructed skeleton graph into a space-time graph convolutional network for action feature extraction to realize human body action recognition; 3, embedding the gating unit recurrent neural network into space-time diagram convolution to optimize the network, and better realizing synchronous extraction of space domain features and time domain features; 4, realizing man-machine interaction in a virtual environment by utilizing 3D modeling software. According to the method, human skeleton action information collected in an NTU RGB + D data set is utilized, a space-time skeleton graph network structure is constructed for time sequence representation of human skeleton joint point positions and a space cooperation relation, and end-to-end human skeleton action recognition based on a space-time diagram convolutional neural network is achieved.

Description

technical field [0001] The invention belongs to the technical field of human behavior recognition, in particular to a human behavior recognition system based on a graph convolutional neural network. Background technique [0002] Human action recognition covers many research topics in computer vision, including human detection in videos, pose estimation and tracking, and analysis and understanding of action sequences. Human action recognition is an important research area due to its wide range of applications, such as patient monitoring, motion analysis, intelligent video surveillance, and human-computer interaction. Traditional human action recognition is mainly based on RGB video, but RGB video has disadvantages such as large amount of calculation, easy to be affected by lighting conditions, and sensitive to background noise. Thanks to the maturity of human body detection algorithms, more and more researches focus on bone-based human action recognition. The human body can...

AI Technical Summary

Problems solved by technology

[0003] Since the skeleton information is represented by the three-dimensional coordinate points of several joints, it is impossible to directly use the traditional CNN for deep feature extraction. The existing CNN method generally converts the skeleton data into an image, and maps the spatial and temporal information on the image texture, Color and other attributes, and then feature extraction, but this process will cause the loss of some key information

Images