Video human body behavior recognition method based on CNN and accumulated hidden layer state ConvLSTM

Abstract

The invention discloses a video human body behavior recognition method based on a CNN and an accumulated hidden layer state ConvLSTM, and the method comprises the steps: employing a 2D CNN which is pre-trained on an Image Net as a backbone model, and inserting an AH-ConvLSTM module behind a plurality of fixed network layer positions in the CNN to construct an overall network; acquiring a data sample set containing video data and labels, and dividing the data sample set into a training sample set and a test sample set; sampling a video in a fragment sampling mode, and using frames obtained through sampling as input to be sent to the whole network; setting a learning rate to train the whole network, and using the training sample set as training data; using a classifier for generating recognition scores of all categories, updating overall network parameters through back propagation, and storing the parameters as weight files; and initializing the whole network by using the weight file with the highest verification accuracy, performing fragment sampling on video frames of the test sample set, and inputting the frames obtained by sampling into the whole network to learn spatio-temporal information in the video so as to obtain an identification result.

Description

technical field [0001] The present invention relates to the field of human behavior recognition in videos, in particular to a convolutional long-short-term memory (Convolutional Long Short-term Memory, abbreviated as ConvLSTM) based on convolutional neural networks (Convolutional Neural Networks, abbreviated as CNN) and accumulated hidden layer states. ) video human behavior recognition method. Background technique [0002] Video action recognition is one of the representative tasks of video understanding. Thanks to the advent of deep learning, video action recognition has made great progress, but it has also encountered new challenges. Modeling remote temporal information in videos results in high computational cost and variability due to differences in datasets and evaluation protocols. One of the most important tasks in video understanding is to understand human behavior, which has many practical application scenarios, including behavior analysis, video retrieval, human...

AI Technical Summary

Problems solved by technology

CNN has a powerful feature extraction ability, and now CNN is very proficient in the application of various computer vision, but the two-dimensional convolutional neural network (referred to as 2D CNN) lacks the ability to model time information, and the three-dimensional convolutional neural network (referred to as 3D CNN), as a natural extension of 2D CNN, has the ability to model temporal and spatial information simultaneously, but it also brings too much computational cost

ConvLSTM, as a variant of Long Short-term Memory (LSTM for short), is very effective for data with sequence characteristics, and video data happens to have sequence characteristics, but in previous work, most of them used stacked ConvLSTM, which has difficulty in convergence, also brings an increase in computational cost, and ConvLSTM only pays attention to the state of the hidden layer at the previous moment, and does not pay attention to the hidden layer at a longer distance, resulting in data redundancy and reducing the utilization rate of data.

Images