Double-flow convolution behavior recognition method based on 3D time flow and parallel spatial flow

Abstract

The invention discloses a double-flow convolution behavior recognition method based on 3D time flow and parallel spatial flow, and the method comprises the following steps: firstly carrying out the optical flow block extraction of an input video; secondly, segmenting an input video, extracting a video frame, and cutting out a human body part; inputting the optical flow block into a 3D convolutional neural network, and inputting the clipped frame into a parallel spatial flow convolutional network; finally, fusing the classification results of the parallel spatial streams and splicing with the scores of the time streams to form a full connection layer, and finally outputing an identification result through an output layer. According to the invention, the human body part cutting and the parallel spatial flow network are utilized to carry out single-frame identification, the single-frame identification accuracy is improved in space, and the 3D convolutional neural network is utilized to carry out action feature extraction of the optical flow, so that the identification accuracy of the time flow part is improved, and decision fusion is carried out by using the final single-layer neuralnetwork in combination with the spatial appearance features and the time action features, so that the overall recognition effect is improved.

Description

technical field [0001] The invention relates to the technical field of human behavior recognition, in particular to a dual-stream convolution behavior recognition method based on 3D time stream and parallel space stream. Background technique [0002] With the development of Internet multimedia, especially the rapid commercialization of 5G technology, large-scale cameras generate and transmit a large number of videos every moment, which will put enormous pressure on public security monitoring. In order to cope with the information explosion, it is very necessary and urgent to analyze and process the video. Human action recognition in videos is an important branch of computer vision and is crucial for public safety analysis and smart city construction. [0003] Before the convolutional neural network was proposed in 2012, the video behavior recognition algorithm was mainly based on traditional algorithms. Among them, the improved dense optical flow method achieved the best re...

AI Technical Summary

Problems solved by technology

However, this method needs to extract the optical flow in advance, so the storage and calculation costs of the optical flow image are too high, and the accuracy is not enough to apply to the actual scene, and the extraction of feature information needs to be improved

In addition, factors such as light intensity and complex scenes in the video scene also affect the accuracy of the model to a certain extent.

Images