A compressed domain video action recognition method and system based on new spatio-temporal feature flow

Abstract

The invention discloses a compressed domain video action recognition method using a new spatiotemporal feature stream, comprising: extracting I frame data in a compressed video sequence, motion vector data and residual data in a P frame, and combining the motion vector data and Residual data are preprocessed. The dual-channel data of the preprocessed motion vector and the single-channel data of the preprocessed residual are respectively used as R / G / B channel fusion to construct a new spatio-temporal feature image and input into the CNN convolutional neural network model for training and testing to obtain motion recognition category scores. The extracted I-frame data and the preprocessed motion vector data are respectively input into the CNN convolutional neural network model for training and testing to obtain the recognition category score of the action, and finally fuse the three data in a ratio of 2:1:1. The recognition category score of the action gets the final action recognition result. The invention can solve the technical problems of low recognition accuracy and complicated calculation process existing in the existing video action recognition method.

Description

technical field [0001] The invention belongs to the technical field of deep learning and pattern recognition, and more specifically, relates to a compressed domain video action recognition method and system based on new spatiotemporal feature streams. Background technique [0002] With the continuous increase of people's demand for artificial intelligence, video action recognition technology has become an important problem in computer vision, which has widely promoted the development of artificial intelligence. [0003] The existing video action recognition methods mainly include the following four types. The first is video action recognition based on artificial features, which mainly extracts and tracks the features of each pixel in the optical flow densely, and classifies them after encoding; the second One is a dual-stream neural network, which divides the video into two parts, space and time, and sends RGB images and optical flow images to two neural networks, and fuses ...

AI Technical Summary

Problems solved by technology

[0004] However, the above-mentioned video action recognition methods have some technical problems that cannot be ignored. For the above-mentioned first method, it has a certain effect on small data sets and specific actions, but when dealing with large-scale data sets, the dense trajectory Features lack certain flexibility and scalability, so more real-time and effective classification cannot be achieved; in the second method above, if optical flow is not used, its recognition accuracy is not high, and if optical flow is used, its calculation The overhead is very high; in the above third method, the data calculation amount of the three-dimensional convolutional neural network is much larger than that of the two-dimensional convolutional neural network, thus greatly occupying the computing resources; the above fourth method is for a specific Actions have recognition effects, but their universality is too low, and they cannot deal with scale changes, so they cannot meet the basic requirements of correctly identifying multiple actions with a certain recognition accuracy guarantee.

Images