Video behavior detection method and device, video behavior response method and device, equipment and medium

Abstract

The invention discloses a video behavior detection method and device, a video behavior response method and device, equipment and a medium. The detection method comprises the steps of extracting a space-time feature map from a video stream; pooling the space-time feature map into a one-dimensional feature map only retaining the time sequence features of the video stream, and evaluating a first probability score that a control region of each time sequence point in the one-dimensional feature map belongs to a target time sequence behavior, evaluating the offset of a candidate box in a time sequence point of the one-dimensional feature map and a second probability score of a target time sequence behavior contained in the candidate box in parallel; optimizing a plurality of candidate boxes according to the first probability score and the second probability score; pooling the candidate boxes into a plurality of feature maps, outputting the feature maps to a full connection layer, completingvideo behavior classification and regression by the full connection layer, and outputting classification result data of the video stream. According to the invention, the video behavior identificationaccuracy of the artificial neural network can be improved, and the video behavior online detection effect is significantly improved.

Description

technical field [0001] The present application relates to image recognition technology, in particular to a video behavior detection and response method and its corresponding neural network implementation device, electronic equipment, and non-volatile storage medium. Background technique [0002] The technology of video behavior recognition using neural network is becoming more and more mature. Nowadays, the more popular video behavior recognition method is realized by R-C3D algorithm. R-C3D is the abbreviation of Region Convolutional 3D Network for TemporalActivity Detection (regional convolutional 3D network for temporal activity detection). Make a proposal (proposal, aiming to provide a time series candidate box), then perform pooling (3D-pooling), and finally perform classification and regression operations. [0003] R-C3D mainly includes a feature extraction network, a time series candidate frame suggestion module, and a behavior recognition network. For the input video...

AI Technical Summary

Problems solved by technology

[0005] However, the R-C3D algorithm is implemented based on the anchor point regression strategy, and generates candidate boxes by assuming multiple multi-anchor point regions of equal length in time series. In this way, it is often impossible to effectively suppress the background time series fragments, which is not conducive to the behavior of variable time length. detection, so RC3D-like methods are usually slightly inferior to non-end-to-end methods on the same dataset

Images