A video-oriented three-stream human motion behavior spatial domain detection method

Abstract

The invention relates to a video-oriented three-stream human motion behavior spatial domain detection method, comprising the following steps: an input preparation step of obtaining an input video, obtaining a corresponding optical flow and a human body semantic partition diagram according to an original image to form a three-stream input; a behavior detection step including detecting,at each timestep, that RGB stream, the flow stream and the pose stream by detectors on respective stream to obtain a detection result, wherein the detection result comprises a classification score and a detectiontubular regression value; a three-stream fusion step of fusing the classification scores on the three streams with unequal weights to obtain fusion scores, taking the detection tubular regression value of the original image as a fusion regression value, and combining the fusion score and the fusion regression value to obtain the behavior detection tubular of each time step; a behavior tube construction step connecting the behavior detection tubes at each time step to construct a final video behavior detection tube. Compared with the prior art, the invention has the advantages of high detection speed and high detection accuracy.

Description

technical field [0001] The invention relates to the field of behavior detection, in particular to a video-oriented three-stream human motion behavior space domain detection method. Background technique [0002] With the rapid development of multimedia technology, video data is growing explosively. Taking Youtube as an example, as of 2018, the total length of videos uploaded on this website in just one minute is as high as more than 500 hours. How to understand and analyze such a huge amount of data, namely Video understanding has become a research topic of increasing interest. As one of the important means of video understanding, spatial domain behavior detection mainly solves the tasks of classifying the behaviors that appear in the video and locating the specific location where the behavior occurs on a specific frame of the video. This task still faces many challenges, such as few suitable training datasets, diverse intra-class variations, noisy video background, and vide...

AI Technical Summary

Problems solved by technology

Most behavior detection methods treat the video as frame by frame. This type of method is confusing for some behaviors to be detected. For example, it is difficult to judge whether a bowing person is standing up or sitting based on one frame alone. Therefore, some methods also use the method of processing multiple frames at a time for behavior detection. However, because these methods do not fully mine video behavior information, there is still room for improvement in detection performance.

Images