Embedded attitude learning method capable of carrying out self supervision on the basis of video time-space relationship

Abstract

The invention puts forward an embedded attitude learning method capable of carrying out self supervision on the basis of a video time-space relationship. The average value of two auxiliary tasks is used for learning the time-space relationship in a video: one time sequence task learns whether two specific figure images are near or not on an aspect of time, and one spatial arrangement task learns ahuman body appearance model from space to enhance a capability of separating a gesture from a background. On the basis of the learning and mining repeated gesture of a course, training is carried outfrom a simple sample, then, the simple sample is iteratively expanded to a harder sample, and meanwhile, inactive video parts are eliminated. Time-space embedding successfully learns the representative characteristics of human gestures in a self-supervision way. By use of the method, a time-space relationship training video is used for carrying out gesture embedding self-supervision learning, manual annotation is not required, cost is lowered, gesture embedding can capture the visual characteristics of the human gestures, and human gesture estimation and retrieval efficiency can be improved.

Description

technical field [0001] The invention relates to the field of video gesture analysis, in particular to a self-supervised embedding gesture learning method based on the temporal-spatial relationship of the video. Background technique [0002] The ability to recognize human poses is essential for describing actions, and different poses in videos form a visual vocabulary similar to text. In computer vision processing systems, finding similar poses in different videos automatically enables many different applications, such as action recognition or video content retrieval. As an emerging topic, posture analysis has practical development in many fields, such as image search, behavior classification, security monitoring, especially in driverless driving in the transportation field, action recognition in smart homes, and human detection in medical diagnosis. Computer interaction and so on have broad application prospects. According to the needs proposed by video embedding poses, ca...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the problem of high cost and time-consuming, the present invention uses spatio-temporal relationship training video to conduct self-supervised learning of pose embedding, without human annotation, reducing costs, pose embedding can capture the visual features of human poses, and improve the efficiency of human pose estimation and retrieval

Images