Real-time three-dimensional gesture estimation method based on single RGB frame

Abstract

The invention provides a real-time three-dimensional gesture estimation method based on a single RGB frame. The method mainly includes 2D joint estimation and 2D joint to 3D posture, and the process is as follows: firstly a color image is taken as input, a hand detector is used to position the hand and generate a likelihood estimated value and coordinates of a hand bounding box, and a pre-trainingnetwork runs to estimate the two-dimensional position of the hand joint and generate a heat map of the hand 2D joint position, and finally nonlinear square minimization is used to fit a 3D model of the hand to the estimated 2D joint position, to restore the 3D hand posture. According to the real-time three-dimensional gesture estimation method, the problem that the hand is quickly moved, self-blocked or obstructed from other objects during operation is solved, initialization is not required, the high-quality detection rate is achieved, and the method can also be extended to other human three-dimensional posture detection applications.

Description

technical field [0001] The invention relates to the field of gesture estimation, in particular to a real-time three-dimensional gesture estimation method based on a single RGB frame. Background technique [0002] As one of the important parts of the human body, the human hand is very important to people in various aspects such as real life, work and study. Human hands are not only able to flexibly complete operational tasks in terms of execution, but also play an auxiliary role in interpersonal communication, such as waving, saluting, inviting, holding hands, etc., and can also be used as sense organs to achieve cognitive purposes. The interactive interface based on natural human hands abandons traditional interactive devices such as keyboards and mice, and uses cameras, data gloves and other equipment as interactive tools to control with free and flexible human hands, which meets the requirements of human-centered modern human-computer interaction. Gesture recognition and ...

AI Technical Summary

Problems solved by technology

[0004] Aiming at problems such as being easily affected by rapid hand movement, self-occlusion or occlusion by other objects during operation, the purpose of the present invention is to provide a real-time 3D gesture estimation method based on a single RGB frame. As input, the hand detector is then used to locate the hand and produce a likelihood estimate and the coordinates of the bounding box of the hand, then run the pre-trained network to estimate the 2D position of the hand joints and generate hotspots for the 2D joint positions of the hand Figure, the final nonlinear square minimization fits the 3D model of the hand to the estimated 2D joint position and restores the 3D hand pose

Images