Hand posture estimation and tracking method based on depth data

Abstract

The invention discloses a hand posture estimation and tracking method based on depth data. According to the method, the hand parameter estimation is obtained through a hand depth image obtained through a depth sensor and by using a semi-supervised hand posture estimation method combining a variational autoencoder (VAE) and a generative adversarial network (GAN), and then the hand action parameterization time series data is obtained by using a gesture tracking method based on a symbolic distance function (SDF). In the practical application, the method is high in calculation efficiency and strong in real-time performance of posture estimation, can obtain a gesture estimation result with higher precision and the hand rapid tracking with better robustness under the constraint of a small number of labeled samples and a large number of non-labeled samples, and the calculation resources and the time can be saved.

Description

technical field [0001] The present invention relates to a hand motion posture estimation and tracking method, in particular to a hand motion posture estimation based on depth data, which is mainly applicable to various applications in the field of human-computer interaction. Background technique [0002] Early hand motion tracking mainly includes methods based on wearable devices and methods based on vision. The most typical representative device for hand posture tracking based on wearable devices is the data glove. The sensors in the glove convert the relevant posture and motion information of the hand into electrical signals and send them to the computer for processing. Although the data glove-based method has the advantages of being fast and accurate, this method requires the user to wear complex data gloves and position trackers, which do not meet the requirements of natural human-computer interaction, and the data gloves are expensive and are currently only suitable for...

AI Technical Summary

Problems solved by technology

In the current hand posture tracking research, the hybrid method based on the advantages of learning [3] and generative model [4] can take into account both the efficiency and the robustness of the algorithm. It is the current hand motion posture estimation and tracking research However, under the framework of the hybrid method [5], how to use the semi-supervised method to solve the difficulties faced by the learning-based method, such as the large hand pose sample space and the time-consuming and labor-intensive manual labeling, is a problem that needs to be solved at present.

[0003] Therefore, the present invention mainly solves the following two problems: (1) There are a large number of local micro-movements in the learner's hand behavior in the experimental environment, and accurately capturing these micro-movements requires the use of model-based hand pose estimation that is more sensitive to subtle changes method, but such methods are usually affected by low computational efficiency, and it is difficult to meet the real-time requirements, especially hand movements are usually more agile than body movements, and it is easy to lose the target when tracking

(2) Compared with the diversity of hand gestures in the virtual-real fusion experimental environment, the lack of labeled samples available for training has become an important factor restricting the accuracy of learning-based methods in gesture estimation. Initialization and re-initialization with large errors will lead to The model-based gesture estimation method consumes more computing resources and time, and destroys the real-time performance of the tracking algorithm

Images