A method and a system for detecting key points of a three-dimensional gesture based on a neural network

Abstract

The invention discloses a three-dimensional gesture key point detection method based on a neural network, which comprises the following steps: acquiring a gesture data set including gesture area information and gesture two-dimensional and three-dimensional key point position information; obtaining a gesture data set containing gesture area information and gesture two-dimensional and three-dimensional key point position information. Training a gesture segmentation network which can detect a gesture region in an RGB image by taking an RGB image containing a gesture as an input; the gesture region detected by the gesture segmentation network is truncated, up-sampled or down-sampled. Training a two-dimensional gesture key point detection network, which can detect a plurality of two-dimensionalgesture key points in a gesture region image; the absolute coordinates of the key points of the three-dimensional gesture are converted into relative coordinates; a 2D to 3D gesture key point mappingnetwork is trained. The network can map multiple 2D gesture key points into 3D space to form 3D gesture key points. The invention can quickly and effectively detect three-dimensional gesture key points from RGB images containing gestures.

Description

technical field [0001] The invention belongs to the technical field of pattern recognition, and more particularly relates to a neural network-based method and system for detecting key points of three-dimensional gestures. Background technique [0002] 3D gesture key point detection technology is one of the important contents in the field of 3D computer vision research, which provides an important basis for various image processing technologies such as dynamic gesture recognition, gesture prediction, abnormal motion detection, virtual reality, etc.; its main task is to input After the image of the gesture is processed by this technology, the coordinates of the key points of the three-dimensional gesture in the given image are output, where the key points of the gesture include the fingertips and joint points of each finger; in the process of detecting the key points of the three-dimensional gesture, the shape of the gesture is random, Gestures are flexible and changeable, eac...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the above defects or improvement needs of the prior art, the present invention provides a method and system for detecting key points of 3D gestures based on neural networks. The position coordinates of the key points of the three-dimensional gestures are estimated, so as to solve the technical problems in the existing dynamic gesture recognition methods, such as small measurement range, high hardware cost, and unsuitability for large-scale application scenarios

Images