Gesture recognition system and method based on VR virtual office

Abstract

The invention provides a gesture recognition system and method based on VR virtual office, and the system comprises: a camera which is disposed above a user operation region, and collects a depth image and an infrared image of the user operation region according to frames; the model construction part that is used for constructing a hand detection model and a hand skeleton point recognition model;the on-line detection tracking part that is used for inputting the images acquired by the camera into the hand detection model and the hand skeleton point identification model to obtain coordinates ofa plurality of skeleton points of the hand under camera coordinates; the first calibration part that is used for transferring the skeleton points of the hand under the camera coordinate system of theskeleton point identification module to the skeleton points of the hand under the helmet-mounted display coordinate system; and the rendering part that is used for rendering the three-dimensional model of the hand corresponding to the current frame in the virtual scene of the helmet-mounted display. According to the gesture recognition system and method, the gesture position information of the hand in the virtual scene is restored with high precision and high stability.

Description

technical field [0001] The present invention relates to the field of human-computer interaction technology, and more specifically, to a gesture recognition system and method based on VR virtual office. Background technique [0002] As we all know, in order to enhance the immersion of VR / AR combination of virtual and real, so that VR / AR users have a better experience, the human-computer interaction module is essential, especially the high-precision real-time restoration of hand gestures in VR / AR scenes is very important. Affect the user's immersion experience in VR / AR scenes. Gesture recognition is very critical in the field of VR / AR, especially in the field of VR virtual office, which has relatively high requirements for accuracy, delay, and environmental compatibility and stability. [0003] The current gesture recognition mostly stays on some simple and conventional gesture operations, such as clicking some menus or buttons, or doing some simple operations on some picture...

AI Technical Summary

Problems solved by technology

For example, American gesture recognition company LeapMotion, but in virtual office scenarios, due to the influence of environmental factors such as keyboards and desktops, the accuracy of hand restoration is not ideal, which cannot meet the needs of this environment, which will greatly reduce the user experience in VR virtual office experience

Images