Method and system for visualizing real hand of participant in helmet-type virtual reality environment

Abstract

The invention discloses a method and a system for visualizing a real hand of a participant in a helmet-type virtual reality environment. The method comprises the following steps of: initializing and realizing virtual and real rectification; calibrating a real camera to obtain real camera parameters comprising a real camera space position, a horizontal viewing field angle and a vertical viewing field angle, and maintaining a virtual camera of a viewpoint position in the virtual environment to be consistent with the real camera parameters; partitioning a hand image, wherein video acquisition equipment acquires a video image shot by the real camera, and a computer carries out skin color detection on each frame of the acquired video image to realize separation of the hand image and the background; fusing virtuality and reality; and fusing the separated hand image into the virtual environment in real time by a dynamic vein mapping method to realize the seamless fusion of a virtual image. The system comprises a computer, a helmet display, the camera and the video acquisition equipment. The invention is efficient and stable and can obtain a vivid stereoscopic vision image.

Description

technical field [0001] The present invention relates to a method and system for visualizing the actual hand of a participant in a helmet-mounted virtual reality system. In the method, a camera is installed in the helmet-mounted display, and video image processing technology and virtual reality graphics technology are comprehensively used to realize real-time segmentation of hand images and Integrating into the virtual environment, it can provide users with realistic and three-dimensional virtual hand visualization images to meet the requirements of human-computer interaction. Background technique [0002] In the closed virtual reality system based on the head-mounted display, the visualization of the virtual hand is an important factor to ensure the user's immersion. Foreign experiments show that the most concerned question of novice users in the virtual environment is: "Where is my hand?" [1]. The quality of the visualization effect will also directly affect the user's vis...

AI Technical Summary

Problems solved by technology

[0003] 1. The shape of the real hand is complex, and it is difficult to achieve accurate geometric modeling. With the improvement of the fidelity of the model, the number of polygons will also increase, the burden on the system will increase, and the real-time performance will also decrease

[0004] 2. It is difficult to achieve the texture characteristics consistent with the real hand, and the visual effect is not good

[0005] 3. The difference in human hands is not considered. The general three-dimensional hand model represents the hands of different people, and the visual effect is not good

[0006] 4. Due to the errors of virtual peripherals such as data gloves and hardware delays, it is impossible to completely match the movements of the real hand and the virtual hand, and the hardware cost is also very high

[0007] 5. Virtual reality peripherals such as data gloves have a sense of foreign body and restraint, etc.

[0009] In the augmented reality flight simulator based on multiple fixed cameras (application number: 200710303603.9), what the head-mounted display shows is also the image after virtual-real fusion, but in its system, multiple cameras fixed above the head are used to shoot the real scene, and The image taken is directly integrated into the virtual environment, so it is different from the layout scheme and technical scheme of the camera of the present invention

Images