A man-machine interaction system and method of virtual reality

Abstract

The invention discloses a virtual reality man-machine interaction system and method, which is based on spatial positioning and gesture recognition, combines virtual and real space coordinate matchingand gesture recognition, can obtain real-time gesture model of user operation real-time change through gesture image acquisition and processing, and can provide visual feeling for user on display of ahelmet-mounted display. The hardware-in-the-loop manipulation equipment calibrated by spatial coordinates is accurately mapped with the manipulation objects in the virtual scene to provide the user with the tactile experience in the operation process. Compared with the common realization mode, the invention has the advantages of direct interaction, close to reality, strong sense of experience andthe like. The human input of the virtual reality system which is displayed by the helmet is free from the restriction of the handle and gloves, so that the human-computer interaction of the virtual reality system is direct and natural, and can obtain the tactile feeling, so as to achieve the effect of closer to the real user experience.

Description

technical field [0001] The invention belongs to the technical field of virtual reality simulation, and in particular relates to a virtual reality human-computer interaction system and method. Background technique [0002] In recent years, virtual reality (Virtual Reality, referred to as VR) technology and VR interactive equipment have been more and more widely used in virtual training and virtual maintenance of large-scale equipment. system, construction machinery virtual operation training system, combat vehicle driving simulation and control system, etc. Most of such VR virtual systems use handles or data gloves for interactive input. Although the purpose of virtual interaction can be achieved, the interactive operation is generally not direct, inconsistent with the process of using the actual equipment, and it cannot provide a tactile experience for interactive operation. The user experience is not strong. On the basis of the development of gesture recognition technolog...

AI Technical Summary

Problems solved by technology

Most of such VR virtual systems use handles or data gloves for interactive input. Although the purpose of virtual interaction can be achieved, the interactive operation is generally not direct, inconsistent with the process of using the actual equipment, and it cannot provide the tactile experience of interactive operation. Poor user experience

On the basis of the development of gesture recognition technology, the combination of gesture recognition and VR technology has emerged, but in almost most applications, the operator's hand only "interacts" with the air in the space, which still cannot give the operator a more realistic reality. Operating experience

Images