IPT simulation training gesture recognition method based on binocular vision

Abstract

The invention discloses an IPT simulation training gesture recognition method based on binocular vision, and the method comprises the steps: improving an interaction mode according to the characteristics of a VR application, and carrying out the innovative improvement in three aspects: gesture segmentation, gesture tracking and gesture recognition; during gesture segmentation, compensating a target loss phenomenon in a foreground segmentation image caused by motion blur through background modeling, target motion trend estimation and other methods; during gesture tracking, using the combinationof chrominance information and depth information, realizing tracking of a motion gesture during operation of the flight simulator through contour fitting; during gesture recognition, providing and using an extended Haar-like feature and an improved Adaboost method for hand shape classification, and defining a state area. The good dynamic gesture recognition rate is obtained, the use experience during operation is improved, an operator can accurately check the hand action during operation, the operation training efficiency is effectively improved, and the training cost is reduced.

Description

technical field [0001] The invention belongs to the field of computer vision / human-computer interaction, and in particular relates to a binocular vision-based IPT simulation training gesture recognition method. Background technique [0002] At present, gesture recognition technology is mainly divided into data glove-based recognition technology and vision-based recognition technology. Data glove-based recognition technology uses sensor devices to convert hand and finger movement information into computer-understandable control commands. This technology collects gestures and gesture movement data through additional sensor devices; vision-based technology uses cameras to collect human Gestures are converted into computer-understandable commands through video image processing and understanding technology to achieve human-computer interaction effects, and natural interaction with computers can be achieved without wearing any additional equipment. [0003] However, the core comp...

AI Technical Summary

Problems solved by technology

[0003] However, the core components of data gloves are usually quite expensive, and they need to be worn to operate the machine, and the interaction after the user wears it is extremely unnatural, which is quite different from the real interactive operation, and cannot bring a good user experience to the operator

However, most of the existing vision-based recognition technologies are based on monocular cameras. When the user moves violently, it is easy to cause the loss of the target gesture in the foreground segmentation image. The tracking of gestures based solely on color is easily affected by the ambient light. The problem of tracking loss, which in turn will lead to a decrease in recognition accuracy

[0004] In the IPT (Integrated Procedures Trainer, an important training device in the aerospace field) system, the traditional tactile interactive operation can only simulate the operation sequence, and cannot simulate the operating environment

Images