Method and system for gesture recognition and man-machine interaction based on 2D video sequence

Abstract

The invention discloses a method and a system for gesture recognition and man-machine interaction based on a 2D video sequence. The method comprises the steps of S1, acquiring a monocular 2D video frame sequence image, and extracting moving foreground in the image; S2, detecting a human hand in the moving foreground, and constructing a joint feature model of the human hand; S3, predicting a location area where a human hand target emerges, searching and positioning the human hand target in the location area by using the joint feature model of the human hand, and acquiring the location of the human hand in the current frame; S4, judging the type of the current operating mode according to the location of the human hand in the current frame; and S5, tracking the human hand, recognizing the posture and the gesture of the human hand in the current operating mode, converting the posture and the gesture into a corresponding instruction so as to realize man-machine interaction. The man-machine interaction method disclosed by the invention not only can realize target hand selection at a complicated background, but also can realize high-accuracy and high-stability tracking for the human hand.

Description

technical field [0001] The invention relates to a gesture recognition and human-computer interaction method and system based on 2D video sequences, belonging to the technical field of human-computer interaction. Background technique [0002] Somatosensory control technology based on gesture recognition has become an important means of human-computer interaction. It collects the user's action picture through a common camera, detects and locates the hand features in the image through the pattern recognition algorithm, and then converts the recognition information into an operation signal by recognizing the gesture and movement trajectory of the human hand. Send it to the smart terminal, and trigger corresponding operation commands, such as switching TV programs, adjusting the volume, zooming in and out of pictures and web pages, and controlling simple somatosensory games, such as cutting fruit, playing ball, driving and other games. Gesture recognition technology is based on ...

AI Technical Summary

Problems solved by technology

However, this application still has the following disadvantages: a) Due to the lack of depth information, it will be difficult to extract human hands in complex environments; b) ordinary 2D cameras are very sensitive to light, and it is necessary to achieve non-rigid, less textured images like human hands. High-precision tracking of the target will face great challenges in a complex environment; c) due to noise, distance, and the different habits of each person, it will also cause certain problems in the recognition of various gestures and gestures of the human hand. Influence; d) It is impossible to realize the recognition of the gestures and gestures of human hands; e) For the recognition of both hands, how to deal with the crossing of two hands is also a difficult problem

Images