Dynamic gesture recognition method and system based on video coding data multi-feature fusion

Abstract

The invention discloses a dynamic gesture recognition method and a dynamic gesture recognition system based on video coding data multi-feature fusion. The data extraction and fusion are carried out by taking a GOP of a coding video as a basic unit, and a good fusion effect is obtained by acquiring an I frame in the GOP and extracting accumulated motion vectors and accumulated residual errors of all P frames in the GOP; feature data is directly extracted from compressed data, the data extraction process does not need too much calculation overhead, accumulation motion vectors are adopted to pay attention to space displacement information of a motion part in an image sequence, and good robustness is achieved for a complex illumination background environment; motion vector data and residual data in video coding are utilized, when a motion gesture video has three-dimensional motion features, very high recognition accuracy can be achieved, clearer motion features and contour information in the video are obtained by accumulating the motion vector data and residual data of all P frames in a single GOP. Therefore, the dynamic gesture recognition can be accurately realized, and the gesture recognition effect is improved.

Description

technical field [0001] The invention belongs to the field of information technology, and in particular relates to a dynamic gesture recognition method and system based on multi-feature fusion of video coding data. Background technique [0002] As an important way of human-computer interaction, dynamic gestures have received more and more attention in recent years. The specific movements performed by the hands are captured and analyzed by the corresponding equipment, and transformed into a series of control commands to make the smart equipment respond, making the human-computer interaction more natural and friendly. In recent years, dynamic gesture recognition has been widely used in somatosensory games, sign language recognition, assisted driving, medical equipment, and smart home appliance control. [0003] Currently in computer vision, most existing action recognition models are large convolutional neural networks (CNNs) that only use raw RGB frames as input. However, pr...

AI Technical Summary

Problems solved by technology

Although the recognition effect can be improved by adding optical flow and depth data, the system resource consumption of calculating optical flow is large, and depth data also requires more input devices, and there is a problem of data alignment

Images