Gesture recognition method and device based on random variation information, equipment and medium

Abstract

The invention discloses a gesture recognition method and device based on random variation information, equipment and a medium, and the method comprises the steps: carrying out the feature extraction of obtained to-be-detected data, and obtaining the feature data in different modes; obtaining a variational information bottleneck; compressing random local information in the feature data through the variational information bottleneck, and reserving residual local information in the feature data to obtain a fusion result of the feature data in different modes; and inputting the fusion results in the different modes into a preset model, and outputting a gesture recognition result. According to the invention, the relevance among multiple modes can be fully utilized, the interference noise is reduced, the recognition accuracy is further improved, and the method can be widely applied to the technical field of image processing.

Description

technical field [0001] The present invention relates to the technical field of image processing, in particular to a gesture recognition method, device, equipment and medium based on stochastic variational information. Background technique [0002] In the current era of rapid technological development, it is of great significance to use artificial intelligence technology to achieve better interaction between humans and machines. With the rapid development of technologies such as chips and communications, people can obtain more and more information. Data obtained from different domains or perspectives for the same object is multimodal data, and each domain or perspective is called a modality. Compared with the traditional single-modal perception, multi-modal learning can better complete the perception task with the help of the relationship between the various modal data. Therefore, Multi-Modal Machine Learning (MMML) has quickly become a current research hotspot and has been...

AI Technical Summary

Problems solved by technology

Ineffective redundancy mainly includes complex and variable environmental noise in some modal data collected in real scenes, which may cause information conflicts with other modalities

Such invalid redundant information will interfere with the gesture recognition model; and improper learning of invalid redundant information by the model will affect the recognition performance of the model

[0005] However, multimodal fusion technology still has challenges such as useful information cannot be effectively utilized, redundant information cannot be better suppressed, and multiple modal information cannot be fine-grained.

The traditional multimodal fusion method based on simple connection has the following disadvantages: 1. The correlation between multimodality is not fully exploited and utilized

2. The interference noise existing between multiple modes has not been effectively suppressed

Images