Human behavior recognition method and system based on multimodal deep Boltzmann machine

Abstract

The invention discloses a human body behavior recognition method and system based on a multimodal deep Boltzmann machine. The method includes the steps of: 1) acquiring data from vision and wearable sensors; 2) establishing vision data and wearable sensor data Modal fusion model; 3) Heterogeneous transfer learning using deep neural network to reconstruct missing data; 4) Classification using softmax regression model classifier; 5) Deep network model generated from public sample data according to individual user characteristics Make adaptive adjustments. The invention can improve the accuracy rate of human body behavior recognition in the complex scene and the data missing situation.

Description

technical field [0001] The invention relates to the technical fields of artificial intelligence and behavior recognition, in particular to a human behavior recognition method and system based on a multimodal deep Boltzmann machine. Background technique [0002] In recent years, the robot industry has shown explosive growth, and the era of "full application" of robots is coming. On the one hand, robots appear in families and daily life. On the other hand, with the development of industrial robots, robots are widely used in various industries such as automobile manufacturing and metal manufacturing to realize human-machine collaboration. Human behavior recognition is widely used in human-computer interaction, human-computer collaboration and other fields. Robots need to understand and recognize human behavior from various abstract layers. The accuracy of recognition will play a major role in the application and development of robot technology. Human behavior recognition by ro...

AI Technical Summary

Problems solved by technology

[0007] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and propose a human behavior recognition method and system based on a multimodal deep Boltzmann machine with high recognition accuracy and strong usability, aiming to build a human behavior recognition system based on vision The multimodal deep neural network model of wearable sensors can improve the accuracy of behavior recognition in complex scenes; the deep Boltzmann machine network is used in the multimodal deep learning model to reduce the impact of missing data on the accuracy of behavior recognition The influence caused by this problem; a method of adjusting the network structure to establish an adaptive common model combined with individual characteristics is proposed, so as to improve the accuracy of the robot's recognition of the specific owner's behavior

Images