Mixed reality open set human body posture recognition method based on deep learning

A technology of human body posture and mixed reality, applied in the field of wearable sensors, to achieve the effect of facilitating subsequent processing operations

Active Publication Date: 2021-11-26
SHANGHAI JIAO TONG UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Aiming at the problem that the existing deep learning algorithm relies on a large amount of artificially marked data and does not pay attention to the imbalance between the number of known classes and unknown classes in the real domain, the present invention proposes a mixed reality open-set human gesture recognition method based on deep learning, which can accurately Identify known human activities while detecting unknown activities not seen in the training set

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mixed reality open set human body posture recognition method based on deep learning
  • Mixed reality open set human body posture recognition method based on deep learning
  • Mixed reality open set human body posture recognition method based on deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] Such as image 3 As shown, this embodiment relates to a mixed reality open-set human gesture recognition method based on wearable sensor devices and deep learning, and the specific steps include:

[0023] Step 1: Build a neural network.

[0024] The input data of the neural network of the mixed reality-based open-set human body posture recognition algorithm adopted in this embodiment are the human body posture data generated by the virtual IMU and the real human body posture data collected by wearable sensors, and are migrated from the virtual domain IMU data to The mixed reality method of real domain IMU data only uses a small amount of labeled real IMU data, and the adversarial training strategy enables the network model to accurately recognize known actions while including the ability to detect unknown human behavior actions.

[0025] Such as figure 2 As shown, the neural network of the open-set human gesture recognition algorithm based on mixed reality includes: ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a mixed reality open-set human body posture recognition method based on deep learning. The method includes using a neural network of an open-set human body posture recognition algorithm based on mixed reality, and setting a training loss function through a decision boundary based on a sample number imbalance problem to obtain a trained neural network; and migrating the domain invariant features learned from the virtual domain to the real domain to guide the real domain to carry out feature extraction, thereby realizing mixed reality human body posture recognition. According to the invention, unknown activities which do not appear in the training set can be detected while known human activities are accurately identified.

Description

technical field [0001] The present invention relates to a technology in the field of wearable sensors, in particular to a mixed reality open-set human gesture recognition method based on wearable sensor equipment and deep learning. Background technique [0002] Wearable sensors such as accelerometers and gyroscopes and other inertial measurement units (IMUs for short) are widely used in Human Activity Recognition (HAR for short) due to their low price and flexibility. Since there will be a variety of unpredictable and unknown behaviors in real life, the existing human gesture recognition based on wearable sensor devices can only solve the closed set problem, that is, it can only recognize human activities that have appeared in the training set. When there are unknown gestures in the test set that have not appeared in the training set, the existing closed-set algorithm will inevitably mistake the unknown class as a certain class in the known actions, which will cause a signif...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/08
CPCG06N3/04G06N3/084G06F18/24G06F18/214
Inventor 张紫璇裴凌储磊夏宋鹏程郁文贤
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap