Human body posture visual identification method of transfer carrying nursing robot

Abstract

The invention discloses a human body posture visual identification method of a transfer carrying nursing robot. According to the method, a two-stage network (the first-stage neural network is PAF andthe second-stage neural network is improved ResNet) based on RGBD (color image RGB + depth image Depth) is utilized to realize close-range higher-precision identification; a ResNet network commonly used for classification is improved, and input and output structures of the ResNet network are modified, so that the ResNet network can solve the problem of human joint recognition, serves as a second-level neural network, and obtains higher precision compared with the existing method. According to the nearest neighbor insect following method, under the condition that only two joint coordinates aredepended on, the armpit contour of the human body is automatically tracked, the armpit contour is further repaired through the convex hull algorithm, the armpit center, namely the armpit point, is finally obtained, and by means of the armpit point recognition method, the dependence of the armpit point on the joint position is reduced, and meanwhile accuracy is improved.

Description

[0001] Technical field: [0002] The invention belongs to the technical field of nursing robots, and in particular relates to visual recognition of human body gestures for transferring and transporting nursing robots. [0003] Background technique: [0004] my country has entered an aging society. At present, there are more than 230 million people over the age of 60, and the degree of aging is becoming more and more serious. With the continuous progress of my country's economy and technology, the demand for intelligent transfer, transportation and nursing robots in the Chinese market is increasing. bigger and bigger. The intelligent perception and understanding of the environment is the key to the intelligentization of the transfer, handling and nursing robots, and human body posture recognition is a key to environmental perception. [0005] At present, human body posture recognition is mostly contact recognition. The posture of the human body is estimated by sticking points. ...

AI Technical Summary

Problems solved by technology

[0005] At present, human body posture recognition is mostly contact recognition. The posture of the human body is estimated by sticking points. Inertial tracker to detect the rotation and extension of the patient's forearm and wrist; Chinese Dalian University of Technology patent 201611272616.X uses 12 data acquisition nodes placed in different positions of the human body to measure the movement posture of different human body; contact The measurement requires the patient to wear a variety of sensors or optical signs, which is not only troublesome, but also affects the movement of the patient, causing psychological discomfort to the patient, which is not conducive to the practical application of fully automatic detection

Non-contact human body posture measurement is mainly based on vision, and the color map human body posture recognition method PAF (PartAffinity Field) (Cao Z, Simon T, Wei S E, et al. Realtime Multi-Person 2D Pose Estimation using PartAffinity Fields[J].2016. ) realizes high-precision and high-reliability human pose recognition, but it can only provide 2D pixel coordinates, and the estimation cannot be transferred to 3D global coordinates; Zimmermann et al. (Zimmermann C, Welschehold T, Dornhege C, et al.3D Human Pose Estimation in RGBD Images for Robotic TaskLearning[J].2018.) A 3D convolutional network based on RGBD information was proposed for the grasping and teaching function of service robots. This algorithm has a large amount of computation, which is not conducive to lightweight hardware and real-time output

The Kinect currently launched by Microsoft is the most common human body posture visual recognition product in the nursing industry: the lifting transfer robot RIBA (MukaiT, Hirano S, Yoshida M, et al. Tactile-based motion adjustment for the Nursing-care assistant robot RIBA[C] / / IEEE International Conference on Robotics&Automation.2011.) Kinect camera and its SDK are used to recognize human body posture, but the algorithm based on random forest is too close to the camera or the human body is partially occluded On the other hand, the Kinect SDK cannot recognize whether the user is front or back, that is, the recognition of the left and right directions of the user's joints cannot be guaranteed.

Images