A method and device for estimating human pose based on 3D

Abstract

The invention discloses a method and device based on three-dimensional human body posture estimation, wherein the method includes S1: using a monocular camera to collect depth images and RGB color images of the human body at different angles; S2: constructing key points of human bones based on the RGB color images Detect the neural network to obtain the key point annotation image; S3: Construct the 2D-3D mapping network of the hand joint nodes; S4: Calibrate the depth image and key point annotation image of the same angle of the human body, and then perform 3D point cloud coloring conversion on the corresponding depth image, Obtain the colored depth image; S5: Based on the key point annotation image and the colored depth image, use the preset learning network to predict the corresponding position of the marked human skeleton key points in the depth image; S6: Merge the output of step S3 and step S5 to realize Refined Estimation for 3D Human Pose Estimation.

Description

technical field [0001] The invention belongs to the application fields of computer vision, image processing, computer graphics and deep learning, and in particular relates to a three-dimensional human body pose estimation method and device. Background technique [0002] The so-called human pose estimation refers to matching the abstract hierarchical features with the human body model, so as to obtain the poses of the target at different times. Human pose estimation is the core problem of human motion capture. The posture expression of the human body includes two aspects, one is the position and direction of the whole human body in the world coordinates; the other is the angle of the joints of each part of the body and the skin deformation affected by the joint angles. The main application areas of human motion pose estimation can be divided into three major directions: monitoring, control and analysis: [0003] (1) In terms of monitoring applications, some traditional appl...

AI Technical Summary

Problems solved by technology

Magnetic or optical markers are attached to human limbs, and their three-dimensional trajectories are used to describe the target movement. These systems are automatic, but their disadvantages are: the equipment is very bulky and expensive, and cannot be widely used

[0011] The second is to use the part detection method to determine the various parts of the body after removing the background other than the human body. This method can be partially automated, but requires strict guarantees for human scene segmentation

The disadvantage of this method is that the tracking of subsequent frames has cumulative errors, and long-term tracking is prone to errors

The disadvantage of this method is that it is difficult to describe the human motion posture with a limited number of states, it depends on prior knowledge, and it can only track a specific set of actions.

However, the conditions for using multi-eye cameras are: calibration is required and it is inconvenient to arrange in ordinary households, which is not conducive to the popularization of motion capture technology to thousands of households

Images