Human body three-dimensional reconstruction method and device based on RGBD single-view images

Abstract

The invention discloses a human body three-dimensional reconstruction method and device based on RGBD single-view images. The human body three-dimensional reconstruction method comprises the steps: collecting RGBD images of a human body through a depth camera, wherein the images comprise a single-view-angle color image and a depth image; respectively obtaining three-dimensional human body model parameters, human body segmentation information and two-dimensional joint point information according to the RGBD image; obtaining human body three-dimensional articulation point information according to the human body segmentation information, the two-dimensional articulation point information and the depth picture so as to restrain articulation points and body shapes of the three-dimensional humanbody model according to the human body three-dimensional articulation point information, and optimizing three-dimensional human body model parameters and the three-dimensional human body model; and rendering a depth picture according to the optimized three-dimensional human body model, and further optimizing the front surface of the optimized three-dimensional human body model by utilizing the single-view color picture to obtain a three-dimensional reconstruction result of the human body. According to the human body three-dimensional reconstruction method, three-dimensional reconstruction ofa human body can be carried out by utilizing single-frame single-view RGBD image information acquired by the depth camera.

Description

technical field [0001] The invention relates to the technical field of three-dimensional reconstruction in computer vision, in particular to a method and device for three-dimensional reconstruction of a human body based on an RGBD single-view image. Background technique [0002] With the continuous development of 3D reconstruction technology in the field of computer vision, it is becoming more and more important to use a depth camera to reconstruct a 3D human body in a lightweight way. On September 13, 2017, Apple released the new model iPhoneX, in which the front True Depth camera has attracted great public attention. According to Apple's official introduction, the iPhone X projects more than 30,000 invisible light points onto the face through the front dot matrix projector, and then calculates the depth map of the face based on the reflected light points received by the infrared lens. It is foreseeable that 3D reconstruction, especially the 3D reconstruction related to th...

AI Technical Summary

Problems solved by technology

[0003] However, at this stage, reconstructing a 3D human body from a single frame of RGBD images is still an underdetermined optimization problem, and the geometric information of the back and sides cannot be directly obtained from the image

And the continuous promotion of depth cameras on mobile platforms will become a trend in the future. The hardware of depth cameras is also likely to be greatly improved, but limited by the impact of algorithms, power and space, at this stage from commercial The depth information directly collected in the depth camera has poor accuracy and contains more noise and less detail information.

Point clouds directly converted from such depth maps are often unsatisfactory

Images