Human body three-dimensional reconstruction method and device based on rgbd single-view image

Abstract

The invention discloses a method and device for three-dimensional reconstruction of a human body based on an RGBD single-view image, wherein the method includes: collecting an RGBD picture of a human body through a depth camera, and the picture includes a single-view color picture and a depth picture; obtaining a three-dimensional human body according to the RGBD picture Model parameters, human body segmentation information and two-dimensional joint point information; according to the human body segmentation information, two-dimensional joint point information and depth image, the three-dimensional joint point information of the human body is obtained, and the joint points and body shape of the three-dimensional human body model are calculated according to the three-dimensional joint point information of the human body Constrain and optimize the 3D human body model parameters and 3D human body model; render the depth image according to the optimized 3D human body model, and use the single-view color image to further optimize the front of the optimized 3D human body model to obtain the 3D reconstruction result of the human body . This method can use the single-frame single-view RGBD image information collected by the depth camera to perform three-dimensional reconstruction of the human body.

Description

technical field [0001] The invention relates to the technical field of three-dimensional reconstruction in computer vision, in particular to a method and device for three-dimensional reconstruction of a human body based on an RGBD single-view image. Background technique [0002] With the continuous development of 3D reconstruction technology in the field of computer vision, it is becoming more and more important to use a depth camera to reconstruct a 3D human body in a lightweight way. On September 13, 2017, Apple released the new model iPhoneX, in which the front True Depth camera has attracted great public attention. According to Apple's official introduction, the iPhone X projects more than 30,000 invisible light points onto the face through the front dot matrix projector, and then calculates the depth map of the face based on the reflected light points received by the infrared lens. It is foreseeable that 3D reconstruction, especially the 3D reconstruction related to th...

AI Technical Summary

Problems solved by technology

[0003] However, at this stage, reconstructing a 3D human body from a single frame of RGBD images is still an underdetermined optimization problem, and the geometric information of the back and sides cannot be directly obtained from the image

And the continuous promotion of depth cameras on mobile platforms will become a trend in the future. The hardware of depth cameras is also likely to be greatly improved, but limited by the impact of algorithms, power and space, at this stage from commercial The depth information directly collected in the depth camera has poor accuracy and contains more noise and less detail information.

Point clouds directly converted from such depth maps are often unsatisfactory

Images