Real-time human body dynamic three-dimensional reconstruction method and system driven by clothes physical model

Abstract

The invention discloses a real-time human body dynamic three-dimensional reconstruction method and system driven by a clothes physical model, and the method comprises the steps: carrying out the photographing of a human body through a depth camera, and obtaining a color-depth image sequence; performing parameterized human body model matching, model surface non-rigid motion tracking and model updating based on depth fusion by using the input depth sequence; performing multi-view three-dimensional human body semantic segmentation based on a deep learning method by using the color image sequenceto obtain independent geometric models of different clothes, and forming a multi-layer human body model in combination with the parameterized human body model; according to the depth sequence input, performing human body motion tracking by using the double-layer human body surface to obtain corresponding human body posture information; conducting physical simulation on clothes movement according to human body movement, constructing depth fitting external force by combining input depth information, and restraining a clothes physical simulation result to be matched with depth input. The method is accurate and efficient in solution, and can realize human body dynamic three-dimensional reconstruction with clothes fine dynamic details.

Description

technical field [0001] The invention relates to the technical fields of computer vision and computer graphics, in particular to a real-time human body dynamic three-dimensional reconstruction method and system driven by a physical model of clothing. Background technique [0002] The dynamic clothing in the existing film and television and game production is generated by manually setting key frames or using clothing physical simulation. At present, there is still a lack of a convenient and efficient method in the industry and academia to directly and finely digitally capture the movement of clothing. . With the help of the human body shape information and clothing geometric information reconstructed by the human body dynamic 3D reconstruction system, when the physical characteristics of the clothing are known, it is very challenging and meaningful to use the physical simulation method to finely capture the motion of the clothing . On the one hand, this direction can greatly...

AI Technical Summary

Problems solved by technology

[0003] The existing 4D reconstruction algorithm based on a single RGBD camera can reconstruct the geometry, movement, surface texture and even surface reflectance of the human body, but the 4D reconstruction results are still not realistic enough

The main reasons are as follows: First, the existing methods all use single-layer geometry for dynamic 4D reconstruction of the human body. Using single-layer geometry to express the human body will topologically connect the human body itself with the clothing it wears, so it cannot track or even describe the clothing and human body Interactive motion (such as the sliding and fluttering of clothing relative to the human body), this single-layer geometric expression also makes the 4D results reconstructed by existing methods unable to be directly edited and driven effectively, so it is difficult to be directly applied to applications such as virtual fittings Second, the fine motion of the clothing worn by the human body is difficult to describe with the commonly used dynamic chain model or the surface non-rigid deformation map based on sparse uniform sampling, resulting in the existing reconstruction system often giving too smooth Clothing dynamic reconstruction results; finally, under the condition of limited single-view input, the existing methods are still unable to reconstruct the dynamic deformation results of clothing in the occluded area

Images