Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Face incompleteness scanning completion method and device based on deep learning

A deep learning and incomplete technology, applied in the field of computer vision, can solve the problems of different face scanning poses and redundant surfaces, and achieve the effect of good completion effect.

Pending Publication Date: 2021-11-19
TSINGHUA UNIV
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] This application provides a method, device and storage medium for face incomplete scan completion based on deep learning to solve the technical problems of different face scan poses and redundant surfaces in the related art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Face incompleteness scanning completion method and device based on deep learning
  • Face incompleteness scanning completion method and device based on deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] figure 1 It is a schematic flow chart of a method for face incomplete scanning and completion based on deep learning provided according to an embodiment of the present application, as shown in figure 1 As shown, can include:

[0031] Step 101, acquiring a depth image and a color image collected by a depth camera.

[0032] Step 102, detect the two-dimensional face feature points in the color image, and generate the three-dimensional face feature points of the face scan according to the camera internal reference and the depth information in the depth image, and roughly align the face scan to the template person according to the three-dimensional feature points In the standard coordinate system where the face is located.

[0033] Among them, the completion method provided by this application uses a 3DMM face model as a template, and defines the coordinate system where the template is located as a standard coordinate system.

[0034] Among them, according to the three-di...

Embodiment 2

[0044] Further, based on the deep learning-based face defect scan completion method provided in the above embodiments, the embodiment of the present application also provides a deep learning-based face defect scan complement device 200, figure 2 It is a schematic structural diagram of a deep learning-based face incomplete scan completion device provided according to an embodiment of the present application, as shown in figure 2 As shown, can include:

[0045] An acquisition module 201, configured to acquire a depth image and a color image collected by a depth camera;

[0046] The rough alignment module 202 is used to detect the two-dimensional face feature points in the color image, and generate the three-dimensional face feature points of the face scan according to the camera internal reference and the depth information in the depth image, and will scan the face according to the three-dimensional feature points Roughly aligned to the standard coordinate system where the te...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a face incompleteness scanning completion method based on deep learning. The method comprises the following steps: taking a depth image and a color image collected by a depth camera; detecting two-dimensional face feature points in the color image, generating three-dimensional face feature points of face scanning according to an internal reference of the camera and depth information in the depth image, and roughly aligning the face scanning to a standard coordinate system where a template face is located according to the three-dimensional feature points; more accurately aligning the face scanning with the template face by using an iterative nearest point algorithm; fitting the template face to the aligned face for scanning by using a Laplace deformation algorithm; setting a distance threshold value on a fitting result, and removing a redundant surface in the face scanning; performing geometric shape completion on incomplete point cloud of a face area by using a PointNet auto-encoder to generate a complete face point cloud. According to the method and the device, the problems of different face scanning poses and redundant surfaces are solved, and geometric face shapes are complemented by using a neural network, so that a better complementation effect is obtained.

Description

technical field [0001] The present application relates to the technical field of computer vision, and in particular to a method, device and storage medium for face incomplete scan completion based on deep learning. Background technique [0002] Depth cameras can only capture faces from a single perspective, and the face scans collected by them are usually incomplete. Therefore, it is usually necessary to complement the incomplete scans of faces. [0003] In related technologies, data sets such as ShapeNet are mainly used to train the completion network to perform the work of point cloud completion. However, the objects in the ShapeNet dataset are usually airplanes, tables and chairs, etc. These objects have more complex geometric structures than human faces. Therefore, using a network carefully designed by these objects to complement human faces may not be very good. [0004] At the same time, other network structures used in point cloud completion work do not take into acc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T5/00G06T17/00G06K9/00G06N3/04
CPCG06T17/00G06T2207/10028G06N3/045G06T5/77
Inventor 徐枫冯铖锃
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com