Face detection and restoration method based on RGBD camera

Abstract

The invention provides a face detection and restoration method based on an RGBD camera. The method comprises the following steps: 1) constructing a binocular camera system based on a RGB camera and aninfrared camera, and obtaining image data from the RGB camera and the infrared camera respectively; 2) carrying out face detection on the two types of image data; 3) determining face feature vectorsin the image data; and 4) calculating an approximate coordinate position on an infrared image of a face key point coordinate vector on an RGB image by utilizing corresponding matrixes of two camera images, and selecting a face key feature vector with the minimum distance from the face key feature vectors detected on the infrared image as an optimal matching result. Since depth images are calculated based on the infrared images, the depth images generally keep a one-to-one correspondence relation with infrared image pixels, and the depth image face and the infrared image face have the same position; and through combined processing of RGB camera images and the depth images, face detection and matching of the two types of data can be finished while collecting the RGBD data, which makes it possible for high-level applications in the RGBD-based face image processing field.

Description

technical field [0001] The invention relates to the field of face recognition, in particular to a face detection and restoration method based on an RGBD camera. Background technique [0002] Face recognition plays an important role in the security field, and in the current face recognition system and related applications, most of them use RGB image data. RGB image data has some inherent defects. As two-dimensional data, it is In the process of projecting 3D information into 2D space, the depth information is lost, and we cannot obtain the 3D information of objects or faces from 2D images. High-precision 3D reconstruction solutions can reconstruct the 3D information of different objects in the scene, but the high cost and low speed prevent such products from being widely used in current common scenes. The depth camera is relatively cheap, has a high data frame rate, and can feed back data in real time. In addition to providing RGB images, it can also provide infrared images ...

AI Technical Summary

Problems solved by technology

But generally speaking, the points on the RGB image and the depth image are not one-to-one correspondence, so when we want to associate the face on the RGB image with the face on the depth image, the depth image will not be found. The problem of the face on the image, or when several faces appear on the RGB image at the same time, we have no way to find the corresponding face on the RGB image on the depth image

Moreover, the face information on the depth image may be missing due to the failure to reflect the laser normally.

The above problems have brought difficulties to RGBD-based face recognition and related applications.

Images