Line-of-sight estimation method based on depth appearance gaze network

Abstract

The invention provides a line-of-sight estimation method based on a depth appearance gaze network. Main content of the method includes a gaze data set, a gaze network, and cross-data set evaluation. The method comprise the following steps: a large number of images from different participants are collected as a gaze data set, face marks are manually annotated on subsets of the data set, face calibration is performed on input images obtained via a monocular RGB camera, a face detection method and a face mark detection method are adopted to position the marks, a general three-dimensional face shape model is fitted to estimate a detected three-dimensional face posture, a spatial normalization technique is applied, the head posture and eye images are distorted to a normalized training space, and a convolutional neural network is used to learn mapping of the head posture and the eye images to three-dimensional gaze in a camera coordinate system. According to the method, a continuous conditional neural network model is employed to detect the face marks and average face shapes for performing three-dimensional posture estimation, the method is suitable for line-of-sight estimation in different environments, and accuracy of estimation results is improved.

Description

technical field [0001] The invention relates to the field of gaze estimation, in particular to a gaze estimation method based on a deep appearance gaze network. Background technique [0002] Line of sight estimation is to estimate the line of sight direction of the human eye in the picture, calculate and return the coordinates of the center position of the eyeballs of both eyes, and the line of sight direction vector of both eyes. Real-time tracking of human eye sight can be realized in the video, which is often used in public security, transportation, medicine, military investigation and other fields. Specifically, in the field of public security, by estimating the direction of the human eye's gaze, it is possible to infer the area or object that the person is concerned about, and further study the person's psychological activities, etc., which can be used for lie detection during the interrogation of prisoners. In the field of traffic, sight estimation automatically monit...

AI Technical Summary

Problems solved by technology

Due to the complex background of face images collected in daily life, it is affected by conditions such as illumination, posture, gaze direction, and personal appearance, and most of the existing methods are only applicable to data sets with specific backgrounds. Once estimated across data sets, the results are easy. There are errors, so it is still a challenge to accurately estimate the line of sight

Images