Eye movement interaction method, system and device based on eye movement tracking technology

Abstract

The invention belongs to the technical field of eye movement tracking, and discloses an eye movement interaction method, system and device based on eye movement tracking technology, and the method comprises the steps: setting a passive adsorption gaze cursor or an eye movement interaction intention of a sensing area and predicting an active adsorption gaze cursor to select a target; setting corresponding sensing areas, namely effective clicking areas, for different targets, when the cursor makes contact with or covers the sensing area of a certain target, whether eye movement behaviors such aseye tremor exists or not, detecting whether the glancing distance exceeds a threshold value or not at the same time, and absorbing or highlighting then the target object. A machine learning algorithmis adopted to train the eye movement behavior data of the user, the data is filtered, processed and analyzed, an eye movement behavior rule is trained, and a subjective consciousness eye movement interaction intention model of the user is obtained. Through the method, the stability and accuracy in the eye movement interaction process are improved, and the user experience of eye movement interaction is improved.

Description

technical field [0001] The invention belongs to the technical field of eye movement tracking, and in particular relates to an eye movement interaction method, system and equipment based on eye movement tracking technology. The system leverages technologies in the fields of human-machine interface (HMI), wearable computing, human physiology, image processing, and deep learning Background technique [0002] The near-eye display device (HMD) or in-vehicle display also includes an electronic display and optical components. HMD can be for example Mixed Reality - MR glasses, Augmented Reality - AR glasses such as displays made by Microsoft, Vuzix and DigiLens; Virtual Reality - VR glasses such as Sony, Samsung, Oculus , displays manufactured by Carl Zeiss; Extended Reality (Extended Reality)—XR glasses or some combination thereof; in-vehicle displays may include but are not limited to head-up displays (HUD) or head-up displays with augmented reality systems (AR-HUD), driver Info...

AI Technical Summary

Problems solved by technology

Such physiological nystagmus will cause the jitter of eye movement coordinates, which is not conducive to accurate target selection in eye movement interaction and affects user experience;

[0008] (3) The existing interaction method of MR glasses is based on the interaction of virtual holographic objects presented by optical displays, and hardly considers the interaction with things in the real physical world

[0009] (4) In the existing technology, the mouse trackball has adsorption technology and methods, but the same method is not suitable for eye movement adsorption. In addition to interaction needs, eye movement interaction also has cognitive needs and viewing needs, so it is no longer reasonable When the mechanism of the system distinguishes these eye needs, it will cause trouble for the user to randomly attach the eye movement cursor

[0012] (6) When there are two or more adjacent tiny interactive buttons, when the geometric center of the eye-moving cursor vibrates between them, it will lead to repeated selection, and the accuracy of cursor pointing (the process of selecting the target) will be reduced

[0013] (7) The operation efficiency of the existing eye movement interaction mode is not high, and the clicking operation using single-eye blinking or double-eye blinking will easily cause eye fatigue, and the loss of the eyeball image signal in the process of blinking and clicking will cause the gaze cursor to deviate. Shifting and dislocation, that is, it can be understood as clicking and confirming through blinking during the eye movement interaction process, while the gaze cursor shifts from the original aligned position, which prevents the user from smoothly interacting with the function buttons

Another example is that different users have different habits of visual search and cognitive bandwidth. For example, people have different cognitive habits when reading books. view), some people are easy to miss when reading, etc.

Images