Mouse rapid control method and system in multi-screen scene based on sight line estimation and computer readable medium

Abstract

The invention discloses a mouse rapid control method and system in a multi-screen scene based on a sight line estimation and a computer readable medium. The method comprises the steps of image acquisition, image processing, coordinate position reading and mouse action control. The image processing comprises the steps of human face detection, facial feature point positioning, head posture estimation, three-dimensional sight line estimation, coordinate transformation and confidence coefficient threshold setting. According to the mouse rapid control method and system in a multi-screen scene based on sight line estimation and the computer readable medium, the problem of mouse following during large-screen or cross-multi-screen information browsing can be effectively solved, particularly, human-computer interaction is more convenient in a working scene of expanding a plurality of large screens, the mouse is controlled to rapidly move in a large range among the plurality of screens, and then, a hand can control the mouse to perform small-range fine control, and the mouse can be quickly, naturally and efficiently controlled among multiple screens through hand-eye matching.

Description

technical field [0001] The invention belongs to the technical field of eye-controlled mouse and human-computer interaction, and relates to a method, a system and a computer-readable medium for quickly controlling a mouse in a multi-screen scene based on line-of-sight estimation. Background technique [0002] The current rapid development of computer technology and multimedia technology provides convenient conditions for obtaining information and human-computer interaction. However, with the increase in the size and number of computer and multimedia screens, the efficiency of the traditional one-handed operation of the mouse for human-computer interaction has dropped significantly, and cannot fully meet the needs of human-computer interaction. Therefore, explore natural, efficient, and intelligent human-computer interaction. It has become one of the important topics in the field of computer research. [0003] In the field of human-computer interaction, the human eye, as an i...

AI Technical Summary

Problems solved by technology

[0004] However, there are relatively few researches on eye-controlled mouse technology at present. Most of the methods use special wearable devices such as glasses and helmets to realize the function of eye-controlled mouse. The cost is high and the support ecology is poor. Features are detected and positioned to achieve the purpose of controlling the mouse (for example, there is a known eye control method based on Kinect camera and depth information), but in the multi-screen scene with the increasing screen size, the line of sight caused by the human eye The range of change is limited, and it is difficult to solve the large-scale cross-screen mouse following technology only by relying on the characteristics of the human eye, and the structure of the human eye is complex, and the physical modeling is difficult. Low, therefore, the accuracy and efficiency of current research on eye-controlled mouse technology is low

Images