Interaction method and interaction device based on lip language

Abstract

The invention discloses an interaction method and an interaction device based on a lip language. The interaction method comprises the following steps of obtaining the depth image, the infrared image and the color image of a target human body object; obtaining the lip part area images of the target human body object from the depth image, the infrared image and the color image respectively; extracting lip part characteristics from the lip part area images, carrying out fusion treatment on lip part characteristics extracted from the depth image, the infrared image and the color image, and then carrying out lip language recognition; converting the result of the lip language recognition into a corresponding operating command, and carrying out interaction according to the operating command. Through the way, relative to a way of conventionally directly obtaining image information through a color camera, the interaction method and the interaction device are difficultly influenced by an environment such as the intensity of a light ray; the hit rate of image recognition can be effectively improved; the hit rate of the lip language recognition is improved; the hit rate of the lip language recognition is further improved; finally, the execution rate and the operation accuracy of the interaction can be effectively ameliorated.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to an interaction method and an interaction device based on lip language. Background technique [0002] In the field of artificial intelligence and image processing, many functions can be realized by using the image information of the target, such as analyzing the user's expression or action information, etc. Image acquisition and recognition has always been a hot research topic, involving many aspects of users' daily life and scientific research. [0003] For example, the technology of recognizing the user's face, such as lip language recognition, can improve the accuracy of somatosensory interaction and voice recognition, thereby further bringing a more comfortable interactive experience. [0004] However, in the prior art, lip language recognition is generally based on a color camera to obtain the user's facial information. This method is easily affected by the environm...

AI Technical Summary

Problems solved by technology

[0004] However, in the prior art, lip language recognition is generally based on a color camera to obtain the user's facial information. This method is easily affected by the environment. For example, if the light is too strong or too weak, it will directly affect the accuracy of image acquisition, resulting in a very low recognition rate. Low, resulting in a low success rate of interaction

Images