Static sign language letter recognition system and method based on Kinect sensor

Abstract

The invention relates to the field of computer vision and intelligent human-machine interaction, in particular to a human-machine interaction system based on machine vision and an interaction method of the system, and provides a method for carrying out static sign language letter recognition based on an improved SURF algorithm by combining a Kinect sensor. The Kinect sensor collects the depth image of a target area to carry out hand pixel area division, and interference caused by illumination changes and complex backgrounds can be eliminated. The improved SURF algorithm is used for extracting feature points, meanwhile, the self-adaption radius r is set, SURF feature points are screened grade by grade in the neighbourhood with r as the radius by comparing the number of the feature points and the feature point distance, the recognition rate is greatly improved, and the robustness of recognition work on the skin color, the illumination changes, the complex backgrounds and other environmental factors, angle changes and scale changes is guaranteed. To solve the problem that SURF feature vector dimensions are high, an SVM one-to-one classification method is adopted, SURF feature descriptors are classified and trained, and a recognition result is obtained.

Description

technical field [0001] The invention relates to the field of computer vision and intelligent human-computer interaction, in particular to a human-computer interaction system and an interaction method based on machine vision. Background technique [0002] With the wide application of computers, human-computer interaction (Human Computer Interaction, HCI) has become an important part of people's daily life. The cognitive habits and forms that humans naturally communicate with the natural world are the development direction of human-computer interaction. Therefore, researchers are also working hard to make future terminals able to hear, see, speak, and feel. Simply put, human-computer interaction is the interaction between humans and computers. From keyboard to mouse control, then from voice to touch, and then to multi-touch, with the expansion of the human-computer interaction mode and the continuous penetration of non-professional people, the human-computer interaction mode...

AI Technical Summary

Problems solved by technology

However, in order to overcome the interference of complex background and illumination changes on the recognition work, this method adopts the depth image as the image to be detected, and the pixel value information of the depth image is converted from the distance value information. After the region segmentation results, the palm part is easy to calculate the wrong feature points due to the extremely similar pixel value information, which will interfere with the next recognition work

At the same time, the SURF eigenvector is 64-dimensional, and the eigenvector with too high dimension will also bring some interference to the recognition work.

Images