Sign language recognition method based on track and random forest

Abstract

The invention provides a sign language recognition method based on a track and a random forest. The sign language recognition method mainly comprises the steps of collecting an original depth image and skeleton data; establishing a classification model based on a random forest and traversing segmentation nodes to calculate depth difference characteristics; judging a left branch or a right branch entering the tree model, and repeatedly executing the traversal step until reaching a segmentation node or a leaf node; extracting a vector and an angle between finger joint points as test features, and calculating a DTW distance between the test features and template features; and outputting an identification result according to the obtained DTW distance. According to the invention, the three-dimensional coordinate point of the finger is estimated according to the depth image; the defect that only weak feature description can be provided for the depth map acquired by the Kinect depth camera isovercome; blurring caused by the similarity between fingers and the area occupied by a hand image in the whole image are usually very small, the low signal-to-noise ratio and the detail loss of finger joint points are easily caused, and various defects of shielding, self-shielding and the like caused by the non-rigid characteristics of the human hand are also overcome.

Description

technical field [0001] The present invention relates to the technical field of sign language recognition, in particular, to a sign language recognition method based on trajectory and random forest. Background technique [0002] Human body perception technology continues to develop, and sign language recognition as an important branch has been widely promoted in many application fields. The development of sign language recognition system has been divided into the following categories by the current commonly used sign language recognition methods: [0003] Based on wearable devices, this method is not suitable for widespread promotion, because its interaction method is unnatural, and the use of data gloves is complicated and expensive, and it is currently only used in special occasions such as laboratory research Based on computer vision, based on RGB-D images The research on sign language recognition has developed rapidly in recent years. With the introduction of depth camer...

AI Technical Summary

Problems solved by technology

However, the resolution and accuracy of the currently launched depth sensors cannot meet the requirements; and due to the non-rigid body characteristics of the human hand, self-occlusion and mutual occlusion of the two hands are prone to occur, and the specific three-dimensional coordinates of the hand joint points cannot be obtained. These are all problems that ordinary depth cameras do not solve

[0004] Based on deep learning, it uses the network to learn a large amount of data, which greatly relies on ideal training samples, and big data is not easy to obtain and the cost is very high

And we have no way of knowing what features it selects, that is, feature selection is opaque, human intervention is impossible, parameter adjustment is difficult, and the selected features may only be applicable to a certain data set

[0005] Traditional sign language recognition does not take into account the detailed movements of the fingers. Random forests are used to input the entire image as a whole, often used as an action classifier. Most of them extract contours, histograms, and shape features from the entire image. sort

However, ordinary depth cameras can only provide weak feature description due to the accuracy and uniform distribution of human skin in color and surface properties, and the hand image usually occupies a small area in the entire image, which easily leads to low signal-to-noise ratio, making The detailed movement features of the fingers are missing

Images