Binocular-vision blind guiding instrument based on reinforcement learning

Abstract

The invention discloses a binocular-vision blind guiding instrument based on reinforcement learning. The binocular-vision blind guiding instrument based on reinforcement learning comprises a binocular-vision module, a voice module, a state set generating module, a reinforcement learning module and a GPS navigation module, wherein the binocular-vision module is used for processing images of a firstimage collecting module and a second image collecting module, generating a real-time depth map and establishing a three-dimensional scene; the voice module comprises a voice input module and a voiceoutput module, the voice input module comprises a voice recognition module, the voice recognition module is used for carrying out voice recognition, and the voice output module is used for outputtingbehavior instructions, positioning information and path navigation information of the reinforcement learning module in a voice manner; the state set generating module extracts environment characteristics of the three-dimensional scene through a convolution neural network, and a state set is generated; the reinforcement learning module comprises a reinforcement learning network, and the reinforcement learning network is used for learning the state conversion strategy with the state set as input to obtain output action instructions; the GPS navigation module is used for positioning and generating path planning according to voice recognition information. Real-time path planning signals of blind persons can be provided, and the action strategy of the blind persons is conveniently guided in real time.

Description

technical field [0001] The invention relates to a binocular visual guide for the blind, in particular to a binocular visual guide for the blind based on reinforcement learning. Background technique [0002] The current mainstream blind guide system is mainly aimed at generating simple voice reminder signals through simple processing of images, or detecting obstacles through distance measurement by ultrasonic transmitters, and sending out warning signals by alarm reminder devices. Using this strategy to correct the actions of the blind cannot provide more in-depth guidance on the actions of the blind, that is, it cannot provide more convenient real-time behavior guidance signals. At the same time, since the current blind guide only processes the data at a shallow level, the correctness and effectiveness of its behavior reminders are not optimistic. [0003] Chinese patent document CN 106859929 discloses a multifunctional blind guide device based on binocular vision, which us...

AI Technical Summary

Problems solved by technology

The walking direction suggestion is generated by simple processing of the image, and the correctness and effectiveness of the reminder are poor

Images