HoloLens-based blind person navigation system and method thereof

Abstract

The invention discloses a HoloLens-based blind person navigation system and a method thereof, which belong to the field of mixed reality. According to the invention, a blind person can be helped to send a voice destination instruction in a strange building, and the blind person is guided to arrive at the destination through voice navigation after the HoloLens recognizes the destination; the blindperson can be helped to detect obstacles ahead in the advancing process of the blind person, and the blind person is reminded to avoid the obstacles through voice. According to the method, a built target building model is imported into Unity 3D to build a virtual scene, and after modules such as a navigation module and an obstacle recognition module are added into a script, an application programis published into HoloLens. The real environment information collected by an environment camera and a depth camera can be processed by utilizing a holographic processing unit of the system, the position and posture of the camera are judged, and obstacle information in the environment is identified. After the blind person wears the HoloLens, the blind can safely arrive at a destination only throughvoice navigation prompt of the glasses without help of others and understanding of the structure of a strange target building, and great convenience is brought to life of the blind person.

Description

technical field [0001] The invention relates to a human-computer interaction technology that uses voice to provide indoor navigation for the blind, in particular to a system and method for blind navigation based on HoloLens. Background technique [0002] As early as the 1960s, people had a preliminary exploration of augmented reality (AR) technology. Thanks to the advancement of hardware, a large number of computing devices with high performance and low power consumption came out one after another. In 2010, AR and wearable devices developed rapidly. In 2012, the first generation of Google Glass came out, and the smart hardware of AR technology became a trend. Giants such as Apple, Intel, Qualcomm, and Microsoft joined in one after another. Augmented reality devices emerged one after another, such as Epson’s Moverio ProBT200 , Meta2, Vuzix M100, Daqri smart helmet, ODG R-7, Liangfengtai HiAR Glasses, etc., the prospect of AR is widely optimistic. In 2015, the first generatio...

AI Technical Summary

Problems solved by technology

In July 2017, Google Glass returned and launched the enterprise version of Google Glass, which has extended battery life, improved comfort, and has advantages such as a more powerful central processing unit and an 8-megapixel camera. However, the calculation process is simple, and the product positioning is still as a mobile phone. , Tablet and other equipment external extension

[0003] It is well known that the life of the blind is inconvenient. Most blind people rely on touch, hearing and other auxiliary tools, such as crutches, guide dogs, etc., for daily activities. These auxiliary facilities are relatively inaccurate and unreliable

For example, a blind guide stick can only detect obstacles in a small area of ​​ground space in front of the blind, and it can also play a certain role in a relatively empty outdoor space. However, in some small indoor spaces, such as bedrooms, kitchens, etc., Blind people cannot accurately perceive the three-dimensional space around them, so it is easy to detect that the feet can move forward with crutches, but they are injured by facilities such as wardrobe doors and cabinets when walking forward.

In addition, blind people cannot view maps, cannot identify indoor target locations, such as using the toilet in an unfamiliar public environment, and blind people cannot recognize whether the front is a wall or a room with a closed door, all of which pose great challenges to the normal life of blind people

Although guide dogs can provide guidance services, guide dogs are also unable to familiarize themselves with unfamiliar places, and dogs cannot accurately understand human thoughts. Moreover, the cost of raising dogs is relatively high, and the time period for training guide dogs is long. Blind people put in a lot of extra effort

Images