Robot navigation method based on pre-processing layer and deep enhanced learning

A technology of reinforcement learning and preprocessing layer, applied in the field of robot navigation, can solve problems such as poor generalization performance, and achieve the effect of excellent transfer performance

Active Publication Date: 2019-06-11
HARBIN INST OF TECH SHENZHEN GRADUATE SCHOOL
View PDF6 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Aiming at the defects or deficiencies in the prior art, the present invention provides a robot navigation method based on preprocessing layer and deep reinforcement learning to solve the problem of poor generalization performance of the navigation algorithm based on deep reinforcement learning from the virtual environment to the real environment , combine the preprocessing layer with deep reinforcement learning, receive environmental information and output correct actions through the preprocessing layer and deep reinforcement learning, and then enable the robot equipped with the above method to obtain navigation capabilities and have a strong ability to migrate from the virtual environment to reality The ability of the environment can be applied to the field of robot navigation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Robot navigation method based on pre-processing layer and deep enhanced learning
  • Robot navigation method based on pre-processing layer and deep enhanced learning
  • Robot navigation method based on pre-processing layer and deep enhanced learning

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment 1

[0085] For the navigation method of the present invention, in combination with Figure 2 to Figure 12 The present invention is further described in specific embodiments in practical application and calculation process:

[0086] build as image 3 In the virtual training environment shown, in the virtual training environment, the virtual sensor uses an RGB camera, the virtual robot uses a virtual TURTLEBOT model, the virtual environment uses GAZEBO (simulation robot software), and the communication layer uses the ROS multi-computer communication method. The settings are printed with The square of the number 9 is an obstacle, the number 2 printed on the wall is set as the target, and the number 4 and 8 are the left and right position information respectively.

[0087] Considering that in the training process there are such Figure 4 The difference between the virtual environment shown and the real environment is too large (the gray value matrix is ​​too different), and when deep...

specific Embodiment 2

[0113] Figure 10-Figure 12 Another specific embodiment of the navigation method of the present invention is specifically applied.

Embodiment 2

[0114] Embodiment 2 is a virtual training environment based on embodiment 1. The same virtual sensor has adopted an RGB camera, and the virtual robot has adopted a virtual TURTLEBOT model. The square with the "fire" picture is set as an obstacle, the rescued person printed on the white paper is set as the target, and the real robot is set as the rescuer.

[0115] Adopt the same method as embodiment 1 to train the rescue robot, the observation statistics are obtained as follows Figure 10 The number of training iteration steps of the virtual robot navigation task in the virtual environment is shown. It can be seen that with the increase of training rounds, the number of steps for the robot to complete the task gradually decreases until the deep reinforcement learning converges at about 120,000 rounds.

[0116] Migrate the results of deep reinforcement learning in the virtual environment to the real environment. Specifically, after the training in the virtual environment is comp...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a robot navigation method based on a pre-processing layer and deep enhanced learning. According to the robot navigation method, a virtual pre-processing layer is arranged in aconstructed virtual training environment, and a realistic pre-processing layer is arranged in a realistic environment; and information with the same significance is output by the realistic pre-processing layer and the virtual pre-processing layer, thus a deep enhanced learning result in the virtual training environment is transplanted onto a robot navigation system in the realistic environment soas to realize navigation. The problem of poor generalization performance in the process that a navigation algorithm based on deep enhanced learning is transferred from the virtual environment to therealistic environment is solved, the pre-processing layer and deep enhanced learning are combined, environment information is received though the pre-processing layer and deep enhanced learning, accurate actions are output, thus the robot carrying the robot navigation method obtains the navigation ability and has the high ability to transfer from the virtual environment to the realistic environment, and the robot navigation method can be applied to the field of robot navigation.

Description

technical field [0001] The invention relates to the technical field of robot navigation, in particular to a robot navigation method based on a preprocessing layer and deep reinforcement learning. Background technique [0002] In the past two decades, robots have become more common and more important in many human activities. [0003] However, due to the complex and unpredictable environment, most robots realize their navigation through manual or semi-automatic operation. Although it provides the possibility to deal with unforeseen environmental conditions. But humans are needed to understand the sensory data obtained by the sensors and make decisions to drive the robot. Therefore, mobile robots require navigation systems with higher levels of intelligence and autonomy to allow them to autonomously make optimal decisions in complex environments. [0004] In deep reinforcement learning, the robot interacts with the environment, that is, by performing actions in the environm...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G05D1/02G05D1/00
Inventor 许杰雄于刚黄思静张畅帅凯鹏蒋境伟
Owner HARBIN INST OF TECH SHENZHEN GRADUATE SCHOOL
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products