A Reinforcement Learning-Based Hierarchical Control Method for Salamander Robot Path Tracking

Abstract

A Reinforcement Learning-Based Hierarchical Control Method for Salamander Robotic Path Following. Aiming at the path tracking problem of the salamander bionic robot, a layered control framework is established, including the upper controller based on reinforcement learning and the lower controller based on inverse kinematics, which realizes the tracking target path of the salamander bionic robot. Specifically, for the upper controller, the state space representation, action space representation and reward function are designed based on the softActor‑Critic (actor‑critic) algorithm, which can improve tracking accuracy and eliminate static errors. For the underlying controllers, a leg controller and a spine controller based on inverse kinematics are established. Finally, the controller trained by the robot in the simulation environment is migrated to the real environment to verify the feasibility and generalization ability of the algorithm. Experimental results show that the present invention can better complete the control target, and shows a better control effect in terms of migration and generalization from simulation to reality.

Description

technical field [0001] The invention belongs to the technical field of bionic robot path tracking control, in particular to a layered control method for salamander robot path tracking based on reinforcement learning. Background technique [0002] The development of robots for field search and rescue has become a hot field of robotics research. The search and rescue scenes are usually narrow and the terrain is complex. These places are dangerous and difficult for rescuers to reach. Using robots to assist rescue teams to explore and obtain information, The efficiency of rescue can be improved. A key characteristic of animals is their ability to move efficiently in their environment. This basic but amazing ability is the result of millions of years of evolution. Its flexibility and energy efficiency far exceed the level of robots. Therefore, the bionic robot is designed according to the body structure of the animal, hoping to achieve the same control effect. The bionic robot i...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to solve the problems involved in the existing salamander robot control methods, such as the parameter optimization is complicated, the algorithm consumes a large amount of computing resources, etc., and to provide a layered control method for the salamander robot path tracking based on reinforcement learning

[0007] In order to solve the problem of salamander robot path tracking, the present invention adopts a hierarchical control method. The upper-layer policy network is trained with reinforcement learning to provide complex global decision-making, while the bottom-layer traditional controller implements commands from the upper-layer controller.

Images