Gait Control Method of Humanoid Robot Based on Model-Dependent Reinforcement Learning

Abstract

The invention discloses a humanoid robot gait control method based on model correlated reinforcement learning. The method comprises steps of 1) defining a reinforcement learning framework for a stable control task in forward and backward movements of a humanoid robot; 2) carrying out gait control of the humanoid robot with a model correlated reinforcement learning method based on the sparse online Gaussian process; and 3) improving a motion selection method of a reinforcement learning humanoid robot controller by a PID controller, and taking the improved operation as an optimizing initial point for the PID controller obtaining the motion selection operation of the reinforcement learning controller. The invention utilizes reinforcement learning to control gaits of the humanoid robot in movement, and thus the movement control of the humanoid robot can be automatically adjusted via interaction with the environment, a better control effect is achieved, and the humanoid robot is enabled to be stable in forward and backward directions.

Description

Technical field [0001] The invention relates to the field of humanoid robot walking stability control and reinforcement learning, in particular to a humanoid robot gait control method based on model-related reinforcement learning. Background technique [0002] When controlling the walking of the humanoid robot, we usually use the theory of forward and inverse kinematics to obtain the static trajectory of each joint of the humanoid robot, and then use these trajectories to control the humanoid robot to walk. It’s just that the robot joint trajectories obtained in this way can only be used for walking on an ideal flat ground, and cannot walk on uneven ground, because these joint trajectories are planned on the assumption that the environment is a flat ground, and nothing else Factor interference, and the contact surface between the sole of the foot and the ground on uneven ground is different from that on flat ground. Therefore, when the robot is walking on an uneven surface, an o...

AI Technical Summary

Problems solved by technology

PID is a linear controller, which requires the environment to be a linear approximation model, but the humanoid robot system is a complex nonlinear model, so the PID controller cannot perfectly meet the control requirements of the system

[0004] In order to better control the walking stability of humanoid robots, the use of reinforcement learning to control humanoid robots has attracted widespread attention, but the application of reinforcement learning to the walking stability control of humanoid robots also faces many problems. The state and control actions of the human robot are continuous, and the space is too large, so the traditional reinforcement learning is not convenient to apply

The experimental cost of humanoid robot is too high, and reinforcement learning needs multiple learning and training to achieve better control effect

Images