Humanoid robot gait control method based on model correlated reinforcement learning

Abstract

The invention discloses a humanoid robot gait control method based on model correlated reinforcement learning. The method comprises steps of 1) defining a reinforcement learning framework for a stable control task in forward and backward movements of a humanoid robot; 2) carrying out gait control of the humanoid robot with a model correlated reinforcement learning method based on the sparse online Gaussian process; and 3) improving a motion selection method of a reinforcement learning humanoid robot controller by a PID controller, and taking the improved operation as an optimizing initial point for the PID controller obtaining the motion selection operation of the reinforcement learning controller. The invention utilizes reinforcement learning to control gaits of the humanoid robot in movement, and thus the movement control of the humanoid robot can be automatically adjusted via interaction with the environment, a better control effect is achieved, and the humanoid robot is enabled to be stable in forward and backward directions.

Description

technical field [0001] The invention relates to the field of walking stability control and reinforcement learning of a humanoid robot, in particular to a gait control method of a humanoid robot based on model-related reinforcement learning. Background technique [0002] When controlling the humanoid robot to walk, we usually use the theory of forward and inverse kinematics to obtain the static trajectory of each joint of the humanoid robot, and then use these trajectories to control the humanoid robot to walk. It’s just that the robot joint trajectories obtained in this way can only be used for walking on an ideal flat ground, but cannot walk on uneven ground, because these joint trajectories assume that the environment is a flat ground when planning, and there is no other The interference of factors, and the contact surface between the sole of the foot and the ground on the uneven ground is different from that on the flat ground. Therefore, when the robot walks on an uneve...

AI Technical Summary

Problems solved by technology

PID is a linear controller, which requires the environment to be a linear approximation model, but the humanoid robot system is a complex nonlinear model, so the PID controller cannot perfectly meet the control requirements of the system

[0004] In order to better control the walking stability of humanoid robots, the use of reinforcement learning to control humanoid robots has attracted widespread attention, but the application of reinforcement learning to the walking stability control of humanoid robots also faces many problems. The state and control actions of the human robot are continuous, and the space is too large, so the traditional reinforcement learning is not convenient to apply

The experimental cost of humanoid robot is too high, and reinforcement learning needs multiple learning and training to achieve better control effect

Images