Mechanical arm motion planning method based on deep reinforcement learning

Abstract

The invention discloses a mechanical arm motion planning method based on deep reinforcement learning. The method comprises the following steps of 1, acquiring an environment image once before a mechanical arm moves, wherein the environment image comprises the mechanical arm in an initial state, a moving target point and an intermediate obstacle; 2, according to the acquired environment image, utilizing the target partitioning algorithm to separate a prohibited area, a working area and a target position from one another, and reconstructing a planning space; 3, dividing the reconstructed planning space into three-dimensional grid spaces, and establishing binarized grid spaces; 4, solving the corresponding analytical solution of each joint of the mechanical arm under known terminal coordinates by utilizing the robot inverse kinematics, and determining the relative positional relationships between the mechanical arm and the planned space boundary, the prohibited area boundary and the moving target under the global coordinate system; and 5, planning the motion strategy for the mechanical arm and acquiring the optimal motion strategy, so that the mechanical arm moves to the target position at the minimum cost under the premise of avoiding the obstacle.

Description

technical field [0001] The invention relates to a dynamic intelligent planning method based on reinforcement learning, in particular to a motion planning method of a 6-axis cooperative mechanical arm. Background technique [0002] The traditional trajectory planning of the manipulator mostly adopts the method of trajectory interpolation, combined with high-order polynomials to smooth the position, speed and acceleration of the manipulator. In this method, the motion scheme of the manipulator is relatively fixed, and it is impossible to actively avoid the environment. of obstacles. As the performance requirements of the robotic arm in the industrial field are getting higher and higher, and the robotic arm is required to complete more interactive or dynamic complex space tasks, the traditional method is no longer applicable. [0003] In recent years, reinforcement learning methods have been increasingly used in robot control tasks. Traditional reinforcement learning methods s...

AI Technical Summary

Problems solved by technology

Before DQN, it was widely believed that learning value functions using large nonlinear function approximators was difficult and unstable

The algorithm combines deep learning with reinforcement learning, and can use a function-like approximator to learn the value function in a stable manner. In order to minimize the correlation between samples, the network uses samples in the buffer for offline training, but it is difficult to cope with High-dimensional continuous action space

The Actor-Critic method can handle continuous action space problems, but the use of random strategies makes it difficult for the network to converge

Images