Cleaning robot optimal path planning method based on approximate model multi-step optimization

Abstract

The invention discloses a cleaning robot optimal path planning method based on approximate model multi-step optimization. The method comprises steps: a model, hyperparameters and an environment are initialized; an exploration strategy is selected; the model is updated with the current sample; a value function, a strategy and the current state are updated; whether the current plot is ended is judged and further, the current sampling trajectory is used to update a trajectory pool; a reconstructed sampling trajectory is used to update the trajectory pool; all trajectories in the trajectory pool are used to update the model; the model is adopted for planning; whether to reach the maximum plot number is judged; and finally, according to a learnt optimal strategy, the optimal cleaning robot planning path is acquired. Through adopting the sampling trajectory and the single sample to update the model, the model learning precision is improved; the model is then used for planning, the learning speeds of the value function, the strategy and the whole algorithm are improved; and the sample utilization efficiency is also improved, and thus, the optimal cleaning robot planning path is acquired by adopting the fewer samples in the shorter time.

Description

technical field [0001] The invention relates to a cleaning robot path planning method, in particular to a cleaning robot optimal path planning method based on approximate model multi-step optimization. Background technique [0002] The autonomous path planning of cleaning robots is a common problem in the field of control. In this problem, all possible states of the robot can be modeled as a state space, all actions that can occur in it can be modeled as an action space, and the next possible state reached after the current state has an action can be modeled as a transition function, and will reach the next possible state. The immediate reward a state obtains is modeled as a reward function, which transforms the problem into a Markov decision process. The conventional idea to solve this problem is to use discrete reinforcement learning methods, such as Q-learning and SARSA algorithm to solve, such as direct discrete state space and action space, that is, divide the state sp...

AI Technical Summary

Problems solved by technology

When the learned model is accurate enough, it can accelerate the convergence of strategies and algorithms. When the learned model is not accurate enough, using the model for planning will hinder the acquisition of the optimal solution of the strategy.

Images