Policy migration method based on probability

Abstract

The invention discloses a probability-based strategy migration method, and belongs to the technical field of artificial intelligence. The environment of continuous control tasks such as robot operation is affected by high dynamic and uncertainty, and actually, it is difficult to use a virtual environment to approach a real environment. The method comprises the following steps: constructing a Q function estimator of a probability through Monte Carlo dropout, and combining the Q function estimator with strategy gradient optimization, so that the algorithm has the capability of identifying environmental uncertainty. Specifically, through virtual environment training data acquisition, uncertainty decomposition and inference, strategy gradient optimization and real environment operation performance evaluation, decomposition and measurement of environment uncertainty are realized, and strategy learning efficiency and strategy operation performance are improved.

Description

technical field [0001] The invention relates to a probability-based strategy migration method, which belongs to the technical field of artificial intelligence. Background technique [0002] The poor performance of virtual-real policy transfer is an important factor restricting the in-depth application of reinforcement learning. For general continuous control learning problems, a common solution is to learn and train in a virtual environment, and transfer the trained policy network to the real environment at the cost of zero or few samples, involving two environments. For problems such as space robot operation, due to the need for a large number of ground reliability tests, at least three environments are required for migration, namely virtual environment, ground test environment, and real space environment. The environment of such tasks is affected by high dynamics and uncertainty. In fact, it is difficult to use the virtual environment to approach the real environment, whi...

AI Technical Summary

Problems solved by technology

The environment of such tasks is affected by high dynamics and uncertainties. In fact, it is difficult to use the virtual environment to approach the real environment, which restricts the further improvement of policy learning efficiency and policy operation performance.

Images