Method for improving continuous control stability of intelligent agent

Abstract

The invention discloses a method for improving the continuous control stability of an intelligent agent, and the method comprises the steps: 1, enabling the intelligent agent to interact with an environment according to a preset continuous control task, collecting interaction data, and enabling the interaction data to serve as training data to be put into an experience playback pool; step 2, training the conservative state scoring network in a mode of taking a minimum value from an objective function of the conservative state strategy evaluation module; step 3, calculating an objective function of the conservative state strategy promotion module in combination with the trained conservative state scoring network, and training a strategy function of the conservative state strategy network by taking a maximum value until a strategy with the strongest stability is obtained; and step 4, the intelligent agent executes subsequent continuous control by using the strategy with the strongest stability. According to the method, the robustness of the strategy obtained by agent reinforcement learning training during migration from a training environment to a real environment is effectively enhanced, and the stability of agent continuous control is improved.

Description

technical field [0001] The invention relates to the field of continuous control of intelligent bodies, in particular to a method for improving the stability of continuous control of intelligent bodies. Background technique [0002] Deep reinforcement learning algorithms have achieved great success in areas such as robot control and game intelligence. In tasks represented by simulated robot control, it is usually necessary to deal with continuous state and action spaces, so this type of problem is usually abstracted as a continuous control problem in agent reinforcement learning. Taking the classic reinforcement learning algorithm soft actor-critic algorithm (soft actor-critic, SAC algorithm) in the continuous control problem as an example, it effectively enhances the sample reuse and The training stability problem is applied to a series of simulation robot control tasks represented by the MuJoCo simulation environment, and excellent performance has been achieved in multiple...

AI Technical Summary

Problems solved by technology

[0006] However, in many real scene tasks, it is often difficult to predict the specific form of the disturbance in advance, and in some tasks it cannot even be assumed that the disturbance follows a certain form

For example, when a robot that needs to be controlled and trained performs tasks in a new environment, it may not be possible to know the differences between the new environment and the training environment; when the difference in the environment comes from complex air flow or random errors in the manufacturing process of mechanical materials, It is even difficult to effectively model these perturbations

In other words, in many task scenarios, it is not possible to enhance the robustness of the trained strategy by introducing the above RMDP-based method in the training environment

Images