Strategy selection method based on Actor-Critic framework in deep reinforcement learning

Abstract

The invention discloses a strategy selection method based on Actor-Critic framework in deep reinforcement learning. According to the method, a plurality of strategies are kept in an Actor at the sametime; in the reinforcement learning training process, an action state value function in Critic is used for estimating the value expectation of a strategy, an advantage strategy in the strategy is selected or integrated in real time according to the expectation to be output or updated, the purpose is to improve the training speed, and an effective local strategy is generated in training. Therefore,the method has the technical effects of improving the sampling efficiency, increasing the parameter search space and the like for the strategy gradient-based reinforcement learning algorithm using the Actor-Critic framework.

Description

technical field [0001] The invention belongs to the technical field of reinforcement learning, and in particular relates to a strategy selection method based on an Actor-Critic framework in deep reinforcement learning. Background technique [0002] A reinforcement learning agent interacts with an environment by receiving observations that characterize the current state of the environment and, in response, performing actions from a predetermined set of actions; some reinforcement learning agents use neural networks to select The action performed. [0003] Actor-Critic is a commonly used framework in reinforcement learning, where Actor is responsible for outputting the current execution action, and Critic is responsible for estimating the value of the current action or the value of the current state; usually there will be an Actor and a Critic, according to the reinforcement learning algorithm used Simultaneously update the parameters of both to train an excellent reinforceme...

AI Technical Summary

Problems solved by technology

But in this way, different low-level strategies are usually trained with different simple tasks first, which requires the definition of additional tasks, increasing the complexity of the complete training process

Images