Deep reinforcement learning training method and computer readable storage medium

Abstract

The invention provides a deep reinforcement learning training method and a computer readable storage medium, and the method comprises the steps: assigning a situation number, and initializing a weight parameter of a deep reinforcement learning multi-head neural network model; carrying out stochastic decision by an intelligent agent and collecting samples and storing the samples in an experience playback buffer area; according to the situation number, adopting an online clustering algorithm to achieve self-adaptive situation division to obtain situation division and situation centers at the end of the current moment; randomly sampling samples from the experience playback buffer area, and distributing each sample to the closest situation; training a shared feature extractor and a weight parameter of a corresponding output head according to a situation corresponding to the sample, synchronously updating the weight parameters of other output heads in combination with knowledge distillation loss, and estimating a value function; and in the next time step, through the intelligent agent, continuing to make a decision according to the value function, making samples collected and stored in the experience playback buffer area, and repeating the steps until the pre-specified number of training times is completed or convergence is achieved. The stability and plasticity of model training are improved.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a deep reinforcement learning training method and a computer-readable storage medium. Background technique [0002] In the field of reinforcement learning, the powerful learning ability of deep neural networks makes it possible for agents to learn effective control strategies directly from high-dimensional continuous environments. In theory, in order to achieve stable training performance, neural networks generally require training data to satisfy the characteristics of independent and identical distribution (i.i.d.), which is almost impossible to hold in the general reinforcement learning paradigm. The training mode of reinforcement learning while exploring and learning makes the training data have highly time-dependent and non-stationary inherent properties. Since the distribution of training data used by the neural network is different before and after ...

AI Technical Summary

Problems solved by technology

[0006] The present invention provides a deep reinforcement learning training method and a computer-readable storage medium to solve the catastrophic interference and forgetting problems commonly encountered in the training process of the existing deep reinforcement learning neural network model

Images