Reinforcement learning exploration method and device based on generative adversarial mechanism

Abstract

The invention discloses a reinforcement learning exploration method and device based on a generative adversarial mechanism, and the method comprises the steps: constructing a first action value network, a second action value network, a state value network, a target state value network, a strategy network, a density model network and an identification network; updating the first action value network, the second action value network, the state value network, the target state value network, the strategy network, the density model network and the identification network based on a generative adversarial mechanism and a learning process of an offline reinforcement learning algorithm; and generating an updated strategy model according to the plurality of updated networks, and testing the strategymodel. According to the method, an exploration algorithm utilizing a correct decision acceleration and stable reinforcement learning training process in an exploration process is designed.

Description

technical field [0001] The invention relates to the technical field of machine learning, in particular to a reinforcement learning exploration method and device based on a generative confrontation mechanism. Background technique [0002] In recent years, computers have made huge leaps in computing and storage performance, triggering the rapid rise of deep learning. Deep learning has not only made great progress in the fields of image classification, speech recognition, and natural language processing, but also provides convenient conditions for the approximation of value functions in reinforcement learning and the expression of agent behavior strategies. Intensive learning combined with neural networks has the ability to process large-scale simulated data and complete iterative learning through gradient updates one after another, making AlphaGo, which defeated the world's Go masters, a reality. [0003] However, reinforcement learning still faces many problems, one of which...

AI Technical Summary

Problems solved by technology

[0005] There are still some deficiencies in the above exploration methods: the exploration method based on random ideas cannot provide sufficient exploration for the agent; the exploration method of designing the intrinsic incentive function causes the value estimation of the agent to fluctuate due to the attenuation nature of the intrinsic incentive, which affects the training stability

Images