Reinforcement learning battle game AI training method based on information bottleneck theory

Abstract

The invention relates to a reinforcement learning battle game AI training method based on an information bottleneck theory. The method comprises the following steps of 1) initializing an AI training model; 2) performing decision interaction in a simulation environment through the game AI to obtain a sample training batch data set; 3) iteratively training an AI training model by adopting a reinforcement learning algorithm according to a sample training batch data set obtained by interaction between the game AI and the environment, and storing parameters of the AI training model in stages; and (4) fixing part of the stored parameters of the AI training models at different stages, training the remaining parameters again by using a reinforcement learning algorithm to perform fine adjustment to obtain the final AI training models of different levels of AIs, and generating an AI file of the battle game. Compared with the prior art, the method has advantages of high sampling efficiency, high training speed, high test flexibility, AI grading and the like.

Description

technical field [0001] The invention relates to the field of game intelligence AI learning, in particular to an AI training method based on information bottleneck theory for reinforcement learning and fighting games. Background technique [0002] In recent years, with the development of deep learning technology, many achievements have been made in the field of deep reinforcement learning. More and more methods combining deep learning and reinforcement learning algorithms (such as DQN, A2C, PPO, DDPG, etc.) are used in video game AI. However, in many cases, in reinforcement learning problems, the interaction cost between the agent and the environment is very high, so it is hoped that the algorithm can converge as quickly as possible to save training costs, that is, through the same sampling rate and learn a higher level of intelligent strategy. [0003] In the existing battle games, the man-machine duel mode is one of the important parts of the game. The existing game AI is ...

AI Technical Summary

Problems solved by technology

[0002] In recent years, with the development of deep learning technology, many achievements have been made in the field of deep reinforcement learning. More and more methods combining deep learning and reinforcement learning algorithms (such as DQN, A2C, PPO, DDPG, etc.) are used in video game AI. However, in many cases, in reinforcement learning problems, the interaction cost between the agent and the environment is very high, so it is hoped that the algorithm can converge as quickly as possible to save training costs, that is, through the same sampling rate, learn a higher level of intelligent strategy

[0003] In the existing battle games, the man-machine duel mode is one of the important parts of the game. The existing game AI is designed by artificially setting the strategy distribution and targeted action mapping, so the routine is often single and different. It has the flexibility of fighting between players. At the same time, in the existing method of reinforcement learning training game AI, using original pixels as input will have a lot of redundant information that will affect the efficiency of network learning and the speed of reinforcement learning algorithms.

In the deep learning experiment, it is shown that during the training process, the neural network first remembers the input through the mutual information of the input layer and the representation layer variables, and then compresses the input information according to the specific learning task to discard the useless redundancy. Remaining information, that is, reducing the mutual information between the input layer and the presentation layer, this process is the Information E-C Process, and the existing reinforcement learning algorithm has not yet optimized this information extraction process

Images