Inverse reinforcement learning method based on probability density estimation

Abstract

The invention discloses an inverse reinforcement learning method based on probability density estimation. The method comprises the following steps: initializing a strategy model; obtaining expert demonstration; estimating the state distribution probability density of experts by utilizing expert demonstration; estimating expert state action joint distribution probability density by utilizing expert demonstration; collecting a strategy state sample by utilizing strategy and environment interaction; estimating strategy state distribution probability density by using a state sample; restoring a reward function by utilizing the expert state distribution probability density and the expert state action joint distribution probability density; optimizing the strategy by using a PPO method; repeating the steps until the model converges; removing model parameters, and outputting a strategy model. The method can be integrated into various existing inverse reinforcement learning frameworks, and the efficiency of various inverse reinforcement learning algorithms is greatly improved; meanwhile, the calculation complexity of various inverse reinforcement learning frames can be remarkably reduced, the sample utilization rate of the inverse reinforcement learning frames is improved, and the convergence speed of the inverse reinforcement learning frames is increased.

Description

technical field [0001] The invention relates to artificial intelligence technology, in particular to an inverse reinforcement learning method based on probability density estimation. Background technique [0002] Reinforcement learning is an artificial intelligence framework that uses interaction with the environment to learn knowledge and skills. In RL, the agent needs to collect the reward signal sent by the environment to find the learning goal. However, in some real-world tasks, the reward function of the environment is difficult to define. The inverse reinforcement learning method is a typical method of designing reward signals. [0003] The inverse reinforcement learning method was first proposed by Ng and Russel. Inverse reinforcement learning uses the help of expert demonstrations to restore the reward function of the environment. [0004] Recently, the inverse reinforcement learning method via generative adversarial networks is a mainstream inverse reinforcement l...

AI Technical Summary

Problems solved by technology

[0006] The main purpose of the present invention is to overcome the defects of the above-mentioned background technology, provide an inverse reinforcement learning method based on probability density estimation, and solve the problems of low training efficiency, high computational complexity and poor convergence performance of the existing inverse reinforcement learning framework

Images