A Deep Value Function Learning Method for Agents Based on State Distribution Perceptual Sampling

Abstract

The invention discloses an agent deep value function learning method based on state distribution perception sampling, which is used for the agent to quickly learn the value function with fewer samples. It specifically includes the following steps: 1) Obtain empirical data for the agent to learn the value function, and define the algorithm goal; 2) Use convolutional neural network to preprocess the empirical data to obtain a feature set with stronger expressive ability; 3) In the In the feature space of the empirical data set, an unsupervised method is used to cluster the empirical data set; 4) According to the state distribution of the empirical data set, the sample state distribution-aware sampling method based on uniform sampling and cluster equal probability sampling interpolation is used for sampling; 5 ) The agent uses the sampled samples to learn the value function. The invention is applicable to the game game problem in the reinforcement learning field, and can achieve better results quickly under the condition of less sample size.

Description

technical field [0001] The invention belongs to the field of enhanced learning, which is a branch of the field of machine learning, and in particular relates to a sample sampling method based on state distribution perception of empirical data. Background technique [0002] Sample selection is an important issue in the field of machine learning, and different selection methods directly affect the quality of model learning. In the field of reinforcement learning, sample sampling from empirical datasets can help overcome the problem of sample correlation and forgetting earlier samples. The goal of sample sampling is to select samples from the sample set that can speed up model convergence and enhance the agent's ability to perceive the environment. The traditional method generally uses random uniform sampling to sample from the experience data set. This method is likely to cause sample imbalance and make the learning speed of the agent slower. [0003] The existing sampling m...

AI Technical Summary

Problems solved by technology

However, this method does not fundamentally solve two problems: 1. The importance of samples in different states is close, but the number of generated is quite different. According to what standard to sample from the empirical data set can avoid redundant samples. Oversampling; 2. Since the sample itself is very high-dimensional, huge in number and constantly generated, it is a key factor to effectively analyze a large number of high-dimensional samples. How to efficiently sample from a large number of continuously generated sample sets

Images