A Network Optimal Tracking Control Method Based on Off-Policy q-Learning

Abstract

A network optimal tracking control method based on non-policy Q-learning relates to a network tracking control method. The present invention proposes a new non-policy Q-based tracking control problem for networked control systems with packet loss. The learning method fully utilizes measurable data, and realizes the system to track the target in an approximately optimal way when the system model parameters are unknown and there is data loss in network communication. The invention does not require the system model parameters to be known, and uses the measurable data of the network control system to learn the optimal tracking control strategy based on the state feedback of the predictor; and the algorithm can ensure the unbiasedness of the iterative Bellman equation solution based on the Q-function. The simulation verifies the effectiveness of the proposed method.

Description

technical field [0001] The invention relates to a network tracking control method, in particular to a network optimal tracking control method based on non-policy Q-learning. Background technique [0002] Reinforcement learning is a learning method that uses "trial and error" interactions with the environment to find strategies that lead to the largest expected cumulative reward. According to whether the behavior strategy is consistent with the target strategy in the learning process, reinforcement learning can be divided into On-policy learning and Off-policy learning. If during the learning process, the behavior strategy of action selection is consistent with the target strategy of learning improvement, the method is called policy learning, otherwise it is called non-policy learning. [0003] Off-policy RL has several advantages over on-policy learning and has desirable properties: (a) It solves the exploration-exploitation dilemma. The system adopts any behavioral strate...

AI Technical Summary

Problems solved by technology

However, the use of off-policy Q-learning methods, compensating for packet loss, and solving the optimal tracking control problem in the case of unknown system model parameters has not been studied, which is the motivation of the present research

Images