Distributed traffic signal control method based on generative adversarial network and reinforcement learning

Abstract

The invention discloses a method for accelerating a reinforcement learning (RL) algorithm by using an improved generative adversarial network (WGAN-GP) and applying the algorithm to regional traffic signal control, and the advantages of the generative adversarial network in the aspect of data generation and the advantages of the reinforcement learning algorithm in the aspect of learning a control strategy are applied to the regional traffic signal control. And the learning speed and effect of the signal control strategy can be effectively improved. The method mainly comprises: giving a control framework of multi-agent reinforcement learning in regional traffic signal control, and meanwhile, defining all elements of reinforcement learning, namely, states, actions, rewards and objective functions; defining a generative adversarial network structure; and proposing a data interaction framework of the generative adversarial network and reinforcement learning.

Description

technical field [0001] The invention relates to the field of traffic management and control, in particular to a distributed traffic signal control method based on generating confrontation networks and reinforcement learning. Background technique [0002] It is generally believed that the adaptive traffic signal control method is one of the effective methods to cope with the increasing traffic demand and relieve road traffic congestion. Compared with the early adaptive signal control methods, such as SCOOT, SCATS, and OPAC, the traffic signal control method based on reinforcement learning can learn the signal control scheme through the interaction with the traffic system without complex calculation formulas. [0003] There have been research attempts to apply actor-critic algorithm (A2C) to distributed traffic signal control. However, there are still several problems: (1) In the distributed signal control, the communication between agents is not enough, which may easily lead...

AI Technical Summary

Problems solved by technology

However, there are still several problems: (1) In the distributed signal control, the communication between agents is not enough, which may easily lead to separate operations between intersections, and the effect of regional joint control is not good; (2) A2C as an online strategy Reinforcement learning algorithms need to accumulate data through real-time interaction with the environment, and then use it for model training, so there are disadvantages of low data utilization efficiency and low model training efficiency; (3) when you want the trained A2C model to continue learning in practical applications When the data accumulation efficiency is low, the model parameters cannot be updated in time according to the traffic status.

Images