Cooperative control method for multi-intersection signal lamp based on Q value migration depth reinforcement learning

Abstract

The invention provides a cooperative control method for multi-intersection signal lamp based on Q value migration depth reinforcement learning, and belongs to the crossing field of machine learning and intelligent traffic. A multi-intersection traffic network of an area is modeled into a multi-Agent system firstly. Each Agent simultaneously considers the influence of adjacent Agent actions at themost recent moment in the learning strategy process, so that multiple Agents can cooperatively conduct signal lamp control of multi-intersection. Each Agent adaptively controls one intersection through a deep Q network, and a network input is a discrete traffic state code of the original state information of the corresponding intersection. An optimal action Q value of the adjacent Agent at the most recent moment is transferred to the loss function of the network in the process of learning. The cooperative control method for multi-intersection signal lamp based on the Q value migration depth reinforcement learning can improve the traffic flow of the regional road network and the utilization rate of the road and can reduce the queuing length of the vehicle to relieve the traffic jam, and hasno limitation on the structure of each intersection.

Description

technical field [0001] The invention belongs to the intersecting field of machine learning and intelligent transportation, and relates to a multi-intersection signal lamp cooperative control method based on Q-value migration deep reinforcement learning. Background technique [0002] Traffic congestion has become an urgent challenge for urban traffic, but the existing infrastructure road facilities are difficult to expand due to space, environmental and economic constraints. Therefore, the optimal control of traffic lights is one of the effective ways to solve this problem. Through the adaptive control of signal lights, the traffic of the regional road network can be optimized to reduce congestion and carbon dioxide emissions. [0003] At present, different machine learning methods have been used in the research of urban traffic light control, mainly including fuzzy logic, evolutionary algorithm and dynamic programming. Control based on fuzzy logic usually establishes a set...

AI Technical Summary

Problems solved by technology

[0009] Aiming at the problems that the traditional signal light control method has difficulty in extracting traffic state features, the lack of effective collaborative strategies among signal lights at multiple intersections, and the algorithm relies too much on the intersection structure, etc., the present invention proposes a cooperative deep Q network with Q value migration (QT -CDQN) for coordinated control of signal lights at multiple intersections

Images