Reinforcement learning-based multi-agent system for network traffic signal control
- 1 January 2010
- journal article
- Published by Institution of Engineering and Technology (IET) in IET Intelligent Transport Systems
- Vol. 4 (2), 128-135
- https://doi.org/10.1049/iet-its.2009.0070
Abstract
A challenging application of artificial intelligence systems involves the scheduling of traffic signals in multi-intersection vehicular networks. This paper introduces a novel use of a multi-agent system and reinforcement learning (RL) framework to obtain an efficient traffic signal control policy. The latter is aimed at minimising the average delay, congestion and likelihood of intersection cross-blocking. A five-intersection traffic network has been studied in which each intersection is governed by an autonomous intelligent agent. Two types of agents, a central agent and an outbound agent, were employed. The outbound agents schedule traffic signals by following the longest-queue-first (LQF) algorithm, which has been proved to guarantee stability and fairness, and collaborate with the central agent by providing it local traffic statistics. The central agent learns a value function driven by its local and neighbours' traffic conditions. The novel methodology proposed here utilises the Q-Learning algorithm with a feedforward neural network for value function approximation. Experimental results clearly demonstrate the advantages of multi-agent RL-based control over LQF governed isolated single-intersection control, thus paving the way for efficient distributed traffic signal control in complex settings.Keywords
This publication has 10 references indexed in Scilit:
- A Collaborative Reinforcement Learning Approach to Urban Traffic Control OptimizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2008
- A Novel Signal-Scheduling Algorithm With Quality-of-Service Provisioning for an Isolated IntersectionIEEE Transactions on Intelligent Transportation Systems, 2008
- Chapter 11 ITS and Traffic ManagementPublished by Elsevier ,2006
- Review of road traffic control strategiesProceedings of the IEEE, 2003
- Reinforcement Learning for True Adaptive Traffic Signal ControlJournal of Transportation Engineering, 2003
- A multivariable regulator approach to traffic-responsive network-wide signal controlControl Engineering Practice, 2002
- Analysis of intersection delay under real-time adaptive signal controlTransportation Research Part C: Emerging Technologies, 1999
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Reinforcement Learning: A SurveyJournal of Artificial Intelligence Research, 1996
- A theory of cerebellar functionMathematical Biosciences, 1971