More Web Proxy on the site http://driver.im/

research-article

Intelligent Traffic Signal Control with Deep Reinforcement Learning at Single Intersection

Authors:

Yayu GaoAuthors Info & Claims

ICCAI '21: Proceedings of the 2021 7th International Conference on Computing and Artificial Intelligence

Pages 399 - 406

https://doi.org/10.1145/3467707.3467767

Published: 24 September 2021 Publication History

Abstract

In this paper, we apply the Proximal Policy Optimization (PPO) algorithm in intelligent traffic signal control at a single intersection with eight lanes and four signal phases. The optimization goal is to minimize the average waiting time of vehicles so as to improve the traffic efficiency of the intersection. Extensive experiments are conducted in Simulation of Urban MObility (SUMO) to evaluate the performance of the proposed algorithm, and compare it with other classic algorithms including Deep Q-network (DQN), Advantage Actor Critic (A2C) and Fixed Time. Simulation results show that the proposed PPO algorithm outperforms the others under various traffic scenarios to different extent. The performance gain is significant under unbalanced traffic where one direction is saturated while the other is not, and becomes marginal when all the directions are saturated or unsaturated. PPO also demonstrates good portability and robustness over time-varying traffic patterns, while implies it could be a preferable option for implementation in real world intelligent traffic signal control systems.

References

[1]

[1] F. Rasheed, K. L. A. Yau, R. M. Noor, C. Wu, and Y. C. Low, “Deep Reinforcement Learning for Traffic Signal Control: A Review,” IEEE Access, vol. 8, pp. 208016–208044, Oct. 2020.

[2]

[2] K. A. Yau, J. Qadir, H. L. Khoo, M. H. Ling, and P. Komisarczuk, “A Survey on Reinforcement Learning Models and Algorithms for Traffic Signal Control,” ACM Computing Surveys, vol. 50, no. 3, pp. 1–38, June 2017.

Digital Library

[3]

[3] B. Abdulhai, R. Pringle, and G. J. Karakoulas, “Reinforcement learning for true adaptive traffic signal control,” Journal of Transportation Engineering, vol. 129, no. 3, pp. 278–285, May 2003.

[4]

[4] W. Marco, V. Jelle, V. Jilles, and K. Arne, “Intelligent traffic light control,” July 2004.

[5]

[5] L. A. Prashanth and B. Shalabh, “Reinforcement Learning With Function Approximation for Traffic Signal Control,” IEEE Transactions on Intelligent Transportation Systems, vol. 12, no. 2, pp. 412–421, June 2011.

[6]

[6] L. Li, Y. Lv, and F. Y. Wang, “Traffic signal timing via deep reinforcement learning,” IEEE/CAA Journal of Automatica Sinica, vol. 3, no. 3, pp. 247–254, July 2016.

[7]

[7] S. M. A. Shabestary, and B. Abdulhai, “Deep Learning vs. Discrete Reinforcement Learning for Adaptive Traffic Signal Control,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 286–293, Nov. 2018.

Digital Library

[8]

[8] C. Wan and M. Hwang, “Value-based deep reinforcement learning for adaptive isolated intersection signal control,” IET Intelligent Transport Systems, vol. 12, no. 9, pp. 1005–1010, Oct. 2018.

[9]

[9] X. Liang, X. Du, G. Wang, and Z. Han, “A Deep Reinforcement Learning Network for Traffic Light Cycle Control,” IEEE Transactions on Vehicular Technology, vol. 68, no. 2, pp. 1243–1253, Feb. 2019.

[10]

[10] V. Mnih, K. Kavukcuoglu, D. Silver, et al. “Playing atari with deep reinforcement learning,” arXiv preprint, Dec. 2013.

[11]

[11] V. Mnih, K. Kavukcuoglu, D. Silver, et al. “Human-level control through deep reinforcement learning,” Nature, pp. 529–533, Feb. 2015.

[12]

[12] G. Deepeka, C. Maria, and V. George, “Deep Reinforcement Learning for Autonomous Traffic Light Control,” in 2018 3rd IEEE International Conference on Intelligent Transportation Engineering (ICITE), pp. 214–218, Sept. 2018.

[13]

[13] H. Wei, G. Zheng, H. Yao, and Z. Li, “IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496–2505, Aug. 2018.

Digital Library

[14]

[14] G. Zheng, Y. Xiong, X. Zang, J. Feng, H. Wei, H. Zhang, Y. Li, K. Xu, and Z. Li, “Learning Phase Competition for Traffic Signal Control,” in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1963-1972, May 2019.

Digital Library

[15]

[15] Y. Du, W. Shangguan, D. Rong, and L. Chai, “RA-TSC: Learning Adaptive Traffic Signal Control Strategy via Deep Reinforcement Learning,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3275–3280, Oct. 2019.

Digital Library

[16]

[16] V. Mnih, A. P. Badia, M. Mirza, et al. “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning, pp. 1928–1937, Feb. 2016.

[17]

[17] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess,T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” Computer ence., Sept. 2015.

[18]

[18] S. G. Rizzo, G. Vantini, and S. Chawla, “Time Critic Policy Gradient Methods for Traffic Signal Control in Complex and Congested Scenarios,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1654–1664, July 2019.

Digital Library

[19]

[19] T. Chu, J. Wang, L. Codecà, and Z. Li, “Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 3, pp. 1086–1095, Mar. 2020.

[20]

[20] T. Tan, F. Bao, Y. Deng, A. Jin, Q. Dai, and J. Wang, “Cooperative Deep Reinforcement Learning for Large-Scale Traffic Grid Signal Control,” IEEE Transactions on Cybernetics, vol. 50, no. 6, pp. 2687–2700, June 2020.

[21]

[21] J. Schulman, F. Wolski, P. Dhariwal, et al. “Proximal policy optimization algorithms,” arXiv preprint, July 2017.

[22]

[22] D. Krajzewicz, J. Erdmann, M. Behrisch, and L. Bieker, “Recent development and applications of SUMO - Simulation of Urban MObility,” Systems and Mearsurements, vol. 5, no. 3&4, pp. 128–138, Dec. 2012. [Online]. Available: http://elib.dlr.de/80843/

[23]

[23] https://github.com/summerleft/ITS

Cited By

Su GYang J(2024)Enhancing the Robustness of Traffic Signal Control with StageLight: A Multiscale Learning ApproachEng10.3390/eng50100075:1(104-115)Online publication date: 8-Jan-2024
https://doi.org/10.3390/eng5010007
Pang AWang MChen YPun MLepech M(2024)Scalable Reinforcement Learning Framework for Traffic Signal Control Under Communication DelaysIEEE Open Journal of Vehicular Technology10.1109/OJVT.2024.33686935(330-343)Online publication date: 2024
https://doi.org/10.1109/OJVT.2024.3368693
Hu TLi Z(2024)A multi‐agent deep reinforcement learning approach for traffic signal coordinationIET Intelligent Transport Systems10.1049/itr2.12521Online publication date: 24-Jun-2024
https://doi.org/10.1049/itr2.12521
Show More Cited By

Recommendations

Traffic Signal Control Using Deep Reinforcement Learning with Multiple Resources of Rewards
PE-WASUN '19: Proceedings of the 16th ACM International Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous Networks

Intelligent traffic signal control is an effective way to solve the traffic congestion problem in the real world. One trend is to use Deep Reinforcement Learning (DRL) to control traffic signals based on the snapshots of traffic states. While most of ...
Feudal Multi-Agent Deep Reinforcement Learning for Traffic Signal Control
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

Reinforcement learning (RL) is a promising technique for optimizing traffic signal controllers that dynamically respond to real-time traffic conditions. Recent efforts that applied Multi-Agent RL (MARL) to this problem have shown remarkable improvement ...
Two-layer coordinated reinforcement learning for traffic signal control in traffic network
Highlights
- Develop a novel two-layer coordinated multi-agent reinforcement learning algorithm.
- Synergize traffic signals through local cooperation and global coordination.
- Optimization of vehicle emissions and traffic congestion as ...
Abstract
Intersection traffic signal control considering vehicle emissions has become an important topic, however, the decision complexity of traffic signal control increases dramatically in a dynamic traffic environment with multi-intersections. It is a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICCAI '21: Proceedings of the 2021 7th International Conference on Computing and Artificial Intelligence

April 2021

498 pages

ISBN:9781450389501

DOI:10.1145/3467707

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICCAI '21

ICCAI '21: 2021 7th International Conference on Computing and Artificial Intelligence

April 23 - 26, 2021

Tianjin, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
220
Total Downloads

Downloads (Last 12 months)68
Downloads (Last 6 weeks)7

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Su GYang J(2024)Enhancing the Robustness of Traffic Signal Control with StageLight: A Multiscale Learning ApproachEng10.3390/eng50100075:1(104-115)Online publication date: 8-Jan-2024
https://doi.org/10.3390/eng5010007
Pang AWang MChen YPun MLepech M(2024)Scalable Reinforcement Learning Framework for Traffic Signal Control Under Communication DelaysIEEE Open Journal of Vehicular Technology10.1109/OJVT.2024.33686935(330-343)Online publication date: 2024
https://doi.org/10.1109/OJVT.2024.3368693
Hu TLi Z(2024)A multi‐agent deep reinforcement learning approach for traffic signal coordinationIET Intelligent Transport Systems10.1049/itr2.12521Online publication date: 24-Jun-2024
https://doi.org/10.1049/itr2.12521
Agafonov AYumaganov AMyasnikov V(2023)Adaptive Traffic Signal Control Based on Neural Network Prediction of Weighted Traffic FlowOptoelectronics, Instrumentation and Data Processing10.3103/S875669902205001658:5(503-513)Online publication date: 3-Mar-2023
https://doi.org/10.3103/S8756699022050016
Agafonov AYumaganov A(2023)Cooperative Control of Traffic Signals and Vehicle TrajectoriesСовместное управление сигналами светофоров и траекториями движения транспортных средствInformatics and AutomationИнформатика и автоматизация10.15622/ia.22.1.122:1(5-32)Online publication date: 27-Jan-2023
https://doi.org/10.15622/ia.22.1.1
Agafonov AYumaganov AMyasnikov V(2023)Efficiency of Adaptive Traffic Signal Control in a Partially Connected Vehicle Environment2023 IX International Conference on Information Technology and Nanotechnology (ITNT)10.1109/ITNT57377.2023.10139039(1-4)Online publication date: 17-Apr-2023
https://doi.org/10.1109/ITNT57377.2023.10139039
Agafonov AYumaganov AMyasnikov V(2022)An Algorithm for Cooperative Control of Traffic Signals and Vehicle Trajectories2022 4th International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA)10.1109/SUMMA57301.2022.9973827(675-680)Online publication date: 9-Nov-2022
https://doi.org/10.1109/SUMMA57301.2022.9973827
Agafonov A(2022)Adaptive Traffic Light Control Using a Distributed Processing Approach2022 IEEE International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON)10.1109/SIBIRCON56155.2022.10016941(186-189)Online publication date: 11-Nov-2022
https://doi.org/10.1109/SIBIRCON56155.2022.10016941

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents