Stackelberg Differential Lane Change Game Based on MPC and Inverse MPC
Authors: 
Qingyu Zhang, Reza Langari, H. Eric Tseng, Shankar Mohan, Steven Szwabowski, Dimitar FilevAuthors Info & Claims
Pages 8473 - 8485
Abstract
A Stackelberg differential game theoretic model predictive controller is proposed for an autonomous highway driving problem. The hierarchical controller’s high-level component is the two-player Stackelberg differential lane change game, where each player uses a model predictive controller (MPC) to control his/her own motion. The differential game is converted into a bi-level optimization problem and is solved with the branch and bound algorithm. Additionally, an inverse MPC algorithm is developed to estimate the weights of the MPC cost function of the target vehicle. The low-level hybrid MPC controls both the autonomous vehicle’s longitudinal motion and its real-time lane determination. Simulations indicate both the inverse MPC’s capability on aggressiveness estimation of target vehicles and DGTMPC’s superior performance in interactive lane change situations.
References
[1]
D. Lin, L. Li, and S. E. Jabari, “Pay to change lanes: A cooperative lane-changing strategy for connected/automated driving,” Transp. Res. C, Emerg. Technol., vol. 105, pp. 550–564, Mar. 2019.
[2]
S. Pan, W. Yafei, and W. Kaizheng, “A game theory-based model predictive controller considering intension for mandatory lane change,” SAE Tech. Paper, 2020-01-5127, 2020.
[3]
J. Yoo and R. Langari, “A Stackelberg game theoretic model of lane-merging,” 2020, arXiv:2003.09786.
[4]
J. Yoo and R. Langari, “A game-theoretic model of human driving and application to discretionary lane-changes,” 2020, arXiv:2003.09783.
[5]
S. Karimi and A. Vahidi, “Receding horizon motion planning for automated lane change and merge using Monte Carlo tree search and level-K game theory,” in Proc. Amer. Control Conf. (ACC), Jul. 2020, pp. 1223–1228.
[6]
G. Su, N. Li, Y. Yildiz, A. Girard, and I. Kolmanovsky, “A traffic simulation model with interactive drivers and high-fidelity car dynamics,” IFAC-PapersOnLine, vol. 51, no. 34, pp. 384–389, 2019.
[7]
S. Zhang, Y. Zhi, R. He, and J. Li, “Research on traffic vehicle behavior prediction method based on game theory and HMM,” IEEE Access, vol. 8, pp. 30210–30222, 2020.
[8]
Q. Dai, X. Xu, W. Guo, S. Huang, and D. Filev, “Towards a systematic computational framework for modeling multi-agent decision-making at micro level for smart vehicles in a smart world,” 2020, arXiv:2009.12213.
[9]
W. Schwarting, A. Pierson, S. Karaman, and D. Rus, “Stochastic dynamic games in belief space,” 2019, arXiv:1909.06963.
[10]
A. Liniger and J. Lygeros, “A noncooperative game approach to autonomous racing,” IEEE Trans. Control Syst. Technol., vol. 28, no. 3, pp. 884–897, May 2020.
[11]
G. Williams, B. Goldfain, P. Drews, J. M. Rehg, and E. A. Theodorou, “Autonomous racing with AutoRally vehicles and differential games,” 2017, arXiv:1707.04540.
[12]
R. Spica, E. Cristofalo, Z. Wang, E. Montijano, and M. Schwager, “A real-time game theoretic planner for autonomous two-player drone racing,” IEEE Trans. Robot., vol. 36, no. 5, pp. 1389–1403, Oct. 2020.
[13]
A. Dreves and M. Gerdts, “A generalized Nash equilibrium approach for optimal control problems of autonomous cars,” Optim. Control Appl. Methods, vol. 39, no. 1, pp. 326–342, Jan. 2018.
[14]
B. J. Goode and M. J. Roan, “A differential game theoretic approach for two-agent collision avoidance with travel limitations,” J. Intell. Robotic Syst., vol. 67, nos. 3–4, pp. 201–218, Sep. 2012.
[15]
N. Li, D. W. Oyler, M. Zhang, Y. Yildiz, I. Kolmanovsky, and A. R. Girard, “Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems,” IEEE Trans. Control Syst. Technol., vol. 26, no. 5, pp. 1782–1797, Sep. 2018.
[16]
Q. Zhang, R. Langari, H. E. Tseng, D. Filev, S. Szwabowski, and S. Coskun, “A game theoretic model predictive controller with aggressiveness estimation for mandatory lane change,” IEEE Trans. Intell. Vehicles, vol. 5, no. 1, pp. 75–89, Nov. 2019.
[17]
P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in Proc. Twenty-first Int. Conf. Mach. Learn. (ICML), 2004, p. 1.
[18]
Q. Zhang, D. Filev, H. E. Tseng, S. Szwabowski, and R. Langari, “Addressing mandatory lane change problem with game theoretic model predictive control and fuzzy Markov chain,” in Proc. Annu. Amer. Control Conf. (ACC), Jun. 2018, pp. 4764–4771.
[19]
R. Isaacs, Differential Games: A Mathematical Theory With Applications to Warfare and Pursuit, Control and Optimization. North Chelmsford, MA, USA: Courier Corporation, 1999.
[20]
P. Liu, Y. Du, L. Wang, and J. Da Young, “Ready to bully automated vehicles on public roads?,” Accident Anal. Prevention, vol. 137, Mar. 2020, Art. no.
[21]
T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, vol. 23. Philadelphia, PA, USA: SIAM, 1999.
[22]
A. Bensoussan, S. Chen, and S. P. Sethi, “The maximum principle for global solutions of stochastic Stackelberg differential games,” SIAM J. Control Optim., vol. 53, no. 4, pp. 1956–1981, 2015.
[23]
J. F. Bard and J. T. Moore, “A branch and bound algorithm for the bilevel programming problem,” SIAM J. Sci. Statist. Comput., vol. 11, no. 2, pp. 281–292, 1990.
[24]
S. T. DeNegre and T. K. Ralphs, “A Branch-and-cut algorithm for integer bilevel linear programs,” in Operations Research and Cyber-Infrastructure. Cham, Switzerland: Springer, 2009, pp. 65–78.
[25]
B. Colson, P. Marcotte, and G. Savard, “A trust-region method for nonlinear bilevel programming: Algorithm and computational experience,” Comput. Optim. Appl., vol. 30, no. 3, pp. 211–227, 2005.
[26]
M. Zhu and S. Martínez, “Stackelberg-game analysis of correlated attacks in cyber-physical systems,” in Proc. Amer. Control Conf., Jun. 2011, pp. 4063–4068.
[27]
F.-L. Meng and X.-J. Zeng, “A Stackelberg game-theoretic approach to optimal real-time pricing for the smart grid,” Soft Comput., vol. 17, no. 12, pp. 2365–2380, Dec. 2013.
[28]
G. Goodwin, M. M. Seron, and J. A. De Doná, Constrained Control and Estimation: An Optimisation Approach. Cham, Switzerland: Springer, 2006.
[29]
M. Simaan and J. B. Cruz, “On the Stackelberg strategy in nonzero-sum games,” J. Optim. Theory Appl., vol. 11, no. 5, pp. 533–555, May 1973.
[30]
Z. H. Gümüş and C. A. Floudas, “Global optimization of nonlinear bilevel programming problems,” J. Global Optim., vol. 20, no. 1, pp. 1–31, 2001.
[31]
J. H. Yoo and R. Langari, “Stackelberg game based model of highway driving,” in Proc. Adapt. Control; Adv. Vehicle Propuls. Syst.; Aerosp. Syst.; Auto. Syst.; Battery Modeling; Biochem. Syst.; Control Over Netw.; Control Syst. Design; Cooperativ, Oct. 2012, pp. 499–508.
[32]
A. Talebpour, H. S. Mahmassani, and S. H. Hamdar, “Modeling lane-changing behavior in a connected environment: A game theory approach,” Transp. Res. Proc., vol. 7, pp. 420–440, Jan. 2015.
[33]
N. Li, M. Zhang, Y. Yildiz, I. Kolmanovsky, and A. Girard, “Game theory-based traffic modeling for calibration of automated driving algorithms,” in Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions. Cham, Switzerland: Springer, 2019, pp. 89–106.
[34]
D. Sadigh, S. Sastry, S. A. Seshia, and A. D. Dragan, “Planning for autonomous cars that leverage effects on human actions,” in Robotics: Science and Systems, vol. 2. RSS Foundation, 2016, pp. 1–9.
[35]
M. C. Priess, J. Choi, and C. Radcliffe, “Determining human control intent using inverse LQR solutions,” in Proc. Dynamic Syst. Control Conf., Oct. 2013, pp. 1–8.
[36]
M. C. Priess, R. Conway, J. Choi, J. M. Popovich, and C. Radcliffe, “Solutions to the inverse LQR problem with application to biological systems analysis,” IEEE Trans. Control Syst. Technol., vol. 23, no. 2, pp. 770–777, Mar. 2015.
[37]
H. El-Hussieny, A. A. Abouelsoud, S. F. M. Assal, and S. M. Megahed, “Adaptive learning of human motor behaviors: An evolving inverse optimal control approach,” Eng. Appl. Artif. Intell., vol. 50, pp. 115–124, Apr. 2016.
[38]
A. Ramadan, J. Choi, and C. J. Radcliffe, “Inferring human subject motor control intent using inverse MPC,” in Proc. Amer. Control Conf. (ACC), Jul. 2016, pp. 5791–5796.
[39]
B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning,” in Proc. AAAI, Chicago, IL, USA, 2008, pp. 1433–1438.
[40]
A. Boularias, J. Kober, and J. Peters, “Relative entropy inverse reinforcement learning,” in Proc. 14th Int. Conf. Artif. Intell. Statist., 2011, pp. 182–189.
[41]
L.-W. Zhang, Y.-E. Ge, and Y. Lu, “An alternating direction method for solving a class of inverse semi-definite quadratic programming problems,” J. Ind. Manage. Optim., vol. 12, no. 1, pp. 317–336, Apr. 2015.
[42]
J. Wu, Y. Zhang, L. Zhang, and Y. Lu, “A sequential convex program approach to an inverse linear semidefinite programming problem,” Asia–Pacific J. Oper. Res., vol. 33, no. 4, Aug. 2016, Art. no.
[43]
M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 62, no. 2, p. 1805, 2000.
[44]
A. Kesting, M. Treiber, and D. Helbing, “Enhanced intelligent driver model to access the impact of driving strategies on traffic capacity,” Philosophical Trans. Roy. Soc. A, Math., Phys. Eng. Sci., vol. 368, no. 1928, pp. 4585–4605, Oct. 2010.
Index Terms
- Stackelberg Differential Lane Change Game Based on MPC and Inverse MPC
Index terms have been assigned to the content through auto-classification.
Recommendations
Application of Stackelberg Game Theory for Shared Steering Torque Control in Lane Change Maneuver
2018 IEEE Intelligent Vehicles Symposium (IV)Human-machine interactions play a crucial role in determining driver-automation conflicts, as well as stabilities of the co-piloting system. Therefore, this paper proposed a Stackelberg-based shared control scheme to describe the driver-automation ...
Kinematic Design for Platoon-Lane-Change Maneuvers
For the lane-change maneuvers under the architectures of automated highway systems, most researchers focus on the maneuvers for a single vehicle to change lanes. However, not only are the lane-change maneuvers for a single vehicle needed, but a platoon-...
Lane Change Maneuvers for Automated Vehicles
By considering a lane change maneuver as primarily a longitudinal motion planning problem, this paper presents a lane change maneuver algorithm with a pragmatic approach to determine an inter-vehicle traffic gap and time instance to perform the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1524-9050 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 01 August 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025