More Web Proxy on the site http://driver.im/

research-article

Approximate Markov-Nash Equilibria for Discrete-Time Risk-Sensitive Mean-Field Games

Authors:

Tamer Başar, Maxim RaginskyAuthors Info & Claims

Mathematics of Operations Research, Volume 45, Issue 4

Pages 1596 - 1620

https://doi.org/10.1287/moor.2019.1044

Published: 01 November 2020 Publication History

Abstract

In this paper, we study a class of discrete-time mean-field games under the infinite-horizon risk-sensitive optimality criterion. Risk sensitivity is introduced for each agent (player) via an exponential utility function. In this game model, each agent is coupled with the rest of the population through the empirical distribution of the states, which affects both the agent’s individual cost and its state dynamics. Under mild assumptions, we establish the existence of a mean-field equilibrium in the infinite-population limit as the number of agents (N) goes to infinity, and we then show that the policy obtained from the mean-field equilibrium constitutes an approximate Nash equilibrium when N is sufficiently large.

References

[1]

Adlakha S, Johari R, Weintraub GY (2015) Equilibria of dynamic games with many players: Existence, approximation, and market structure. J. Econom. Theory 156:269–316.

[2]

Aliprantis CD, Border KC (2006) Infinite Dimensional Analysis, 3rd ed. (Springer, Berlin).

[3]

Avila-Godoy G, Brau A, Fernandez-Gaucherand E (1997) Controlled Markov chains with discounted risk-sensitive criteria: Applications to machine replacement. Proc. 36th IEEE Conf. Decision Control, vol. 2 (IEEE, Piscataway, NJ), 1115–1120.

[4]

Bauerle N, Rieder U (2014) More risk-sensitive Markov decision processes. Math. Oper. Res. 39(1):105–120.

Digital Library

[5]

Bensoussan A, Frehse J, Yam P (2013) Mean Field Games and Mean Field Type Control Theory (Springer, New York).

[6]

Billingsley P (1999) Convergence of Probability Measures, 2nd ed. (Wiley, New York).

[7]

Biswas A (2015) Mean field games with ergodic cost for discrete time Markov processes. Preprint, submitted October 30, https://arxiv.org/abs/1510.08968.

[8]

Bogachev VI (2007) Measure Theory, vol. II (Springer, Berlin).

[9]

Budhiraja A, Majumder AP (2015) Long time results for a weakly interacting particle system in discrete time. Stochastic Anal. Appl. 33(3):429–463.

[10]

Cardaliaguet P (2011) Notes on mean-field games. Working paper, Université Paris-Dauphine, Paris.

[11]

Carmona R, Delarue F (2013) Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51(4):2705–2734.

Digital Library

[12]

Chung K, Sobel MJ (1987) Discounted MDP’s: Distribution functions and exponential utility maximization. SIAM J. Control Optim. 25(1):49–62.

Digital Library

[13]

Dai Pra P, Meneghini L, Runggaldier WJ (1996) Connections between stochastic control and dynamic games. Math. Control Signals Systems 9(4):303–326.

[14]

Djehiche B, Tembine H (2016) Risk-sensitive mean-field type control under partial observation. Benth FE, Di Nunno G, eds. Stochastics of Environmental and Financial Economics (Springer, Cham, Switzerland), 243–263.

[15]

Dudley RM (2004) Real Analysis and Probability (Cambridge University Press, New York).

[16]

Elliot R, Li X, Ni Y (2013) Discrete time mean-field stochastic linear-quadratic optimal control problems. Automatica J. IFAC 49(11):3222–3233.

[17]

Gomes DA, Saúde J (2014) Mean field games models - a brief survey. Dynam. Games Appl. 4(2):110–154.

[18]

Gomes DA, Mohr J, Souza RR (2010) Discrete time, finite state space mean field games. J. Math. Pures Appl. 93(3):308–328.

[19]

Hernandez-Hernandez D, Marcus SI (1996) Risk sensitive control of Markov processes in countable state space. Systems Control Lett. 29(3):147–155.

Digital Library

[20]

Hernández-Lerma O, Lasserre JB (1996) Discrete-Time Markov Control Processes: Basic Optimality Criteria (Springer, New York).

[21]

Howard RA, Matheson JE (1972) Risk-sensitive Markov decision processes. Management Sci. 18(7):356–369.

Digital Library

[22]

Huang M (2010) Large-population LQG games involving major player: The Nash certainity equivalence principle. SIAM J. Control Optim. 48(5):3318–3353.

[23]

Huang M, Caines PE, Malhamé RP (2007) Large-population cost coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralizedε-Nash equilibria. IEEE Trans. Automat. Control 52(9):1560–1571.

[24]

Huang M, Malhamé RP, Caines PE (2006) Large population stochastic dynamic games: Closed loop McKean-Vlasov sysyems and the Nash certainity equivalence principle. Comm. Inform. Systems 6(3):221–252.

[25]

Jaskiewicz A (2007) Average optimality for risk-sensitive control with general state space. Ann. Appl. Probab. 17(2):654–675.

[26]

Jovanovic B, Rosenthal RW (1988) Anonymous sequential games. J. Math. Econom. 17(1):77–87.

[27]

Langen HJ (1981) Convergence of dynamic programming models. Math. Oper. Res. 6(4):493–512.

Digital Library

[28]

Lasry J, Lions P (2007) Mean field games. Japanese J. Math. 2(1):229–260.

[29]

Moon J, Başar T (2015) Discrete-time decentralized control using the risk-sensitive performance criterion in the large population regime: a mean field approach. 2015 Amer. Control Conf. (ACC) (IEEE, Piscataway, NJ), 4779–4784.

[30]

Moon J, Başar T (2016) Discrete-time mean field Stackelberg games with a large number of followers. Proc. 2016 IEEE 55th Conf. Decision Control (IEEE, Piscataway, NJ), 3578–3583.

[31]

Moon J, Başar T (2016) Robust mean field games for coupled Markov jump linear systems. Internat. J. Control 89(7):1367–1381.

[32]

Moon J, Başar T (2017) Linear quadratic risk-sensitive and robust mean field games. IEEE Trans. Automat. Control 62(3):1062–1077.

[33]

Nourian M, Nair GN (2013) Linear-quadratic-Gaussian mean field games under high rate quantization. Proc. 52nd IEEE Conf. Decision Control (IEEE, Piscataway, NJ), 1898–1903.

[34]

Parthasarathy KR (1967) Probability Measures on Metric Spaces (American Mathematical Society, Providence, RI).

[35]

Saldi N, Başar T, Raginsky M (2019) Approximate Nash equilibria in partially observed stochastic games with mean-field interactions. Math. Oper. Res. 44(3):1006–1033.

[36]

Saldi N, Başar T, Raginsky M (2018) Markov-Nash equilibria in mean-field games with discounted cost. SIAM J. Control Optim. 56(6):4256–4287.

Digital Library

[37]

Şen N, Caines PE (2016) Mean field game theory with a partially observed major agent. SIAM J. Control Optim. 54(6):3174–3224.

Digital Library

[38]

Şen N, Caines PE (2016) On mean field games and nonlinear filtering for agents with individual-state partial observations. 2016 American Control Conf. (IEEE, Piscataway, NJ), 4681–4686.

[39]

Serfozo R (1982) Convergence of Lebesgue integrals with varying measures. Sankhya Ser. A 44(3):380–402.

[40]

Tembine H (2015) Risk-sensitive mean-field-type games with Lp-norm drifts. Automatica J. IFAC 59:224–237.

Digital Library

[41]

Tembine H, Zhu Q, Başar T (2014) Risk-sensitive mean field games. IEEE Trans. Automat. Control 59(4):835–850.

[42]

Villani C (2009) Optimal Transport: Old and New (Springer, Berlin).

Cited By

Dianetti JFerrari GFischer MNendel M(2023)A Unifying Framework for Submodular Mean Field GamesMathematics of Operations Research10.1287/moor.2022.131648:3(1679-1710)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1287/moor.2022.1316
Mao WQiu HWang CFranke HKalbarczyk ZIyer RBaşar TKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)A mean-field game approach to cloud resource management with function approximationProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602896(36243-36258)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602896

Index Terms

Approximate Markov-Nash Equilibria for Discrete-Time Risk-Sensitive Mean-Field Games

Index terms have been assigned to the content through auto-classification.

Recommendations

Markov--Nash Equilibria in Mean-Field Games with Discounted Cost

In this paper, we consider discrete-time dynamic games of the mean-field type with a finite number $N$ of agents subject to an infinite-horizon discounted-cost optimality criterion. The state space of each agent is a Polish space. At each time, the agents ...
Approximate Nash Equilibria in Partially Observed Stochastic Games with Mean-Field Interactions
Establishing the existence of Nash equilibria for partially observed stochastic dynamic games is known to be quite challenging, with the difficulties stemming from the noisy nature of the measurements available to individual players (agents) and the ...
Learning mean-field games with discounted and average costs

We consider learning approximate Nash equilibria for discrete-time mean-field games with stochastic nonlinear state dynamics subject to both average and discounted costs. To this end, we introduce a mean-field equilibrium (MFE) operator, whose fixed ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Mathematics of Operations Research

Mathematics of Operations Research Volume 45, Issue 4

November 2020

429 pages

ISSN:0364-765X

DOI:10.1287/moor.2020.45.issue-4

Issue’s Table of Contents

Copyright © 2020, INFORMS.

Publisher

INFORMS

Linthicum, MD, United States

Publication History

Published: 01 November 2020

Accepted: 16 October 2019

Received: 03 February 2019

Author Tags

Author Tags

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dianetti JFerrari GFischer MNendel M(2023)A Unifying Framework for Submodular Mean Field GamesMathematics of Operations Research10.1287/moor.2022.131648:3(1679-1710)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1287/moor.2022.1316
Mao WQiu HWang CFranke HKalbarczyk ZIyer RBaşar TKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)A mean-field game approach to cloud resource management with function approximationProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602896(36243-36258)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602896

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents