[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2101.01385.html
   My bibliography  Save this paper

Recurrent Neural Networks for Stochastic Control Problems with Delay

Author

Listed:
  • Jiequn Han
  • Ruimeng Hu
Abstract
Stochastic control problems with delay are challenging due to the path-dependent feature of the system and thus its intrinsic high dimensions. In this paper, we propose and systematically study deep neural networks-based algorithms to solve stochastic control problems with delay features. Specifically, we employ neural networks for sequence modeling (\emph{e.g.}, recurrent neural networks such as long short-term memory) to parameterize the policy and optimize the objective function. The proposed algorithms are tested on three benchmark examples: a linear-quadratic problem, optimal consumption with fixed finite delay, and portfolio optimization with complete memory. Particularly, we notice that the architecture of recurrent neural networks naturally captures the path-dependent feature with much flexibility and yields better performance with more efficient and stable training of the network compared to feedforward networks. The superiority is even evident in the case of portfolio optimization with complete memory, which features infinite delay.

Suggested Citation

  • Jiequn Han & Ruimeng Hu, 2021. "Recurrent Neural Networks for Stochastic Control Problems with Delay," Papers 2101.01385, arXiv.org, revised Jun 2021.
    • Handle: RePEc:arx:papers:2101.01385
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2101.01385
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Asea, Patrick K. & Zak, Paul J., 1999. "Time-to-build and cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 23(8), pages 1155-1175, August.
    2. F. Gozzi & C. Marinelli & S. Savin, 2009. "On Controlled Linear Diffusions with Delay in a Model of Optimal Advertising under Uncertainty with Memory Effects," Journal of Optimization Theory and Applications, Springer, vol. 142(2), pages 291-321, August.
    3. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    4. Jiequn Han & Ruimeng Hu, 2019. "Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games," Papers 1912.01809, arXiv.org, revised Jun 2020.
    5. Harald Bauer & Ulrich Rieder, 2005. "Stochastic control problems with delay," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 62(3), pages 411-427, December.
    6. Li Chen & Zhen Wu & Zhiyong Yu, 2012. "Delayed Stochastic Linear-Quadratic Control Problem and Related Applications," Journal of Applied Mathematics, Hindawi, vol. 2012, pages 1-22, September.
    7. Salvatore Federico, 2011. "A stochastic control problem with delay arising in a pension fund model," Finance and Stochastics, Springer, vol. 15(3), pages 421-459, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Papers 2102.09851, arXiv.org, revised Feb 2021.
    2. Carl Remlinger & Joseph Mikael & Romuald Elie, 2022. "Robust Operator Learning to Solve PDE," Working Papers hal-03599726, HAL.
    3. Maximilien Germain & Huyên Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance ," Working Papers hal-03115503, HAL.
    4. Nacira Agram & Bernt Øksendal & Jan Rems, 2024. "Deep learning for quadratic hedging in incomplete jump market," Digital Finance, Springer, vol. 6(3), pages 463-499, September.
    5. Robert Balkin & Hector D. Ceniceros & Ruimeng Hu, 2023. "Stochastic Delay Differential Games: Financial Modeling and Machine Learning Algorithms," Papers 2307.06450, arXiv.org.
    6. William Lefebvre & Enzo Miller, 2021. "Linear-Quadratic Stochastic Delayed Control and Deep Learning Resolution," Journal of Optimization Theory and Applications, Springer, vol. 191(1), pages 134-168, October.
    7. Luca Di Persio & Emanuele Lavagnoli & Marco Patacca, 2022. "Calibrating FBSDEs Driven Models in Finance via NNs," Risks, MDPI, vol. 10(12), pages 1-19, November.
    8. Maximilien Germain & Huyên Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance ," Post-Print hal-03115503, HAL.
    9. Nacira Agram & Bernt {O}ksendal & Jan Rems, 2024. "Deep learning for quadratic hedging in incomplete jump market," Papers 2407.13688, arXiv.org.
    10. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Post-Print hal-03145949, HAL.
    11. Maximilien Germain & Huy^en Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance," Papers 2101.08068, arXiv.org, revised Apr 2021.
    12. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Working Papers hal-03145949, HAL.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Working Papers hal-03145949, HAL.
    2. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Papers 2102.09851, arXiv.org, revised Feb 2021.
    3. Fabbri Giorgio & Federico Salvatore, 2014. "On the Infinite-Dimensional Representation of Stochastic Controlled Systems with Delayed Control in the Diffusion Term," Mathematical Economics Letters, De Gruyter, vol. 2(3-4), pages 33-43, November.
    4. Frédéric Zumer & Jacques Le Cacheux & Marc Flandreau, 1998. "Stability without a pact? Lessons from the European Gold Standard, 1880-1913," Sciences Po publications n°98-01, Sciences Po.
    5. William Lefebvre & Enzo Miller, 2021. "Linear-quadratic stochastic delayed control and deep learning resolution," Post-Print hal-03145949, HAL.
    6. William Lefebvre & Enzo Miller, 2021. "Linear-Quadratic Stochastic Delayed Control and Deep Learning Resolution," Journal of Optimization Theory and Applications, Springer, vol. 191(1), pages 134-168, October.
    7. Salvatore Federico, 2011. "A stochastic control problem with delay arising in a pension fund model," Finance and Stochastics, Springer, vol. 15(3), pages 421-459, September.
    8. Ulrich Brandt-Pollmann & Ralph Winkler & Sebastian Sager & Ulf Moslener & Johannes Schlöder, 2008. "Numerical Solution of Optimal Control Problems with Constant Control Delays," Computational Economics, Springer;Society for Computational Economics, vol. 31(2), pages 181-206, March.
    9. Mauro Bambi & Cristina Girolami & Salvatore Federico & Fausto Gozzi, 2017. "Generically distributed investments on flexible projects and endogenous growth," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 63(2), pages 521-558, February.
    10. Kitagawa, Akiomi & Shibata, Akihisa, 2001. "Long gestation in an overlapping generations economy: endogenous cycles and indeterminacy of equilibria," Journal of Mathematical Economics, Elsevier, vol. 35(1), pages 99-127, February.
    11. Bambi, Mauro & Gori, Franco, 2014. "Unifying Time-To-Build Theory," Macroeconomic Dynamics, Cambridge University Press, vol. 18(8), pages 1713-1725, December.
    12. Mauro Bambi, 2006. "Endogenous growth and time to build: the AK case," Computing in Economics and Finance 2006 77, Society for Computational Economics.
    13. Heinzel, Christoph & Winkler, Ralph, 2006. "Gradual versus structural technological change in the transition to a low-emission energy industry: How time-to-build and differing social and individual discount rates influence environmental and tec," Dresden Discussion Paper Series in Economics 09/06, Technische Universität Dresden, Faculty of Business and Economics, Department of Economics.
    14. Giorgio Fabbri & Fausto Gozzi & Andrzej Swiech, 2017. "Stochastic Optimal Control in Infinite Dimensions - Dynamic Programming and HJB Equations," Post-Print hal-01505767, HAL.
    15. Ralph Winkler, 2008. "Optimal compliance with emission constraints: dynamic characteristics and the choice of technique," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 39(4), pages 411-432, April.
    16. Fabbri, Giorgio, 2017. "International borrowing without commitment and informational lags: Choice under uncertainty," Journal of Mathematical Economics, Elsevier, vol. 68(C), pages 103-114.
    17. Hippolyte d'Albis & Jean-Pierre Drugeon, 2020. "On Investment and Cycles in Explicitely Solved Vintage Capital Models," PSE Working Papers halshs-02570648, HAL.
    18. De Cesare, Luigi & Sportelli, Mario, 2022. "A non-linear approach to Kalecki’s investment cycle," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 193(C), pages 57-70.
    19. Winkler, Ralph & Brandt-Pollmann, Ulrich & Moslener, Ulf & Schlöder, Johannes, 2005. "On the Transition from Instantaneous to Time-Lagged Capital Accumilation: The Case of Leontief Type Production Functions," ZEW Discussion Papers 05-30, ZEW - Leibniz Centre for European Economic Research.
    20. Olivier Menoukeu Pamen, 2015. "Optimal Control for Stochastic Delay Systems Under Model Uncertainty: A Stochastic Differential Game Approach," Journal of Optimization Theory and Applications, Springer, vol. 167(3), pages 998-1031, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2101.01385. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.