Portfolio dynamic trading strategies using deep reinforcement learning
Abstract
Using the constituent stocks of the iShares MSCI US ESG Select Index ETF, a matrix of technical indicators, returns, and covariance is incorporated to represent the inherent information characteristics of the stock market. In this study, based on the proposed Deep Reinforcement Learning for Portfolio Management on Environmental, Social, and Governance (DRLPMESG) architecture model, investors who use active portfolio management reap the greatest rewards, as the portfolio with 5 stocks performing the best, with an annualized return of 46.58%, a Sharpe ratio of 1.37, and a cumulative return of 115.18%, indicating that the results have the potential to win the market and generate excess profits. In contrast to the efficient market hypothesis, this new understanding of proven effectiveness in obtaining satisfactory rewards would help improve investment strategies for portfolio management. Furthermore, this study proposed that holding 5 stocks in a portfolio can lead to higher returns, laying the foundation for future research on the number of holdings. Moreover, when compared to previous static strategies, this model offering a dynamic strategy may generate a more stable return in the face of market fluctuations.
References
[1]
Aggarwal S and Aggarwal S Deep investment in financial markets using deep learning models Int J Comput Appl 2017 162 40-43
[2]
Ahmed S, Alshater MM, ElAmmari A, and Hammami H Artificial intelligence and machine learning in finance: a bibliometric review Res Int Bus Finance 2022 61 101646
[3]
Al-Aradi A and Jaimungal S Active and passive portfolio management with latent factors Quant Finance 2021 21 1437-1459
[4]
Alexander C Market risk analysis, practical financial econometrics 2008 Wiley
[5]
Arulkumaran K, Deisenroth MP, Brundage M, and Bharath AA Deep reinforcement learning: a brief survey IEEE Signal Process Mag 2017 34 26-38
[6]
Betancourt C and Chen W-H Deep reinforcement learning for portfolio management of markets with a dynamic number of assets Expert Syst Appl 2021 164 114002
[7]
Bodnar T and Schmid W Econometrical analysis of the sample efficient frontier Eur J Finance 2009 15 317-335
[8]
Castiglioni I, Rundo L, Codari M, et al. AI applications to medical images: From machine learning to deep learning Phys Med 2021 83 9-24
[9]
Chandra P Investment analysis and portfolio management 2017 New York McGraw-Hill Education
[10]
Charpentier A, Elie R, Remlinger C (2023) Reinforcement learning in economics and Finance. Comput Econ 62:425–462
[11]
Chen YF and Huang SH Sentiment-influenced trading system based on multimodal deep reinforcement learning Appl Soft Comput 2021 112 107788
[12]
Chen R and Ren J Do AI-powered mutual funds perform better? Finance Res Lett 2021 47 102616
[13]
Chen J, Luo C, Pan L, and Jia Y Trading strategy of structured mutual fund based on deep learning network Expert Syst Appl 2021 183 115390
[14]
Choueifaty Y and Coignard Y Toward Maximum Diversification J Portf Manag 2008 35 40-51
[15]
Craja P, Kim A, and Lessmann S Deep learning for detecting financial statement fraud Decis Support Syst 2020 139 113421
[16]
Da Silva AS, Lee W, and Pornrojnangkool B The Black-Litterman model for active portfolio management J Portf Manag 2009 35 61
[17]
Darapaneni N, Basu A, Savla S, et al (2020) Automated portfolio rebalancing using Q-learning. In: 2020 11th IEEE annual ubiquitous computing, electronics & mobile communication conference (UEMCON). IEEE, pp 0596–0602
[18]
DeMiguel V, Gil-Bazo J, Nogales FJ, Santos AA (2021) Can machine learning help to select portfolios of mutual funds?, SSRN
[19]
Deng Y, Bao F, Kong YY, et al. Deep direct reinforcement learning for financial signal representation and trading IEEE Trans Neural Netw Learn Syst 2017 28 653-664
[20]
Escrig-Olmedo E, Muñoz-Torres MJ, and Fernandez-Izquierdo MA Socially responsible investing: sustainability indices, ESG rating and information provider agencies Int J Sustain Dev 2010 2 442-461
[21]
Evans JL and Archer SH Diversification and the reduction of dispersion: an empirical analysis J Finance 1968 23 761-767
[22]
Huang G, Zhou X, Song Q (2020) Deep reinforcement learning for portfolio management. arXiv preprint arXiv:201213773
[23]
Ieda M, Fujino N, and Sasaki H Active portfolio management with conditioning information J Investig 2019 28 51-65
[24]
Kocmanová A and Dočekalová M Construction of the economic indicators of performance in relation to environmental, social and corporate governance (ESG) factors Acta Universitatis Agriculturae Et Silviculturae Mendelianae Brunensis 2012 60 195-206
[25]
Liang Z, Chen H, Zhu J, et al (2018) Adversarial deep reinforcement learning in portfolio management. arXiv preprint arXiv:180809940
[26]
Lim QYE, Cao Q, and Quek C Dynamic portfolio rebalancing through reinforcement learning Neural Comput Appl 2022 34 7125-7139
[27]
Lintner J The valuation of risk assets and the selection of risky investments in stock portfolios and capital budgets Rev Econ Stat 1965 47 13-37
[28]
Liu FR, Li Y, Li BT, et al. Bitcoin transaction strategy construction based on deep reinforcement learning Appl Soft Comput 2021 113 107952
[29]
Lucarelli G and Borrotti M A deep Q-learning portfolio management framework for the cryptocurrency market Neural Comput Appl 2020 32 17229-17244
[30]
Ma YL, Han RZ, and Wang WZ Portfolio optimization with return prediction using deep learning and machine learning Expert Syst Appl 2021 165 113973
[31]
Malkiel BG The efficient market hypothesis and its critics J Econ Perspect 2003 17 59-82
[32]
Markowitz H Portfolio selection J Finance 1952 7 77-91
[33]
Meng TL and Khushi M Reinforcement learning in financial markets Data 2019 4 110
[34]
Ozbayoglu AM, Gudelek MU, and Sezer OB Deep learning for financial applications: a survey Appl Soft Comput 2020 93 106384
[35]
Peterson S (2012) Active portfolio management. In: Investment theory and risk management, pp 187–196
[36]
Pham U, Luu Q, and Tran H Multi-agent reinforcement learning approach for hedging portfolio problem Soft Comput 2021 25 7877-7885
[37]
Reilly FK and Akhtar RA The benchmark error problem with global capital markets J Portf Manag 1995 22 33
[38]
Rundo F Deep LSTM with reinforcement learning layer for financial trend prediction in FX high frequency trading systems Appl Sci 2019 9 4460
[39]
Sharpe WF Capital asset prices: a theory of market equilibrium under conditions of risk J Finance 1964 19 425-442
[40]
Shi S, Li JJ, Li GH, et al. GPM: a graph convolutional network based reinforcement learning framework for portfolio management Neurocomputing 2022 498 14-27
[41]
Stoilov T, Stoilova K, and Vladimirov M Application of modified Black-Litterman model for active portfolio management Expert Syst Appl 2021 186 115719
[42]
Sutton RS and Barto AG Reinforcement learning: an introduction 1998 Cambridge MIT Press
[43]
Taghian M, Asadi A, and Safabakhsh R Learning financial asset-specific trading rules via deep reinforcement learning Expert Syst Appl 2022 195 116523
[44]
Théate T and Ernst D An application of deep reinforcement learning to algorithmic trading Expert Syst Appl 2021 173 114632
[45]
Torrente ML, Uberti P (2022) A rescaling technique to improve numerical stability of portfolio optimization problems. Soft Comput.
[46]
Vo NNY, He XZ, Liu SW, and Xu GD Deep learning for decision making and the optimization of socially responsible investments and portfolio Decis Support Syst 2019 124 113097
[47]
Wang HN, Liu N, Zhang YY, et al. Deep reinforcement learning: a survey Front Inf Technol Electron Eng 2020 21 1726-1744
[48]
Wang JZ, Zhang HP, and Luo H Research on the construction of stock portfolios based on multiobjective water cycle algorithm and KMV algorithm Appl Soft Comput 2022 115 108186
[49]
Weng LG, Sun XD, Xia M, et al. Portfolio trading system of digital currencies: a deep reinforcement learning with multidimensional attention gating mechanism Neurocomputing 2020 402 171-182
[50]
Wu X, Chen H, Wang J, et al. Adaptive stock trading strategies with deep reinforcement learning methods Inf Sci 2020 538 142-158
[51]
Wu ME, Syu JH, Lin JCW, and Ho JM Portfolio management system in equity market neutral using reinforcement learning Appl Intell 2021 51 8119-8131
[52]
Xu ZY, Zhang J, Wang JY, and Xu ZM Prediction research of financial time series based on deep learning Soft Comput 2020 24 8295-8312
[53]
Yu XM, Wu WJ, Liao XC, and Han Y Dynamic stock-decision ensemble strategy based on deep reinforcement learning Appl Intell 2023 53 2452-2470
[54]
Yun H, Lee M, Kang YS, and Seok J Portfolio management via two-stage deep learning with a joint cost Expert Syst Appl 2020 143 113041
[55]
Zhang Z, Zohren S, and Roberts S Deep reinforcement learning for trading J Financ Data Sci 2020 2 25-40
[56]
Zhu X and Wu X Class noise vs. attribute noise: a quantitative study Artif Intell Rev 2004 22 177
Index Terms
- Portfolio dynamic trading strategies using deep reinforcement learning
Index terms have been assigned to the content through auto-classification.
Recommendations
Multi-type data fusion framework based on deep reinforcement learning for algorithmic trading
AbstractIn recent years, research on algorithmic trading based on machine learning has been increasing. One challenge faced is getting an accurate representation of the stock market environment from multi-type data. Most existing algorithmic trading ...
Deep reinforcement learning for stock portfolio optimization by connecting with modern portfolio theory
Highlights- Combined a reinforcement learning and modern portfolio theory for portfolio.
- ...
AbstractWith artificial intelligence and data quality development, portfolio optimization has improved rapidly. Traditionally, researchers in the financial market have utilized the modern portfolio theory for portfolio optimization; however, ...
Multi-agent reinforcement learning approach for hedging portfolio problem
AbstractDeveloping a hedging strategy to reduce risk of losses for a given set of stocks in a portfolio is a difficult task due to cost of the hedge. In Vietnam stock market, cross-hedge is involved hedging a long position of a stock because there is no ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 30 July 2023
Accepted: 01 July 2023
Author Tags
Qualifiers
- Research-article
Funding Sources
- Ministry of Science and Technology, Taiwan
- National Taipei University, Taiwan
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025