FreQuant: A Reinforcement-Learning based Adaptive Portfolio Optimization with Multi-frequency Decomposition
Pages 1211 - 1221
Abstract
How can we leverage inherent frequency features of stock signals for effective portfolio optimization? Portfolio optimization in the domain of finance revolves around strategically allocating assets to maximize returns. Recent advancements highlight the efficacy of deep learning and reinforcement learning (RL) in capturing temporal asset patterns for portfolio optimization. However, previous methodologies focusing on time-domain often fail to detect sudden market shifts and abrupt events because their models are overly tailored to prevalent patterns, resulting in significant losses.
In this paper, we propose FreQuant (Adaptive Portfolio Optimization via Multi-<u>Fre</u>quency <u>Quant</u>itative Analysis), an effective deep RL framework for portfolio optimization that fully operates in the frequency domain, tackling the limitations of time domain-focused models. By bringing the analysis into the frequency domain with the Discrete Fourier Transform, our framework captures both prominent and subtle market frequencies, enhancing its adaptability and stability in response to market shifts. This approach allows FreQuant to adeptly identify primary asset patterns while also effectively responding to less common and abrupt market events, providing a more accurate and comprehensive asset representation. Empirical validation on diverse real-world trading datasets underscores the remarkable performance of FreQuant, showing its superiority in terms of profitability. Notably, FreQuant achieves up to 2.1x higher Annualized Rate of Return and 2.9x higher Portfolio Value than the best-performing competitors.
Supplemental Material
MP4 File - rtp0061 2-minute-long promotional video
Presentation Video - This video provides a concise introduction to the paper titled "FreQuant: A Reinforcement Learning-Based Adaptive Portfolio Optimization with Multi-Frequency Decomposition". It discusses the complexity and practicality of the portfolio optimization problem, offering a clear overview of the motivation and the three core concepts driving the proposed method. This introduction serves as a foundation for understanding the subsequent content.
- Download
- 21.82 MB
References
[1]
Carlo Altavilla, Luca Brugnolini, Refet S Gürkaynak, Roberto Motto, and Giuseppe Ragusa. 2019. Measuring euro area monetary policy. Journal of Monetary Economics, Vol. 108 (2019), 162--179.
[2]
Vitor Azevedo and Christopher Hoegner. 2023. Enhancing stock market anomalies with machine learning. Review of Quantitative Finance and Accounting, Vol. 60, 1 (2023), 195--230.
[3]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
[4]
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018).
[5]
L. Blume and D. Easley. 1992. Evolution and market behavior. Journal of Economic Theory, Vol. 58 (1992), 9--40. https://doi.org/10.1016/0022-0531(92)90099-4
[6]
James W Cooley and John W Tukey. 1965. An algorithm for the machine calculation of complex Fourier series. Mathematics of computation, Vol. 19, 90 (1965), 297--301.
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional Transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[8]
Andrea Flori, S. Giansante, C. Girardone, and F. Pammolli. 2019. Banks' business strategies on the edge of distress. Annals of Operations Research, Vol. 299 (2019), 481--530. https://doi.org/10.1007/s10479-019-03383-z
[9]
Scott Fujimoto, Herke van Hoof, and David Meger. 2018. Addressing Function Approximation Error in Actor-Critic Methods. In International Conference on Machine Learning. https://api.semanticscholar.org/CorpusID:3544558
[10]
Siyu Gao, Yunbo Wang, and Xiaokang Yang. 2023. Stockformer: Learning hybrid trading machines with predictive coding. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23. 4766--4774.
[11]
Ramazan Genccay, Faruk Selccuk, and Brandon J Whitcher. 2001. An introduction to wavelets and other filtering methods in finance and economics. Elsevier.
[12]
Tuomas Haarnoja, Aurick Zhou, P. Abbeel, and Sergey Levine. 2018. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. ArXiv, Vol. abs/1801.01290 (2018). https://api.semanticscholar.org/CorpusID:28202810
[13]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[14]
Akira Hirose. 2012. Complex-valued neural networks. Vol. 400. Springer Science & Business Media.
[15]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, Vol. 33 (2020), 6840--6851.
[16]
Thomas Hollis, Antoine Viscardi, and Seung Eun Yi. 2018. A comparison of LSTMs and attention mechanisms for forecasting financial time series. arXiv preprint arXiv:1812.07699 (2018).
[17]
Harrison Hong and Jiang Wang. 2000. Trading and returns under periodic market closures. The Journal of Finance, Vol. 55, 1 (2000), 297--354.
[18]
Min Hou, Chang Xu, Yang Liu, Weiqing Liu, Jiang Bian, Le Wu, Zhi Li, Enhong Chen, and Tie-Yan Liu. 2021. Stock trend prediction with multi-granularity data: A contrastive learning approach with adaptive fusion. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 700--709.
[19]
Narasimhan Jegadeesh and Sheridan Titman. 1993. Returns to buying winners and selling losers: Implications for stock market efficiency. The Journal of finance, Vol. 48, 1 (1993), 65--91.
[20]
Zhengyao Jiang, Dixing Xu, and Jinjun Liang. 2017. A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059 (2017).
[21]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012).
[22]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, Vol. 521, 7553 (2015), 436--444.
[23]
Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015).
[24]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature, Vol. 518, 7540 (2015), 529--533.
[25]
John Moody and Matthew Saffell. 1998. Reinforcement learning for trading. Advances in Neural Information Processing Systems, Vol. 11 (1998).
[26]
Hui Niu, Siyuan Li, and Jian Li. 2022. MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 1573--1583.
[27]
Yong-chan Park, Jun-Gi Jang, and U Kang. 2021. Fast and accurate partial fourier transform for time series data. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1309--1318.
[28]
James M Poterba and Lawrence H Summers. 1988. Mean reversion in stock prices: Evidence and implications. Journal of financial economics, Vol. 22, 1 (1988), 27--59.
[29]
Juan Carlos Rodriguez. 2007. Measuring financial contagion: A copula approach. Journal of empirical finance, Vol. 14, 3 (2007), 401--423.
[30]
Simone Scardapane, Steven Van Vaerenbergh, Amir Hussain, and Aurelio Uncini. 2018. Complex-valued neural networks with nonparametric activation functions. IEEE Transactions on Emerging Topics in Computational Intelligence, Vol. 4, 2 (2018), 140--150.
[31]
Si Shi, Jianjun Li, Guohui Li, Peng Pan, and Ke Liu. 2021. Xpm: An explainable deep reinforcement learning framework for portfolio management. In Proceedings of the 30th ACM international conference on information & knowledge management. 1661--1670.
[32]
David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. 2014. Deterministic policy gradient algorithms. In International conference on machine learning. Pmlr, 387--395.
[33]
Yejun Soun, Jaemin Yoo, Minyong Cho, Jihyeong Jeon, and U Kang. 2022. Accurate Stock Movement Prediction with Self-supervised Learning from Sparse Noisy Tweets. In 2022 IEEE International Conference on Big Data (Big Data). IEEE, 1691--1700.
[34]
Shuo Sun, Molei Qin, Haochong Xia, Chuqiao Zong, Jie Ying, Yonggang Xie, Lingxuan Zhao, Xinrun Wang, Bo An, et al. 2023. TradeMaster: A Holistic Quantitative Trading Platform Empowered by Reinforcement Learning. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
[35]
Shuo Sun, Xinrun Wang, Wanqi Xue, Xiaoxuan Lou, and Bo An. 2023. Mastering Stock Markets with Efficient Mixture of Diversified Trading Experts. In KDD. ACM, 2109--2119.
[36]
Shuo Sun, Wanqi Xue, Rundong Wang, Xu He, Junlei Zhu, Jian Li, and Bo An. 2022. DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 1858--1867.
[37]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1--9.
[38]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, Vol. 30 (2017).
[39]
Daixin Wang, Zhiqiang Zhang, Yeyu Zhao, Kai Huang, Yulin Kang, and Jun Zhou. 2023. Financial Default Prediction via Motif-preserving Graph Neural Network with Curriculum Learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2233--2242.
[40]
Haozhe Wang, Chao Du, Panyan Fang, Li He, Liang Wang, and Bo Zheng. 2023. Adversarial Constrained Bidding via Minimax Regret Optimization with Causality-Aware Reinforcement Learning. In KDD. ACM, 2314--2325.
[41]
Heyuan Wang, Tengjiao Wang, Shun Li, Jiayi Zheng, Shijie Guan, and Wei Chen. 2022. Adaptive Long-Short Pattern Transformer for Stock Investment Selection. In Proc. 31st Int. Joint Conf. Artif. Intell. 3970--3977.
[42]
Heyuan Wang, Tengjiao Wang, and Yi Li. 2020. Incorporating expert-based investment opinion signals in stock prediction: A deep learning framework. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 971--978.
[43]
Jingyuan Wang, Yang Zhang, Ke Tang, Junjie Wu, and Zhang Xiong. 2019. Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 1900--1908.
[44]
Rundong Wang, Hongxin Wei, Bo An, Zhouyan Feng, and Jun Yao. 2021. Commission fee is not enough: A hierarchical reinforced framework for portfolio management. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 626--633.
[45]
Zhicheng Wang, Biwei Huang, Shikui Tu, Kun Zhang, and Lei Xu. 2021. DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 643--650.
[46]
Mengyuan Yang, Xiaolin Zheng, Qianqiao Liang, Bing Han, and Mengying Zhu. 2022. A Smart Trader for Portfolio Management based on Normalizing Flows. IJCAI.
[47]
Yunan Ye, Hengzhi Pei, Boxin Wang, Pin-Yu Chen, Yada Zhu, Ju Xiao, and Bo Li. 2020. Reinforcement-learning based portfolio management with augmented asset movement prediction states. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 1112--1119.
[48]
Jaemin Yoo and U Kang. 2021. Attention-Based Autoregression for Accurate and Efficient Multivariate Time Series Forecasting. In Proceedings of the 2021 SIAM International Conference on Data Mining, SDM 2021, Virtual Event, April 29 - May 1, 2021, Carlotta Demeniconi and Ian Davidson (Eds.). SIAM, 531--539.
[49]
Jaemin Yoo, Yejun Soun, Yong-chan Park, and U Kang. 2021. Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2037--2045.
[50]
Liheng Zhang, Charu Aggarwal, and Guo-Jun Qi. 2017. Stock price prediction via discovering multi-frequency trading patterns. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 2141--2149.
[51]
Zihao Zhang, Stefan Zohren, and Stephen Roberts. 2020. Deep learning for portfolio optimization. The Journal of Financial Data Science (2020).
[52]
Lifan Zhao, Shuming Kong, and Yanyan Shen. 2023. DoubleAdapt: A Meta-learning Approach to Incremental Learning for Stock Trend Forecasting. In KDD. ACM, 3492--3503.
Index Terms
- FreQuant: A Reinforcement-Learning based Adaptive Portfolio Optimization with Multi-frequency Decomposition
Index terms have been assigned to the content through auto-classification.
Recommendations
Dynamic portfolio rebalancing through reinforcement learning
AbstractPortfolio managements in financial markets involve risk management strategies and opportunistic responses to individual trading behaviours. Optimal portfolios constructed aim to have a minimal risk with highest accompanying investment returns, ...
Hedging International Foreign Exchange Risks via Option Based Portfolio Insurance
This paper proposes an option-based portfolio insurance method for international foreign exchange risk hedging. Each investor is assumed to maximize the expected utility of his/her portfolio which includes international risky assets and foreign currency ...
Portfolio optimization using predictive auxiliary classifier generative adversarial networks
AbstractIn financial engineering, portfolio optimization has been of consistent interest. Portfolio optimization is a process of modulating asset distributions to maximize expected returns and minimize risks. Despite numerous studies on ...
Highlights- Proposed a novel portfolio optimization algorithm and a deep learning model.
- ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2024
6901 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 25 - 29, 2024
Barcelona, Spain
Acceptance Rates
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 523Total Downloads
- Downloads (Last 12 months)523
- Downloads (Last 6 weeks)181
Reflects downloads up to 11 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in