More Web Proxy on the site http://driver.im/

research-article

Multi-type data fusion framework based on deep reinforcement learning for algorithmic trading

Authors:

Caiming ZhangAuthors Info & Claims

Applied Intelligence, Volume 53, Issue 2

Pages 1683 - 1706

https://doi.org/10.1007/s10489-022-03321-w

Published: 30 April 2022 Publication History

Abstract

In recent years, research on algorithmic trading based on machine learning has been increasing. One challenge faced is getting an accurate representation of the stock market environment from multi-type data. Most existing algorithmic trading studies analyze the stock market based on a relatively single data source. However, with the complicated stock market environment, different types of data reflect the changes in the stock market from different perspectives, and how to obtain the temporal features of different types of data and integrate them to obtain a deeper representation of the stock market environment are still problems to be solved. To tackle these problems, in this study, we combine deep learning and reinforcement learning (RL) and propose a multi-type data fusion framework with deep reinforcement learning (MSF-DRL) that integrates stock data, technical indicators and candlestick charts, in which technical indicators can reduce the impact of noise in stock data. In the process of learning trading strategies under the MSF-DRL framework, the temporal features of stock data and technical indicators are extracted through a long short-term memory (LSTM) network, and a convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) are successively used to extract the features of the candlestick chart. The fused features are used as the input of the RL module, which makes trading decisions on this basis. To verify the effectiveness of the MSF-DRL framework, we conducted comparative experiments on datasets composed of Chinese stocks and some stocks of the S&P 500 stock market index. Compared with the other trading strategies, our trading strategy can obtain more profits and a higher Sharpe ratio.

References

[1]

Fister D, Perc M, and Jagrič T Two robust long short-term memory frameworks for trading stocks Appl Intell 2021 51 10 7177-7195

[2]

Soleymani F and Paquet E Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder—DeepBreath Expert Syst Appl 2020 156 113456

[3]

Thakkar A and Chaudhari K Fusion in stock market prediction: a decade survey on the necessity, recent developments, and potential future directions Inf Fusion 2021 65 95-107

[4]

Li L, Zhu F, Sun H, et al. Multi-source information fusion and deep-learning-based characteristics measurement for exploring the effects of peer engagement on stock price synchronicity Inf Fusion 2021 69 1-21

[5]

Ozbayoglu AM, Gudelek MU, and Sezer OB Deep learning for financial applications: a survey Appl Soft Comput 2020 93 106384

[6]

Du W and Ding S A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications Artif Intell Rev 2021 54 3215-3238

[7]

Silver D, Hubert T, Schrittwieser J, et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play Science 2018 362 1140-1144

[8]

Liu S, Cao J, Wang Y, et al. Self-play reinforcement learning with comprehensive critic in computer games Neurocomputing 2021 449 207-213

[9]

Zielinski KMC, Hendges LV, Florindo JB, et al. Flexible control of discrete event systems using environment simulation and reinforcement learning Appl Soft Comput 2021 111 107714

[10]

Apolinarska AA, Pacher M, Li H, et al. Robotic assembly of timber joints using reinforcement learning Autom Constr 2021 125 103569

[11]

Chakole JB, Kolhe MS, Mahapurush GD, et al. A Q-learning agent for automated trading in equity stock markets Expert Syst Appl 2021 163 113761

[12]

Arulkumaran K, Deisenroth MP, Brundage M, and Bharath AA Deep reinforcement learning: a brief survey IEEE Signal Process Mag 2017 34 6 26-38

[13]

Sezer OB and Ozbayoglu AM Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach Appl Soft Comput 2018 70 525-538

[14]

Deng Y, Bao F, Kong Y, Ren Z, and Dai Q Deep direct reinforcement learning for financial signal representation and trading IEEE Trans Neural Netw Learn Syst 2017 28 3 653-664

[15]

Théate T and Ernst D An application of deep reinforcement learning to algorithmic trading Expert Syst Appl 2021 173 114632

[16]

Lei K, Zhang B, Li Y, et al. Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading Expert Syst Appl 2020 140 112872

[17]

Lee J, Koh H, and Choe HJ Learning to trade in financial time series using high-frequency through wavelet transformation and deep reinforcement learning Appl Intell 2021 51 6202-6223

[18]

Ma C, Zhang J, Liu J, et al. A parallel multi-module deep reinforcement learning algorithm for stock trading Neurocomputing 2021 449 290-302

[19]

Wu X, Chen H, Chen C, et al. The autonomous navigation and obstacle avoidance for USVs with ANOA deep reinforcement learning method Knowl-Based Syst 2020 196 105201

[20]

Fan G, Hua Z, and Li J Multi-scale depth information fusion network for image dehazing Appl Intell 2021 51 10 7262-7280

[21]

Liu M, Yan X, Wang C, and Wang K Segmentation mask-guided person image generation Appl Intell 2021 51 2 1161-1176

[22]

Lippi M, Montemurro MA, Degli Esposti M, and Cristadoro G Natural language statistical features of lstm-generated texts IEEE Transactions on Neural Networks and Learning Systems 2019 30 3326-3337

[23]

Feng Q, He D, Liu Z, Wang H, and Choo KR SecureNLP: A System for Multi-Party Privacy-Preserving Natural Language Processing IEEE Transactions on Information Forensics and Security 2020 15 3709-3721

[24]

Gridach M A framework based on (probabilistic) soft logic and neural network for NLP Appl Soft Comput 2020 93 106232

[25]

Xie Y, Liang R, Liang Z, et al. Speech emotion classification using attention-based LSTM IEEEACM Trans Audio Speech Lang Process 2019 27 1675-1685

[26]

Tu YH, Du J, and Lee CH Speech enhancement based on Teacher–Student deep learning using improved speech presence probability for Noise-Robust speech recognition IEEEACM Trans Audio Speech Lang Process 2019 27 2080-2091

[27]

Li S, Xing X, Fan W, et al. Spatiotemporal and frequential cascaded attention networks for speech emotion recognition Neurocomputing 2021 448 238-248

[28]

Fischer T and Krauss C Deep learning with long short-term memory networks for financial market predictions Eur J Oper Res 2018 270 2 654-669

[29]

Baek Y and Kim HY Modaugnet: A new forecasting framework for stock market index value with an overfitting prevention LSTM module and a prediction LSTM module Expert Syst Appl 2018 113 457-480

[30]

Zhou F, Zhou H, Yang Z, and Yang L EMD2FNN: A strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction Expert Syst Appl 2019 115 136-151

[31]

Lu Y, Chen Y, Zhao D, and Li D MGRL: Graph Neural network based inference in a Markov network with reinforcement learning for visual navigation Neurocomputing 2021 421 140-150

[32]

Wu M-E, Syu J-H, Lin JC-w, Ho J-M (2021) Portfolio management system in equity market neutral using reinforcement learning. Appl Intell

[33]

Wu X, Chen H, Wang J, Troiano L, Loia V, and Fujita H Adaptive stock trading strategies with deep reinforcement learning methods Inf Sci 2020 538 142-158

[34]

Carta S, Corriga A, Ferreira A, Podda AS, and Recupero AS A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning Appl Intell 2021 51 2 889-905

[35]

Shynkevich Y, McGinnity TM, Coleman SA, et al. Forecasting price movements using technical indicators: Investigating the impact of varying input window length Neurocomputing 2017 264 71-88

[36]

Lee SH, Levin D, Finley PD, and Heilig CM Chief complaint classification with recurrent neural networks J Biomed Inform 2019 93 103158

[37]

Mammone N, Ieracitano C, and Morabito FC A deep CNN approach to decode motor preparation of upper limbs from time–frequency maps of EEG signals at source level Neural Netw 2020 124 357-372

[38]

Liu M, Lu Y, Long S, et al. An attention-based CNN-biLSTM hybrid neural network enhanced with features of discrete wavelet transformation for fetal acidosis classification Expert Syst Appl 2021 186 115714

[39]

Kim HY and Won CH Forecasting the volatility of stock price index: a hybrid model integrating LSTM with multiple GARCH-type models Expert Syst Appl 2018 103 25-37

Cited By

Wang LWang X(2024)An adaptive financial trading strategy based on proximal policy optimization and financial signal representationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109365138:PAOnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.109365
Kwon YLee Z(2024)A hybrid decision support system for adaptive trading strategiesDecision Support Systems10.1016/j.dss.2023.114100177:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.dss.2023.114100
Liu WGu YGe Y(2024)Multi-factor stock trading strategy based on DQN with multi-BiGRU and multi-head ProbSparse self-attentionApplied Intelligence10.1007/s10489-024-05463-554:7(5417-5440)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05463-5

Recommendations

Portfolio dynamic trading strategies using deep reinforcement learning
Abstract
Using the constituent stocks of the iShares MSCI US ESG Select Index ETF, a matrix of technical indicators, returns, and covariance is incorporated to represent the inherent information characteristics of the stock market. In this study, based on ...
Algorithmic trading using continuous action space deep reinforcement learning▪
Abstract
Finding a more efficient trading strategy has always been one of the main concerns in financial market trading. In order to create trading strategies that lead to higher profits, historical data must be used. Due to a large amount of financial ...
Highlights
- Proposing an autonomous trading algorithm.
- Providing smart trading capability in stock markets.
- Employing a continuous action space DRL model.
- Facilitating Money management.
A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets
Abstract
Algorithmic trading based on machine learning is a developing and promising field of research. Financial markets have a complex, uncertain, and dynamic nature, making them challenging for trading. Some financial theories, such as the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Applied Intelligence

Applied Intelligence Volume 53, Issue 2

Jan 2023

1252 pages

ISSN:0924-669X

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 30 April 2022

Accepted: 28 January 2022

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
National Natural Science Foundation of China
Natural Science Foundation of Shandong Province
Natural Science Foundation of Shandong Province
Natural Science Foundation of Shandong Province
NSFC-Zhejiang Joint Fund of the Integration of Informatization and Industrialization
Key Research and Development Project of Shandong Province
Key Research and Development Project of Shandong Province
Science and technology plan for young talents in Colleges and universities of Shandong Province
Natural Science Foundation of Shandong Province

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang LWang X(2024)An adaptive financial trading strategy based on proximal policy optimization and financial signal representationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109365138:PAOnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.109365
Kwon YLee Z(2024)A hybrid decision support system for adaptive trading strategiesDecision Support Systems10.1016/j.dss.2023.114100177:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.dss.2023.114100
Liu WGu YGe Y(2024)Multi-factor stock trading strategy based on DQN with multi-BiGRU and multi-head ProbSparse self-attentionApplied Intelligence10.1007/s10489-024-05463-554:7(5417-5440)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05463-5

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents