Short-Term Power Load Forecasting Using a VMD-Crossformer Model
<p>DSW embedding.</p> "> Figure 2
<p>Two-stage attention layer.</p> "> Figure 3
<p>Architecture of hierarchical encoder–decoder.</p> "> Figure 4
<p>Model prediction process.</p> "> Figure 5
<p>Original data: (<b>a</b>) I dataset; (<b>b</b>) II dataset. The datasets were split in a 7:1:2 ratio between training, validation, and testing. By training the model on the training set, the model could learn and recognize the laws and patterns of the time series data; by performing model evaluation and adjusting the parameters on the validation set, the model’s predictive capacity could be further enhanced to raise the model’s accuracy and stability; and by assessing the predicted outcomes of the model on a test set, the model’s generalizability was objectively assessed and its ability to accurately forecast electric loads for real-world applications was determined.</p> "> Figure 5 Cont.
<p>Original data: (<b>a</b>) I dataset; (<b>b</b>) II dataset. The datasets were split in a 7:1:2 ratio between training, validation, and testing. By training the model on the training set, the model could learn and recognize the laws and patterns of the time series data; by performing model evaluation and adjusting the parameters on the validation set, the model’s predictive capacity could be further enhanced to raise the model’s accuracy and stability; and by assessing the predicted outcomes of the model on a test set, the model’s generalizability was objectively assessed and its ability to accurately forecast electric loads for real-world applications was determined.</p> "> Figure 6
<p>VMD decomposition results: (<b>a</b>) I dataset; (<b>b</b>) II dataset.</p> "> Figure 7
<p>Comparison of simple model predictions: (<b>a</b>) I dataset; (<b>b</b>) II dataset.</p> "> Figure 8
<p>Comparison of complex model predictions: (<b>a</b>) I dataset; (<b>b</b>) II dataset.</p> ">
Abstract
:1. Introduction
- (1)
- Finding the ideal number of decomposition layers is made easier by adjusting the VMD parameter with the help of the PCC. This avoids problems with too many or too few VMD layers.
- (2)
- The original data are disaggregated by the VMD. Reconstructing the submodal data with the original data, the submodal data act as data augmentation that allows for feature highlighting of the original data. The network can then be trained with this new set of data to improve its ability for grasping the relationships between the variables.
- (3)
- The Crossformer network based on MTS prediction has a strong prediction capability. Among them, the DSW embedding mainly exploits cross-dimensional dependencies, and the TSA layer is equipped with an attention mechanism with temporal and dimensional phases to capture cross-temporal dependencies and cross-dimensional dependencies. The HED structure integrates information from different time scales for the final prediction.
- (4)
- The proposed VMD-Crossformer prediction model combines the strengths of each module. The VMD allows the submodal data to be used as data augmentation, while the Crossformer network captures the relationship between the submodalities and the original data in time and dimension. When comparing this model with other models on two datasets, this model had a higher prediction accuracy.
2. Fundamental Doctrine
2.1. Improved Variational Modal Decomposition
2.2. Crossformer Network
2.2.1. Dimensionally Segmented Embedding
2.2.2. Two-Stage Attention Layer
2.2.3. Hierarchical Encoder–Decoder
3. Construction of the VMD-Crossformer Prediction Model
4. Experimental Results and Analysis
4.1. Experimental Datasets
4.2. Evaluation Criteria
4.3. VMD
4.4. Analysis of Experimental Results
4.4.1. Parametric Optimization Results
4.4.2. Comparison of Forecast Results
5. Conclusions
- (1)
- The optimal number of decomposition layers matching the original signal was found using the VMD parameter optimization approach, which was based on the PCC. This method helped to prevent problems such as under-decomposition and over-decomposition of the signal caused by setting an inappropriate number of VMD modes.
- (2)
- The complex power load data were broken down into relatively simple submodal components using the VMD algorithm. Each modal component reflects the characteristics of the original signal in various frequency ranges, and each modal component is reconstructed with the original signal and then input into the prediction network, which can greatly improve the prediction accuracy.
- (3)
- The Crossformer network utilized cross-dimensional dependencies and information at different scales to capture the relationship between data more comprehensively and accurately predict the power load data.
- (4)
- Taking the GEFCom2014 dataset and the load dataset of the Belgium Power Grid Company as an example, the prediction based on VMD-Crossformer showed a higher prediction accuracy and better performance than other models.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Jahan, I.S.; Snasel, V.; Misak, S. Intelligent systems for power load forecasting: A study review. Energies 2020, 13, 6105. [Google Scholar] [CrossRef]
- Ahmad, N.; Ghadi, Y.; Adnan, M.; Ali, M. Load forecasting techniques for power system: Research challenges and survey. IEEE Access 2022, 10, 71054–71090. [Google Scholar] [CrossRef]
- Song, K.-B.; Baek, Y.-S.; Hong, D.H.; Jang, G. Short-term load forecasting for the holidays using fuzzy linear regression method. IEEE Trans. Power Syst. 2005, 20, 96–101. [Google Scholar] [CrossRef]
- Dudek, G.; Pełka, P.; Smyl, S. A hybrid residual dilated LSTM and exponential smoothing model for midterm electric load forecasting. IEEE Trans. Neural Netw. 2021, 33, 2879–2891. [Google Scholar] [CrossRef] [PubMed]
- Zhou, Y.; Ding, F. Modeling nonlinear processes using the radial basis function-based state-dependent autoregressive models. IEEE Signal Process Lett. 2020, 27, 1600–1604. [Google Scholar] [CrossRef]
- Pappas, S.S.; Ekonomou, L.; Karampelas, P.; Karamousantas, D.; Katsikas, S.; Chatzarakis, G.; Skafidas, P. Electricity demand load forecasting of the Hellenic power system using an ARMA model. Electr. Power Syst. Res. 2010, 80, 256–264. [Google Scholar] [CrossRef]
- Dai, Y.; Zhao, P. A hybrid load forecasting model based on support vector machine with intelligent methods for feature selection and parameter optimization. Appl. Energy 2020, 279, 115332. [Google Scholar] [CrossRef]
- Khwaja, A.S.; Anpalagan, A.; Naeem, M.; Venkatesh, B. Joint bagged-boosted artificial neural networks: Using ensemble machine learning to improve short-term electricity load forecasting. Electr. Power Syst. Res. 2020, 179, 106080. [Google Scholar] [CrossRef]
- Zhang, P.; Wu, X.; Wang, X.; Bi, S. Short-term load forecasting based on big data technologies. CSEE J. Power Energy Syst 2015, 1, 59–67. [Google Scholar] [CrossRef]
- Fan, G.-F.; Zhang, L.-Z.; Yu, M.; Hong, W.-C.; Dong, S.-Q. Applications of random forest in multivariable response surface for short-term load forecasting. Int. J. Electr. Power Energy Syst. 2022, 139, 108073. [Google Scholar] [CrossRef]
- Chen, B.; Wang, Y. Short-term electric load forecasting of integrated energy system considering nonlinear synergy between different loads. IEEE Access 2021, 9, 43562–43573. [Google Scholar] [CrossRef]
- Chodakowska, E.; Nazarko, J.; Nazarko, L. ARIMA models in electrical load forecasting and their robustness to noise. Energies 2021, 14, 7952. [Google Scholar] [CrossRef]
- Niu, D.; Yu, M.; Sun, L.; Gao, T.; Wang, K. Short-term multi-energy load forecasting for integrated energy systems based on CNN-BiGRU optimized by attention mechanism. Appl. Energy 2022, 313, 118801. [Google Scholar] [CrossRef]
- Shi, H.; Xu, M.; Li, R. Deep learning for household load forecasting—A novel pooling deep RNN. IEEE Trans. Smart Grid 2017, 9, 5271–5280. [Google Scholar] [CrossRef]
- Zhan, X.; Kou, L.; Xue, M.; Zhang, J.; Zhou, L. Reliable long-term energy load trend prediction model for smart grid using hierarchical decomposition self-attention network. IEEE Trans. Reliab. 2022, 72, 607–621. [Google Scholar] [CrossRef]
- Li, J.; Deng, D.; Zhao, J.; Cai, D.; Hu, W.; Zhang, M.; Huang, Q. A novel hybrid short-term load forecasting method of smart grid using MLR and LSTM neural network. IEEE Trans. Ind. Inf. 2020, 17, 2443–2452. [Google Scholar] [CrossRef]
- Li, D.; Sun, G.; Miao, S.; Gu, Y.; Zhang, Y.; He, S. A short-term electric load forecast method based on improved sequence-to-sequence GRU with adaptive temporal dependence. Int. J. Electr. Power Energy Syst. 2022, 137, 107627. [Google Scholar] [CrossRef]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017. [Google Scholar]
- Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. Proc. AAAI Conf. Artif. Intell. 2021, 35, 11106–11115. [Google Scholar] [CrossRef]
- Liu, S.; Yu, H.; Liao, C.; Li, J.; Lin, W.; Liu, A.X.; Dustdar, S. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In Proceedings of the International Conference on Learning Representations (ICLR 2021), Vienna, Austria, 4 May 2021. [Google Scholar]
- Zhang, Y.; Yan, J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In Proceedings of the The 11th International Conference on Learning Representations, Virtual Event, 25–29 April 2022. [Google Scholar]
- Han, X.; Su, J.; Hong, Y.; Gong, P.; Zhu, D. Mid-to long-term electric load forecasting based on the EMD–Isomap–Adaboost model. Sustainability 2022, 14, 7608. [Google Scholar] [CrossRef]
- Dragomiretskiy, K.; Zosso, D. Variational mode decomposition. IEEE Trans. Signal Process. 2013, 62, 531–544. [Google Scholar] [CrossRef]
- Zhu, S.; Xia, H.; Yin, W.; Wang, Z.; Zhang, J. Fault feature identification for rotor nonstationary signals based on VMD-HT. J. Harbin Eng. Univ. 2024, 45, 825–832. [Google Scholar]
- Wang, G.; Wang, X.; Wang, Z.; Ma, C.; Song, Z. A VMD–CISSA–LSSVM based electricity load forecasting model. Mathematics 2021, 10, 28. [Google Scholar] [CrossRef]
- Zhuang, Z.; Zheng, X.; Chen, Z.; Jin, T. A reliable short-term power load forecasting method based on VMD-IWOA-LSTM algorithm. IEEJ Trans. Electr. Electron. Eng. 2022, 17, 1121–1132. [Google Scholar] [CrossRef]
- Ribeiro, M.H.D.M.; da Silva, R.G.; Moreno, S.R.; Canton, C.; Larcher, J.H.K.; Stefenon, S.F.; Mariani, V.C.; Coelho, L.d.S. Variational mode decomposition and bagging extreme learning machine with multi-objective optimization for wind power forecasting. Appl. Intell. 2024, 54, 3119–3134. [Google Scholar] [CrossRef]
- Qin, L.; Yang, G.; Sun, Q. Maximum correlation Pearson correlation coefficient deconvolution and its application in fault diagnosis of rolling bearings. Measurement 2022, 205, 112162. [Google Scholar] [CrossRef]
- Xie, M.; Chai, C.; Guo, H.; Wang, M. Household electricity load forecasting based on pearson correlation coefficient clustering and convolutional neural network. J. Phys. Conf. Ser. 2020, 1601, 022012. [Google Scholar] [CrossRef]
- Peng, B.-S.; Xia, H.; Liu, Y.-K.; Yang, B.; Guo, D.; Zhu, S.-M. Research on intelligent fault diagnosis method for nuclear power plant based on correlation analysis and deep belief network. Prog. Nucl. Energy 2018, 108, 419–427. [Google Scholar] [CrossRef]
- Aksan, F.; Suresh, V.; Janik, P.; Sikorski, T. Load Forecasting for the Laser Metal Processing Industry Using VMD and Hybrid Deep Learning Models. Energies 2023, 16, 5381. [Google Scholar] [CrossRef]
- Sun, Q.; Cai, H. Short-term power load prediction based on VMD-SG-LSTM. IEEE Access 2022, 10, 102396–102405. [Google Scholar] [CrossRef]
- Tang, Y.; Cai, H. Short-term power load forecasting based on VMD-Pyraformer-Adan. IEEE Access 2023, 11, 61958–61967. [Google Scholar] [CrossRef]
Reference | Study Methodology | Field | Study Location | Datasets | Advantages | Shortcomings |
---|---|---|---|---|---|---|
[22] | EMDIA | Long-term electric load forecasting | Hong Kong | Total electricity consumption, meteorological data, and economic data (2000–2021) | Use of combinatorial models | EMD suffers from modal aliasing |
[25] | VMD-CISSA-LSSVM | Electricity load forecasting | A region in Shandong | Real historical load data (30 April 2007 to 12 September 2007) | Circumvention of EMD and algorithmic optimization of network parameters | Only one dataset was used and the effects of other factors were not taken into account |
[26] | VMD-IWOA-LSTM | Short-term power load forecasting | A particular heat region | Total electrical load, weather data, and special data | The data are reconstructed and fed into the network, and an algorithm is used to optimize the network parameters | Incomplete information in the dataset and high dependence on high-quality meteorological data inputs |
PCC | Degree of Relevance |
---|---|
0.8 < |p| ≤ 1.0 | Extremely relevant |
0.6 < |p| ≤ 0.8 | Highly relevant |
0.4 < |p| ≤ 0.6 | Moderately relevant |
0.2 < |p| ≤ 0.4 | Weak correlation |
0.0 ≤ |p| ≤ 0.2 | Hardly relevant |
(a) | ||||
---|---|---|---|---|
K | IMF1 | IMF2 | IMF3 | IMF4 |
2 | 0.540 | 0.798 | ||
3 | 0.526 | 0.765 | 0.371 | |
4 | 0.526 | 0.765 | 0.370 | 0.097 |
(b) | ||||
K | IMF1 | IMF2 | IMF3 | IMF4 |
2 | 0.874 | 0.575 | ||
3 | 0.809 | 0.469 | 0.420 | |
4 | 0.807 | 0.463 | 0.409 | 0.160 |
Segment Lengths | I Dataset | II Dataset | ||
---|---|---|---|---|
MAE/MW | MAPE/% | MAE/MW | MAPE/% | |
4 | 71.872 | 2.144 | 83.603 | 1.016 |
6 | 72.179 | 2.155 | 80.260 | 0.975 |
8 | 71.604 | 2.130 | 80.155 | 0.973 |
12 | 73.888 | 2.198 | 83.823 | 1.026 |
24 | 74.435 | 2.213 | 83.115 | 1.012 |
Attention Heads | I Dataset | II Dataset | ||
---|---|---|---|---|
MAE/MW | MAPE/% | MAE/MW | MAPE/% | |
2 | 74.976 | 2.237 | 90.290 | 1.102 |
4 | 71.604 | 2.130 | 80.155 | 0.973 |
6 | 73.197 | 2.194 | 78.268 | 0.952 |
8 | 74.558 | 2.225 | 78.900 | 0.956 |
Number of Routers | I Dataset | II Dataset | ||
---|---|---|---|---|
MAE/MW | MAPE/% | MAE/MW | MAPE/% | |
2 | 72.849 | 2.169 | 81.211 | 0.989 |
4 | 72.657 | 2.165 | 79.886 | 0.970 |
6 | 73.364 | 2.184 | 80.007 | 0.974 |
8 | 70.865 | 2.112 | 76.398 | 0.930 |
10 | 71.604 | 2.130 | 78.268 | 0.952 |
Batch Sizes | I Dataset | II Dataset | ||
---|---|---|---|---|
MAE/MW | MAPE/% | MAE/MW | MAPE/% | |
4 | 61.532 | 1.841 | 69.638 | 0.851 |
8 | 62.491 | 1.863 | 68.906 | 0.847 |
16 | 68.457 | 2.044 | 72.687 | 0.894 |
32 | 70.865 | 2.112 | 76.398 | 0.930 |
64 | 78.312 | 2.352 | 84.183 | 1.023 |
Predictive Model | I Dataset | II Dataset | ||||
---|---|---|---|---|---|---|
MAE/MW | MAPE/% | RMSE/MW | MAE/MW | MAPE/% | RMSE/MW | |
① GRU | 138.261 | 4.233 | 187.112 | 461.713 | 6.006 | 641.123 |
② Informer | 119.922 | 3.557 | 165.043 | 419.288 | 5.182 | 562.025 |
③ Crossformer | 110.201 | 3.271 | 154.903 | 374.571 | 4.636 | 510.108 |
④ EMD-Crossformer | 82.885 | 2.502 | 112.657 | 227.041 | 2.766 | 299.396 |
⑤ VMD-Crossformer | 61.532 | 1.841 | 84.486 | 68.906 | 0.847 | 89.209 |
Predictive Model | I Dataset | II Dataset | ||||
---|---|---|---|---|---|---|
MAE/MW | MAPE/% | RMSE/MW | MAE/MW | MAPE/% | RMSE/MW | |
VMD-CNN-LSTM | 104.410 | 3.135 | 150.349 | 122.817 | 1.541 | 163.344 |
VMD-SG-LSTM | 68.139 | 2.045 | 97.945 | 82.890 | 1.032 | 109.183 |
VMD-Pyraformer-Adan | 65.867 | 1.978 | 93.013 | 76.554 | 0.953 | 100.784 |
VMD-Crossformer | 61.532 | 1.841 | 84.486 | 68.906 | 0.847 | 89.209 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, S.; Cai, H. Short-Term Power Load Forecasting Using a VMD-Crossformer Model. Energies 2024, 17, 2773. https://doi.org/10.3390/en17112773
Li S, Cai H. Short-Term Power Load Forecasting Using a VMD-Crossformer Model. Energies. 2024; 17(11):2773. https://doi.org/10.3390/en17112773
Chicago/Turabian StyleLi, Siting, and Huafeng Cai. 2024. "Short-Term Power Load Forecasting Using a VMD-Crossformer Model" Energies 17, no. 11: 2773. https://doi.org/10.3390/en17112773
APA StyleLi, S., & Cai, H. (2024). Short-Term Power Load Forecasting Using a VMD-Crossformer Model. Energies, 17(11), 2773. https://doi.org/10.3390/en17112773