FusionSF: Fuse Heterogeneous Modalities in a Vector Quantized Framework for Robust Solar Power Forecasting
Authors: 
Ziqing Ma, Wenwei Wang, Tian Zhou, Chao Chen, Bingqing Peng, Liang Sun, Rong JinAuthors Info & Claims
Pages 5532 - 5543
Abstract
Accurate solar power forecasting is crucial to integrate photovoltaic plants into the electric grid, schedule and secure the power grid safety. This problem becomes more demanding for those newly installed solar plants which lack sufficient operational data. Current research predominantly relies on historical solar power data or numerical weather prediction in a single-modality format, ignoring the complementary information provided in different modalities. In this paper, we propose a multi-modality fusion framework to integrate historical power data, numerical weather prediction, and satellite images, significantly improving forecast performance. We introduce a vector quantized framework that aligns modalities with varying information densities, striking a balance between integrating sufficient information and averting model overfitting. Our framework demonstrates strong zero-shot forecasting capability, which is especially useful for those newly installed plants. Moreover, we collect and release a multi-modal solar power (MMSP) dataset from real-world plants to further promote the research of multi-modal solar forecasting algorithms. Our extensive experiments show that our model not only operates with robustness but also boosts accuracy in both zero-shot forecasting and scenarios rich with training data, surpassing leading models. We have incorporated it into our eForecaster platform and deployed it for more than 300 solar plants with a total capacity of over 15GW. Our code and dataset are accessible at https://github.com/DAMO-DI-ML/FusionSF.git.
Supplemental Material
MP4 File - FusionSF_promotion_video
a promotion video for FusionSF: Fuse Heterogeneous Modalities in a Vector Quantized Framework for Robust Solar Power Forecasting
- Download
- 4.81 MB
References
[1]
Anas Al-lahham, Obaidah Theeb, Khaled Elalem, Tariq A. Alshawi, and Saleh A. Alshebeili. 2020. Sky Imager-Based Forecast of Solar Irradiance Using Machine Learning. Electronics, Vol. 9, 10 (2020). https://doi.org/10.3390/electronics9101700
[2]
Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. CoRR, Vol. abs/1308.3432 (2013). showeprint[arXiv]1308.3432 http://arxiv.org/abs/1308.3432
[3]
Kotaro BESSHO, Kenji DATE, Masahiro HAYASHI, Akio IKEDA, Takahito IMAI, Hidekazu INOUE, Yukihiro KUMAGAI, Takuya MIYAKAWA, Hidehiko MURATA, Tomoo OHNO, Arata OKUYAMA, Ryo OYAMA, Yukio SASAKI, Yoshio SHIMAZU, Kazuki SHIMOJI, Yasuhiko SUMIDA, Masuo SUZUKI, Hidetaka TANIGUCHI, Hiroaki TSUCHIYAMA, Daisaku UESAWA, Hironobu YOKOTA, and Ryo YOSHIDA. 2016. An Introduction to Himawari-8/9 mdash; Japan rsquo;s New-Generation Geostationary Meteorological Satellites. Journal of the Meteorological Society of Japan. Ser. II, Vol. 94, 2 (2016), 151--183. https://doi.org/10.2151/jmsj.2016-009
[4]
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. 2023. Accurate medium-range global weather forecasting with 3D neural networks. Nat., Vol. 619, 7970 (2023), 533--538. https://doi.org/10.1038/S41586-023-06185--3
[5]
Oussama Boussif, Ghait Boukachab, Dan Assouline, Stefano Massaroli, Tianle Yuan, Loubna Benabbou, and Yoshua Bengio. 2023. What if We Enrich day-ahead Solar Irradiance Time Series Forecasting with Spatio-Temporal Context? CoRR, Vol. abs/2306.01112 (2023). https://doi.org/10.48550/ARXIV.2306.01112 showeprint[arXiv]2306.01112
[6]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3--7, 2021. OpenReview.net. https://openreview.net/forum?id=YicbFdNTTy
[7]
Michael Emmanuel, Kate Doubleday, Burcin Cakir, Marija Markovi?, and Bri-Mathias Hodge. 2020. A review of power system planning and operational models for flexibility assessment in high solar energy penetration scenarios. Solar Energy, Vol. 210 (2020), 169--180. https://doi.org/10.1016/j.solener.2020.07.017 Special Issue on Grid Integration.
[8]
Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Wang, Mu Li, and Dit-Yan Yeung. 2022. Earthformer: Exploring Space-Time Transformers for Earth System Forecasting. In NeurIPS.
[9]
Zhangyang Gao, Cheng Tan, Lirong Wu, and Stan Z. Li. 2022. SimVP: Simpler yet Better Video Prediction. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18--24, 2022. IEEE, 3160--3170. https://doi.org/10.1109/CVPR52688.2022.00317
[10]
D González-Zeas, B Erazo, P Lloret, B De Bièvre, S Steinschneider, and Olivier Dangles. 2019. Linking global climate change. Science of the Total Environment, Vol. 650 (2019), 2577--2586.
[11]
Yanyi He, Kaicun Wang, and Fei Feng. 2021. Improvement of ERA5 over ERA-Interim in simulating surface incident solar radiation throughout China. Journal of Climate, Vol. 34, 10 (2021), 3853--3867.
[12]
Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Mu noz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. 2020. The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, Vol. 146, 730 (2020), 1999--2049.
[13]
William F Holmgren, Clifford W Hansen, and Mark A Mikofski. 2018. pvlib python: A python package for modeling solar energy systems. Journal of Open Source Software, Vol. 3, 29 (2018), 884.
[14]
Tao Hong, Pierre Pinson, Shu Fan, Hamidreza Zareipour, Alberto Troccoli, and Rob J. Hyndman. 2016. Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond. International Journal of Forecasting, Vol. 32, 3 (2016), 896--913. https://doi.org/10.1016/j.ijforecast.2016.02.001
[15]
Pierre Ineichen. 2016. Validation of models that estimate the clear sky global and beam solar irradiance. Solar Energy, Vol. 132 (2016), 332--344.
[16]
Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, et al. 2023. Learning skillful medium-range global weather forecasting. Science (2023), eadi2336.
[17]
David A Lavers, Adrian Simmons, Freja Vamborg, and Mark J Rodwell. 2022. An evaluation of ERA5 precipitation for climate monitoring. Quarterly Journal of the Royal Meteorological Society, Vol. 148, 748 (2022), 3152--3165.
[18]
Zhuo Li, Kejie Wang, Chenchen Li, Miao Zhao, and Jiannong Cao. 2019. Multimodal Deep Learning for Solar Irradiance Prediction. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). 784--792. https://doi.org/10.1109/iThings/GreenCom/CPSCom/SmartData.2019.00144
[19]
Zhuo Li, Kejie Wang, Chenchen Li, Miao Zhao, and Jiannong Cao. 2019. Multimodal Deep Learning for Solar Irradiance Prediction. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). 784--792. https://doi.org/10.1109/iThings/GreenCom/CPSCom/SmartData.2019.00144
[20]
Tariq Limouni, Reda Yaagoubi, Khalid Bouziane, Khalid Guissi, and El Houssain Baali. 2023. Accurate one step and multistep forecasting of very short-term PV power using LSTM-TCN model. Renewable Energy, Vol. 205 (2023), 1010--1024. https://doi.org/10.1016/j.renene.2023.01.118
[21]
Jingxuan Liu, Haixiang Zang, Lilin Cheng, Tao Ding, Zhinong Wei, and Guoqiang Sun. 2023. A Transformer-based multimodal-learning framework using sky images for ultra-short-term solar irradiance forecasting. Applied Energy, Vol. 342 (2023), 121160. https://doi.org/10.1016/j.apenergy.2023.121160
[22]
Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6--9, 2019. OpenReview.net. https://openreview.net/forum?id=Bkg6RiCqY7
[23]
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1--5, 2023. OpenReview.net. https://openreview.net/pdf?id=Jbdc0vTOcol
[24]
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. 2021. Meta-Learning Framework with Applications to Zero-Shot Time-Series Forecasting. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2--9, 2021. AAAI Press, 9242--9250. https://doi.org/10.1609/AAAI.V35I10.17115
[25]
Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar. 2022. FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators. CoRR, Vol. abs/2202.11214 (2022). showeprint[arXiv]2202.11214 https://arxiv.org/abs/2202.11214
[26]
Elissaios Sarmas, Nikos Dimitropoulos, Vangelis Marinakis, Zoi Mylona, and Haris Doukas. 2022. Transfer learning strategies for solar power forecasting under data scarcity. Nature Scientific Reports 12 (2022), 14643. https://doi.org/10.1038/s41598-022--18516-x
[27]
Xingjian SHI, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-kin Wong, and Wang-chun WOO. 2015. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. In Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2015/file/07563a3fe3bbe7e3ba84431ad9d055af-Paper.pdf
[28]
Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. 2021. RoFormer: Enhanced Transformer with Rotary Position Embedding. CoRR, Vol. abs/2104.09864 (2021). showeprint[arXiv]2104.09864 https://arxiv.org/abs/2104.09864
[29]
Esra Suel, Samir Bhatt, Michael Brauer, Seth Flaxman, and Majid Ezzati. 2021. Multimodal deep learning from satellite and street-level imagery for measuring income, overcrowding, and environmental deprivation in urban areas. Remote Sensing of Environment, Vol. 257 (2021), 112339. https://doi.org/10.1016/j.rse.2021.112339
[30]
Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4--9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 6306--6315. https://proceedings.neurips.cc/paper/2017/hash/7a98af17e63a0ac09ce2e96d03992fbc-Abstract.html
[31]
Qingsong Wen, Linxiao Yang, Tian Zhou, and Liang Sun. 2022. Robust time series analysis and applications: An industrial perspective. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4836--4837.
[32]
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. 2023. Transformers in Time Series: A Survey. 6778--6786. https://doi.org/10.24963/ijcai.2023/759
[33]
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. 2023. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1--5, 2023. OpenReview.net. https://openreview.net/pdf?id=ju_Uqw384Oq
[34]
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). 101--112.
[35]
Dazhi Yang, Wenting Wang, Christian A. Gueymard, Tao Hong, Jan Kleissl, Jing Huang, Marc J. Perez, Richard Perez, Jamie M. Bright, Xiang'ao Xia, Dennis van der Meer, and Ian Marius Peters. 2022. A review of solar forecasting, its dependence on atmospheric sciences and implications for grid integration: Towards carbon neutrality. Renewable and Sustainable Energy Reviews, Vol. 161 (2022), 112348. https://doi.org/10.1016/j.rser.2022.112348
[36]
Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2022. SoundStream: An End-to-End Neural Audio Codec. IEEE ACM Trans. Audio Speech Lang. Process., Vol. 30 (2022), 495--507. https://doi.org/10.1109/TASLP.2021.3129994
[37]
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2023. Are Transformers Effective for Time Series Forecasting?. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7--14, 2023, Brian Williams, Yiling Chen, and Jennifer Neville (Eds.). AAAI Press, 11121--11128. https://doi.org/10.1609/AAAI.V37I9.26317
[38]
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. 2022. Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures. CoRR, Vol. abs/2207.01186 (2022). https://doi.org/10.48550/ARXIV.2207.01186 showeprint[arXiv]2207.01186
[39]
Yunhao Zhang and Junchi Yan. 2023. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1--5, 2023. OpenReview.net. https://openreview.net/pdf?id=vSVLM2j9eie
[40]
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, Vol. 35. 11106--11115.
[41]
Tian Zhou, Ziqing Ma, Xue Wang, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, and Rong Jin. 2022. FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting. In NeurIPS. http://papers.nips.cc/paper_files/paper/2022/hash/524ef58c2bd075775861234266e5e020-Abstract-Conference.html
[42]
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In 39th International Conference on Machine Learning (ICML).
[43]
Tian Zhou, PeiSong Niu, Xue Wang, Liang Sun, and Rong Jin. 2023. One Fits All:Power General Time Series Analysis by Pretrained LM. arxiv: 2302.11939 [cs.LG]
[44]
Zhaoyang Zhu, Weiqi Chen, Rui Xia, Tian Zhou, Peisong Niu, Bingqing Peng, Wenwei Wang, Hengbo Liu, Ziqing Ma, Xinyue Gu, Jin Wang, Qiming Chen, Linxiao Yang, Qingsong Wen, and Liang Sun. 2023. Energy forecasting with robust, flexible, and explainable machine learning algorithms. AI Magazine, Vol. 44, 4 (dec 2023), 377--393. https://doi.org/10.1002/aaai.12130
[45]
Øyvind Sommer Klyve, Magnus Moe Nygård, Heine Nygard Riise, Jonathan Fagerström, and Erik Stensrud Marstein. 2023. The value of forecasts for PV power plants operating in the past, present and future Scandinavian energy markets. Solar Energy, Vol. 255 (2023), 208--221. https://doi.org/10.1016/j.solener.2023.03.044
Index terms have been assigned to the content through auto-classification.
Recommendations
Spatio-Temporal Consistency Enhanced Differential Network for Interpretable Indoor Temperature Prediction
KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data MiningIndoor temperature prediction is crucial for decision-making in central heating systems. Beyond accuracy, predictions shall be interpretable, i.e. conform to the laws of physics; otherwise, it may lead to system failures or unsafe conditions. However, ...
UrbanGPT: Spatio-Temporal Large Language Models
KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data MiningSpatio-temporal prediction aims to forecast and gain insights into the ever-changing dynamics of urban environments across both time and space. Its purpose is to anticipate future patterns, trends, and events in diverse facets of urban life, including ...
UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction
KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data MiningUrban spatio-temporal prediction is crucial for informed decision-making, such as traffic management, resource optimization, and emergence response. Despite remarkable breakthroughs in pretrained natural language models that enable one model to handle ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2024
6901 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 25 - 29, 2024
Barcelona, Spain
Acceptance Rates
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 186Total Downloads
- Downloads (Last 12 months)186
- Downloads (Last 6 weeks)22
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in