Abstract
The field of smart meter data analytics is a relatively young field that recently grew due to the wealth of data generated from smart meters. Recent progress in high-performance computing made it possible to efficiently use deep neural networks to extract more actionable knowledge and patterns from massive data collected within the smart grid. Examining the available literature, there is a need for a comprehensive survey that covers all aspects of smart meter data analytics. This review paper provides a broad overview of the current research spectrum within smart meter analysis, while identifying future challenges for smart meter data analytic through a detailed taxonomy. The proposed taxonomy identifies the main domains within the field, as well as the various functions that are conducted within each domain. To show the strength of deep learning techniques in smart meter analytics, an experiment is conducted comparing various classical load profiling techniques with deep clustering. The results of the experiment conducted on an open large dataset show the performance of deep clustering to be better than classical techniques. Furthermore, the review is performed from the perspective of data science; emphasizing the data tasks such as data reprocessing, analytic algorithms, and evaluation methods for each domain. Furthermore, the paper discusses deep neural network techniques used for each domain of smart meter analysis and identifies gaps within each domain, including obstacles and potential opportunities for research.
Similar content being viewed by others
References
Eskandarnia E, Al-Ammal HM. A taxonomy of smart meter analytics: forecasting, knowledge discovery, and power management. Int J Comput Digit Syst. 2022;11:851–9.
Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge: MIT Press; 2016.
Amasyali K, El-Gohary NM. A review of data-driven building energy consumption prediction studies. Renew Sustain Energy Rev. 2018;81:1192–205.
Beaudin M, Zareipour H. Home energy management systems: a review of modelling and complexity. Renew Sustain Energy Rev. 2015;45:318–35.
Khuntia SR, Tuinema BW, Rueda JL, van der Meijden MA. Time-horizons in the planning and operation of transmission networks: an overview. IET Gener Trans Distrib. 2016;10(4):841–8.
Srivastava AK, Pandey AS, Singh D. Short-term load forecasting methods: a review. Int Conf Emerg Trends Electr Electron Sustain Energy Syst (ICETEESES). 2016. https://doi.org/10.1109/ICETEESES.2016.7581373.
Sun L, Zhou K, Yang S. Regional difference of household electricity consumption: an empirical study of Jiangsu, China. J Clean Prod. 2018;171:1415–28. https://doi.org/10.1016/j.jclepro.2017.10.123.
Swan LG, Ugursal VI. Modeling of end-use energy consumption in the residential sector: a review of modeling techniques. Renew Sustain Energy Rev. 2009;13(8):1819–35.
Kakran S, Chanana S. Smart operations of smart grids integrated with distributed generation: a review. Renew Sustain Energy Rev. 2018;81:524–35.
Nafi NS, Ahmed K, Gregory MA, Datta M. A survey of smart grid architectures, applications, benefits and standardization. J Netw Comput Appl. 2016;76:23–36.
Wang Y, Chen Q, Kang C, Zhang M, Wang K, Zhao Y. Load profiling and its application to demand response: a review. Tsinghua Sci Technol. 2015;20(2):117–29.
Deb C, Zhang F, Yang J, Lee SE, Shah KW. A review on time series forecasting techniques for building energy consumption. Renew Sustain Energy Rev. 2017;74:902–24. https://doi.org/10.1016/j.rser.2017.02.085.
Raza MQ, Khosravi A. A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings. Renew Sustain Energy Rev. 2015;50:1352–72. https://doi.org/10.1016/j.rser.2015.04.065.
Vrablecová P, Bou Ezzeddine A, Rozinajová V, Šárik S, Sangaiah AK. Smart grid load forecasting using online support vector regression. Comput Electr Eng. 2018. https://doi.org/10.1016/j.compeleceng.2017.07.006.
Wei Y, Zhang X, Shi Y, Xia L, Pan S, Wu J, Han M, Zhao X. A review of data-driven approaches for prediction and classification of building energy consumption. Renew Sustain Energy Rev. 2018;82:1027–47.
Zor K, Timur O, Teke A. A state-of-the-art review of artificial intelligence techniques for short-term electric load forecasting. In: Energy (IYCE) 2017 6th International Youth Conference on IEEE, 2017, pp. 1–7.
Alahakoon D, Yu X. Smart electricity meter data intelligence for future energy systems: a survey. IEEE Trans Ind Inf. 2016;12(1):425–36.
Yildiz B, Bilbao JI, Dore J, Sproul AB. Recent advances in the analysis of residential electricity consumption and applications of smart meter data. Appl Energy. 2017;208:402–27. https://doi.org/10.1016/j.apenergy.2017.10.014.
Liu X, Golab L, Golab W, Ilyas IF, Jin S. Smart meter data analytics: systems, algorithms, and benchmarking. ACM Trans Database Syst (TODS). 2017;42(1):2.
Yin X, Li X, Zhang Y, Zhang T, Lu C, Ai Q, Li Z, Sun Z. A survey of deep learning and its application in distribution network. In: 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), IEEE, 2020, pp. 643–6.
Ibrahim MS, Dong W, Yang Q. Machine learning driven smart electric power systems: current trends and new perspectives. Appl Energy. 2020;272:115–237.
Wang Y, Chen Q, Hong T, Kang C. Review of smart meter data analytics: applications, methodologies, and challenges. IEEE Trans Smart Grid. 2018;10:3125–48.
Petersen K, Feldt R, Mujtaba S, Mattsson M. Systematic mapping studies in software engineering. Int Conf Eval Assess Softw Eng (EASE). 2008;12:1–10.
Iskandarnia E, Al-Amal H, Al-Madany W. Load forecasting in different scale and horizon-a review. In: 3rd Smart Cities Symposium (SCS 2020), Vol. 2020. IET, 2020.
Khuntia SR, Rueda JL, van der Meijden MA. Forecasting the load of electrical power systems in mid-and long-term horizons: a review. IET Gener Trans Distrib. 2016;10(16):3971–7.
Kong W, Dong ZY, Hill DJ, Luo F, Xu Y. Short-term residential load forecasting based on resident behaviour learning. IEEE Trans Power Syst. 2018;33(1):1087–8. https://doi.org/10.1109/TPWRS.2017.2688178.
Kuster C, Rezgui Y, Mourshed M. Electrical load forecasting models: a critical systematic review. Sustain Cities Soc. 2017;35:257–70. https://doi.org/10.1016/j.scs.2017.08.009.
Yildiz B, Bilbao JI, Sproul AB. A review and analysis of regression and machine learning models on commercial building electricity load forecasting. Renew Sustain Energy Rev. 2017;73:1104–22. https://doi.org/10.1016/j.rser.2017.02.023.
Dedinec A, Filiposka S, Dedinec A, Kocarev L. Deep belief network based electricity load forecasting: an analysis of macedonian case. Energy. 2016;115:1688–700.
Marino DL, Amarasinghe K, Manic M. Building energy load forecasting using deep neural networks. In: Industrial Electronics Society, IECON 2016-42nd Annual Conference of the IEEE, 2016, pp. 7046–51.
Ma W, Fang S, Liu G, Zhou R. Modeling of district load forecasting for distributed energy system. Appl Energy. 2017;204:181–205. https://doi.org/10.1016/j.apenergy.2017.07.009.
Kim M, Park S, Lee J, Joo Y, Choi JK. Learning-based adaptive imputation method with kNN algorithm for missing power data. Energies. 2017. https://doi.org/10.3390/en10101668.
Li S, Wang P, Goel L. Short-term load forecasting by wavelet transform and evolutionary extreme learning machine. Electr Power Syst Res. 2015;122:96–103.
Dang-Ha T-H, Bianchi FM, Olsson R. Local short term electricity load forecasting: Automatic approaches. In: Neural Networks (IJCNN), 2017 International Joint Conference on IEEE, 2017, pp. 4267–74.
Chandrashekar G, Sahin F. A survey on feature selection methods. Comput Electr Eng. 2014;40(1):16–28.
Ghamisi P, Benediktsson JA. Feature selection based on hybridization of genetic algorithm and particle swarm optimization. IEEE Geosci Remote Sens Lett. 2015;12(2):309–13.
Hu Z, Bao Y, Chiong R, Xiong T. Mid-term interval load forecasting using multi-output support vector regression with a memetic algorithm for feature selection. Energy. 2015;84:419–31. https://doi.org/10.1016/j.energy.2015.03.054.
Koprinska I, Rana M, Agelidis VG. Correlation and instance based feature selection for electricity load forecasting. Knowl Based Syst. 2015;82:29–40. https://doi.org/10.1016/j.knosys.2015.02.017.
Eskandarnia E, AlHammad M. Predication of future energy consumption using SARIMAX. 2021, pp. 657–62.
Eskandar EM, Alammal H, Almadany W. What prophet says about electrical consumption-forecasting techniques for big temporal data. 2021, pp. 543–48.
Liu C, Jin Z, Gu J, Qiu C. Short-term load forecasting using a long short-term memory network. IEEE PES Innov Smart Grid Technol Conf Eur (ISGT-Europe). 2017. https://doi.org/10.1109/ISGTEurope.2017.8260110.
Chen T, Wang Y-C. Long-term load forecasting by a collaborative fuzzy-neural approach. Int J Electr Power Energy Syst. 2012;43(1):454–64.
Chen T. A collaborative fuzzy-neural approach for long-term load forecasting in Taiwan. Comput Ind Eng. 2012;63(3):663–70.
Chaouch M. Clustering-based improvement of nonparametric functional time series forecasting: application to intra-day household-level load curves. IEEE Trans Smart Grid. 2014;5(1):411–9.
Lee W-J, Hong J. A hybrid dynamic and fuzzy time series model for mid-term power load forecasting. Int J Electr Power Energy Syst. 2015;64:1057–62. https://doi.org/10.1016/j.ijepes.2014.08.006.
de Oliveira EM, Cyrino Oliveira FL. Forecasting mid-long term electric energy consumption through bagging ARIMA and exponential smoothing methods. Energy. 2018;144:776–88. https://doi.org/10.1016/j.energy.2017.12.049.
Ghofrani M, Hassanzadeh M, Etezadi-Amoli M, Fadali MS. Smart meter based short-term load forecasting for residential customers. In: North American Power Symposium (NAPS), IEEE, 2011, pp. 1–5.
Hong T, Wilson J, Xie J. Long term probabilistic load forecasting and normalization with hourly information. IEEE Trans Smart Grid. 2014;5(1):456–62. https://doi.org/10.1109/TSG.2013.2274373.
Lusis P, Khalilpour KR, Andrew L, Liebman A. Short-term residential load forecasting: impact of calendar effects and forecast granularity. Appl Energy. 2017;205:654–69. https://doi.org/10.1016/j.apenergy.2017.07.114.
Amjady N, Keynia F. Mid-term load forecasting of power systems by a new prediction method. Energy Convers Manag. 2008;49(10):2678–87. https://doi.org/10.1016/j.enconman.2008.04.008.
Yuan L, Ma J, Gu J, Wen H, Jin Z. Featuring periodic correlations via dual granularity inputs structured RNNS ensemble load forecaster. Int Trans Electr Energy Syst. 2020;30(11):12571.
Yang Y, Hong W, Li S. Deep ensemble learning based probabilistic load forecasting in smart grids. Energy. 2019;189:116–324.
Fahiman F, Erfani SM, Rajasegarar S, Palaniswami M, Leckie C. Improving load forecasting based on deep learning and k-shape clustering. In: 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, 2017, pp. 4134–41.
Yang Y, Li W, Gulliver TA, Li S. Bayesian deep learning-based probabilistic load forecasting in smart grids. IEEE Trans Ind Inf. 2019;16(7):4703–13.
Sun M, Zhang T, Wang Y, Strbac G, Kang C. Using bayesian deep learning to capture uncertainty for residential net load forecasting. IEEE Trans Power Syst. 2019;35(1):188–201.
Fekri MN, Patel H, Grolinger K, Sharma V. Deep learning for load forecasting with smart meter data: online adaptive recurrent neural network. Appl Energy. 2021;282:116–77.
Razavi SE, Arefi A, Ledwich G, Nourbakhsh G, Smith D, Minakshi M. From load to net energy forecasting: short-term residential forecasting for the blend of load and pv behind the meter. IEEE Access. 2020;8:224343–53.
Madhure, R.U., Raman, R., Singh, S.K.: Cnn-lstm based electricity theft detector in advanced metering infrastructure. In: 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), IEEE, 2020, pp. 1–6
Khan ZA, Hussain T, Ullah A, Rho S, Lee M, Baik SW. Towards efficient electricity forecasting in residential and commercial buildings: a novel hybrid cnn with a lstm-ae based framework. Sensors. 2020;20(5):1399.
Bouktif S, Fiaz A, Ouni A, Serhani MA. Multi-sequence lstm-rnn deep learning and metaheuristics for electric load forecasting. Energies. 2020;13(2):391.
Aghabozorgi S, Seyed Shirkhorshidi A, Ying Wah T. Time-series clustering: a decade review. Inf Syst. 2015;53:16–38. https://doi.org/10.1016/j.is.2015.04.007.
Arbelaitz O, Gurrutxaga I, Muguerza J, Pérez JM, Perona I. An extensive comparative study of cluster validity indices. Pattern Recogn. 2013;46(1):243–56.
Eskandarnia E, Al-Ammal HM, Ksantini R. An embedded deep-clustering-based load profiling framework. Sustain Cities Soc. 2021;78: 103618.
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, PMLR, 2016, pp. 478–87.
Mrabah N, Bouguessa M, Ksantini R. Adversarial deep embedded clustering: on a better trade-off between feature randomness and feature drift. IEEE Trans Knowl Data Eng. 2020. https://doi.org/10.1109/TKDE.2020.2997772.
Aljalbout E, Golkov V, Siddiqui Y, Strobel M, Cremers D. Clustering with deep learning: Taxonomy and new methods. 2018. arXiv preprint arXiv:1801.07648
Feng C, Mehmani A, Zhang J. Deep learning-based real-time building occupancy detection using ami data. IEEE Trans Smart Grid. 2020;11(5):4490–501.
Chen H, Wang Y-H, Fan C-H. A convolutional autoencoder-based approach with batch normalization for energy disaggregation. J Supercomput. 2020;73:2961–78.
Paresh S, Thokala NK, Chandra MG. Electrical load disaggregation using a two-stage deep learning approach. In: Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, 2019, pp. 366–67.
He W, Chai Y. An empirical study on energy disaggregation via deep learning. In: International conference on artificial intelligence and industrial engineering (AIIE 2016). London: Atlantis Press; 2016. p. 338–42.
Ullah A, Haydarov K, Ul Haq I, Muhammad K, Rho S, Lee M, Baik SW. Deep learning assisted buildings energy consumption profiling using smart meter data. Sensors. 2020;20(3):873.
Zhang P, Cheng H, Zou B, Dai P, Ye C. Load data mining based on deep learning method. In: Proceedings of the 3rd International Conference on Computer Science and Application Engineering, 2019, pp. 1–5.
Gong X, Tang B, Zhu R, Liao W, Song L. Data augmentation for electricity theft detection using conditional variational auto-encoder. Energies. 2020;13(17):4291.
Kelly J, Knottenbelt W. Neural nilm: Deep neural networks applied to energy disaggregation. In: Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-efficient Built Environments, 2015, pp. 55–64.
Nabil M, Ismail M, Mahmoud M, Shahin M, Qaraqe K, Serpedin E. Deep learning-based detection of electricity theft cyber-attacks in smart grid ami networks. In: Deep learning applications for cyber security. Berlin: Springer; 2019. p. 73–102.
Wang Y, Chen Q, Gan D, Yang J, Kirschen DS, Kang C. Deep learning-based socio-demographic information identification from smart meter data. IEEE Trans Smart Grid. 2018;10:2593–602.
Garcia FCC, Macabebe EQB. Inferring appliance energy usage from smart meters using fully convolutional encoder decoder networks. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017, pp. 2075–8.
Li J, Wang F. Non-technical loss detection in power grids with statistical profile images based on semi-supervised learning. Sensors. 2020;20(1):236.
Hassani H, Farajzadeh-Zanjani M, Razavi-Far R, Saif M, Palade V. Design of a cost-effective deep convolutional neural network–based scheme for diagnosing faults in smart grids. In: 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), IEEE, 2019, pp. 1420–25.
Singh S, Majumdar A. Deep sparse coding for non-intrusive load monitoring. IEEE Trans Smart Grid. 2017;9(5):4669–78.
Devlin MA, Hayes BP. Load identification and classification of activities of daily living using residential smart meter data. In: 2019 IEEE Milan PowerTech, IEEE, 2019, pp. 1–6.
Devlin MA, Hayes BP. Non-intrusive load monitoring and classification of activities of daily living using residential smart meter data. IEEE Trans Consum Electron. 2019;65(3):339–48.
Farkas M, Lacko P. Using advanced audio generating techniques to model electrical energy load. In: International conference on engineering applications of neural networks. Berlin: Springer; 2017. p. 39–48.
Singhal V, Maggu J, Majumdar A. Simultaneous detection of multiple appliances from smart-meter measurements via multi-label consistent deep dictionary learning and deep transform learning. IEEE Trans Smart Grid. 2018;10(3):2969–78.
Muratori M, Rizzoni G. Residential demand response: dynamic energy management and time-varying electricity pricing. IEEE Trans Power Syst. 2016;31(2):1108–17.
Mahmud AA, Sant P. Real-time price savings through price suggestions for the smart grid demand response model. In: Smart Grid and Cities Congress and Fair (ICSG), 2017 5th International Istanbul, IEEE, 2017, pp. 65–9.
Längkvist M, Karlsson L, Loutfi A. A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn Lett. 2014;42:11–24.
Hosseini SS, Agbossou K, Kelouwani S, Cardenas A. Non-intrusive load monitoring through home energy management systems: a comprehensive review. Renew Sustain Energy Rev. 2017;79:1266–74. https://doi.org/10.1016/j.rser.2017.05.096.
Esa NF, Abdullah MP, Hassan MY. A review disaggregation method in non-intrusive appliance load monitoring. Renew Sustain Energy Rev. 2016;66:163–73.
Basu K, Debusschere V, Bacha S, Maulik U, Bondyopadhyay S. Nonintrusive load monitoring: a temporal multilabel classification approach. IEEE Trans Ind Inf. 2015;11(1):262–70.
Azaza M, Wallin F. Evaluation of classification methodologies and features selection from smart meter data. Energy Proced. 2017;142:2250–6.
Alkhulaifi A, Aljohani AJ. Investigation of deep learning-based techniques for load disaggregation, low-frequency approach. Int J Adv Comput Sci Appl. 2020;11:701–6.
Messinis GM, Hatziargyriou ND. Review of non-technical loss detection methods. Electric Power Systems Research. 2018;158:250–66.
Vercamer D, Steurtewagen B, Van den Poel D, Vermeulen F. Predicting consumer load profiles using commercial and open data. IEEE Trans Power Syst. 2016;31(5):3693–701.
Christensen MH, Nozal DC, Kavadakis I, Pinson P. Data-driven learning from dynamic pricing data-classification and forecasting. In: 2019 IEEE Milan PowerTech, IEEE, 2019, pp. 1–6.
Rodriguez Fernandez M, Garcia AC, Alonso IG, Casanova EZ. Using the big data generated by the smart home to improve energy efficiency management. Energy Eff. 2016;9(1):249–60. https://doi.org/10.1007/s12053-015-9361-3.
Ye Y, Qiu D, Wu X, Strbac G, Ward J. Model-free real-time autonomous control for a residential multi-energy system using deep reinforcement learning. IEEE Trans Smart Grid. 2020;11(4):3068–82.
Shateri M, Messina F, Piantanida P, Labeau F. Privacy-cost management in smart meters using deep reinforcement learning. In: 2020 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe), 2020, pp. 929–33.
Patyn C, Reymond M, Rădulescu R, Deconinck G, Nowé A. Reinforcement learning for demand response of domestic household appliances. In: Adaptive Learning Agents 2018 Proceedings, 2018, pp. 1–7.
Lu R, Hong SH, Zhang X. A dynamic pricing demand response algorithm for smart grid: reinforcement learning approach. Appl Energy. 2018;220:220–30.
Zhou X, Canady R, Li Y, Koutsoukos X, Gokhale A. Overcoming stealthy adversarial attacks on power grid load predictions through dynamic data repair. In: International conference on dynamic data driven application systems. Berlin: Springer; 2020. p. 102–9.
Jahromi AN, Sakhnini J, Karimpour H, Dehghantanha A. A deep unsupervised representation learning approach for effective cyber-physical attack detection and identification on highly imbalanced data. In: Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering, 2019, pp. 14–23.
Marulli F, Visaggio CA. Adversarial deep learning for energy management in buildings. In: Summer Sim, 2019, pp. 1–50.
Zhou L, Ouyang X, Ying H, Han L, Cheng Y, Zhang T. Cyber-attack classification in smart grid via deep neural network. In: Proceedings of the 2nd International Conference on Computer Science and Application Engineering, 2018, pp. 1–5.
Kong X, Kong D, Yao J, Bai L, Xiao J. Online pricing of demand response based on long short-term memory and reinforcement learning. Appl Energy. 2020;271: 114945.
Liu M, Liu D, Sun G, Zhao Y, Wang D, Liu F, Fang X, He Q, Xu D. Deep learning detection of inaccurate smart electricity meters: a case study. IEEE Ind Electron Mag. 2020;14(4):79–90.
Niu X, Li J, Sun J, Tomsovic K. Dynamic detection of false data injection attack in smart grid using deep learning. In: 2019 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference (ISGT), IEEE, 2019, pp. 1–6.
Jindal A, Aujla GS, Kumar N, Prodan R, Obaidat MS. Drums: demand response management in a smart city using deep learning and svr. In: 2018 IEEE Global Communications Conference (GLOBECOM), IEEE, 2018, pp. 1–6.
Asghar MR, Dán G, Miorandi D, Chlamtac I. Smart meter data privacy: a survey. IEEE Commun Surv Tutor. 2017;19(4):2820–35.
Soykan EU, Bilgin Z, Ersoy MA, Tomur E. Differentially private deep learning for load forecasting on smart grid. In: 2019 IEEE Globecom Workshops (GC Wkshps), IEEE, 2019, pp. 1–6.
Sun L, Zhou K, Yang S. An ensemble clustering based framework for household load profiling and driven factors identification. Sustain Cities Soc. 2020;53: 101958.
Sehovac L, Nesen C, Grolinger K. Forecasting building energy consumption with deep learning: a sequence to sequence approach. In: 2019 IEEE International Congress on Internet of Things (ICIOT), IEEE, pp. 108–116.
Taïk A, Cherkaoui S. Electrical load forecasting using edge computing and federated learning. In: ICC 2020-2020 IEEE International Conference on Communications (ICC), IEEE, pp. 1–6.
Acknowledgements
We thank Michael Hutton of the ōbex project for language editing support.
Funding
Not applicable
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Eskandarnia, E., Al-Ammal, H., Ksantini, R. et al. Deep Learning Techniques for Smart Meter Data Analytics: A Review. SN COMPUT. SCI. 3, 243 (2022). https://doi.org/10.1007/s42979-022-01161-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42979-022-01161-6