Abstract
Reinforcement learning is commonly applied in residential energy management, particularly for optimizing energy costs. However, RL agents often face challenges when dealing with deceptive and sparse rewards in the energy control domain, especially with stochastic rewards. In such situations, thorough exploration becomes crucial for learning an optimal policy. Unfortunately, the exploration mechanism can be misled by deceptive reward signals, making thorough exploration difficult. Go-Explore is a family of algorithms which combines planning methods and reinforcement learning methods to achieve efficient exploration. We use the Go-Explore algorithm to solve the cost-saving task in residential energy management problems and achieve an improvement of up to 19.84% compared to the well-known reinforcement learning algorithms.
Supported by Irish Research Council & University of Galway.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barker, S., et al.: Smart*: an open data set and tools for enabling research in sustainable homes. SustKDD 111(112), 108 (2012)
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K.O., Clune, J.: Go-explore: a new approach for hard-exploration problems. arXiv preprint arXiv:1901.10995 (2019)
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K.O., Clune, J.: First return, then explore. Nature 590(7847), 580–586 (2021)
Glavic, M., Fonteneau, R., Ernst, D.: Reinforcement learning for electric power system decision and control: past considerations and perspectives. IFAC-PapersOnLine 50(1), 6918–6927 (2017)
Haq, E.U., Lyu, C., Xie, P., Yan, S., Ahmad, F., Jia, Y.: Implementation of home energy management system based on reinforcement learning. Energy Rep. 8, 560–566 (2022)
Huang, C., Zhang, H., Wang, L., Luo, X., Song, Y.: Mixed deep reinforcement learning considering discrete-continuous hybrid action space for smart home energy management. J. Mod. Power Syst. Clean Energy 10(3), 743–754 (2022)
Ilager, S., Ramamohanarao, K., Buyya, R.: Thermal prediction for efficient energy management of clouds using machine learning. IEEE Trans. Parallel Distrib. Syst. 32(5), 1044–1056 (2020)
Lu, J., Mannion, P., Mason, K.: A multi-objective multi-agent deep reinforcement learning approach to residential appliance scheduling. IET Smart Grid 5(4), 260–280 (2022)
Lu, R., Hong, S.H., Yu, M.: Demand response for home energy management using reinforcement learning and artificial neural network. IEEE Trans. Smart Grid 10(6), 6629–6639 (2019)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
PJM: 2021 PJM dataset (2021). https://dataminer2.pjm.com/feed/rt_fivemin_mnt_lmps. https://www.pjm.com/markets-and-operations
Ren, M., Liu, X., Yang, Z., Zhang, J., Guo, Y., Jia, Y.: A novel forecasting based scheduling method for household energy management system based on deep reinforcement learning. Sustain. Urban Areas 76, 103207 (2022)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Shuvo, S.S., Yilmaz, Y.: Home energy recommendation system (HERS): a deep reinforcement learning method based on residents’ feedback and activity. IEEE Trans. Smart Grid 13(4), 2812–2821 (2022)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (2018)
Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., Lai, C.S.: A multi-agent reinforcement learning-based data-driven method for home energy management. IEEE Trans. Smart Grid 11(4), 3201–3211 (2020)
Yu, L., et al.: Deep reinforcement learning for smart home energy management. IEEE Internet Things J. 7(4), 2751–2762 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, J., Mannion, P., Mason, K. (2024). Go-Explore for Residential Energy Management. In: Nowaczyk, S., et al. Artificial Intelligence. ECAI 2023 International Workshops. ECAI 2023. Communications in Computer and Information Science, vol 1948. Springer, Cham. https://doi.org/10.1007/978-3-031-50485-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-50485-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50484-6
Online ISBN: 978-3-031-50485-3
eBook Packages: Computer ScienceComputer Science (R0)