Abstract
The growth of scale and complexity of software as well as the complex environment with high dynamic lead to the uncertainties in self-adaptive software’s decision making at run time. Self-adaptive software needs the ability to avoid negative effects of uncertainties to the quality attributes of the software. However, existing planning methods cannot handle the two types of runtime uncertainties caused by complexity of system and running environment. This paper proposes a planning method to handle these two types of runtime uncertainties based on reinforcement learning. To handle the uncertainty from the system, the proposed method can exchange ineffective self-adaptive strategies to effective ones according to the iterations of execution effects at run time. It can plan dynamically to handle uncertainty from environment by learning knowledge of relationship between system states and actions. This method can also generate new strategies to deal with unknown situations. Finally, we use a complex distributed e-commerce system, Bookstore, to validate the ability of proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Krupitzer, C., Roth, F.M., Vansyckel, S., et al.: A survey on engineering approaches for self-adaptive systems. Pervasive Mob. Comput. 17(PB), 184–206 (2015)
Cheng, S.W., Garlan, D.: Handling uncertainty in autonomic systems. In: International Workshop on Living with Uncertainties (2007)
Esfahani, N., Kouroshfar, E., Malek, S., et al.: Taming uncertainty in self-adaptive software. In: 13th European conference on Foundations of Software Engineering, pp. 234–244. ACM (2011)
Elkhodary, A., Esfahani, N., Malek, S., et al.: FUSION: a framework for engineering self-tuning self-adaptive software systems. In: 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 7–16. ACM (2010)
Mao, X., Dong, M., Liu, L., et al.: An integrated approach to developing self-adaptive software. J. Inf. Sci. Eng. 30(4), 1071–1085 (2014)
Amoui, M., Salehie, M., Mirarab, S., et al.: Adaptive action selection in autonomic software using reinforcement learning. In: International Conference on Autonomic and Autonomous Systems, pp. 175–181. IEEE Computer Society (2008)
Acknowledgement
This work is supported by the Projects (61672401) supported by the National Natural Science Foundation of China; Projects (315***10101, 315**0102) supported by the Pre-Research Project of the “Thirteenth Five-Year-Plan” of China.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, T., Li, Q., Wang, L., He, L., Li, Y. (2018). Using Reinforcement Learning to Handle the Runtime Uncertainties in Self-adaptive Software. In: Mazzara, M., Ober, I., Salaün, G. (eds) Software Technologies: Applications and Foundations. STAF 2018. Lecture Notes in Computer Science(), vol 11176. Springer, Cham. https://doi.org/10.1007/978-3-030-04771-9_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-04771-9_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04770-2
Online ISBN: 978-3-030-04771-9
eBook Packages: Computer ScienceComputer Science (R0)