More Web Proxy on the site http://driver.im/

article

URL: A unified reinforcement learning approach for autonomic cloud management

Authors:

Cheng-Zhong Xu,

Xiangping BuAuthors Info & Claims

Journal of Parallel and Distributed Computing, Volume 72, Issue 2

Pages 95 - 105

https://doi.org/10.1016/j.jpdc.2011.10.003

Published: 01 February 2012 Publication History

Abstract

Cloud computing is emerging as an increasingly important service-oriented computing paradigm. Management is a key to providing accurate service availability and performance data, as well as enabling real-time provisioning that automatically provides the capacity needed to meet service demands. In this paper, we present a unified reinforcement learning approach, namely URL, to automate the configuration processes of virtualized machines and appliances running in the virtual machines. The approach lends itself to the application of real-time autoconfiguration of clouds. It also makes it possible to adapt the VM resource budget and appliance parameter settings to the cloud dynamics and the changing workload to provide service quality assurance. In particular, the approach has the flexibility to make a good trade-off between system-wide utilization objectives and appliance-specific SLA optimization goals. Experimental results on Xen VMs with various workloads demonstrate the effectiveness of the approach. It can drive the system into an optimal or near-optimal configuration setting in a few trial-and-error iterations.

References

[1]

Abdelzaher, T.F., Shin, K.G. and Bhatti, N., Performance guarantees for web server end-systems: a control-theoretical approach. IEEE Transactions on Parallel and Distributed Systems. v13 i1. 80-96.

[2]

C.G. Atkeson, J.C. Santamar'ia, A comparison of direct and model-based reinforcement learning. in: In International Conference on Robotics and Automation, 1997.

[3]

A. Bar-Hillel, A. Di-Nur, L. Ein-Dor, R. Gilad-Bachrach, Y. Ittach, Workstation capacity tuning using reinforcement learning. in: SC, 2007.

[4]

T. Bittman, The future of infrastructure and operations: the engine of cloud computing, in: Gartner 27th Annual Data Center Conference, December 2008.

[5]

X. Bu, J. Rao, C.-Z. Xu, A reinforcement learning approach to online web system autoconfiguration, in: Proc. of Int. Conf. on Distributed Computing Systems (ICDCS), 2009.

[6]

X. Bu, J. Rao, C.-Z. Xu, A model-free learning approach for coordinated configuration of virtual machines and appliances, in: Proc. of MASCOTS, 2011.

Digital Library

[7]

I.-H. Chung, J.K. Hollingsworth, Automated cluster-based web service performance tuning, in: HPDC, pp. 36-44, 2004.

[8]

C. Clark, K. Fraser, S. Hand, J.G. Hansen, E. Jul, C. Limpach, I. Pratt, A. Warfield, Live migration of virtual machines, in: NSDI, 2005.

[9]

Y. Diao, J. Hellerstein, S. Parekh, H. Shaikh, M. Surendra, Controlling quality of service in multi-tier web applications, in: Proc. of ICDCS, 2006.

[10]

D. Gupta, L. Cherkasova, R. Gardner, A. Vahdat, Enforcing performance isolation across virtual machines in xen, in: Middleware, 2006.

[11]

Huebscher, M.C. and Mccann, J.A., A survey of autonomic computing: degrees, models, and applications. ACM Computing Surveys. v40 i3.

[12]

IDC. Virtualization and multicore innovations disrupt the worldwide server market. document number: 206035, 2007.

[13]

E. Ipek, O. Mutlu, J.F. Martinez, R. Caruana, Self-optimizing memory controllers: a reinforcement learning approach, in: ISCA, 2008.

[14]

Kaelbling, L.P., Littman, M.L. and Moore, A.W., Reinforcement learning: a survey. Journal of Artificial Intelligence Research. v4. 237-285.

[15]

A. Kamra, V. Misra, E. Nahum, Yaksha: a self-tuning controller for managing the performance of 3-tiered websites, in: Proc. of IWQoS'04, pp. 47-56, 2004.

[16]

Kephart, J.O. and Chess, D.M., The vision of autonomic computing. IEEE Computer.

[17]

E. Kwan, S. Lightstone, A. Storm, A. Wu, Automatic configuratoin of ibm db2 universal database, 2002.

[18]

X. Liu, L. Sha, Y. Diao, S. Froehlich, J.L. Hellerstein, S.S. Parekh, Online response time optimization of apache web server, in: IWQoS, pp. 461-478, 2003.

[19]

Lu, C., Lu, Y., Abdelzaher, T., Stankovic, J. and Son, S., Feedback control architecture and design methodology for service delay guarantees in web servers. IEEE Transactions on Parallel and Distributed Systems. v17 i9. 1014-1027.

[20]

D. Ongaro, A.L. Cox, S. Rixner, Scheduling i/o in virtual machine monitors, in: Poc. of ACM/Usenix Int. Conf. on Virtual Execution Environment (VEE), 2008.

[21]

P. Padala, K.G. Shin, X. Zhu, M. Uysal, Z. Wang, S. Singhal, A. Merchant, K. Salem, Adaptive control of virtualized resources in utility computing environments, in: EuroSys, 2007.

Digital Library

[22]

J. Rao, X. Bu, C.-Z. Xu, K. Wang, A distributed self-learning approach for elastic provisioning of virtualized cloud resources, in: Proc. of MASCOTS, 2011.

Digital Library

[23]

J. Rao, X. Bu, C.-Z. Xu, L.Y. Wang, G. Yin, Vconf: a reinforcement learning approach for virtual machine autoconfiguration, in: Proc. of Int. Conf. on Autonomic Computing (ICAC), 2009.

[24]

J. Rao, C.-Z. Xu, Online measurement of the capacity of multi-tier websites using hardware counters, in: Proc. of ICDCS, 2008. (An extended version is to appear in IEEE Trans. on Parallel and Distributed Systems).

[25]

L. Roderick, E. Zamost, J. Anderson, Vmmark: a scalable benchmark for virtualized systems. Technical Report VMware-TR-2006-002, VMware, Inc., 2006. www.vmware.com/products/vmmark.

[26]

A.A. Soror, U.F. Minhas, A. Aboulnaga, K. Salem, P. Kokosielis, S. Kamath, Automatic virtual machine configuration for database workloads, in: SIGMOD Conference, 2008.

[27]

Sutton, R.S. and Barto, A.G., Reinforcement Learning: An Introduction. 1998. MIT Press.

[28]

G. Tesauro, Online resource allocation using decompositional reinforcement learning, in: AAAI, 2005.

Digital Library

[29]

Tesauro, G., Das, R., Chan, H., Kephart, J., Levine, D., Rawson, F. and Lefurgy, C., Managing power consumption and performance of computing systems using reinforcement learning. Advances in Neural Information Processing Systems. v20.

[30]

Tesauro, G., Jong, N.K., Das, R. and Bennani, M.N., On the use of hybrid reinforcement learning for autonomic resource allocation. Cluster Computing.

[31]

Wei, J. and Xu, C.-Z., eQoS: provisioning of client-perceived end-to-end QoS guarantees in web servers. IEEE Transactions on Computers. v55 i12. 1543-1556.

[32]

Wei, J., Zhou, X. and Xu, C.-Z., Robust processing rate allocation for proportional slowdown differentiation on Internet servers. IEEE Transactions on Computers. v54 i8. 964-977.

[33]

A. Whitaker, R.S. Cox, S.D. Gribble, Configuration debugging as search: finding the needle in the haystack, in: OSDI, 2004.

[34]

B. Xi, Z. Liu, M. Raghavachari, C.H. Xia, L. Zhang, A smart hill-climbing algorithm for application server configuration, in: WWW, pp. 287-296, 2004.

[35]

Xu, C.-Z., Wei, J. and Liu, F., Model predictive feedback control for end-to-end qos guarantees in web servers. IEEE Computer. v41 i3. 66-72.

[36]

Yin, G., Xu, C.-Z. and Wang, L., Q-learning algorithms with random truncation bounds and applications to effective parallel computing. Journal of Optimization and Applications. v137 i2. 435-452.

[37]

Y. Zhang, W. Qu, A. Liu, Automatic performance tuning for j2ee application server systems, in: WISE, pp. 520-527, 2005.

[38]

W. Zheng, R. Bianchini, T.D. Nguyen, Automatic configuration of internet services, in: EuroSys, pp. 219-229, 2007.

Digital Library

[39]

Zhou, X., Wei, J. and Xu, C.-Z., Resource allocation for session-based 2d service differentiation on e-commerce servers. IEEE Transactions on Parallel and Distributed Systems. v17 i8. 838-850.

Cited By

Theodoros AAndreas CDimitris C(2023)Reinforcement Learning Techniques for Optimizing System Configuration on the Cloud: A Taxonomy and Open ProblemsProceedings of the 2023 International Conference on embedded Wireless Systems and Networks10.5555/3639940.3639995(345-350)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.5555/3639940.3639995
Zhou GTian WBuyya R(2023)Multi-search-routes-based methods for minimizing makespan of homogeneous and heterogeneous resources in Cloud computingFuture Generation Computer Systems10.1016/j.future.2022.11.031141:C(414-432)Online publication date: 15-Feb-2023
https://dl.acm.org/doi/10.1016/j.future.2022.11.031
Panwar SRauthan MBarthwal V(2022)A systematic review on effective energy utilization management strategies in cloud data centersJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-022-00368-511:1Online publication date: 17-Dec-2022
https://dl.acm.org/doi/10.1186/s13677-022-00368-5
Show More Cited By

URL: A unified reinforcement learning approach for autonomic cloud management

Recommendations

VCONF: a reinforcement learning approach to virtual machines auto-configuration
ICAC '09: Proceedings of the 6th international conference on Autonomic computing

Virtual machine (VM) technology enables multiple VMs to share resources on the same host. Resources allocated to the VMs should be re-configured dynamically in response to the change of application demands or resource supply. Because VM execution ...
Personalized Service Delivery using Reinforcement Learning in Fog and Cloud Environment
iiWAS2019: Proceedings of the 21st International Conference on Information Integration and Web-based Applications & Services

The ability to fulfil the resource demand in runtime is encouraging the businesses to migrate to cloud. Recently, to provide real-time cloud services and to save network resources, fog computing is introduced. To further improve the quality of service ...
Reinforcement Learning Based Service Provisioning for a Greener Cloud
ICECCS '14: Proceedings of the 2014 3rd International Conference on Eco-friendly Computing and Communication Systems

Cloud computing is an emerging distributed computing model consisting of massive data enters for making different services available to the users. In the current scenario where energy consumption and wastage in the IT field is looked upon with growing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Parallel and Distributed Computing

Journal of Parallel and Distributed Computing Volume 72, Issue 2

February, 2012

228 pages

ISSN:0743-7315

Issue’s Table of Contents

Copyright © Elsevier Inc. © 2011.

Publisher

Academic Press, Inc.

United States

Publication History

Published: 01 February 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Theodoros AAndreas CDimitris C(2023)Reinforcement Learning Techniques for Optimizing System Configuration on the Cloud: A Taxonomy and Open ProblemsProceedings of the 2023 International Conference on embedded Wireless Systems and Networks10.5555/3639940.3639995(345-350)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.5555/3639940.3639995
Zhou GTian WBuyya R(2023)Multi-search-routes-based methods for minimizing makespan of homogeneous and heterogeneous resources in Cloud computingFuture Generation Computer Systems10.1016/j.future.2022.11.031141:C(414-432)Online publication date: 15-Feb-2023
https://dl.acm.org/doi/10.1016/j.future.2022.11.031
Panwar SRauthan MBarthwal V(2022)A systematic review on effective energy utilization management strategies in cloud data centersJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-022-00368-511:1Online publication date: 17-Dec-2022
https://dl.acm.org/doi/10.1186/s13677-022-00368-5
Zhong ZXu MRodriguez MXu CBuyya R(2022)Machine Learning-based Orchestration of Containers: A Taxonomy and Future DirectionsACM Computing Surveys10.1145/351041554:10s(1-35)Online publication date: 13-Sep-2022
https://dl.acm.org/doi/10.1145/3510415
Panwar RSupriya M(2022)Dynamic resource provisioning for service-based cloud applicationsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2022.06.001168:C(90-107)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.jpdc.2022.06.001
Soni DKumar N(2022)Machine learning techniques in emerging cloud computing integrated paradigmsJournal of Network and Computer Applications10.1016/j.jnca.2022.103419205:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.jnca.2022.103419
Funika WKoperek PKitowski J(2022)Automated cloud resources provisioning with the use of the proximal policy optimizationThe Journal of Supercomputing10.1007/s11227-022-04924-379:6(6674-6704)Online publication date: 10-Nov-2022
https://dl.acm.org/doi/10.1007/s11227-022-04924-3
Padakandla S(2021)A Survey of Reinforcement Learning Algorithms for Dynamically Varying EnvironmentsACM Computing Surveys10.1145/345999154:6(1-25)Online publication date: 13-Jul-2021
https://dl.acm.org/doi/10.1145/3459991
Asghari ASohrabi MYaghmaee F(2021)Task scheduling, resource provisioning, and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithmThe Journal of Supercomputing10.1007/s11227-020-03364-177:3(2800-2828)Online publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1007/s11227-020-03364-1
Caviglione LGaggero MPaolucci MRonco R(2021)Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacentersSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-020-05462-x25:19(12569-12588)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1007/s00500-020-05462-x
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents