Article

Self-managed decentralised systems using K-components and collaborative reinforcement learning

Authors:

Jim Dowling,

Vinny CahillAuthors Info & Claims

WOSS '04: Proceedings of the 1st ACM SIGSOFT workshop on Self-managed systems

Pages 39 - 43

https://doi.org/10.1145/1075405.1075413

Published: 31 October 2004 Publication History

Get Access

Abstract

Components in a decentralised system are faced with uncertainty as how to best adapt to a changing environment to maintain or optimise system performance. How can individual components learn to adapt to recover from faults in an uncertain environment? How can a decentralised system coordinate the adaptive behaviour of its components to realise system optimisation goals given problems establishing consensus in dynamic environments? This paper introduces a self-adaptive component model, called K-Components, that enables individual components adapt to a changing environment and a decentralised coordination model, called collaborative reinforcement learning, that enables groups of components to learn to collectively adapt their behaviour to establish and maintain system-wide properties in a changing environment.

References

[1]

N. H. Minsky, "On conditions for self-healing in distributed software systems," Proceedngs of the Autonomic Computing Workshop, AMS '03, 2003.

Google Scholar

[2]

A. Montresor, H. Meling, and O. Babaoglu, "Towards self-organizing, self-repairing and resilient distributed systems," Future Directions in Distributed Computing, vol. LNCS 2584, 2003.

Digital Library

Google Scholar

[3]

M. Hayden, "The ensemble system," PhD Thesis, Cornell University: Dept. of Computer Science, 1997.

Digital Library

Google Scholar

[4]

D. Garlan and B. Schmerl, "Model-based adaptation for self-healing systems," in Proceedings of the first workshop on Self-healing systems, pp. 27--32, ACM Press, 2002.

Digital Library

Google Scholar

[5]

R. Khare and R. N. Taylor, "Extending the representational state transfer (rest) architectural style for decentralized systems," in Proceedings of the International Conference on Software Engineering (ICSE), 2004.

Digital Library

Google Scholar

[6]

I. Clarke, S. Miller, T. Hong, O. Sandberg, and B. Wiley, "Protecting free expression online with freenet," IEEE Internet Computing, Jan/Feb, 2002.

Digital Library

Google Scholar

[7]

E. Curran and J. Dowling, "Sample: An on-demand probabilistic routing protocol for ad-hoc networks," Technical Report: Department of Computer Science, Trinity College Dublin, 2004.

Google Scholar

[8]

J. Kennedy and R. Eberhart, Swarm Intelligence. San Francisco, California: Morgan Kaufmann, 2001.

Digital Library

Google Scholar

[9]

E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm Intelligence: from natural to artificial systems. New York: Oxford University Press, 1999.

Digital Library

Google Scholar

[10]

J. Dowling and V. Cahill, "The k-component architecture meta-model for self-adaptive software," Proceedings of Reflection 2001, 2001.

Digital Library

Google Scholar

[11]

R. Sutton and A. Barto, Reinforcement Learning. MIT Press, 1998.

Digital Library

Google Scholar

[12]

J. Dowling, E. Curran, R. Cunningham, and V. Cahill., "Collaborative reinforcement learning of autonomic behaviour," 2nd International Workshop on Self-Adaptive and Autonomic Computing Systems, 2004.

Digital Library

Google Scholar

[13]

S. Camazine, J. Deneubourg, N. Franks, J. Sneyd, G. Theraulaz, and E. Bonabeau, Self-Organization in Biological Systems. Princeton University Press, 2003.

Digital Library

Google Scholar

[14]

M. Jelasity, A. Montresor, and O. Babaoglu, "A modular paradigm for building self-organizing peer-to-peer applications," Proceedings of ESOP03: International Workshop on Engineering Self-Organising Applications, 2003.

Google Scholar

Cited By

View all

Arellanes D(2021)Self-Organizing Software Models for the Internet of Things: Complex Software Structures That Emerge Without a Central ControllerIEEE Systems, Man, and Cybernetics Magazine10.1109/MSMC.2021.30628227:3(4-9)Online publication date: Jul-2021
https://doi.org/10.1109/MSMC.2021.3062822
de Lemos RGrześ MLitoiu MClarke STei K(2019)Self-adaptive artificial intelligenceProceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems10.1109/SEAMS.2019.00028(155-156)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/SEAMS.2019.00028
Christi AGroce AGopinath R(2019)Evaluating Fault Localization for Resource Adaptation via Test-Based Software Modification2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS.2019.00017(26-33)Online publication date: Jul-2019
https://doi.org/10.1109/QRS.2019.00017
Show More Cited By

Recommendations

Decentralized and partially decentralized multi-agent reinforcement learning
Reinforcement Learning: With Open AI, TensorFlow and Keras Using Python
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Recent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

WOSS '04: Proceedings of the 1st ACM SIGSOFT workshop on Self-managed systems

October 2004

119 pages

ISBN:1581139896

DOI:10.1145/1075405

Editors:
David Garlan
Carnegie Mellon University
,
Jeff Kramer
Imperial College
,
Alexander Wolf
University of Colorado, University of Lugano

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

WOSS04

Sponsor:

SIGSOFT

WOSS04: Workshop on Self-Healing Systems [co-located with ACM SIGSOFT 2004 )

October 31 - November 1, 2004

California, Newport Beach

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
590
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Arellanes D(2021)Self-Organizing Software Models for the Internet of Things: Complex Software Structures That Emerge Without a Central ControllerIEEE Systems, Man, and Cybernetics Magazine10.1109/MSMC.2021.30628227:3(4-9)Online publication date: Jul-2021
https://doi.org/10.1109/MSMC.2021.3062822
de Lemos RGrześ MLitoiu MClarke STei K(2019)Self-adaptive artificial intelligenceProceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems10.1109/SEAMS.2019.00028(155-156)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/SEAMS.2019.00028
Christi AGroce AGopinath R(2019)Evaluating Fault Localization for Resource Adaptation via Test-Based Software Modification2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS.2019.00017(26-33)Online publication date: Jul-2019
https://doi.org/10.1109/QRS.2019.00017
Magableh BAlmiani M(2019)A deep recurrent Q network towards self‐adapting distributed microservice architectureSoftware: Practice and Experience10.1002/spe.277850:2(116-135)Online publication date: 28-Nov-2019
https://doi.org/10.1002/spe.2778
Hayek EBen Yahia IArellanes DLau K(2018)Analysis of component-based approaches toward componentized 5G2018 21st Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN)10.1109/ICIN.2018.8401611(1-5)Online publication date: Feb-2018
https://doi.org/10.1109/ICIN.2018.8401611
Krupitzer CPfannemuller MKaddour JBecker C(2018)SATISFy: Towards a Self-Learning Analyzer for Time Series Forecasting in Self-Improving Systems2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W)10.1109/FAS-W.2018.00045(182-189)Online publication date: Sep-2018
https://doi.org/10.1109/FAS-W.2018.00045
Zhou YChen T(2017)ReferencesSoftware Adaptation in an Open Environment10.1201/9781315181721-11(195-212)Online publication date: 28-Apr-2017
https://doi.org/10.1201/9781315181721-11
Mattos DBosch JOlsson H(2017)Your System Gets Better Every Day You Use It: Towards Automated Continuous Experimentation2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2017.15(256-265)Online publication date: Aug-2017
https://doi.org/10.1109/SEAA.2017.15
Tzafilkou KProtogeros NKoumpis A(2017)User-centred cloud service adaptationInternational Journal of Computer Integrated Manufacturing10.1080/0951192X.2015.103069730:4-5(472-482)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1080/0951192X.2015.1030697
Ismail ACardellini V(2015)Decentralized Planning for Self-Adaptation in Multi-cloud EnvironmentAdvances in Service-Oriented and Cloud Computing10.1007/978-3-319-14886-1_9(76-90)Online publication date: 28-Feb-2015
https://doi.org/10.1007/978-3-319-14886-1_9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Decentralized and partially decentralized multi-agent reinforcement learning

Reinforcement Learning: With Open AI, TensorFlow and Keras Using Python

Reward Shaping in Episodic Reinforcement Learning