More Web Proxy on the site http://driver.im/

research-article

Four enhancements to automateddistributed system experimentation methods

Authors:

Antonio Carzaniga,

Alexander L. WolfAuthors Info & Claims

ICSE '08: Proceedings of the 30th international conference on Software engineering

Pages 491 - 500

https://doi.org/10.1145/1368088.1368155

Published: 10 May 2008 Publication History

Abstract

Experimentation is an essential tool employed by the developers of software systems, especially distributed systems. In prior work we developed a model-driven framework for automating various experimentation tasks, such as workload generation, and demonstrated that it gives the engineer a cost-effective means to conduct large-scale experiments on distributed testbeds. We have enhanced the methods underlying the framework in four significant ways: (1) increasing the expressiveness of workloads by allowing for conditional and reactive behaviors; (2) supporting the repeatability of experiments through the creation of environment workloads that can control the operational context; (3) enabling the composability of application and environment workloads to obtain a broader class of experiments; and (4) extending the scope of experiment management to include control over multiple runs. We use the enhancements to conduct a series of interesting new experiments. Specifically, the enhancements allow us to manipulate a fixed-wired testbed so that it simulates a mobile-wireless environment, and to selectively and maliciously inject faults into a system.

References

[1]

3rd Generation Partnership Project (3GPP). General Packet Radio Service (GPRS); Service description; Stage 2, June 2006. TS 23.060, V. 7.1.0.

[2]

J. Albrecht, C. Tuttle, A. C. Snoeren, and A. Vahdat. Planetlab application management using Plush. ACM Operating Systems Review, 40(1):33--40, Jan. 2006.

Digital Library

[3]

R. Chandra, R. M. Lefever, M. Cukier, and W. H. Sanders. Loki: A state-driven fault injector for distributed systems. In Proceedings of the International Conference on Dependable Systems and Networks, pages 237--242, New York, NY, June 2000.

Digital Library

[4]

B. N. Chun. DART: Distributed automated regression testing for large-scale network applications. In Proceedings of the 8th International Conference on Principles of Distributed Systems, pages 20--36, Grenoble, France, Dec. 2004.

Digital Library

[5]

F. Dabek, M. F. Kaashoek, D. Karger, R. Morris, and I. Stoica. Wide-area cooperative storage with CFS. In Proceedings of the 18th ACM Symposium on Operating Systems Principles, pages 202--215, Chateau Lake Louise, Banff, Canada, Oct. 2001.

Digital Library

[6]

E. Eide, L. Stoller, and J. Lepreau. An experimentation workbench for replayable networking research. In Proceedings of 4th USENIX Symposium on Networked Systems Design and Implementation, pages 215--228, Cambridge, MA, Apr. 2007.

Digital Library

[7]

M.-C. Hsueh, T. K. Tsai, and R. K. Iyer. Fault injection techniques and tools. IEEE Computer, 30(4):75--82, 1997.

Digital Library

[8]

E. Hyytia and J. Virtamo. Random waypoint mobility model in cellular networks, 2005.

[9]

D. Johnson, T. Stack, R. Fish, D. M. Flickinger, L. Stoller, R. Ricci, and J. Lepreau. Mobile Emulab: A robotic wireless and sensor network testbed. In Proceedings of the 25th Conference on Computer Communications, Barcelona, Catalunya, Spain, Apr. 2006.

[10]

T. Miyachi, K. ichi Chinen, and Y. Shinoda. Automatic configuration and execution of Internet experiments on an actual node-based testbed. In Proceedings of the First International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, pages 274--282, Trento, Italy, Feb. 2005.

Digital Library

[11]

D. Oppenheimer, V. Vatkovskiy, and D. A. Patterson. Towards a framework for automated robustness evaluation of distributed services. In Proceedings of the 2nd Bertinoto Workshop on Future Directions in Distributed Computing: Survivability: Obstacles and Solutions, Bertinoro, Italy, June 2004.

[12]

S. PalChaudhuri, J.-Y. L. Boudec, and M. Vojnovic. Perfect simulations for random trip mobility models. In Proceedings of the 38th Annual Simulation Symposium, San Diego, CA, Apr. 2005.

Digital Library

[13]

L. Peterson, T. Anderson, D. Culler, and T. Roscoe. A blueprint for introducing disruptive technology into the Internet. ACM SIGCOMM Computer Communication Review, 33(1):59--64, 2003.

Digital Library

[14]

S. Sivasubramanian, G. Pierre, M. van Steen, and G. Alonso. Analysis of caching and replication strategies for web applications. IEEE Internet Computing, 11(1):60--66, Jan. 2007.

Digital Library

[15]

N. Spring, L. Peterson, A. Bavier, and V. Pai. Using PlanetLab for network research: Myths, realities, and best practices. SIGOPS Operating Systems Review, 40(1):17--24, Jan. 2006.

Digital Library

[16]

I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for Internet applications. In Proceedings of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pages 149--160, San Diego, CA, Aug. 2001.

Digital Library

[17]

I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for Internet applications. In Proceedings of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pages 149--160, San Diego, CA, Aug. 2001.

Digital Library

[18]

I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for Internet applications. In Proceedings of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pages 149--160, San Diego, CA, Aug. 2001.

Digital Library

[19]

B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar. An integrated experimental environment for distributed systems and networks. In Proc. of the Fifth Symposium on Operating Systems Design and Implementation, pages 255--270, Boston, MA, Dec. 2002. USENIX Association.

Digital Library

Cited By

de Resende HSchimuneck MBoth CWickboldt JMarquez-Barja J(2020)COPA: Experimenter-Level Container Orchestration for Networking TestbedsIEEE Access10.1109/ACCESS.2020.30356198(201781-201798)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3035619
(2017)Improving the performance and reproducibility of experiments on large-scale testbeds with k-coresComputer Communications10.1016/j.comcom.2017.05.016110:C(35-47)Online publication date: 15-Sep-2017
https://dl.acm.org/doi/10.1016/j.comcom.2017.05.016
Padurariu MRosca BIvanciu ILuchian ERus ADobrota V(2016)Automatic control of an OpenFlow-based network using Lua scripts and SPLAY2016 International Conference on Communications (COMM)10.1109/ICComm.2016.7528286(299-302)Online publication date: Jun-2016
https://doi.org/10.1109/ICComm.2016.7528286
Show More Cited By

Index Terms

Four enhancements to automateddistributed system experimentation methods
1. Software and its engineering
  1. Software creation and management
    1. Designing software
      1. Software implementation planning
        Software design techniques
    2. Software development process management

Recommendations

Automating experimentation on distributed testbeds
ASE '05: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering

Engineering distributed systems is a challenging activity. This is partly due to the intrinsic complexity of distributed systems, and partly due to the practical obstacles that developers face when evaluating and tuning their design and implementation ...
Experimentation in the operating system: the windows experimentation platform
ICSE-SEIP '19: Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice

Online controlled experiments are the gold standard for evaluating improvements and accelerating innovations in online and app worlds. However, little is known about applicability, implementation, and efficacy of experimentation for operating systems (...
A wired-wireless testbed architecture for network layer experimentation based on ORBIT and VINI
WinTECH '07: Proceedings of the second ACM international workshop on Wireless network testbeds, experimental evaluation and characterization

Renewed interest has emerged in novel internet architectures that among other aspects can better address the specific requirements of wireless networks, such as increased host mobility, router mobility, or flow control over fast-changing links. Most ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '08: Proceedings of the 30th international conference on Software engineering

May 2008

558 pages

ISBN:9781605580791

DOI:10.1145/1368088

General Chair:
Wilhelm Schäfer
University of Paderborn
,
Program Chairs:
Matthew B. Dwyer
University of Nebraska
,
Volker Gruhn
University of Leipzig

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 May 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '08

Sponsor:

ICSE '08: International Conference on Software Engineering

May 10 - 18, 2008

Leipzig, Germany

Acceptance Rates

ICSE '08 Paper Acceptance Rate 56 of 370 submissions, 15%;

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

de Resende HSchimuneck MBoth CWickboldt JMarquez-Barja J(2020)COPA: Experimenter-Level Container Orchestration for Networking TestbedsIEEE Access10.1109/ACCESS.2020.30356198(201781-201798)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3035619
(2017)Improving the performance and reproducibility of experiments on large-scale testbeds with k-coresComputer Communications10.1016/j.comcom.2017.05.016110:C(35-47)Online publication date: 15-Sep-2017
https://dl.acm.org/doi/10.1016/j.comcom.2017.05.016
Padurariu MRosca BIvanciu ILuchian ERus ADobrota V(2016)Automatic control of an OpenFlow-based network using Lua scripts and SPLAY2016 International Conference on Communications (COMM)10.1109/ICComm.2016.7528286(299-302)Online publication date: Jun-2016
https://doi.org/10.1109/ICComm.2016.7528286
Buchert TRuiz CNussbaum LRichard O(2015)A survey of general-purpose experiment management tools for distributed systemsFuture Generation Computer Systems10.1016/j.future.2014.10.00745:C(1-12)Online publication date: 1-Apr-2015
https://dl.acm.org/doi/10.1016/j.future.2014.10.007
Basu AFleming SStanier JNaicken SWakeman IGurbani V(2013)The state of peer-to-peer network simulatorsACM Computing Surveys10.1145/2501654.250166045:4(1-25)Online publication date: 30-Aug-2013
https://dl.acm.org/doi/10.1145/2501654.2501660
Jayasinghe DKimball JChoudhary SZhu TPu C(2013)An automated approach to create, store, and analyze large-scale experimental data in clouds2013 IEEE 14th International Conference on Information Reuse & Integration (IRI)10.1109/IRI.2013.6642493(357-364)Online publication date: Aug-2013
https://doi.org/10.1109/IRI.2013.6642493
Jayasinghe DKimball JZhu TChoudhary SCalton P(2013)An infrastructure for automating large-scale performance studies and data processing2013 IEEE International Conference on Big Data10.1109/BigData.2013.6691573(187-192)Online publication date: Oct-2013
https://doi.org/10.1109/BigData.2013.6691573
Cadar CPietzuch PWolf ARoman GSullivan K(2010)Multiplicity computingProceedings of the FSE/SDP workshop on Future of software engineering research10.1145/1882362.1882380(81-86)Online publication date: 7-Nov-2010
https://dl.acm.org/doi/10.1145/1882362.1882380
Leonini LRivière ÉFelber PRexford JSirer E(2009)SPLAYProceedings of the 6th USENIX symposium on Networked systems design and implementation10.5555/1558977.1558990(185-198)Online publication date: 22-Apr-2009
https://dl.acm.org/doi/10.5555/1558977.1558990

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents