More Web Proxy on the site http://driver.im/

Article

Flexible, wide-area storage for distributed systems with WheelFS

Authors:

Jeremy Stribling,

M. Frans Kaashoek,

Robert MorrisAuthors Info & Claims

NSDI'09: Proceedings of the 6th USENIX symposium on Networked systems design and implementation

Pages 43 - 58

Published: 22 April 2009 Publication History

Abstract

WheelFS is a wide-area distributed storage system intended to help multi-site applications share data and gain fault tolerance. WheelFS takes the form of a distributed file system with a familiar POSIX interface. Its design allows applications to adjust the tradeoff between prompt visibility of updates from other sites and the ability for sites to operate independently despite failures and long delays. WheelFS allows these adjustments via semantic cues, which provide application control over consistency, failure handling, and file and replica placement.

WheelFS is implemented as a user-level file system and is deployed on PlanetLab and Emulab. Three applications (a distributed Web cache, an email service and large file distribution) demonstrate that WheelFS's file system interface simplifies construction of distributed applications by allowing reuse of existing software. These applications would perform poorly with the strict semantics implied by a traditional file system interface, but by providing cues to WheelFS they are able to achieve good performance. Measurements show that applications built on WheelFS deliver comparable performance to services such as CoralCDN and BitTorrent that use specialized wide-area storage systems.

References

[1]

Amazon Simple Storage System. http://aws.amazon. com/s3/.

[2]

ADYA, A., BOLOSKY, W. J., CASTRO, M., CERMAK, G., CHAIKEN, R., DOUCEUR, J. R., HOWELL, J., LORCH, J. R., THEIMER, M., AND WATTENHOFER, R. P. FARSITE: Federated, available, and reliable storage for an incompletely trusted environment. In Proceedings of the 5th OSDI (Dec. 2002).

Digital Library

[3]

AGUILERA, M. K., MERCHANT, A., SHAH, M., VEITCH, A., AND KARAMANOLIS, C. Sinfonia: A new paradigm for building scalable distributed systems. In Proceedings of the 21st SOSP (Oct. 2007).

Digital Library

[4]

ALLCOCK, W., BRESNAHAN, J., KETTIMUTHU, R., LINK, M., DUMITRESCU, C., RAICU, I., AND FOSTER, I. The Globus striped GridFTP framework and server. In Proceedings of the 2005 Super Computing (Nov. 2005).

Digital Library

[5]

ANDERSON, T. E., DAHLIN, M. D., NEEFE, J. M., PATTERSON, D. A., ROSELLI, D. S., AND WANG, R. Y. Serverless network file systems. In Proceedings of the 15th SOSP (Dec. 1995).

Digital Library

[6]

ANNAPUREDDY, S., FREEDMAN, M. J., AND MAZIÈRES, D. Shark: Scaling file servers via cooperative caching. In Proceedings of the 2nd NSDI (May 2005).

Digital Library

[7]

BAVIER, A., BOWMAN, M., CHUN, B., CULLER, D., KARLIN, S., MUIR, S., PETERSON, L., ROSCOE, T., SPALINK, T., AND WAWRZONIAK, M. Operating systems support for planetary-scale network services. In Proceedings of the 1st NSDI (Mar. 2004).

Digital Library

[8]

BELARAMANI, N., DAHLIN, M., GAO, L., NAYATE, A., VENKATARAMANI, A., YALAGANDULA, P., AND ZHENG, J. PRACTI replication. In Proceedings of the 3rd NSDI (2006).

Digital Library

[9]

BELARAMANI, N., ZHENG, J., NAYATE, A., SOULÉ, R., DAHLIN, M., AND GRIMM, R. PADS: A policy architecture for building distributed storage systems. In Proceedings of the 6th NSDI (Apr. 2009).

Digital Library

[10]

BHAGWAN, R., TATI, K., CHENG, Y.-C., SAVAGE, S., AND VOELKER, G. M. Total Recall: System support for automated availability management. In Proceedings of the 1st NSDI (Mar. 2004).

Digital Library

[11]

BURROWS, M. The Chubby lock service for loosely-coupled distributed systems. In Proceedings of the 7th OSDI (Nov. 2006).

Digital Library

[12]

C. THEKKATH, T. MANN, E. L. Frangipani: A scalable distributed file system. In Proceedings of the 16th SOSP.

Digital Library

[13]

CHANG, F., DEAN, J., GHEMAWAT, S., HSIEH, W. C., WALLACH, D. A., BURROWS, M., CHANDRA, T., FIKES, A., AND GRUBER, R. E. Bigtable: A distributed storage system for structured data. In Proceedings of the 7th OSDI (Nov. 2006).

Digital Library

[14]

COHEN, B. Incentives build robustness in BitTorrent. In Proceedings of the Workshop on Economics of Peer-to-Peer Systems (June 2003).

[15]

DABEK, F., COX, R., KAASHOEK, F., AND MORRIS, R. A decentralized network coordinate system. In Proceedings of the 2004 SIGCOMM (2004).

Digital Library

[16]

DECANDIA, G., HASTORUN, D., JAMPANI, M., KAKULAPATI, G., LAKSHMAN, A., PILCHIN, A., SIVASUBRAMANIAN, S., VOSSHALL, P., AND VOGELS, W. Dynamo: Amazon's highly available key-value store. In Proceedings of the 21st SOSP (Oct. 2007).

Digital Library

[17]

Dovecot IMAP server. http://www.dovecot.org/.

[18]

FEKETE, A., GUPTA, D., LUCHANGCO, V., LYNCH, N., AND SCHVARTSMAN, A. Eventually-serializable data services. Theoretical Computer Science (June 1999).

Digital Library

[19]

FREEDMAN, M. J., FREUDENTHAL, E., AND MAZIÈRES, D. Democratizing content publication with Coral. In Proceedings of the 1st NSDI (Mar. 2004).

Digital Library

[20]

FREEDMAN, M. J., LAKSHMINARAYANAN, K., AND MAZIÈRES, D. OASIS: Anycast for any service. In Proceedings of the 3rd NSDI (May 2006).

Digital Library

[21]

Filesystem in user space. http://fuse.sourceforge. net/.

[22]

GHEMAWAT, S., GOBIOFF, H., AND LEUNG, S.-T. The Google file system. In Proceedings of the 19th SOSP (Dec. 2003).

Digital Library

[23]

GILBERT, S., AND LYNCH, N. Brewer's conjecture and the feasibility feasibility of consistent, available, partition tolerant web services. In ACM SIGACT News (June 2002), vol. 33.

Digital Library

[24]

KOHLER, E., MORRIS, R., CHEN, B., JANNOTTI, J., AND KAASHOEK, M. F. The Click modular router. ACM Trans. on Computer Systems (Aug. 2000).

Digital Library

[25]

LAMPORT, L. The part-time parliament. ACM Transactions on Computer Systems 16, 2 (1998), 133-169.

Digital Library

[26]

LI, J., KROHN, M., MAZIÈRES, D., AND SHASHA, D. Secure Untrusted data Repository (SUNDR). In Proceedings of the 6th OSDI (Dec. 2004).

Digital Library

[27]

MUTHITACHAROEN, A., MORRIS, R., GIL, T., AND CHEN, B. Ivy: A read/write peer-to-peer file system. In Proceedings of the 5th OSDI (2002).

Digital Library

[28]

NIGHTINGALE, E. B., AND FLINN, J. Energy-efficiency and storage flexibility in the Blue File System. In Proceedings of the 6th OSDI (Dec. 2004).

Digital Library

[29]

PARK, K., AND PAI, V. S. Scale and performance in the CoBlitz large-file distribution service. In Proceedings of the 3rd NSDI (May 2006).

Digital Library

[30]

RHEA, S., EATON, P., GEELS, D., WEATHERSPOON, H., ZHAO, B., AND KUBIATOWICZ, J. Pond: The OceanStore prototype. In Proceedings of the 2nd FAST (Mar. 2003).

Digital Library

[31]

SAITO, Y., BERSHAD, B., AND LEVY, H. Manageability, availability and performance in Porcupine: A highly scalable internet mail service. ACM Transactions of Computer Systems (2000).

Digital Library

[32]

SAITO, Y., KARAMONOLIS, C., KARLSSON, M., AND MAHALINGAM, M. Taming aggressive replication in the Pangaea wide-area file system. In Proceedings of the 5th OSDI (2002).

Digital Library

[33]

SANDBERG, R., GOLDBERG, D., KLEIMAN, S., WALSH, D., AND LYON, B. Design and implementation of the Sun Network Filesystem. In Proceedings of the Summer 1985 USENIX (June 1985).

[34]

SATYANARAYANAN, M., HOWARD, J., NICHOLS, D., SIDEBOTHAM, R., SPECTOR, A., AND WEST, M. The ITC distributed file system: Principles and design. In Proceedings of the 10th SOSP (1985).

Digital Library

[35]

SATYANARAYANAN, M., KISTLER, J., KUMAR, P., OKASAKI, M., SIEGEL, E., AND STEERE, D. Coda: A highly available file system for a distributed workstation environment. IEEE Trans. on Comp. 4, 39 (Apr 1990), 447-459.

Digital Library

[36]

SIT, E., MORRIS, R., AND KAASHOEK, M. F. UsenetDHT: A low-overhead design for Usenet. In Usenix NSDI (2008).

Digital Library

[37]

STRIBLING, J. PlanetLab All-Pairs-Pings. http://pdos. csail.mit.edu/~strib/pl_app/.

[38]

STRIBLING, J., SIT, E., KAASHOEK, M. F., LI, J., AND MORRIS, R. Don't give up on distributed file systems. In Proceedings of the 6th IPTPS (2007).

[39]

TERRY, D., THEIMER, M., PETERSEN, K., DEMERS, A., SPREITZER, M., AND HAUSER, C. Managing update conflicts in Bayou, a weakly connected replicated storage system. In Proceedings of the 15th SOSP (1995).

Digital Library

[40]

VON BEHREN, J. R., CZERWINSKI, S., JOSEPH, A. D., BREWER, E. A., AND KUBIATOWICZ, J. Ninjamail: the design of a high-performance clustered, distributed e-mail system. In Proceedings of the ICPP '00 (2000).

Digital Library

[41]

WEIL, S. A., BRANDT, S. A., MILLER, E. L., LONG, D. D. E., AND MALTZAHN, C. Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7th OSDI (Nov. 2006).

Digital Library

[42]

WHITE, B., LEPREAU, J., STOLLER, L., RICCI, R., GURUPRASAD, S., NEWBOLD, M., HIBLER, M., BARB, C., AND JOGLEKAR, A. An integrated experimental environment for distributed systems and networks. In Proceedings of the 5th OSDI (Dec. 2002).

Digital Library

[43]

YU, H., AND VAHDAT, A. Design and evaluation of a conit-based continuous consistency model for replicated services. ACM TOCS 20, 3 (Aug. 2002), 239-282.

Digital Library

Cited By

Macko PHennessey J(2022)Survey of Distributed File System Design ChoicesACM Transactions on Storage10.1145/346540518:1(1-34)Online publication date: 2-Mar-2022
https://dl.acm.org/doi/10.1145/3465405
Zhang IRaybuck APatel POlynyk KNelson JLeija OMartinez ALiu JSimpson AJayakar SPenna PDemoulin MChoudhury PBadam A(2021)The Demikernel Datapath OS Architecture for Microsecond-scale Datacenter SystemsProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles10.1145/3477132.3483569(195-211)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3477132.3483569
Frate MMarczuk MVerdi F(2019)OrchFlowJournal of Network and Systems Management10.1007/s10922-018-9476-x27:3(551-572)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s10922-018-9476-x
Show More Cited By

Index Terms

Flexible, wide-area storage for distributed systems with WheelFS
1. General and reference
  1. Cross-computing tools and techniques
    1. Performance
2. Software and its engineering
  1. Software organization and properties

Recommendations

Building distributed, wide-area applications with wheelFS
GPC'08: Proceedings of the 3rd international conference on Advances in grid and pervasive computing

It is a challenge to build applications that need to share data and are distributed across hundreds or thousands of computers in a wide-area network (e.g., PlanetLab or on a Grid). In order to cope with high latency, throughput bottlenecks, and ...
Storage management in large distributed object-based storage systems
Wide-area cooperative storage with CFS

The Cooperative File System (CFS) is a new peer-to-peer read-only storage system that provides provable guarantees for the efficiency, robustness, and load-balance of file storage and retrieval. CFS does this with a completely decentralized architecture ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

NSDI'09: Proceedings of the 6th USENIX symposium on Networked systems design and implementation

April 2009

480 pages

Program Chairs:
Jennifer Rexford
Princeton University
,
Emin Gün Sirer
Cornell University

Sponsors

USENIX Assoc: USENIX Assoc

Publisher

USENIX Association

United States

Publication History

Published: 22 April 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Macko PHennessey J(2022)Survey of Distributed File System Design ChoicesACM Transactions on Storage10.1145/346540518:1(1-34)Online publication date: 2-Mar-2022
https://dl.acm.org/doi/10.1145/3465405
Zhang IRaybuck APatel POlynyk KNelson JLeija OMartinez ALiu JSimpson AJayakar SPenna PDemoulin MChoudhury PBadam A(2021)The Demikernel Datapath OS Architecture for Microsecond-scale Datacenter SystemsProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles10.1145/3477132.3483569(195-211)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3477132.3483569
Frate MMarczuk MVerdi F(2019)OrchFlowJournal of Network and Systems Management10.1007/s10922-018-9476-x27:3(551-572)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s10922-018-9476-x
Zhang ILebeck NFonseca PHolt BCheng RNorberg AKrishnamurthy ALevy HKeeton KRoscoe T(2016)DiamondProceedings of the 12th USENIX conference on Operating Systems Design and Implementation10.5555/3026877.3026933(723-738)Online publication date: 2-Nov-2016
https://dl.acm.org/doi/10.5555/3026877.3026933
Blial OBen Mamoun MBenaini R(2016)An Overview on SDN Architectures with Multiple ControllersJournal of Computer Networks and Communications10.1155/2016/93965252016Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1155/2016/9396525
Holt BBornholt JZhang IPorts DOskin MCeze L(2016)Disciplined Inconsistency with Consistency TypesProceedings of the Seventh ACM Symposium on Cloud Computing10.1145/2987550.2987559(279-293)Online publication date: 5-Oct-2016
https://dl.acm.org/doi/10.1145/2987550.2987559
Liao J(2016)Server-side prefetching in distributed file systemsConcurrency and Computation: Practice & Experience10.1002/cpe.343228:2(294-310)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1002/cpe.3432
Tang HLiu FShen GJin YGuo CLea RGopalakrishnan STilevich EMurphy ABlackstock M(2015)UniDriveProceedings of the 16th Annual Middleware Conference10.1145/2814576.2814729(137-148)Online publication date: 24-Nov-2015
https://dl.acm.org/doi/10.1145/2814576.2814729
Marconett DYoo S(2015)FlowBrokerJournal of Network and Systems Management10.1007/s10922-014-9325-523:2(328-359)Online publication date: 1-Apr-2015
https://dl.acm.org/doi/10.1007/s10922-014-9325-5
Lin PBi JWang Y(2015)WEBridgeSecurity and Communication Networks10.1002/sec.10308:10(1926-1942)Online publication date: 10-Jul-2015
https://dl.acm.org/doi/10.1002/sec.1030
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten