More Web Proxy on the site http://driver.im/

research-article

pNFS, POSIX, and MPI-IO: a tale of three semantics

Authors:

Dean Hildebrand,

Roger HaskinAuthors Info & Claims

PDSW '09: Proceedings of the 4th Annual Workshop on Petascale Data Storage

Pages 32 - 36

https://doi.org/10.1145/1713072.1713082

Published: 14 November 2009 Publication History

Abstract

MPI-IO is emerging as the standard mechanism for file I/O within HPC applications. While pNFS demonstrates high-performance I/O for bulk data transfers, its performance and scalability with MPI-IO is unproven. To attain success, the consistency semantics and interfaces of pNFS, POSIX, and MPI-IO must all be reconciled and efficiently translated. This paper investigates and discusses the challenges of using pNFS to support the consistency semantics of HPC applications.

References

[1]

W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster, "The Globus Striped GridFTP Framework and Server," in Proceedings of Supercomputing '05, Seattle, WA, 2005.

Digital Library

[2]

B. Welch, M. Unangst, Z. Abbasi, G. Gibson, B. Mueller, J. Small, J. Zelenka, and B. Zhou, "Scalable Performance of the Panasas Parallel File System," in Proceedings of the 6th USENIX Conference on File and Storage Technologies, San Jose, CA, 2008.

Digital Library

[3]

F. Schmuck and R. Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters," in Proceedings of the USENIX Conference on File and Storage Technologies, San Francisco, CA, 2002.

Digital Library

[4]

Cluster File Systems Inc., "Lustre: A Scalable, High-Performance File System," www.lustre.org, 2002.

[5]

W. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir, MPI: The Complete Reference, volume 2--The MPI-2 Extensions. Cambridge, MA: MIT Press, 1998.

Digital Library

[6]

NCSA, "HDF5 ", hdf.ncsa.uiuc.edu/HDF5.

[7]

J. Li, W. Liao, A. Choudhary, R. Ross, R. Thakur, W. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale, "Parallel netCDF: A Scientific High-Performance I/O Interface," in Proceedings of Supercomputing '03, Phoenix, AZ, 2003.

Digital Library

[8]

R. Thakur, W. Gropp, and E. Lusk, "Data Sieving and Collective I/O in ROMIO," in Proceedings of the 7th Symposium on the Frontiers of Massively Parallel Computation, 1999.

Digital Library

[9]

S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M. Eisler, and D. Noveck, "NFS Version 4 Protocol Specification," RFC 3530, 2003.

[10]

S. Shepler, M. Eisler, and D. Noveck, "NFSv4 Minor Version 1," Internet Draft, 2008.

[11]

D. Hildebrand, P. Andrews, M. Eshel, R. Haskin, P. Kovatch, and J. White, "Deploying pNFS across the WAN: First Steps in HPC Grid Computing," in Proceedings of the 9th LCI International Conference on High-Performance Clustered Computing, Urbana, IL, 2008.

[12]

D. Hildebrand and P. Honeyman, "Exporting Storage Systems in a Scalable Manner with pNFS," in Proceedings of the 22nd IEEE/13th NASA Goddard Conference on Mass Storage Systems and Technologies, Monterey, CA, 2005.

Digital Library

[13]

G. Grider, L. Ward, R. Ross, and G. Gibson, "A Business Case for Extensions to the POSIX I/O API for High End, Clustered, and Highly Concurrent Computing," www.opengroup.org/platform/hecewg, 2006.

[14]

"MPI-Forum," www.mpi-forum.org/docs/_mpi-20-html/node215.htm.

[15]

R. Thakur, W. Gropp, and E. Lusk, "An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces," in Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation, 1996.

Digital Library

[16]

R. Thakur, E. Lusk, and W. Gropp, "Users Guide for ROMIO: A High-Performance, Portable MPI-IO Implementation," Technical Memorandum ANL/MCS-TM-234, Mathematics and Computer Science Division, Argonne National Laboratory, Revised May 2004.

[17]

M. Oberg, H. M. Tufo, and M. Woitaszek, "Exploration of Parallel Storage Architectures for a Blue Gene/L on the TeraGrid," in Proceedings of the 9th LCI International Conference on High-Performance Clustered Computing, Urbana, IL, 2008.

[18]

D. Hildebrand, L. Ward, and P. Honeyman, "Large Files, Small Writes, and pNFS," in Proceedings of the 20th ACM International Conference on Supercomputing, Cairns, Australia, 2006.

Digital Library

[19]

Sun Microsystems Inc., "Lustre File System," White-paper, 2007.

[20]

J. P. Prost, R. Treumann, R. Hedges, B. Jia, and A. E. Koniges, "MPI-IO/GPFS, an Optimized Implementation of MPI-IO on top of GPFS," in Proceedings of Supercomputing '01, Denver, CO, 2001.

Digital Library

[21]

P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur, "PVFS: A Parallel File System for Linux Clusters," in Proceedings of the 4th Annual Linux Showcase and Conference, Atlanta, GA, 2000.

Digital Library

[22]

I. Chavis, D. Coutts, J. Huie, S. Liu, S. Qualters, B. Demkowicz, and D. L Turkenkopf, "A Guide to the IBM Clustered Network File System," IBM Redbooks, 2008.

[23]

IBM Corp., "IBM Storage Optimization and Integration Services-scale out file services," datasheet, 2007.

[24]

M. Eisler, P. Corbett, M. Kazar, D. Nydick, and C. Wagner, "Data ONTAP GX: A Scalable Storage Cluster," in Proceedings of the 5th USENIX conference on File and Storage Technologies, San Jose, CA, 2007.

Digital Library

Cited By

Dong BBez JByna SButt AMi NChard K(2023)AIIO: Using Artificial Intelligence for Job-Level and Automatic I/O Performance Bottleneck DiagnosisProceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3588195.3592986(155-167)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3588195.3592986
Dong BWu KByna SDong BWu KByna S(2021)IntroductionUser-Defined Tensor Data Analysis10.1007/978-3-030-70750-7_1(1-8)Online publication date: 22-Feb-2021
https://doi.org/10.1007/978-3-030-70750-7_1
Dong BByna SWu KPrabhat Johansen HJohnson JKeen N(2016)Data Elevator: Low-Contention Data Movement in Hierarchical Storage System2016 IEEE 23rd International Conference on High Performance Computing (HiPC)10.1109/HiPC.2016.026(152-161)Online publication date: Dec-2016
https://doi.org/10.1109/HiPC.2016.026
Show More Cited By

Index Terms

pNFS, POSIX, and MPI-IO: a tale of three semantics
1. General and reference
  1. Cross-computing tools and techniques
    1. Performance
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Communications management
        Input / output
        File systems management
    2. Extra-functional properties
      1. Software performance

Recommendations

Large files, small writes, and pNFS
ICS '06: Proceedings of the 20th annual international conference on Supercomputing

Workload characterization studies highlight the prevalence of small and sequential data requests in scientific applications. Parallel file systems excel at large data transfers but sometimes at the expense of small I/O performance. pNFS is an NFSv4.1 ...
pNFS/PVFS2 over InfiniBand: early experiences
PDSW '07: Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07

The computing power of clusters has been rapidly growing up towards petascale capability, which requires petascale I/O systems to provide data in a sustained high-throughput manner. Network File System (NFS), a ubiquitous standard used in most existing ...
Direct-pNFS: scalable, transparent, and versatile access to parallel file systems
HPDC '07: Proceedings of the 16th international symposium on High performance distributed computing

Grid computations require global access to massive data stores. To meet this need, the GridNFS project aims to provide scalable, high-performance, transparent, and secure wide-area data management as well as a scalable and agile name space.

While ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

PDSW '09: Proceedings of the 4th Annual Workshop on Petascale Data Storage

November 2009

58 pages

ISBN:9781605588834

DOI:10.1145/1713072

Conference Chair:
Garth A. Gibson
Carnegie Mellon University and Panasas Inc.

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SC '09

Sponsor:

SIGARCH

SC '09: International Conference for High Performance Computing, Networking, Storage and Analysis

November 14, 2009

Oregon, Portland

Acceptance Rates

Overall Acceptance Rate 17 of 41 submissions, 41%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
271
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dong BBez JByna SButt AMi NChard K(2023)AIIO: Using Artificial Intelligence for Job-Level and Automatic I/O Performance Bottleneck DiagnosisProceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3588195.3592986(155-167)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3588195.3592986
Dong BWu KByna SDong BWu KByna S(2021)IntroductionUser-Defined Tensor Data Analysis10.1007/978-3-030-70750-7_1(1-8)Online publication date: 22-Feb-2021
https://doi.org/10.1007/978-3-030-70750-7_1
Dong BByna SWu KPrabhat Johansen HJohnson JKeen N(2016)Data Elevator: Low-Contention Data Movement in Hierarchical Storage System2016 IEEE 23rd International Conference on High Performance Computing (HiPC)10.1109/HiPC.2016.026(152-161)Online publication date: Dec-2016
https://doi.org/10.1109/HiPC.2016.026
Mickens JNightingale EElson JNareddy KGehring DFan BKadav AChidambaram VKhan OMahajan RStoica I(2014)BlizzardProceedings of the 11th USENIX Conference on Networked Systems Design and Implementation10.5555/2616448.2616473(257-273)Online publication date: 2-Apr-2014
https://dl.acm.org/doi/10.5555/2616448.2616473
Dong BByna SWu K(2014)Parallel query evaluation as a Scientific Data Service2014 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER.2014.6968765(194-202)Online publication date: Sep-2014
https://doi.org/10.1109/CLUSTER.2014.6968765

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents