[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Effective Nonblocking MPI-I/O in Remote I/O Operations Using a Multithreaded Mechanism

  • Conference paper
Parallel and Distributed Processing and Applications (ISPA 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3358))

Abstract

A flexible intermediate library named Stampi realizes seamless MPI operations on interconnected parallel computers. Dynamic process creation and MPI-I/O operations both inside a computer and among computers are available with it. MPI-I/O operations to a remote computer are realized by MPI-I/O processes of the Stampi library which are invoked on a remote computer using a vendor-supplied MPI-I/O library. If the vendor-supplied one is not available, a single MPI-I/O process is invoked on a remote computer, and it uses UNIX I/O functions instead of the vendor-supplied one. In nonblocking MPI-I/O functions with multiple user processes, the single MPI-I/O process carries out I/O operations required by the processes sequentially. This results in small overlap of computation by the user processes with I/O operations by the MPI-I/O process. Therefore performance of the nonblocking functions is poor with multiple user processes. To realize effective I/O operations, a Pthreads library has been implemented in the MPI-I/O mechanism, and multithreaded I/O operations have been realized. The newly implemented MPI-I/O mechanism has been evaluated on inter-connected PC clusters, and higher overlap of the computation with the I/O operations has been achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 71.50
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 89.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Message Passing Interface Forum: MPI: A message-passing interface standard (1995)

    Google Scholar 

  2. Message Passing Interface Forum: MPI-2: Extensions to the message-passing interface standard (1997)

    Google Scholar 

  3. Imamura, T., Tsujita, Y., Koide, H., Takemiya, H.: An architecture of stampi: MPI library on a cluster of parallel computers. In: Dongarra, J., Kacsuk, P., Podhorszki, N. (eds.) PVM/MPI 2000. LNCS, vol. 1908, pp. 200–207. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  4. Tsujita, Y., Imamura, T., Takemiya, H., Yamagishi, N.: Stampi-I/O: A flexible parallel-I/O library for heterogeneous computing environment. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J., Volkert, J. (eds.) PVM/MPI 2002. LNCS, vol. 2474, pp. 288–295. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  5. Tsujita, Y.: Flexible intermediate library for MPI-2 support on an sCore cluster system. In: Li, M., et al. (eds.) GCC 2003. LNCS, vol. 3033, pp. 129–136. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  6. Institute of Electrical, Electronic Engineers: Information Technology – Portable Operating Systems Interface – Part 1: System Application Program Interface (API) – Amendment 2: Threads Extensions [C Languages] (1995)

    Google Scholar 

  7. LinuxThreads: http://pauillac.inria.fr/~xleroy/linuxthreads/

  8. PC Cluster Consortium: http://www.pccluster.org/

  9. Matsuda, M., Kudoh, T., Ishikawa, Y.: Evaluation of MPI implementations on grid-connected clusters using an emulated WAN environment. In: Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2003), Tokyo, Japan, pp. 10–17. IEEE Computer Society, Los Alamitos (2003)

    Chapter  Google Scholar 

  10. Gropp, W., Lusk, E., Doss, N., Skjellum, A.: A high-performance, portable implementation of the MPI Message-Passing Interface standard. Parallel Computing 22, 789–828 (1996)

    Article  MATH  Google Scholar 

  11. Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pp. 23–32 (1999)

    Google Scholar 

  12. Dickens, P., Thakur, R.: Improving collective I/O performance using threads. In: Proceedings of the Joint International Parallel Processing Symposium and IEEE Symposium on Parallel and Distributed Processing, pp. 38–45 (1999)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Tsujita, Y. (2004). Effective Nonblocking MPI-I/O in Remote I/O Operations Using a Multithreaded Mechanism. In: Cao, J., Yang, L.T., Guo, M., Lau, F. (eds) Parallel and Distributed Processing and Applications. ISPA 2004. Lecture Notes in Computer Science, vol 3358. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30566-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30566-8_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-24128-7

  • Online ISBN: 978-3-540-30566-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics