Article

Issues in developing a thread-safe MPI implementation

Authors:

EuroPVM/MPI'06: Proceedings of the 13th European PVM/MPI User's Group conference on Recent advances in parallel virtual machine and message passing interface

Pages 12 - 21

https://doi.org/10.1007/11846802_11

Published: 17 September 2006 Publication History

Publisher Site

Abstract

The MPI-2 Standard has carefully specified the interaction between MPI and user-created threads, with the goal of enabling users to write multithreaded programs while also enabling MPI implementations to deliver high performance. In this paper, we describe and analyze what the MPI Standard says about thread safety and what it implies for an implementation. We classify the MPI functions based on their thread-safety requirements and discuss several issues to consider when implementing thread safety in MPI. We use the example of generating new context ids (required for creating new communicators) to demonstrate how a simple solution for the single-threaded case cannot be used when there are multiple threads and how a naïve thread-safe algorithm can be expensive. We then present an algorithm for generating context ids that works efficiently in both single-threaded and multithreaded cases.

References

[1]

Analysis of thread safety needs of MPI routines. http://www.mcs.anl.gov/mpi/mpich2/developer/design/threadlist.htm.

Google Scholar

[2]

Erik D. Demaine. A threads-only MPI implementation for the development of parallel programs. In Proc. of the 11th International Symposium on High Performance Computing Systems, pages 153-163, July 1997.

Google Scholar

[3]

Francisco García, Alejandro Calderón, and Jesús Carretero. MiMPI: A multithread-safe implementation of MPI. In Proc. of 6th European PVM/MPI Users' Group Meeting, pages 207-214, September 1999.

Crossref

Google Scholar

[4]

Edward A. Lee. The problem with threads. Computer, 39(5):33-42, May 2006.

Crossref

Google Scholar

[5]

Message Passing Interface Forum. MPI-2: Extensions to the Message-Passing Interface, July 1997. http://www.mpi-forum.org/docs/docs.html.

Google Scholar

[6]

MPICH2. http://www.mcs.anl.gov/mpi/mpich2.

Google Scholar

[7]

Tomas Plachetka. (Quasi-) thread-safe PVM and (quasi-) thread-safe MPI without active polling. In Proc. of 9th European PVM/MPI Users' Group Meeting, pages 296-305, September 2002.

Crossref

Google Scholar

[8]

Boris V. Protopopov and Anthony Skjellum. A multithreaded message passing interface (MPI) architecture: Performance and program issues. Journal of Parallel and Distributed Computing, 61(4):449-466, April 2001.

Crossref

Google Scholar

[9]

Anthony Skjellum, Boris Protopopov, and Shane Hebert. A thread taxonomy for MPI. In Proc. of the 2nd MPI Developers Conference, pages 50-57, June 1996.

Crossref

Google Scholar

[10]

Hong Tang and Tao Yang. Optimizing threaded MPI execution on SMP clusters. In Proc. of the 15th ACM International Conference on Supercomputing, pages 381- 392, June 2001.

Crossref

Google Scholar

Cited By

View all

Ross CCarothers CMubarak MRoss RLi JMa KJohansson BJain S(2018)Leveraging shared memory in the ross time warp simulator for complex network simulationsProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320974(3837-3848)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320974
Dang HSnir MGropp WDongarra JHolmes DCollis ALarsson Träff JSmith L(2016)Towards millions of communicating threadsProceedings of the 23rd European MPI Users' Group Meeting10.1145/2966884.2966914(1-14)Online publication date: 25-Sep-2016
https://dl.acm.org/doi/10.1145/2966884.2966914
Potluri SRossetti DBecker DPoole DGorentla Venkata MHernandez OShamis PLopez MBaker MPoole W(2015)Exploring OpenSHMEM Model to Program GPU-based Extreme-Scale SystemsRevised Selected Papers of the Second Workshop on OpenSHMEM and Related Technologies. Experiences, Implementations, and Technologies - Volume 939710.1007/978-3-319-26428-8_2(18-35)Online publication date: 4-Aug-2015
https://dl.acm.org/doi/10.1007/978-3-319-26428-8_2
Show More Cited By

Recommendations

Thread-safety in an MPI implementation: Requirements and analysis

The MPI-2 Standard has carefully specified the interaction between MPI and user-created threads. The goal of this specification is to allow users to write multithreaded MPI programs while also allowing MPI implementations to deliver high performance. ...
A Thread Taxonomy for MPI
MPIDC '96: Proceedings of the Second MPI Developers Conference

Abstract: In 1994, we presented extensions to MPI and offered an early paper on potential thread extensions to MPI as well as non-blocking collective extensions to MPI. The present paper is a thorough review of thread issues in MPI, including ...
(Quasi-) Thread-Safe PVM and (Quasi-) Thread-Safe MPI without Active Polling
Proceedings of the 9th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface

PVM (the current version 3.4) as well as many current MPI implementations force application programmers to use active polling (also known as busy waiting) in larger parallel programs. This serious problem is related to thread-unsafety of these ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

EuroPVM/MPI'06: Proceedings of the 13th European PVM/MPI User's Group conference on Recent advances in parallel virtual machine and message passing interface

September 2006

411 pages

ISBN:354039110X

Editors:
Bernd Mohr
Forschungszentrum Jülich GmbH, Zentralinstitut für Angewandte Mathematik, Jülich, Germany
,
Jesper Larsson Träff
C&C Research Laboratories NEC Europe Ltd., Sankt Augustin, Germany
,
Joachim Worringen
Dolphin Interconnect Solutions ASA, R&D Germany, Wachtberg, Germany
,
Jack Dongarra
University of Tennessee, Computer Science Department, Knoxville, TN

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 17 September 2006

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ross CCarothers CMubarak MRoss RLi JMa KJohansson BJain S(2018)Leveraging shared memory in the ross time warp simulator for complex network simulationsProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320974(3837-3848)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320974
Dang HSnir MGropp WDongarra JHolmes DCollis ALarsson Träff JSmith L(2016)Towards millions of communicating threadsProceedings of the 23rd European MPI Users' Group Meeting10.1145/2966884.2966914(1-14)Online publication date: 25-Sep-2016
https://dl.acm.org/doi/10.1145/2966884.2966914
Potluri SRossetti DBecker DPoole DGorentla Venkata MHernandez OShamis PLopez MBaker MPoole W(2015)Exploring OpenSHMEM Model to Program GPU-based Extreme-Scale SystemsRevised Selected Papers of the Second Workshop on OpenSHMEM and Related Technologies. Experiences, Implementations, and Technologies - Volume 939710.1007/978-3-319-26428-8_2(18-35)Online publication date: 4-Aug-2015
https://dl.acm.org/doi/10.1007/978-3-319-26428-8_2
Hoefler TBronevetsky GBarrett BDe Supinski BLumsdaine A(2010)Efficient MPI support for advanced hybrid programming modelsProceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface10.5555/1894122.1894129(50-61)Online publication date: 12-Sep-2010
https://dl.acm.org/doi/10.5555/1894122.1894129
Kamal HMirtaheri SWagner AHariri SKeahey K(2010)Scalability of communicators and groups in MPIProceedings of the 19th ACM International Symposium on High Performance Distributed Computing10.1145/1851476.1851507(264-275)Online publication date: 21-Jun-2010
https://dl.acm.org/doi/10.1145/1851476.1851507
Carribault PPérache MJourdren H(2010)Enabling low-overhead hybrid MPI/OpenMP parallelism with MPCProceedings of the 6th international conference on Beyond Loop Level Parallelism in OpenMP: accelerators, Tasking and more10.1007/978-3-642-13217-9_1(1-14)Online publication date: 14-Jun-2010
https://dl.acm.org/doi/10.1007/978-3-642-13217-9_1
Patrick CSon SKandemir M(2008)Comparative evaluation of overlap strategies with study of I/O overlap in MPI-IOACM SIGOPS Operating Systems Review10.1145/1453775.145378442:6(43-49)Online publication date: 1-Oct-2008
https://dl.acm.org/doi/10.1145/1453775.1453784
Hoefler TKambadur PGraham RShipman GLumsdaine A(2007)A case for standard non-blocking collective operationsProceedings of the 14th European conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface10.5555/2396095.2396122(125-134)Online publication date: 30-Sep-2007
https://dl.acm.org/doi/10.5555/2396095.2396122
Thakur RGropp W(2007)Test suite for evaluating performance of MPI implementations that support MPI_THREAD_MULTIPLEProceedings of the 14th European conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface10.5555/2396095.2396111(46-55)Online publication date: 30-Sep-2007
https://dl.acm.org/doi/10.5555/2396095.2396111
Thakur RGropp W(2007)Open issues in MPI implementationProceedings of the 12th Asia-Pacific conference on Advances in Computer Systems Architecture10.5555/2392163.2392194(327-338)Online publication date: 23-Aug-2007
https://dl.acm.org/doi/10.5555/2392163.2392194
Show More Cited By

Abstract

References

Cited By

Recommendations

Thread-safety in an MPI implementation: Requirements and analysis

A Thread Taxonomy for MPI

(Quasi-) Thread-Safe PVM and (Quasi-) Thread-Safe MPI without Active Polling

Comments

Information

Published In

Sponsors

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations