More Web Proxy on the site http://driver.im/

research-article

Open access

Towards a Hybrid MPI Correctness Benchmark Suite

Authors:

Alexander Hück,

Jan-Patrick Lehr,

Joachim Protze,

Simon Schwitanski,

Christian BischofAuthors Info & Claims

EuroMPI/USA '22: Proceedings of the 29th European MPI Users' Group Meeting

Pages 46 - 56

https://doi.org/10.1145/3555819.3555853

Published: 14 September 2022 Publication History

All formats PDF

Abstract

High-performance computing codes often combine the Message-Passing Interface (MPI) with a shared-memory programming model, e.g., OpenMP, for efficient computations. These so-called hybrid models may issue MPI calls concurrently from different threads at the highest level of MPI thread support. The correct use of either MPI or OpenMP can be complex and error-prone. The hybrid model increases this complexity even further. While correctness analysis tools exist for both programming paradigms, for hybrid models, a new set of potential errors exist, whose detection requires combining knowledge of MPI and OpenMP primitives. Unfortunately, correctness tools do not fully support the hybrid model yet, and their current capabilities are also hard to assess. In previous work, to enable structured comparisons of correctness tools and improve their coverage, we proposed the MPI-CorrBench test suite for MPI. Likewise, others proposed the DataRaceBench test suite for OpenMP. However, for the particular error classes of the hybrid model, no such test suite exists. Hence, we propose a hybrid MPI-OpenMP test suite to (1) facilitate the correctness tool development in this area and, subsequently, (2) further encourage the use of the hybrid model at the highest level of MPI thread support. To that end, we discuss issues with this hybrid model and the knowledge correctness tools need to combine w.r.t. MPI and OpenMP to detect these. In our evaluation of two state-of-the-art correctness tools, we see that for most cases of concurrent and conflicting MPI operations, these tools can cope with the added complexity of OpenMP. However, more intricate errors, where user code interferes with MPI, e.g., a data race on a buffer, still evade tool analysis.

References

[1]

2022. MPI Forum Issue Tracker. https://github.com/mpi-forum/mpi-issues/issues/117. Accessed: 2022-08-15.

[2]

2022. ThreadSanitizer. https://clang.llvm.org/docs/ThreadSanitizer.html. Accessed: 2022-08-15.

[3]

Simone Atzeni, Ganesh Gopalakrishnan, Zvonimir Rakamaric, Dong H Ahn, Ignacio Laguna, Martin Schulz, Gregory L Lee, Joachim Protze, and Matthias S Müller. 2016. ARCHER: Effectively Spotting Data Races in Large OpenMP Applications. In 2016 IEEE international parallel and distributed processing symposium (IPDPS). IEEE, 53–62.

[4]

David R Butenhof. 1997. Programming with POSIX threads. Addison-Wesley Professional.

Digital Library

[5]

Tim Cramer, Felix Münchhalfen, Christian Terboven, Tobias Hilbrich, and Matthias S Müller. 2016. Extending MUST to Check Hybrid-Parallel Programs for Correctness Using the OpenMP Tools Interface. In Tools for High Performance Computing 2015. Springer, 85–101.

[6]

L. Dagum and R. Menon. 1998. OpenMP: An Industry Standard API for Shared-Memory Programming. IEEE Computational Science and Engineering 5, 1 (1998), 46–55. https://doi.org/10.1109/99.660313

Digital Library

[7]

Alexander Droste, Michael Kuhn, and Thomas Ludwig. 2015. MPI-checker: Static Analysis for MPI. In 2nd Workshop on the LLVM Compiler Infrastructure in HPC. 1–10.

Digital Library

[8]

William Gropp and Rajeev Thakur. 2007. Thread-safety in an MPI implementation: Requirements and analysis. Parallel Comput. 33, 9 (2007), 595–604. https://doi.org/10.1016/j.parco.2007.07.002 Selected Papers from EuroPVM/MPI 2006.

Digital Library

[9]

Yizi Gu and John Mellor-Crummey. 2018. Dynamic Data Race Detection for OpenMP Programs. In SC18: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 767–778.

[10]

Tobias Hilbrich, Bronis R de Supinski, Andreas Knüpfer, Robert Dietrich, Christian Terboven, Felix Münchhalfen, and Wolfgang E Nagel. 2015. A Deadlock Detection Concept for OpenMP Tasks and Fully Hybrid MPI-OpenMP Applications. http://sc15.supercomputing.org/sites/all/themes/SC15images/tech_poster/poster_files/post149s2-file3.pdf.

[11]

Tobias Hilbrich, Martin Schulz, Bronis R de Supinski, and Matthias S Müller. 2010. MUST: A scalable approach to runtime error detection in MPI programs. In Tools for high performance computing 2009. Springer, 53–66.

[12]

Torsten Hoefler, Greg Bronevetsky, Brian Barrett, Bronis R. de Supinski, and Andrew Lumsdaine. 2010. Efficient MPI Support for Advanced Hybrid Programming Models. In Recent Advances in the Message Passing Interface, Rainer Keller, Edgar Gabriel, Michael Resch, and Jack Dongarra (Eds.). Springer, Berlin, Heidelberg, 50–61. https://doi.org/10.1007/978-3-642-15646-5_6

[13]

Alexander Hück, Jan-Patrick Lehr, Sebastian Kreutzer, Joachim Protze, Christian Terboven, Christian Bischof, and Matthias S. Müller. 2018. Compiler-aided type tracking for correctness checking of MPI applications. In IEEE/ACM 2nd Intl. Workshop on Software Correctness for HPC Applications (Correctness). IEEE, 51–58. https://doi.org/10.1109/Correctness.2018.00011

[14]

Intel. 2022. Intel Inspector. https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/inspector.html. Accessed: 2022-08-15.

[15]

Intel. 2022. Intel Trace Analyzer and Collector. https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/trace-analyzer.html. Accessed: 2022-08-15.

[16]

Ignacio Laguna, Ryan Marshall, Kathryn Mohror, Martin Ruefenacht, Anthony Skjellum, and Nawrin Sultana. 2019. A Large-Scale Study of MPI Usage in Open-Source HPC Applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, Colorado) (SC ’19). ACM, Article 31, 14 pages. https://doi.org/10.1145/3295500.3356176

Digital Library

[17]

Mathieu Laurent, Emmanuelle Saillard, and Martin Quinson. 2021. The MPI Bugs Initiative: a Framework for MPI Verification Tools Evaluation. In 2021 IEEE/ACM 5th International Workshop on Software Correctness for HPC Applications (Correctness). 1–9. https://doi.org/10.1109/Correctness54621.2021.00008

[18]

Jan-Patrick Lehr, Tim Jammer, and Christian Bischof. 2020. MPI-CorrBench: Towards an MPI Correctness Benchmark Suite. In Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing (Virtual Event, Sweden) (HPDC ’21). ACM, 69–80. https://doi.org/10.1145/3431379.3460652

Digital Library

[19]

Chunhua Liao, Pei-Hung Lin, Joshua Asplund, Markus Schordan, and Ian Karlin. 2017. DataRaceBench: a benchmark suite for systematic evaluation of data race detection tools. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1–14.

Digital Library

[20]

Chi-Keung Luk, Robert Cohn, Robert Muth, Harish Patil, Artur Klauser, Geoff Lowney, Steven Wallace, Vijay Janapa Reddi, and Kim Hazelwood. 2005. Pin: Customized Program Analysis Tools with Dynamic Instrumentation. ACM SIGPLAN Notices 40, 6 (2005), 190–200.

Digital Library

[21]

Message Passing Interface Forum. 2021. MPI: A Message-Passing Interface Standard Version 4.0. https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf.

[22]

Jan Felix Münchhalfen, Tobias Hilbrich, Joachim Protze, Christian Terboven, and Matthias S. Müller. 2014. Classification of Common Errors in OpenMP Applications. In Using and Improving OpenMP for Devices, Tasks, and More, Luiz DeRose, Bronis R. de Supinski, Stephen L. Olivier, Barbara M. Chapman, and Matthias S. Müller (Eds.). Springer International Publishing, Cham, 58–72.

[23]

Nicholas Nethercote and Julian Seward. 2007. Valgrind: a framework for heavyweight dynamic binary instrumentation. ACM Sigplan notices 42, 6 (2007), 89–100.

Digital Library

[24]

OpenMP Architecture Review Board. 2020. OpenMP - Application Programming Interface Version 5.1. https://www.openmp.org/wp-content/uploads/OpenMP-API-Specification-5-1.pdf.

[25]

[25] Paradyn Project.[n.d.].

[26]

Joachim Protze, Jonas Hahnfeld, Dong H. Ahn, Martin Schulz, and Matthias S. Müller. 2017. OpenMP Tools Interface: Synchronization Information for Data Race Detection. In IWOMP(Lecture Notes in Computer Science, Vol. 10468). Springer, 249–265.

[27]

R. Rabenseifner, G. Hager, and G. Jost. 2009. Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes. In 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing. 427–436. https://doi.org/10.1109/PDP.2009.43

Digital Library

[28]

Emmanuelle Saillard, Patrick Carribault, and Denis Barthou. 2014. PARCOACH: Combining static and dynamic validation of MPI collective communications. The International Journal of High Performance Computing Applications 28, 4(2014), 425–434.

Digital Library

[29]

Konstantin Serebryany, Alexander Potapenko, Timur Iskhodzhanov, and Dmitriy Vyukov. 2011. Dynamic Race Detection with LLVM Compiler. In International Conference on Runtime Verification. Springer, 110–114.

[30]

Jeff A. Stuart, Pavan Balaji, and John D. Owens. 2011. Extending MPI to Accelerators. In Proceedings of the 1st Workshop on Architectures and Systems for Big Data (Galveston Island, Texas, USA) (ASBD ’11). ACM, New York, NY, USA, 19–23. https://doi.org/10.1145/2377978.2377981

Digital Library

[31]

Michael Süß and Claudia Leopold. 2005. Common mistakes in OpenMP and how to avoid them. In International Workshop on OpenMP. Springer, 312–323.

[32]

Jesper Larsson Träff. 2016. A Library for Advanced Datatype Programming. In EuroMPI. ACM, 98–107.

[33]

J.S. Vetter and B.R. de Supinski. 2000. Dynamic Software Testing of MPI Applications with Umpire. In SC ’00: Proceedings of the 2000 ACM/IEEE Conference on Supercomputing. 51–51. https://doi.org/10.1109/SC.2000.10055

Cited By

Saad SFadel EAlzamzami OEassa FAlghamdi A(2025)Temporal-Logic-Based Testing Tool for Programs Using the Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) Programming ModelsIEEE Access10.1109/ACCESS.2025.352557813(4171-4187)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3525578
Saad SFadel EAlzamzami OEassa FAlghamdi A(2024)Temporal-Logic-Based Testing Tool Architecture for Dual-Programming Model SystemsComputers10.3390/computers1304008613:4(86)Online publication date: 25-Mar-2024
https://doi.org/10.3390/computers13040086
Tronge JSchuchart JDalcin LPritchard H(2024)Improving MPI Language Support Through Custom Datatype SerializationProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00062(414-424)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00062
Show More Cited By

Index Terms

Towards a Hybrid MPI Correctness Benchmark Suite
1. Computing methodologies
  1. Parallel computing methodologies
2. Software and its engineering
  1. Software organization and properties
    1. Software functional properties
      1. Correctness

Recommendations

RMARaceBench: A Microbenchmark Suite to Evaluate Race Detection Tools for RMA Programs
SC-W '23: Proceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis

Parallel programming models with Remote Memory Access (RMA), such as MPI RMA, OpenSHMEM, and GASPI, allow processes to modify the memory of other processes directly. Special care is required to avoid concurrent conflicting accesses that lead to data ...
MPI-CorrBench: Towards an MPI Correctness Benchmark Suite
HPDC '21: Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing

The Message Passing Interface (MPI) is the de-facto standard for distributed memory computing in high-performance computing (HPC). To aid developers write correct MPI programs, different tools have been proposed, e.g., Intel Trace Analyzer and Collector ...
MPI correctness checking for OpenMP/MPI applications

The MPI interface is the de-facto standard for message passing applications, but it is also complex and defines several usage patterns as erroneous. A current trend is the investigation of hybrid programming techniques that use MPI processes and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

EuroMPI/USA '22: Proceedings of the 29th European MPI Users' Group Meeting

September 2022

78 pages

ISBN:9781450397995

DOI:10.1145/3555819

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Deutsche Forschungsgemeinschaft

Conference

EuroMPI/USA'22

EuroMPI/USA'22: 29th European MPI Users' Group Meeting

September 26 - 28, 2022

TN, Chattanooga, USA

Acceptance Rates

Overall Acceptance Rate 66 of 139 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
376
Total Downloads

Downloads (Last 12 months)161
Downloads (Last 6 weeks)35

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Saad SFadel EAlzamzami OEassa FAlghamdi A(2025)Temporal-Logic-Based Testing Tool for Programs Using the Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) Programming ModelsIEEE Access10.1109/ACCESS.2025.352557813(4171-4187)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3525578
Saad SFadel EAlzamzami OEassa FAlghamdi A(2024)Temporal-Logic-Based Testing Tool Architecture for Dual-Programming Model SystemsComputers10.3390/computers1304008613:4(86)Online publication date: 25-Mar-2024
https://doi.org/10.3390/computers13040086
Tronge JSchuchart JDalcin LPritchard H(2024)Improving MPI Language Support Through Custom Datatype SerializationProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00062(414-424)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00062
Pätzold CSchwitanski SJenke JTomski FMüller M(2024)Correctness Checking of MPI+OpenMP Applications Using Vector Clocks in MUSTProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00035(227-231)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00035
Jammer TSchwitanski SSaillard EHück AJenke JVinayagame RBischof C(2024)Designing Quality MPI Correctness Benchmarks: Insights and MetricsProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00034(222-226)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00034
Hück AZiegler TSchwitanski SJenke JBischof C(2024)Compiler-Aided Correctness Checking of CUDA-Aware MPI ApplicationsProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00032(204-213)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00032
Lehr JHalkenhäuser MChakrabarti DIslam SPalermo DLieberman R(2024)ompTest - Unit Testing with OMPTProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00031(196-203)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SCW63240.2024.00031
Hück AJammer TJenke JBischof C(2023)Investigating the Real-World Applicability of MPI Correctness BenchmarksProceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624091(230-233)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624091
Schwitanski SJenke JKlotz SMüller M(2023)RMARaceBench: A Microbenchmark Suite to Evaluate Race Detection Tools for RMA ProgramsProceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624087(205-214)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624087
Tronge JPritchard HBrown J(2023)Improving MPI Safety for Modern LanguagesProceedings of the 30th European MPI Users' Group Meeting10.1145/3615318.3615328(1-11)Online publication date: 11-Sep-2023
https://dl.acm.org/doi/10.1145/3615318.3615328
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten