Globalizing selectively: shared-memory efficiency with address-space separation
Article No.: 17, Pages 1 - 12
Abstract
It has become common for MPI-based applications to run on shared-memory machines. However, MPI semantics do not allow leveraging shared memory fully for communication between processes from within the MPI library. This paper presents an approach that combines compiler transformations with a specialized runtime system to achieve zero-copy communication whenever possible by proving certain properties statically and globalizing data selectively by altering the allocation and deallocation of communication buffers. The runtime system provides dynamic optimization, when such proofs are not possible statically, by copying data only when there are write-write or read-write conflicts. We implemented a prototype compiler, using ROSE, and evaluated it on several benchmarks. Our system produces code that performs better than MPI in most cases and no worse than MPI, tuned for shared memory, in all cases.
References
[1]
D. H. Bailey, E. Barszcz, J. T. Barton, D. S. Browning, R. L. Carter, L. Dagum, R. A. Fatoohi, P. O. Frederickson, T. A. Lasinski, R. S. Schreiber, H. D. Simon, V. Venkatakrishnan, and S. K. Weeratunga. The NAS parallel benchmarks. International Journal of High Performance Computing Applications, 5(3):63--73, 1991.
[2]
Boost C++ libraries. http://www.boost.org/.
[3]
G. Bronevetsky. Communication-sensitive static dataflow for parallel message passing applications. In Proceedings of the 7th annual IEEE/ACM International Symposium on Code Generation and Optimization, CGO '09, pages 1--12. IEEE Computer Society, 2009.
[4]
D. Buntinas, G. Mercier, and W. D. Gropp. Implementation and shared-memory evaluation of MPICH2 over the Nemesis communication subsystem. In PVM/MPI'06, pages 86--95, 2006.
[5]
A. Danalis, L. Pollock, M. Swany, and J. Cavazos. MPI-aware compiler optimizations for improving communication-computation overlap. In Proceedings of the 23rd International Conference on Supercomputing (ICS), pages 316--325, 2009.
[6]
A. Friedley, T. Hoefler, G. Bronevetsky, A. Lumsdaine, and C.-C. Ma. Ownership passing: Efficient distributed memory programming on multi-core systems. In Proceedings of the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP), pages 177--186, 2013.
[7]
G. Gopalakrishnan, R. M. Kirby, S. Siegel, R. Thakur, W. Gropp, E. Lusk, B. R. D. Supinski, M. Schulz, and G. Bronevetsky. Formal analysis of MPI-based parallel programs. Communications of the ACM, 54(12):82--91, Dec. 2011.
[8]
P. Haller and M. Odersky. Capabilities for uniqueness and borrowing. In Proceedings of the 24th European conference on Object-oriented programming, ECOOP'10, pages 354--378, Berlin, Heidelberg, 2010. Springer-Verlag.
[9]
C. A. R. Hoare. Communicating sequential processes. Communications of the ACM, 21(8):666--677, Aug. 1978.
[10]
F. Jiao, N. Mahajan, J. Willcock, A. Chauhan, and A. Lumsdaine. Partial globalization of partitioned address spaces for zero-copy communication with shared memory. In Proceedings of the 18th International Conference on High Performance Computing (HiPC), 2011.
[11]
K. Kennedy and A. Sethi. Resource-based communication placement analysis. In Proceedings of the 9th Workshop on Language and Compilers for Parallel Computing (LCPC), pages 369--388. Springer-Verlag, 1996.
[12]
Message Passing Interface Forum, University of Tennessee, Knoxville, Tennessee. MPI: A Message-Passing Interface Standard, Version 2.2, Sept. 2009.
[13]
S. Negara, R. K. Karmani, and G. Agha. Inferring ownership transfer for efficient message passing. In Proceedings of the 16th ACM Symposium on Principles and Practice of Parallel Programming (PPoPP), pages 81--90, 2011.
[14]
D. Quinlan, C. Liao, T. Panas, R. Matzke, M. Schordan, R. Vuduc, and Q. Yi. ROSE User Manual: A Tool for Building Source-to-Source Translators. Lawrence Livermore National Laboratory, Livermore, CA 94550, version 0.9.5a edition, Nov. 2010.
[15]
A. Silberschatz, P. B. Galvin, and G. Gagne. Operating System Concepts. Wiley, 8th edition, 2008.
[16]
H. Tang, K. Shen, and T. Yang. Program transformation and runtime support for threaded MPI execution on shared-memory machines. ACM Transactions on Programming Languages and Systems (TOPLAS), 22(4):673--700, July 2000.
- Globalizing selectively: shared-memory efficiency with address-space separation
Recommendations
OpenMP for Networks of SMPs
In this paper, we present the first system that implements OpenMP on a network of shared-memory multiprocessors. This system enables the programmer to rely on a single, standard, shared-memory API for parallelization within a multiprocessor and between ...
Extended commitment ordering, or guaranteeing global serializability by applying commitment order selectively to global transactions
PODS '93: Proceedings of the twelfth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systemsThe Extended Commitment Ordering (ECO) property of transaction histories (schedules) generalizes the Commitment Ordering (CO) property defined in [Raz 90]. In a multi resource manager (RM) environment ECO guarantees global serializability when supported ...
The performance and scalability of SHMEM and MPI-2 one-sided routines on a SGI Origin 2000 and a Cray T3E-600: Performances
This paper compares the performance and scalability of SHMEM and MPI-2 one-sided routines on different communication patterns for a SGI Origin 2000 and a Cray T3E-600. The communication tests were chosen to represent commonly used communication patterns ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2013
1123 pages
Copyright © 2013 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 November 2013
Check for updates
Qualifiers
- Research-article
Funding Sources
Conference
SC13: International Conference for High Performance Computing, Networking, Storage and Analysis
November 17 - 21, 2013
Colorado, Denver
Acceptance Rates
SC '13 Paper Acceptance Rate 91 of 449 submissions, 20%;
Overall Acceptance Rate 1,516 of 6,373 submissions, 24%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 133Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in