Comparative Evaluation of Call Graph Generation by Profiling Tools
Pages 213 - 232
Abstract
Call graphs generated by profiling tools are critical to dissecting the performance of parallel programs. Although many mature and sophisticated profiling tools record call graph data, each tool is different in its runtime overheads, memory consumption, and output data generated. In this work, we perform a comparative evaluation study on the call graph data generation capabilities of several popular profiling tools – Caliper, HPCToolkit, TAU, and Score-P. We evaluate their runtime overheads, memory consumption, and generated call graph data (size and quality). We perform this comparison empirically by executing several proxy applications, AMG, LULESH, and Quicksilver on a parallel cluster. Our results show which tool results in the lowest overheads and produces the most meaningful call graph data under different conditions.
References
[1]
Adhianto L et al. HPCTOOLKIT: tools for performance analysis of optimized parallel programs Concurr. Comput. Pract. Exp. 2010 22 6 685-701
[2]
Adhianto, L., Mellor-Crummey, J., Tallent, N.R.: Effectively presenting call path profiles of application performance. In: 2010 39th International Conference on Parallel Processing Workshops, pp. 179–188. IEEE (2010)
[3]
Bell R, Malony AD, and Shende S Kosch H, Böszörményi L, and Hellwagner H ParaProf: a portable, extensible, and scalable tool for parallel performance profile analysis Euro-Par 2003 Parallel Processing 2003 Heidelberg Springer 17-26
[4]
Bhatele, A., Brink, S., Gamblin, T.: Hatchet: pruning the overgrowth in parallel profiles. In: Proceedings of the ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2019, November 2019. lLNL-CONF-772402
[5]
Boehme, D., et al.: Caliper: performance introspection for HPC software stacks. In: SC 2016: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 550–560 (2016).
[6]
Henson, V.E., Yang, U.M.: BoomerAMG: a parallel algebraic multigrid solver and preconditioner. Appl. Numer. Math. 41(1), 155–177 (2002). https://www.sciencedirect.com/science/article/pii/S0168927401001155. Developments and Trends in Iterative Methods for Large Systems of Equations - in Memorium Rudiger Weiss
[7]
Karlin, I., Keasler, J., Neely, R.: Lulesh 2.0 updates and changes. Technical report LLNL-TR-641973, August 2013
[8]
Knobloch M and Mohr B Tools for GPU computing-debugging and performance analysis of heterogenous HPC applications Supercomput. Front. Innov. 2020 7 1 91-111
[9]
Knüpfer A et al. Brunst H, Müller MS, Nagel WE, Resch MM, et al. Score-p: a joint performance measurement run-time infrastructure for periscope, Scalasca, TAU, and Vampir Tools for High Performance Computing 2011 2012 Heidelberg Springer 79-91
[10]
Leko, A., Sherburne, H., Su, H., Golden, B., George, A.D.: Practical experiences with modern parallel performance analysis tools: an evaluation. In: Parallel and Distributed Processing, IPDPS 2008 IEEE Symposium, pp. 14–18 (2008)
[11]
Lindlan, K.A., et al.: A tool framework for static and dynamic analysis of object-oriented software with templates. In: SC 2000: Proceedings of the 2000 ACM/IEEE Conference on Supercomputing, p. 49. IEEE (2000)
[12]
Liu X and Mellor-Crummey J A tool to analyze the performance of multithreaded programs on NUMA architectures ACM Sigplan Not. 2014 49 8 259-272
[13]
Madsen JR et al. Sadayappan P, Chamberlain BL, Juckeland G, Ltaief H, et al. Timemory: modular performance analysis for HPC High Performance Computing 2020 Cham Springer 434-452
[14]
Malony, A.D., Huck, K.A.: General hybrid parallel profiling. In: 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, pp. 204–212. IEEE (2014)
[15]
Mellor-Crummey J, Fowler R, and Marin G HPCView: a tool for top-down analysis of node performance J. Supercomput. 2002 23 81-101
[16]
Mohr B Scalable parallel performance measurement and analysis tools-state-of-the-art and future challenges Supercomput. Front. Innov. 2014 1 2 108-123
[17]
Nataraj A, Sottile M, Morris A, Malony AD, and Shende S Kermarrec A-M, Bougé L, and Priol T TAUoverSupermon: low-overhead online parallel performance monitoring Euro-Par 2007 Parallel Processing 2007 Heidelberg Springer 85-96
[18]
Nethercote, N.: Dynamic binary analysis and instrumentation. Technical report, University of Cambridge, Computer Laboratory (2004)
[19]
Richards, D.F., Bleile, R.C., Brantley, P.S., Dawson, S.A., McKinley, M.S., O’Brien, M.J.: Quicksilver: a proxy app for the monte Carlo transport code mercury. In: 2017 IEEE International Conference on Cluster Computing (CLUSTER), pp. 866–873. IEEE (2017)
[20]
Saviankou P, Knobloch M, Visser A, and Mohr B Cube v4: from performance report explorer to performance analysis tool Procedia Comput. Sci. 2015 51 1343-1352
[21]
Shende S and Malony AD Integration and application of TAU in parallel Java environments Concurr. Comput. Pract. Exp. 2003 15 3–5 501-519
[22]
Shende SS and Malony AD The TAU parallel performance system Int. J. High Perform. Comput. Appl. 2006 20 2 287-311
[23]
Tallent NR, Mellor-Crummey JM, and Fagan MW Binary analysis for measurement and attribution of program performance ACM Sigplan Not. 2009 44 6 441-452
Index Terms
- Comparative Evaluation of Call Graph Generation by Profiling Tools
Index terms have been assigned to the content through auto-classification.
Recommendations
An Overhead Analysis of MPI Profiling and Tracing Tools
PERMAVOST '22: Proceedings of the 2nd Workshop on Performance EngineeRing, Modelling, Analysis, and VisualizatiOn StrategyMPI performance analysis tools are important instruments for finding performance bottlenecks in large-scale MPI applications. These tools commonly support either the profiling or the tracing of parallel applications. Depending on the type of analysis, ...
Comparing call graphs
PASTE '07: Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineeringComparing program analysis results from different static and dynamic analysis tools is difficult and therefore too rare, especially when it comes to qualitative comparison. Analysis results can be strongly affected by specific details of programs being ...
Google-Wide Profiling: A Continuous Profiling Infrastructure for Data Centers
Google-Wide Profiling (GWP), a continuous profiling infrastructure for data centers, provides performance insights for cloud applications. With negligible overhead, GWP provides stable, accurate profiles and a datacenter-scale tool for traditional ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2022
382 pages
ISBN:978-3-031-07311-3
DOI:10.1007/978-3-031-07312-0
© Springer Nature Switzerland AG 2022.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 29 May 2022
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025