More Web Proxy on the site http://driver.im/

article

Misleading Performance Reporting in the Supercomputing Field

Author:

David H. BaileyAuthors Info & Claims

Scientific Programming, Volume 1, Issue 2

Pages 141 - 151

https://doi.org/10.1155/1992/328793

Published: 01 April 1992 Publication History

Abstract

In a previous humorous note, I outlined 12 ways in which performance figures for scientific supercomputers can be distorted. In this paper, the problem of potentially misleading performance reporting is discussed in detail. Included are some examples that have appeared in recent published scientific papers. This paper also includes some proposed guidelines for reporting performance, the adoption of which would raise the level of professionalism and reduce the level of confusion in the field of supercomputing.

References

[1]

D. H. Bailey. "Twelve ways to food the masses when giving performance results on parallel computers." Supercomput. Rev. vol. 4, no. 8, pp. 54-55, 1991. (Also published in Supercomputer, pp. 4-7, 1991.)

[2]

J. Markoff. "Measuring how fast computers really are." New York Times. September 22. 1991. p. 14f.

[3]

Paper A.

[4]

Paper B.

[5]

Paper C.

[6]

Paper D.

[7]

O. M. Lubeck, M. L. Simmons, and H. J. Wasserman. Proceedings of Supercomputing 1992. IEEE. pp. 403-412.

[8]

Paper G.

[9]

Paper E.

[10]

Paper F.

[11]

J. N. Shadid and R. S. Tuminaro. Proceedings of the Fifth SIAM Conference on Parallel Processing for Scientific Computing. Philadelphia. PA: SIAM. 1992. pp. 123-129.

[12]

J. Gustafson, D. Rover, S. Elbert, and M. Carter, "SLALOM: surviving adolescence." Supercomput. Rev. vol. 4, no. 12, pp. 54-61, 1991.

[13]

D. H. Bailey. "Extra-high speed matrix multiplication on the Cray-2." SIAM J. Scientific Statistical Comput. vol. 9, pp. 603-607, 1988.

Digital Library

[14]

D. H. Bailey, K. Lee, and H. D. Simon, "Using Strassen's algorithm to accelerate the solution of linear systems." J. Supercomput., vol. 4, pp. 357-371, 1991.

Digital Library

[15]

N. J. Higham. "Exploiting fast matrix multiplication within the level 3 BLAS." ACM Trans. Mathematical Software. vol. 16, pp. 352-368, 1990.

Digital Library

[16]

E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. DuCroz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchoy, and D. Sorenson. The LAPACK Users Guide. Philadelphia PA: SIAM. 1992.

[17]

Paper H.

[18]

J. Gustafson, G. R. Montry, and R. E. Benner. "Development of parallel methods for a 1024- processor hypercube." SIAM J. Scientific Statistical Comput., vol. 9. pp. 609-638. 1988.

Digital Library

[19]

D. H. Bailey and P. O. Frederickson. Proceedings of Supercomputing'91. Los. Alamitos. CA: IEEE. 1991. pp. 166-173.

[20]

M. Garbey and D. Levine. Proceedings of the Fourth SIAM Conference on Parallel Processing for Scientific Computing. Philadelphia. PA: SIAM. 1990. pp. 340-345.

[21]

Paper I.

Cited By

Kramer WLathrop SCosta JKramer W(2011)How to measure useful, sustained performanceState of the Practice Reports10.1145/2063348.2063351(1-18)Online publication date: 12-Nov-2011
https://dl.acm.org/doi/10.1145/2063348.2063351
Ireland KJezak JLevitan SChiarulli DPalesi MKumar S(2009)Scalable arbitration of partitioned bus interconnection networks in 3D-IC systemsProceedings of the 2nd International Workshop on Network on Chip Architectures10.1145/1645213.1645231(77-82)Online publication date: 12-Dec-2009
https://dl.acm.org/doi/10.1145/1645213.1645231
Levitan SChiarulli D(2009)Massively parallel processingProceedings of the 46th Annual Design Automation Conference10.1145/1629911.1630050(534-538)Online publication date: 26-Jul-2009
https://dl.acm.org/doi/10.1145/1629911.1630050
Show More Cited By

Misleading Performance Reporting in the Supercomputing Field
1. Information systems

Recommendations

Hungarian Supercomputing Grid
ICCS '02: Proceedings of the International Conference on Computational Science-Part II

The main objective of the paper is to describe the main goals and activities within the newly formed Hungarian Supercomputing Grid (H-SuperGrid) which will be used as a high-performance and highthroughput Grid. In order to achieve these two features ...
Misleading performance claims in parallel computations
DAC '09: Proceedings of the 46th Annual Design Automation Conference

In a previous humorous note entitled "Twelve Ways to Fool the Masses ...," I outlined twelve common ways in which performance figures for technical computer systems can be distorted. In this paper and accompanying conference talk, I give a reprise of ...
Misleading performance in the supercomputing field
Supercomputing '92: Proceedings of the 1992 ACM/IEEE conference on Supercomputing

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Scientific Programming

Scientific Programming Volume 1, Issue 2

April 1992

102 pages

Issue’s Table of Contents

Publisher

IOS Press

Netherlands

Publication History

Published: 01 April 1992

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kramer WLathrop SCosta JKramer W(2011)How to measure useful, sustained performanceState of the Practice Reports10.1145/2063348.2063351(1-18)Online publication date: 12-Nov-2011
Ireland KJezak JLevitan SChiarulli DPalesi MKumar S(2009)Scalable arbitration of partitioned bus interconnection networks in 3D-IC systemsProceedings of the 2nd International Workshop on Network on Chip Architectures10.1145/1645213.1645231(77-82)Online publication date: 12-Dec-2009
Levitan SChiarulli D(2009)Massively parallel processingProceedings of the 46th Annual Design Automation Conference10.1145/1629911.1630050(534-538)Online publication date: 26-Jul-2009
Bailey D(2009)Misleading performance claims in parallel computationsProceedings of the 46th Annual Design Automation Conference10.1145/1629911.1630049(528-533)Online publication date: 26-Jul-2009
Culler DSingh JGupta A(1998)Parallel Computer ArchitectureundefinedOnline publication date: 29-Sep-1998
Gurd J(1994)Supercomputing: big bang or steady state growth?ACM SIGARCH Computer Architecture News10.1145/181799.18180222:3(3-13)Online publication date: 1-Jun-1994

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents