More Web Proxy on the site http://driver.im/

research-article

An empirical survey of performance and energy efficiency variation on Intel processors

Authors:

Aniruddha Marathe,

Grayson Blanks,

Nirmal Kumbhare,

Ghaleb Abdulla,

Barry RountreeAuthors Info & Claims

E2SC'17: Proceedings of the 5th International Workshop on Energy Efficient Supercomputing

Article No.: 9, Pages 1 - 8

https://doi.org/10.1145/3149412.3149421

Published: 12 November 2017 Publication History

Abstract

Traditional HPC performance and energy characterization approaches assume homogeneity and predictability in the performance of the target processor platform. Consequently, processor performance variation has been considered to be a secondary issue in the broader problem of performance characterization. In this work, we present an empirical survey of the variation in processor performance and energy efficiency on several generations of HPC-grade Intel processors. Our study shows that, compared to the previous generation of Intel processors, the problem of performance variation has become worse on more recent generation of Intel processors. Specifically, the performance variation across processors on a large-scale production HPC cluster at LLNL has increased to 20% and the run-to-run variation in the performance of individual processors has increased to 15%. We show that this variation is further magnified under a hardware-enforced power constraint, potentially due to the increase in number of cores, inconsistencies in the chip manufacturing process and their combined impact on processor's energy management functionality. Our experimentation with a hardware-enforced processor power constraint shows that the variation in processor performance and energy efficiency has increased by up to 4x on the latest Intel processors.

References

[1]

Great internet mersenne prime search. https://www.mersenne.org/download/.

[2]

Intel turbo technology 2.0. https://www.intel.com/content/www/us/en/architecture-and-technology/turbo-boost/turbo-boost-technology.html.

[3]

B. Acun, P. Miller, and L. V. Kale. Variation among processors under turbo boost in hpc systems. In Proceedings of the 2016 International Conference on Supercomputing, page 6. ACM, 2016.

Digital Library

[4]

D. H. Bailey, E. Barszcz, J. T. Barton, D. S. Browning, R. L. Carter, L. Dagum, R. A. Fatoohi, P. O. Frederickson, T. A. Lasinski, R. S. Schreiber, et al. The NAS parallel benchmarks summary and preliminary results. In Supercomputing'91, Proceedings of the 1991 ACM/IEEE Conference on, pages 158-165. IEEE, 1991.

Digital Library

[5]

A. Bhatele, K. Mohror, S. H. Langer, and K. E. Isaacs. There goes the neighborhood: performance degradation due to nearby jobs. In Supercomputing'13, Proceedings of the 2013 ACM/IEEE Conference on, page 41. ACM, 2013.

Digital Library

[6]

S. Borkar, T. Karnik, S. Narendra, J. Tschanz, A. Keshavarzi, and V. De. Parameter variations and impact on circuits and microarchitecture. In Proceedings of the 40th annual Design Automation Conference, pages 338--342. ACM, 2003.

Digital Library

[7]

D. Chasapis, M. Casas, M. Moretó, M. Schulz, E. Ayguadé, J. Labarta, and M. Valero. Runtime-guided mitigation of manufacturing variability in power-constrained multi-socket numa nodes. In Proceedings of the 2016 International Conference on Supercomputing, page 5. ACM, 2016.

Digital Library

[8]

J. J. Dongarra, J. Du Croz, S. Hammarling, and I. S. Duff. A set of level 3 basic linear algebra subprograms. ACM Transactions on Mathematical Software, pages 1--17, 1990.

Digital Library

[9]

N. Gholkar, F. Mueller, and B. Rountree. Power tuning HPC jobs on power-constrained systems. In Proceedings of the 2016 International Conference on Parallel Architectures and Compilation, pages 179--191. ACM, 2016.

Digital Library

[10]

D. Hackenberg, R. Oldenburg, D. Molka, and R. Schöne. Introducing firestarter: A processor stress test utility. In Green Computing Conference, 2013 International, pages 1--9. IEEE, 2013.

[11]

S. Herbert and D. Marculescu. Variation-aware dynamic voltage/frequency scaling. In High Performance Computer Architecture, 2009. HPCA 2009. IEEE 15th International Symposium on, pages 301--312. IEEE, 2009.

[12]

Y. Inadomi, T. Patki, K. Inoue, M. Aoyagi, B. Rountree, M. Schulz, D. Lowenthal, Y. Wada, K. Fukazawa, M. Ueda, et al. Analyzing and mitigating the impact of manufacturing variability in power-constrained supercomputing. In Supercomputing'15, Proceedings of the ACM/IEEE Conference on, page 78. ACM, 2015.

Digital Library

[13]

E. A. Leon, I. Karlin, and A. T. Moody. System noise revisited: Enabling application scalability and reproducibility with smt. In 2016 IEEE International Parallel and Distributed Processing Symposium, pages 596--607, May 2016.

[14]

A. Marathe, H. Gahvari, J.-S. Yeom, and A. Bhatele. Libpowermon: A lightweight profiling framework to profile program context and system-level metrics. In Parallel and Distributed Processing Symposium Workshops, 2016 IEEE International, pages 1132--1141. IEEE, 2016.

[15]

J. D. McCalpin. Memory bandwidth and machine balance in current high performance computers. IEEE Computer Society Technical Committee on Computer Architecture Newsletter, pages 19-25, Dec. 1995.

[16]

E. Rosenthal, E. A. León, and A. T. Moody. Mitigating system noise with simultaneous multi-threading. Proceedings of SC13 (poster), 2013.

[17]

B. Rountree, D. H. Ahn, B. R. De Supinski, D. K. Lowenthal, and M. Schulz. Beyond dvfs: A first look at performance under a hardware-enforced power bound. In Parallel and Distributed Processing Symposium Workshops & PhD Forum, 2012 IEEE 26th International, pages 947--953. IEEE, 2012.

Digital Library

[18]

K. Shoga, B. Rountree, M. Schulz, and J. Shafer. Whitelisting msrs with msr-safe.

[19]

H. Shoukourian, T. Wilde, H. Huber, and A. Bode. Analysis of the efficiency characteristics of the first high-temperature direct liquid cooled petascale supercomputer and its cooling infrastructure. Journal of Parallel and Dist. Computing, pages 87--100, 2017.

[20]

R. Teodorescu and J. Torrellas. Variation-aware application scheduling and power management for chip multiprocessors. In Computer Architecture, 2008. 35th International Symposium on, pages 363--374. IEEE.

Digital Library

[21]

O. Tuncer, E. Ates, Y. Zhang, A. Turk, J. Brandt, V. J. Leung, M. Egele, and A. K. Coskun. Diagnosing performance variations in HPC applications using machine learning. In International Supercomputing Conference, pages 355--373, 2017.

[22]

L. Zhang, L. S. Bai, R. P. Dick, L. Shang, and R. Joseph. Process variation characterization of chip-level multiprocessors. In Proceedings of the 46th Annual Design Automation Conference, pages 694--697. ACM, 2009.

Digital Library

Cited By

Juffinger JKalinin SGruss DMueller FTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)SUIT: Secure Undervolting with Instruction TrapsProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3620665.3640373(1128-1145)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620665.3640373
Kusaba TAwaki YYoshida KMiwa SYamaki HHanawa THonda H(2024)Power-Efficiency Variation on A64FX Supercomputers and its Application to System Operation2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops)10.1109/CLUSTERWorkshops61563.2024.00018(55-65)Online publication date: 24-Sep-2024
https://doi.org/10.1109/CLUSTERWorkshops61563.2024.00018
Cui MPapadopoulou NPericàs M(2023)Analysis and Characterization of Performance Variability for OpenMP RuntimeProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624239(1614-1622)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624239
Show More Cited By

Index Terms

An empirical survey of performance and energy efficiency variation on Intel processors
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
2. Hardware
  1. Power and energy
    1. Power estimation and optimization
      1. Platform power issues

Recommendations

Resource management techniques for performance and energy efficiency in multithreaded processors
Exploring SIMD for Molecular Dynamics, Using Intel® Xeon® Processors and Intel® Xeon Phi Coprocessors
IPDPS '13: Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing

We analyse gather-scatter performance bottlenecks in molecular dynamics codes and the challenges that they pose for obtaining benefits from SIMD execution. This analysis informs a number of novel code-level and algorithmic improvements to Sandia's ...
Improving performance and energy efficiency of embedded processors via post-fabrication instruction set customization

Encapsulating critical computation subgraphs as application-specific instruction set extensions is an effective technique to enhance the performance and energy efficiency of embedded processors. However, the addition of custom functional units to the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

E2SC'17: Proceedings of the 5th International Workshop on Energy Efficient Supercomputing

November 2017

84 pages

ISBN:9781450351324

DOI:10.1145/3149412

Copyright © 2017 ACM.

© 2017 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SC '17

Sponsor:

SIGHPC

SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2017

CO, Denver, USA

Acceptance Rates

E2SC'17 Paper Acceptance Rate 10 of 21 submissions, 48%;

Overall Acceptance Rate 17 of 33 submissions, 52%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
367
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)4

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Juffinger JKalinin SGruss DMueller FTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)SUIT: Secure Undervolting with Instruction TrapsProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3620665.3640373(1128-1145)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620665.3640373
Kusaba TAwaki YYoshida KMiwa SYamaki HHanawa THonda H(2024)Power-Efficiency Variation on A64FX Supercomputers and its Application to System Operation2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops)10.1109/CLUSTERWorkshops61563.2024.00018(55-65)Online publication date: 24-Sep-2024
https://doi.org/10.1109/CLUSTERWorkshops61563.2024.00018
Cui MPapadopoulou NPericàs M(2023)Analysis and Characterization of Performance Variability for OpenMP RuntimeProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624239(1614-1622)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624239
Aksar BSencan ESchwaller BAaziz OLeung VBrandt JKulis BEgele MCoskun AMohror KArnold DBadia R(2023)Prodigy: Towards Unsupervised Anomaly Detection in Production HPC SystemsProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3607076(1-14)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3581784.3607076
Kambli ADey P(2023)A critical review on recent developments and applications of microchannels in the field of heat transfer and energyHeat and Mass Transfer10.1007/s00231-023-03358-859:9(1707-1747)Online publication date: 18-Apr-2023
https://doi.org/10.1007/s00231-023-03358-8
Yoshida KSageyama RMiwa SYamaki HHonda H(2022)Analyzing Performance and Power-Efficiency Variations among NVIDIA GPUsProceedings of the 51st International Conference on Parallel Processing10.1145/3545008.3545084(1-12)Online publication date: 29-Aug-2022
https://dl.acm.org/doi/10.1145/3545008.3545084
Abdelhafez HHalawa HAlmoallim AAhmadi APattabiraman KRipeanu M(2022)Characterizing Variability in Heterogeneous Edge Systems: A Methodology & Case Study2022 IEEE/ACM 7th Symposium on Edge Computing (SEC)10.1109/SEC54971.2022.00016(107-121)Online publication date: Dec-2022
https://doi.org/10.1109/SEC54971.2022.00016
Wilson DAl-rawi ALawson LJana SArdanaz FEastep JCoskun A(2022)Guiding Hardware-Driven Turbo with Application Performance Awareness2022 IEEE 13th International Green and Sustainable Computing Conference (IGSC)10.1109/IGSC55832.2022.9969356(1-8)Online publication date: 24-Oct-2022
https://doi.org/10.1109/IGSC55832.2022.9969356
Ardebili MBartolini AAcquaviva ABenini L(2022)Rule-Based Thermal Anomaly Detection for Tier-0 HPC SystemsHigh Performance Computing. ISC High Performance 2022 International Workshops10.1007/978-3-031-23220-6_18(262-276)Online publication date: 29-May-2022
https://dl.acm.org/doi/10.1007/978-3-031-23220-6_18
Zhang YWilson DPaschalidis ICoskun A(2021)A Data Center Demand Response Policy for Real-World Workload Scenarios in HPC2021 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE51398.2021.9474075(282-287)Online publication date: 1-Feb-2021
https://doi.org/10.23919/DATE51398.2021.9474075
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents