[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3447545.3451176acmconferencesArticle/Chapter ViewAbstractPublication PagesicpeConference Proceedingsconference-collections
Work in Progress

Towards a Benchmark for Software Resource Efficiency

Published: 19 April 2021 Publication History

Abstract

Data centers already account for over 250TWh of energy usage every year and their energy demand will grow above 1PWh until 2030 even in the best-case scenarios of some studies. As this demand cannot be met with renewable sources as of today, this growth will lead to a further increase of CO2 emissions. The data center growth is mainly driven by software resource usage but most of the energy efficiency improvements are nowadays done on hardware level that cannot compensate the demand. To reduce the resource demand of software in data centers one needs to be able to quantify its resource usage. Therefore, we propose a benchmark to assess the resource consumption of data center software (i.e., cloud applications) and make the resource usage of standard application types comparable between vendors. This benchmark aims to support three main goals (i) software vendors should be able to get an understanding of the resource consumption of their software; (ii) software buyers should be able to compare the software of different vendors; and (iii) spark competition between the software vendors to make their software more efficient and thus, in the long term, reduce the data center growth as the software systems require less resources.

References

[1]
Anders Andrae and Tomas Edler. 2015. On Global Electricity Usage of Communication Technology: Trends to 2030. Challenges, Vol. 6, 1 (Apr 2015), 117--157.
[2]
David Bermbach, Erik Wittern, and Stefan Tai. 2017. Cloud Service Benchmarking 1 ed.). Springer International Publishing.
[3]
Reinhard Brandl, Martin Bichler, and Michael Ströbel. 2007. Cost accounting for shared IT infrastructures. Wirtschaftsinformatik, Vol. 49, 2 (2007), 83--94.
[4]
Andreas Brunnert and Helmut Krcmar. 2017. Continuous performance evaluation and capacity planning using resource profiles for enterprise applications. Journal of Systems and Software, Vol. 123 (2017), 239 -- 262.
[5]
Andreas Brunnert, Kilian Wischer, and Helmut Krcmar. 2014. Using Architecture-Level Performance Models as Resource Profiles for Enterprise Applications. In Proceedings of the 10th International ACM Sigsoft Conference on Quality of Software Architectures (QoSA '14). Association for Computing Machinery, New York, NY, USA, 53--62.
[6]
Eugenio Capra, Chiara Francalanci, and Sandra A. Slaughter. 2012. Is software “green”? Application development environments and energy efficiency in open source applications. Information and Software Technology, Vol. 54, 1 (2012), 60 -- 71.
[7]
Bob Cramblitt. 2019. SPEC releases new version of CPU benchmark suite. Press Release http://www.spec.org/cpu2017/press/v1_1_release.html.
[8]
Stefan Valentin Gheorghita, Henk Corporaal, and Twan Basten. 2005. Iterative compilation for energy reduction. Journal of Embedded Computing, Vol. 1, 4 (2005), 509--520.
[9]
Lee Gillam, Bin Li, John O'Loughlin, and Anuz Pratap Singh Tomar. 2013. Fair benchmarking for cloud computing systems. Journal of Cloud Computing: Advances, Systems and Applications, Vol. 2, 1 (2013), 1--45.
[10]
James Glanz. 2012. Power, Pollution and the Internet. New York Times (September 23rd 2012).
[11]
Jens Gröger, Andreas Köhler, Stefan Naumann, Andreas Filler, Achim Guldner, Eva Kern, Lorenz Hilty, and Yuliyan Maksimov. 2018. Entwicklung und Anwendung von Bewertungsgrundlagen für ressourceneffiziente Software unter Berücksichtigung bestehender Methodik-Abschlussbericht. UBA TEXTE, Vol. 105 (2018).
[12]
Nikolas Herbst. 2018. Methods and Benchmarks for Auto-Scaling Mechanisms in Elastic Cloud Environments. Ph.D. Dissertation. University of Würzburg, Germany.
[13]
Karl Huppler. 2009. The Art of Building a Good Benchmark. In Performance Evaluation and Benchmarking, Raghunath Nambiar and Meikel Poess (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 18--30.
[14]
Karl Huppler, Klaus-Dieter Lange, and John Beckett. 2012. SPEC: Enabling Efficiency Measurement. In Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE '12). Association for Computing Machinery, New York, NY, USA, 257--258.
[15]
J. Jang, M. Jeon, H. Kim, H. Jo, J. Kim, and S. Maeng. 2011. Energy Reduction in Consolidated Servers through Memory-Aware Virtual Machine Scheduling. IEEE Trans. Comput., Vol. 60, 4 (2011), 552--564.
[16]
Yichao Jin, Yonggang Wen, and Qinghua Chen. 2012. Energy Efficiency and Server Virtualization in Data Centers: An Empirical Investigation. In 2012 IEEE Conference on Computer Communications Workshops. 133--138.
[17]
Samuel Kounev, Klaus-Dieter Lange, and Jóakim von Kistowski. 2020. Systems Benchmarking 1 ed.). Springer International Publishing.
[18]
Eric Masanet, Arman Shehabi, Nuoa Lei, Sarah Smith, and Jonathan Koomey. 2020. Recalibrating global data center energy-use estimates. Science, Vol. 367, 6481 (2020), 984--986.
[19]
Ricardo Nobre, Lu'i s Reis, and Jo a o M. P. Cardoso. 2018. Compiler Phase Ordering as an Orthogonal Approach for Reducing Energy Consumption. CoRR, Vol. abs/1807.00638 (2018). arxiv: 1807.00638 http://arxiv.org/abs/1807.00638
[20]
Stuart Oskamp. 2000. A sustainable future for humanity? How can psychology help? The American psychologist, Vol. 55 5 (2000), 496--508.
[21]
G. Rocha, F. Castor, and G. Pinto. 2019. Comprehending Energy Behaviors of Java I/O APIs. In 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1--12.
[22]
Andrew R Sanderford, Andrew P. McCoy, and Matthew J. Keefe. 2017. Adoption of Energy Star certifications: theory and evidence compared. Building Research and Information (6 Jan. 2017), 1--13.
[23]
Norbert Schmitt, James Bucek, Klaus-Dieter Lange, and Samuel Kounev. 2020. Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite. In Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020). ACM, New York, NY, USA, 4.
[24]
Connie U. Smith. 2007. Introduction to Software Performance Engineering: Origins and Outstanding Problems .Springer Berlin Heidelberg, Berlin, Heidelberg, 395--428.
[25]
Christian Vögele, André van Hoorn, Eike Schulz, Wilhelm Hasselbring, and Helmut Krcmar. 2018. WESSBAS: extraction of probabilistic workload specifications for load testing and performance prediction--a model-driven approach for session-based application systems. Software & Systems Modeling, Vol. 17, 2 (2018), 443--477.
[26]
Jóakim von Kistowski, Jeremy A. Arnold, Karl Huppler, Klaus-Dieter Lange, John L. Henning, and Paul Cao. 2015. How to Build a Benchmark. In Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015) (ICPE '15). ACM, New York, NY, USA.
[27]
C. Murray Woodside, Greg Franks, and Dorina C. Petriu. 2007. The Future of Software Performance Engineering. Future of Software Engineering (FOSE '07) (2007), 171--187.

Cited By

View all
  • (2022)Systematic analysis of software development in cloud computing perceptionsJournal of Software: Evolution and Process10.1002/smr.248536:2Online publication date: 29-Jun-2022

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering
April 2021
198 pages
ISBN:9781450383318
DOI:10.1145/3447545
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2021

Check for updates

Author Tags

  1. benchmark
  2. cloud
  3. data center
  4. resource efficiency
  5. resource utilization
  6. software development
  7. software engineering
  8. sustainability

Qualifiers

  • Work in progress

Conference

ICPE '21

Acceptance Rates

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)16
  • Downloads (Last 6 weeks)1
Reflects downloads up to 01 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Systematic analysis of software development in cloud computing perceptionsJournal of Software: Evolution and Process10.1002/smr.248536:2Online publication date: 29-Jun-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media