More Web Proxy on the site http://driver.im/

research-article

Concern-driven Reporting of Software Performance Analysis Results

Authors:

Dušan Okanović,

André van Hoorn,

Christoph Zorn,

Vincenzo Ferme,

Jürgen WalterAuthors Info & Claims

ICPE '19: Companion of the 2019 ACM/SPEC International Conference on Performance Engineering

Pages 1 - 4

https://doi.org/10.1145/3302541.3313103

Published: 27 March 2019 Publication History

Abstract

State-of-the-art approaches for reporting performance analysis results rely on charts providing insights on the performance of the system, often organized in dashboards. The insights are usually data-driven, i.e., not directly connected to the performance concern leading the users to execute the performance engineering activity, thus limiting the understandability of the provided result. A cause is that the data is presented without further explanations.

To solve this problem, we propose a concern-driven approach for reporting of performance evaluation results, shaped around a performance concern stated by a stakeholder and captured by state-of-the-art declarative performance engineering specifications. Starting from the available performance analysis, the approach automatically generates a customized performance report providing a chart- and natural-language-based answer to the concern. In this paper, we introduce the general concept of concern-driven performance analysis reporting and present a first prototype implementation of the approach. We envision that, by applying our approach, reports tailored to user concerns reduce the effort to analyze performance evaluation results.

References

[1]

ISO/IEC/IEEE Systems and software engineering -- Architecture description. ISO/IEC/IEEE 42010:2011(E), 2011.

[2]

S. Baltes, O. Moseler, F. Beck, and S. Diehl. Navigate, understand, communicate: How developers locate performance bugs. In Proc. Int. Symp. on Empirical Soft. Eng. and Measurement, ESEM, pages 1--10, 2015.

[3]

F. Beck, O. Moseler, S. Diehl, and G. D. Rey. In situ understanding of performance bottlenecks through visually augmented code. In Proc. Int. Conf. on Program Comprehension, ICPC, pages 63--72, 2013.

[4]

F. Beck, H. A. Siddiqui, A. Bergel, and D. Weiskopf. Method Execution Reports: Generating text and visualization to describe program behavior. In Proc. IEEE Working Conf. on Soft. Visualization, VISSOFT, pages 1--10, 2017.

[5]

L. F. Cortés-Coy, M. Linares-Vásquez, J. Aponte, and D. Poshyvanyk. On automatically generating commit messages via summarization of source code changes. In Proc. Int. Working Conf. on Source Code Analysis and Manipulation, SCAM, pages 275--284, 2014.

Digital Library

[6]

A. Gatt and E. Krahmer. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. Journal of Artificial Intelligence Research, 61:65--170, 2018.

Digital Library

[7]

C. Haight and F. D. Silva. Gartner's magic quadrant for application performance monitoring suites. http://www.gartner.com/, 2016.

[8]

C. Heger, A. van Hoorn, M. Mann, and D. Okanovic. Application performance management: State of the art and challenges for the future. In Proc. Int. Conf. on Perf. Eng., ICPE, pages 429--432. ACM, 2017.

Digital Library

[9]

K. E. Isaacs, A. Giménez, I. Jusufi, T. Gamblin, A. Bhatele, M. Schulz, B. Hamann, and P.-T. Bremer. State of the Art of Performance Visualization. In EuroVis - STARs, 2014.

[10]

Z. M. Jiang and A. E. Hassan. A Survey on Load Testing of Large-Scale Software Systems. IEEE Transactions on Software Engineering, 41(11):1091--1118, 2015.

[11]

P. W. McBurney and C. McMillan. Automatic documentation generation via source code summarization of method context. In Proc. Int. Conf. on Program Comprehension, ICPC, pages 279--290. ACM, 2014.

Digital Library

[12]

L. Moreno, J. Aponte, G. Sridhara, A. Marcus, L. Pollock, and K. Vijay-Shanker. Automatic generation of natural language summaries for Java classes. In Proc. IEEE Int. Conf. on Program Comprehension, ICPC, pages 23--32. IEEE, 2013.

[13]

L. Moreno, G. Bavota, M. Di Penta, R. Oliveto, A. Marcus, and G. Canfora. Automatic generation of release notes. In Proc. Int. Symp. on Foundations of Soft. Eng., FSE, pages 484--495. ACM, 2014.

Digital Library

[14]

D. Okanoviç, A. van Hoorn, C. Heger, A. Wert, and S. Siegl. Towards performance tooling interoperability: An open format for representing execution traces. In Proc. Eur. Workshop Comp. Perf. Eng., pages 94--108, 2016.

[15]

H. Schulz, D. Okanović, A. van Hoorn, V. Ferme, and C. Pautasso. Behavior-driven load testing using contextual knowledge--approach and experiences. In Proc. Int. Conf. on Perf. Eng., 2019.

Digital Library

[16]

G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker. Towards automatically generating summary comments for Java methods. In Proc. IEEE/ACM Int. Conf. on Automated Soft. Eng., ASE, pages 43--52, 2010.

Digital Library

[17]

J. Walter, S. Eismann, J. Grohmann, D. Okanović, and S. Kounev. Tools for declarative performance engineering. In Int. Conf. on Perf. Eng., pages 53--56, 2018.

Digital Library

[18]

J. Walter, M. König, S. Eismann, and S. Kounev. PAVO: A Framework for the Visualization of Performance Analyses Results. In Proc. Symp. on Soft. Perf., SSP, 2016.

[19]

J. Walter, A. van Hoorn, and S. Kounev. Automated and adaptable decision support for software performance engineering. In Proc. Int. Conf. on Perf. Evaluation Methodologies and Tools, VALUETOOLS, pages 66--73, 2017.

Digital Library

[20]

J. Walter, A. van Hoorn, H. Koziolek, D. Okanoviç, and S. Kounev. Asking what?, automating the how?: The vision of declarative performance engineering. In Proc. Int. Conf. on Perf Eng., ICPE, pages 91--94. ACM, 2016.

Digital Library

Cited By

Frank SBrott JKesim DHolz HEschhold Mvan Hoorn AVieira MCardellini VDi Marco ATuma P(2023)dqualizer: Domain-Centric Runtime Quality Analysis of Business-Critical Application SystemsCompanion of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578245.3584853(397-404)Online publication date: 15-Apr-2023
https://dl.acm.org/doi/10.1145/3578245.3584853
Okanović DBeck SMerz LZorn CMerino Lvan Hoorn ABeck FAmaral JKoziolek ATrubiani CIosup A(2020)Can a Chatbot Support Software Engineers with Load Testing? Approach and ExperiencesProceedings of the ACM/SPEC International Conference on Performance Engineering10.1145/3358960.3375792(120-129)Online publication date: 20-Apr-2020
https://dl.acm.org/doi/10.1145/3358960.3375792
Trubiani CAleti AGoodwin SJamshidi Pvan Hoorn AGratzl S(2020)VisArch: Visualisation of Performance-based Architectural RefactoringsSoftware Architecture10.1007/978-3-030-58923-3_12(182-190)Online publication date: 8-Sep-2020
https://doi.org/10.1007/978-3-030-58923-3_12
Show More Cited By

Index Terms

Concern-driven Reporting of Software Performance Analysis Results
1. Human-centered computing
  1. Visualization
    1. Visualization application domains
      1. Information visualization
2. Software and its engineering
  1. Software organization and properties
    1. Extra-functional properties
      1. Software performance
      2. Software usability

Recommendations

Tools for Declarative Performance Engineering
ICPE '18: Companion of the 2018 ACM/SPEC International Conference on Performance Engineering

Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper,...
Performance engineering for software architectures
COMPSAC '97: Proceedings of the 21st International Computer Software and Applications Conference

Software Performance Engineering (SPE) is a method for constructing software systems that meet performance goals. SPE includes techniques for gathering data, coping with uncertainty, constructing and evaluating performance models, evaluating ...
Software performance in the real world: personal lessons from the performance trauma team
WOSP '07: Proceedings of the 6th international workshop on Software and performance

In the nine years that we have been involved in software performance engineering (SPE) and performance testing engagements we have learned several things. Across numerous eCommerce applications and an enterprise CRM product suite, our knowledge base ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '19: Companion of the 2019 ACM/SPEC International Conference on Performance Engineering

March 2019

99 pages

ISBN:9781450362863

DOI:10.1145/3302541

General Chairs:
Varsha Apte
IIT Bombay, India
,
Antinisca Di Marco
University of L'Aquila, Italy
,
Program Chairs:
Marin Litoiu
York University, Canada
,
José Merseguer
Universidad de Zaragoza, Spain

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

German Research Foundation
German Federal Ministry of Education and Research
Baden-Württemberg Stiftung
Swiss National Science Foundation
Deutsche Forschungsgemeinschaft

Conference

ICPE '19

Sponsor:

ICPE '19: Tenth ACM/SPEC International Conference on Performance Engineering

April 7 - 11, 2019

Mumbai, India

Acceptance Rates

Overall Acceptance Rate 252 of 851 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Frank SBrott JKesim DHolz HEschhold Mvan Hoorn AVieira MCardellini VDi Marco ATuma P(2023)dqualizer: Domain-Centric Runtime Quality Analysis of Business-Critical Application SystemsCompanion of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578245.3584853(397-404)Online publication date: 15-Apr-2023
https://dl.acm.org/doi/10.1145/3578245.3584853
Okanović DBeck SMerz LZorn CMerino Lvan Hoorn ABeck FAmaral JKoziolek ATrubiani CIosup A(2020)Can a Chatbot Support Software Engineers with Load Testing? Approach and ExperiencesProceedings of the ACM/SPEC International Conference on Performance Engineering10.1145/3358960.3375792(120-129)Online publication date: 20-Apr-2020
https://dl.acm.org/doi/10.1145/3358960.3375792
Trubiani CAleti AGoodwin SJamshidi Pvan Hoorn AGratzl S(2020)VisArch: Visualisation of Performance-based Architectural RefactoringsSoftware Architecture10.1007/978-3-030-58923-3_12(182-190)Online publication date: 8-Sep-2020
https://doi.org/10.1007/978-3-030-58923-3_12
Schulz HOkanović Dvan Hoorn AFerme VPautasso CApte VDi Marco ALitoiu MMerseguer J(2019)Behavior-driven Load Testing Using Contextual Knowledge - Approach and ExperiencesProceedings of the 2019 ACM/SPEC International Conference on Performance Engineering10.1145/3297663.3309674(265-272)Online publication date: 4-Apr-2019
https://dl.acm.org/doi/10.1145/3297663.3309674
De Gyves Avila SCano PMejia AMoreno ILepe A(2019)A Data Driven Platform for Improving Performance Assessment of Software Defined Storage SolutionsTrends and Applications in Software Engineering10.1007/978-3-030-33547-2_20(266-275)Online publication date: 17-Oct-2019
https://doi.org/10.1007/978-3-030-33547-2_20

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents