More Web Proxy on the site http://driver.im/

research-article

A framework for measurement based performance modeling

Authors:

Dharmesh Thakkar,

Ahmed E. Hassan,

Gilbert Hamann,

Parminder FloraAuthors Info & Claims

WOSP '08: Proceedings of the 7th international workshop on Software and performance

Pages 55 - 66

https://doi.org/10.1145/1383559.1383567

Published: 23 June 2008 Publication History

Abstract

Techniques for performance modeling are broadly classified into measurement, analytical and simulation based techniques. Measurement based performance modeling is commonly adopted in practice. Measurement based modeling requires the execution of a large number of performance tests to build accurate performance models. These performance tests must be repeated for every release or build of an application. This is a time consuming and error-prone manual process.

In this paper, we present a framework for the systematic and automated building of measurement based performance models. The framework is based on our experience in performance modeling of two large applications: the DVD Store application by Dell and another larger enterprise application. We use the Dell DVD Store application as a running example to demonstrate the various steps in our framework. We present the benefits and shortcomings of our framework. We discuss the expected reduction in effort due to adopting our framework.

References

[1]

Sankarasetty, J., Mobley, K., Foster, L., Hammer, T., and Calderone, T. 2007. Software performance in the real world: personal lessons from the performance trauma team. In Proceedings of the 6th international Workshop on Software and Performance (Buenes Aires, Argentina, February 05 - 08, 2007). WOSP '07. ACM, New York, NY, 201--208.

Digital Library

[2]

Jaffe, D., Muirhead T. 2005. The Open Source DVD Store Application. http://www.dell.com/downloads/global/power/ps3q05-20050217-Jaffe-OE.pdf

[3]

Goldsmith, S. F., Aiken, A. S., and Wilkerson, D. S. 2007. Measuring empirical computational complexity. In Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (Dubrovnik, Croatia, September 03 - 07, 2007). ESEC-FSE '07. ACM, New York, NY, 395--404.

Digital Library

[4]

Jain R. 1992. The art of computer systems performance analysis. John Wiley.

[5]

Yang, J., Evans, D., Bhardwaj, D., Bhat, T., and Das, M. 2006. Perracotta: mining temporal API rules from imperfect traces. In Proceeding of the 28th international Conference on Software Engineering (Shanghai, China, May 20 - 28, 2006). ICSE '06. ACM, New York, NY, 282--291.

Digital Library

[6]

Muirhead T., Jaffe, D. 2005. Migrating enterprise databases from Sun servers to the Dell PowerEdge 2850 running Microsoft Windows Server 2003. http://www.dell.com/downloads/global/power/ps1q05-20040270-Jaffe.pdf

[7]

Woodside, M., Franks, G., and Petriu, D. C. 2007. The Future of Software Performance Engineering. In 2007 Future of Software Engineering (May 23 - 25, 2007). International Conference on Software Engineering. IEEE Computer Society, Washington, DC, 171--187.

Digital Library

[8]

Gunther, N. J. 2006 Guerrilla Capacity Planning: a Tactical Approach to Planning for Highly Scalable Applications and Services. Springer-Verlag New York, Inc.

Digital Library

[9]

Sopitkamol, M. and Menascé, D. A. 2005. A method for evaluating the impact of software configuration parameters on e-commerce sites. In Proceedings of the 5th international Workshop on Software and Performance (Palma, Illes Balears, Spain, July 12 - 14, 2005). WOSP '05. ACM, New York, NY, 53--64.

Digital Library

[10]

Yilmaz, C., Krishna, A. S., Memon, A., Porter, A., Schmidt, D. C., Gokhale, A., and Natarajan, B. 2005. Main effects screening: a distributed continuous quality assurance process for monitoring performance degradation in evolving software systems. In Proceedings of the 27th international Conference on Software Engineering (St. Louis, MO, USA, May 15 - 21, 2005). ICSE '05. ACM, New York, NY, 293--302.

Digital Library

[11]

Research In Motion. Capacity calculator for BlackBerry Enterprise Server 4.1 for Microsoft Exchange. http://www.blackberry.com/select/toolkit/dls/BlackBerry_Enterprise_Server_Version_4.1.0_for_Microsoft_Exchange_Capacity_Calculator.xls

[12]

Mania D. and Murphy J. 2002. Framework for predicting the performance of component-based systems. In Proceedings of IEEE 10th International Conference on Software, Telecommunications and Computer Networks (Italy, October 2002). SoftCOM 2002. pp. 46--50, ISBN 953 6114 52 6.

[13]

WebLOAD load testing stress testing tool. http://www.webload.org/

[14]

HP LoadRunner Software. https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-126-17%5E8_4000_100__

[15]

http://www.clarkware.com/software/JUnitPerf.html

[16]

http://www.junit.org

[17]

Israr, T. A., Lau, D. H., Franks, G., and Woodside, M. 2005. Automatic generation of layered queuing software performance models from commonly available traces. In Proceedings of the 5th international Workshop on Software and Performance (Palma, Illes Balears, Spain, July 12 - 14, 2005). WOSP '05. ACM, New York, NY, 147--158.

Digital Library

[18]

S. E. Sim. 1998. Supporting multiple program comprehension strategies during software maintenance. Masters thesis, University of Toronto, 1998.

[19]

Research In Motion. BlackBerry Enterprise Server for Microsoft Exchange version 4.1 performance benchmarking. http://www.blackberry.com/knowledgecenterpublic/livelink.exe/fetch/2000/8067/645045/7963/7965/1180408/Performance_Benchmarking_Guide.pdf?nodeid=1367404&vernum=0

[20]

Pentakalos, O. and Friedman, M. 2002. Windows 2000 Performance Guide: Help for Windows 2000 Administrators. UMI Order Number: 4665., O'Reilly & Associates, Inc.

Digital Library

[21]

Smith, C. U., Lladó, C. M., Cortellessa, V., Marco, A. D., and Williams, L. G. 2005. From UML models to software performance results: an SPE process based on XML interchange formats. In Proceedings of the 5th international Workshop on Software and Performance (Palma, Illes Balears, Spain, July 12 - 14, 2005). WOSP '05. ACM, New York, NY, 87--98.

Digital Library

[22]

Xie, T., Marinov, D., and Notkin, D. 2004. Rostra: A Framework for Detecting Redundant Object-Oriented Unit Tests. In Proceedings of the 19th IEEE international Conference on Automated Software Engineering (September 20 - 24, 2004). Automated Software Engineering. IEEE Computer Society, Washington, DC, 196--205.

Digital Library

[23]

Rothermel, G. and Harrold, M. J. 1997. A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6, 2 (Apr. 1997), 173--210.

Digital Library

[24]

The R project for statistical computing. http://www.r-project.org/

[25]

Juran, J. M., Godfrey A. B. 1988. Juran's Quality Handbook. McGraw-Hill Professional.

Cited By

Alipour MMoghaddam MVaidhyanathan KKjærgaard M(2023)Emoticontrol: Emotions-based Control of User-Interfaces AdaptationsProceedings of the ACM on Human-Computer Interaction10.1145/35932277:EICS(1-29)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3593227
Tarvo AReiss S(2018)Automatic performance prediction of multithreaded programsAutomated Software Engineering10.1007/s10515-017-0214-525:1(101-155)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1007/s10515-017-0214-5
Divan M(2017)Data-driven decision making2017 International Conference on Infocom Technologies and Unmanned Systems (Trends and Future Directions) (ICTUS)10.1109/ICTUS.2017.8285973(50-56)Online publication date: Dec-2017
https://doi.org/10.1109/ICTUS.2017.8285973
Show More Cited By

Index Terms

A framework for measurement based performance modeling
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
      1. Modeling methodologies
2. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
    2. Metrics

Recommendations

Automated analysis of multithreaded programs for performance modeling
Performance evaluation review

We present an approach for building performance models of multithreaded programs automatically. We use a combination of static and a dynamic analyses of a single representative run of the program to build its model. The model can predict performance of ...
Rule-based automatic software performance diagnosis and improvement

There are many advantages to analyzing performance at the design level, rather than waiting until system testing. However the necessary expertise in making and interpreting performance models may not be available, and the time for the analysis may be ...
Eclipse modeling framework for document management
DocEng '05: Proceedings of the 2005 ACM symposium on Document engineering

The lifecycle of document management applications typically comprises a set of loosely coupled subsystems that provide capture, index, search, workflow, fulfillment and archival features. However, there exists no standard model for composing these ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WOSP '08: Proceedings of the 7th international workshop on Software and performance

June 2008

218 pages

ISBN:9781595938732

DOI:10.1145/1383559

General Chair:
Alberto Avritzer
Siemens Corporate Research, USA
,
Program Chairs:
Elaine Weyuker
ATT Labs - Research, USA
,
Murray Woodside
Carleton University, Canada

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WOSP '08

Sponsor:

WOSP '08: Workshop on Software and Performance

June 23 - 26, 2008

NJ, Princeton, USA

Acceptance Rates

Overall Acceptance Rate 149 of 241 submissions, 62%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
510
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Alipour MMoghaddam MVaidhyanathan KKjærgaard M(2023)Emoticontrol: Emotions-based Control of User-Interfaces AdaptationsProceedings of the ACM on Human-Computer Interaction10.1145/35932277:EICS(1-29)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3593227
Tarvo AReiss S(2018)Automatic performance prediction of multithreaded programsAutomated Software Engineering10.1007/s10515-017-0214-525:1(101-155)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1007/s10515-017-0214-5
Divan M(2017)Data-driven decision making2017 International Conference on Infocom Technologies and Unmanned Systems (Trends and Future Directions) (ICTUS)10.1109/ICTUS.2017.8285973(50-56)Online publication date: Dec-2017
https://doi.org/10.1109/ICTUS.2017.8285973
Malik HShakshuki E(2017)Performance evaluation of counter selection techniques to detect discontinuity in large-scale-systemsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-017-0525-19:1(43-59)Online publication date: 4-Jul-2017
https://doi.org/10.1007/s12652-017-0525-1
Malik HShakshuki E(2017)Detecting performance anomalies in large-scale software systems using entropyPersonal and Ubiquitous Computing10.1007/s00779-017-1036-y21:6(1127-1137)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1007/s00779-017-1036-y
Pachidi SSpruit M(2016)The Performance Mining MethodBig Data10.4018/978-1-4666-9840-6.ch009(181-199)Online publication date: 2016
https://doi.org/10.4018/978-1-4666-9840-6.ch009
Malik HShakshuki E(2016)Towards Identifying Performance AnomaliesProcedia Computer Science10.1016/j.procs.2016.04.14083(621-627)Online publication date: 2016
https://doi.org/10.1016/j.procs.2016.04.140
Malik HDavis IGodfrey MNeuse DManskovskii S(2016)Connecting the dots: anomaly and discontinuity detection in large-scale systemsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-016-0381-47:4(509-522)Online publication date: 16-Jun-2016
https://doi.org/10.1007/s12652-016-0381-4
Pachidi SSpruit M(2015)The Performance Mining MethodInternational Journal of Business Intelligence Research10.4018/IJBIR.20150101026:1(11-29)Online publication date: Jan-2015
https://doi.org/10.4018/IJBIR.2015010102
Ekwem ENisar K(2014)An Experimental StudyInternational Journal of Advanced Pervasive and Ubiquitous Computing10.4018/ijapuc.20140701036:3(35-53)Online publication date: 1-Jul-2014
https://doi.org/10.4018/ijapuc.2014070103
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents