More Web Proxy on the site http://driver.im/

research-article

On the practical feasibility of software monitoring: a framework for low-impact execution tracing

Authors:

Ingrid NunesAuthors Info & Claims

SEAMS '19: Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems

Pages 169 - 180

https://doi.org/10.1109/SEAMS.2019.00030

Published: 25 May 2019 Publication History

Abstract

In order for a software system to self-adapt, it often needs to be aware of its behavior. A typical way of achieving this is by means of the runtime collection of execution traces, which requires the interception of the execution of, e.g. methods, and record information about them. Although this is simple in theory, in practice, it can be very costly because it might have a significant impact on the application performance or require huge amounts of memory or storage. This becomes a significant issue mainly in real-time applications, which are time-sensitive and must often meet deadlines in resource-constrained environments. We thus in this paper propose a two-phase tracing framework to cope with the monitoring overhead. In its first phase, the application is monitored in a lightweight fashion providing information for the second phase, which identifies an adaptive configuration and samples traces according to the current configuration. The adaptive configuration is determined by a set of criteria, specified through a proposed domain-specific language. We empirically evaluate our framework by instantiating it as a reduced-overhead monitoring solution, integrated into an existing automated application-level caching approach. We demonstrate that our solution reduces the overhead caused by monitoring, without compromising the performance improvements provided by the caching approach.

References

[1]

Y. Qin, C. Xu, P. Yu, and J. Lu, "SIT: Sampling-based interactive testing for self-adaptive apps," Journal of Systems and Software, vol. 120, pp. 70--88, 2016.

Digital Library

[2]

M. Tanabe, K. Tei, Y. Fukazawa, and S. Honiden, "Learning environment model at runtime for self-adaptive systems," in Proceedings of the Symposium on Applied Computing - SAC '17. New York, New York, USA: ACM Press, 2017, pp. 1198--1204. {Online}. Available: http://dl.acm.org/citation.cfm?doid=3019612.3019776

Digital Library

[3]

J. Mertz and I. Nunes, "Automation of application-level caching in a seamless way," Software - Practice and Experience, vol. 48, no. 6, pp. 1218--1237, 2018. {Online}. Available: http://onlinelibrary.wiley.com/doi/10.1002/spe.2571/abstract

[4]

Y. Feng, K. Dreef, J. A. Jones, and A. van Deursen, "Hierarchical abstraction of execution traces for program comprehension," in Proceedings of the 26th Conference on Program Comprehension - ICPC '18. Association for Computing Machinery ACM ACM, 2018, pp. 86--96. {Online}. Available: http://dl.acm.org/citation.cfm?doid=3196321.3196343

Digital Library

[5]

H. Pirzadeh, S. Shanian, A. Hamou-Lhadj, and A. Mehrabian, "The concept of stratified sampling of execution traces," in IEEE International Conference on Program Comprehension. IEEE, jun 2011, pp. 225--226. {Online}. Available: http://ieeexplore.ieee.org/document/5970192/

[6]

E. Yuan, N. Esfahani, and S. Malek, "Automated mining of software component interactions for self-adaptation," in Proceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems - SEAMS 2014. New York, New York, USA: ACM Press, jun 2014, pp. 27--36. {Online}. Available: http://dl.acm.org/citation.cfm?id=2593929.2593934http://dl.acm.org/citation.cfm?doid=2593929.2593934

[7]

G. Reger and K. Havelund, "What is a trace? A runtime verification perspective," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Cham, oct 2016, vol. 9953 LNCS, pp. 339--355.

[8]

V. Horký, J. Kotrč, P. Libic, and P. Tma, "Analysis of Overhead in Dynamic Java Performance Monitoring," in Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering - ICPE '16. New York, New York, USA: ACM Press, 2016, pp. 275--286. {Online}. Available: http://dl.acm.org/citation.cfm?doid=2851553.2851569

Digital Library

[9]

P. Kang, "Function call interception techniques," Software - Practice and Experience, vol. 48, no. 3, pp. 385--401, may 2018. {Online}. Available:

[10]

A. Miranskyy, A. Hamou-Lhadj, E. Cialini, and A. Larsson, "Operational-Log Analysis for Big Data Systems: Challenges and Solutions," IEEE Software, vol. 33, no. 2, pp. 52--59, 2016.

Digital Library

[11]

E. Bartocci, Y. Falcone, A. Francalanza, and G. Reger, "Introduction to runtime verification," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Cham, 2018, vol. 10457 LNCS, pp. 1--33.

[12]

L. Della Toffola, M. Pradel, and T. R. Gross, "Performance problems you can fix: a dynamic analysis of memoization opportunities," in Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications - OOPSLA 2015. New York, New York, USA: ACM Press, oct 2015, pp. 607--622. {Online}. Available

[13]

R. Wilhelm, T. Mitra, F. Mueller, I. Puaut, P. Puschner, J. Staschulat, P. Stenström, J. Engblom, A. Ermedahl, N. Holsti, S. Thesing, D. Whalley, G. Bernat, C. Ferdinand, and R. Heckmann, "The worst-case execution-time problem overview of methods and survey of tools," ACM Transactions on Embedded Computing Systems, vol. 7, no. 3, pp. 1--53, apr 2008. {Online}. Available: http://portal.acm.org/citation.cfm?doid=1347375.1347389

Digital Library

[14]

B. Cornelissen, A. Zaidman, A. van Deursen, L. Moonen, and R. Koschke, "A Systematic Survey of Program Comprehension through Dynamic Analysis," IEEE Transactions on Software Engineering, vol. 35, no. 5, pp. 684--702, sep 2009. {Online}. Available: http://ieeexplore.ieee.org/document/4815280/

Digital Library

[15]

P. Derbeko, S. Dolev, E. Gudes, and J. D. Ullman, "Concise essence-preserving big data representation," in Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. IEEE, dec 2016, pp. 3662--3665. {Online}. Available: http://ieeexplore.ieee.org/document/7841033/

[16]

R. Rabiser, S. Guinea, M. Vierhauser, L. Baresi, and P. Grünbacher, "A comparison framework for runtime monitoring approaches," Journal of Systems and Software, vol. 125, pp. 309--321, 2017. {Online}. Available: http://www.sciencedirect.com/science/article/pii/S0164121216302618

Digital Library

[17]

A. Chan, R. Holmes, G. C. Murphy, and A. T. Ying, "Scaling an object-oriented system execution visualizer through sampling," in Proceedings - IEEE Workshop on Program Comprehension, vol. 2003-May. IEEE Comput. Soc, 2003, pp. 237--244. {Online}. Available: http://ieeexplore.ieee.org/document/1199207/

Digital Library

[18]

P. Dugerdil, "Using trace sampling techniques to identify dynamic clusters of classes," in Proceedings of the 2007 Conference of the Center for Advanced Studies on Collaborative Research, CASCON '07. New York, New York, USA: ACM Press, 2007, pp. 306--314.

Digital Library

[19]

T. Apiwattanapong and M. J. Harrold, "Selective path profiling," ACM SIGSOFT Software Engineering Notes, 2003.

Digital Library

[20]

R. Santelices, S. Sinha, and M. J. Harrold, "Subsumption of program entities for efficient coverage and monitoring," Proceedings of the 3rd international workshop on Software quality assurance - SOQUA '06, 2006.

Digital Library

[21]

L. Fei and S. P. Midkiff, "Artemis," in Proceedings of the 2006 ACM SIGPLAN conference on Programming language design and implementation - PLDI '06, vol. 41, no. 6. New York, New York, USA: ACM Press, 2006, p. 84. {Online}. Available: http://portal.acm.org/citation.cfm?doid=1133981.1133992

Digital Library

[22]

M. Sridharan, S. J. Fink, and R. Bodik, "Thin Slicing," PLDI '07: Proceedings of the ACM SIGPLAN 2007 Conference on Programming Language Design and Implementation, 2007.

Digital Library

[23]

H. Narayanappa, M. S. Bansal, and H. Rajan, "Property-aware program sampling," in Proceedings of the 9th ACM SIGPLANSIGSOFT workshop on Program analysis for software tools and engineering PASTE 10. New York, New York, USA: ACM Press, 2010, p. 45. {Online}. Available: http://portal.acm.org/citation.cfm?doid=1806672.1806682

Digital Library

[24]

H. Pirzadeh, S. Shanian, A. Hamou-Lhadj, L. Alawneh, and A. Shafiee, "Stratified sampling of execution traces: Execution phases serving as strata," Science of Computer Programming, vol. 78, no. 8, pp. 1099--1118, aug 2013. {Online}. Available: http://linkinghub.elsevier.com/retrieve/pii/S0167642312002080

Digital Library

[25]

M. B. Dwyer, A. Kirmeer, and S. Elbaum, "Adaptive Online Program Analysis," 29th International Conference on Software Engineering (ICSE'07), 2007.

Digital Library

[26]

J. J. Thomas, S. Fischmeister, and D. Kumar, "Lowering overhead in sampling-based execution monitoring and tracing," ACM SIGPLAN Notices, vol. 46, no. 5, p. 101, apr 2011. {Online}. Available: http://dl.acm.org/citation.cfm?doid=2016603.1967692

Digital Library

[27]

S. Fischmeister and Y. Ba, "Sampling-based program execution monitoring," in ACM SIGPLAN Notices, vol. 45, no. 4. New York, New York, USA: ACM Press, 2010, p. 133. {Online}. Available: http://portal.acm.org/citation.cfm?doid=1755951.1755908

Digital Library

[28]

M. Hauswirth and T. M. Chilimbi, "Low-overhead memory leak detection using adaptive statistical profiling," in Proceedings of the 11th international conference on Architectural support for programming languages and operating systems - ASPLOS-XI, vol. 39, no. 11. New York, New York, USA: ACM Press, 2004, pp. 156--164. {Online}. Available: http://portal.acm.org.www.library.gatech.edu:2048/citation.cfm?id=1024393.1024412

Digital Library

[29]

H. Daoud, N. Ezzati-Jivan, and M. R. Dagenais, "Dynamic trace-based sampling algorithm for memory usage tracking of enterprise applications," in 2017 IEEE High Performance Extreme Computing Conference, HPEC 2017. IEEE, sep 2017, pp. 1--7. {Online}. Available: http://ieeexplore.ieee.org/document/8091061/

[30]

A. Hamou-Lhadj and T. C. Lethbridge, "A Survey of Trace Exploration Tools and Techniques," pp. 42--55, 2004. {Online}. Available: http://dl.acm.org/citation.cfm?id=1034914.1034918

Digital Library

[31]

S. Reiss and M. Renieris, "Encoding program executions," in Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001. IEEE Comput. Soc, 2001, pp. 221--230. {Online}. Available: http://ieeexplore.ieee.org/document/919096/

Digital Library

[32]

A. Hamou-Lhadj and T. C. Lethbridge, "Compression techniques to simplify the analysis of large execution traces," in Proceedings - IEEE Workshop on Program Comprehension, vol. 2002-Janua. IEEE Comput. Soc, 2002, pp. 159--168. {Online}. Available: http://ieeexplore.ieee.org/document/1021337/

Digital Library

[33]

S. Alimadadi, A. Mesbah, and K. Pattabiraman, "Inferring hierarchical motifs from execution traces," in Proceedings of the 40th International Conference on Software Engineering - ICSE '18. New York, New York, USA: ACM Press, 2018, pp. 776--787. {Online}. Available

Digital Library

[34]

K. Kouame, N. Ezzati-Jivan, and M. R. Dagenais, "A Flexible Data-Driven Approach for Execution Trace Filtering," in Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015. IEEE, jun 2015, pp. 698--703. {Online}. Available: http://ieeexplore.ieee.org/document/7207296/

Digital Library

[35]

A. Zaidman and S. Demeyer, "Managing trace data volume through a heuristical clustering process based on event execution frequency," Eighth European Conference on Software Maintenance and Reengineering, 2004. CSMR 2004. Proceedings., 2004.

Digital Library

[36]

S. P. Reiss, "Dynamic Detection and Visualization of Software Phases," Proceedings of the third international workshop on Dynamic analysis (WODA 2005), vol. 30, no. 4, p. 1, 2005.

Digital Library

[37]

A. Hamou-Lhadj and T. Lethbridge, "Summarizing the content of large traces to facilitate the understanding of the behaviour of a software system," in IEEE International Conference on Program Comprehension, vol. 2006, 2006, pp. 181--190.

Digital Library

[38]

B. Cornelissen, A. Zaidman, D. Holten, L. Moonen, A. van Deursen, and J. J. van Wijk, "Execution trace analysis through massive sequence and circular bundle views," Journal of Systems and Software, vol. 81, no. 12, pp. 2252--2268, 2008.

Digital Library

[39]

A. V. Miranskyy, N. H. Madhavji, M. S. Gittens, M. Davison, M. Wilding, D. Godwin, and C. A. Taylor, "Sift: A Scalable Iterative-Unfolding Technique for Filtering Execution Traces," in Proceedings of the 2008 conference of the center for advanced studies on collaborative research meeting of minds - CASCON '08. New York, New York, USA: ACM Press, 2008, p. 274. {Online}. Available: http://portal.acm.org/citation.cfm?doid=1463788.1463817

Digital Library

[40]

F. Wininger, N. Ezzati-Jivan, and M. R. Dagenais, "A declarative framework for stateful analysis of execution traces," Software Quality Journal, vol. 25, no. 1, pp. 201--229, mar 2017. {Online}. Available: http://link.springer.com/10.1007/s11219-016-9311-0

Digital Library

[41]

J. Hassine, A. Hamou-Lhadj, and L. Alawneh, "A framework for the recovery and visualization of system availability scenarios from execution traces," Information and Software Technology, vol. 96, pp. 78--93, apr 2018. {Online}. Available: https://www.sciencedirect.com/science/article/pii/S0950584917303166

[42]

H. Eichelberger and K. Schmid, "Flexible resource monitoring of Java programs," Journal of Systems and Software, 2014.

[43]

J. Mertz and I. Nunes, "Understanding Application-Level Caching in Web Applications," ACM Computing Surveys, vol. 50, no. 6, pp. 1--34, 2017. {Online}. Available: http://dl.acm.org/citation.cfm?doid=3161158.3145813

Digital Library

[44]

Z. Chen, B. Chen, L. Xiao, X. Wang, L. Chen, Y. Liu, and B. Xu, "Speedoo," in Proceedings of the 40th International Conference on Software Engineering - ICSE '18. New York, New York, USA: ACM Press, 2018, pp. 811--821. {Online}. Available: http://dl.acm.org/citation.cfm?doid=3180155.3180229

Digital Library

[45]

J. Mertz and I. Nunes, "A Qualitative Study of Application-Level Caching," IEEE Transactions on Software Engineering, vol. 43, no. 9, pp. 798--816, sep 2017. {Online}. Available /

Cited By

Reichelt DKühne SHasselbring WVieira MCardellini VDi Marco ATuma P(2023)Towards Solving the Challenge of Minimal Overhead MonitoringCompanion of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578245.3584851(381-388)Online publication date: 15-Apr-2023
https://dl.acm.org/doi/10.1145/3578245.3584851
Mertz JNunes I(2023)Software runtime monitoring with adaptive sampling rate to collect representative samples of execution tracesJournal of Systems and Software10.1016/j.jss.2023.111708202:COnline publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111708
da Silva MBezerra CCoutinho EMaia P(2021)A Catalog of Performance Measures for Self-Adaptive SystemsProceedings of the XX Brazilian Symposium on Software Quality10.1145/3493244.3493259(1-10)Online publication date: 8-Nov-2021
https://dl.acm.org/doi/10.1145/3493244.3493259

Recommendations

Software runtime monitoring with adaptive sampling rate to collect representative samples of execution traces
Abstract
Monitoring software systems at runtime is key for understanding workloads, debugging, and self-adaptation. It typically involves collecting and storing observable software data, which can be analyzed online or offline. Despite the usefulness of ...
Highlights
- An adaptive process to dynamically adjust the sampling rate in software monitoring.
- Algorithms with statistical foundations to improve the representativeness of samples.
- The sample error achieved by the approach is 9%–54% lower ...
Sampling-based program execution monitoring
LCTES '10: Proceedings of the ACM SIGPLAN/SIGBED 2010 conference on Languages, compilers, and tools for embedded systems

For its high overall cost during product development, program debugging is an important aspect of system development. Debugging is a hard and complex activity, especially in time-sensitive systems which have limited resources and demanding timing ...
Sampling-based program execution monitoring
LCTES '10

For its high overall cost during product development, program debugging is an important aspect of system development. Debugging is a hard and complex activity, especially in time-sensitive systems which have limited resources and demanding timing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SEAMS '19: Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems

May 2019

223 pages

General Chair:
Marin Litoiu
York University, Canada
,
Program Chairs:
Siobhán Clarke
Trinity College Dublin, Ireland
,
Kenji Tei
Waseda University / National Institute of Informatics, Japan, Japan

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

IEEE Press

Publication History

Published: 25 May 2019

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '19

Sponsor:

SIGSOFT
IEEE-CS

ICSE '19: 41st International Conference on Software Engineering

May 25 - 26, 2019

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 17 of 31 submissions, 55%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
83
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Reichelt DKühne SHasselbring WVieira MCardellini VDi Marco ATuma P(2023)Towards Solving the Challenge of Minimal Overhead MonitoringCompanion of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578245.3584851(381-388)Online publication date: 15-Apr-2023
https://dl.acm.org/doi/10.1145/3578245.3584851
Mertz JNunes I(2023)Software runtime monitoring with adaptive sampling rate to collect representative samples of execution tracesJournal of Systems and Software10.1016/j.jss.2023.111708202:COnline publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111708
da Silva MBezerra CCoutinho EMaia P(2021)A Catalog of Performance Measures for Self-Adaptive SystemsProceedings of the XX Brazilian Symposium on Software Quality10.1145/3493244.3493259(1-10)Online publication date: 8-Nov-2021
https://dl.acm.org/doi/10.1145/3493244.3493259

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents