More Web Proxy on the site http://driver.im/

article

Free access

Software error analysis: a real case study involving real faults and mutations

Authors:

Pascale Thévenod-FosseAuthors Info & Claims

ACM SIGSOFT Software Engineering Notes, Volume 21, Issue 3

Pages 158 - 171

https://doi.org/10.1145/226295.226313

Published: 01 May 1996 Publication History

Abstract

The paper reports on a first experimental comparison of software errors generated by real faults and by 1st-order mutations. The experiments were conducted on a program developed by a student from the industrial specification of a critical software from the civil nuclear field. Emphasis was put on the analysis of errors produced upon activation of 12 real faults by focusing on the mechanisms of error creation, masking, and propagation up to failure occurrence, and on the comparison of these errors with those created by 24 mutations. The results involve a total of 3730 errors recorded from program execution traces: 1458 errors were produced by the real faults, and the 2272 others by the mutations. They are in favor of a suitable consistency between errors generated by mutations and by real faults: 85% of the 2272 errors due to the mutations were also produced by the real faults. Moreover, it was observed that although the studied mutations were simple faults, they can create erroneous behaviors as complex as those identified for the real faults. This lends support to the representativeness of errors due to mutations.

References

[1]

Bishop P.G. and Pullen F.D., "Error Masking: a Source of Failure Dependency in Multi-Version Programs", Proc./st IFIP Working Conference on Dependable Computing for Critical Applications (DCCA-1), Santa-Barbara, USA, pp. 53-73, 1989.

[2]

DeMillo R.A., Lipton R.J. and Sayward F.G., "Hints on Test Data Selection: Help for the Practicing Programmer", Computer, vol. 11, no. 4, pp. 34-41, 1978.

Digital Library

[3]

DeMillo R.A. and Mathur A.P., "On the Use of Software Artifacts to Evaluate the Effectiveness of Mutation Analysis for Detecting Errors in Production Software", Software Engineering Research Center report, Purdue University, W. Lafayette, USA, 1994.

[4]

Goradia T., "Dynamic Impact Analysis : A Cost-effective Technique to Enforce Errorpropagation", Proc. 1 st International Symposium on Software Testing and Analysis (ISSTA), Cambridge, USA, pp. 171-181, 1993.

Digital Library

[5]

Laprie J.C. (Ed.), Dependability: Basic Concepts and Terminology, Springer-Verlag, vol. 5, Dependable Computing and Fault Tolerance Systems, Springer-Verlag, 1992.

Digital Library

[6]

Murrill B.W. and Morell L., "An Experimental Approach to Analyzing Software Semantics Using Error Flow Information", Proc. 2nd International Symposium on Software Testing and Analysis (ISSTA), Seattle, USA, p. 200, 1994.

Digital Library

[7]

Offutt A.J., "The Coupling Effect: Fact or Fiction?", Proc. 3rd Symposium on Testing, Analysis and Verification (TAV 3), Key West, USA, pp. 131-140, 1989.

Digital Library

[8]

Podgurski A. and Clarke L.A., "A Formal Model of Program Dependences and Its Implications for Software Testing, Debugging, and Maintenance", IEEE Transactions on Software Engineering, vol. 16, no. 9, pp. 965- 979, 1990.

Digital Library

[9]

Richardson D.j. and Thompson M.C., "The RELAY Model of Error Detection and its Application", Proc. 2nd Workshop on Software Testing, Verification and Analysis, Banff, Canada, pp. 223-230, 1988.

[10]

Richardson D.J. and Thompson M.C., "An Analysis of Test Data Selection Criteria Using the RELAY Model of Fault Detection", IEEE Transactions on Software Engineering, vol. 19, no. 6, pp. 533-553, 1993.

Digital Library

[11]

Th6venod-Fosse P. and Waeselynck H., "STATEMATE Applied to Statistical Software Testing", Proc. 1st International Symposium on Software Testing and Analysis (ISSTA), Cambridge, USA, pp. 99-109, 1993.

Digital Library

[12]

Th6venod-Fosse P. and Crouzet Y., "On the Adequacy of Functional Test Criteria Based on Software Behaviour Models", Proc. 5th IFIP Working Conference on Dependable Computing for Critical Applications (DCCA-5), Urbana- Champaign, USA, pp. 176-187, 1995.

[13]

Thompson M.C., Richardson D.J. and Clarke L.A., "An Information Flow Model of Fault Detection", Proc. 1st international Symposium on Software Testing and Analysis (ISSTA), Cambridge, USA, pp. 182-192, 1993.

Digital Library

[14]

Voas J.M. and Miller K.W., "The Revealing Power of a Test Case", Journal of Software Testing, Verification and Reliability, vol. 2, pp. 25-42, 1992.

[15]

Voas J.M., Morell L.M. and Miller K., "Predicting Where Faults Can Hide from Testing", IEEE Software, vol. 8, no. 2, pp. 41- 48, 1992.

Digital Library

[16]

Zeil S.J., "Perturbation Techniques for Detecting Domain Errors", IEEE Transactions on Software Engineering, vol. 15, no. 6, pp. 737-746, 1989.

Digital Library

Cited By

Kuznetsov KGambi ADhiddi SHess JGopinath R(2024)Empirical Evaluation of Frequency Based Statistical Models for Estimating Killable MutantsProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686669(61-71)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686669
Ahmed ZMakedonski PCombemale BWimmer MChechik MEgyed A(2024)Exploring the Fundamentals of Mutations in Deep Neural NetworksProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems10.1145/3652620.3687426(227-233)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3652620.3687426
Mansur RShaffer CEdwards SDorodchi MZhange MCooper S(2024)Mutating Matters: Analyzing the Influence of Mutation Testing in Programming CoursesProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 110.1145/3649165.3690110(151-157)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649165.3690110
Show More Cited By

Index Terms

Software error analysis: a real case study involving real faults and mutations
1. Software and its engineering
  1. Software creation and management

Recommendations

Software error analysis: a real case study involving real faults and mutations
ISSTA '96: Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis

The paper reports on a first experimental comparison of software errors generated by real faults and by 1st-order mutations. The experiments were conducted on a program developed by a student from the industrial specification of a critical software from ...
Systematic t-Error Correcting/All Unidirectional Error Detecting Codes

In this paper we give methods for the construction of systematic t-random error correcting and all unidirectional error detecting codes. Also we give the encoding/decoding algorithms and discuss their implementation.
Error injection aimed at fault removal in fault tolerance mechanisms-criteria for error selection using field data on software faults
ISSRE '96: Proceedings of the The Seventh International Symposium on Software Reliability Engineering

Fault injection allows a detailed study of complex interactions between faults and fault handling mechanisms. It can be a useful complement to analytical modeling and formal verification techniques in the testing of fault tolerant systems. However, work ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGSOFT Software Engineering Notes

ACM SIGSOFT Software Engineering Notes Volume 21, Issue 3

May 1996

293 pages

ISSN:0163-5948

DOI:10.1145/226295

Editors:
Steven J. Zeil
Old Dominion Univ., Norfolk, VA
,
Will Tracz
Loral Federal Systems, Owego, NY

Issue’s Table of Contents

ISSTA '96: Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis
May 1996
294 pages
ISBN:0897917871
DOI:10.1145/229000
Editors:
Steve J. Zeil
Old Dominion Univ., Norfolk, VA
,
Will Tracz
Loral Federal Systems, Owego, NY

Copyright © 1996 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 1996

Published in SIGSOFT Volume 21, Issue 3

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

129
Total Citations
View Citations
1,664
Total Downloads

Downloads (Last 12 months)198
Downloads (Last 6 weeks)23

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kuznetsov KGambi ADhiddi SHess JGopinath R(2024)Empirical Evaluation of Frequency Based Statistical Models for Estimating Killable MutantsProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686669(61-71)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686669
Ahmed ZMakedonski PCombemale BWimmer MChechik MEgyed A(2024)Exploring the Fundamentals of Mutations in Deep Neural NetworksProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems10.1145/3652620.3687426(227-233)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3652620.3687426
Mansur RShaffer CEdwards SDorodchi MZhange MCooper S(2024)Mutating Matters: Analyzing the Influence of Mutation Testing in Programming CoursesProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 110.1145/3649165.3690110(151-157)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649165.3690110
Du HPalepu VJones JRoychoudhury APaiva AAbreu RStorey M(2024)Ripples of a Mutation — An Empirical Study of Propagation Effects in Mutation TestingProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639179(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639179
Kim YChoi JYoo SPark M(2024)Propagation and Masking Characteristics of Errors During Software Testing in Synchronous ProgrammingIEEE Access10.1109/ACCESS.2024.346972912(146009-146021)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3469729
Mao RZhang LZhang X(2024)Mutation‐based data augmentation for software defect predictionJournal of Software: Evolution and Process10.1002/smr.263436:6Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1002/smr.2634
Gay GSalahirad A(2023)How Closely are Common Mutation Operators Coupled to Real Faults?2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00021(129-140)Online publication date: Apr-2023
https://doi.org/10.1109/ICST57152.2023.00021
Tian ZChen JZhu QYang JZhang L(2022)Learning to Construct Better Mutation FaultsProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3556949(1-13)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3556949
Kaufman SFeatherman RAlvin JKurtz BAmmann PJust RDwyer MDamian DZeller A(2022)Prioritizing mutants to guide mutation testingProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510187(1743-1754)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510187
Escobar-Velasquez CLinares-Vasquez MBavota GTufano MMoran KDi Penta MVendome CBernal-Cardenas CPoshyvanyk D(2022)Enabling Mutant Generation for Open- and Closed-Source Android AppsIEEE Transactions on Software Engineering10.1109/TSE.2020.298263848:1(186-208)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TSE.2020.2982638
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents