Article

Metamorphic fault tolerance: an automated and systematic methodology for fault tolerance in the absence of test oracle

Authors:

Huai Liu,

Iman I. Yusuf,

Heinz W. Schmidt,

Tsong Yueh ChenAuthors Info & Claims

ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

Pages 420 - 423

https://doi.org/10.1145/2591062.2591109

Published: 31 May 2014 Publication History

Get Access

Abstract

A system may fail due to an internal bug or a fault in its execution environment. Incorporating fault tolerance strategies enables such system to complete its function despite the failure of some of its parts. Prior to the execution of some fault tolerance strategies, failure detection is needed. Detecting incorrect output, for instance, assumes the existence of an oracle to check the correctness of program outputs given an input. However, in many practical situations, oracle does not exist or is extremely difficult to apply. Such an oracle problem is a major challenge in the context of software testing. In this paper, we propose to apply metamorphic testing, a software testing method that alleviates the oracle problem, into fault tolerance. The proposed technique supports failure detection without the need of oracles.

References

[1]

P. E. Ammann and J. C. Knight. Data diversity: An approach to software fault tolerance. IEEE T Comput, 37(4):418–425, 1988.

Digital Library

Google Scholar

[2]

A. Avizienis. The N-version approach to fault-tolerant software. IEEE T Software Eng, 11(12):1491–1501, 1985.

Digital Library

Google Scholar

[3]

A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and secure computing. IEEE T Depend Secure, 1(1):11–33, 2004.

Digital Library

Google Scholar

[4]

T. Y. Chen, F.-C. Kuo, H. Liu, and S. Wang. Conformance testing of network simulators based on metamorphic testing technique. In Proceedings of FORTE’09, pages 243–248, 2009.

Digital Library

Google Scholar

[5]

T. Y. Chen, T. H. Tse, and Z. Zhou. Semi-proving: An integrated method for program proving, testing, and debugging. IEEE T Software Eng, 37(1):109–125, 2011.

Digital Library

Google Scholar

[6]

P. Hu, Z. Zhang, W. K. Chan, and T. H. Tse. An empirical comparison between direct and indirect test result checking approaches. In Proceedings of SOQUA’06, pages 6–13, 2006.

Digital Library

Google Scholar

[7]

J. C. Knight and N. G. Leveson. An experimental evaluation of the assumption of independence in multiversion programming. IEEE T Software Eng, 12(1):96–109, 1986.

Digital Library

Google Scholar

[8]

H. Liu, F.-C. Kuo, D. Towey, and T. Y. Chen. How effectively does metamorphic testing alleviate the oracle problem? IEEE T Software Eng, in press.

Google Scholar

[9]

G. J. Myers. The Art of Software Testing. John Wiley and Sons, second edition, 2004.

Digital Library

Google Scholar

[10]

D. S. Rosenblum. A practical approach to programming with assertions. IEEE T Software Eng, 21(1):19–31, 1995.

Digital Library

Google Scholar

[11]

C.-A. Sun, G. Wang, B. Mu, H. Liu, Z. Wang, and T. Y. Chen. Metamorphic testing for web services: Framework and a case study. In Proceedings of ICWS’11, pages 283–290, 2011.

Digital Library

Google Scholar

[12]

P. Wu. Iterative metamorphic testing. In Proceedings of COMPSAC’05, volume 1, pages 19–24, 2005.

Digital Library

Google Scholar

[13]

X. Xie, W. E. Wong, T. Y. Chen, and B. Xu. Metamorphic slice: An application in spectrum-based fault localization. Inform Software Tech, 55(5):866–879, 2013.

Digital Library

Google Scholar

[14]

I. I. Yusuf, H. W. Schmidt, and I. D. Peake. Architecture-based fault tolerance support for grid applications. In Proceedings of QoSA’11, pages 177–182, 2011.

Digital Library

Google Scholar

Cited By

View all

Zheng ZRen DLiu HChen TLi T(2024)Identifying the Failure-Revealing Test Cases in Metamorphic Testing: A Statistical ApproachACM Transactions on Software Engineering and Methodology10.1145/369599034:2(1-26)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3695990
Zheng ZRen DLiu HChen T(2024)Metamorphic Fault Tolerance: Addressing the Oracle Problem of Reliability Assurance for Contemporary Software SystemsComputer10.1109/MC.2024.339075957:7(77-86)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/MC.2024.3390759
Fu LLei YYan MXu LXu ZZhang X(2023)MetaFLIET Software10.1049/sfw2.1210217:2(137-153)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1049/sfw2.12102
Show More Cited By

Index Terms

Metamorphic fault tolerance: an automated and systematic methodology for fault tolerance in the absence of test oracle
1. General and reference
  1. Cross-computing tools and techniques
    1. Reliability
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software organization and properties
    1. Extra-functional properties
      1. Software reliability

Recommendations

Fault Tolerance in Multiprocessor Systems Without Dedicated Redundancy

An algorithm called RAFT (recursive algorithm for fault tolerance) for achieving fault tolerance in multiprocessor systems is described. Through the use of a combination of dynamic space- and time- redundancy techniques, RAFT achieves fault tolerance in ...
On the Hardness of Adding Nonmasking Fault Tolerance
This paper investigates the complexity of adding nonmasking fault tolerance, where a nonmasking fault-tolerant program guarantees recovery from states reached due to the occurrence of faults to states from where its specifications are satisfied. We first ...
Using dynamic task level redundancy for OpenMP fault tolerance
ARCS'12: Proceedings of the 25th international conference on Architecture of Computing Systems

Obtaining fault tolerant applications and systems is one of today's most important topics of research. Fault tolerance is becoming more and more essential in shared memory parallel programs and in multi/many core architectures due to the decreasing size ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

May 2014

741 pages

ISBN:9781450327688

DOI:10.1145/2591062

General Chair:
Pankaj Jalote
IIIT-Delhi, India
,
Program Chairs:
Lionel Briand
University of Luxembourg, Luxembourg
,
André van der Hoek
University of California, Irvine, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE '14

Sponsor:

SIGSOFT

ICSE '14: 36th International Conference on Software Engineering

May 31 - June 7, 2014

Hyderabad, India

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
226
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zheng ZRen DLiu HChen TLi T(2024)Identifying the Failure-Revealing Test Cases in Metamorphic Testing: A Statistical ApproachACM Transactions on Software Engineering and Methodology10.1145/369599034:2(1-26)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3695990
Zheng ZRen DLiu HChen T(2024)Metamorphic Fault Tolerance: Addressing the Oracle Problem of Reliability Assurance for Contemporary Software SystemsComputer10.1109/MC.2024.339075957:7(77-86)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/MC.2024.3390759
Fu LLei YYan MXu LXu ZZhang X(2023)MetaFLIET Software10.1049/sfw2.1210217:2(137-153)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1049/sfw2.12102
Rehman FIzurieta C(2022)An Approach For Verifying And Validating Clustering Based Anomaly Detection Systems Using Metamorphic Testing2022 IEEE International Conference On Artificial Intelligence Testing (AITest)10.1109/AITest55621.2022.00011(12-18)Online publication date: Aug-2022
https://doi.org/10.1109/AITest55621.2022.00011
Yin YRuan JLi YLi YPan Z(2021)Syntax‐based metamorphic relation prediction via the bagging frameworkExpert Systems10.1111/exsy.1290239:6Online publication date: 15-Dec-2021
https://doi.org/10.1111/exsy.12902
Guo HTao CHuang Z(2020)Metamorphic Testing for Plant Identification Mobile Applications Based on Test ContextsMobile Computing, Applications, and Services10.1007/978-3-030-64214-3_15(209-223)Online publication date: 19-Dec-2020
https://doi.org/10.1007/978-3-030-64214-3_15
Sun CFu APoon PXie XLiu HChen T(2019)METRIC+: A Metamorphic Relation Identification Technique Based on Input plus Output DomainsIEEE Transactions on Software Engineering10.1109/TSE.2019.2934848(1-1)Online publication date: 2019
https://doi.org/10.1109/TSE.2019.2934848
Patel KHierons R(2019)A partial oracle for uniformity statisticsSoftware Quality Journal10.1007/s11219-019-09459-027:4(1419-1447)Online publication date: 20-Aug-2019
https://doi.org/10.1007/s11219-019-09459-0
Chen TKuo FLiu HPoon PTowey DTse TZhou Z(2018)Metamorphic TestingACM Computing Surveys10.1145/314356151:1(1-27)Online publication date: 4-Jan-2018
https://dl.acm.org/doi/10.1145/3143561
Segura SFraser GSanchez ARuiz-Cortes A(2016)A Survey on Metamorphic TestingIEEE Transactions on Software Engineering10.1109/TSE.2016.253287542:9(805-824)Online publication date: 1-Sep-2016
https://doi.org/10.1109/TSE.2016.2532875
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Fault Tolerance in Multiprocessor Systems Without Dedicated Redundancy

On the Hardness of Adding Nonmasking Fault Tolerance

Using dynamic task level redundancy for OpenMP fault tolerance