Abstract
Function Block Diagram (FBD), one of the PLC programming languages, is a graphical modeling language which has been increasingly used to implement safety-critical software such as nuclear reactor protection software. With increased importance of structural testing for FBD models, FBD model-based test coverage criteria have been introduced. In this paper, we empirically evaluate the fault detection effectiveness of the FBD coverage criteria using mutation analysis. We produce 1800 test suites satisfying the FBD criteria and generate more than 600 mutants automatically for the target industrial FBD models. Then we evaluate mutant detection of the test suites to assess the fault detection effectiveness of the coverage criteria. Based on the experimental results, we analyze strengths and weaknesses of the FBD coverage criteria, and suggest possible improvements for the test coverage criteria.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
International Electrotechnical Commission: Programmable controllers : Part 3: Programming languages. IEC, Geneva (2003)
Doosan Heavy Industry and Construction: KNICS-RPS-SDS231-01, Rev. 01., Software Design Specification for the Bistable Processor of the Reactor Protection System (2006)
USNRC: Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants, Regulatory Guide 1.171 (1997)
Jee, E., Yoo, J., Cha, S., Bae, D.: A data flow-based structural testing technique for FBD programs. Information and Software Technology 51, 1131–1139 (2009)
Andrews, J.H., Briand, L.C., Labiche, Y., Namin, A.S.: Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria. IEEE Transactions on Software Engineering 32, 608–624 (2006)
King, K.N., Offutt, A.J.: A fortran language system for mutation-based software testing. Software: Practice and Experience 21, 685–718 (1991)
Jia, Y., Harman, M.: An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 649–678 (2011)
Hutchins, M., Foster, H., Goradia, T., Ostrand, T.: Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In: 16th International Conference on Software Engineering, pp. 191–200. IEEE Computer Society Press, Los Alamitos (1994)
Li, N., Praphamontripong, U., Offutt, A.J.: An experimental comparison of four unit test criteria: Mutation, edge-pair, all-uses and prime path coverage. In: International Conference on Software Testing, Verification and Validation Workshops, pp. 220–229. IEEE Press, New York (2009)
Offutt, J., Abdurazik, A.: Generating Tests from UML Specifications. In: France, R., Rumpe, B. (eds.) UML 1999. LNCS, vol. 1723, pp. 416–429. Springer, Heidelberg (1999)
Andrews, A., France, R., Ghosh, S., Craig, G.: Test adequacy criteria for UML design models. Software Testing, Verification and Reliability 13, 95–127 (2003)
Mcquillan, J.A., Power, J.F.: A survey of UML-based coverage criteria for software testing. Department of Computer Science. NUI Maynooth, Co. Kildare, Ireland (2005)
Lakehal, A., Parissis, I.: Structural test coverage criteria for lustre programs. In: 10th International Workshop on Formal Methods for Industrial Critical Systems, pp. 35–43. ACM Press, New York (2005)
Mader, A.: A classification of PLC models and applications. In: 5th International Workshop on Discrete Event Systems –Discrete Event Systems, Analysis and Control, pp. 239–247. Springer, Heidelberg (2000)
Jee, E.: A Data Flow-Based Structural Testing Technique for FBD Programs. Ph.D Thesis. KAIST Press, Republic of Korea (2009)
Yices SMT solver, http://yices.csl.sri.com
Oh, Y., Yoo, J., Cha, S., Seong Son, H.: Software safety analysis of function block diagrams using fault trees. Reliability Engineering & System Safety 88, 215–228 (2005)
Offutt, A.J., Lee, A., Rothermel, G., Untch, R.H., Zapf, C.: An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology 5, 99–118 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shin, D., Jee, E., Bae, DH. (2012). Empirical Evaluation on FBD Model-Based Test Coverage Criteria Using Mutation Analysis. In: France, R.B., Kazmeier, J., Breu, R., Atkinson, C. (eds) Model Driven Engineering Languages and Systems. MODELS 2012. Lecture Notes in Computer Science, vol 7590. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33666-9_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-33666-9_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33665-2
Online ISBN: 978-3-642-33666-9
eBook Packages: Computer ScienceComputer Science (R0)