Abstract
The question of when different programmers tend to commit the same errors is a critical issue for achieving fault diversity in fault tolerance. This problem is interdisciplinary and related to theories of human error in cognitive psychology. This paper proposes a psychological framework that combines Rasmussen’s performance levels with cross-level errors, represented by post-completion error, to model situations in which different programmers are prone to making the same errors. To validate the framework, we conducted an experiment where 200 student programmers independently solved the same problem, with the same tool and language. The results indicate that programmers unlikely commit the same errors in skill-based performances, most likely make the same errors in rule-based performances. These findings suggest that natural independent development may be less effective in preventing common errors in functions involving rule-based performance and post-completion scenarios, whereas it could be effective in preventing common errors in skill-based and knowledge-based performances. The results provided new insights into the strategies for avoiding coincident faults in N-version programming, from a human factor perspective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lyu, M.R.: Handbook of Software Reliability Engineering. IEEE Computer Society Press, California (1996)
Littlewood, B., Popov, P., Strigini, L.: Modeling software design diversity: a review. ACM Comput. Surv. 33, 177–208 (2001)
Lyu, M.R., Chen, J.-H., Avizienis, A.: Experience in metrics and measurements for N-version programming. Int. J. Reliab. Qual. Saf. Eng. 1, 41–62 (1994)
Xu, H., Chen, Z., Wu, W., Jin, Z., Kuo, S.-y., Lyu, M.: NV-DNN: towards fault-tolerant DNN systems with N-version programming. In: 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 44–47. IEEE (2019)
Levitin, G., Xing, L., Xiang, Y.: Optimal early warning defense of N-version programming service against co-resident attacks in cloud system. Reliab. Eng. Syst. Saf. 201, 106969 (2020)
Eckhardt, D.E., et al.: An experimental evaluation of software redundancy as a strategy for improving reliability. IEEE Trans. Software Eng. 17(7), 692–702 (1991). https://doi.org/10.1109/32.83905
John, C.K., Leveson, N.G.: An experimental evaluation of the assumption of independence in multi-version programming. IEEE Trans. Software Eng. 12, 96–109 (1986)
Avzenis, A., Lyu, M.R., Schutz, W.: In search of effective diversity: a six-language study of fault-tolerant flight control software. In: Proceedings of the 18th International Symposium on Fault-Tolerant Computing, pp. 15–22. (1988)
Feldt, R.: Generating diverse software versions with genetic programming: an experimental study. IEE Proc., Softw. 145, 228–236 (1998)
Yoo, C.S., Seong, P.H.: Experimental analysis of specification language diversity impact on NPP software diversity. J. Syst. Softw. 62, 111–122 (2002)
Meine, J.P., van der Meulen, M.A.R.: Correlations between internal software metrics and software dependability in a large population of small C/C++ programs. In: 18th IEEE International Symposium on Software Reliability Engineering, pp. 203–208. IEEE Computer Society (2007)
Reason, J.: Human Error. Cambridge University Press, Cambridge, UK (1990)
Rasmussen, J.: Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans. Syst. Man Cybern. 13, 257–266 (1983)
Byrne, M.D., Bovair, S.: A working memory model of a common procedural error. Cogn. Sci. 21, 31–61 (1997)
Palanque, P., Cockburn, A., Gutwin, C.: A classification of faults covering the human-computer interaction loop. In: 39th International Conference Computer Safety, Reliability, and Security, pp. 434–448. Springer (2020)
Mohanani, R., Salman, I., Turhan, B., Rodríguez, P., Ralph, P.: Cognitive biases in software engineering: a systematic mapping study. IEEE Trans. Software Eng. 46, 1318–1339 (2018)
Huang, F., Liu, B., Huang, B.: A taxonomy system to identify human error causes for software defects. In: The 18th international conference on reliability and quality in design, pp. 44–49. International Society of Science and Applied Technologies (2012)
Huang, F., Strigini, L.: HEDF: a method for early forecasting software defects based on human error mechanisms. IEEE Access 11, 3626–3652 (2023)
Li, S.Y., Blandford, A., Cairns, P., Young, R.M.: Post-completion errors in problem solving. In: Proceedings of the Twenty-Seventh Annual Conference of the Cognitive Science Society. Citeseer (2005)
Huang, F.: Post-completion error in software development. In: The 9th International Workshop on Cooperative and Human Aspects of Software Engineering, ICSE 2016, pp. 108–113. ACM (2016)
Gisev, N., Bell, J.S., Chen, T.F.: Interrater agreement and interrater reliability: key concepts, approaches, and applications. Res. Social Adm. Pharm. 9, 330–338 (2013)
Popov, P., Stankovic, V., Strigini, L.: An empirical study of the effectiveness of “forcing” diversity based on a large population of diverse programs. In: 23rd International Symposium on Software Reliability Engineering, pp. 41–50. IEEE (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, B., Song, Y., Xu, W., Huang, F. (2023). A Cognitive Framework for Modeling Coincident Software Faults: An Experimental Study. In: Guiochet, J., Tonetta, S., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2023. Lecture Notes in Computer Science, vol 14181. Springer, Cham. https://doi.org/10.1007/978-3-031-40923-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-40923-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40922-6
Online ISBN: 978-3-031-40923-3
eBook Packages: Computer ScienceComputer Science (R0)