Summary
We consider two interpretations for what it means for test data to demonstrate correctness. For each interpretation, we examine under what conditions data sufficient to demonstrate correctness exists, and whether it can be automatically detected and/or generated. We establish the relation between these questions and the problem of deciding equivalence of two programs.
Similar content being viewed by others
References
Angluin, D.: On the complexity of minimum inference of regular sets. Information and Control39(3): 337–350 (1978)
Baldwin, D., Sayward, F.: Heuristics for determining equivalence of program mutations. Technical Report Number 161, Yale University 1979
Barzdin, J.M., Bicevskis, J.J., Kalninsh, A.A.: Construction of complete sample system for correctness testing. In: Lecture Notes in Computer Science, Vol. 32, pp. 1–12. Berlin: Springer-Verlag 1975
Biermann, A.W., Krishnaswamy, R.: Constructing programs from example computations. IEEE Transactions on Software Engineering SE-2(3): 141–153 (1976)
Blum, M.: On effective procedures for speeding up algorithms. Journal of the ACM18(2): 290–305 (1971)
Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control28(2): 125–155 (1975)
Brooks, M.: Automatic generation of test data for recursive programs having simple errors. PhD Thesis, Stanford University 1980
Budd, T.A., Lipton, R.J., DeMillo, R.A., Sayward, F.G.: Mutation analysis. Technical Report Number 155, Yale University 1979
Budd, T.A.: Mutation analysis of program test data. PhD Thesis, Yale University 1980
Chow, T.S.: Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering SE-4(3): 178–187 (1978)
Clarke, L.A.: A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering SE-2(3): 215–222 (1976)
DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: Help for the practicing programmer. Computer11(4): 34–43 (1978)
DeMillo, R.A., Lipton, R.J.: A probabilistic remark on algebraic program testing. Information Processing Letters7(4): 193–195 (1978)
Foster, K.A.: Error sensitive test cases. IEEE Transactions on Software Engineering SE-6(3): 258–264 (1980)
Geller, M.: Test data as an aid in proving program correctness. Communications of the ACM21(5): 368–375 (1978)
Goodenough, J.B., Gerhart, S.L.: Towards a theory of test data selection. IEEE Transactions on Software Engineering SE-1(2):156–173 (1975)
Goodenough, J.B.: A survey of program testing issues. pp. 316–340. In: Research Directions in Software Technology (P. Wegner, ed.), pp. 316–340. Cambridge, MA: MIT Press, 1979
Hamlet, R.: Testing programs with the aid of a compiler. IEEE Transactions on Software Engineering SE-3(4): 279–290 (1977)
Hamlet, R.: Testing programs with finite sets of data. The Computer Journal20(3): 232–237 (1977)
Hamlet, R.: Test reliability and software maintenance. In: Proceeding of the Compsac conference, pp. 315–320. IEEE 1978
Hamlet, R.: Critique of reliability theory. In: Digest of the Workshop on Software Testing and Test Documentation, pp. 57–69. Fort Lauderdale 1978
Hantler, S.L., King, J.C.: An introduction to proving the correctness of programs. ACM Computer Surveys8(3): 331–353 (1976)
Hopcroft, J.E., Ullman, J.D.: Formal languages and their relation to automata. New York: Addison-Wesley 1969
Howden, W.E.: Reliability of the path analysis testing strategy. IEEE Transactions on Software Engineering SE-2(3): 208–214 (1976)
Howden, W.E.: Lindenmayer grammers and symbolic testing. Information Processing L etters7(1): 36–39 (1978)
Howden, W.E.: Algebraic program testing. Acta Informat.10(1): 53–66 (1978)
Howden, W.E.: Functional program testing. IEEE Transactions on Software Engineering SE-6(2): 162–169 (1980)
Kernighan, B.W., Plauger, P.J.: Software tools. New York: Addison-Wesley 1976
Miller, E.F., Melton, R.A.: Automated generation of testcase datasets. In: Proceeding 1975 International Conference on Reliable Software, pp. 51–58. Los Angeles 1975
Ostrand, T.J., Weyuker, E.J.: Remarks on the theory of test data selection. In: Digest of the Workshop on Software Testing and Test Documentation, pp. 1–18. Fort Lauderdale 1978
Rogers, H. Jr.: Theory of recursive functions and effective computability. New York: McGraw-Hill 1967
Rowland, J.H., Davis, P.J.: On the selection of test data for recursive mathematical sub-routines. Siam Journal on Computing10(1): 59–72 (1981)
White, L.J., Cohen, E.I.: A domain strategy for computer program testing. IEEE Transactions on Software Engineering SE-6(3): 247–257 (1980)
Author information
Authors and Affiliations
Additional information
The work of this author was supported in part by the National Science Foundation under grant MCS-8109547
Rights and permissions
About this article
Cite this article
Budd, T.A., Angluin, D. Two notions of correctness and their relation to testing. Acta Informatica 18, 31–45 (1982). https://doi.org/10.1007/BF00625279
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00625279