[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

An Empirical Method for Selecting Software Reliability Growth Models

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Estimating remaining defects (or failures) in software can help test managers make release decisions during testing. Several methods exist to estimate defect content, among them a variety of software reliability growth models (SRGMs). SRGMs have underlying assumptions that are often violated in practice, but empirical evidence has shown that many are quite robust despite these assumption violations. The problem is that, because of assumption violations, it is often difficult to know which models to apply in practice. We present an empirical method for selecting SRGMs to make release decisions. The method provides guidelines on how to select among the SRGMs to decide on the best model to use as failures are reported during the test phase. The method applies various SRGMs iteratively during system test. They are fitted to weekly cumulative failure data and used to estimate the expected remaining number of failures in software after release. If the SRGMs pass proposed criteria, they may then be used to make release decisions. The method is applied in a case study using defect reports from system testing of three releases of a large medical record system to determine how well it predicts the expected total number of failures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abdel-Ghaly, A., Chan, P., and Littlewood, B. 1986. Evaluation of Competing Software Reliability Predictions: IEEE Transactions on Software Engineering SE-12(9): 950–967.

    Google Scholar 

  • AIAA SBOS/COS Software Reliability Working Group, AIAA Software Reliability Engineering Recommended Practice, R-013-1992, American Institute of Aeronautics and Astronautics.

  • Basili, V., and Zelkowitz, M. 1978. Analyzing medium-scale software development. Proceedings of the Third International Conference on Software Engineering 116–123.

  • Biyani, S., and Santhanam, P. 1998. Exploring defect data from development and customer usage on software modules over multiple releases. Proceedings of the Ninth International Conference on Software Reliability Engineering. Paderborn, Germany, pp. 316–320.

  • Briand, L., El Emam, K., and Freimut, B. 1998. A comparison and integration of capture- recapture models and the detection profile method. Proceedings of the Ninth International Conference on Software Reliability Engineering. Paderborn, Germany, pp. 32–41.

  • Briand, L., El Emam, K., Freimut, B., Laitenberger, B., and Laitenberger, O. 1997. Quantitative evaluation of capture- recapture models to control software inspections. Proceedings of the Eighth International Conference on Software Reliability Engineering. Albuquerque, NM, pp. 234–244.

  • Brocklehurst, S., and Littlewood, B. 1996. Techniques for Prediction, Analysis, and Recalibration, In: M. Lyu (ed.): Handbook of Software Reliability Engineering. Los Alamitos, CA: McGraw-Hill and IEEE Computer Society press Chapter 4, pp. 119–166.

    Google Scholar 

  • Brocklehurst, S., Chan, P., Littlewood, B., and Snell, J. 1990. “Recalibrating software reliability models. IEEE Transactions on Software Engineering SE-16(4): pp. 458–470.

    Google Scholar 

  • Cai, K., Wen, C., and Zhang, M. 1991. A critical review on software reliability modeling. Reliability Engineering and System Safety 32: 357–371.

    Google Scholar 

  • Conte, S., Dunsmore, H., and Shen, V. 1986. Software Engineering Metrics and Models. Menlo Park, California: The Benjamin/Cummings Publishing Company, Inc.

    Google Scholar 

  • Eick, S., Loader, C., Long, M., Votta, L., and VanderWeil, S. 1992. Estimating software fault content before coding. Proceedings of the International Conference on Software Engineering. Melbourne, Australia, pp. 59–65.

  • Farr, W., and Smith, O. 1993. Statistical Modeling and Estimation of Reliability Functions for Software (SMERFS) User's Guide. Dahlgren, VA: Naval Surface Warfare Center.

    Google Scholar 

  • Frankl, P., Hamlet, R., Littlewood, B., and Strigini, L. 1998. Evaluating testing methods by delivered reliability. IEEE Transactions on Software Engineering 24(8): 586–601.

    Google Scholar 

  • Gaudoin, O., Xie, M., and Yang, B. 2002. A simple goodness-of-fit test for the power-law process, based on the Duane plot. IEEE Transactions on Reliability (in press).

  • Goel, A. L. 1985. Software reliability models: Assumptions, limitations, and applicability. IEEE Transactions on Reliability 11(12): 1411–1421.

    Google Scholar 

  • Goel, A. L., and Okumoto, K. 1979. A time dependent error detection model for software reliability and other performance measures. IEEE Transactions on Reliability 28(3): 206–211.

    Google Scholar 

  • Iannino, A., Musa, J., Okumoto, K., and Littlewood, B. 1984. Criteria for software model comparisons. IEEE Transactions on Software Engineering SE-10(6): 687–691.

    Google Scholar 

  • Kececioglu, D. 1991. Reliability Engineering Handbook, Vol. 2, Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Keiller, P., Littlewood, B., Miller, D., and Sofer, A. 1983. Comparison of software reliability predictions. Proceedings of the Thirteenth International Symposium on Fault-tolerant Computing. pp. 128–134.

  • Khoshgoftaar, T., and Woodcock, T. 1991. Software reliabiity model selection: A case study. Proceedings of the Second International Symposium on Software Reliability Engineering. IEEE Computer Society Press, Austin, TX: pp. 183–191.

    Google Scholar 

  • Lyu, M. (ed.): 1996. Handbook of Software Reliability Engineering. New York: McGraw-Hill.

    Google Scholar 

  • Lyu, M., and Nikora, A. 1992. CASREA—A computer-aided software reliability estimation tool, Proceedings of the Fifth International Workshop on Computer-Aided Software Engineering, Montreal, CA, pp. 264–275.

  • Musa, J. 1998. Applying failure data to guide decisions. Software Reliability Engineering. New York: McGraw-Hill.

    Google Scholar 

  • Musa, J., and Ackerman, A. 1989. Quantifying software validation: When to stop testing. IEEE Software. 19–27.

  • Musa, J., Iannino, A., and Okumoto, K. 1987. Software Reliability: Measurement, Prediction, Application. New York: McGraw-Hill.

    Google Scholar 

  • Rigdon, S. 2000. Statistical Methods for the Reliability of Repairable Systems. New York: Wiley.

    Google Scholar 

  • Runeson, P., and Wohlin, C. 1998. An experimental evaluation of an experience-based capture-recapture method in software code inspections. Empirical Software Engineering: An International Journal 3(4): 381–406.

    Google Scholar 

  • Stringfellow, C. 2000. An integrated method for improving testing effectiveness and efficiency. PhD Dissertation, Colorado State University.

  • Trachtenberg, M. 1990. A general theory of software-reliability modeling. IEEE Transactions on Reliability 39(1): 92–96.

    Google Scholar 

  • Vander Wiel, S., and Votta, L. 1993. Assessing software designs using capture- recapture methods. IEEE Transactions on Software Engineering 19(11): 1045–1054.

    Google Scholar 

  • Wohlin, C., and Runeson, P. 1995. An experimental evaluation of capture- recapture in software inspections. Journal of Software Testing, Verification and Reliability 5(4): 213–232.

    Google Scholar 

  • Wohlin, C., and Runeson, P. 1998. Defect content estimations from review data. Proceedings of the International Conference on Software Engineering. Kyoto, Japan, pp. 400–409.

  • Wood, A. 1996. Predicting software reliability. IEEE Computer 29(11): 69–78.

    Google Scholar 

  • Wood, A. 1997. Software reliability growth models: Assumptions vs. reality. Proceedings of the International Symposium on Software Reliability Engineering 23(11): 136–141.

    Google Scholar 

  • Yamada, S., Ohba, M., and Osaki, S. 1983. S-shaped reliability growth modeling for software error detection. IEEE Transactions on Reliability 32(5): 475–478.

    Google Scholar 

  • Yamada, S., Ohba, M., and Osaki, S. 1985. Software reliability growth modeling: Models and applications. IEEE Transactions on Reliability 11(12): 1431–1437.

    Google Scholar 

  • Yamada, S., Ohtera, H., and Narihisa, H. 1986. Software reliability growth models with testing effort. IEEE Transactions on Reliability 35(1): 19–23.

    Google Scholar 

  • Yang, M., and Chao, A. 1995. Reliability-estimation & stopping-rules for software testing based on repeated appearances of bugs. IEEE Transactions on Reliability 44(2): 315–321.

    Google Scholar 

  • Yu, T., Shen, V., and Dunsmore, H. 1988. An analysis of several software defect models. IEEE Transactions on Software Engineering 14(9): 1261–1270.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Midwestern State University, Wichita Falls, TX, 76308

    C. Stringfellow

  2. School of Electrical Engineering and Computer Science, Washingron State University, Pullman, WA, 99164

    A. Amschler Andrews

Authors
  1. C. Stringfellow
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Amschler Andrews
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Stringfellow, C., Andrews, A.A. An Empirical Method for Selecting Software Reliability Growth Models. Empirical Software Engineering 7, 319–343 (2002). https://doi.org/10.1023/A:1020515105175

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1020515105175

Search

Navigation