[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/1810295.1810313acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Transparent combination of expert and measurement data for defect prediction: an industrial case study

Published: 01 May 2010 Publication History

Abstract

Defining strategies on how to perform quality assurance (QA) and how to control such activities is a challenging task for organizations developing or maintaining software and software-intensive systems. Planning and adjusting QA activities could benefit from accurate estimations of the expected defect content of relevant artifacts and the effectiveness of important quality assurance activities. Combining expert opinion with commonly available measurement data in a hybrid way promises to overcome the weaknesses of purely data-driven or purely expert-based estimation methods. This article presents a case study of the hybrid estimation method HyDEEP for estimating defect content and QA effectiveness in the telecommunication domain. The specific focus of this case study is the use of the method for gaining quantitative predictions. This aspect has not been empirically analyzed in previous work. Among other things, the results show that for defect content estimation, the method performs significantly better statistically than purely data-based methods, with a relative error of 0.3 on average (MMRE).

References

[1]
Aurum, A., Petersson, H., Wohlin, C., "State-of-the-Art: Software Inspections after 25 years", Software Testing, Verification and Reliability, vol. 12, pp. 131--154, 2002
[2]
Briand, L. C., El Emam, K., Bomarius, F. CoBRA: A Hybrid Method for Software Cost Estimation, Benchmarking and Risk Assessment, 20th International Conference on Software Engineering, pp. 390--399, 1998
[3]
Briand, L., Wuest, J., Daly, J. W., Porter, V., "Exploring the Relationships between Design Measures and Software Quality in Object-Oriented Systems", Journal of Systems and Software, vol. 51, pp. 245--273, 2000
[4]
Catal, C., Diri, B., "A systematic review of software fault predictions studies," Expert Systems with Applications, vol. 36, pp. 7346--7354, 2009
[5]
Chulani S., and Boehm B., "Modeling Software Defect Introduction and Removal: COQUALMO," Technical Report USC-CSE-99-510, University of Southern California, Center for Software Engineering, 1999
[6]
Conte, S. D., Dunsmore, H. E., Shen V. Y., Software engineering Metrics and Models, The Benjamin-Cummings Publishing Company, 1986
[7]
Fenton, N. E., Pfleeger, S. L., Software Metrics -- A Rigorous & Practical Approach, PWS Publishing Company, 1998
[8]
Graves, T. L., Karr, A. F., Marron, J. S., Siy, "Predicting fault incidence using software change history," Transactions on Software Engineering, vol. 26, pp. 653--661, 2000
[9]
Jacobs, J., Moll, J., Kusters, R., Trienekens, J., Brombacher, A., "Identification of factors that influence defect injection and detection in development of software intensive products," Information and Software Technology, vol. 49, pp. 774--789, 2007
[10]
Juristo, N., Moreno, A. M., Vegas, S., "A Survey on Testing Technique Empirical Studies: How limited is our Knowledge?", 1st International Symposium on Empirical Software Engineering, pp. 161--172, 2002
[11]
Kan, S. H., Metrics and Models in Software Quality Engineering, Addison-Wesley Professional, 2002.
[12]
Kläs, M., Elberzhager, F., Nakao, H., "Managing Software Quality through a Hybrid Defect Content and Effectiveness Model," 2nd International Symposium on Empirical Software Engineering and Measurement, pp. 321--323, 2008
[13]
Kläs, M., Nakao, H., Elberzhager, F., Münch, J., "Predicting Defect Content and Quality Assurance Effectiveness by Combining Expert Judgment and Defect Data - A Case Study," 19th International Symposium on Software Reliability Engineering, pp. 17--26, 2008
[14]
Kläs, M., Nakao, H., Elberzhager, E., Münch, J., "Support Planning and Controlling of Early Quality Assurance by Combining Expert Judgment and Defect Data - A Case Study," Empirical Software Engineering Journal, 2009.
[15]
Kläs, M., Trendowicz, A., Wickenkamp, A., Münch, J., Kikuchi, N., Ishigai, Y., "The use of simulation techniques for hybrid software cost estimation and risk analysis," Advances in computers, Elsevier, vol. 74, pp. 115--174, 2008
[16]
Lyu, M. R., Encyclopedia of Software Engineering. John Wiley & Sons, chapter Software Reliability Theory, 2002
[17]
Menzies, T., Greenwald, J., Frank, A., "Data Mining Static Code Attributes to Learn Defect Predictors," Transactions on Software Engineering, vol. 33, pp. 2--13, 2007
[18]
Mueller, M., Pfahl, D., "Simulation Methods," In: Advanced Topics in Empirical Software Engineering: A Handbook, Springer, pp. 117--152, 2008
[19]
Nagappan, N., Ball, T., Zeller, A., "Mining metrics to predict component failures," 28th International Conference on Software Engineering, pp. 452--461, May 20--28, 2006
[20]
Petersson, H., Thelin, T., Runeson, P., and Wohlin, C., "Capture-Recapture in Software Inspections after 10 years Research - Theory, Evaluation and Application," Journal of Systems and Software, vol. 72, pp. 249--264, 2004
[21]
Scott, H. and Wohlin, C., "Capture-recapture in software unit testing: a case study," Proceedings of the 2nd International Symposium on Empirical Software Engineering and Measurement. ACM, New York, pp. 32--40, 2008
[22]
Trendowicz, A., Heidrich, J., Münch, J., Ishigai, Y., Yokoyama, K., Kikuchi, N., "Development of a hybrid cost estimation model in an iterative manner," 28th International Conference on Software Engineering, pp. 331--340, 2006
[23]
Wagner, S., "A model and sensitivity analysis of the quality economics of defect-detection techniques," International Symposium on Software Testing and Analysis, 2006
[24]
Walia, G. S., Carver, J. C., Nagappan, N., "The effect of the number of inspectors on the defect estimates produced by capture-recapture models," ACM/IEEE 30th International Conference on Software Engineering, pp. 331--340, 2008
[25]
Wohlin, C., Runeson, P., Host. M., Ohlsson, M. C., Regnell, B., Wesslen, A., Experimentation in software engineering - an introduction, Kluwer, 2000
[26]
Zhang, H., Kitchenham, B., Pfahl, D., "Reflections on 10 Years of Software Process Simulation Modeling: A Systematic Review," International Conference on Software Process, Springer, pp. 345--356, 2008.

Cited By

View all
  • (2022)A Systematic Review of Attributes and Techniques for Open Source Software Evolution AnalysisResearch Anthology on Agile Software, Software Development, and Testing10.4018/978-1-6684-3702-5.ch006(84-106)Online publication date: 2022
  • (2022)RELMP-MM: an approach to cross project fault prediction using improved regularized extreme learning machine and identical matched metricsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-022-03820-114:10(13523-13542)Online publication date: 31-Mar-2022
  • (2022)Test case prioritization using test case diversification and fault-proneness estimationsAutomated Software Engineering10.1007/s10515-022-00344-y29:2Online publication date: 1-Aug-2022
  • Show More Cited By

Index Terms

  1. Transparent combination of expert and measurement data for defect prediction: an industrial case study

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2
    May 2010
    554 pages
    ISBN:9781605587196
    DOI:10.1145/1810295
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 May 2010

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. HyDEEP
    2. defect content
    3. effectiveness
    4. hybrid estimation

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ICSE '10
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 276 of 1,856 submissions, 15%

    Upcoming Conference

    ICSE 2025

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 09 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)A Systematic Review of Attributes and Techniques for Open Source Software Evolution AnalysisResearch Anthology on Agile Software, Software Development, and Testing10.4018/978-1-6684-3702-5.ch006(84-106)Online publication date: 2022
    • (2022)RELMP-MM: an approach to cross project fault prediction using improved regularized extreme learning machine and identical matched metricsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-022-03820-114:10(13523-13542)Online publication date: 31-Mar-2022
    • (2022)Test case prioritization using test case diversification and fault-proneness estimationsAutomated Software Engineering10.1007/s10515-022-00344-y29:2Online publication date: 1-Aug-2022
    • (2021)A Systematic Review of Attributes and Techniques for Open Source Software Evolution AnalysisResearch Anthology on Usage and Development of Open Source Software10.4018/978-1-7998-9158-1.ch001(1-23)Online publication date: 2021
    • (2020)WR-ELM: Weighted Regularization Extreme Learning Machine for Imbalance Learning in Software Fault PredictionIEEE Transactions on Reliability10.1109/TR.2020.2996261(1-21)Online publication date: 2020
    • (2018)A Systematic Review of Attributes and Techniques for Open Source Software Evolution AnalysisOptimizing Contemporary Application and Processes in Open Source Software10.4018/978-1-5225-5314-4.ch001(1-23)Online publication date: 2018
    • (2017)An Extensive Analysis of Efficient Bug Prediction ConfigurationsProceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering10.1145/3127005.3127017(107-116)Online publication date: 8-Nov-2017
    • (2016)Open Source Software EvolutionInternational Journal of Open Source Software and Processes10.4018/IJOSSP.20160101027:1(28-48)Online publication date: 1-Jan-2016
    • (2016)Understanding Open Source Software Evolution Using Fuzzy Data Mining Algorithm for Time Series DataAdvances in Fuzzy Systems10.1155/2016/14796922016(1)Online publication date: 1-Sep-2016
    • (2016)Fuzzy analysis and prediction of commit activity in open source software projectsIET Software10.1049/iet-sen.2015.008710:5(136-146)Online publication date: Oct-2016
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media