[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/1985374.1985392acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Program slicing-based cohesion measurement: the challenges of replicating studies using metrics

Published: 24 May 2011 Publication History

Abstract

It is important to develop corpuses of data to test out the efficacy of using metrics. Replicated studies are an important contribution to corpuses of metrics data. There are few replicated studies using metrics reported in software engineering.
To contribute more data to the body of evidence on the use of novel program slicing-based cohesion metrics.
We replicate a very well regarded study by Meyers and Binkley [15, 16] which analyses the cohesion of open source projects using program slicing-based metrics.
Our results are very different from Meyers and Binkley's original results. This suggests that there are a variety of opportunities for inconsistently to creep into the collection and analysis of metrics data during replicated studies.
We conclude that researchers using metrics data must present their work with sufficient detail for replication to be possible. Without this detail it is difficult for subsequent researchers to accurately replicate a study such that consistent and reliable data can be added to a body of evidence.

References

[1]
Andersson, C. Runeson, P. (2007). A Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems. Software Engineering, IEEE Transactions on, 33(5), p273--286
[2]
Bowes, D., Counsell, S., Hall, T. (2008). Calibrating program slicing metrics for practical use, TAIC PART Conference, Windsor, Computer Society Press.
[3]
Brooks, A., Roper, M. et al (2007). Replication's role in software engineering. In: Shull, F., Singer, J., Sjoberg, D. I. K. (eds). Guide to advanced empirical software engineering. Springer, London, pp 365--379.
[4]
Fenton, N. E.; Ohlsson, N. (2000) "Quantitative analysis of faults and failures in a complex software system," Software Engineering, IEEE Transactions on, vol. 26, no. 8, pp. 797--814
[5]
Fenton, N. E., Pfleeger, S. L. (1997). Software Metrics: A Rigorous and Practical Approach (2nd ed.). Boston, MA: PWS Publishing Company.
[6]
Garcia Campos, C. (2009, April). CVSAnalY Manual. Retrieved May 16, 2010, from http://gsyc.es/~carlosgc/files/cvsanaly.pdf
[7]
GrammaTech Inc. (2007). CodeSurfer User Guide and Technical Reference.
[8]
GrammaTech Inc. (2010). CodeSurfer product page. Retrieved May 16, 2010, from GrammaTech Web site: http://www.grammatech.com/products/codesurfer/overview.html
[9]
Green, P., Lane, P., Rainer, A., Scholz, S.-B. (2009). An Introduction to Slice-Based Cohesion and Coupling Metrics. Technical Report No. 488, University of Hertfordshire, School of Computer Science.
[10]
Horwitz, S., Reps, T., Binkley, D. (1990). Interprocedural Slicing Using Dependence Graphs. ACM Transactions on Programming Languages and Systems, 12 (1), 26--60.
[11]
Kerr, M. (2010). Can Slice-Based Metrics be Used to Predict the Amount of Maintenance that will be Required on Each Module of a Program?
[12]
Kitchenham, B. A.; Pfleeger, S. L.; Pickard, L. M.; Jones, P. W.; Hoaglin, D. C.; El Emam, K.; Rosenberg, J.;, "Preliminary guidelines for empirical research in software engineering," Software Engineering, IEEE Transactions on, vol. 28, no. 8, pp. 721--734, Aug 2002.
[13]
Lonchampt, G., L. Bonnetain, Hieter, P. (1996). Reproduction of Fleischmann and Pons Experiments. in Sixth International Conference on Cold Fusion, Progress in New Hydrogen Energy. 1996. Lake Toya, Hokkaido, Japan: New Energy and Industrial Technology Development Organization, Tokyo Institute of Technology, Tokyo, Japan.
[14]
H. Longworth (1984). Slice based program metrics. Master's thesis, Computer Science, Michigan Technical University, Michigan, USA.
[15]
Meyers, T. M., Binkley, D. (2004). A Longitudinal and Comparative Study of Slice-Based Metrics. International Software Metrics Symposium, Chicargo, USA, IEEE Procs
[16]
Meyers, T. M., Binkley, D. (2007). An Empirical Study of Slice-Based Cohesion and Coupling Metrics. ACM Transactions on SoftwareMaintenance, 17(1), pp. 1--25.
[17]
Ott, L. M., & Thuss, J. J. (1993). Slice Based Metrics for Estimating Cohesion. In Proceedings of Internationl Software Metrics Symposium, Proceedings of the IEEE-CS, 71--81
[18]
Pan, K., Kim, S., Whitehead, J. Bug classification using program slicing metrics. In SCAM '06: Proceedings of the Sixth IEEE International Workshop on Source Code Analysis and Manipulation, Washington, DC, USA, 2006. IEEE Computer Society, p. 31--42.
[19]
Robles, G. (2010) Replicating MSR: A study of the potential replicability of papers published in the Mining Software Repositories Proceedings. Mining Software Repositories (MSR), 2010 7th IEEE Working Conference on, Computer Society Press, 171--180.
[20]
Shull, F.; Basili, V.; Carver, J.; Maldonado, J. C.; Travassos, G. H.; Mendonca, M.; Fabbri, S. (2002) Replicating software engineering experiments: addressing the tacit knowledge problem. Empirical Software Engineering, Proceedings in International Symposium. p. 7--16.
[21]
Shull, F., Carver, J., Vegas, S., Juristo, N., "The role of replications in Empirical Software Engineering," Empirical Software Engineering, vol. 13, no. 2, pp. 211--218, 2008.
[22]
Weiser, M. (1981). Program Slicing. Proceedings of the 5th International Conference on Software Engineering (pp. 439--449). San Diego: IEEE Press.
[23]
Weiser, M. (1984). Program Slicing. IEEE Transon Software Engineering, SE-10 (4), 352--357.

Index Terms

  1. Program slicing-based cohesion measurement: the challenges of replicating studies using metrics

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WETSoM '11: Proceedings of the 2nd International Workshop on Emerging Trends in Software Metrics
    May 2011
    90 pages
    ISBN:9781450305938
    DOI:10.1145/1985374
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 May 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. data quality
    2. empirical software engineering
    3. slicing metrics

    Qualifiers

    • Research-article

    Conference

    ICSE11
    Sponsor:
    ICSE11: International Conference on Software Engineering
    May 24, 2011
    HI, Waikiki, Honolulu, USA

    Upcoming Conference

    ICSE 2025

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 156
      Total Downloads
    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Dec 2024

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media