research-article

Sensitivity of results to different data quality meta-data criteria in the sample selection of projects from the ISBSG dataset

Authors:

Marta Fernández-Diego,

Mónica Martínez-Gómez,

José-María Torralba-MartínezAuthors Info & Claims

PROMISE '10: Proceedings of the 6th International Conference on Predictive Models in Software Engineering

Article No.: 13, Pages 1 - 9

https://doi.org/10.1145/1868328.1868348

Published: 12 September 2010 Publication History

Get Access

Abstract

Background: Most prediction models, e.g. effort estimation, require preprocessing of data. Some datasets, such as ISBSG, contain data quality meta-data which can be used to filter out low quality cases from the analysis. However, an agreement has not been reached yet between researchers about these data quality selection criteria.

Aims: This paper aims to analyze the influence of data quality meta-data criteria in the number of selected projects, which can have influence in the models obtained. For this, a case study has been selected to gain a more complete understanding of what might be important to focus in future research.

Method: Data quality meta-data selection criteria of some works based on ISBSG dataset which propose prediction models were reviewed first. Considerable attention has been paid to two data quality meta-data variables in ISBSG dataset Release 11 which are Data Quality Rating and Unadjusted Function Point Rating. Secondly, this paper considers data from 830 projects which have been collected from the ISBSG dataset after a preliminary screening. This first screening leads mainly to a subset of projects with comparable definitions in size and effort. Then data quality meta-data criteria are applied in order to infer their influence.

Results: Overall, it seems that data selection criteria, regardless data quality meta-data concerns, involve an important reduction in sample size. From 5052 projects, only 830 are really considered. Then 262 projects remain for analysis if the maximum quality rate is applied for both data quality meta-data variables. But, since the initial data preparation focuses the problem of missingness for a certain purpose, data quality criteria seem not to be the clue for the analysis results. However, some variability has been observed.

Conclusions: Whilst this analysis is supported by a case study, it is hoped that it contributes to a better understanding of the subject. In fact, results found suggest that in those studies where the selection criteria of projects are not very strictly applied, these data quality criteria must be carefully taken into account.

References

[1]

}}Ahmed, F., Bouktif, S., Serhani, A., and Khalil, I. Integrating Function Point Project Information for Improving the Accuracy of Effort Estimation. 2008 The Second International Conference on Advanced Engineering Computing and Applications in Sciences, (2008), 193--198.

Abstract

References

Cited By

Index Terms

Recommendations

Data quality: cinderella at the software metrics ball?

Validation of the Arizona Governor's Quality Award criteria: a test of the Baldrige criteria

A human morning routine dataset

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations