[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2961111.2962619acmconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
short-paper

Survey Guidelines in Software Engineering: An Annotated Review

Published: 08 September 2016 Publication History

Abstract

Background: Survey is a method of research aiming to gather data from a large population of interest. Despite being extensively used in software engineering, survey-based research faces several challenges, such as selecting a representative population sample and designing the data collection instruments.
Objective: This article aims to summarize the existing guidelines, supporting instruments and recommendations on how to conduct and evaluate survey-based research.
Methods: A systematic search using manual search and snowballing techniques were used to identify primary studies supporting survey research in software engineering. We used an annotated review to present the findings, describing the references of interest in the research topic.
Results: The summary provides a description of 15 available articles addressing the survey methodology, based upon which we derived a set of recommendations on how to conduct survey research, and their impact in the community.
Conclusion: Survey-based research in software engineering has its particular challenges, as illustrated by several articles in this review. The annotated review can contribute by raising awareness of such challenges and present the proper recommendations to overcome them.

References

[1]
C. Bennett, S. Khangura, J. C. Brehaut, I. D. Graham, D. Moher, B. K. Potter, and J. M. Grimshaw. Reporting guidelines for survey research: an analysis of published guidance and reporting practices. PLoS Med, 8(8):e1001069, 2011.
[2]
L. Bourque and E. P. Fielder. How to conduct self-administered and mail surveys, volume 3. Sage, 2003.
[3]
A. Cater-Steel, M. Toleman, and T. Rout. Addressing the challenges of replications of surveys in software engineering research. In Empirical Software Engineering, 2005. 2005 International Symposium on, pages 10--pp. IEEE, 2005.
[4]
M. Ciolkowski, O. Laitenberger, S. Vegas, and S. Biffl. Practical experiences in the design and conduct of surveys in empirical software engineering. Springer, 2003.
[5]
R. Conradi, J. Li, O. P. N. Slyngstad, V. B. Kampenes, C. Bunse, M. Morisio, and M. Torchiano. Reflections on conducting an international survey of software engineering. In Empirical Software Engineering, 2005. 2005 International Symposium on, pages 10--pp. IEEE, 2005.
[6]
R. M. de Mello, P. C. da Silva, P. Runeson, and G. H. Travassos. Towards a framework to support large scale sampling in software engineering surveys. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, page 48. ACM, 2014.
[7]
R. M. de Mello, P. C. da Silva, and G. H. Travassos. Sampling improvement in software engineering surveys. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, page 13. ACM, 2014.
[8]
R. M. de Mello, P. C. Da Silva, and G. H. Travassos. Investigating probabilistic sampling approaches for large-scale surveys in software engineering. Journal of Software Engineering Research and Development, 3(1):1--26, 2015.
[9]
R. M. de Mello and G. H. Travassos. Would sociable software engineers observe better? In Empirical Software Engineering and Measurement, 2013 ACM/IEEE International Symposium on, pages 279--282. IEEE, 2013.
[10]
R. M. de Mello and G. H. Travassos. Characterizing sampling frames in software engineering surveys. In Proc. 12th Workshop on Experimental Software Engineering (ESELAW), 2015.
[11]
T. Dybå. An empirical investigation of the key factors for success in software process improvement. Software Engineering, IEEE Transactions on, 31(5):410--424, 2005.
[12]
A. Fink. The survey handbook, volume 1. Sage, 2003.
[13]
F. J. Fowler Jr. Survey research methods. Sage publications, 2013.
[14]
J. Ji, J. Li, R. Conradi, C. Liu, J. Ma, and W. Chen. Some lessons learned in conducting software engineering surveys in china. In Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, pages 168--177. ACM, 2008.
[15]
M. Kasunic. Designing an effective survey. Technical report, DTIC Document, 2005.
[16]
B. Kitchenham, S. Linkman, and D. Law. Desmet: a methodology for evaluating software engineering methods and tools. Computing & Control Engineering Journal, 8(3):120--126, 1997.
[17]
B. A. Kitchenham, D. Budgen, and P. Brereton. Surveys. In Evidence-Based Software Engineering and Systematic Reviews, pages 234--242. CRC Press, 2015.
[18]
B. A. Kitchenham and S. L. Pfleeger. Principles of survey research: parts 1-6. ACM SIGSOFT Software Engineering Notes, 26--28, 2001-2003.
[19]
B. A. Kitchenham and S. L. Pfleeger. Personal opinion surveys. In Guide to Advanced Empirical Software Engineering, pages 63--92. Springer, 2008.
[20]
B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. Software Engineering, IEEE Transactions on, 28(8):721--734, 2002.
[21]
J. Linåker, S. M. Sulaman, R. Maiani de Mello, and M. Höst. Guidelines for conducting surveys in software engineering. 2015.
[22]
M. S. Litwin. How to measure survey reliability and validity, volume 7. Sage Publications, 1995.
[23]
M. Petre. Uml in practice. In Proceedings of the 2013 International Conference on Software Engineering, pages 722--731. IEEE Press, 2013.
[24]
T. Punter, M. Ciolkowski, B. Freimut, and I. John. Conducting on-line surveys in software engineering. In Empirical Software Engineering, 2003. ISESE 2003. Proceedings. 2003 International Symposium on, pages 80--88. IEEE, 2003.
[25]
A. Rindfleisch, A. J. Malter, S. Ganesan, and C. Moorman. Cross-sectional versus longitudinal survey research: Concepts, findings, and guidelines. Journal of Marketing Research, 45(3):261--279, 2008.
[26]
J. Ropponen and K. Lyytinen. Components of software development risk: how to address them? a project manager survey. Software Engineering, IEEE Transactions on, 26(2):98--112, 2000.
[27]
D. I. Sjøberg, J. E. Hannay, O. Hansen, V. B. Kampenes, A. Karahasanovic, N.-K. Liborg, and A. C. Rekdal. A survey of controlled experiments in software engineering. Software Engineering, IEEE Transactions on, 31(9):733--753, 2005.
[28]
C. Zhang and D. Budgen. A survey of experienced user perceptions about software design patterns. Information and Software Technology, 55(5):822--835, 2013.

Cited By

View all
  • (2024)Key Performance Indicators in Brazilian Software Development Companies: a SurveyProceedings of the XXIII Brazilian Symposium on Software Quality10.1145/3701625.3701646(60-68)Online publication date: 5-Nov-2024
  • (2024)Can ChatGPT emulate humans in software engineering surveys?Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3690744(414-419)Online publication date: 24-Oct-2024
  • (2024)The Role of Generative AI in Software Development Productivity: A Pilot Case StudyProceedings of the 1st ACM International Conference on AI-Powered Software10.1145/3664646.3664773(131-138)Online publication date: 10-Jul-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEM '16: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
September 2016
457 pages
ISBN:9781450344272
DOI:10.1145/2961111
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 September 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Survey
  2. annotated review
  3. guidelines

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Conference

ESEM '16
Sponsor:

Acceptance Rates

ESEM '16 Paper Acceptance Rate 27 of 122 submissions, 22%;
Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)233
  • Downloads (Last 6 weeks)25
Reflects downloads up to 30 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Key Performance Indicators in Brazilian Software Development Companies: a SurveyProceedings of the XXIII Brazilian Symposium on Software Quality10.1145/3701625.3701646(60-68)Online publication date: 5-Nov-2024
  • (2024)Can ChatGPT emulate humans in software engineering surveys?Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3690744(414-419)Online publication date: 24-Oct-2024
  • (2024)The Role of Generative AI in Software Development Productivity: A Pilot Case StudyProceedings of the 1st ACM International Conference on AI-Powered Software10.1145/3664646.3664773(131-138)Online publication date: 10-Jul-2024
  • (2024)Charting a Path to Efficient Onboarding: The Role of Software VisualizationProceedings of the 2024 IEEE/ACM 17th International Conference on Cooperative and Human Aspects of Software Engineering10.1145/3641822.3641873(133-143)Online publication date: 14-Apr-2024
  • (2024)Qualitative Surveys in Software Engineering Research: Definition, Critical Review, and GuidelinesIEEE Transactions on Software Engineering10.1109/TSE.2024.3474173(1-16)Online publication date: 2024
  • (2024)A Dive Into the State of the Practice of the Brazilian Game Software EcosystemIEEE Transactions on Games10.1109/TG.2023.324221716:1(195-204)Online publication date: Mar-2024
  • (2024)Investigating the Impact of Developer Proficiency on Bug-Fixing Efficiency and Accuracy2024 8th International Conference on Information Technology (InCIT)10.1109/InCIT63192.2024.10810579(427-432)Online publication date: 14-Nov-2024
  • (2024)Continuous Conformance of Software Architectures2024 IEEE 21st International Conference on Software Architecture (ICSA)10.1109/ICSA59870.2024.00019(112-122)Online publication date: 4-Jun-2024
  • (2024)A Comprehensive Taxonomy of Social Engineering Attacks and Defense Mechanisms: Toward Effective Mitigation StrategiesIEEE Access10.1109/ACCESS.2024.340319712(72224-72241)Online publication date: 2024
  • (2024)Augmented Reality Navigation: A SurveyInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2431757(1-17)Online publication date: 28-Nov-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media