[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1109/CESSER-IP.2019.00010acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Mind the gap: are practitioners and researchers in software testing speaking the same language?

Published: 27 May 2019 Publication History

Abstract

Context. Software testing is the area of software engineering focused on determining whether a software meets the planned requirements and on evaluating its quality. Lately, academic researchers have increased their attention in this topic due to the impact of its success on software projects. However, recent studies have discussed that practitioners and researchers might have different views regarding what is important to explore and study in order to improve the software testing process. Goal. This study aims to investigate the differences of interests between academic researchers and practitioners in software testing, pointing out observable convergences and divergences between the two communities. Method. A mixed-method approach based on a mapping study, a quantitative study and a focus group was applied to collect quantitative and qualitative data from professionals and academic sources. Results. Our results confirm the existence of a gap between the two communities and the findings suggest that, while researchers are mainly focused on the proposition of novel tools and techniques, practitioners are more interested in issues related to the evaluation and discussions of existing approaches, tools and techniques. Therefore, academic researchers might consider identify, understand and modify the existing tools and strategies, instead of building new ones. Conclusion. In general, the distinction between the two groups is noticeable and there is only one strong mutual interest between both practitioners and researchers, namely, test automation. Therefore, there is a need for the development of strategies that reduce the gap between academia and industrial practice and bring them closer in order to increase the quality of the software testing processes.

References

[1]
Bourque, P. and Fairley, R.E., 2014. Guide to the software engineering body of knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society Press.
[2]
Tuteja, M., & Dubey, G. (2012). A research study on importance of testing and quality assurance in software development life cycle (SDLC) models. International Journal of Soft Computing and Engineering (IJSCE), 2(3), 251--257.
[3]
Sandhu, G.K. and Kaur, R., 2016. Effective test case generation for load testing of web server using river formation dynamics (Doctoral dissertation).
[4]
Kurokawa, T., & Shinagawa, M. (2008). Technical trends and challenges of software testing. NISTEP Science & Technology Foresight Center.
[5]
Garousi, V., & Herkiloglu, K. (2016, April). Selecting the right topics for industry-academia collaborations in software testing: an experience report. In Software Testing, Verification and Validation (ICST), 2016 IEEE International Conference on (pp. 213--222). IEEE.
[6]
Garousi, V., Felderer, M., Kuhrmann, M., & Herkiloğlu, K. (2017, June). What industry wants from academia in software testing?: Hearing practitioners' opinions. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering (pp. 65--69). ACM.
[7]
S. Beecham, P. O'Leary, S. Baker, I. Richardson, and J. Noll. 2014. Making Software Engineering Research Relevant. Computer 47, 4 (Apr 2014), 810--83.
[8]
V. Garousi, K. Petersen, and B. Özkan. 2016. Challenges and Best Practices in Industry-academia Collaborations in Software Engineering. Information and Software Technology 79, C (Nov. 2016), 106--127.
[9]
Garousi, V., & Felderer, M. (2017). Worlds apart: Industrial and academic focus areas in software testing. IEEE Software, (5), 38--45.
[10]
Cartaxo, B., Pinto, G., & Soares, S. (2018). Towards a model to transfer knowledge from software engineering research to practice. Information and Software Technology, 97, 80--82.
[11]
Arcuri, A. (2017). An experience report on applying software testing academic results in industry: we need usable automated test generation. Empirical Software Engineering, 1--23.
[12]
Marculescu, B., Feldt, R., Torkar, R., & Poulding, S. (2018). Transferring interactive search-based software testing to industry. Journal of Systems and Software, 142, 156--170.
[13]
Moran, K., Linares-Vásquez, M., & Poshyvanyk, D. (2017, May). Automated GUI testing of Android apps: from research to practice. In Proceedings of the 39th International Conference on Software Engineering Companion (pp. 505--506). IEEE Press.
[14]
Arksey, H. and O'Malley, L. 2005. Scoping studies: towards a methodological framework. Int J Social Research Methodology. 8(1):19--32.
[15]
da Silva, F. Q. B., Santos, A. L. M., Soares, S. C. B., França, A. C. C., Monteiro, C. V. F. and Maciel, F. F. 2011. Six years of systematic literature reviews in Software Engineering: An updated tertiary study. Information and Software Technology 53, 899--913.
[16]
Begel, A., Bosch, J. and Storey, M.A., 2013. Social networking meets software development: Perspectives from github, msdn, stack exchange, and topcoder. IEEE Software, 30(1), pp.52--66
[17]
Cartaxo, B., Pinto, G., Ribeiro, D., Kamei, F., Santos, R.E., da Silva, F.Q. and Soares, S., 2017, May. Using Q&A websites as a method for assessing systematic reviews. In Proceedings of the 14th International Conference on Mining Software Repositories (pp. 238--242). IEEE Press.
[18]
Astle, D. and Durnil, D., 2004. OpenGL-ES game development (game development series). Budapest: Premier Press.
[19]
Furtado, A., Andrade, N., Oliveira, N. and Brasileiro, F., 2013, February. Contributor profiles, their dynamics, and their importance in five Q&A sites. In Proceedings of the 2013 conference on Computer supported cooperative work (pp. 1237--1252). ACM.
[20]
Kitzinger, J. (1995). Qualitative research: introducing focus groups. Bmj, 311(7000), 299--302.
[21]
Seaman, C. B. Qualitative methods in empirical studies of Software Engineering. IEEE Transactions on Software Engineering, 25, 4, 557--572, 1999.
[22]
Petticrew, M., and Roberts, H. 2006. Systematic Reviews in the Social Sciences. Blackwell Publishing. 336
[23]
Kitchenham, B. and Charters, S. Guidelines for performing systematic literature reviews in software engineering. 2007. Technical Report EBSE-2007--01, School of Computer Science and Mathematics, Keele University.
[24]
Santos, R., Magalhaes, C., Capretz, L. F., Neto, J. C., da Silva, F. Q. B., & Saher, A. (2018). Computer games are serious business and so is their quality: particularities of software testing in game development from the perspective of practitioners. arXiv preprint arXiv:1812.05164.
[25]
Tuteja, M., & Dubey, G. (2012). A research study on importance of testing and quality assurance in software development life cycle (SDLC) models. International Journal of Soft Computing and Engineering (IJSCE), 2(3), 251--257.
[26]
Sandhu, G.K. and Kaur, R., 2016. Effective test case generation for load testing of web server using river formation dynamics (Doctoral dissertation).
[27]
Kurokawa, T., & Shinagawa, M. (2008). Technical trends and challenges of software testing. NISTEP Science & Technology Foresight Center.
[28]
Kontio, Jyrki, Laura Lehtola, and Johanna Bragge. "Using the focus group method in software engineering: obtaining practitioner and user experiences." In Empirical Software Engineering, 2004. ISESE'04. Proceedings. 2004 International Symposium on, pp. 271--280. IEEE, 2004.
[29]
Powell, Richard A., and Helen M. Single. "Focus groups." International journal for quality in health care 8.5 (1996): 499--504.
[30]
Baldassarre, M.T., Caivano, D., Visaggio, G. Empirical studies for innovation dissemination: Ten years of experience (2013) EASE - ACM International Conference Proceeding Series, pp. 144--152.
[31]
Ardimento, P., Caivano, D., Cimitile, M., Visaggio, G. Empirical investigation of the efficacy and efficiency of tools for transferring software engineering knowledge (2008) Journal of Information and Knowledge Management, 7 (3), pp. 197--207.
[32]
Santos, R. E. S.; Da Silva, F. Q. B.; Baldassarre, M. T.; Magalhães, C. V. C. Benefits and Limitations of Project-to-Project Job Rotation in Software Organizations: A Synthesis of Evidence. Information and Software Technology, 2017.

Cited By

View all
  • (2024)Testing the past: can we still run tests in past snapshots for Java projects?Empirical Software Engineering10.1007/s10664-024-10530-z29:5Online publication date: 30-Jul-2024
  • (2023)Myths and Facts About a Career in Software Testing: A Comparison Between Students’ Beliefs and Professionals’ ExperienceIEEE Software10.1109/MS.2023.326729640:5(76-84)Online publication date: 1-Sep-2023
  • (2019)Implications of Resurgence in Artificial Intelligence for Research Collaborations in Software EngineeringACM SIGSOFT Software Engineering Notes10.1145/3356773.335681344:3(68-70)Online publication date: 14-Nov-2019

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CESSER-IP '19: Proceedings of the Joint 7th International Workshop on Conducting Empirical Studies in Industry and 6th International Workshop on Software Engineering Research and Industrial Practice
May 2019
57 pages

Sponsors

Publisher

IEEE Press

Publication History

Published: 27 May 2019

Check for updates

Author Tags

  1. industry
  2. software engineering
  3. software testing

Qualifiers

  • Research-article

Conference

ICSE '19
Sponsor:

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)0
Reflects downloads up to 10 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Testing the past: can we still run tests in past snapshots for Java projects?Empirical Software Engineering10.1007/s10664-024-10530-z29:5Online publication date: 30-Jul-2024
  • (2023)Myths and Facts About a Career in Software Testing: A Comparison Between Students’ Beliefs and Professionals’ ExperienceIEEE Software10.1109/MS.2023.326729640:5(76-84)Online publication date: 1-Sep-2023
  • (2019)Implications of Resurgence in Artificial Intelligence for Research Collaborations in Software EngineeringACM SIGSOFT Software Engineering Notes10.1145/3356773.335681344:3(68-70)Online publication date: 14-Nov-2019

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media