Perceptions on the Utility of Community Question and Answer Websites Like Stack Overflow to Software Developers
Pages 2413 - 2425
Abstract
Software developers make use of on crowdsourcing during development. Beyond learning from others, developers use online portals such as Stack Overflow as a vehicle for collaboration. However, little is known about developers’ experiences on such platforms, particularly around problems that are encountered online. Such insights could benefit software developers in terms of recommendations for pitfalls to avoid, ways to exploit crowdsourced knowledge, and the provision of insights to improve online code sharing communities. We interviewed 50 practitioners to fill this gap, where outcomes show that software developers’ use of online portals is targeted, and such portals are a lifeline to modern software development. Practitioners are facilitated with code solutions and debugging, often in a very timely fashion. While these experiences are largely positive, practitioners also encounter negative experiences online, some of which could be significantly deleterious to the community. We discuss the implications of these findings, such as creating awareness of the quality and reliability of code snippets, improving code searches, code validation and outdated code detection and attribution of code snippets.
References
[1]
S. Beyer, C. Macho, M. Di Penta, and M. Pinzger, “Automatically classifying posts into question categories on stack overflow,” in Proc. IEEE/ACM 26th Int. Conf. on Prog. Comprehension, 2018, pp. 211–21 110.
[2]
E. C. Wenger and W. M. Snyder, “Communities of practice: The organizational frontier,” Harvard Bus. Rev., vol. 78, no. 1, pp. 139–146, 2000.
[3]
S. Sengupta and C. Haythornthwaite, “Learning with comments: An analysis of comments and community on stack overflow,” in Proc. 53rd Hawaii Int. Conf. Syst. Sci., 2020, pp. 2898–2907.
[4]
D. Yu, Z. Zhou, and Y. Wang, “Crowdsourcing software task assignment method for collaborative development,” IEEE Access, vol. 7, pp. 35 743–35 754, 2019.
[5]
P. Chatterjee, M. Kong, and L. Pollock, “Finding help with programming errors: An exploratory study of novice software engineers’ focus in stack overflow posts,” J. Syst. Softw., vol. 159, 2020, Art. no.
[6]
U. F. Campos, G. Smethurst, J. P. Moraes, R. Bonifácio, and G. Pinto, “Mining rule violations in JavasCript code snippets,” in Proc. IEEE/ACM 16th Int. Conf. Mining Softw. Repositories, 2019, pp. 195–199.
[7]
S. Penoyer, B. Reynolds, B. Marshall, and P. W. Cardon, “Impact of users’motivation on gamified crowdsourcing systems: A case of stackoverflow,” Issues Informat. Syst., vol. 19, no. 2, 2018, pp. 33–40.
[8]
C. Ragkhitwetsagul, J. Krinke, M. Paixao, G. Bianco, and R. Oliveto, “Toxic code snippets on stack overflow,” IEEE Trans. Softw. Eng., vol. 47, no. 3, pp. 560–581, Mar. 2021.
[9]
I. G. Ndukwe, S. A. Licorish, A. Tahir, and S. G. MacDonell, “How have views on software quality differed over time? Research and practice viewpoints,” J. Syst. Softw., vol. 195, 2022, Art. no.
[10]
R. Abdalkareem, E. Shihab, and J. Rilling, “On code reuse from stackoverflow: An exploratory study on Android apps,” Informat. Softw. Technol., vol. 88, pp. 148–158, 2017.
[11]
X. Xia, L. Bao, D. Lo, P. S. Kochhar, A. E. Hassan, and Z. Xing, “What do developers search for on the web?,” Empir. Softw. Eng., vol. 22, no. 6, pp. 3149–3185, 2017.
[12]
D. Palomera and A. Figueroa, “Leveraging linguistic traits and semi-supervised learning to single out informational content across how-to community question-answering archives,” Informat. Sci., vol. 381, pp. 20–32, 2017.
[13]
C. Parnin and C. Treude, “Measuring API documentation on the web,” in Proc. 2nd Int. Workshop Web 2.0 Softw. Eng., 2011, pp. 25–30.
[14]
A. Barua, S. W. Thomas, and A. E. Hassan, “What are developers talking about? An analysis of topics and trends in stack overflow,” Empir. Softw. Eng., vol. 19, no. 3, pp. 619–654, 2014.
[15]
M. Squire, ““should we move to stack overflow?” Measuring the utility of social media for developer support,” in Proc. IEEE/ACM 37th Int. Conf. Softw. Eng., 2015, pp. 219–228.
[16]
Q. Huang, X. Xia, Z. Xing, D. Lo, and X. Wang, “API method recommendation without worrying about the task-API knowledge gap,” in Proc. IEEE/ACM 33rd Int. Conf. Automated Softw. Eng., 2018, pp. 293–304.
[17]
S. Meldrum, S. A. Licorish, and B. T. R. Savarimuthu, “Exploring research interest in stack overflow–a systematic mapping study and quality evaluation,” 2020,.
[18]
F. Fischeret al., “Stack overflow considered harmful? The impact of copy&paste on Android application security,” in Proc. IEEE Symp. Secur. Privacy, 2017, pp. 121–136.
[19]
Y. Acar, M. Backes, S. Fahl, D. Kim, M. L. Mazurek, and C. Stransky, “You get where you’re looking for: The impact of information sources on code security,” in Proc. IEEE Symp. Secur. Privacy, 2016, pp. 289–305.
[20]
S. M. Nasehi, J. Sillito, F. Maurer, and C. Burns, “What makes a good code example?: A study of programming Q&A in stackoverflow,” in Proc. IEEE 28th Int. Conf. Softw. Maintenance, 2012, pp. 25–34.
[21]
C. Treude and M. P. Robillard, “Augmenting API documentation with insights from stack overflow,” in Proc. IEEE/ACM 38th Int. Conf. Softw. Eng., 2016, pp. 392–403.
[22]
L. Ponzanelli, G. Bavota, M. Di Penta, R. Oliveto, and M. Lanza, “Mining stackoverflow to turn the IDE into a self-confident programming prompter,” in Proc. 11th Work. Conf. Mining Softw. Repositories, 2014, pp. 102–111.
[23]
Y. Jin, X. Yang, R. G. Kula, E. Choi, K. Inoue, and H. Iida, “Quick trigger on stack overflow: A study of gamification-influenced member tendencies,” in Proc. IEEE/ACM 12th Work. Conf. Mining Softw. Repositories, 2015, pp. 434–437.
[24]
D. Anand and S. Ravichandran, “Investigations into the goodness of posts in Q&A forums–popularity versus quality,” in Information Systems Design and Intelligent Applications. Berlin, Germany: Springer, 2015, pp. 639–647.
[25]
G. Digkas, N. Nikolaidis, A. Ampatzoglou, and A. Chatzigeorgiou, “Reusing code from stackoverflow: The effect on technical debt,” in Proc. IEEE 45th Euromicro Conf. Softw. Eng. Adv. Appl., 2019, pp. 87–91.
[26]
C. Treude and M. P. Robillard, “Understanding stack overflow code fragments,” in Proc. IEEE Int. Conf. Softw. Maintenance Evol., 2017, pp. 509–513.
[27]
Y. Wu, S. Wang, C.-P. Bezemer, and K. Inoue, “How do developers utilize source code from stack overflow?,” Empir. Softw. Eng., vol. 24, no. 2, pp. 637–673, 2019.
[28]
C. Shah, S. Oh, and J. S. Oh, “Research agenda for social Q&A,” Library Informat. Sci. Res., vol. 31, no. 4, pp. 205–209, 2009.
[29]
C. Shah, V. Kitzie, and E. Choi, “Modalities, motivations, and materials–investigating traditional and social online Q&A services,” J. Informat. Sci., vol. 40, no. 5, pp. 669–687, 2014.
[30]
I. Srba and M. Bielikova, “A comprehensive survey and classification of approaches for community question answering,” ACM Trans. Web, vol. 10, no. 3, pp. 1–63, 2016.
[31]
S. E. Hove and B. Anda, “Experiences from conducting semi-structured interviews in empirical software engineering research,” in Proc. IEEE 11th Int. Softw. Metrics Symp., 2005, pp. 10–pp.
[32]
P. Runeson and M. Höst, “Guidelines for conducting and reporting case study research in software engineering,” Empir. Softw. Eng., vol. 14, no. 2, pp. 131–164, 2009.
[33]
W. Bai, O. Akgul, and M. L. Mazurek, “A qualitative investigation of insecure code propagation from online forums,” in Proc. IEEE Cybersecurity Develop., 2019, pp. 34–48.
[34]
S. Agostinho, “Naturalistic inquiry in e-learning research,” Int. J. Qualitative Methods, vol. 4, no. 1, pp. 13–26, 2005.
[35]
M. Q. Patton, Qualitative Evaluation and Research Methods. Newbury Park, CA, USA: SAGE, 1990.
[36]
V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qualitative Res. Psychol., vol. 3, no. 2, pp. 77–101, 2006.
[37]
F. K. Thang and J. H. L. Koh, “Deepening and transferring twenty-first century learning through a lower secondary integrated science module,” Learn.: Res. Pract., vol. 3, no. 2, pp. 148–162, 2017.
[38]
H.-F. Hsieh and S. E. Shannon, “Three approaches to qualitative content analysis,” Qualitative Health Res., vol. 15, no. 9, pp. 1277–1288, 2005.
[39]
W. J. Potter and D. Levine-Donnerstein, “Rethinking validity and reliability in content analysis,” J. Appl. Commun. Res., vol. 27, pp. 258–284, 1999.
[40]
E. R. Babbie, The Practice of Social Research. Boston, MA, USA: Cengage learning, 2020.
[41]
J. Cohen, Statistical Power Analysis for the Behavioral Sciences. New York, NY, USA: Academic, 2013.
[42]
L. Martie, A. V. D. Hoek, and T. Kwak, “Understanding the impact of support for iteration on code search,” in Proc. 11th Joint Meeting Found. Softw. Eng., 2017, pp. 774–785.
[43]
S. P. Reiss, “Semantics-based code search,” in Proc. IEEE 31st Int. Conf. Softw. Eng., 2009, pp. 243–253.
[44]
S. E. Sim, M. Umarji, S. Ratanotayanon, and C. V. Lopes, “How well do search engines support code retrieval on the web?,” ACM Trans. Softw. Eng. Methodol., vol. 21, no. 1, pp. 1–25, 2011.
[45]
H. Zhang, S. Wang, T.-H. Chen, and A. E. Hassan, “Reading answers on stack overflow: Not enough!,” IEEE Trans. Softw. Eng., vol. 47, no. 11, pp. 2520–2533, Nov. 2021.
[46]
P. Chatterjee, “Automatic identification of informative code in stack overflow posts,” in Proc. IEEE/ACM 1st Int. Workshop Natural Lang.Based Softw. Eng., 2022, pp. 21–24.
[47]
S. Nadi and C. Treude, “Essential sentences for navigating stack overflow answers,” in Proc. IEEE 27th Int. Conf. Softw. Anal., Evol. Reengineering, 2020, pp. 229–239.
[48]
B. Xu, Z. Xing, X. Xia, and D. Lo, “AnswerBot: Automated generation of answer summary to developers’ technical questions,” in Proc. IEEE/ACM 32nd Int. Conf. Automated Softw. Eng., 2017, pp. 706–716.
[49]
M. Asaduzzaman, A. S. Mashiyat, C. K. Roy, and K. A. Schneider, “Answering questions about unanswered questions of stack overflow,” in Proc. IEEE 10th Work. Conf. Mining Softw. Repositories, 2013, pp. 97–100.
[50]
F. Calefato, F. Lanubile, and N. Novielli, “How to ask for technical help? Evidence-based guidelines for writing questions on stack overflow,” Informat. Softw. Technol., vol. 94, pp. 186–207, 2018.
[51]
T. Althoff, C. Danescu-Niculescu-Mizil, and D. Jurafsky, “How to ask for a favor: A case study on the success of altruistic requests,” in Proc. Int. AAAI Conf. Web Social Media, 2014, pp. 12–21.
[52]
C. Treude, O. Barzilay, and M.-A. Storey, “How do programmers ask and answer questions on the web?(Nier track),” in Proc. 33rd Int. Conf. Softw. Eng., 2011, pp. 804–807.
[53]
R. Abdalkareem, E. Shihab, and J. Rilling, “What do developers use the crowd for? A study using stack overflow,” IEEE Softw., vol. 34, no. 2, pp. 53–60, 2017.
[54]
C. S. Peterson, “Investigating the effect of polyglot programming on developers,” in Proc. IEEE Symp. Vis. Lang. Hum.-Centric Comput., 2021, pp. 1–2.
[55]
N. Shrestha, C. Botta, T. Barik, and C. Parnin, “Here we go again: Why is it difficult for developers to learn another programming language?,” in Proc. IEEE/ACM 42nd Int. Conf. Softw. Eng., 2020, pp. 691–701.
[56]
A. Lotter, S. A. Licorish, B. T. R. Savarimuthu, and S. Meldrum, “Code reuse in stack overflow and popular open source Java projects,” in Proc. IEEE 25th Australas. Softw. Eng. Conf., 2018, pp. 141–150.
[57]
M. Choetkiertikul, D. Avery, H. K. Dam, T. Tran, and A. Ghose, “Who will answer my question on stack overflow?,” in Proc. IEEE 24th Australas. Softw. Eng. Conf., 2015, pp. 155–164.
[58]
E. Zolduoarrati, S. A. Licorish, and N. Stanger, “Impact of individualism and collectivism cultural profiles on the behaviour of software developers: A study of stack overflow,” J. Syst. Softw., vol. 192, 2022, Art. no.
[59]
E. Zolduoarrati and S. A. Licorish, “On the value of encouraging gender tolerance and inclusiveness in software engineering communities,” Informat. Softw. Technol., vol. 139, 2021, Art. no.
[60]
R. P. Buse and W. Weimer, “Synthesizing api usage examples,” in Proc. IEEE 34th Int. Conf. Softw. Eng., 2012, pp. 782–792.
[61]
D. Yang, A. Hussain, and C. V. Lopes, “From query to usable code: An analysis of stack overflow code snippets,” in Proc. IEEE/ACM 13th Work. Conf. Mining Softw. Repositories, 2016, pp. 391–401.
[62]
S. Meldrum, S. A. Licorish, C. A. Owen, and B. T. R. Savarimuthu, “Understanding stack overflow code quality: A recommendation of caution,” Sci. Comput. Program., vol. 199, 2020, Art. no.
[63]
P. Xia, M. Matsushita, N. Yoshida, and K. Inoue, “Studying reuse of out-dated third-party code in open source projects,” Informat. Media Technol., vol. 9, no. 2, pp. 155–161, 2014.
[64]
V. Terragni, Y. Liu, and S.-C. Cheung, “CSNIPPEX: Automated synthesis of compilable code snippets from Q&A sites,” in Proc. 25th Int. Symp. Softw. Testing Anal., 2016, pp. 118–129.
[65]
S. Zhou, B. Shen, and H. Zhong, “Lancer: Your code tell me what you need,” in Proc. IEEE/ACM 34th Int. Conf. Automated Softw. Eng., 2019, pp. 1202–1205.
[66]
S. A. Licorish and M. Wagner, “Combining gin and PMD for code improvements,” in Proc. Genet. Evol. Comput. Conf. Companion, 2022, pp. 790–793.
[67]
S. Baltes and S. Diehl, “Usage and attribution of stack overflow code snippets in GitHub projects,” Empir. Softw. Eng., vol. 24, no. 3, pp. 1259–1295, 2019.
[68]
E. Kalliamvakou, D. Damian, K. Blincoe, L. Singer, and D. M. German, “Open source-style collaborative development practices in commercial projects using GitHub,” in Proc. IEEE/ACM 37th Int. Conf. Softw. Eng., 2015, pp. 574–585.
[69]
S. Wang, T.-H. Chen, and A. E. Hassan, “How do users revise answers on technical Q&A websites? A case study on stack overflow,” IEEE Trans. Softw. Eng., vol. 46, no. 9, pp. 1024–1038, Sep. 2020.
[70]
M. N. Marshall, “Sampling for qualitative research,” Fam. Pract., vol. 13, no. 6, pp. 522–526, 1996.
[71]
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” 2018,.
[72]
Introducing the world's largest open multilingual language model: Bloom. Accessed: Aug.31, 2022. [Online]. Available: https://huggingface.co/blog/bloom
Recommendations
Towards exploring the code reuse from stack overflow during software development
ICPC '22: Proceedings of the 30th IEEE/ACM International Conference on Program ComprehensionAs one of the most well-known programmer Q&A websites, Stack Overflow (i.e., SO) is serving tens of thousands of developers every day. Previous work has shown that many developers reuse the code snippets on SO when they find an answer (from SO) that ...
How do developers utilize source code from stack overflow?
Technical question and answer Q&A platforms, such as Stack Overflow, provide a platform for users to ask and answer questions about a wide variety of programming topics. These platforms accumulate a large amount of knowledge, including hundreds of ...
Improving the quality of code snippets in stack overflow
SAC '16: Proceedings of the 31st Annual ACM Symposium on Applied ComputingQuestion and answer (Q&A) websites like Stack Overflow are one of the important sources of code examples in which developers can ask their questions and leave their answers about programming issues. Since the number of programmers who use these websites ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
0098-5589 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 01 April 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Jan 2025