More Web Proxy on the site http://driver.im/

research-article

Supporting Developers in Addressing Human-Centric Issues in Mobile Apps

Authors:

Hourieh Khalajzadeh,

Mojtaba Shahin,

Humphrey O. Obie,

Pragya Agrawal,

John GrundyAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 49, Issue 4

Pages 2149 - 2168

https://doi.org/10.1109/TSE.2022.3212329

Published: 01 April 2023 Publication History

Abstract

Failure to consider the characteristics, limitations, and abilities of diverse end-users during mobile app development may lead to problems for end-users, such as accessibility and usability issues. We refer to this class of problems as <italic>human-centric issues</italic>. Despite their importance, there is a limited understanding of the types of human-centric issues that are encountered by end-users and taken into account by the developers of mobile apps. In this paper, we examine what human-centric issues end-users report through Google App Store reviews, what human-centric issues are a topic of discussion for developers on GitHub, and whether end-users and developers discuss the same human-centric issues. We then investigate whether an automated tool might help detect such human-centric issues and whether developers would find such a tool useful. To do this, we conducted an empirical study by extracting and manually analysing a random sample of 1,200 app reviews and 1,200 issue comments from 12 diverse projects that exist on both Google App Store and GitHub. Our analysis led to a taxonomy of human-centric issues that characterises human-centric issues into three-high level categories: App Usage, Inclusiveness, and User Reaction. We then developed machine learning and deep learning models that are promising in automatically identifying and classifying human-centric issues from app reviews and developer discussions. A survey of mobile app developers shows that the automated detection of human-centric issues has practical applications. Guided by our findings, we highlight some implications and possible future work to further understand and better incorporate addressing human-centric issues into mobile app development.

References

[1]

J. Grundy, H. Khalajzadeh, and J. Mcintosh, “Towards human-centric model-driven software engineering,” in Proc. 15th Int. Conf. Eval. Novel Approaches Softw. Eng., 2020, pp. 229–238.

[2]

K. Hartzel, “How self-efficacy and gender issues affect software adoption and use,” Commun. ACM, vol. 46, no. 9, pp. 167–171, 2003.

Digital Library

[3]

T. Miller, S. Pedell, A. A. Lopez-Lorca, A. Mendoza, L. Sterling, and A. Keirnan, “Emotion-led modelling for people-oriented requirements engineering: The case study of emergency systems,” J. Syst. Softw., vol. 105, pp. 54–71, 2015.

Digital Library

[4]

S. E. Stock, D. K. Davies, M. L. Wehmeyer, and S. B. Palmer, “Evaluation of cognitively accessible software to increase independent access to cellphone technology for people with intellectual disability,” J. Intellectual Disabil. Res., vol. 52, no. 12, pp. 1155–1164, 2008.

[5]

S. Wirtz, E.-M. Jakobs, and M. Ziefle, “Age-specific usability issues of software interfaces,” in Proc. 17th World Congr. Ergonom., 2009.

[6]

O. Kulyk, R. Kosara, J. Urquiza, and I. Wassink, “Human-centered aspects,” in Human-Centered Visualization Environments. Berlin, Germany: Springer, 2007, pp. 13–75.

[7]

J. Brunet, G. C. Murphy, R. Terra, J. Figueiredo, and D. Serey, “Do developers discuss design?,” in Proc. 11th Work. Conf. Mining Softw. Repositories, 2014, pp. 340–343.

[8]

J. Tsay, L. Dabbish, and J. Herbsleb, “Let's talk about it: Evaluating contributions through discussion in GitHub,” in Proc. 22nd ACM SIGSOFT Int. Symp. Found. Softw. Eng., 2014, pp. 144–154.

[9]

W. Mo, B. Shen, Y. Chen, and J. Zhu, “TBIL: A tagging-based approach to identity linkage across software communities,” in Proc. IEEE Asia-Pacific Softw. Eng. Conf., 2015, pp. 56–63.

[10]

H. Khalajzadeh, M. Shahin, H. O. Obie, and J. Grundy, “How are diverse end-user human-centric issues discussed on GitHub?,” 2022,.

[11]

H. Khalajzadeh, M. Shahin, H. Obie, P. Agrawal, and J. Grundy, “Supporting developers in addressing human-centric issues in mobile apps [Data set],” 2022. [Online]. Available: https://doi.org/10.5281/zenodo.6982529

[12]

J. Grundy, H. Khalajzadeh, T. Kanij, and I. Mueller, “HumaniSE: Approaches to achieve more human-centric software engineering,” in Proc. 15th Int. Conf. Eval. Novel Approaches Softw. Eng., 2020, Art. no.

[13]

A. R. Nasabet al., “Automated identification of security discussions in microservices systems: Industrial surveys and experiments,” J. Syst. Softw., vol. 181, 2021, Art. no.

[14]

G. A. A. Prana, C. Treude, F. Thung, T. Atapattu, and D. Lo, “Categorizing the content of GitHub README files,” Empir. Softw. Eng., vol. 24, no. 3, pp. 1296–1327, 2019.

Digital Library

[15]

S. Abualhaija, C. Arora, M. Sabetzadeh, L. C. Briand, and M. Traynor, “Automated demarcation of requirements in textual specifications: A machine learning-based approach,” Empir. Softw. Eng., vol. 25, no. 6, pp. 5454–5497, 2020.

Digital Library

[16]

A. Mazuera-Rozo, C. Trubiani, M. Linares-Vásquez, and G. Bavota, “Investigating types and survivability of performance bugs in mobile apps,” Empir. Softw. Eng., vol. 25, no. 3, pp. 1644–1686, 2020.

Digital Library

[17]

Open-source Android apps project. Jul.2021. [Online]. Available: https://github.com/pcqpcq/open-source-android-apps

[18]

Open-source IOS apps. Jul.2021. [Online]. Available: https://github.com/dkhamsing/open-source-ios-apps

[19]

C. Li, H. O. Obie, and H. Khalajzadeh, “A first step towards detecting values-violating defects in Android APIs,” 2021,.

[20]

B. G. Glaser and A. L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research. Evanston, IL, USA: Routledge, 2017.

[21]

N. Humbatova, G. Jahangirova, G. Bavota, V. Riccio, A. Stocco, and P. Tonella, “Taxonomy of real faults in deep learning systems,” in Proc. IEEE/ACM 42nd Int. Conf. Softw. Eng., 2020, pp. 1110–1121.

[22]

J.-Y. Mao, K. Vredenburg, P. W. Smith, and T. Carey, “The state of user-centered design practice,” Commun. ACM, vol. 48, no. 3, pp. 105–109, 2005.

Digital Library

[23]

S. Gupta and A. Gupta, “Dealing with noise problem in machine learning data-sets: A systematic review,” Procedia Comput. Sci., vol. 161, pp. 466–474, 2019.

Digital Library

[24]

GeeksforGeeks, “Snowball stemmer – NLP,” 2020. [Online]. Available: https://www.geeksforgeeks.org/snowball-stemmer-nlp/

[25]

S. Paul, “A detailed case study on multi-label classification with machine learning algorithms and predicting movie tags based on plot summaries!,” 2019. [Online]. Available: https://medium.com/@saugata.paul1010/a-detailed-case-study-on-multi-label-classification-with-machine-learning-algorithms-and-72031742c9aa

[26]

D. S. S. Exchange, “Word2Vec embeddings with TF-IDF,” 2018. [Online]. Available: https://datascience.stackexchange.com/questions/28598/word2vec-embeddings-with-tf-idf

[27]

C. Prathibhamol, K. Jyothy, and B. Noora, “Multi label classification based on logistic regression (MLC-LR),” in Proc. IEEE Int. Conf. Adv. Comput. Commun. Informat., 2016, pp. 2708–2712.

[28]

K. P. Murphy, Probabilistic Machine Learning: An Introduction. Cambridge, MA, USA: MIT Press, 2022.

[29]

J. Read, B. Pfahringer, G. Holmes, and E. Frank, “Classifier chains for multi-label classification,” Mach. Learn., vol. 85, no. 3, pp. 333–359, 2011.

Digital Library

[30]

H. Liu, S. Zhang, and X. Wu, “MLSLR: Multilabel learning via sparse logistic regression,” Inf. Sci., vol. 281, pp. 310–320, 2014.

Digital Library

[31]

T. Li, C. Zhang, and S. Zhu, “Empirical studies on multi-label classification,” in Proc. IEEE 18th Int. Conf. Tools Artif. Intell., 2006, pp. 86–92.

[32]

S. Kouchakiet al., “Multi-label random forest model for tuberculosis drug resistance classification and mutation ranking,” Front. Microbiol., vol. 11, 2020, Art. no.

[33]

M. Chen, Q. Liu, S. Chen, Y. Liu, C.-H. Zhang, and R. Liu, “XGBoost-based algorithm interpretation and application on post-fault transient stability status prediction of power system,” IEEE Access, vol. 7, pp. 13 149–13 158, 2019.

[34]

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” 2018,.

[35]

Y. Liuet al., “RoBERTa: A robustly optimized BERT pretraining approach,” 2019,.

[36]

V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” 2019,.

[37]

P. Bhargava, A. Drozd, and A. Rogers, “Generalization in NLI: Ways (not) to go beyond simple heuristics,” 2021,.

[38]

Huggingface, “Preprocessing data,” 2021. [Online]. Available: https://huggingface.co/docs/transformers/preprocessing

[39]

C. Mathewset al., “AH-CID: A tool to automatically detect human-centric issues in app,” in Proc. 16th Int. Conf. Softw. Technol., 2021, pp. 386–397.

[40]

M. S. Sorower, “A literature survey on algorithms for multi-label learning,” Oregon State University, Corvallis, vol. 18, pp. 1–25, 2010.

[41]

P. Jadhav, “Valuation metrics for multi label classification,” 2022. [Online]. Available: https://medium.datadriveninvestor.com/a-survey-of-evaluation-metrics-for-multilabel-classification-bb16e8cd41cd

[42]

B. A. Kitchenham and S. L. Pfleeger, “Personal opinion surveys,” in Guide to Advanced Empirical Software Engineering, Berlin, Germany: Springer, 2008, pp. 63–92.

[43]

D. N. Palacio, D. McCrystal, K. Moran, C. Bernal-Cárdenas, D. Poshyvanyk, and C. Shenefiel, “Learning to identify security-related issues using convolutional neural networks,” in Proc. IEEE 35th Int. Conf. Softw. Maintenance Evol., 2019, pp. 140–144.

[44]

G. Ramos, C. Ponting, J. P. Labao, and K. Sobowale, “Considerations of diversity, equity, and inclusion in mental health apps: A scoping review of evaluation frameworks,” Behav. Res. Ther., vol. 147, 2021, Art. no.

[45]

R. S. Valdez, M. C. Gibbons, E. R. Siegel, R. Kukafka, and P. F. Brennan, “Designing consumer health it to enhance usability among different racial and ethnic groups within the united states,” Health Technol., vol. 2, no. 4, pp. 225–233, 2012.

[46]

N. S. M. Yusop, J. Grundy, and R. Vasa, “Reporting usability defects: A systematic literature review,” IEEE Trans. Softw. Eng., vol. 43, no. 9, pp. 848–867, Sep. 2017.

Digital Library

[47]

K. Huynhet al., “Improving human-centric software defect evaluation, reporting, and fixing,” in Proc. IEEE 45th Annu. Comput. Softw. Appl. Conf., 2021, pp. 408–417.

[48]

A. Alshayban, I. Ahmed, and S. Malek, “Accessibility issues in Android apps: State of affairs, sentiments, and ways forward,” in Proc. IEEE/ACM 42nd Int. Conf. Softw. Eng., 2020, pp. 1323–1334.

[49]

A. di Sorboet al., “What would users change in my app? Summarizing app reviews for recommending software changes,” in Proc. 24th ACM SIGSOFT Int. Symp. Found. Softw. Eng., 2016, pp. 499–510.

[50]

S. Malgaonkar, S. A. Licorish, and B. T. R. Savarimuthu, “Prioritizing user concerns in app reviews–A study of requests for new features, enhancements and bug fixes,” Inf. Softw. Technol., vol. 144, 2022, Art. no.

[51]

Y. Liu, C. Xu, and S.-C. Cheung, “Characterizing and detecting performance bugs for smartphone applications,” in Proc. 36th Int. Conf. Softw. Eng., 2014, pp. 1013–1024.

[52]

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering, Berlin, Germany: Springer, 2012.

[53]

F. Peters, T. T. Tun, Y. Yu, and B. Nuseibeh, “Text filtering and ranking for security bug report prediction,” IEEE Trans. Softw. Eng., vol. 45, no. 6, pp. 615–631, Jun. 2019.

[54]

Y. Xu and R. Goodacre, “On splitting training and validation set: A comparative study of cross-validation, bootstrap and systematic sampling for estimating the generalization performance of supervised learning,” J. Anal. Testing, vol. 2, no. 3, pp. 249–262, 2018.

[55]

M. Ortuet al., “The emotional side of software developers in JIRA,” in Proc. IEEE/ACM 13th Int. Conf. Mining Softw. Repositories, 2016, pp. 480–483.

[56]

E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian, “The promises and perils of mining GitHub,” in Proc. 11th Work. Conf. Mining Softw. Repositories, 2014, pp. 92–101.

[57]

M. Roccetti, C. Prandi, S. Mirri, and P. Salomoni, “Designing human-centric software artifacts with future users: A case study,” Hum.-Centric Comput. Inf. Sci., vol. 10, pp. 1–17, 2020.

[58]

I. Rauf, D. van der Linden, M. Levine, J. Towse, B. Nuseibeh, and A. Rashid, “Security but not for security's sake: The impact of social considerations on app developers’ choices,” in Proc. IEEE/ACM 42nd Int. Conf. Softw. Eng. Workshops, 2020, pp. 141–144.

[59]

A.-S. Cheng and K. R. Fleischmann, “Developing a meta-inventory of human values,” in Proc. 73rd ASIS&T Annu. Meeting Navigating Streams Inf. Ecosyst., 2010, Art. no.

[60]

J. Grundyet al., “Addressing the influence of end user human aspects on software engineering,” in Proc. Int. Conf. Eval. Novel Approaches Softw. Eng., 2022, pp. 241–264.

[61]

W. Hussainet al., “Human values in software engineering: Contrasting case studies of practice,” IEEE Trans. Softw. Eng., vol. 48, no. 5, pp. 1818–1833, May 2022.

Digital Library

[62]

J. Whittle, M. A. Ferrario, W. Simm, and W. Hussain, “A case for human values in software engineering,” IEEE Softw., vol. 38, no. 1, pp. 106–113, Jan./Feb. 2021.

Digital Library

[63]

W. Hussainet al., “How can human values be addressed in AgileMethods a case study on SAFe,” IEEE Trans. Softw. Eng., early access, Jan. 11, 2022.

[64]

A. Nurwidyantoroet al., “Towards a human values dashboard for software development: An exploratory study,” in Proc. IEEE/ACM Int. Symp. Empir. Softw. Eng. Meas., 2021, pp. 23:1–23:12.

[65]

E. Winter, S. Forshaw, and M. A. Ferrario, “Measuring human values in software engineering,” in Proc. IEEE/ACM 12th Int. Symp. Empir. Softw. Eng. Meas., 2018, pp. 1–4.

[66]

D. Pletea, B. Vasilescu, and A. Serebrenik, “Security and emotion: Sentiment analysis of security discussions on GitHub,” in Proc. 11th Work. Conf. Mining Softw. Repositories, 2014, pp. 348–351.

[67]

A. J. Ko and P. K. Chilana, “Design, discussion, and dissent in open bug reports,” in Proc. iConf., 2011, pp. 106–113.

[68]

M. B. Twidale and D. M. Nichols, “Exploring usability discussions in open source development,” in Proc. IEEE 38th Annu. Hawaii Int. Conf. Syst. Sci., 2005, pp. 198c–198c.

[69]

M. S. Andreasen, H. V. Nielsen, S. O. Schrøder, and J. Stage, “Usability in open source software development: Opinions and practice,” Inf. Technol. Control, vol. 35, no. 3, pp. 303–312, 2006.

[70]

L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb, “Social coding in GitHub: Transparency and collaboration in an open software repository,” in Proc. ACM Conf. Comput. Supported Cooperative Work, 2012, pp. 1277–1286.

[71]

H. K. Dam, B. T. R. Savarimuthu, D. Avery, and A. Ghose, “Mining software repositories for social norms,” in Proc. IEEE/ACM 37th Int. Conf. Softw. Eng., 2015, pp. 627–630.

[72]

F. Barcellini, F. Détienne, J.-M. Burkhardt, and W. Sack, “A socio-cognitive analysis of online design discussions in an open source software community,” Interacting Comput., vol. 20, no. 1, pp. 141–165, 2008.

Digital Library

[73]

M. Ortu, G. Destefanis, B. Adams, A. Murgia, M. Marchesi, and R. Tonelli, “The JIRA repository dataset: Understanding social aspects of software development,” in Proc. 11th Int. Conf. Predictive Models Data Analytics Softw. Eng., 2015, Art. no.

[74]

L. A. Cabrera-Diego, N. Bessis, and I. Korkontzelos, “Classifying emotions in stack overflow and JIRA using a multi-label approach,” Knowl.-Based Syst., vol. 195, 2020, Art. no. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0950705120300939

[75]

A. Murgia, P. Tourani, B. Adams, and M. Ortu, “Do developers feel emotions? An exploratory analysis of emotions in software artifacts,” in Proc. 11th Work. Conf. Mining Softw. Repositories, 2014, pp. 262–271.

[76]

B. Fredrickson, “The role of positive emotions in positive psychology. The broaden-and-build theory of positive emotions,” Amer. Psychol., vol. 56, no. 3, pp. 218–26, 2001.

[77]

F. P. Brooks, “No silver bullet essence and accidents of software engineering,” Computer, vol. 20, no. 4, pp. 10–19, Apr. 1987.

Digital Library

[78]

H. Khalid, E. Shihab, M. Nagappan, and A. E. Hassan, “What do mobile app users complain about?,” IEEE Softw., vol. 32, no. 3, pp. 70–77, May/Jun. 2015.

Digital Library

[79]

S. McIlroy, N. Ali, H. Khalid, and A. E. Hassan, “Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews,” Empir. Softw. Eng., vol. 21, no. 3, pp. 1067–1106, 2016.

Digital Library

[80]

Q. Chen, C. Chen, S. Hassan, Z. Xing, X. Xia, and A. E. Hassan, “How should I improve the UI of my app? A study of user reviews of popular apps in the Google Play,” ACM Trans. Softw. Eng. Methodol., vol. 30, no. 3, pp. 1–38, 2021.

Digital Library

[81]

H. O. Obieet al., “A first look at human values-violation in app reviews,” in Proc. IEEE/ACM 43rd Int. Conf. Softw. Eng.: Softw. Eng. Soc., 2021, pp. 29–38.

[82]

M. Fazziniet al., “Characterizing human aspects in reviews of COVID-19 apps,” in Proc. IEEE/ACM 9th Int. Conf. Mobile Softw. Eng. Syst., 2022, pp. 38–49.

[83]

N. Genc-Nayebi and A. Abran, “A systematic literature review: Opinion mining studies from mobile app store user reviews,” J. Syst. Softw., vol. 125, pp. 207–219, 2017.

Cited By

Oliveira AEler M(2024)Exploring Accessibility of Mobile Applications Through User Feedback: Insights from App Reviews in a Systematic Literature ReviewProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702094(1-15)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702094
Shamsujjoha MGrundy JKhalajzadeh HLu QLi L(2024)Developer and End-User Perspectives on Addressing Human Aspects in Mobile eHealth AppsInformation and Software Technology10.1016/j.infsof.2023.107353166:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107353

Recommendations

Serving Mobile Apps: A Slice at a Time
EuroSys '19: Proceedings of the Fourteenth EuroSys Conference 2019

End users wanting to do more and more with mobile apps has led to explosive growth in the number of available apps. This has widened the gap between developers making apps available and end users being able to install all the apps they want on their ...
An Explorative Study of the Mobile App Ecosystem from App Developers' Perspective
WWW '17: Proceedings of the 26th International Conference on World Wide Web

With the prevalence of smartphones, app markets such as Apple App Store and Google Play has become the center stage in the mobile app ecosystem, with millions of apps developed by tens of thousands of app developers in each major market. This paper ...
Spam Mobile Apps: Characteristics, Detection, and in the Wild Analysis

The increased popularity of smartphones has attracted a large number of developers to offer various applications for the different smartphone platforms via the respective app markets. One consequence of this popularity is that the app markets are also ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 49, Issue 4

April 2023

1635 pages

ISSN:0098-5589

Issue’s Table of Contents

0098-5589 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 April 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Oliveira AEler M(2024)Exploring Accessibility of Mobile Applications Through User Feedback: Insights from App Reviews in a Systematic Literature ReviewProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702094(1-15)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702094
Shamsujjoha MGrundy JKhalajzadeh HLu QLi L(2024)Developer and End-User Perspectives on Addressing Human Aspects in Mobile eHealth AppsInformation and Software Technology10.1016/j.infsof.2023.107353166:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107353

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents