[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

What and How long: Prediction of Mobile App Engagement

Published: 08 September 2021 Publication History

Abstract

User engagement is crucial to the long-term success of a mobile app. Several metrics, such as dwell time, have been used for measuring user engagement. However, how to effectively predict user engagement in the context of mobile apps is still an open research question. For example, do the mobile usage contexts (e.g., time of day) in which users access mobile apps impact their dwell time? Answers to such questions could help mobile operating system and publishers to optimize advertising and service placement. In this article, we first conduct an empirical study for assessing how user characteristics, temporal features, and the short/long-term contexts contribute to gains in predicting users’ app dwell time on the population level. The comprehensive analysis is conducted on large app usage logs collected through a mobile advertising company. The dataset covers more than 12K anonymous users and 1.3 million log events. Based on the analysis, we further investigate a novel mobile app engagement prediction problem—can we predict simultaneously what app the user will use next and how long he/she will stay on that app? We propose several strategies for this joint prediction problem and demonstrate that our model can improve the performance significantly when compared with the state-of-the-art baselines. Our work can help mobile system developers in designing a better and more engagement-aware mobile app user experience.

References

[1]
Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 19–26.
[2]
Ricardo Baeza-Yates, Di Jiang, Fabrizio Silvestri, and Beverly Harrison. 2015. Predicting the next app that you are going to use. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. ACM, 285–294.
[3]
Mikhail Bilenko and Ryen W. White. 2008. Mining the search trails of surfing crowds: identifying relevant websites from user activity. In Proceedings of the 17th International Conference on World Wide Web. ACM, 51–60.
[4]
Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning. springer.
[5]
Veronika Bogina and Tsvi Kuflik. 2017. Incorporating dwell time in session-based recommendations with recurrent neural networks. In Proceedings of the RecTemp@ RecSys. 57–59.
[6]
Matthias Böhmer, Brent Hecht, Johannes Schöning, Antonio Krüger, and Gernot Bauer. 2011. Falling asleep with angry birds, facebook and kindle: A large scale study on mobile application usage. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, 47–56.
[7]
Leo Breiman. 2001. Random forests. Machine Learning 45, 1 (2001), 5–32.
[8]
Christopher J. C. Burges. 1998. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 2 (1998), 121–167.
[9]
Hong Cao and Miao Lin. 2017. Mining smartphone data for app usage prediction and recommendations: A survey. Pervasive and Mobile Computing 37, C (2017), 1–22.
[10]
Juan Pablo Carrascal and Karen Church. 2015. An in-situ study of mobile app & mobile search interactions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2739–2748.
[11]
Thomas M. Cover and Peter E. Hart. 1967. Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13, 1 (1967), 21–27.
[12]
David Roxbee Cox. 1966. The statistical analysis of series of events. Monographs on Applied Probability and Statistics. Springer.
[13]
Ernesto Diaz-Aviles, Hoang Thanh Lam, Fabio Pinelli, Stefano Braghin, Yiannis Gkoufas, Michele Berlingerio, and Francesco Calabrese. 2014. Predicting user engagement in twitter with collaborative ranking. In Proceedings of the 2014 Recommender Systems Challenge. ACM, 41–46.
[14]
Trinh Minh Tri Do and Daniel Gatica-Perez. 2014. Where and what: Using smartphones to predict next locations and applications in daily life. Pervasive and Mobile Computing 12 (2014), 79–91.
[15]
Harris Drucker, Corinna Cortes, Lawrence D. Jackel, Yann LeCun, and Vladimir Vapnik. 1994. Boosting and other ensemble methods. Neural Computation 6, 6 (1994), 1289–1301.
[16]
Claudia Ehmke and Stephanie Wilson. 2007. Identifying web usability problems from eye-tracking data. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 1. British Computer Society, 119–128.
[17]
Hossein Falaki, Ratul Mahajan, Srikanth Kandula, Dimitrios Lymberopoulos, Ramesh Govindan, and Deborah Estrin. 2010. Diversity in smartphone usage. In Proceedings of the 8th International Conference on Mobile Systems, Applications, and Services. ACM, 179–194.
[18]
Denzil Ferreira, Jorge Goncalves, Vassilis Kostakos, Louise Barkhuus, and Anind K. Dey. 2014. Contextual experience sampling of mobile application micro-usage. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services. ACM, 91–100.
[19]
Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve web search. ACM Transactions on Information Systems 23, 2 (2005), 147–168.
[20]
Andrew Gelman. 2008. Scaling regression inputs by dividing by two standard deviations. Statistics in Medicine 27, 15 (2008), 2865–2873.
[21]
Google. 2018. Select a Category for Your App or Game.
[22]
Balázs Hidasi and Domonkos Tikk. 2016. General factorization framework for context-aware recommendations. Data Mining and Knowledge Discovery 30, 2 (2016), 342–371.
[23]
Ryosuke Homma, Keiichi Soejima, Mitsuo Yoshida, and Kyoji Umemura. 2018. Analysis of user dwell time on non-news pages. In Proceedings of the 2018 IEEE International Conference on Big Data. IEEE, 4333–4338.
[24]
David W. Hosmer Jr, Stanley Lemeshow, and Rodney X. Sturdivant. 2013. Applied Logistic Regression. John Wiley & Sons.
[25]
Jeff Huang, Ryen W. White, and Susan Dumais. 2011. No clicks, no problem: Using cursor movements to understand and improve search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1225–1234.
[26]
Ke Huang, Chunhui Zhang, Xiaoxiao Ma, and Guanling Chen. 2012. Predicting mobile application usage using contextual information. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 1059–1065.
[27]
Diane Kelly and Nicholas J. Belkin. 2004. Display time as implicit feedback: understanding task effects. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 377–384.
[28]
Young Hoon Kim, Dan J. Kim, and Kathy Wachter. 2013. A study of mobile user engagement (MoEN): Engagement motivations, perceived value, satisfaction, and continued engagement intention. Decision Support Systems 56, C (2013), 361–370.
[29]
Farshad Kooti, Mihajlo Grbovic, Luca Maria Aiello, Eric Bax, and Kristina Lerman. 2017. iPhone’s digital marketplace: Characterizing the big spenders. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining. ACM, 13–21.
[30]
Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. 2012. Models of user engagement. In Proceedings of the International Conference on User Modeling, Adaptation, and Personalization. Springer, 164–175.
[31]
Huoran Li and Xuan Lu. 2017. Mining device-specific apps usage patterns from large-scale android users. arXiv:1707.09252. Retrieved from https://arxiv.org/abs/1707.09252.
[32]
Huoran Li, Xuan Lu, Xuanzhe Liu, Tao Xie, Kaigui Bian, Felix Xiaozhu Lin, Qiaozhu Mei, and Feng Feng. 2015. Characterizing smartphone usage patterns from millions of android users. In Proceedings of the 2015 Internet Measurement Conference. ACM, 459–472.
[33]
Zhung-Xun Liao, Po-Ruey Lei, Tsu-Jou Shen, Shou-Chung Li, and Wen-Chih Peng. 2012. Mining temporal profiles of mobile applications for usage prediction. In Proceedings of the 2012 IEEE 12th International Conference onData Mining Workshops. IEEE, 890–893.
[34]
Zhung-Xun Liao, Shou-Chung Li, Wen-Chih Peng, S. Yu Philip, and Te-Chuan Liu. 2013. On the feature discovery for app usage prediction in smartphones. In Proceedings of the 2013 IEEE 13th International Conference onData Mining. IEEE, 1127–1132.
[35]
Zhung-Xun Liao, Yi-Chin Pan, Wen-Chih Peng, and Po-Ruey Lei. 2013. On mining mobile apps usage behavior for predicting apps usage in smartphones. In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management. ACM, 609–618.
[36]
Akhil Mathur, Nicholas D. Lane, and Fahim Kawsar. 2016. Engagement-aware computing: Modelling user engagement from mobile contexts. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 622–633.
[37]
Abhinav Mehrotra, Veljko Pejovic, Jo Vermeulen, Robert Hendley, and Mirco Musolesi. 2016. My phone and me: Understanding people’s receptivity to mobile notifications. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1021–1032.
[38]
Cataldo Musto, Giovanni Semeraro, Marco De Gemmis, and Pasquale Lops. 2015. Word embedding techniques for content-based recommender systems: An empirical evaluation. In Proceedings of the Recsys Posters.
[39]
Klaas Nelissen, Monique Snoeck, Seppe Vanden Broucke, and Bart Baesens. 2018. Swipe and tell: Using implicit feedback to predict user engagement on tablets. ACM Transactions on Information Systems 36, 4 (2018), 1–36.
[40]
Wei Pan, Nadav Aharony, and Alex Pentland. 2011. Composite social network for predicting mobile apps installation. Proceedings of the AAAI Conference on Artificial Intelligence 25, 1 (2011).
[41]
Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe: Designing intelligent prompting mechanisms for pervasive applications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 897–908.
[42]
Thanasis Petsas, Antonis Papadogiannakis, Michalis Polychronakis, Evangelos P. Markatos, and Thomas Karagiannis. 2017. Measurement, modeling, and analysis of the mobile app ecosystem. ACM Transactions on Modeling and Performance Evaluation of Computing Systems 2, 2 (2017), 1–33.
[43]
Martin Pielot, Karen Church, and Rodrigo De Oliveira. 2014. An in-situ study of mobile phone notifications. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services. ACM, 233–242.
[44]
Mary Lou Quinlan. 2003. Just Ask a Woman: Cracking the Code of What Women Want and How They Buy. John Wiley & Sons.
[45]
Janeaya Revels, Dewi Tojib, and Yelena Tsarenko. 2010. Understanding consumer intention to use mobile services. Australasian Marketing Journal 18, 2 (2010), 74–80.
[46]
Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural Networks 61 (2015), 85–117.
[47]
Yoshifumi Seki and Mitsuo Yoshida. 2018. Analysis of user dwell time by category in news application. In Proceedings of the 2018 IEEE/WIC/ACM International Conference on Web Intelligence. IEEE, 732–735.
[48]
Suranga Seneviratne, Aruna Seneviratne, Prasant Mohapatra, and Anirban Mahanti. 2015. Your installed apps reveal your gender and more!ACM SIGMOBILE Mobile Computing and Communications Review 18, 3 (2015), 55–61.
[49]
Choonsung Shin, Jin-Hyuk Hong, and Anind K. Dey. 2012. Understanding and prediction of mobile application usage for smart phones. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 173–182.
[50]
Iwan Syarif, Ed Zaluska, Adam Prugel-Bennett, and Gary Wills. 2012. Application of bagging, boosting and stacking to intrusion detection. In Proceedings of the International Workshop on Machine Learning and Data Mining in Pattern Recognition. Springer, 593–602.
[51]
Chang Tan, Qi Liu, Enhong Chen, and Hui Xiong. 2012. Prediction for mobile application usage patterns. In Proceedings of the Nokia MDC Workshop, Vol. 12.
[52]
Maryam Tavakol and Ulf Brefeld. 2014. Factored MDPs for detecting topics of user sessions. In Proceedings of the 8th ACM Conference on Recommender Systems. 33–40.
[53]
Yuan Tian, Ke Zhou, Mounia Lalmas, Yiqun Liu, and Dan Pelleg. 2020. Cohort modeling based app category usage prediction. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 248–256.
[54]
Yuan Tian, Ke Zhou, Mounia Lalmas, and Dan Pelleg. 2020. Identifying tasks from mobile app usage patterns. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2357–2366.
[55]
Steven Van Canneyt, Marc Bron, Andy Haines, and Mounia Lalmas. 2017. Describing patterns and disruptions in large scale mobile app usage data. In Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 1579–1584.
[56]
Henriette C. van Vugt, Elly A. Konijn, Johan F. Hoorn, I. Keur, and Anton Eliéns. 2007. Realism is not all! user engagement with task-related interface characters. Interacting with Computers 19, 2 (2007), 267–280.
[57]
Theodore Vasiloudis, Hossein Vahabi, Ross Kravitz, and Valery Rashkov. 2017. Predicting session length in media streaming. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 977–980.
[58]
Tianxin Wang, Jingwu Chen, Fuzhen Zhuang, Leyu Lin, Feng Xia, Lihuan Du, and Qing He. (2020). Capturing attraction distribution: Sequential attentive network for dwell time prediction. In Proceedings of the 24th European Conference on Artificial Intelligence.
[59]
Siqi Wu, Marian-Andrei Rizoiu, and Lexing Xie. 2018. Beyond views: Measuring and predicting engagement in online videos. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 12.
[60]
Qiang Xu, Jeffrey Erman, Alexandre Gerber, Zhuoqing Mao, Jeffrey Pang, and Shobha Venkataraman. 2011. Identifying diverse usage behaviors of smartphone apps. In Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference. ACM, 329–344.
[61]
Shijian Xu, Wenzhong Li, Xiao Zhang, Songcheng Gao, Tong Zhan, and Sanglu Lu. 2020. Predicting and recommending the next smartphone apps based on recurrent neural network. CCF Transactions on Pervasive Computing and Interaction 2, 4 (2020), 314–328.
[62]
Ye Xu, Mu Lin, Hong Lu, Giuseppe Cardone, Nicholas Lane, Zhenyu Chen, Andrew Campbell, and Tanzeem Choudhury. 2013. Preference, context and communities: A multi-faceted approach to predicting smartphone app usage patterns. In Proceedings of the 2013 International Symposium on Wearable Computers. ACM, 69–76.
[63]
Tingxin Yan, David Chu, Deepak Ganesan, Aman Kansal, and Jie Liu. 2012. Fast app launching for mobile devices using predictive user context. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services. ACM, 113–126.
[64]
Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014. Beyond clicks: Dwell time for personalization. In Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 113–120.
[65]
Elad Yom-Tov, Mounia Lalmas, Ricardo Baeza-Yates, Georges Dupret, Janette Lehmann, and Pinar Donmez. 2013. Measuring inter-site engagement. In Proceedings of the 2013 IEEE International Conference on Big Data. IEEE, 228–236.
[66]
Chunhui Zhang, Xiang Ding, Guanling Chen, Ke Huang, Xiaoxiao Ma, and Bo Yan. 2012. Nihao: A predictive smartphone application launcher. In Proceedings of the International Conference on Mobile Computing, Applications, and Services. Springer, 294–313.
[67]
Sha Zhao, Zhiling Luo, Ziwen Jiang, Haiyan Wang, Feng Xu, Shijian Li, Jianwei Yin, and Gang Pan. 2019. AppUsage2Vec: Modeling smartphone app usage for prediction. In Proceedings of the 2019 IEEE 35th International Conference on Data Engineering. IEEE, 1322–1333.
[68]
Sha Zhao, Julian Ramos, Jianrong Tao, Ziwen Jiang, Shijian Li, Zhaohui Wu, Gang Pan, and Anind K. Dey. 2016. Discovering different kinds of smartphone users through their application usage behaviors. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 498–509.
[69]
Tengfei Zhou, Hui Qian, Zebang Shen, Chao Zhang, Chengwei Wang, Shichen Liu, and Wenwu Ou. 2018. Jump: A joint predictor for user click and dwell time. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. AAAI Press. 3704–3710.
[70]
Xun Zou, Wangsheng Zhang, Shijian Li, and Gang Pan. 2013. Prophet: What app you wish to use next. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication. ACM, 167–170.

Cited By

View all
  • (2025)Exploring the Effect of an 8-Week AI-Composed Exercise Program on Pain Intensity and Well-Being in Patients With Spinal Pain: Retrospective Cohort AnalysisJMIR Formative Research10.2196/578269(e57826-e57826)Online publication date: 18-Feb-2025
  • (2025)Appformer: A novel framework for mobile app usage prediction leveraging progressive multi-modal data fusion and feature extractionExpert Systems with Applications10.1016/j.eswa.2024.125903265(125903)Online publication date: Mar-2025
  • (2024)Towards the design of a mobile application with persuasive elements to increase energy flexibilityProceedings of the European Conference on Cognitive Ergonomics 202410.1145/3673805.3673825(1-4)Online publication date: 8-Oct-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems
ACM Transactions on Information Systems  Volume 40, Issue 1
January 2022
599 pages
ISSN:1046-8188
EISSN:1558-2868
DOI:10.1145/3483337
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Received: 01 October 2021
Published: 08 September 2021
Accepted: 01 April 2021
Revised: 01 March 2021
Published in TOIS Volume 40, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Mobile apps
  2. user engagement
  3. app usage
  4. dwell time
  5. next app prediction
  6. app engagement prediction
  7. demographics
  8. behavior modeling
  9. user modeling

Qualifiers

  • Research-article
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)251
  • Downloads (Last 6 weeks)29
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Exploring the Effect of an 8-Week AI-Composed Exercise Program on Pain Intensity and Well-Being in Patients With Spinal Pain: Retrospective Cohort AnalysisJMIR Formative Research10.2196/578269(e57826-e57826)Online publication date: 18-Feb-2025
  • (2025)Appformer: A novel framework for mobile app usage prediction leveraging progressive multi-modal data fusion and feature extractionExpert Systems with Applications10.1016/j.eswa.2024.125903265(125903)Online publication date: Mar-2025
  • (2024)Towards the design of a mobile application with persuasive elements to increase energy flexibilityProceedings of the European Conference on Cognitive Ergonomics 202410.1145/3673805.3673825(1-4)Online publication date: 8-Oct-2024
  • (2024)ComBoost: An Instruction Complexity Aware DTM Technique for Edge DevicesProceedings of the 29th ACM/IEEE International Symposium on Low Power Electronics and Design10.1145/3665314.3670820(1-6)Online publication date: 5-Aug-2024
  • (2024)MAPLEProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435148:1(1-25)Online publication date: 6-Mar-2024
  • (2024)Efficient Group Collaboration for Sensing Time Redundancy Optimization in Mobile CrowdsensingIEEE Internet of Things Journal10.1109/JIOT.2024.339353211:15(26091-26103)Online publication date: 1-Aug-2024
  • (2024)Understanding critical service factors in neobanks: crafting strategies through text miningJournal of Modelling in Management10.1108/JM2-01-2024-002120:3(894-922)Online publication date: 8-Oct-2024
  • (2023)TPP: Accelerate Application Launch via Two-Phase Prefetching on Smartphone2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10136908(1-6)Online publication date: Apr-2023
  • (2023)SLED: Structure Learning based Denoising for RecommendationACM Transactions on Information Systems10.1145/361138542:2(1-31)Online publication date: 8-Nov-2023
  • (2023)Examining User Heterogeneity in Digital ExperimentsACM Transactions on Information Systems10.1145/357893141:4(1-34)Online publication date: 22-Mar-2023
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media