More Web Proxy on the site http://driver.im/

research-article

Predicting and Diagnosing User Engagement with Mobile UI Animation via a Data-Driven Approach

Authors:

Xiaojuan MaAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 13

https://doi.org/10.1145/3313831.3376324

Published: 23 April 2020 Publication History

Abstract

Animation, a common design element in user interfaces (UI), can impact user engagement (UE) with mobile applications. To avoid impairing UE due to improper design of animation, designers rely on resource-intensive evaluation methods like user studies or expert reviews. To alleviate this burden, we propose a data-driven approach to assisting designers in examining UE issues with their animation designs. We first crowdsource UE assessments of mobile UI animations. Based on the collected data, we then build a novel deep learning model that captures both spatial and temporal features of animations to predict their UE levels. Evaluations show that our model achieves a reasonable accuracy. We further leverage the animation feature encoded by our model and a sample set of expert reviews to derive potential UE issues of a particular animation. Finally, we develop a proof-of-concept tool and evaluate its potential usage in actual design practices with experts

References

[1]

2016. UX animation: When to use it, when to lose it. https://techbeacon.com/app-dev-testing/ ux-animation-when-use-it-when-lose-it. (Febuary 2016).

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).

[3]

Saeideh Bakhshi, David A. Shamma, and Eric Gilbert. 2014. Faces Engage Us: Photos with Faces Attract More Likes and Comments on Instagram. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 965--974.

Digital Library

[4]

Nikola Banovic, Antti Oulasvirta, and Per Ola Kristensson. 2019. Computational Modeling in Human-Computer Interaction. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, W26.

[5]

John J Bartko. 1966. The intraclass correlation coefficient as a measure of reliability. Psychological reports 19, 1 (1966), 3--11.

[6]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101.

[7]

Jonathan Carlton, Andy Brown, Caroline Jay, and John Keane. 2019. Inferring User Engagement from Interaction Data. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, NY, NY, USA, Article LBW1212, 6 pages.

Digital Library

[8]

Bay-Wei Chang and David Ungar. 1995. Animation: from cartoons to the user interface. (1995).

[9]

Aline Chevalier and Melody Y Ivory. 2003. Web site designs: Influences of designer's expertise and design constraints. International Journal of Human-Computer Studies 58, 1 (2003), 57--87.

Digital Library

[10]

Fanny Chevalier, Nathalie Henry Riche, Catherine Plaisant, Amira Chalbi, and Christophe Hurter. 2016. Animations 25 years later: New roles and opportunities. In Proceedings of the International Working Conference on Advanced Visual Interfaces. ACM, 280--287.

Digital Library

[11]

Domenic V Cicchetti. 1994. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological assessment 6, 4 (1994), 284.

[12]

Rene V Dawis. 1987. Scale construction. Journal of Counseling Psychology 34, 4 (1987), 481.

[13]

Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. 2017. Rico: A Mobile App Dataset for Building Data-Driven Design Applications. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, NY, NY, USA, 845--854.

Digital Library

[14]

Cyril Goutte and Eric Gaussier. 2005. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In European Conference on Information Retrieval. Springer, 345--359.

Digital Library

[15]

Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 6645--6649.

[16]

David Gunning. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web (2017).

[17]

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in neural information processing systems. 8527--8537.

[18]

Jan Hartmann, Alistair Sutcliffe, and Antonella De Angeli. 2008. Towards a Theory of User Judgment of Aesthetics and User Interface Quality. ACM Trans. Comput.-Hum. Interact. 15, 4, Article 15 (Dec. 2008), 30 pages.

Digital Library

[19]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[20]

Jeffrey Heer and George Robertson. 2007. Animated transitions in statistical data graphics. IEEE transactions on visualization and computer graphics 13, 6 (2007), 1240--1247.

Digital Library

[21]

Elad Hoffer and Nir Ailon. 2015. Deep metric learning using triplet network. In International Workshop on Similarity-Based Pattern Recognition. Springer, 84--92.

[22]

Weiyin Hong, James YL Thong, and Kar Yan Tam. 2007. How do Web users respond to non-banner-ads animation? The effects of task type and user experience. Journal of the American Society for Information Science and Technology 58, 10 (2007), 1467--1482.

Digital Library

[23]

Matt C Howard. 2016. A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction 32, 1 (2016), 51--62.

[24]

Jussi Huhtala, Ari-Heikki Sarjanoja, Jani Mäntyjärvi, Minna Isomursu, and Jonna Häkkilä. 2010. Animated UI Transitions and Perception of Time: A User Study on Animated Effects on a Mobile Screen. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, NY, NY, USA, 1339--1342.

Digital Library

[25]

Kevin Keeker. 1997. Improving web site usability and appeal. Retrieved May 19 (1997), 2002.

[26]

Young Hoon Kim, Dan J Kim, and Kathy Wachter. 2013. A study of mobile user engagement (MoEN): Engagement motivations, perceived value, satisfaction, and continued engagement intention. Decision Support Systems 56 (2013), 361--370.

Digital Library

[27]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[28]

Jonas F Kraft and Jörn Hurtienne. 2017. Transition animations support orientation in mobile interfaces without increased user effort. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 17.

Digital Library

[29]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 60, 6 (May 2017), 84--90.

Digital Library

[30]

Brenda Laurel and S Joy Mountford. 1990. The art of human-computer interface design. Addison-Wesley Longman Publishing Co., Inc.

Digital Library

[31]

Yang Li, Samy Bengio, and Gilles Bailly. 2018. Predicting human performance in vertical menu selection using deep learning. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 29.

Digital Library

[32]

Daniel Liddle. 2016a. Emerging Guidelines for Communicating with Animation in Mobile User Interfaces. In Proceedings of the 34th ACM International Conference on the Design of Communication (SIGDOC '16). ACM, NY, NY, USA, Article 16, 9 pages.

Digital Library

[33]

Daniel Liddle. 2016b. Emerging guidelines for communicating with animation in mobile user interfaces. In Proceedings of the 34th ACM International Conference on the Design of Communication. ACM, 16.

Digital Library

[34]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579--2605.

[35]

Aliaksei Miniukovich and Antonella De Angeli. 2015. Computation of Interface Aesthetics. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 1163--1172.

Digital Library

[36]

David Novick, Joseph Rhodes, and Wervyn Wert. 2011. The communicative functions of animation in user interfaces. In Proceedings of the 29th ACM international conference on Design of communication. ACM, 1--8.

Digital Library

[37]

Heather L O'Brien. 2011. Exploring user engagement in online news interactions. Proceedings of the American society for information science and technology 48, 1 (2011), 1--10.

[38]

Heather L O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies 112 (2018), 28--39.

[39]

Jeeyun Oh and S Shyam Sundar. 2016. User engagement with interactive media: A communication perspective. In Why Engagement Matters. Springer, 177--198.

[40]

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).

[41]

Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. 2014. Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596 (2014).

[42]

Katharina Reinecke and Krzysztof Z. Gajos. 2014. Quantifying Visual Preferences Around the World. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 11--20.

Digital Library

[43]

George G Robertson, Stuart K Card, and Jock D Mackinlay. 1993. Information visualization using 3D interactive animation. Commun. ACM 36, 4 (1993), 56--72.

Digital Library

[44]

Ruth Rosenholtz, Amal Dorai, and Rosalind Freeman. 2011. Do predictions of visual perception aid design? ACM Transactions on Applied Perception (TAP) 8, 2 (2011), 12.

[45]

Holly M Rus and Linda D Cameron. 2016. Health communication in social media: message features predicting user engagement on diabetes-related Facebook pages. Annals of behavioral medicine 50, 5 (2016), 678--689.

[46]

Céline Schlienger, Stéphane Conversy, Stéphane Chatty, Magali Anquetil, and Christophe Mertz. 2007. Improving users' comprehension of changes with animation and sound: An empirical assessment. In IFIP Conference on Human-Computer Interaction. Springer, 207--220.

[47]

Karen Simonyan and Andrew Zisserman. 2014. Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems. 568--576.

[48]

Amanda Swearngin and Yang Li. 2019. Modeling Mobile Interface Tappability Using Crowdsourcing and Deep Learning. arXiv preprint arXiv:1902.11247 (2019).

[49]

Bruce H Thomas and Paul Calder. 2001. Applying cartoon animation techniques to graphical user interfaces. ACM Transactions on Computer-Human Interaction (TOCHI) 8, 3 (2001), 198--222.

Digital Library

[50]

Noam Tractinsky, Ohad Inbar, Omer Tsimhoni, and Thomas Seder. 2011. Slow Down, You Move Too Fast: Examining Animation Aesthetics to Promote Eco-driving. In Proceedings of the 3rd International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '11). ACM, NY, NY, USA, 193--202.

Digital Library

[51]

Marcus Trapp and René Yasmin. 2013. Addressing animated transitions already in mobile app storyboards. In International Conference of Design, User Experience, and Usability. Springer, 723--732.

Digital Library

[52]

Barbara Tversky, Julie Bauer Morrison, and Mireille Betrancourt. 2002. Animation: can it facilitate? International journal of human-computer studies 57, 4 (2002), 247--262.

[53]

Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. 2016. Temporal segment networks: Towards good practices for deep action recognition. In European conference on computer vision. Springer, 20--36.

[54]

Jane Webster and Hayes Ho. 1997. Audience engagement in multimedia presentations. ACM SIGMIS Database: the DATABASE for Advances in Information Systems 28, 2 (1997), 63--77.

Digital Library

[55]

Ziming Wu, Taewook Kim, Quan Li, and Xiaojuan Ma. 2019. Understanding and Modeling User-Perceived Brand Personality from Mobile Application UIs. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 213, 12 pages.

Digital Library

[56]

Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: Generating Structured Feedback on Visual Designs Using a Crowd of Non-experts. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, NY, NY, USA, 1433--1444.

Digital Library

[57]

Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03). 912--919.

Digital Library

Cited By

Duan PCheng CLi GHartmann BLi Y(2024)UICrit: Enhancing Automated Design Evaluation with a UI Critique DatasetProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676381(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676381
Chen QLiu SHuang KWang XMa XZhu JPeng Z(2024)RetAssist: Facilitating Vocabulary Learners with Generative Images in Story Retelling PracticesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661581(2019-2036)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661581
Rashik MJasim MKucher KSarvghad AMahyar N(2024)Beyond Text and Speech in Conversational Agents: Mapping the Design Space of AvatarsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661563(1875-1894)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661563
Show More Cited By

Index Terms

Predicting and Diagnosing User Engagement with Mobile UI Animation via a Data-Driven Approach
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User models

Recommendations

Data driven cloth animation
SIGGRAPH '07: ACM SIGGRAPH 2007 sketches

We present a new method for cloth animation based on data driven synthesis. In contrast to approaches that focus on physical simulation, we animate cloth by manipulating short sequences of existing cloth animation. While our source of data is cloth ...
Data-driven 2D effects animation
ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing

Making plausible, high quality visual effects, like water splashes or fire, in traditional 2D animation pipelines require an animator to draw many frames of phenomena that are very difficult to recreate manually. We present a technique that uses a ...
Self adaptive animation based on user perspective

In this paper we present a new character animation technique in which the animation adapts itself based on the change in the user’s perspective, so that when the user moves and their point of viewing the animation changes, then the character animation ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Research Grants Council of Hong Kong

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
1,302
Total Downloads

Downloads (Last 12 months)200
Downloads (Last 6 weeks)20

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Duan PCheng CLi GHartmann BLi Y(2024)UICrit: Enhancing Automated Design Evaluation with a UI Critique DatasetProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676381(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676381
Chen QLiu SHuang KWang XMa XZhu JPeng Z(2024)RetAssist: Facilitating Vocabulary Learners with Generative Images in Story Retelling PracticesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661581(2019-2036)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661581
Rashik MJasim MKucher KSarvghad AMahyar N(2024)Beyond Text and Speech in Conversational Agents: Mapping the Design Space of AvatarsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661563(1875-1894)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661563
Nie LSaid KHu M(2024)SoK: An Exhaustive Taxonomy of Display Issues for Mobile ApplicationsProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645193(537-548)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645193
Wu WZhou XShi XWu PLin XYang JHe L(2024)The Effect of Individual-Level Factors and Task Features on Interface Design for Rule-Verification Crowdsourcing TasksInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2332031(1-28)Online publication date: 16-Apr-2024
https://doi.org/10.1080/10447318.2024.2332031
Lee LYau YHui P(2024)Perceived User Reachability in Mobile UIs Using Data Analytics and Machine LearningInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2327199(1-24)Online publication date: 25-Mar-2024
https://doi.org/10.1080/10447318.2024.2327199
Pontillo VPalomba FFerrucci F(2024)Test Code Flakiness in Mobile Apps: The Developer’s PerspectiveInformation and Software Technology10.1016/j.infsof.2023.107394168(107394)Online publication date: Apr-2024
https://doi.org/10.1016/j.infsof.2023.107394
Gao SWang YLiu H(2024)UiAnalyzer: Evaluating whether the UI of apps is at the risk of violating the design conventions in terms of function layoutExpert Systems with Applications10.1016/j.eswa.2023.122408239(122408)Online publication date: Apr-2024
https://doi.org/10.1016/j.eswa.2023.122408
Heraz ABhyravabhottla KSajith N(2024)Predicting user engagement levels through emotion-based gesture analysis of initial impressionsElectronic Commerce Research10.1007/s10660-024-09915-5Online publication date: 23-Oct-2024
https://doi.org/10.1007/s10660-024-09915-5
Almaral Martinez MBerna Martinez JMacia Perez FLorenzo Fonseca IPascual Caceres C(2023)Understanding the importance of animation in human-computer interfaces and its relation to user experienceProceedings of the 2023 5th International Conference on Image, Video and Signal Processing10.1145/3591156.3591180(164-171)Online publication date: 24-Mar-2023
https://dl.acm.org/doi/10.1145/3591156.3591180
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents