[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3411764.3445497acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Open access

Adapting User Interfaces with Model-based Reinforcement Learning

Published: 07 May 2021 Publication History

Abstract

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user – for example, due to surprise or relearning effort – or “trap” the process to a suboptimal design immaturely. However, effects on users are hard to predict as they depend on factors that are latent and evolve over the course of interaction. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy: It finds beneficial changes when there are such and avoids changes when there are none. Our model-based reinforcement learning method plans sequences of adaptations and consults predictive HCI models to estimate their effects. We present empirical and simulation results from the case of adaptive menus, showing that the method outperforms both a non-adaptive and a frequency-based policy.

References

[1]
John R Anderson and Christian J Lebiere. 2014. The atomic components of thought. Psychology Press. https://doi.org/10.4324/9781315805696
[2]
Mauricio Araya, Olivier Buffet, Vincent Thomas, and Françcois Charpillet. 2010. A POMDP Extension with Belief-dependent Rewards. In Advances in Neural Information Processing Systems 23, J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta (Eds.). Curran Associates, Inc., 64–72. https://doi.org/10.1109/TCIAIG.2012.2186810
[3]
Gilles Bailly, Eric Lecolinet, and Laurence Nigay. 2008. Flower Menus: A New Type of Marking Menu with Large Menu Breadth, within Groups and Efficient Expert Mode Memorization. In Proceedings of the Working Conference on Advanced Visual Interfaces(AVI ’08). 15–22. https://doi.org/10.1145/1385569.1385575
[4]
Gilles Bailly, Eric Lecolinet, and Laurence Nigay. 2016. Visual Menu Techniques. ACM Comput. Surv. 49, 4, Article 60 (Dec. 2016), 41 pages. https://doi.org/10.1145/3002171
[5]
Gilles Bailly, Antti Oulasvirta, Duncan P. Brumby, and Andrew Howes. 2014. Model of Visual Search and Selection Time in Linear Menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). 3865–3874. https://doi.org/10.1145/2556288.2557093
[6]
Gilles Bailly, Antti Oulasvirta, Timo Kötzing, and Sabrina Hoppe. 2013. MenuOptimizer: Interactive Optimization of Menu Systems. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology(UIST ’13). 331–342. https://doi.org/10.1145/2501988.2502024
[7]
Richard E Bellman and Lotfi Asker Zadeh. 1970. Decision-making in a fuzzy environment. Management science 17, 4 (1970), B–141. https://doi.org/10.1287/mnsc.17.4.B141
[8]
Mark G Brockington. 1996. A taxonomy of parallel game-tree search algorithms. ICGA Journal 19, 3 (1996), 162–174.
[9]
Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games 4, 1(2012), 1–43. https://doi.org/10.1109/TCIAIG.2012.2186810
[10]
Michael D. Byrne. 2001. ACT-R/PM and menu selection: applying a cognitive architecture to HCI. International Journal of Human-Computer Studies 55, 1 (2001), 41 – 84. https://doi.org/10.1006/ijhc.2001.0469
[11]
Yuan Cai and Xiang Ji. 2018. ASA-routing: A-Star Adaptive Routing Algorithm for Network-on-Chips. In Algorithms and Architectures for Parallel Processing, Jaideep Vaidyaand Jin Li (Eds.). Springer International Publishing, Cham, 187–198.
[12]
Andy Cockburn and Carl Gutwin. 2009. A Predictive Model of Human Performance With Scrolling and Hierarchical Lists. Human–Computer Interaction 24, 3 (2009), 273–314. https://doi.org/10.1080/07370020902990402
[13]
Andy Cockburn, Carl Gutwin, and Saul Greenberg. 2007. A Predictive Model of Menu Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’07). 627–636. https://doi.org/10.1145/1240624.1240723
[14]
Peng Dai, Christopher H Lin, Daniel S Weld, 2013. POMDP-based control of workflows for crowdsourcing. Artificial Intelligence 202 (2013), 52–85.
[15]
John J. Dudley, Jason T. Jacques, and Per Ola Kristensson. 2019. Crowdsourcing Interface Feature Design with Bayesian Optimization. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300482
[16]
Leah Findlater and Joanna McGrenere. 2004. A Comparison of Static, Adaptive, and Adaptable Menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for Computing Machinery, New York, NY, USA, 89–96. https://doi.org/10.1145/985692.985704
[17]
Leah Findlater, Karyn Moffatt, Joanna McGrenere, and Jessica Dawson. 2009. Ephemeral Adaptation: The Use of Gradual Onset to Improve Menu Selection Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1655–1664. https://doi.org/10.1145/1518701.1518956
[18]
Stephen Fitchett and Andy Cockburn. 2012. AccessRank: Predicting What Users Will Do Next. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 2239–2242. https://doi.org/10.1145/2207676.2208380
[19]
Peter A Frensch. 1994. Composition during serial learning: A serial position effect. J. Exp. Psychol. Learn. Mem. Cogn. 20 (1994), 423–423.
[20]
Christoph Gebhardt, Brian Hecox, Bas van Opheusden, Daniel Wigdor, James Hillis, Otmar Hilliges, and Hrvoje Benko. 2019. Learning Cooperative Personalized Policies from Gaze Data. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 197–208. https://doi.org/10.1145/3332165.3347933
[21]
Camille Gobert, Kashyap Todi, Gilles Bailly, and Antti Oulasvirta. 2019. SAM: A Modular Framework for Self-Adapting Web Menus. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 481–484. https://doi.org/10.1145/3301275.3302314
[22]
Saul Greenberg and Ian H. Witten. 1985. Adaptive personalized interfaces—A question of viability. Behaviour & Information Technology 4, 1 (1985), 31–45. https://doi.org/10.1080/01449298508901785
[23]
Jesse Hoey, Axel Von Bertoldi, Pascal Poupart, and Alex Mihailidis. 2007. Assisting persons with dementia during handwashing using a partially observable Markov decision process. In International Conference on Computer Vision Systems: Proceedings (2007).
[24]
Anthony J. Hornof and David E. Kieras. 1997. Cognitive Modeling Reveals Menu Search in Both Random and Systematic. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems(CHI ’97). 107–114. https://doi.org/10.1145/258549.258621
[25]
Bowen Hui, Grant Partridge, and Craig Boutilier. 2009. A Probabilistic Mental Model for Estimating Disruption. In Proceedings of the 14th International Conference on Intelligent User Interfaces (Sanibel Island, Florida, USA) (IUI ’09). Association for Computing Machinery, New York, NY, USA, 287–296. https://doi.org/10.1145/1502650.1502691
[26]
Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. 2019. When to trust your model: Model-based policy optimization. In Advances in Neural Information Processing Systems. 12498–12509.
[27]
Jussi P. P. Jokinen, Sayan Sarcar, Antti Oulasvirta, Chaklam Silpasuwanchai, Zhenxin Wang, and Xiangshi Ren. 2017. Modelling Learning of New Keyboard Layouts. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 4203–4215. https://doi.org/10.1145/3025453.3025580
[28]
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996), 237–285.
[29]
Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, 2019. Model-based reinforcement learning for atari. arXiv:1903.00374 (2019).
[30]
Sunjun Kim, Byungjoo Lee, Thomas van Gemert, and Antti Oulasvirta. 2020. Optimal Sensor Position for a Computer Mouse. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376735
[31]
Levente Kocsis and Csaba Szepesvári. 2006. Bandit Based Monte-Carlo Planning. In Proceedings of the 17th European Conference on Machine Learning (Berlin, Germany) (ECML’06). Springer-Verlag, Berlin, Heidelberg, 282–293. https://doi.org/10.1007/11871842_29
[32]
Pat Langley. 1997. Machine learning for adaptive user interfaces. In Annual Conference on Artificial Intelligence(KI 1997). 53–62. https://doi.org/10.1007/3540634932_3
[33]
Talia Lavie and Joachim Meyer. 2010-08. Benefits and Costs of Adaptive User Interfaces. Int. J. Hum.-Comput. Stud. 68, 8 (2010-08), 508–524. https://doi.org/10.1016/j.ijhcs.2010.01.004
[34]
Dong-Seok Lee and Wan Chul Yoon. 2004. Quantitative results assessing design issues of selection-supportive menus. International Journal of Industrial Ergonomics 33, 1(2004), 41–52.
[35]
Katri Leino, Kashyap Todi, Antti Oulasvirta, and Mikko Kurimo. 2019. Computer-Supported Form Design Using Keystroke-Level Modeling with Reinforcement Learning. In Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion(IUI ’19). 85–86. https://doi.org/10.1145/3308557.3308704
[36]
Luis Leiva. 2012. Interaction-Based User Interface Redesign. In Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces (Lisbon, Portugal) (IUI ’12). Association for Computing Machinery, New York, NY, USA, 311–312. https://doi.org/10.1145/2166966.2167028
[37]
Wanyu Liu, Gilles Bailly, and Andrew Howes. 2017. Effects of Frequency Distribution on Linear Menu Performance. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1307–1312. https://doi.org/10.1145/3025453.3025707
[38]
J. Derek Lomas, Jodi Forlizzi, Nikhil Poonwala, Nirmal Patel, Sharan Shodhan, Kishan Patel, Ken Koedinger, and Emma Brunskill. 2016. Interface Design Optimization as a Multi-Armed Bandit Problem. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 4142–4153. https://doi.org/10.1145/2858036.2858425
[39]
I. Scott MacKenzie. 1992. Fitts’ Law as a Research and Design Tool in Human-Computer Interaction. Hum.-Comput. Interact. 7, 1 (March 1992), 91–139. https://doi.org/10.1207/s15327051hci0701_3
[40]
Kent L Norman. 1991. The psychology of menu selection: Designing cognitive control at the human/computer interface. Intellect Books.
[41]
Antti Oulasvirta. 2018. Computational interaction. Oxford University Press. https://doi.org/10.1093/oso/9780198799603.001.0001
[42]
Antti Oulasvirta, Niraj Ramesh Dayama, Morteza Shiripour, Maximilian John, and Andreas Karrenbauer. 2020. Combinatorial Optimization of Graphical User Interface Designs. Proc. IEEE 108, 3 (2020), 434–464. https://doi.org/10.1109/JPROC.2020.2969687
[43]
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.
[44]
Angel R. Puerta, Henrik Eriksson, John H. Gennari, and Mark A. Musen. 1994. Model-Based Automated Generation of User Interfaces. In Proceedings of the Twelfth National Conference on Artificial Intelligence (Vol. 1) (Seattle, Washington, USA) (AAAI ’94). American Association for Artificial Intelligence, USA, 471–477.
[45]
Reyhaneh Raissi, Evanthia Dimara, Jacquelyn H. Berry, Wayne D. Gray, and Gilles Bailly. 2020. Retroactive Transfer Phenomena in Alternating User Interfaces. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376538
[46]
Andrew Sears and Ben Shneiderman. 1994. Split Menus: Effectively Using Selection Frequency to Organize Menus. ACM Trans. Comput.-Hum. Interact. 1, 1 (March 1994), 27–51. https://doi.org/10.1145/174630.174632
[47]
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. 2015. Taking the human out of the loop: A review of Bayesian optimization. Proc. IEEE 104, 1 (2015), 148–175.
[48]
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 (2016). https://doi.org/10.1038/nature16961
[49]
Harold Soh, Scott Sanner, Madeleine White, and Greg Jamieson. 2017. Deep Sequential Recommendation for Personalized Adaptive User Interfaces. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (Limassol, Cyprus) (IUI ’17). Association for Computing Machinery, New York, NY, USA, 589–593. https://doi.org/10.1145/3025171.3025207
[50]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
[51]
Kashyap Todi, Jussi Jokinen, Kris Luyten, and Antti Oulasvirta. 2018. Familiarisation: Restructuring Layouts with Visual Learning Models. In 23rd International Conference on Intelligent User Interfaces(IUI ’18). 547–558. https://doi.org/10.1145/3172944.3172949
[52]
Kashyap Todi, Jussi Jokinen, Kris Luyten, and Antti Oulasvirta. 2019. Individualising Graphical Layouts with Predictive Visual Search Models. ACM Trans. Interact. Intell. Syst. 10, 1, Article 9 (Aug. 2019), 24 pages. https://doi.org/10.1145/3241381
[53]
Kashyap Todi, Daryl Weir, and Antti Oulasvirta. 2016. Sketchplore: Sketch and Explore with a Layout Optimiser. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS ’16). Association for Computing Machinery, New York, NY, USA, 543–555. https://doi.org/10.1145/2901790.2901817
[54]
Theophanis Tsandilas and m. c. schraefel. 2007. Bubbling Menus: A Selective Mechanism for Accessing Hierarchical Drop-down Menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 1195–1204. https://doi.org/10.1145/1240624.1240806
[55]
Jean Vanderdonckt, Sara Bouzit, Gaëlle Calvary, and Denis Chêne. 2019. Exploring a Design Space of Graphical Adaptive Menus: Normal vs. Small Screens. ACM Trans. Interact. Intell. Syst. 10, 1, Article 2 (July 2019), 40 pages. https://doi.org/10.1145/3237190
[56]
Roel Vertegaal. 2003. Attentive User Interfaces. Commun. ACM 46, 3 (March 2003), 30–33. https://doi.org/10.1145/636772.636794
[57]
Ying Wu and Thomas S. Huang. 1999. Vision-Based Gesture Recognition: A Review. In Gesture-Based Communication in Human-Computer Interaction, Annelies Braffort, Rachid Gherbi, Sylvie Gibet, Daniel Teil, and James Richardson(Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 103–115.
[58]
Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013. POMDP-based statistical spoken dialog systems: A review. Proc. IEEE 101, 5 (2013), 1160–1179.
[59]
Arianna Yuan and Yang Li. 2020. Modeling Human Visual Search Performance on Realistic Webpages Using Analytical and Deep Learning Methods. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376870

Cited By

View all
  • (2024)Mobile User Interface Adaptation Based on Usability Reward Model and Multi-Agent Reinforcement LearningMultimodal Technologies and Interaction10.3390/mti80400268:4(26)Online publication date: 24-Mar-2024
  • (2024)Towards Co-designing a Human-Agent Interface: A Case Study in Online Grooming Detection.Proceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3690893(390-392)Online publication date: 24-Nov-2024
  • (2024)User-controlled Form Adaptation by Unsupervised LearningAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685431(1-8)Online publication date: 13-Oct-2024
  • Show More Cited By

Index Terms

  1. Adapting User Interfaces with Model-based Reinforcement Learning
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
      May 2021
      10862 pages
      ISBN:9781450380966
      DOI:10.1145/3411764
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 May 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Adaptive User Interfaces
      2. Monte Carlo Tree Search
      3. Predictive Models
      4. Reinforcement Learning

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      • Academy of Finland

      Conference

      CHI '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Upcoming Conference

      CHI 2025
      ACM CHI Conference on Human Factors in Computing Systems
      April 26 - May 1, 2025
      Yokohama , Japan

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)1,400
      • Downloads (Last 6 weeks)177
      Reflects downloads up to 13 Dec 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Mobile User Interface Adaptation Based on Usability Reward Model and Multi-Agent Reinforcement LearningMultimodal Technologies and Interaction10.3390/mti80400268:4(26)Online publication date: 24-Mar-2024
      • (2024)Towards Co-designing a Human-Agent Interface: A Case Study in Online Grooming Detection.Proceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3690893(390-392)Online publication date: 24-Nov-2024
      • (2024)User-controlled Form Adaptation by Unsupervised LearningAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685431(1-8)Online publication date: 13-Oct-2024
      • (2024)MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIsProceedings of the ACM on Human-Computer Interaction10.1145/36611478:EICS(1-27)Online publication date: 17-Jun-2024
      • (2024)Reinforcement Learning-Based Framework for the Intelligent Adaptation of User InterfacesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661329(40-48)Online publication date: 24-Jun-2024
      • (2024)The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video MeetingsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661507(1638-1657)Online publication date: 1-Jul-2024
      • (2024)Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement LearningExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651059(1-7)Online publication date: 11-May-2024
      • (2024)CoExplorer: Generative AI Powered 2D and 3D Adaptive Interfaces to Support Intentionality in Video MeetingsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650797(1-10)Online publication date: 11-May-2024
      • (2024)Heads-Up Multitasker: Simulating Attention Switching On Optical Head-Mounted DisplaysProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642540(1-18)Online publication date: 11-May-2024
      • (2024)Simulation-based Optimization of User Interfaces for Quality-assuring Machine Learning Model PredictionsACM Transactions on Interactive Intelligent Systems10.1145/359455214:1(1-32)Online publication date: 9-Jan-2024
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media