BendAide: A Deformable Interface to Augment Touchscreen Mobile Devices
Authors: 
Alexander Keith Eady, Alfrancis Guerrero, Victor Cheung, 
David Thue, 
Audrey GirouardAuthors Info & Claims
Article No.: 2, Pages 1 - 10
Abstract
The uses of handheld mobile devices are diverse, yet interaction is not; touchscreens are the singular primary interface on most mobiles. Touch interaction has usability issues (e.g., the “fat fingers problem”) which impair the fine control of small interface elements, such as when working with text. Beyond text entry, this includes tasks like placing the in-text cursor (caret), text selection, and copy/paste. Current solutions for touch usability issues do not address complex uses like working with text. We propose deformable interaction, specifically bend, added alongside touch to support working with text on mobile. We explore this through a study of BendAide, a novel deformable 3D printed case for mobiles that adds bend interaction to the device. We found that people perceive different advantages between bend and touch and that they will alternate between these inputs based on task demands and their personal abilities. Adding alternate input options to mobile could reduce the complexity of on-display interfaces and interactions and give people more choice in how they use their devices.
References
[1]
Teemu T Ahmaniemi, Johan Kildal, and Merja Haveri. 2014. What is a device bend gesture really good for? In SIGCHI Conference on Human Factors in Computing Systems, ACM, 3503–3512.
[2]
Jennifer M. Allen, Leslie A. McFarlin, and Thomas Green. 2008. An In-Depth Look into the Text Entry User Experience on the iPhone. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 52, 5 (September 2008), 508–512.
[3]
Ahmed Sabbir Arif, Sunjun Kim, Wolfgang Stuerzlinger, Geehyuk Lee, and Ali Mazalek. 2016. Evaluation of a Smart-Restorable Backspace Technique to Facilitate Text Entry Error Correction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16, ACM Press, New York, New York, USA, 5151–5162.
[4]
Ahmed Sabbir Arif, Wolfgang Stuerzlinger, Ali Mazalek, Sunjun Kim, and Geehyuk Lee. 2015. A smart-restorable backspace to facilitate text entry error correction. In CHI 2015 Workshop on Text Entry on the Edge, Seoul, South Korea.
[5]
Oscar Kin-Chung Au, Xiaojun Su, and Rynson W.H. Lau. 2014. LinearDragger: a Linear Selector for One-finger Target Acquisition. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 2607–2616.
[6]
Daniel Avrahami. 2015. The Effect of Edge Targets on Touch Performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, ACM Press, New York, New York, USA, 1837–1846.
[7]
Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services - MobileHCI ’12, ACM Press, New York, New York, USA, 251.
[8]
Joanna Bergstrom-Lehtovirta and Antti Oulasvirta. 2014. Modeling the functional area of the thumb on mobile touchscreen surfaces. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 1991–2000.
[9]
Xiaojun Bi, Yang Li, and Shumin Zhai. 2013. FFitts law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1363–1372.
[10]
Jesse Burstyn, Amartya Banerjee, and Roel Vertegaal. 2013. FlexView: an evaluation of depth navigation on deformable mobile devices. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI ’13, ACM Press, New York, New York, USA, 193.
[11]
Luis Cavazos Quero, Garam Lee, Jing Yang, and Jundong Cho. 2017. H-Slate. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing - CSCW ’17 Companion, ACM Press, New York, New York, USA, 159–162.
[12]
Tom Djajadiningrat, Kees Overbeeke, and Stephan Wensveen. 2002. But how, Donald, tell us how?: on the creation of meaning in interaction design through feedforward and inherent feedback. In Proceedings of the conference on Designing interactive systems processes, practices, methods, and techniques - DIS ’02, New York, New York, USA, 285.
[13]
Trinh-Minh-Tri Do and Daniel Gatica-Perez. 2010. By their apps you shall understand them. Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia - MUM ’10 (2010), 1–10.
[14]
Alexander Keith Eady. 2019. Deformable interactions to improve the usability of handheld mobile devices. Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019 (October 2019).
[15]
Alexander Keith Eady and Audrey Girouard. 2015. Caret Manipulation using Deformable Input in Mobile Devices. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, ACM, New York, NY, USA, 587–591.
[16]
Rachel Eardley, Anne Roudaut, Steve Gill, and Stephen J. Thompson. 2017. Understanding Grip Shifts: How Form Factors Impact Hand Movements on Mobile Phones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17, ACM Press, New York, New York, USA, 4680–4691.
[17]
Matthew Ernst and Audrey Girouard. 2016. Bending Blindly: Exploring Bend Gestures for the Blind. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, ACM Press, New York, New York, USA, 2088–2096.
[18]
Matthew Ernst and Audrey Girouard. 2016. Exploring Haptics for Learning Bend Gestures for the Blind. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16 (2016), 2097–2104.
[19]
Matthew Ernst, Travis Swan, Victor Cheung, and Audrey Girouard. 2017. Typhlex: Exploring Deformable Input for Blind Users Controlling a Mobile Screen Reader. IEEE Pervasive Comput 16, 4 (October 2017), 28–35.
[20]
Elias Fares, Victor Cheung, and Audrey Girouard. 2017. Effects of bend gesture training on learnability and memorability in a mobile game. In Interactive Surfaces and Spaces, ACM, 240–245.
[21]
Vittorio Fuccella and Benoît Martin. 2017. TouchTap. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter - CHItaly ’17, ACM Press, New York, New York, USA, 1–6.
[22]
Audrey Girouard and Alexander Keith Eady. 2018. Deformable User Interfaces: Using Flexible Electronics for Human Computer Interaction. In 2018 International Flexible Electronics Technology Conference (IFETC), IEEE, 1–3.
[23]
Audrey Girouard, Jessica Lo, Md Riyadh, Farshad Daliri, Alexander Keith Eady, and Jerome Pasquero. 2015. One-Handed Bend Interactions with Deformable Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, ACM Press, New York, 1509–1518.
[24]
Audrey Girouard, Jessica Lo, Md Riyadh, Farshad Daliri, Alexander Keith Eady, and Jerome Pasquero. 2015. One-Handed Bend Interactions with Deformable Smartphones. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 1509–1518.
[25]
Antonio Gomes, Lahiru Priyadarshana, Juan Pablo Carrascal, and Roel Vertegaal. 2016. WhammyPhone: Exploring Tangible Audio Manipulation Using Bend Input on a Flexible Smartphone. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST ’16 Adjunct, ACM, New York, New York, USA, 159–161.
[26]
Daniel Gotsch, Xujing Zhang, Juan Pablo Carrascal, and Roel Vertegaal. 2016. HoloFlex: A Flexible Light-Field Smartphone with a Microlens Array and a P-OLED Touchscreen. Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST ’16 (2016), 69–79.
[27]
David Holman, Roel Vertegaal, Mark Altosaar, Nikolaus Troje, and Derek Johns. 2005. PaperWindows: Interaction Techniques for Digital Paper. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’05, Berlin, Germany, 591.
[28]
Steven Hoober. 2013. How Do Users Really Hold Mobile Devices? UXMatters. Retrieved September 20, 2018 from https://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php
[29]
Amy K Karlson and Benjamin B Bederson. 2008. One-handed Touchscreen Input for Legacy Applications. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2008), 1399–1408.
[30]
Kleomenis Katevas, Ioannis Arapakis, and Martin Pielot. 2018. Typical Phone Use Habits: Intense Use Does Not Predict Negative Well-Being. (2018), 1–13.
[31]
Andreas Komninos, Mark Dunlop, Kyriakos Katsaris, and John Garofalakis. 2018. A glimpse of mobile text entry errors and corrective behaviour in the wild. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - MobileHCI ’18, ACM Press, New York, New York, USA, 221–228.
[32]
Byron Lahey, Audrey Girouard, Winslow Burleson, and Roel Vertegaal. 2011. PaperPhone: Understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In SIGCHI Conference on Human Factors in Computing Systems (CHI ’11), ACM, 1303–1312.
[33]
Jianwei Lai, Navid Rajabi, and Elahe Javadi. 2019. A Shortcut for Caret Positioning on Touch-Screen Phones. (2019). In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '19). Association for Computing Machinery, New York, NY, USA, Article 35, 1–6.
[34]
Jianwei Lai, Navid Rajabi, and Elahe Javadi. 2019. A Shortcut for Caret Positioning on Touch-Screen Phones. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, ACM, New York, NY, USA, 1–6.
[35]
Huy Viet Le, Patrick Bader, Thomas Kosch, and Niels Henze. 2016. Investigating Screen Shifting Techniques to Improve One-Handed Smartphone Usage. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction - NordiCHI ’16, ACM Press, New York, New York, USA, 1–10.
[36]
Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers’ Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18 (2018), 1–12.
[37]
Huy Viet Le, Sven Mayer, Katrin Wolf, and Niels Henze. 2016. Finger Placement and Hand Grasp during Smartphone Interaction. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’16, ACM Press, New York, New York, USA, 2576–2584.
[38]
Sang-Su Lee, Sohyun Kim, Bipil Jin, Eunji Choi, Boa Kim, Xu Jia, Daeeop Kim, and Kun-pyo Lee. 2010. How users manipulate deformable displays as input devices. In SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, New York, USA, 1647–1656.
[39]
Wing Ho Andy Li and Hongbo Fu. 2013. BezelCursor: bezel-initiated cursor for one-handed target acquisition on mobile touch screens. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications on - SA ’13, ACM Press, New York, New York, USA, 1–1.
[40]
Google LLC. Gboard - the Google Keyboard. Retrieved from https://play.google.com/store/apps/details?id=com.google.android.inputmethod.latin&hl=en_CA
[41]
Jessica Lo and Audrey Girouard. 2014. Fabricating Bendy: Design and Development of Deformable Prototypes. IEEE Pervasive Computing Special Issue on Fabrication and Printing 13, 3 (2014), 40–46.
[42]
Jessica Lo and Audrey Girouard. 2017. Bendy: An exploration into gaming with mobile flexible devices. In International Conference on Tangible, Embedded, and Embodied Interaction, ACM, 163–172.
[43]
Fabrice Matulic and Daniel Vogel. 2018. Multiray: Multi-finger raycasting for large displays. Conference on Human Factors in Computing Systems - Proceedings 2018-April, (April 2018).
[44]
Steven Nagels, Raf Ramakers, Kris Luyten, and Wim Deferme. 2018. Silicone Devices. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, ACM Press, New York, New York, USA, 1–13.
[45]
Vinh P. Nguyen, Sang Ho Yoon, Ansh Verma, and Karthik Ramani. 2014. BendID: Flexible interface for localized deformation recognition. In International Joint Conference on Pervasive and Ubiquitous Computing, ACM, 553–557.
[46]
Alan Poston. 2000. Human Engineering Design Data Digest. Washington, DC. Retrieved from https://www.acq.osd.mil/rd/hptb/hfetag/products/documents/HE_Design_Data_Digest.pdf
[47]
Christian Rendl, David Kim, Patrick Parzer, Sean Fanello, Martin Zirkl, Gregor Scheipl, Michael Haller, and Shahram Izadi. 2016. FlexCase: Enhancing Mobile Interaction with a Flexible Sensing and Display Cover. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 5138–5150.
[48]
Anne Roudaut, Stéphane Huot, and Eric Lecolinet. 2008. TapTap and MagStick. In Proceedings of the working conference on Advanced visual interfaces - AVI ’08, ACM Press, New York, New York, USA, 146.
[49]
Frederik Rudeck and Patrick Baudisch. 2012. Rock-Paper-Fibers: Bringing physical affordance to mobile touch devices. Conference on Human Factors in Computing Systems - Proceedings (2012), 1929–1932.
[50]
Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. 2004. Gummi: A Bendable Computer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, 263–270.
[51]
Paden Shorey and Audrey Girouard. 2017. Bendtroller: An exploration of in-game action mappings with a deformable game controller. In SIGCHI Conference on Human Factors in Computing Systems, ACM, 1447–1458.
[52]
Amanda L. Smith and Barbara S. Chaparro. 2015. Smartphone Text Input Method Performance, Usability, and Preference With Younger and Older Adults. Human Factors: The Journal of the Human Factors and Ergonomics Society 57, 6 (September 2015), 1015–1028.
[53]
Kenji Suzuki, Kazumasa Okabe, Ryuuki Sakamoto, and Daisuke Sakamoto. 2016. Fix and slide: Caret Navigation with Movable Background. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI ’16, ACM Press, New York, New York, USA, 478–482.
[54]
Taichi Tajika, Tomoko Yonezawa, and Noriaki Mitsunaga. 2008. Intuitive page-turning interface of e-books on flexible e-paper based on user studies. In Proceeding of the 16th ACM international conference on Multimedia - MM ’08, New York, New York, USA, 793.
[55]
Marc Teyssier, Gilles Bailly, Catherine Pelachaud, Eric Lecolinet, Andrew Conn, and Anne Roudaut. 2019. Skin-On Interfaces: A Bio-Driven Approach for Artificial Skin Design to Cover Interactive Devices. (2019).
[56]
Giovanni Maria Troiano, Esben Warming Pedersen, and Kasper Hornbæk. 2014. User-defined gestures for elastic, deformable displays. In International Working Conference on Advanced Visual Interfaces, ACM, 1–8.
[57]
Matthieu B. Trudeau, Deanna S. Asakawa, Devin L. Jindrich, and Jack T. Dennerlein. 2016. Two-handed grip on a mobile phone affords greater thumb motor performance, decreased variability, and a more extended thumb posture than a one-handed grip. Appl Ergon 52, (January 2016), 24–28.
[58]
Roel Vertegaal and Ivan Poupyrev. 2008. Organic User Interfaces. Commun ACM 51, 6 (June 2008), 26–30.
[59]
Daniel Vogel and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’07, ACM Press, New York, New York, USA, 657.
[60]
Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous keyboard for small mobile devices. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services - MobiSys ’14, ACM Press, New York, New York, USA, 14–27.
[61]
Kristen Warren, Jessica Lo, Vaibhav Vadgama, and Audrey Girouard. 2013. Bending the rules: Bend gesture classification for flexible displays. In SIGCHI Conference on Human Factors in Computing Systems, ACM, 607–610.
[62]
Chihiro Watanabe, Alvaro Cassinelli, Yoshihiro Watanabe, and Masatoshi Ishikawa. 2014. Generic method for crafting deformable interfaces to physically augment smartphones. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA ’14, ACM Press, New York, New York, USA, 1309–1314.
[63]
Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. Uncertain text entry on mobile devices. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, ACM Press, New York, New York, USA, 2307–2316.
[64]
Neng-Hao Yu, Da-Yuan Huang, Jia-Jyun Hsu, and Yi-Ping Hung. 2013. Rapid Selection of Hard-to-Access Targets by Thumb on Mobile Touch-Screens. Proc. MobileHCI (2013), 400.
[65]
Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: Reimagining Keyboard Interaction. Commun ACM 55, 9 (2012), 91.
Index Terms
- BendAide: A Deformable Interface to Augment Touchscreen Mobile Devices
Index terms have been assigned to the content through auto-classification.
Recommendations
Deformable Interactions to Improve the Usability of Handheld Mobile Devices
MobileHCI '19: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and ServicesHandheld mobile device use is dominated by touch interfaces; keyboards and other physical inputs are disappearing. In parallel, there is a trend towards more sophisticated devices supporting complex use. While touch devices are usable, the user ...
Evaluating reading and analysis tasks on mobile devices: a case study of tilt and flick scrolling
OZCHI '09: Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7Flick scrolling is a natural scrolling method for mobile touch devices such as the iPhone™. It is useful not only for its performance but perhaps even more so for its ease of use and user experience. Tilt scrolling instead uses the device's tilt to ...
Integrating a multitouch kiosk system with mobile devices and multimodal interaction
ITS '10: ACM International Conference on Interactive Tabletops and SurfacesWe present Calisto, a service-oriented information kiosk system for public places, like museums or hotel lobbies. Calisto supports collaboration between multiple users. They can connect their mobile devices to the large public terminal and share ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
June 2024
437 pages
ISBN:9798400718281
DOI:10.1145/3670947
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
GI '24
Acceptance Rates
Overall Acceptance Rate 206 of 508 submissions, 41%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 28Total Downloads
- Downloads (Last 12 months)28
- Downloads (Last 6 weeks)10
Reflects downloads up to 20 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format