[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3678957.3685749acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

NearFetch: Enhancing Touch-Based Mobile Interaction on Public Displays with an Embedded Programmable NFC Array

Published: 04 November 2024 Publication History

Abstract

Public displays are becoming ubiquitous in urban spaces but face challenges such as display blindness, interaction blindness, and attention competition. As mobile devices continue to evolve, people increasingly rely on smartphones as their primary means of accessing information. Recent studies suggest that integrating public displays into today’s hyperconnected society by considering their interactions with personal mobile devices will enhance public display interactions. This paper presents NearFetch, an interaction technique that enhances public displays by embedding programmable Near Field Communication (NFC) arrays within the screen. NearFetch transforms public displays into interactive surfaces for mobile devices, enabling users to access relevant information and services by simply touching the displayed content with their smartphones. The key technical contribution of NearFetch lies in the design and implementation of embedded programmable NFC arrays, which ensure consistency between Radio Frequency (RF) signals and display content. Experimental results demonstrate that NearFetch achieves an average recognition accuracy of 94.33%. System Usability Scale (SUS) and NASA Task Load Index (NASA-TLX) questionnaire surveys indicate that NearFetch provides robust usability. The system enables seamless access to information and services on public displays, while reducing user configuration efforts, enhancing privacy, and improving interaction efficiency.

Supplemental Material

MP4 File
Presentation video - short version

References

[1]
Christopher Ackad, Andrew Clayphan, Martin Tomitsch, and Judy Kay. 2015. An in-the-wild study of learning mid-air gestures to browse hierarchical information at a large interactive public display. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Osaka, Japan) (UbiComp ’15). Association for Computing Machinery, New York, NY, USA, 1227–1238. https://doi.org/10.1145/2750858.2807532
[2]
Asma Almutairi, Mateusz Mikusz, Hassam Niaz, Ludwig Trotter, and Nigel Davies. 2019. Why simple is best: lessons from designing an emergency system for public displays. In Proceedings of the 8th ACM International Symposium on Pervasive Displays (Palermo, Italy) (PerDis ’19). Association for Computing Machinery, New York, NY, USA, Article 13, 7 pages. https://doi.org/10.1145/3321335.3324949
[3]
Florian Alt, Alireza Sahami Shirazi, Thomas Kubitza, and Albrecht Schmidt. 2013. Interaction techniques for creating and exchanging content with public displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 1709–1718. https://doi.org/10.1145/2470654.2466226
[4]
Craig Anslow and B. L. William Wong. 2017. Effects of the Display Angle and Physical Size on Large Touch Displays in the Work Place. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (Brighton, United Kingdom) (ISS ’17). Association for Computing Machinery, New York, NY, USA, 318–323. https://doi.org/10.1145/3132272.3135080
[5]
Lisa Anthony, Kathryn A. Stofer, Annie Luc, and Jacob O. Wobbrock. 2016. Gestures by Children and Adults on Touch Tables and Touch Walls in a Public Science Center. In Proceedings of the The 15th International Conference on Interaction Design and Children (Manchester, United Kingdom) (IDC ’16). Association for Computing Machinery, New York, NY, USA, 344–355. https://doi.org/10.1145/2930674.2930682
[6]
C. Ardito, M. F. Costabile, R. Lanzilotti, A. De Angeli, and G. Desolda. 2012. A field study of a multi-touch display at a conference. In Proceedings of the International Working Conference on Advanced Visual Interfaces (Capri Island, Italy) (AVI ’12). Association for Computing Machinery, New York, NY, USA, 580–587. https://doi.org/10.1145/2254556.2254664
[7]
Gilbert Beyer, Vincent Binder, Nina Jäger, and Andreas Butz. 2014. The puppeteer display: attracting and actively shaping the audience with an interactive public banner display. In Proceedings of the 2014 Conference on Designing Interactive Systems (Vancouver, BC, Canada) (DIS ’14). Association for Computing Machinery, New York, NY, USA, 935–944. https://doi.org/10.1145/2598510.2598575
[8]
Omair Shahzad Bhatti, Michael Barz, and Daniel Sonntag. 2021. EyeLogin - Calibration-free Authentication Method for Public Displays Using Eye Gaze. In ACM Symposium on Eye Tracking Research and Applications (Virtual Event, Germany) (ETRA ’21 Short Papers). Association for Computing Machinery, New York, NY, USA, Article 14, 7 pages. https://doi.org/10.1145/3448018.3458001
[9]
Victor Cheung and Stacey D. Scott. 2015. Studying Attraction Power in Proxemics-Based Visual Concepts for Large Public Interactive Displays. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (Madeira, Portugal) (ITS ’15). Association for Computing Machinery, New York, NY, USA, 93–102. https://doi.org/10.1145/2817721.2817749
[10]
Keith Cheverst, Alan Dix, Daniel Fitton, Chris Kray, Mark Rouncefield, Corina Sas, George Saslis-Lagoudakis, and Jennifer G. Sheridan. 2005. Exploring bluetooth based mobile phone interaction with the hermes photo display. In Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services (Salzburg, Austria) (MobileHCI ’05). Association for Computing Machinery, New York, NY, USA, 47–54. https://doi.org/10.1145/1085777.1085786
[11]
Jorgos Coenen, Sandy Claes, and Andrew Vande Moere. 2017. The concurrent use of touch and mid-air gestures or floor mat interaction on a public display. In Proceedings of the 6th ACM International Symposium on Pervasive Displays (Lugano, Switzerland) (PerDis ’17). Association for Computing Machinery, New York, NY, USA, Article 9, 9 pages. https://doi.org/10.1145/3078810.3078819
[12]
Emmanuel Courtoux, Caroline Appert, and Olivier Chapuis. 2023. SurfAirs: Surface + Mid-air Input for Large Vertical Displays. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (, Hamburg, Germany, ) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 644, 15 pages. https://doi.org/10.1145/3544548.3580877
[13]
Travis Cox, Marcus Carter, and Eduardo Velloso. 2016. Public DisPLAY: social games on interactive public screens. In Proceedings of the 28th Australian Conference on Computer-Human Interaction (Launceston, Tasmania, Australia) (OzCHI ’16). Association for Computing Machinery, New York, NY, USA, 371–380. https://doi.org/10.1145/3010915.3010917
[14]
Nicholas S. Dalton, Emily Collins, and Paul Marshall. 2015. Display Blindness? Looking Again at the Visibility of Situated Displays using Eye-tracking. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 3889–3898. https://doi.org/10.1145/2702123.2702150
[15]
Linda Di Geronimo, Andrea Canonica, Maria Husmann, and Moira C. Norrie. 2017. Continuous tilting interaction techniques on mobile devices for controlling public displays. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Lisbon, Portugal) (EICS ’17). Association for Computing Machinery, New York, NY, USA, 21–26. https://doi.org/10.1145/3102113.3102120
[16]
Shenfeng Fei, Andruid Kerne, Ajit Jain, Andrew M. Webb, and Yin Qu. 2013. Positioning Portals with Peripheral NFC Tags to Embody Trans-Surface Interaction. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (St. Andrews, Scotland, United Kingdom) (ITS ’13). Association for Computing Machinery, New York, NY, USA, 317–320. https://doi.org/10.1145/2512349.2514593
[17]
Patrick Tobias Fischer and Eva Hornecker. 2012. Urban HCI: spatial aspects in the design of shared encounters for media facades. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 307–316. https://doi.org/10.1145/2207676.2207719
[18]
Marcus Foth, Martin Tomitsch, Laura Forlano, M. Hank Haeusler, and Christine Satchell. 2016. Citizens breaking out of filter bubbles: urban screens as civic media. In Proceedings of the 5th ACM International Symposium on Pervasive Displays (Oulu, Finland) (PerDis ’16). Association for Computing Machinery, New York, NY, USA, 140–147. https://doi.org/10.1145/2914920.2915010
[19]
Hans Gellersen, Carl Fischer, Dominique Guinard, Roswitha Gostner, Gerd Kortuem, Christian Kray, Enrico Rukzio, and Sara Streng. 2009. Supporting device discovery and spontaneous interaction with spatial references. Personal Ubiquitous Comput. 13, 4 (may 2009), 255–264. https://doi.org/10.1007/s00779-008-0206-3
[20]
John Hardy, Enrico Rukzio, and Nigel Davies. 2011. Real world responses to interactive gesture based public displays. In Proceedings of the 10th International Conference on Mobile and Ubiquitous Multimedia (Beijing, China) (MUM ’11). Association for Computing Machinery, New York, NY, USA, 33–39. https://doi.org/10.1145/2107596.2107600
[21]
Robert Hardy and Enrico Rukzio. 2008. Touch & interact: touch-based interaction of mobile phones with displays. In Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (Amsterdam, The Netherlands) (MobileHCI ’08). Association for Computing Machinery, New York, NY, USA, 245–254. https://doi.org/10.1145/1409240.1409267
[22]
Luke Hespanhol and Martin Tomitsch. 2015. Strategies for Intuitive Interaction in Public Urban Spaces. Interacting with Computers 27, 3 (2015), 311–326. https://doi.org/10.1093/iwc/iwu051
[23]
Philip Speranza Jason O. Germany and Dan Anthony. 2019. Eliciting Public Display Awareness and Engagement: An Experimental Study and Semantic Strategy. International Journal of Human–Computer Interaction 35, 20 (2019), 1975–1985. https://doi.org/10.1080/10447318.2019.1597572 arXiv:https://doi.org/10.1080/10447318.2019.1597572
[24]
Huiliang Jin, Tao Xu, Bertrand David, and René Chalon. 2014. Direct migrator: Eliminating borders between personal mobile devices and pervasive displays. In 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS). 557–562. https://doi.org/10.1109/PerComW.2014.6815267
[25]
Mohamed Khamis, Florian Alt, and Andreas Bulling. 2016. Challenges and design space of gaze-enabled public displays. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (Heidelberg, Germany) (UbiComp ’16). Association for Computing Machinery, New York, NY, USA, 1736–1745. https://doi.org/10.1145/2968219.2968342
[26]
Mohamed Khamis, Andreas Bulling, and Florian Alt. 2015. Tackling challenges of interactive public displays using gaze. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (Osaka, Japan) (UbiComp/ISWC’15 Adjunct). Association for Computing Machinery, New York, NY, USA, 763–766. https://doi.org/10.1145/2800835.2807951
[27]
Hannu Kukka, Heidi Oja, Vassilis Kostakos, Jorge Gonçalves, and Timo Ojala. 2013. What makes you click: exploring visual signals to entice interaction on public displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 1699–1708. https://doi.org/10.1145/2470654.2466225
[28]
Rui Neves Madeira, Pedro Albuquerque Santos, and Nuno Correia. 2020. Using Personalisation to improve User Experience in Public Display Systems with Mobile Interaction. In Proceedings of the 17th International Conference on Advances in Mobile Computing & Multimedia (Munich, Germany) (MoMM2019). Association for Computing Machinery, New York, NY, USA, 3–12. https://doi.org/10.1145/3365921.3365934
[29]
Ville Mäkelä, Sumita Sharma, Jaakko Hakulinen, Tomi Heimonen, and Markku Turunen. 2017. Challenges in Public Display Deployments: A Taxonomy of External Factors. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 3426–3475. https://doi.org/10.1145/3025453.3025798
[30]
Jörg Müller, Dennis Wilmsmann, Juliane Exeler, Markus Buzeck, Albrecht Schmidt, Tim Jay, and Antonio Krüger. 2009. Display Blindness: The Effect of Expectations on Attention towards Digital Signage. In Proceedings of the 7th International Conference on Pervasive Computing (Nara, Japan) (Pervasive ’09). Springer-Verlag, Berlin, Heidelberg, 1–8. https://doi.org/10.1007/978-3-642-01516-8_1
[31]
Omar Namnakani, Penpicha Sinrattanavong, Yasmeen Abdrabou, Andreas Bulling, Florian Alt, and Mohamed Khamis. 2023. GazeCast: Using Mobile Devices to Allow Gaze-based Interaction on Public Displays. In Proceedings of the 2023 Symposium on Eye Tracking Research and Applications (Tubingen, Germany) (ETRA ’23). Association for Computing Machinery, New York, NY, USA, Article 92, 8 pages. https://doi.org/10.1145/3588015.3589663
[32]
Pai Chet Ng, James She, Kang Eun Jeon, and Matthias Baldauf. 2017. When Smart Devices Interact With Pervasive Screens: A Survey. ACM Trans. Multimedia Comput. Commun. Appl. 13, 4, Article 55 (aug 2017), 23 pages. https://doi.org/10.1145/3115933
[33]
Hiroaki Ogata, Chengiu Yin, Moushir M El-Bishouty, and Yoneo Yano. 2010. Computer supported ubiquitous learning environment for vocabulary learning. International Journal of Learning Technology 5, 1 (2010), 5–24.
[34]
Timo Ojala, Vassilis Kostakos, Hannu Kukka, Tommi Heikkinen, Tomas Linden, Marko Jurmu, Simo Hosio, Fabio Kruger, and Daniele Zanni. 2012. Multipurpose Interactive Public Displays in the Wild: Three Years Later. Computer 45, 5 (2012), 42–49. https://doi.org/10.1109/MC.2012.115
[35]
Laxmi Pandey, Khalad Hasan, and Ahmed Sabbir Arif. 2021. Acceptability of Speech and Silent Speech Input Methods in Private and Public. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (, Yokohama, Japan, ) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 251, 13 pages. https://doi.org/10.1145/3411764.3445430
[36]
Callum Parker, Martin Tomitsch, Nigel Davies, Nina Valkanova, and Judy Kay. 2020. Foundations for Designing Public Interactive Displays that Provide Value to Users. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (, Honolulu, HI, USA, ) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376532
[37]
Callum Parker, Martin Tomitsch, and Judy Kay. 2018. Does the Public Still Look at Public Displays? A Field Observation of Public Displays in the Wild. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 2, Article 73 (jul 2018), 24 pages. https://doi.org/10.1145/3214276
[38]
Sara Price and Yvonne Rogers. 2004. Let’s get physical: The learning benefits of interacting in digitally augmented physical spaces. Computers & Education 43, 1-2 (2004), 137–151.
[39]
Jun Rekimoto. 1997. Pick-and-drop: a direct manipulation technique for multiple computer environments. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology (Banff, Alberta, Canada) (UIST ’97). Association for Computing Machinery, New York, NY, USA, 31–39. https://doi.org/10.1145/263407.263505
[40]
Jun Rekimoto, Brygg Ullmer, and Haruo Oba. 2001. DataTiles: a modular platform for mixed physical and graphical interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’01). Association for Computing Machinery, New York, NY, USA, 269–276. https://doi.org/10.1145/365024.365115
[41]
Isabel Benavente Rodriguez and Nicolai Marquardt. 2017. Gesture Elicitation Study on How to Opt-in & Opt-out from Interactions with Public Displays. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (Brighton, United Kingdom) (ISS ’17). Association for Computing Machinery, New York, NY, USA, 32–41. https://doi.org/10.1145/3132272.3134118
[42]
Iván Sánchez, Marta Cortés, Jukka Riekki, and Mika Oja. 2011. NFC-based interactive learning environments for children. In Proceedings of the 10th international conference on interaction design and children. 205–208.
[43]
Corina Sas and Alan Dix. 2008. Designing and evaluating mobile phone-based interaction with public displays. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy) (CHI EA ’08). Association for Computing Machinery, New York, NY, USA, 3941–3944. https://doi.org/10.1145/1358628.1358962
[44]
Dominik Schmidt, Fadi Chehimi, Enrico Rukzio, and Hans Gellersen. 2010. PhoneTouch: a technique for direct phone interaction on surfaces. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machinery, New York, NY, USA, 13–16. https://doi.org/10.1145/1866029.1866034
[45]
Stefan Schneegass. 2015. There is more to Interaction with Public Displays than Kinect: Using Wearables to Interact with Public Displays. In Proceedings of the 4th International Symposium on Pervasive Displays (Saarbruecken, Germany) (PerDis ’15). Association for Computing Machinery, New York, NY, USA, 243–244. https://doi.org/10.1145/2757710.2776803
[46]
Khoovirajsingh Seewoonauth, Enrico Rukzio, Robert Hardy, and Paul Holleis. 2009. Touch & connect and touch & select: interacting with a computer by touching it with a mobile phone. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (Bonn, Germany) (MobileHCI ’09). Association for Computing Machinery, New York, NY, USA, Article 36, 9 pages. https://doi.org/10.1145/1613858.1613905
[47]
James She, Jon Crowcroft, Hao Fu, and Flora Li. 2014. Convergence of interactive displays with smart mobile devices for effective advertising: A survey. ACM Trans. Multimedia Comput. Commun. Appl. 10, 2, Article 17 (feb 2014), 16 pages. https://doi.org/10.1145/2557450
[48]
Edward Tse, Mark Hancock, and Saul Greenberg. 2007. Speech-filtered bubble ray: improving target acquisition on display walls. In Proceedings of the 9th International Conference on Multimodal Interfaces (Nagoya, Aichi, Japan) (ICMI ’07). Association for Computing Machinery, New York, NY, USA, 307–314. https://doi.org/10.1145/1322192.1322245
[49]
Radu-Daniel Vatavu. 2013. On designing interactivity awareness for ambient displays. Multimedia tools and applications 66 (2013), 59–80.
[50]
Tim Weißker, Andreas Berst, Johannes Hartmann, and Florian Echtler. 2016. The massive mobile multiuser framework: enabling ad-hoc realtime interaction on public displays with mobile devices. In Proceedings of the 5th ACM International Symposium on Pervasive Displays (Oulu, Finland) (PerDis ’16). Association for Computing Machinery, New York, NY, USA, 168–174. https://doi.org/10.1145/2914920.2915004
[51]
Yan Zhai, Guoying Zhao, Toni Alatalo, Janne Heikkilä, Timo Ojala, and Xinyuan Huang. 2013. Gesture interaction for wall-sized touchscreen display. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication (Zurich, Switzerland) (UbiComp ’13 Adjunct). Association for Computing Machinery, New York, NY, USA, 175–178. https://doi.org/10.1145/2494091.2494148
[52]
Yi Zhao, Joshua R. Smith, and Alanson Sample. 2015. NFC-WISP: an open source software defined near field RFID sensing platform. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (Osaka, Japan) (UbiComp/ISWC’15 Adjunct). Association for Computing Machinery, New York, NY, USA, 369–372. https://doi.org/10.1145/2800835.2800912
[53]
Yu Zheng, Licia Capra, Ouri Wolfson, and Hai Yang. 2014. Urban Computing: Concepts, Methodologies, and Applications. ACM Trans. Intell. Syst. Technol. 5, 3, Article 38 (sep 2014), 55 pages. https://doi.org/10.1145/2629592
[54]
Jamie Zigelbaum, Adam Kumpf, Alejandro Vazquez, and Hiroshi Ishii. 2008. Slurp: tangibility spatiality and an eyedropper. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy) (CHI EA ’08). Association for Computing Machinery, New York, NY, USA, 2565–2574. https://doi.org/10.1145/1358628.1358713

Index Terms

  1. NearFetch: Enhancing Touch-Based Mobile Interaction on Public Displays with an Embedded Programmable NFC Array

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICMI '24: Proceedings of the 26th International Conference on Multimodal Interaction
    November 2024
    725 pages
    ISBN:9798400704628
    DOI:10.1145/3678957
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 November 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Hyperconnected society
    2. Input Techniques
    3. Interactive Surfaces
    4. Mobile Interaction
    5. NFC
    6. Public Displays
    7. Public Interactive Displays

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICMI '24
    ICMI '24: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
    November 4 - 8, 2024
    San Jose, Costa Rica

    Acceptance Rates

    Overall Acceptance Rate 453 of 1,080 submissions, 42%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 55
      Total Downloads
    • Downloads (Last 12 months)55
    • Downloads (Last 6 weeks)55
    Reflects downloads up to 12 Dec 2024

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media