[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3290605.3300721acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Gaze-Guided Narratives: Adapting Audio Guide Content to Gaze in Virtual and Real Environments

Published: 02 May 2019 Publication History

Abstract

Exploring a city panorama from a vantage point is a popular tourist activity. Typical audio guides that support this activity are limited by their lack of responsiveness to user behavior and by the difficulty of matching audio descriptions to the panorama. These limitations can inhibit the acquisition of information and negatively affect user experience. This paper proposes Gaze-Guided Narratives as a novel interaction concept that helps tourists find specific features in the panorama (gaze guidance) while adapting the audio content to what has been previously looked at (content adaptation). Results from a controlled study in a virtual environment (n=60) revealed that a system featuring both gaze guidance and content adaptation obtained better user experience, lower cognitive load, and led to better performance in a mapping task compared to a classic audio guide. A second study with tourists situated at a vantage point (n=16) further demonstrated the feasibility of this approach in the real world.

Supplementary Material

MP4 File (paper491p.mp4)
Preview video
MP4 File (pn3933.mp4)
Supplemental video

References

[1]
Gregory D Abowd, Christopher G Atkeson, Jason Hong, Sue Long, Rob Kooper, and Mike Pinkerton. 1997. Cyberguide: A mobile contextaware tour guide. Wireless networks 3, 5 (1997), 421--433.
[2]
Vasileios-Athanasios Anagnostopoulos, Michal Havlena, Peter Kiefer, Ioannis Giannopoulos, Konrad Schindler, and Martin Raubal. 2017. Gaze-Informed Location Based Services. International Journal of Geographical Information Science 31, 9 (2017), 1770--1797.
[3]
Maoz Azaryahu and Kenneth E. Foote. 2008. Historical space as narrative medium: on the configuration of spatial narratives of time at historical sites. GeoJournal 73, 3 (Nov 2008), 179--194.
[4]
Reynold Bailey, Ann McNamara, Nisha Sudarsanam, and Cindy Grimm. 2009. Subtle gaze direction. ACM Transactions on Graphics (TOG) 28, 4 (2009), 100.
[5]
Rafael Ballagas, André Kuntze, and Steffen P Walz. 2008. Gaming tourism: Lessons from evaluating REXplorer, a pervasive game for tourists. In International Conference on Pervasive Computing. Springer, 244--261.
[6]
Stephen Brewster, Joanna Lumsden, Marek Bell, Malcolm Hall, and Stuart Tasker. 2003. Multimodal 'eyes-free' interaction techniques for wearable devices. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 473--480.
[7]
John Brooke. 1996. SUS - A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.
[8]
Andreas Bulling, Andrew T Duchowski, and Päivi Majaranta. 2011. PETMEI 2011: the 1st international workshop on pervasive eye tracking and mobile eye-based interaction. In Proceedings of the 13th international conference on Ubiquitous computing. ACM, 627--628.
[9]
Andreas Bulling, Jamie A. Ward, Hans Gellersen, and Gerhard Troster. 2011. Eye Movement Analysis for Activity Recognition Using Electrooculography. IEEE Trans. Pattern Anal. Mach. Intell. 33, 4 (April 2011), 741--753.
[10]
Marc Cavazza and David Pizzi. 2006. Narratology for interactive storytelling: A critical introduction. In International Conference on Technologies for Interactive Digital Storytelling and Entertainment. Springer, 72--83.
[11]
Keith Cheverst, Nigel Davies, Keith Mitchell, and Adrian Friday. 2000. Experiences of developing and deploying a context-aware tourist guide: the GUIDE project. In Proceedings of the 6th annual international conference on Mobile computing and networking. ACM, 20--31.
[12]
Keith William John Cheverst, Ian Norman Gregory, and Helen Turner. 2016. Encouraging Visitor Engagement and Reflection with the Landscape of the English Lake District: Exploring the potential of Locative Media. In International Workshop on'Unobtrusive User Experiences with Technology in Nature'. 1--5.
[13]
Kenny R Coventry, Thora Tenbrink, and John Bateman. 2009. Spatial language and dialogue. Vol. 3. OUP Oxford.
[14]
Steven Dow, Jaemin Lee, Christopher Oezbek, Blair Maclntyre, Jay David Bolter, and Maribeth Gandy. 2005. Exploring spatial narratives and mixed reality experiences in Oakland Cemetery. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology. ACM, 51--60.
[15]
Lex Fridman, Heishiro Toyoda, Sean Seaman, Bobbie Seppelt, Linda Angell, Joonbum Lee, Bruce Mehler, and Bryan Reimer. 2017. What Can Be Predicted from Six Seconds of Driver Glances?. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 2805--2813.
[16]
Brian F. Goldiez, Ali M. Ahmad, and Peter A. Hancock. 2007. Effects of Augmented Reality Display Settings on Human Wayfinding Performance. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37, 5 (Sept 2007), 839--845.
[17]
Jonna Häkkilä, Keith Cheverst, Johannes Schöning, Nicola J. Bidwell, Simon Robinson, and Ashley Colley. 2016. NatureCHI: Unobtrusive User Experiences with Technology in Nature. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 3574--3580.
[18]
Dai-In Han, Timothy Jung, and Alex Gibson. 2013. Dublin AR: implementing augmented reality in tourism. In Information and communication technologies in tourism 2014. Springer, 511--523.
[19]
Dan Witzner Hansen and Arthur E.C. Pece. 2005. Eye tracking in the wild. Computer Vision and Image Understanding 98, 1 (2005), 155 -- 181. Special Issue on Eye Detection and Tracking.
[20]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASATLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139 -- 183.
[21]
Mary Hegarty. 2002. Development of a self-report measure of environmental spatial ability. Intelligence 30, 5 (2002), 425--447.
[22]
Florian Heller, Aaron Krämer, and Jan Borchers. 2014. Simplifying Orientation Measurement for Mobile Audio Augmented Reality Applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 615--624.
[23]
Kenneth Holmqvist, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. OUP Oxford.
[24]
Michael Xuelin Huang, Jiajia Li, Grace Ngai, and Hong Va Leong. 2017. ScreenGlint: Practical, In-situ Gaze Estimation on Smartphones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 2546--2557.
[25]
Jari Kangas, Deepak Akkil, Jussi Rantala, Poika Isokoski, Päivi Majaranta, and Roope Raisamo. 2014. Gaze Gestures and Haptic Feedback in Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 435--438.
[26]
Moritz Kassner, William Patera, and Andreas Bulling. 2014. Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gazebased Interaction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp '14 Adjunct). ACM, New York, NY, USA, 1151--1160.
[27]
Mohamed Khamis, Axel Hoesl, Alexander Klimczak, Martin Reiss, Florian Alt, and Andreas Bulling. 2017. EyeScout: Active Eye Tracking for Position and Movement Independent Gaze Interaction with Large Public Displays. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 155--166.
[28]
Peter Kiefer, Ioannis Giannopoulos, Dominik Kremer, Christoph Schlieder, and Martin Raubal. 2014. Starting to get bored: An outdoor eye tracking study of tourists exploring a city panorama. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '14). ACM, New York, NY, USA, 315--318.
[29]
Izabela Krejtz, Agnieszka Szarkowska, Krzysztof Krejtz, Agnieszka Walczak, and Andrew Duchowski. 2012. Audio Description As an Aural Guide of Children's Visual Attention: Evidence from an Eye-tracking Study. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12). ACM, New York, NY, USA, 99--106.
[30]
Krzysztof Krejtz, Andrew Duchowski, Izabela Krejtz, Agnieszka Szarkowska, and Agata Kopacz. 2016. Discerning Ambient/Focal Attention with Coefficient K. ACM Trans. Appl. Percept. 13, 3, Article 11 (May 2016), 20 pages.
[31]
Kuno Kurzhals, Emine Cetinkaya, Yongtao Hu, Wenping Wang, and Daniel Weiskopf. 2017. Close to the Action: Eye-Tracking Evaluation of Speaker-Following Subtitles. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6559--6568.
[32]
Pupil Labs. 2017. Hololens and BT300 eye tracking add-ons. https://pupil-labs.com/blog/2017-03/ hololens-and-bt300-eye-tracking-add-ons/. Accessed: 2018--12--25.
[33]
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and Evaluation of a User Experience Questionnaire. In HCI and Usability for Education and Work, Andreas Holzinger (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 63--76.
[34]
Viktor Losing, Lukas Rottkamp, Michael Zeunert, and Thies Pfeiffer. 2014. Guiding visual search tasks using gaze-contingent auditory feedback. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, 1093--1102.
[35]
Patrick Luley, Roland Perko, Johannes Weinzerl, Lucas Paletta, and Alexander Almer. 2012. Mobile Augmented Reality for Tourists MARFT. Advances in Location-Based Services Lecture Notes in Geoinformation and Cartography (2012), 21--36.
[36]
Päivi Majaranta and Andreas Bulling. 2014. Eye Tracking and EyeBased Human--Computer Interaction. Springer London, London, 39--65.
[37]
Ann McNamara, Thomas Booth, Srinivas Sridharan, Stephen Caffey, Cindy Grimm, and Reynold Bailey. 2012. Directing gaze in narrative art. In Proceedings of the ACM Symposium on Applied Perception. ACM, 63--70.
[38]
NaturalSoft Ltd. 2018. NaturalReader Commercial. https://www. naturalreaders.com/commercial.html {Online}.
[39]
Marcel Neuenhaus and Maha Aly. 2017. Empathy Up. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, New York, NY, USA, 86--92.
[40]
Valeria Orso, Alessandra Varotto, Stefano Rodaro, Anna Spagnolli, Giulio Jacucci, Salvatore Andolina, Jukka Leino, and Luciano Gamberini. 2017. A Two-step, User-centered Approach to Personalized Tourist Recommendations. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter (CHItaly '17). ACM, New York, NY, USA, Article 7, 5 pages.
[41]
Seonwook Park, Xucong Zhang, Andreas Bulling, and Otmar Hilliges. 2018. Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications (ETRA '18). ACM, New York, NY, USA, Article 21, 10 pages.
[42]
Daniela Petrelli, Nick Dulake, Mark T. Marshall, Anna Pisetti, and Elena Not. 2016. Voices from the War: Design As a Means of Understanding the Experience of Visiting Heritage. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1033--1044.
[43]
Martin Pielot, Benjamin Poppinga, Wilko Heuten, and Susanne Boll. 2012. PocketNavigator: Studying Tactile Navigation Systems In-situ. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 3131--3140.
[44]
Pernilla Qvarfordt, David Beymer, and Shumin Zhai. 2005. RealTourist -- A Study of Augmenting Human-Human and Human-Computer Dialogue with Eye-Gaze Overlay. In Human-Computer Interaction - INTERACT 2005, Maria Francesca Costabile and Fabio Paternò (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 767--780.
[45]
Patrick Renner and Thies Pfeiffer. 2017. Attention guiding techniques using peripheral vision and eye tracking for feedback in augmentedreality-based assistance systems. In 2017 IEEE Symposium on 3D User Interfaces (3DUI). 186--194.
[46]
Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: An efficient alternative to SIFT or SURF. In 2011 International Conference on Computer Vision. 2564--2571.
[47]
Christian Sailer, Peter Kiefer, and Raubal Martin. 2015. An integrated learning management system for location-based mobile learning. Proceedings of the 11th International Conference on Mobile Learning 2015, 118--122.
[48]
Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying Fixations and Saccades in Eye-tracking Protocols. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (ETRA '00). ACM, New York, NY, USA, 71--78.
[49]
Thiago Santini, Hanna Brinkmann, Luise Reitstätter, Helmut Leder, Raphael Rosenberg, Wolfgang Rosenstiel, and Enkelejda Kasneci. 2018. The art of pervasive eye tracking: unconstrained eye tracking in the Austrian Gallery Belvedere. In Proceedings of the 7th Workshop on Pervasive Eye Tracking and Mobile Eye-Based Interaction. ACM, 5.
[50]
Simon Schenk, Marc Dreiser, Gerhard Rigoll, and Michael Dorr. 2017. GazeEverywhere: Enabling Gaze-only User Interaction on an Unmodified Desktop PC in Everyday Scenarios. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3034--3044.
[51]
Albrecht Schmidt. 2000. Implicit human computer interaction through context. Personal Technologies 4, 2 (01 Jun 2000), 191--199.
[52]
Johannes Schöning, Brent Hecht, and Nicole Starosielski. 2008. Evaluating automatically generated location-based stories for tourists. In CHI'08 extended abstracts on Human factors in computing systems. ACM, 2937--2942.
[53]
Tobii. 2018. Enhanced PC Games with Eye Tracking. https:// tobiigaming.com/games/. Accessed: 2018--12--25.
[54]
Waldo R. Tobler. 1994. Bidimensional Regression. Geographical Analysis 26, 3 (1994), 187--212.
[55]
Takumi Toyama, Thomas Kieninger, Faisal Shafait, and Andreas Dengel. 2011. Museum Guide 2.0 -- An Eye-Tracking based Personal Assistant for Museums and Exhibits. 1 (05 2011), 103--110.
[56]
Tom Tullis and Bill Albert. 2013. Chapter 6 - Self-Reported Metrics. In Measuring the User Experience (Second Edition) (second edition ed.), Tom Tullis and Bill Albert (Eds.). Morgan Kaufmann, Boston, 121 -- 161.
[57]
Wikimedia Foundation, Inc. 2018. Wikipedia: The free encyclopedia. https://www.wikipedia.org {Online}.

Cited By

View all
  • (2024)ViRgilites: Multilevel Feedforward for Multimodal Interaction in VRProceedings of the ACM on Human-Computer Interaction10.1145/36586458:EICS(1-24)Online publication date: 17-Jun-2024
  • (2024)Making maps & visualizations for mobile devices: A research agenda for mobile-first and responsive cartographic designJournal of Location Based Services10.1080/17489725.2023.2251423(1-71)Online publication date: 3-Apr-2024
  • (2024)Gaze-based detection of mind wandering during audio-guided panorama viewingScientific Reports10.1038/s41598-024-79172-x14:1Online publication date: 14-Nov-2024
  • Show More Cited By

Index Terms

  1. Gaze-Guided Narratives: Adapting Audio Guide Content to Gaze in Virtual and Real Environments

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
        May 2019
        9077 pages
        ISBN:9781450359702
        DOI:10.1145/3290605
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 02 May 2019

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. gaze-guided narratives
        2. outdoor eye tracking
        3. tourist guide

        Qualifiers

        • Research-article

        Funding Sources

        • ETH Zurich Research Grant

        Conference

        CHI '19
        Sponsor:

        Acceptance Rates

        CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;
        Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

        Upcoming Conference

        CHI 2025
        ACM CHI Conference on Human Factors in Computing Systems
        April 26 - May 1, 2025
        Yokohama , Japan

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)110
        • Downloads (Last 6 weeks)13
        Reflects downloads up to 17 Dec 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)ViRgilites: Multilevel Feedforward for Multimodal Interaction in VRProceedings of the ACM on Human-Computer Interaction10.1145/36586458:EICS(1-24)Online publication date: 17-Jun-2024
        • (2024)Making maps & visualizations for mobile devices: A research agenda for mobile-first and responsive cartographic designJournal of Location Based Services10.1080/17489725.2023.2251423(1-71)Online publication date: 3-Apr-2024
        • (2024)Gaze-based detection of mind wandering during audio-guided panorama viewingScientific Reports10.1038/s41598-024-79172-x14:1Online publication date: 14-Nov-2024
        • (2024)The Role of Audio in Immersive Storytelling: a Systematic Review in Cultural HeritageMultimedia Tools and Applications10.1007/s11042-024-19288-4Online publication date: 26-Jun-2024
        • (2024)KD-Eye: Lightweight Pupil Segmentation for Eye Tracking on VR Headsets via Knowledge DistillationWireless Artificial Intelligent Computing Systems and Applications10.1007/978-3-031-71464-1_18(209-220)Online publication date: 13-Nov-2024
        • (2023)On the Effect of User Tracking on Perceived Source Positions in Mobile Audio Augmented RealityProceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter10.1145/3605390.3605422(1-9)Online publication date: 20-Sep-2023
        • (2023)Gazealytics: A Unified and Flexible Visual Toolkit for Exploratory and Comparative Gaze AnalysisProceedings of the 2023 Symposium on Eye Tracking Research and Applications10.1145/3588015.3589844(1-7)Online publication date: 30-May-2023
        • (2023)iBall: Augmenting Basketball Videos with Gaze-moderated Embedded VisualizationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581266(1-18)Online publication date: 19-Apr-2023
        • (2023)Potential of eye-tracking for interactive geovisual exploration aided by machine learningInternational Journal of Cartography10.1080/23729333.2022.21503799:2(150-172)Online publication date: 10-Jan-2023
        • (2023)Unobtrusive interaction: a systematic literature review and expert surveyHuman–Computer Interaction10.1080/07370024.2022.216240439:5-6(380-416)Online publication date: Feb-2023
        • Show More Cited By

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media