[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3411764.3445339acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation

Published: 07 May 2021 Publication History

Abstract

Explicitly alerting users is not always an optimal intervention, especially when they are not motivated to obey. For example, in video-based learning, learners who are distracted from the video would not follow an alert asking them to pay attention. Inspired by the concept of Mindless Computing, we propose a novel intervention approach, Mindless Attractor, that leverages the nature of human speech communication to help learners refocus their attention without relying on their motivation. Specifically, it perturbs the voice in the video to direct their attention without consuming their conscious awareness. Our experiments not only confirmed the validity of the proposed approach but also emphasized its advantages in combination with a machine learning-based sensing module. Namely, it would not frustrate users even though the intervention is activated by false-positive detection of their attentive state. Our intervention approach can be a reliable way to induce behavioral change in human–AI symbiosis.

Supplementary Material

VTT File (3411764.3445339_videofigurecaptions.vtt)
VTT File (3411764.3445339_videopreviewcaptions.vtt)
MP4 File (3411764.3445339_videofigure.mp4)
Supplemental video
MP4 File (3411764.3445339_videopreview.mp4)
Preview video

References

[1]
Alexander Travis Adams, Jean Marcel dos Reis Costa, Malte F. Jung, and Tanzeem Choudhury. 2015. Mindless computing: designing technologies to subtly influence behavior. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, New York, NY, 719–730. https://doi.org/10.1145/2750858.2805843
[2]
Saleema Amershi, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, Eric Horvitz, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, and Paul N. Bennett. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 37th ACM CHI Conference on Human Factors in Computing Systems, Vol. 3. ACM, New York, NY, 1–13. https://doi.org/10.1145/3290605.3300233
[3]
Riku Arakawa, Shinnnosuke Takamichi, and Hiroshi Saruwatari. 2019. Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device. In Proceedings of the 10th ISCA Speech Synthesis Workshop. ISCA, Baixas, France, 93–98. https://doi.org/10.21437/SSW.2019-17
[4]
Ryan Shaun J. D. Baker, Sidney K. D’Mello, Ma. Mercedes T. Rodrigo, and Arthur C. Graesser. 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 4 (2010), 223–241. https://doi.org/10.1016/j.ijhcs.2009.12.003
[5]
Meltem Huri Baturay. 2015. An Overview of the World of MOOCs. Procedia – Social and Behavioral Sciences 174 (2015), 427–433. https://doi.org/10.1016/j.sbspro.2015.01.685
[6]
Maude Beauchemin, Louis De Beaumont, Phetsamone Vannasing, Aline Turcotte, Claudine Arcand, Pascal Belin, and Maryse Lassonde. 2006. Electrophysiological markers of voice familiarity. The European Journal of Neuroscience 23, 11 (2006), 3081–—3086. https://doi.org/10.1111/j.1460-9568.2006.04856.x
[7]
Jonathan Bidwell and Henry Fuchs. 2011. Classroom analytics: Measuring student engagement with automated gaze tracking. Technical Report. Department of Computer, University of North Carolina at Chapel Hill, Chapel Hill, NC. 1–17pages. https://doi.org/10.13140/RG.2.1.4865.6242
[8]
James C. Byers, Alvah C. Bittner, and Susan G. Hill. 1989. Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary?. In Proceedings of the 1989 Annual International Industrial Ergonomics and Safety Conference. Taylor & Francis, Philadelphia, PA, 481–485.
[9]
Vanessa Cobus, Hannah Meyer, Swamy Ananthanarayan, Susanne Boll, and Wilko Heuten. 2018. Towards reducing alarm fatigue: peripheral light pattern design for critical care alarms. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction. ACM, New York, NY, 654–663. https://doi.org/10.1145/3240167.3240218
[10]
Tatiana Conde, Óscar F. Gonçalves, and Ana P. Pinheiro. 2015. Paying attention to my voice or yours: An ERP study with words. Biological Psychology 111 (2015), 40–52. https://doi.org/10.1016/j.biopsycho.2015.07.014
[11]
Crispin Coombs. 2020. Will COVID-19 be the tipping point for the Intelligent Automation of work? A review of the debate and implications for research. International Journal of Information Management 102182 (2020), 1–4. https://doi.org/10.1016/j.ijinfomgt.2020.102182
[12]
Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13, 3 (1989), 319–340. https://doi.org/10.2307/249008
[13]
Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020. A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores. In Proceedings of the 38th ACM CHI Conference on Human Factors in Computing Systems, Vol. 509. ACM, New York, NY, 1–12. https://doi.org/10.1145/3313831.3376638
[14]
Jiankang Deng, Jia Guo, Evangelos Ververas, Irene Kotsia, and Stefanos Zafeiriou. 2020. RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild. In Proceeedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, New York, NY, 5202–5211. https://doi.org/10.1109/CVPR42600.2020.00525
[15]
Berkeley J. Dietvorst, Joseph P. Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General 144, 1 (2015), 114–126. https://doi.org/10.1037/xge0000033
[16]
Sidney D’Mello, Andrew Olney, Claire Williams, and Patrick Hays. 2012. Gaze tutor: A gaze-reactive intelligent tutoring system. International Journal of Human-Computer Studies 70, 5 (2012), 377–398. https://doi.org/10.1016/j.ijhcs.2012.01.004
[17]
Haoshu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu. 2017. RMPE: Regional Multi-person Pose Estimation. In Proceedings of the 2017 IEEE International Conference on Computer Vision. IEEE, New York, NY, 2353–2362. https://doi.org/10.1109/ICCV.2017.256
[18]
Robert S. Fish, Robert E. Kraut, Robert W. Root, and Ronald E. Rice. 1992. Evaluating Video as a Technology for Informal Communication. In Procedings of the 10th ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 37–48. https://doi.org/10.1145/142750.142755
[19]
Reza Ghoddoosian, Marnim Galib, and Vassilis Athitsos. 2019. A Realistic Dataset and Baseline Temporal Model for Early Drowsiness Detection. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, New York, NY, 178–187. https://doi.org/10.1109/cvprw.2019.00027
[20]
Karen Goldschmidt. 2020. The COVID-19 Pandemic: Technology use to Support the Wellbeing of Children. Journal of pediatric nursing 53 (2020), 88–90. https://doi.org/10.1016/j.pedn.2020.04.013
[21]
Linsong Guo, Qin Zhang, and Shuxia Han. 2002. Agricultural Machinery Safety Alert System Using Ultrasonic Sensors. Journal of Agricultural Safety and Health 8, 4 (2002), 385–396. https://doi.org/10.13031/2013.10219
[22]
Philip J. Guo, Juho Kim, and Rob Rubin. 2014. How video production affects student engagement: an empirical study of MOOC videos. In Proceedings of the 1st ACM Conference on Learning @ Scale. ACM, New York, NY, 41–50. https://doi.org/10.1145/2556325.2566239
[23]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52 (1988), 139–183. https://doi.org/10.1016/s0166-4115(08)62386-9
[24]
Stephen Hutt, Caitlin Mills, Nigel Bosch, Kristina Krasich, James R. Brockmole, and Sidney K. D’Mello. 2017. “Out of the Fr-Eye-ing Pan”: Towards Gaze-Based Models of Attention during Learning with Technology in the Classroom. In Proceedings of the 25th ACM International Conference on User Modeling, Adaptation and Personalization. ACM, New York, NY, 94–103. https://doi.org/10.1145/3079628.3079669
[25]
Toastmasters International. 2011. Your Speaking Voice: Tips for Adding Strength and Authority to Your Speaking Voice. https://www.toastmasters.org/resources/your-speaking-voice. Accessed: August 19, 2019.
[26]
Seita Kayukawa, Keita Higuchi, João Guerreiro, Shigeo Morishima, Yoichi Sato, Kris Kitani, and Chieko Asakawa. 2019. BBeep: A Sonic Collision Avoidance System for Blind Travellers and Nearby Pedestrians. In Proceedings of the 37th ACM CHI Conference on Human Factors in Computing Systems, Vol. 52. ACM, New York, NY, 1–12. https://doi.org/10.1145/3290605.3300282
[27]
Michael Kerres. 2020. Against All Odds: Education in Germany Coping with COVID-19. Postdigital Science and Education 2, 3 (2020), 690–694. https://doi.org/10.1007/s42438-020-00130-7
[28]
Mitsuru Kodama. 2020. Digitally transforming work styles in an era of infectious disease. International Journal of Information Management 102172 (2020), 1–6. https://doi.org/10.1016/j.ijinfomgt.2020.102172
[29]
Anastasia Kuzminykh and Sean Rintel. 2020. Classification of Functional Attention in Video Meetings. In Proceedings of the 38th ACM CHI Conference on Human Factors in Computing Systems, Vol. 419. ACM, New York, NY, 1–13. https://doi.org/10.1145/3313831.3376546
[30]
Anastasia Kuzminykh and Sean Rintel. 2020. Low Engagement As a Deliberate Practice of Remote Participants in Video Meetings. In Extended Abstracts of the 38th ACM CHI Conference on Human Factors in Computing Systems, Vol. 321. ACM, New York, NY, 1–9. https://doi.org/10.1145/3334480.3383080
[31]
Seungyon Claire Lee and Thad Starner. 2010. BuzzWear: alert perception in wearable tactile displays on the wrist. In Proceedings of the 28th ACM CHI International Conference on Human Factors in Computing Systems. ACM, New York, NY, 433–442. https://doi.org/10.1145/1753326.1753392
[32]
Fred G. Martin. 2012. Will massive open online courses change how we teach?Commununications of the ACM 55, 8 (2012), 26–28. https://doi.org/10.1145/2240236.2240246
[33]
Stephen C. Nettelhorst and Laura A. Brannon. 2012. The effect of advertisement choice, sex, and need for cognition on attention. Computers in Human Behavior 28, 4 (2012), 1315–1320. https://doi.org/10.1016/j.chb.2012.02.015
[34]
Rachel S. Oeppen, Graham Shaw, and Peter A. Brennan. 2020. Human factors recognition at virtual meetings and video conferencing: how to get the best performance from yourself and others. British Journal of Oral and Maxillofacial Surgery 58, 6 (2020), 643–646. https://doi.org/10.1016/j.bjoms.2020.04.046
[35]
Fernando Poyatos. 1993. Paralanguage: A Linguistic and Interdisciplinary Approach to Interactive Speech and Sounds. John Benjamins Publishing Company, Amsterdam, Netherlands.
[36]
Julie Saint-Lot, Jean-Paul Imbert, and Frédéric Dehais. 2020. Red Alert: A Cognitive Countermeasure to Mitigate Attentional Tunneling. In Proceedings of the 38th ACM CHI Conference on Human Factors in Computing Systems, Vol. 580. ACM, New York, NY, 1–6. https://doi.org/10.1145/3313831.3376709
[37]
Samiha Samrose, Ru Zhao, Jeffery White, Vivian Li, Luis Nova, Yichen Lu, Mohammad Rafayet Ali, and Mohammed E. Hoque. 2017. CoCo: Collaboration Coach for Understanding Team Dynamics during Video Conferencing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2017), 160:1–160:24. https://doi.org/10.1145/3161186
[38]
Gianluca Schiavo, Alessandro Cappelletti, Eleonora Mencarini, Oliviero Stock, and Massimo Zancanaro. 2014. Overt or subtlefi: supporting group conversations with automatically targeted directives. In Proceedings of the 19th ACM International Conference on Intelligent User Interfaces. ACM, New York, NY, 225–234. https://doi.org/10.1145/2557500.2557507
[39]
Kshitij Sharma, Hamed S. Alavi, Patrick Jermann, and Pierre Dillenbourg. 2016. A gaze-based learning analytics model: in-video visual feedback to improve learner’s attention in MOOCs. In Proceedings of the 6th ACM International Conference on Learning Analytics & Knowledge. ACM, New York, NY, 417–421. https://doi.org/10.1145/2883851.2883902
[40]
Cary Stothart, Ainsley Mitchum, and Courtney Yehnert. 2015. The attentional cost of receiving a cell phone notification.Journal of Experimental Psychology: Human Perception and Performance 41, 4(2015), 893–897. https://doi.org/10.1037/xhp0000100
[41]
Joseph W. Sullivan and Frances Degen Horowitz. 1983. The effects of intonation on infant attention: the role of the rising intonation contour. Journal of Child Language 10, 3 (1983), 521–534. https://doi.org/10.1017/s0305000900005341
[42]
Chinchu Thomas and Dinesh Babu Jayagopi. 2017. Predicting student engagement in classrooms using facial behavioral cues. In Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education. ACM, New York, NY, 33–40. https://doi.org/10.1145/3139513.3139514
[43]
Tomoki Toda, Takashi Muramatsu, and Hideki Banno. 2012. Implementation of Computationally Efficient Real-Time Voice Conversion. In Proceedings of the 13th Annual Conference of the International Speech Communication Association. ISCA, Baixas, France, 94–97.
[44]
George L. Trager. 1958. Paralanguage: A first approximation. Studies in Linguistics 13 (1958), 1–12.
[45]
Narayanan Veliyath, Pradipta De, Andrew A. Allen, Charles B. Hodges, and Aniruddha Mitra. 2019. Modeling Students’ Attention in the Classroom using Eyetrackers. In Proceedings of the 2019 ACM Southeast Regional Conference. ACM, New York, NY, 2–9. https://doi.org/10.1145/3299815.3314424
[46]
Viswanath Venkatesh, Michael G. Morris, Gordon B. Davis, and Fred D. Davis. 2003. User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly 27, 3 (2003), 425–478. https://doi.org/10.2307/30036540
[47]
Brian Wansink and Koert van Ittersum. 2007. Portion Size Me: Downsizing Our Consumption Norms. Journal of the American Dietetic Association 107, 7(2007), 1103–1106. https://doi.org/10.1016/j.jada.2007.05.019
[48]
Mark Weiser and John Seely Brown. 1995. Designing calm technology. Technical Report. Xerox PARC, Palo Alto, CA. 1–5 pages.
[49]
Mark Weiser and John Seely Brown. 1997. The coming age of calm technology. In Beyond calculation. Springer, New York, NY, 75–85. https://doi.org/10.1007/978-1-4612-0685-9_6
[50]
Steve Whittaker. 1995. Rethinking video as a technology for interpersonal communications: theory and design implications. International Journal of Human-Computer Studies 42, 5 (1995), 501–529. https://doi.org/10.1006/ijhc.1995.1022
[51]
Xiang Xiao and Jingtao Wang. 2016. Context and cognitive state triggered interventions for mobile MOOC learning. In Proceedings of the 18th ACM International Conference on Multimodal Interaction6. ACM, New York, NY, 378–385. https://doi.org/10.1145/2993148.2993177
[52]
Yi Xu. 2005. Speech melody as articulatorily implemented communicative functions. Speech Communication 46, 3-4 (2005), 220–251. https://doi.org/10.1016/j.specom.2005.02.014
[53]
Kun Yu, Shlomo Berkovsky, Ronnie Taib, Dan Conway, Jianlong Zhou, and Fang Chen. 2017. User Trust Dynamics: An Investigation Driven by Differences in System Performance. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. ACM, New York, NY, 307–317. https://doi.org/10.1145/3025171.3025219
[54]
Janez Zaletelj and Andrej Kosir. 2017. Predicting students’ attention in the classroom from Kinect facial and body features. EURASIP Journal on Image and Video Processing 2017 (2017), 80. https://doi.org/10.1186/s13640-017-0228-8
[55]
Robert J. Zatorre and Jackson T. Gandour. 2007. Neural specializations for speech and pitch: moving beyond the dichotomies. Philosophical Transactions of the Royal Society B: Biological Sciences 363, 1493 (2007), 1087–1104. https://doi.org/10.1098/rstb.2007.2161
[56]
Jianlong Zhou, Syed Z. Arshad, Simon Luo, and Fang Chen. 2017. Effects of Uncertainty and Cognitive Load on User Trust in Predictive Decision Making. In Proceeding of the 16th IFIP TC 13 International Conference on Human-Computer Interaction. Springer, Cham, Switzerland, 23–39. https://doi.org/10.1007/978-3-319-68059-0_2

Cited By

View all
  • (2024)AudioMove: Applying the Spatial Audio to Multi-Directional Limb Exercise GuidanceProceedings of the ACM on Human-Computer Interaction10.1145/36764898:MHCI(1-26)Online publication date: 24-Sep-2024
  • (2024)PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a SmartwatchProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676350(1-16)Online publication date: 13-Oct-2024
  • (2024)Investigation of Unconscious Gaze and Head Direction in Videoconference - Turning off Microphone/Camera by Unconscious Gaze and Head Direction -Proceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655645(1-3)Online publication date: 4-Jun-2024
  • Show More Cited By

Index Terms

  1. Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Please enable JavaScript to view thecomments powered by Disqus.

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
          May 2021
          10862 pages
          ISBN:9781450380966
          DOI:10.1145/3411764
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 07 May 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Badges

          • Honorable Mention

          Author Tags

          1. Computational intervention
          2. Human attention
          3. Machine learning-based sensing
          4. Mindless computing
          5. Video-based learning

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          Conference

          CHI '21
          Sponsor:

          Acceptance Rates

          Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

          Upcoming Conference

          CHI 2025
          ACM CHI Conference on Human Factors in Computing Systems
          April 26 - May 1, 2025
          Yokohama , Japan

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)443
          • Downloads (Last 6 weeks)62
          Reflects downloads up to 02 Mar 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)AudioMove: Applying the Spatial Audio to Multi-Directional Limb Exercise GuidanceProceedings of the ACM on Human-Computer Interaction10.1145/36764898:MHCI(1-26)Online publication date: 24-Sep-2024
          • (2024)PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a SmartwatchProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676350(1-16)Online publication date: 13-Oct-2024
          • (2024)Investigation of Unconscious Gaze and Head Direction in Videoconference - Turning off Microphone/Camera by Unconscious Gaze and Head Direction -Proceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655645(1-3)Online publication date: 4-Jun-2024
          • (2024)Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-HearingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642287(1-18)Online publication date: 11-May-2024
          • (2023)Brain-Computer Integration: A Framework for the Design of Brain-Computer Interfaces from an Integrations PerspectiveACM Transactions on Computer-Human Interaction10.1145/360362130:6(1-48)Online publication date: 25-Sep-2023
          • (2023)A Generative Framework for Designing Interactions to Overcome the Gaps between Humans and Imperfect AIs Instead of Improving the Accuracy of the AIsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3577036(1-5)Online publication date: 19-Apr-2023
          • (2023)CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative ModelsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581133(1-19)Online publication date: 19-Apr-2023
          • (2022)Unconscious Other’s Impression Changer: A Method to Manipulate Cognitive Biases That Subtly Change Others’ Impressions Positively/Negatively by Making AI Bias in Emotion Estimation AISensors10.3390/s2224996122:24(9961)Online publication date: 17-Dec-2022
          • (2022)Exploring Tactile Stimuli from a Wrist-Worn Device to Manipulate Subjective Time Based on the Filled-Duration IllusionSensors10.3390/s2219719422:19(7194)Online publication date: 22-Sep-2022
          • (2022)Mindless Memorization Booster: A Method to Influence Memorization Power Using Attention Induction Phenomena Caused by Visual Interface Modulation and Its Application to Memorization Support for English Vocabulary LearningElectronics10.3390/electronics1114227611:14(2276)Online publication date: 21-Jul-2022
          • Show More Cited By

          View Options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Login options

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media