More Web Proxy on the site http://driver.im/

poster

Inter-brain Synchrony and Eye Gaze Direction During Collaboration in VR

Authors:

Ihshan Gumilar,

Prasanth Sasikumar,

Mark Billinghurst,

Ashkan F. Hayati,

Abdul MominAuthors Info & Claims

CHI EA '22: CHI Conference on Human Factors in Computing Systems Extended Abstracts

Article No.: 345, Pages 1 - 7

https://doi.org/10.1145/3491101.3519746

Published: 28 April 2022 Publication History

Abstract

Brain activity sometimes synchronises when people collaborate together on real world tasks. Understanding this process could to lead to improvements in face to face and remote collaboration. In this paper we report on an experiment exploring the relationship between eye gaze and inter-brain synchrony in Virtual Reality (VR). The experiment recruited pairs who were asked to perform finger-tracking exercises in VR with three different gaze conditions: averted, direct, and natural, while their brain activity was recorded. We found that gaze direction has a significant effect on inter-brain synchrony during collaboration for this task in VR. This shows that representing natural gaze could influence inter-brain synchrony in VR, which may have implications for avatar design for social VR. We discuss implications of our research and possible directions for future work.

Supplementary Material

MP4 File (3491101.3519746-talk-video.mp4)

Talk Video

Download
19.94 MB

References

[1]

Anaël Ayrolles, Florence Brun, Phoebe Chen, Amir Djalovski, Yann Beauxis, Richard Delorme, Thomas Bourgeron, Suzanne Dikker, and Guillaume Dumas. 2021. HyPyP: a Hyperscanning Python Pipeline for inter-brain connectivity analysis. Social Cognitive and Affective Neuroscience 16, 1-2 (Jan. 2021), 72–83. https://doi.org/10.1093/scan/nsaa141

[2]

Fabio Babiloni and Laura Astolfi. 2014. Social neuroscience and hyperscanning techniques: Past, present and future. Neuroscience & Biobehavioral Reviews 44 (July 2014), 76–93. https://doi.org/10.1016/j.neubiorev.2012.07.006

[3]

Fabio Babiloni, Febo Cincotti, Donatella Mattia, Marco Mattiocco, Fabrizio De Vico Fallani, Andrea Tocci, Luigi Bianchi, Maria Grazia Marciani, and Laura Astolfi. 2006. Hypermethods for EEG hyperscanning. In 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. 3666–3669. https://doi.org/10.1109/IEMBS.2006.260754 ISSN: 1557-170X.

[4]

Adrian P. Burgess. 2013. On the interpretation of synchronization in EEG hyperscanning studies: a cautionary note. Frontiers in Human Neuroscience 7 (2013). https://doi.org/10.3389/fnhum.2013.00881

[5]

Andrea Cavallo, Ovidiu Lungu, Cristina Becchio, Caterina Ansuini, Aldo Rustichini, and Luciano Fadiga. 2015. When gaze opens the channel for communication: Integrative role of IFG and MPFC. NeuroImage 119 (Oct. 2015), 63–69. https://doi.org/10.1016/j.neuroimage.2015.06.025

[6]

Laurence Conty, Karim N’Diaye, Charles Tijus, and Nathalie George. 2007. When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing. Neuropsychologia 45, 13 (Jan. 2007), 3024–3037. https://doi.org/10.1016/j.neuropsychologia.2007.05.017

[7]

Artur Czeszumski, Sara Eustergerling, Anne Lang, David Menrath, Michael Gerstenberger, Susanne Schuberth, Felix Schreiber, Zadkiel Zuluaga Rendon, and Peter König. 2020. Hyperscanning: a valid method to study neural inter-brain underpinnings of social interaction. Frontiers in Human Neuroscience 14 (2020), 39.

[8]

G. Dumas, F. Lachat, J. Martinerie, J. Nadel, and N. George. 2011. From social behaviour to brain synchronization: Review and perspectives in hyperscanning. IRBM 32, 1 (Feb. 2011), 48–53. https://doi.org/10.1016/j.irbm.2011.01.002

[9]

Guillaume Dumas, Jacqueline Nadel, Robert Soussignan, Jacques Martinerie, and Line Garnero. 2010. Inter-Brain Synchronization during Social Interaction. PLoS ONE 5, 8 (Aug. 2010), e12166. https://doi.org/10.1371/journal.pone.0012166 cites: dumasInterBrainSynchronizationSocial2010.

[10]

Elaine Fox, Andrew Mathews, Andrew J Calder, and Jenny Yiend. 2007. Anxiety and sensitivity to gaze direction in emotionally expressive faces.Emotion 7, 3 (2007), 478.

[11]

John Foxe and Adam Snyder. 2011. The Role of Alpha-Band Brain Oscillations as a Sensory Suppression Mechanism during Selective Attention. Frontiers in Psychology 2 (2011), 154. https://doi.org/10.3389/fpsyg.2011.00154

[12]

Maia Garau, Mel Slater, Vinoba Vinayagamoorthy, Andrea Brogni, Anthony Steed, and M. Angela Sasse. 2003. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’03). Association for Computing Machinery, New York, NY, USA, 529–536. https://doi.org/10.1145/642611.642703

Digital Library

[13]

N. George and L. Conty. 2008. Facing the gaze of others. Neurophysiologie Clinique/Clinical Neurophysiology 38, 3 (June 2008), 197–207. https://doi.org/10.1016/j.neucli.2008.03.001

[14]

Alexandre Gramfort, Martin Luessi, Eric Larson, Denis A. Engemann, Daniel Strohmeier, Christian Brodbeck, Roman Goj, Mainak Jas, Teon Brooks, Lauri Parkkonen, and Matti Hämäläinen. 2013. MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience 0 (2013). https://doi.org/10.3389/fnins.2013.00267 Publisher: Frontiers.

[15]

Tobias Grossmann, Mark H. Johnson, Teresa Farroni, and Gergely Csibra. 2007. Social perception in the infant brain: gamma oscillatory activity in response to eye gaze. Social Cognitive and Affective Neuroscience 2, 4 (Dec. 2007), 284–291. https://doi.org/10.1093/scan/nsm025

[16]

Ihshan Gumilar, Ekansh Sareen, Reed Bell, Augustus Stone, Ashkan Hayati, Jingwen Mao, Amit Barde, Anubha Gupta, Arindam Dey, Gun Lee, and Mark Billinghurst. 2021. A comparative study on inter-brain synchrony in real and virtual environments using hyperscanning. Computers & Graphics 94 (Feb. 2021), 62–75. https://doi.org/10.1016/j.cag.2020.10.003

[17]

Jari K. Hietanen, Jukka M. Leppänen, Mikko J. Peltola, Kati Linna-aho, and Heidi J. Ruuhiala. 2008. Seeing direct and averted gaze activates the approach–avoidance motivational brain systems. Neuropsychologia 46, 9 (July 2008), 2423–2430. https://doi.org/10.1016/j.neuropsychologia.2008.02.029

[18]

Jonne O. Hietanen, Mikko J. Peltola, and Jari K. Hietanen. 2020. Psychophysiological responses to eye contact in a live interaction and in video call. Psychophysiology 57, 6 (2020), e13587. https://doi.org/10.1111/psyp.13587 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/psyp.13587.

[19]

Janet Hui-wen Hsiao and Garrison Cottrell. 2008. Two Fixations Suffice in Face Recognition. Psychological Science 19, 10 (Oct. 2008), 998–1006. https://doi.org/10.1111/j.1467-9280.2008.02191.x Publisher: SAGE Publications Inc.

[20]

S. Rao Jammalamadaka and Ambar Sengupta. 2001. Topics in Circular Statistics. World Scientific, Singapore.

[21]

Michelle Jarick and Alan Kingstone. 2015. The duality of gaze: eyes extract and signal social information during sustained cooperative and competitive dyadic gaze. Frontiers in Psychology 6 (2015), 1423. https://doi.org/10.3389/fpsyg.2015.01423

[22]

JASP Team. 2021. JASP (Version 0.16)[Computer software]. https://jasp-stats.org/

[23]

Ladislav Kesner, Dominika Grygarová, Iveta Fajnerová, Jiří Lukavskỳ, Tereza Nekovářová, Jaroslav Tintěra, Yuliya Zaytseva, and Jiří Horáček. 2018. Perception of direct vs. averted gaze in portrait paintings: An fMRI and eye-tracking study. Brain and cognition 125(2018), 88–99.

[24]

Takahiko Koike, Motofumi Sumiya, Eri Nakagawa, Shuntaro Okazaki, and Norihiro Sadato. 2019. What Makes Eye Contact Special? Neural Substrates of On-Line Mutual Eye-Gaze: A Hyperscanning fMRI Study. eNeuro 6, 1 (Feb. 2019), ENEURO.0284–18.2019. https://doi.org/10.1523/ENEURO.0284-18.2019

[25]

Takahiko Koike, Hiroki C. Tanabe, Shuntaro Okazaki, Eri Nakagawa, Akihiro T. Sasaki, Koji Shimada, Sho K. Sugawara, Haruka K. Takahashi, Kazufumi Yoshihara, Jorge Bosch-Bayard, and Norihiro Sadato. 2016. Neural substrates of shared attention as social memory: A hyperscanning functional magnetic resonance imaging study. NeuroImage 125 (Jan. 2016), 401–412. https://doi.org/10.1016/j.neuroimage.2015.09.076

[26]

Ryo Kuboshita, Takashi X. Fujisawa, Kai Makita, Ryoko Kasaba, Hidehiko Okazawa, and Akemi Tomoda. 2020. Intrinsic brain activity associated with eye gaze during mother–child interaction. Scientific Reports 10, 1 (Nov. 2020), 18903. https://doi.org/10.1038/s41598-020-76044-y Bandiera_abtest: a Cc_license_type: cc_by Cg_type: Nature Research Journals Number: 1 Primary_atype: Research Publisher: Nature Publishing Group Subject_term: Human behaviour;Psychology;Social behaviour Subject_term_id: human-behaviour;psychology;social-behaviour.

[27]

Fanny Lachat, Laurent Hugueville, Jean-Didier Lemaréchal, Laurence Conty, and Nathalie George. 2012. Oscillatory Brain Correlates of Live Joint Attention: A Dual-EEG Study. Frontiers in Human Neuroscience 6 (2012). https://doi.org/10.3389/fnhum.2012.00156

[28]

A. Louwerse, J. N. van der Geest, J. H. M. Tulen, J. van der Ende, A. R. Van Gool, F. C. Verhulst, and K. Greaves-Lord. 2013. Effects of eye gaze directions of facial images on looking behaviour and autonomic responses in adolescents with autism spectrum disorders. Research in Autism Spectrum Disorders 7, 9 (Sept. 2013), 1043–1053. https://doi.org/10.1016/j.rasd.2013.04.013

[29]

P Montague. 2002. Hyperscanning: Simultaneous fMRI during Linked Social Interactions. NeuroImage 16, 4 (Aug. 2002), 1159–1164. https://doi.org/10.1006/nimg.2002.1150

[30]

J. Panero and M. Zelnik. 1979. Human dimension & interior space: a source book of design reference standards. Watson-Guptill.

[31]

Daisuke N. Saito, Hiroki C. Tanabe, Keise Izuma, Masamichi J. Hayashi, Yusuke Morito, Hidetsugu Komeda, Hitoshi Uchiyama, Hirotaka Kosaka, Hidehiko Okazawa, Yasuhisa Fujibayashi, and Norihiro Sadato. 2010. “Stay Tuned”: Inter-Individual Neural Synchronization During Mutual Gaze and Joint Attention. Frontiers in Integrative Neuroscience 4 (2010). https://doi.org/10.3389/fnint.2010.00127

[32]

Nicolina Sciaraffa, Gianluca Borghini, Pietro Aricò, Gianluca Di Flumeri, Alfredo Colosimo, Anastasios Bezerianos, Nitish V Thakor, and Fabio Babiloni. 2017. Brain interaction during cooperation: Evaluating local properties of multiple-brain network. Brain sciences 7, 7 (2017), 90.

[33]

Atsushi Senju, Yoshikuni Tojo, Kiyoshi Yaguchi, and Toshikazu Hasegawa. 2005. Deviant gaze processing in children with autism: an ERP study. Neuropsychologia 43, 9 (2005), 1297–1306.

[34]

Caroline Szymanski, Ana Pesquita, Allison A. Brennan, Dionysios Perdikis, James T. Enns, Timothy R. Brick, Viktor Müller, and Ulman Lindenberger. 2017. Teams on the same wavelength perform better: Inter-brain phase synchronization constitutes a neural substrate for social facilitation. NeuroImage 152 (May 2017), 425–436. https://doi.org/10.1016/j.neuroimage.2017.03.013

[35]

Jlenia Toppi, Gianluca Borghini, Manuela Petti, Eric J. He, Vittorio De Giusti, Bin He, Laura Astolfi, and Fabio Babiloni. 2016. Investigating Cooperative Behavior in Ecological Settings: An EEG Hyperscanning Study. PLOS ONE 11, 4 (April 2016), e0154236. https://doi.org/10.1371/journal.pone.0154236

[36]

Jason Trees, Joseph Snider, Maryam Falahpour, Nick Guo, Kun Lu, Douglas C. Johnson, Howard Poizner, and Thomas T. Liu. 2014. Game controller modification for fMRI hyperscanning experiments in a cooperative virtual reality environment. MethodsX 1(2014), 292–299. https://doi.org/10.1016/j.mex.2014.10.009

[37]

Roberto Valenti, Nicu Sebe, and Theo Gevers. 2012. Combining Head Pose and Eye Location Information for Gaze Estimation. IEEE Transactions on Image Processing 21, 2 (Feb. 2012), 802–815. https://doi.org/10.1109/TIP.2011.2162740 Conference Name: IEEE Transactions on Image Processing.

Digital Library

[38]

David W. L. Wu, Walter F. Bischof, and Alan Kingstone. 2014. Natural gaze signaling in a social context. Evolution and Human Behavior 35, 3 (May 2014), 211–218. https://doi.org/10.1016/j.evolhumbehav.2014.01.005

[39]

Shan Xu, Shen Zhang, and Haiyan Geng. 2018. The Effect of Eye Contact Is Contingent on Visual Awareness. Frontiers in Psychology 9 (2018), 93. https://doi.org/10.3389/fpsyg.2018.00093

[40]

Ayumi Yoshioka, Hiroki C Tanabe, Motofumi Sumiya, Eri Nakagawa, Shuntaro Okazaki, Takahiko Koike, and Norihiro Sadato. 2021. Neural substrates of shared visual experiences: a hyperscanning fMRI study. Social Cognitive and Affective Neuroscience (June 2021), nsab082. https://doi.org/10.1093/scan/nsab082

[41]

Kyongsik Yun, Katsumi Watanabe, and Shinsuke Shimojo. 2012. Interpersonal body and neural synchronization as a marker of implicit social interaction. Scientific Reports 2, 1 (Dec. 2012), 959. https://doi.org/10.1038/srep00959

Cited By

Podkosova IBrument H(2024)Towards Full Body Co-Embodiment of Human and Non-Human Avatars in Virtual Reality2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00084(432-435)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00084
Chang EBillinghurst MYoo B(2023)Brain activity during cybersickness: a scoping reviewVirtual Reality10.1007/s10055-023-00795-y27:3(2073-2097)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1007/s10055-023-00795-y
Sasikumar PPai YBai HBillinghurst M(2022)PSCVR: Physiological Sensing in Collaborative Virtual Reality2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00137(663-666)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00137

Recommendations

Connecting the Brains via Virtual Eyes : Eye-Gaze Directions and Inter-brain Synchrony in VR
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Hyperscanning is an emerging method for measuring two or more brains simultaneously. This method allows researchers to simultaneously record neural activity from two or more people. While this method has been extensively implemented over the last five ...
A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Supporting natural communication cues is critical for people to work together remotely and face-to-face. In this paper we present a Mixed Reality (MR) remote collaboration system that enables a local worker to share a live 3D panorama of his/her ...
An assessment of eye-gaze potential within immersive virtual environments

In collaborative situations, eye gaze is a critical element of behavior which supports and fulfills many activities and roles. In current computer-supported collaboration systems, eye gaze is poorly supported. Even in a state-of-the-art video ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

3066 pages

ISBN:9781450391566

DOI:10.1145/3491101

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster
Research
Refereed limited

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
527
Total Downloads

Downloads (Last 12 months)150
Downloads (Last 6 weeks)20

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Podkosova IBrument H(2024)Towards Full Body Co-Embodiment of Human and Non-Human Avatars in Virtual Reality2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00084(432-435)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00084
Chang EBillinghurst MYoo B(2023)Brain activity during cybersickness: a scoping reviewVirtual Reality10.1007/s10055-023-00795-y27:3(2073-2097)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1007/s10055-023-00795-y
Sasikumar PPai YBai HBillinghurst M(2022)PSCVR: Physiological Sensing in Collaborative Virtual Reality2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00137(663-666)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00137

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents