More Web Proxy on the site http://driver.im/

research-article

Public Access

Can Biosignals be Expressive?: How Visualizations Affect Impression Formation from Shared Brain Activity

Authors:

Geoff KaufmanAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 1, Issue CSCW

Article No.: 71, Pages 1 - 21

https://doi.org/10.1145/3134706

Published: 06 December 2017 Publication History

Abstract

We are exploring the concept of expressive biosignals: leveraging wearable technologies to introduce sensed physiological data as cues for social perception. Biosignals can help us achieve a deeper understanding of each other by revealing or clarifying the psychological processes that underlie our subjective experience. We conducted an exploratory study investigating expressive biosignals, comparing the influence of a variety of brain activity visualizations on impression formation. Results revealed that while participants readily infer emotional and cognitive states from visualized brain activity, the ambiguity of the data can lead to diverse perceptions and interpretations. Participants also expressed concerns that the observation of another individual's data during interaction might be invasive or distracting. We present a set of design considerations addressing issues of interpretability, integration, and privacy of biosignals in interpersonal contexts.

References

[1]

Maher Abujelala, Cheryl Abellanoza, Aayush Sharma, and Fillia Makedon.2016. Brain-ee: Brain enjoyment evaluation using commercial eeg headband Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 33.

Digital Library

[2]

Gordon W Allport.1933. Studies in Expressive Movement. The Macmillan Company.

[3]

Nalini Ambady and Robert Rosenthal.1992. Thin Slices of Expressive Behavior as Predictors of Interpersonal Consequences: A Meta-Analysis. Psychological Bulletin Vol. 111, 2 (1992), 256--274.

[4]

Yadid Ayzenberg, Javier Hernandez Rivera, and Rosalind Picard.2012. FEEL: frequent EDA and event logging--a mobile social interaction stress monitoring system Proceedings of the CHI 2012 Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2357--2362.

Digital Library

[5]

Fabio Babiloni and Laura Astolfi.2014. Social neuroscience and hyperscanning techniques: past, present and future. Neuroscience & Biobehavioral Reviews Vol. 44 (2014), 76--93.

[6]

Pouya Bashivan, Irina Rish, and Steve Heisig.2015. Mental State Recognition via Wearable EEG. (2015).

[7]

Chris Berka, Daniel J Levendowski, Milenko M Cvetinovic, Miroslav M Petrovic, Gene Davis, Michelle N Lumicao, Vladimir T Zivkovic, Miodrag V Popovic, and Richard Olmstead.2004. Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. International Journal of Human-Computer Interaction, Vol. 17, 2 (2004), 151--170.

[8]

Franco Curmi, Maria Angela Ferrario, Jen Southern, and Jon Whittle.2013. HeartLink: open broadcast of live biometric data to social networks Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1749--1758.

Digital Library

[9]

Bella M DePaulo.1992. Nonverbal behavior and self-presentation. Psychological Bulletin Vol. 111, 2 (1992), 203.

[10]

Empatica.2017. Monitor stress, seizures, activity, sleep. (2017). Retrieved April 24, 2017 from https://www.empatica.com/product-embrace.

[11]

Jérémy Frey.2016. Remote Heart Rate Sensing and Projection to Renew Traditional Board Games and Foster Social Interactions. In Proceedings of the CHI 2016 Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1865--1871.

Digital Library

[12]

Laurent George and Anatole Lécuyer.2010. An overview of research on "passive" brain-computer interfaces for implicit human-computer interaction. In International Conference on Applied Bionics and Biomechanics ICABB 2010-Workshop W1 "Brain-Computer Interfacing and Virtual Reality".

[13]

Robert Gifford.1994. A lens-mapping framework for understanding the encoding and decoding of interpersonal dispositions in nonverbal behavior. Journal of Personality and Social Psychology, Vol. 66, 2 (1994), 398.

[14]

Daniel T Gilbert, Brett W Pelham, and Douglas S Krull.1988. On cognitive busyness: When person perceivers meet persons perceived. Journal of Personality and Social Psychology, Vol. 54, 5 (1988), 733.

[15]

Erving Goffman.1959. The presentation of self in everyday life. Anchor, Garden City, NY.

[16]

Samuel D Gosling, Peter J Rentfrow, and William B Swann.2003. A very brief measure of the Big-Five personality domains. Journal of Research in Personality Vol. 37, 6 (2003), 504--528.

[17]

Mariam Hassib, Daniel Buschek, PawelW W Wozniak, and Florian Alt.2017 a. HeartChat: Heart Rate Augmented Mobile Chat to Support Empathy and Awareness Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2239--2251.

Digital Library

[18]

Mariam Hassib, Stefan Schneegass, Philipp Eiglsperger, Niels Henze, Albrecht Schmidt, and Florian Alt.2017 b. EngageMeter: A System for Implicit Audience Engagement Sensing Using Electroencephalography Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5114--5119.

Digital Library

[19]

Javier Hernandez, Ivan Riobo, Agata Rozga, Gregory D Abowd, and Rosalind W Picard.2014. Using electrodermal activity to recognize ease of engagement in children during social interactions. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 307--317.

Digital Library

[20]

Noura Howell, Laura Devendorf, Rundong Kevin Tian, Tomás Vega Galvez, Nan-Wei Gong, Ivan Poupyrev, Eric Paulos, and Kimiko Ryokai.2016. Biosignals as Social Cues: Ambiguity and Emotional Interpretation in Social Displays of Skin Conductance. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM, 865--870.

Digital Library

[21]

Interaxon.2015. Available Data - Muse Developers. (2015). Retrieved November 11, 2016 from http://developer.choosemuse.com/research-tools/available-data

[22]

Joris H Janssen, Jeremy N Bailenson, Wijnand A IJsselsteijn, and Joyce HDM Westerink.2010. Intimate heartbeats: Opportunities for affective communication technology. IEEE Transactions on Affective Computing Vol. 1, 2 (2010), 72--80.

Digital Library

[23]

Markus Koppensteiner and Karl Grammer.2010. Motion patterns in political speech and their influence on personality ratings. Journal of Research in Personality Vol. 44, 3 (2010), 374--379.

[24]

Megan N Kozak, Abigail A Marsh, and Daniel M Wegner.2006. What do I think you're doing? Action identification and mind attribution. Journal of Personality and Social Psychology, Vol. 90, 4 (2006), 543.

[25]

Sylvia D Kreibig.2010. Autonomic nervous system activity in emotion: A review. Biological Psychology Vol. 84, 3 (2010), 394--421.

[26]

Erik Andreas Larsen.2011. Classification of EEG signals in a brain-computer interface system. (2011).

[27]

Lucian Leahu and Phoebe Sengers.2014. Freaky: performing hybrid human-machine emotion. Proceedings of the 2014 ACM Conference on Designing Interactive Systems. ACM, 607--616.

Digital Library

[28]

Robert W Levenson and Anna M Ruef.1997. Physiological aspects of emotional knowledge and rapport. Empathic Accuracy, bibfieldeditorWilliam J Ickes (Ed.). Guilford Press, Chapter 2, 44--72.

[29]

Rashima Mahajan, Dipali Bansal, and Shweta Singh.2014. A Real Time Set Up for Retrieval of Emotional States from Human Neural Responses. International Journal of Medical, Health, Pharmaceutical and Biomedical Engineering, Vol. 8, 3 (2014), 142--147.

[30]

Banu Manav.2007. Color-emotion associations and color preferences: A case study for residences. Color Research & Application Vol. 32, 2 (2007), 144--150.

[31]

Jennifer Mankoff, Anind K Dey, Gary Hsieh, Julie Kientz, Scott Lederer, and Morgan Ames.2003. Heuristic evaluation of ambient displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 169--176.

Digital Library

[32]

Gerald Matthews and Manfred Amelang.1993. Extraversion, arousal theory and performance: A study of individual differences in the EEG. Personality and Individual Differences Vol. 14, 2 (1993), 347--363.

[33]

Nick Merrill and Coye Cheshire.2016. Habits of the Heart (rate): Social Interpretation of Biosignals in Two Interaction Contexts Proceedings of the 19th International Conference on Supporting Group Work. ACM, 31--38.

Digital Library

[34]

Nick Merrill and Coye Cheshire.2017. Trust Your Heart: Assessing Cooperation and Trust with Biosignals in Computer-Mediated Interactions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 2--12.

Digital Library

[35]

Crysta Metcalf, Gunnar Harboe, Joe Tullio, Noel Massey, Guy Romano, Elaine M Huang, and Frank Bentley.2008. Examining presence and lightweight messaging in a social television experience. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 4, 4 (2008), 27.

Digital Library

[36]

Hyeryung Christine Min and Tek-Jin Nam.2014. Biosignal sharing for affective connectedness. In Proceedings of the CHI 2014 Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2191--2196.

Digital Library

[37]

Helmuth Petsche, K Linder, Peter Rappelsberger, and Gerold Gruber.1988. The EEG: An adequate method to concretize brain processes elicited by music. Music Perception: An Interdisciplinary Journal, Vol. 6, 2 (1988), 133--159.

[38]

Ronald E Riggio and Howard S Friedman.1986. Impression formation: the role of expressive behavior. Journal of Personality and Social Psychology, Vol. 50, 2 (1986), 421.

[39]

Louis A Schmidt and Laurel J Trainor.2001. Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognition & Emotion, Vol. 15, 4 (2001), 487--500.

[40]

Holger Schn"adelbach, Stefan Rennick Egglestone, Stuart Reeves, Steve Benford, Brendan Walker, and Michael Wright.2008. Performing thrill: designing telemetry systems and spectator interfaces for amusement rides Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1167--1176.

Digital Library

[41]

Mark S Schwartz and Frank Andrasik.2015. Biofeedback: A Practitioner's Guide. Guilford Publications.

[42]

Petr Slovák, Joris Janssen, and Geraldine Fitzpatrick.2012. Understanding heart rate sharing: towards unpacking physiosocial space Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 859--868.

Digital Library

[43]

Jaime Snyder, Mark Matthews, Jacqueline Chien, Pamara F Chang, Emily Sun, Saeed Abdullah, and Geri Gay.2015. Moodlight: Exploring personal and social implications of ambient display of biosensor data. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 143--153.

Digital Library

[44]

A Strauss and J Corbin.1998. Basics of qualitative research techniques. (1998).

[45]

Chiew Seng Sean Tan, Johannes Schöning, Kris Luyten, and Karin Coninx.2014. Investigating the effects of using biofeedback as visual stress indicator during video-mediated collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 71--80.

Digital Library

[46]

Martin Tanis and Tom Postmes.2003. Social cues and impression formation in CMC. Journal of Communication Vol. 53, 4 (2003), 676--693.

[47]

Kristin Taraldsen, Sebastien FM Chastin, Ingrid I Riphagen, Beatrix Vereijken, and Jorunn L Helbostad.2012. Physical activity monitoring by use of accelerometer-based body-worn sensors in older adults: a systematic literature review of current knowledge and applications. Maturitas, Vol. 71, 1 (2012), 13--19.

[48]

Wouter Walmink, Danielle Wilde, and Florian'Floyd' Mueller.2014. Displaying heart rate data on a bicycle helmet to support social exertion experiences Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. ACM, 97--104.

Digital Library

[49]

Julia Werner, Reto Wettach, and Eva Hornecker.2008. United-pulse: feeling your partner's pulse. In Proceedings of the 10th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 535--538.

Digital Library

[50]

Amanda L Wheat and Kevin T Larkin.2010. Biofeedback of heart rate variability and related physiology: A critical review. Applied Psychophysiology and Biofeedback Vol. 35, 3 (2010), 229--242.

[51]

Michele A Williams, Asta Roseway, Chris O'Dowd, Mary Czerwinski, and Meredith Ringel Morris.2015. Swarm: an actuated wearable for mediating affect. Proceedings of the 9th International Conference on Tangible, Embedded, and Embodied Interaction. ACM, 293--300.

Digital Library

Cited By

Zechner OSchrom-Feiertag HWespi RPretolesi DNguyen QTscheligi M(2024)Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)JMIR XR and Spatial Computing10.2196/57655Online publication date: 27-Feb-2024
https://doi.org/10.2196/57655
Huang ZCao XZhang YMa X(2024)Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic ChillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642818(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642818
Sasikumar PHajika RGupta KGunasekaran TPai YBai HNanayakkara SBillinghurst M(2024)A User Study on Sharing Physiological Cues in VR Assembly Tasks2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00096(765-773)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00096
Show More Cited By

Index Terms

Can Biosignals be Expressive?: How Visualizations Affect Impression Formation from Shared Brain Activity
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
  2. Visualization
    1. Empirical studies in visualization

Recommendations

The Effect of Expressive Biosignals on Empathy and Closeness for a Stigmatized Group Member

We explore the potential for expressive biosignals, or displays of sensed physiological data as social cues, to influence perceptions of a member of a stigmatized group. In a between-subjects experiment (N=62) participants read a fictional interview with ...
Expressive Biosignals: Authentic Social Cues for Social Connection
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

My research introduces expressive biosignals as a novel social cue to improve interpersonal communication. Expressive biosignals are sensed physiological data revealed between people to provide a deeper understanding of each other's psychological ...
SynKin: A Game for Intentionally Synchronizing Biosignals
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Inter-individual synchronization of motor activity, such as gestures, posture and speech rhythm during face-to-face interaction is a well-established phenomenon. Recent investigations have revealed that similar synchronization during interaction also ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 1, Issue CSCW

November 2017

2095 pages

EISSN:2573-0142

DOI:10.1145/3171581

Editors:
Clifford Lampe
University of Michigan
,
Jeff Nichols
Google
,
Karrie Karahalios
University of Illinois Urbana-Champaign
,
Geraldine Fitzpatrick
Vienna University of Technology
,
Uichin Lee
KAIST
,
Andres Monroy-Hernandez
Snap Inc.
,
Wolfgang Stuerzlinger
Simon Fraser University

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2017

Published in PACMHCI Volume 1, Issue CSCW

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
893
Total Downloads

Downloads (Last 12 months)187
Downloads (Last 6 weeks)22

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zechner OSchrom-Feiertag HWespi RPretolesi DNguyen QTscheligi M(2024)Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)JMIR XR and Spatial Computing10.2196/57655Online publication date: 27-Feb-2024
https://doi.org/10.2196/57655
Huang ZCao XZhang YMa X(2024)Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic ChillsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642818(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642818
Sasikumar PHajika RGupta KGunasekaran TPai YBai HNanayakkara SBillinghurst M(2024)A User Study on Sharing Physiological Cues in VR Assembly Tasks2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00096(765-773)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00096
Sharma KGiannakos M(2023)Carry-Forward Effect: Early scaffolding learning processesProceedings of the 2023 Symposium on Learning, Design and Technology10.1145/3594781.3594786(43-52)Online publication date: 23-Jun-2023
https://dl.acm.org/doi/10.1145/3594781.3594786
Meijer WVerhoeff BVerma HBourgeois J(2023)Fast Drink: Mediating Empathy for Gig WorkersProceedings of the 2nd Empathy-Centric Design Workshop10.1145/3588967.3588975(1-6)Online publication date: 23-Apr-2023
https://dl.acm.org/doi/10.1145/3588967.3588975
Di Lodovico CColombo SRapp A(2023)Ambiguity for Social Self-tracking Practices: Exploring an Emerging Design SpaceCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3606989(144-148)Online publication date: 14-Oct-2023
https://dl.acm.org/doi/10.1145/3584931.3606989
Xue MAn PLiang RGuo ZHu JHansen PFeijs L(2023)Co-constructing Stories Based on Users Lived Experiences to Investigate Visualization Design for Collective Stress ManagementProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596118(652-663)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596118
Chen SCheng HSitu JHuang Y(2023)Mirror Hearts: Exploring the (Mis-)Alignment between AI-Recognized and Self-Reported EmotionsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585607(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585607
El Ali AStepanova EPalande SMader ACesar PJansen K(2023)BreatheWithMe: Exploring Visual and Vibrotactile Displays for Social Breath Awareness during Colocated, Collaborative TasksExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585589(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585589
Jiang YDing XMa XSun ZGu N(2023)IntimaSea: Exploring Shared Stress Display in Close RelationshipsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581000(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581000
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents