Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips
<p>Participant-to-group physiological synchrony for each participant’s HR (<b>a</b>) and EDA (<b>b</b>). In each window, each marker refers to a participant. Filled markers depict inter-subject correlations exceeding chance-level correlations based on 500 trials of circular shuffle (depicted by the grey distributions); open markers depict inter-subject correlations not exceeding chance level. x depicts missing data.</p> "> Figure 2
<p>Percentage of participants with significant inter-subject correlations for HR (<b>top</b>; <b>a</b>,<b>b</b>) and EDA (<b>bottom</b>; <b>c</b>,<b>d</b>) as a function of stimulus duration (<b>left</b>; <b>a</b>,<b>c</b>) and participant group size (<b>right</b>; <b>b</b>,<b>d</b>), averaged over the six movie orders and 50 subsets of participant combinations. The color of the lines refers to the group size in the left plots and to the stimulus duration in the right plots, as do the numbers on the right side of some of the lines.</p> "> Figure 3
<p>Standard deviation across movies (<b>left a</b>,<b>c</b>) and subgroups (<b>right</b>; <b>b</b>,<b>d</b>) of the percentage of participants with significant inter-subject correlations for HR (<b>top</b>); (<b>a</b>,<b>b</b>) and EDA (<b>bottom</b>; <b>c</b>,<b>d</b>) as a function of stimulus duration (<b>left</b>; <b>a</b>,<b>c</b>) and participant group size (<b>right</b>; <b>b</b>,<b>d</b>). The color of the lines refers to the group size in the left plots and to the stimulus duration in the right plots.</p> "> Figure 4
<p>Fraction of participants with significant inter-subject correlations for HR (<b>a</b>) and EDA (<b>b</b>) as a function of the total minutes of data included in analysis, varied through varying stimulus duration and group size. For each datapoint, the stimulus duration is reflected by the marker size and the group size is reflected by the marker color.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. Participants
2.2. Materials
2.3. Design
2.4. Analysis
2.4.1. Pre-Processing
2.4.2. Inter-Subject Correlations
2.4.3. Circular Shuffle-Based Significance Test
2.4.4. Effect of Stimulus Duration and Group Size on Inter-Subject Correlation Significance
3. Results
3.1. Effect of Movie and Condition on Inter-Subject Correlation
3.2. Significance of Inter-Subject Correlations
3.3. Dependency of Significant Inter-Subject Correlation on Stimulus Duration and Group Size
3.4. Comparing Effects of Stimulus Duration and Group Size on Significant Inter-Subject Correlation
3.5. Correlation between Performance Measures and Inter-Subject Correlation
4. Discussion and Conclusion
4.1. Significance of Inter-Subject Correlations for Different Stimuli
4.2. Significance of Inter-Subject Correlations with Varying Amounts of Data Included
4.3. Inter-Subject Correlations as Measure of Attentional Engagement
4.4. Is Higher Inter-Subject Correlation Always Better?
4.5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Ki, J.J.; Kelly, S.P.; Parra, L.C. Attention Strongly Modulates Reliability of Neural Responses to Naturalistic Narrative Stimuli. J. Neurosci. 2016, 36, 3092–3101. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Stuldreher, I.V.; Thammasan, N.; Van Erp, J.B.F.; Brouwer, A.M. Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Reflects Shared Selective Auditory Attention. J. Neural Eng. 2020, 17, 046028. [Google Scholar] [CrossRef]
- Pérez, P.; Madsen, J.; Banellis, L.; Türker, B.; Raimondo, F.; Perlbarg, V.; Valente, M.; Niérat, M.C.; Puybasset, L.; Naccache, L.; et al. Conscious Processing of Narrative Stimuli Synchronizes Heart Rate between Individuals. Cell Rep. 2021, 36, 109692. [Google Scholar] [CrossRef] [PubMed]
- Dmochowski, J.P.; Sajda, P.; Dias, J.; Parra, L.C. Correlated Components of Ongoing EEG Point to Emotionally Laden Attention—A Possible Marker of Engagement? Front. Hum. Neurosci. 2012, 6, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Dmochowski, J.P.; Bezdek, M.A.; Abelson, B.P.; Johnson, J.S.; Schumacher, E.H.; Parra, L.C. Audience Preferences Are Predicted by Temporal Reliability of Neural Processing. Nat. Commun. 2014, 5, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Cohen, S.S.; Parra, L.C. Memorable Audiovisual Narratives Synchronize Sensory and Supramodal Neural Responses. eNeuro 2016, 3, 1–11. [Google Scholar] [CrossRef] [Green Version]
- Cohen, S.S.; Madsen, J.; Touchan, G.; Robles, D.; Lima, S.F.A.; Henin, S.; Parra, L.C. Neural Engagement with Online Educational Videos Predicts Learning Performance for Individual Students. Neurobiol. Learn. Mem. 2018, 155, 60–64. [Google Scholar] [CrossRef]
- Hasson, U.; Nir, Y.; Levy, I.; Fuhrmann, G.; Malach, R. Intersubject Synchronization of Cortical Activity during Natural Vision. Science 2004, 303, 1634–1640. [Google Scholar] [CrossRef]
- Liu, Y.; Ayaz, H.; Shewokis, P.A. Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, FNIRS, and Physiological Measures. Front. Hum. Neurosci. 2017, 11, 389. [Google Scholar] [CrossRef] [Green Version]
- Parra, L.C.; Haufe, S.; Dmochowski, J.P. Correlated Components Analysis—Extracting Reliable Dimensions in Multivariate Data. Neurons, Behav. Data Anal. Theory 2019, 2. [Google Scholar] [CrossRef]
- Poulsen, A.T.; Kamronn, S.; Dmochowski, J.; Parra, L.C.; Hansen, L.K. EEG in the Classroom: Synchronised Neural Recordings during Video Presentation. Sci. Rep. 2017, 7, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Stuldreher, I.V.; Thammasan, N.; van Erp, J.B.F.; Brouwer, A.M. Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Detects Attentionally Relevant Events in Time. Front. Neurosci. 2020, 14, 1–11. [Google Scholar] [CrossRef] [PubMed]
- Schmälzle, R.; Häcker, F.E.K.; Honey, C.J.; Hasson, U. Engaged Listeners: Shared Neural Processing of Powerful Political Speeches. Soc. Cogn. Affect. Neurosci. 2014, 10, 1137–1143. [Google Scholar] [CrossRef] [PubMed]
- Critchley, H.D. Electrodermal Responses: What Happens in the Brain. Neuroscientist 2002, 8, 132–142. [Google Scholar] [CrossRef]
- Bradley, M.M.; Lang, P.J. Affective Reactions to Acoustic Stimuli. Psychophysiology 2000, 37, 204–215. [Google Scholar] [CrossRef]
- Lang, P.J.; Bradley, M.M.; Cuthbert, B.N. Emotion, Motivation, and Anxiety: Brain Mechanisms and Psychophysiology. Biol. Psychiatry 1998, 44, 1248–1263. [Google Scholar] [CrossRef] [PubMed]
- Najafi, T.A.; Affanni, A.; Rinaldo, R.; Zontone, P. Driver Attention Assessment Using Physiological Measures from EEG, ECG, and EDA Signals †. Sensors 2023, 23, 2039. [Google Scholar] [CrossRef]
- Awais, M.; Badruddin, N.; Drieberg, M. A Hybrid Approach to Detect Driver Drowsiness Utilizing Physiological Signals to Improve System Performance and Wearability. Sensors 2017, 17, 1991. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Golland, Y.; Keissar, K.; Levit-Binnun, N. Studying the Dynamics of Autonomic Activity during Emotional Experience. Psychophysiology 2014, 51, 1101–1111. [Google Scholar] [CrossRef]
- Steiger, B.K.; Kegel, L.C.; Spirig, E.; Jokeit, H. Dynamics and Diversity of Heart Rate Responses to a Disaster Motion Picture. Int. J. Psychophysiol. 2019, 143, 64–79. [Google Scholar] [CrossRef]
- Madsen, J.; Parra, L.C. Cognitive Processing of a Common Stimulus Synchronizes Brains, Hearts, and Eyes. PNAS Nexus 2022, 1, 1–14. [Google Scholar] [CrossRef] [PubMed]
- Liu, Y.; Wang, T.; Wang, K.; Zhang, Y. Collaborative Learning Quality Classification Through Physiological Synchrony Recorded by Wearable Biosensors. Front. Psychol. 2021, 12, 674369. [Google Scholar] [CrossRef] [PubMed]
- Gashi, S.; Di Lascio, E.; Santini, S. Using Unobtrusive Wearable Sensors to Measure the Physiological Synchrony Between Presenters and Audience Members. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2019, 3, 1–19. [Google Scholar] [CrossRef]
- Brouwer, A.M.; Zander, T.O.; van Erp, J.B.F.; Korteling, J.E.; Bronkhorst, A.W. Using Neurophysiological Signals That Reflect Cognitive or Affective State: Six Recommendations to Avoid Common Pitfalls. Front. Neurosci. 2015, 9, 136. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Algumaei, M.; Hettiarachchi, I.; Veerabhadrappa, R.; Bhatti, A. Physiological Synchrony Predict Task Performance and Negative Emotional State during a Three-Member. Sensors 2023, 23, 2268. [Google Scholar] [CrossRef]
- Borovac, A.; Stuldreher, I.V.; Thammasan, N.; Brouwer, A.M. Validation of Wearables for Electrodermal Activity (EdaMove) and Heart Rate (Wahoo Tickr). In Proceedings of the 12th International Conference on Methods and Techniques in Behavioral Research and 6th Seminar on Behavioral Methods, Kraków, Poland, 13–15 October 2021. [Google Scholar]
- Van Beers, J.J.; Stuldreher, I.V.; Thammasan, N.; Brouwer, A.M. A Comparison between Laboratory and Wearable Sensors in the Context of Physiological Synchrony. In Proceedings of the 2020 International Conference on Multimodal Interaction, Utrecht, The Netherlands, 25–29 October 2020; pp. 604–608. [Google Scholar] [CrossRef]
- Thomann, J.; Baumann, C.R.; Landolt, H.; Werth, E. Psychomotor Vigilance Task Demonstrates Impaired Vigilance. J. Clincal Sleep Med. 2014, 10, 1019–1024. [Google Scholar] [CrossRef]
- Thammasan, N.; Stuldreher, I.V.; Schreuders, E.; Giletta, M.; Brouwer, A.M. A Usability Study of Physiological Measurement in School Using Wearable Sensors. Sensors 2020, 20, 5380. [Google Scholar] [CrossRef]
- Benedek, M.; Kaernbach, C. A Continuous Measure of Phasic Electrodermal Activity. J. Neurosci. Methods 2010, 190, 80–91. [Google Scholar] [CrossRef] [Green Version]
- Fit Sdk Release Notes. Available online: https://www.thisisant.com/developer/fit-sdk-release-notes (accessed on 16 October 2020).
- Cheung, M.N. Detection of and Recovery from Errors in Cardiac Interbeat Intervals. Psychophysiology 1981, 18, 341–346. [Google Scholar] [CrossRef]
- Salmi, J.; Roine, U.; Glerean, E.; Lahnakoski, J.; Nieminen-Von Wendt, T.; Tani, P.; Leppämäki, S.; Nummenmaa, L.; Jääskeläinen, I.P.; Carlson, S.; et al. The Brains of High Functioning Autistic Individuals Do Not Synchronize with Those of Others. NeuroImage Clin. 2013, 3, 489–497. [Google Scholar] [CrossRef]
- Hasson, U.; Avidan, G.; Gelbard, H.; Vallines, I.; Harel, M.; Minshew, N.; Behrmann, M. Shared and Idiosyncratic Cortical Activation Patterns in Autism Revealed under Continuous Real-Life Viewing Conditions. Autism Res. 2009, 2, 220–231. [Google Scholar] [CrossRef] [PubMed]
- Guo, C.C.; Nguyen, V.T.; Hyett, M.P.; Parker, G.B.; Breakspear, M.J. Out-of-Sync: Disrupted Neural Activity in Emotional Circuitry during Film Viewing in Melancholic Depression. Sci. Rep. 2015, 5, 11605. [Google Scholar] [CrossRef] [Green Version]
- Mäntylä, T.; Nummenmaa, L.; Rikandi, E.; Lindgren, M.; Kieseppä, T.; Hari, R.; Suvisaari, J.; Raij, T.T. Aberrant Cortical Integration in First-Episode Psychosis During Natural Audiovisual Processing. Biol. Psychiatry 2018, 84, 655–664. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hedge, C.; Powell, G.; Sumner, P. The Reliability Paradox: Why Robust Cognitive Tasks Do Not Produce Reliable Individual Differences. Behav. Res. Methods 2018, 50, 1166–1186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Finn, E.S.; Glerean, E.; Khojandi, A.Y.; Nielson, D.; Molfese, P.J.; Handwerker, D.A.; Bandettini, P.A. Idiosynchrony: From Shared Responses to Individual Differences during Naturalistic Neuroimaging. Neuroimage 2020, 215, 116828. [Google Scholar] [CrossRef]
Name | Duration | URL (Accessed on 30 November 2020) |
---|---|---|
Chauffeur | 09:45 | https://www.youtube.com/watch?v=jaFmvyH7dW8 |
El Mourabbi | 09:04 | https://www.youtube.com/watch?v=X9bJou2gKxo |
De Chinese Muur | 09:50 | https://www.youtube.com/watch?v=yjGFuhPy3Qo |
One of the Boys | 10:58 | https://www.youtube.com/watch?v=PsGAuhgQ97k |
Samual | 09:45 | https://www.youtube.com/watch?v=VUseoqCVnj4 |
Turn it Around | 09:26 | https://www.youtube.com/watch?v=beC7IpQpTz4 |
Movie | HR | EDA |
---|---|---|
Chauffeur | , | , |
El Mourabbi | , | , |
De Chinese Muur | , | , |
One of the Boys | , | , |
Samual | , | , |
Turn it Around | , | , |
Overall | , | , |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Stuldreher, I.V.; van Erp, J.B.F.; Brouwer, A.-M. Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips. Sensors 2023, 23, 3006. https://doi.org/10.3390/s23063006
Stuldreher IV, van Erp JBF, Brouwer A-M. Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips. Sensors. 2023; 23(6):3006. https://doi.org/10.3390/s23063006
Chicago/Turabian StyleStuldreher, Ivo V., Jan B. F. van Erp, and Anne-Marie Brouwer. 2023. "Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips" Sensors 23, no. 6: 3006. https://doi.org/10.3390/s23063006
APA StyleStuldreher, I. V., van Erp, J. B. F., & Brouwer, A.-M. (2023). Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips. Sensors, 23(6), 3006. https://doi.org/10.3390/s23063006