Abstract
Reliable applications of affective brain-computer interfaces (aBCI) in realistic, multi-modal environments require a detailed understanding of the processes involved in emotions. To explore the modality-specific nature of affective responses, we studied neurophysiological responses (i.e., EEG) of 24 participants during visual, auditory, and audiovisual affect stimulation. The affect induction protocols were validated by participants’ subjective ratings and physiological responses (i.e., ECG). Coherent with literature, we found modality-specific responses in the EEG: posterior alpha power decreases during visual stimulation and increases during auditory stimulation, anterior alpha power tends to decrease during auditory stimulation and to increase during visual stimulation. We discuss the implications of these results for multi-modal aBCI.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aftanas, L.I., Reva, N.V., Varlamov, A.A., Pavlov, S.V., Makhnev, V.P.: Analysis of evoked eeg synchronization and desynchronization in conditions of emotional activation in humans: temporal and topographic characteristics. Neuroscience and Behavioral Physiology 34(8), 859–867 (2004)
Baumgartner, T., Esslen, M., Jancke, L.: From emotion perception to emotion experience: Emotions evoked by pictures and classical music. International Journal of Psychophysiology 60(1), 34–43 (2006)
Bradley, M.M., Lang, P.J.: Affective reactions to acoustic stimuli. Psychophysiology 37(2), 204–215 (2000)
Bradley, M.M., Lang, P.J.: The international affective digitized sounds (IADS-2): Affective ratings of sounds and instruction manual. Technical report, University of Florida, Center for Research in Psychophysiology, Gainesville, Fl, USA (2007)
Brefczynski-Lewis, J., Lowitszch, S., Parsons, M., Lemieux, S., Puce, A.: Audiovisual non-verbal dynamic faces elicit converging fMRI and ERP responses. Brain Topography 21(3-4), 193–206 (2009)
Codispoti, M., Ferrari, V., Bradley, M.M.: Repetitive picture processing: autonomic and cortical correlates. Brain Research 1068(1), 213–220 (2006)
Ethofer, T., Van De Ville, D., Scherer, K., Vuilleumier, P.: Decoding of emotional information in voice-sensitive cortices. Current Biology 19(12), 1028–1033 (2009)
Grandjean, D., Sander, D., Pourtois, G., Schwartz, S., Seghier, M.L., Scherer, K., Vuilleumier, P.: The voices of wrath: brain responses to angry prosody in meaningless speech. Nature Neuroscience 8(2), 145–146 (2005)
Guntekin, B., Basar, E.: Emotional face expressions are differentiated with brain oscillations. International Journal of Psychophysiology 64(1), 91–100 (2007)
Hadjikhani, N., de Gelder, B.: Seeing fearful body expressions activates the fusiform cortex and amygdala. Current Biology 13(24), 2201–2205 (2003)
Hajcak, G., MacNamara, A., Olvet, D.M.: Event-related potentials, emotion, and emotion regulation: An integrative review. Developmental Neuropsychology 35(2), 129–155 (2010)
Hari, R., Salmelin, R., Makela, J.P., Salenius, S., Helle, M.: MEG cortical rhythms. International Journal of Psychophysiology 26, 51–62 (1997)
Jensen, O., Mazaheri, A.: Shaping functional architecture by oscillatory alpha activity: gating by inhibition. Frontiers in Human Neuroscience 4 (2010)
Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(12), 2067–2083 (2008)
Kober, H., Feldman Barrett, L., Joseph, J., Bliss-Moreau, E., Lindquist, K., Wager, T.D.: Functional grouping and cortical-subcortical interactions in emotion: a meta-analysis of neuroimaging studies. NeuroImage 42(2), 998–1031 (2008)
Krause, C.M.: Cognition- and memory-related erd/ers responses in the auditory stimulus modality. Progress in brain research 159, 197–207 (2006)
Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Technical manual and affective ratings. Technical report, University of Florida, Center for Research in Psychophysiology, Gainesville, Fl, USA (1999)
Näätänena, R., Paavilainen, P., Rinne, T., Alho, K.: The mismatch negativity (MMN) in basic research of central auditory processing: A review. Clinical Neurophysiology 118(12), 2544–2590 (2007)
Panksepp, J., Bernatzky, G.: Emotional sounds and the brain: the neuro-affective foundations of musical appreciation. Behavioral Processes 60, 133–155 (2002)
Pfurtscheller, G., Lopes da Silva, F.H.: Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology 110(11), 1842–1857 (1999)
Picard, R.W.: Affective Computing. The MIT Press, Cambridge (1997)
Pourtois, G., Vuilleumier, P.: Dynamics of emotional effects on spatial attention in the human visual cortex. Progress in Brain Research, vol. 156, pp. 67–91. Elsevier, Amsterdam (2006)
Sabatinelli, D., Lang, P.J., Keil, A., Bradley, M.M.: Emotional perception: Correlation of functional MRI and event-related potentials. Cerebral Cortex 17(5), 1085–1091 (2007)
Sander, D., Grandjean, D., Scherer, K.R.: A systems approach to appraisal mechanisms in emotion. Neural Networks 18(4), 317–352 (2005)
Vuilleumier, P.: How brains beware: neural mechanisms of emotional attention. Trends in Cognitive Sciences 9(12), 585–594 (2005)
Wang, J., Nicol, T., Skoe, E., Sams, M., Kraus, N.: Emotion modulates early auditory response to speech. Journal of Cognitive Neuroscience 21(11), 2121–2128 (2009)
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1), 39–58 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mühl, C., van den Broek, E.L., Brouwer, AM., Nijboer, F., van Wouwe, N., Heylen, D. (2011). Multi-modal Affect Induction for Affective Brain-Computer Interfaces. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)