More Web Proxy on the site http://driver.im/

research-article

Emotion and attention: predicting electrodermal activity through video visual descriptors

Authors:

A. Hernández-García,

F. Fernández-Martínez,

F. Díaz-de-MaríaAuthors Info & Claims

WI '17: Proceedings of the International Conference on Web Intelligence

Pages 914 - 923

https://doi.org/10.1145/3106426.3109418

Published: 23 August 2017 Publication History

Abstract

This paper contributes to the field of affective video content analysis through the novel employment of electrodermal activity (EDA) measurements as ground truth for machine learning algorithms. The variation of the electrical properties of the skin, known as EDA, is a psychophysiological indicator widely used in medicine, psychology and neuroscience which can be considered a somatic marker of the emotional and attentional reaction of subjects towards stimuli. One of its main advantages is that the recorded information is not biased by the cognitive process of giving an opinion or a score to characterize the subjective perception. In this work, we predict the levels of emotion and attention, derived from EDA records, by means of a small set of low-level visual descriptors computed from the video stimuli. Linear regression experiments show that our descriptors predict significantly well the sum of emotion and attention levels, reaching a coefficient of determination R² = 0.25. This result sets a promising path for further research on the prediction of emotion and attention from videos using EDA.

References

[1]

Montserrat Aiger, María Palacín, and José-Manuel Cornejo. 2013. Electrodermal signal by Sociograph: methodology to measure the Group Activity. Revista de Psicología Social 28, 3 (2013), 333--347.

[2]

Masato Asahina, Atsuya Suzuki, Masahiro Mori, Toshihide Kanesaka, and Takamichi Hattori. 2003. Emotional sweating response in a patient with bilateral amygdala damage. International Journal of Psychophysiology 47, 1 (2003), 87 -- 93.

[3]

Martha Augoustinos, Iain Walker, and Ngaire Donaghue. 2014. Social cognition: An integrated introduction. Sage. 114--115 pages.

[4]

D. Ayata, Y. Yaslan, and M. Kamaşak. 2016. Emotion recognition via random forest and galvanic skin response: Comparison of time based feature sets, window sizes and wavelet approaches. (Oct 2016), 1--4.

[5]

David Bordwell, Kristin Thompson, and Jeremy Ashton. 1997. Film art: An introduction. Vol. 7. McGraw-Hill New York.

[6]

Wolfram Boucsein. 2012. Electrodermal activity. Springer Science & Business Media.

[7]

Steven J. Breckler. 1984. Empirical validation of affect, behavior, and cognition as distinct components of attitude. Journal of Personality and Social Psychology 47, 6 (December 1984), 1191--1205.

[8]

Christine Brinckmann. 2014. Color and empathy: essays on two aspects of film. Amsterdam University Press.

[9]

Roberto Cabeza and Lars Nyberg. 2000. Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies. J. Cognitive Neuroscience 12, 1 (Jan. 2000), 1--47.

Digital Library

[10]

Antonio R. Damasio. 1994. Descartes' error: emotion, reason, and the human brain. New York: Avon Books.

[11]

Charles Darwin. 1872. The expression of the emotions in man and animals. London, UK: John Marry (1872).

[12]

Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang. 2006. Studying Aesthetics in Photographic Images Using a Computational Approach. In Proceedings of the 9th European Conference on Computer Vision - Volume Part III (ECCV'06). Springer-Verlag, Berlin, Heidelberg, 288--301.

Digital Library

[13]

Michael E. Dawson and Keith H. Nuechterlein. 1984. Psychophysiological Dysfunctions in the Developmental Course of Schizophrenic Disorders. Schizophrenia Bulletin 10, 2 (1984), 204--232.

[14]

M. E. Dawson, A. M. Schell, and D. L. Filion. 2000. The electrodermal system. In Handbook of Psychophysiology, Second Edition, J. T. Cacioppo, L. G. Tassinary, and G. G. Berntson (Eds.). Cambridge University Press, Cambridge, 200--223.

[15]

Ch Fere. 1888. Note sur les modifications de la résistance électrique sous l'influence des excitations sensorielles et des émotions. CR Soc. Biol 5 (1888), 217--219.

[16]

F. Fernández-Martínez, A. Hernández-García, and F. Díaz-de-María. 2015. Succeeding metadata based annotation scheme and visual tips for the automatic assessment of video aesthetic quality in car commercials. Expert Systems with Applications (2015), 293--305.

Digital Library

[17]

F. Fernández-Martínez, A. Hernández-García, A. Gallardo-Antolín, and F. Díaz de María. 2014. Combining audio-visual features for viewers' perception classification of Youtube car commercials. In Proceedinggs of Workshop on Speech, Language and Audio in Multimedia (SLAM).

[18]

J. Fleureau, P. Guillotel, and I. Orlac. 2013. Affective Benchmarking of Movies Based on the Physiological Responses of a Real Audience. In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on. 73--78.

Digital Library

[19]

Don C. Fowles, Margaret J. Christie, Robert Edelberg, William W. Grings, David T. Lykken, and Peter H. Venables. 1981. Publication Recommendations for Electrodermal Measurements. Psychophysiology 18, 3 (1981), 232--239.

[20]

Maria Gendron and Lisa Feldman Barrett. 2009. Reconstructing the past: A century of ideas about emotion in psychology. Emotion review 1, 4 (2009), 316--339.

[21]

A Hernández-García, F Fernández-Martínez, and F Díaz-de María. 2016. Comparing visual descriptors and automatic rating strategies for video aesthetics prediction. Signal Processing: Image Communication 47 (2016), 280--288.

Digital Library

[22]

Ernest R. Hilgard. 1980. The trilogy of mind: Cognition, affection, and conation. Journal of the History of the Behavioral Sciences 16, 2 (1980), 107--117. <107::AID-JHBS2300160202>3.0.CO;2-Y

[23]

William James. 1884. What is an emotion? Mind 34 (1884), 188--205.

[24]

William James. 1890. The principles of psychology. New York: Dover.

[25]

Sander Koelstra, Christian Muhl, Mohammad Soleymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, and Ioannis Patras. 2012. DEAP: A Database for Emotion Analysis ;Using Physiological Signals. IEEE Trans. Affect. Comput. 3, 1 (Jan. 2012), 18--31.

Digital Library

[26]

Mathieu Lajante, Olivier Droulers, Thibaut Dondaine, and David Amarantini. 2012. Opening the "black box" of electrodermal activity in consumer neuroscience research. Journal of Neuroscience, Psychology, and Economics 5, 4 (2012), 238.

[27]

Peter J Lang, Mark K Greenwald, Margaret M Bradley, and Alfons O Hamm. 1993. Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology 30, 3 (1993), 261--273.

[28]

Joseph LeDoux. 2012. Rethinking the emotional brain. Neuron 73, 4 (2012), 653--676.

[29]

J. L. Martínez and E. Garrido. 2003. Sistema para la medición de reacciones emocionales en grupos sociales. 2 168 928. Patente de invención A6113 5116, 2003-10-1. (2003).

[30]

J. L. Martínez, Sergio Monge, and M. Isabel Valdunquillo. 2012. Medición de las respuestas psicofisiológicas grupales para apoyar el análisis de discursos políticos. Trípodos 29 (2012), 53--72.

[31]

Anush K. Moorthy, Pere Obrador, and Nuria Oliver. 2010. Towards Computational Models of the Visual Aesthetic Appeal of Consumer Videos. In Proceedings of the 11th European Conference on Computer Vision: Part V (ECCV'10). Springer-Verlag, Berlin, Heidelberg, 1--14. http://dl.acm.org/citation.cfm?id=1888150.1888152

Digital Library

[32]

Arne Öhman. 1999. Distinguishing Unconscious from Conscious Emotional Processes: Methodological Considerations and Theoretical Implications. In Handbook of Cognition and Emotion. John Wiley & Sons, Ltd, 321--352.

[33]

International Commission on Illumination. 2004. Colorimetry: technical report. Commission internationale de l'Eclairage, CIE Central Bureau. http://books.google.es/books?id=P1NkAAAACAAJ

[34]

Michael Ondaatje and Walter Murch. 2002. The conversations: Walter Murch and the art of editing film. A&C Black.

[35]

Luiz Pessoa. 2008. On the relationship between emotion and cognition. Nature Reviews Neuroscience 9, 2 (2008), 148--158.

[36]

Michael I. Posner, M. Rosario Rueda, and Philipp Kanske. 2007. Probing the Mechanisms of Attention. In Handbook of Psychophysiology (third ed.), John T. Cacioppo, Louis G. Tassinary, and Gary Berntson (Eds.). Cambridge University Press, 410--432. Cambridge Books Online.

[37]

William F. Prokasy and David C. Raskin (Eds.). 1973. Electrodermal Activity in Psychological Research. Academic Press.

[38]

M. J. Rosenberg and C. I. Hovland. 1960. Cognitive, affective, and behavioral components of attitude. In Attitude organization and change, M.J. Rosenberg, C. I. Hovland, McGuire W., Abelson R., and J. Brehm (Eds.). New Haven, CT: Yale University Press.

[39]

Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. 1998. A Metric for Distributions with Applications to Image Databases. In Proceedings of the Sixth International Conference on Computer Vision (ICCV '98). IEEE Computer Society, Washington, DC, USA, 59--. http://dl.acm.org/citation.cfm?id=938978.939133

Digital Library

[40]

Klaus R Scherer. 2005. What are emotions? And how can they be measured? Social science information 44, 4 (2005), 695--729.

[41]

Henrique Sequeira, Pascal Hot, Laetitia Silvert, and Sylvain Delplanque. 2009. Electrical autonomic correlates of emotion. International journal of psychophysiology 71, 1 (2009), 50--56.

[42]

Alvy Ray Smith. 1978. Color Gamut Transform Pairs. SIGGRAPH Comput. Graph. 12, 3 (Aug. 1978), 12--19.

Digital Library

[43]

Mohammad Soleymani, Guillaume Chanel, Joep J. M. Kierkels, and Thierry Pun. 2008. Affective Characterization of Movie Scenes Based on Multimedia Content Analysis and User's Physiological Emotional Responses. In ISM. IEEE Computer Society, 228--235. http://dblp.uni-trier.de/db/conf/ism/ism2008.html#SoleymaniCKP08

Digital Library

[44]

Mohammad Soleymani, Jeroen Lichtenauer, Thierry Pun, and Maja Pantic. 2012. A Multimodal Database for Affect Recognition and Implicit Tagging. Affective Computing, IEEE Transactions on 3, 1 (2012), 42--55.

Digital Library

[45]

Kenneth L. Subotnik, Anne M. Schell, Mark S. Chilingar, Michael E. Dawson, Joseph Ventura, Kimberle A. Kelly, Gerhard S. Hellemann, and Keith H. Nuechterlein. 2012. The interaction of electrodermal activity and expressed emotion in predicting symptoms in recent-onset schizophrenia. Psychophysiology 49, 8 (2012), 1035--1038.

[46]

Ed S Tan (Ed.). 1996. Emotion and the Structure of Narrative Film: Film as an Emotion Machine. Lawrence Erlbaum Associates, Inc.

[47]

Daniel Tranel. 2000. Electrodermal activity in cognitive neuroscience: Neuroanatomical and neuropsychological correlates. In Cognitive neuroscience of emotion. Series in affective science. Oxford University Press, 192--224.

[48]

Yong Jian Wang and Michael S Minor. 2008. Validity, reliability, and applicability of psychophysiological techniques in marketing research. Psychology & Marketing 25, 2 (2008), 197--232.

[49]

Boon-Lock Yeo and Bede Liu. 1995. Rapid Scene Analysis on Compressed Video. IEEE Trans. Cir. and Sys. for Video Technol. 5, 6 (Dec. 1995), 533--544.

Digital Library

[50]

R. B. Zajonc. 1980. Feeling and thinking: Preferences need no inferences. American Psychologist 35, 2 (1980), 151--175.

Cited By

Naghizade EJi KTag BSalim FKostakos VKay JHoang T(2024)Inside Out or Not: Privacy Implications of Emotional DisclosureCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677598(121-125)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677598
Kalaganis FGeorgiadis KOikonomou VLaskaris NNikolopoulos SKompatsiaris I(2021)Unlocking the Subconscious Consumer Bias: A Survey on the Past, Present, and Future of Hybrid EEG Schemes in NeuromarketingFrontiers in Neuroergonomics10.3389/fnrgo.2021.6729822Online publication date: 17-May-2021
https://doi.org/10.3389/fnrgo.2021.672982
García-Faura ÁHernández-García AFernández-Martínez FDíaz-de-María FSan-Segundo R(2019)Emotion and attention: Audiovisual models for group-level skin response recognition in short moviesWeb Intelligence10.3233/WEB-19039817:1(29-40)Online publication date: 22-Feb-2019
https://doi.org/10.3233/WEB-190398
Show More Cited By

Recommendations

Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios

Following the idea of using virtual environments (VEs) as mood induction procedures (MIPs), this study set out to examine whether five different virtual park scenarios would each elicit a specific affective state (i.e., joy, sadness, boredom, anger and ...
Virtual proximity and facial expressions of computer agents regulate human emotions and attention
CASA' 2010 Special Issue

Emotion- and attention-related subjective and physiological responses to virtual proximity and facial expressions of embodied computer agents (ECA) were studied. Thirty participants viewed female and male characters with a neutral, unpleasant, or ...
Do all facial emojis communicate emotion? The impact of facial emojis on perceived sender emotion and text processing
Abstract
Facial emojis can express a variety of positive and negative emotions, and are commonly used in digital, written communication. However, little is known about how emojis impact text processing and how different emoji-text combinations give rise ...
Highlights
- We investigated how facial emojis impact text comprehension using behavioral ratings and EEG.
- In ratings, negative emojis communicate specific mental states, while positive emojis communicate general positive emotion.
- In ERPs, the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

WI '17: Proceedings of the International Conference on Web Intelligence

August 2017

1284 pages

ISBN:9781450349512

DOI:10.1145/3106426

Conference Chair:
Amit Sheth
Wright State University GuoROLE@GENERAL CHAIR
,
General Chairs:
Axel Ngonga
Leipzig University, Germany
,
yin Wang
Chongqing University of Posts and Telecommunications, China
,
Elizabeth Chang
The University of New South Wales, Australia
,
Dominik Ślęzak
Infobright Inc. & University of Warsaw, Poland
,
Bogdan Franczyk
Leipzig University, Germany
,
Program Chairs:
Rainer Alt
Leipzig University, Germany
,
Xiaohui Tao
University of Southern Queensland, Australia

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
TCII: IEEE Computer Society Technical Committee on Intelligent Informatics
Web Intelligence Consortium

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WI '17

Sponsor:

SIGAI
TCII

WI '17: International Conference on Web Intelligence 2017

August 23 - 26, 2017

Leipzig, Germany

Acceptance Rates

WI '17 Paper Acceptance Rate 118 of 178 submissions, 66%;

Overall Acceptance Rate 118 of 178 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
268
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)2

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Naghizade EJi KTag BSalim FKostakos VKay JHoang T(2024)Inside Out or Not: Privacy Implications of Emotional DisclosureCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677598(121-125)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677598
Kalaganis FGeorgiadis KOikonomou VLaskaris NNikolopoulos SKompatsiaris I(2021)Unlocking the Subconscious Consumer Bias: A Survey on the Past, Present, and Future of Hybrid EEG Schemes in NeuromarketingFrontiers in Neuroergonomics10.3389/fnrgo.2021.6729822Online publication date: 17-May-2021
https://doi.org/10.3389/fnrgo.2021.672982
García-Faura ÁHernández-García AFernández-Martínez FDíaz-de-María FSan-Segundo R(2019)Emotion and attention: Audiovisual models for group-level skin response recognition in short moviesWeb Intelligence10.3233/WEB-19039817:1(29-40)Online publication date: 22-Feb-2019
https://doi.org/10.3233/WEB-190398
Gashi SDi Lascio ESantini S(2019)Using Unobtrusive Wearable Sensors to Measure the Physiological Synchrony Between Presenters and Audience MembersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33144003:1(1-19)Online publication date: 29-Mar-2019
https://dl.acm.org/doi/10.1145/3314400

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents