To Our Emotions, with Love: How Affective Should Affective Computing Be?

“Affective computing” has become the rallying call for a heterogeneous group of researchers that, among other goals, tries to improve the interaction of humans and machines via the development of affective and multimodal intelligent systems. This development appears logical based on the popular notion that emotions play an important part in social interactions. In fact, research shows that humans missing the possibility to express or perceive/interpret emotions, seem to have difficulties navigating social conventions and experience a negative impact on their relationships.

However, while emotions are certainly somewhat important, the desire to implement affect in machines might also be influenced by romantic notions, echoed in the plight of iconic science fiction characters, such as Star Trek’s Data, who struggles to achieve humanity via emotions. The emphasis on emotions in the psychological research on nonverbal communication is in part due to theoretical discussions in that community. However, taken out of this context, there is the risk to overestimate the importance of discrete emotional expressions. Inversely, behaviors relevant to successful interaction will be underestimated in psychology because they might be culturally variable and perhaps even idiosyncratic for smaller groups or individuals. In other words, what might be noise for the emotion theorist could be the data to create believable conversational agents.

I will discuss how much emotion might or might not be needed when trying to build emotional or emotion-savvy systems, depending on the type of application that is desired, based on a multi-level approach. At one level of analysis, a clear distinction of encoding and decoding processes is required to know what (real) people actually show in certain situations, or what people might in fact perceive. It is not obvious how much information is actually “read” from faces, as opposed to “read” into faces. In other words, context plays a large role for the interpretation of nonverbal behavior. Some of this context is verbal, but some is situational.

At a different level of analysis, interactive characteristics in conversation need to be considered. This refers to issues such as responsiveness, synchrony, or imitation that are often neglected in affective computing applications and in basic psychological research. For example, an artificial system that will only react to observed patterns of verbal/nonverbal behavior might be too slow and create strange delayed effects, as opposed to systems that seem to react, but that, in fact, anticipate the interactant’s reactions. It is these areas where much interesting work is, should be, and will be happening in the next few years.

Authors and Affiliations

1. School of Humanities and Social Sciences, Jacobs University Bremen, Campus Ring 1, 28759, Bremen, Germany

   Arvid Kappas

Editors and Affiliations

1. University of Memphis, 202 Psychology Building, 38152, Memphis, TN, USA

   Sidney D’Mello  & Arthur Graesser  & 

2. Technische Universität München, Arcisstraße 21, 80333, München, Germany

   Björn Schuller

3. Laboratoire d’Informatique pour la Mécanique et les Sciences de l’Ingénieur (LIMSI-CNRS), Bâtiment 508, 91403, Orsay Cedex, France

   Jean-Claude Martin

Reprints and permissions

© 2011 Springer-Verlag Berlin Heidelberg

Policies and ethics