Uncovering the Nuanced Structure of Expressive Behavior Across Modalities

Guided by semantic space theory, large-scale computational studies have advanced our understanding of the structure and function of expressive behavior. I will integrate findings from experimental studies of facial expression (N=19,656), vocal bursts (N=12,616), speech prosody (N=20,109), multimodal reactions (N=8,056), and an ongoing study of dyadic interactions (N=1,000+). These studies combine methods from psychology and computer science to yield new insights into what expressive behaviors signal, how they are perceived, and how they shape social interaction. Using machine learning to extract cross-cultural dimensions of behavior while minimizing biases due to demographics and context, we arrive at objective measures of the structural dimensions that make up human expression. Expressions are consistently found to be high-dimensional and blended, with their meaning across cultures being efficiently conceptualized in terms of a wide range of specific emotion concepts. Altogether, these findings generate a comprehensive new atlas of expressive behavior, which I will explore through a variety of visualizations. This new taxonomy departs from models such as the basic six and affective circumplex, suggesting a new way forward for expression understanding and sentiment analysis.