More Web Proxy on the site http://driver.im/

poster

On the relationship between head pose, social attention and personality prediction for unstructured and dynamic group interactions

Authors:

Ramanathan Subramanian,

Jacopo Staiano,

Nicu SebeAuthors Info & Claims

ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

Pages 3 - 10

https://doi.org/10.1145/2522848.2522862

Published: 09 December 2013 Publication History

Abstract

Correlates between social attention and personality traits have been widely acknowledged in social psychology studies. Head pose has commonly been employed as a proxy for determining the social attention direction in small group interactions. However, the impact of head pose estimation errors on personality estimates has not been studied to our knowledge.

In this work, we consider the unstructured and dynamic cocktail party scenario where the scene is captured by multiple, large field-of-view cameras. Head pose estimation is a challenging task under these conditions owing to the uninhibited motion of persons (due to which facial appearance varies owing to perspective and scale changes), and the low resolution of captured faces. Based on proxemic and social attention features computed from position and head pose annotations, we first demonstrate that social attention features are excellent predictors of the Extraversion and Neuroticism personality traits. We then repeat classification experiments with behavioral features computed from automated estimates-- obtained experimental results show that while prediction performance for both traits is affected by head pose estimation errors, the impact is more adverse for Extraversion.

References

[1]

A. Airola, T. Pahikkala, W. Waegeman, B. D. Baets, and T. Salakoski. A comparison of auc estimators in small-sample studies. Journal of Machine Learning Research - Proceedings Track, 8:3--13, 2010.

[2]

N. Ambady and R. Rosenthal. Thin slices' of expressive behaviors as predictors of interpersonal consequences. a meta analysis. Psychological Bulletin, 111:156--274, 1992.

[3]

S. O. Ba and J.-M. Odobez. Recognizing visual focus of attention from head pose in natural meetings. IEEE Transactions on Systems, Man, and Cybernetics--Part B: Cybernetics, 39(1):16--33, 2009.

Digital Library

[4]

D. R. Carney, C. R. Colvin, and J. A. Hall. A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, 41:1054--1072, 2007.

[5]

C. Cortes and V. Vapnik. Support-vector networks. In Machine Learning, pages 273--297, 1995.

Digital Library

[6]

N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, pages 886--893, 2005.

Digital Library

[7]

S. De Julio and K. Duffy. Neuroticism and proxemic behavior. Perception and Motor Skills, 45(1):51--55, 1977.

[8]

J. F. Dovidio and S. L. Ellyson. Decoding visual dominance: Attributions of power based on relative percentages of looking while speaking and looking while listening. Social Psychology Quarterly, 45(2):106--113, 1982.

[9]

T. Evgeniou and M. Pontil. Regularized multi--task learning. In Int'l conference on Knowledge Discovery and Data Mining, pages 109--117, 2004.

Digital Library

[10]

T. Evgeniou and M. Pontil. Regularized multi-task learning. In ACM Int'l Conference on Knowledge Discovery and Data Mining, 2004.

Digital Library

[11]

E. Frank, M. Hall, and B. Pfahringer. Locally weighted naive bayes. In Uncertainty in Artificial Intelligence, pages 249--256, 2003.

Digital Library

[12]

E. T. Hall. The hidden dimension. Anchor Books, 1963.

[13]

H. Hung, D. B. Jayagopi, S. Ba, J.-M. Odobez, and D. Gatica-Perez. Investigating automatic dominance estimation in groups from visual attention and speaking activity. In Int'l Conference on Multimodal Interfaces, pages 233--236, 2008.

Digital Library

[14]

D. B. Jayagopi, H. Hung, C. Yeo, and D. Gatica-Perez. Modeling dominance in group conversations using nonverbal activity cues. IEEE Trans. Audio, Speech and Lang. Proc.- Special issue on multimodal processing in speech-based interactions, 17(3):501--513, 2009.

Digital Library

[15]

S. R. Langton, R. J. Watt, and I. Bruce. Do the eyes have it? cues to the direction of social attention. Trends in Cognitive Science, 4(2):50--59, 2000.

[16]

O. Lanz and R. Brunelli. Dynamic head location and pose from video. In Int'l Conference on Multisensor Fusion and Integration for Intelligent Systems, pages 47--52, 2006.

[17]

B. Lepri, J. Staiano, G. Rigato, K. Kalimeri, A. Finnerty, F. Pianesi, N. Sebe, and A. Pentland. The sociometric badges corpus: A multilevel behavioral dataset for social behavior in complex organizations. In Int'l Conference on Social Computing, pages 623--628, 2012.

Digital Library

[18]

B. Lepri, R. Subramanian, K. Kalimeri, J. Staiano, F. Pianesi, and N. Sebe. Connecting meeting behavior with extraversion - a systematic study. IEEE Transactions on Affective Computing, 3(4):443--455, 2012.

Digital Library

[19]

L. Liang and V. Cherkassky. Connection between svm

[20]

and multi-task learning. In Int'l Joint Conference on Neural Networks, 2008.

[21]

J.-M. Odobez and S. O. Ba. A Cognitive and Unsupervised MAP Adaptation Approach to the Recognition of the Focus of Attention from Head Pose. In Int'l Conference on Multi-Media & Expo, 2007.

[22]

M. Perugini and L. Di Blas. Analyzing personality-related adjectives from an eticemic perspective: the big five marker scale (bfms) and the italian ab5c taxonomy. Big Five Assessment, pages 281--304, 2002.

[23]

A. K. Rajagopal, R. Subramanian, R. L. Vieriu, E. Ricci, O. Lanz, K. Ramakrishnan, and N. Sebe. An adaptation framework for head-pose classification in dynamic multi-view scenarios. In Asian conference on Computer Vision, pages 652--666, 2012.

Digital Library

[24]

J. Staiano, B. Lepri, R. Subramanian, N. Sebe, and F. Pianesi. Automatic modeling of personality states in small group interactions. In ACM Int'l conference on Multimedia, pages 989--992, 2011.

Digital Library

[25]

L. B. Statistics and L. Breiman. Random forests. In Machine Learning, pages 5--32, 2001.

Digital Library

[26]

R. Stiefelhagen, J. Yang, and A. Waibel. Modeling focus of attention for meeting indexing based on multiple cues. IEEE Transactions on Neural Networks, 13(4):928--938, 2002.

Digital Library

[27]

R. Subramanian, J. Staiano, K. Kalimeri, N. Sebe, and F. Pianesi. Putting the pieces together: multimodal analysis of social attention in meetings. In Int'l Conference on Multimedia, pages 659--662, 2010.

Digital Library

[28]

M. Voit and R. Stiefelhagen. Deducing the visual focus of attention from head pose estimation in dynamic multi-view meeting scenarios. In Int'l Conference on Multimodal interfaces, pages 173--180, 2008.

Digital Library

[29]

G. Zen, B. Lepri, E. Ricci, and O. Lanz. Space speaks: towards socially and personality aware visual surveillance. In ACM Int'l Workshop on Multimodal Pervasive Video Analysis, pages 37--42, 2010.

Digital Library

Cited By

Madan SGahalawat MGuha TGoecke RSubramanian R(2025)Explainable human-centered traits from head motion and facial expression dynamicsPLOS ONE10.1371/journal.pone.031388320:1(e0313883)Online publication date: 17-Jan-2025
https://doi.org/10.1371/journal.pone.0313883
Malik HDhillon HParameshwara RGoecke RSubramanian R(2023)Examining the Influence of Personality and Multimodal Behavior on Hireability ImpressionsProceedings of the Fourteenth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3627631.3627658(1-9)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3627631.3627658
Beyan CVinciarelli ABue A(2023)Co-Located Human–Human Interaction Analysis Using Nonverbal Cues: A SurveyACM Computing Surveys10.1145/362651656:5(1-41)Online publication date: 25-Nov-2023
https://dl.acm.org/doi/10.1145/3626516
Show More Cited By

Recommendations

Tracking head pose and focus of attention with multiple far-field cameras
ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

In this work we present our recent approach on estimating head orientations and foci of attention of multiple people in a smart room, which is equipped with several cameras to monitor the room. In our approach, we estimate each person's head orientation ...
Head pose estimation with one camera, in uncalibrated environments
EGIHMI '10: Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction

Head pose together with eye gaze are a reliable indication regarding the estimate of the focus of attention of a person standing in front of a camera, with applications ranging from driver's attention estimation to meeting environments. As gaze ...
Continuous Emotion Recognition in Videos by Fusing Facial Expression, Head Pose and Eye Gaze
ICMI '19: 2019 International Conference on Multimodal Interaction

Continuous emotion recognition is of great significance in affective computing and human-computer interaction. Most of existing methods for video based continuous emotion recognition utilize facial expression. However, besides facial expression, other ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

December 2013

630 pages

ISBN:9781450321297

DOI:10.1145/2522848

General Chairs:
Julien Epps
The University of New South Wales, Australia
,
Fang Chen
National ICT Australia, Australia
,
Sharon Oviatt
Incaa Designs, USA
,
Kenji Mase
Nagoya University, Japan
,
Program Chairs:
Andrew Sears
Rochester Institute of Technology, USA
,
Kristiina Jokinen
University of Helsinki, Finland
,
Björn Schuller
Technische Universität München, Germany

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 December 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICMI '13

Sponsor:

SIGCHI

ICMI '13: 2013 International Conference on Multimodal Interaction

December 9 - 13, 2013

Sydney, Australia

Acceptance Rates

ICMI '13 Paper Acceptance Rate 49 of 133 submissions, 37%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
368
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)4

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Madan SGahalawat MGuha TGoecke RSubramanian R(2025)Explainable human-centered traits from head motion and facial expression dynamicsPLOS ONE10.1371/journal.pone.031388320:1(e0313883)Online publication date: 17-Jan-2025
https://doi.org/10.1371/journal.pone.0313883
Malik HDhillon HParameshwara RGoecke RSubramanian R(2023)Examining the Influence of Personality and Multimodal Behavior on Hireability ImpressionsProceedings of the Fourteenth Indian Conference on Computer Vision, Graphics and Image Processing10.1145/3627631.3627658(1-9)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3627631.3627658
Beyan CVinciarelli ABue A(2023)Co-Located Human–Human Interaction Analysis Using Nonverbal Cues: A SurveyACM Computing Surveys10.1145/362651656:5(1-41)Online publication date: 25-Nov-2023
https://dl.acm.org/doi/10.1145/3626516
Madan SJain RSharma GSubramanian RDhall AEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group SettingsProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612858(9526-9530)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612858
Jacques Junior JGucluturk YPerez MGuclu UAndujar CBaro XEscalante HGuyon Ivan Gerven Mvan Lier REscalera S(2022)First Impressions: A Survey on Vision-Based Apparent Personality Trait AnalysisIEEE Transactions on Affective Computing10.1109/TAFFC.2019.293005813:1(75-95)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TAFFC.2019.2930058
Tan STax DHung H(2021)Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction DynamicsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34481225:1(1-22)Online publication date: 30-Mar-2021
https://dl.acm.org/doi/10.1145/3448122
Romeo MHernández García DHan TCangelosi AJokinen K(2021)Predicting apparent personality from body language: benchmarking deep learning architectures for adaptive social human–robot interactionAdvanced Robotics10.1080/01691864.2021.1974941(1-13)Online publication date: 27-Oct-2021
https://doi.org/10.1080/01691864.2021.1974941
Ijuin KJokinen K(2020)Exploring Gaze Behaviour and Perceived Personality TraitsSocial Computing and Social Media. Design, Ethics, User Behavior, and Social Network Analysis10.1007/978-3-030-49570-1_35(504-512)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-49570-1_35
Celiktutan OSkordos EGunes H(2019)Multimodal Human-Human-Robot Interactions (MHHRI) Dataset for Studying Personality and EngagementIEEE Transactions on Affective Computing10.1109/TAFFC.2017.273701910:4(484-497)Online publication date: 1-Oct-2019
https://doi.org/10.1109/TAFFC.2017.2737019
Sun WFan YMin XPeng SMa SZhai G(2019)LPHD: A Large-Scale Head Pose Dataset for RGB Images2019 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME.2019.00190(1084-1089)Online publication date: Jul-2019
https://doi.org/10.1109/ICME.2019.00190
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten