More Web Proxy on the site http://driver.im/

research-article

Emotion Recognition In The Wild Challenge 2014: Baseline, Data and Protocol

Authors:

Tom GedeonAuthors Info & Claims

ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

Pages 461 - 466

https://doi.org/10.1145/2663204.2666275

Published: 12 November 2014 Publication History

Abstract

The Second Emotion Recognition In The Wild Challenge (EmotiW) 2014 consists of an audio-video based emotion classification challenge, which mimics the real-world conditions. Traditionally, emotion recognition has been performed on data captured in constrained lab-controlled like environment. While this data was a good starting point, such lab controlled data poorly represents the environment and conditions faced in real-world situations. With the exponential increase in the number of video clips being uploaded online, it is worthwhile to explore the performance of emotion recognition methods that work `in the wild'. The goal of this Grand Challenge is to carry forward the common platform defined during EmotiW 2013, for evaluation of emotion recognition methods in real-world conditions. The database in the 2014 challenge is the Acted Facial Expression In Wild (AFEW) 4.0, which has been collected from movies showing close-to-real-world conditions. The paper describes the data partitions, the baseline method and the experimental protocol.

References

[1]

M. S. Bartlett, G. Littlewort, M. G. Frank, C. Lainscsek, I. R. Fasel, and J. R. Movellan. Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia, 1(6):22--35, 2006.

[2]

A. Dhall, R. Goecke, J. Joshi, M. Wagner, and T. Gedeon. Emotion recognition in the wild challenge 2013. In Proceedings of the ACM on International Conference on Multimodal Interaction (ICMI), pages 509--516, 2013.

Digital Library

[3]

A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Acted Facial Expressions in the Wild Database. In Technical Report, 2011.

[4]

A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Static Facial Expression Analysis In Tough Conditions: Data, Evaluation Protocol And Benchmark. In Proceedings of the IEEE International Conference on Computer Vision and Workshops BEFIT, pages 2106--2112, 2011.

[5]

A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Collecting large, richly annotated facial-expression databases from movies. IEEE Multimedia, 19(3):0034, 2012.

Digital Library

[6]

A. Dhall, J. Joshi, I. Radwan, and R. Goecke. Finding Happiest Moments in a Social Context. In Proceedings of the Asian Conference on Computer Vision (ACCV), pages 613--626, 2012.

Digital Library

[7]

A. Dhall, K. Sikka, G. Littlewort, R. Goecke, and M. Bartlett. A Discriminative Parts Based Model Approach for Fiducial Points Free and Shape Constrained Head Pose Normalisation In The Wild. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pages 1--8, 2014.

[8]

F. Eyben, M. Wollmer, and B. Schuller. Openear-introducing the munich open-source emotion and affect recognition toolkit. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII), pages 1--6, 2009.

[9]

F. Eyben, M. Wöllmer, and B. Schuller. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the ACM International Conference on Multimedia (MM), pages 1459--1462, 2010.

Digital Library

[10]

P. F. Felzenszwalb and D. P. Huttenlocher. Pictorial Structures for Object Recognition. International Journal on Computer Vision, 61(1):55--79, 2005.

Digital Library

[11]

R. Gross, I. Matthews, J. F. Cohn, T. Kanade, and S. Baker. Multi-PIE. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (FG), pages 1--8, 2008.

[12]

S. E. Kahou, C. Pal, X. Bouthillier, P. Froumenty, S. Jean, K. R. Konda, P. Vincent, A. Courville, and Y. Bengio. Combining modality specific deep neural networks for emotion recognition in video. In Proceedings of the ACM on International Conference on Multimodal Interaction (ICMI), pages 543--550, 2013.

Digital Library

[13]

T. Kanade, J. F. Cohn, and Y. Tian. Comprehensive database for facial expression analysis. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (FG), pages 46--53, 2000.

Digital Library

[14]

M. Liu, R. Wang, Z. Huang, S. Shan, and X. Chen. Partial least squares regression on grassmannian manifold for emotion recognition. In Proceedings of the ACM on International Conference on Multimodal Interaction (ICMI), pages 525--530, 2013.

Digital Library

[15]

S. Nowee. Facial Expression Recognition in the Wild: The Influence of Temporal Information. PhD thesis, University of Amsterdam, 2014.

[16]

B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. A. Müller, and S. S. Narayanan. The interspeech 2010 paralinguistic challenge. In INTERSPEECH, pages 2794--2797, 2010.

[17]

B. Schuller, M. Valstar, F. Eyben, G. McKeown, R. Cowie, and M. Pantic. Avec 2011--the first international audio/visual emotion challenge. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII), pages 415--424, 2011.

Digital Library

[18]

K. Sikka, A. Dhall, and M. Bartlett. Weakly supervised pain localization using multiple instance learning. Proceedings of the IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pages 1--8, 2013.

[19]

K. Sikka, K. Dykstra, S. Sathyanarayana, G. Littlewort, and M. Bartlett. Multiple kernel learning for emotion recognition in the wild. In Proceedings of the ACM on International Conference on Multimodal Interaction (ICMI), pages 517--524, 2013.

Digital Library

[20]

F. Wallhoff. Facial expressions and emotion database, 2006. http://www.mmk.ei.tum.de/ waf/fgnet/feedtum.html.

[21]

J. Whitehill, G. Littlewort, I. R. Fasel, M. S. Bartlett, and J. R. Movellan. Toward Practical Smile Detection. IEEE Transaction on Pattern Analysis and Machine Intelligence, 31(11):2106--2111, 2009.

Digital Library

[22]

X. Xiong and F. De la Torre. Supervised descent method and its applications to face alignment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 532--539, 2013.

Digital Library

[23]

G. Zhao and M. Pietikainen. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Transaction on Pattern Analysis and Machine Intelligence, 29(6):915--928, 2007.

Digital Library

[24]

X. Zhu and D. Ramanan. Face detection, pose estimation, and landmark localization in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2879--2886, 2012.

Digital Library

Cited By

Liu YZhang XKauttonen JZhao G(2024)Uncertain Facial Expression Recognition via Multi-Task Assisted CorrectionIEEE Transactions on Multimedia10.1109/TMM.2023.330120926(2531-2543)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3301209
Wang RWu XXu THu CKittler J(2024)Deep Metric Learning on the SPD Manifold for Image Set ClassificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.319045034:2(663-680)Online publication date: Feb-2024
https://doi.org/10.1109/TCSVT.2022.3190450
Cai ZWang RXu TWu XKittler J(2024)A Riemannian Residual Learning Mechanism for SPD Network2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651149(1-9)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651149
Show More Cited By

Index Terms

Emotion Recognition In The Wild Challenge 2014: Baseline, Data and Protocol
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Hardware
  1. Communication hardware, interfaces and storage
    1. Signal processing systems

Recommendations

EmotiW 2016: video and group-level emotion recognition challenges
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

This paper discusses the baseline for the Emotion Recognition in the Wild (EmotiW) 2016 challenge. Continuing on the theme of automatic affect recognition `in the wild', the EmotiW challenge 2016 consists of two sub-challenges: an audio-video based ...
Video and Image based Emotion Recognition Challenges in the Wild: EmotiW 2015
ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

The third Emotion Recognition in the Wild (EmotiW) challenge 2015 consists of an audio-video based emotion and static image based facial expression classification sub-challenges, which mimics real-world conditions. The two sub-challenges are based on ...
From individual to group-level emotion recognition: EmotiW 5.0
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

Research in automatic affect recognition has come a long way. This paper describes the fifth Emotion Recognition in the Wild (EmotiW) challenge 2017. EmotiW aims at providing a common benchmarking platform for researchers working on different aspects ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

November 2014

558 pages

ISBN:9781450328852

DOI:10.1145/2663204

General Chairs:
Albert Ali Salah
Boğaziçi University, Turkey
,
Jeffrey Cohn
University of Pittsburgh, USA
,
Björn Schuller
University of Passau, Germany and Imperial College London, UK
,
Program Chairs:
Oya Aran
Idiap Research Institute, Switzerland
,
Louis-Philippe Morency
University of Southern California, USA
,
Philip R. Cohen
Adapx, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMI '14

Sponsor:

SIGCHI

ICMI '14: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 12 - 16, 2014

Istanbul, Turkey

Acceptance Rates

ICMI '14 Paper Acceptance Rate 51 of 127 submissions, 40%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

166
Total Citations
View Citations
877
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)8

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YZhang XKauttonen JZhao G(2024)Uncertain Facial Expression Recognition via Multi-Task Assisted CorrectionIEEE Transactions on Multimedia10.1109/TMM.2023.330120926(2531-2543)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3301209
Wang RWu XXu THu CKittler J(2024)Deep Metric Learning on the SPD Manifold for Image Set ClassificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.319045034:2(663-680)Online publication date: Feb-2024
https://doi.org/10.1109/TCSVT.2022.3190450
Cai ZWang RXu TWu XKittler J(2024)A Riemannian Residual Learning Mechanism for SPD Network2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651149(1-9)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651149
Zeng XGuo JWei YZhuo Y(2024)Deep hybrid manifold for image set classificationImage and Vision Computing10.1016/j.imavis.2024.104935(104935)Online publication date: Feb-2024
https://doi.org/10.1016/j.imavis.2024.104935
Peng LLiu YWei YCui JQi M(2024)Reliability-aware label distribution learning with attention-rectified for facial expression recognitionApplied Intelligence10.1007/s10489-024-05999-655:1Online publication date: 25-Nov-2024
https://doi.org/10.1007/s10489-024-05999-6
Alphonse AAbinaya SArikumar K(2023)A Novel Monogenic Sobel Directional Pattern (MSDP) and Enhanced Bat Algorithm-Based Optimization (BAO) with Pearson Mutation (PM) for Facial Emotion RecognitionElectronics10.3390/electronics1204083612:4(836)Online publication date: 7-Feb-2023
https://doi.org/10.3390/electronics12040836
Lima PMaranhão DSoares C(2023)Automatic Emotion Detection in the Learning of AlgorithmsProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617032(56-64)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3617023.3617032
Lian ZSun HSun LChen KXu MWang KXu KHe YLi YZhao JLiu YLiu BYi JWang MCambria EZhao GSchuller BTao JEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised LearningProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612836(9610-9614)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612836
Dhall ASingh MGoecke RGedeon TZeng DWang YIkeda K(2023)EmotiW 2023: Emotion Recognition in the Wild ChallengeProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3616545(746-749)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3616545
Wei DShen XSun QGao XRen Z(2023)Sparse Representation Classifier Guided Grassmann Reconstruction Metric Learning With Applications to Image Set AnalysisIEEE Transactions on Multimedia10.1109/TMM.2022.317353525(4307-4322)Online publication date: 2023
https://doi.org/10.1109/TMM.2022.3173535
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents