Hierarchical facial expression recognition
Authors: 
Xiang Zhang, Yu Gu, Huan Yan, Yantong Wang, Fuji Ren, Yusheng JiAuthors Info & Claims
Pages 119 - 123
Abstract
Learning facial expressions constitutes a challenging job due to the uncertainties caused by the ambiguity of facial expressions. To address this issue, we propose a simple yet efficient Coarse-to-Fine Network (CFNet) inspired by human being's cognitive mode for suppressing such uncertainties. A child learns quickly whether his behavior is allowed by reading adults' coarse facial expressions like positive or negative, and then adjusts accordingly via further interpreting its fine meaning like happy or angry. Similarly, CFNet first aggregates basic facial expressions into 3 coarse categories based on their distributions in the Valence-Arousal (VA) emotion space. Then, CFNet leverages fine labels with the coarse classification results for fine-grained facial expression recognition with discriminative loss handling high intra-class variations within coarse categories. Experiments on benchmark datasets demonstrate the superiority of our method over the state-of-art rivals.
References
[1]
Samuel Albanie, Arsha Nagrani, Andrea Vedaldi, and Andrew Zisserman. 2018. Emotion recognition in speech using cross-modal transfer in the wild. In Proceedings of the 26th ACM international conference on Multimedia. 292--301.
[2]
Panagiotis Antoniadis, Panagiotis P Filntisis, and Petros Maragos. 2021. Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition. arXiv preprint arXiv:2106.03487 (2021).
[3]
Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang. 2016. Training deep networks for facial expression recognition with crowd-sourced label distribution. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 279--283.
[4]
Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), Vol. 1. Ieee, 886--893.
[5]
Paul Ekman. 1993. Facial expression and emotion. American psychologist 48, 4 (1993), 384.
[6]
Amir Hossein Farzaneh and Xiaojun Qi. 2021. Facial expression recognition in the wild via deep attentive center loss. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2402--2411.
[7]
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. Msceleb-1m: A dataset and benchmark for large-scale face recognition. In European conference on computer vision. Springer, 87--102.
[8]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[9]
Sheheryar Khan, Lijiang Chen, and Hong Yan. 2017. Co-clustering to reveal salient facial features for expression recognition. IEEE Transactions on Affective Computing 11, 2 (2017), 348--360.
[10]
Hanting Li, Mingzhe Sui, Feng Zhao, Zhengjun Zha, and Feng Wu. 2021. MViT: Mask Vision Transformer for Facial Expression Recognition in the wild. arXiv preprint arXiv:2106.04520 (2021).
[11]
Shan Li, Weihong Deng, and JunPing Du. 2017. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2852--2861.
[12]
Ling Lo, Hong Xia Xie, Hong-Han Shuai, and Wen-Huang Cheng. 2021. Facial Chirality: Using Self-Face Reflection to Learn Discriminative Features for Facial Expression Recognition. In 2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1--6.
[13]
Ali Mollahosseini, Behzad Hasani, and Mohammad H Mahoor. 2017. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing 10, 1 (2017), 18--31.
[14]
Rosalind W Picard. 2000. Affective computing. MIT press.
[15]
Delian Ruan, Yan Yan, Shenqi Lai, Zhenhua Chai, Chunhua Shen, and Hanzi Wang. 2021. Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7660--7669.
[16]
James A Russell. 1980. A circumplex model of affect. Journal of personality and social psychology 39, 6 (1980), 1161.
[17]
Liam Schoneveld, Alice Othmani, and Hazem Abdelkawy. 2021. Leveraging recent advances in deep learning for audio-Visual emotion recognition. Pattern Recognition Letters (2021).
[18]
Jiahui She, Yibo Hu, Hailin Shi, Jun Wang, Qiu Shen, and Tao Mei. 2021. Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6248--6257.
[19]
Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao. 2020. Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6897--6906.
[20]
Kai Wang, Xiaojiang Peng, Jianfei Yang, Debin Meng, and Yu Qiao. 2020. Region attention networks for pose and occlusion robust facial expression recognition. IEEE Transactions on Image Processing 29 (2020), 4057--4069.
[21]
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A discriminative feature learning approach for deep face recognition. In European conference on computer vision. Springer, 499--515.
[22]
Zhengyao Wen, Wenzhong Lin, Tao Wang, and Ge Xu. 2021. Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition. arXiv preprint arXiv:2109.07270 (2021).
[23]
Haifeng Zhang, Wen Su, Jun Yu, and Zengfu Wang. 2020. Weakly Supervised Local-Global Relation Network for Facial Expression Recognition. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, Christian Bessiere (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1040--1046.
[24]
Yuhang Zhang, Chengrui Wang, and Weihong Deng. 2021. Relative Uncertainty Learning for Facial Expression Recognition. Advances in Neural Information Processing Systems 34 (2021).
[25]
Zengqun Zhao, Qingshan Liu, and Feng Zhou. 2021. Robust Lightweight Facial Expression Recognition Network with Label Distribution Training. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3510--3519.
[26]
Lin Zhong, Qingshan Liu, Peng Yang, Bo Liu, Junzhou Huang, and Dimitris N Metaxas. 2012. Learning active facial patches for expression analysis. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2562--2569.
Index Terms
- Hierarchical facial expression recognition
Recommendations
Expression-invariant face recognition by facial expression transformations
In this paper, we present a method of expression-invariant face recognition that transforms input face image with an arbitrary expression into its corresponding neutral facial expression image. When a new face image with an arbitrary expression is ...
Facial expression recognition with Convolutional Neural Networks
Facial expression recognition has been an active research area in the past 10 years, with growing application areas including avatar animation, neuromarketing and sociable robots. The recognition of facial expressions is not an easy problem for machine ...
Pose-Robust Facial Expression Recognition Using View-Based 2D + 3D AAM
This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D 3D active appearance model (AAM) that extends the 2D 3D AAM to the view-based approach, where one independent face model is used for a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2022
208 pages
ISBN:9781450393980
DOI:10.1145/3538641
- Conference Chair:
- Peng Li,
- Program Chairs:
- Junyoung Heo,
- Tomas Cerny
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 20 October 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
RACS '22
Sponsor:
RACS '22: International Conference on Research in Adaptive and Convergent Systems
October 3 - 6, 2022
Virtual Event, Japan
Acceptance Rates
Overall Acceptance Rate 393 of 1,581 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 78Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)3
Reflects downloads up to 26 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in