More Web Proxy on the site http://driver.im/

research-article

Progressive ShallowNet for large scale dynamic and spontaneous facial behaviour analysis in children

Authors:

Moona MazherAuthors Info & Claims

Volume 119, Issue C

https://doi.org/10.1016/j.imavis.2022.104375

Published: 01 March 2022 Publication History

Abstract

Highlights

•

Progressive weight ShallowNet for analysis of spontaneous facial emotions.

•

Limit the alternative path for the gradient at the earlier stage and increase gradually.

•

The network is able to explore more feature space and vulnerable to perturbations.

Abstract

COVID-19 has severely disrupted every aspect of society and left negative impact on our life. Resisting the temptation in engaging face-to-face social connection is not as easy as we imagine. Breaking ties within social circle makes us lonely and isolated, that in turns increase the likelihood of depression related disease and even can leads to death by increasing the chance of heart disease. Not only adults, children's are equally impacted where the contribution of emotional competence to social competence has long term implications. Early identification skill for facial behaviour emotions, deficits, and expression may help to prevent the low social functioning. Deficits in young children's ability to differentiate human emotions can leads to social functioning impairment. However, the existing work focus on adult emotions recognition mostly and ignores emotion recognition in children. By considering the working of pyramidal cells in the cerebral cortex, in this paper, we present progressive lightweight shallow learning for the classification by efficiently utilizing the skip-connection for spontaneous facial behaviour recognition in children. Unlike earlier deep neural networks, we limit the alternative path for the gradient at the earlier part of the network by increase gradually with the depth of the network. Progressive ShallowNet is not only able to explore more feature space but also resolve the over-fitting issue for smaller data, due to limiting the residual path locally, making the network vulnerable to perturbations. We have conducted extensive experiments on benchmark facial behaviour analysis in children that showed significant performance gain comparatively.

References

[1]

I. Abbasnejad, S. Sridharan, D. Nguyen, S. Denman, C. Fookes, S. Lucey, Using synthetic data to improve facial expression analysis with 3d convolutional networks, in: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 1609–1618.

[2]

D.A. Al Chanti, A. Caplier, Deep learning for spatio-temporal modeling of dynamic spontaneous emotions, IEEE Trans. Affect. Comput. 12 (2) (2021) 363–376,.

Digital Library

[3]

P. Barros, S. Wermter, Developing crossmodal expression recognition based on a deep neural model, Adapt. Behav. 24 (5) (2016) 373–396.

[4]

Spatio-Temporal, Samples using deep. Audio-visual emotion recognition system for variable length spatio-temporal samples using deep transfer-learning., in: Business Information Systems: 23rd International Conference, BIS 2020, CO, USA June 8-10, 2020, Proceedings, volume 389, Colorado Springs, Springer, 2020, p. 434.

[5]

S.A. Denham, E.A. Couchoud, Young preschoolers’ understanding of emotions, Child Stud. J. 20 (3) (1990) 171–192.

[6]

S.A. Denham, M. McKinley, E.A. Couchoud, R. Holt, Emotional and behavioral predictors of preschool peer ratings, Child Dev. 61 (4) (1990) 1145–1152.

[7]

Y. Fan, X. Lu, D. Li, Y. Liu, Video-based emotion recognition using cnn-rnn and c3d hybrid networks, in: Proceedings of the 18th ACM International Conference on Multimodal Interaction, 2016, pp. 445–450.

[8]

K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: surpassing human-level performance on imagenet classification, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1026–1034.

Digital Library

[9]

S.M. Jones, J.L. Brown, W.L. Hoglund, J.L. Aber, A school-randomized clinical trial of an integrated social-emotional learning and literacy intervention: Impacts after 1 school year, J. Consult. Clin. Psychol. 78 (6) (2010) 829.

[10]

S.M. Jones, J.L. Brown, J.L. Aber, Two-year impacts of a universal school-based social-emotional and literacy intervention: an experiment in translational developmental research, Child Dev. 82 (2) (2011) 533–554.

[11]

R.A. Khan, A. Crenn, A. Meyer, S. Bouakaz, A novel database of children's spontaneous facial expressions (LIRIS-CSE), Image Vision Comput. 83 (2019) 61–69.

[12]

D.H. Kim, W.J. Baddar, J. Jang, Y.M. Ro, Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition, IEEE Trans. Affect. Comput. 10 (2) (2017) 223–236.

[13]

Y.-H. Lai, S.-H. Lai, Emotion-preserving representation learning via generative adversarial network for multi-view facial expression recognition, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), IEEE, 2018, pp. 263–270.

[14]

P. Liu, S. Han, Z. Meng, Y. Tong, Facial expression recognition via a boosted deep belief network, in: Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition, 2014, pp. 1805–1812.

[15]

Y. Lv, Z. Feng, C. Xu, Facial expression recognition via deep learning, in: 2014 International Conference on Smart Computing, IEEE, 2014, pp. 303–308.

[16]

D.J. McDowell, R. O’Neil, R.D. Parke, Display Rule Application In A Disappointing Situation and Children’s Emotional Reactivity: Relations With Social Competence. Merrill-Palmer Quarterly (1982-), American Psychological Association, USA, 2000, pp. 306–324.

[17]

S. Rifai, Y. Bengio, A. Courville, P Vincent, M. Mirza, Disentangling factors of variation for facial expression recognition, in: European Conference on Computer Vision, Springer, 2012, pp. 808–822.

[18]

M. Sprung, H.M. Münch, P.L. Harris, C. Ebesutani, S.G. Hofmann, Children's emotion understanding: a meta-analysis of training studies, Dev. Rev. 37 (2015) 41–65.

[19]

D. Tran, L. Bourdev, R. Fergus, L. Torresani, M. Paluri, Learning spatiotemporal features with 3d convolutional networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 4489–4497.

[20]

V. Vielzeuf, S. Pateux, F. Jurie, Temporal multimodal fusion for video emotion classification in the wild, in: Proceedings of the 19th ACM International Conference on Multimodal Interaction, 2017, pp. 569–576.

[21]

H. Yang, Z. Zhang, L. Yin, Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), IEEE, 2018, pp. 294–301.

[22]

F. Zhang, T. Zhang, Q. Mao, C. Xu, Joint pose and expression modeling for facial expression recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3359–3368.

[23]

J. Zhao, X. Mao, J. Zhang, Learning deep facial expression features from image and optical flow sequences using 3d cnn, Visual Comput. 34 (10) (2018) 1461–1475.

Cited By

Qayyum ARazzak ITanveer MMazher M(2022)Spontaneous Facial Behavior Analysis Using Deep Transformer-based Framework for Child–computer InteractionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/353957720:2(1-17)Online publication date: 26-May-2022
https://dl.acm.org/doi/10.1145/3539577
Gurpinar CTakir SBicer EUluer PArica NKose H(2022)Contrastive learning based facial action unit detection in children with hearing impairment for a socially assistive robot platformImage and Vision Computing10.1016/j.imavis.2022.104572128:COnline publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.imavis.2022.104572

Index Terms

Progressive ShallowNet for large scale dynamic and spontaneous facial behaviour analysis in children
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Children can accurately recognize facial emotions from emoticons

Developmental studies on children's ability to recognize emotions have focused on communication in physical, but not virtual space. Emotion recognition in real life is based on observation of facial expression whereas in virtual communication emotions ...
Design for self-reporting psychological health in children with intellectual disabilities
IDC '15: Proceedings of the 14th International Conference on Interaction Design and Children

In this paper, we describe the development and initial results of an interactive touch based questionnaire aiming to support self-reporting of the psychological health among children with intellectual disabilities (ID) aged 12-15. The questionnaire was ...
Emotion Recognition Model Based on Facial Expressions, Ethnicity and Gender Using Backpropagation Neural Network

Many emotion recognition approaches are built using facial expressions, but few of them use both the ethnicity and gender as attributes. The authors have developed an approach based on Artificial Neural Networks ANN using backpropagation algorithm to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Image and Vision Computing

Image and Vision Computing Volume 119, Issue C

Mar 2022

138 pages

ISSN:0262-8856

Issue’s Table of Contents

Elsevier B.V.

Publisher

Butterworth-Heinemann

United States

Publication History

Published: 01 March 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qayyum ARazzak ITanveer MMazher M(2022)Spontaneous Facial Behavior Analysis Using Deep Transformer-based Framework for Child–computer InteractionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/353957720:2(1-17)Online publication date: 26-May-2022
https://dl.acm.org/doi/10.1145/3539577
Gurpinar CTakir SBicer EUluer PArica NKose H(2022)Contrastive learning based facial action unit detection in children with hearing impairment for a socially assistive robot platformImage and Vision Computing10.1016/j.imavis.2022.104572128:COnline publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.imavis.2022.104572

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents