[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Combining filtered dictionary representation based deep subspace filter learning with a discriminative classification criterion for facial expression recognition

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Automatic facial expression recognition is an active research area that has attracted much attention from both academics and practitioners of different fields. However, in reality, the problem of noise interference and cross-dataset expression recognition generally degrade the performance of recognition methods, we investigate the problems above, and propose a facial expression recognition approach from the perspective of deep subspace filter learning combined with discriminative classification criterion. Specifically, to derive an effective expression-related feature representation, we construct the filtered dictionaries based on deep subspace filter learning structure that corresponds to extract different expressions. Also, considering the similarities and discriminations existed in the filtered dictionaries, we further present a flexible classification criterion that adopt a dynamic weight to increase the adaptation between filtered dictionaries. To sum up, the proposed approach has more discriminative power from the aspect of representation and classification. Comprehensive experiments carried out using several public datasets, including JAFFE, CK+, and KDEF datasets, confirm that the proposed approach is superior compared to several state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Abate AF, Cimmino L, Narducci F, Pero C (2020) Biometric face recognition based on landmark dynamics, In: IEEE international conference on dependable, autonomic and secure computing, international conference on pervasive intelligence and computing, international conference on cloud and big data computing, international conference on cyber science and technology congress (DASC/PiCom/CBDCom/CyberSciTech), pp 601–605

  • Abate Andrea F, Castiglione A (2020) Michele Nappi, Ignazio Passero, DELEX: a DEep Learning Emotive eXperience: Investigating empathic HCI. In: International conference on advanced visual interfaces, AVI

  • Al-Laith A, Shahbaz M (2021) Tracking sentiment towards news entities from arabic news on social media. Future Generation Computer Systems. Article in press

  • Alphonse AS, Shankar K, Jeyasheela Rakkini MJ et al (2021) A multi-scale and rotation-invariant phase pattern (MRIPP) and a stack of restricted Boltzmann machine (RBM) with preprocessing for facial expression classification. J Ambient Intell Human Comput 12:3447–3463

    Google Scholar 

  • Andre TL, Edilson A, Alberto FDS, Thiago OS (2017) Facial expression recognition with convolutional neural network: coping with few data and the training sample. Pattern Recogn 61:610–628

    Google Scholar 

  • Bashar F, Khan A, Ahmed F, Kabir MH (2013) Robust facial expression recognition based on median ternary pattern (mtp). Int Conf Elect Inf Commun Technol (EICT) 2014:1–5

    Google Scholar 

  • Chan TH, Jia K, Gao S et al (2015) PCANet: a simple deep learning baseline for image classification. IEEE T Image Process 24:5017–5032

    MathSciNet  MATH  Google Scholar 

  • Dahmane M, Meunier J (2011) Emotion recognition using dynamic grid-based hog features. In: Face and gesture, pp 884-888

  • Dhall A, Asthana A, Goecke R, Gedeon T (2011) Emotion recognition using phog and lpq features. In: Face and Gesture, pp. 878-883

  • Dharanya V, Raj Alex Noel Joseph, Gopi Varun P (2021) Facial Expression recognition through person-wise regeneration of expressions using auxiliary classifier generative adversarial network (AC-GAN) based model. J Vis Commun Image Represent 77:103110

    Google Scholar 

  • Du L, Hu H (2019) Weighted patch-based manifold regularization dictionary pair learning model for facial expression recognition using iterative optimization classification strategy. Comput Vis Image Understand 186:13–24

    Google Scholar 

  • Goeleven E, De R Verschuere R (2008) The Karolinska directed emotional faces: a validation study. Cogn Emot 22:1094–1118

    Google Scholar 

  • Gund M, Bharadwaj AR, Nwogu I (2021) “Interpretable emotion classification using temporal convolutional models,” In: 2020 25th international conference on pattern recognition (ICPR), pp 6367-6374

  • Gund M, Bharadwaj AR, Nwogu I (2021) Interpretable emotion classification using temporal convolutional models, In: 2020 25th international conference on pattern recognition (ICPR), pp. 6367-6374

  • Guo S, Feng L, Feng ZB, Li YH, Wang Y, Liu SL, Qiao H (2019) Multi-view Laplacian least squares for human emotion recognition. Neurocomputing 370:78–87

    Google Scholar 

  • Han Y, Wu F, Tao D, Shao J, Zhuang Y et al (2012) Sparse unsupervised dimensionality reduction for multiple view data. IEEE Trans Circuits Syst Video Technol 22:1485–1496

    Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37:346–361

    Google Scholar 

  • Hinton G, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507

    MathSciNet  MATH  Google Scholar 

  • Hu Y, Mian AS, Owens R (2011) Sparse approximated nearest points for image set classification. In: IEEE conference on computer vision and pattern recognition, Colarado, Colombia, USA, pp 121–128

  • Kansizoglou I, Bampis L, Gasteratos A (2019) An active learning paradigm for online audio-visual emotion recognition. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2019.2961089

    Article  Google Scholar 

  • Kas M, El merabet Y, Ruichek Y, Messoussi R (2021) New framework for person-independent facial expression recognition combining textural and shape analysis through new feature extraction approach. Inf Sci 549:200–220

    MathSciNet  Google Scholar 

  • Li Z, Li R, Jin G (2020) Sentiment analysis of Danmaku videos based on Naïve Bayes and sentiment dictionary. IEEE Access 8:75073–75084

    Google Scholar 

  • Li Z, Wang C, Liu X et al (2021) Facial expression description and recognition based on fuzzy semantic concepts. Futur Gener Comput Syst 114:619–628

    Google Scholar 

  • Li S, Deng W (2020) Deep facial expression recognition: a survey, IEEE Trans Affect Comput. Article in Press

  • Liu J, Feng Y, Wang H (2021) Facial expression recognition using pose-guided face alignment and discriminative features based on deep learning. IEEE Access 9:69267–69277

    Google Scholar 

  • Lucey P, Jeffrey FC, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression, In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition-workshops, pp. 94-101

  • Lyons M, Budynek J, Akamatsu S (1999) Automatic classification of single facial images. IEEE Trans Pattern Anal Mach Intell 21:1357–1362

    Google Scholar 

  • Mahesh VGV, Chen C, Rajangam V, Raj ANJ, Krishnan PT (2021) Shape and texture aware facial expression recognition using spatial pyramid Zernike moments and law’s textures feature set. IEEE Access 9:52509–52522

    Google Scholar 

  • Meng D, Peng X, Wang K, Qiao Y (2019) Frame attention networks for facial expression recognition in videos. IEEE Int Conf Image Process (ICIP) 2019:3866–3870

    Google Scholar 

  • Minaee S, Minaei M, Abdolrashidi A (2021) Deep-emotion: facial expression recognition using attentional convolutional network. Sensors 21:3046

    Google Scholar 

  • Olague Gustavo, Olague Matthieu, Jacobo-Lopez Angel R, Ibarra-Vazquez Gerardo (2021) “Less is more pursuing the visual turing test with the Kuleshov effect,” In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) workshops, pp 1553–1561

  • Owusu E, Wiafe I (2021) An advance ensemble classification for object recognition. Neural Comput Appl 33:11661–11672

  • Perveen N, Roy D, Mohan CK (2018) Spontaneous expression recognition using universal attribute model. IEEE Trans Image Process 27:5575–5584

    MathSciNet  Google Scholar 

  • Poursaberi A, Noubari HA, Gavrilova M, Yanushkevich SN (2012) Gauss-Laguerre wavelet textural feature fusion with geometrical information for facial expression identification. EURASIP J Image Video Process 1:17

    Google Scholar 

  • Senechal T, Rapp V, Salam H, Seguier R, Bailly K, Prevost L (2012) Facial action recognition combining heterogeneous features via multikernel learning. IEEE Trans Syst Man Cybern Part B (Cybernetics) 42:993–1005

    Google Scholar 

  • Shan C, Gong S, Mcowan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27:803–816

    Google Scholar 

  • Shan C, Gong S, Mcowan PW (2005) Robust facial expression recognition using local binary patterns, In: Proceedings of the IEEE international conference on image processing, pp 914–917

  • Shi Z, Ye Y, Wu Y (2016) Rank-based pooling for deep convolutional neural networks. Neural Netw 83:21–31

    Google Scholar 

  • Siddiqi MH, Ali R, Khan AM et al (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE T Image Process 24:1386–1398

    MathSciNet  MATH  Google Scholar 

  • St A (2017) Emotion recognition: the influence of texture’s descriptors on classification accuracy, In: Beyond databases, architectures and structures. Towards efficient solutions for data analysis and knowledge representation: 13th international conference, BDAS 2017, Ustroń, Poland, May 30–June 2, pp 427–438

  • Sun Z, Hu Z-P, Wang M, Zhao S-H (2017) Discriminative feature learning-based pixel difference representation for facial expression recognition. IET Comput Vision 11:675–682

    Google Scholar 

  • Sun Z, Hu ZP, Chiong R, Wang M, He W (2018a) Combining the Kernel collaboration representation and deep subspace learning for facial expression recognition. J Circuits Syst Comput 27:1850121

    Google Scholar 

  • Sun Z, Chiong R, Hu ZP (2018b) An extended dictionary representation approach with deep subspace learning for facial expression recognition. Neurocomputing 316:1–9

    Google Scholar 

  • Sun Z, Hu Z, Zhao M (2019) Automatically query active features based on pixel-level for facial expression recognition. IEEE Access 7:104630–104641

    Google Scholar 

  • Sun Z, Chiong R, Hu Z-P (2020) Self-adaptive feature learning based on a priori knowledge for facial expression recognition, Knowledge-Based Systems, 204, 106124

  • Truong HP, Nguyen, TP, Kim YG (2021) Weighted statistical binary patterns for facial feature representation. Appl Intell

  • Turan C, Lam KM (2018) Histogram-based local descriptors for facial expression recognition (fer): a comprehensive study. J Visual Commun Image Represent 55:331–341

    Google Scholar 

  • Umer S, Rout RK, Pero C et al. (2021) Facial expression recognition with trade-offs between data augmentation and deep learning features. J Ambient Intell Humanized Comput. Article in Press

  • Wang R, Shan S, Chen X, Chen J, Gao W (2011) Maximal linear embedding for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 33:1776–1792

    Google Scholar 

  • Wang SJ, Yan WJ, Zhao G et al (2014) Micro-expression recognition using robust principle component analysis and local spatiotemporal directional features. Lect Notes Comput Sci 8925:325–338

    Google Scholar 

  • Waqas J, Yi Z, Lei Z (2013) Collaborative neighbor representation based classification using l2-minimization approach. Pattern Recogn Lett 34:201–208

    Google Scholar 

  • Wright J, Yang A, Ganesh A, Shastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227

    Google Scholar 

  • Wu B-F, Lin C-H (2018) Adaptive feature mapping for customizing deep learning based facial expression recognition model. IEEE Access 6:12451–12461

    Google Scholar 

  • Yan Q, Song N, Huang R (2015) Accurate and robust facial expressions recognition by fusing multiple sparse representation based classifiers. Neurocomputing 149:71–78

    Google Scholar 

  • Yang P, Liu Q, Cui X et al. (2007) Facial expression recognition using encoded dynamic features, In: IEEE international conference on multimedia and expo, Beijing, China, July 2-5, pp 1107–1110

  • Yang M, Zhang D, Wang S (2012) Relaxed collaborative representation for pattern classification, In: IEEE conference on computer vision & pattern recognition, pp 2224–2231

  • Yousaf A et al (2021) Emotion recognition by textual tweets classification using voting classifier (LR-SGD). IEEE Access 9(2021):6286–6295

    Google Scholar 

  • Yu M, Zheng H, Peng Z, Dong J, Du H (2020) Facial expression recognition based on a multi-task global-local network. Pattern Recognit Lett 131:166–171

    Google Scholar 

  • Zavarez MV, Berriel RF, Oliveira-Santos T (2017) Cross-database facial expression recognition based on fine-tuned deep convolutional network, In: IEEE 30th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), pp 405–412

  • Zeng N, Zhang H, Song B, Liu W et al (2018) Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273:643–649

    Google Scholar 

  • Zhang H, Nasrabadi NM, Zhang Y, Huang TS (2012) Multi-view automatic target recognition using joint sparse representation. IEEE Trans Aerosp Electron Syst 48:2481–2497

    Google Scholar 

  • Zhang L, Yang M, Feng XC (2011) Sparse representation or collaborative representation which helps face recognition? In ICCV

  • Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expression. IEEE Trans Pattern Anal Mach Intell 29:915–928

    Google Scholar 

  • Zhu W, Peng B (2020) Sparse and low-rank regularized deep subspace clustering, Knowledge-Based Systems, Volume 204, 106199, ISSN 0950-7051

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grants 62001413 and 61771420, the Natural Science Foundation of Hebei Province under Grants F2020203064, and the Doctoral Foundation of Yanshan University under Grant BL18033.

Author information

Authors and Affiliations

  1. Department of Information Science and Engineering, Yanshan University, Qinhuangdao, 066000, Hebei, China

    Zhe Sun, Hehao Zhang, Suwei Ma & Zhengping Hu

Authors
  1. Zhe Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hehao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Suwei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengping Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengping Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Z., Zhang, H., Ma, S. et al. Combining filtered dictionary representation based deep subspace filter learning with a discriminative classification criterion for facial expression recognition. Artif Intell Rev 55, 6547–6566 (2022). https://doi.org/10.1007/s10462-022-10160-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-022-10160-1

Keywords

Search

Navigation