[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Multiple classifiers fusion for facial expression recognition

  • Original Paper
  • Published:
Granular Computing Aims and scope

Abstract

Human facial expression recognition has been treated as a multi-class classification problem in the field of artificial intelligence. The main difficulty lies in how to distinguish the different categories of expression features. In this paper, we identify common facial expressions by fusing multiple weak classifiers. It compensates for the disadvantage of single classifier in weak generalization ability and low recognition rate for different datasets and different environments. This paper integrates the prediction results of each classifier through improved weighted mean value method and proposes an expression feature extraction method based on keypoint detection. Classifier fusion methods enable each classifier to perform at its best in order to improve overall expression recognition. Keypoint detection is used to improve the model’s attention on the expression features. Convolution neural network is selected as the model for feature extraction and classification, and the model structure is adjusted. Experiments show that the recognition accuracy of this method used on datasets FER 2013 and CK+ are 70.7% and 95.4% respectively, which are better than that of a single classifier, which shows that the keypoint extraction feature and classifier fusion method used in this paper have a good effect on facial expression recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Bera S, Roy SK (2020) Fuzzy rough soft set and its application to lattice. Granular Comput 5(2):217–223

    Article  Google Scholar 

  • Dange AD, Momin B (2019) The cnn and dpm based approach for multiple object detection in images. In: 2019 International conference on intelligent computing and control systems (ICCS), IEEE, pp 1106–1109

  • Egrioglu E, Yolcu U, Bas E (2019) Intuitionistic high-order fuzzy time series forecasting method based on pi-sigma artificial neural networks trained by artificial bee colony. Granular Comput 4(4):639–654

    Article  Google Scholar 

  • Ejegwa PA (2020) Generalized triparametric correlation coefficient for pythagorean fuzzy sets with application to mcdm problems. Granular Comput (1–2)

  • Ekman P, Friesen W (1978) A technique for the measurement of facial actions. Rivista DI Psichiatria 47(2):126–138

    Google Scholar 

  • Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:170404861

  • Kido S, Hirano Y, Hashimoto N (2018) Detection and classification of lung abnormalities by use of convolutional neural network (cnn) and regions with cnn features (r-cnn). In: 2018 International workshop on advanced image technology (IWAIT), IEEE, pp 1–4

  • Kong Q, Zhang X, Xu W (2019) Operation properties and algebraic properties of multi-covering rough sets. Granular Comput 4(3):377–390

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  • Lecun Y, Bottou L (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  • Liang M, Mi J, Feng T (2019) Optimal granulation selection for multi-label data based on multi-granulation rough sets. Granular Comput 4(3):323–335

    Article  Google Scholar 

  • Lin JT (1997) Granular computing. Announcement of the BISC Special Interest Group on Granular Computing

  • Mehrabian A, Russell JA (1974) An approach to environmental psychology. The MIT Press, Chicago

    Google Scholar 

  • Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1–10

  • Pawlak Z (1982) Rough sets. Int J Comput Inform Sci 11(5):341–356

    Article  Google Scholar 

  • Shi D, Zhang X (2019) Probabilistic decision making based on rough sets in interval-valued fuzzy information systems. Granular Comput 4(3):391–405

    Article  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:14091556

  • Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405

    Article  Google Scholar 

  • Wu F, Yan S, Smith JS, Zhang B (2019) Deep multiple classifier fusion for traffic scene recognition. Granular Comput pp 1–12

  • Xu LL, Zhang SM, Zhao JL (2019) Expression recognition algorithm for parallel convolutional neural network. J Image Graph 24:227–236

    Google Scholar 

  • Yang B, Cao JM, Jiang DP, Lv JD (2018) Facial expression recognition based on dual-feature fusion and improved random forest classifier. Multimed Tool Appl 77:20477–20499

    Article  Google Scholar 

  • Yao J (2005a) Information granulation and granular relationships. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 326–329

  • Yao JT, Vasilakos AV, Pedrycz W (2013) Granular computing: perspectives and challenges. IEEE Trans Cybern 43(6):1977–1989

    Article  Google Scholar 

  • Yao MZ, Huang GW (2020) Facial expression recognition based on convolutional neural network. Comput Knowl Techol 16(16):19–23

    Google Scholar 

  • Yao Y (2005b) Perspectives of granular computing. In: 2005 IEEE international conference on granular computing, IEEE, vol 1, pp 85–90

  • Yurtkan K, Demirel H (2014) Entropy-based feature selection for improved 3d facial expression recognition. SIViP 8(2):267–277

    Article  Google Scholar 

  • Zadeh LA (1965) Fuzzy sets. Inf. Control 8(3):338–353

    Article  MathSciNet  Google Scholar 

  • Zadeh LA (1979) Fuzzy sets and information granularity. Adv Fuzzy Set Theory Appl 11:3–18

    MathSciNet  Google Scholar 

  • Zadeh LA (1996) Key roles of information granulation and fuzzy logic in human reasoning, concept formulation and computing with words. In: Proceedings of IEEE 5th international fuzzy systems, IEEE, vol 1, pp 1–1

  • Zadeh LA (1998) Some reflections on soft computing, granular computing and their roles in the conception, design and utilization of information/intelligent systems. Soft Comput 2(1):23–25

    Article  Google Scholar 

  • Zhang W, Wang X, Yang X, Chen X, Wang P (2019) Neighborhood attribute reduction for imbalanced data. Granular Comput 4(3):301–311

    Article  Google Scholar 

  • Zhao H, Liu H (2020) Multiple classifiers fusion and cnn feature extraction for handwritten digits recognition. Granular Comput 5(3):411–418

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is supported by ‘Chenguang Program’ supported by Shanghai Education Development Fo-undation and Shanghai Municipal Education Commission under grant number 18CG54. Furthermore, this work is also sponsored by Project funded by China Postdoctoral Science Foundation under Grant Number 2019M651576, National Natural Science Foundation of China (CN) under Grant Number 61602296, Natural Science Foundation of Shanghai (CN) Under Grant Number 16ZR1414500. The authors would like to thank their supports.

Author information

Authors and Affiliations

  1. College of Information Engineering, Shanghai Maritime University, Shanghai, 201306, China

    Chuanjie Zhang & Changming Zhu

Authors
  1. Chuanjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changming Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Zhu, C. Multiple classifiers fusion for facial expression recognition. Granul. Comput. 7, 171–181 (2022). https://doi.org/10.1007/s41066-021-00258-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41066-021-00258-2

Keywords

Search

Navigation