Abstract
Facial emotion recognition has been extensively studied over the last decade due to its various applications in the fields such as human-computer interaction and data analytics. In this paper, we develop a facial emotion recognition approach to classify seven emotional states (joy, sadness, surprise, anger, fear, disgust and neutral). Seventeen action units tracked by Kinect v2 sensor have been used as features. Classification of emotions was performed by artificial neural networks (ANNs). Six subjects took part in the experiment. We have achieved average accuracy of 95.8% for the case in which we tested our approach with the same volunteers took part in our data generation process. We also evaluated the performance of the network with additional volunteers who were not part of the training data and achieved 67.03% classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Mehrabian, A.: Communication without words. In: Mortensen, C.D. (ed.) Communication Theory, pp. 193–200. Transaction Publishers, New Brunswick (2008)
Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologists Press, Stanford University, Palo Alto (1977)
Wang, P., Barrett, F., Martin, E., Milonova, M., Gur, R.E., Gur, R.C., Kohler, C., Verma, R.: Automated video-based facial expression analysis of neuropsychiatric disorders. J. Neurosci. Methods 168(1), 224–238 (2008)
Tong, Y., Chen, R., Cheng, Y.: Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle. Optik-Int. J. Light Electron. Optics 125, 4186–4189 (2014)
Jabid, T., Kabir, M.H., Chae, O.: Facial expression recognition using local directional pattern (IDP). In: 17th IEEE International Conference on Image Processing (ICIP). IEEE (2010)
Guo, Y., Tian, Y., Gao, X., Zhang, X.: Micro-expression recognition based on local binary patterns from three orthogonal planes and nearest neighbor method. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2014)
Gizatdinova, Y., Surakka, V., Zhao, G., Makinen, E., Raisamo, R.: Facial expression classification based on local spatiotemporal edge and texture descriptors. In: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research. ACM (2010)
Kabir, M.H., Jabid, T., Chae, O.: Local directional pattern variance (IDPV): a robust feature descriptor for facial expression recognition. Int. Arab. J. Inf. Technol. 9(4), 382–391 (2012)
Mao, Q., Pan, X., Zhan, Y., Shen, X.: Using Kinect for real-time emotion recognition via facial expressions. Front. Inf. Technol. Electron. Eng. 16(4), 272–282 (2015)
Youssef, A.E., Aly, S.F., Ibrahim, A.S., Abbott, A.L.: Auto-optimized multimodal expression recognition framework using 3D kinect data for ASD therapeutic aid. Int. J. Model. Optim. 3(2), 112–115 (2013)
Zhang, Z., Cui, L., Liu, X., Zhu, T.: Emotion detection using Kinect 3D facial points. In: International Conference on Web Intelligence (2016)
Tarnowski, P., Kolodziej, M., Majkowkski, A., Rak, R.: Emotion recognition using facial expressions. In: International Conference on Computer Science, ICCS (2017)
Microsoft SDK for Face Tracking. http://msdn.microsoft.com/enus/library/jj130970.aspx
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Yağış, E., Unel, M. (2018). Facial Expression Based Emotion Recognition Using Neural Networks. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds) Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science(), vol 10882. Springer, Cham. https://doi.org/10.1007/978-3-319-93000-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-93000-8_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92999-6
Online ISBN: 978-3-319-93000-8
eBook Packages: Computer ScienceComputer Science (R0)