Abstract
Face recognition has becoming an important and popular authentication technology for web services and mobile applications in recent years. The quality of facial obstruction removal is a critical component of face recognition, especially for mission critical applications such as facial recognition based authentication systems. It is well known that some facial obstructions may severely affect the extraction and recognition quality and accuracy of facial features, which in turn disturbs the prediction accuracy of facial recognition model and algorithms. In this paper, we propose a Facial Obstructions Removal Scheme (FORS) based on an Enhanced Cycle-Consistent Generative Adversarial Networks (ECGAN) for face recognition. By training a convolution neural network based facial image classifier, we identify those images that contain facial obstructions. Then the images with facial obstructions are processed by using the facial image converter of FORS and the ECGAN model, which removes facial obstructions seamlessly while preserving the facial features. Our experimental results show that the proposed FORS scheme improves the face recognition accuracy over some existing state of art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Phillips, P.J., Wechsler, H., Huang, J., Rauss, P.J.: The feret database and evaluation procedure for face-recognition algorithms. Image Vis. Comput. 16, 295–306 (1998)
Phillips, P.J., Grother, P.J., Micheals, R.J., Blackburn, D.M., Tabassi, E., Bone, M.: Face recognition vendor test 2002: evaluation report. Technical report 6965, NIST Interagency/Internal Report (NISTIR) (2003)
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Yang, S., Luo, P., Loy, C.C., Tang, X.: From facial parts responses to face detection: a deep learning approach. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3676–3684 (2015)
Saito, Y., Kenmochi, Y., Kotani, K.: Estimation of eyeglassless facial images using principal component analysis. In: Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348), vol. 4, pp. 197–201 (1999)
Guo, P., Su, F.: Enhanced PCA reconstruction method for eyeglass frame auto-removal. In: 2014 4th IEEE International Conference on Network Infrastructure and Digital Content, pp. 359–363, September 2014
Wang, Y.K., Jang, J.H., Tsai, L.W., Fan, K.C.: Improvement of face recognition by eyeglass removal. In: 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 228–231, October 2010
Wu, C., Liu, C., Shum, H.Y., Xy, Y.Q., Zhang, Z.: Automatic eyeglasses removal from face images. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 322–336 (2004)
Cheng, W.C., Liao, H.C., Pan, M.H., Chen, C.C.: A fatigue detection system with eyeglasses removal. In: 2013 15th International Conference on Advanced Communication Technology (ICACT), pp. 331–335. IEEE (2013)
Chen, W., Wang, B.: Methods of eyeglasses detection and frame removal for face image. Comput. Eng. Appl. 15, 178–182 (2016)
Yi, D., Li, S.Z.: Learning sparse feature for eyeglasses problem in face recognition. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 430–435, March 2011
Fernández, A., García, R., Usamentiaga, R., Casado, R.: Glasses detection on real images based on robust alignment. Mach. Vis. Appl. 26(4), 519–531 (2015)
Liang, M., Xue, Y., Xue, K., Yang, A.: Deep convolution neural networks for automatic eyeglasses removal. DEStech Transactions on Computer Science and Engineering (AIEA) (2017)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks (2017). arXiv preprint arXiv:1703.10593
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 2672–2680. Curran Associates, Inc. (2014)
Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821. IEEE (2017)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: Advances in Neural Information Processing Systems, pp. 2234–2242 (2016)
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint (2016)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)
Acknowledgement
The authors from Huazhong University of Science and Technology, Wuhan, China, are supported by the Chinese university Social sciences Data Center (CSDC) construction projects (2017–2018) from the Ministry of Education, China. The first author, Dr. Yuming Wang, is currently a visiting scholar at the School of Computer Science, Georgia Institute of Technology, funded by China Scholarship Council (CSC) for the visiting period of one year from December 2017 to December 2018. Prof. Ling Liu’s research is partially supported by the USA National Science Foundation CISE grant 1564097 and an IBM faculty award. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Wang, Y., Ou, X., Tu, L., Liu, L. (2018). Effective Facial Obstructions Removal with Enhanced Cycle-Consistent Generative Adversarial Networks. In: Aiello, M., Yang, Y., Zou, Y., Zhang, LJ. (eds) Artificial Intelligence and Mobile Services – AIMS 2018. AIMS 2018. Lecture Notes in Computer Science(), vol 10970. Springer, Cham. https://doi.org/10.1007/978-3-319-94361-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-94361-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94360-2
Online ISBN: 978-3-319-94361-9
eBook Packages: Computer ScienceComputer Science (R0)