Abstract
Synthesizing realistic multi-view face images from a single-view input is an effective and cheap way for data augmentation. In addition it is promising for more efficiently training deep pose-invariant models for large-scale unconstrained face recognition. It is a challenging generative learning problem due to the large pose discrepancy between the synthetic and real face images, and the need to preserve identity after generation. We propose IP-GAN, a framework based on Generative Adversarial Networks to disentangle the identity and pose of faces, such that we can generate face images of a specific person with a variety of poses, or images of different identities with a particular pose. To rotate a face, our framework requires one input image of that person to produce an identity vector, and any other input face image to extract a pose embedding vector. Then we recombine the identity vector and the pose vector to synthesize a new face of the person with the extracted pose. Two learning pathways are introduced, the generation and the transformation, where the generation path focuses on learning complete representation in the latent embedding space. While the transformation path focuses on synthesis of new face images with target poses. They collaborate and compete in a parameter-sharing manner, and in an unsupervised settings. The experimental results demonstrate the effectiveness of the proposed framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. In: IEEE FG (2018). https://doi.org/10.1109/fg.2018.00020
Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_6
Chen, D., Cao, X., Wen, F., Sun, J.: Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. https://doi.org/10.1109/cvpr.2013.389
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR (2015). https://doi.org/10.1109/cvpr.2015.7298682
Cao, K., Rong, Y., Li, C., Tang, X., Loy, C.C.: Pose-Robust face recognition via deep residual equivariant mapping. In: CVPR (2018). https://doi.org/10.1109/cvpr.2018.00544
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH 1999. ACM Press (1999). https://doi.org/10.1145/311535.311556
Huang, R., Zhang, S., Li, T., He, R.: Beyond face rotation: global and local perception GAN for Photorealistic and identity preserving frontal view synthesis. In: ICCV (2017). https://doi.org/10.1109/iccv.2017.267
Yin, X., Yu, X., Sohn, K., Liu, X., Chandraker, M.: Towards Large-pose face frontalization in the wild. In: ICCV (2017). https://doi.org/10.1109/iccv.2017.430
Tran, L., Yin, X., Liu, X.: Disentangled representation learning GAN for pose-invariant face recognition. In: CVPR (2017). https://doi.org/10.1109/cvpr.2017.141
Tian, Y., Peng, X., Zhao, L., Zhang, S., Metaxas, D.N.: CR-GAN: learning complete representations for multi-view generation (2018). https://doi.org/10.24963/ijcai.2018/131
Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS (2014). https://arxiv.org/abs/1406.2661
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015). https://arxiv.org/abs/1409.1556
Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-PIE. Image Vision Comput. 28, 807–813 (2010). https://doi.org/10.1016/j.imavis.2009.08.002
Huang, X., Liu, M.-Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 179–196. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_11
Wu, X., He, R., Sun, Z., Tan, T.: A light CNN for deep face representation with noisy labels (2018). https://doi.org/10.1109/tifs.2018.2833032
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE (2008). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.457.7213
Acknowledgements
This work was financially supported by the Government of the Russian Federation (Grant 08-08).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zeno, B., Kalinovskiy, I., Matveev, Y. (2019). IP-GAN: Learning Identity and Pose Disentanglement in Generative Adversarial Networks. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Workshop and Special Sessions. ICANN 2019. Lecture Notes in Computer Science(), vol 11731. Springer, Cham. https://doi.org/10.1007/978-3-030-30493-5_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-30493-5_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30492-8
Online ISBN: 978-3-030-30493-5
eBook Packages: Computer ScienceComputer Science (R0)