Abstract
Reshaping faces is an interesting thing and a common demand in today’s media industry and cosmetic industry. Especially, the reshaped face images contribute a lot to special effects industry and plastic-beauty industry. In this paper, we provide a novel parametric representation of 3D face model that describes the face shape as a linear combination of semantic and non-semantic bases. The semantic (local) bases correspond to each individual face part and they are used to semantically edit each face part, while the non-semantic bases explain other shape variations to span the subspace of the face shape. First, we build a sparse and spatially localized parametric face model from a dataset of 3D face models by sparse principal component analysis. Second, to define the semantic bases, we train a regression model to correlate semantically significant values like nose height and mouth width. Finally, the bases of the resulting parametric face model is orthogonalized to all defined semantic bases by the Gram–Schmidt algorithm for generating the novel parametric representation of 3D face model. The novel representation can be applied in reshaping face in the images. The experiment results demonstrate that our representation not only retains the accuracy for 3D face reconstruction but also provides users a user-friendly tool to edit facial parts for desired facial shapes.
Graphical abstract
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alvarez FJA, Parra EBB, Tubio FM (2017) Improving graphic expression training with 3D models. J Vis 20(4):889–904
Bach F, Jenatton R, Mairal J, Obozinski G (2012) Optimization with sparsity-inducing penalties. Found Trends Mach Learn 4(1):1–106
Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th annual conference on computer graphics and interactive techniques. ACM, pp 187–194
Blanz V, Basso C, Poggio T, Vetter T (2003) Reanimating faces in images and video. Comput Gr Forum (Blackwell Publishing, Inc) 22(3):641–650
Blanz V, Scherbaum K, Vetter T, Seidel H-P (2004) Exchanging faces in images. Comput Gr Forum (Blackwell Publishing, Inc) 23(3):669–676
Cao C, Weng Y, Zhou S, Tong Y, Zhou K (2014) Facewarehouse: a 3d facial expression database for visual computing. Vis Comput Gr IEEE Trans 20(3):413–425
Chalas I, Urbanova P, Jurik V, Ferkova Z, Jandova M, Sochor J, Kozlikova B (2017) Generating various composite human faces from real 3D facial images. V Comput 33(4):443–458
Chou J-K, Yang C-K, Gong S-D (2012) Face-off: automatic alteration of facial features. Multimed Tools Appl 56(3):569–596
Fan X, Chai Z, Feng Y, Wang Y, Wang S, Lou Z (2016) An efficient mesh-based face beautifier on mobile devices. Neurocomputing. 172:134–142
Golub GH, Vanloan CF (1996) Matrix computations (Johns Hopkins studies in mathematical sciences). The Johns Hopkins University Press, Baltimore
Guo Y, Zhang J, Cai J, Jiang B, Zheng J (2017) 3DFaceNet: real-time dense face reconstruction via synthesizing photo-realistic face images. arXiv:1708.00980
Hasler N, Stoll C, Sunkel M, Rosenhahn B, Seidel H-P (2009) A statistical model of human pose and body shape. Comput Gr Forum 28(2):337–346
Kang D, Tian F, Sahandi R (2017) Interactive illustration of collage for children with folktale e-book. J Vis 20(3):639–650
Kavan L, Sloan PP, O’Sullivan C (2010) Fast and efficient skinning of animated meshes. Comput Gr Forum (Blackwell Publishing Ltd) 29(2):327–336
Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1867–1874
Kemelmacher-Shlizerman I (2016) Transfiguring portraits. ACM Trans Gr (TOG) 35(4):94 (1–94:8)
Klum S, Han H, Jain AK, Klara B (2013) Sketch based face recognition: forensic vs. composite sketches. In: Proceedings of the 6th international conference on biometrics. IEEE, pp 1–8
Lewis JP, Anjyo K, Rhee T, Zhang M, Pighin FH, Deng Z (2014) Practice and theory of blendshape facial models. Eurographics (State Art Rep) 1(8)
Leyvand T, Cohen-Or D, Dror G, Lischinski D (2008) Data-driven enhancement of facial attractiveness. ACM Trans Gr (TOG) 27(3):38
Li H, Yu J, Ye Y, Bregler C (2013) Realtime facial animation with on-the-fly correctives. ACM Trans Gr (TOG) 32(4):42 (1–42:10)
Liao Q, Jin X, Zeng W (2012) Enhancing the symmetry and proportion of 3D face geometry. Vis Comput Gr IEEE Trans 18(10):1704–1716
Lu J, Sunkavalli K, Carr N, Hadap S, Forsyth D (2016) A visual representation for editing face images. arXiv:1612.00522
Lu Z, Yang J, Liu Q (2017) Face image retrieval based on shape and texture feature fusion. Comput Visual Media 3(4):359–368
Ma DS, Correll J, Wittenbrink B (2015) The Chicago face database: a free stimulus set of faces and norming data. Behav Res Methods 47(4):1122–1135
Mansouri S, Ebrahimnezhad H (2016) Segmentation-based semi-regular remeshing of 3D models using curvature-adapted subdivision surface fitting. J Vis 19(1):141–155
Neumann T, Varanasi K, Wenger S, Wacker M, Magnor M, Theobalt C (2013) Sparse localized deformation components. ACM Trans Gr (TOG) 32(6):179
Nirkin Y, Masi I, Tran AT, Hassner T, Medioni G (2017) On face segmentation, face swapping, and face perception. arXiv:1704.06729
Paysan P, Knothe R, Amberg B (2009) A 3D face model for pose and illumination invariant face recognition. In: 6th IEEE international conference on advanced video and signal based surveillance (AVSS’09), IEEE, Genova, Italy, pp 296–301
Ramamoorthi R, Hanrahan P (2001) An efficient representation for irradiance environment maps. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques. ACM, pp 497–500
Shamir A, Sorkine O (2009) Visual media retargeting. ACM SIGGRAPH ASIA 2009 courses. ACM, p 11
Streuber S, Quiros-Ramirez MA, Hill MQ, Hahn CA, Zuffi S, O’Toole A, Black MJ (2016) Body talk: crowdshaping realistic 3D avatars with words. ACM Trans Gr (TOG) 35(4):54
Suryanarayana GK, Dubey R (2017) Image analyses of supersonic air-intake buzz and control by natural ventilation. J Vis 1–17
Thies J, Zollhofer M, Stamminger M, Theobalt C, Niebner M (2016) Face2face: real-time face capture and reenactment of RGB videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 2387–2395
Tran AT, Hassner T, Masi I, Medioni G (2016) Regressing robust and discriminative 3D morphable models with a very deep neural network. arXiv:1612.04904
Vlasic D, Brand M, Pfister H, Popovic J (2005) Face transfer with multilinear models. ACM Trans Gr (TOG) 24(3):426–433
Zhang Y, Lin W, Zhou B, Chen Z, Sheng B, Wu J (2014) Facial expression cloning with elastic and muscle models. J Vis Commun Image Represent 25(5):916–927
Zhou S, Fu H, Liu L, Cohen-Or D, Han X (2010) Parametric reshaping of human bodies in images. ACM Trans Gr (TOG) 29(4):126
Acknowledgements
The research is supported in part by NSFC (61572424) and the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme FP7 (2007-2013) under REA Grant Agreement no. 612627-”AniNex”. Min Tang is supported in part by NSFC (61572423) and Zhejiang Provincial NSFC (LZ16F020003).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Du, J., Song, D., Tang, Y. et al. “Edutainment 2017” a visual and semantic representation of 3D face model for reshaping face in images. J Vis 21, 649–660 (2018). https://doi.org/10.1007/s12650-018-0476-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-018-0476-4