Abstract
In this paper we present a method to derive 3D shape and surface texture of a human face from a single image. The method draws on a general flexible 3D face model which is “learned” from examples of individual 3D-face data (Cyberware-scans). In an analysis-by-synthesis loop, the flexible model is matched to the novel face image.
From the coloured 3D model obtained by this procedure, we can generate new images of the face across changes in viewpoint and illumination. Moreover, nonrigid transformations which are represented within the flexible model can be applied, for example changes in facial expression.
The key problem for generating a flexible face model is the computation of dense correspondence between all given 3D example faces. A new correspondence algorithm is described which is a generalization of common algorithms for optic flow computation to 3D-face data.
Chapter PDF
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
C. Choi, T. Okazaki, H. Harashima, and T. Takebe, “A system of analyzing and synthesizing facial images,” in Proc. IEEE Int. Symposium of Circuit and Syatems (ISCAS91), pp. 2665–2668, 1991.
D. Beymer, A. Shashua, and T. Poggio, “Example-based image analysis and synthesis,” A.I. Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1993.
A. Lanitis, C. Taylor, T. Cootes, and T. Ahmad, “Automatic interpretation of human faces and hand gestures using flexible models,” in Proc. International Workshop on Face and Gesture Recognition (M.Bichsel, ed.), (Zürich, Switzerland), pp. 98–103, 1995.
T. Vetter and T. Poggio, “Linear objectclasses and image synthesis from a single example image,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 733–742, 1997.
D. Beymer and T. Poggio, “Image representation for visual learning,” Science, vol. 272, pp. 1905–1909, 1996.
F. Parke, “A parametric model of human faces,” doctoral thesis, University of Utah, Salt Lake City, 1974.
T. Akimoto, Y. Suenaga, and R. Wallace, “Automatic creation of 3D facial models,” IEEE Computer Graphics and Applications, vol. 13, no. 3, pp. 16–22, 1993.
J. Barron, D. Fleet, and S. Beauchemin, “Performance of optical flow techniques,” Int. Journal of Computer Vision, pp. 43–77, 1994.
J. Bergen, P. Anandan, K. Hanna, and R. Hingorani, “Hierarchical model-based motion estimation,” in Proceedings of the European Conference on Computer Vision, (Santa Margherita Ligure, Italy), pp. 237–252, 1992.
J. Bergen and R. Hingorani, “Hierarchical motion-based frame rate conversion,” technical report, David Sarnoff Research Center Princeton NJ 08540, 1990.
P. Burt and E. Adelson, “The Laplacian pyramide as a compact image code,” IEEE Transactions on Communications, no. 31, pp. 532–540, 1983.
B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artificial Intelligence, vol. 17, pp. 185–203, 1981.
E. Hildreth, The Measurement of Visual Motion. Cambridge: MIT Press, 1983.
H. H. Nagel, “Displacement vectors derived from second order intensity variations in image sequences.,” Computer Vision, Graphics and Image Processing, vol. 21, pp. 85–117, 1983.
M. Jones and T. Poggio, “Model-based matching by linear combination of prototypes,” a.i. memo no., Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1996.
T. Vetter, M. J. Jones, and T. Poggio, “A bootstrapping algorithm for learning linear models of object classes,” in IEEE Conference on Computer Vision and Pattern Recognition — CVPR'97, (Puerto Rico, USA), IEEE Computer Society Press, 1997.
P. Viola, “Alignment by maximization of mutual information,” A.I. Memo No. 1548, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1995.
T. Vetter, “Synthestis of novel views from a single face image,” International Journal of Computer Vision, no. in press.
P. Burt and E. Adelson, “Merging images through pattern decomposition,” in Applications of Digital Image Processing VIII, no. 575, pp. 173–181, SPIE The International Society for Optical Engeneering, 1985.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vetter, T., Blanz, V. (1998). Estimating coloured 3D face models from single images: An example based approach. In: Burkhardt, H., Neumann, B. (eds) Computer Vision — ECCV’98. ECCV 1998. Lecture Notes in Computer Science, vol 1407. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0054761
Download citation
DOI: https://doi.org/10.1007/BFb0054761
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64613-6
Online ISBN: 978-3-540-69235-5
eBook Packages: Springer Book Archive