Abstract
Model-based face analysis is a general paradigm with applications that include face recognition, expression recognition, lip-reading, head pose estimation, and gaze estimation. A face model is first constructed from a collection of training data, either 2D images or 3D range scans. The face model is then fit to the input image(s) and the model parameters used in whatever the application is. Most existing face models can be classified as either 2D (e.g. Active Appearance Models) or 3D (e.g. Morphable Models). In this paper we compare 2D and 3D face models along three axes: (1) representational power, (2) construction, and (3) real-time fitting. For each axis in turn, we outline the differences that result from using a 2D or a 3D face model.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Baker, S., Gross, R., and Matthews, I. 2003. Lucas-Kanade 20 years on: A unifying framework: Part 3. Technical Report CMU-RI-TR-03-35, Carnegie Mellon University Robotics Institute.
Baker, S., Gross, R., and Matthews. I. 2004. Lucas-Kanade 20 years on: Part 4. Technical Report CMU-RI-TR-04-14, Carnegie Mellon University Robotics Institute.
Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1090–1097.
Baker, S. and Matthews, I. 2004. Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56:221–255.
Baker, S., Patil, R., Cheung, K.M., and Matthews, I. 2004. Lucas-Kanade 20 years on: Part 5. Technical Report CMU-RI-TR-04-64, Carnegie Mellon University Robotics Institute.
Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. In Proceedings of EUROGRAPHICS.
Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Computer Graphics, Annual Conference Series (SIGGRAPH), pp. 187–194.
Brand, M. 2001. Morphable 3D models from video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 456–463.
Bregler, C., Hertzmann, A., and Biermann, H. 2000. Recovering non-rigid 3D shape from image streams. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Cootes, T., Edwards, G., and Taylor, C. 2001 Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):681–685.
Gross, R., Matthews, I., and Baker, S. 2004. Constructing and fitting active appearance models with occlusion. In Proceedings of the IEEE Workshop on Face Processing in Video, pp. 72–79.
Hager, G. and Belhumeur, P. 1998. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025–1039.
Ishikawa, T., Baker, S., Matthews, I., and Kanade, T. 2004. Passive driver gaze tracking with active appearance models. In Proceedings of the 11th World Congress on Intelligent Transportation Systems, URL: http://www.ri.cmu.edu/pubs/pub_4705.html.
Kanade, T. 1973. Picture Processing System by Computer Complex and Recognition of Human Faces. PhD thesis, Kyoto University, November.
Koterba, S., Baker, S., Matthews, I., Hu, C., Xiao, J., Cohn, J., and Kanade. T. 2005. Multi-view AAM fitting and camera calibration. In Proceedings of the IEEE International Conference on Computer Vision, pp. 511–518.
Lanitis, A., Taylor, C., and Cootes, T. 1997. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742–756.
Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679.
Matthews, I. and Baker, S. 2004. Active Appearance Models revisited. International Journal of Computer Vision, 60(2):135–164.
Matthews, I., Cootes, T., Bangham, A., Cox, S., and Harvery, R. 2002. Extraction of visual features for lipreading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2):198–213.
Matthews, I., Xiao, I., and Baker, S. 2006. On the dimensionality of deformable face models. Technical Report CMU-RI-TR-06-12, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.
Romdhani, S. and Vetter, T. 2003. Efficient, robust and accurate fitting of a 3D morphable model. In Proceedings of the International Conference on Computer Vision, 1:59–55.
Rowley, H.A., Baluja, S., and Kanade, T. 1998. Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38.
Shum, H.Y. and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1):63–84.
Tian, Y., Kanade, T., and Cohn, J. 2001. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97–115.
Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137–154.
Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. Tracking and modeling non-rigid objects with rank constraints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1:493–500.
Xiao, J., Baker, S., Matthews, I., and Kanade, T. 2004a. Real-time combined 2D Plus; 3D Active Appearance Models}. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 535–542.
Xiao, J., Chai, J., and Kanade, T. 2004b. A closed-form solution to non-rigid shape and motion recovery. In Proceedings of the European Conference on Computer Vision, Vol. 4, pp. 573–587.
Xiao, J., Chai, J., and Kanade, T. 2006. A closed-form solution to non-rigid shape and motion recovery. International Journal of Computer Vision, 67(2):233–246.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Matthews, I., Xiao, J. & Baker, S. 2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting. Int J Comput Vis 75, 93–113 (2007). https://doi.org/10.1007/s11263-007-0043-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0043-2