More Web Proxy on the site http://driver.im/

research-article

High-quality single-shot capture of facial geometry

Authors:

Paul Beardsley,

Markus GrossAuthors Info & Claims

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

Article No.: 40, Pages 1 - 9

https://doi.org/10.1145/1833349.1778777

Published: 26 July 2010 Publication History

Abstract

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography.

Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.

Supplementary Material

Supplemental material. (040.zip)

Download
164.97 MB

References

[1]

Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The Digital Emily Project: Photoreal facial modeling and animation. ACM Trans. Graph.

[2]

Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single shot capture of facial geometry: Implementation details. Tech. Rep. 671, ETH Zurich.

[3]

Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph.

Digital Library

[4]

Chen, T., Goesele, M., and Seidel, H. 2006. Mesostructure from specularity. CVPR.

Digital Library

[5]

DI3D. 2009. Dimensional imaging. http://www.di3d.com.

[6]

Donner, C., and Jensen, H. 2006. A spectral bssrdf for shading human skin. Eurographics Symposium on Rendering.

[7]

Donner, C., Weyrich, T., d'Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. 2008. A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Trans. Graph.

Digital Library

[8]

Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. CVPR.

[9]

Glencross, M., Ward, G., Melendez, F., Jay, C., Liu, J., and Hubbold, R. 2008. A perceptually validated model for surface depth hallucination. ACM Trans. Graph.

Digital Library

[10]

Golovinskiy, A., Matusik, W., Pfister, H., Rusinkiewicz, S., and Funkhouser, T. 2006. A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph.

Digital Library

[11]

Hartley, R., and Zisserman, A. 2000. Multiple View Geometry, second ed. Cambridge University Press.

Digital Library

[12]

Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Shadows in three-source photometric stereo. ECCV.

Digital Library

[13]

Hiep, V., Keriven, R., Labatut, P., and Pons, J. 2009. Towards high-resolution large-scale multi-view stereo. CVPR.

[14]

Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. SIGGRAPH 2005 Courses.

Digital Library

[15]

Intel. 2001. Opencv reference manual. http://developer.intel.com.

[16]

Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In SGP.

Digital Library

[17]

Ma, W., Hawkins, T., Peers, P., Chabert, C., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques.

Digital Library

[18]

Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J., Yang, R., Nister, D., and Pollefeys, M. 2007. Real-time visibility-based fusion of depth maps. ICCV.

[19]

Meyer, M., Desbrun, M., Schröder, P., and Barr, A. H. 2003. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III.

[20]

Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph.

Digital Library

[21]

Parke, F. 1974. A parametric model for human faces. PhD Thesis, University of Utah.

Digital Library

[22]

Pighin, F., and Lewis, J. 2005. Digital face cloning. ACM Trans. Graph.

[23]

Robert, L., and Deriche, R. 1996. Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. In ECCV.

Digital Library

[24]

Scharstein, D., and Szeliski, R. 1996. Stereo matching with non-linear diffusion. CVPR.

Digital Library

[25]

Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV.

Digital Library

[26]

Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. CVPR.

Digital Library

[27]

Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph.

Digital Library

[28]

Svoboda, T. Multi camera self-calibration. http://cmp.felk.cvut.cz/svoboda/SelfCal/index.html.

[29]

Torralba, A., and Freeman, W. 2003. Properties and applications of shape recipes. CVPR.

[30]

Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. CVPR.

[31]

Weise, T., Li, H., Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. SCA.

Digital Library

[32]

Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph.

Digital Library

[33]

Woodford, O., Torr, P., Reid, I., and Fitzgibbon, A. 2008. Global stereo reconstruction under second order smoothness priors. CVPR.

Cited By

Yang HZheng MMa CLai YWan PHuang H(2024)VRMM: A Volumetric Relightable Morphable Head ModelACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657406(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657406
Han YLyu JXu F(2024)High-Quality Facial Geometry and Appearance Capture at Home2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00073(697-707)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00073
Saito SSchwartz GSimon TLi JNam G(2024)Relightable Gaussian Codec Avatars2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00021(130-141)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00021
Show More Cited By

Index Terms

High-quality single-shot capture of facial geometry
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Special purpose systems
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
  2. Computer graphics
    1. Graphics systems and interfaces
    2. Image manipulation
      1. Texturing

Recommendations

High-quality single-shot capture of facial geometry

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-...
Epipolar geometry for prism-based single-lens stereovision

In order to simplify the design and implementation of a stereo vision system, prism has been used to capture stereo images with a single camera. This kind of system not only provides advantages over traditional two-camera stereo, but also reduces the ...
Single-shot high-quality facial geometry and skin appearance capture

We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

July 2010

984 pages

ISBN:9781450302104

DOI:10.1145/1833349

Conference Chair:
Tony DeRose,
Editor:
Hugues Hoppe
ACM Transactions on Graphics

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SIGGRAPH '10

Sponsor:

SIGGRAPH

SIGGRAPH '10: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 26 - 30, 2010

California, Los Angeles

Acceptance Rates

SIGGRAPH '10 Paper Acceptance Rate 103 of 390 submissions, 26%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

120
Total Citations
View Citations
4,064
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)3

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang HZheng MMa CLai YWan PHuang H(2024)VRMM: A Volumetric Relightable Morphable Head ModelACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657406(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657406
Han YLyu JXu F(2024)High-Quality Facial Geometry and Appearance Capture at Home2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00073(697-707)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00073
Saito SSchwartz GSimon TLi JNam G(2024)Relightable Gaussian Codec Avatars2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00021(130-141)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00021
Li HLiu X(2024)Strand-accurate multi-view facial hair reconstruction and trackingThe Visual Computer10.1007/s00371-024-03465-540:7(4713-4724)Online publication date: 14-Jun-2024
https://doi.org/10.1007/s00371-024-03465-5
Deb DTripathi SPuri P(2023)MUNCH: Modelling Unique 'N Controllable HeadsProceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3623264.3624470(1-11)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3623264.3624470
Yang HZheng MFeng WHuang HLai YWan PWang ZMa C(2023)Towards Practical Capture of High-Fidelity Relightable AvatarsSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618138(1-11)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618138
Ling JWang ZLu MWang QQian CXu F(2023)Semantically Disentangled Variational Autoencoder for Modeling 3D Facial DetailsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316666629:8(3630-3641)Online publication date: 1-Aug-2023
https://doi.org/10.1109/TVCG.2022.3166666
Chen SLai YXia SRosin PGao L(2023)3D Face Reconstruction and Gaze Tracking in the HMD for Virtual InteractionIEEE Transactions on Multimedia10.1109/TMM.2022.315682025(3166-3179)Online publication date: 2023
https://doi.org/10.1109/TMM.2022.3156820
Shanbhag AChinara C(2023)Face to Face Augmented Reality - Broadening the Horizons of AR Communication2023 International Conference on Electrical, Computer and Energy Technologies (ICECET)10.1109/ICECET58911.2023.10389393(1-5)Online publication date: 16-Nov-2023
https://doi.org/10.1109/ICECET58911.2023.10389393
Kumar RLuo JPang ADavis J(2023)Disjoint Pose and Shape for 3D Face Reconstruction2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)10.1109/ICCVW60793.2023.00336(3107-3117)Online publication date: 2-Oct-2023
https://doi.org/10.1109/ICCVW60793.2023.00336
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents