More Web Proxy on the site http://driver.im/

research-article

High resolution passive facial performance capture

Authors:

Wolfgang Heidrich,

Alla ShefferAuthors Info & Claims

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

Article No.: 41, Pages 1 - 10

https://doi.org/10.1145/1833349.1778778

Published: 26 July 2010 Publication History

Abstract

We introduce a purely passive facial capture approach that uses only an array of video cameras, but requires no template facial geometry, no special makeup or markers, and no active lighting. We obtain initial geometry using multi-view stereo, and then use a novel approach for automatically tracking texture detail across the frames. As a result, we obtain a high-resolution sequence of compatibly triangulated and parameterized meshes. The resulting sequence can be rendered with dynamically captured textures, while also consistently applying texture changes such as virtual makeup.

Supplementary Material

Supplemental material. (041.zip)

Final-MainVideo.mp4 - the main paper video. Final-Additional.mp4 - Video of additional material.

Download
87.29 MB

MP4 File (tp049-10.mp4)

Download
157.82 MB

References

[1]

Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The digital emily project: photoreal facial modeling and animation. In ACM SIGGRAPH Courses, 1--15.

Digital Library

[2]

Beeler, T., Bickel, B., Sumner, R., Beardsley, P., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graphics (Proc. SIGGRAPH).

Digital Library

[3]

Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graphics (Proc. SIGGRAPH), 33.

Digital Library

[4]

Blanz, V., Basso, C., Vetter, T., and Poggio, T. 2003. Reanimating faces in images and video. Computer Graphics Forum (Proc. Eurographics) 22, 3, 641--650.

[5]

Borshukov, G., Piponi, D., Larsen, O., Lewis, J. P., and Tempelaar-Lietz, C. 2003. Universal capture: image-based facial animation for "the matrix reloaded". In ACM SIGGRAPH Sketches & Applications.

Digital Library

[6]

Bouguet, J.-Y. 1999. Pyramidal implementation of the lucas kanade feature tracker: Description of the algorithm. Tech. rep., Intel Corporation, Microprocessor Research Labs.

[7]

Bradley, D., and Heidrich, W. 2010. Binocular camera calibration using rectification error. In Canadian Conference on Computer and Robot Vision.

Digital Library

[8]

Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.

[9]

Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM Trans. Graphics (Proc. SIGGRAPH) 27, 3, 99.

Digital Library

[10]

Bradley, D., Atcheson, B., Ihrke, I., and Heidrich, W. 2009. Synchronization and rolling shutter compensation for consumer video camera arrays. In International Workshop on Projector-Camera Systems (PROCAMS 2009).

[11]

Canny, J. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6, 679--698.

Digital Library

[12]

de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., eidel, H.-P. S., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graphics (Proc. SIGGRAPH), 98.

Digital Library

[13]

DeCarlo, D., and Metaxas, D. 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation. In CVPR, 231.

Digital Library

[14]

DeCarlo, D., and Metaxas, D. 2000. Optical flow constraints on deformable models with applications to face tracking. Int. Journal of Computer Vision 38, 2, 99--127.

Digital Library

[15]

Essa, I., Basu, S., Darrell, T., and Pentland, A. 1996. Modeling, tracking and interactive animation of faces and heads using input from video. In Proceedings of Computer Animation, 68.

Digital Library

[16]

Fiala, M., and Shu, C. 2008. Self-identifying patterns for plane-based camera calibration. Machine Vision and Applications 19, 4, 209--216.

Digital Library

[17]

Furukawa, Y., and Ponce, J. 2009. Dense 3d motion capture for human faces. In CVPR.

[18]

Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In SIGGRAPH '98: Proceedings of the 25th annual conference on Computer graphics and interactive techniques, 55--66.

Digital Library

[19]

Kadir, T., and Brady, M. 2001. Saliency, scale and image description. Int. J. Comput. Vision 45, 2, 83--105.

Digital Library

[20]

Lévy, B., Petitjean, S., Ray, N., and Maillot, J. 2002. Least squares conformal maps for automatic texture atlas generation. ACM Trans. Graph. 21, 3, 362--371.

Digital Library

[21]

Li, H., Roivainen, P., and Forcheimer, R. 1993. 3-d motion estimation in model-based facial image coding. IEEE Trans. Pattern Anal. Mach. Intell. 15, 6, 545--555.

Digital Library

[22]

Lin, I.-C., and Ouhyoung, M. 2005. Mirror mocap: Automatic and efficient capture of dense 3d facial motion parameters from video. The Visual Computer 21, 6, 355--372.

[23]

Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Eurographics Symposium on Rendering, 183--194.

Digital Library

[24]

Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., and Debevec, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 27, 5, 121.

Digital Library

[25]

Mohammed, U., Prince, S. J. D., and Kautz, J. 2009. Visiolization: generating novel facial images. ACM Trans. Graph. 28, 3, 57.

Digital Library

[26]

Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313--318.

Digital Library

[27]

Pighin, F. H., Szeliski, R., and Salesin, D. 1999. Resynthesizing facial animation through 3d model-based tracking. In ICCV, 143--150.

[28]

Shewchuk, J. 1996. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. In Applied Computational Geometry: Towards Geometric Engineering, M. C. Lin and D. Manocha, Eds., vol. 1148 of Lecture Notes in Computer Science. Springer-Verlag, 203--222.

Digital Library

[29]

Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dyanmic 3-d facial expressions. In Computer Graphics Forum, 677--686.

[30]

Williams, L. 1990. Performance-driven facial animation. SIGGRAPH Comput. Graph. 24, 4, 235--242.

Digital Library

[31]

Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graphics (Proc. SIGGRAPH), 548--558.

Digital Library

Cited By

Rasras FPidhorskyi SSimon TPaz HWen HSaragih JRomero J(2024)The Lips, the Teeth, the tip of the Tongue: LTT TrackingSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687691(1-11)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687691
Somraj NChoudhary KMupparaju SSoundararajan R(2024)Factorized Motion Fields for Fast Sparse Input Dynamic View SynthesisACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657498(1-12)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657498
Yang HZheng MMa CLai YWan PHuang H(2024)VRMM: A Volumetric Relightable Morphable Head ModelACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657406(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657406
Show More Cited By

Index Terms

High resolution passive facial performance capture

Recommendations

High-quality single-shot capture of facial geometry
SIGGRAPH '10: ACM SIGGRAPH 2010 papers

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-...
High resolution passive facial performance capture

We introduce a purely passive facial capture approach that uses only an array of video cameras, but requires no template facial geometry, no special makeup or markers, and no active lighting. We obtain initial geometry using multi-view stereo, and then ...
Facial hair tracking for high fidelity performance capture

Facial hair is a largely overlooked topic in facial performance capture. Most production pipelines in the entertainment industry do not have a way to automatically capture facial hair or track the skin underneath it. Thus, actors are asked to shave clean ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

July 2010

984 pages

ISBN:9781450302104

DOI:10.1145/1833349

Conference Chair:
Tony DeRose,
Editor:
Hugues Hoppe
ACM Transactions on Graphics

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGGRAPH '10

Sponsor:

SIGGRAPH

SIGGRAPH '10: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 26 - 30, 2010

California, Los Angeles

Acceptance Rates

SIGGRAPH '10 Paper Acceptance Rate 103 of 390 submissions, 26%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

63
Total Citations
View Citations
2,712
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)4

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rasras FPidhorskyi SSimon TPaz HWen HSaragih JRomero J(2024)The Lips, the Teeth, the tip of the Tongue: LTT TrackingSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687691(1-11)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687691
Somraj NChoudhary KMupparaju SSoundararajan R(2024)Factorized Motion Fields for Fast Sparse Input Dynamic View SynthesisACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657498(1-12)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657498
Yang HZheng MMa CLai YWan PHuang H(2024)VRMM: A Volumetric Relightable Morphable Head ModelACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657406(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657406
Saito SSchwartz GSimon TLi JNam G(2024)Relightable Gaussian Codec Avatars2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00021(130-141)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00021
Li HLiu X(2024)Strand-accurate multi-view facial hair reconstruction and trackingThe Visual Computer10.1007/s00371-024-03465-540:7(4713-4724)Online publication date: 14-Jun-2024
https://doi.org/10.1007/s00371-024-03465-5
Li XCheng YRen XJia HXu DZhu WYan Y(2024)Topo4D: Topology-Preserving Gaussian Splatting for High-fidelity 4D Head CaptureComputer Vision – ECCV 202410.1007/978-3-031-72754-2_8(128-145)Online publication date: 31-Oct-2024
https://doi.org/10.1007/978-3-031-72754-2_8
Wagner NBotsch MSchwanecke U(2023)SoftDECA: Computationally Efficient Physics-Based Facial AnimationsProceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3623264.3624439(1-11)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3623264.3624439
Yang HZheng MFeng WHuang HLai YWan PWang ZMa C(2023)Towards Practical Capture of High-Fidelity Relightable AvatarsSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618138(1-11)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618138
Zheng MZhang HYang HHuang D(2023)NeuFace: Realistic 3D Neural Face Rendering from Multi-View Images2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01618(16868-16877)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01618
Bolkart TLi TBlack M(2023)Instant Multi-View Head Capture through Learnable Registration2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00081(768-779)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00081
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents