[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

Sparse localized deformation components

Published: 01 November 2013 Publication History

Abstract

We propose a method that extracts sparse and spatially localized deformation modes from an animated mesh sequence. To this end, we propose a new way to extend the theory of sparse matrix decompositions to 3D mesh sequence processing, and further contribute with an automatic way to ensure spatial locality of the decomposition in a new optimization framework. The extracted dimensions often have an intuitive and clear interpretable meaning. Our method optionally accepts user-constraints to guide the process of discovering the underlying latent deformation space. The capabilities of our efficient, versatile, and easy-to-implement method are extensively demonstrated on a variety of data sets and application contexts. We demonstrate its power for user friendly intuitive editing of captured mesh animations, such as faces, full body motion, cloth animations, and muscle deformations. We further show its benefit for statistical geometry processing and biomechanically meaningful animation editing. It is further shown qualitatively and quantitatively that our method outperforms other unsupervised decomposition methods and other animation parameterization approaches in the above use cases.

Supplementary Material

ZIP File (a179-neumann.zip)
Supplemental material.

References

[1]
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: shape completion and animation of people. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3.
[2]
Bach, F. R., Jenatton, R., Mairal, J., and Obozinski, G. 2012. Optimization with sparsity-inducing penalties. Found. Trends Mach. Learn. 4, 1.
[3]
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. (Proc. SIGGRAPH) 30.
[4]
Boyd, S., Parikh, N., Chu, E., Peleato, B., and Eckstein, J. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1.
[5]
Cao, Y., Shapiro, A., Faloutsos, P., and Pighin, F. 2007. Motion editing with independent component analysis. Visual Computer.
[6]
Cashman, T. J., and Hormann, K. 2012. A continuous, editable representation for deforming mesh sequences with separate signals for time, pose and shape. Comp. Graph. Forum (Proc. EG) 31, 2.
[7]
Crane, K., Weischedel, C., and Wardetzky, M. 2013. Geodesics in heat: A new approach to computing distance based on heat flow. ACM Trans. Graph., to appear.
[8]
de Aguiar, E., Theobalt, C., Thrun, S., and Seidel, H.-P. 2008. Automatic conversion of mesh animations into skeleton-based animations. Comp. Graph. Forum (Proc. EG) 27, 2.
[9]
de Aguiar, E., Sigal, L., Treuille, A., and Hodgins, J. K. 2009. Stable spaces for real-time clothing. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 3.
[10]
Deng, B., Bouaziz, S., Deuss, M., Zhang, J., Schwartzburg, Y., and Pauly, M. 2013. Exploring Local Modifications for Constrained Meshes. Comp. Graph. Forum (Proc. EG) 32, 2.
[11]
Feng, W.-W., Kim, B.-U., and Yu, Y. 2008. Real-time data driven deformation using kernel canonical correlation analysis. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3.
[12]
Fröhlich, S., and Botsch, M. 2011. Example-driven deformations based on discrete shells. Comp. Graph. Forum 30, 8.
[13]
Guan, P., Reiss, L., Hirshberg, D., Weiss, A., and Black, M. J. 2012. DRAPE: DRessing Any PErson. ACM Trans. on Graphics (Proc. SIGGRAPH) 31, 4.
[14]
Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. Comp. Graph. Forum (Proc. EG) 2, 28.
[15]
Hasler, N., Thormählen, T., Rosenhahn, B., and Seidel, H.-P. 2010. Learning skeletons for shape and pose. In Proc. of I3D.
[16]
Havaldar, P. 2006. Performance driven facial animation. In ACM SIGGRAPH 2006 Course 30 Notes.
[17]
Hyvärinen, A., Karhunen, J., and Oja, E. 2001. Independent component analysis. John Wiley & Sons.
[18]
Jenatton, R. 2011. Structured Sparsity-Inducing Norms: Statistical and Algorithmic Properties with Applications to Neuroimaging. PhD thesis, École Normale Supérieure Cachan.
[19]
Jolliffe, I. T., Trendafilov, N. T., and Uddin, M. 2003. A modified principal component technique based on the LASSO. J. Comp. Graph. Stat. 12, 3.
[20]
Kavan, L., Sloan, P.-P., and O'Sullivan, C. 2010. Fast and efficient skinning of animated meshes. Comp. Graph. Forum (Proc. EG) 29, 2.
[21]
Kavan, L., Gerszewski, D., Bargteil, A., and Sloan, P.- P. 2011. Physics-inspired upsampling for cloth simulation in games. ACM Trans. Graph. 30, 4.
[22]
Kim, D., Koh, W., Narain, R., Fatahalian, K., Treuille, A., and O'Brien, J. F. 2013. Near-exhaustive precomputation of secondary cloth effects. ACM Trans. Graph. (Proc. SIGGRAPH) 32, 4.
[23]
Kircher, S., and Garland, M. 2009. Free-form motion processing. ACM Trans. Graph. 27, 2.
[24]
Kry, P. G., James, D. L., and Pai, D. K. 2000. EigenSkin: Real time large deformation character skinning in hardware. In Proc. SCA.
[25]
Lau, M., Chai, J., Xu, Y.-Q., and Shum, H.-Y. 2009. Face poser: Interactive modeling of 3D facial expressions using facial priors. ACM Trans. Graph. 29, 1.
[26]
Le, B., and Deng, Z. 2012. Smooth skinning decomposition with rigid bones. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 31, 6.
[27]
Lee, D. D., and Seung, H. S. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401.
[28]
Levine, S., Wang, J. M., Haraux, A., Popovíc, Z., and Koltun, V. 2012. Continuous character control with low-dimensional embeddings. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4.
[29]
Lewis, J. P., and Anjyo, K. 2010. Direct-manipulation blendshapes. IEEE CGAA 30, 4.
[30]
Li, H., Weise, T., and Pauly, M. 2010. Example-based facial rigging. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 4.
[31]
Mackey, L. 2009. Deflation methods for sparse pca. In Adv. NIPS.
[32]
Mairal, J., Bach, F., Ponce, J., and Sapiro, G. 2009. Online dictionary learning for sparse coding. In Proc. ICML.
[33]
Meyer, M., and Anderson, J. 2007. Key point subspace acceleration and soft caching. ACM Trans. Graph. (Proc. SIGGRAPH)26, 3.
[34]
Miguel, E., Bradley, D., Thomaszewski, B., Bickel, B., Matusik, W., Otaduy, M. A., and Marschner, S. 2012. Data-driven estimation of cloth simulation models. Comp. Graph. Forum (Proc. EG) 31, 2.
[35]
Mohr, A., and Gleicher, M. 2003. Building efficient, accurate character skins from examples. ACM Trans. Graph. (Proc. SIGGRAPH).
[36]
Neumann, T., Varanasi, K., Hasler, N., Wacker, M., Magnor, M., and Theobalt, C. 2013. Capture and Statistical Modeling of Arm-Muscle Deformations. Comp. Graph. Forum (Proc. EG) 32, 2.
[37]
Olshausen, B., and Field, D. J. 1997. Sparse coding with an overcomplete basis set: a strategy employed by v1? Vision Research 37, 23.
[38]
Osipa, J. 2003. Stop Staring: Facial modeling and animation done right, second ed. Sybex.
[39]
Pokrass, J., Bronstein, A. M., Bronstein, M. M., Sprechmann, P., and Sapiro, G. 2013. Sparse Modeling of Intrinsic Correspondences. Comp. Graph. Forum (Proc. EG) 32, 2.
[40]
Schumacher, C., Thomaszewski, B., Coros, S., Martin, S., Sumner, R., and Gross, M. 2012. Efficient simulation of example-based materials. In Proc. SCA.
[41]
Seo, J., Irving, G., Lewis, J. P., and Noh, J. 2011. Compression and direct manipulation of complex blendshape models. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 30, 6.
[42]
Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3.
[43]
Sumner, R. W., Schmid, J., and Pauly, M. 2007. Embedded deformation for shape manipulation. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3.
[44]
Tena, J. R., De la Torre, F., and Matthews, I. 2011. Interactive region-based linear 3D face models. ACM Trans. Graph. (Proc. SIGGRAPH) 30, 4.
[45]
Theobalt, C., de Aguiar, E., Stoll, C., Seidel, H.-P., and Thrun, S. 2010. Image and geometry processing for 3D Cinematography. Springer, ch. Performance capture from multi-view video.
[46]
Tournier, M., and Reveret, L. 2012. Principal Geodesic Dynamics. In Proc. SCA.
[47]
Valgaerts, L., Wu, C., Bruhn, A., Seidel, H.-P., and Theobalt, C. 2012. Lightweight binocular facial performance capture under uncontrolled lighting. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 31, 6.
[48]
Vlasic, D., Brand, M., Pfister, H., and Popovíc, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3.
[49]
Weber, O., Sorkine, O., Lipman, Y., and Gotsman, C. 2007. Context-aware skeletal shape deformation. Comp. Graph. Forum (Proc. EG) 26, 3.
[50]
Wright, S., Nowak, R., and Figueiredo, M. 2009. Sparse reconstruction by separable approximation. Signal Processing, IEEE Transactions on 57, 7.
[51]
Wu, C., Varanasi, K., Liu, Y., Seidel, H.-P., and Theobalt, C. 2011. Shading-based dynamic shape refinement from multi-view video under general illumination. In Proc. ICCV.
[52]
Zhang, L., Snavely, N., Curless, B., and Seitz, S. 2004. Spacetime faces: High-resolution capture for modeling and animation. ACM Trans. Graph. (Proc. SIGGRAPH).
[53]
Zou, H., Hastie, T., and Tibshirani, R. 2006. Sparse principal component analysis. J. Comp. Graph. Stat. 15, 2.

Cited By

View all
  • (2024)FC-4DFS: Frequency-controlled Flexible 4D Facial Expression SynthesizingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681455(10882-10890)Online publication date: 28-Oct-2024
  • (2024)Compressed Skinning for Facial BlendshapesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657477(1-9)Online publication date: 13-Jul-2024
  • (2024)Face shape transfer via semantic warpingVisual Intelligence10.1007/s44267-024-00058-72:1Online publication date: 3-Sep-2024
  • Show More Cited By

Index Terms

  1. Sparse localized deformation components

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 32, Issue 6
    November 2013
    671 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/2508363
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 November 2013
    Published in TOG Volume 32, Issue 6

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. data-driven animation
    2. dimensionality reduction
    3. editing captured animations
    4. mesh deformation

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)42
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 11 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)FC-4DFS: Frequency-controlled Flexible 4D Facial Expression SynthesizingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681455(10882-10890)Online publication date: 28-Oct-2024
    • (2024)Compressed Skinning for Facial BlendshapesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657477(1-9)Online publication date: 13-Jul-2024
    • (2024)Face shape transfer via semantic warpingVisual Intelligence10.1007/s44267-024-00058-72:1Online publication date: 3-Sep-2024
    • (2024)Variational autoencoders for 3D data processingArtificial Intelligence Review10.1007/s10462-023-10687-x57:2Online publication date: 8-Feb-2024
    • (2024)ShapeFusion: A 3D Diffusion Model for Localized Shape EditingComputer Vision – ECCV 202410.1007/978-3-031-72630-9_5(72-89)Online publication date: 5-Dec-2024
    • (2023)Learning a crowd-powered perceptual distance metric for facial blendshapesJournal on Image and Video Processing10.1186/s13640-023-00609-w2023:1Online publication date: 15-May-2023
    • (2023)HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance FieldACM Transactions on Graphics10.1145/362631643:1(1-16)Online publication date: 30-Nov-2023
    • (2023)3D Face Recognition: Two Decades of Progress and ProspectsACM Computing Surveys10.1145/361586356:3(1-39)Online publication date: 5-Oct-2023
    • (2023)Local Deformation for Interactive Shape EditingACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591501(1-10)Online publication date: 23-Jul-2023
    • (2023)Improved Projective Dynamics Global Using Snapshots-based Reduced BasesACM SIGGRAPH 2023 Posters10.1145/3588028.3603665(1-2)Online publication date: 23-Jul-2023
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media