Super-Resolution Cloth Animation with Spatial and Temporal Coherence
Authors: 
Jiawang Yu, 
Zhendong WangAuthors Info & Claims
Article No.: 105, Pages 1 - 14
Abstract
Creating super-resolution cloth animations, which refine coarse cloth meshes with fine wrinkle details, faces challenges in preserving spatial consistency and temporal coherence across frames. In this paper, we introduce a general framework to address these issues, leveraging two core modules. The first module interleaves a simulator and a corrector. The simulator handles cloth dynamics, while the corrector rectifies differences in low-frequency features across various resolutions. This interleaving ensures prompt correction of spatial errors from the coarse simulation, effectively preventing their temporal propagation. The second module performs mesh-based super-resolution for detailed wrinkle enhancements. We decompose garment meshes into overlapping patches for adaptability to various styles and geometric continuity. Our method achieves an 8× improvement in resolution for cloth animations. We showcase the effectiveness of our method through diverse animation examples, including simple cloth pieces and intricate garments.
Supplementary Material
supplemental
- Download
- 590.28 MB
References
[1]
Ferran Alet, Adarsh Keshav Jeewajee, Maria Bauza Villalonga, Alberto Rodriguez, Tomas Lozano-Perez, and Leslie Kaelbling. 2019. Graph element networks: adaptive, structured computation and memory. In International Conference on Machine Learning. PMLR, 212--222.
[2]
Saeed Anwar, Salman Khan, and Nick Barnes. 2020. A Deep Journey into Super-resolution: A Survey. ACM Comput. Surv. 53, 3, Article 60 (may 2020), 34 pages.
[3]
David Baraff and Andrew Witkin. 1998. Large Steps in Cloth Simulation. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98). Association for Computing Machinery, New York, NY, USA, 43--54.
[4]
Miklos Bergou, Max Wardetzky, David Harmon, Denis Zorin, and Eitan Grinspun. 2006. A Quadratic Bending Model for Inextensible Surfaces. In Proceedings of the Fourth Eurographics Symposium on Geometry Processing (Cagliari, Sardinia, Italy) (SGP '06). Eurographics Association, Goslar, DEU, 227--230.
[5]
Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2022. Neural Cloth Simulation. ACM Trans. Graph. 41, 6, Article 220 (nov 2022), 14 pages.
[6]
Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective Dynamics: Fusing Constraint Projections for Fast Simulation. ACM Trans. Graph. (SIGGRAPH) 33, 4, Article 154 (July 2014), 11 pages.
[7]
Shaked Brody, Uri Alon, and Eran Yahav. 2022. How Attentive are Graph Attention Networks?. In International Conference on Learning Representations.
[8]
Jiong Chen, Florian Schäfer, Jin Huang, and Mathieu Desbrun. 2021b. Multiscale Cholesky Preconditioning for Ill-Conditioned Problems. ACM Trans. Graph. 40, 4, Article 81 (jul 2021), 13 pages.
[9]
Yunuo Chen, Tianyi Xie, Cem Yuksel, Danny Kaufman, Yin Yang, Chenfanfu Jiang, and Minchen Li. 2023b. Multi-Layer Thick Shells. In ACM SIGGRAPH 2023 Conference Proceedings (SIGGRAPH '23). Association for Computing Machinery, New York, NY, USA, Article 25, 9 pages.
[10]
Zhen Chen, Hsiao-Yu Chen, Danny M. Kaufman, Mélina Skouras, and Etienne Vouga. 2021a. Fine Wrinkling on Coarsely Meshed Thin Shells. ACM Trans. Graph. 40, 5, Article 190 (aug 2021), 32 pages.
[11]
Zhen Chen, Danny Kaufman, Mélina Skouras, and Etienne Vouga. 2023a. Complex Wrinkle Field Evolution. ACM Trans. Graph. 42, 4, Article 72 (jul 2023), 19 pages.
[12]
Kwang-Jin Choi and Hyeong-Seok Ko. 2002. Stable but responsive cloth. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (San Antonio, Texas) (SIGGRAPH '02). Association for Computing Machinery, New York, NY, USA, 604--611.
[13]
Edilson de Aguiar, Leonid Sigal, Adrien Treuille, and Jessica K. Hodgins. 2010. Stable spaces for real-time clothing. In ACM SIGGRAPH 2010 Papers (Los Angeles, California) (SIGGRAPH '10). Association for Computing Machinery, New York, NY, USA, Article 106, 9 pages.
[14]
Wei-Wen Feng, Yizhou Yu, and Byung-Uck Kim. 2010. A Deformation Transformer for Real-Time Cloth Animation. In ACM SIGGRAPH 2010 Papers (Los Angeles, California) (SIGGRAPH '10). Association for Computing Machinery, New York, NY, USA, Article 108, 9 pages.
[15]
Russell Gillette, Craig Peters, Nicholas Vining, Essex Edwards, and Alla Sheffer. 2015. Real-Time Dynamic Wrinkling of Coarse Animated Cloth. In Proceedings of the 14th ACM SIGGRAPH / Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA '15). Association for Computing Machinery, New York, NY, USA, 17--26.
[16]
Eitan Grinspun, Anil N. Hirani, Mathieu Desbrun, and Peter Schröder. 2003. Discrete Shells. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (San Diego, California) (SCA '03). Eurographics Association, Goslar, DEU, 62--67.
[17]
Xianfeng Gu, Steven J. Gortler, and Hugues Hoppe. 2002. Geometry images. ACM Trans. Graph. 21, 3 (jul 2002), 355?361.
[18]
Peng Guan, Loretta Reiss, David A. Hirshberg, Alexander Weiss, and Michael J. Black. 2012. DRAPE: DRessing Any PErson. ACM Trans. Graph. 31, 4, Article 35 (jul 2012), 10 pages.
[19]
Jiequn Han, Arnulf Jentzen, and Weinan E. 2018. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences 115, 34 (2018), 8505--8510.
[20]
Tero Karras. 2012. Maximizing parallelism in the construction of BVHs, octrees, and k-d trees. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics conference on High-Performance Graphics. 33--37.
[21]
Ladislav Kavan, Dan Gerszewski, Adam W. Bargteil, and Peter-Pike Sloan. 2011. Physics-Inspired Upsampling for Cloth Simulation in Games., Article 93 (2011), 10 pages.
[22]
Byungsoo Kim, Vinicius C Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. 2019. Deep fluids: A generative network for parameterized fluid simulations. 38, 2 (2019), 59--70.
[23]
Byung-Uck Kim and Farzam Farbiz. 2013. Wrinkle Flow for Compact Representation of Predefined Clothing Animation. In ACM SIGGRAPH 2013 Posters (Anaheim, California) (SIGGRAPH '13). Association for Computing Machinery, New York, NY, USA, Article 15, 1 pages.
[24]
Scott Kircher and Michael Garland. 2005. Progressive Multiresolution Meshes for Deforming Surfaces. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA '05). Association for Computing Machinery, New York, NY, USA, 191?200.
[25]
Zorah Lahner, Daniel Cremers, and Tony Tung. 2018. Deepwrinkles: Accurate and realistic clothing modeling. In Proceedings of the European conference on computer vision (ECCV). 667--684.
[26]
Lei Lan, Minchen Li, Chenfanfu Jiang, Huamin Wang, and Yin Yang. 2023. Second-Order Stencil Descent for Interior-Point Hyperelasticity. ACM Trans. Graph. 42, 4, Article 108 (jul 2023), 16 pages.
[27]
Lei Lan, Guanqun Ma, Yin Yang, Changxi Zheng, Minchen Li, and Chenfanfu Jiang. 2022. Penetration-Free Projective Dynamics on the GPU. ACM Trans. Graph. 41, 4, Article 69 (jul 2022), 16 pages.
[28]
Yongjoon Lee, Sung eui Yoon, Seungwoo Oh, Duksu Kim, and Sunghee Choi. 2010. Multi-Resolution Cloth Simulation. Computer Graphics Forum (Pacific Graphics) 29, 7 (2010), 2225--2232.
[29]
Cheng Li, Min Tang, Ruofeng Tong, Ming Cai, Jieyi Zhao, and Dinesh Manocha. 2020. P-Cloth: Interactive Complex Cloth Simulation on Multi-GPU Systems Using Dynamic Matrix Assembly and Pipelined Implicit Integrators. ACM Trans. Graph. 39, 6, Article 180 (nov 2020), 15 pages.
[30]
Minchen Li, Ming Gao, Timothy Langlois, Chenfanfu Jiang, and Danny M. Kaufman. 2019. Decomposed Optimization Time Integrator for Large-Step Elastodynamics. ACM Trans. Graph. 38, 4, Article 70 (July 2019), 10 pages.
[31]
Yinxiao Li, Yan Wang, Yonghao Yue, Danfei Xu, Michael Case, Shih-Fu Chang, Eitan Grinspun, and Peter K Allen. 2018. Model-driven feedforward prediction for manipulation of deformable objects. IEEE Transactions on Automation Science and Engineering 15, 4 (2018), 1621--1638.
[32]
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE international conference on computer vision. 1833--1844.
[33]
Hongying Liu, Zhubo Ruan, Peng Zhao, Chao Dong, Fanhua Shang, Yuanyuan Liu, Linlin Yang, and Radu Timofte. 2022. Video super-resolution based on deep learning: a comprehensive survey. Artificial Intelligence Review 55, 8 (2022), 5981--6035.
[34]
Hsueh-Ti Derek Liu, Jiayi Eris Zhang, Mirela Ben-Chen, and Alec Jacobson. 2021b. Surface Multigrid via Intrinsic Prolongation. ACM Trans. Graph. 40, 4, Article 80 (jul 2021), 13 pages.
[35]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021a. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE international conference on computer vision. 10012--10022.
[36]
Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B Tenenbaum, Tao Du, Chuang Gan, and Wojciech Matusik. 2023. Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics. arXiv preprint arXiv:2304.14369 (2023).
[37]
Matthias Müller. 2008. Hierarchical Position Based Dynamics. In Proceedings of VRIPHYS. 1--10.
[38]
Matthias Müller and Nuttapong Chentanez. 2010. Wrinkle Meshes. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Madrid, Spain) (SCA '10). Eurographics Association, Goslar, DEU, 85--92.
[39]
Young Jin Oh, Tae Min Lee, and In-Kwon Lee. 2018. Hierarchical Cloth Simulation Using Deep Neural Networks. In Proceedings of Computer Graphics International 2018 (Bintan, Island, Indonesia) (CGI 2018). Association for Computing Machinery, New York, NY, USA, 139--146.
[40]
Xiaoyu Pan, Jiaming Mai, Xinwei Jiang, Dongxue Tang, Jingxiang Li, Tianjia Shao, Kun Zhou, Xiaogang Jin, and Dinesh Manocha. 2022. Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH '22). Association for Computing Machinery, New York, NY, USA, Article 11, 10 pages.
[41]
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. 2020. Learning Mesh-Based Simulation with Graph Networks. In International Conference on Learning Representations.
[42]
Damien Rohmer, Tiberiu Popa, Marie-Paule Cani, Stefanie Hahmann, and Alla Sheffer. 2010. Animation Wrinkling: Augmenting Coarse Cloth Simulations with Realistic-Looking Wrinkles. ACM Trans. Graph. 29, 6, Article 157 (dec 2010), 8 pages.
[43]
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. 2020. Learning to simulate complex physics with graph networks. In International conference on machine learning. PMLR, 8459--8468.
[44]
Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia. 2018. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning. PMLR, 4470--4479.
[45]
Igor Santesteban, Miguel A Otaduy, and Dan Casas. 2022. Snug: Self-supervised neural dynamic garments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8140--8150.
[46]
Igor Santesteban, Nils Thuerey, Miguel A Otaduy, and Dan Casas. 2021. Self-supervised collision handling via generative 3d garment models for virtual try-on. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11763--11773.
[47]
Han Shao, Libo Huang, and Dominik L. Michels. 2022. A Fast Unsmoothed Aggregation Algebraic Multigrid Framework for the Large-Scale Simulation of Incompressible Flow. ACM Trans. Graph. 41, 4, Article 49 (jul 2022), 18 pages.
[48]
Rasmus Tamstorf, Toby Jones, and Stephen F. McCormick. 2015. Smoothed Aggregation Multigrid for Cloth Simulation. ACM Trans. Graph. (SIGGRAPH Asia) 34, 6, Article 245 (Oct. 2015), 13 pages.
[49]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[50]
Huamin Wang. 2021. GPU-Based Simulation of Cloth Wrinkles at Submillimeter Levels. ACM Trans. Graph. 40, 4, Article 169 (jul 2021), 14 pages.
[51]
Huamin Wang, Florian Hecht, Ravi Ramamoorthi, and James F. O'Brien. 2010a. Example-Based Wrinkle Synthesis for Clothing Animation. ACM Trans. Graph. 29, 4, Article 107 (jul 2010), 8 pages.
[52]
Huamin Wang, James O'Brien, and Ravi Ramamoorthi. 2010b. Multi-Resolution Isotropic Strain Limiting. ACM Trans. Graph. (SIGGRAPH Asia) 29, 6, Article 156 (Dec. 2010), 156:1--156:10 pages.
[53]
Huamin Wang and Yin Yang. 2016. Descent Methods for Elastic Body Simulation on the GPU. ACM Trans. Graph. (SIGGRAPH Asia) 35, 6, Article 212 (Nov. 2016), 10 pages.
[54]
Zhihao Wang, Jian Chen, and Steven CH Hoi. 2020. Deep learning for image super-resolution: A survey. IEEE transactions on pattern analysis and machine intelligence 43, 10 (2020), 3365--3387.
[55]
Zhendong Wang, Longhua Wu, Marco Fratarcangeli, Min Tang, and Huamin Wang. 2018. Parallel Multigrid for Nonlinear Cloth Simulation. Computer Graphics Forum (Pacific Graphics) 37, 7 (2018), 131--141.
[56]
Zhendong Wang, Yin Yang, and Huamin Wang. 2023. Stable Discrete Bending by Analytic Eigensystem and Adaptive Orthotropic Geometric Stiffness. ACM Trans. Graph. 42, 6, Article 175 (dec 2023), 16 pages.
[57]
Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, and Klaus Hildebrandt. 2023. A Fast Geometric Multigrid Method for Curved Surfaces. In ACM SIGGRAPH 2023 Conference Proceedings (Los Angeles, CA, USA) (SIGGRAPH '23). Association for Computing Machinery, New York, NY, USA, Article 1, 11 pages.
[58]
Botao Wu, Zhendong Wang, and Huamin Wang. 2022. A GPU-Based Multilevel Additive Schwarz Preconditioner for Cloth and Deformable Body Simulation. ACM Trans. Graph. 41, 4, Article 63 (jul 2022), 14 pages.
[59]
Zangyueyang Xian, Xin Tong, and Tiantian Liu. 2019. A Scalable Galerkin Multigrid Method for Real-Time Simulation of Deformable Objects. ACM Trans. Graph. 38, 6, Article 162 (nov 2019), 13 pages.
[60]
Jiayi Eris Zhang, Jérémie Dumas, Yun (Raymond) Fei, Alec Jacobson, Doug L. James, and Danny M. Kaufman. 2022b. Progressive Simulation for Cloth Quasistatics. ACM Trans. Graph. 41, 6, Article 218 (nov 2022), 16 pages.
[61]
Meng Zhang, Duygu Ceylan, and Niloy J. Mitra. 2022a. Motion Guided Deep Dynamic 3D Garments. ACM Trans. Graph. 41, 6, Article 219 (nov 2022), 12 pages.
[62]
Meng Zhang, Tuanfeng Wang, Duygu Ceylan, and Niloy J Mitra. 2021a. Deep detail enhancement for any garment. Computer Graphics Forum 40, 2 (2021), 399--411.
[63]
Meng Zhang, Tuanfeng Y. Wang, Duygu Ceylan, and Niloy J. Mitra. 2021b. Dynamic Neural Garments. ACM Trans. Graph. 40, 6, Article 235 (dec 2021), 15 pages.
[64]
Xindong Zhang, Hui Zeng, Shi Guo, and Lei Zhang. 2022c. Efficient long-range attention network for image super-resolution. In European Conference on Computer Vision. Springer, 649--667.
[65]
Fang Zhao, Zekun Li, Shaoli Huang, Junwu Weng, Tianfei Zhou, Guo-Sen Xie, Jue Wang, and Ying Shan. 2023. Learning Anchor Transformations for 3D Garment Animation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 491--500.
[66]
Javier S Zurdo, Juan P Brito, and Miguel A Otaduy. 2013. Animating wrinkles by example on non-skinned cloth. IEEE Transactions on Visualization and Computer Graphics 19, 1 (2013), 149--158.
Index Terms
- Super-Resolution Cloth Animation with Spatial and Temporal Coherence
Recommendations
Faking dynamics of cloth animation for animated films
AMDO'10: Proceedings of the 6th international conference on Articulated motion and deformable objectsIn this paper we argue for the concept of fake dynamics to allow animators to interactively create visually pleasing animations of cloth models while keeping him/her in full control of the animation process. Existing animation and simulation techniques ...
Data driven cloth animation
SIGGRAPH '07: ACM SIGGRAPH 2007 sketchesWe present a new method for cloth animation based on data driven synthesis. In contrast to approaches that focus on physical simulation, we animate cloth by manipulating short sequences of existing cloth animation. While our source of data is cloth ...
Faking Dynamics of Cloth Animation for Animated Films
6th International Conference on Articulated Motion and Deformable Objects - Volume 6169In this paper we argue for the concept of fake dynamics to allow animators to interactively create visually pleasing animations of cloth models while keeping him/her in full control of the animation process. Existing animation and simulation techniques ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 July 2024
Published in TOG Volume 43, Issue 4
Check for updates
Qualifiers
- Research-article
Funding Sources
- Key R&D Program of Zhejiang
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 297Total Downloads
- Downloads (Last 12 months)297
- Downloads (Last 6 weeks)28
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in