[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3309486.3340245acmconferencesArticle/Chapter ViewAbstractPublication PagesscaConference Proceedingsconference-collections
research-article

Subspace neural physics: fast data-driven interactive simulation

Published: 26 July 2019 Publication History

Abstract

Data-driven methods for physical simulation are an attractive option for interactive applications due to their ability to trade precomputation and memory footprint in exchange for improved runtime performance. Yet, existing data-driven methods fall short of the extreme memory and performance constraints imposed by modern interactive applications like AAA games and virtual reality. Here, performance budgets for physics simulation range from tens to hundreds of micro-seconds per frame, per object. We present a data-driven physical simulation method that meets these constraints. Our method combines subspace simulation techniques with machine learning which, when coupled, enables a very efficient subspace-only physics simulation that supports interactions with external objects - a longstanding challenge for existing subspace techniques. We also present an interpretation of our method as a special case of subspace Verlet integration, where we apply machine learning to efficiently approximate the physical forces of the system directly in the subspace. We propose several practical solutions required to make effective use of such a model, including a novel training methodology required for prediction stability, and a GPU-friendly subspace decompression algorithm to accelerate rendering.

Supplementary Material

ZIP File (a6-holden.zip)
Supplemental material.

References

[1]
Steven S. An, Theodore Kim, and Doug L. James. 2008. Optimizing Cubature for Efficient Integration of Subspace Deformations. In ACM SIGGRAPH Asia 2008 Papers (SIGGRAPH Asia '08). ACM, New York, NY, USA, Article 165, 10 pages.
[2]
Jernej Barbič and Doug L. James. 2005. Real-Time Subspace Integration for St. Venant-Kirchhoff Deformable Models. In ACM SIGGRAPH 2005 Papers (SIGGRAPH '05). ACM, New York, NY, USA, 982--990.
[3]
Jan Bender, Matthias Müller, and Miles Macklin. 2017. Position-Based Simulation Methods in Computer Graphics. In EUROGRAPHICS 2017 Tutorials. Eurographics Association.
[4]
Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective Dynamics: Fusing Constraint Projections for Fast Simulation. ACM Trans. Graph. 33, 4, Article 154 (July 2014), 11 pages.
[5]
Christopher Brandt, Elmar Eisemann, and Klaus Hildebrandt. 2018. Hyper-reduced Projective Dynamics. ACM Trans. Graph. 37, 4, Article 80 (July 2018), 13 pages.
[6]
KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. CoRR abs/1409.1259 (2014). arXiv:1409.1259 http://arxiv.org/abs/1409.1259
[7]
Mengyu Chu and Nils Thuerey. 2017. Data-Driven Synthesis of Smoke Flows with CNN-based Feature Descriptors. CoRR abs/1705.01425 (2017). arXiv:1705.01425 http://arxiv.org/abs/1705.01425
[8]
Simon Clavet. 2016. Motion Matching and The Road to Next-Gen Animation. In Proc. of GDC 2016.
[9]
Edilson de Aguiar, Leonid Sigal, Adrien Treuille, and Jessica K. Hodgins. 2010. Stable Spaces for Real-time Clothing. In ACM SIGGRAPH 2010 Papers (SIGGRAPH '10). ACM, New York, NY, USA, Article 106, 9 pages.
[10]
P. Dollár, P. Welinder, and P. Perona. 2010. Cascaded pose regression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1078--1085.
[11]
Lawson Fulton, Vismay Modi, David Duvenaud, David I. W. Levin, and Alec Jacobson. 2019. Latent-space Dynamics for Reduced Deformable Simulation. Computer Graphics Forum (2019).
[12]
Yarin Gal and Zoubin Ghahramani. 2016. A Theoretically Grounded Application of Dropout in Recurrent Neural Networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc., USA, 1027--1035. http://dl.acm.org/citation.cfm?id=3157096.3157211
[13]
Nico Galoppo, Miguel Otaduy, Serhat Tekin, Markus Gross, and Ming Lin. 2007. Soft Articulated Characters with Fast Contact Handling. Comput. Graph. Forum 26 (09 2007), 243--253.
[14]
Nico Galoppo, Miguel A. Otaduy, William Moss, Jason Sewall, Sean Curtis, and Ming C. Lin. 2009. Controlling Deformable Material with Dynamic Morph Targets. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (I3D '09). ACM, New York, NY, USA, 39--47.
[15]
Fabian Hahn, Bernhard Thomaszewski, Stelian Coros, Robert W. Sumner, Forrester Cole, Mark Meyer, Tony DeRose, and Markus Gross. 2014. Subspace Clothing Simulation Using Adaptive Bases. ACM Trans. Graph. 33, 4, Article 105 (July 2014), 9 pages.
[16]
David Harmon and Denis Zorin. 2013. Subspace Integration with Local Deformations. ACM Trans. Graph. 32, 4, Article 107 (July 2013), 10 pages.
[17]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. 9, 8 (Nov. 1997), 1735--1780.
[18]
J. Huang, Y. Tong, K. Zhou, H. Bao, and M. Desbrun. 2011. Interactive Shape Interpolation through Controllable Dynamic Deformation. IEEE Transactions on Visualization and Computer Graphics 17, 7 (July 2011), 983--992.
[19]
Thomas Jakobsen. 2001. Advanced character physics. In Game Developers Conference.
[20]
Doug L. James and Dinesh K. Pai. 2002. DyRT: Dynamic Response Textures for Real Time Deformation Simulation with Graphics Hardware. ACM Trans. Graph. 21, 3 (July 2002), 582--585.
[21]
Ning Jin, Yilin Zhu, Zhenglin Geng, and Ronald Fedkiw. 2018. A Pixel-Based Framework for Data-Driven Clothing. CoRR abs/1812.01677 (2018). arXiv:1812.01677 http://arxiv.org/abs/1812.01677
[22]
Angjoo Kanazawa, Michael J. Black, David W. Jacobs, and Jitendra Malik. 2017. End-to-end Recovery of Human Shape and Pose. CoRR abs/1712.06584 (2017). arXiv:1712.06584 http://arxiv.org/abs/1712.06584
[23]
Byungsoo Kim, Vinicius C. Azevedo, Nils Thuerey, Theodore Kim, Markus H. Gross, and Barbara Solenthaler. 2018. Deep Fluids: A Generative Network for Parameterized Fluid Simulations. CoRR abs/1806.02071 (2018). arXiv:1806.02071 http://arxiv.org/abs/1806.02071
[24]
Doyub Kim, Woojong Koh, Rahul Narain, Kayvon Fatahalian, Adrien Treuille, and James F. O'Brien. 2013. Near-exhaustive Precomputation of Secondary Cloth Effects. ACM Trans. Graph. 32, 4, Article 87 (July 2013), 8 pages.
[25]
Theodore Kim and Doug L. James. 2009. Skipping Steps in Deformable Simulation with Online Model Reduction. In ACM SIGGRAPH Asia 2009 Papers (SIGGRAPH Asia '09). ACM, New York, NY, USA, Article 123, 9 pages.
[26]
Theodore Kim and Doug L. James. 2011. Physics-based Character Skinning Using Multi-domain Subspace Deformations. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '11). ACM, New York, NY, USA, 63--72.
[27]
Tae-Yong Kim, Nuttapong Chentanez, and Matthias Müller-Fischer. 2012. Long Range Attachments - a Method to Simulate Inextensible Clothing in Computer Games. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '12). Eurographics Association, Goslar Germany, Germany, 305--310. http://dl.acm.org/citation.cfm?id=2422356.2422399
[28]
Paul G. Kry, Doug L. James, and Dinesh K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '02). ACM, New York, NY, USA, 153--159.
[29]
Lubor Ladicky, SoHyeon Jeong, Nemanja Bartolovic, Marc Pollefeys, and Markus Gross. 2017. Physicsforests: Real-time Fluid Simulation Using Machine Learning. In ACM SIGGRAPH 2017 Real Time Live! (SIGGRAPH '17). ACM, New York, NY, USA, 22--22.
[30]
L'ubor Ladický, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven Fluid Simulations Using Regression Forests. ACM Trans. Graph. 34, 6, Article 199 (Oct. 2015), 9 pages.
[31]
Zorah Lähner, Daniel Cremers, and Tony Tung. 2018. DeepWrinkles: Accurate and Realistic Clothing Modeling. CoRR abs/1808.03417 (2018). arXiv:1808.03417 http://arxiv.org/abs/1808.03417
[32]
Tiantian Liu, Adam W. Bargteil, James F. O'Brien, and Ladislav Kavan. 2013. Fast Simulation of Mass-spring Systems. ACM Trans. Graph. 32, 6, Article 214 (Nov. 2013), 7 pages.
[33]
Ran Luo, Tianjia Shao, Huamin Wang, Weiwei Xu, Xiang Chen, Kun Zhou, and Yin Yang. 2018. NNWarp: Neural Network-based Nonlinear Deformation. IEEE Transactions on Visualization and Computer Graphics PP (11 2018), 1--1.
[34]
Miles Macklin and Matthias Müller. 2013. Position Based Fluids. ACM Trans. Graph. 32, 4, Article 104 (July 2013), 12 pages.
[35]
Miles Macklin, Matthias Müller, and Nuttapong Chentanez. 2016. XPBD: Position-based Simulation of Compliant Constrained Dynamics. In Proceedings of the 9th International Conference on Motion in Games (MIG '16). ACM, New York, NY, USA, 49--54.
[36]
Matthias Müller and Nuttapong Chentanez. 2011. Solid Simulation with Oriented Particles. In ACM SIGGRAPH 2011 Papers (SIGGRAPH'11). ACM, New York, NY, USA, Article 92, 10 pages.
[37]
Matthias Müller, Nuttapong Chentanez, Tae-Yong Kim, and Miles Macklin. 2014. Strain Based Dynamics. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '14). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 149--157. http://dl.acm.org/citation.cfm?id=2849517.2849542
[38]
Matthias Müller, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. 2007. Position based dynamics. Journal of Visual Communication and Image Representation 18, 2 (2007), 109--118.
[39]
Vinod Nair and Geoffrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML'10). Omnipress, USA, 807--814. http://dl.acm.org/citation.cfm?id=3104322.3104425
[40]
Rahul Narain, Matthew Overby, and George E. Brown. 2016. ADMM ⊇ Projective Dynamics: Fast Simulation of General Constitutive Models. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '16). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 21--28. http://dl.acm.org/citation.cfm?id=2982818.2982822
[41]
Markus Oberweger, Paul Wohlhart, and Vincent Lepetit. 2016. Training a Feedback Loop for Hand Pose Estimation. CoRR abs/1609.09698 (2016). arXiv:1609.09698 http://arxiv.org/abs/1609.09698
[42]
Zherong Pan, Hujun Bao, and Jin Huang. 2015. Subspace Dynamic Simulation Using Rotation-strain Coordinates. ACM Trans. Graph. 34, 6, Article 242 (Oct. 2015), 12 pages.
[43]
Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. 2018. On the Convergence of Adam and Beyond. In International Conference on Learning Representations. https://openreview.net/forum?id=ryQu7f-RZ
[44]
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. 1986. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. MIT Press, Cambridge, MA, USA, Chapter Learning Internal Representations by Error Propagation, 318--362. http://dl.acm.org/citation.cfm?id=104279.104293
[45]
Eftychios Sifakis and Jernej Barbic. 2012. FEM Simulation of 3D Deformable Solids: A Practitioner's Guide to Theory, Discretization and Model Reduction. In ACM SIGGRAPH 2012 Courses (SIGGRAPH '12). ACM, New York, NY, USA, Article 20, 50 pages.
[46]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15 (2014), 1929--1958. http://jmlr.org/papers/v15/srivastava14a.html
[47]
J. Stam. 2009. Nucleus: Towards a unified dynamics solver for computer graphics. In 2009 11th IEEE International Conference on Computer-Aided Design and Computer Graphics. 1--11.
[48]
Yun Teng, Mark Meyer, Tony De Rose, and Theodore Kim. 2015. Subspace Condensation: Full Space Adaptivity for Subspace Deformations. ACM Trans. Graph. 34, 4, Article 76 (July 2015), 9 pages.
[49]
Yun Teng, Miguel A. Otaduy, and Theodore Kim. 2014. Simulating Articulated Subspace Self-contact. ACM Trans. Graph. 33, 4, Article 106 (July 2014), 9 pages.
[50]
Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2016. Accelerating Eulerian Fluid Simulation With Convolutional Networks. CoRR abs/1607.03597 (2016). arXiv:1607.03597 http://arxiv.org/abs/1607.03597
[51]
Adrien Treuille, Andrew Lewis, and Zoran Popović. 2006. Model Reduction for Realtime Fluids. In ACM SIGGRAPH 2006 Papers (SIGGRAPH '06). ACM, New York, NY, USA, 826--834.
[52]
Alexis Vaisse. 2016. Ubisoft Cloth Simulation: Performance Postmortem and Journey from C++ to Compute Shaders. In GDC 2016 Talks.
[53]
Philipp von Radziewsky, Elmar Eisemann, Hans-Peter Seidel, and Klaus Hildebrandt. 2016. Optimized Subspaces for Deformation-based Modeling and Shape Interpolation. Comput. Graph. 58, C (Aug. 2016), 128--138.
[54]
Christoph von Tycowicz, Christian Schulz, Hans-Peter Seidel, and Klaus Hildebrandt. 2013. An Efficient Construction of Reduced Deformable Objects. ACM Trans. Graph. 32, 6, Article 213 (Nov. 2013), 10 pages.
[55]
Huamin Wang, Florian Hecht, Ravi Ramamoorthi, and James F. O'Brien. 2010. Example-based Wrinkle Synthesis for Clothing Animation. In ACM SIGGRAPH 2010 Papers (SIGGRAPH '10). ACM, New York, NY, USA, Article 107, 8 pages.
[56]
Yu Wang, Alec Jacobson, Jernej Barbič, and Ladislav Kavan. 2015. Linear Subspace Design for Real-time Shape Deformation. ACM Trans. Graph. 34, 4, Article 57 (July 2015), 11 pages.
[57]
Marcel Weiler, Dan Koschier, and Jan Bender. 2016. Projective Fluids. In Proceedings of the 9th International Conference on Motion in Games (MIG '16). ACM, New York, NY, USA, 79--84.
[58]
Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2018. Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow. CoRR abs/1802.10123 (2018). arXiv:1802.10123 http://arxiv.org/abs/1802.10123
[59]
You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. 2018. tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow. CoRR abs/1801.09710 (2018). arXiv:1801.09710 http://arxiv.org/abs/1801.09710
[60]
Hongyi Xu and Jernej Barbič. 2016. Pose-space Subspace Dynamics. ACM Trans. Graph. 35, 4, Article 35 (July 2016), 14 pages.
[61]
Weiwei Xu, Nobuyuki Umentani, Qianwen Chao, Jie Mao, Xiaogang Jin, and Xin Tong. 2014. Sensitivity-optimized Rigging for Example-based Real-time Clothing Synthesis. ACM Trans. Graph. 33, 4, Article 107 (July 2014), 11 pages.
[62]
Yin Yang, Dingzeyu Li, Weiwei Xu, Yuan Tian, and Changxi Zheng. 2015. Expediting Precomputation for Reduced Deformable Simulation. ACM Trans. Graph. 34, 6, Article 243 (Oct. 2015), 13 pages.
[63]
Y. Yang, W. Xu, X. Guo, K. Zhou, and B. Guo. 2013. Boundary-Aware Multidomain Subspace Deformation. IEEE Transactions on Visualization and Computer Graphics 19, 10 (Oct 2013), 1633--1645.

Cited By

View all
  • (2024)Deep Neural Network-Based Cloth Collision Detection AlgorithmScientific Programming10.1155/2024/78892782024Online publication date: 1-Jan-2024
  • (2024)Resolving Collisions in Dense 3D Crowd AnimationsACM Transactions on Graphics10.1145/368726643:5(1-14)Online publication date: 6-Sep-2024
  • (2024)ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment SimulationsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657408(1-10)Online publication date: 13-Jul-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
SCA '19: Proceedings of the 18th annual ACM SIGGRAPH/Eurographics Symposium on Computer Animation
July 2019
83 pages
ISBN:9781450366779
DOI:10.1145/3309486
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cloth simulation
  2. collision detection
  3. data-driven simulation
  4. machine learning
  5. model reduction
  6. neural networks

Qualifiers

  • Research-article

Funding Sources

  • Natural Sciences and Engineering Research Council of Canada

Conference

SCA '19
Sponsor:

Acceptance Rates

Overall Acceptance Rate 183 of 487 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)179
  • Downloads (Last 6 weeks)21
Reflects downloads up to 20 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Deep Neural Network-Based Cloth Collision Detection AlgorithmScientific Programming10.1155/2024/78892782024Online publication date: 1-Jan-2024
  • (2024)Resolving Collisions in Dense 3D Crowd AnimationsACM Transactions on Graphics10.1145/368726643:5(1-14)Online publication date: 6-Sep-2024
  • (2024)ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment SimulationsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657408(1-10)Online publication date: 13-Jul-2024
  • (2024)Neural Garment Dynamics via Manifold‐Aware TransformersComputer Graphics Forum10.1111/cgf.1502843:2Online publication date: 30-Apr-2024
  • (2024)Predictive Dead Reckoning for Online Peer-to-Peer GamesIEEE Transactions on Games10.1109/TG.2023.323794316:1(173-184)Online publication date: Mar-2024
  • (2024)Optimal Control of Granular Material2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10611171(13234-13241)Online publication date: 13-May-2024
  • (2024)Physics-Encoded Graph Neural Networks for Deformation Prediction under Contact2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610366(17160-17166)Online publication date: 13-May-2024
  • (2024)IIPC: Intra-Inter Patch Correlations for Garment Collision Handling2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10687452(1-6)Online publication date: 15-Jul-2024
  • (2024)Neural Modes: Self-supervised Learning of Nonlinear Modal Subspaces2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02185(23158-23167)Online publication date: 16-Jun-2024
  • (2024)Workflow for Creating Animated Digital Replicas of Historical ClothingIEEE Access10.1109/ACCESS.2024.341367412(83707-83718)Online publication date: 2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media