[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3468267.3470616acmconferencesArticle/Chapter ViewAbstractPublication PagespascConference Proceedingsconference-collections
research-article
Open access

Fast scalable implicit solver with convergence of equation-based modeling and data-driven learning: earthquake city simulation on low-order unstructured finite element

Published: 26 August 2021 Publication History

Abstract

We developed a new approach in converging equation-based modeling and data-driven learning on high-performance computing resources to accelerate physics-based earthquake simulations. Here, data-driven learning based on data generated while conducting equation-based modeling was used to accelerate the convergence process of an implicit low-order unstructured finite-element solver. This process involved a suitable combination of data-driven learning for estimating high-frequency components and coarsened equation-based models for estimating low-frequency components of the problem. The developed solver achieved a 12.8-fold speedup over the state-of-art solver with a 96.4% size-up scalability up to 24,576 nodes (98,304 MPI processes × 12 OpenMP threads = 1,179,648 CPU cores) of Fugaku with 126,581,788,413 degrees-of-freedom, leading to solving a huge city earthquake shaking analysis in a 10.1-fold shorter time than the previous state-of-the-art solver. Furthermore, to show that the developed method attains high performance on variety of systems with small implementation costs, we ported the developed method to recent GPU systems by use of directive based methods (OpenACC). The equation based modeling and the data-driven learning are of utterly different characteristics, and hence they are rarely combined. The developed approach of combining them is effective, and remarkable results mentioned above are achieved.

References

[1]
N. J. Lindsey, T. C. Dawe, J. B. Ajo-Franklin, Illuminating seafloor faults and ocean dynamics with dark fiber distributed acoustic sensing, Science 29 Nov 2019, Vol. 366, Issue 6469, pp. 1103--1107, 2019.
[2]
Z. E. Ross, D. T. Trugman, E. Hauksson, P. M. Shearer, Searching for hidden earthquakes in Southern California, Science 24 May 2019, Vol. 364, Issue 6442, pp. 767--771, 2019.
[3]
K. J. Bergen, P. A. Johnson, M. V. de Hoop, G. C. Beroza, Machine learning for data-driven discovery in solid Earth geoscience, Science 22 Mar 2019, Vol. 363, Issue 6433, eaau0323, 2019.
[4]
K. Obara, A. Kato, Connecting slow earthquakes to huge earthquakes, Science 15 Jul 2016, Vol. 353, Issue 6296, pp. 253--257, 2016.
[5]
T. Nishikawa, T. Matsuzawa, K. Ohta, N. Uchida, T. Nishimura, S. Ide, The slow earthquake spectrum in the Japan Trench illuminated by the S-net seafloor observatories, Science, 23 Aug 2019, Vol. 365, Issue 6455, pp. 808--813, 2019.
[6]
T. H. Jordan, SC15 Invited Talk Spotlight: Societal Impact of Earthquake Simulations at Extreme Scale.
[7]
J. de la Puente, J. E. Rodriguez, M. Monterrubio-Velasco, O. Rojas, A. Folch, Urgent Supercomputing of Earthquakes: Use Case for Civil Protection, In Proceedings of the Platform for Advanced Scientific Computing Conference (PASC '20), ACM, 2020.
[8]
K. Fujita, T. Ichimura, K. Koyama, M. Horikoshi, H. Inoue, L. Meadows, S. Tanaka, M. Hori, L. Maddegedara, T. Hori, A Fast Implicit Solver with Low Memory Footprint and High Scalability for Comprehensive Earthquake Simulation System, Research Poster for SC16: International Conference for High Performance Computing, Networking, Storage and Analysis, 2016.
[9]
K. Fujita, T. Ichimura, K. Koyama, H. Inoue, M. Hori, L. Maddegedara, Fast and Scalable Low-order Implicit Unstructured Finite-element Solver for Earth's Crust Deformation Problem, In Proceedings of the Platform for Advanced Scientific Computing Conference (PASC' 17), 2017.
[10]
T. Yamaguchi, K. Fujita, T. Ichimura, A. Naruse, J. C. Wells, C. J. Zimmer, T. P. Straatsma, M. Hori, L. Maddegedara, N. Ueda, Low-Order Finite Element Solver with Small Matrix-Matrix Multiplication Accelerated by AI-Specific Hardware for Crustal Deformation Computation, In Proceedings of the Platform for Advanced Scientific Computing Conference (PASC '20), 2020.
[11]
L. Carrington, D. Komatitsch, M. Laurenzano, M. Tikir, D. Michea, N.L. Goff, A. Snavely, J. Tromp, High-frequency simulations of global seismic wave propagation using SPECFEM3DGLOBE, Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC'08), IEEE Press, pp. 1--11, 2008.
[12]
A. Heinecke, A. Breuer, S. Rettenberger, M. Bader, A-A. Gabriel, C. Pelties, A. Bode, W. Barth, X-K. Liao, K. Vaidyanathan, M. Smelyanskiy, P. Dubey, Petascale High Order Dynamic Rupture Earthquake Simulations on Heterogeneous Supercomputers, Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC'14), IEEE Press, 2014.
[13]
H. Fu, C. He, B. Chen, Z. Yin, Z. Zhang, W. Zhang, T. Zhang, W. Xue, W. Liu, W. Yin, G. Yang, X. Chen, 15-Pflops Nonlinear Earthquake Simulation on Sunway TaihuLight: Enabling Depiction of Realistic 10 Hz Scenarios, SC '17 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Article No. 2, 2017.
[14]
T. Ichimura, K. Fujita, T. Yamaguchi, M. Hori, M. Lalith, N. Ueda, AI with Super-Computed Data for Monte Carlo Earthquake Hazard Classification, Research Poster for SC17: International Conference for High Performance Computing, Networking, Storage and Analysis, 2017.
[15]
C. Uphoff, S. Rettenberger, M. Bader, E. H. Madden, T. Ulrich, S. Wollherr, A. Gabriel, Extreme scale multi-physics simulations of the tsunamigenic 2004 sumatra megathrust earthquake, In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '17), 2017.
[16]
T. Ichimura, K. Fujita, S. Tanaka, M. Hori, M. Lalith, Y. Shizawa, H. Kobayashi, Physics-based urban earthquake simulation enhanced by 10.7 BlnDOF x 30 K time-step unstructured FE non-linear seismic wave simulation. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC'14). IEEE Press, pp. 15--26, 2014.
[17]
T. Ichimura, K. Fujita, P. E. B. Quinay, L. Maddegedara, M. Hori, S. Tanaka, Y. Shizawa, H. Kobayashi, K. Minami, Implicit Nonlinear Wave Simulation with 1.08T DOF and 0.270T Unstructured Finite Elements to Enhance Comprehensive Earthquake Simulation. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC'15), ACM, 2015.
[18]
T. Ichimura, K. Fujita, T. Yamaguchi, A. Naruse, J. C. Wells, T. C. Schulthess, T. P. Straatsma, C. J. Zimmer, M. Martinasso, K. Nakajima, M. Hori, L. Maddegedara, A fast scalable implicit solver for nonlinear time-evolution earthquake city problem on low-ordered unstructured finite elements with artificial intelligence and transprecision computing, Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC'18). ACM, Article No. 49, 2018.
[19]
T. Ichimura, K. Fujita, K. Koyama, R. Kusakabe, K. Minami, H. Inoue, S. Nishizawa, M. Tsuji, T. Nishiki, M. Hori, L. Maddegedara, N. Ueda, Fast Scalable Implicit Solver with Convergence of Physics-Based Simulation and Data-Driven Learning: Toward High-Fidelity Simulation with Digital Twin City, Research Poster for the International Conference on High Performance Computing, Networking, Storage and Analysis (SC'20), 2020.
[20]
K. Fujita, K. Katsushima, T. Ichimura, M. Horikoshi, K. Nakajima, M. Hori, L. Maddegedara, Wave Propagation Simulation of Complex Multi-Material Problems with Fast Low-Order Unstructured Finite-Element Meshing and Analysis, Proceedings of the International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2018), 2018.
[21]
J. M. Winget, T. J. Hughes, Solution algorithms for nonlinear transient heat conduction analysis employing element-by-element iterative strategies, Computer Methods in Applied Mechanics and Engineering, vol. 52(1--3), 1985.
[22]
J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, J. N. Kutz, On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 1(2): 391--421, 2014.
[23]
I. Mezić, Analysis of fluid flows via spectral properties of the Koopman operator, Annu. Rev. Fluid Mech., 2013.
[24]
R. T. Q. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud, Neural Ordinary Differential Equations, 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, Canada, 2018.
[25]
G. H. Golub, Q. Ye, Inexact conjugate gradient method with inner-outer iteration, SIAM Journal on Scientific Computing, vol. 21(4), pp. 1305--1320, 1997.
[26]
K. Fujita, K. Koyama, K. Minami, H. Inoue, S. Nishizawa, M. Tsuji, T. Nishiki, T. Ichimura, M. Hori, L. Maddegedara, High-fidelity nonlinear low-order unstructured implicit finite-element seismic simulation of important structures by accelerated element-by-element method, Journal of Computational Science, 2020.
[27]
Supercomputer Fugaku, RIKEN Center for Computational Science [Online]. https://www.r-ccs.riken.jp/en/postk/project
[28]
T. Yoshida, Fujitsu High Performance CPU for the Post-K Computer, IEEE Hot Chips: A Symposium on High Performance Chips, San Jose, August 21, 2018.
[29]
Y. Ajima, et al., The Tofu Interconnect D, IEEE Cluster 2018, 2018.
[30]
Oakbridge-CX. [Online]. https://www.cc.u-tokyo.ac.jp/en/supercomputer/obcx/system.php
[31]
A. Kumar, S. Kottapalli, I. M. Steiner, B. Valentine, I. Hirsh, G. Vedaraman, L. P. Looi, M. Arafa, A. Rudoff, S. Mandava, B. Fahim, and S. A. Vora, Future Intel Xeon Scalable Processor (Codename: Cascade Lake-SP), Hot Chips 30, [Online]. https://www.hotchips.org/hc30/2conf/2.15_Intel_Cascade_Lake_HC30.Intel.Akhilesh.CLXCPU.Final.pdf
[32]
Big Data and Extreme-scale Computing. [Online]. https://www.exascale.org/bdec/
[33]
T. Yamaguchi, T. Ichimura, K. Fujita, M. Hori, L. Wijerathne, N. Ueda, Data-Driven Approach to Inversion Analysis of Three-dimensional Inner Soil Structure via Wave Propagation Analysis, Computational Science - ICCS 2020, Lecture Notes in Computer Science, 2020.

Cited By

View all
  • (2023)69.7-PFlops Extreme Scale Earthquake Simulation with Crossing Multi-faults and Topography on SunwayProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3613209(1-15)Online publication date: 12-Nov-2023

Index Terms

  1. Fast scalable implicit solver with convergence of equation-based modeling and data-driven learning: earthquake city simulation on low-order unstructured finite element

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      PASC '21: Proceedings of the Platform for Advanced Scientific Computing Conference
      July 2021
      186 pages
      ISBN:9781450385633
      DOI:10.1145/3468267
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      In-Cooperation

      • CSCS: Swiss National Supercomputing Centre

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 26 August 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. data driven methods
      2. finite-element modeling
      3. iterative solver
      4. scalability

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      PASC '21
      Sponsor:

      Acceptance Rates

      PASC '21 Paper Acceptance Rate 17 of 33 submissions, 52%;
      Overall Acceptance Rate 109 of 221 submissions, 49%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)140
      • Downloads (Last 6 weeks)18
      Reflects downloads up to 07 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)69.7-PFlops Extreme Scale Earthquake Simulation with Crossing Multi-faults and Topography on SunwayProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1145/3581784.3613209(1-15)Online publication date: 12-Nov-2023

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media