Latent assimilation with implicit neural representations for unknown dynamics
References
[1]
M. Amendola, R. Arcucci, L. Mottet, C.Q. Casas, S. Fan, C. Pain, P. Linden, Y.K. Guo, Data assimilation in the latent space of a convolutional autoencoder, in: M. Paszynski, D. Kranzlmüller, V.V. Krzhizhanovskaya, J.J. Dongarra, P.M. Sloot (Eds.), Computational Science – ICCS 2021, Springer International Publishing, Cham, 2021, pp. 373–386. https://doi.org/10.1007/978-3-030-77977-1_30.
[2]
T. Arcomano, I. Szunyogh, J. Pathak, A. Wikner, B.R. Hunt, E. Ott, A machine learning-based global atmospheric forecast model, Geophys. Res. Lett. 47 (9) (2020),.
[3]
R. Arcucci, L. Mottet, C. Pain, Y.K. Guo, Optimal reduced space for variational data assimilation, J. Comput. Phys. 379 (2019) 51–69,.
[4]
G. Artana, A. Cammilleri, J. Carlier, E. Mémin, Strong and weak constraint variational assimilations for reduced order fluid flow modeling, J. Comput. Phys. 231 (8) (2012) 3264–3288,.
[5]
M. Asch, M. Bocquet, M. Nodet, Data Assimilation, Society for Industrial and Applied Mathematics, Philadelphia, PA, 2016,.
[6]
K. Atkinson, W. Han, Spherical Harmonics and Approximations on the Unit Sphere: An Introduction, Lecture Notes in Mathematics, Springer, 2012.
[7]
T. Bachlechner, B.P. Majumder, H. Mao, G. Cottrell, J. McAuley, Rezero is all you need: fast convergence at large depth, in: Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, PMLR, 2021, pp. 1352–1361.
[8]
D. Bank, N. Koenigstein, R. Giryes, Autoencoders, in: Machine Learning for Data Science Handbook: Data Mining and Knowledge Discovery Handbook, 2023, pp. 353–374.
[9]
M. Bemana, K. Myszkowski, H.P. Seidel, T. Ritschel, X-fields: implicit neural view-, light- and time-image interpolation, ACM Trans. Graph. 39 (6) (2020),.
[10]
G. Berkooz, P. Holmes, J.L. Lumley, The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech. 25 (1) (1993) 539–575,.
[11]
K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, Q. Tian, Accurate medium-range global weather forecasting with 3D neural networks, Nature 619 (7970) (2023) 533–538,.
[12]
S.L. Brunton, J.L. Proctor, J.N. Kutz, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. 113 (15) (2016) 3932–3937,.
[13]
K.J. Burns, G.M. Vasil, J.S. Oishi, D. Lecoanet, B.P. Brown, Dedalus: a flexible framework for numerical simulations with spectral methods, Phys. Rev. Res. 2 (2020),.
[14]
Y. Cao, J. Zhu, Z. Luo, I. Navon, Reduced-order modeling of the upper tropical Pacific Ocean model using proper orthogonal decomposition, Comput. Math. Appl. 52 (8) (2006) 1373–1386,.
[15]
Chen, K.; Han, T.; Gong, J.; Bai, L.; Ling, F.; Luo, J.J.; Chen, X.; Ma, L.; Zhang, T.; Su, R.; et al. (2023): Fengwu: pushing the skillful global medium-range weather forecast beyond 10 days lead. preprint arXiv:2304.02948.
[16]
P.Y. Chen, J. Xiang, D.H. Cho, Y. Chang, G.A. Pershing, H.T. Maia, M.M. Chiaramonte, K.T. Carlberg, E. Grinspun, CROM: continuous reduced-order modeling of PDEs using implicit neural representations, in: International Conference on Learning Representations, 2023.
[17]
R.T. Chen, Y. Rubanova, J. Bettencourt, D.K. Duvenaud, Neural ordinary differential equations, Adv. Neural Inf. Process. Syst. 31 (2018).
[18]
R.T. Chen, B. Amos, M. Nickel, Learning neural event functions for ordinary differential equations, in: International Conference on Learning Representations, 2021.
[19]
Z. Chen, H. Zhang, Learning implicit fields for generative shape modeling, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5939–5948.
[20]
S. Cheng, J. Chen, C. Anastasiou, P. Angeli, O.K. Matar, Y.K. Guo, C.C. Pain, R. Arcucci, Generalised latent assimilation in heterogeneous reduced spaces with machine learning surrogate models, J. Sci. Comput. 94 (1) (2023),.
[21]
Chung, J.; Gülçehre, Ç.; Cho, K.; Bengio, Y. (2014): Empirical evaluation of gated recurrent neural networks on sequence modeling. preprint arXiv:1412.3555.
[22]
P. Cizmas, A. Palacios, T. O'Brien, M. Syamlal, Proper-orthogonal decomposition of spatio-temporal patterns in fluidized beds, Chem. Eng. Sci. 58 (19) (2003) 4417–4427,.
[23]
M.C. Clare, O. Jamil, C.J. Morcrette, Combining distribution-based neural networks to predict weather forecast probabilities, Q. J. R. Meteorol. Soc. 147 (741) (2021) 4337–4357,.
[24]
R.A. DeVore, R. Howard, C. Micchelli, Optimal nonlinear approximation, Manuscr. Math. 63 (1989) 469–478,.
[25]
E. Dupont, H. Kim, S.M.A. Eslami, D.J. Rezende, D. Rosenbaum, From data to functa: your data point is a function and you can treat it like one, in: Proceedings of the 39th International Conference on Machine Learning, 2022.
[26]
E. Dupont, H. Loya, M. Alizadeh, A. Goliński, Y.W. Teh, A. Doucet, Coin++: neural compression across modalities, Trans. Mach. Learn. Res. (2022).
[27]
G. Evensen, Data Assimilation, 2nd edn, Springer, Berlin, Heidelberg, 2009.
[28]
A. Farchi, P. Laloyaux, M. Bonavita, M. Bocquet, Using machine learning to correct model error in data assimilation and forecast applications, Q. J. R. Meteorol. Soc. 147 (739) (2021) 3067–3084,.
[29]
R. Fathony, A.K. Sahu, D. Willmott, J.Z. Kolter, Multiplicative filter networks, in: International Conference on Learning Representations, 2020.
[30]
A. Fillion, M. Bocquet, S. Gratton, S. Gürol, P. Sakov, An iterative ensemble Kalman smoother in presence of additive model error, SIAM/ASA J. Uncertain. Quantificat. 8 (1) (2020) 198–228,.
[31]
S. Fresca, L. Dede', A. Manzoni, A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized pdes, J. Sci. Comput. 87 (2021) 1–36,.
[32]
J. Galewsky, R.K. Scott, L.M. Polvani, An initial-value problem for testing numerical models of the global shallow-water equations, Tellus, Ser. A Dyn. Meteorol. Oceanogr. (2004),.
[33]
A. Glaws, R. King, M. Sprague, Deep learning for in situ data compression of large turbulent flow simulations, Phys. Rev. Fluids 5 (2020),.
[34]
A. Gruber, M. Gunzburger, L. Ju, Z. Wang, A comparison of neural network architectures for data-driven reduced-order modeling, Comput. Methods Appl. Mech. Eng. 393 (2022),.
[35]
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[36]
H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Horányi, J. Muñoz-Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers, A. Simmons, C. Soci, S. Abdalla, X. Abellan, G. Balsamo, P. Bechtold, G. Biavati, J. Bidlot, M. Bonavita, G. De Chiara, P. Dahlgren, D. Dee, M. Diamantakis, R. Dragani, J. Flemming, R. Forbes, M. Fuentes, A. Geer, L. Haimberger, S. Healy, R.J. Hogan, E. Hólm, M. Janisková, S. Keeley, P. Laloyaux, P. Lopez, C. Lupu, G. Radnoti, P. de Rosnay, I. Rozum, F. Vamborg, S. Villaume, J.N. Thépaut, The era5 global reanalysis, Q. J. R. Meteorol. Soc. 146 (730) (2020) 1999–2049,.
[37]
S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput. 9 (8) (1997) 1735–1780,.
[38]
C.M. Jiang, S. Esmaeilzadeh, K. Azizzadenesheli, K. Kashinath, M. Mustafa, H.A. Tchelepi, P. Marcus, Anandkumar A. Prabhat, Meshfreeflownet: a physics-constrained deep continuous space-time super-resolution framework, in: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, IEEE, 2020, pp. 1–15. https://doi.org/10.1109/SC41405.2020.00013.
[39]
E. Kalnay, Atmospheric Modeling, Data Assimilation and Predictability, Cambridge University Press, 2002,.
[40]
S. Kotsuki, Y. Sato, T. Miyoshi, Data assimilation for climate research: model parameter estimation of large-scale condensation scheme, J. Geophys. Res., Atmos. 125 (1) (2020),.
[41]
W.A. Lahoz, P. Schneider, Data assimilation: making sense of Earth observation, Front. Environ. Sci. 2 (2014) 16.
[42]
R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu, A. Merose, S. Hoyer, G. Holland, O. Vinyals, J. Stott, A. Pritzel, S. Mohamed, P. Battaglia, et al., Learning skillful medium-range global weather forecasting, Science 382 (6677) (2023) 1416–1421,.
[43]
Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444,.
[44]
K. Lee, K.T. Carlberg, Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys. 404 (2020),.
[45]
Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Fourier neural operator for parametric partial differential equations, in: International Conference on Learning Representations, 2021.
[46]
Lipton, Z.C.; Berkowitz, J.; Elkan, C. (2015): A critical review of recurrent neural networks for sequence learning. preprint arXiv:1506.00019.
[47]
Z. Long, Y. Lu, B. Dong, Pde-net 2.0: learning pdes from data with a numeric-symbolic hybrid deep network, J. Comput. Phys. 399 (2019),.
[48]
L. Lu, P. Jin, G. Pang, Z. Zhang, G.E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nat. Mach. Intell. 3 (3) (2021) 218–229,.
[49]
M. Lukoševičius, H. Jaeger, Reservoir computing approaches to recurrent neural network training, Comput. Sci. Rev. 3 (3) (2009) 127–149,.
[50]
J.L. Lumley, Stochastic Tools in Turbulence, Academic Press, 1970.
[51]
Y. Luo, K. Ogle, C. Tucker, S. Fei, C. Gao, S. LaDeau, J.S. Clark, D.S. Schimel, Ecological forecasting and data assimilation in a data-rich era, Ecol. Appl. 21 (5) (2011) 1429–1442,.
[52]
A. Maas, A. Hannun, A. Ng, Rectifier nonlinearities improve neural network acoustic models, in: Proceedings of the International Conference on Machine Learning, 2013.
[53]
I. Mehta, M. Gharbi, C. Barnes, E. Shechtman, R. Ramamoorthi, M. Chandraker, Modulated periodic activations for generalizable local functional representations, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14214–14223. https://doi.org/10.1109/ICCV48922.2021.01395.
[54]
P. Nadler, R. Arcucci, Y.K. Guo, Data assimilation for parameter estimation in economic modelling, in: 2019 15th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), IEEE, 2019, pp. 649–656. https://doi.org/10.1109/SITIS.2019.00106.
[55]
S. Niu, Y. Luo, M.C. Dietze, T.F. Keenan, Z. Shi, J. Li, F. Stuart Chapin III, The role of data assimilation in predictive ecology, Ecosphere 5 (5) (2014),.
[56]
Ohlberger, M.; Rave, S. (2015): Reduced basis methods: success, limitations and future challenges. preprint arXiv:1511.02021.
[57]
J.J. Park, P. Florence, J. Straub, R. Newcombe, S. Lovegrove, Deepsdf: learning continuous signed distance functions for shape representation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 165–174.
[58]
Pathak, J.; Subramanian, S.; Harrington, P.; Raja, S.; Chattopadhyay, A.; Mardani, M.; Kurth, T.; Hall, D.; Li, Z.; Azizzadenesheli, K.; Hassanzadeh, P.; Kashinath, K.; Anandkumar, A. (2022): Fourcastnet: a global data-driven high-resolution weather model using adaptive Fourier neural operators. preprint arXiv:2202.11214.
[59]
S. Pawar, O. San, Equation-free surrogate modeling of geophysical flows at the intersection of machine learning and data assimilation, J. Adv. Model. Earth Syst. 14 (11) (2022),.
[60]
S.G. Penny, T.A. Smith, T.C. Chen, J.A. Platt, H.Y. Lin, M. Goodliff, H.D.I. Abarbanel, Integrating recurrent neural networks with data assimilation for scalable data-driven state estimation, J. Adv. Model. Earth Syst. 14 (3) (2022),.
[61]
E. Perez, F. Strub, H. de Vries, V. Dumoulin, A.C. Courville, Film: visual reasoning with a general conditioning layer, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
[62]
M. Peyron, A. Fillion, S. Gürol, V. Marchais, S. Gratton, P. Boudier, G. Goret, Latent space data assimilation by using deep learning, Q. J. R. Meteorol. Soc. 147 (740) (2021) 3759–3777,.
[63]
A. Pinkus, Basic properties of n-widths, Springer Berlin Heidelberg, Berlin, Heidelberg, 1985, pp. 9–38. https://doi.org/10.1007/978-3-642-69894-1_2.
[64]
M. Raissi, P. Perdikaris, G.E. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378 (2019) 686–707,.
[65]
S. Rasp, P.D. Dueben, S. Scher, J.A. Weyn, S. Mouatadid, N. Thuerey, Weatherbench: a benchmark data set for data-driven weather forecasting, J. Adv. Model. Earth Syst. 12 (11) (2020),.
[66]
S. Rasp, N. Thuerey, Data-driven medium-range weather prediction with a resnet pretrained on climate simulations: a new model for weatherbench, J. Adv. Model. Earth Syst. 13 (2) (2021),.
[67]
F. Regazzoni, L. Dedè, A. Quarteroni, Machine learning for fast and reliable solution of time-dependent differential equations, J. Comput. Phys. 397 (2019),.
[68]
S. Scher, G. Messori, Ensemble methods for neural network-based weather forecasts, J. Adv. Model. Earth Syst. 13 (2) (2021),.
[69]
P.J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech. 656 (2010) 5–28,.
[70]
M.G. Schultz, C. Betancourt, B. Gong, F. Kleinert, M. Langguth, L.H. Leufen, A. Mozaffari, S. Stadtler, Can deep learning beat numerical weather prediction?, Philos. Trans. R. Soc. A, Math. Phys. Eng. Sci. 379 (2194) (2021),.
[71]
J. Sirignano, K. Spiliopoulos, Dgm: a deep learning algorithm for solving partial differential equations, J. Comput. Phys. 375 (2018) 1339–1364,.
[72]
L. Sirovich, Turbulence and the dynamics of coherent structures part I: coherent structures, Q. Appl. Math. 45 (3) (1987) 561–571,.
[73]
V. Sitzmann, J. Martel, A. Bergman, D. Lindell, G. Wetzstein, Implicit neural representations with periodic activation functions, Adv. Neural Inf. Process. Syst. 31 (2020) 7462–7473.
[74]
O. Talagrand, 4D-VAR: four-dimensional variational assimilation, in: Advanced Data Assimilation for Geosciences: Lecture Notes of the Les Houches School of Physics: Special Issue, Oxford University Press, 2014, https://doi.org/10.1093/acprof:oso/9780198723844.003.0001.
[75]
M. Tancik, P.P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J.T. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, Adv. Neural Inf. Process. Syst. 33 (2020) 7537–7547.
[76]
A. Tangborn, S.Q. Zhang, Wavelet transform adapted to an approximate Kalman filter system, Appl. Numer. Math. 33 (1) (2000) 307–316,.
[77]
S.M. Udrescu, M. Tegmark, Ai Feynman: a physics-inspired method for symbolic regression, Sci. Adv. 6 (16) (2020),.
[78]
P. Virtanen, R. Gommers, T.E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S.J. van der Walt, M. Brett, J. Wilson, K.J. Millman, N. Mayorov, A.R.J. Nelson, E. Jones, R. Kern, E. Larson, C.J. Carey, İ. Polat, Y. Feng, E.W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E.A. Quintero, C.R. Harris, A.M. Archibald, A.H. Ribeiro, F. Pedregosa, P. van Mulbregt, SciPy 1.0 Contributors: SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nat. Methods 17 (2020) 261–272,.
[79]
E. Weinan, B. Yu, The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat. 6 (1) (2018) 1–12,.
[80]
H. Weller, J. Thuburn, C.J. Cotter, Computational modes and grid imprinting on five quasi-uniform spherical c grids, Mon. Weather Rev. 140 (8) (2012) 2734–2755,.
[81]
J.A. Weyn, D.R. Durran, R. Caruana, Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere, J. Adv. Model. Earth Syst. 12 (9) (2020),.
[82]
Ye, Z.; Liu, H.; Wang, Z.; Dong, B. (2023): Analysis of the decoder width for parametric partial differential equations. preprint arXiv:2306.14390.
[83]
Y. Yin, M. Kirchmeyer, J.Y. Franceschi, A. Rakotomamonjy, P. Gallinari, Continuous pde dynamics forecasting with implicit neural representations, in: International Conference on Learning Representations, 2023.
[84]
J. Zhai, S. Zhang, J. Chen, Q. He, Autoencoder and its various variants, in: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, 2018, pp. 415–419. https://doi.org/10.1109/SMC.2018.00080.
Recommendations
A novel neural network training framework with data assimilation
AbstractIn recent years, the prosperity of deep learning has revolutionized the Artificial Neural Networks. However, the dependence of gradients and the offline training mechanism in the learning algorithms prevents the Artificial Neural Networks from ...
Going Off-Grid: Continuous Implicit Neural Representations for 3D Vascular Modeling
Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge PapersAbstractPersonalised 3D vascular models are valuable for diagnosis, prognosis and treatment planning in patients with cardiovascular disease. Traditionally, such models have been constructed with explicit representations such as meshes and voxel masks, or ...
Implicit neural representation steganography by neuron pruning
AbstractRecently, implicit neural representation (INR) has started to be applied in image steganography. However, the quality of stego and secret images represented by INR is generally low. In this paper, we propose an implicit neural representation ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Elsevier Inc.
Publisher
Academic Press Professional, Inc.
United States
Publication History
Published: 17 July 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Dec 2024