More Web Proxy on the site http://driver.im/

research-article

Insertion of PETSc in the OpenFOAM Framework

Authors:

Xuejun YangAuthors Info & Claims

ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), Volume 2, Issue 3

Article No.: 16, Pages 1 - 19

https://doi.org/10.1145/3098821

Published: 08 August 2017 Publication History

Abstract

OpenFOAM is a widely used open source framework for simulation in several areas of computational fluid dynamics and engineering. As a partial differential equation (PDE)-based framework, OpenFOAM suffers from a performance bottleneck in solving large-scale sparse linear systems of equations. To address the problem, this article proposes a novel OpenFOAM-PETSc framework by inserting PETSc, a dedicated numerical solving package, into the OpenFOAM to speed up the process of solving linear equation systems. The design of the OpenFOAM-PETSc framework is described, and the implementation of an efficient matrix conversion algorithm is given as a case study. Validation tests on a high-performance computing cluster show that OpenFOAM-PETSc reduces the time of solving PDEs by about 27% in the lid-driven cavity flow case and by more than 50% in flow around the cylinder case in comparison with OpenFOAM, without compromising the scalability. In addition, this article also gives a preliminary performance analysis of different numerical solution methods, which may provide guidelines for further optimizations.

References

[1]

Amani AlOnazi, David Keyes, Alexey Lastovetsky, and Vladimir Rychkov. 2015. Design and optimization of OpenFOAM-based CFD applications for hybrid and heterogeneous HPC platforms. arXiv:1505.07630.

[2]

Allison H. Baker, Martin Schulz, and Ulrike M. Yang. 2011. On the performance of an algebraic multigrid solver on multicore clusters. In High Performance Computing for Computational Science—VECPAR 2010. Springer, 102--115.

Digital Library

[3]

Satish Balay, Shrirang Abhyankar, Mark F. Adams, Jed Brown, Peter Brune, Kris Buschelman, Lisandro Dalcin, et al. 2015. PETSc Home Page. Retrieved July 13, 2017, from http://www.mcs.anl.gov/petsc.

[4]

Massimiliano Culpo. 2011. Current Bottlenecks in the Scalability of OpenFOAM on Massively Parallel Clusters. Retrieved July 13, 2017, from http://www.prace-ri.eu/IMG/pdf/Current_Bottlenecks_in_the_Scalability_of_OpenFOAM_on_Massively_Parallel_Clusters.pdf.

[5]

Paride Dagnaa and Joerg Hertzerb. 2013. Evaluation of Multi-Threaded OpenFOAM Hybridization for Massively Parallel Architectures. Technical Report. Available at http://www.prace-ri.eu/IMG/pdf/wp98,pdf.

[6]

L. D’Amore, A. Murli, V. Boccia, and L. Carracciuolo. 2014. Insertion of PETSc in the NEMO stack software driving NEMO towards exascale computing. In Proceedings of the 2014 International Conference on High Performance Computing and Simulation (HPCS’14). IEEE, Los Alamitos, CA, 724--731.

[7]

Hans De Sterck, Ulrike Meier Yang, and Jeffrey J. Heys. 2006. Reducing complexity in parallel algebraic multigrid preconditioners. SIAM Journal on Matrix Analysis and Applications 27, 4, 1019--1039.

Digital Library

[8]

Robert Falgout and Ulrike Yang. 2002. Hypre: A library of high performance preconditioners. In Computational Science—ICCS 2002. Springer, 632--641.

Digital Library

[9]

Björn Gmeiner, Ulrich Rüde, Holger Stengel, Christian Waluga, and Barbara Wohlmuth. 2015. Performance and scalability of hierarchical hybrid multigrid solvers for Stokes systems. SIAM Journal on Scientific Computing 37, 2, C143--C168.

[10]

W. D. Gropp, D. K. Kaushik, D. E. Keyes, and B. F. Smith. 2000. Performance modeling and tuning of an unstructured mesh CFD application. In Proceedings of the 2000 ACM/IEEE Conference on Supercomputing (SC’00). 1--15.

Digital Library

[11]

William D. Gropp, Dinesh K. Kaushik, David E. Keyes, and Barry F. Smith. 2001. High-performance parallel implicit CFD. Parallel Computing 27, 4, 337--362.

Digital Library

[12]

Raad I. Issa. 1986. Solution of the implicitly discretised fluid flow equations by operator-splitting. Journal of Computational Physics 62, 1, 40--65.

Digital Library

[13]

Hasnat Jamil. 2010. Performance test of PETSc library for the solution of fully coupled velocity-pressure formulation for an unstructured finite volume RANSE solver (Master’s thesis). Retrieved from http://www.cimne.com/cvdata/cntr2/spc31/dtos/img/mdia/thesis/MasterThesisJamilHasnat.pdf.

[14]

Murat Manguoglu. 2012. A General Sparse Sparse Linear System Solver and Its Application in OpenFOAM. Retrieved July 13, 2017, from http://prace-ri-eu/IMG/pdf/A_General_Sparse_Sparse_Linear_System_Solver_and_Its_Application_in_OpenFOAM-2.pdf.

[15]

Nicholas Nethercote and Julian Seward. 2007. Valgrind: A framework for heavyweight dynamic binary instrumentation. ACM SIGPLAN Notices 42, 89--100.

Digital Library

[16]

Jarunan Panyasantisuk. 2009. Integration of PETSc linear solver package into ISIS-CFD flow solver (Master’s thesis). Retrieved from http://www.cimne.com/cvdata/cntr2/spc31/dtos/img/mdia/thesis/MasterThesisJarunanPanyasantisuk.pdf.

[17]

Gavin J. Pringle. 2010. Porting OpenFOAM to HECToR: A dCSE Project. Technical Report. EPCC, University of Edinburgh.

[18]

Orlando Rivera and Karl Fürlinger. 2011. Parallel aspects of OpenFOAM with large eddy simulations. In Proceedings of the 2011 IEEE 13th International Conference on High Performance Computing and Communications (HPCC’11). IEEE, Los Alamitos, CA, 389--396.

Digital Library

[19]

Sebastian Rodriguez. 2013. External Incompressible 3D Flow CFD Analysis OpenFOAM Tutorial. Retrieved July 13, 2017, from http://www.libremechanics.com/?q=node/29.

[20]

J. W. Ruge and K. Stüben. 1987. Algebraic multigrid. Multigrid Methods 3, 13, 73--130.

[21]

Yousef Saad. 2003. Iterative Methods for Sparse Linear Systems. Siam.

Digital Library

[22]

D. Skinner. 2005. Integrated performance monitoring: A portable profiling infrastructure for parallel applications. In Proceedings of the 2005 International Supercomputing Conference (SC’05).

[23]

X. H. Sun and L. M. Ni. 1990. Another view on parallel speedup. In Proceedings of the 1990 International Supercomputing Conference (SC’90). 324--333.

Digital Library

[24]

Mottaqiallah Taouil. 2009. A Hardware Accelerator for the OpenFOAM Sparse Matrix-Vector Product. Ph.D. Dissertation. TU Delft, Delft University of Technology.

[25]

Miao Wang, Yuhua Tang, Xiaowei Guo, and Xiaoguang Ren. 2012. Performance analysis of the graph-partitioning algorithms used in OpenFOAM. In Proceedings of the 2012 IEEE 5th International Conference on Advanced Computational Intelligence (ICACI’12). IEEE, Los Alamitos, CA, 99--104.

[26]

Ren Xiaoguang, Xu Xinhai, Tang Yuhua, Fang Xudong, and Sen Ye. 2013. An application-level synchronous checkpoint-recover method for parallel CFD simulation. In Proceedings of the 2013 IEEE 16th International Conference on Computational Science and Engineering (CSE’13). IEEE, Los Alamitos, CA, 58--65.

Digital Library

Cited By

Liu YWan ZChen QHu BSun RXu CYu HZan J(2024)A Coupled Fully Implicit Method for Solving Drag-adjoint Equations of Steady Incompressible FlowsJournal of Computational Physics10.1016/j.jcp.2024.113251(113251)Online publication date: Jul-2024
https://doi.org/10.1016/j.jcp.2024.113251
Kumar MMangadoddy NBanerjee RSreedhar GRaparla SKumar R(2021)Hydrodynamic force analysis of magnetite medium inside dense medium cyclone using multiphase GPU parallelized ASM modelMinerals Engineering10.1016/j.mineng.2021.107061170(107061)Online publication date: Aug-2021
https://doi.org/10.1016/j.mineng.2021.107061
Lin ZYang WZhou HXu XSun LZhang YTang Y(2018)Communication Optimization for Multiphase Flow Solver in the Library of OpenFOAMWater10.3390/w1010146110:10(1461)Online publication date: 16-Oct-2018
https://doi.org/10.3390/w10101461
Show More Cited By

Index Terms

Insertion of PETSc in the OpenFOAM Framework
1. Applied computing
  1. Physical sciences and engineering
    1. Physics
2. Mathematics of computing
  1. Mathematical software
    1. Mathematical software performance
    2. Solvers

Recommendations

Using a homogeneous equilibrium model for the study of the inner nozzle flow and cavitation pattern in convergent-divergent nozzles of diesel injectors

In this paper, the behaviour of the internal nozzle flow and cavitation phenomenon are numerically studied for non-conventional Diesel convergent-divergent nozzles in order to assess their potential in terms of flow characteristics. The used nozzles ...
Performance assessment of OpenFOAM and FLOW-3D in the numerical modeling of a low Reynolds number hydraulic jump

A comparative performance analysis of the CFD platforms OpenFOAM and FLOW-3D is presented, focusing on a 3D swirling turbulent flow: a steady hydraulic jump at low Reynolds number. Turbulence is treated using RANS approach RNG k-ε. A Volume Of Fluid (...
A quantification method for numerical dissipation in quasi-DNS and under-resolved DNS, and effects of numerical dissipation in quasi-DNS and under-resolved DNS of turbulent channel flows

LES for industrial applications with complex geometries is mostly characterised by: a) a finite volume CFD method using a non-staggered arrangement of the flow variables and second order accurate spatial and temporal discretisation schemes, b) an ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Performance Evaluation of Computing Systems

ACM Transactions on Modeling and Performance Evaluation of Computing Systems Volume 2, Issue 3

September 2017

135 pages

ISSN:2376-3639

EISSN:2376-3647

DOI:10.1145/3119902

Editors:
Sem Borst
Nokia Bell Labs / Eindhoven University of Technology, Netherlands
,
Carey Williamson
University of Calgary, Canada

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 August 2017

Accepted: 01 May 2017

Revised: 01 January 2017

Received: 01 July 2016

Published in TOMPECS Volume 2, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

open fund from the State Key Laboratory of High Performance Computing
National Key Research and Development Program of China
Science Challenge Project

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
463
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)6

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu YWan ZChen QHu BSun RXu CYu HZan J(2024)A Coupled Fully Implicit Method for Solving Drag-adjoint Equations of Steady Incompressible FlowsJournal of Computational Physics10.1016/j.jcp.2024.113251(113251)Online publication date: Jul-2024
https://doi.org/10.1016/j.jcp.2024.113251
Kumar MMangadoddy NBanerjee RSreedhar GRaparla SKumar R(2021)Hydrodynamic force analysis of magnetite medium inside dense medium cyclone using multiphase GPU parallelized ASM modelMinerals Engineering10.1016/j.mineng.2021.107061170(107061)Online publication date: Aug-2021
https://doi.org/10.1016/j.mineng.2021.107061
Lin ZYang WZhou HXu XSun LZhang YTang Y(2018)Communication Optimization for Multiphase Flow Solver in the Library of OpenFOAMWater10.3390/w1010146110:10(1461)Online publication date: 16-Oct-2018
https://doi.org/10.3390/w10101461
Lin ZXu XYang WLi HFeng YZhang Y(2018)A Layered Communication Optimization Method Based on OpenFOAM2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS)10.1109/HPCC/SmartCity/DSS.2018.00084(400-407)Online publication date: Jun-2018
https://doi.org/10.1109/HPCC/SmartCity/DSS.2018.00084

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents