Computational

Large-Eddy Combustion Simulations with CharLESx

CharLESx is the FX Lab’s primary large eddy simulation (LES) solver. It uses the finite volume method to solve the fully compressible Navier-Stokes equations and has a multitude of capabilities which make it well-suited to both fundamental research and applied engineering problems.

Features

Performance:

  • Massively parallel, scaling to hundreds of thousands of CPU cores
  • Automatic mesh partitioning
  • Dynamic load rebalancing in finite-rate chemistry and PEC simulations

Numerics:

  • Structured and unstructured mesh support
  • Low-dissipation numerics and NSCBC boundary conditions for accurate acoustic treatment
  • Split schemes with semi-implicit integration of stiff ODE systems
  • Shock capturing
  • Wall models

State-of-the-art combustion modeling:

  • Finite rate chemistry
  • Multiple tabulated chemistry models
  • PEC, the fidelity-adaptive chemistry framework
  • Radiative heat loss

Advanced flow physics capabilities:

  • Real fluid equations of state for transcritical and supercritical flows
  • Lagrangian spray particles for multiphase flows

Discontinuous Galerkin Methods

Discontinuous Galerkin (DG) methods offer high orders of accuracy with good numerical dispersion and dissipation properties, especially on unstructured grids. They provide geometric flexibility and hp-adaptability, which allows different orders of accuracy to be used in different solution regions for better accuracy. This allows DG solvers to better capture small scale flow structures, flame fronts, and other features common in turbulent reacting flows.

Quail (Ching et al., 2022) is a lightweight, open-source discontinuous Galerkin code written in-house for teaching and prototyping. Vectorized Python operations are used to improve the computational efficiency while retaining code clarity and modularity. Currently, Quail solves first-order and second-order nonlinear systems of partial differential equations common to the flow physics problems tackled in the lab.


Machine Learning

Numerical simulations rely on models to describe physical phenomena. Machine learning techniques offer powerful means to improve the accuracy and computational efficiency of these models. At the FX Lab, these methods have been applied to reactive flow fields for super-resolution problems and on supercritical fluids for prediction of critical properties. Physics-informed neural networks have also been developed to enhance the training procedure of these techniques, bridging the gap between data-driven models and fundamental physics.

BLASTNet

BLASTNet (Bearable Large Accessible Scientific Training Network-of-Datasets) (Chung et al., 2023) is an initiative to curate large amounts of high-fidelity fluid dynamics data in a convenient format for machine learning model applications. There is now over 13 TB of data, which are useful for fluid flows in a wide range of applications tied to automotive, propulsion, energy and the environment. Specifically, scientific engineering tasks related to these domains may include turbulent closure modeling, spatio-temporal modeling, and inverse modeling. In addition, the FX Lab also hosts regular seminars and competitions to disseminate ML for flow physics.


Wildfire Prediction with SwirlLM on Tensor Processing Units (TPUs)

The FX Lab leverages Tensor Processing Units (TPUs) and Graphics Processing Units (GPUs) for machine learning and fire spread simulations. The SwirlLM solver, developed in collaboration with Google Research, is a next-generation, open-source fire/atmosphere solver that captures critical physics of coupled fire/atmosphere interactions.

Performance:

  • Massively parallel, (almost) linear scaling to thousands of TPU cores
  • Code is compiled by the Accelerated Linear Algebra (XLA) compiler with just-in-time (JIT) approach
  • Three-dimensional data structure and numerical operators are designed to optimize the performance within the TensorFlow programming paradigm

Numerics:

  • Semi-implicit finite difference solver with second order accuracy in space and time (Wang et al., 2022)
  • Low-Mach approach with prescribed hydrostatic reference state to allow for larger time steps
  • Two-phase representation of atmospheric fluid flow and solid fuel
  • Lagrangian Particle Tracking for ember transport
  • Cartesian meshes
  • Terrain representation as immersed boundary

Advanced combustion modeling

  • Solid wood combustion model for pyrolysis and gas-phase combustion
  • Evaporation model for fuel moisture
  • Grey gas model for radiative heat transfer

Advanced atmosphere model

  • Low-cloud resolving (Chammas et al., 2023)
  • Anelastic equations for moist air
  • Atmospheric model with three phases: water vapor, liquid water, and ice

References

2023

  1. Turbulence in focus: benchmarking scaling behavior of 3D volumetric super-resolution with BLASTNet 2.0 data
    Wai Tong Chung, Bassem Akoush, Pushan Sharma, Alex Tamkin, Ki Sung Jung, Jacqueline H. Chen, Jack Guo, Davy Brouzet, Mohsen Talei, Bruno Savard, Alexei Y. Poludnenko, and Matthias Ihme
    In Proceedings of the 37th International Conference on Neural Information Processing Systems, New Orleans, LA, USA, Dec 2023
  2. Accelerating Large-Eddy Simulations of Clouds With Tensor Processing Units
    Sheide Chammas, Qing Wang, Tapio Schneider, Matthias Ihme, Yi-fan Chen, and John Anderson
    Journal of Advances in Modeling Earth Systems, Oct 2023

2022

  1. Quail: A lightweight open-source discontinuous Galerkin code in Python for teaching and prototyping
    Eric J. Ching, Brett Bornhoft, Ali Lasemi, and Matthias Ihme
    SoftwareX, Oct 2022
  2. A TensorFlow simulation framework for scientific computing of fluid flows on tensor processing units
    Qing Wang, Matthias Ihme, Yi-Fan Chen, and John Anderson
    Computer Physics Communications, May 2022