machine learning tasks

regression, classification, clustering, ...

Think in terms of machine learning tasks (regression, classification, ...) rather than specific algorithms (neural networks, Gaussian processes, ...).

regression: matching inputs and continuous outputs

classification: matching inputs and discrete outputs

dim. reduction: finding low-dim. representations

clustering: grouping similar data points

reinforcement learning: sequential decision making (control) under uncertainty

optimizing GAMG settings

with Bayesian optimization (BayesOpt)

GAMG - generalized geometric algebraic multigrid

full GAMG entry in fvSolution

						p
{
	solver                    GAMG;
	smoother                  DICGaussSeidel;
	tolerance                 1e-06;
	relTol                    0.01;
	cacheAgglomeration        yes;
	nCellsInCoarsestLevel     10;
	processorAgglomerator     none;
	nPreSweeps                0;
	preSweepsLevelMultiplier  1;
	maxPreSweeps              10;
	nPostSweeps               2;
	postSweepsLevelMultiplier 1;
	maxPostSweeps             10;
	nFinestSweeps             2;
	interpolateCorrection     no;
	scaleCorrection           yes;
	directSolveCoarsest       no;
	coarsestLevelCorr
	{
		solver          PCG;
		preconditioner  DIC;
		tolerance       1e-06;
		relTol          0.01;
	}
}
						
					

optimal settings depend on

$\rightarrow$ high-dim. search space with uncertainty

Bayesian optimization
optimization of black-box objective function guided by probabilistic surrogate model (Gaussian process)

idea
use BayesOpt to find optimal GAMG settings
for class of simulations

elapsed time for 50 steps; 2D cylinder flow

implementation outline

• Ax for BayesOpt
• SmartSim for orchestration
• ~100 lines Python script

search space definition in config.yaml

						smoother:
	name: "smoother"
	type: "choice"
	value_type: "str"
	is_ordered: False
	sort_values: False
	values: ["FDIC", "DIC", "DICGaussSeidel", "symGaussSeidel", "nonBlockingGaussSeidel", "GaussSeidel"]
nFinestSweeps:
	name: "nFinestSweeps"
	type: "range"
	value_type: "int"
	bounds: [1, 10]
...
						
					

~15% runtime reduction

modeling a turbulent flow

with dynamic mode decomposition (DMD)

DMD = dimensionality reduction + (linear) regression

some notation

dimensionality reduction via SVD

$$\mathbf{M} \approx \mathbf{U}_r\mathbf{\Sigma}_r\mathbf{V}_r^T$$

reduced data matrix

$$\tilde{\mathbf{M}} = \mathbf{U}_r^T\mathbf{M} = \mathbf{\Sigma}_r\mathbf{V}_r^T,\quad \tilde{\mathbf{M}} \in \mathbb{R}^{r\times N}$$

$$N \ll M,\quad r\ll N$$

classical DMD

$$\tilde{\mathbf{x}}_{n+1} = \tilde{\mathbf{A}} \tilde{\mathbf{x}}_{n}$$

noise-corrupted reduced state vector

noise leads to decaying dynamics

forward-backward-consistent DMD

if $\ \tilde{\mathbf{x}}_{n+1} = \tilde{\mathbf{A}} \tilde{\mathbf{x}}_{n}\ $ then $\ \tilde{\mathbf{A}}^{-1}\tilde{\mathbf{x}}_{n+1} = \tilde{\mathbf{x}}_{n}$

$$\underset{\tilde{\mathbf{A}}}{\mathrm{argmin}}\left|\left| \tilde{\mathbf{Y}}-\tilde{\mathbf{A}}\tilde{\mathbf{X}} \right|\right|_F + \left|\left| \tilde{\mathbf{X}}-\tilde{\mathbf{A}}^{-1}\tilde{\mathbf{Y}} \right|\right|_F$$

reference: O. Azencot et al. (2019)

improved robustness, small frequency error

error propagation - "optimized" DMD

$\tilde{\mathbf{x}}_n \approx \tilde{\mathbf{\Phi}}\tilde{\mathbf{\Lambda}}^{n-1}\tilde{\mathbf{b}}$, $\ \tilde{\mathbf{A}}=\tilde{\mathbf{\Phi}}\tilde{\mathbf{\Lambda}}\tilde{\mathbf{\Phi}}^{-1}$, $\ \tilde{\mathbf{b}} = \tilde{\mathbf{\Phi}}^{-1}\tilde{\mathbf{x}}_1$

$$\underset{\tilde{\mathbf{\lambda}},\tilde{\mathbf{\Phi}}_\mathbf{b}}{\mathrm{argmin}}\left|\left| \tilde{\mathbf{M}}-\tilde{\mathbf{\Phi}}_\mathbf{b}\tilde{\mathbf{V}}_{\mathbf{\lambda}} \right|\right|_F$$

reference: T. Askham, J. N. Kutz (2018)

improved robustness, optimization challenging

multiple constraints and noise identification

$ \tilde{\mathbf{x}}_n = \hat{\mathbf{x}}_n + \tilde{\mathbf{n}}_n $

$\hat{\bar{\mathbf{x}}}_{n+h} = \tilde{\mathbf{\Phi}}\tilde{\mathbf{\Lambda}}^{n+h-1}\tilde{\mathbf{\Phi}}^{-1} \hat{\mathbf{x}}_n$

$L_f = \sum\limits_{n=1}^{N-H}\sum\limits_{h=1}^{H} \left|\tilde{\mathbf{x}}_{n+h} - \hat{\bar{\mathbf{x}}}_{n+h}\right|$

$L_b = \sum\limits_{n=H+1}^{N}\sum\limits_{h=1}^{H} \left|\tilde{\mathbf{x}}_{n-h} - \hat{\bar{\mathbf{x}}}_{n-h}\right|$

final optimization problem

$\underset{\tilde{\mathbf{\lambda}},\tilde{\mathbf{\Phi}}, \tilde{\mathbf{N}}}{\mathrm{argmin}} \left(L_f + L_b\right)$

optimization via
automatic differentiation + gradient descent

early stopping prevents overadjustment to data

prediction error for increasing noise level

$|\mathbf{u}^\prime|$ at $Re=dU_\mathrm{in}/\nu=3700$; DNS setup based on
O. Lehmkuhl et al. (2013)

PSD of force coefficients (p-Welch, 4 segments)
$St=fT_\mathrm{conv}$ and $T_\mathrm{conv}=d/U_\mathrm{in}$

relative and cumulative explained variance

$|\mathbf{u}^\prime|$ prediction; $r=10$, $H=5f_\mathrm{vs}$

$|\mathbf{u}^\prime|$ vortex shedding mode

closed-loop flow control

with model-based deep reinforcement learning

closed-loop control benchmark, $Re=100$

evaluation of optimal policy (control law)

separation control, B. Font et al. (2025)

• LES, higher-order spectral elements
• 8 simulations in parallel
• 96 episodes (iterations)
• 6 days turnaround time
• 1152 GPUh (A100)
• $4$ EUR/GPUh $\rightarrow 5$ kEUR

Training cost DrivAer model

• $5$ hours/simulation (1000 MPI ranks)
• $10$ parallel simulations
• $100$ iterations $\rightarrow 20$ days turnaround time
• $20\times 24\times 10\times 1000 \approx 5\times 10^6 $ CPUh
• $0.01-0.05$ EUR/CPUh $\rightarrow 0.5-2$ mEUR

CFD simulations are expensive!

Idea: replace CFD with model(s) in some episodes

Challenge: dealing with surrogate model errors

for e in episodes:
	if not models_reliable():
		sample_trajectories_from_simulation()
		update_models()
	else:
		sample_trajectories_from_models()
	update_policy()
						

auto-regressive surrogate models with weights $\theta_m$

$$ m_{\theta_m} : (\underbrace{S_{n-d}, \ldots, S_{n-1}, S_n}_{\hat{S}_n}, A_n) \rightarrow (S_{n+1}, R_{n+1}) $$

When are the models reliable?

1. evaluate policy for every model
2. compare to previous policy loss
3. switch if loss did not decrease for
   at least $N_\mathrm{thr}$ of the models

normalized training time for MEPPO variants