New Algorithm Slashes Simulation Costs for Bayesian Inverse Problems

Engineers who calibrate expensive physics-based simulations against experimental data often face a brutal trade-off: rigorous Bayesian inference demands thousands of model evaluations, but each evaluation of a demanding simulation can take hours. The work, published in Advanced Modeling and Simulation in Engineering Sciences, targets a class of problems that has long resisted efficient statistical treatment.

The researchers, led by Gil Robalo Rei together with Christoph P. Schmidt, Jonas Nitzler, Maximilian Dinkel and Wolfgang A. Wall, developed an algorithm they call adaptive batch-sized regulated importance sampling, or ABRIS. It builds on black box variational inference, a technique that turns the task of approximating a posterior distribution into an optimization problem. The key constraint is that the method never requires derivatives of the underlying simulation model, which means it can be applied to proprietary or legacy codes whose internals are inaccessible.

Bayesian inverse problems arise whenever parameters of a physical model, such as material constants or boundary conditions, must be inferred from noisy measurements. Rather than producing a single best-fit parameter set, Bayesian methods return a full posterior distribution that quantifies uncertainty. The catch is computational: evaluating the likelihood requires running the forward model, and for sophisticated multiphysics simulations each run can be enormously expensive. Depending on the problem, accurate posterior approximation may require thousands of model calls, which can make the inference process prohibitively costly.

Gradient-based variational inference is often the most efficient option in terms of model evaluations, but it demands derivatives of the probabilistic model. Obtaining those derivatives for legacy simulation codes is frequently impractical: automatic differentiation may be incompatible with the existing code base, the adjoint method requires additional problem-specific modeling, and finite differences multiply the number of evaluations by the parameter dimension. The Munich team therefore built their approach on score function estimators, which approximate the gradient of the variational objective using only model evaluations at prescribed inputs, fulfilling the black box property.

The central innovation of ABRIS lies in how it handles the samples already collected. In standard variational inference, the gradient estimate at each iteration requires samples drawn from the current variational distribution, so evaluations from earlier iterations cannot be reused directly. ABRIS circumvents this with importance sampling: the proposal distribution is constructed as a mixture of the variational distributions from previous iterations at which the model was actually evaluated, and reweighting corrects for the mismatch. This allows the algorithm to reuse existing simulation results without introducing bias, reducing the variance of the gradient estimate and, crucially, allowing many parameter updates to proceed without any new model calls at all.

Deciding when new simulations are genuinely needed is the delicate part. If no new samples are ever drawn, the parameter space is no longer explored and the posterior approximation becomes biased. ABRIS therefore combines three criteria. The first is the effective sample size, a standard diagnostic for importance sampling quality. The second is a new criterion tailored to variational inference: the algorithm compares the error of the importance-sampling-based gradient estimate against a reference estimate computed from freshly drawn samples, which are free because they require no model evaluation, and triggers new simulations only when the reused sample set performs worse. The third is a periodic condition that forces new model calls at regular intervals, acting as a safety layer to enforce exploration. In addition, a batch-sequential sampling loop adaptively increases the number of model evaluations per iteration when the available samples prove insufficient, reverting in the worst case to a large batch resembling classical black box variational inference.

In benchmark studies matching Gaussian distributions across dimensions from one to 32, the method consistently outperformed its plain black box variational inference counterpart for dimensions up to eight, reducing the number of model calls by roughly two orders of magnitude at fixed accuracy. The window size parameter had little effect on the required calls, making it easy to set. In higher dimensions, larger batch sizes and window sizes were needed for reliable convergence, and the authors recommend avoiding single-sample batches. Notably, in the eight-dimensional case the sampling loop rarely needed more than two iterations, and toward the end of optimization the periodic criterion dominated, enforcing exploration even near a local optimum while most parameter updates remained virtually free.

The method was then tested against Markov chain Monte Carlo and sequential Monte Carlo on a generalized Poisson calibration problem, in which a spatially varying coefficient field of 20 latent dimensions was inferred from noisy observations of a finite element solution. Averaged over five runs, ABRIS outperformed the alternatives once the number of model calls exceeded 2000. To reach a relative error of approximately five percent in the posterior mean, ABRIS required about 6000 simulation calls, compared with roughly 150,000 for the Metropolis-Hastings Markov chain Monte Carlo scheme and about 161,000 for sequential Monte Carlo, a reduction by factors of roughly 200 and 26 respectively. The authors note that this was achieved without modifying the probabilistic model itself, operating directly on the expensive forward model.

The most demanding demonstration involved Bayesian calibration of a nonlinear electro-chemo-mechanical model of a solid-state lithium-ion battery, a multiphysics continuum model coupling mechanics, charge conservation and mass conservation, solved monolithically with finite elements in the open-source code 4C. Four physical parameters were calibrated from synthetic observations of cell voltage, state-of-charge and lithium amount-of-substance during a discharge cycle, with the observations corrupted by noise. Because certain parameter combinations caused simulations to diverge, the team bounded the parameters and cleverly transformed discrete quantities, such as the number of active material particles and the separator thickness, into continuous variables, exploiting the black box nature of the inference to avoid discrete random variables. A reference posterior was computed with a waste-free sequential Monte Carlo approach using 512 particles, which required 41,472 model calls and roughly 104 hours of wall time per reference solution.

Compared against a smaller sequential Monte Carlo configuration targeting around 1000 forward model calls, the ABRIS approach achieved posterior approximations whose global modes were closer to the ground truth, and its marginal posterior distributions agreed with the reference far better on the separator thickness parameter, with a Cauchy-Schwarz divergence of 0.05 to 0.07 versus 0.80, while using approximately 30 percent fewer forward model calls. The variational approximation did tend to overestimate uncertainty relative to the reference solution, an effect the authors trace to the interaction between the Gaussian mixture variational family, the variational objective and a Tikhonov regularizer introduced to prevent mode collapse; they argue that for calibration problems, conservative overestimation of uncertainty is preferable to overconfident underestimation. The authors caution that the achievable accuracy depends on the choice of variational family, and that the method is best suited to posteriors that simple distributions can approximate well, even in moderately high-dimensional problems up to the 32-dimensional cases studied. The code and data are publicly available on GitHub, and the approach requires no forward model gradients, no surrogate models and no modification of the simulation software, making it applicable wherever validated but computationally demanding physics-based models must be confronted with experimental data.

Subject of Research: A black box variational inference algorithm that reduces simulation calls in Bayesian inverse problems with expensive physics-based models

Article Title: A black box variational inference scheme for inverse problems with demanding physics-based models

Article References: Rei, G. R., Schmidt, C. P., Nitzler, J., Dinkel, M., & Wall, W. A. (2026). A black box variational inference scheme for inverse problems with demanding physics-based models. Advanced Modeling and Simulation in Engineering Sciences, 13(1), Article 19. https://doi.org/10.1186/s40323-026-00335-8

Image Credits: AI Generated

DOI: 10.1186/s40323-026-00335-8

Keywords: Bayesian inference, variational inference, inverse problems, importance sampling, black box models, computational modeling, solid-state batteries, Markov chain Monte Carlo, sequential Monte Carlo, uncertainty quantification, model calibration, scientific computing