Bayesian Gaussian Processes Improve Physical Surrogate Models Under Data Location Uncertainty

Computational scientists have introduced a Bayesian framework that incorporates input data location uncertainties into Gaussian process regression for approximating mathematical functions and partial differential equations. Developed by researchers Dongwei Ye, Weihao Yan, Christoph Brune, and Mengwu Guo, the method addresses engineering scenarios where experimental observations are collected at imprecise spatial or temporal coordinates. Rather than assuming that input coordinates possess absolute certainty, the framework operates on two distinct data classes: noise-corrupted outputs with certain inputs, and noise-corrupted outputs with uncertain inputs governed by prior probability distributions. By applying Bayes’ rule alongside a Gaussian process prior over the unknown latent function, the framework infers a posterior distribution for the uncertain input coordinates. These quantified location uncertainties are subsequently incorporated into surrogate predictions through marginalization. Numerical demonstrations across one-dimensional functions, the heat equation, and the Allen–Cahn reaction-diffusion equation confirmed that the method achieves consistent generalization performance and substantially reduces predictive uncertainties compared to conventional models.

Data-driven approximation techniques have been extensively developed in scientific computing to serve as alternatives for solving ordinary and partial differential equations. These statistical models achieve accurate predictions by learning latent structures embedded within training data, and many modern formulations incorporate known physical constraints to prevent nonphysical artifacts while improving generalizability. Within scientific machine learning, uncertainty quantification forms a critical component of model validation because mathematical representations inevitably involve incertitude arising from incomplete physical knowledge or intrinsic system variability. In physics-based simulations, uncertainties typically stem from governing model assumptions, empirical parameters, and boundary or initial conditions. Conversely, uncertainties in data-driven modeling originate primarily from training datasets corrupted by numerical discrepancies, physical measurement errors, or stochastic noise. Gaussian process regression has become a leading methodology for data-driven modeling because its probabilistic, kernel-based formulation provides closed-form predictive distributions that quantify predictive uncertainty. By incorporating differential operators directly into kernel definitions, Gaussian processes can serve as physics-constrained surrogate models that characterize relationships between external source terms and system states.

Despite the widespread utility of Gaussian process surrogates, most conventional models assume that all input locations have full certainty. In standard regression formulations, uncertainty is modeled exclusively as an additive white Gaussian noise term corrupting output responses, with variance calibrated as a statistical hyperparameter. However, this assumption frequently proves too strong in practical engineering scenarios. During predictive monitoring of engineering assets, sensors deployed to capture mechanical behaviors can deviate from designated positions due to human installation errors, structural deformation, or ambient environmental interference. Observational data are consequently recorded at uncertain input locations, a difficulty mirrored in robotic navigation mapping and marine system identification. When constructing surrogate models for partial differential equations, exact synthetic data can be generated for known source terms and initial or boundary conditions, but state solution measurements gathered via sensors or experiments remain vulnerable to both coordinate discrepancies and observational measurement noise.

Existing methods addressing uncertain inputs in Gaussian processes generally follow two categories, yet both face practical challenges. The first category adjusts covariance kernel functions directly, either by using second-order Taylor series expansions around input coordinates or by marginalizing input distributions within covariance functions. The second category treats input uncertainty as an additional heteroscedastic noise term on outputs, reflecting that input errors most heavily impact areas where outputs change rapidly. Alternatively, input locations are optimized alongside hyperparameters using gradient descent on the variational evidence lower bound. Crucially, because both paradigms quantify uncertainty directly using specific structural assumptions, they struggle to handle heterogeneous scenarios where input uncertainty affects only a subset of data, a setting typical of partial differential equation surrogates.

To resolve this limitation, the authors developed a Bayesian approach accommodating heterogeneous datasets by dividing training data into certain and uncertain groups. The certain data collection pairs exact coordinate vectors with observed scalar outputs. The uncertain collection pairs noisy scalar outputs with input locations modeled as continuous random variables. Each uncertain location is assigned an independent Gaussian prior distribution defined by a mean vector and a diagonal covariance matrix with uncorrelated coordinates. The prior mean represents an initial engineering guess of the location, while plus or minus two standard deviations define a ninety-five percent confidence interval. The latent function is governed by a zero-mean Gaussian process prior with a covariance kernel parameterized by hyperparameters. When observations contain independent Gaussian white noise, the joint distribution of observed responses and unobserved test outputs follows a normal distribution. Hyperparameters and noise variances are calibrated by maximizing the log-marginal likelihood across certain data or across combined data incorporating prior means.

Posterior distributions of uncertain input locations are inferred via Bayes’ rule by combining coordinate priors with the joint Gaussian likelihood across all observed responses. Because calculating the marginal likelihood requires integrating across uncertain coordinate space and is intractable, the researchers utilize Markov Chain Monte Carlo sampling, specifically the Metropolis algorithm, to draw posterior coordinate samples. To make prediction computationally feasible, the authors designed a two-stage conditioning scheme. Constructing a fresh Gaussian process from scratch for each posterior sample would impose severe computational costs by repeatedly inverting large covariance matrices. Instead, the researchers first condition the Gaussian process solely on the fixed certain observations to obtain intermediate mean and covariance functions. The algorithm then conditions this intermediate process on each posterior coordinate sample. The marginal predictive mean and variance at test points are evaluated via Monte Carlo integration across these samples, substantially reducing computational overhead while rigorously propagating coordinate uncertainty.

The framework extends directly to linear partial differential equations where a linear differential operator acts on an unknown state driven by an external source term. Because linear differential operations preserve Gaussianity, applying the operator directly to the state covariance kernel produces analytical kernel representations for the source term as well as cross-covariance functions linking the state and source. Training data are partitioned into three distinct sets: noise-free synthetic source term evaluations, noise-free state samples from initial and boundary conditions, and experimental state measurements corrupted by noise at uncertain coordinates. By formulating a joint Gaussian likelihood across all three observable sets, the framework leverages known boundary conditions and synthetic physical constraints to infer the posterior distribution of uncertain measurement locations. Maximizing the log-marginal likelihood calibrates kernel hyperparameters and noise variances, allowing physical differential relationships to refine coordinate distributions and enhance the predictive fidelity of the resulting surrogate.

The methodology was evaluated across multiple one-dimensional mathematical benchmark functions, using mean squared error to assess coordinate estimation and mean squared prediction error over one hundred test points to evaluate surrogate accuracy. In an eight-point interpolation test based on a sinusoidal function over a domain up to eight pi, four points had certain locations while four had uncertain locations generated via a Sobol sequence, perturbed by uniform noise with a prior variance of four. A standard Gaussian process trained on uncorrected prior means was severely misled, producing distorted predictions. In contrast, the Bayesian framework applied Metropolis sampling to infer posterior coordinate distributions, driving coordinate mean estimates closer to ground truth while contracting prior variances. When predictions were marginalized across posterior coordinates, the mean squared prediction error dropped from 143.2 under the prior down to 11.9. Subsequent trials across four additional one-dimensional functions using thirty certain and thirty uncertain observations confirmed consistent reductions in predictive variance and superior curve tracking.

The framework was subsequently applied to a physical diffusion problem governed by the one-dimensional heat equation with unit diffusion coefficient, sinusoidal initial conditions, and homogeneous Dirichlet boundary conditions. Synthetic source term values were generated at Chebyshev nodes of the first kind, while boundary and initial conditions were evaluated at Chebyshev nodes of the second kind. Experiments demonstrated that increasing source term samples from 16 to 64 decreased absolute prediction error by approximately an order of magnitude, with 256 samples achieving high approximation accuracy. Using 256 source points, eight uncertain solution observations were introduced, four uncertain in both space and time and four uncertain in space alone, each assigned a prior standard deviation of 0.04. Metropolis sampling drew 2,600 posterior states in 736.09 seconds following 53.95 seconds of hyperparameter optimization using Adam in Jax on a Dell R750xa server with an NVIDIA L40 GPU. Posterior coordinate distributions exhibited substantial variance reductions, particularly along the spatial coordinate where periodic wave patterns provided strong local sensitivity, whereas near-zero sensitivity along the temporal dimension offered limited corrective power. Predictions combining posterior coordinates yielded substantially lower absolute errors and reduced predictive variances.

To examine nonlinear dynamics, the researchers implemented the framework on a one-dimensional Allen–Cahn reaction-diffusion equation with diffusion coefficient 0.01, reaction parameter 5, and a cubic reaction term. Although the reaction term introduces nonlinearity, a joint Gaussian distribution was constructed between the state and linear differential operator by evaluating source data directly from the negative reaction term. With 256 evenly spaced samples capturing phase separation gradients, the maximum baseline error dropped to approximately 0.055. Metropolis sampling generated 2,600 posterior draws in 850.44 seconds following 89.5 seconds of hyperparameter optimization. Posterior means for points in steep gradient regions converged closely with ground truth coordinates, whereas points in flat regions where local sensitivity approached zero showed minimal improvement, confirming that physical gradients dictate location identifiability. In concluding, the authors noted that when uncertain observations dominate, hyperparameter estimation via log-marginal likelihood risks overfitting noise. They suggested future investigations explore fully Bayesian hyperparameter marginalization, variational inference, hybrid empirical Bayes, and sparse inducing-point approximations, publishing all code openly on GitHub.

Subject of Research: Bayesian Gaussian process surrogate modeling with uncertain input data locations

Article Title: PDE-constrained Gaussian process surrogate modeling with uncertain data locations

Article References: Ye, D., Yan, W., Brune, C., & Guo, M. (2025). PDE-constrained Gaussian process surrogate modeling with uncertain data locations. Advanced Modeling and Simulation in Engineering Sciences, 12(1), Article 33. https://doi.org/10.1186/s40323-025-00308-3

Image Credits: AI Generated

DOI: 10.1186/s40323-025-00308-3

Keywords: Gaussian process regression, surrogate modeling, Bayesian inference, uncertainty quantification, partial differential equations, scientific machine learning, Markov Chain Monte Carlo, Allen-Cahn equation, PDE-constrained, Gaussian, process, surrogate