Applied mathematicians Esteban G. Tabak, Giulio Trigila, and Wenjun Zhao have introduced a data-driven computational methodology called the hierarchical barycenter to simulate conditional probability densities across complex datasets characterized by structured, non-uniform, or partially missing covariates. In many empirical scenarios across statistical learning, observations do not share a uniform set of descriptive features, presenting significant hurdles for classical regression and estimation techniques. The authors address this fundamental challenge by extending the mathematical framework of optimal transport barycenters into a hierarchical structure capable of processing heterogeneous observation groups. By formulating conditional density simulation through measure-preserving transportation maps, the proposed technique extracts shared information across diverse data subsets without forcing artificial uniform vector representations. Published in the journal Machine Learning, the methodology establishes a unified mathematical foundation for conditional distribution simulation, demonstrating clear advantages over classical optimal transport formulations in both controlled synthetic experiments and empirical biomedical measurements.
The central problem stems from the widespread prevalence of non-standard feature vectors in multi-source datasets. In typical conditional density estimation, practitioners aim to determine the conditional probability distribution of an outcome variable given a specific target vector of covariates. Classical statistical tools, such as kernel conditional density estimators, inherently rely on the assumption that every observation possesses an identical, globally defined vector of explanatory factors. However, in practical settings such as multi-category recommendation systems, covariates naturally branch into specialized hierarchies. For example, rating data collected for books may include specific attributes such as page count, whereas records for bicycles encompass entirely distinct descriptors such as component color. Similarly, in clinical research, treatment outcomes are recorded alongside condition-specific physiological qualifiers, such as blood glucose concentrations in diabetic cohorts or core body temperatures during respiratory illnesses, making uniform vectorization mathematically unnatural and practically unfeasible.
Segmenting such heterogeneous datasets into independent sub-problems based on shared covariates represents a rudimentary workaround, but it severely degrades statistical power. When analysts divide training data into isolated clusters corresponding to specific item types, the effective sample size of each subgroup diminishes drastically. This fragmentation prevents models from leveraging cross-cutting factors, such as demographic age, which exert consistent systemic effects across different categories. Furthermore, partitioning data fails to address common scenarios involving missing covariate values, where predictors are uniformly defined across the target population but remain unrecorded for specific samples due to observational limitations. The hierarchical barycenter overcomes these data-fragmentation dilemmas by establishing an optimal transport formulation that explicitly bridges partially overlapping feature subsets, allowing observations from feature-rich categories to systematically inform estimation tasks within data-constrained or incompletely observed subpopulations without discarding partial records.
The theoretical foundation of the framework extends the distributional barycenter problem previously developed in optimal transport literature. In the standard distributional barycenter formulation, one considers a set of paired observations drawn from an unknown joint probability distribution defined as the product of a marginal covariate density and a conditional outcome density. The objective is to identify a deterministic transport map that transforms the outcome variables into a new latent representation while conditioning on the observed covariates. Crucially, this transport map is designed to eliminate all variability in the outcome variable that can be explained by the covariates, and strictly only that variability. This deformation is constrained such that the resulting latent variable is statistically independent of the conditioning factors while minimizing a transport deformation cost, which is selected as the expected squared Euclidean norm between the original observations and their mapped representations.
Once the optimal transport map is determined, sampling from the conditional probability distribution for any arbitrary target covariate value proceeds by inverting the transformation. The latent variables, which represent the intrinsic variation of the system stripped of covariate dependency, are pushed back through the inverted map parameterized by the target covariate values. In the hierarchical generalization developed by the authors, defining statistical independence between the latent variables and the covariates requires accommodating variables that exist only on localized subsets of the domain. The authors resolve this structural ambiguity by modeling the covariate space as a directed tree or a broader graphical network, introducing auxiliary categorical indicator variables that explicitly delineate whether a particular covariate branch is active or unobserved for each individual record. With this covariate extension, the problem formulation reduces to a regular barycenter problem where the latent coordinates remain independent of both the branch identifiers and the branch-specific features.
To enforce statistical independence between the transformed variables and complex covariate graphs, the authors implement an objective function grounded in information-theoretic penalties. While independence can theoretically be characterized through weak formulations with measurable test functions or conditional density criteria, the researchers focus on the vanishing of mutual information between the transformed variables and designated subsets of covariates. Formulating independence as a non-negative mutual information penalty yields significant structural advantages, allowing the optimization problem to be solved through standard penalty regularizations. Moreover, because high-dimensional joint distributions often suffer from sparse sample coverage, global independence constraints can become empirically inaccurate. Decomposing the full covariate vector into smaller subsets enables the model to penalize mutual information across manageable lower-dimensional projections, including individual covariates, thereby enforcing rigorous marginal independence without succumbing to the curse of dimensionality in data-sparse domains.
To operationalize the continuous optimal transport problem on finite empirical datasets, the authors reformulate the objective using sample averages and non-parametric kernel estimators. The theoretical transport cost is approximated directly by the empirical mean of pairwise Euclidean distances between sample points and their mapped targets. Simultaneously, the mutual information terms governing statistical independence are estimated using kernel density approximations, where continuous variables are smoothed via Gaussian kernels and discrete or categorical indicators are handled using exact Kronecker delta functions. When computing the gradient of the resulting kernelized loss function with respect to the mapped points, the authors establish that derivatives taken with respect to the kernel centers represent zero-mean random variables with vanishing variance in the large-sample limit. This key mathematical property permits a simplified, statistically consistent gradient formulation that neglects inner kernel center derivatives, significantly decreasing computational overhead during iterative updates.
Numerical minimization of the resulting empirical objective is executed through regularized gradient descent accelerated by a preconditioning procedure derived from implicit gradient descent. Standard second-order Newton-type methods require inverting dense Hessian matrices whose dimensions scale with the total number of training samples, creating an insurmountable computational bottleneck for iterative optimization. To retain the regularizing benefits of implicit descent without incurring full matrix inversions, the authors employ a preconditioner containing exclusively the diagonal elements of the Hessian matrix. The iteration is paired with an adaptive learning rate strategy that maintains small step sizes far from local extrema, replicating stable standard gradient descent, while dynamically expanding step sizes in the vicinity of the minimum. This adaptive acceleration converges smoothly toward quasi-Newton behavior, delivering robust convergence rates without risking numerical instability across complex optimization surfaces.
A notable operational benefit of the proposed architecture is that map inversion does not require auxiliary regression models, nearest-neighbor interpolations, or neural approximations. Instead, the first-order optimality condition of the variational objective function yields a closed-form analytic expression for the inverse transport map. Because the transport cost adopts a canonical squared Euclidean metric, setting the gradient of the loss with respect to the mapped variables to zero produces an explicit relationship defining the original outcome variable as a smooth function of the latent coordinates and the covariate values. Consequently, generating new conditional samples for unobserved target covariates simply involves evaluating this explicit analytical formula across the latent barycentric coordinates. Hyperparameter selection for the kernel bandwidths is performed using Silverman’s rule-of-thumb adjusted for sub-sample cardinalities, while penalty coefficients governing mutual information are systematically tuned through cross-validation on validation log-likelihood metrics.
The empirical efficacy of the hierarchical barycenter was established through rigorous synthetic evaluations and real-world clinical benchmarks. In synthetic tests across Gaussian systems exhibiting additive means, non-additive interactions, and heterogeneous standard deviations, the method achieved significantly lower Kullback-Leibler divergence than complete-case analysis or nearest-neighbor imputation, approaching the accuracy of fully observed baselines. In clinical evaluations on spinal bone mineral density measurements across 485 North American adolescents, where covariates of age and sex were masked across subgroups, the hierarchical solver attained the highest average test log-likelihood across 30 random realizations. Crucially, the model faithfully captured the physiological heteroscedasticity of female bone density decline with age, outperforming standard barycentric baselines that failed to resolve subtle conditional variance shifts.
The framework also demonstrated substantial utility in out-of-domain extrapolation and latent factor identification across structured cofactor configurations. When integrating auxiliary datasets that contained non-overlapping predictor values, the hierarchical solver accurately simulated conditional distributions across regions lacking in-sample observations, while successfully uncovering bimodal latent noise structures obscured by known covariates. Runtime measurements across 200 optimization iterations confirmed that solve times scale roughly quadratically with sample size, matching the expected computational demands of Gram matrix manipulations in kernelized mutual information estimations. The authors acknowledge that scaling this methodology to massive industrial datasets requires further mathematical adaptations to mitigate Gram matrix operations, noting that recent structural reformulations of independence constraints offer promising pathways toward achieving large-scale empirical scalability.
Subject of Research: Optimal transport methodology for conditional probability density simulation with structured and missing covariates
Article Title: The Hierarchical Barycenter: Conditional Probability Simulation with Structured and Unobserved Covariates
Article References: Tabak, E. G., Trigila, G., & Zhao, W. (2026). The Hierarchical Barycenter: Conditional Probability Simulation with Structured and Unobserved Covariates. Machine Learning, 115(11), Article 258. https://doi.org/10.1007/s10994-026-07186-9
Image Credits: AI Generated
DOI: 10.1007/s10994-026-07186-9
Keywords: hierarchical barycenter, optimal transport, conditional probability simulation, missing covariates, structured cofactors, distributional barycenter, mutual information, kernel density estimation, preconditioned gradient descent, map inversion, bone mineral density, latent factor discovery, heteroscedasticity, Gram matrix operations, transfer learning, multi-task learning, Kullback-Leibler divergence
Cite Scienmag News
Courtney Benton. (November 20, 2026). Hierarchical Barycenters Enable Conditional Probability Simulation from Heterogeneous and Incomplete Data. Scienmag. https://scienmag.com/hierarchical-barycenters-enable-conditional-probability-simulation-from-heterogeneous-and-incomplete-data/
Courtney Benton. "Hierarchical Barycenters Enable Conditional Probability Simulation from Heterogeneous and Incomplete Data." Scienmag, 20 November 2026, https://scienmag.com/hierarchical-barycenters-enable-conditional-probability-simulation-from-heterogeneous-and-incomplete-data/. Accessed 20 November 2026.
Courtney Benton. "Hierarchical Barycenters Enable Conditional Probability Simulation from Heterogeneous and Incomplete Data." Scienmag. November 20, 2026. https://scienmag.com/hierarchical-barycenters-enable-conditional-probability-simulation-from-heterogeneous-and-incomplete-data/








