In a paper published in Nonlinear Processes in Geophysics, V. Mithlesh Kumar, Anil Yildiz, and Julia Kowalski of RWTH Aachen University introduce a Bayesian data selection workflow that quantifies, in a rigorous information-theoretic sense, how much each candidate dataset actually contributes to calibrating a landslide runout model. Their central finding is striking: more data does not automatically mean better calibration. What matters is whether an observation captures the specific dynamics that a given parameter governs. An observation that seems superficially relevant — say, the total runout distance, when the ultimate goal is predicting impact area — may be nearly useless for constraining certain parameters, while a less obvious choice, such as a time series of velocity, can be dramatically more informative.

The technical backbone of the approach is Bayesian calibration. Rather than producing a single best-fit parameter value, Bayesian methods represent uncertainty about a parameter as a probability distribution, called the prior, and then update it with observational data to yield a narrower posterior distribution. For landslide models, whose parameters are conceptual rather than physically measurable, this framework is especially attractive because it explicitly handles both measurement noise and model inadequacy — the two dominant sources of discrepancy between what a model predicts and what is actually observed. The catch is computational: Bayesian calibration requires enormous numbers of forward model evaluations, which is why the field has increasingly turned to fast surrogate models, such as Gaussian process emulators, to stand in for the expensive simulation.

Kumar and colleagues go a step further by asking a question that earlier studies had only addressed qualitatively: given several possible observational datasets, which one is most informative? Their answer draws on the Kullback-Leibler divergence, a classic measure from information theory that quantifies how much one probability distribution differs from another. In the Bayesian setting, the KL divergence between the posterior and the prior has a direct interpretation as information gain — it measures how much the data reduced uncertainty about the parameters. A dataset that produces a large KL divergence has done more work; one that leaves the posterior barely changed has contributed little, no matter how abundant or precise it appears.

Computing this quantity is far from trivial, because it involves intractable integrals over posterior distributions. The team sidestepped this with a k-nearest-neighbor-based divergence estimator, and they made the whole pipeline feasible by embedding it in a three-phase workflow built on the Python package PSimPy: first a Gaussian process surrogate is trained on model outputs and validated by cross-validation, then Markov Chain Monte Carlo sampling produces posterior distributions for each candidate dataset, and finally the KL divergences are computed and compared. The result is a matrix that, for every combination of parameter and observation type, quantifies the informational value of that pairing — a systematic tool for deciding where scarce measurement resources should be spent.

To demonstrate the method, the researchers used an idealized lumped mass model governed by the Voellmy rheology, which combines a dry Coulomb friction coefficient that resists basal motion with a velocity-dependent turbulent friction term. They generated synthetic observations with known ground-truth parameters, deliberately isolating the effect of data selection from model error. The experiments compared aggregated observations — maximum velocity and runout distance — against time series of velocity and position, and probed how the length and temporal resolution of the time series influenced the outcome.

The results revealed a clean division of labor. Calibrating with maximum velocity strongly constrained the turbulent friction coefficient but left the Coulomb coefficient nearly untouched, while runout distance did the opposite: it pinned down the Coulomb coefficient while leaving the turbulent coefficient badly biased, with its most probable estimate falling far from the true value. The explanation lies in the physics — runout distance varies almost entirely with the Coulomb friction coefficient, whereas peak velocity is governed by the turbulent term. Time series data, whether of velocity or position, outperformed both aggregated datasets, delivering substantial information about both parameters simultaneously, because they capture the evolving acceleration and deceleration phases that the friction parameters actually control.

Perhaps the most practically valuable insight concerns diminishing returns. As the researchers lengthened the velocity time series used for calibration, information gain rose — but the rate of increase fell steeply and plateaued. Remarkably, the optimal observation window corresponded to the time the sliding mass needed to reach its maximum velocity, the point at which the system’s dynamics have evolved and stabilized. Data collected beyond that threshold added progressively less. This echoes findings from hydrology, where longer calibration records and finer temporal resolution improve parameter inference only up to a point, and it suggests that monitoring campaigns could be designed around the dynamics that matter rather than around maximal data volume.

The workflow also proved capable of inferring the noise characteristics of the observations themselves. When the discrepancy parameters of the statistical noise model were calibrated alongside the friction coefficients, the estimates closely matched the known noise levels used to generate the synthetic data — evidence that rich, time-resolved observations can help quantify the very uncertainty they carry, offsetting the effects of imperfect data quality.

The authors are candid about the limitations of their demonstration. The study relies on synthetic data and idealized topography, and it assumes negligible model discrepancy — an assumption that may not hold for real field observations, where structural model errors can cause posteriors to contract toward incorrect parameter regions, inflating KL divergence in misleading ways. They recommend complementing information-theoretic metrics with posterior predictive checks in such cases. Still, the framework is model-agnostic and transferable to complex models and real terrain, and it offers geohazard practitioners something they have lacked: a way to assess, before an expensive field campaign, which measurements will actually sharpen their predictions. For a hazard community working with sparse, costly data, knowing which observations carry the most information may prove as important as the models themselves.

Subject of Research: Bayesian data selection and information-theoretic quantification of observational data value for calibrating landslide runout model parameters

Article Title: Bayesian data selection to quantify the value of data for landslide runout calibration

Article References: Kumar, V. M., Yildiz, A., & Kowalski, J. (2026). Bayesian data selection to quantify the value of data for landslide runout calibration. Nonlinear Processes in Geophysics, 33(3), 425-453. https://doi.org/10.5194/npg-33-425-2026

Image Credits: AI Generated

DOI: 10.5194/npg-33-425-2026

Keywords: landslide runout, Bayesian calibration, Kullback-Leibler divergence, data selection, information gain, Gaussian process emulator, MCMC, Voellmy rheology, friction parameters, natural hazards, uncertainty quantification, surrogate modeling

The study, published open access in the journal Machine Learning by Sintija Stevanoska, Christian L. Camacho Villalón, Sašo Džeroski and Katharina Dost, tackles a well-known pain point in applied machine learning. Labels are expensive: healthcare systems accumulate millions of patient records, yet expert-annotated outcomes remain scarce. Self-supervised learning promises to bridge that gap by defining pretext tasks from the intrinsic structure of data — masking features and reconstructing them, corrupting values and asking a model to spot the damage, or learning invariant representations across contrastive views. For tables, however, these methods inherit an awkward problem. Unlike images or sentences, tabular rows lack spatial or sequential structure, mix numerical and categorical features, and often contain missing values and tangled inter-dependencies. And pretraining on very large unlabeled pools is computationally expensive while potentially importing noisy or unrepresentative samples that actively degrade learning.

The researchers asked a deceptively simple question: is a small, well-chosen subset of unlabeled data sufficient for effective pretraining? To answer it, they ran what they describe as the first large-scale evaluation of unlabeled data subsampling for tabular self-supervised learning, spanning 25 benchmark datasets drawn from the OpenML-CC18 suite and covering domains from biology and finance to medicine and climate science. Four representative self-supervised models were tested — TabNet, VIME, SCARF and SAINT — deliberately chosen to span the field’s three main paradigms. TabNet and VIME learn by masking or corrupting features and reconstructing them, SCARF relies on contrastive views generated by resampling features from their empirical distributions, and SAINT combines contrastive and denoising objectives inside a transformer that attends over both rows and columns. This diversity of pretext tasks and encoder architectures ensured the conclusions would not hinge on one particular model family.

At the heart of the study lies a family of selection strategies borrowed and adapted from active learning, the branch of machine learning concerned with identifying the most informative examples to label. The team compared random sampling against three principled approaches. Disagreement sampling, based on the classic query-by-committee method, trains a committee of 100 decision trees on bootstrap samples of the labeled data, restricting each tree to a random subset of features in the manner of random forests, and scores every unlabeled point by the normalized entropy of the committee’s votes — a value from 0, indicating complete agreement, to 1, indicating uniform disagreement. Diversity sampling applies k-means clustering to the unlabeled pool and picks one representative per cluster, the point closest to each centroid, enforcing a minimum cluster size to avoid crowning representatives of tiny outlier groups. Optimal transport sampling treats the labeled and unlabeled sets as empirical distributions, computes an entropically regularized transport plan between them with the Sinkhorn algorithm, and assigns each unlabeled point its expected transport cost to the labeled distribution — a low score signaling strong alignment with the labeled data, a high score signaling underrepresented regions of the feature space.

Crucially, all of these strategies are deliberately model-agnostic. They are computed before any self-supervised pretraining begins, using no gradients, losses, embeddings or augmentations from the downstream model. Score-based methods partition the unlabeled pool into percentile brackets — low, medium and high — and samples are then drawn from a chosen bracket, either uniformly at random or via the diversity-based cluster representatives. This design allowed the authors to evaluate the sampling approaches consistently across all four pretext objectives, isolating the effect of data selection from the idiosyncrasies of any particular architecture.

The experimental scale was formidable. Labeled subsets of 100, 500 and 1000 examples were drawn under both unbiased, stratified conditions and a deliberately biased regime in which labeled examples were preferentially selected from one branch of a decision stump, simulating real-world selection bias. Unlabeled subsets of 100 to 1000 examples were drawn under every combination of strategy, subset size, bracket and five random seeds. Performance was measured through three lenses: downstream classification quality using macro-averaged area under the precision-recall curve, total wall-clock time including both selection and training, and representation quality assessed by linear probing — training a simple logistic regression classifier on frozen pretrained embeddings to test whether class information is linearly encoded in the learned latent space.

The headline finding is striking. Subsampling only 100 to 1000 unlabeled examples often matched or exceeded full-data pretraining across both labeled-data regimes. In the low-label, unbiased setting, the best sampling strategies reached up to 119 percent of the full-data baseline and 114 percent of a purely supervised baseline that skipped pretraining entirely. Under biased labeled selection the gains grew even more pronounced, climbing to 123 percent of the full-data baseline and 115 percent of supervised learning. The effect was strongest exactly where it matters most: when labels were scarcest, and the choice of pretraining data had the greatest opportunity to shape what the model learned. Optimal transport distance and disagreement-based sampling performed most consistently, but the humble random sampler was surprisingly competitive, hinting that for small labeled sets, simply reducing redundancy and avoiding overfitting to the unlabeled distribution may itself act as a form of regularization.

The computational story is equally compelling. Pretraining on the full unlabeled pool consistently demanded 60 to 70 percent more time than the sampling strategies, and the selection step itself added a negligible share of the total runtime. Random sampling typically finished in 29 to 42 percent of full-data time, with optimal transport close behind; diversity-based selection was the most expensive to compute, consuming up to 70 percent of the full-data baseline’s runtime without delivering consistent accuracy dividends. In other words, competitive or better predictive performance was achieved at roughly 30 to 40 percent of the computational cost — a margin that alone would justify subsampling in resource-constrained environments even without accuracy improvements. Linear probing reinforced the picture: targeted subsampling consistently matched or exceeded the representation quality of full-data pretraining, and in many configurations full-data self-supervision actually fell below the supervised baseline, an adverse effect the selective strategies appeared to mitigate, particularly under label bias.

To understand when and why subsampling pays off, the team conducted a meta-analysis linking dataset characteristics to observed gains. They extracted statistical and complexity-related metafeatures for each dataset — among them the performance of a nearest-neighbor classifier built on the single most important feature, the stability of local neighborhood structure across cross-validation folds, the minimum support of decision-tree leaves, the median absolute deviation of the features, and trimmed means capturing distributional asymmetry. The patterns that emerged were model-specific rather than universal. TabNet and SCARF gained most from subsampling on datasets where a single dominant feature carries the predictive signal, while SAINT and VIME tended to benefit more on structurally complex tasks; datasets containing a feature with strongly asymmetric central mass favored using all the data. No single metafeature predicted gains across all architectures, pointing to a decisive interplay between dataset geometry and each model’s inductive bias.

The authors are candid about limitations and open questions. The number and size of unlabeled subsets and the granularity of the score brackets were fixed by design, so the study does not determine an optimal data budget for a given dataset, and the meta-analysis, based on 25 datasets, is exploratory rather than confirmatory. Future directions include adaptive budget selection guided by validation performance or meta-learned dataset properties, curriculum-style ordering of the unlabeled pool, incorporating self-supervised model feedback into the selection loop, and training a meta-learner to recommend strategy and subset size automatically. Even so, the message for practitioners is already actionable and, in its own way, subversive: in tabular self-supervised learning, the brute-force assumption that more unlabeled data is always better does not survive scrutiny. A few hundred well-chosen — or even randomly chosen — examples can deliver richer representations and better predictions at a fraction of the cost, a result that could reshape how resource-conscious teams approach pretraining on the structured data that quietly runs the world.

Subject of Research: Unlabeled data selection for efficient self-supervised learning on tabular data

Article Title: Less is More: Unlabeled Data Selection for Efficient Tabular Self-supervised Learning

Article References: Stevanoska, S., Camacho Villalón, C. L., Džeroski, S., & Dost, K. (2026). Less is More: Unlabeled Data Selection for Efficient Tabular Self-supervised Learning. Machine Learning, 115(9), Article 208. https://doi.org/10.1007/s10994-026-07133-8

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07133-8

Keywords: self-supervised learning, tabular data, data selection, unlabeled data, active learning, optimal transport, query-by-committee, computational efficiency, representation learning, machine learning benchmarks, TabNet, VIME