Massive hyperspectral dataset teaches AI to spot bare soil from the sky

Bare soil may not look like much, but for scientists trying to feed a growing planet, it is one of the most valuable signals in the sky. When a field is stripped of vegetation, satellites and aircraft can read its surface directly, estimating moisture, nutrients, organic matter and texture from the way sunlight reflects off the ground. The catch is that before any of that chemistry can be inferred, algorithms must first decide which pixels actually show soil and which show roads, buildings, shadows or stubble. A new benchmark called HyBEAR, published in Earth System Science Data, now gives the research community its largest and most rigorously labeled resource for exactly that task, and it could reshape how precision agriculture is done from orbit.

The dataset, assembled by a team from the Silesian University of Technology, Opole University of Technology, AGH University of Kraków, KP Labs, QZ Solutions and the European Space Agency’s ϕ-Lab, contains 1,954 hyperspectral image patches covering more than 43,000 hectares of southern Poland. That translates to roughly 108 million labeled pixels, each carrying 430 spectral bands spanning the visible, near-infrared and short-wave infrared ranges from 414 to 2,357 nanometers. According to the authors, it is the largest and most heterogeneous collection of its kind for bare soil detection, and the first to provide pixel-level annotations for entire agricultural parcels rather than isolated, spatially unmoored pixels.

The imagery was acquired on 3 March 2021 under cloudless, windless skies, using a HySpex VS-725 system flown aboard a Piper PA-31 Navajo aircraft at altitudes between 2,550 and 2,700 meters. Two sensors worked in tandem: a VNIR-1800 capturing 186 bands at 3.26 nanometer resolution across 400 to 1,000 nanometers, and a SWIR-384 capturing 288 bands at 5.45 nanometer resolution across 930 to 2,500 nanometers. The resulting ground sampling distance of 2 meters is fine enough to resolve individual field boundaries, hedgerows and dirt tracks, which matters enormously when the goal is to delineate whole fields rather than rough statistical averages.

What makes HyBEAR genuinely novel is its labeling philosophy. Most existing collections treat bare soil detection as a pixel-by-pixel classification problem, which can mislead downstream applications. Fertilization planning, tillage monitoring and erosion assessment all operate at the level of entire fields, so what farmers and agronomists actually need is an algorithm that says: this parcel, as a whole, is free of vegetation. HyBEAR’s annotators therefore labeled complete parcels, applying a strict rule that a field qualified as SOIL only when at least roughly 85 percent of its surface showed visible bare ground with no signs of mature vegetation. Ambiguous cases were flagged as MAYBE-SOIL and resolved through consensus among three experts with one, four and ten years of remote sensing experience, with the most senior scientist arbitrating the hardest calls.

Getting those labels right was harder than it sounds. Vegetation indices such as NDVI, which flag chlorophyll activity, can miss sparse or freshly emerging plants, while roads, rooftops and artificial surfaces often show equally low index values and can be mistaken for soil. Plain RGB composites proved similarly treacherous, since bare earth, dirt roads and shadowed ground can look nearly identical depending on flight altitude and sun angle. The team combined automated index analysis with careful visual inspection of both natural-color and color-infrared renderings, explicitly excluding dirt roads, infrastructure, deep shadows near tree lines and areas where crop residues complicated the spectral signature. The result is a ground truth designed to survive contact with messy reality.

Heterogeneity was built in deliberately. The two source scenes, P1 near the village of Przeworno in Lower Silesia and P2 south of Głubczyce in the Opolskie region, lie more than 60 kilometers apart yet were flown within an hour of each other. Because the sun and clouds shifted between acquisitions, lighting and reflectance conditions differ measurably between the two areas, forcing any trained model to generalize rather than memorize. The 250-by-250-pixel patches were divided into five spatially disjoint folds, and the benchmark prescribes a five-fold cross-validation protocol in which each fold serves once as the test set. This design directly probes whether an algorithm tuned on one landscape can transfer its knowledge to another, a question that haunts every operational Earth observation system.

To give future researchers a starting line, the authors ran six classic machine learning models through the protocol: logistic regression, L2-regularized linear models, AdaBoost, linear-kernel support vector machines, decision trees and random forests, each operating on the full 430-dimensional spectral vector of every pixel. The results were revealing. Logistic regression and support vector machines led the pack, with average accuracy around 0.926 to 0.927 and F-scores approaching 0.9. More striking was what happened on Fold 0, the only fold drawn from the P1 scene. Non-linear models such as random forests, AdaBoost and decision trees, which had apparently overfitted the spectral character of the larger P2 scene, degraded sharply when tested across locations, while the simpler linear models held up. In machine learning, the humblest method winning is often the most instructive outcome.

The failure cases are as informative as the successes. Qualitative inspection of patches like IMG_0022_F0 shows that errors cluster around physical confounders: tree shadows that mimic dark soil, dirt roads that share its texture, and crop residues that spectrally mix with the underlying ground. Fold 0 also carries the lowest soil pixel ratio in the dataset, at 28.3 percent in the full version, compounding the difficulty. The authors argue that these weaknesses point toward deep learning approaches capable of learning spatial context and richer representations, rather than judging each pixel in isolation. They also note that the benchmark’s parcel-level annotations remain largely untapped by the current pixel-wise baselines, leaving a clear opening for field-level segmentation methods.

Beyond agriculture, HyBEAR has an unexpected role to play in spacecraft engineering. As hyperspectral instruments proliferate on satellites, the bottleneck is no longer acquisition but transmission and on-board computing. The authors describe bare soil detection as a form of smart data compression: by pruning everything that is not soil before downlink, a satellite can slash the volume of data it ships to the ground and reserve its limited memory and compute for pixels that actually matter. Extracting soil parameters from non-soil pixels would produce noisy, inherently incorrect estimates anyway, so filtering early is both an efficiency gain and an accuracy safeguard. Related work from the team, including in-orbit vegetation index detection on the Intuition-1 mission and even quantum-kernel support vector machines, suggests this pipeline is already moving toward flight hardware.

The benchmark ships with reproducibility baked in. Alongside the imagery and labels, the Zenodo release includes wavelength metadata for all 430 bands, Jupyter notebooks covering preprocessing, normalization and the full cross-validation logic, and the ten trained baseline models so that anyone can independently verify the reported numbers. The authors are candid about limitations: the data represents a single early-spring snapshot of one region, so global soil diversity and seasonal variation remain untested, and the evaluation primarily addresses regional rather than worldwide generalization. Future expansions are planned to bring in more geographic zones and timeframes. Even so, HyBEAR fills a gap the community has long acknowledged, replacing ad hoc comparisons with a shared yardstick, and giving the fast-growing field of AI-driven soil mapping something it has rarely had: a fair, expert-verified and genuinely difficult test.

Subject of Research: A large-scale hyperspectral benchmark dataset with expert-verified parcel-level annotations for bare soil detection in precision agriculture

Article Title: HyBEAR: a hyperspectral benchmark for bare soil detection

Article References: Wijata, A. M., Ruszczak, B., Niepala, A., Gumiela, M., Smykala, K., Longépé, N., & Nalepa, J. (2026). HyBEAR: a hyperspectral benchmark for bare soil detection. Earth System Science Data, 18(10), 7301-7317. https://doi.org/10.5194/essd-18-7301-2026

Image Credits: AI Generated

DOI: 10.5194/essd-18-7301-2026

Keywords: hyperspectral imaging, bare soil detection, precision agriculture, remote sensing, machine learning benchmark, Earth observation, soil mapping, cross-validation, satellite data compression, agricultural datasets, HyBEAR, hyperspectral