Marine scientists analyzing vast archives of underwater seafloor imagery face a persistent operational bottleneck: turning millions of raw seabed photographs into biologically meaningful ecological assessments requires painstaking manual annotation by human taxonomic experts. In a comparative evaluation evaluating automated image interpretation under strict labeling constraints, researchers demonstrated that modern vision foundation models, specifically a lightweight DINOv3 architecture paired with a linear support vector classifier, can effectively categorize benthic organisms and seafloor substrates using only tiny pools of labeled reference images. Under evaluated annotation-time assumptions, this few-shot learning approach is estimated to reduce total annotation time on two datasets, offering potential annotation-support advantages without requiring extensive labeled training libraries.
The study, conducted by researchers at the Department of Marine Technology at the Norwegian University of Science and Technology, evaluated the performance of several major pre-trained vision representations across three distinct benthic imagery collections sourced from the Squidle+ platform. The growing use of remotely operated vehicles and autonomous underwater vehicles enables increasingly extensive benthic imaging surveys. However, underwater imagery exhibits substantial visual variability caused by visibility conditions, light scattering, color attenuation, motion blur, and viewpoint changes. These environmental factors create a substantial domain gap between standard terrestrial natural image datasets and subsea observations, which historically impaired the direct transfer of standard convolutional neural networks and vision transformers.
To overcome this data scarcity challenge without relying on time-consuming de novo model training, the investigators formulated a systematic few-shot classification pipeline. The workflow first identifies localized candidate regions using either spatial point layouts or automated segmentation masks generated by the Segment Anything Model. These localized image crops are subsequently passed through frozen vision backbones to extract high-dimensional visual feature embeddings. Rather than updating the internal weights of the deep neural network across long optimization cycles, the few-shot framework constructs a compact support set of annotated examples to train lightweight downstream classifiers. The resulting decision boundaries are then deployed across the remaining unlabeled query crops to generate automated taxonomic predictions that serve as annotation suggestions for human verification and quality control.
In benchmarking model architectures, the research team compared four distinct visual backbone families spanning self-supervised vision transformers, vision-language representations, and standard supervised convolutional baselines. Specifically, the evaluation tested the small DINOv3 vision transformer with sixteen-by-sixteen pixel patches containing 21 million parameters, the small DINOv2 vision transformer with fourteen-by-fourteen patches, standard ResNet-50 models comprising 23.5 million parameters, and OpenAI CLIP variants utilizing both ResNet-50 and Vision Transformer backbones. Embeddings extracted from these frozen encoders were tested across support set budgets ranging from 20 up to 100 labeled samples. The small DINOv3 model consistently demonstrated the strongest overall classification accuracy across the evaluated datasets, establishing that self-supervised representations pre-trained on diverse web-scale imagery without restrictive textual constraints retain rich, transferable low-level and high-level visual features directly applicable to underwater biology.
Crucially, the empirical results revealed that scaling up the parameter count of vision foundation models does not reliably yield superior performance in low-shot benthic classification. When comparing the 21-million-parameter small DINOv3 model against its 86-million-parameter base variant and 300-million-parameter large counterpart, the smallest network frequently matched or exceeded the performance of the larger models. On an Australian sponge-dominated dataset, the small DINOv3 architecture achieved superior accuracy and macro-averaged F1 scores across nearly all sample budgets. While the large model generally achieved the highest macro-F1 and accuracy on Dataset C, the small backbone delivered a favorable balance between classification performance and parameter efficiency.
The study also examined support-set construction strategies, contrasting standard class-balanced random selection against a prioritized sampling framework based on embedding-space clustering. Because real-world marine imagery exhibits severe class imbalance where common abiotic substrates dwarf rare taxa, naive random sampling often fails to capture the visual diversity of the scene. The proposed prioritized strategy partitions unlabeled embeddings via k-means clustering, identifying cluster centroids as initial representatives and using a farthest-first Euclidean distance criterion to select subsequent diverse samples within each cluster without requiring initial ground-truth labels. On Datasets B and C, prioritized sampling improved classification accuracy over class-balanced random sampling, yielding average gains of 7.62 and 5.02 percentage points, respectively, with confidence intervals excluding zero.
However, the researchers observed a nuanced trade-off: these accuracy gains did not consistently improve macro-averaged F1 scores, which treat all taxonomic classes equally regardless of prevalence. Because prioritized clustering targets dense geometric regions in the feature space without explicit knowledge of class frequencies, it can favor visually dominant categories while underrepresenting subtle or sparse organisms. In the four-class discrete object dataset, prioritized sampling improved both overall accuracy and macro-F1 scores, but on datasets with higher class imbalances or subtle taxonomic boundaries, macro-F1 scores occasionally stagnated or declined. This outcome highlights that while prioritized clustering provides reliable overall accuracy improvements that directly minimize total correction effort, human operators monitoring rare indicator species must balance representative embedding coverage against explicit class-balancing protocols.
In comparing adaptation algorithms for mapping extracted embeddings to final discrete categories, the researchers analyzed four inference methods: cosine similarity, k-nearest neighbors, non-linear support vector classifiers with radial basis functions, and linear support vector classifiers. Across the experimental conditions, linear support vector classification proved to be the most dependable and consistent adaptation method. While non-linear support vector classifiers became competitive at higher labeling budgets on some datasets, LinearSVC provided the most consistent performance, achieving higher accuracy on Datasets A and C and superior macro-F1 across all three datasets. Cosine similarity and k-nearest neighbor approaches generally achieved lower performance than LinearSVC across the datasets.
Beyond accuracy metrics, the authors introduced an analytical annotation-time modeling framework to assess the practical economic feasibility of deploying few-shot classification into human-in-the-loop annotation platforms. Incorporating parameters for manual point annotation time, model inference latency, rapid visual verification time, and misclassification correction costs, the mathematical model compared few-shot workflows against fully manual annotation and traditional fine-tuned models requiring hundreds of labeled training examples. The analysis established that few-shot classification is conditionally advantageous rather than universally optimal. For small-to-medium surveys where initial manual labeling budgets are constrained, few-shot classification reduced total annotation time by an estimated 6.7 percent and 21.7 percent compared to fine-tuning on two target datasets, while both machine-assisted pipelines reduced human time by over 40 to 57 percent compared to purely manual labeling.
The researchers noted essential operational limitations, underscoring that few-shot classification functions under a closed-set assumption where unrepresented categories in the support set cannot be correctly identified at inference time. Furthermore, experimental evaluations relied on crop-level data splits, which reflect within-dive or within-survey human-in-the-loop workflows where an expert rapidly labels a small fraction of newly collected images to infer the remainder of the same survey. Because crops from the same photographic transect may share identical lighting and water visibility conditions, the reported metrics reflect within-dataset adaptation rather than zero-shot cross-ocean domain transfer. The authors concluded that foundation-model-based few-shot learning offers marine institutions an immediate, computationally lean mechanism to clear image backlogs, provided deployment strategies dynamically align model selection, support-set sampling, and verification costs with specific ecological survey objectives.
Subject of Research: Few-shot classification and vision foundation models for benthic seafloor habitat image annotation
Article Title: A comparative study of vision foundation models for few-shot classification in benthic imagery
Article References: Elkholy, H., Pizarro, O., & Ludvigsen, M. (2026). A comparative study of vision foundation models for few-shot classification in benthic imagery. Frontiers in Marine Science, 13, 1925773. https://doi.org/10.3389/fmars.2026.1925773
Image Credits: AI Generated
DOI: 10.3389/fmars.2026.1925773
Keywords: benthic imagery, few-shot learning, vision foundation models, DINOv3, marine ecology, image annotation, LinearSVC, ocean observation, underwater robotics, machine learning, habitat mapping, support set selection, clustering, remotely operated vehicles, autonomous underwater vehicles, marine conservation
Cite Scienmag News
Violet Maxwell. (November 20, 2026). Vision Foundation Models Support Few-Shot Classification for Benthic Imagery Annotation. Scienmag. https://scienmag.com/vision-foundation-models-support-few-shot-classification-for-benthic-imagery-annotation/
Violet Maxwell. "Vision Foundation Models Support Few-Shot Classification for Benthic Imagery Annotation." Scienmag, 20 November 2026, https://scienmag.com/vision-foundation-models-support-few-shot-classification-for-benthic-imagery-annotation/. Accessed 20 November 2026.
Violet Maxwell. "Vision Foundation Models Support Few-Shot Classification for Benthic Imagery Annotation." Scienmag. November 20, 2026. https://scienmag.com/vision-foundation-models-support-few-shot-classification-for-benthic-imagery-annotation/








