The scaffolds at the heart of the study are made of hydroxyapatite, a mineral chemically similar to natural bone, arranged as an orthogonal lattice of rods with pores between 100 and 400 micrometers wide. Implanted into mandibular bone defects in Yucatan minipigs, these structures invite new bone to grow into their macropores, and quantifying how much bone has infiltrated each scaffold is the key to comparing different lattice designs and biological additives. The problem is that bone and hydroxyapatite share nearly identical mineral composition, so their X-ray attenuation values overlap almost completely in micro-CT scans. Ingrown bone also overlaps in brightness with the surrounding soft tissue and noise. Manual labeling of the roughly 1,000 images per scaffold is prohibitively time consuming across studies involving tens of scaffolds, which is exactly where the U-Net convolutional neural network enters the picture.

The U-Net architecture, first introduced in 2015, remains a favorite in biomedical imaging because it delivers strong segmentation from small training sets. Its U-shaped design works like an autoencoder with a memory: an encoding path progressively shrinks the image while doubling its feature channels to detect patterns, a bottleneck connects the two halves, and a decoding path restores the original dimensions, with skip connections threading fine spatial detail from encoder to decoder at every level. The Illinois team used a four-level network starting with eight feature channels and reaching 128 at the bottleneck, ultimately classifying every pixel in a 512-by-512 patch as scaffold, bone, or background. Rather than inventing a new architecture, the researchers held the network constant and varied only the loss function, the mathematical rule that tells the model how wrong its predictions are during training.

Three loss functions were pitted against each other under identical conditions. Cross-entropy loss, the workhorse of classification, evaluates every pixel individually regardless of class balance. Soft Dice loss instead optimizes the overlap between prediction and ground truth, a strategy favored when one class, like bone, occupies only a small fraction of the image. A third, less common option called Accuracy loss directly rewards correct classification of the entire patch. The team trained each model for 1,000 epochs on 225 patches derived from 31 manually labeled images, using four NVIDIA V100 GPUs and confirming that all three losses converged without overfitting. A second expert annotator independently relabeled a subset to gauge human variability, finding a bone Dice score of 0.78 between annotators and an average difference in bone growth fraction of just 1.2 percentage points.

Then came the surprise. Standard evaluation metrics, the numbers researchers typically report to demonstrate that a segmentation model works, produced results that seemed catastrophic but were actually meaningless. Some patches scored a Dice coefficient of zero and a Precision of zero, values usually interpreted as total failure, yet visual inspection showed those patches were segmented perfectly well. The catch: those patches contained no bone in the ground truth at all. When the model predicted a handful of spurious bone pixels in a bone-free image, the Dice and Precision formulas collapsed to zero even though the biological error was negligible. Similar distortions plagued Recall, which tanked on patches where a few missed pixels represented a large fraction of a tiny bone area. Under extreme class imbalance, the familiar metrics were measuring the imbalance, not the model.

To escape this trap, the researchers invented an application-specific endpoint they call delta BGF: simply the difference between the bone growth fraction predicted by the model and the bone growth fraction in the ground truth. This is the number that actually matters for the science, because the entire purpose of segmenting the scans is to quantify bone regeneration and compare treatment groups. Bland-Altman analysis showed the systematic bias between predicted and true bone growth fraction was less than 1.5 percentage points for all three models, comparable to the disagreement between two human experts. When the team correlated the standard metrics against the absolute value of delta BGF, Accuracy emerged as the clear winner, with correlation coefficients reaching negative 0.8 for the Accuracy-loss model and negative 0.78 for the Dice model. A Steiger test confirmed Accuracy’s correlation was significantly stronger than those of Dice, Precision, and Recall in eight of nine comparisons, and bootstrap resampling across 10,000 samples showed the relationship was stable, with confidence intervals lying entirely below zero.

The loss function also shaped where the errors landed, which matters because different regions of the image carry different scientific weight. The Accuracy-loss model tended to overestimate bone, scattering false positives inside the scaffold boundary, precisely where the ingrown bone quantification happens. The cross-entropy and Dice models instead missed bone in the dense ring surrounding the scaffold, errors that are conveniently located outside the region of interest and can be corrected automatically, since scaffold pixels cannot legitimately exist beyond the scaffold boundary. Given this geography of mistakes, the cross-entropy model emerged as the recommended choice for future in vivo studies quantifying spatial variation in bone growth, combining clean class edges within the scaffold with correctable peripheral errors.

Robustness testing added another layer of confidence. The team created augmented test sets by rotating patches 15, 30, and 45 degrees, shifting contrast in both directions, and degrading resolution by factors of 0.66 and 0.5, simulating the natural variation in micro-CT scanning parameters and scaffold orientation. None of these perturbations produced a statistically significant change in Accuracy, Dice, Precision, Recall, or delta BGF. The systematic bias remained below 2.6 percentage points across all augmentations, with contrast changes producing the widest limits of agreement and rotation the narrowest, suggesting that intensity shifts stress the model more than orientation changes. For a field where scanning protocols differ between labs and even between experiments, this resilience means the trained models can plausibly be applied to new datasets without retraining from scratch.

Perhaps the most consequential result is biological rather than computational. Comparing against the group’s earlier in vivo studies, the smallest treatment differences those studies could detect were on the order of 8 to 20 percentage points in bone fraction, vastly larger than the 1.5 percentage point error of the AI segmentation. In other words, the model’s mistakes are too small to obscure genuine biological differences between scaffold designs, meaning future comparisons of bone regeneration should remain statistically sound. The study’s limitations are acknowledged candidly: the 54 test patches derive from only six images, so patch-level statistics treat clustered data as independent, and the authors frame their correlation findings as descriptive rather than confirmatory. Still, the message radiates well beyond bone scaffolds. Whenever deep learning is deployed to extract a quantitative measurement rather than a pretty picture, the evaluation metric should be the measurement itself. A model that aces the standard report card can still flunk the science, and a model that appears to fail it may be quietly doing excellent work.

Subject of Research: Evaluating U-Net deep learning segmentation of bone ingrowth in micro-CT images of hydroxyapatite scaffolds using loss functions and application-specific metrics

Article Title: Segmenting scaffolds with ingrown bone in micro-CT images: considerations of evaluation metrics and loss functions

Article References: Aboutaleb, S., Haug, N., Keni-McCray, P., McCray, A. R. C., Phillips, H., Sharping, S., Cohen, D., Norato, J., & Wagoner Johnson, A. (2026). Segmenting scaffolds with ingrown bone in micro-CT images: considerations of evaluation metrics and loss functions. Medical & Biological Engineering & Computing. https://doi.org/10.1007/s11517-026-03686-x

Image Credits: AI Generated

DOI: 10.1007/s11517-026-03686-x

Keywords: U-Net, semantic segmentation, micro-CT, bone regeneration, hydroxyapatite scaffold, loss functions, evaluation metrics, Dice coefficient, cross-entropy loss, Bland-Altman analysis, deep learning, biomedical imaging

The research, led by Hanh Nguyen Thi of Thuyloi University and Hanoi Architectural University together with colleagues at CMC University, the Posts and Telecommunications Institute of Technology, the Vietnam Academy of Science and Technology, and Hanoi University of Science and Technology, introduces a framework called HCV-MLO, short for Hybrid CNN-ViT with Multi-Loss Optimization. The work extends a conference paper the team presented at MAPR 2025, and it arrives at a moment when content-based image retrieval, known in the field as CBIR, is under growing pressure from ever-larger image collections and increasingly subtle queries.

To understand why the hybrid approach matters, it helps to look at what each architecture does well. Convolutional neural networks, the workhorses of deep learning since the breakthrough ImageNet results of 2012, scan images with filters that detect local patterns: edges, textures, corners, and small motifs that build up into recognizable objects. They are exceptionally good at capturing these fine-grained visual signatures. But convolutions look through a narrow window at each step, so a CNN can struggle to relate a detail in the top-left corner of an image to a feature in the bottom-right, especially when that relationship spans a large distance.

Vision transformers, introduced by Dosovitskiy and colleagues in 2021 with the memorable paper title declaring that an image is worth sixteen by sixteen words, take a different route. They chop an image into patches, treat each patch like a word in a sentence, and use self-attention to let every patch weigh its relationship to every other patch. This gives ViTs a natural talent for modeling long-range dependencies and global context, the kind of scene-level understanding that tells a retrieval system a dog on a beach belongs with other outdoor animal scenes even if the local textures differ. The catch is that transformers, on their own, can be less sensitive to the fine local detail that CNNs capture so effortlessly.

The Vietnamese team’s insight was that these weaknesses are complementary, and that a retrieval system should not have to choose. HCV-MLO runs parallel CNN and ViT branches over the same input image and merges their feature representations into a single embedding, a compact numerical vector that encodes the image’s semantic content. In a retrieval system, similarity between images is computed as distance between these vectors, so the entire game is to learn embeddings in which images of the same category cluster tightly while images of different categories spread apart. The hybrid design gives the embedding both the local texture sensitivity of the convolutional branch and the global relational awareness of the transformer branch.

But architecture alone, the researchers argue, is only half the story. The second pillar of HCV-MLO is its multi-loss optimization strategy, which trains the network by simultaneously minimizing three distinct loss functions: triplet loss, contrastive loss, and cross-entropy loss. Each of these objectives shapes the embedding space in a different way. Triplet loss, made famous by the FaceNet face recognition system in 2015, pulls an anchor image closer to a positive example of the same class while pushing it away from a negative example, directly enforcing the relative ordering that retrieval depends on. Contrastive loss works on pairs of images, rewarding the network when similar images map to nearby points and dissimilar images map to distant ones. Cross-entropy loss, the standard objective for classification, anchors the embedding to clear category boundaries and provides a stable supervisory signal.

To find out what each loss actually contributes, the team built three single-loss variants of their hybrid model: CNN-ViT-CE using cross-entropy alone, CNN-ViT-Contrastive using contrastive loss alone, and CNN-ViT-Triplet using triplet loss alone. This ablation-style analysis, more comprehensive than what appeared in their earlier conference version, allowed them to isolate the effect of each objective on the learned representations. The experiments covered three widely used benchmark datasets that together span a demanding range of retrieval challenges: CIFAR-100, with its one hundred diverse object categories in small, low-resolution images; CUB-200-2011, a fine-grained dataset of two hundred bird species where distinguishing one sparrow from another requires exquisite attention to subtle detail; and Stanford Dogs, another fine-grained benchmark covering one hundred twenty dog breeds.

The results were consistent across all three datasets. The full HCV-MLO model outperformed both single-network baselines, meaning pure CNN or pure ViT systems, and the single-loss variants, meaning the hybrid architecture trained with only one of the three objectives. The headline figure comes from CIFAR-100, where HCV-MLO achieved a mean average precision at rank one hundred, or mAP@100, of 81.99, a clear improvement over recent competitive methods in the field. Mean average precision is a strict metric for retrieval because it rewards systems that place all the relevant results near the top of the ranked list, not merely somewhere in the first hundred. The fact that the gains held up on fine-grained datasets like birds and dogs, where inter-class differences are tiny, suggests that the combination of local and global features with multi-loss training produces embeddings that are genuinely more discriminative, not just better tuned to one benchmark.

The study’s broader message resonates with a trend running through recent deep metric learning research, a field that has produced a rich catalog of loss functions including proxy-based objectives, multi-similarity loss, ranked list loss, and circle loss. Rather than betting on any single recipe, HCV-MLO demonstrates that stacking complementary objectives on top of a complementary architecture yields a practical and effective solution for embedding quality. The authors also note that the datasets used in the study, CIFAR-100, CUB-200-2011, and Stanford Dogs, are all publicly available, and that no new datasets were generated, making the work straightforward for other groups to reproduce and compare against.

For everyday applications, the implications are tangible. Better retrieval embeddings mean medical image archives where a clinician can find prior cases resembling a new scan, e-commerce platforms where a shopper’s camera snapshot surfaces the right product, and digital libraries where a rough sketch or photo unlocks visually related material. As image collections continue to balloon, the hybrid CNN-ViT strategy with multi-loss optimization offers a blueprint for systems that see both the trees and the forest, capturing the fine texture of an individual leaf while never losing sight of the shape of the whole canopy. The research was carried out without specific grant funding, and the authors report no competing financial interests, leaving the door open for the broader community to build on their hybrid, multi-loss recipe.

Subject of Research: A hybrid CNN-ViT feature embedding framework with multi-loss optimization for content-based image retrieval

Article Title: Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization

Article References: Thi, H. N., Huu, Q. N., Thuy, Q. D. T., Le, N. T. T., Van, T. N., & Huu, H. N. (2026). Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization. International Journal of Machine Learning and Cybernetics, 17(9), Article 446. https://doi.org/10.1007/s13042-026-03271-6

Image Credits: AI Generated

DOI: 10.1007/s13042-026-03271-6

Keywords: content-based image retrieval, convolutional neural networks, vision transformers, hybrid architecture, triplet loss, contrastive loss, cross-entropy loss, deep metric learning, feature embedding, CIFAR-100, fine-grained retrieval, HCV-MLO