The research, led by Hanh Nguyen Thi of Thuyloi University and Hanoi Architectural University together with colleagues at CMC University, the Posts and Telecommunications Institute of Technology, the Vietnam Academy of Science and Technology, and Hanoi University of Science and Technology, introduces a framework called HCV-MLO, short for Hybrid CNN-ViT with Multi-Loss Optimization. The work extends a conference paper the team presented at MAPR 2025, and it arrives at a moment when content-based image retrieval, known in the field as CBIR, is under growing pressure from ever-larger image collections and increasingly subtle queries.

To understand why the hybrid approach matters, it helps to look at what each architecture does well. Convolutional neural networks, the workhorses of deep learning since the breakthrough ImageNet results of 2012, scan images with filters that detect local patterns: edges, textures, corners, and small motifs that build up into recognizable objects. They are exceptionally good at capturing these fine-grained visual signatures. But convolutions look through a narrow window at each step, so a CNN can struggle to relate a detail in the top-left corner of an image to a feature in the bottom-right, especially when that relationship spans a large distance.

Vision transformers, introduced by Dosovitskiy and colleagues in 2021 with the memorable paper title declaring that an image is worth sixteen by sixteen words, take a different route. They chop an image into patches, treat each patch like a word in a sentence, and use self-attention to let every patch weigh its relationship to every other patch. This gives ViTs a natural talent for modeling long-range dependencies and global context, the kind of scene-level understanding that tells a retrieval system a dog on a beach belongs with other outdoor animal scenes even if the local textures differ. The catch is that transformers, on their own, can be less sensitive to the fine local detail that CNNs capture so effortlessly.

The Vietnamese team’s insight was that these weaknesses are complementary, and that a retrieval system should not have to choose. HCV-MLO runs parallel CNN and ViT branches over the same input image and merges their feature representations into a single embedding, a compact numerical vector that encodes the image’s semantic content. In a retrieval system, similarity between images is computed as distance between these vectors, so the entire game is to learn embeddings in which images of the same category cluster tightly while images of different categories spread apart. The hybrid design gives the embedding both the local texture sensitivity of the convolutional branch and the global relational awareness of the transformer branch.

But architecture alone, the researchers argue, is only half the story. The second pillar of HCV-MLO is its multi-loss optimization strategy, which trains the network by simultaneously minimizing three distinct loss functions: triplet loss, contrastive loss, and cross-entropy loss. Each of these objectives shapes the embedding space in a different way. Triplet loss, made famous by the FaceNet face recognition system in 2015, pulls an anchor image closer to a positive example of the same class while pushing it away from a negative example, directly enforcing the relative ordering that retrieval depends on. Contrastive loss works on pairs of images, rewarding the network when similar images map to nearby points and dissimilar images map to distant ones. Cross-entropy loss, the standard objective for classification, anchors the embedding to clear category boundaries and provides a stable supervisory signal.

To find out what each loss actually contributes, the team built three single-loss variants of their hybrid model: CNN-ViT-CE using cross-entropy alone, CNN-ViT-Contrastive using contrastive loss alone, and CNN-ViT-Triplet using triplet loss alone. This ablation-style analysis, more comprehensive than what appeared in their earlier conference version, allowed them to isolate the effect of each objective on the learned representations. The experiments covered three widely used benchmark datasets that together span a demanding range of retrieval challenges: CIFAR-100, with its one hundred diverse object categories in small, low-resolution images; CUB-200-2011, a fine-grained dataset of two hundred bird species where distinguishing one sparrow from another requires exquisite attention to subtle detail; and Stanford Dogs, another fine-grained benchmark covering one hundred twenty dog breeds.

The results were consistent across all three datasets. The full HCV-MLO model outperformed both single-network baselines, meaning pure CNN or pure ViT systems, and the single-loss variants, meaning the hybrid architecture trained with only one of the three objectives. The headline figure comes from CIFAR-100, where HCV-MLO achieved a mean average precision at rank one hundred, or mAP@100, of 81.99, a clear improvement over recent competitive methods in the field. Mean average precision is a strict metric for retrieval because it rewards systems that place all the relevant results near the top of the ranked list, not merely somewhere in the first hundred. The fact that the gains held up on fine-grained datasets like birds and dogs, where inter-class differences are tiny, suggests that the combination of local and global features with multi-loss training produces embeddings that are genuinely more discriminative, not just better tuned to one benchmark.

The study’s broader message resonates with a trend running through recent deep metric learning research, a field that has produced a rich catalog of loss functions including proxy-based objectives, multi-similarity loss, ranked list loss, and circle loss. Rather than betting on any single recipe, HCV-MLO demonstrates that stacking complementary objectives on top of a complementary architecture yields a practical and effective solution for embedding quality. The authors also note that the datasets used in the study, CIFAR-100, CUB-200-2011, and Stanford Dogs, are all publicly available, and that no new datasets were generated, making the work straightforward for other groups to reproduce and compare against.

For everyday applications, the implications are tangible. Better retrieval embeddings mean medical image archives where a clinician can find prior cases resembling a new scan, e-commerce platforms where a shopper’s camera snapshot surfaces the right product, and digital libraries where a rough sketch or photo unlocks visually related material. As image collections continue to balloon, the hybrid CNN-ViT strategy with multi-loss optimization offers a blueprint for systems that see both the trees and the forest, capturing the fine texture of an individual leaf while never losing sight of the shape of the whole canopy. The research was carried out without specific grant funding, and the authors report no competing financial interests, leaving the door open for the broader community to build on their hybrid, multi-loss recipe.

Subject of Research: A hybrid CNN-ViT feature embedding framework with multi-loss optimization for content-based image retrieval

Article Title: Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization

Article References: Thi, H. N., Huu, Q. N., Thuy, Q. D. T., Le, N. T. T., Van, T. N., & Huu, H. N. (2026). Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization. International Journal of Machine Learning and Cybernetics, 17(9), Article 446. https://doi.org/10.1007/s13042-026-03271-6

Image Credits: AI Generated

DOI: 10.1007/s13042-026-03271-6

Keywords: content-based image retrieval, convolutional neural networks, vision transformers, hybrid architecture, triplet loss, contrastive loss, cross-entropy loss, deep metric learning, feature embedding, CIFAR-100, fine-grained retrieval, HCV-MLO

The technical heart of the challenge lies in what is called re-identification. Object detection systems can draw boxes around every turkey in a single video frame, but behavior analysis demands something more: the ability to follow the same animal across frames, through occlusions and dense crowding, under a consistent identity. Multi-object tracking algorithms such as DeepSORT accomplish this by combining motion predictions with an appearance embedding—a numerical fingerprint, typically 128 numbers long, that captures what an individual looks like. The trouble is that the appearance model shipped by default with DeepSORT, a network called mars-small128 with roughly 2.8 million parameters, was trained on pedestrians. Humans are easy to tell apart by clothing, texture, and body structure; turkeys in a barn, uniform in color and shape, offer almost none of these cues. The result is that the generic model loses much of its discriminative power the moment it is pointed at poultry.

Sen and Lutz asked whether a compact, domain-specific embedding, trained on actual turkey data, could recover the discrimination that generic models lose while remaining light enough for on-farm deployment. Their answer is a Siamese convolutional neural network of just 714,000 parameters—about a quarter the size of the pedestrian baseline—that maps a cropped image of a turkey to a 128-dimensional vector on the unit hypersphere. The architecture was not designed in one stroke but arrived at through a careful, ablation-driven refinement process, in which each design decision had to justify itself through measurable gains in retrieval accuracy and training stability on held-out, unseen identities.

Three design elements survived that process. First, the network’s residual blocks use pre-activation ordering, a arrangement in which batch normalization and activation functions precede each convolution, keeping the skip connection free of non-linear transformations and improving gradient flow during training. Second, every residual block is augmented with squeeze-and-excitation channel attention, a small gating module that learns to emphasize the most informative feature channels—in this case, apparently, the subtle plumage coloration and body markings that do distinguish one turkey from another. Third, and most impactful, the channel count of the network’s deepest stage was doubled from 64 to 128, placing representational capacity where it matters most rather than in the final projection layer. An embedding head built around a batch-normalization neck, a technique borrowed from person re-identification that decouples metric and classification feature spaces, rounds out the design.

Training combined two complementary objectives: a batch-hard triplet loss, which pulls images of the same bird together in embedding space while pushing images of different birds apart by mining the hardest positive and negative pairs within each batch, and an auxiliary identity-classification loss with label smoothing, which regularizes the network on the modest training set of 74 turkey identities. The underlying dataset, extracted from overhead video of working commercial barns, comprised 106 unique turkey identities and 709 image crops, split so that the validation and test animals were never seen during training—a realistic open-set protocol, since any deployed system will inevitably encounter birds it has never learned from.

The results are striking. On an exhaustive pairwise evaluation of unseen test identities, the domain-specific embedding outperformed mars-small128 on every discrimination metric. The area under the receiver operating characteristic curve rose from 0.890 to 0.943, accuracy from 0.849 to 0.911, and recall by more than ten percent. Most telling was the separation between the similarity distributions of same-bird and different-bird pairs: the gap between their means grew by 73.5 percent, and a separability index improved by roughly half. Because false identity associations in tracking stem directly from overlap between these distributions, this cleaner separation is precisely the property that should translate into more stable identities downstream. Bootstrap confidence intervals confirmed that the gains were not an artifact of the small test pool. A t-SNE visualization of the embedding space showed tight, well-separated clusters for most individuals, with residual overlap confined to the genuinely most similar birds—an honest reflection of the problem’s intrinsic difficulty.

Just as important is what the small model does not cost. With 3.9 times fewer parameters and a footprint of 2.80 MiB versus 10.72 MiB, the custom network runs slightly faster per embedding than the baseline while using comparable GPU memory. Interpretability analyses using Grad-CAM showed that the network concentrates its attention on the bird’s body—the head–neck junction and upper back in standing birds, the breast and flank in resting ones—rather than on barn litter or background clutter, consistent with the squeeze-and-excitation design steering the model toward plumage and body-structure cues.

When the embedding was plugged into a DeepSORT tracking pipeline and tested on three commercial barn sequences, identity-related metrics improved consistently: mean IDF1, a strict measure of identity preservation, rose from 81.3 to 83.2 percent, and association accuracy improved, while detection accuracy remained unchanged—confirming the gain came from better association, not better detection. A controlled stress test that randomly dropped 10 to 30 percent of detections revealed a subtlety: the proposed embedding prefers a more permissive association threshold than the baseline, and once each model was evaluated at its own optimal setting, the domain-specific model won in five of six tested conditions. The authors are candid that the improvement in these particular sequences is moderate, largely because the footage contains few prolonged occlusions—the very regime where a strong appearance model matters most.

They are equally candid about the study’s limits. Identity fragmentation remains severe in absolute terms: even the improved tracker used about 90 track identities to cover roughly 57 real animals, meaning a farm relying on raw track counts would still over-count its flock. The authors identify this fragmentation, along with the need for larger-scale validation across more barns, flocks, and lighting conditions, as the primary remaining obstacle to reliable individual-level monitoring. Still, the central message stands and carries well beyond poultry: appearance representations learned for humans do not transfer to animals without loss, and the remedy is not a larger generic backbone but a smaller, domain-adapted one. For precision-livestock applications where compute and power are constrained, better identity discrimination and lower cost, it turns out, need not be in tension.

Subject of Research: A lightweight domain-specific appearance embedding for individual turkey re-identification in dense commercial barn environments using computer vision and multi-object tracking.

Article Title: A lightweight domain-specific appearance embedding for Turkey re-identification in dense barn environments

Article References: Sen, D., & Lutz, T. (2026). A lightweight domain-specific appearance embedding for Turkey re-identification in dense barn environments. Smart Agricultural Technology, 15, Article 102549. https://doi.org/10.1016/j.atech.2026.102549

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102549

Keywords: turkey re-identification, precision livestock farming, computer vision, multi-object tracking, DeepSORT, Siamese network, appearance embedding, deep metric learning, triplet loss, squeeze-and-excitation attention, animal welfare monitoring, smart agriculture