The research, led by Hanh Nguyen Thi of Thuyloi University and Hanoi Architectural University together with colleagues at CMC University, the Posts and Telecommunications Institute of Technology, the Vietnam Academy of Science and Technology, and Hanoi University of Science and Technology, introduces a framework called HCV-MLO, short for Hybrid CNN-ViT with Multi-Loss Optimization. The work extends a conference paper the team presented at MAPR 2025, and it arrives at a moment when content-based image retrieval, known in the field as CBIR, is under growing pressure from ever-larger image collections and increasingly subtle queries.

To understand why the hybrid approach matters, it helps to look at what each architecture does well. Convolutional neural networks, the workhorses of deep learning since the breakthrough ImageNet results of 2012, scan images with filters that detect local patterns: edges, textures, corners, and small motifs that build up into recognizable objects. They are exceptionally good at capturing these fine-grained visual signatures. But convolutions look through a narrow window at each step, so a CNN can struggle to relate a detail in the top-left corner of an image to a feature in the bottom-right, especially when that relationship spans a large distance.

Vision transformers, introduced by Dosovitskiy and colleagues in 2021 with the memorable paper title declaring that an image is worth sixteen by sixteen words, take a different route. They chop an image into patches, treat each patch like a word in a sentence, and use self-attention to let every patch weigh its relationship to every other patch. This gives ViTs a natural talent for modeling long-range dependencies and global context, the kind of scene-level understanding that tells a retrieval system a dog on a beach belongs with other outdoor animal scenes even if the local textures differ. The catch is that transformers, on their own, can be less sensitive to the fine local detail that CNNs capture so effortlessly.

The Vietnamese team’s insight was that these weaknesses are complementary, and that a retrieval system should not have to choose. HCV-MLO runs parallel CNN and ViT branches over the same input image and merges their feature representations into a single embedding, a compact numerical vector that encodes the image’s semantic content. In a retrieval system, similarity between images is computed as distance between these vectors, so the entire game is to learn embeddings in which images of the same category cluster tightly while images of different categories spread apart. The hybrid design gives the embedding both the local texture sensitivity of the convolutional branch and the global relational awareness of the transformer branch.

But architecture alone, the researchers argue, is only half the story. The second pillar of HCV-MLO is its multi-loss optimization strategy, which trains the network by simultaneously minimizing three distinct loss functions: triplet loss, contrastive loss, and cross-entropy loss. Each of these objectives shapes the embedding space in a different way. Triplet loss, made famous by the FaceNet face recognition system in 2015, pulls an anchor image closer to a positive example of the same class while pushing it away from a negative example, directly enforcing the relative ordering that retrieval depends on. Contrastive loss works on pairs of images, rewarding the network when similar images map to nearby points and dissimilar images map to distant ones. Cross-entropy loss, the standard objective for classification, anchors the embedding to clear category boundaries and provides a stable supervisory signal.

To find out what each loss actually contributes, the team built three single-loss variants of their hybrid model: CNN-ViT-CE using cross-entropy alone, CNN-ViT-Contrastive using contrastive loss alone, and CNN-ViT-Triplet using triplet loss alone. This ablation-style analysis, more comprehensive than what appeared in their earlier conference version, allowed them to isolate the effect of each objective on the learned representations. The experiments covered three widely used benchmark datasets that together span a demanding range of retrieval challenges: CIFAR-100, with its one hundred diverse object categories in small, low-resolution images; CUB-200-2011, a fine-grained dataset of two hundred bird species where distinguishing one sparrow from another requires exquisite attention to subtle detail; and Stanford Dogs, another fine-grained benchmark covering one hundred twenty dog breeds.

The results were consistent across all three datasets. The full HCV-MLO model outperformed both single-network baselines, meaning pure CNN or pure ViT systems, and the single-loss variants, meaning the hybrid architecture trained with only one of the three objectives. The headline figure comes from CIFAR-100, where HCV-MLO achieved a mean average precision at rank one hundred, or mAP@100, of 81.99, a clear improvement over recent competitive methods in the field. Mean average precision is a strict metric for retrieval because it rewards systems that place all the relevant results near the top of the ranked list, not merely somewhere in the first hundred. The fact that the gains held up on fine-grained datasets like birds and dogs, where inter-class differences are tiny, suggests that the combination of local and global features with multi-loss training produces embeddings that are genuinely more discriminative, not just better tuned to one benchmark.

The study’s broader message resonates with a trend running through recent deep metric learning research, a field that has produced a rich catalog of loss functions including proxy-based objectives, multi-similarity loss, ranked list loss, and circle loss. Rather than betting on any single recipe, HCV-MLO demonstrates that stacking complementary objectives on top of a complementary architecture yields a practical and effective solution for embedding quality. The authors also note that the datasets used in the study, CIFAR-100, CUB-200-2011, and Stanford Dogs, are all publicly available, and that no new datasets were generated, making the work straightforward for other groups to reproduce and compare against.

For everyday applications, the implications are tangible. Better retrieval embeddings mean medical image archives where a clinician can find prior cases resembling a new scan, e-commerce platforms where a shopper’s camera snapshot surfaces the right product, and digital libraries where a rough sketch or photo unlocks visually related material. As image collections continue to balloon, the hybrid CNN-ViT strategy with multi-loss optimization offers a blueprint for systems that see both the trees and the forest, capturing the fine texture of an individual leaf while never losing sight of the shape of the whole canopy. The research was carried out without specific grant funding, and the authors report no competing financial interests, leaving the door open for the broader community to build on their hybrid, multi-loss recipe.

Subject of Research: A hybrid CNN-ViT feature embedding framework with multi-loss optimization for content-based image retrieval

Article Title: Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization

Article References: Thi, H. N., Huu, Q. N., Thuy, Q. D. T., Le, N. T. T., Van, T. N., & Huu, H. N. (2026). Hybrid CNN-ViT feature embedding for image retrieval with multi-loss optimization. International Journal of Machine Learning and Cybernetics, 17(9), Article 446. https://doi.org/10.1007/s13042-026-03271-6

Image Credits: AI Generated

DOI: 10.1007/s13042-026-03271-6

Keywords: content-based image retrieval, convolutional neural networks, vision transformers, hybrid architecture, triplet loss, contrastive loss, cross-entropy loss, deep metric learning, feature embedding, CIFAR-100, fine-grained retrieval, HCV-MLO

The study, published in Multimedia Tools and Applications by researchers at the Federal University of Maranhão and the Pontifical Catholic University of Rio de Janeiro, tackles a problem that has long frustrated automated analysis of geological imagery. Content-based image retrieval, or CBIR, allows computers to find images that look similar to a query image without relying on text labels. The approach has proven its worth in face recognition, e-commerce and medical imaging, but rock tomography presents a uniquely hostile environment for such systems. Slices from different rock samples can look strikingly alike, while subtle variations in pore connectivity and grain texture separate samples that matter enormously for fluid flow predictions. A retrieval system that cannot distinguish these fine differences is worse than useless, because it returns plausible-looking but geologically irrelevant results.

The researchers’ solution draws on deep metric learning, a family of techniques that trains neural networks not simply to classify images but to arrange them in a mathematical space where distance equals meaning. In such an embedding space, slices from the same rock volume cluster tightly together while slices from different volumes are pushed far apart. When a user submits a query slice, the system encodes it into this space and returns the nearest neighbors by Euclidean distance. The critical question is how to train the network to build this well-organized space, and this is where the choice of loss function becomes decisive.

Earlier metric learning approaches relied on contrastive loss, which compares pairs of images, or triplet loss, which compares an anchor image against one positive example and one negative example simultaneously. Both formulations suffer from a well-known weakness: they are highly sensitive to which samples happen to appear in each training batch, and they require careful mining of difficult negative examples to work well. N-pair loss, introduced by Google researcher Kihyuk Sohn in 2016, takes a different tack. Instead of contrasting an anchor against a single negative, it pits the anchor-positive pair against multiple negatives at once, using a softmax-like exponential formulation. Each training step therefore delivers a far richer gradient signal, encouraging compact clusters and clear separation between classes without the instability that plagues simpler pairwise methods.

To test whether this multi-negative strategy could handle the subtleties of rock imagery, the team trained models on two publicly available datasets from the Digital Rocks Portal. The first, the 16 Brazilian Pre-Salt Carbonates dataset, contains micro-CT volumes of carbonate rock from the pre-salt formations, captured at voxel resolutions ranging from 6 to 64 micrometers. The second, the 11 Sandstones dataset, includes 1000 slices from each of eleven sandstone varieties, from Bandera Gray to Bentheimer. The researchers preprocessed the volumes into 128-by-128-pixel slices and, crucially, split each volume sequentially along its slicing axis, reserving the first half of every volume for training and the second half for testing. This conservative design prevents the near-identical adjacent slices that characterize tomographic data from leaking between training and evaluation, a pitfall that could artificially inflate performance.

The architecture itself combined well-known convolutional backbones, including EfficientNetB0, EfficientNetV2M, ResNet101, DenseNet201 and MobileNet, all pretrained on ImageNet, with an embedding layer that projects extracted features into a lower-dimensional latent space. The team varied the number of negative samples from two to five and the embedding dimensionality from 16 upward, running each configuration three times to report mean and standard deviation. The results were striking. On the carbonate dataset, a DenseNet201 model with five negative pairs and a 16-dimensional embedding achieved an F1-score of 98.87 plus or minus 1.17 percent and a mean average precision of 98.83 plus or minus 1.22 percent. On the sandstone dataset, an EfficientNetB0 configuration pushed even higher, reaching 99.71 percent F1-score and 99.64 percent mAP, meaning that nearly every retrieved slice genuinely belonged to the same rock volume as the query.

Perhaps the most interesting finding is that no single backbone dominated across both materials. The deep, hierarchical architectures of ResNet101 and DenseNet201 excelled on the heterogeneous, irregularly porous carbonates, while the lighter EfficientNetB0, with its balanced scaling of depth, width and resolution, proved most stable on the more uniform, grain-dominated sandstones. The researchers attribute this to the alignment between each architecture’s inductive biases and the visual character of the material. DenseNet201, which emphasizes fine textural continuity through its densely connected feature reuse, actually became unstable on sandstones, where discrimination depends more on subtle global structural differences than on local texture. The lesson for practitioners is that backbone selection should follow the geology, not the leaderboard.

The team also demonstrated that the gains were not an artifact of their training setup. Compared against a triplet loss model previously published for rock tomography retrieval, a double Siamese network that incorporates porosity and permeability data, and a conventional classification-based retrieval baseline, the N-pair loss approach outperformed all of them on both datasets. On the sandstones, it beat the DSNN approach by roughly seven percentage points of F1-score, even though that competitor had access to extra physical metadata. An ablation study confirmed that adding L2 regularization to the loss, which penalizes overly large feature vector magnitudes, was essential: without it, carbonate performance dropped by more than five percentage points, suggesting that constraining embedding geometry promotes genuine generalization rather than memorization.

Most compelling of all were the cross-dataset experiments, in which models trained exclusively on sandstones were tested on carbonates and vice versa. Performance inevitably fell, to roughly 77 to 80 percent F1-score, but the fact that the models retained any meaningful retrieval ability across such radically different lithologies indicates that the learned embeddings encode transferable structural and textural descriptors rather than dataset-specific shortcuts. The researchers note that if the near-perfect intra-dataset scores had merely reflected the controlled nature of the tomography data, a far larger collapse would have been expected under this severe distribution shift. Instead, the models appear to have captured general morphological features of porous media.

The practical implications extend well beyond academic benchmarks. In oil and gas exploration, identifying and categorizing rock formations directly shapes resource management and extraction decisions, and a retrieval system that instantly surfaces structurally similar samples from vast tomographic archives could accelerate comparative analysis, quality assessment and geological interpretation. The method also requires no porosity or permeability measurements, only images, making it applicable wherever non-visual metadata is scarce. Limitations remain: the validation covered only rock datasets, the training used random rather than adaptive negative sampling, and the model still confuses closely related Berea sandstone subtypes whose petrophysical properties genuinely overlap. The authors point to hard negative mining, margin-based constraints and hybrid metric-classification objectives as promising next steps. For now, though, the work establishes that multi-negative metric learning, long proven on natural image benchmarks, can master the subtle, highly correlated structures of the subsurface, bringing automated digital rock analysis a significant step closer to the drilling rig.

Subject of Research: Deep metric learning with N-pair loss for content-based retrieval of rock tomography images

Article Title: Rock tomography image retrieval: a deep metric learning approach with N-pair loss

Article References: Silva, A., Farias, M., Souza, J., Pinto, D., Belfort, F., Freitas, M., Araújo, A., Pessoa, A., Paiva, A., Silva, A., Rodrigues, A., & Albuquerque, M. (2026). Rock tomography image retrieval: a deep metric learning approach with N-pair loss. Multimedia Tools and Applications, 85(10), Article 772. https://doi.org/10.1007/s11042-026-21936-w

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21936-w

Keywords: content-based image retrieval, rock tomography, N-pair loss, deep metric learning, digital rock analysis, micro-CT, pre-salt carbonates, sandstones, convolutional neural networks, embedding space, petroleum geoscience, machine learning

The research, carried out by Junhan Yang and Xin Zhao of Xi’an University of Science and Technology in China, introduces a scheme called DBRA-Net, short for Dual-Branch Residual Attention Network. The work addresses one of the most persistent tensions in secure multimedia computing: the trade-off between security, retrieval accuracy, and computational cost. Systems that prioritize strong encryption often deliver poor search precision, while systems that achieve high precision frequently rely on heavy neural networks that are impractical to deploy at cloud scale. DBRA-Net is designed to break that compromise, and the reported experimental results suggest it makes genuine progress on all three fronts simultaneously.

The first half of the scheme is a multi-level encryption architecture built for resistance against the attacks that plague naive image ciphers. Images are subjected to pixel confusion, which shuffles the spatial positions of pixel values, and coefficient scrambling, which randomizes the frequency-domain coefficients that compact image content into a small number of visually significant components. The dual approach matters because statistical attacks often exploit structure that survives single-layer scrambling: histograms that leak information about brightness distributions, or correlations between adjacent pixels that betray the underlying scene. By destroying both spatial and frequency-domain structure, the authors report markedly improved resistance against differential and statistical attacks, the two standard families of cryptanalysis applied to image encryption schemes.

Encryption is only as strong as its keys, and the scheme devotes considerable engineering to key management. A key embedding mechanism ties cryptographic keys tightly into the encryption process, while a decentralized key management architecture avoids the single point of failure that a centralized key store would represent. The system also incorporates a key update mechanism, allowing keys to be rotated over time so that long-lived encrypted archives are not protected by the same secret indefinitely. Together, these mechanisms form what the authors describe as a guarantee of key security across the entire lifecycle of the encrypted data, from initial upload through years of cloud storage.

The second, and arguably more novel, half of the work is the neural network that performs retrieval directly on encrypted imagery. Rather than searching for exact pixel matches, modern image retrieval relies on content-based features, compact numerical descriptors that capture the semantic essence of a picture. DBRA-Net generates these descriptors from cipher images using two parallel processing branches, each equipped with a residual attention mechanism. Residual connections, popularized by deep convolutional networks, allow information to bypass layers and gradients to flow efficiently during training, while attention mechanisms let the network selectively emphasize the visual patterns most useful for distinguishing one image from another. The combination enables deeper discriminative feature extraction without the exploding parameter counts that typically accompany deep architectures.

Two further components refine the network’s output. A Grouped Attention Module partitions features into groups and learns separate weighting for each, sharpening the emphasis on the most informative dimensions of the representation. An Adaptive Weighted Fusion mechanism then merges the feature vectors produced by the two branches, weighing their contributions dynamically rather than treating both branches as equally reliable regardless of the image content. This dual-branch philosophy echoes a broader trend in computer vision, where multiple specialized pathways often outperform a single monolithic feature extractor, because different branches can specialize in complementary cues such as texture, color statistics, or structural layout.

The most striking feature of DBRA-Net may be its size. The model contains merely 5.9 million parameters when configured for the Corel-10K dataset, a figure that is small by the standards of contemporary deep networks, some of which carry hundreds of millions or billions of weights. The authors achieved this by adopting lightweight attention computations, exploiting residual connections for representational efficiency, and strictly controlling both network depth and parameter count. The payoff is a network that retains strong representational capacity while imposing low computational overhead, a property the researchers argue is essential for practical deployment on cloud servers, where millions of queries and petabytes of storage demand efficiency as much as intelligence.

The experimental evaluation covers two widely used retrieval benchmarks. On Corel-10K, a challenging collection of ten thousand photographs spanning diverse semantic categories, the scheme achieves a mean average precision, or mAP, of 0.752. On UKbench, a dataset frequently used to evaluate instance-level image retrieval, it reaches an mAP of 0.863. In both cases, the authors report that the results outperform existing retrieval methods, including prior encrypted-image retrieval schemes that relied on handcrafted features, homomorphic encryption, bag-of-words models, and earlier deep learning approaches. The comparison is meaningful because mAP captures not only whether the system finds relevant images but how highly it ranks them among the returned results, a critical property for user-facing search experiences.

The broader significance of the work lies in its alignment with where cloud computing is heading. Privacy regulations, corporate confidentiality requirements, and rising public awareness of data harvesting are pushing more sensitive imagery, from medical scans to industrial photographs, onto encrypted storage. Retrieval over ciphertext, sometimes called searchable encryption for images, has evolved from histogram-based tricks over encrypted JPEG coefficients toward end-to-end deep learning systems that operate on fully encrypted pixels. DBRA-Net represents the latest step in that trajectory, and its lightweight design addresses a criticism that has dogged the field: that deep-learning-based encrypted retrieval is too expensive for real-world cloud deployments.

Challenges remain, as they always do in a young field. The evaluation covers two benchmark datasets, and real-world archives with billions of heterogeneous images would stress any retrieval system further. The security analysis, while strong against differential and statistical attacks, will inevitably be tested by future cryptanalysts probing the scheme’s chaos-based and scrambling components. Nevertheless, the study demonstrates that the long-standing trade-off between searching securely and searching well is not immutable. With careful attention to key management, disciplined network design, and attention mechanisms that squeeze discriminative power out of every parameter, it appears possible to let a cloud server find your images without ever being able to look at them. As more of the world’s visual memory migrates to infrastructure owned by strangers, that capability may shift from a research curiosity to an everyday expectation.

Subject of Research: Privacy-preserving cipher image retrieval in cloud computing using a dual-branch residual attention network

Article Title: DBRA-Net: privacy-preserving cipher image retrieval via dual-branch residual attention network

Article References: Yang, J., & Zhao, X. (2026). DBRA-Net: privacy-preserving cipher image retrieval via dual-branch residual attention network. International Journal of Machine Learning and Cybernetics, 17(10), Article 467. https://doi.org/10.1007/s13042-026-03303-1

Image Credits: AI Generated

DOI: 10.1007/s13042-026-03303-1

Keywords: cipher image retrieval, privacy preservation, cloud computing, image encryption, residual attention, deep learning, feature fusion, dual-branch network, key management, mean average precision, searchable encryption, content-based image retrieval