The research, led by Yawen Liu and Wei Tang with colleagues Zhen Zhang and Xinrong Chen, addresses a persistent bottleneck in robotic manipulation. Conventional approaches to force sensing rely on physical instruments such as force and torque sensors or tactile arrays mounted directly on the gripper. Rigid-body models calculate grasping force through explicit mechanical equations, while soft-body models attempt to capture the nonlinear deformations of compliant structures. Although advances in materials science, including conductive hydrogels that endow soft robots with human-like sensory capabilities, have pushed the field forward, physical sensors bring inherent drawbacks: they add system complexity and cost, and their measurement range is limited, making it difficult to adapt to the diverse and dynamic demands of different grasping tasks.

Vision-based pressure estimation has emerged as an elegant alternative. By pointing cameras at the interaction between a gripper and an object, computer vision techniques can infer the magnitude and distribution of applied forces without any physical contact with the measured surface. This eliminates the need for structural modifications to the gripper and offers flexible, scalable perception. Yet most existing methods, including the current state-of-the-art system known as HiVPE, rely on single RGB images. That restriction means the network sees only surface appearance, missing explicit three-dimensional geometric deformation and high-level semantic context, both of which become crucial when dealing with severe occlusions or the complex hyperelastic behavior of soft materials.

The new framework breaks free of that single-modality constraint by weaving together four distinct streams of information. The first is the ordinary RGB image capturing the interaction between the gripper and a high-resolution pressure sensing array. The second is a binary segmentation mask that precisely delineates the gripper’s contact regions, generated by a pre-trained deep segmentation model that isolates the hand from the background. The third is a depth map produced from the RGB image by a monocular depth estimation network, encoding the geometric and spatial relationships of the contact. The fourth is perhaps the most surprising: a short textual prompt, either the word Straight or Roll, describing the gripper’s action, which is converted into a semantic embedding by the CLIP text encoder, the same contrastive language-image model that underpins many modern multimodal AI systems.

Architecturally, the system proceeds in three stages: feature extraction, feature fusion, and decoding. During extraction, a dual-branch hierarchical encoding scheme employs two parallel SE-ResNeXt50 networks. The first processes the raw RGB image to produce five feature maps at progressively abstract scales, spanning downsampling factors from 2 to 32 and channel dimensions from 64 to 2048, capturing everything from low-level textures to high-level semantics. The second encoder jointly processes the concatenated segmentation and depth inputs, yielding a parallel set of features rich in structural and geometric cues. In this way, one branch learns what the scene looks like while the other learns how it is shaped in space.

The fusion stage is where the framework’s distinctive machinery comes into play. Multi-head self-attention, with four attention heads of 64 dimensions each operating on 256-dimensional features, first refines the highest-resolution feature maps within each modality, strengthening intra-modal contextual dependencies through residual connections and feed-forward networks. A bidirectional cross-attention operator then lets the two streams talk to each other: the appearance features absorb structural context from the segmentation and depth branch, while the structural features incorporate appearance cues from the RGB branch. Finally, the 512-dimensional textual embedding is projected through a multilayer perceptron into a 2880-dimensional feature space, reshaped to align spatially with the visual features and concatenated to form a unified multimodal representation. A Feature Pyramid Network decoder then performs top-down aggregation, fusing high-level semantic cues with low-level spatial details before a segmentation head produces the final dense pressure prediction.

Training and evaluation were carried out on a publicly available benchmark containing roughly 650,000 synchronized RGB-pressure frames from two very different soft grippers, one tendon-actuated and one pneumatic. Ground-truth pressure annotations came from a high-resolution Sensel Morph pressure sensing array, spatially aligned with the camera images through calibration and homography transformation. The dataset covers four interaction types: contact, slide, close, and no-contact, the last serving as adversarial samples. The authors discretized continuous pressure values into nine logarithmically spaced categories, from zero background to a top bin of 64 kilopascals and above, and optimized the network with a pixel-wise weighted cross-entropy loss that penalizes underrepresented contact classes to counteract the imbalance between vast non-contact regions and sparse pressure-bearing areas. All results were averaged over six independent runs to verify statistical stability.

The results are striking for the tendon-actuated gripper, where the multimodal method surpassed the previous state of the art on every metric. It achieved a temporal accuracy of 96.57 percent, a contact area intersection-over-union of 77.20 percent, a volumetric IoU of 62.87 percent, and a mean absolute error of just 4.33 pascals, compared with 5.0 pascals for HiVPE and 5.3 for the earlier VPEC-Net. For the pneumatic gripper, the method attained the highest temporal accuracy and volumetric IoU, though it fell marginally short of HiVPE on contact IoU and mean absolute error, a shortfall the authors attribute to multimodal fusion occasionally introducing noise when soft surfaces deform so severely that depth and segmentation maps become blurred. Ablation experiments confirmed that each component earns its place: segmentation masks alone improved performance, adding attention strengthened contact-region predictions, depth maps reduced error further, and textual prompts delivered the final gains in spatial-pressure consistency.

The implications extend well beyond a leaderboard. Because the approach requires no embedded sensors, it could make force-aware grasping accessible to cheap, commodity robotic hardware, from laboratory manipulators to warehouse pickers handling fragile goods. The demonstration that a single descriptive word about the gripper’s action can measurably sharpen pressure predictions hints at a future where robots are instructed in natural language and perceive the physical consequences of their movements through vision alone. The authors are candid about the current limitations: the framework has so far been validated on static, offline benchmark data rather than deployed on physical robots in dynamic manipulation, where real-time sensor noise, unpredictable physical interactions, and inference latency constraints all come into play. They also note that soft grippers, with their complex nonlinear hyperelastic deformation, remain the hardest case, and they point toward more advanced depth estimation, adaptive fusion architectures that dynamically re-weight unreliable modalities, and explicit deformation modeling as the next frontiers. If those challenges are met, the line between seeing and touching may grow ever thinner, bringing robots one step closer to hands that truly understand what they hold.

Subject of Research: Vision-based multimodal pressure estimation for robotic gripper manipulators

Article Title: Vision-Based multimodal pressure estimation for robotic gripper manipulators

Article References: Liu, Y., Tang, W., Zhang, Z., & Chen, X. (2026). Vision-Based multimodal pressure estimation for robotic gripper manipulators. Results in Engineering, 32, Article 113295. https://doi.org/10.1016/j.rineng.2026.113295

Image Credits: AI Generated

DOI: 10.1016/j.rineng.2026.113295

Keywords: robotics, pressure estimation, multimodal AI, soft grippers, computer vision, deep learning, attention mechanism, CLIP, tactile sensing, depth estimation, feature pyramid network, robotic manipulation

The researchers call their model HA-DETR, short for Hierarchical Adaptive Feature Enhancement Detection Transformer. It builds on the detection transformer family of architectures, which replaced the hand-engineered region proposal pipelines of earlier detectors with an end-to-end attention mechanism that reasons about an entire image at once. Since the original DETR appeared in 2020, the family has evolved rapidly through variants such as Deformable DETR, DAB-DETR, and DINO, and recent work has shown that detection transformers can now rival or exceed the one-stage YOLO detectors that long dominated real-time applications. Liu and Jiao’s contribution is not a new backbone or a new loss function, but a set of three modules that intervene at specific points in the feature hierarchy, each tailored to the particular kind of degradation that dominates at that level.

The key observation motivating the architecture is that underwater degradations affect low-level details and high-level semantic representations differently. At the bottom of a convolutional or transformer feature pyramid, the network encodes fine-grained structure: edges, textures, and the delicate outlines of fins, tentacles, and shells. These are precisely the cues that scattering and blur destroy first. Higher up the pyramid, features become more abstract and semantic, encoding what an object is rather than where its boundaries lie, but they become vulnerable to a different problem: attention drifts toward the water column, sediment, and background clutter instead of locking onto the sparse object regions that matter. A single, uniform enhancement strategy applied to all levels therefore tends to help one level at the expense of another, which is why previous enhancement-plus-detection pipelines have delivered mixed results.

HA-DETR’s first component, the Frequency-Domain Feature Enhancement module, addresses the low-level problem by moving the computation into the frequency domain. The idea draws on a growing body of work showing that frequency analysis can separate image content in ways that spatial processing cannot: blur and haze tend to suppress high-frequency components, while object contours and textures concentrate there. By transforming features with operations related to the fast Fourier transform and adaptively reweighting frequency bands, the module restores the perception of blurred objects before the information is lost to deeper layers. This follows a broader trend in vision research, from frequency-aware transformers for image restoration to discrete cosine transform hybrids, that treats spectral information as a first-class citizen rather than a curiosity.

The second component, the Polarity-Aware Feature Interaction module, works at the opposite end of the hierarchy. Its job is to refine high-level semantic features so that the model’s attention concentrates on object regions rather than on the visually dominant but uninformative background. The term polarity refers to the module’s ability to distinguish and separately handle the opposing contributions of foreground and background signals during feature interaction, strengthening the positive evidence for objects while suppressing the negative pull of the surrounding water. In an underwater scene where a fish may occupy only a few percent of the pixels, this foreground-background asymmetry is the difference between a confident detection and a network that dutifully describes the emptiness around the fish.

The third component, the Hierarchical Attention Feature Pyramid Network, tackles the fusion problem that sits between the two extremes. Feature pyramid networks and their many descendants are the standard machinery for combining multi-scale features so that small objects benefit from high-resolution detail and large objects benefit from semantic depth. HA-DETR’s variant uses attention to optimize how information flows across scales and reinforces semantic-guided representation, ensuring that the enhanced low-level detail from the frequency module and the sharpened semantics from the polarity module are combined coherently rather than averaged into mush. The three modules together perform what the authors describe as differentiated processing and adaptive enhancement across semantic levels, allowing the network to preserve fine-grained structures and maintain robust semantic discrimination simultaneously.

The experimental evidence comes from two widely used underwater benchmarks. On DUO, a dataset of underwater object detections assembled from marine imagery with annotations for creatures such as fish, jellyfish, and sea urchins, HA-DETR achieves an average precision of 70.7 percent. On UTDAC2020, a dataset collected in a real aquaculture environment where turbidity and crowding are severe, it reaches 53.2 percent. Both figures represent the model’s superiority over competing approaches reported in the study, and the gap is particularly meaningful on the aquaculture data, where low contrast and heavy occlusion have historically dragged detection scores down. The authors report no competing interests, and the work was accepted by Cluster Computing after revisions in June 2026 and published on 18 September 2026.

What makes the approach notable beyond its benchmark numbers is the way it reframes the relationship between image enhancement and object detection. Much prior work treats underwater enhancement as a preprocessing step, using generative or physics-based models to produce a cleaner image before detection begins. That strategy can help, but it adds computational cost, can introduce artifacts that mislead the detector, and optimizes for visual quality rather than detection performance. HA-DETR instead embeds enhancement directly inside the detection architecture, applying it selectively where the feature hierarchy needs it. This echoes a recent line of research, including edge-aware DETR variants and CNN-transformer hybrids for underwater scenes, that argues detection-aware feature processing beats generic image restoration for this task.

The practical stakes are considerable. Automated monitoring of fish populations, coral reef health, and invasive species depends on reliable detection in conditions no human photographer would choose. Aquaculture operations use cameras to track feeding behavior, growth, and disease outbreaks in pens where visibility is often measured in meters. Autonomous underwater vehicles conducting infrastructure inspection or deep-sea surveying need to identify objects in real time with limited bandwidth back to shore. A detector that maintains accuracy under blur, low contrast, and occlusion extends the operational envelope of all of these systems, potentially reducing the need for human divers in hazardous or remote environments.

There are, of course, caveats. The reported results come from two datasets, and performance on other water bodies, lighting conditions, and sensor types remains to be demonstrated. Transformer-based detectors also carry computational costs that matter for the power-constrained embedded platforms on which many underwater vehicles run, although the field is moving quickly toward efficient attention designs. Still, the hierarchical adaptive principle at the heart of HA-DETR, matching the enhancement strategy to the semantic level being processed, is a general one, and the authors’ results suggest that respecting the structure of the feature hierarchy is one of the most effective ways to make machine vision work where the water refuses to cooperate.

Subject of Research: Transformer-based hierarchical adaptive feature enhancement for underwater object detection

Article Title: HA-DETR: Hierarchical adaptive feature enhancement for underwater object detection

Article References: Liu, T., & Jiao, G. (2026). HA-DETR: Hierarchical adaptive feature enhancement for underwater object detection. Cluster Computing, 29(13), Article 768. https://doi.org/10.1007/s10586-026-06577-w

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06577-w

Keywords: underwater object detection, detection transformer, HA-DETR, frequency-domain enhancement, feature pyramid network, attention mechanism, computer vision, marine monitoring, aquaculture, deep learning, multi-scale feature fusion, image degradation

The system, called FUS-DETR, was developed by Siyuan Duan, Geng Zhang, Xin Li, Shang Wang, Zhimin Mao, Bingliang Hu, and Xuebin Liu at the Xi’an Institute of Optics and Precision Mechanics of the Chinese Academy of Sciences, together with the University of Chinese Academy of Sciences. Writing in the journal Applied Intelligence, the team reports that their transformer-based detector achieves state-of-the-art results on two demanding air-to-air drone detection benchmarks, reaching 96.6 percent mean average precision at a 50 percent overlap threshold on the Det-Fly dataset and 98.1 percent on the UAV-Eagle dataset.

Those numbers matter because the air-to-air detection scenario is uniquely brutal for computer vision. When a camera drone is used to hunt another drone, the target often occupies only a handful of pixels in the frame. At long range, a quadcopter can shrink to a few dozen pixels across, losing nearly all of the distinctive shape cues that ground-based detectors rely on. Worse, the hunting drone’s own camera is moving, so the background — buildings, trees, roads, clouds, birds — streams past and creates a constantly shifting field of clutter that can masquerade as a target or swallow a real one entirely.

Traditional approaches to this problem have leaned heavily on convolutional neural networks, particularly the YOLO family of real-time detectors, which have dominated practical object detection for years. But convolutional networks process images through local receptive fields, which limits their ability to reason about the global context of a scene. A transformer-based detector, by contrast, uses self-attention mechanisms that let the model relate any part of the image to any other part, making it easier to distinguish a small intruder from its surroundings based on scene-wide patterns rather than purely local appearance.

FUS-DETR builds on the DETR, or Detection Transformer, architecture — specifically the real-time RT-DETR lineage that has recently begun to challenge YOLO models on speed and accuracy. The researchers introduced three key innovations. The first is a Feature-Focused Diffusion Pyramid Network, or FFDP, which tackles a chronic weakness of feature pyramid networks: as information flows between different scales of the image representation, fine details tend to be lost. The FFDP combines feature focus modules with a feature diffusion mechanism that enriches multi-scale feature maps with contextual information, deliberately spreading information across scales so that the faint signature of a small drone is not diluted or discarded along the way.

The second innovation is a Multilevel Attention Fusion Module, or MAF. Detection networks typically blend features from several layers, each capturing different levels of detail — shallow layers hold fine-grained edges and textures, while deeper layers encode broader semantic context. The MAF uses a hierarchical attention mechanism to adaptively fuse local and global features, learning on the fly which combination best captures the fine details of a small target while suppressing the background noise that would otherwise trigger false detections. In effect, the module lets the network decide, for every part of every image, how much to trust close-up texture versus wide-angle context.

The third component consists of three independent information fusion blocks arranged in concatenation and parallel structures. These blocks aggregate multi-scale information efficiently, giving the detection head a richer, more complete representation of each candidate target. Together, the three modules form a pipeline that is explicitly engineered around the two defining difficulties of drone detection from the air: extreme target smallness and heavy background clutter.

The experimental results are notable not just for their accuracy but for the rigor behind them. The team benchmarked FUS-DETR against a broad field of competitors, including classic detectors such as Faster R-CNN, SSD, and RetinaNet, the full YOLO series from YOLOv3 through YOLOv10, and the latest transformer-based detectors including RT-DETRv2 and DEIM. In an unusual display of transparency, the researchers disclosed in supplementary materials that they had initially reproduced some baseline results with an insufficient 50-epoch training schedule; they retrained RT-DETRv2 and DEIM under the full 100-epoch protocol and updated the comparison tables accordingly. All baselines were run under unified, fully documented training settings with the same 640 by 640 input resolution, ensuring that the comparison reflects the architectures rather than differences in training recipes.

The researchers also published detailed module-level complexity analyses, breaking down the parameter counts and computational costs of the FocusBlock and MAF components, including the multi-kernel depth-wise convolutions with kernel sizes of 5, 7, 9, and 11 that drive the feature diffusion process. This kind of accounting matters for a practical reason the authors highlight directly: achieving an optimal balance between top-tier detection performance and a model size suitable for aerial deployment remains a critical challenge. A detector that runs only on a data-center GPU is useless for a drone that must spot intruders in real time using onboard compute. The team’s attention to lightweight, re-parameterizable convolutional refinement operators suggests the design was shaped by deployment constraints from the start.

The datasets underpinning the results are publicly available — Det-Fly on GitHub and UAV-Eagle in an open repository — which means other groups can verify the numbers and build on them. The code is available from the corresponding author upon request. The work was supported by the Open Research Fund of the Shaanxi Key Laboratory of Optical Remote Sensing and Intelligent Information Processing, and the authors declare no competing financial interests.

If the reported performance holds up in independent testing, the implications extend well beyond counter-drone applications. The core techniques — context-preserving feature diffusion across pyramid scales, adaptive local-global attention fusion, and efficient multi-scale aggregation — address the general problem of small object detection in cluttered scenes, which also plagues medical imaging, satellite reconnaissance, wildlife monitoring, and autonomous driving. As drones multiply in the airspace, the ability of one flying machine to reliably see another may become as fundamental to aviation safety as radar was to the last century. FUS-DETR offers a glimpse of what that machine vision might look like: fast, compact, and sharp-eyed enough to find a speck of a drone in a sky full of noise.

Subject of Research: Transformer-based deep learning for detecting small unmanned aerial vehicles in cluttered airborne images

Article Title: FUS-DETR: a robust algorithm for drone detection from a cluttered background in airborne images

Article References: Duan, S., Zhang, G., Li, X., Wang, S., Mao, Z., Hu, B., & Liu, X. (2026). FUS-DETR: a robust algorithm for drone detection from a cluttered background in airborne images. Applied Intelligence, 56(15), Article 424. https://doi.org/10.1007/s10489-026-07448-y

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07448-y

Keywords: UAV detection, FUS-DETR, transformer, object detection, small object detection, air-to-air detection, computer vision, feature pyramid network, attention mechanism, deep learning, drone safety, Applied Intelligence