Improved grasp detection via global feature enhancement and skip-connected fusion

Robots have long struggled with one of the simplest human skills: picking things up. In cluttered, unpredictable environments—piles of tools, jumbled boxes in a warehouse, household objects scattered on a table—a robot must decide not only what to grasp but exactly where, at what angle, and with how much confidence. A research team in China has now unveiled a new neural network architecture that pushes robotic grasping detection to near-perfect accuracy on standard benchmarks while maintaining the real-time performance needed for practical deployment.

The system, called GFESF-Grasp, was developed by Shengjun Xu, Xiaoyi Wang, and colleagues at Xi’an University of Architecture and Technology, with collaborators at Northwestern Polytechnical University and Xi’an Jiaotong University. Writing in the journal Autonomous Robots, the team reports accuracy of 98.9% on the Cornell grasping dataset, 96.1% on the much larger Jacquard dataset, and 87.0% on OCID-Grasp, which presents especially noisy, cluttered conditions. In real-world experiments involving objects the robot had never seen before, the system achieved a grasping success rate of 92.2% in complex scenarios—a figure the researchers say validates its practicality in challenging, unstructured environments.

The central problem the team attacked is one that plagues virtually all vision-based robotics: distinguishing the signal of graspable objects from the noise of background clutter. Convolutional neural networks, the workhorses of computer vision, tend to lose fine detail as an image passes through successive layers of downsampling. By the time the network reaches its deepest layers, where semantic understanding is strongest, the precise geometric information needed to position a gripper—object edges, thin handles, boundaries between overlapping items—has often been washed away. Conversely, early layers preserve fine spatial detail but lack the contextual understanding to interpret it. Existing grasp detection methods struggle to suppress background noise and highlight the critical regions where a successful grip is possible, leading to false detections or positioning errors when scenes get messy.

GFESF-Grasp tackles this through two complementary innovations. The first is a Global Feature Enhancement (GFE) module placed at the bottleneck layer of the network—the point where the feature maps are most compact and semantically rich. The GFE module combines parallel dilated convolutions with differing dilation rates. Dilated convolutions insert gaps between the elements of a convolutional kernel, allowing a small kernel to cover a much larger area of the image without increasing the number of parameters. By running several such convolutions in parallel with different dilation rates, the module captures both fine local details and broad global context simultaneously, effectively expanding the network’s receptive field. This expanded receptive field enhances the network’s ability to perceive cross-scale geometric features—crucial when a single scene might contain everything from a small screw to a large carton.

But context alone is not enough; the network also needs to know where to look. To that end, the GFE module integrates a Mamba-Like Linear Attention (MLLA) mechanism, drawing on the linear-time sequence modeling ideas popularized by the Mamba architecture. Unlike conventional transformer attention, whose computational cost grows quadratically with input size, linear attention approaches scale far more gently, making them viable for real-time robotics. The MLLA component strengthens feature responses in the regions that matter most for grasping—object edges and boundaries in particular—sharpening the network’s discriminative power when backgrounds are noisy and objects are visually entangled.

The second innovation addresses the other side of the encoder-decoder architecture: how information flows back up during decoding. The team’s Skip-Connected Feature Fusion (SF) module adaptively merges features from different levels of the network. Skip connections, in which shallow high-resolution features are carried forward and combined with deep low-resolution features, have been a staple of segmentation and detection architectures since the days of U-Net. But naive fusion—simply adding or concatenating feature maps—can inject redundant or contradictory information. The SF module instead learns to weight the incoming features adaptively, suppressing redundant information while preserving the key geometric cues needed to compute an accurate grasp pose. The result is a decoder that retains the sharp spatial detail of early layers without being drowned in noise.

The task the network performs is generative grasp detection in the plane. Given an RGB-D image—a color photograph paired with a depth map capturing the 3-D geometry of the scene—the network outputs a dense, per-pixel prediction of grasp quality and geometry. Each pixel is assigned a graspability score along with the orientation and width a parallel-jaw gripper should adopt at that location. From this dense prediction, the robot’s planning system selects the highest-quality grasp, converts it into a 3-D pose using the depth information, and executes the motion. This formulation, pioneered by earlier work such as the Generative Grasping Convolutional Neural Network (GG-CNN), enables reactive grasping at camera frame rates, but its accuracy has historically lagged behind heavier, slower two-stage methods that first detect objects and then propose grasps.

GFESF-Grasp demonstrates that a lightweight single-stage network need not sacrifice accuracy. The team evaluated the system on three widely used benchmarks. Cornell, with its 885 RGB-D images of household objects, is the classic test of planar grasp detection, and a score of 98.9% approaches the practical ceiling for the task. Jacquard, a large-scale synthetic dataset with more than 54,000 images and over a million labeled grasps, tests generalization across far greater object diversity; 96.1% there is a strong result. OCID-Grasp, built on the Object Clutter Indoor Dataset, is the sternest examination: depth images of heavily cluttered bins, where occlusion and sensor noise make edge detection genuinely difficult. Achieving 87.0% under those conditions suggests the GFE and SF modules are doing their intended work of separating object signal from clutter noise.

Physical experiments reinforced the benchmark results. When confronted with complex scenes containing unknown objects—items never represented in the training data—the system achieved a 92.2% grasp success rate. Generalization to novel objects is arguably the most important metric for real-world deployment, since a warehouse or domestic robot cannot be retrained for every item it encounters. The dense, geometry-driven nature of the grasp representation helps here: rather than recognizing specific object categories, the network learns general visual and geometric signatures of graspability, which transfer to novel items.

The implications extend across the rapidly expanding field of robot manipulation. E-commerce fulfillment centers, where millions of heterogeneous items must be picked and packed, are an obvious application; so are recycling and waste sorting, where objects arrive in genuinely chaotic clutter. Agricultural robotics, hospital service robots, and laboratory automation all face the same fundamental problem of grasping unmodeled objects in unstructured settings. A detection network that is accurate, fast, and robust to clutter moves the field closer to systems that can operate reliably outside the carefully engineered cells of traditional industrial automation.

The researchers have also made the work unusually accessible: code for GFESF-Grasp is publicly available on GitHub, allowing other laboratories to reproduce the results and build on the architecture. The work was supported by the National Natural Science Foundation of China and the Natural Science Foundation of Shaanxi Province, among other funders.

The architecture also reflects a broader trend in machine learning research: the migration of ideas from sequence modeling into computer vision. The Mamba family of architectures, originally proposed as efficient alternatives to transformers for long sequences of text, has proven remarkably adaptable to visual tasks, and GFESF-Grasp’s use of a Mamba-like linear attention module within a convolutional grasp detector is a concrete example of that cross-pollination. For robotics, where compute budgets are tight and latency budgets are measured in milliseconds, efficiency-oriented attention mechanisms may prove far more consequential than their raw-capacity transformer cousins.

There remain, of course, challenges ahead. Planar grasp detection with parallel-jaw grippers is only a slice of the manipulation problem; dexterous hands, articulated objects, deformables, and task-oriented grasping—where the robot must grasp an object in a way that supports its subsequent use—all demand richer representations. The authors acknowledge that the work targets grasping pose estimation in unstructured environments, and future work will likely push toward six-degrees-of-freedom grasp poses, dynamic scenes, and integration with large language models that can supply semantic and task-level reasoning.

But the core achievement stands: a network that looks at a cluttered scene and, in the time it takes a camera to blink, finds a place to put its fingers with better-than-90% reliability. As robots step out of laboratories and into the messiness of the human world, that combination of precision, speed, and robustness is precisely what the field has been waiting for. The gap between how effortlessly a human hand clears a cluttered table and how a robot does it just narrowed a little further.