The research was presented in Frontiers in Robotics and Artificial Intelligence in 2026 and evaluated both an artificial-intelligence perception system and its integration with a physical arm. The machine is designed for household-assistance applications, in which a user might ask a robot to retrieve one particular item from a table surrounded by clutter. Its arm weighs 2.2 kg, has a payload capacity of 1.5 kg and ends in a one-degree-of-freedom gripper that closes its thumb and four remaining fingers. The arm comprises three coupled tendon-driven modules: a three-degree-of-freedom shoulder, a two-degree-of-freedom elbow and a two-degree-of-freedom wrist. Its lightweight, elastic tendons were chosen partly to suit operation close to people.

To perceive the scene, the researchers converted an RGB-D image into a three-dimensional point cloud and used only each point’s spatial coordinates, excluding color information to reduce sensitivity to texture and lighting. A shared backbone extracted features for two linked tasks: estimating full six-degree-of-freedom grasp poses and assigning points to semantic object categories. The latter task links each possible grasp to a recognized object, allowing a user to select a target category in a graphical interface. The system then executes the highest-scoring grasp associated with that category, defined by the candidate point, approach direction, rotation around the approach axis, gripper width and gripper depth.

The central network innovation was a 3D Mixture-of-Experts UNet based on the MinkUNet architecture. Rather than activating an entire enlarged model, the design used eight parallel expert networks and a gating mechanism that selected the top two for each input. The experts replaced convolutional layers in the two deepest residual blocks of the encoder, where features represent broad object geometry and scene context. An auxiliary load-balancing loss discouraged the gate from routing nearly all inputs to a small subset of experts. Evaluation showed that the eight experts were selected with broadly balanced frequency, indicating that expert collapse did not occur, although tests of alternative expert counts, routing arrangements and block placements found no configuration that consistently dominated the chosen design.

The multi-task model contained 24.2 million parameters in total, of which 17.1 million were active during inference because only two experts ran at once. By comparison, a separate semantic-segmentation model using the study’s conventional 3D UNet backbone would have required at least 14.7 million additional active parameters. The model was trained for 40 epochs with a batch size of four on a single NVIDIA GeForce RTX 4080 SUPER graphics processor. It used cross-entropy losses for objectness and semantic classification, smooth L1 loss for continuous grasp quantities, and several task-specific terms for approach direction, gripper dimensions and grasp quality. Label smoothing was applied to segmentation with a coefficient of 0.1.

Benchmark testing used GraspNet-1Billion, a dataset of 97,280 RGB-D images from 190 cluttered real-world scenes containing 88 general objects and more than 1.1 billion grasp-pose annotations. The 190 scenes were divided into 100 training scenes and 90 test scenes, with the test scenes further separated into seen, similar and novel object groups. Because many of the objects had similar shapes, the researchers assigned the 88 objects to seven geometric categories: pump bottle, tube, container, irregular-shaped object, spherical object, elongated or cylindrical object, and cuboid object. Camera-specific train and test sets were evaluated separately for RealSense and Kinect data. The proposed backbone generally improved grasp estimation and segmentation over 3D UNet and PointNet++ alternatives, with gains reaching 5.24 average-precision points for grasp estimation and 3.42 mean intersection-over-union points for segmentation.

Those aggregate improvements did not apply uniformly. Compared with the 3D UNet backbone, the Mixture-of-Experts model improved RealSense grasp-pose average precision by 2.43 points on seen objects, 1.98 on similar objects and 2.14 on novel objects. For Kinect data, it improved seen and similar results by 5.24 and 2.25 points, respectively, but the novel-object result fell by 0.65 points. Segmentation similarly declined by 1.38 mIoU points on the RealSense novel subset, even as grasp-pose performance improved on that subset, suggesting a trade-off between the tasks’ shared representations. The authors associated weaker Kinect performance with sensor characteristics such as depth noise, missing points and differences in point-cloud density. On the similar and novel test sets, segmentation results were affected by systematic inter-class confusion: cuboid objects could be classified as containers, while pump bottles could be confused with tubes.

The arm itself used a geometry-based inverse-kinematics method to resolve its redundant seven-degree-of-freedom structure. When the end-effector pose is fixed, the elbow can occupy multiple positions, so the method introduced an arm angle and searched its permitted range in fixed increments. It selected the angle that minimized the sum of the absolute joint angles, producing a complete set of joint commands. The motion planner then interpolated position linearly and orientation with spherical linear interpolation. In the physical workflow, the end-effector first moved to a pre-grasp pose offset opposite the estimated approach vector, advanced to the selected grasp, lifted the object vertically, transferred it to a predefined placement location and released it.

The laboratory evaluation involved 10 target objects in scenes containing at least four objects each, with object numbers, positions and orientations varied manually between trials. Each target received 10 pick-and-place attempts, and success required grasping the specified object without toppling it, lifting and transporting it, then releasing it without a drop. All three seen objects achieved 100% success. Among unseen objects, the individual rates ranged from 90% to zero: objects 4 and 6 each succeeded in nine trials, objects 7 and 9 in seven, object 5 in four, object 8 in one, and object 10 in none. The 52.9% average for unseen objects therefore concealed substantial differences associated with object geometry, size and material. The researchers identified several contributing constraints, including incomplete depth measurements from translucent plastic, limited contact options imposed by the one-degree-of-freedom gripper, and difficulty avoiding the supporting surface when few stable grasp regions existed.

The complete system, including the robotic-arm controller, used 5.1 GB of RAM and 3.12 GB of video memory and processed images at an average of 20 FPS in the experimental setup. These measurements suggest that the workload could fit on a compact edge computer such as a Jetson Orin2, but its performance on such hardware was not tested. The prototype also had important functional limits. Semantic categories allowed a user to request a broad object type, yet two instances of the same type could not be selected separately without instance-level segmentation. The study proposes future work on more adaptable multi-degree-of-freedom hands, improved depth acquisition for translucent objects, better mixture-of-experts tuning and instance segmentation. It also suggests voice or electromyography-based target selection for users with limited physical mobility, an application direction that was proposed rather than evaluated.

Subject of Research: A 7-DOF robotic system for autonomous perception and grasping of user-selected general objects

Article Title: Development of a 7-DOF robotic arm control system for autonomous grasping of user-specified general objects

Article References: Bai, S., Jiang, Y., Yokoi, H., & Togo, S. (2026). Development of a 7-DOF robotic arm control system for autonomous grasping of user-specified general objects. Frontiers in Robotics and AI, 13, 1955232. https://doi.org/10.3389/frobt.2026.1955232

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1955232

Keywords: robotic grasping, assistive robots, 7-DOF arm, point-cloud perception, grasp pose estimation, semantic segmentation, mixture of experts, inverse kinematics, tendon-driven robot, pick-and-place, Development, robotic