Advances in Prosthetic Hand Control: A Multimodal Approach to Dynamic Tool Handling
In the ongoing quest to enhance the functionality and realism of prosthetic hands, researchers at the Beijing University of Posts and Telecommunications have developed a groundbreaking system that transcends traditional static grasping. Their innovative approach, termed the Tactile, Kinesthetic, and EMG Bionic Gripping Controller (TKE-BGC), integrates multimodal sensory feedback into real-time control algorithms, enabling prostheses to handle complex tool manipulation tasks with unprecedented stability and efficiency.
Human hands are sophisticated machines finely tuned by the brain to adjust grip force dynamically in response to environmental feedback. This sensorimotor integration allows humans to rarely drop tools during rigorous or rapid tasks—a feature that current prosthetic hands lack. The TKE-BGC emulates this biological feedback loop by incorporating three critical data streams: electromyographic (EMG) signals from residual limb muscles, tactile contact forces from the prosthetic fingertips, and kinesthetic information derived from joint angles. This multidimensional sensory fusion informs a biomimetic control strategy that dynamically adapts grip posture and force during task execution.
To generate the robust dataset necessary for training the TKE-BGC, researchers enlisted able-bodied participants who donned a data glove equipped with embedded tactile sensors and bend sensors, alongside EMG electrodes placed to capture residual muscle signals. Participants engaged in two primary tasks—hammering nails and sawing wooden strips—each embodying complex interaction dynamics characterized by rigid impacts and varying load distributions. These recordings offered a comprehensive portrait of how natural hands negotiate sudden perturbations, providing executable templates for the prosthetic control model.
At the heart of this system lies a sophisticated neural architecture. A Transformer encoder processes the multimodal inputs, leveraging cross-attention mechanisms where tactile and joint-angle data serve as queries that modulate the EMG feature representations. This design mirrors the biological sensorimotor loop, allowing tactile and kinesthetic feedback to dynamically influence motor commands. Subsequently, a multilayer perceptron predicts the necessary joint angle adjustments, ensuring real-time adaptability of the prosthetic hand during manipulation tasks.
The TKE-BGC’s performance was benchmarked against two established prosthetic grip control methods: fixed force (FiF) and force follows (FoF). FiF applies a constant preload without adaptation, while FoF modulates grip force based on lagging load measurements, often resulting in delayed or excessive responses. During offline analysis using the demonstration datasets, TKE-BGC outperformed both, achieving significantly lower root-mean-square error in predicting joint movements, particularly under the impact-intensive hammering conditions. This evidences its superior ability to anticipate and adapt to dynamic interaction forces.
Subsequent online studies involved six able-bodied individuals fitted with an extended limb apparatus and three transradial amputees. Each participant performed four tasks: the two “seen” tasks from training (hammering and sawing) and two unseen tasks involving different tools (peeling and desktop item organization). All control strategies were tested in randomized sequences. Remarkably, TKE-BGC consistently minimized tool drops and reduced task completion times across all conditions. In contrast, FiF and FoF frequently failed to maintain stable grasps, resulting in multiple drops and longer durations.
Beyond performance metrics, the TKE-BGC demonstrated physiological and user experience advantages. EMG amplitude analyses revealed markedly lower muscle activation during TKE-BGC operation (average 0.0023) compared to FoF (average 0.0124), implying reduced muscular effort and potentially less fatigue. Integrated EMG measurements aligned with these findings. Subjective feedback via the USE questionnaire highlighted higher user satisfaction, ease of use, and natural control sensations when employing TKE-BGC. Conversely, some users described FoF as reactive and resisting external forces, undermining intuitiveness.
A key triumph of the research is the system’s ability to generalize effectively. Despite training on data from a single able-bodied individual, TKE-BGC delivered consistent performance across diverse users, including amputees, and extended seamlessly to novel tasks without further training. This adaptability underscores the robustness of the multimodal fusion and the transformative potential of integrating tactile feedback—a modality whose removal in ablation studies caused the most significant degradation in predictive accuracy.
Current tactile feedback in the system is relatively sparse, with only nine contact force measurement points on the prosthetic contact surface. The researchers acknowledge that human skin provides dense, multidimensional sensory input that conveys nuanced information critical for manipulation. Plans are underway to incorporate high-density tactile sensor arrays coupled with advanced neural network architectures capable of interpreting local tactile imagery, promising richer sensory representations and finer control.
Moreover, the team aims to expand their dataset to include diverse manipulation styles from multiple participants, thereby capturing personalized strategies and enhancing adaptive prosthetic behavior. They also anticipate introducing optimization-based mapping techniques that can dynamically align joint postures for individualized anatomical and task requirements, further personalizing control and broadening applicability.
This pioneering research marks a significant paradigm shift in prosthetic hand technology, moving beyond static grasping to dynamic, context-aware manipulation. By harnessing the synergy of multimodal sensory integration and advanced computational models inspired by human neurophysiology, the TKE-BGC controller offers an empowering tool for amputees. Its implications extend from improving everyday living functionality to enabling vocational rehabilitation, ultimately fostering greater independence and quality of life.
The study was spearheaded by Professors Bin Fang and colleagues including Boao Li, Shuhui Wu, Ting You, among others, at the School of Artificial Intelligence, Beijing University of Posts and Telecommunications. The research received support from the Brain Science and Brain-like Intelligence Technology–National Science and Technology Major Project (grant no. 2025ZD0215600) and the National Natural Science Foundation of China (grant nos. 62573063, 62536001). Their findings, published in the May 2026 issue of Cyborg and Bionic Systems, showcase the promising trajectory of prosthetic hand research and its potential to reshape human-machine interactions.
Subject of Research: Prosthetic hand control and dynamic tool manipulation using multimodal sensory integration
Article Title: Dynamic Manipulation Skill Learning for Tactile Myoelectric Prosthetic Hands in Tool Handling
News Publication Date: May 13, 2026
Web References: DOI: 10.34133/cbsystems.0572
Image Credits: Bin Fang, School of Artificial Intelligence, Beijing University of Posts and Telecommunications
Keywords: Prosthetic hands, multimodal sensory integration, electromyography, tactile sensing, kinesthetic feedback, transformer encoder, real-time control, biomimetic gripping, tool manipulation, muscle fatigue reduction, dynamic grasp stability, sensorimotor fusion
