Cyborg insects, typically cockroaches or beetles fitted with tiny electronic backpacks that deliver electrical stimulation to their antennae or sensory organs, have attracted intense attention in recent years as potential search-and-rescue scouts, environmental monitors, and infrastructure inspection agents. The appeal is straightforward. Millions of years of evolution have produced locomotion systems that no human engineer can match at small scales: insects can climb rubble, squeeze through crevices, recover from falls, and sustain operation on meager energetic resources. By interfacing electrodes with the insect’s nervous system, researchers can, in principle, command the animal to turn left, turn right, accelerate, or pause. But commanding an insect is only half the problem. To close the control loop, the system must know continuously where the insect actually is and where it is pointed, a measurement task that has historically required either bulky laboratory motion-capture systems or reflective markers glued to the animal’s body.

The traditional marker-based approach carries hidden costs. A small frame of reflective markers mounted on the insect’s back adds payload to an animal that may itself weigh only a few grams, and it changes the aerodynamics, center of mass, and natural gait of the creature. Worse, the researchers found, physical marker frames can snag on walls and obstacles, entangling the insect as it navigates cluttered terrain. In experiments comparing the new markerless method with earlier marker-based setups, no obstacle entanglement was observed with the markerless approach, while the marker-based configurations risked exactly the kind of snagging that disrupts both the experiment and the insect’s welfare. For a field whose long-term vision is deploying these animals in disaster zones full of debris, eliminating protruding hardware is not a cosmetic improvement but a functional necessity.

The new framework, described in the International Journal of Intelligent Robotics and Applications, relies on a single inexpensive overhead web camera pointed at the experimental arena. The core challenge is one of scale: a cockroach viewed from directly above occupies only about 0.13 percent of the image frame, making it one of the smallest conceivable detection targets. To handle this, the team built their system around a lightweight YOLO detector, a family of neural network architectures famous for performing object detection in a single forward pass, which makes it fast enough for real-time video processing even on modest hardware. The detector locates each cyborg insect within every frame of the camera feed, even in arenas cluttered with obstacles that would confound simpler background-subtraction methods.

Detection alone, however, does not reveal the insect’s heading, the orientation of its body axis, which is essential for closed-loop steering. The researchers therefore augmented the YOLO detector with three different pose-estimation strategies and compared them head to head. The first strategy extracted keypoints directly from the YOLO network, effectively asking the same architecture that finds the insect to also identify anatomically meaningful points on its body. The second employed a dedicated convolutional neural network that predicts heatmaps, probability distributions over the image in which each anatomical landmark lights up as a bright blob whose weighted centroid yields the keypoint position; this approach descends from human pose-estimation techniques that regress part heatmaps rather than coordinates directly. The third benchmark was DeepLabCut, the widely adopted open-source pose-estimation toolkit originally developed for neuroscience, which has become a de facto standard for markerless tracking of animals in behavioral laboratories.

The comparative evaluation produced a clear winner. When the team ran single and pairs of cyborg cockroaches through both obstacle-free and obstacle-filled arenas, the YOLO-based keypoint estimator achieved the most favorable balance of accuracy, robustness, and speed, delivering inference in just 32 milliseconds per frame. The CNN heatmap model and DeepLabCut, while capable, were slower and in some conditions less reliable at maintaining continuous measurement of a target so small and so erratically moving. Critically, in the two-insect experiments, the system maintained stable identity preservation, correctly distinguishing which tracked animal was which over time, a nontrivial feat when two nearly identical cockroaches cross paths or pass close to one another in a confined arena.

Real-time performance mattered for a second reason: the tracking system had to be integrated with the wireless stimulation backpack carried by each insect. The complete pipeline, camera capture, neural inference, trajectory computation, and command transmission, ran at 23 frames per second, fast enough to synchronize the insect’s observed kinematics with the electrical commands being issued. The researchers report a clear correlation between the stimulation commands sent to the antennae and the measured changes in the insect’s position and heading, meaning that for the first time in their experimental setup, feedback control of cyborg cockroach locomotion could proceed continuously without any external commercial motion-capture equipment and without any physical attachment beyond the functional backpack itself.

The technical significance of the work lies partly in its demonstration that low-cost, commodity hardware can substitute for laboratory-grade tracking infrastructure. Traditional approaches to measuring insect locomotion include tethered treadmills and spherical trackballs, servo-driven spheres on which a fixed insect walks while sensors measure ball rotation, as well as high-end optical motion capture requiring calibrated multi-camera arrays. Each of these constrains the experiment: trackballs prevent free navigation through clutter, and motion-capture systems demand reflective markers and precisely controlled lighting. A single web camera paired with an efficient neural network removes those constraints, opening the door to locomotion experiments in environments that genuinely resemble the unstructured terrain, rubble piles, dense vegetation, collapsed structures, where cyborg insects would one day be deployed.

There are also implications for animal welfare and experimental validity. Payload constraints on small insects are severe; every additional gram of marker hardware alters the animal’s energy expenditure and gait, potentially confounding the very locomotion data researchers are trying to collect. By measuring the insect purely through vision, the system supports natural locomotion of backpack-equipped animals, and the elimination of snag-prone marker frames reduces both experimental failures and physical harm to the insects. The team notes that the proposed system is well suited to real-time locomotion tracking experiments in controlled laboratory arenas, a modest but important scope claim: this is a laboratory validation, with overhead cameras and defined arenas, rather than a field-ready deployment.

The broader context is a rapidly maturing cyborg-insect ecosystem. Recent years have seen swarm navigation of cyborg insects through unknown obstructed terrain, automated robotic assembly lines for mounting electrodes on live insects, onboard solar cells that keep backpack batteries charged without restricting mobility, and 3D-printed ergonomic harnesses designed to minimize invasiveness. What has lagged behind is the sensing layer, the ability to know, cheaply and continuously, what the insects are doing. Machine-learning-based behavioral tracking has already reached human-level accuracy in other domains, outperforming some commercial solutions in neuroscience laboratories, and this study extends that trajectory to one of the most demanding tracking regimes imaginable: millimeter-scale targets, from above, in real time, among obstacles.

The research team, which also included Hafiz Akbar Simanjorang, M. Munadi, and Rifky Ismail of Diponegoro University and Ade Kurniawan of Institut Teknologi Sains Bandung, was funded by Indonesia’s Ministry of Higher Education, Science, and Technology. Their results suggest that the pieces of a fully autonomous cyborg-insect control system, wireless stimulation, markerless perception, and closed-loop feedback, can now be assembled from components that cost a small fraction of traditional motion-capture installations. As the field moves toward multi-insect swarms searching for earthquake survivors or mapping hazardous interiors, the ability to watch many small animals at once, with ordinary cameras and fast neural networks, may prove as decisive as the electrodes that steer them.

For now, the system remains confined to laboratory arenas with a single overhead viewpoint, and the authors are careful to frame it as a foundation rather than a finished product. Raw data are available from the corresponding author upon request, and the team emphasizes that the framework’s low latency, markerless operation, and demonstrated identity preservation across multiple animals collectively remove several long-standing bottlenecks in cyborg-insect experimentation. If biohybrid robots are to leave the laboratory, they will need systems like this one: unobtrusive, fast, and indifferent to clutter, watching from above while the insects do what insects do best.

The research team was spearheaded by John Zimmerman, a postdoctoral fellow at Harvard SEAS. He was joined by a distinguished group of researchers, including Ryoma Ishii, a medical and health informatics scientist at NTT Research, and Kevin Kit Parker, the Tarr Family Professor of Bioengineering and Applied Physics at Harvard SEAS. The collaborative effort also included contributions from the Harvard SEAS Disease Biophysics Group, which Parker leads. Their joint efforts culminated in the publication of a paper in the journal Science Robotics titled “Bioinspired Design of a Tissue Engineered Ray with Machine Learning.”

The core of this research addresses a pressing question in the development of biohybrid robots, particularly focusing on the marine ray model: How do researchers select fin geometries that are effective in novel working environments while still adhering to the natural scaling laws associated with swimming speed and efficiency? Ishii articulated this query, emphasizing the challenges faced in designing biohybrid robots that effectively emulate real biological forms.

Traditional biomimetic approaches in engineering biohybrids often involve replicating existing biological structures, an approach that can come with inherent limitations. For example, when designing biohybrid equivalents of batoid fishes—like skates and rays—engineers grapple with the vast range of natural aspect ratios and fin shapes. Determining which characteristics to replicate can be daunting, and existing models may sometimes overlook essential biomechanical principles that dictate swimming performance. This can be detrimental, leading to designs that squander muscle mass or fail to achieve optimal swimming velocities.

Given these obstacles, the research team sought an innovative pathway to determine fin geometries that can excel under varying conditions, without violating the principles of natural scaling laws. They turned to the realm of machine learning as a potential solution to optimize the design process effectively. The interdisciplinary nature of this endeavor meant that traditional computational modeling techniques would likely require excessive computational power. This led the team to the hypothesis that ML-DO could permit a more systematic and efficient exploration of design options that maximize swimming speeds.

To validate their hypothesis, the researchers undertook a structured approach that unfolded in three pivotal steps. Initially, they developed an algorithm capable of expressing a wide array of fin geometries. This was followed by the formulation of a generalized ML-DO framework aimed at navigating the expansive and often disjointed configuration space of available designs. Finally, the team employed this methodology to pinpoint biohybrid fin geometries that favorably impacted performance by allowing for smooth and efficient aquatic movement.

The insightful results gleaned from the ML-DO approach provided quantitative evidence regarding fin structure-function correlations while simultaneously reconstructing prevalent patterns found in the morphology of open-sea batoid species. Significantly, the team emerged with a superior design concept: fins characterized by large aspect ratios paired with delicately tapered tips, which proved to be versatile across various swimming scales. Utilizing these findings, the team successfully engineered biohybrid mini-rays that utilized cardiac muscle tissue and demonstrated impressive self-propelled swimming capabilities at the millimeter scale. These mini-rays showcased swimming efficiencies that were approximately double those observed in previous biomimetic designs.

While the outcomes from the current study are commendable, researchers acknowledge that further advancements must be made to reconcile the discrepancies between engineered devices and naturally occurring marine organisms—particularly regarding efficiency. The devices showcased in the study, though superior to their biomimetic counterparts, still exhibited a slightly lower efficiency compared to natural marine life, highlighting the room for continued refinement in their designs.

Looking forward, the research team anticipates ongoing developments in biohybrid robotics that will have a significant impact on various applications. These include but are not limited to deploying remote sensors, creating probes for hostile environments, and designing vehicles for therapeutic delivery. The ML-DO-informed methodology is hoped to provide insights that parallel the selective pressures faced by natural evolution, giving researchers a deeper understanding of the factors that shape biological tissues in both healthy and pathological states.

Furthermore, this research is poised to contribute to the burgeoning field of 3D organ biofabrication, aiding in the eventual goal of creating complex structures such as biohybrid hearts. Parker, highlighting the collaborative nature of this research, noted that a joint research agreement established two years ago with NTT Research aimed to enhance understanding of cardiac physiology while driving the development of biohybrid devices. He expressed excitement regarding the progress made thus far and the potential future accomplishments stemming from this partnership.

In 2022, a significant three-year joint research agreement was formalized between NTT Research and Harvard SEAS, focusing on engineering a model of the human heart while investigating the fundamental principles that govern muscular pumps. The goal is to leverage their findings to contribute to a cardiovascular bio digital twin model—an innovative fusion of biological understanding and engineering prowess that could pave the way for new advancements in medical technology.

This research not only illuminates new avenues in bioengineering but emphasizes the transformative potential of machine learning in optimizing biological designs. As researchers continue to explore, the marriage of biological inspiration and artificial intelligence may yield unprecedented innovations, leading to breakthroughs not only in robotics but also in our comprehension of biological systems and their applications in medicine and human health.

Subject of Research: Development of biohybrid robots using machine-learning directed optimization
Article Title: Bioinspired Design of a Tissue Engineered Ray with Machine Learning
News Publication Date: October 2023
Web References: Science Robotics
References: None provided
Image Credits: None provided

Keywords: Machine learning, biohybrid robots, biomimetics, tissue engineering, robotics, cardiac muscle, swimming efficiency, design optimization.