Vision-based tactile sensors have become one of the most popular architectures in robotic touch research over the past several years. The basic recipe is elegant. A small camera sits inside a rigid housing, pointing at a soft, translucent elastomer pad. When an object presses against the pad, the elastomer deforms, and internal illumination reveals the deformation as shifting patterns of light and shadow. Sophisticated computer-vision algorithms then reconstruct the contact geometry, estimate forces, and even identify textures from the image sequences. Sensors built this way can deliver spatial resolutions that far exceed those of conventional arrays of pressure-sensitive elements, producing dense contact images that resemble miniature photographs of whatever the robot is touching.

The problem is that those miniature photographs come at a cost. A typical vision-based tactile sensor streams images continuously, and every one of those frames must travel from the sensor, across a data bus, and into a separate processor before any meaning can be extracted from it. For a single fingertip the data rates are manageable; for a robotic hand with multiple fingers, each equipped with one or more sensors and running at frame rates high enough to catch fast slipping events, the pipeline quickly becomes saturated. Latency creeps in, power consumption climbs, and the perception loop slows to the point where a robot may react to a slip only after the object has already fallen. This separation between where data are sensed and where data are processed has been called the memory and bandwidth bottleneck, and it is a familiar villain across the broader field of machine perception.

The research reported in Communications Engineering tackles that bottleneck head-on by moving the computation into the sensor itself, an approach known as in-sensor computing. Instead of shuttling raw images to an external computer, the sensing device performs meaningful processing at or near the imaging plane, so that what leaves the sensor is not a flood of pixels but a compact stream of perceptual information. In the most advanced implementations of this idea, the image sensor’s own hardware performs operations such as convolution, feature extraction, or event generation during or immediately after image acquisition. The result is a dramatic reduction in the volume of data that must be transmitted, along with lower latency and lower energy consumption per perception event.

Applying this paradigm to touch is a natural but technically demanding step. Unlike a camera observing a static scene, a tactile sensor operates in a regime of constant, dynamic deformation. The elastomer surface is always moving under load, and the most decision-relevant information, such as the onset of slip, a sudden change in contact area, or the fine vibration signature of a textured surface, is embedded in rapid temporal changes rather than in any single frame. A sensing-and-computing architecture intended for touch therefore has to preserve fast temporal dynamics while still compressing the data enormously. The new work describes an integrated design in which the sensing elements and the computing elements are conceived together from the start, rather than bolted together after the fact, which the authors argue is the key to achieving truly seamless tactile sensing and perception.

According to the published description, the system integrates the image-capture function with on-chip processing so that tactile features are extracted as the data are generated. Early-stage in-sensor designs in this field have relied on a variety of mechanisms: pixel-level analog computation, in-memory computing arrays that multiply and accumulate signals where the data are stored, and neuromorphic approaches in which individual pixels fire asynchronously only when they detect change, mimicking the behavior of biological mechanoreceptors in human skin. Each strategy trades off some combination of accuracy, speed, power, and fabrication complexity. The value of the paradigm demonstrated here lies in treating those trade-offs as a co-design problem: the sensor’s optical geometry, its readout electronics, and its processing primitives are optimized jointly so that the perceptual tasks a robot actually needs, such as contact localization, force estimation, texture recognition, and slip detection, can be executed within the sensor’s own footprint.

The implications for robotics are substantial. Dexterous manipulation, the ability to grasp, turn, insert, and assemble objects with humanlike facility, is widely regarded as the next frontier for industrial and service robots, and researchers in the field consistently identify touch as the missing sense. A human hand contains tens of thousands of tactile nerve endings, and the somatosensory system performs an extraordinary amount of preprocessing in the peripheral nerves and spinal cord before signals ever reach the brain. In-sensor computing for robots echoes that biological architecture: low-level feature extraction happens locally, and only compact, meaningful signals ascend to the robot’s central controller. This hierarchical organization is precisely what allows biological hands to react to a slipping cup within tens of milliseconds, far faster than any conventional camera-processor pipeline could manage.

The practical consequences extend beyond manipulation speed. Because in-sensor processing reduces the data bandwidth per fingertip by orders of magnitude, the approach makes it feasible to instrument an entire robotic hand, or even two hands working together, with dense tactile coverage without overwhelming the robot’s compute budget. Lower data volumes also translate into lower power draw, a critical consideration for mobile robots, humanoid platforms, and prosthetic devices that must operate for hours on limited batteries. For prosthetics in particular, a tactile sensor that outputs interpretable perceptual signals rather than raw video could simplify the interface between the device and the user’s nervous system, bringing sensory feedback in artificial limbs closer to clinical reality.

Challenges remain before such sensors become routine hardware. Fabricating computation directly into or alongside an imaging array adds manufacturing complexity and cost, and any processing fixed in hardware must be versatile enough to serve many different tasks, from gripping a fragile egg to identifying a bolt by its knurled surface. Calibration and long-term stability are perennial concerns for soft elastomer components, which can fatigue or creep over millions of contact cycles. The field will also need standardized benchmarks so that in-sensor architectures can be compared fairly against conventional sensing-plus-processing pipelines on accuracy, latency, and energy. The authors position their work as a demonstration of the paradigm rather than the final word on its engineering, and the trajectory of the broader literature suggests rapid iteration: vision-based tactile sensing has matured from laboratory curiosities to commercially available fingertips in under a decade, and the addition of native computation is likely to accelerate that curve rather than interrupt it.

If that trajectory holds, the significance of the work may ultimately lie less in any single sensor design than in the architectural principle it validates: that the future of robotic touch belongs not to better cameras paired with faster computers, but to sensing devices that think. As robots move out of cages and factories and into homes, hospitals, and fields, they will need to handle the world with the same effortless, instantaneous responsiveness that human skin provides. Seamless tactile sensing and perception, computed where contact happens, is a major step toward hands that do not merely record touch but understand it.

Subject of Research: Vision-based tactile sensing with in-sensor computing for robotic touch perception

Article Title: A vision-based tactile sensor with in-sensor computing paradigm for seamless tactile sensing and perception

Article References: Fan, W., Zheng, J., Liu, Y., & Zhang, D. (2026). A vision-based tactile sensor with in-sensor computing paradigm for seamless tactile sensing and perception. Communications Engineering. https://doi.org/10.1038/s44172-026-00770-w

Image Credits: AI Generated

DOI: 10.1038/s44172-026-00770-w

Keywords: tactile sensing, in-sensor computing, robotics, robotic touch, vision-based sensors, tactile perception, dexterous manipulation, slip detection, neuromorphic sensing, robotic prosthetics, vision-based, tactile

The device was developed by researchers led by Xiaoxian Zhang and Yongsheng Wang at Beijing Jiaotong University, in collaboration with Yuchao Yang and Yaoyu Tao at Peking University. Their approach combines an ambipolar semiconductor, tungsten diselenide, or WSe₂, with a thin ferroelectric layer made from hafnium zirconium oxide, known as HfₓZr₁₋ₓO₂ or HZO. The resulting architecture is designed to function not merely as a light sensor, but as a programmable optical computing element capable of changing how it responds to incoming light.

At the heart of the device is a sub-20-nanometer HZO ferroelectric film integrated with a split-gate structure. Ferroelectric materials possess a switchable internal electric polarization. Once that polarization is changed, it can continue influencing the electronic behavior of a device even after the programming voltage is removed. In this photodiode, the polarization of the HZO layer modifies the electrical environment of the WSe₂ channel, allowing the researchers to control the polarity of the device and reconfigure its photocurrent response.

WSe₂ is particularly useful for this purpose because it is ambipolar. Depending on the surrounding electric field and gate conditions, it can conduct through either electrons or positively charged holes. The split-gate design takes advantage of this dual behavior to create a reconfigurable homojunction, a junction formed within the same semiconductor system rather than between conventional materials with different electronic properties. By switching the ferroelectric polarization, the researchers can alter the junction configuration and reverse the direction of the photocurrent.

One of the most notable features of the device is that it can switch its photocurrent polarity without requiring an external bias during operation. In conventional photodetectors, an applied voltage is often needed to drive current or tune the response, adding to energy consumption and complicating circuit integration. The reported architecture instead uses the stored polarization of the ferroelectric layer to establish the required electrostatic conditions internally. This enables light detection and signal modulation under zero external bias, an important step toward low-power sensing systems.

The programming process is also designed to be highly energy efficient. The researchers report a programming energy below one femtojoule, a scale that is extraordinarily small compared with the energy typically associated with moving data between a sensor and a processor. The device switches between states in approximately 50 microseconds and retains its programmed weight for more than 100 seconds. In this context, the “weight” represents the adjustable strength or sign of the device response, allowing it to act as an analog computational element rather than a simple on-or-off detector.

The team demonstrated that the photodiode could perform in-situ preprocessing of optical signals, including matrix-vector multiplication, a fundamental operation in artificial intelligence and neural-network algorithms. Matrix-vector multiplication normally requires large numbers of data transfers between memory and processing units. When implemented directly through the physical responses of devices, the operation can occur in parallel as light is detected, reducing the need for repeated digital conversion and memory access. The reconfigurable photocurrent of the WSe₂-HZO device provides a physical means of applying adjustable computational weights to optical inputs.

To test its practical potential, the researchers used the photodiode as a physical convolution kernel in simulated image edge-detection tasks. Convolution kernels apply mathematical weight patterns to neighboring pixels to identify features such as boundaries, contours, and fine structures. The device produced edge maps that were nearly indistinguishable from those generated by ideal software calculations. At its optimal programmed weight state, the system achieved a normalized mean squared error of approximately 3.2 × 10⁻⁴, indicating a close match between the hardware-generated and software-generated results.

The researchers say the work addresses several limitations that have slowed the development of neuromorphic vision hardware. Many existing devices provide only a unidirectional photocurrent, limiting their ability to represent positive and negative computational weights. Others depend on continuous external bias or require high programming voltages. Device concepts based on Schottky barriers or polymer ferroelectrics such as PVDF can also face challenges involving reliability, manufacturing compatibility, and scaling. By using HZO, a ferroelectric material more closely aligned with established semiconductor processing, the new design points toward a potentially CMOS-compatible route for integrating sensing, memory, and computation.

The result does not yet represent a complete commercial vision processor, but it provides a compact building block for future systems in which pixels detect, remember, and interpret optical information at the same physical location. Such architectures could be valuable in cameras that need to respond rapidly while operating on limited power, from edge-AI devices and smart sensors to wearable systems and autonomous machines. The study, published in Nano Research, illustrates how ferroelectric polarization and two-dimensional semiconductors can be combined to make photodetectors programmable, computational, and substantially more energy efficient.

Subject of Research: Reconfigurable ferroelectric photodiodes and in-sensor computing using HZO and ambipolar WSe₂.

Article Title: Achieving energy-efficient in-Sensor computing via ferroelectric reconfigurable homojunctions

News Publication Date: 12-May-2026

Web References: https://doi.org/10.26599/NR.2026.94908610; https://www.sciopen.com/journal/1998-0124

References: Nano Research, DOI: 10.26599/NR.2026.94908610

Image Credits: Nano Research, Tsinghua University Press

Keywords

In-sensor computing, neuromorphic vision, ferroelectric electronics, HfₓZr₁₋ₓO₂, HZO, WSe₂, ambipolar semiconductors, reconfigurable photodiodes, CMOS-compatible devices, edge detection, optical computing, low-power electronics