Conventional machine vision architectures channel visual information through physically segregated imaging, photodetection, analog-to-digital conversion, and memory storage stages before transferring digital arrays to backend electronic processors. This sequential pipeline imposes severe data movement penalties, elevated thermal footprints, and processing latencies that constrain real-time robotic and embodied intelligence platforms. To mitigate these bottlenecks, researchers are shifting computational workloads upstream toward the point of optical data capture. Writing in PhotoniX Synergy, an academic research team led by Zheng Huang and Hongwei Chen has reviewed the technological trajectory of optical pre-sensor computing. This emerging operational framework embeds analog computational operators directly into propagating light fields prior to sensor photodetection, restructuring the traditional imaging-centric pipeline into an intrinsically computing-centric architecture. By extracting task-relevant visual representations within the optical domain, pre-sensor computing suppresses raw spatial redundancy before electronic digitization takes place. The paradigm harnesses the fundamental physical attributes of photons—namely propagation velocity, massive spatial parallelism, broad spectral bandwidth, and sub-nanosecond processing latencies—to execute front-end feature extraction and compression.
At the foundational level of pre-sensor computing, engineered optical operators execute elementary mathematical transformations directly across spatial wavefronts during unconstrained propagation. Elementary operators routinely perform spatial convolution, differentiation, integration, and temporal filtering without drawing active electrical power during the optical transfer process. Convolutional operations encode spatial kernels through mechanisms such as angle-selective diffractive layers, metasurface-engineered point spread functions, and polarization-multiplexed optical modulation schemes. These optical arrangements execute parallel local feature filtering across high-resolution visual inputs before photon absorption occurs. Spatial differentiation operators enable edge enhancement and boundary extraction by exploiting physical phenomena including the photonic spin Hall effect, the Brewster angle effect, photonic-crystal mode coupling, geometric-phase metasurfaces, and multilayer planar dielectric structures. These configurations support first-order, second-order, and higher-order spatial derivatives that feed directly into downstream electronic classification networks. Although these discrete operators streamline localized feature processing, their fixed mathematical transformations remain insufficient for fully autonomous, end-to-end perceptual tasks, driving researchers to develop more sophisticated optical neural networks.
Diffractive optical neural networks form the most widely investigated architecture for multi-layer optical front ends, emulating deep neural networks through cascaded phase and amplitude modulation surfaces. Light incident on a diffractive layer generates secondary wavelets whose phase delays and transmission amplitudes are tailored across wavelength-scale features, allowing Huygens-Fresnel coherent superposition to direct energy toward targeted spatial coordinates. Since the initial demonstration of three-dimensionally printed five-layer diffractive architectures for character classification, diffractive designs have expanded into dynamic, reconfigurable systems. Researchers have demonstrated reconfigurable diffractive processing units combining digital micromirror devices, spatial light modulators, and complementary metal oxide semiconductor sensors, establishing diffractive recurrent configurations for temporal processing alongside adaptive error-correction training routines. Diffractive networks have executed all seven fundamental logic operations, performed five-layer holographic image recovery across tens of thousands of parameters per layer, reconstructed scenes through complex scattering media, and carried out multi-dimensional Fourier transformations. Recent architectures have further adapted coherent matrix transforms to operate under spatially incoherent monochromatic light by mapping intensity distributions, allowing passive optical inference under uncoordinated illumination.
Metasurface engineering has accelerated this diffractive paradigm by condensing bulky free-space assemblies into flat, subwavelength nanostructures that manipulate optical polarization, amplitude, and phase within micron-scale envelopes. Multilayer metasurface arrays integrated with field-programmable gate arrays have achieved dynamic modulation ranges reaching thirty-five decibels, supporting online reinforcement learning for adaptive multibeam focusing. Metasurface diffractive systems trained using proximal policy optimization algorithms have extended optical inference into complex, game-theoretic decision-making environments, exhibiting performance benchmarks comparable to human players. Beyond discriminative classification, metasurface platforms have entered generative computing regimes through all-optical semantic generation chips like LightGen, which integrates metasurface encoders and generators with single-mode fiber latent-space conversion units. This architecture executes high-resolution semantic image generation, neural radiance field three-dimensional modeling, and high-definition video synthesis, yielding throughput and energy efficiency gains two orders of magnitude above conventional electronic microprocessors. Parallel investigations using silicon metasurfaces have demonstrated robust anti-interference diffractive recognition in cluttered, multi-object environments while compressing optical footprints down to the sub-millimeter regime.
Fourier-optics systems utilize the intrinsic mathematical properties of optical lenses to execute spatial-frequency transformations across standardized four-focal-length arrangements. In these configurations, an initial lens converts input spatial signals into frequency distributions at the Fourier transform plane, where spatial light modulators or phase masks apply targeted filtering before an inverse-transform lens reconstructs the spatial output. By modulating frequency planes, four-f system neural networks perform high-throughput convolutions and channel-tiled multidimensional filtering. Reconfigurable amplitude modulation based on digital micromirrors in Fourier planes has demonstrated processing throughputs on the order of peta-operations per second with latencies down to one hundred microseconds while mitigating coherent noise. Specialized four-f configurations also implement non-trainable spatial pooling operations, such as optical average and motion pooling, reducing parameter bloat and suppressing overfitting. Recent hybrid frameworks alternate lightweight digital neural approximations with optical diffraction learning to accelerate network convergence. Furthermore, introducing controlled partial incoherence through spatial light modulators bearing dynamic random phase profiles has suppressed speckle noise and diffraction artifacts, strengthening model robustness under fluctuating illumination environments.
Alternative pre-sensor architectures exploit complex scattering phenomena and ray-based geometrical optics to circumvent the physical constraints of conventional wave diffraction. Scattering networks harness the deterministic multiple scattering that occurs when coherent or incoherent wavefronts traverse disordered media, mapping input signals into high-dimensional linear representations. Researchers have physically instantiated random projection matrices using dopant scatterers in nanophotonic silica, rotating adhesive tape disks optimized via genetic algorithms, and living three-dimensional tumor spheroids where cellular structures serve as biological scattering neurons. In parallel, geometrical-optics neural networks rely on refraction, reflection, and amplitude modulation rather than wave interference. Lensless optoelectronic systems using passive amplitude masks achieve single-shot optical convolution and dilated feature extraction under ambient incoherent illumination. Multi-layer geometrical platforms combine spatial light modulators, microlens arrays, and image intensifiers to duplicate scenes across sixteen hundred input neurons, executing optical fan-out and fan-in while achieving input compression ratios reaching eight hundred to one. Optoelectronic conversion layers pairing photodetector arrays with light-emitting diodes provide intermediate signal re-amplification, whereas spiropyran thin films enable fluorescence-based matrix multiplication and ultraviolet in-memory visual display computing.
Translating pre-sensor optical computing from laboratory breadboards to operational hardware requires direct monolithic integration onto semiconductor image sensor substrates. Researchers have demonstrated compact optical decryptors utilizing high-density optical linear perceptrons nanoprinted directly onto complementary metal oxide semiconductor sensor dies. These nanoprinted structures attain neuron densities exceeding five hundred million neurons per square centimeter—six orders of magnitude above conventional macroscopic diffractive networks—while executing passive inference across a working distance of sixty-two point eight micrometers. Multifunctional metasurface diffractive chips integrated onto image sensors enable polarization-multiplexed parallel processing across visible wavelengths. In waveguide architectures, optical parallel computational array chips integrate high-quality-factor resonators coupled to bus waveguides, achieving end-to-end optical image processing and reconstruction with a six-nanosecond response time across a one-hundred-and-sixty-nanometer optical bandwidth. Furthermore, integrated spectral optical neural network chips combine metasurface convolutional layers directly with sensor pixels to execute snapshot hyperspectral feature extraction. When deployed for biomedical diagnosis of meibomian gland dysfunction, this integrated spectral chip achieved an accuracy of ninety-six point twenty-two percent, exceeding standard three-channel digital color imaging models within a compact physical package.
Because unperturbed optical propagation and passive phase modulation represent purely linear transformations, cascaded optical layers without nonlinear activation collapse mathematically into a single equivalent linear operation, severely restricting model capacity. To overcome this limitation, optical neural networks incorporate either optoelectronic or all-optical nonlinear activation mechanisms across two-dimensional spatial planes. Optoelectronic approaches exploit square-law photodetector detection, intermediate amplification circuits driving light-emitting diodes or vertical-cavity surface-emitting lasers, microchannel plate saturation in image intensifiers, or self-modulating transparent molybdenum disulfide phototransistors coupled to liquid crystal layers. Conversely, all-optical nonlinear activation relies on intrinsic optical material dynamics, avoiding electrical conversion delays and power overheads. Saturable absorbers, reverse saturable absorbers, two-dimensional graphene, rubidium vapor cells, photorefractive crystals, and cadmium selenide quantum dot films provide thresholding and saturation responses resembling rectified linear units or sigmoid functions. In cadmium selenide quantum dot multilayer networks, photoluminescence thresholding shifted up to ninety-five percent of the computational burden into the optical domain under ambient light. In addition, multi-reflection cavities and folding arrangements within linear diffractive platforms have generated higher-order nonlinear transformations without dedicated active materials.
Recent architectural developments emphasize optoelectronic fusion schemes that bridge pre-sensor optical front ends with near-sensor and in-sensor electronic processing arrays. An analog photoelectronic computing chip demonstrated in twenty-twenty-three combined diffractive optical feature extraction for high-resolution imagery with an electronic array of one thousand and twenty-four photodiodes executing fully connected vector operations, delivering an energy efficiency of seventy-four point eight peta-operations per watt and throughput of four point six peta-operations per second. Bio-inspired optoelectronic spatiotemporal image sensors developed in twenty-twenty-six emulate retinal cone cells, the eye lens, and the visual cortex by integrating pre-sensor, in-sensor, and near-sensor functions into a three-layer binary convolutional network, achieving three hundred and sixty-seven giga-operations per watt. Commercially oriented hardware initiatives, including the Tulkas T100 optical processing unit developed by Austin-based startup Neurophos, reportedly generate four thousand peta-operations per second while drawing one percent of the power consumed by standard electronic chips through folded optical compression paths. These cooperative hybrid architectures unite the zero-power spatial filtering of free-space optics with the reconfigurability, high sensitivity, and precise electrical signal routing characteristic of advanced integrated complementary metal oxide semiconductor electronics.
Despite these experimental milestones, deploying optical pre-sensor computing across autonomous vehicles, robotics, and embodied physical artificial intelligence systems requires overcoming critical operational hurdles. Physical AI platforms operate under strict size, weight, and power envelopes while demanding millisecond-scale perceptual feedback for closed-loop motor control. Transitioning beyond controlled laboratory benches necessitates robust compatibility with broadband, spatially incoherent natural sunlight rather than artificial coherent laser sources, eliminating redundant optoelectronic scene-encoding stages. Furthermore, static passive elements must evolve into low-power adaptive systems; replacing power-hungry spatial light modulators with nonvolatile optical materials or lightweight electronic parameter tuning, analogous to low-rank adaptation in large foundation models, could provide low-overhead task reconfigurability. Looking forward, researchers envision an end-to-end processing pipeline that couples free-space pre-sensor visual front ends directly into post-sensor on-chip photonic integrated circuits based on microring arrays or Mach-Zehnder interferometer meshes. Overcoming coupling losses, unifying disparate signal representations, and implementing co-optimized end-to-end training across multi-scale optical regimes will be essential to establish optical pre-sensor computing as a foundational perception framework for next-generation intelligent physical machines.
Subject of Research: Optical pre-sensor computing paradigms and neural architectures for front-end visual processing
Article Title: Review of optical pre-sensor computing
Article References: Huang, Z., Wang, C., Shi, W., Wu, S., He, Y., Li, Y., Zhang, C., & Chen, H. (2026). Review of optical pre-sensor computing. PhotoniX Synergy, 1(1), Article 7. https://doi.org/10.1007/s44519-026-00009-3
Image Credits: AI Generated
DOI: 10.1007/s44519-026-00009-3
Keywords: optical computing, pre-sensor computing, machine vision, diffractive deep neural networks, metasurface optics, Fourier optics, photonic neural networks, spatial light modulators, nonlinear optical activation, optoelectronic fusion, physical AI, in-sensor computing, free-space optics, analog optical processing, hyperspectral feature extraction, optical operators, embodied intelligence
Cite Scienmag News
Elena Sutton. (November 18, 2026). Optical Pre-Sensor Computing Processes Visual Data Directly Within Incident Light Fields. Scienmag. https://scienmag.com/optical-pre-sensor-computing-processes-visual-data-directly-within-incident-light-fields/
Elena Sutton. "Optical Pre-Sensor Computing Processes Visual Data Directly Within Incident Light Fields." Scienmag, 18 November 2026, https://scienmag.com/optical-pre-sensor-computing-processes-visual-data-directly-within-incident-light-fields/. Accessed 18 November 2026.
Elena Sutton. "Optical Pre-Sensor Computing Processes Visual Data Directly Within Incident Light Fields." Scienmag. November 18, 2026. https://scienmag.com/optical-pre-sensor-computing-processes-visual-data-directly-within-incident-light-fields/








