Sensor-token network rebuilds photoacoustic images without the system matrix

Photoacoustic tomography has long promised the best of two worlds: the rich optical contrast of hemoglobin, melanin, and lipids, and the deep penetration and fine spatial resolution of ultrasound. A nanosecond laser pulse heats tissue slightly, the tissue expands thermoelastically, and broadband acoustic waves race outward to an array of ultrasound transducers. Recovering the initial pressure distribution from those time-resolved signals, however, is a notoriously ill-posed inverse problem, especially when the sensor array is sparse. A new study published in Frontiers in Applied Mathematics and Statistics proposes a strikingly direct answer: a Transformer-style neural network that treats each sensor’s full time series as a token and maps raw measurements to a reconstructed image in a single forward pass, never touching the system matrix at inference time.

The physics behind the problem is compact. Under the assumption of a homogeneous, lossless medium with constant sound speed, the forward problem can be written as y = Hx₀ + n, where H is the discretized system operator, x₀ is the vectorized image, and n is noise. In practice, H is severely under-determined: the 71 transducers in this study form a closed square aperture that fully surrounds the 64 × 64 pixel imaging domain but samples it only at every fourth boundary grid point, a sparse-view configuration. Naive inversion yields streaking artifacts and poor resolution of fine vascular detail, so the field has invested heavily in compressive-sensing solvers such as the Iterative Shrinkage-Thresholding Algorithm (ISTA) and split-Bregman total variation (SBTV), which exploit the spatial sparsity of vessel-like structures.

Both classical iterative solvers and modern unrolled deep networks share a costly bottleneck: they require repeated multiplication by H and its adjoint Hᵀ at inference time. Each matrix-vector product costs on the order of 4.65 × 10⁷ floating-point operations for the chosen grid, and ISTA alone needs 500 iterations per reconstruction. Algorithm unrolling, in which a fixed number of iterative steps is unfolded into a trainable network with each layer corresponding to one iteration, preserves the physics while exploiting data-driven priors, but every unrolled layer still performs an Hx and Hᵀx product, so the per-image cost scales as K·O(MN), where K is the unrolled depth. For clinically realistic grid resolutions, this remains a serious obstacle to real-time use.

The new architecture, named the Sensor-Token Reconstruction Network (STRN), belongs to a different family: direct, end-to-end reconstruction that bypasses the system matrix entirely at inference. Each sensor’s 160-sample time series is projected into a 128-dimensional token embedding by a shared linear layer. Sinusoidal positional encodings convey sensor ordering, and the tokens are processed by two stacked Transformer encoder blocks with four attention heads each. Multi-head self-attention allows every sensor to interact with every other sensor, capturing the nonlocal acoustic relationships encoded in the PAT forward model. The resulting token matrix is flattened and decoded into the reconstructed image through three fully connected layers, with a sigmoid output matching the normalized image range. Training minimizes a vessel-weighted mean squared error, assigning a penalty weight of 8 to vessel pixels and 1 to background, and the network contains roughly 18 million trainable parameters.

Crucially, the analytical system matrix is not discarded but repurposed. The authors constructed an analytical k-space H-matrix in a single pass, exploiting the spatial Fourier decomposition of the initial pressure field, and validated it against the k-Wave pseudo-spectral time-domain solver under matched geometry. Two targeted regularizations were needed: Hann k-space apodization to suppress Gibbs-like aliasing from DFT truncation, and Gaussian temporal damping with a time constant set to 90 percent of the measurement duration to eliminate non-physical late-time oscillations. Applied together, these regularizations reduced the energy-normalized mismatch against k-Wave by 49 percent, from an NMSE of 0.496 to 0.254. The mean per-sensor Pearson correlation between analytical and k-Wave signals reached 0.919 ± 0.049, with all 71 sensors exceeding the acceptance threshold of r = 0.80, confirming that the analytical model faithfully reproduces the temporal waveform structure of the numerical reference.

The training corpus comprised 191 grayscale radiological images from Radiopaedia.org, selected for vessel-like branching structures and treated as surrogate initial-pressure maps. Each was resized to the 64 × 64 grid, contrast-enhanced, and augmented by small spatial translations, yielding 488 training samples. The network was trained for 300 epochs with the Adam optimizer on a single commodity laptop CPU, an Intel Core i7-1255U with 16 GB of RAM, with no GPU acceleration. On a strictly held-out test set of 46 samples, STRN achieved a mean SSIM of 0.522 ± 0.225, a mean PSNR of 22.09 ± 7.42 dB, and a mean NMSE of 0.233 ± 0.172. Twenty-one of the 46 test samples exceeded SSIM 0.5, ten exceeded 0.7, and the best sample reached 0.965.

The benchmark comparison against three baselines on the same test set was decisive. ISTA, run with λ = 0.009 and 500 iterations, achieved a mean SSIM of only 0.212 and a PSNR of 10.49 dB, recovering the gross vessel layout but producing diffuse, low-amplitude reconstructions with streak artifacts. SBTV performed even worse on structural metrics, with SSIM 0.173 and PSNR 9.36 dB, introducing staircase artifacts characteristic of total-variation regularization. LISTA, the strongest baseline, reached SSIM 0.502, statistically indistinguishable from STRN on that single metric, but fell decisively behind on every other: STRN’s PSNR was 7.83 dB higher, its NMSE 3.5 times lower, and its Pearson correlation 0.41 higher, all significant at p < 10⁻⁸ under paired t-tests and Wilcoxon signed-rank tests. The pattern is consistent with a known limitation of unrolled networks trained on limited data: the learned proximal operator can become overly aggressive at sparsifying the iterate, leaving incomplete coverage on multi-vessel phantoms.

Two validation analyses addressed the validity and the mechanism of the approach. First, an inverse-crime concern arises when training and testing share a forward operator, so the trained network was evaluated, without retraining, on measurements produced independently by the k-Wave solver. Zero-shot, SSIM fell from 0.522 to 0.192 while Pearson correlation was largely preserved at 0.786, a 94 percent retention, indicating that the gross spatial mapping transfers across forward models even though fine amplitude calibration does not. The gap proved rapidly closable: roughly 20 fine-tuning samples nearly doubled the zero-shot SSIM, and roughly 160 samples recovered about 90 percent of the in-distribution fidelity, supporting the interpretation of the analytical H-matrix as an efficient training surrogate rather than a source of an inverse crime.

Second, and perhaps most strikingly, a parameter-matched ablation revealed that the self-attention sub-layers are not the source of the network’s accuracy. Replacing multi-head self-attention with a parameter-matched position-wise MLP, with no inter-sensor mixing, matched or exceeded the full attention model on every metric and every seed, with SSIM 0.479 ± 0.003 versus 0.458 ± 0.012 under the matched 150-epoch budget. The dominant parameter block is the fully connected spatial decoder, which accounts for roughly half of the network’s parameters, and a decoder-size sweep confirmed that attention conferred a measurable benefit only when the decoder was starved to the point of uselessness. Sensor identity, not attention, is what the network exploits: removing positional encoding lowered SSIM to 0.438, and randomly permuting sensor ordering at test time collapsed SSIM from 0.522 to 0.074. The authors report this negative result openly, framing STRN as a direct, matrix-free architecture whose fidelity is driven by the learned decoder over a sensor-token representation, and pointing toward simpler and cheaper per-token designs for fixed detector arrays.

The computational payoff is independent of the reconstruction-quality argument. Each iterative baseline performs at least one H matrix-vector product per iteration, with ISTA using 500, SBTV 76 outer iterations, and LISTA 16 unrolled layers, while STRN performs a single forward pass dominated by dense matrix multiplications on the order of 10⁷ operations. On the same commodity CPU hardware, this single-pass cost is at least an order of magnitude below the cumulative per-iteration cost of all three baselines. A robustness study further characterized the trained model under realistic measurement imperfections: the network proved most robust to per-channel detector-gain variation, tolerating a 20 percent gain spread with only a 0.036 SSIM drop, moderately robust to additive noise down to 20 dB, and most sensitive to per-channel timing jitter, where a one-sample jitter already lowered SSIM to 0.461. This ordering is consistent with the mechanism of a direct time-of-flight mapping and identifies inter-channel timing calibration as the dominant requirement for experimental deployment. The authors acknowledge that the training corpus remains limited to a single public source, that the analytical forward model assumes a homogeneous lossless medium, and that scaling with image resolution, sensor count, and grid spacing has not yet been characterized. Future work includes extending the architecture to three-dimensional PAT with volumetric decoders, integrating physics-informed loss terms, and investigating whether the sensor-as-token paradigm generalizes to related tomographic modalities such as thermoacoustic tomography and ultrasound computed tomography, where any fixed sensor array producing sparse measurements might benefit from the same matrix-free shortcut.

Subject of Research: Matrix-free direct photoacoustic image reconstruction with a sensor-token Transformer network

Article Title: Matrix-free photoacoustic image reconstruction via sensor-token networks

Article References: John, M., Said, S., Barhumi, I., Turaev, S., & Yahia, M. (2026). Matrix-free photoacoustic image reconstruction via sensor-token networks. Frontiers in Applied Mathematics and Statistics, 12, Article 1952041. https://doi.org/10.3389/fams.2026.1952041

Image Credits: AI Generated

DOI: 10.3389/fams.2026.1952041

Keywords: photoacoustic tomography, image reconstruction, Transformer, sensor tokens, matrix-free inference, deep learning, sparse-view imaging, inverse problems, compressive sensing, domain generalization, architectural ablation, k-Wave simulation