AI Framework Rebuilds Missing Urban Air Pollution Data While Quantifying Its Own Uncertainty

A new artificial intelligence framework can reconstruct missing air-pollution measurements across a city while telling researchers how much to trust its own estimates, according to a study published in Scientific Reports. The system, called CAT-GAN-UKF, was developed by Lior Tobaly, Zeev Zalevsky and Eyal Yaniv and tested on real monitoring data from the New York City metropolitan area, where sensor outages and intermittent data loss are a persistent problem for urban air-quality networks. The problem the researchers set out to address is structural rather than incidental.

Urban monitoring networks are inherently incomplete: measurements disappear intermittently, sustained outages occur, and the reliability of the remaining evidence changes with the pollutant involved, the location of the sensor, the time of day and the overall state of the network. CAT-GAN-UKF combines three technical ingredients. The first is a set of pollutant-specific conditional Wasserstein generative adversarial networks with gradient penalty, a form of adversarial machine learning trained to model the distributions of process and observation residuals, meaning the discrepancies between predicted and actual values.

Crucially, these networks were trained only on signed residual distributions derived from the chronological training period, so no information from later data could leak into the model. The second ingredient is a nonlinear network unscented Kalman filter, a state-estimation algorithm that propagates pollutant levels through space and time. The third is a masked recursive Bayesian updating scheme that ties the pieces together.

The innovation lies in how the two components communicate. The Monte Carlo variance of residuals generated by the adversarial networks defines time-varying diagonal covariance matrices, denoted Q and R, which are injected directly into the Kalman filter. In plain terms, the generative model learns, moment by moment and location by location, how noisy the process and the sensors are likely to be, and the filter uses that knowledge to weight new measurements against its own internal predictions.

The primary model uses causal temporal features, lagged values, rolling statistics, missingness indicators and spatial context from neighbouring stations. The evaluation relied on 153,260 hourly station-time records drawn from an OpenAQ-derived 2024 dataset covering the New York metropolitan area. After predefined quality control, 27 of 33 monitoring locations were retained.

All learned parameters were fitted on training data only, with a separate validation interval used for pollutant-specific multiplicative interval calibration, and the held-out test period was never used for fitting or calibration, a discipline that guards against optimistic performance estimates. The team assessed robustness under four random-masking levels, four contiguous outage lengths, and an empirical-gap replay protocol whose outage-length distribution was learned from naturally occurring gaps in the training period.

Each stress test was repeated over 30 masking seeds, and the framework was compared against matched linear Kalman filters, fixed-covariance and heuristic adaptive unscented Kalman filters, as well as machine-learning baselines including Extra Trees, random forest, histogram gradient boosting, a multilayer perceptron, ridge and lasso regressions, persistence and temporal means. After Holm correction for multiple comparisons, CAT-GAN-UKF significantly improved root-mean-square error over the adaptive UKF in seven of nine scenarios, over the linear Kalman filter in all nine, over Extra Trees in all nine, and over simple persistence in eight of nine.

Against a strong fixed-covariance UKF, however, point-estimation accuracy was generally comparable rather than uniformly superior, an honest nuance the authors report directly. Where the new framework clearly pulled ahead was in uncertainty quantification. Relative to the fixed-covariance UKF, it reduced the average width of 95 percent prediction intervals by approximately 38 percent across the nine scenarios while keeping mean empirical coverage near 95 percent. Narrower intervals at the same coverage mean the system is not just accurate but efficiently honest: it wastes less of its uncertainty budget on cases it can pin down confidently and reserves wider bounds for genuinely ambiguous situations.

Ablation analysis revealed that spatial coupling between stations is the dominant structural contribution to performance. Removing it increased RMSE by approximately 5.9 percent under 10 percent random masking and by 42.7 percent under a 24-hour outage, indicating that information from neighbouring sensors is what carries the reconstruction through long gaps. Adding meteorological covariates made things worse: temperature and humidity were evaluated only in a sensitivity analysis because they did not improve reconstruction in this benchmark, and including them worsened RMSE throughout.

The authors are careful about scope. They describe CAT-GAN-UKF as an uncertainty-efficient research framework for spatial reconstruction under degraded sensing, and they note that real-time latency, memory use and resource-constrained deployment feasibility were not evaluated. The findings apply to the New York City benchmark tested here, not to operational forecasting systems.

Subject of Research: Uncertainty-aware machine learning reconstruction of missing urban air-quality monitoring data

Article Title: CAT-GAN-UKF: uncertainty-aware spatiotemporal reconstruction for resilient urban air-quality monitoring under partial observability

Article References: Tobaly, L., Zalevsky, Z., & Yaniv, E. (2026). CAT-GAN-UKF: uncertainty-aware spatiotemporal reconstruction for resilient urban air-quality monitoring under partial observability. Scientific Reports. https://doi.org/10.1038/s41598-026-77620-4

Image Credits: AI Generated

DOI: 10.1038/s41598-026-77620-4

Keywords: air quality, machine learning, generative adversarial networks, Kalman filter, uncertainty quantification, New York City, sensor networks, data reconstruction, environmental monitoring, Scientific Reports, CAT-GAN-UKF, uncertainty-aware