Machine learning fills missing air pollution data in Delhi urban study

Delhi, one of the most polluted megacities on Earth, has long presented scientists with a frustrating paradox: the air quality data needed to understand and combat its pollution crisis are frustratingly incomplete. Now, a new study published in Environmental Science and Pollution Research offers a powerful solution, using an army of machine learning models to fill the decade-long gaps in Delhi’s pollution records and, in doing so, reconstruct the most complete picture yet of how the city’s toxic air has evolved between 2014 and 2024.

The research, conducted by Sedra Shafi and Nicola Scafetta of the Department of Earth Sciences, Environment and Georesources at the University of Naples Federico II in Italy, tackles a problem that quietly undermines air pollution science worldwide. Monitoring networks in fast-growing megacities are rarely as complete as researchers would like. Instruments fail. Stations open late or close early. Power outages, maintenance schedules, and budget constraints all conspire to leave holes in the datasets that scientists and policymakers depend upon. In Delhi, the problem was particularly acute: the city operated only four monitoring stations in 2014, expanding gradually to 45 by 2024. That fragmentation means that any attempt to assess decade-scale trends in pollutants such as fine particulate matter, nitrogen dioxide, ozone, sulfur dioxide, and carbon monoxide has been built on shaky statistical ground.

The stakes could hardly be higher. Chronic exposure to PM2.5—particles smaller than 2.5 micrometers that penetrate deep into the lungs and enter the bloodstream—is linked to cardiovascular disease, respiratory illness, lung cancer, and reduced life expectancy. Global studies have estimated that air pollution shortens average human life expectancy by a margin rivaling that of smoking. For Delhi, consistently ranked among the world’s worst cities for air quality, reliable long-term records are essential for evaluating whether policy interventions—from vehicle restrictions to the banning of certain fuels—have actually worked. Without complete data, trend analysis and policy evaluation become exercises in guesswork.

The core insight of the new study is elegant in its simplicity: pollution measured at nearby monitoring stations is not independent. Stations that are geographically close tend to experience similar weather, similar emission patterns, and similar atmospheric conditions, which means their readings are strongly correlated. If one station’s record has a gap, its well-correlated neighbors contain the statistical information needed to reconstruct what the missing station would likely have measured. The challenge lies in doing this accurately and systematically across thousands of missing values, six different pollutants, and 45 stations over eleven years.

Shafi and Scafetta’s answer is an iterative, multi-model machine learning workflow implemented in MATLAB’s Regression Learner application. For every single missing data point, the algorithm first identifies the four stations most strongly correlated with the target station for that particular pollutant—the assumption being that these stations share the most similar environmental conditions. The observations from these four neighboring stations then serve as predictor variables in a regression problem, where the goal is to predict the concentration at the gapped station from the concentrations measured elsewhere.

But rather than committing to a single algorithm, the researchers let the data decide. Their workflow evaluates 35 different machine learning regression models for each imputation task, spanning an impressive range of statistical approaches. These include decision tree methods such as Fine Tree ensembles and Bagged Trees, support vector machines with Fine Gaussian kernels, ensemble methods including Optimizable Ensembles, and kernel-based regressions employing Rational Quadratic and Exponential kernels. For each missing value, the model that delivers the best reconstruction accuracy on the available data is selected, the gap is filled, and the procedure moves on. Because filling one gap can change the statistical landscape for subsequent gaps, the process iterates until every missing value across the entire network has been reconstructed.

The results reveal a clear hierarchy among the competing algorithms. Tree-based methods—Fine Tree and Bagged Trees—along with Optimizable Ensemble, Fine Gaussian SVM, and the Rational Quadratic and Exponential kernel regressions, consistently outperformed traditional multilinear regression. This finding echoes a growing body of evidence across the atmospheric sciences: the relationships between pollution levels at different locations are rarely linear. Nonlinear effects arising from atmospheric chemistry, complex wind patterns, seasonal meteorology, and heterogeneous emission sources mean that flexible, nonlinear learners capture the underlying structure of urban pollution fields far better than classical linear models.

To validate the approach, the researchers conducted artificial-gap experiments, deliberately removing known data points across different pollutants, stations, and years, then testing how accurately their workflow could recover the true values. The results were encouraging but nuanced. The method performed well under the tested conditions for PM2.5 and other pollutants whose concentrations are driven largely by regional processes—broad-scale weather systems, seasonal dynamics like the winter crop-residue burning that blankets northern India in smoke each autumn, and city-wide meteorological conditions. When pollution is regionally coherent, neighboring stations genuinely do carry redundant information, and machine learning can exploit that redundancy effectively.

The story was different for pollutants dominated by highly local sources. Species whose concentrations are controlled by nearby traffic, individual industrial emitters, or street-level microenvironments show strong local variability that neighboring stations simply cannot capture. For these pollutants, reconstruction accuracy dropped. The authors are candid about this limitation: their approach is well suited to long-term regional reconstruction, but it cannot conjure information about hyper-local emission events that no nearby sensor recorded. This honesty matters, because imputation methods that silently overstate their reliability can bias downstream analyses in dangerous ways.

Beyond the technical achievement, the reconstructed datasets enabled something Delhi has never had before: ensemble-mean pollutant records computed across all 45 stations for the full 2014–2024 period. By averaging across a complete, spatially distributed network, the researchers produced a more realistic and bias-corrected representation of the city’s air quality evolution than any sparse or incomplete dataset could provide. Notably, this analysis indicates that Delhi’s air quality has shown modest improvement over the decade—a finding that would have been impossible to establish with confidence from the fragmented original records, and one with direct implications for assessing the effectiveness of the city’s air quality management efforts.

The study also situates itself within a rapidly evolving field. Machine learning has become a staple of air pollution research, applied to forecasting particulate concentrations, estimating PM2.5 from satellite data across China and the United States, and building spatiotemporal prediction models from land-use random forests to extreme gradient boosting. Deep learning approaches to missing data imputation, including bidirectional recurrent networks and generative adversarial frameworks, have shown promise in time series applications. What distinguishes the new work is its focus on long-term reconstruction across a dense urban network—and its pragmatic, competition-based model selection, which avoids locking in any single algorithm a priori.

The implications extend well beyond Delhi. Rapidly urbanizing cities across South and Southeast Asia, Africa, and Latin America share the same pathology: young monitoring networks with short, patchy records, superimposed on severe pollution problems. The workflow developed by Shafi and Scafetta offers a template for such cities—demonstrating that with only the correlation structure among existing stations and a bank of regression models, researchers can salvage scientifically usable decadal records from otherwise unusable fragments. The authors have made their MATLAB scripts implementing the four-step iterative procedure, along with the model-selection settings for all 35 regression algorithms, available in the supplementary materials of the paper, lowering the barrier for other research groups to adopt and adapt the method.

There are, of course, boundaries to what imputation can achieve. Reconstructed data are statistical estimates, not measurements, and any analysis built on them inherits the uncertainties of the reconstruction process. The authors’ own validation makes clear that performance varies by pollutant, by station, and by year, and that pollutants with strong local emission signatures remain the weak point of any neighbor-based approach. Future refinements might incorporate meteorological variables, satellite retrievals, or chemical transport model outputs as additional predictors, potentially improving performance for locally driven species.

Nevertheless, the study represents a meaningful step forward in the unglamorous but essential work of data rescue. As governments worldwide face mounting pressure to demonstrate progress on air quality—and as the health toll of pollution continues to mount—the ability to construct trustworthy long-term records from imperfect monitoring networks becomes a scientific necessity. For Delhi, a city whose struggle with air pollution has become emblematic of the global urban air quality crisis, the machine learning reconstruction offers both a clearer view of the past and a firmer foundation for the decisions that will shape its atmospheric future. The air over Delhi may remain toxic, but the record of that toxicity is now, for the first time, whole.