Dueling Imputations: Deterministic Framework Sharpens AI Time-Series Forecasting

Missing data is one of the oldest and most stubborn enemies of artificial intelligence. In sensor networks that monitor air quality, energy grids, hospitals, and industrial plants, readings vanish for all sorts of mundane reasons: a sensor fails, a transmission drops, a report arrives late, or extreme weather interrupts collection. For time-series forecasting systems, those gaps are more than an inconvenience. They break the temporal continuity on which predictive models depend, and the way engineers fill them can quietly determine whether a forecast is trustworthy or wildly off the mark. A new study published in Discover Informatics argues that the fix lies not in smarter models but in smarter, and strikingly disciplined, data repair.

Researchers led by Agung Bella Putra Utama, Aji Prasetya Wibawa, and Anik Nur Handayani of Universitas Negeri Malang, together with Andrew Nafalski of Adelaide University, have introduced a deterministic duel-based imputation framework that puts forecasting performance itself in charge of deciding how missing values should be filled. The central insight is deceptively simple: most imputation methods are judged by how statistically similar their reconstructions are to the original data, not by whether they actually help a forecasting model make better predictions. A filled-in series can look numerically plausible on paper and still wreck a downstream forecast by smoothing away the very peaks and rhythms the model needs to learn.

The framework works like a structured tournament. For every missing observation, the system generates a set of candidate values using familiar baseline techniques, including mean, median, and mode substitution, K-Nearest Neighbors, Multiple Imputation by Chained Equations, and Last-Observation-Carried-Forward. Each candidate is then scored with a composite loss function that combines three complementary forecasting metrics: Mean Absolute Percentage Error, which captures proportional error; Root Mean Square Error, which penalizes large absolute deviations; and the coefficient of determination, or R-squared, which measures how much of the variance in the real series the reconstruction explains. The weights are set at 0.4, 0.4, and 0.2 respectively, so error-based criteria dominate while explanatory power still plays a supporting role.

What separates this approach from conventional candidate ranking is the way winners are chosen. Instead of collapsing every candidate into a single aggregated score and picking the top one, the framework stages deterministic pairwise duels. Candidates face each other one-on-one, and a sensitivity threshold of 0.001 prevents negligible score differences from flipping outcomes. After all comparisons, candidates are ranked by cumulative wins, and the top three advance to a final aggregation stage. Four aggregation modes are available: averaging the survivors for low-variance stability, taking the median for outlier robustness, taking the maximum to emphasize peak responsiveness, or a winner-take-all selection of the candidate with the lowest forecasting loss. Because every step follows fixed rules, the same input always produces the same output, a property the authors argue is essential for auditing, validation, and accountability in operational systems.

The team evaluated the framework on the Beijing PM2.5 dataset, a widely used benchmark of hourly meteorological and air-pollution measurements collected between 2010 and 2014. After cleaning and temporal alignment, 41,776 valid records remained out of 43,800, with roughly 2,068 missing values concentrated in the PM2.5 variable itself. Crucially, the missingness is not random: gaps cluster in cold seasons and during periods of rapid atmospheric change, exactly the conditions where forecasting matters most and where naive imputation does the most damage. The dataset includes dew point, temperature, pressure, cumulative wind speed, snowfall, rainfall, and temporal indicators as explanatory variables, with PM2.5 concentration serving as the prediction target.

The imputed datasets were then fed into four recurrent forecasting architectures trained under identical conditions: a vanilla Recurrent Neural Network, a Long Short-Term Memory network, a Bidirectional LSTM, and a Gated Recurrent Unit. Hyperparameters for all models were tuned with Particle Swarm Optimization, though the researchers are careful to note that this stochastic tuning happens entirely outside the imputation stage and does not compromise its determinism. The optimized configurations converged on surprisingly lightweight designs: three hidden layers of 24 neurons each, sigmoid activations, the Adam optimizer with mean squared error loss, a batch size of 32, 46 training epochs, and a dropout rate of 0.2.

The results are striking. Compared with conventional imputation methods, the duel-based framework reduced MAPE by up to 70 percent and RMSE by 12 to 15 percent, while maintaining R-squared values above 0.95 across all four architectures. The Mean aggregation variant delivered the strongest overall performance, preserving both proportional variation and the amplitude structure of the signal. Dropping missing observations entirely, by contrast, produced the worst forecasts, confirming that simply discarding incomplete rows destroys the temporal coherence recurrent models rely on. Even BRITS, a sophisticated deep-learning imputation method, showed less consistent gains, suggesting that reconstruction quality alone does not guarantee forecasting quality.

Statistical testing reinforced the case. Paired t-tests and Wilcoxon signed-rank tests comparing the proposed Mean variant against KNN, the strongest conventional baseline, produced p-values below 0.01 across all architectures, with each experiment repeated ten times under fixed random seeds. Visual analysis of predicted versus observed PM2.5 trajectories showed close alignment across smooth and volatile segments alike, with the Bi-LSTM achieving the highest R-squared values and the GRU delivering the most efficient runtime while matching LSTM accuracy. The framework’s computational overhead scales quadratically with the number of candidates but linearly with data size, and because the candidate set is small and fixed, runtimes remain predictable even as data volume grows.

To test generalizability, the researchers extended the evaluation beyond air quality to three additional domains: the Heart Disease dataset, which has weak temporal structure; the KEDS e-journal dataset, which shows irregular behavioral patterns and moderate sparsity; and the Sunspot dataset, which exhibits strong periodic behavior. The framework achieved lower MAPE and RMSE than baselines including drop-missing, mean imputation, KNN, and BRITS across all of them, with the largest gains appearing in datasets with strong sequential patterns such as Sunspot. The authors acknowledge limitations, including the quadratic cost of pairwise comparison for large candidate pools and the fixed forecasting horizons tested, and they point to cluster-based candidate reduction and online learning as future directions. But the broader message stands: imputation should not be a passive preprocessing chore. Treated as a forecasting-driven decision process, it becomes a strategic lever for building AI systems that are not only accurate but consistent, transparent, and worthy of trust.

Subject of Research: A deterministic duel-based imputation framework that integrates forecasting performance metrics into missing-data selection for reliable AI-driven time-series forecasting.

Article Title: A deterministic duel-based imputation framework for reliable AI-driven time-series forecasting

Article References: Utama, A. B. P., Wibawa, A. P., Handayani, A. N., & Nafalski, A. (2026). A deterministic duel-based imputation framework for reliable AI-driven time-series forecasting. Discover Informatics, 1(1), Article 3. https://doi.org/10.1007/s44564-026-00001-6

Image Credits: AI Generated

DOI: 10.1007/s44564-026-00001-6

Keywords: time-series imputation, missing data, AI forecasting, deterministic imputation, deep learning, LSTM, GRU, RNN, PM2.5, forecasting reliability, machine learning, data preprocessing