The findings matter because Kelmarsh data have supported several dozen wind-energy studies since 2022. Conventional quality control generally addresses questionable values within a record, but it cannot by itself reveal that measurements begin late in a file, that the meaning of a channel changed, or that an entire year has been divided into overlapping blocks. In the audit, these problems remained hidden when researchers merely filtered implausible observations. The authors therefore treated file structure, provenance and measurement definitions as components of data quality rather than assuming that a successfully parsed time series was internally consistent and complete.

The first defect affects the apparent timing of data availability. For most measurements, the record starts partway through the year rather than on January 1. These onset points occur at 12 different steps between 2016 and 2023, indicating a pattern across files rather than a single isolated interruption. If an entire annual file is classified as missing before its signals actually begin, calculated missing-data rates can exceed the true gap by as much as 73 percentage points. The audit produced a signal-onset manifest to document when each measurement becomes available, allowing users to distinguish a genuine absence of observations from the earlier beginning of the stored record.

A second problem emerges in the main wind-speed measurement between November 2017 and April 2018. During that interval, the channel changes from representing the average of two sensors to representing a single sensor. This is a change in what the measurement means, not simply a fluctuation in performance. When the two definitions were compared, the change made the turbines appear between 1.2% and 9.1% more productive at a given wind speed. By contrast, the single-sensor series alone changed by between −0.6% and +1.5%. The contrast shows why a mixed series can create the appearance of improved productivity: the apparent shift is tied to sensor configuration rather than a comparable change within one sensor.

The largest structural error occurs in the 2023 and 2024 files. Instead of storing each year as one continuous sequence, the authors found 81 overlapping segments for each year. Measurements are present only in the first copy of each time step, leaving 97.6% of rows without wind-speed or power values. The result is an array that looks highly incomplete if all rows are retained and subjected to ordinary missing-value filtering. If the first copy of each duplicated time step is kept, the year can be recovered; if the last copy is kept, less than 1% of the data remains. That second outcome produces annual capacity factors—annual output expressed as a share of the maximum possible output—that are overstated by 25 to 53 percentage points.

This pattern was not limited to one year. Files tested from a second wind farm owned by the same operator share the segmented structure, indicating that the issue may reflect the owner’s file-production process rather than an isolated defect in the Kelmarsh release. That observation does not establish the precise cause or prove that every file from the operator is affected, but it broadens the potential scope of the problem. It also demonstrates why provenance information matters: file layout, not only numerical values, can determine whether a dataset represents a complete operating year or a collection of mostly empty overlapping records.

The audit also found smaller inconsistencies and placeholder values in records for energy, voltage, wind speed and temperature. These defects are distinct from the three large structural failures because they occur within otherwise identifiable series rather than changing the coverage or definition of the whole record. The authors provide a table that flags suspect values alongside reconstructed files for 2023 and 2024, a manifest of signal onset dates, and the code and outputs used for the analysis. Their supplied dataset is identified as version 4 of the Kelmarsh SCADA data, while the analysis package is released separately so that the checks and reconstructions can be reproduced.

Not every pre-specified prediction survived the audit. Of the predictions reported in the preprint, one failed, seven failed as originally worded, and one could not be evaluated as specified. The authors state that all outcomes are reported rather than narrowing the discussion to the successful cases. This distinction is important for interpreting the reconstruction: the work is a provenance audit supported by reproducible checks, not a demonstration that a single method can diagnose every possible dataset defect. Its strongest conclusions come from directly documented structures in the files, including the onset dates, sensor-definition switch, overlapping blocks and their measurable effects on coverage and calculated summaries.

For downstream research, the results show that open-data reuse requires checks that go beyond deleting extreme or physically implausible values. Analysts need to verify when each signal starts, determine whether a channel retains the same measurement definition over time, and inspect how timestamps and values are distributed across repeated segments. Deduplication also requires a deliberate rule: choosing the first populated copy recovers the 2023 and 2024 measurements, whereas selecting the final copy produces severe data loss and inflated capacity factors. The audit’s code, corrected files and flag table are intended to support those choices, but the preprint itself remains open for discussion and under journal review, so its findings should be understood as results reported by the authors rather than a settled consensus.

Subject of Research: Provenance defects and downstream effects in the Kelmarsh wind farm SCADA dataset

Article Title: Structural defects in the Kelmarsh open SCADA dataset: a provenance audit with file reconstruction and downstream consequences

Article References: Park, A., Yin, L., Lee, C., Wong, A., & Choi, Y. (2026). Structural defects in the Kelmarsh open SCADA dataset: a provenance audit with file reconstruction and downstream consequences. https://doi.org/10.5194/wes-2026-166

Image Credits: AI Generated

DOI: 10.5194/wes-2026-166

Keywords: Kelmarsh wind farm, open wind data, SCADA datasets, data provenance, data reconstruction, wind speed sensors, missing data, capacity factor, quality control, reproducible research, Structural, defects

The study draws on the European Injury Database, a multinational surveillance system that collects standardized information on injury presentations at emergency departments across participating countries. Between 2008 and 2023, more than 62,000 attendances for intentional self-harm were recorded from 18 countries, but the researchers faced a fundamental obstacle: the vast majority of records, over 86 percent, carried an unknown value for the outcome they most wanted to understand, namely whether the patient had a documented history of prior self-harm. Rather than discarding this wealth of data, the team set out to understand why the information was missing in the first place, and whether the missingness itself followed a pattern that could be modeled and worked around.

What they found was striking. When they tested how well the missingness of the outcome could be predicted from the available variables, country membership alone explained a remarkable share of the variation, achieving a pseudo-R-squared of 0.537 and an area under the curve of 0.948. In plain terms, knowing which country a record came from was almost sufficient to predict whether the prior self-harm field would be documented at all. This reflects deep structural differences in how national surveillance systems operate, how EDs record clinical information, and how data are transmitted to the central database. Documentation practices, not clinical reality, were driving the gaps.

The analysis also revealed a subtler and more consequential pattern: documentation was positively biased. Across countries, the sparser the overall documentation, the higher the proportion of positive flags among the records that were documented at all, with a Spearman correlation of 0.882. This suggests that clinicians or coders in systems with low documentation rates may only record the prior self-harm field when they have a particularly strong reason to do so, meaning that documented cases are not a random sample of the true prevalence. Any model trained naively on such data would learn to recognize documentation behavior as much as clinical reality.

To navigate these problems, the researchers adopted what they describe as a three-step strategy. First, they excluded three countries whose outcome protocols were structurally absent, meaning the field was never or almost never completed. Second, they redefined the prediction target: instead of trying to predict the true underlying history, the model would predict whether a record carries a documented positive flag versus its absence. Third, they retained records with known outcome status, yielding an analytical dataset of 26,554 records with a prevalence of 13.7 percent. This approach preserved roughly three times as much data as a complete-case analysis would have allowed, which matters because statistical power and the diversity of training examples both shrink rapidly when entire records are discarded.

The modeling approach centered on XGBoost, a gradient boosting algorithm that builds an ensemble of decision trees, each successive tree correcting the errors of those before it. Gradient boosting has become a workhorse of structured clinical data prediction because it handles mixed variable types, captures nonlinear relationships and interactions without manual specification, and performs well on moderately sized datasets. Crucially for a clinical application, the researchers paired the model with SHAP values, a technique rooted in cooperative game theory that assigns each feature a quantified contribution to every individual prediction. This transforms the model from an opaque score generator into something clinicians can inspect, question, and audit.

Performance was solid but appropriately modest. On the held-out test set, the model achieved an area under the receiver operating characteristic curve of 0.800, with a confidence interval running from 0.783 to 0.816. The area under the precision-recall curve was 0.380, a figure that reflects the inherent difficulty of the task given the 13.7 percent prevalence, and the Brier score of 0.100 indicated reasonably well-behaved probability estimates. The model significantly outperformed a logistic regression baseline, with a DeLong test p-value of 3.93 multiplied by ten to the power of minus ten, confirming that the nonlinear structure of the boosted trees captured relationships the linear model could not.

Threshold selection is where statistical performance meets clinical reality. At the optimized operating point of 0.080, the model detected 80.8 percent of documented positives while correctly clearing 62.8 percent of negatives, and the negative predictive value reached 95.4 percent. These trade-offs matter enormously in an emergency setting. A high-sensitivity configuration means fewer at-risk patients slip through, but it also flags many patients who will turn out to have no documented history, generating additional workload. The negative predictive value is particularly relevant for triage: when the model says a patient is unlikely to have a documented prior self-harm flag, it is right more than nineteen times out of twenty.

Recognizing that a model is only useful if it can run at the moment a patient arrives, the researchers built a second version restricted to variables available at time zero, meaning at triage before any extended clinical workup. This time-zero model achieved an AUC of 0.762, only modestly below the full model, and still comfortably ahead of the linear baseline at 0.729. The small performance sacrifice is a worthwhile price for a tool that can actually inform the first clinical encounter rather than arriving after decisions have already been made.

The SHAP analysis pointed to proximal risk factor documentation, place of occurrence, and patient age as the dominant drivers of predictions. This raised an important methodological concern: the proximal risk factor variable could be circular, since documentation of risk factors might itself be correlated with documentation of prior self-harm history, meaning the model could partly be learning documentation habits rather than clinical patterns. To test this, the team reran the analysis with that variable excluded. Performance dipped only slightly, to an AUC of 0.765, confirming that the model’s core signal does not depend on the potentially circular feature.

Perhaps the most methodologically revealing exercise was the country prediction test. When the researchers trained a model to guess which country a record came from using only the clinical features, it achieved 95.3 percent accuracy, fourteen times better than random chance. This demonstrates that the clinical variables in the database encode substantial geographic information, whether through differing coding conventions, injury mechanisms, demographic profiles, or health system characteristics. This implicit geographic encoding offers a plausible explanation for a troubling pattern in the validation results: the model scored 0.800 on the randomly split test set, but only 0.674 under temporal validation and 0.658 under group-aware cross-validation, where entire countries or time periods were held out. The gap between random and grouped validation is a classic signature of leakage, where the model exploits site-specific quirks that will not transfer to new settings.

Calibration, the agreement between predicted probabilities and observed frequencies, showed a similarly nuanced picture. In most countries with sufficient data, the model’s probability estimates were adequately calibrated, meaning a predicted risk of twenty percent roughly corresponded to twenty percent of such patients having a documented flag. In low-volume countries, however, calibration broke down, likely because the model had fewer examples to learn country-specific patterns and because local documentation practices differed from the training distribution. This is a familiar challenge in multinational health data: a model that is well calibrated on average can be badly miscalibrated for specific subpopulations.

The fairness analysis surfaced what the authors identify as a critical limitation. Sensitivity was substantially lower in older adults, at 68.0 percent, with an AUC in that subgroup of 0.722 and a wide confidence interval stretching from 0.617 to 0.822. Older patients who self-harm are a population of particular concern, since late-life self-harm is associated with especially high risk of completed suicide, often with higher lethality of methods and less prior contact with mental health services. A model that under-detects documented history in this group could inadvertently direct attention away from some of the patients who need it most. The authors are candid that this disparity must be addressed before any deployment.

The authors are equally careful about what the model is and is not doing. This is a retrospective pattern-recognition task, not a crystal ball. The model does not predict future suicide attempts, and it does not infer undocumented clinical history. It learns which combinations of injury characteristics, demographics, and clinical features recorded at ED attendance are associated with a positive prior self-harm flag in the surveillance database. The clinical rationale is that patients whose observable patterns strongly resemble those of patients with documented prior histories may warrant closer attention, but the output is a flag for consideration, not a diagnosis.

The path from these results to clinical practice runs through several gates that the authors spell out. Local validation is essential, because the geographic encoding demonstrated in the country prediction test means a model trained on pooled European data may behave very differently in any single health system. Country-specific recalibration would adjust the model’s probability outputs to match local documentation prevalence and coding practices. And perhaps most importantly, the model would need to be embedded within structured post-identification care pathways, because the evidence from large-scale ED studies is clear that screening alone, whether by questionnaire or algorithm, does not by itself reduce suicidal behavior. Identification only saves lives when it connects to follow-up: safety planning, rapid outpatient linkage, means restriction counseling, and repeated contact.

There are also broader lessons here for the field of clinical machine learning. The study demonstrates that formal missingness analysis should precede model building when outcomes are inconsistently recorded, that the choice between complete-case analysis and target redefinition can multiply usable data several-fold, and that validation must go beyond random splits to include temporal and group-aware designs that expose leakage. The finding that a model can look excellent on a random test set while degrading sharply under realistic validation conditions is a cautionary tale for any predictive tool trained on multinational data. Equally, the demonstration that SHAP-based interpretability can coexist with strong discrimination, and that sensitivity analyses can probe potentially circular features, offers a template for transparent model development in sensitive clinical domains.

For emergency departments weighing whether such tools belong in their workflows, the realistic framing is as a decision support layer rather than an autonomous gatekeeper. A model that flags a presentation as consistent with prior self-harm history could prompt a targeted question at triage, trigger a review of available records, or cue a brief psychosocial assessment, all actions that remain under clinician control. Used this way, even a model with imperfect sensitivity adds value by making the invisible visible at the moment it matters. The study’s contribution is not a finished product but a rigorous demonstration that routine surveillance data, handled with methodological care, can support earlier recognition of one of the highest-risk populations walking through the emergency department door, provided the surrounding systems of care are ready to respond.

Subject of Research: Machine learning prediction of documented prior intentional self-harm history in European emergency department injury surveillance data

Article Title: Predicting documented prior intentional self-harm in emergency departments: an interpretable gradient boosting approach using European injury surveillance data

Article References: Giustini, M., Pitidis, A., Fazio, C., Alves, T., Fian, T., Valkenberg, H., Nijman, S., Papadakaki, M., & Carannante, A. (2026). Predicting documented prior intentional self-harm in emergency departments: an interpretable gradient boosting approach using European injury surveillance data. Frontiers in Digital Health, 8, Article 1964834. https://doi.org/10.3389/fdgth.2026.1964834

Image Credits: AI Generated

DOI: 10.3389/fdgth.2026.1964834

Keywords: machine learning, XGBoost, self-harm, suicide prevention, emergency department, EU-IDB, SHAP, predictive modelling, missing data, algorithmic fairness, injury surveillance, digital health

The impact of missing data on the validity of study estimates depends heavily on why the data are missing. Pregnant women from lower socioeconomic backgrounds tend to access antenatal care later in pregnancy and enroll later in research studies, which means they accumulate more missing data early in pregnancy. This is particularly problematic because socioeconomic status is itself an important determinant of drinking behavior during pregnancy. Women also modify their consumption after recognizing they are pregnant, and the timing of that recognition varies widely. Some stop or reduce drinking immediately upon learning of the pregnancy, while some heavy drinkers continue to binge in the first trimester or maintain heavy drinking throughout gestation. Standard approaches to handling missing values, such as last value carry forward or mean imputation, are poorly suited to this setting because drinking behavior during pregnancy varies by gestational timing and by day of the week, and because the pattern of missingness is itself linked to drinking recency.

A team of researchers led by Ayesha Sania of Columbia University Irving Medical Center and the New York State Psychiatric Institute, working with colleagues at NYU Grossman School of Medicine, Avera Health, the University of South Dakota, and Stellenbosch University in Cape Town, has now demonstrated a machine learning solution to this problem. Writing in Advances in Drug and Alcohol Research, the team applied the k-nearest neighbor algorithm, or k-NN, to impute missing daily alcohol consumption data from the Safe Passage Study, a prospective cohort investigation conducted by the Prenatal Alcohol and SIDS and Stillbirth Network. The Safe Passage Study enrolled 11,892 pregnant women between August 2007 and January 2015 from antenatal clinics in the Northern Plains of the United States and in Cape Town, South Africa, with 11,083 women included in the imputation analysis. Participants were eligible if they were pregnant with one or two fetuses, aged 16 years or older, at least six weeks gestational age at recruitment, and spoke English or Afrikaans. Women were followed throughout pregnancy and for one year postnatally, with ethical approval obtained from institutional review boards including Stellenbosch University, Sanford Health, the Indian Health Service, and participating Tribal Nations.

Alcohol exposure data in the Safe Passage Study were collected using a modified version of the Timeline Followback method, a validated self-report instrument in which participants report drinking on their last known drinking day and then for the 30 days prior. The instrument captured remarkable detail: types and number of drinks, container sizes, the amount of ice in each drink, how many people shared a drink, and the duration of drinking episodes. These data were converted into estimates of total alcohol consumed and the number of standard drinks on each reported drinking day. Yet the design of the method inherently generates a variable amount of missing data per participant. Women who never drank, or whose last drinking day preceded their last menstrual period, had complete records. Women who drank but quit within 30 days of a collection period had few or no gaps. But women who continued to drink and reported their most recent 30 days of intake left unrecorded intervals stretching back toward the start of pregnancy. Frequent drinkers consequently accumulated more missing days than infrequent drinkers, meaning that simple summation of reported drinks would systematically underestimate true consumption and bias any analysis built on that metric.

The k-NN algorithm addresses this challenge through pattern recognition rather than statistical modeling. As a non-parametric method, it makes no distributional assumptions about the underlying data, an advantage given that alcohol consumption during pregnancy is highly skewed, with most drinking concentrated in the first trimester before pregnancy recognition. The algorithm works by measuring the similarity between the drinking pattern of a participant with missing data and the patterns of other participants in the database. Similarity is quantified using cosine distance, which captures the shape of a drinking pattern regardless of its absolute magnitude. Once the k most similar participants are identified, the weighted average of their drinking values for the missing day serves as the best estimate, with nearer neighbors exerting greater influence than distant ones. The team also scaled imputed values to each individual’s consumption level, ensuring that a participant with a similar pattern but heavier overall intake was not matched to a lighter drinker’s absolute values.

Data preparation required careful engineering. The team converted the records so that each participant occupied a single row, with drinking values as separate variables for each day from 15 days before the last menstrual period through day 310 of pregnancy. Because drinking behavior varies by day of the week, participants were temporally aligned so that, for example, a participant whose pregnancy began on a Wednesday was compared with others whose corresponding days also fell on Wednesdays. Outlier drinking values were Winsorized, or capped, at three standard deviations to limit their influence on imputed values. Participants lacking any first-trimester data or missing more than 200 days were excluded, since their drinking patterns could not be reliably established. A crucial methodological insight shaped the reference pool: women with completely missing-free records were more likely to be nondrinkers or light drinkers, so using only them as neighbors would underestimate true drinking. Instead, the team included segments of non-missing data from all participants as reference material.

Validation proceeded by randomly deleting data for 5 to 15 consecutive days from the first-trimester records of participants with no missing data, then comparing imputed values against the actual deleted values. The first trimester was chosen because both the proportion of women drinking and the magnitude of their drinking are highest then, particularly before pregnancy recognition. The team examined root mean squared error across values of k from 1 to 10 and across reference segment lengths. They found that five nearest neighbors and reference segments of 55 days, roughly two months, provided imputed values with the least error. Shorter segments did not meaningfully increase the available reference pool, while the 55-day length preserved the algorithm’s ability to characterize specific drinking patterns. For nondrinking segments, a single neighbor produced the lowest error, but multiple neighbors improved prediction of drinking segments, making k equal to 5 a reasonable compromise for both.

The results were striking. Across 3.2 million person-days of observation, 0.36 million person-days, or 11.4 percent, were missing. About 45 percent of participants had alcohol data for every single day of pregnancy, while the remaining 55 percent had at least one day missing. Sixty-two percent of participants were drinkers, consuming at least one drink during pregnancy. After deleting data segments from complete first-trimester records, the algorithm produced no difference between actual and predicted values for 64 percent of deleted segments. For another 31 percent of segments, imputed values fell within plus or minus one drink per day of the actual values. The algorithm exactly predicted drinking status for 76 percent of drinking segments in the combined analysis. Accuracy varied by study site: nondrinking segments were predicted within one drink for 78.6 percent of deleted segments in South Africa and 67.6 percent in the Northern Plains, reflecting differences in the magnitude of drinking and the proportion of missing data between sites.

Imputation also shifted population-level estimates in informative ways. Following imputation, the mean number of drinks in the first trimester increased by an average of two drinks in South Africa, while the increase for the Northern Plains sites was just below one drink. The Northern Plains had fewer missing data points overall, with over half of participants having 30 or fewer missing days, and lower proportions of drinkers and drinking volumes. Most missing data at the South African site came from early trimesters, reflecting later enrollment there, whereas most missing data in the Northern Plains occurred in the third trimester. This distinction matters because early-pregnancy missing data are more likely to represent active drinking periods, given that many women reduce or stop drinking after learning of the pregnancy.

The study is, to the authors’ knowledge, the first to describe this method for imputing missing alcohol consumption data in a longitudinal study of pregnant women. Prior comparative work had found that multiple imputation and full information maximum likelihood yielded less biased estimates than complete case analysis or last observation carried forward, but even those methods underestimated actual drinking levels, and imputing irregularly spaced longitudinal data would require complex extensions of standard multiple imputation. The non-parametric nature of k-NN sidesteps the normality assumptions embedded in most standard software packages, making it well suited to skewed, high-volume daily data. The k-NN algorithm is increasingly used to impute missing data in high-volume research areas such as genetics and metabolomics, where it has often produced the smallest imputation error compared with mean and median imputation, Bayesian linear regression, and clustering approaches. The authors note that the algorithm cannot overcome bias introduced when participants misreport drinking days as nondrinking days, and that future studies should evaluate alternative distance measures and other machine learning approaches. The methodological details presented here hold relevance well beyond alcohol research, extending to substance use and nutrition studies that suffer from informative missing data in longitudinal designs.

Subject of Research: Machine learning imputation of missing prenatal alcohol exposure data

Article Title: K-nearest neighbor algorithm for imputing missing longitudinal prenatal alcohol data

Article References: Sania, A., Pini, N., Nelson, M. E., Myers, M. M., Shuffrey, L. C., Lucchini, M., Elliott, A. J., Odendaal, H. J., & Fifer, W. P. (2025). K-nearest neighbor algorithm for imputing missing longitudinal prenatal alcohol data. Advances in Drug and Alcohol Research, 4, Article 13449. https://doi.org/10.3389/adar.2024.13449

Image Credits: AI Generated

DOI: 10.3389/adar.2024.13449

Keywords: k-nearest neighbor, machine learning, data imputation, prenatal alcohol exposure, missing data, longitudinal studies, Safe Passage Study, Timeline Followback, epidemiology, alcohol research, K-nearest, neighbor

The system, called IndoorCART, short for Indoor Context-Aware RSSI Transformer, was developed by Balaram Singh, Rajdeep Chatterjee, Mahendra Kumar Gourisaria, Manoj Sahni, and Ernesto León-Castro, with the first three authors based at KIIT Deemed to be University in India. It is an encoder-only transformer, the same family of architecture that underpins modern language models, but retooled for a very different kind of data. Instead of words, the model receives tokens representing the received signal strength indicator, or RSSI, from each Wi-Fi access point in a building. Each token is paired with a learnable identity embedding so the network knows which access point it is hearing, without imposing any artificial order on the measurements. Because Wi-Fi fingerprints have no natural sequence, the authors argue that self-attention, which lets every token interact with every other token in parallel, is a better fit than recurrent or convolutional designs that assume locality or time.

The most distinctive piece of the architecture is what the researchers call Adaptive Attention. In a typical fingerprint, only a fraction of the access points are actually detected; on the widely used UJIIndoorLoc benchmark, more than 62 percent of the possible measurements are missing. Conventional pipelines either fill these gaps with placeholder constants, which can masquerade as genuine weak signals, or discard the missing entries entirely, throwing away the spatial hints that the pattern of absence carries. IndoorCART does neither. Every undetected access point is assigned a learnable absent-signal embedding, a shared null state that tells the encoder the signal was not observed rather than weakly received. A custom attention mask then down-weights interactions involving absent tokens, and a density-adaptive dropout scheme increases regularization when few signals are visible, protecting the model from overfitting to sparse observations. The authors are careful to frame this as a domain-specific adaptation of standard scaled dot-product attention rather than a wholly new attention mechanism, and they note that a uniform null embedding is a conservative choice because the benchmark datasets do not record the physical locations of the access points.

After the encoder stacks have contextualized the tokens, IndoorCART aggregates them with a hybrid strategy that combines a learnable CLS token, which summarizes global fingerprint context, with global average pooling across all access-point tokens, which preserves distributed spatial information. A regression head then maps this representation directly to continuous coordinates. That last choice matters more than it might seem. Many recent transformer-based localization systems frame the problem as classification over discrete reference points, predicting which survey location a fingerprint belongs to and reporting errors relative to label centroids. Direct coordinate regression is a harder task, because the model must predict positions anywhere in the physical space and is scored against true coordinates in meters, but it is also the formulation that matters for real deployments.

The evaluation is unusually thorough for this field. The team compared IndoorCART against nine reproducible regression baselines, including linear regression, k-nearest neighbors, support vector regression, random forests, XGBoost, multilayer perceptrons, one-dimensional convolutional networks, LSTMs, and GRUs, all trained under identical preprocessing, splits, and metrics. They also reimplemented six recent transformer-based localization systems, including AATS, the L-SwiGLU Transformer, HyTra, VTIL, VITAL, and a Siamese-encoder transformer, under the same controlled protocol. Across three public benchmarks, IndoorCART achieved mean absolute errors of 4.79 meters on UJIIndoorLoc, 0.18 meters on TUJI1, and 1.33 meters on SODIndoorLoc, with coefficients of determination of 0.99, 0.98, and 0.99 respectively. On SODIndoorLoc, a sprawling multi-building dataset collected at Shandong Jianzhu University in China, the improvement over the strongest reproduced transformer baseline, HyTra, reached nearly 30 percent.

The ablation studies show where the gains come from. Swapping the transformer encoder for an MLP, a CNN, or a BiLSTM increased error by roughly 12 to 30 percent across the benchmarks, confirming that global self-attention captures cross-access-point structure that local or sequential models miss. Removing the adaptive components entirely, meaning the absent-signal embeddings, the custom masking, and the density-adaptive dropout, raised the mean error from 4.79 to 5.67 meters on UJIIndoorLoc and from 1.33 to 1.89 meters on SODIndoorLoc. A worst-case analysis using the 95th-percentile error showed that adaptive masking also reduces severe localization failures, not just average error, which matters for safety-critical applications such as emergency response inside buildings.

Notably, the authors resist the temptation to oversell. On UJIIndoorLoc, the margin over a simple k-nearest-neighbor regressor was only 0.11 meters, a 2.24 percent relative improvement, and several baselines matched the same rounded coefficient of determination. The team attributes this to the dataset’s dense sampling, with more than fifteen fingerprints per location, and its strong local spatial autocorrelation, which lets similarity-based methods perform near saturation. They even estimate a theoretical error floor of about 4.45 meters for the dataset given its low signal-to-noise ratio, suggesting IndoorCART is already close to what the data can support. Similarly, the striking 0.18-meter result on TUJI1 comes with an explicit caveat: because the study used the dataset’s fixed split without enforcing device-disjoint or location-disjoint testing, the figure should be read as interpolation performance under that split, not as proof of generalization to unseen devices or reference points.

The paper also tackles an uncomfortable gap between its own numbers and those reported elsewhere in the literature. Several published transformer systems claim sub-meter errors on UJIIndoorLoc, roughly an order of magnitude below IndoorCART’s 4.79 meters. The authors identify six factors behind the discrepancy, including classification-based formulations that cannot localize between reference points, smaller proprietary testbeds, cross-validation protocols that can yield optimistic estimates, and differing error metrics. When they reproduced HyTra, L-SwiGLU, and VITAL under their own standardized protocol, none matched IndoorCART, and the much-cited 0.355-meter result for L-SwiGLU could not be reproduced on the full UJIIndoorLoc test set. The comparison, they stress, positions IndoorCART as the strongest model within this controlled study rather than as a universal state-of-the-art claim.

Deployment practicality receives its own analysis. The base model carries 8.2 million parameters, occupies 32.8 megabytes, and infers a position in 0.234 milliseconds, well within real-time requirements and, remarkably, using only 0.08 gigabytes of RAM at inference because it does not store the training data the way k-nearest-neighbor and random-forest methods must. Pruning and INT8 quantization shrink the model to 5.7 megabytes and cut inference time to 0.052 milliseconds with only a 2.7 percent accuracy penalty, opening the door to edge and mobile deployment. The authors recommend the full model for sparse or structurally complex environments, simpler methods for dense, well-structured radio maps, and hybrid local-global schemes as a promising middle ground.

The broader significance is twofold. For engineers, IndoorCART offers a template for treating missingness as a first-class signal in tabular sensing problems, a lesson that likely extends beyond Wi-Fi to Bluetooth, ultra-wideband, and emerging 6G integrated sensing systems. For the research community, the paper is a case study in honest evaluation: it distinguishes controlled reimplementations from literature-reported numbers, quantifies uncertainty with bootstrap confidence intervals and paired statistical tests, and states plainly where its advantages shrink. As demand grows for centimeter-to-meter accuracy in the GPS-denied spaces where people actually live and work, frameworks that combine architectural adaptation with this level of methodological candor may prove as influential as the accuracy numbers themselves.

Subject of Research: A transformer-based framework for Wi-Fi fingerprinting indoor localization that models missing RSSI signals

Article Title: IndoorCART: a spatially-aware transformer framework for Wi-Fi fingerprinting indoor localization

Article References: Singh, B., Chatterjee, R., Gourisaria, M. K., Sahni, M., & León-Castro, E. (2026). IndoorCART: a spatially-aware transformer framework for Wi-Fi fingerprinting indoor localization. Frontiers in Signal Processing, 6, Article 1884243. https://doi.org/10.3389/frsip.2026.1884243

Image Credits: AI Generated

DOI: 10.3389/frsip.2026.1884243

Keywords: indoor localization, Wi-Fi fingerprinting, transformer, RSSI, self-attention, coordinate regression, UJIIndoorLoc, missing data, Adaptive Attention, deep learning, positioning, signal processing

The core difficulty is structural. GDP evolves jointly with labor-market conditions, investment, trade, prices, exchange rates, and fiscal variables, and these relationships shift across countries and over time. Recent shocks, including the COVID-19 pandemic, inflationary surges, supply-chain disruptions, and large exchange-rate movements, have only sharpened the instability and nonlinearity of macroeconomic dynamics. Meanwhile, cross-country economic databases are rarely balanced. Historical coverage differs across variables, countries, and sources; some country-year observations are simply absent. The conventional responses, dropping incomplete variables or filling gaps with imputed values, either discard useful information or blur the crucial distinction between what was actually observed and what was manufactured by an algorithm. The MCT takes a different route: it keeps the missingness visible throughout the entire forecasting pipeline.

Technically, the framework rests on four integrated components. First, an explicit binary observation mask accompanies every input, telling the network which of the entries in a 10-year by 14-variable window are genuine measurements and which are unavailable. Second, a reconstruction pathway is trained jointly with the forecaster, but its supervision is applied only where verified ground truth exists, namely originally observed values that are deliberately hidden during training. Naturally unavailable entries are never assigned unverified reconstruction targets. Third, cross-variable interactions are modeled through an asymmetric cosine-normalized query–key similarity attention mechanism rather than the conventional scaled dot-product attention used in standard Transformers. Because the query and key projections are learned independently, the resulting similarity matrix is directional, and the authors are careful to stress that these are internal neural similarities, not statistical correlations or causal relationships. Fourth, an input-adaptive dynamic feedforward module blends three learnable transformation kernels with weights generated from a global summary of the current representation, while a similarity-structure self-distillation loss encourages the forecasting pathway to preserve the cross-variable interaction structure learned by the reconstruction pathway.

The empirical design is deliberately restrictive, which is part of what makes the results notable. The dataset assembles 14 macroeconomic variables, from inflation and unemployment to capital stock and the human-capital index, for 14 countries including the United States, China, India, Brazil, and Nigeria, drawing on the IMF World Economic Outlook, the IMF International Financial Statistics, and the Penn World Table. The target is nominal GDP at current prices in U.S. dollars. Neural models are trained globally on pooled country-specific 10-year input windows, with country-balanced sampling so that economies with longer histories do not dominate optimization. No country identifier or embedding is supplied; the model must learn shared dynamics across very different economies. Training covers 1980 to 2015, validation runs from 2016 to 2019, and testing is reserved for 2020 through 2024, a period chosen, knowingly, to include some of the most turbulent macroeconomic conditions in decades.

Evaluation proceeds from a single forecast origin at the end of 2019. The 2020 forecast is generated directly, while forecasts for 2021 through 2024 are produced recursively, with each model’s own predictions appended to the rolling input window and no realized test-period outcomes fed back into the forecasting path. This fixed-origin recursive protocol yields 70 forecast errors per model, 14 countries times five horizons, and the authors treat those errors as the dependent panel they are. Rather than assuming independence, the primary statistical comparisons use standard errors clustered simultaneously by country and target year, with Holm adjustment across the seven original benchmarks. All pairwise squared-loss improvements favor the MCT, with Holm-adjusted p-values below 0.05, and two complementary sensitivity analyses, target-year-block inference and a 10,000-replicate country-cluster bootstrap, reach the same qualitative conclusion.

The benchmark sweep is unusually broad. Beyond the original statistical models, exponential smoothing, ARIMA, and vector autoregression, and recurrent neural networks including CNN, LSTM, bidirectional LSTM, and encoder–decoder LSTM, the expanded comparison adds five simple rules such as persistence and random walk with drift, five contemporary architectures including PatchTST, TimesNet, Crossformer, and TiDE, and six conventional machine-learning methods including CatBoost, XGBoost, and Random Forest. None reproduces the MCT’s point-forecast accuracy. Among simple rules, the best performer, an AR(1) model fitted to log GDP growth, records an RMSE of 0.536350, nearly four times the MCT figure. CatBoost leads the machine-learning family at 0.239600. The MCT also posts the lowest Theil’s U, at 0.159992, indicating the largest relative reduction in error compared with a no-change persistence reference.

Ablation experiments probe which components matter. Replacing the complete MCT block with a standard Transformer produces the largest deterioration, a 15.60 percent increase in RMSE, followed by swapping the normalized-similarity attention for conventional dot-product attention, which costs 12.42 percent. Removing the reconstruction pathway raises error by 8.68 percent, while dropping the dynamic feedforward module or the self-distillation objective costs 6.77 and 5.50 percent respectively. The authors caution that some of these comparisons also change parameter counts, so the percentages should not be read as isolated causal effect sizes. Robustness was further tested under additional random corruption at 10, 30, and 50 percent masking rates and under six structured synthetic missingness scenarios, including block, ragged-edge, variable-specific, country-specific, and MAR-like and MNAR-like mechanisms. The MCT retained the lowest RMSE in every condition, with its advantage over a standard Transformer widening to roughly 27 percent under the heaviest random corruption.

The reconstruction audit offers a rare moment of verified honesty in the imputation literature. Under a fixed 25 percent audit mask, the model recovered 1,591 deliberately withheld observations with an RMSE of 0.062474 and a pooled R-squared of 0.958807, results that are consistent across countries. But no accuracy claim is made for the 383 naturally unavailable entries, because no verified ground truth exists for them. Attention diagnostics reinforce the theme of careful interpretation. The learned similarity matrices are substantially asymmetric, with a mean maximum asymmetry of 0.6966, confirming they are not Pearson-type correlation matrices. Descriptive attention profiles are dispersed rather than concentrated: exports, GDP, employment, exchange rates, and capital receive the highest mean incoming attention, with stable rankings across five random seeds, but the authors explicitly reject any causal reading of these weights.

The study is equally candid about its limits. The evidence rests on a single forecast origin, five target years, 14 countries, and annual data, and the evaluation is retrospective rather than a genuine real-time-vintage experiment. The 2025 to 2030 outputs, which project, for example, U.S. nominal GDP rising from 28.720 to 32.270 trillion dollars and China’s from 18.995 to 26.020 trillion, are reported separately as post-evaluation model-based projections with empirically calibrated uncertainty bands, not as validated forecasts. The model is also deterministic, so no formal probabilistic prediction intervals are offered, and its accuracy comes at a computational price: more parameters, more memory, and longer training than simpler benchmarks, even though its estimated operation count per sample is about 60 percent lower than PatchTST’s. Still, the central message stands on its own terms. By jointly modeling what is observed, what is missing, and how fourteen macroeconomic variables move together, the MCT demonstrates that treating missingness as information rather than nuisance can measurably improve GDP forecasting in exactly the settings, short, incomplete, cross-country annual data, where forecasters need help most.

Subject of Research: Deep-learning-based GDP forecasting from incomplete multivariate macroeconomic time series

Article Title: A mask-aware multivariate correlation transformer for GDP forecasting with incomplete macroeconomic time series

Article References: Omidian, A., Keivan, Z., Omidian, A., & Mirzamohammadi, M. (2026). A mask-aware multivariate correlation transformer for GDP forecasting with incomplete macroeconomic time series. Discover Artificial Intelligence, 6(1), Article 1454. https://doi.org/10.1007/s44163-026-02538-9

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02538-9

Keywords: GDP forecasting, transformer, missing data, macroeconomic time series, attention mechanism, machine learning, multivariate forecasting, reconstruction, self-distillation, IMF data, Penn World Table, econometrics

The most common response to incomplete records is deceptively simple: throw them away. Analysts who delete every row containing a missing entry, an approach known as complete-case analysis, end up with a dataset that is smaller, potentially biased and statistically less powerful. If the people who failed to answer a questionnaire differ systematically from those who did, the surviving sample no longer represents the population it was drawn from. Even when the missingness is benign, discarding data inflates uncertainty and weakens statistical estimates, a cost that grows rapidly as the proportion of missing entries climbs.

Imputation offers an alternative. Instead of deleting incomplete cases, imputation methods replace each missing value with a plausible estimate inferred from the rest of the data. Done well, this preserves sample size, retains information that deletion would destroy and yields more reliable downstream estimates. The catch is that imputation is itself a prediction problem: to fill in a missing number, an algorithm must model how that number relates to everything else in the record. The quality of the fill-in therefore depends entirely on the quality of the predictive model doing the filling, which is precisely where the new study concentrates its effort.

Asif and Samart’s proposal, Layered Stacking, is built on the idea of stacking, an ensemble strategy in which multiple different models are trained to solve the same problem and their predictions are then combined by a higher-level model that learns when to trust each one. Rather than relying on a single imputation algorithm, Layered Stacking organizes several base learners into layers, letting each layer refine the estimates produced by the one beneath it. The architecture is designed specifically for continuous missing values, the numeric fields where small imputation errors can compound into distorted means, variances and regression coefficients.

To find out whether the added complexity pays off, the researchers ran comprehensive simulation studies built on four distinct data-generating processes. Each simulated dataset was generated under the missing-at-random mechanism, the widely assumed scenario in which the probability that an entry is missing depends on other observed variables but not on the missing value itself. The experiments spanned sample sizes of 500, 1,000, 5,000 and 10,000 observations, and missingness rates of 10, 20, 30 and 40 percent, a grid that captures everything from small studies with modest gaps to large datasets with substantial holes.

The benchmark competition was demanding. Layered Stacking was pitted against five imputation techniques that are widely used in practice: k-nearest neighbors, which averages the values of the most similar complete records; support vector regression, which fits flexible boundaries in a transformed feature space; decision trees, which partition the data through a cascade of yes-or-no questions; random forests, which average many decorrelated trees; and extreme gradient boosting, the sequential ensemble method that has dominated tabular machine-learning competitions for years. Each method was asked to impute the same missing entries under the same conditions, allowing a direct, like-for-like comparison.

Performance was scored with two complementary metrics. The mean squared imputation error, or MSIE, penalizes large mistakes heavily and therefore rewards methods that rarely produce wildly wrong estimates. The mean absolute percentage imputation error, or MAPIE, expresses the average error as a percentage of the true value, making results comparable across variables measured on different scales. The authors also reported the standard deviation of the errors across replications, along with 95 percent confidence intervals, so that differences between methods could be judged against the noise inherent in repeated simulation runs rather than a single lucky or unlucky draw.

The headline result is that Layered Stacking generally achieved lower MSIE than all five benchmark models, with its advantage becoming particularly pronounced on larger datasets. That pattern makes intuitive sense: ensemble methods that learn how to combine base learners need enough data to estimate the combination reliably, so the payoff of stacking grows as sample size increases. For analysts working with the large, messy datasets that characterize contemporary research, the finding suggests that the extra computational cost of a layered ensemble may be repaid in measurably more accurate imputations, which in turn feed into more trustworthy statistical models built on top of the completed data.

Simulations alone can flatter a method in ways real data never do, so the authors also tested Layered Stacking on two real-world datasets. These datasets bring irregularities that no data-generating process fully captures: skewed distributions, awkward correlations between variables and missingness patterns shaped by human behavior rather than mathematical rules. The real-data evaluation served as a sanity check on the simulation findings, and the study reports that the method’s performance carried over, reinforcing the case that the gains are not an artifact of carefully controlled synthetic conditions.

The broader significance of the work lies in what it says about the state of imputation practice. Methods like k-nearest neighbors and random forest imputation have become default choices in statistical software, and extreme gradient boosting has been increasingly adopted for its raw predictive power. Showing that a layered stacking ensemble can beat these established baselines, especially at scale, nudges the field toward treating imputation not as a preprocessing afterthought but as a modeling problem worthy of the same ensemble sophistication applied to prediction tasks. For anyone who has watched a regression analysis wobble because a fifth of the data contained holes, the message is appealing: the blanks in a dataset need not be a permanent tax on statistical accuracy, and the tools for filling them are getting better.

Subject of Research: Machine learning methods for imputing missing continuous data

Article Title: A layered stacking ensemble approach for missing data imputation

Article References: Asif, M., & Samart, K. (2026). A layered stacking ensemble approach for missing data imputation. PLOS One, 21(10), e0359843. https://doi.org/10.1371/journal.pone.0359843

Image Credits: AI Generated

DOI: 10.1371/journal.pone.0359843

Keywords: missing data, imputation, stacking ensemble, machine learning, PLOS One, simulation study, missing at random, random forest, gradient boosting, statistics, data science, mean squared error

Time series data, sequences of measurements ordered in time, underpin countless real-world applications, including clinical diagnostics, air quality monitoring, electricity load forecasting, anomaly detection, and emergency event prediction. Because missing values are both common and inevitable, the quality of any downstream analysis depends heavily on how well those gaps can be filled. Traditional approaches relied on statistical techniques such as ARIMA or simple substitution with zeros or means, while later machine learning models like BRITS and GRU-D brought recurrent neural networks to the task. More recently, Transformer-based architectures such as SAITS and ImputeFormer have set strong benchmarks. Yet all of these methods share a common limitation: they treat imputation as a deterministic estimation problem, producing a single best guess while ignoring the genuine uncertainty that surrounds any missing measurement.

Diffusion models, the same generative technology behind today’s most striking image generators, offer a natural alternative. They work by gradually adding Gaussian noise to data in a forward process and then learning to reverse that process, step by step, transforming pure noise into realistic samples. Because they model entire probability distributions rather than point estimates, diffusion models can capture the uncertainty of missing values and even provide confidence bounds around their predictions. Models like CSDI, SSSD, and SPD have already applied this idea to time series imputation with promising results. But according to the Tokyo team, these existing approaches suffer from two fundamental weaknesses that undermine the consistency of their output.

The first weakness is what the researchers call padding-induced distribution shift. To train a neural network on incomplete data, missing entries must be filled with placeholder values, typically zeros, so that the data can flow through the network’s mathematical machinery. The trouble is that diffusion models learn to approximate the distribution of whatever they are fed during training. When a large fraction of the training inputs consists of artificial zeros rather than genuine measurements, the model’s learned target distribution drifts away from the true data distribution. The effect is far from trivial: the team’s exploratory analysis showed that simply swapping zero-padding for one-padding degraded performance by 54.7 percent on one benchmark, while switching from sample-mean padding to linear interpolation improved results by 62.8 percent. In other words, the seemingly mundane choice of how to fill in blanks before training can swing accuracy dramatically.

The second weakness lies in how diffusion models are optimized. Standard diffusion training is entirely noise-driven: the network learns to predict the Gaussian noise that was added at each step, following the distributional dynamics of the reverse process. But this scheme provides no direct supervision from the actual observed values in the data. The result, the researchers show, is an unstable denoising trajectory, the sequence of intermediate states the model passes through as it walks from pure noise back to a clean sample. Visualizations of the popular CSDI model reveal oscillatory, erratic trajectories with large errors persisting deep into the reverse process, which ultimately produces imputations that can be inconsistent with the observed data surrounding the gaps.

ITCD tackles both problems head-on with a two-part design. The first component is a two-stage diffusion architecture. In the Refinement Stage, the model trains on padding-corrupted samples but uses its own probabilistic generations to replace the artificial padding with distribution-consistent values. These refined samples then serve as training inputs for the Core Stage, a second diffusion model that learns on data far closer to the true underlying distribution. Because the Core Stage is the only component used at inference time, the Refinement Stage adds no computational cost when the trained model is deployed. Ablation experiments confirm the value of this design: adding the Refinement Stage reduced error by 18.6 percent on the PeMS traffic dataset and 65.4 percent on the Electricity dataset in one comparison, and by 14.3 percent on Beijing Air quality data in another.

The second component, Implicit Trajectory-Constrained Optimization, reshapes how the denoising process itself is guided. It introduces two complementary constraints. The Intermediate Constraint supervises the predicted noise at every step of the reverse process, and crucially, it does so across all positions in the data, not just the missing ones. Existing methods typically focus only on the missing regions, discarding the rich distributional information contained in the observed values. By minimizing the mean absolute error between predicted and added noise everywhere, the model is pushed to align its entire denoising trajectory with the true data distribution. The Terminal Constraint then supervises the fully reconstructed sample at the final reverse step, combining an Observed Reconstruction Task that learns directly from visible measurements with a Masked Imputation Task that forces accurate prediction of artificially hidden values. Together, these constraints implicitly steer the model along a coherent path from noise to data, without requiring any extra learnable parameters.

The architecture also includes a clever structural safeguard during sampling. Before each denoising step, the entries corresponding to observed positions are explicitly refreshed with the true conditional values, preventing inaccurate intermediate estimates from contaminating the missing regions and reducing error accumulation along the trajectory. A small amount of stochastic noise is deliberately retained at each step to keep the trajectory from collapsing into a rigid deterministic path, which strengthens the robustness of the imposed constraints. The denoising network itself uses attention mechanisms in two directions simultaneously, across time and across features, with a learned gating mechanism that blends the two perspectives and incorporates information about the current diffusion step.

The experimental evidence is striking. Tested on eight datasets spanning air quality, traffic, electricity, and healthcare, including Beijing Air, Italy Air, PeMS, Melbourne Pedestrian, ETTh1, Electricity, and two PhysioNet clinical benchmarks, ITCD outperformed a broad field of baselines built on Transformers, recurrent networks, convolutional networks, and generative models. Under 10 percent point missingness, it cut error by 35.6 percent on Italy Air, 22.7 percent on PeMS, and 12.5 percent on ETTh1 relative to the best competitors. Its advantage grew as missingness increased: while CSDI’s error on Beijing Air ballooned from 0.144 to 0.640 as missing ratios rose from low to 90 percent, ITCD’s curve stayed remarkably flat. The method also proved robust across point, subsequence, and block missing patterns, and across repeated random sampling runs it showed substantially lower variance and better distributional alignment with ground truth than CSDI, as measured by the Wasserstein distance.

Perhaps most surprising for a diffusion model, ITCD is fast. Because the trajectory constraints stabilize the reverse process, the model needs only about ten diffusion steps rather than the fifty typically used by competitors, and it still matches or exceeds their accuracy. That translates into an 89.1 percent reduction in inference time on the ETT dataset and a 52.8-fold acceleration on PhysioNet2012, and it even beats the recurrent BRITS model on most benchmarks. As a probabilistic model, ITCD also provides meaningful uncertainty bounds around its predictions, giving practitioners a more honest picture of what remains unknown. The researchers, whose work was supported by the Japan Science and Technology Agency, have released their code publicly and plan to explore how imputation quality affects downstream tasks and to investigate explicit trajectory search strategies. For a field drowning in incomplete data, a method that is simultaneously more accurate, more stable, and dramatically faster is likely to attract attention well beyond the machine learning community.

Subject of Research: Diffusion-based deep learning for consistent imputation of missing values in multivariate time series

Article Title: ITCD: implicit trajectory-constrained diffusion for consistent time series imputation

Article References: Ma, S., Xu, Z., & Ichise, R. (2026). ITCD: implicit trajectory-constrained diffusion for consistent time series imputation. Applied Intelligence, 56(14), Article 397. https://doi.org/10.1007/s10489-026-07427-3

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07427-3

Keywords: time series imputation, diffusion models, missing data, deep learning, denoising trajectory, generative models, machine learning, uncertainty quantification, neural networks, Applied Intelligence, conditional diffusion, data science

The problem the team targets is known as spatio-temporal forecasting, a task that sits at the heart of applications ranging from traffic management and air quality prediction to energy grid optimization. In these settings, the value recorded at one location at one moment is statistically entangled with values recorded at neighboring locations and at earlier moments, and exploiting that entanglement is precisely what allows modern neural networks to anticipate the future. The difficulty is that most existing spatio-temporal models were built under a convenient but unrealistic assumption: that every sensor reports at every time step. When that assumption collapses, prediction quality can degrade dramatically, because the network’s learned representations of spatial and temporal context become contaminated by gaps it does not know how to interpret.

MGCRN addresses the gap problem through two tightly coupled components. The first is what the authors call spatio-temporal interactive perception attention, an attention mechanism that explicitly perceives which data points are missing while it models contextual information. Attention mechanisms, which rose to prominence in natural language processing, allow a model to weigh the relevance of different inputs dynamically rather than treating all inputs equally. In MGCRN, the attention module does double duty: it captures the dependencies linking observations across space and time, and it simultaneously accounts for the missing-value pattern itself, so that a gap in the record informs how the model interprets the surrounding data rather than silently corrupting it. This means the network can, in effect, reason about what it does not see, using the structure of the missingness as a signal in its own right.

The second pillar of the architecture is an adaptive graph convolutional network that leans into a property the authors describe as spatio-temporal heterogeneity. Real-world sensor networks are not uniform: the relationship between two traffic intersections, two air quality monitors, or two power substations changes depending on where they are, what time it is, and what conditions prevail. Many conventional graph-based forecasting methods fix the connections between nodes in advance, for example by wiring together sensors that are physically adjacent on a road network. MGCRN instead constructs dynamic graphs on the fly, using the inherent heterogeneity of the data to rebuild a reasonable spatio-temporal dependency structure as conditions evolve. The graph that connects the network’s nodes is therefore not a static scaffold but a living structure that adapts to the regime the system is currently in.

Graph convolution itself deserves a brief technical unpacking, because it is the mathematical engine that makes spatial reasoning possible in neural networks. Just as a standard convolutional layer slides a filter across an image to aggregate information from neighboring pixels, a graph convolutional layer aggregates feature information from a node’s neighbors, as defined by the edges of a graph. Stacking such layers allows information to propagate across multiple hops, so that a sensor can eventually be influenced by conditions at sensors several steps away in the network topology. When the graph is adaptive, the model learns which connections matter and how strongly, which is far more flexible than relying on geographic distance alone. Two monitors on opposite sides of a city may be more tightly correlated than two monitors on the same street if a highway or an industrial zone shapes their behavior, and an adaptive graph can discover such non-obvious relationships from the data.

The recurrent component of MGCRN supplies the temporal dimension. Recurrent networks process sequences step by step, maintaining an internal hidden state that serves as a compressed memory of everything observed so far. By embedding graph convolutions inside a recurrent framework, the architecture can update its spatial understanding at every time step while carrying forward the temporal context needed for multi-step prediction. The attention mechanism threads through this process, ensuring that at each step the model knows which of its inputs are trustworthy observations and which are missing entries that must be handled through learned inference rather than naive substitution. This integration of missing-awareness into the core forecasting loop, rather than treating imputation as a separate preprocessing stage, is one of the study’s central design choices.

The empirical case for the approach rests on experiments across four real-world datasets, a standard proving ground for spatio-temporal methods that typically include traffic flow benchmarks collected from urban sensor networks. According to the authors, MGCRN maintains high forecasting accuracy even under high missing-rate scenarios, conditions under which many competing methods falter. The model, they report, significantly outperforms existing spatio-temporal forecasting methods on all four datasets. That combination of results is notable because robustness to missing data has often been purchased at the cost of predictive sharpness: methods that impute aggressively can smooth away the very dynamics that make forecasting valuable, while methods that forecast sharply tend to assume clean inputs. MGCRN’s results suggest that perceiving missingness within the attention mechanism, rather than patching it beforehand, allows the model to have both.

The study situates itself within a rapidly growing literature. The authors’ reference list traces the evolution of the field from early diffusion convolutional recurrent networks for data-driven traffic forecasting, through attention-based spatial-temporal graph convolutional networks, to recent work on dynamic graph structures, transformers adapted for traffic prediction, and generative approaches such as conditional score-based diffusion models for time series imputation. Recent years have also seen dedicated frameworks for forecasting with missing values, including end-to-end multivariate models designed for variable missingness and pre-trained models for incomplete time series. MGCRN distinguishes itself in this crowded landscape by unifying three concerns that are usually handled separately: the perception of missing data, the modeling of spatio-temporal dependencies, and the construction of adaptive graphs that respect heterogeneity.

The practical implications extend well beyond the benchmark datasets. Intelligent transportation systems depend on continuous streams from loop detectors, cameras, and GPS probes, and any city engineer knows that these streams are riddled with outages. Environmental monitoring networks face similar realities, with instruments going offline for calibration or damage precisely during extreme events when forecasts matter most. Industrial processes monitored by distributed sensor arrays encounter sensor drift and failure as a matter of routine. A forecasting backbone that degrades gracefully as missingness increases could therefore make downstream systems, from traffic signal control to pollution alerts, more reliable in the messy conditions of deployment rather than the tidy conditions of the laboratory. The work was supported in part by the National Natural Science Foundation of China, the Zhejiang Provincial Natural Science Foundation, and industry research funds, reflecting the applied orientation of the collaboration between academia and Zhejiang Supcon Information.

Looking forward, the study contributes to a broader shift in how the machine learning community thinks about imperfect data. Rather than treating missing values as a nuisance to be eliminated before modeling begins, architectures like MGCRN fold the uncertainty of incomplete observation into the learning process itself, letting the network calibrate its confidence in each piece of evidence. As sensor networks proliferate through smart cities, autonomous logistics, and climate monitoring, the volume of data will keep growing, but so will the volume of gaps. Methods that can perceive what is absent, adapt their internal representation of how the world connects, and still deliver sharp forecasts may become the default foundation for the next generation of predictive systems. For now, MGCRN offers a concrete, empirically validated demonstration that acknowledging missingness is not a concession to reality but a source of predictive power.

Subject of Research: A missing-data-aware graph convolutional recurrent network for spatio-temporal forecasting

Article Title: MGCRN: Missing-aware Graph Convolutional Recurrent Network for spatio-temporal forecasting

Article References: Liu, W., Deng, Z., Zhao, Z., Shen, G., Xu, Z., & Kong, X. (2026). MGCRN: Missing-aware Graph Convolutional Recurrent Network for spatio-temporal forecasting. Cluster Computing, 29(13), Article 762. https://doi.org/10.1007/s10586-026-06604-w

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06604-w

Keywords: spatio-temporal forecasting, missing data, graph convolutional network, attention mechanism, recurrent neural network, traffic forecasting, adaptive graph, deep learning, time series, sensor networks, Cluster Computing, machine learning

The power of graph neural networks comes from a deceptively simple mechanism called message passing. In this paradigm, the representation of each node is updated iteratively by gathering and combining information from its neighbors. Through repeated rounds of exchange, every node acquires a low-dimensional embedding, a compact numerical vector that summarizes both its own attributes and the structure of its local neighborhood. This allows the model to reason about entities in context rather than in isolation, which is precisely why graph neural networks excel where ordinary neural networks struggle. Yet the mechanism has a well-known Achilles heel: it assumes the graph is complete. When some node features or structural relationships are missing, as is common in real-world data collected from noisy sensors, privacy-restricted databases, or partially observed systems, the quality of the messages degrades, and performance can collapse.

Incompleteness is not a rare edge case. In practical deployments, node features may be unrecorded for large portions of a network, and edges may be missing simply because the observation process was limited. To cope with this, researchers have developed graph completion learning approaches, which attempt to recover and reconstruct the missing node features or structural relationships before or during learning. Several methods have been proposed for feature completion alone and for structure completion alone, and these have proven effective within their narrow scopes. However, two serious limitations have persisted. First, existing graph completion models typically depend on a large number of labeled nodes to guide the reconstruction, which restricts their usefulness in scenarios where labeling is expensive and only a handful of nodes carry labels. Second, most methods address only one type of incompleteness at a time, leaving the far more challenging situation, in which features and structure are both incomplete simultaneously, largely unsolved.

A new study published in Volume 13, Issue 07 of the IEEE/CAA Journal of Automatica Sinica on August 3, 2026, tackles both problems at once. The work introduces a general graph completion learning framework called the Extremely Weak Supervision–Robust Graph Completion Network, abbreviated EWS-RGCN. According to author Chengxiang Lei of the Electric Power Research Institute of Guangdong Power Grid Co., Ltd., in China, the central insight is architectural: the model separates feature completion and structure completion into two independent channels. This separation, Lei explains, alleviates the mutual interference between missing node features and missing structural relationships that arises when a single message-passing process is asked to handle both kinds of gaps at once. The design directly confronts the way incomplete features and incomplete edges contaminate each other inside conventional graph neural networks.

The two-channel architecture works as follows. In the feature channel, missing node features are reconstructed using a trainable parameter matrix that is optimized jointly with the rest of the model, so the imputation of absent attributes is learned end to end rather than fixed in advance. In the structure channel, a personalized PageRank algorithm is introduced to reconstruct the missing structure based on the structural relationships that do exist. Personalized PageRank, a well-established technique for measuring node proximity in a graph, allows the model to infer plausible connections from the observed topology. As Lei notes, this bifurcation strategy ensures that the channels do not interfere at the initial stage and that each channel focuses on extracting pertinent information independently. Only after each channel has produced its own reconstruction does the model bring the two views of the graph together.

The second pillar of the framework is its answer to the label scarcity problem. Instead of relying on abundant labeled nodes, EWS-RGCN employs a multi-level contrastive graph mask autoencoder to extract effective supervision from the data itself. Autoencoders learn by encoding an input into a compressed representation and then decoding it back, and the masking variant deliberately hides parts of the input so the network must infer them, forcing it to internalize the underlying structure of the data. Contrastive learning adds another layer of self-supervision by teaching the model to distinguish between representations that should be similar and those that should differ. By combining these ideas at multiple levels, the framework reduces its dependence on labeled nodes dramatically, enabling learning under what the authors call extremely weak supervision, a regime in which conventional graph completion models struggle.

The encoding and decoding details reveal how the two channels operate in parallel. For the feature channel, encoding proceeds in two steps: structure-guided feature diffusion, which is aimed at effective message propagation during the message-passing process, followed by feature transformation through a multi-layer perceptron. The diffusion step lets feature information flow along the observed graph, while the transformation step reshapes it into a useful representation. For the structure channel, positional encoding based on graph convolutional networks is employed to generate node embeddings that capture where each node sits within the topology. In both channels, the decoding process involves masking the embeddings and feeding them into a multi-layer perceptron decoder, so the model repeatedly practices reconstructing hidden information. The node embeddings from the two channels are then fused using attention mechanisms, which weight the contribution of each channel adaptively to support the final classification task.

One further component ties the whole system together: an inter-channel information cooperation module that enhances mutual learning between the feature and structure completion channels. This module allows the two reconstructions to inform each other without the destructive interference that plagues monolithic designs, striking a balance between independence during early reconstruction and cooperation during final representation learning. The result is a model that can handle both missing features and missing structures simultaneously, even when the number of labeled nodes is extremely limited, a combination of robustness and label efficiency that previous graph completion approaches have not achieved together.

The empirical evaluation was deliberately demanding. The researchers tested EWS-RGCN on six benchmark datasets under a range of feature and structure missing rates and with limited labeled nodes, simulating the harsh conditions that real deployments often face. Across all of these scenarios, the approach outperformed existing graph completion learning methods, demonstrating that the dual-channel design and the self-supervised autoencoder translate from theory into measurable gains. As Lei remarks, the approach can handle both missing features and structures simultaneously, even with extremely limited labeled nodes, which is precisely the combination of difficulties that has limited earlier methods in practice.

The significance of this work extends beyond a single benchmark table. By combining graph completion with extremely weak supervision and separate feature and structure processing, EWS-RGCN offers a potential path toward making graph neural networks robust in the messy, partially observed settings where they are actually deployed, from power grid monitoring to social and biological networks. The study, titled Training Robust Graph Completion Networks with Extremely Weak Supervision on Graphs with Incomplete Features and Structure, was supported in part by the National Natural Science Foundation of China under grant number 62575116. As graph-based machine learning continues to spread into safety-critical and data-scarce domains, frameworks like this one suggest that the missing pieces of a graph no longer have to mean missing performance.

Subject of Research: Robust graph completion learning for graph neural networks on graphs with incomplete features and structure under extremely weak supervision

Article Title: An innovative, robust approach for reconstructing graphs with incomplete information

Article References: An innovative, robust approach for reconstructing graphs with incomplete information. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: graph neural networks, graph completion learning, missing data, weak supervision, contrastive learning, autoencoder, personalized PageRank, message passing, node classification, self-supervised learning, IEEE/CAA Journal of Automatica Sinica, machine learning

Now a team of researchers from Meenakshi Medical College Hospital and Research Institute in Tamil Nadu, India, has published a formal commentary in the same journal challenging that interpretation. In a Matters Arising contribution, R. Saravanan, Paramanantham Madhavan, R. Sivayogana, and K. Anusha do not dispute the value of the underlying registry analysis, which they describe as an important framework for evaluating rare cancers using population-level data. Instead, they raise two methodological concerns that, if correct, could substantially alter how the German findings should be read. The first concerns whether the apparent rise in multivisceral resection reflects a real change in surgical practice or simply an improvement in how operations were documented and coded over time. The second concerns the handling of missing data on tumor grade and metastatic status, variables that were absent in a large fraction of cases yet still used to stratify survival analyses. Both concerns strike at the heart of a growing movement in oncology: the use of registry data to draw clinical conclusions about diseases too rare for randomized trials.

The argument about coding is technically subtle but conceptually compelling. The German analysis of organ resections relied on a subset of only 1,318 patients with codeable resection procedures, a considerably smaller group than the full surgical cohort, and it depended entirely on whether the Operationen- und Prozedurenschlüssel, the German procedure coding system, captured a code for every organ removed during each operation. The commentary points out that the original paper itself acknowledged a parallel phenomenon: the rise in annual case numbers over the same period was attributed to improving case ascertainment and documentation completeness following the 2014 legal framework for cancer registration, not to a true increase in disease incidence. If overall case documentation improved across two decades, the Indian authors argue, it is plausible and untested that procedure-level documentation improved as well. Under that scenario, earlier multivisceral resections may have been underdocumented, with only the single most prominent procedure code recorded, while later cases benefited from more complete coding that captured every organ removed in the same operation. Part or all of the observed temporal trend would then reflect better paperwork rather than bolder surgery.

The distinction is far more than methodological tidiness. If the trend is substantially a documentation artifact, the commentary notes, then the implicit message to clinicians and registry stewards, that German surgical practice has progressively converged on international multivisceral resection guidelines for liposarcoma, would be overstated. The more accurate takeaway would concern the maturation of registry coding rather than the maturation of surgical decision-making. The authors propose a concrete diagnostic test: reporting the proportion of surgical cases with at least one codeable resection procedure by diagnosis year, alongside the existing organ-count trend, would allow readers to judge whether the apparent rise in multivisceral resection tracks improving procedure documentation or represents an independent change in clinical practice. If the two curves move together, the documentation explanation gains weight; if they diverge, the practice-change interpretation becomes more credible. Such a check costs nothing beyond additional analysis of data already in hand, which is precisely why the commentary frames its absence as a missed opportunity.

The second major concern involves missing data in the survival analyses. According to the commentary, grading information was missing in twenty-four percent of cases, and distant metastasis status was missing in fifty-five percent, yet both variables were used to stratify the survival analyses and the primary treatment description without any sensitivity analysis addressing whether the missingness was informative. This matters because of a well-known statistical hazard: complete-case analysis, in which patients lacking a given variable are simply excluded, produces unbiased results only if the missing data occur randomly. If, as the original paper’s own discussion suggests, earlier-era cases suffered from less complete documentation, then those cases may be disproportionately represented among patients with missing grading or staging information. The Kaplan-Meier survival curves and treatment-pattern tables restricted to complete cases would then systematically overrepresent more recent, better-documented tumors, which may differ from the excluded cases in surgical approach, tumor biology ascertainment, or case mix. The resulting curves could paint a rosier or otherwise distorted picture of outcomes across the full registry cohort.

The commentary applies this concern specifically to the Kaplan-Meier estimates stratified by grade, which were calculated only among the roughly three-quarters of cases with recorded grading. No comparison of baseline characteristics between graded and ungraded cases was provided to establish that the two groups were otherwise similar. The authors suggest a remedy that mirrors their proposal on procedure coding: reporting whether missingness in grading and metastasis status varies systematically by diagnosis year, and providing a basic comparison of age, sex, and histology distribution between complete and incomplete cases. Such comparisons would allow readers to judge whether the survival and treatment-pattern findings generalize to the full registry cohort or are specific to the more thoroughly documented subset. The commentary also notes an internal inconsistency in the original paper’s logic: the study elsewhere treats improving documentation completeness as a plausible explanation for other temporal patterns in the dataset, yet does not extend that same reasoning to the grading and staging variables underlying the survival analysis.

What makes this exchange notable is its constructive tone and its broader relevance. The Indian authors explicitly state that neither concern displaces the substantial contribution of the German study, which established a reproducible, histology- and location-specific framework for extracting a rare tumor cohort from national registry data. They call that framework a genuinely useful methodological template for future rare-cancer registry research well beyond retroperitoneal sarcoma. The stakes they identify are correspondingly high: distinguishing genuine practice change from improved documentation completeness, and characterizing whether missing data correlate with diagnosis era, would allow clinicians and health-policy readers to rely on reported trends as evidence of evolving surgical practice with greater confidence, rather than treating documentation artifacts as clinical signals. In an era when health systems increasingly mine administrative and registry data to guide cancer care, the commentary is a reminder that the quality of the data pipeline can be as consequential as the quality of the medicine it records.

The technical issues at play are familiar to epidemiologists but deserve wider understanding. Procedure coding systems like the German OPS are designed for billing and administration, not research, and their granularity depends on local coding habits, staffing, and evolving regulations. Studies of complex surgical specimens in other fields have documented substantial variability in how thoroughly multi-organ procedures are captured in coded records, and the commentary cites work on procedural terminology optimization in genitourinary surgery as an illustration. Similarly, the problem of informative missingness in registry-based survival analysis has prompted a growing literature on imputation techniques for Kaplan-Meier estimation, which the commentary references. Neither problem is unique to the German study; both are endemic to registry research. The commentary’s contribution is to show how these general hazards map onto the specific claims of a high-profile national analysis, and to propose analyses that would resolve the ambiguity without requiring new data collection.

For patients with retroperitoneal sarcoma, the practical implications are indirect but real. Multivisceral resection is the single most important determinant of long-term survival in this disease, and international guidelines recommend aggressive compartmental surgery for liposarcoma whenever feasible. If German practice has genuinely converged on those guidelines over two decades, that is encouraging news for the roughly one in a hundred thousand people diagnosed each year. If instead the trend partly reflects coding maturation, the appropriate response is not complacency about surgery but investment in registry quality, so that future analyses can distinguish the two. The commentary, published open access in October 2026, ultimately asks for methodological consistency: the same skepticism about documentation quality that the original authors applied to case counts should be applied to procedure codes, tumor grades, and staging data. Whether the German group responds with the requested sensitivity analyses will determine how confidently the oncology community can cite this study as evidence that surgical practice, and not merely surgical record-keeping, has changed.

Subject of Research: Methodological evaluation of German cancer registry data on retroperitoneal sarcoma treatment trends

Article Title: Comment on “Treatment of retroperitoneal sarcoma in Germany between 2000 and 2022: a retrospective analysis from the German Cancer Registry Group”

Article References: Saravanan, R., Madhavan, P., Sivayogana, R., & Anusha, K. (2026). Comment on “Treatment of retroperitoneal sarcoma in Germany between 2000 and 2022: a retrospective analysis from the German Cancer Registry Group”. Journal of Cancer Research and Clinical Oncology, 152(10), Article 189. https://doi.org/10.1007/s00432-026-06591-w

Image Credits: AI Generated

DOI: 10.1007/s00432-026-06591-w

Keywords: retroperitoneal sarcoma, cancer registry, multivisceral resection, registry completeness, missing data, survival analysis, surgical oncology, liposarcoma, rare cancers, clinical epidemiology, procedure coding, Germany