Time-series foundation models fall short of specialized models in digital twin predictive maintenance benchmark

Digital twins promise a single virtual replica of a physical asset that can answer an open, evolving stream of questions throughout its lifecycle: Is the engine behaving normally? What is the specific failure mode? How long until critical failure? In practice, however, most deployed architectures cannot deliver on that promise. They rely on a dedicated, single-purpose analytical model for every distinct query, task, physical scale, and operating context, and the engineering overhead of training and maintaining that vast model library threatens the scalability of large-scale deployments. A new study from Carnegie Mellon University, published in Frontiers in Aerospace Engineering, puts a potential solution—time-series foundation models, or TSFMs—to one of the most rigorous tests it has yet faced, and the verdict is nuanced: these general-purpose backbones can generalize across tasks, but they do not yet match specialized models trained directly on the target data.

The research team, led by Sizhe Ma with Katherine A. Flanigan and Mario Bergés, benchmarked three architecturally distinct multi-task TSFMs—MOMENT, UniTS, and FFTS—across four downstream tasks within a predictive maintenance context: imputation, anomaly detection, classification, and regression. The testbed was the Commercial Modular Aero-Propulsion System Simulation, or C-MAPSS, a NASA-developed dataset that simulates a large commercial turbofan engine of the 90,000-pound thrust class across varying altitudes, speeds, and throttle settings. By systematically injecting deterioration into the engine’s five rotating components—the fan, low-pressure compressor, high-pressure compressor, high-pressure turbine, and low-pressure turbine—the simulation produces complete run-to-failure trajectories, something real-world operational datasets typically cannot provide because they are truncated to prevent catastrophic failure. The model outputs 21 noisy sensor streams measuring gas-path temperatures, pressures, and shaft speeds, of which 14 sensors responsive to degradation were used for the evaluation.

The choice of predictive maintenance as the case study is deliberate. Under the Open System Architecture for Condition-Based Maintenance standard, predictive maintenance is a hierarchical process whose modules map naturally onto the four benchmarked tasks. Data manipulation requires imputation to recover missing sensor information, simulating sensor failure or transmission loss common in industrial environments. State detection is computationally framed as anomaly detection, identifying deviations from normal behavior. Health assessment diagnoses the specific root cause of degradation through classification. Prognostics assessment projects the future trajectory of the asset to estimate remaining useful life, or RUL, via regression. Because these requirements for interpreting complex, multivariate sensor streams mirror the fundamental data challenges found across digital twin applications, performance on this benchmark serves as a proxy for the analytical layer that any mature digital twin would host.

The experimental design is a controlled test of generalization. All three models were evaluated under identical data preprocessing, splitting, and evaluation frameworks, with ten-fold cross-validation partitioned at the level of the engine unit rather than the window, so that no window spans a group boundary. The test set was held out entirely and never used for training, hyperparameter selection, or epoch selection. Three adaptation regimes were compared: zero-shot inference, in which the pre-trained backbone is applied with no parameter update; lightweight adaptation, which freezes the backbone and updates only a small set of parameters such as task-specific tokens or a linear head; and full fine-tuning, which updates all parameters. Critically, no model was initialized with random weights—every experiment started from a pre-trained backbone derived from official repositories or a reproduction of the original pre-training procedure, as in the case of FFTS, which releases no checkpoint and was pre-trained from scratch using its published federated strategy over 18 datasets from the Monash repository.

The headline finding is a trade-off between generalizability and precision. Although the TSFMs generalized to every evaluated task, they rarely surpassed specialized, domain-specific baselines trained only on C-MAPSS. In imputation, the best TSFM configuration—UniTS under full fine-tuning—reached a mean squared error of 0.123, against 0.105 for the specialized SAITS baseline. In classification, the same configuration reached an accuracy of 0.978, against 0.988 for the DA-Net baseline. In regression, UniTS full fine-tuning achieved an RMSE of 14.21, against 12.90 for the specialized Bi-LSTM Attention baseline. Anomaly detection was the closest contest of the four: the best TSFM configuration—MOMENT in zero-shot mode—reached an adjusted F1 of 0.872 and an AUC of 0.983, against 0.882 and 0.987 for the Anomaly Transformer baseline, meaning the frozen backbone fell within 0.004 AUC of a model trained directly on the target data without any parameter update.

One of the most striking results concerns the task absent from all three models’ pre-training objectives. TSFM pre-training in the time-series domain has centered overwhelmingly on forecasting, driven by the abundance of unlabeled temporal data available for self-supervision. RUL estimation, by contrast, is a regression task that maps an input window to an exogenous scalar target—the number of cycles to failure. The benchmark reveals that the backbone combining channel-mixing with explicit task tokens, UniTS, adapts best to this out-of-pre-training objective, and most visibly when adaptation is restricted to a small set of parameters. From a window width of 64 onward, UniTS prompt learning reached the best lightweight result of the three models on both regression metrics; at a width of 128 it achieved an RMSE of 14.93, against 17.61 for MOMENT linear probing and 19.02 for FFTS linear probing. Remarkably, this frozen configuration also surpassed the fully fine-tuned MOMENT and FFTS and the specialized LSTM baseline, all while leaving the pre-trained weights untouched.

The study also uncovered a counterintuitive pattern in anomaly detection. For all three models, fine-tuning outperformed zero-shot inference at narrow window widths of 16 and 32, but fell below it from a width of 64 onward. The authors offer a plausible explanation rooted in the unsupervised training strategy: because the reconstruction objective is fitted on the entire run-to-failure training data, a longer window exposes the model to more degraded behavior, and a backbone with sufficient capacity can learn to reconstruct that degraded behavior as faithfully as the healthy behavior it is meant to characterize. The share of anomalous training data rises from roughly 9 percent at the narrowest width to approximately 45 percent at the widest, so both the proportion of contaminated samples and the degraded content of each sample increase with window width. The practical implication reverses the pattern observed in the other three tasks: in reconstruction-based monitoring, zero-shot or lightweight use of a TSFM may be preferable to full adaptation.

Window width itself emerged as a decisive constraint. All three models degraded at the shortest tested window width of 16, and the degradation was largest for UniTS in every task. A candidate explanation lies in patch-based tokenization: MOMENT uses a patch size of 8 and can resolve a complete patch at width 16, whereas UniTS and FFTS employ a patch size of 16 and are fed input sequences shorter than a single patch—a scenario unseen during their pre-training. The evidence is mixed, however, since FFTS shares UniTS’s patch size yet degrades least of the three, reaching the best zero-shot adjusted F1 in anomaly detection and the best lightweight RMSE in regression at that width. The authors therefore report patch size as a plausible contributor rather than an established cause, and note that isolating it would require an ablation varying patch size within a single backbone. The practical lesson stands regardless: for monitoring high-frequency assets at short window widths, the alignment between a model’s tokenization and the inference window length should be considered per model rather than assumed.

In the zero-shot regime, no single backbone led across tasks, and the results were only partially consistent with an effect of pre-training data scale. MOMENT, pre-trained on the largest corpus—the 1.23-billion-time-point Time Series Pile—reached the best zero-shot imputation error at every window width and the best zero-shot AUC in anomaly detection at two widths. Yet at other widths, FFTS, whose corpus is less than half the size, reached the higher zero-shot adjusted F1. Because the three models differ simultaneously in architecture, parameter count, pre-training corpus and objective, and patch size, the authors are careful to frame these as patterns observed under one shared benchmarking framework rather than as isolated causes. An appendix examining the pre-training corpora confirms that all three reach only the modality level of familiarity with C-MAPSS: temperatures, pressures, and rate-like quantities occur in all three corpora, but none contains gas-path instrumentation of any engine, and C-MAPSS itself is absent from all three pre-training datasets.

The authors are equally careful about scope: the evaluation was conducted on raw sensor series alone, without the semantic or physics-based components of a mature digital twin, so the findings establish an empirical baseline for the analytical layer such a system would host rather than an evaluation of digital twin integration itself. Looking forward, they argue that closing the gap to specialized models may require moving beyond purely data-driven pre-training toward knowledge-guided TSFMs that are supplied with the physical constraints, system topology, and failure logic a digital twin already holds, rather than expecting the backbone to recover them from raw sensor data. Whether that strategy closes the gap remains an open question—but this benchmark provides the raw-data reference point against which it can finally be asked.

Subject of Research: Artificial Intelligence

Article Title: Benchmarking time-series foundation models for multitask predictive maintenance in digital twins

Article References: Ma, S., Flanigan, K. A., & Bergés, M. (2026). Benchmarking time-series foundation models for multitask predictive maintenance in digital twins. Frontiers in Aerospace Engineering, 5, Article 1937868. https://doi.org/10.3389/fpace.2026.1937868

Image Credits: AI Generated

DOI: 10.3389/fpace.2026.1937868

Keywords: digital twins, time-series foundation models, predictive maintenance, machine learning, aerospace engineering, remaining useful life, C-MAPSS, benchmarking, time-series, foundation, models, multitask