Pediatric Intensive Care AI Models Face Limits from Missing Physiological Data

Machine learning models trained on the Pediatric Intensive Care database can achieve moderate-to-high statistical discrimination for hospital mortality, yet their clinical utility is severely hampered by low sensitivity and missing physiological data. Researchers Maria A. Bravo, Judy Ximena Ramos Garzon, and Carlos A. Fajardo evaluated the open-access pediatric dataset to assess its predictive integrity. Although the top-performing model achieved an area under the receiver operating characteristic curve of 0.8521, its sensitivity reached only 42 percent at the selected operating threshold. This critical diagnostic shortfall indicates that more than half of the non-survivors would be incorrectly classified as low risk. The investigation demonstrates that while the database contains informative biochemical and laboratory markers, its absence of foundational bedside physiological and neurological variables places hard boundaries on the real-world deployment of artificial intelligence in pediatric critical care risk stratification.

Mortality within pediatric intensive care units represents a critical public health priority, as facilities admit young patients suffering from complex cardiovascular, renal, and respiratory failures alongside life-threatening infections such as sepsis. Traditionally, clinical teams have relied on established prognostic tools, including the Pediatric Risk of Mortality, the Pediatric Index of Mortality, and the Pediatric Logistic Organ Dysfunction score, to guide triage and optimize critical care resources. While these conventional scoring systems remain clinically meaningful and broadly deployed, researchers have noted that their practical discrimination may suffer from population-specific bias, calibration drift, and an inherent inability to capture subtle, non-linear interactions across diverse clinical indicators. In adult intensive care, large-scale open-access resources such as MIMIC-III and the eICU Collaborative Research Database have accelerated transparent, reproducible artificial intelligence research. In pediatric critical care, however, comparable public repositories remain exceptionally rare, leaving the PIC database as the sole open-access resource focused exclusively on critically ill children.

To systematically assess whether the PIC database can support dependable predictive modeling, the investigative team analyzed records originating from the Children’s Hospital of Zhejiang University School of Medicine. The raw repository encompasses 13,499 hospital admissions corresponding to 12,881 unique pediatric patients aged zero to eighteen years who were admitted between 2010 and 2018, capturing more than 1,298 clinical variables across 16 interlinked tables. In accordance with ethical standards and the Health Insurance Portability and Accountability Act, all direct patient identifiers were removed prior to controlled research distribution under data use agreements. The authors extracted core demographic, admission, and stay details from the primary patients, admissions, and intensive care stay tables. To establish a rigorous study cohort, the researchers restricted inclusion to patients aged one month to seventeen years, excluding neonates due to distinct physiological characteristics, and retained only the first hospital admission per individual to avoid correlation bias.

The analytical design further confined data extraction to the initial 24 hours following admission to the intensive care unit, aligning with standard severity scoring protocols like PRISM III to evaluate early risk while preventing information leakage from subsequent clinical interventions or disease progression. After applying these strict inclusion criteria, the final study cohort comprised 4,272 pediatric patients, among whom 4,056 survived and 216 died during their hospital stay, establishing a baseline in-hospital mortality rate of 5.06 percent. Candidate variables were screened through literature reviews and clinical expert consultation, evaluating measurement unit consistency, clinical definitions, and biological plausibility. To protect model integrity from excessive data imputation, variables exhibiting more than 60 percent missing values across the 24-hour window were excluded entirely from the predictor set. The remaining missing entries were imputed using the Miceforest algorithm, an approach based on LightGBM designed to preserve complex intervariable correlations.

Through this systematic preprocessing pipeline, the authors identified 49 clinically validated predictors, consisting of 45 laboratory measurements, three vital signs, and one demographic variable. Summary metrics were subsequently engineered to reflect longitudinal bedside recordings: vital signs were represented through their minimum, mean, and maximum values; laboratory assays were summarized using observed minimum and maximum concentrations; and the demographic variable provided a single baseline entry. This engineering process expanded the analytical feature space to 100 derived variables for predictive modeling and interpretability analysis. Before undertaking statistical modeling, the distributional properties of all variables were examined using the Shapiro-Wilk test. Because none of the clinical features conformed to a normal Gaussian distribution, the investigators utilized non-parametric comparative methods, employing the Mann-Whitney U test for continuous parameters and the Chi-square test for categorical attributes to map foundational baseline differences between survivors and non-survivors.

To quantify the independent epidemiological association between individual clinical predictors and pediatric mortality, the investigators performed univariate odds ratio analyses using logistic regression models. In these individual evaluations, each clinical variable served as a solitary predictor to estimate how incremental changes altered the odds of patient death, reporting point estimates alongside 95 percent confidence intervals. The strongest statistical associations were concentrated almost exclusively among laboratory measurements. Specifically, maximum phosphate displayed the highest univariate association with an odds ratio of 2.681 and a 95 percent confidence interval spanning 2.099 to 3.424, followed closely by minimum phosphate at an odds ratio of 2.474 with a confidence interval between 1.938 and 3.159. Minimum international normalized ratio and prothrombin time also demonstrated marked predictive association, exhibiting an odds ratio of 1.989 with a confidence interval from 1.492 to 2.652, while mean respiratory rate was the sole vital sign among the top-ranked associations.

The investigation evaluated nine distinct machine learning architectures representing linear, ensemble, boosting, kernel, and neural network paradigms: Logistic Regression, Support Vector Classifier, Decision Trees, Random Forest, Gradient Boosting Machine, Multilayer Perceptron, XGBoost, LightGBM, and CatBoost. To ensure unbiased evaluation, the cohort was divided using stratified random sampling based on mortality outcome into a training partition of 3,417 patients and an independent hold-out test partition of 855 patients, fixing random seeds to maintain exact reproducibility. Model development and validation relied on five-fold stratified cross-validation within the training cohort, utilizing the area under the receiver operating characteristic curve as the primary evaluation metric alongside sensitivity, specificity, and F1-score. CatBoost achieved the highest discrimination during cross-validation and was selected for Bayesian hyperparameter optimization and decision-threshold tuning aimed at maximizing the F1-score before final evaluation on the untouched test partition.

Prior to hyperparameter optimization on the hold-out test partition, CatBoost demonstrated an area under the receiver operating characteristic curve of 0.8426, with a 95 percent confidence interval between 0.8317 and 0.8536, outperforming alternative architectures while Random Forest, Gradient Boosting, XGBoost, and LightGBM also exhibited robust discrimination. Following Bayesian hyperparameter tuning, CatBoost achieved a test-set area under the curve of 0.8521 with a 95 percent confidence interval of 0.8414 to 0.8627. Despite this moderate-to-high discriminatory power, adjusting the classification decision threshold to optimize the F1-score resulted in a test-set sensitivity of merely 42 percent. This modest sensitivity underscores a pronounced limitation in detecting non-survivors within practical clinical environments, where false-negative predictions carry severe operational consequences by mistakenly categorizing deteriorating pediatric patients as low-risk individuals who might consequently receive delayed therapeutic intervention.

To examine how various algorithms utilized the 100 derived variables, the researchers implemented model interpretability techniques using Shapley Additive Explanations across seven computationally stable architectures, including Decision Trees, Random Forest, Gradient Boosting, XGBoost, LightGBM, Logistic Regression, and the optimized CatBoost model. TreeSHAP was applied to tree ensembles to generate exact explanations, while LinearSHAP evaluated logistic regression under linear assumptions. The authors avoided direct comparisons of absolute numerical SHAP values across distinct algorithms, implementing instead a consensus-based majority-voting framework tracking the top twenty variables ranked by mean absolute SHAP values. This consensus analysis demonstrated that minimum hemoglobin and maximum lactate appeared within the top twenty features across all seven models. Minimum gamma-glutamyl transferase, minimum lactate, and maximum partial pressure of carbon dioxide were selected by six models, whereas minimum indirect bilirubin, maximum white blood cell count, and mean respiratory rate were prioritized by five models.

Within the optimized CatBoost model, maximum partial pressure of carbon dioxide emerged as the most influential feature by mean absolute SHAP value, followed by minimum lactate, maximum lactate, minimum creatine kinase, and maximum creatine kinase. Detailed beeswarm plots revealed that elevated carbon dioxide and lactate values drove higher predicted mortality risks, whereas decreased hemoglobin and pH measurements corresponded to increased risk. However, comparing the top twenty univariate odds ratio predictors with CatBoost SHAP rankings revealed only partial convergence, with merely five shared variables: minimum lactate, maximum lactate, mean respiratory rate, maximum hematocrit, and minimum urea. Highly ranked univariate predictors such as phosphate, potassium, and coagulation indicators did not appear within CatBoost’s top twenty SHAP features, while maximum partial pressure of carbon dioxide, creatine kinase, and pH were elevated only in multivariable model interpretations, highlighting differences between marginal statistical associations and complex multivariable non-linear interactions.

When benchmarked against established clinical scoring systems, the discrimination achieved by the CatBoost classifier aligns closely with pooled literature values for PRISM, PIM, and PELOD, which report summary area under the curve values between 0.82 and 0.84, though those tools typically maintain higher sensitivities and specificities near 0.75 to 0.78. However, direct equivalence remains constrained because the PIC database completely lacks fundamental bedside variables required by conventional scoring systems, including systemic blood pressure, Glasgow Coma Scale assessments, pupillary reflexes, capillary refill time, and hourly urinary output. The authors concluded that while the PIC repository provides meaningful biochemical prognostic signals, its restricted clinical scope, single-center provenance, and missing physiological parameters fundamentally limit the reliability of machine learning for actionable pediatric mortality prediction, emphasizing an urgent necessity for richer, multicenter critical care databases.

Subject of Research: Machine learning mortality prediction and data completeness evaluation using the Pediatric Intensive Care database

Article Title: Evaluating the AI-predictive potential of the PIC database for pediatric ICU mortality

Article References: Bravo, M. A., Garzon, J. X. R., & Fajardo, C. A. (2026). Evaluating the AI-predictive potential of the PIC database for pediatric ICU mortality. Pediatric Research. https://doi.org/10.1038/s41390-026-05451-0

Image Credits: AI Generated

DOI: 10.1038/s41390-026-05451-0

Keywords: pediatric intensive care, machine learning, mortality prediction, CatBoost, PIC database, clinical data quality, SHAP analysis, critical care, Evaluating, AI-predictive, potential, database