The clinical question at the heart of the study is one of the most consequential decision points in pediatric emergency medicine. Surviving Sepsis Campaign guidelines call for prompt fluid resuscitation in children with septic shock, typically in measured boluses of isotonic crystalloid, but a substantial minority of patients do not respond and progress to fluid-refractory shock, where timely initiation of vasoactive medications becomes critical. Evidence from adult and pediatric studies suggests that delayed vasopressor initiation is associated with worse outcomes, yet clinicians currently lack objective tools to predict, at the bedside, which children will fail fluid resuscitation. Guidelines offer limited guidance on this specific transition, leaving the decision to continue boluses or start vasoactives dependent largely on clinical judgment, which can vary widely from one practitioner, shift, or institution to the next.

The stakes of that judgment cut in both directions. Continuing to administer fluid boluses to a child who will never respond risks positive fluid balance, which has been linked in pediatric studies to higher mortality and prolonged mechanical ventilation, as well as fluid overload and its complications, including respiratory compromise from pulmonary edema and tissue edema that can impair organ function. Escalating too early to vasoactive medications, conversely, exposes children who might have responded to fluids alone to invasive monitoring and potent drugs, each of which carries its own risks of complications, dosing errors, and the need for specialized critical care resources that may not be immediately available in every emergency department. The new study was designed to address exactly this fork in the road: whether, after a child has received two fluid boluses, routinely available clinical data can predict who will subsequently require vasopressor support.

To build the tool, the investigators conducted a retrospective analysis of electronic health record data from five pediatric emergency departments: Johns Hopkins Children’s Center, Johns Hopkins All Children’s Hospital, Cincinnati Children’s Hospital Medical Center, Children’s National Hospital, and Children’s Healthcare of Atlanta. This multicenter design is an important strength, because models trained on data from a single institution risk learning site-specific patterns of care that do not generalize. They included children aged 3 months to 17 years who were screened for sepsis, received at least two fluid boluses, and were vasopressor-naïve at the time of the second bolus—a restriction that ensures the model is predicting future escalation rather than simply detecting medication already underway. To ensure a physiologically meaningful analytic cohort, they required that patients have abnormal age-adjusted vital signs before the first bolus and documented vital signs after the second bolus, so that the model’s inputs reflect a genuine resuscitation trajectory rather than fragmentary documentation. Of 645 eligible patients, 341 met these analytic criteria, and 88 of them—25.8 percent—went on to receive vasoactive medications, a rate consistent with the roughly one in four proportion of fluid-refractory cases reported in earlier pediatric sepsis literature.

The modeling approach was deliberately systematic. The team began with 41 candidate variables and used recursive feature elimination to narrow the field to eight predictors, favoring a parsimonious model built from data that clinicians already capture in the course of routine care. Parsimony matters for more than elegance: models with fewer inputs are easier to compute reliably, less vulnerable to missing data, and simpler to audit for the clinicians who must trust their output. To predict subsequent vasopressor administration, they employed a super learner framework, an ensemble method that evaluated 13 different algorithms and selected the best-performing combination. Rather than committing in advance to a single modeling technique, the super learner approach lets the data determine which algorithm or weighted blend of algorithms best captures the underlying relationships, a strategy increasingly favored in clinical prediction work because it reduces the risk that a poorly chosen method will underperform. Random Forest emerged as the optimal model, achieving an area under the receiver operating characteristic curve of 0.827 (95 percent confidence interval 0.777–0.876) and an area under the precision-recall curve of 0.661—a level of discrimination the authors describe as accurate for this clinical task.

Feature importance analysis revealed that hemodynamic measures dominated the predictive signal. Post-second-bolus mean arterial pressure and the severity of the baseline mean arterial pressure before resuscitation were the strongest predictors of subsequent vasoactive need, a finding that aligns with clinical intuition: children whose blood pressure remains depressed after two boluses, or who presented with more profoundly abnormal pressures to begin with, were far more likely to progress to fluid-refractory shock. This convergence between the model’s internal logic and established physiology is reassuring, since machine-learning models can sometimes achieve high performance by exploiting artifacts of documentation or care patterns rather than genuine biology. Notably, blood urea nitrogen was the only laboratory variable retained in the final model, suggesting that the core predictive information resides in vital signs rather than in an extensive laboratory panel—an attribute that enhances the tool’s practicality in busy emergency departments and in settings with limited laboratory turnaround, where results of blood cultures, lactate, or chemistry panels may take hours to return.

Beyond raw discrimination, the team evaluated calibration and translated the model’s continuous risk scores into four clinically interpretable risk tiers. Discrimination alone—the model’s ability to rank patients correctly—is not sufficient for bedside use; clinicians need to know whether a predicted probability of, say, 40 percent actually corresponds to outcomes observed roughly 40 percent of the time. Observed vasoactive medication rates across these tiers ranged from 6.6 percent in the lowest-risk group to 63.6 percent in the highest, a 9.6-fold gradient. Calibration plots comparing observed versus predicted rates demonstrated that the model’s probability estimates tracked actual outcomes across the tiers, an important property for any tool intended to inform decisions rather than merely rank-order patients. The authors report following contemporary reporting standards for clinical prediction models, including the TRIPOD+AI guidance, which was developed to improve the transparency, completeness, and reproducibility of artificial intelligence–based prediction studies, and used permutation-based methods to assess feature importance across multiple algorithms, a technique that perturbs each variable in turn and measures how much model performance degrades.

The study sits within a growing body of work applying machine learning to pediatric sepsis. Previous efforts have targeted earlier stages of care, such as predicting sepsis at triage or identifying children with severe sepsis using the Phoenix criteria, the recently updated international consensus framework for defining sepsis in children. What distinguishes the current model is its focus on a specific, high-stakes juncture: the moment after initial fluid resuscitation when the clinician must choose between continuing fluids and escalating to vasoactive support. By anchoring the prediction to this decision point and restricting inputs to variables available at that moment, the tool is designed to be actionable rather than retrospective—a distinction that matters, because prediction models are only useful in practice when their inputs exist at the time the decision must be made.

The authors and observers note several limitations inherent to the study design. The analysis is retrospective, meaning the model predicts what clinicians actually did—vasopressor administration—rather than a gold-standard physiologic endpoint of fluid refractoriness, and treatment decisions at the participating centers may themselves have been influenced by local practices, thresholds, and resource availability. The cohort of 341 patients, though adequate for model development, is modest, and the model was developed and evaluated within the same multicenter dataset rather than being externally validated in entirely new institutions or prospectively, leaving open the possibility that performance will be somewhat lower in unseen settings—a well-documented phenomenon in clinical machine learning. Generalizability to community emergency departments, resource-limited settings, or patient populations underrepresented in the data remains to be established. The researchers also caution that the tool is intended to support, not replace, clinical judgment.

Nevertheless, the implications are significant. A calibrated, eight-variable model that can be computed from data already flowing into the electronic health record could be embedded in clinical decision support systems, flagging children in the highest risk tier for earlier critical care consultation, earlier central access preparation, or closer hemodynamic monitoring—while reassuring clinicians that lower-risk patients may reasonably continue fluid-based resuscitation. Such triage support could also standardize care across centers and help address documented sociodemographic disparities in pediatric sepsis outcomes by making escalation decisions less dependent on individual judgment alone, since standardized, data-driven prompts applied uniformly may blunt the influence of unconscious bias in who gets escalated quickly.

The research was supported by the National Institutes of Health through the Small Business Technology Transfer program (Award Number 5R41AI167224), and the institutional review board at Children’s National Hospital determined the project did not constitute human subjects research. The datasets and code underlying the model have been made publicly available in a GitHub repository, a transparency measure that should facilitate independent validation and allow other research groups to test the model on their own data before any clinical deployment. The study team, led by first author Tom Velez and corresponding author Ioannis Koutroulis of Children’s National Hospital, included collaborators from pediatric emergency and critical care divisions across the five participating institutions.

The authors write that the next steps involve prospective evaluation of the tool in live clinical environments, where its effect on the timing of vasopressor initiation, fluid balance, and patient-centered outcomes can be measured directly. Such before-and-after or stepped-wedge evaluations are the recognized pathway from promising retrospective models to tools that actually change care, and they will reveal whether real-time implementation introduces new failure modes, such as alert fatigue or overreliance on the model’s output. If those studies confirm the model’s performance, the four-tier risk framework could give emergency clinicians something they currently lack at the second-bolus decision point: an objective, evidence-based estimate of the probability that a child’s shock will prove fluid-refractory, delivered in time to act on it.

Drawing upon an extensive dataset comprising electronic health records and registry information from five hospitals spread across four varied Chinese provinces, the investigation leveraged a robust cohort of 9,221 pediatric ICU encounters documented over an 11-year period, from January 2012 through December 2023. This considerable sample size afforded a rich tapestry of clinical variables and patient demographics reflective of real-world heterogeneity, critical for the authentic assessment of scoring systems. The study population exhibited an in-hospital mortality rate of 13.4%, underscoring the severity and pressing need for accurate mortality forecasting mechanisms in this vulnerable group.

The original PSS, originally conceptualized to unify pediatric sepsis assessment by quantifying organ dysfunction across four core systems—cardiovascular, respiratory, coagulation, and neurological—was scrutinized using two pivotal performance metrics: the Area Under the Receiver Operating Characteristic curve (AUROC) and the Area Under the Precision-Recall Curve (AUPRC). Across various clinically pertinent temporal windows, the PSS demonstrated only moderate predictive power for in-hospital mortality, consistently yielding AUROC values approximating 0.60. These results signified a substantial limitation, indicating that within the Chinese ICU context, the PSS may lack the discriminative precision requisite for confident clinical decision-making regarding mortality risk.

Importantly, the findings illuminate the nuanced reality that prognostic tools such as the PSS, though internationally proposed, are not universally transferable without meticulous validation. Variability in baseline patient health status, prevalence of underlying chronic pathologies, differential resource availability, and distinctive clinical practices—including the operational definition of “suspected infection”—profoundly influence cohort characteristics and consequently the predictive success of scoring algorithms. This heterogeneity necessitates prudence when translating such tools across diverse healthcare landscapes, especially when repurposed primarily for mortality risk stratification rather than initial sepsis detection.

Confronted with these limitations, the research team innovatively pursued a modification pathway that emphasized the retention of the PSS’s conceptual integrity while enhancing predictive accuracy and clinical interpretability. They harnessed extreme gradient boosting (XGBoost), a powerful machine learning technique adept at modeling complex, nonlinear relationships, to detect candidate mortality predictors within the dataset. To elucidate and rank the relevance of these predictors, SHapley Additive exPlanations (SHAP) were employed, providing transparent interpretability of the model’s decision-making process.

Crucially, the selection of predictors was not left to algorithmic discretion alone; rather, it was grounded firmly in clinical relevance and practicality. Only variables that clinicians could readily assess based on routine practice, which also held clear pathophysiological significance and were interpretable at the bedside, were incorporated. This careful balance between data-centric methods and expert clinical judgment led to the emergence of a modified scoring system termed PSS+. PSS+ uniquely combines original PSS organ system indicators with select demographic factors, pre-existing comorbidities, and vital signs, crafted into a parsimonious logistic regression model.

To rigorously evaluate the generalizability and robustness of the PSS+, a site-aware validation strategy was deployed. Four hospitals participated in model training and internal validation, each further divided into training and holdout datasets to minimize overfitting risk. Critically, a fifth hospital in Fujian Province contributed an entirely independent external validation cohort, providing a stringent test of transportability. Across both internal test sets and the external cohort, the PSS+ markedly outperformed the original PSS variants and the pediatric Sequential Organ Failure Assessment (pSOFA) score, delivering AUROC values of 0.75 and 0.71, respectively. These enhancements indicate a substantially elevated capacity to accurately discern high-risk pediatric patients likely to experience fatal outcomes.

Further analytic techniques underscored the statistical solidity of the modified model. Multicollinearity assessment confirmed the absence of significant inter-variable redundancy, thereby affirming the stability and interpretability of PSS+. Intriguingly, the presentation of PSS+ as a nomogram—an intuitive graphical tool—lowers barriers to clinical adoption, enabling bedside clinicians to easily compute mortality risk estimates without necessitating complex computational resources.

This pioneering study underscores a pivotal lesson in critical care and predictive analytics: internationally standardized scoring systems, while foundational, require local contextualization and tailoring to optimize performance within distinct healthcare milieus. The improved discrimination demonstrated by PSS+ exemplifies how integrating data-driven insights with thoughtful clinical acumen can yield risk stratification tools that balance predictive accuracy with usability, ultimately enhancing patient outcomes through timely therapeutic interventions.

Beyond its immediate clinical implications, this investigation offers a valuable blueprint for future research endeavors across global health domains. It highlights the indispensable role of diverse data sources, machine learning interpretability frameworks, and multidisciplinary collaboration to navigate the intricate interplay of biology, healthcare infrastructure, and algorithmic modeling. Moreover, it offers a timely reminder that risk scores must be continuously revalidated as medical knowledge evolves and healthcare systems transform.

In sum, this comprehensive study reveals both the potential and limitations intrinsic to pediatric sepsis scoring systems. By innovatively enhancing the Phoenix Sepsis Score to create the modified PSS+, the authors deliver a powerful, context-aware tool that promises to better support early, accurate mortality risk stratification in children with suspected infection admitted to ICUs. These advancements herald a new era in the precision management of pediatric sepsis—one that marries standardized rigor with adaptive flexibility to meet the complexities of global critical care.

As pediatric critical care continues to grapple with the challenge of sepsis, the emergence of tools like PSS+ portends increased survival chances through more informed clinical decision-making. Evaluating and validating such scoring modifications across different countries and healthcare settings will be paramount. Meanwhile, this study sets a striking example of how international guidelines can be thoughtfully recalibrated, ensuring they serve diverse patient populations with maximum efficacy and interpretability.

Subject of Research: People

Article Title: Validation and modification of the phoenix sepsis score for predicting in-hospital mortality in children with suspected infection admitted to the intensive care unit

News Publication Date: 19-Mar-2026

Web References:
https://www.doi.org/10.1097/CM9.0000000000003988

References:
DOI: 10.1097/CM9.0000000000003988

Image Credits: Chinese Medical Journal

Keywords

Sepsis, Pediatrics, Infectious diseases, Artificial intelligence, Machine learning, Risk assessment, Clinical research, Data analysis, Public health