Why Promising Immune Biomarkers Collapse in the Clinic: New Framework Predicts Failure Before Validation

Every year, cancer researchers report dozens of new immune biomarkers that appear to predict which patients will respond to immunotherapy. Genes associated with T cell activity, interferon signaling, and cytotoxic killing of tumor cells routinely achieve impressive statistical performance in bulk RNA-sequencing datasets. Yet when these markers move from retrospective discovery cohorts into prospective clinical validation, a large fraction quietly fail. A study published in BMC Bioinformatics by Hoang Minh Quan Pham, Po-Hao Feng, Chia-Ling Chen, Kang-Yun Lee, and Chiou-Feng Lin of Taipei Medical University and collaborators now offers a systematic explanation for this recurring disappointment, along with a computational toolkit designed to flag doomed biomarkers before expensive validation studies are ever launched.

The central problem the researchers identify is what they call phenotypic inflation: the systematic gap between how well a biomarker performs in a discovery dataset and how much genuine clinical predictive value it actually carries. When the team plotted phenotypic performance against clinical performance across their cohorts, they found a striking and reproducible pattern. Every 0.1-unit increase in the phenotypic area under the curve, a standard measure of classification accuracy, predicted a 0.079-unit larger drop in clinical performance, a relationship that explained roughly 37 percent of the variance and held with overwhelming statistical significance. Most remarkably, above a phenotypic AUC of approximately 0.55, a threshold that 95 percent of observations in their dataset exceeded, additional phenotypic performance conferred no additional clinical predictive value whatsoever.

This finding strikes at the heart of conventional biomarker discovery practice. If most published immune biomarkers already sit above the 0.55 phenotypic AUC threshold, then the endless pursuit of ever-higher classification accuracy within discovery datasets is, according to this analysis, largely a waste of effort. The uncoupled model, which removed the systematic relationship, explained only about 4 percent of variance, confirming that the inflation phenomenon is not statistical noise but a structured, quantitatively reproducible feature of bulk transcriptomic biomarker evaluation. The relationship replicated independently across ten external RNA-sequencing cohorts and held when the analysis was stratified by cancer type, where it explained more than half of the observed variance.

To understand why this inflation occurs, the authors turned to the biology of the tumor microenvironment. Bulk RNA sequencing measures the average gene expression across a mixture of tumor cells, immune cells, fibroblasts, and other stromal components. A gene that appears to be a strong marker of antitumor immunity in bulk data may in fact be expressed by a completely different cell type than assumed, or its apparent signal may be driven entirely by shifts in the proportions of cell populations rather than by genuine changes in per-cell expression. Either scenario can produce a biomarker that looks excellent in one cohort but collapses in another where the cellular composition differs.

The team’s first diagnostic instrument addresses this problem directly. The Source Fidelity Index quantifies how faithfully a gene’s expression can be attributed to T cells and natural killer cells, drawing on single-cell RNA-sequencing data from 2.2 million individual cells across 18 datasets. The results were sobering. Of 101 immune genes commonly invoked in biomarker studies, only 35, or 34.7 percent, showed high T/NK fidelity with an SFI of 0.6 or greater. In other words, roughly two-thirds of the genes that researchers routinely treat as proxies for cytotoxic immune activity cannot be reliably attributed to the immune cells they are assumed to represent. Well-known markers such as granzyme A, perforin 1, granulysin, and NKG7 fared better than many others, but the overall picture suggests that a substantial portion of the immune biomarker literature rests on shaky cellular attribution.

The second metric, the Biomarker Fragility Index, tackles the complementary problem of composition sensitivity. Using three-dimensional simulation across 82 synthetic cellular compositions for each gene, the framework measures how much a gene’s bulk expression signal changes when the mixture of cell types shifts. A fragile biomarker is one whose bulk-level behavior is dominated by composition rather than by biology, making it inherently unreliable across patient cohorts with different immune infiltration patterns. When the researchers tested whether fragility carried information beyond what spatial methods could capture, they found that BFI provided 18.6 to 27.3 percent independent explanatory variance for clinical outcomes even after controlling for a spatially derived source-fidelity estimate, with significance in four of five phenotype definitions. Notably, BFI also correlated independently with spatial myeloid co-localization, suggesting that the local positioning of myeloid cells relative to other populations contributes to how fragile a biomarker’s bulk signal becomes.

Combining these metrics with additional risk factors, the authors assembled a six-factor Operational Risk Stratification framework and validated it in 754 samples from eight immune checkpoint blockade cohorts. The composition-boundary rule, a simple decision criterion derived from the fragility analysis, achieved 82.8 percent accuracy in classifying whether a biomarker’s phenotypic behavior was reliable. The full six-factor framework recalled 93.3 percent of phenotypic failures, and this performance remained stable under leave-one-cohort-out validation, with mean recall of 95.0 percent across held-out cohorts. Perhaps most practically, BFI-guided selection reduced the risk of phenotypic validation failure by 82 percent compared with random biomarker selection, a difference that reached permutation-based significance at p equals 0.007. For research groups deciding which candidate genes to carry forward into costly validation programs, this represents a potentially enormous saving of time and resources.

The framework’s robustness was tested against a common criticism of such approaches: sensitivity to the statistical assumptions embedded in the analysis. The authors report that BFI’s classification performance was invariant across all tested distributional assumptions, meaning the tool does not quietly depend on a particular model of how gene expression data are distributed. To demonstrate generalizability, the team applied the framework pan-cancer across The Cancer Genome Atlas, identifying 168 gene-cancer combinations in the Unreliable tier spanning 32 cancer types. This catalog provides an immediate, actionable resource: any researcher considering one of these combinations as a biomarker in a given cancer type now has a quantitative warning that the marker’s bulk-level signal is likely to be composition-driven and fragile.

The clinical context of this work is the growing reliance on immune checkpoint blockade, the class of therapies that unleashes T cells against tumors and has transformed outcomes in melanoma, lung cancer, and other malignancies. Biomarkers such as PD-L1 expression, the cytolytic activity score, and gene signatures built from T cell markers are widely used to guide patient selection, yet their performance varies frustratingly across tumor types and treatment settings. The new study does not invalidate these tools, but it reframes the question researchers should ask. Rather than asking whether a biomarker achieves high accuracy in a discovery cohort, the framework argues, investigators should first ask whether the biomarker’s signal is genuinely attributable to the intended cell type and whether it can survive changes in cellular composition. A biomarker that passes those tests may justify validation; one that fails them is likely to join the long list of phenotypically impressive markers that never reach the clinic.

The implications extend beyond immunotherapy. Bulk transcriptomics remains the workhorse of large-scale biomarker discovery across medicine, from autoimmune disease to cardiology, and the composition-driven fragility mechanism the authors describe is not unique to cancer. Any bulk measurement drawn from heterogeneous tissue is vulnerable to the same inflation phenomenon, in which mixture effects manufacture apparent signal that dissolves under cohort variation. By providing open, quantitative diagnostics, the Source Fidelity Index and Biomarker Fragility Index offer a template for pre-validation screening that could be adapted to other fields. The authors, who note that generative artificial intelligence was used only for language refinement while all scientific contributions were human-generated, funded by Taiwan’s National Science and Technology Council, present the work as a practical addition to the biomarker development pipeline. If the framework’s predictions hold as widely as its validation suggests, the era of discovering biomarkers that look brilliant on paper and fail in patients may finally be drawing to a close, replaced by a more honest accounting of what bulk gene expression can and cannot tell us about the biology of disease.

Subject of Research: Computational diagnostics for predicting translation failure of bulk RNA-sequencing immune biomarkers in cancer immunotherapy

Article Title: Phenotypic Inflation and composition-driven fragility: a pre-validation diagnostic framework for immune biomarker translation failure

Article References: Phenotypic Inflation and composition-driven fragility: a pre-validation diagnostic framework for immune biomarker translation failure. (n.d.). https://doi.org/10.1186/s12859-026-06691-x

Image Credits: AI Generated

DOI: 10.1186/s12859-026-06691-x

Keywords: immune biomarkers, phenotypic inflation, biomarker fragility, bulk RNA-sequencing, tumor microenvironment, immune checkpoint blockade, single-cell RNA sequencing, biomarker translation, cancer immunotherapy, TCGA, transcriptomics, BMC Bioinformatics