New Python Package Puts Biomarker Discoveries to the Test Before They Reach the Clinic

Every year, thousands of biomedical studies claim to have discovered a new biomarker—a measurable biological signature that could predict how a patient will respond to a drug, whether a tumor will resist immunotherapy, or which treatment path offers the best odds of survival. Yet a striking fraction of these discoveries never survives contact with new data. When researchers rerun the same analysis on a slightly different patient cohort, the celebrated molecular markers often vanish, replaced by an entirely different set of genes or proteins. The problem is rarely the biology. More often, it is the feature selection method itself, applied without adequate scrutiny, that manufactures fragile, irreproducible results. A new open-source software package called ROOFS, short for RObust biOmarker Feature Selection, published in BMC Bioinformatics by a team led by Anastasiia Bakhmach and Sébastien Benzekry of the COMPutational pharmacology and clinical Oncology team at the Centre Inria d’Université Côte d’Azur in Marseille, aims to change that by forcing every feature selection method to prove its worth on the researcher’s own data before any biomarker claim is made.

Feature selection is the computational step in which an algorithm sifts through potentially thousands of measured variables—gene expression levels, protein abundances, clinical measurements—and decides which subset carries the genuine predictive signal. It is essential for biomarker discovery and for building clinical predictive models, because models trained on too many noisy variables tend to memorize the quirks of their training data rather than learn generalizable patterns. Over the past decades, the methodological literature on feature selection has become rich and mature, offering a wide spectrum of algorithmic approaches, from filter methods that rank variables by statistical association, to embedded methods like LASSO that perform selection during model training, to wrapper methods that repeatedly retrain models on different variable subsets. But the ROOFS team observed a persistent gap: much of this methodological progress has not been fully adopted in applied biomedical research. Practitioners often pick a single familiar method, run it once, and report the top-ranked features, without asking whether those features would be stable, robust, or even real.

ROOFS tackles this problem with a benchmarking philosophy. Rather than prescribing one best method, the Python package takes the user’s own dataset and runs multiple feature selection methods on it in parallel, currently supporting classification tasks. It then generates reports summarizing a comprehensive set of evaluation metrics that together paint a far more honest picture of what each method has found. The central idea is that the choice of a feature selection method should be data-driven, adapted to the specific problem at hand, rather than inherited from habit or convenience. By automating this comparative evaluation, ROOFS lowers the technical barrier that has kept rigorous methodological benchmarking out of reach for many applied research groups.

Among the metrics ROOFS evaluates, downstream predictive performance is arguably the most important. A feature selection method is only useful if the features it selects actually improve the ability of a predictive model to classify new samples correctly. But estimating this performance naively is a well-known statistical trap. If the same data are used both to select features and to evaluate the resulting model, the reported accuracy is inflated by a statistical bias called optimism. ROOFS addresses this by estimating downstream predictive performance using optimism correction, a technique that quantifies how much the apparent performance overstates the true generalization ability and adjusts the estimate accordingly. This correction is what separates a defensible biomarker model from one that merely flatters its own training data.

Stability is the second pillar of the ROOFS evaluation. A genuine biomarker should appear consistently in the selected feature set regardless of small perturbations in the data—whether a few patients are added or removed, or the analysis is rerun with a different random seed. Methods that select wildly different features on nearly identical datasets are producing artifacts, not biology. ROOFS quantifies stability by repeatedly applying each feature selection method across resampled versions of the data and measuring how consistent the selected features remain. This matters enormously for translational value: a biomarker panel that cannot be reproduced across cohorts has little hope of surviving validation, let alone reaching clinical use.

Robustness of individual features adds a third dimension. Beyond asking whether the overall feature set is stable, ROOFS examines how reliable each individual feature is—how consistently a particular gene or protein earns its place in the selection across repeated analyses. This distinction is technically meaningful because a method can produce a stable overall panel while individual members fluctuate, or vice versa. For researchers deciding which candidates to carry forward into expensive experimental validation, knowing which individual features are robust and which are marginal can save years of wasted laboratory effort and millions in research funding.

Perhaps the most technically inventive component of ROOFS is its use of semi-synthetic data with a simulated outcome to estimate true positive and false positive rates. In real biomarker studies, researchers never know the ground truth—which features are genuinely predictive and which are noise—so they cannot directly measure how often a method correctly identifies real signals or falsely promotes random variables. ROOFS circumvents this by constructing semi-synthetic datasets in which the outcome is simulated and the truly predictive features are known by construction. Running each feature selection method on these controlled datasets reveals its true positive rate—how reliably it detects genuine signals—and its false positive rate—how often it promotes pure noise to the status of biomarker. This gives researchers a calibrated understanding of each method’s detection characteristics, something no single real-world analysis can provide.

The ROOFS team demonstrated the package’s utility on data from the PIONeeR biomarker study, a clinical research program aimed at identifying predictors of resistance to anti-PD-(L)1 immunotherapy in lung cancer. Immunotherapy with PD-1 and PD-L1 inhibitors has transformed the treatment of non-small cell lung cancer, but a substantial proportion of patients derive little benefit, and identifying who will resist treatment remains one of the most pressing unsolved problems in oncology. The PIONeR study, sponsored by Assistance Publique Hôpitaux de Marseille and conducted in accordance with Good Clinical Practice guidelines with approval from the French ethics committee, generated exactly the kind of high-dimensional clinical and molecular data where feature selection choices carry maximum consequence. Applying ROOFS to this data allowed the researchers to benchmark competing feature selection methods and evaluate which candidates for resistance biomarkers could withstand scrutiny across stability, robustness, and predictive performance metrics.

The implications reach well beyond lung cancer immunotherapy. The authors argue that comprehensive benchmarking with ROOFS has the potential to improve the reproducibility of feature selection discoveries and increase the translational value of clinical models. In an era when machine learning is being deployed across every domain of medicine—from radiology to genomics to drug response prediction—the reproducibility crisis in biomarker research represents both a scientific failure and a barrier to patient benefit. Tools that institutionalize rigorous evaluation at the point of discovery could reshape how applied biomedical research is conducted, making it standard practice to ask not just which features a method selects, but whether those features deserve to be trusted. The package is written in Python, supports classification tasks, and is freely available on the GitLab repository of Inria, the French national research institute for digital sciences, lowering the entry cost for research groups worldwide.

ROOFS arrives at a moment of reckoning for computational biomedicine. The methodological literature on feature selection is mature and abundant; what has been missing is the bridge that carries that maturity into everyday applied research. By automating multi-method benchmarking, correcting for statistical optimism, quantifying stability and robustness, and calibrating detection performance on semi-synthetic ground truth, ROOFS offers researchers a way to make feature selection choices that are defensible, transparent, and reproducible. For a field where fragile biomarker claims have too often collapsed under validation, that bridge could not come soon enough. Whether it succeeds in changing practice will depend on adoption—but the tool itself, open access and openly licensed, is now available for any research team willing to put its biomarker discoveries to the test.

Subject of Research: Biomarker discovery

Article Title: ROOFS: RObust biOmarker Feature Selection

Article References: Bakhmach, A., Dufossé, P., Charpigny, S., Monville, F., Greillier, L., Barlési, F., & Benzekry, S. (2026). ROOFS: RObust biOmarker Feature Selection. BMC Bioinformatics. https://doi.org/10.1186/s12859-026-06667-x

Image Credits: AI Generated

DOI: 10.1186/s12859-026-06667-x

Keywords: feature selection, biomarker discovery, machine learning, reproducibility, lung cancer, immunotherapy, BMC Bioinformatics, Python, ROOFS, RObust, biOmarker, Feature