New Global Benchmark Aims to End the Reproducibility Crisis in AI-Driven Peptide Drug Discovery

Computational models that predict how peptides bind to their protein targets have become one of the most promising engines of modern drug discovery, but the field has quietly been running on shifting ground. Different research groups have measured success with incompatible metrics, built their test sets in contradictory ways, and sampled their negative examples with such abandon that headline performance figures often say more about dataset construction than about molecular recognition. A new consensus framework, published in Frontiers in Bioinformatics, now attempts to end this chaos with a single, internationally ratified rulebook for evaluating peptide-target interaction prediction.

The framework, called PTI-TAPE (Peptide-Target Interaction—Tasks Assessing Peptide Engagement), emerged from a structured three-round eDelphi consensus study led by researchers at Imperial College London, the University of Nebraska Medical Center, and the University of Pennsylvania, together with the PTI-TAPE Consortium Committee. Fifteen international experts spanning computational biology, machine learning, drug discovery, peptide chemistry, and regulatory science worked through ten weeks of iterative anonymous surveys, ultimately ratifying 26 definitive evaluation standards across eight domains. Every standard had to clear a demanding threshold: agreement from more than 80 percent of responding panellists, combined with low inter-respondent variance.

The stakes are considerable. More than eighty peptide therapeutics have already received global approval, and over 170 candidates are in clinical development. Peptides occupy a structurally privileged therapeutic niche, combining the binding specificity of large biologics with the synthetic accessibility of small molecules. Deep learning architectures, from graph neural networks and SE(3)-equivariant frameworks to transformers adapted for molecular interaction tasks, have accelerated every stage of the peptide pipeline, while AlphaFold 3 has made it possible to model peptide-protein complexes with previously unattainable structural accuracy. Yet without agreed evaluation standards, the field cannot tell which of these advances are real.

The scale of the problem is documented in detail by the systematic review underlying PTI-TAPE. Performance has been reported with metrics as divergent as AUROC, Matthews correlation coefficient, mean squared error, Spearman rank correlation, and root mean square deviation, with no consensus on which should be mandatory. Negative sampling strategies have swung between random, clustered, and sequence-similarity-filtered approaches, and the more permissive methods demonstrably inflate performance estimates. Because most public repositories catalogue binders far more completely than confirmed non-binders, a model can achieve high apparent accuracy simply by exploiting compositional artefacts of the negative set rather than learning the true determinants of binding. The small-molecule virtual screening literature learned this lesson long ago; peptide benchmarking has largely ignored it.

Among the ratified standards, several stand out as technically consequential. For affinity estimation, the panel mandated reporting Pearson and Spearman correlation alongside root mean squared error, each with 95 percent confidence intervals, and—critically—required demonstration of correct rank ordering across the full affinity range, from high-affinity binders through to confirmed non-binders. This provision directly blocks a common reporting artefact in which correlations computed over a few points spanning a narrow affinity range appear strong yet carry no evidence of genuine ranking ability. For structural modelling, ligand RMSD from the crystallographic reference is now the required accuracy measure. For binary classification, AUROC became a required primary metric, with AUPRC elevated to co-primary status for imbalanced tasks after three rounds of debate, and the Matthews correlation coefficient endorsed as recommended but not mandatory.

The negative sampling hierarchy is arguably the most consequential output of the entire consensus process. Experimental non-binders—peptides measured not to bind a given target—were ranked first by a wide margin, yet panellists simultaneously recognised that such data are critically underrepresented in existing repositories. The practical resolution is a tiered fallback hierarchy: experimental non-binders as the gold standard, followed by literature-derived non-binding peptides, then structurally non-compatible pairs, and finally shuffled sequences. The framework also imposes three practical requirements: the provenance of negatives must be reported as explicitly as that of positives, the quantity and ratio of negatives to positives must be declared, and negatives should be used to test monotonic scoring across the affinity continuum rather than merely to populate a classification task. Only consistent ordering across diverse targets provides evidence that a method has captured the physics of association rather than dataset-specific structure.

Temporal dataset splits are now mandatory for all benchmark datasets, a direct response to the near-invisible problem of data leakage, where future interaction data contaminate training sets in ways that are impossible to audit under current reporting norms. Dataset size requirements proved the most contentious issue of all: Round 1 responses spanned five orders of magnitude, from fewer than 100 to more than a million binding pairs. Rather than treating this as noise, the organisers recognised it as genuine task-dependence—classification tasks demand large datasets, affinity estimation suffers from scarce high-quality binding constants, and structural modelling may suffice with hundreds of well-resolved complexes. The task-stratified reframing introduced in Round 2 collapsed the variance and produced consensus, with a minimum of 10,000 binding pairs recommended for binary classification.

The entire edifice is governed by a new organising framework called STRIDE: Standardisation, Transparency, Representativeness, Integration, Discovery, and Evidence. Standardisation demands identical evaluation protocols across all submitted methods. Transparency mandates full disclosure of methods, data, code, and limitations sufficient for independent reproduction, including open-source evaluation code, pre-trained baseline models, API access, and tutorial materials. Representativeness requires balanced coverage across peptide biology, chemistry, structure, and target diversity—a demanding ask given the chemical diversity of therapeutic peptides, which span linear, cyclic, lipidated, stapled, and non-natural amino acid variants. Integration connects computational predictions to experimental validation, while Discovery protects methodological innovation, and Evidence aligns evaluation with clinical utility and regulatory requirements.

The authors are candid about the framework’s limitations. The panel of fifteen experts, while internally diverse, falls at the lower bound of the target range and below the sample sizes of some comparable Delphi initiatives; with thirteen respondents in Round 2, a single participant shifting two points on the Likert scale moves the mean by roughly 0.15, which is material relative to the gap between consensus categories. The panel also skewed toward computational and academic fields, leaving experimental peptide chemists, clinical investigators, and regulatory scientists underrepresented. And the study reports consensus achieved through deliberation but cannot yet demonstrate that adopting the standards improves real-world outcomes—prospective evaluation comparing reproducibility between compliant and non-compliant studies remains a necessary next step.

PTI-TAPE is explicitly designed as a living platform rather than a static document, with biennial standard reviews coordinated by an elected, rotating multi-disciplinary steering committee and versioned releases to track how standards evolve. The authors map out an ambitious implementation roadmap: journal and funder adoption progressing from encouragement to mandatory compliance within three years, integration with high-throughput experimental validation platforms such as phage and mRNA display, compliance certification for contract research organisations, and—most consequentially—the potential for PTI-TAPE compliance to serve as a credibility signal within regulatory submissions. The framework is careful to state that compliance documents methodological rigour, not regulatory readiness. But the direction of travel is clear: a field that could not compare two methods across independent benchmarks now possesses a ratified, quantitatively anchored, and community-endorsed evaluation infrastructure, and the gap between computational PTI prediction and genuine therapeutic benefit has never looked narrower.

Subject of Research: Standardised benchmarking protocols for computational peptide-target interaction prediction

Article Title: Establishing standardized evaluation protocols for peptide-target interaction prediction: the PTI-TAPE consensus framework eDelphi

Article References: Waldock, W. J., Wang, G., Darzi, A., de la Fuente-Nunez, C., Ashrafian, H., & the PTI-TAPE Consortium Committee (2026). Establishing standardized evaluation protocols for peptide-target interaction prediction: the PTI-TAPE consensus framework eDelphi. Frontiers in Bioinformatics, 6, Article 1894725. https://doi.org/10.3389/fbinf.2026.1894725

Image Credits: AI Generated

DOI: 10.3389/fbinf.2026.1894725

Keywords: peptide-target interaction, PTI-TAPE, eDelphi consensus, benchmarking, drug discovery, machine learning, evaluation standards, negative sampling, temporal splits, STRIDE framework, therapeutic peptides, reproducibility