The core problem is deceptively simple. When researchers evaluate a toxicity model, they typically shuffle their dataset randomly and hold out a fraction for testing. But chemical datasets are full of close molecular relatives, analogs that differ only by a small substituent yet share nearly identical structures. Under random splitting, a near-twin of a test compound often sits in the training data, and the model effectively memorizes the answer rather than learning generalizable chemistry. ToxBench was designed to measure this inflation directly rather than assume it away.

The benchmark assembles three widely used toxicology datasets: Tox21, covering 7,538 compounds across 12 toxicity endpoints; ClinTox, with 1,379 compounds and 2 tasks focused on clinical toxicity; and SIDER, containing 1,350 compounds and 27 side-effect tasks. All three were processed through a transparent standardization pipeline in which the removal of compounds with conflicting labels is explicitly reported, an unusual level of bookkeeping that makes the benchmark reproducible from end to end.

Four model classes were put through their paces: Random Forest, XGBoost, a multilayer perceptron, and a Graph Neural Network that reads molecules as molecular graphs. Each model was trained under both random and Bemis-Murcko scaffold-based splits, the latter grouping compounds by their shared molecular scaffold so that entire structural families fall on one side of the divide. Every configuration was repeated across five independent random seeds, yielding 120 distinct experimental conditions, a deliberate hedge against the luck-of-the-draw variability that plagues small molecular datasets.

The headline finding is stark. Moving from random to scaffold splitting reduced the area under the receiver operating characteristic curve, or AUROC, by 0.057 to 0.079 points across all four model classes on Tox21, with a mean drop of 0.070. On SIDER, three of the four models lost 0.031 to 0.035 AUROC points. In a field where claimed improvements between competing algorithms are often smaller than that margin, the implication is uncomfortable: a substantial share of the progress reported in predictive toxicology may reflect leakage rather than genuine learning.

Crucially, the authors refused to treat scaffold splitting as a magic fix. Instead of assuming that grouping by scaffold eliminates leakage, they audited the residual structural similarity between training and test sets directly. On Tox21, the fraction of test compounds with a nearest-neighbor Tanimoto similarity above 0.6 to the training set falls from 43.9 percent under random splitting to 12.1 percent under scaffold splitting. That is a dramatic reduction, but it means roughly one test compound in eight still has a close chemical cousin in the training data, and the benchmark quantifies this residue rather than ignoring it.

Not every dataset behaved as expected. ClinTox, the smallest of the three, showed a reversed performance ordering between split types, and its seed-to-seed variance of plus or minus 0.085 to 0.160 AUROC points confirmed that results on this dataset are dominated by sampling noise rather than by the split strategy itself. With extreme class imbalance and only 1,379 compounds, ClinTox serves as a cautionary tale: benchmarks built on small, skewed datasets can produce rankings that are essentially statistical artifacts.

The study also tackled a question that receives far less attention than raw accuracy: whether a model’s predicted probabilities mean what they say. When a toxicity model reports an 80 percent chance of hepatotoxicity, regulators and medicinal chemists need that number to be calibrated, meaning that among compounds assigned that score, roughly 80 percent really are toxic. Post-hoc calibration cut the expected calibration error by 67 to 68 percent on ClinTox, showing that reliable probability estimates are achievable with simple corrections. On Tox21 and SIDER, raw Random Forest predictions were already well calibrated, with expected calibration errors of just 0.018 and 0.057 respectively, and post-hoc methods added nothing. Scaffold splitting, however, consistently worsened calibration across all three datasets, meaning the harder, more honest evaluation regime also produces less trustworthy probabilities.

The benchmark’s third pillar addresses a practical question for anyone deploying these models: when should they be trusted at all? Applicability domain analysis revealed a consistent positive relationship between the Tanimoto similarity of a test compound to its nearest training neighbor and predictive reliability, with low-similarity compounds showing notably reduced AUROC under scaffold splitting. In other words, models are most trustworthy for compounds that resemble their training data, and performance degrades predictably as inputs drift into unfamiliar chemical space. Ensemble-based uncertainty estimates proved useful here too: filtering out predictions the model itself was not confident about improved AUROC by up to 0.025 points on SIDER, offering a simple operational rule for flagging unreliable outputs before they reach a decision-maker.

Together, these analyses turn ToxBench into more than a leaderboard. The authors argue that scaffold splitting, multi-seed evaluation, and applicability domain filtering should become standard practice in toxicity prediction benchmarking, and their leakage audit provides a template for how to verify that a split is genuinely doing its job. For a field whose predictions increasingly inform drug safety decisions, the message is clear: reported accuracy is only as credible as the split that produced it, and knowing when a model is guessing outside its comfort zone may matter as much as how often it is right.

Subject of Research: A leakage-audited multi-task benchmark evaluating machine learning models for predictive toxicology with calibration, uncertainty, and applicability domain analysis

Article Title: ToxBench: a leakage-audited multi-task benchmark for predictive toxicology with calibration, uncertainty, and applicability domain analysis

Article References: Kartic, Seo, Y., Yi, S., & Park, T.-S. (2026). ToxBench: a leakage-audited multi-task benchmark for predictive toxicology with calibration, uncertainty, and applicability domain analysis. BMC Bioinformatics. https://doi.org/10.1186/s12859-026-06621-x

Image Credits: AI Generated

DOI: 10.1186/s12859-026-06621-x

Keywords: predictive toxicology, machine learning, data leakage, scaffold splitting, ToxBench, probability calibration, uncertainty quantification, applicability domain, Tox21, ClinTox, SIDER, graph neural networks

Small molecules face a formidable structural challenge, however. The PD-1/PD-L1 interface is a broad, shallow protein-protein contact surface, long considered a difficult target for drug-like compounds. Progress has come from chemotypes such as the biphenyl-based inhibitors developed by Bristol-Myers Squibb, including BMS-202, which bind PD-L1 and induce dimerization that blocks PD-1 recognition. Compounds such as INCB086550 and the oral modulator CA-170 have entered early clinical evaluation, proving that small-molecule checkpoint inhibition is feasible. Still, only a small fraction of explored scaffolds has advanced to the clinic, and screening the relevant chemical space experimentally is costly and slow. Computational triage has therefore become essential for deciding which candidates deserve synthesis and bioassay follow-up.

A team led by Monsin Sangsawat and Pornchai Rojsitthisak at Chulalongkorn University, with collaborators including Worathat Thitikornpong, Cong Feng, and Ming Chen, has now published an integrated solution in the journal Results in Chemistry. Their study reports an open-source, end-to-end cheminformatics framework for predicting the potency of small-molecule PD-L1 inhibitors, built entirely in the KNIME Analytics Platform. Every processing step, from data retrieval to model deployment, is encoded as an explicit node sequence, and the executable workflow file, curated datasets, and fixed split identifiers have been deposited in a public GitHub repository. The design directly confronts a persistent problem in the field: many published machine-learning models for PD-L1 are difficult to reproduce because training data, preprocessing details, or workflow code are inaccessible, and few are evaluated across independent data sources.

The foundation of the framework is a rigorously curated dataset. The researchers queried ChEMBL, targeting the human PD-L1 entry CHEMBL4523993, and retrieved more than 15,000 bioactivity records. To reduce label noise, they applied assay-level filters restricting the modeling set to homogeneous Alpha and HTRF assay formats, the two readouts with the most consistent annotation and the broadest activity coverage. After removing ambiguous qualifiers, duplicates, and non-numeric annotations, and after converting IC50 values to logarithmic pIC50 units, 2413 unique compounds remained. These were stratified into three potency tiers: active compounds with IC50 below 10 nanomolar, moderate inhibitors between 10 and 100 nanomolar, and weak or inactive compounds above 100 nanomolar. The curated set spans 953 unique Bemis-Murcko scaffolds, preserving the chemical diversity reported for PD-L1 inhibitors, from biphenyls and combretastatin analogs to cyclic peptides.

Molecular representation was treated as a central experimental variable. Each compound was encoded with 119 RDKit physicochemical descriptors, which were filtered for zero variance, pruned for high inter-correlation, and refined using recursive feature elimination with cross-validation, ultimately yielding 23 descriptors covering lipophilicity, ring content, van der Waals surface areas, and molecular quantum numbers. In parallel, two fingerprint families were generated: MACCS structural keys and Morgan circular fingerprints with a radius of two, together with a hybrid concatenation of the two. Four regression algorithms were benchmarked across the three representations, producing twelve configurations: Random Forest, a Keras deep neural network, Gradient Boosting, and XGBoost. Each configuration was trained under four fixed random seeds, and results were reported as means with standard deviations to capture stochastic sensitivity.

The Random Forest consistently delivered the most reliable balance between fit and generalization. Using the hybrid MACCS-Morgan representation, it achieved a coefficient of determination of 0.871 on the held-out test set and 0.863 on a strictly external validation set of 403 compounds that were never touched during model selection. Cross-validated Q2 values ranged from 0.888 to 0.902 across descriptor sets, and external Qext2 values reached as high as 0.919 with MACCS fingerprints alone. The deep neural network, although it attained higher training performance for some representations, showed a larger decline on held-out data, suggesting a greater tendency toward overfitting at the current sample size. Importantly, the team verified that no test or external compound shared an identical canonical SMILES with any training compound, and performance was unchanged after excluding high-similarity nearest neighbors, ruling out information leakage through duplicates.

Generalization was then stress-tested with a scaffold-disjoint partition, in which entire Bemis-Murcko scaffold families were assigned exclusively to training or held-out sets. Under this stricter regime, the hybrid Random Forest model achieved a mean test R2 of 0.858 and external validation R2 of 0.851, closely matching the random-split results. This indicates that the reported accuracy was not inflated by scaffold sharing and that the model genuinely extends to structural families absent from training. Prediction reliability was strongest in the moderate-to-potent activity range most relevant to virtual screening, while weak inhibitors with pIC50 below 7 showed larger errors and a modest tendency toward over-prediction, a pattern attributed to the relative scarcity of weak compounds in the curated data.

A distinctive strength of the study is its emphasis on knowing when to trust the model. Two applicability-domain diagnostics were implemented: a distance-based criterion derived from the mean and standard deviation of minimum training-set distances, and a leverage-based Williams plot with the classical threshold of three times the number of descriptors divided by the number of training compounds. Nearly all test and external-validation compounds fell inside the domain, and out-of-domain compounds in the independent BindingDB application set showed markedly larger errors than in-domain ones, with mean absolute errors of 1.884 versus 0.809 under the distance criterion. The researchers applied the model to 124 unique BindingDB compounds after removing 444 overlapping entries. Compounds sharing biphenyl scaffolds or cyclic peptide chemotypes with the training distribution showed small residuals, while structurally distinct pyrazolone derivatives and arylindanyloxypyridine frameworks showed substantially larger errors, pinpointing precisely where future data expansion would pay off.

The framework also translates statistical features into medicinal chemistry insight. SHAP analysis of the Random Forest identified the number of aromatic heterocycles, molar refractivity, and lipophilicity as the most influential descriptors, with threshold-like dependencies. The strongest potency-associated MACCS key simply records the presence of chlorine, but the corresponding Morgan features resolve it more sharply as a chlorinated aryl ring system and a chloro-substituted biaryl, discriminating potency with correlations up to 0.75. Conversely, a potency-decreasing CH2-O linkage characteristic of dibenzyl ether motifs is captured by MACCS but not the leading Morgan features, explaining why the two fingerprint blocks carry non-redundant, oppositely signed information. Scaffold analysis showed simple dibenzyl-ether and stilbene-like cores averaging pIC50 values of 6.3 to 7.3, while phenylpyridine and phenylpyrazine cores bearing basic amines averaged 8.4 to 10.2. These trends map directly onto the predominantly hydrophobic PD-L1 binding cleft, defined by residues such as Tyr56, Met115, and Tyr123, where halogenated aromatic extension and nitrogen heterocycles favor productive binding, and outlier compounds flagged by the model were examined through GNINA docking at this cleft against the reference inhibitor BMS-202.

The authors are candid about limits. Assay-format annotations come from ChEMBL descriptions rather than independently verified conditions, which may bias the set toward chemotypes preferentially tested in Alpha and HTRF formats, and a comparison against the published PDL1-inhi.predictor showed that an external model captured the BindingDB chemical space somewhat better. Yet the study’s contribution extends beyond any single metric: it delivers a transparent, inspectable, and adaptable platform that couples potency prediction with uncertainty quantification, supports pre-docking prioritization, and can be extended with uncertainty-guided active learning. The team suggests that high-ranking compounds within the applicability domain, especially those representing novel chemotypes, could be selected for prospective testing in the same Alpha or HTRF assays, followed by model retraining. For a field where reproducibility is often the missing ingredient, an open workflow anyone can audit and rerun on an ordinary laptop may prove as valuable as the model itself.

Subject of Research: Machine learning-guided discovery and prioritization of small-molecule PD-L1 inhibitors using an open-source KNIME workflow with curated bioactivity data and applicability-domain assessment

Article Title: Machine-learning-guided discovery and prioritization of PD-L1 inhibitors: An open-source KNIME workflow with curated bioactivity data, molecular fingerprints, and applicability-domain assessment

Article References: Sangsawat, M., Watpuang, B., Nathapinthu, T., Srisiriroj, P., Nalinratana, N., Taechawattananant, P., Thitikornpong, W., Feng, C., Chen, M., & Rojsitthisak, P. (2026). Machine-learning-guided discovery and prioritization of PD-L1 inhibitors: An open-source KNIME workflow with curated bioactivity data, molecular fingerprints, and applicability-domain assessment. Results in Chemistry, 30, Article 103877. https://doi.org/10.1016/j.rechem.2026.103877

Image Credits: AI Generated

DOI: 10.1016/j.rechem.2026.103877

Keywords: PD-L1, PD-1/PD-L1 immune checkpoint, machine learning, KNIME workflow, Random Forest, molecular fingerprints, MACCS, Morgan fingerprints, applicability domain, ChEMBL, BindingDB, cancer immunotherapy