The research, led by Alan Victor de Souza Pinho and Rosalvo Ferreira de Oliveira Neto of the Federal University of San Francisco Valley, together with Edilson Beserra de Alencar Filho, tackles a stubborn bottleneck in environmental risk assessment. Experimental toxicity testing is expensive, slow, and ethically fraught, and for pollinators the available data are especially sparse. Quantitative structure-activity relationship, or QSAR, modeling offers a computational alternative, predicting a compound’s biological effects directly from its molecular structure. But QSAR models are only as good as the molecular representations fed into them, and building those representations traditionally requires laborious feature engineering with specialized descriptor libraries.

The team drew its data from ApisTox, a recently released open-access benchmark containing curated toxicity information for 1,035 compounds, of which 296 are classified as toxic to honey bees and 739 as non-toxic. The classification follows the United States Environmental Protection Agency’s regulatory convention: a compound is deemed toxic if its acute LD50 value, whether by oral or contact exposure, is at or below 11 micrograms per bee. The dataset aggregates information from authoritative sources including the EPA’s ECOTOX knowledgebase and the Pesticide Properties DataBase, and for each chemical it retains the most toxic exposure route, a conservative strategy designed to capture the highest potential risk to pollinators.

Into this benchmark the researchers introduced three competing ways of describing molecules. The first used PaDEL, a widely adopted open-source software that calculates hundreds of handcrafted molecular descriptors. The second used Morgan fingerprints computed with RDKit, circular substructure patterns that encode the atomic neighborhoods of each molecule at fixed radii. The third and most novel approach extracted embeddings from MolFormer, a BERT-style chemical language model developed by IBM that treats SMILES strings, the text-based notation for molecular structures, as a language. MolFormer was pretrained on millions of molecules from the PubChem and ZINC databases using a masked language modeling objective, learning to predict hidden tokens from their context and thereby absorbing structural, semantic, and physicochemical information without any labeled toxicity data.

Crucially, the researchers did not fine-tune the massive model. Instead, they adopted a feature-based transfer learning strategy, using the pretrained network as a frozen feature extractor that converts each SMILES string into a dense 768-dimensional vector. These embeddings then served as input to three classical machine learning classifiers: Random Forest, Support Vector Machine, and a Multilayer Perceptron. All models were evaluated with five-fold cross-validation using the area under the receiver operating characteristic curve, or ROC-AUC, a threshold-independent metric well suited to the dataset’s class imbalance. Notably, the embeddings came from a publicly available reduced-scale MolFormer checkpoint trained on only about 100 million molecules, roughly ten percent of the combined ZINC and PubChem corpora, since the full-scale checkpoints trained on the reported 1.1 billion molecules are not publicly available.

The results delivered a clear verdict. Random Forest paired with Morgan fingerprints achieved the best overall discrimination, with a mean ROC-AUC of 0.866, confirming that substructure-based representations remain formidable baselines. Yet Support Vector Machine combined with MolFormer embeddings came astonishingly close, reaching a ROC-AUC of 0.859, a gap of just 0.007. Considering that the embeddings originated from a comparatively lightweight pretraining regime, the near-parity is striking, and it raises an tantalizing question the authors pose explicitly: had the full-scale model trained on the complete 1.1-billion-molecule corpus been accessible, the remaining gap might have narrowed further or even reversed.

Against the older PaDEL descriptors, the verdict was unambiguous. MolFormer embeddings outperformed PaDEL across all three classifiers, with ROC-AUC gains ranging from 0.005 to 0.021, the largest improvement appearing for the Support Vector Machine. The authors attribute this to the alignment between MolFormer’s learned representations and margin-based decision boundaries. More fundamentally, the advantage reflects a difference in representational philosophy. Morgan fingerprints encode the presence of local substructures without capturing the global molecular context in which a functional group sits, and their binary hashing is vulnerable to bit collisions. MolFormer’s bidirectional self-attention, by contrast, encodes every atomic position in relation to all other tokens in the SMILES sequence simultaneously, producing a holistic representation in which toxicophoric groups are described within the context of the complete molecular architecture.

The qualitative analysis of individual predictions illuminated this distinction vividly. Examining cases where the embedding-based model succeeded while the fingerprint-based model failed, the researchers found compounds rich in phosphate esters and carbamate groups, or polyhalogenated aromatic systems, structural motifs associated with acetylcholinesterase inhibition and modulation of neuronal ion channels, the classic mechanisms of insecticidal action. Conversely, when the fingerprint model won, the compounds tended to belong to well-defined agrochemical classes with established low bee toxicity, such as triazine herbicides and triazole fungicides, where simple class-defining substructures are the dominant signal and local encoding suffices. The two models, in other words, see different things in the same molecules.

That complementarity was quantified as well. Although the two best models achieved similar average performance, the Pearson correlation between their predicted probabilities was only 0.7483, indicating partially overlapping but non-identical predictive patterns. In some cross-validation folds the correlation dropped as low as 0.6417 without any degradation in predictive accuracy, evidence that each representation captures distinct aspects of chemical space. The authors argue that this makes the embedding-based model a strong candidate for ensemble frameworks, in which majority voting, probability averaging, or stacking with a meta-learner could combine the strengths of both representations, potentially alongside graph neural network architectures, to build unified predictive platforms accessible through online tools and mobile applications.

The practical implications extend well beyond the leaderboard. By eliminating the need to integrate multiple descriptor calculation libraries and perform complex variable selection, the transfer learning approach substantially simplifies the modeling pipeline, lowering the barrier for ecotoxicological screening under limited labeled data, precisely the conditions that prevail in pollinator protection. The authors caution that their models were developed and evaluated exclusively within the chemical space of the ApisTox dataset, and they recommend formal applicability domain analysis, using tools such as leverage statistics or nearest-neighbor distances in embedding space, before any regulatory deployment. Future work, they note, should explore full-scale pretrained models and incorporate chronic toxicity endpoints. As honey bee populations continue to decline worldwide under the combined pressures of pesticide exposure, habitat loss, and climate change, tools that can rapidly and reliably flag dangerous chemicals before they reach the field may prove not just convenient but essential.

Subject of Research: Transfer learning with chemical language models for predicting honey bee toxicity

Article Title: Transfer learning for honey bee toxicity prediction: MolFormer versus classical QSAR representations

Article References: de Souza Pinho, A. V., de Alencar Filho, E. B., & de Oliveira Neto, R. F. (2026). Transfer learning for honey bee toxicity prediction: MolFormer versus classical QSAR representations. Ecotoxicology, 35(7), Article 166. https://doi.org/10.1007/s10646-026-03149-x

Image Credits: AI Generated

DOI: 10.1007/s10646-026-03149-x

Keywords: transfer learning, MolFormer, QSAR, honey bee toxicity, Apis mellifera, ApisTox, chemical language models, Morgan fingerprints, PaDEL descriptors, ecotoxicology, pesticide risk assessment, machine learning