A new study published in the journal Molecular Diversity by Rujia Zhao, Tianzhu Shen, and Tongzhou Huang tackles this challenge with an ambitious computational pipeline that fuses machine learning, molecular docking, molecular dynamics simulations, and pharmacokinetic prediction into a single end-to-end workflow. Rather than relying on any one technique, the researchers built a layered funnel that begins with a curated dataset of 3,660 known SERT inhibitors and ends with three structurally distinct lead molecules whose predicted binding to the transporter is, on paper, stronger than that of escitalopram, one of the most widely prescribed antidepressants in the world. The work is entirely computational, but it offers a rigorous demonstration of how artificial intelligence can compress years of trial-and-error chemistry into a matter of weeks.

The foundation of the study is data curation. The team harvested bioactivity data from the ChEMBL database, targeting the human serotonin transporter under the identifier CHEMBL228, and applied strict filters: only entries reporting IC50 values with precise standard relations and nanomolar units were retained. After removing redundant structures and those lacking usable chemical notation, 3,045 unique molecules remained. To broaden the chemical diversity of the training set, the researchers manually extracted an additional 615 data points from seven representative patent documents, rendering the molecular structures in ChemDraw and cross-referencing them with reported potencies. The final dataset of 3,660 compounds was converted to pIC50 values, the negative decadic logarithm of IC50, to ensure a distribution suitable for regression modeling.

Representing molecules for machine learning required a dual strategy. Each compound was encoded with Morgan circular fingerprints, with a radius of 3 and a bit length of 1,024, which capture local atomic neighborhoods by iteratively traversing bonds and hashing the resulting substructures into fixed-length bit vectors. These fingerprints were concatenated with thirteen physicochemical descriptors, including molecular weight, lipophilicity, hydrogen bond donor and acceptor counts, topological polar surface area, rotatable bond counts, and the quantitative estimate of drug-likeness known as QED. The resulting feature vectors bridge structural topology and pharmacological behavior, giving the algorithms both the fine detail of chemical motifs and the coarse properties that govern absorption and binding.

Twelve regression algorithms were benchmarked across four families: linear and regularized models, kernel-based and instance-based methods, tree-based ensembles, and a multilayer perceptron neural network. The clear winners were the gradient-boosting and bagging ensembles. LightGBM achieved the best performance with an R-squared of 0.7138 and a mean squared error of 0.3969 on the held-out test set, followed closely by Random Forest at 0.7001 and XGBoost at 0.6920. Linear models fared poorly, with ordinary least squares regression producing particularly large prediction errors, a result the authors interpret as evidence that the structure-activity relationship of SERT inhibitors is fundamentally nonlinear. Importantly, the researchers are candid about the limits of these numbers: an R-squared near 0.7 is adequate for ranking and enriching compound libraries but not for quantitatively predicting exact potency, and the random data split used may overestimate generalization to entirely novel chemotypes. The models were therefore treated as prioritization engines rather than oracle-like activity predictors.

With three validated models in hand, the team unleashed them on a staggering library of 11,526,814 commercially available compounds drawn from eight chemical databases. A conservative consensus rule was applied: only compounds predicted by all three models to have a pIC50 greater than 7, corresponding to an estimated IC50 below 100 nanomolar, were allowed through. This filter reduced the library to 10,647 candidates. The authors acknowledge the trade-off inherent in such stringency, since true actives correctly flagged by only one or two models would be discarded as false negatives, but the consensus approach suppresses model-specific errors and enriches for genuinely high-affinity chemotypes. An applicability-domain analysis using Tanimoto similarity to the training molecules confirmed that the screening output remained chemically anchored: under a moderately inclusive threshold, more than 80 percent of the consensus candidates showed at least moderate similarity to known training compounds, while several top-ranked leads displayed lower similarity, consistent with desirable scaffold novelty.

The next stage was structure-based. The 10,647 candidates were prepared in three dimensions, with tautomers and protonation states generated at physiological pH, and docked into the crystal structure of human SERT, PDB entry 5I71, using a hierarchical Glide protocol that progressed from high-throughput virtual screening through standard precision to extra precision. This cascade winnowed the pool to 288 compounds, which were then refined with the Prime MM/GBSA method to estimate binding free energies. A cutoff of minus 55 kilocalories per mole yielded 58 candidates, and in silico ADMET profiling of those compounds ultimately produced a shortlist of 24 hits with generally acceptable predicted pharmacokinetic profiles. As a sanity check, the models predicted pIC50 values for escitalopram between 7.585 and 7.739, comfortably within the experimentally reported range of 2.1 to 55 nanomolar for that drug, lending credibility to the entire framework.

Five compounds emerged as the most promising: PB49039490 from UkrOrgSynthesis, compound 19,835,875 from ChemBridge, and Z105894806, Z2215663922, and Z310319934 from Enamine. Docking analysis revealed that all five occupy the same orthosteric central S1 pocket of SERT and share a conserved interaction network despite their diverse scaffolds. Each ligand forms ionic or hydrogen-bond contacts with Asp98, a residue long recognized as critical for antidepressant recognition, and engages in pi-pi stacking or cation-pi interactions with aromatic residues including Tyr95, Tyr176, Phe335, and Phe341. Quantitative similarity analysis against a reference panel of 1,439 experimentally validated high-affinity SERT inhibitors showed that 22 of the 24 hits had nearest-neighbor Tanimoto coefficients below 0.40, and similarity to escitalopram itself was strikingly low, with a median of just 0.089. In other words, these are genuinely novel scaffolds, not minor variations on existing drugs.

To test whether these binding poses would survive the thermal noise of a real molecular environment, the researchers ran molecular dynamics simulations in AMBER 22, solvating each protein-ligand complex in a TIP3P water box and performing three independent 100-nanosecond production runs per complex, for a total of 300 nanoseconds of sampling each. The results were reassuring. Protein backbone RMSD values equilibrated at approximately 2.4 to 3.3 angstroms over the final 50 nanoseconds without progressive drift, ligand RMSD values settled into a narrow window of roughly 0.8 to 1.3 angstroms, and the radius of gyration remained stable at about 24.0 to 24.5 angstroms, indicating no structural expansion or destabilization of the transporter. MM/GBSA binding free energies calculated from the trajectories placed Z2215663922 at minus 51.69 kilocalories per mole, compound 19,835,875 at minus 52.17, and Z310319934 at minus 44.48, all more favorable than the minus 40.25 kilocalories per mole computed for escitalopram. Per-residue energy decomposition highlighted Tyr95, Ile172, and Tyr176 as the dominant energetic contributors, with Tyr95 and Ile172 each contributing more than 4 kilocalories per mole in the Z2215663922 complex.

The study is honest about its limitations: the simulations used a single crystal structure without an explicit lipid membrane, entropy terms were omitted from the free-energy calculations, and no laboratory experiments were performed. ADMET predictions also flagged potential hERG cardiac liabilities for two candidates, Z2215663922 and Z105894806, whose predicted potassium channel inhibition values approach the conventional concern threshold. Still, the work delivers exactly what early-stage discovery needs: a transparent, multi-layered prioritization framework and three structurally diverse lead molecules that now merit in vitro validation in serotonin uptake assays, followed by selectivity profiling against the related noradrenaline and dopamine transporters. If even one of these computational leads survives the transition from silicon to bench, it would stand as a compelling case study for machine learning as a genuine engine of antidepressant drug discovery.

Subject of Research: Machine learning-based virtual screening for novel serotonin transporter inhibitors as potential antidepressant drug candidates

Article Title: Identification of novel serotonin transporter (SERT) inhibitors via large-scale machine learning-based virtual screening and molecular dynamics simulations

Article References: Identification of novel serotonin transporter (SERT) inhibitors via large-scale machine learning-based virtual screening and molecular dynamics simulations. (n.d.). https://doi.org/10.1007/s11030-026-11705-0

Image Credits: AI Generated

DOI: 10.1007/s11030-026-11705-0

Keywords: serotonin transporter, SERT inhibitors, machine learning, virtual screening, molecular dynamics, drug discovery, depression, LightGBM, XGBoost, molecular docking, MM/GBSA, ADMET