Researchers wrestle with accuracy of AI technology used to create new drug candidates

CHAPEL HILL, N.C. – Artificial intelligence (AI) has numerous applications in healthcare, from analyzing medical imaging to optimizing the execution of clinical trials, and even facilitating drug discovery.

AlphaFold2, an artificial intelligence system that predicts protein structures, has made it possible for scientists to identify and conjure an almost infinite number of drug candidates for the treatment of neuropsychiatric disorders. However recent studies have sown doubt about the accuracy of AlphaFold2 in modeling ligand binding sites, the areas on proteins where drugs attach and begin signaling inside cells to cause a therapeutic effect, as well as possible side effects.

In a new paper, Bryan Roth, MD, PhD, the Michael Hooker Distinguished Professor of Pharmacology and director of the NIMH Psychoactive Drug Screening Program at the University of North Carolina School of Medicine, and colleagues at UCSF, Stanford and Harvard determined that AlphaFold2 can yield accurate results for ligand binding structures, even when the technology has nothing to go off of. Their results were published in Science.

“Our results suggest that AF2 structures can be useful for drug discovery,” said Roth, senior author who holds a joint appointment at the UNC Eshelman School of Pharmacy. “With a nearly infinite number of possibilities to create drugs that hit their intended target to treat a disease, this sort of AI tool can be invaluable.”

AlphaFold2 and Prospective Modeling

Much like weather forecasting or stock market prediction, AlphaFold2 works by pulling from a massive database of known proteins to create models of protein structures. Then, it can simulate how different molecular compounds (like drug candidates) fit into the protein’s binding sites and produce wanted effects. Researchers can use the resulting combinations to better understand protein interactions and create new drug candidates.

To determine the accuracy of AlphaFold2, researchers had to compare the results of a retrospective study against that of a prospective study. A retrospective study involves researchers feeding the prediction software compounds they already know bind to the receptor. Whereas, a prospective study requires researchers to use the technology as a fresh slate, and then feed the AI platform information about compounds that may or may not interact with the receptor.

Researchers used two proteins, sigma-2 and 5-HT2A, for the study. These proteins, which belong to two different protein families, are important in cell communication and have been implicated in neuropsychiatric conditions such as Alzheimer’s disease and schizophrenia. The 5-HT2A serotonin receptor is also the main target for psychedelic drugs which show promise for treating a large number of neuropsychiatric disorders.

Roth and colleagues selected these proteins because AlphaFold2 had no prior information about sigma-2 and 5-HT2A or the compounds that might bind to them. Essentially, the technology was given two proteins for which it wasn’t trained on — essentially giving the researchers a “blank slate.”

First, researchers fed the AlphaFold system the protein structures for sigma-2 and 5-HT2A, creating a prediction model. Researchers then accessed physical models of the two proteins that were produced using complex microscopy and x-ray crystallography techniques. With a press of a button, as many as 1.6 billion potential drugs were targeted to the experimental models and AlphaFold2 models. Interestingly, every model had a different drug candidate outcome.

Successful Hit Rates

Despite the models having differing results, they show great promise for drug discovery. Researchers determined that the proportion of compounds that actually altered protein activity for each of the models were around 50% and 20% for the sigma-2 receptor and 5-HT2A receptors, respectively. A result greater than 5% is exceptional.

Out of the hundreds of millions of potential combinations, 54% of the drug-protein interactions using the sigma-2 AlphaFold2 protein models were successfully activated through a bound drug candidate. The experimental model for sigma-2 produced similar results with a success rate of 51%.

“This work would be impossible without collaborations among several leading experts at UCSF, Stanford, Harvard, and UNC-Chapel Hill,” Roth said. “Going forward we will test whether these results might be applicable to other therapeutic targets and target classes.”