The analogy is more than a metaphor. DNA is written in an alphabet of four nucleotide letters, proteins in an alphabet of twenty amino acids, and RNA in a four-letter code of its own. Just as words derive meaning from their context in a sentence, the function of a nucleotide or amino acid depends on its position within a sequence and on the sequences that surround it. The transformer architecture, introduced in 2017 in the landmark paper Attention Is All You Need, excels precisely at capturing such long-range contextual dependencies. By applying self-attention mechanisms across millions of biological sequences, models can learn statistical patterns that reflect evolutionary constraints, structural motifs, and functional sites without any explicit programming.

The review organizes the field into biological foundation models, general-purpose systems pretrained on massive unlabeled sequence databases, and specialized models tailored to particular problems. On the protein side, models such as ProtTrans and the Evolutionary Scale Modeling family demonstrated that unsupervised learning on hundreds of millions of sequences yields internal representations from which structure and function can be read off. This line of work culminated in AlphaFold and its successors, which achieved atomic-level accuracy in protein structure prediction and, with AlphaFold 3, extended to predicting interactions between proteins, nucleic acids, and small molecules. Generative protein LLMs such as ProGen and ProGen2 have gone further, producing entirely novel amino acid sequences that fold into functional proteins, a capability the review highlights as one of the most consequential advances of the past decade.

DNA language models have followed a parallel trajectory. The Nucleotide Transformer, built and evaluated on human and multi-species genomes, showed that robust foundation models could be trained at nucleotide resolution. Subsequent work demonstrated that such models are powerful predictors of genome-wide variant effects, meaning they can estimate whether a mutation in a patient’s genome is likely to be harmful, a task central to clinical genetics. The review also points to Evo and its successor Evo 2, genome-scale models capable of modeling and designing sequences across all domains of life, from bacteria to humans. These systems operate at a scale that was unthinkable for genomics only a few years ago, processing contexts long enough to encompass entire genes and their regulatory neighborhoods.

RNA has proven a particularly fertile ground because its structure and function are notoriously difficult to predict from sequence alone. Classical approaches relied on thermodynamic folding algorithms, but deep learning models such as UFold and RNAformer have shown that learned representations can match or exceed physics-based methods for secondary structure prediction. More recently, language model-based deep learning approaches have achieved accurate RNA 3D structure prediction, and motif-aware pretraining strategies have produced multi-purpose RNA models that handle splicing prediction, binding site identification, and function annotation within a single framework. The review notes that self-supervised learning on millions of primary RNA sequences from dozens of vertebrate species has substantially improved sequence-based splicing prediction, with direct implications for interpreting disease-causing mutations in untranslated regions and splice sites.

Beyond the three canonical sequence types, the review surveys a rapidly growing ecosystem of specialized applications. Interaction prediction models forecast which proteins bind which RNAs, which RNAs pair with other RNAs, and how mutations perturb these contacts. Structure prediction systems now cover proteins, RNA, and biomolecular complexes at near-experimental accuracy. Perhaps most striking is the single-cell revolution: foundation models such as scGPT, scBERT, and Geneformer treat gene expression profiles across thousands of individual cells as a kind of language, learning universal representations that support cell type annotation, perturbation response prediction, and the integration of massive transcriptomic datasets. Studies assessing GPT-4 for cell type annotation suggest that general-purpose chatbots can also contribute, though specialized models currently hold the edge on most benchmarks.

The authors are candid about the obstacles. Hallucination, the tendency of generative models to produce plausible but false outputs, is a documented failure mode of natural language generation that carries serious risks when outputs inform drug design or clinical interpretation. Benchmarking remains immature: evaluation suites such as DART-Eval for regulatory DNA and PertEval-scFM for perturbation prediction reveal that performance gains do not always transfer across datasets and cell types. Computational cost is another barrier, since genome-scale attention models demand enormous memory, motivating architectural innovations such as the Hyena hierarchy and Mamba-based state space models that the review identifies as promising directions for long-context biological modeling. Data quality, batch effects in single-cell experiments, and the interpretability of learned representations round out the open challenges.

What distinguishes this review from many surveys of artificial intelligence in biology is its educational dimension. Because it appears in a journal focused on digital education, the authors devote substantial attention to integrating LLMs into genomics teaching and learning. They propose practical projects in which students fine-tune pretrained models on small datasets, use conversational agents to simplify bioinformatics workflows, and learn to critically evaluate model outputs. Tools such as BioMANIA, which allows researchers to interrogate bioinformatics pipelines through natural language conversation, and BioCoder, a benchmark for bioinformatics code generation, illustrate how LLMs can lower the technical barriers that traditionally separate biologists from computational analysis. The authors argue that fluency in these tools is becoming as essential to the modern life scientist as statistical literacy.

The trajectory the review describes is remarkably compressed. In roughly four years, language models have progressed from proof-of-concept embeddings to systems that simulate 500 million years of evolutionary history, design functional proteins, predict the pathogenicity of genetic variants, and annotate cell atlases containing millions of cells. The Wuhan University team frames the current moment as an inflection point: the same scaling laws and reasoning enhancements that drive progress in general-purpose AI, including reinforcement learning approaches that sharpen chain-of-thought reasoning, are poised to accelerate biological discovery further. If the challenges of reliability, evaluation, and interpretability can be met, the authors suggest, the fusion of language models and genomics may redefine both how biology is researched and how the next generation of scientists is trained, turning the genome from an inscrutable code into a text that machines can read, annotate, and even rewrite.

Subject of Research: Applications of large language models in genomics, including biological foundation models for protein, DNA, and RNA sequence analysis

Article Title: AI-Empowered Genome Decoding: Applications of Large Language Models in Genomics

Article References: Li, S., Fan, W., & Zhou, Y. (2025). AI-Empowered Genome Decoding: Applications of Large Language Models in Genomics. Frontiers of Digital Education, 2(1), Article 14. https://doi.org/10.1007/s44366-025-0051-1

Image Credits: AI Generated

DOI: 10.1007/s44366-025-0051-1

Keywords: large language models, genomics, deep learning, protein structure prediction, DNA language models, RNA secondary structure, single-cell analysis, foundation models, AlphaFold, transformer architecture, bioinformatics education, variant effect prediction