The system, called NucleicBERT, applies a transformer-based language model to ribonucleic acid sequences, training it to predict masked positions in nucleotide strings drawn from large public databases. The approach deliberately avoids labels. Instead of being told which sequences are ribozymes, which are microRNAs, or which bind particular proteins, the model is simply asked to fill in the blanks across millions of natural sequences. In doing so, it is forced to internalise the statistical regularities of real RNA: which nucleotides tend to co-occur, which motifs recur across distant branches of life, and which combinations essentially never appear. Those patterns, the authors contend, encode a compressed representation of the physical and evolutionary constraints that shape functional RNA.

The technical foundation is the bidirectional encoder architecture popularised by models such as BERT. In natural language, such models read text in both directions and learn contextual embeddings, so that the meaning of a word depends on its neighbours. NucleicBERT imports that idea wholesale into molecular biology. Each nucleotide in an RNA sequence is treated as a token, and the encoder produces a vector for every position that reflects its biological context. A cytosine embedded in a stem-loop of a transfer RNA acquires a different representation from the same cytosine sitting in the loop of a riboswitch, even though the raw letter is identical. This context sensitivity is precisely what hand-crafted features, position-weight matrices and simple motif searches have historically lacked.

Pretraining proceeds with a masked-language objective. Random positions in each training sequence are hidden, and the model must reconstruct them from surrounding context. Because the training corpus spans diverse RNA families and organisms, the network cannot succeed by memorising shallow patterns; it must capture deeper regularities such as compensatory mutations in paired regions, conserved loops, and the compositional biases of different RNA classes. The resulting embeddings can then be transferred downstream: a relatively small amount of labelled data is sufficient to fine-tune the pretrained network for specific prediction tasks, a strategy that has transformed fields from computer vision to protein biochemistry.

The practical payoff comes in the form of benchmark performance on tasks that matter to RNA biologists. According to the paper, NucleicBERT embeddings improve predictive accuracy on problems including the classification of non-coding RNA families, the identification of RNA-binding protein sites, and the assessment of sequence variants that disrupt splicing or translation. In each case the pretrained model outperforms baselines trained from scratch on the same labelled data, and the advantage is largest precisely where labelled examples are scarcest. That pattern is the classic signature of useful pretraining: the model arrives at a task already fluent in the underlying vocabulary, so supervision only needs to teach the final grammar.

What makes the work conceptually significant is not merely the benchmark numbers but the interpretability experiments layered on top of them. The authors probe what the model has learned by examining attention patterns and embedding geometry. Sequences with related structures and functions cluster together in the embedding space even when their nucleotide identities differ substantially, suggesting the model has discovered homology that raw sequence comparison misses. Attention heads, the internal components that let a transformer weigh relationships between positions, turn out to concentrate on regions that biologists recognise as structurally or functionally meaningful, such as paired stems and conserved catalytic motifs. In effect, the network rediscovers, from raw data alone, some of the hard-won knowledge that RNA biochemists assembled over half a century.

The study also confronts one of the central puzzles of RNA biology: the sheer size of sequence space. An RNA molecule of only 100 nucleotides has 4 to the power of 100 possible sequences, a number that dwarfs the number of atoms in the observable universe. Natural RNA occupies a vanishingly sparse subset of that space, organised into families shaped by common ancestry and common physics. Language models are, in a formal sense, tools for modelling the distribution of data, and NucleicBERT can therefore be read as a statistical map of where functional RNA lives within the vast combinatorial wilderness. Sequences the model assigns high likelihood are, heuristically, sequences that look like biology; sequences it assigns low likelihood are candidates for exotic synthetic designs, or for failure.

That map has immediate applications in engineering. RNA therapeutics, from messenger RNA vaccines to small interfering RNAs and antisense oligonucleotides, all depend on the properties of sequence: how stably a molecule folds, how efficiently it is translated, how recognisable it is to the innate immune system, and how long it survives in the cell. The authors report that NucleicBERT representations correlate with measurable properties such as secondary-structure stability and expression level, offering drug developers a way to screen and optimise candidate sequences in silico before expensive synthesis and testing begin. The same representations can guide the design of synthetic riboswitches and regulatory elements for synthetic biology, where designers currently iterate through costly build-and-test cycles.

The researchers are candid about limitations. RNA databases are biased towards well-studied model organisms and abundant RNA classes, so the model’s fluency is strongest where data are richest and weakest for rare transcripts and poorly characterised clades. The masked-language objective captures linear sequence context directly and higher-order structure only indirectly, so tasks that hinge on detailed three-dimensional folding may still require complementary physics-based or structure-specific models. And like all deep networks, NucleicBERT offers correlations rather than mechanisms: its embeddings are a powerful substrate for prediction, but turning them into causal explanations of why a particular fold catalyses a particular reaction remains future work. The authors frame the model not as a replacement for biochemical experiment but as a hypothesis engine that tells experimentalists where to look.

Even with those caveats, the arrival of a mature nucleic-acid language model marks a turning point in how the life sciences approach sequence data. For twenty years, genome annotation has leaned on alignment-based tools that compare new sequences against known ones, a strategy that fails for molecules with no recognisable relatives. Self-supervised models offer a different epistemology: knowledge distilled from the totality of observed sequences, applicable even to orphans with no evolutionary cousins. As sequencing technologies continue to generate data far faster than any human can annotate them, systems like NucleicBERT are likely to become standard equipment in the computational biology toolkit, reading the genome’s least understood language at a pace no human reader could match and pointing the way to RNA molecules that biology has not yet invented.

Subject of Research: Self-supervised language modelling of RNA sequence space with the NucleicBERT neural network

Article Title: NucleicBERT interprets RNA sequence space through self-supervised language modelling

Article References: Upadhyay, U., Herold, J., Götz, M., & Schug, A. (2026). NucleicBERT interprets RNA sequence space through self-supervised language modelling. Nature Machine Intelligence. https://doi.org/10.1038/s42256-026-01295-9

Image Credits: AI Generated

DOI: 10.1038/s42256-026-01295-9

Keywords: NucleicBERT, RNA, self-supervised learning, language models, machine learning, transformers, non-coding RNA, RNA therapeutics, sequence biology, computational biology, embeddings, RNA structure

The research, led by Ying Li of Kunming University of Science and Technology in Yunnan, China, together with colleagues at the same institution and at Yunnan Observatories of the Chinese Academy of Sciences, introduces a tracking architecture called BF-TrackFormer. The name reflects its core innovation: a bidirectional, autoregressive approach to tracking that works both forward and backward in time. The team built the system specifically to address a problem that has long frustrated automated filament monitoring, namely that filaments do not behave like tidy, well-behaved objects. They split, merge, twist, and fragment, and conventional tracking algorithms routinely lose track of them when their outlines break apart or when a detector momentarily fails to recognize them.

To train and evaluate the new method, the researchers assembled a dataset of 630 H-alpha full-disk solar images drawn from three distinct observational sources: the Big Bear Solar Observatory in California, the Solar Magnetic Activity Research Telescope operated by Kyoto University in Japan, and the Global Oscillation Network Group, a worldwide network of six stations operated by the U.S. National Solar Observatory that provides nearly continuous coverage of the Sun. The dataset was divided into one training set and five testing sets, a deliberate design choice meant to expose the algorithm to different data sources, time intervals, and observing conditions. This diversity matters because H-alpha imagery varies considerably from one instrument to another in resolution, calibration, and image quality, and a tracking system that only works on data resembling its training set would be of limited practical value for operational forecasting centers.

The technical foundation of BF-TrackFormer builds on TrackFormer, a transformer-based multi-object tracking framework originally developed for computer vision applications. Transformers, the same architectural family that powers modern language models, process images by attending to relationships between different regions of the frame, allowing a detector to identify objects and assign them consistent identities across successive frames in a single pass. The Chinese team adapted this paradigm to the peculiar challenges of solar imagery, where filaments appear as dark absorption features against the bright chromosphere and where their boundaries can be diffuse, elongated, and highly irregular compared with the compact objects typical of pedestrian or vehicle tracking benchmarks.

The genuinely novel component is the backward autoregressive branch, which consists of two modules the authors call the ID Embedding Head and the Recheck Network. In a standard forward-only tracker, each new frame is processed in sequence, and objects are matched to tracks established in previous frames. If a filament splits into two pieces, the detector may capture only one fragment, or if a filament temporarily fades below the detection threshold, the track is broken and the object reappears later as a brand-new entity with a new identity. The backward branch effectively gives the algorithm a second chance: it revisits frames where objects were missed or misclassified as background, re-embedding identity information so that fragments of a split filament can be reassociated with their parent track, and discontinuous segments of a single filament’s lifecycle can be stitched back together into one coherent trajectory.

This capability is scientifically significant because fragmented filament evolution is not merely a nuisance for software; it is a physical signal. Observational studies in recent years have documented that filament splitting, bifurcation, and reconstruction are often driven by magnetic reconnection and the emergence of new magnetic flux, processes that can precede or accompany eruption. A filament that breaks apart in H-alpha images may be undergoing the very magnetic restructuring that signals instability. If an automated system fragments a single physical filament into several spurious tracks, or loses it entirely at the critical moment, the observational clues that forecasters and researchers rely on are corrupted. By maintaining identity through splitting events, BF-TrackFormer preserves the integrity of the lifecycle record, which is precisely what statistical studies of filament behavior and eruption prediction models require as input.

The performance figures reported in the paper illustrate both the promise and the remaining difficulty of the task. Averaged across the five testing sets, the method achieved an IDF1 score of 70.1 percent, a measure of how consistently identities are maintained over time; a MOTA of 46.6 percent, the standard multi-object tracking accuracy metric that combines false positives, misses, and identity switches; precision of 75.1 percent; recall of 70.9 percent; and an identity survival rate of 10.1 percent. The authors note a clear trend: as the time interval between consecutive images decreases, all of these metrics improve. This makes intuitive sense, since filaments evolve slowly by human forecasting standards but can change enough between widely spaced frames to confuse any tracker, and denser temporal sampling gives the autoregressive machinery more reliable continuity to work with.

The relatively low identity survival rate underscores how hard the problem remains. Filaments can persist on the Sun for days or weeks, and tracking one across its full lifetime requires surviving changing illumination, projection effects as solar rotation carries features across the disk, overlapping structures, and the episodic fragmentation that motivated the study in the first place. Even so, the authors report that BF-TrackFormer performs particularly well on fragmented filaments, the case that conventional methods handle worst. In the ecosystem of automated solar monitoring, where earlier systems relied on classical image processing techniques such as morphological operations, region growing, and hand-tuned thresholds, the shift to learned, identity-aware tracking represents a generational change in how filament catalogs can be built.

The broader context is the growing recognition that space weather is an operational hazard demanding the same vigilance as terrestrial weather. Filament eruptions are closely associated with coronal mass ejections, the massive expulsions of plasma and magnetic field that are the primary drivers of major geomagnetic storms. Statistical studies spanning multiple solar cycles have established that a large fraction of filament disappearances coincide with coronal mass ejections, making filaments one of the most informative precursors available to forecasters. Automated detection and tracking systems, running continuously on ground-based H-alpha networks, can in principle flag filament activation, disappearance, and fragmentation in near real time, extending the reach of human observers who cannot watch every active region simultaneously.

The authors acknowledge the contributions of the observatories whose data made the work possible, including Big Bear Solar Observatory, the Global Oscillation Network Group, the Solar Magnetic Activity Research Telescope, and the CHASE satellite mission supported by the China National Space Administration. As solar maximum approaches its most active phases and humanity’s dependence on space-based infrastructure deepens, tools like BF-TrackFormer point toward a future in which the Sun’s most dangerous structures are watched continuously, their every split and twist logged by algorithms that never blink, and their eruptive potential assessed with the statistical rigor that only complete, unbroken lifecycle records can provide. The research is a reminder that forecasting the storm begins with faithfully following the calm before it.

Subject of Research: A bidirectional autoregressive deep learning method for detecting and tracking solar filaments across their lifecycles to improve space weather forecasting.

Article Title: Bidirectional Autoregressive Tracking Method for Solar Filaments

Article References: Li, Y., Yang, Y., Zhang, X., Feng, S., Dai, W., Liang, B., & Xiong, J. (2026). Bidirectional Autoregressive Tracking Method for Solar Filaments. Solar Physics, 301(9), Article 136. https://doi.org/10.1007/s11207-026-02737-8

Image Credits: AI Generated

DOI: 10.1007/s11207-026-02737-8

Keywords: solar filaments, space weather, deep learning, multi-object tracking, H-alpha imaging, coronal mass ejections, BF-TrackFormer, solar physics, transformers, Big Bear Solar Observatory, GONG, filament eruptions