In a breakthrough that reads like a heist movie script, where an artificial intelligence learns to pick locks to nature’s most secure vaults of atomic architecture, researchers have fundamentally reengineered how we discover new crystalline materials. A team led by Hyunsoo Park and Aron Walsh at Imperial College London has developed a framework where a generative model, initially trained to propose plausible crystal structures, is subsequently guided by a reinforcement learning agent that rewards it not just for stability, but for the twin jewels of diversity and novelty. This dual-agent system, described in a landmark paper in Nature Machine Intelligence, doesn’t simply spit out variations of known materials. Instead, it actively probes the uncharted badlands of the periodic table, uncovering stable crystals that no human intuition, no brute-force simulation, and no conventional generative algorithm had ever conceived. The work shatters a longstanding bottleneck in materials science: the tendency of generative models to collapse into a narrow mode of safe, already-documented structures, ignoring the vast combinatorial wilderness where transformative materials—room-temperature superconductors, ultra-efficient battery cathodes, and next-generation thermoelectrics—are believed to be hiding.
The central drama of computational materials discovery has always revolved around a paradoxical tension between validity and creativity. Traditional methods, such as density functional theory (DFT) calculations coupled with evolutionary algorithms like USPEX, can optimize atomic configurations to find low-energy structures. Yet they are inherently local searchers, starting from known prototypes and making small, incremental adjustments—swapping an atom here, distorting a lattice there. They climb downhill on the energy landscape with remarkable precision but rarely, if ever, traverse the high-energy passes to reach a completely different valley where an even deeper, more exotic minimum awaits. Enter generative deep learning models, particularly variational autoencoders (VAEs), generative adversarial networks (GANs), and diffusion models, which learn to sample from the distribution of known stable crystals and can hallucinate entirely new ones. The problem, however, is that these models inherit the biases of their training data. Fed a diet of the Inorganic Crystal Structure Database (ICSD), they become gastronomically conservative, regurgitating compounds that look statistically similar to what has been synthesized and characterized over the past century. Park and Walsh recognized that to break this creative logjam, the generative model needed a detached, relentless critic—an artificial curator that could define an objective function for exploration itself and relentlessly push the generator into the unknown.
This is where the paper’s conceptual masterstroke resides: recasting the process of crystal generation as a sequential decision-making problem amenable to reinforcement learning. In their framework, the generative model is treated as a stochastic policy that constructs a crystal step-by-step—selecting a space group, defining lattice parameters, populating Wyckoff positions with chemical elements, and adjusting atomic coordinates. Each of these actions defines a state transition in a Markov decision process (MDP). The reinforcement learning agent, typically implemented using a policy gradient method like REINFORCE or a more advanced actor-critic architecture, then observes the fully constructed crystal and assigns a reward. Crucially, this reward function is a carefully engineered multi-objective signal that balances traditional thermodynamic stability (the energy above the convex hull, as computed by a surrogate model or DFT) with explicit metrics for diversity and novelty. Novelty is quantified by the distance to the nearest neighbor in the training set in a learned, structure-aware latent space, while diversity is measured by the coverage of the generated set across the crystal landscape. The agent then backpropagates this reward to adjust the generative policy, teaching it which sequences of atomic decisions lead to materials that are simultaneously stable, dissimilar to anything previously known, and spread across the structural spectrum.
The technical implementation is a tour de force of tight integration between crystallographic constraints and modern deep learning. The generative backbone likely leverages a crystal graph neural network or a transformer-based model that respects the invariances of crystalline systems, such as periodic boundary conditions and rotation equivariance of the lattice. Rather than generating atomic coordinates in a naive Euclidean space, the model operates on the fractional coordinates and lattice matrices, ensuring that output structures are always valid crystals. The reinforcement learning loop then superimposes a non-differentiable reward signal—since computing the convex hull distance involves solving a global optimization problem over all competing phases—onto the differentiable generator. This is handled elegantly with policy gradient methods, which estimate the gradient of the expected reward with respect to the policy parameters through Monte Carlo rollouts, allowing the system to circumvent the credit assignment problem over long sequences of actions. The authors further introduce a critical regularizer: an intrinsic motivation bonus inspired by psychological theories of curiosity, where the agent receives additional reward for visiting regions of the crystal space where its predictive model of the reward has high uncertainty. This prevents the system from getting stuck in a local rut of moderate novelty, compelling it to probe regions that are truly foreign.
To appreciate the magnitude of this achievement, one must consider the sheer immensity of the crystal structure universe. The space of possible periodic arrangements of atoms is combinatorially explosive. For a modest unit cell with 30 atoms and a choice of 50 elements from the periodic table, the number of possible configurations dwarfs the number of atoms in the observable universe. Even when filtered through the strict rules of charge neutrality, symmetry constraints, and local coordination preferences, the subspace of potentially synthesizable crystals remains mind-bogglingly vast. Traditional high-throughput screening, epitomized by projects like the Materials Project and AFLOW, has cataloged hundreds of thousands of compounds, yet this represents only a sliver surveyed along well-trodden paths of human chemical intuition. The Park-Walsh method, by contrast, is akin to equipping an explorer not with a map of known territories but with a compass whose needle points toward the unfamiliar yet stable. In their demonstrations, the reinforcement-guided generative model consistently outperformed purely generative baselines, unearthing compounds with novel stoichiometries, unexpected coordination environments, and layered structures that defied the empirical rules of thumb encoded in databases. Many of these candidates were subsequently validated by DFT, confirming their energetic viability.
One of the most compelling case studies from their work involves the discovery of previously unknown phases in the Li-Zn-N system, a chemical space of immense interest for solid-state battery electrolytes. A standard generative model trained on known nitrides predominantly churned out structures resembling the anti-fluorite or stuffed silica types that dominate the literature, implicitly assuming these motifs are nature’s only stable solutions. The reinforcement learning-driven model, however, sensed that this mode was overcrowded and learned to deliberately avoid it, instead proposing a series of compounds with intricate diamond-like frameworks where lithium and zinc atoms formed interpenetrating networks within a nitride lattice. These structures were not only new but also displayed ion migration barriers computed by nudged elastic band methods that were significantly lower than those of the benchmark materials. The agent had, in effect, autonomously deduced that channel-like topologies conducive to superionic conduction existed in a corner of the phase diagram that chemists had neglected. It was an act of artificial serendipity, guided not by chance but by a mathematically defined aspiration for novelty.
The viral potential of this research lies not merely in the new crystals it unearthed, but in the fundamental paradigm shift it heralds for the entire field of scientific discovery. For decades, the dominant philosophy of data-driven materials science has been one of prediction: train a model to approximate the quantum mechanical energy of a given structure with high fidelity, then search. The valley of success for such models is narrow, focusing on accuracy within known domains. Park and Walsh invert this philosophy, proposing that the grandest scientific prizes lie in the unknown, and that our models must be explicitly taught to seek out their own ignorance. This is an idea with profound resonance across disciplines, from drug design, where molecular generative models notoriously suffer from a lack of scaffold diversity, to the search for topological materials and quantum spin liquids. The paper’s approach provides a mathematical lingua franca for operationalizing curiosity, transforming the nebulous concept of “thinking outside the box” into a differentiable objective that a machine can relentlessly optimize over millions of iterations while researchers sleep.
Delving deeper into the algorithmic details, the paper’s reinforcement learning setup grapples with the nuance of sparse and delayed rewards. In a typical crystal generation episode, the agent commits to an entire sequence of element choices and lattice constructions before receiving the final reward, making it exceedingly difficult to discern which early decisions steered the outcome toward high diversity. The authors tackle this credit assignment problem by employing a value function network that estimates the expected future reward from any intermediate state of the partially constructed crystal. This critic network is trained jointly with the policy, using temporal difference learning to bootstrap from partial rewards that are heuristically defined at each step—for example, a penalty for selecting a combination of elements that has been heavily overrepresented in the generated set thus far. This step-level feedback transforms the generative process from a form of blind combinatorial gambling into a strategic tree search, where the agent implicitly learns to abandon unfruitful branches of the generation tree and allocate its computational mindset toward promising, risky investigations.
The architecture also features a savvy integration of crystallographic symmetry to collapse the problem’s dimensionality. Instead of generating each atom independently, which would lead to an intractable state space, the model first selects a space group from the 230 possible types, then populates the symmetry-unique Wyckoff sites. This dramatically reduces the combinatorial degrees of freedom while ensuring that every generated structure is a proper crystal with well-defined translational order. The reinforcement learning agent leverages this hierarchy: it learns that certain space groups—say, those with high multiplicity general positions—are flexible playgrounds that permit vast atomic substitutions and yield high diversity, whereas tightly packed, low-symmetry groups tend to produce incremental variations on known themes. Over the course of training, the policy develops a non-uniform preference for space groups that are underexplored in the database, thus acting as an AI geometer that redraws the map of allowable symmetries.
What truly sets this work apart from prior attempts to inject diversity into generative models—such as simple temperature sampling, latent space interpolation, or adversarial training—is that reinforcement learning provides a principled mechanism for optimizing non-differentiable objectives over sequential construction. While a GAN can be trained to fool a discriminator into believing generated crystals come from a dataset, the discriminator’s notion of novelty is myopically tied to the training distribution; it encourages the generator to fill gaps within the distribution, not to leap outside it. The Park-Walsh RL framework, with its explicit distance-based novelty bonus, turns the generator into a cartographer of the unknown, incentivized to stake claims as far from any training example as possible, provided stability is maintained. This results in a phase transition in the model’s behavior: where conventional generative models exhibit a sharp drop in validity as they are pushed to extrapolate, the RL-trained generator maintains a high fraction of stable outputs even as it ventures into structurally unprecedented territories.
The implications for battery technology alone are staggering, given the world’s insatiable hunger for dense, safe energy storage. Consider the search for solid electrolytes to replace flammable liquid electrolytes in lithium-ion batteries. The design space includes elements from across the p-block of the periodic table—sulfur, phosphorus, halogens—combined with lithium and a host of framework cations. The sheer number of possible quaternary and quinary compositions is astronomical, and each composition may adopt dozens of distinct structure types. Screening this space with DFT would require a metropolis of supercomputers running for years. The RL-guided generative model, by contrast, acts as a computationally frugal scout, sending out probes into the most promising, uncharted regions and returning with a select few candidates for DFT validation. In their lithium thiophosphate case study, the method rediscovered the well-known argyrodite and LGPS-type superionic conductors, but more importantly, it identified completely new families where the traditional distinction between framework and mobile-ion sublattices breaks down, yielding a quasi-liquid arrangement of lithium within a crystalline scaffold—a condition theorized to unlock room-temperature superionic conductivity.
The paper also introduces a pivotal concept of “diversity-conditioned” generation, which goes beyond finding one oddity. The reinforcement learning reward function includes an entropy term computed over a batch of generated structures, encouraging the policy to produce a portfolio of materials that are mutually distinct. This batch-wise optimization, reminiscent of how a venture capitalist builds a diversified portfolio of high-risk, high-reward startups, prevents the model from simply finding a single novel pocket and then overexploiting it. Instead, the agent learns to partition its exploration budget across the unexplored frontier of phase space, systematically plumbing the stability of novel quasicrystals, incommensurate phases, and layered heterostructures. It is a form of collective intelligence where each crystal’s value depends on the company it keeps, pushing the frontier of materials knowledge outward in a coordinated, multi-pronged advance.
The validation pipeline is as rigorous as the generation strategy is flashy. Candidate crystals from the RL-guided model are first relaxed using a fast universal interatomic potential, such as a graph network trained on the Materials Project, to filter out catastrophically unstable geometries. Survivors are then subjected to full DFT geometry optimizations with stringent convergence criteria, and their energies are plotted relative to the convex hull of all known competing phases in that chemical space. A material is deemed stable if it lies within a few millielectronvolts per atom of the hull, the threshold for possible experimental synthesis under controlled conditions. The authors report that the RL-enhanced generator maintains a hull distance distribution that is statistically indistinguishable from that of the baseline generative model, debunking any notion that pushing for novelty necessarily sacrifices stability. This is the holy grail: expanding the Pareto frontier of stability and novelty simultaneously.
The broader intellectual lineage of this work connects to the emerging field of “artificial curiosity” and intrinsic motivation in artificial intelligence, made famous by the game-playing agents that mastered Montezuma’s Revenge by rewarding themselves for seeing new pixels. Materials science, it turns out, is a far more formidable game than any Atari console. The state space is continuous, high-dimensional, and governed by the cruel and subtle laws of quantum mechanics. By demonstrating that curiosity-driven reinforcement learning can be tamed and directed in this domain, Park and Walsh provide a blueprint for how the next generation of AI scientists will operate: not as passive predictors but as active explorers, generating their own hypotheses and designing their own experimental validation loops. The paper is poised to become a founding text for a new subfield at the intersection of autonomous materials design and active learning, where the algorithm’s core mandate is not to imitate human knowledge but to surpass it.
The authors are careful to delineate the current limitations, which serve as a roadmap for the community. The reinforcement learning reward function, while effective, relies on a battery of heuristic choices—the distance metric in latent space, the coefficient balancing stability versus novelty, and the entropy penalty. Tuning these hyperparameters for a new chemical family remains more art than science, and a poorly chosen metric can drive the agent toward an island of false novelty, where generated structures are distinct in a trivial sense but chemically nonsensical. Furthermore, the surrogate models that predict energy in the inner loop must be both fast and accurate enough to not mislead the policy. The authors foresee a future where the entire loop is tightened, with the reinforcement learning agent directly controlling a robotic synthesis platform, feeding back not just computational but real experimental observables—a closed-loop autonomous laboratory hunting for functional crystals in real time.
Looking at the scientific reaction already brewing, the paper has struck a nerve. In conference halls and departmental seminars, materials scientists and computer scientists alike are buzzing about the audacity of the approach. The method doesn’t just produce a list of new materials; it produces a new kind of AI behavior, one that is reminiscent of the exploratory play witnessed in humans and animals. There is something deeply satisfying about watching an algorithm, initially fumbling in the dark, gradually develop a taste for the exotic—learning to steer clear of sodium chloride look-alikes and boldly venture into the territory of penta-silicides and inverse-halide perovskites that textbooks never predicted. This narrative of an AI developing a form of scientific aesthetic is what will propel this research into the broader public imagination, sparking discussions about machine creativity and the role of human scientists in an age when the frontier of knowledge is increasingly mapped by digital minds.
The open-source release of the codebase and the trained policies, a hallmark of the Walsh group’s commitment to open science, ensures that this viral idea will immediately propagate into laboratories worldwide. A graduate student working on thermoelectrics can now download the model, specify a target chemical system, and within hours have a list of candidate crystals that are both stable and structurally fresh, curated by a tireless reinforcement learning agent. This democratization of high-level discovery tools is as important as the algorithmic advance itself. It changes the nature of graduate education in materials science, moving from an era where students mainly learned to run DFT calculations on hand-chosen candidates to one where they train and interrogate generative agents, interpreting the surprising outputs as profound suggestions from a non-human intelligence that has combed spaces they could not.
The philosophical implications are as weighty as the practical ones. For centuries, the discovery of new materials has been guided by human pattern recognition—the seasoned intuition that elements in the same column behave similarly, that certain radius ratios favor certain coordinations, and that certain packing motifs reappear. This intuition, crystallized in the Pauling rules and the work of generations of solid-state chemists, has been an astonishingly powerful guide. Yet it is also a cage, steering us toward what we already half-know. The Park-Walsh AI, unburdened by textbook prejudice, proposes structures that bend or break these rules, revealing that the space of permissible crystal chemistry is far more permissive and strange than we imagined. Some of these structures may be laboratory dead ends, too kinetically inaccessible to ever nucleate. But others will undoubtedly seed new families of materials, and the process of sifting through the AI’s dreams will itself strain and enrich our understanding, forcing us to articulate why certain configurations that are computationally stable have never been seen in nature.
The work invites us to reimagine the role of the human scientist as a curator and interrogator of machine-proposed hypotheses, a relationship far more thrilling than the old model of human as calculator. Instead of spending months on a single chemical intuition that may be a dead end, a researcher can now set the objective of a novel ion conductor, specify the wildness of the exploration, and then pour their cognitive effort into understanding why the AI proposed a peculiar layered perovskite with an unorthodox cation ordering. That reverse-engineering process—extracting new chemical rules from the machine’s surprises—constitutes a new kind of dialogue, a Socratic method for the periodic table where the machine proposes, and the human, in awe, learns. This research, therefore, is not just an engineering tool; it is a telescope for the mind, bringing into focus the vast nebula of structural possibility that lies beyond the narrow spectrum of our current synthesized world, and daring us to step inside and explore.
Subject of Research: A computational framework that integrates reinforcement learning with deep generative models to discover diverse and novel crystal structures, explicitly rewarding the generation of materials that are both thermodynamically stable and structurally distinct from known databases.
Article Title: Guiding generative models to uncover diverse and novel crystals via reinforcement learning
Article References:
Park, H., Walsh, A. Guiding generative models to uncover diverse and novel crystals via reinforcement learning.
Nat Mach Intell (2026). https://doi.org/10.1038/s42256-026-01262-4
Image Credits: AI Generated
DOI: https://doi.org/10.1038/s42256-026-01262-4
Keywords: Crystal structure prediction, generative models, reinforcement learning, materials discovery, diversity-driven generation, autonomous laboratories, solid-state batteries, structure novelty, policy gradient, computational materials science.
