Large language models have reached a striking new milestone in astronomy: on several of the world’s most demanding student examinations, the best-performing systems achieved results within the gold-medal range. But a detailed analysis shows that the apparent breakthrough comes with an important warning. Although advanced models can solve many difficult theoretical astronomy problems, they remain unreliable at the geometric reasoning, spatial visualization and conceptual interpretation required for genuine scientific research.
The findings come from a systematic benchmark of five state-of-the-art large language models on examinations from the International Olympiad on Astronomy and Astrophysics, or IOAA. Unlike conventional AI tests that ask short factual questions, the Olympiad papers require students to combine physics, mathematics and astronomy across multiple stages of reasoning. Problems may involve deriving equations, interpreting physical systems, analyzing observations and translating a diagram or image into a quantitative solution. The exams therefore provide a more demanding test of whether an AI system can reason through astronomy rather than simply recall astronomical facts.
Researchers evaluated the models on four IOAA theory examinations held between 2022 and 2025, as well as on data-analysis examinations. The theory papers are designed to probe the foundations of astrophysics, including mechanics, gravitation, radiation, celestial coordinates and the behavior of astronomical systems. Solving such problems generally requires more than applying a familiar formula. A model must identify the relevant physical assumptions, select a mathematical approach, follow a chain of deductions and check whether the final result is consistent with the situation described.
Two models stood out in the theoretical tests. Gemini 2.5 Pro achieved an average score of 85.6 percent across the four examinations, while GPT-5 reached 84.2 percent. Those results placed both systems at a level comparable to, or higher than, the strongest students in examination groups of approximately 200 to 300 participants. According to the researchers, the scores fall within the range typically associated with gold-medal performance, suggesting that frontier AI systems can now handle many Olympiad-style astronomy problems with a degree of competence once associated only with elite human competitors.
The result is particularly significant because Olympiad problems are not ordinary classroom exercises. They often conceal the essential physics inside a complicated description, requiring the solver to decide what can be neglected and what must be retained. A successful solution may depend on recognizing a symmetry, selecting an appropriate reference frame, interpreting a limiting case or connecting an observed quantity to an underlying physical parameter. In this setting, a correct answer is evidence of substantial problem-solving ability, even when the model reaches it through a different internal process than a human student.
Yet high average scores do not mean that the models have mastered astronomy in a general sense. Performance varied substantially from problem to problem, and the researchers’ error analysis identified recurring weaknesses across all of the systems. Conceptual reasoning, geometric reasoning and spatial visualization remained difficult, with accuracy in these categories ranging from roughly 52 to 79 percent. These failures are important because they can appear even when a model knows the relevant equations. An AI may reproduce a familiar formula correctly but misunderstand the physical arrangement of objects, confuse an angle or distance, or apply an equation outside the conditions in which it is valid.
Spatial reasoning is especially central to astronomy. Many astronomical systems cannot be examined directly and must instead be reconstructed from projected images, changing brightness, apparent motion or the relative positions of objects on the sky. A diagram may represent a three-dimensional orbit in two dimensions, while an observation may encode information about inclination, orientation or line-of-sight motion. To solve the problem, a system must build an internal geometric model and manipulate it consistently. The benchmark suggests that language-based competence and symbolic calculation do not automatically provide this capacity.
The contrast became sharper on the data-analysis examinations. GPT-5 achieved an average score of 88.5 percent, a performance comparable to that of top-ten human participants. Other models scored considerably lower, with average results ranging from 48 to 76 percent. Data-analysis tests typically require candidates to extract patterns from tables, plots or observational measurements, estimate uncertainties and connect empirical trends to an astrophysical explanation. They can expose weaknesses that remain hidden in text-only reasoning, because a model must first identify what the data represent before it can calculate or interpret anything.
This divergence between models and examination types shows why simple benchmark scores can be misleading. An AI system may perform exceptionally well on carefully formatted theoretical questions yet struggle when information is distributed across a figure, a graph and a written description. Even when a model can process images, multimodal input does not guarantee reliable scientific interpretation. The system must understand scales, axes, coordinate systems, units, uncertainties and the physical meaning of a visual pattern. A small misreading at the beginning of the analysis can propagate through every subsequent calculation and produce an answer that appears mathematically polished but is scientifically wrong.
The study therefore presents a mixed picture of AI’s future in astronomy. On one hand, the leading models have demonstrated an extraordinary ability to perform multistep derivations and answer advanced theoretical questions at near-elite human levels. They could become useful assistants for checking calculations, explaining standard methods, exploring alternative solutions or helping researchers navigate established knowledge. On the other hand, autonomous research requires more than solving isolated examination problems. Scientific agents must decide which questions are meaningful, design analyses, recognize ambiguous evidence, track uncertainty and detect when their assumptions have failed. The Olympiad results indicate that these capabilities cannot be inferred from high scores alone.
For astronomy, the distinction matters because modern research increasingly depends on complex, multimodal evidence. Telescopes generate images, spectra, time series and catalogs containing millions of measurements. A research assistant that can manipulate equations but misinterprets geometry could draw the wrong conclusion about an orbit, a stellar population or the structure of a distant galaxy. Likewise, a system that produces confident explanations without recognizing uncertainty could make errors difficult to detect. The researchers’ conclusion is not that LLMs are incapable of scientific reasoning, but that their current strengths are unevenly distributed and that critical gaps remain before they can operate as autonomous astronomers.
The benchmark offers a new way to measure those gaps. By moving beyond short factual questions and testing derivation, multimodal interpretation and sustained reasoning, it brings AI evaluation closer to the realities of scientific work. The gold-medal-level theory scores show how rapidly language models have advanced. The weaker and more variable results in conceptual, geometric and data-driven tasks show why headline performance must be treated cautiously. The next generation of astronomy AI will need not only broader knowledge and better calculation, but also stronger spatial models, more dependable physical intuition and the ability to recognize when an apparently elegant solution does not describe the universe.
Subject of Research: Large language models’ performance on advanced astronomy and astrophysics examinations.
Article Title: Gold-medal performance by LLMs at the International Olympiad on Astronomy and Astrophysics
Article References: Carrit Delgado Pinheiro, L., Chen, Z., Caixeta Piazza, B. et al. “Gold-medal performance by LLMs at the International Olympiad on Astronomy and Astrophysics.” Nature Astronomy (2026). https://doi.org/10.1038/s41550-026-02964-w
Image Credits: AI Generated
DOI: https://doi.org/10.1038/s41550-026-02964-w
Keywords: large language models, artificial intelligence, astronomy, astrophysics, International Olympiad on Astronomy and Astrophysics, Gemini 2.5 Pro, GPT-5, multimodal reasoning, scientific AI, spatial visualization
