The proposal, published in October 2022, introduces the Formal Model as the first of three sequential publication steps: the Formal Model itself, followed by implementation documentation, and finally evaluation and testing. Until now, the first formal step of model development has typically been folded into a single research article covering development, implementation, and application all at once. That approach works tolerably for small models, the authors argue, but becomes unwieldy as models grow larger, more complicated, and more interconnected, drowning reviewers in detail and leaving crucial design decisions unexamined until after the analysis is complete.

The rationale is straightforward. Under the current system, a model is constructed, calibrated, tested, and applied, and only then does a peer-reviewed publication appear. Reviewers who question modelling decisions at that late stage can only suggest redesigns and reruns after enormous amounts of work have already been invested. The Formal Model moves the first point of peer review to the beginning of the modelling cycle, occupying roughly the first quarter of the first iteration. It is a reasoned statement of intent, deliberately separated from the model implementation in code, allowing reviewers to flag problems and suggest alternatives before resources are committed to a final version.

The format draws inspiration from the preregistration and registered report movements that have reshaped experimental science. Preregistration allows researchers to register their plans and generate a digital object identifier, creating accountability and a living document that distinguishes pre-diction from post-diction. But preregistration alone lacks peer review, meaning no one can catch flaws in the experimental design before the work proceeds. Registered reports solve this by having journals review the plans and agree in principle to publish the results, whatever they turn out to be. Preregistration has been adapted for mathematical and small computational cognitive models in psychology, yet the authors argue it cannot comfortably accommodate complex model design, where a modeller may iteratively implement, calibrate, and then apply a model to a host of analyses before any hypothesis testing begins.

The Formal Model is not the first attempt to tame model documentation. Well-commented code and tools such as Doxygen have long served implementation, but they say nothing about model structure and aims. The Overview, Design concepts, and Details protocol, developed for individual-based models, originally separated description from implementation, though later versions moved toward specifying code details, and the ODdox variant links the two together. Electronic notebooks have been suggested for the development process. In software engineering, the Formal Methods Model offers a precisely defined description of components and relationships for planning development, and the European Food Safety Authority once defined formal models as those in which model variables and parameters are linked into mathematical equations or algorithms, building on prior problem definition and conceptual model steps. The new format absorbs all three of those EFSA steps into a single standard yet flexible structure spanning disciplines.

So what does a Formal Model actually contain? The proposed template mirrors the familiar IMRAD structure of standard papers to make reading predictable, while retaining flexibility for different model types. It opens with an introduction laying out the reasons for creating the model, the supporting theory, and the modelling approach, rather than problems and hypotheses per se. An aim and purpose section serves as the problem formulation. A theoretical framework section declares the discipline and perspective, for example foraging theory or the metabolic theory of ecology, and explains why the chosen approach, such as agent-based methodology, is the best fit. A framing section then makes explicit the external influences knowingly left out of the model, a deliberate strategy to avoid false inclusion and false exclusion errors and to embrace what the authors call a modest approach to modelling that avoids strong claims.

The heart of the document is an overview of processes, followed by detailed descriptions of each component. For every process, the modeller reviews the current state of knowledge, citing the literature, tables, and diagrams used to develop the concepts, and then presents the planned implementation with formal representations such as equations or flow charts. State variables, spatial and temporal scales, and units must be specified, often tabulated with symbols, predetermined values, and sources. An example drawn from the agent-based model ApisRAM describes how every bee consumes resources and generates heat according to its metabolic rate, with temperature increase defined by a simple equation relating heat production to the amount of nectar burned and the heat capacity of the bee. An overview of components and their connections, typically with an overall diagram, serves as a roadmap to the details, and an optional section on model properties and behaviour can explore expected functioning under controlled conditions. A discussion covering strengths, weaknesses, exclusions, and the model’s place in the literature closes the format.

The authors are candid that the format is no silver bullet. It adds another documentation burden on top of normal documentation and user guides, and substantial documents require real commitment; the ApisRAM formal model, though not in the new format, runs to 58 pages. There is a risk that nobody reads it afterwards, and a risk of low adoption that would undermine any long-term improvement. Yet these downsides are likely counterbalanced by the reviewed publication status, the improvements gained through dialogue with reviewers, and the fact that the article will be cited whenever the model is used, increasing visibility and impact. If writing the Formal Model is folded into the design process itself, only a little extra effort should be needed to complete it.

Versioning addresses another drawback: many large models undergo several modelling cycles as new information arrives or new applications are needed, and a static reviewed document could confuse rather than clarify. The published Formal Model is therefore associated with a defined model version, typically 1.0. For smaller changes, authors could publish unreviewed updates as minor versions in journals that accept such formats or on a model website. For major upgrades, a second major version of the Formal Model may contain enough material for a full publication. Accompanying each revision with a version number represents the model’s development over time and can even form the basis for dialogue between modellers and stakeholders when versions are published in journals allowing comments.

Beyond transparency and reproducibility, the format carries a quieter ambition: giving modellers credit for the considerable work of extensive literature review, careful design, and ingenuity that often goes unacknowledged, with documented consequences for rigour and scientific quality. It also opens collaboration to people without modelling skills, since the conceptual groundwork becomes a publishable contribution in its own right. The Food and Ecological Systems Modelling Journal is currently the only journal offering this set of formats, though publication of all three steps is not mandatory, and an unfollowed Formal Model could even invite others to develop the proposed model. The authors hope that, through use, the format will improve and be improved by the modelling community, beginning a journey toward better models through better documentation.

Subject of Research: A proposed Formal Model publication format for peer review of modelling intent and data foundations before model implementation.

Article Title: The Formal Model article format: justifying modelling intent and a critical review of data foundations through publication

Article References: Topping, C. J., Kondrup Marcussen, L., Thomsen, P., & Chetcuti, J. (2022). The Formal Model article format: justifying modelling intent and a critical review of data foundations through publication. Food and Ecological Systems Modelling Journal, 3, Article e91024. https://doi.org/10.3897/fmj.3.91024

Image Credits: AI Generated

DOI: 10.3897/fmj.3.91024

Keywords: Formal Model, model documentation, peer review, preregistration, agent-based modelling, reproducibility, transparency, scientific modelling, Food and Ecological Systems Modelling Journal, model versioning, Formal, Model

The international team, led by Min Chen and Zhiyi Zhu of Nanjing Normal University together with Thorsten Wagener, Niklas Boers and more than twenty colleagues across Europe, Asia, Australia and North America, argues that reproducibility must be rethought for the artificial intelligence era. In traditional Earth system modelling, reproducibility largely meant running the same code, on comparable hardware, with the same inputs, and obtaining the same outputs. Hybrid models break that tidy picture. Their numerical behaviour depends not only on deterministic physical solvers but also on stochastic training procedures, opaque learned components, versioned datasets and specialized computing environments that are difficult to document and share.

The authors identify three compounding problems. The first is numerical instability: neural network components embedded in climate and hydrological simulations can behave chaotically across tiny changes in hardware, software libraries or floating-point precision, so that two nominally identical runs produce divergent results. The second is procedural opacity: when a deep learning module replaces a physically derived parameterization of clouds, convection or runoff, the reasoning behind a prediction can become effectively inaccessible, making verification and error diagnosis far harder. The third is asymmetric access to computational resources: training frontier models demands GPU clusters and curated datasets that many institutions, particularly in lower-income countries, simply cannot obtain, narrowing who can check and extend published work.

The stakes extend beyond academic housekeeping. Hybrid Earth system models increasingly inform climate risk assessments, disaster early-warning systems and national adaptation strategies. If their results cannot be traced, tested and re-derived, the authors contend, the models risk becoming weakly verifiable black boxes. That would erode model traceability, weaken the cumulative build-up of knowledge across research communities and ultimately narrow the evidential basis available to bodies such as the Intergovernmental Panel on Climate Change and the policymakers who rely on its assessments.

At the core of the Perspective is a conceptual reframing. Rather than treating reproducibility as a post hoc checklist applied when a paper is published, the team proposes that it should be understood as a structural property of the modelling process itself. They argue for a theory of reproducibility assessment calibrated to the epistemology of hybrid systems, in which the question is not merely whether a numerical output can be replicated bit-for-bit, but whether the full chain of evidence — data provenance, training procedure, physical coupling, uncertainty characterization and code environment — can be independently reconstructed and evaluated.

To translate that principle into practice, the researchers introduce RHEM, short for Reproducibility in hybrid Earth system models, a reference guideline intended to govern transparent, trustworthy and reproducible model development. RHEM is conceived as a shared framework against which modelling groups can design, document and audit their workflows. Its criteria cover the life cycle of a hybrid model, from the definition of research questions and the curation of training data through the coupling of machine learning components with physical cores, and finally to release, versioning and community reuse.

Complementing the framework, the Perspective lays out practical pathways that operationalize its criteria into actionable measures. These include comprehensive documentation strategies for geosimulation experiments, provenance capture using standards such as W3C PROV, containerized software environments that pin library versions and hardware assumptions, and modular architectures that clearly separate learned components from physical ones so each can be tested independently. The authors also highlight physically informed constraints on neural networks, which enforce conservation laws and other analytic requirements to stabilize learned parameterizations, and benchmark datasets such as WeatherBench that allow head-to-head comparison of competing forecasting approaches.

The proposal draws on a wide body of prior work in computational science. The authors situate their argument in the broader reproducibility crisis documented across disciplines, citing surveys in which a majority of surveyed scientists reported failing to reproduce others’ experiments, along with manifestos for reproducible science, guidelines for artificial intelligence transparency, and studies showing that much computational hydrology has historically resisted independent replication. They also note existing community infrastructure — component-based modelling frameworks, virtual laboratories, FAIR principles for research software and open modelling portals — as building blocks on which RHEM-style governance can be constructed rather than starting from scratch.

What distinguishes the new contribution, the authors argue, is the explicit coupling of a conceptual structure with operational guidance tailored to hybrid systems. Earlier reproducibility efforts addressed either classical numerical models or machine learning research in isolation. Hybrid Earth system models inherit the pathologies of both: the chaos sensitivity and tuning culture of climate simulation, and the nondeterminism, dataset dependence and interpretability gaps of deep learning. By treating these as an integrated design problem, the team hopes reproducibility can move from an aspirational virtue to an engineered characteristic, embedded in how models are built, tested and documented.

The Perspective closes with a call for coordinated community action. Modelling centres, funding agencies, journals and cyberinfrastructure providers each hold part of the solution, from mandating reproducibility statements at submission to sustaining shared computing resources that level the field between well-funded and resource-constrained groups. If such measures take hold, the authors conclude, reproducibility can be established as a foundational principle of Earth system science in the artificial intelligence era — ensuring that as models grow more powerful, they also grow more accountable, traceable and worthy of the public trust placed in the climate science they support.

The reproducibility dilemma confronting hybrid models has deep historical roots in Earth science. As early as 1994, philosophers and modellers cautioned that numerical models of the Earth could never be verified in the strict sense familiar from laboratory science, because the systems they describe are open, unique and only partially observable. That insight, once confined to philosophical debates about validation, has acquired new urgency now that learned components trained on finite observational records sit inside simulation codes whose outputs shape billion-dollar adaptation decisions. The Perspective effectively extends an old epistemological caution into the computational present, arguing that the looseness of verification once tolerated in physical modelling cannot be compounded by the additional opacity of machine learning.

Computational studies illustrate how fragile reproducibility can be even without neural networks. Research on the EC-Earth3 climate model showed that a change in computing environment could alter simulation outputs in ways that complicated replication, while work on atmospheric regional modelling identified hydrodynamic noise as a fundamental limit on reproducibility, distinguishing irreducible chaos from avoidable procedural failures. These findings matter for the hybrid era because they supply a baseline: some divergence between runs is physically expected, and a sensible reproducibility standard must therefore separate meaningful nondeterminism from instability introduced by sloppy engineering, unstated library versions or uncontrolled floating-point behaviour. The roadmap proposed by Chen and colleagues builds directly on this distinction, seeking to pin down everything that is controllable so that remaining variability can be interpreted scientifically rather than dismissed as noise.

The timing of the Perspective reflects how quickly artificial intelligence has penetrated the field. Differentiable programming frameworks now allow entire Earth system processes to be embedded in gradient-based training loops, generative adversarial networks constrained by physics have been used to sharpen precipitation fields, and scale-adaptive generative models can downscale coarse model output with quantified uncertainty. Commercial and academic systems alike have demonstrated skillful medium-range global weather forecasting learned purely from data, and a foundation model for the Earth system has been proposed as a general-purpose substrate for downstream tasks. Each of these advances widens the surface area of reproducibility risk: more pretrained weights, more data pipelines, more hardware dependencies and more points at which undocumented choices silently enter the scientific record.

The authors’ emphasis on asymmetric access also connects to a broader equity conversation in geospatial and climate research. Surveys of reproducibility practices across the sciences have repeatedly found that most researchers regard replication as difficult, and geospatial researchers have documented how data licensing, proprietary platforms and uneven computational capacity compound the problem in their field. Explainable artificial intelligence for Earth observation has been examined partly through regulatory and societal lenses, underscoring that transparency is not merely a technical preference but increasingly a governance expectation. Framing reproducibility as a structural property rather than a publication-time obligation aligns with these developments, since transparency measures embedded early in a workflow are far cheaper and more effective than forensic reconstruction after the fact.

There is also a constructive reading of the moment. The same machine learning techniques that create reproducibility hazards can support the remedy: automated provenance capture, differentiable pipelines that make couplings explicit, and uncertainty-aware generative methods that state their own limitations. The Perspective’s insistence on coupling a theory of assessment with practical pathways acknowledges this duality. By treating documentation, containerization, modular separation of learned and physical components, and physically informed constraints as parts of a single design discipline, the framework aims to make trustworthiness a property that can be engineered, audited and incrementally improved — a stance the authors present as essential if the accelerating capabilities of hybrid Earth system models are to translate into durable, verifiable scientific knowledge.

Subject of Research: Reproducibility challenges and governance frameworks for hybrid Earth system models that integrate artificial intelligence

Article Title: Enhancing reproducibility in hybrid Earth system models

Article References: Chen, M., Zhu, Z., Wagener, T., Boers, N., Müller, R. D., Strobl, J., Camps-Valls, G., Batty, M., Jakeman, A. J., Kolditz, O., Nativi, S., Brovelli, M. A., Creutzig, F., Kumar, P., Whitehead, P., Barton, C. M., Liu, D., Ma, P., Ma, Z., … Lü, G. (2026). Enhancing reproducibility in hybrid Earth system models. Nature Machine Intelligence. https://doi.org/10.1038/s42256-026-01299-5

Image Credits: AI Generated

DOI: 10.1038/s42256-026-01299-5

Keywords: Earth system models, artificial intelligence, reproducibility, hybrid modelling, machine learning, climate science, Nature Machine Intelligence, RHEM framework, scientific transparency, model documentation, climate risk, computational reproducibility