The study set out to test replicability, a concept that is closely related to but distinct from reproducibility. Reproducibility typically asks whether the same code, run again, produces the same numbers. Replicability asks something harder: whether an independent implementation of the same model, built from the same specification, produces the same results. This distinction matters enormously in agent-based modeling, where a model is rarely a single equation but rather a set of rules describing how autonomous agents interact, move through disease states, and drive emergent population-level dynamics. Because so many discretionary choices are involved in turning written rules into running code, the researchers suspected that the act of implementation itself might inject variability that no amount of careful specification could eliminate.

To find out, the team designed a rigorous and unusually ambitious experiment. They chose a SEIRS epidemiological model — a classic framework in which individuals cycle through Susceptible, Exposed, Infectious, and Recovered states, with the crucial twist that recovered individuals eventually lose their immunity and return to the susceptible pool, allowing epidemic waves to recur. The model’s dynamics are well understood: an initial sharp infection peak followed by a series of progressively damped oscillations as the population settles toward equilibrium. The researchers then wrote a common formal specification of the model using the ODD protocol — Overview, Design concepts, and Details — which is the closest thing the agent-based modeling community has to a standardized blueprint for describing models.

With the specification in hand, seven independent modelers each implemented the identical model on a different platform or in a different programming language. The roster spans the full breadth of the simulation ecosystem: hand-coded implementations in C++, Julia, and Python; the dedicated agent-based modeling platforms NetLogo, GAMA, and Cormas; and PythonPDEVS, a framework rooted in the Discrete Event System Specification formalism. Each implementation was developed without coordination among the modelers, mimicking the real-world conditions under which scientific software is typically produced. Every implementation was then run thirty times with different random seeds, generating a total of 210 simulation runs whose outputs could be compared statistically across implementations and within each implementation’s own stochastic variation.

The first, reassuring finding was qualitative. All seven implementations reproduced the expected epidemic trajectory: a sharp initial infection peak followed by damped oscillations converging toward a stable equilibrium. In other words, every team had correctly captured the essential structure of the model. Had the researchers stopped there, the story would have been one of success. But when they examined quantitative metrics — the amplitude of the infection peak, the timing of that peak, and other key characteristics of the epidemic curve — statistically significant differences emerged between implementations. Simulations that were supposed to be computationally equivalent were, in numerical terms, telling measurably different stories about the same hypothetical outbreak.

The magnitude of those differences is what elevates the finding from a technical footnote to a genuine concern. The variation between implementations turned out to be roughly 7.6 times larger than the stochastic variation within a single implementation across its thirty replications. To appreciate what this means, consider that stochastic noise is the variation modelers routinely expect and account for: run the same simulation many times, and random events produce a spread of outcomes. The divergence between independently written implementations was nearly eight times that expected, well-understood noise. In practical terms, an uncertainty band that a modeling team would report from their own repeated runs would dramatically understate the true uncertainty surrounding the model’s predictions, because it would exclude the variability introduced by implementation choices.

One immediate objection is that the experiment confounds two factors: the modeling platform and the person using it. Because each implementation was produced by a different modeler on a different technology, the observed variance reflects the combined effect of tool and developer, and the study’s design cannot separate the two. The researchers addressed this partially with a complementary analysis: several independent developers worked in a single language, either C++ or Java, removing the platform variable. Even under these controlled conditions, the individual developer alone induced statistically significant and large differences between implementations. This confirms that the human element — the countless small interpretive decisions each programmer makes when translating a specification into code — is itself an important source of divergence. Whether the technology contributes an additional, separable effect remains an open question that future work will need to disentangle.

The most actionable part of the study came from a controlled ablation analysis, in which the researchers systematically traced the divergence back to its origins. Much of it, they found, could be attributed to just two under-specified execution details. The first concerns how residence times — the durations individuals spend in the Exposed, Infectious, or Recovered states — are discretized into whole days. A specification that says individuals remain infectious for, say, five days on average leaves open whether that duration is drawn from a continuous distribution and rounded, truncated, or sampled as an integer directly, and each choice subtly shifts the epidemic dynamics. The second detail concerns whether the state-transition test is strict: when a model checks whether an individual should move from one disease state to another, small differences in how that probabilistic test is implemented can accumulate across thousands of agents and hundreds of time steps.

Neither of these details is exotic or obscure. They are precisely the kinds of decisions that fall through the cracks of even careful model documentation, because they seem too minor to mention. Yet the ablation analysis showed that simply specifying them explicitly in the ODD protocol would remove much of the observed divergence between implementations. This is a genuinely hopeful conclusion. It suggests that the replicability problem in agent-based modeling is not an intractable consequence of complexity, but rather a specification gap that the community can close. The practical prescription is clear: model descriptions should pin down not only the conceptual rules of a model but also the execution-level details — how continuous quantities are discretized, how probabilistic transitions are evaluated, and at what point in the simulation loop each operation occurs.

The broader implications extend well beyond epidemiology. Agent-based models are used to study economics, ecology, urban systems, and social dynamics, and their results increasingly inform real decisions. The reproducibility crisis that has shaken psychology, medicine, and other empirical fields now has a computational analogue, and this study provides some of the clearest quantitative evidence of its scale within simulation science. The authors emphasize that reproducibility is not merely a technical virtue but a cornerstone of scientific credibility, and their findings argue for rigorous practices at every stage: precise specifications, independent replication as a standard check, and open sharing of code and data. In a welcome demonstration of that principle, the team has made all source code, raw simulation outputs, and analysis scripts publicly available on GitHub, allowing anyone to scrutinize, reuse, or extend the experiment. As simulation models take on ever greater roles in policy and prediction, this study stands as both a warning and a roadmap: divergence between implementations is real and large, but with sufficiently precise specifications, it is largely preventable.

Subject of Research: Replicability of agent-based epidemiological simulations across programming platforms and languages

Article Title: Replicability of agent-based simulations: a case study with a SEIRS epidemiological model

Article References: Antunes, B., Havion, H., Delay, É., Foucher, C., Hill, D. R. C., Mazel, C., Bisgambiglia, P.-A., Scriban, A., Zaitsev, O., & Duboz, R. (2026). Replicability of agent-based simulations: a case study with a SEIRS epidemiological model. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-026-02546-3

Image Credits: AI Generated

DOI: 10.1007/s40747-026-02546-3

Keywords: agent-based modeling, replicability, reproducibility crisis, SEIRS model, epidemiological simulation, ODD protocol, NetLogo, GAMA, Cormas, simulation platforms, computational science, model specification

Photon correlation spectroscopy serves as a fundamental technique for probing the microscopic motion of particles within fluids and gels. The method relies on detecting fluctuations in the intensity of light scattered by particles, allowing researchers to infer diffusion coefficients and other dynamic properties. However, the raw data generated by modern detectors often consists of long sequences of photon arrival times, known as time-of-arrival records. Processing these extensive datasets requires sophisticated algorithms to convert raw event streams into meaningful correlation and structure functions, a task that demands substantial computational power.

In the previous iteration of the software, the entire segmentation process was executed within the LabVIEW environment. While this approach was suitable for shorter acquisition windows, it became increasingly inefficient as data lengths extended into minutes. The computational load associated with manipulating large arrays of photon events in a single environment led to prohibitively long processing times. In some instances, the software could not complete the analysis of the entire dataset, forcing researchers to truncate their data or rely on less precise approximations.

To overcome these limitations, the developers introduced a hybrid LabVIEW and MATLAB processing scheme in the new release. LabVIEW continues to manage the user interface, data acquisition, and the final stages of correlation analysis. However, the computationally intensive task of segmenting the photon event sequence has been offloaded to MATLAB routines. This strategic division of labor leverages the strengths of both platforms, allowing the software to handle significantly longer records without exceeding the memory or processing capabilities of standard computing systems.

The second major modification in version V2.1 involves a streamlined method for calculating the structure function. The structure function provides a robust alternative to the standard correlation function, particularly useful in regimes where signal-to-noise ratios are challenging. In the original implementation, this calculation required the parallel evaluation of second-order count moments for both detection channels. This process involved storing and manipulating large intermediate datasets, which further increased the computational burden and the risk of memory overflow during long analyses.

The updated software eliminates the need for explicit calculation of these second-order moments. Instead, it derives the necessary zero-lag information directly from the initial region of the normalized correlation function. The developers employ a second-order polynomial fit to the first several points of the correlation curve to estimate the correlation contrast, also known as the coherence factor. By extrapolating this polynomial fit to zero lag, the software obtains a precise estimate of the initial correlation value without requiring separate moment calculations.

This mathematical refinement simplifies the algorithm significantly. The structure function is now evaluated using a formulation that depends only on the measured mean photon counts and the estimated correlation contrast. This approach reduces the amount of intermediate data that must be stored and processed during the analysis. Consequently, the software requires fewer computational resources and operates more smoothly, even when dealing with the most demanding experimental datasets. The simplification also makes the software more intuitive for users, as the configuration options have been reduced to streamline the workflow.

The validation of the original software version was performed using data acquired from model colloidal nanoparticles embedded in polymer matrices. These systems provide a well-characterized benchmark for testing the accuracy of photon counting algorithms. The new version maintains this rigorous standard, ensuring that the computational shortcuts introduced do not compromise the physical accuracy of the results. The ability to process longer records without loss of precision is a substantial improvement for experimentalists studying slow dynamics or low-concentration samples.

The release of this updated toolbox represents a meaningful step forward in the accessibility of advanced optical analysis techniques. By reducing the technical barriers associated with processing large photon datasets, the software encourages broader adoption of photon correlation spectroscopy in various scientific fields. Researchers working with soft matter, biological fluids, and complex fluids can now utilize this tool to extract high-quality dynamic information from their experiments with greater confidence and efficiency.

The software is available under the GNU General Public License, ensuring that it remains open and accessible to the global scientific community. The developers have provided comprehensive documentation and a user manual to guide researchers through the installation and operation of the updated toolbox. This commitment to open science facilitates reproducibility and allows for further development by other groups in the field. The integration of MATLAB and LabVIEW sets a precedent for hybrid software solutions in experimental physics, demonstrating how combining different programming environments can yield superior performance for specialized scientific applications.

Subject of Research: Software optimization for photon correlation spectroscopy data analysis

Article Title: V2.1 – PhotonSTR-18: A LabVIEW toolbox for photon correlation spectroscopy

Article References: García-Cadena, C. A., Aguilar-Uribe, A. D. J., & Rojas-Ochoa, L. F. (2026). V2.1 – PhotonSTR-18: A LabVIEW toolbox for photon correlation spectroscopy. SoftwareX, 36, Article 103071. https://doi.org/10.1016/j.softx.2026.103071

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103071

Keywords: Photon Correlation Spectroscopy, LabVIEW, MATLAB, Soft Matter Physics, Computational Science, Optical Scattering, Data Analysis, Open Source Software, Colloidal Dynamics, Signal Processing, PhotonSTR-18, toolbox

Assistant Professor Nasir Eisty, a research software engineering and quality assurance expert at the University of Tennessee, Knoxville, has received a $430,000 grant to lead this project. Eisty serves in the Min H. Kao Department of Electrical Engineering and Computer Science, where he specializes in the intersection of software engineering and scientific computing. His work focuses on the practical application of AI agents in the creation and verification of research software, a field that is evolving rapidly as autonomous systems become more capable of interacting with external tools and executing complex tasks.

The core of the problem lies in the nature of agentic AI systems, which can autonomously interact with external tools to achieve programmed goals. In the context of scientific research, these systems are increasingly used to generate code from scratch, execute tests, and validate outputs. However, Eisty points out a significant risk: an AI can generate a test that appears sophisticated, runs successfully, and passes, yet it may be testing the wrong scientific assumption. This discrepancy is particularly dangerous because different scientific domains, such as climate modeling and biological simulation, have distinct definitions of correctness and underlying theoretical constraints.

Traditionally, research software engineers (RSEs) verified simulations by manually identifying important scenarios, encoding them as tests, and evaluating whether the results matched known outcomes. This process was time-consuming and required deep expertise in both the specific scientific domain and software engineering. With the advent of generative AI, scientists can now ask AI systems to analyze scenarios and generate tests, or even have agentic AIs run and modify tests based on results. This capability is valuable for researchers who are domain experts first and software developers second, as it makes good software engineering practices accessible to teams without dedicated engineering resources.

Despite these advantages, the convenience of AI-generated tests comes with a cost. AI systems may fail to capture important edge cases or domain-specific theoretical constraints, leading to incorrect assumptions and irreproducible results. If the scientific assumptions underlying a simulation are not properly accounted for, the credibility of the entire research output is compromised. Eisty’s Software Analytics and Intelligence Lab (SAIL) is investigating how to leverage AI in the scientific software pipeline while maintaining reproducibility, correctness, and long-term utility. The goal is to determine where AI is useful, where it is unreliable, and how humans can remain appropriately involved in the validation process.

The project will involve a detailed study of current RSEs and scientists to understand how they are using generative agentic AIs to design and test their software. The team will then deliberately introduce multiple types of defects into existing research software to evaluate how well AI-generated tests detect these injected issues. By working with domain scientists, the researchers will assess whether the tests reflect the scientific assumptions and questions underlying the software. This approach allows for a controlled environment in which the reliability of AI-generated tests can be measured against known ground truths.

One of the most challenging aspects of this research is defining what it means for an AI-generated test to be scientifically correct. To address this, Eisty and his team plan to study 30 to 40 research software projects across at least five scientific domains, including computational biology, astronomy, climate modeling, machine learning, and computational social science. This diverse set of projects will provide a broad basis for understanding how AI performance varies across different scientific contexts. The findings will help identify patterns in AI reliability and highlight areas where human oversight is most critical.

Upon completion of the experiments, the team will create a curated benchmark suite, open-source tools, and guidelines for responsible AI-assisted software testing. These resources will include a tool to help researchers provide AIs with appropriate scientific context during prompting and a test-development framework that records important information about the AI model and other factors involved in creating each test. The framework will be model-agnostic, allowing RSEs to continue creating reliable and reproducible research software as new AI systems emerge. This ensures that the solutions developed are not tied to a specific technology but are adaptable to future advancements.

The project also includes support for two PhD students who will work with Eisty throughout the research process. This aspect of the grant is crucial for training the next generation of researchers who can help shape the responsible use of AI in scientific software development. Eisty expressed gratitude to the Sloan Foundation for supporting this work, noting that AI is changing software engineering very quickly. The project provides an opportunity not only to study this transformation but also to equip researchers with the skills needed to navigate it responsibly.

Ultimately, this initiative is about finding the right balance between leveraging AI’s ability to accelerate software development and preserving the human judgment, transparency, and rigorous validation that science requires. As AI continues to play a larger role in scientific discovery, ensuring the integrity of the tools used to generate and verify results will be essential. By developing robust frameworks and guidelines, Eisty and his team aim to future-proof research software, ensuring that it remains a reliable foundation for scientific progress in an era of rapid technological change.

Subject of Research: Validation of AI-generated research software for scientific reproducibility

Article Title: With Sloan Foundation Award, Eisty future-proofs research software

Article References: With Sloan Foundation Award, Eisty future-proofs research software. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: Artificial Intelligence, Research Software Engineering, Scientific Reproducibility, Alfred P. Sloan Foundation, University of Tennessee, Quality Assurance, Agentic AI, Computational Science, Software Testing, Data Integrity, Sloan, Foundation