Why 96% of Biomedical Papers Keep Their Code Secret, and How to Fix It

Biomedical science has never been more powerful or more fragile. Researchers can download vast public datasets—insurance claims, electronic health records, national surveys—and interrogate them with sophisticated statistical code in seconds. Yet the very tools that have accelerated discovery have also created a quiet crisis: results that other scientists cannot verify, errors that go undetected, and analyses that hinge on dozens of undocumented judgment calls. A new viewpoint published in the Journal of General Internal Medicine argues that the reproducibility problem in biomedical research is not a matter of individual sloppiness but a systemic failure, and it offers a concrete prescription borrowed from an unlikely source: the software engineering industry.

The authors—Gray Babbs and Alyssa Bilinski of Brown University School of Public Health and Ishani Ganguli of Brigham and Women’s Hospital and Harvard Medical School—begin with a sobering audit. Examining three months of Original Investigations and Research Letters in JAMA, JAMA Internal Medicine, and JAMA Pediatrics, they found that only 4 percent of the 134 publications shared their analysis code, and just 5 percent deposited data in public repositories. Code was listed as available on request for another 10 percent of papers and explicitly unavailable for 86 percent. On the data side, 16 percent of articles reported using public data without providing it in a replication package, 39 percent said data were available on request, and 40 percent listed data as unavailable. In other words, roughly 96 percent of papers in three of the most prestigious medical journals in the world cannot be independently re-run from their published materials.

Reproducibility, as the authors define it following the National Academies’ 2019 consensus report, means that new researchers can obtain the same result using the same data and methods as the original team. That definition sounds modest, but meeting it requires more than good intentions. It requires that the exact code used to clean, transform, and analyze the data be preserved, documented, and shared—along with the data itself or a lawful pathway to access it. The authors argue that the pervasive failure to do so points to structural problems: gaps in training, unclear standards, misaligned incentives, and legitimate fears about misuse of shared resources.

The first barrier is educational. Modern biomedical research increasingly demands programming, but biomedical curricula rarely teach sound coding practices. None of the three authors, they note candidly, were ever taught to test their code as part of their formal training. Writing code that behaves as expected is notoriously difficult even for professional software engineers; for researchers learning on the fly, it is harder still. In practice, coding is often delegated to a single graduate student or junior analyst with limited oversight. Collaborators may scrutinize the output—the tables and figures—but formal review of the code that produced them is rare. The result is an invisible layer of the research process that almost no one checks.

The second barrier is the absence of clear standards. Even when research teams do implement internal review processes, they seldom describe them in their manuscripts, in sharp contrast to the meticulous documentation of data sources and statistical methods that journals demand. Reporting checklists such as CONSORT and CHEERS, which guide methodological transparency across medical journals, contain no requirements about code quality or availability. The third barrier is incentive-related: preparing data and code for public release takes time that publish-or-perish career pressures make easy to deprioritize, especially when peers are skipping it too. The National Institutes of Health now requires Data Management and Sharing Plans for applications that generate scientific data, but it does not oversee implementation, and the requirements do not cover studies that use secondary data sources—a category that includes much of modern health services research.

Fear also plays a role. Authors may worry that shared code could be misinterpreted or repurposed. The authors offer a pointed example: algorithms designed to identify transgender beneficiaries in insurance claims data for research purposes could, in the wrong hands, be used to discriminate against trans patients. The rise of large language models adds fresh anxieties about intellectual property and control. And there is a collective-action problem: researchers who share their data and code expose themselves to criticism that colleagues who share nothing avoid. Transparency, perversely, can feel like a penalty for good behavior.

The prescription draws on three practices that are routine in software engineering but exotic in academic medicine. The first is code testing. Biomedical researchers typically rely on two implicit checks: that the code runs without throwing errors, and that the output looks plausible. Both are ad hoc and vulnerable to confirmation bias, since researchers tend to scrutinize code hardest when results surprise them. Systematic testing instead defines the expected behavior of each function in advance and verifies that given inputs produce correct outputs. The authors give a concrete example: when collapsing person-year-level data to the person level, a test could confirm that the number of unique individuals is preserved. Designing tests that anticipate what could go wrong is a skill that requires practice, but when done well it catches errors that would otherwise slip through unnoticed.

The second practice is code review: a careful examination of the data cleaning and analysis code by someone who did not write it. Standard in industry, it remains rare in academia. Its value lies not only in catching errors but in the discipline it imposes—code written with the expectation that a colleague will read it tends to be cleaner and better documented. While software companies review small units of code frequently, the authors suggest that reviewing a complete analytic pipeline at a project’s end may be more realistic in an academic setting. A side benefit is that reviewed code is largely ready for public dissemination. The third and most resource-intensive practice is repeat coding, in which multiple researchers independently write code for the same analysis. When two independent implementations disagree, the discrepancy may reveal a bug—but it may also expose differing analytic assumptions, such as how to handle missing values or which observations to exclude. Either way, the disagreement is informative, surfacing hidden judgment calls that never appear in the methods section.

Large language models, the authors argue, could dramatically lower the cost of all three practices. Beyond accelerating initial code development, LLMs can generate tests, flag common errors, serve as a first-pass code reviewer, or assist with repeat coding. Creative application of these tools, they suggest, can make the gap between current and best practice far less daunting. But individual virtue is not enough without institutional reinforcement. Journal editors, as de facto standard-setters for biomedical research, are well positioned to raise expectations—enforcing code-sharing at the revised manuscript stage at minimum. Some fields already go further: the American Economic Review requires authors to submit full replication packages with code and data, and when data are non-public, authors must state whether a private version can be made available to a Data Editor or designated third-party replicator. The journal Bioinformatics requires peer review of new software, algorithms, and code. Journals could also deploy AI-powered review tools to ease the burden on human referees.

Funders, finally, have a role to play: financing the preparation of replication packages, supporting double coding for high-stakes analyses, and building shared infrastructure—perhaps including HIPAA-compliant LLM tools for replicating studies that rely on restricted data such as Medicare claims. The authors close on a note of measured optimism. The barriers to reproducibility are substantial but not insurmountable, and the practices they describe have been proven in other fields. Standardizing code testing, review, and sharing, they argue, is not merely a matter of technical rigor. It is a concrete mechanism for reinforcing the credibility of scientific results—and, at a moment when public trust in science is under strain, that may be the most important result of all.

Subject of Research: Reproducibility and code-sharing practices in biomedical research

Article Title: Reproducibility in Biomedical Research: A Systems Prescription

Article References: Babbs, G., Ganguli, I., & Bilinski, A. (2026). Reproducibility in Biomedical Research: A Systems Prescription. Journal of General Internal Medicine. https://doi.org/10.1007/s11606-026-10536-x

Image Credits: AI Generated

DOI: 10.1007/s11606-026-10536-x

Keywords: reproducibility, biomedical research, code sharing, data sharing, code review, code testing, large language models, research integrity, journal editors, NIH data policy, replication, software engineering