“After our Lancet audit, the question we heard most often was whether this was a problem particular to medicine. It is not,” Topaz says. “A reference is a promise to the reader that a source exists. In a growing number of papers across the sciences, that promise is not being kept.” The audit measured how often that promise fails in fields far removed from clinical research. The team drew 500,000 papers published from January 2023 through March 2026 from OpenAlex, a large database of scholarly works, covering five fields: chemistry, physics, materials science, engineering, and computer science. From this corpus, an automated verification tool checked the 21.6 million references that carried a digital object identifier, or DOI. The tool classified 1,135 references across 619 papers as fabricated when the cited title could not be found in online databases.

To test the accuracy of that automated classification, the team had two independent experts review a random sample of 400 flagged references; the experts confirmed that 85.8 percent of them were indeed fabricated. That validation step provides a measured estimate of how often the automated flag was correct, since a title that is temporarily hard to locate online is not the same as a title that never existed. The discipline-by-discipline breakdown shows that no field examined was spared. Physics had the highest rate, with 25.6 fabricated references per 10,000 papers, followed by materials science at 18.4 per 10,000. Computer science and artificial intelligence registered 11.1 per 10,000, engineering 10.3 per 10,000, and chemistry 4.3 per 10,000. These are, in absolute terms, small numbers: the vast majority of papers in every field contained no fabricated references at all.

The trend over time, however, is what the authors describe as accelerating. The share of affected papers rose from 6.3 per 10,000 in 2023, or about one in 1,600, to 15.4 per 10,000 in 2025, about one in 650, and then to 23.0 per 10,000 in the first quarter of 2026, about one in 435. In roughly three years, the measured prevalence more than tripled. The distribution of fabricated references within affected papers was also uneven, and that unevenness carries practical consequences for how the problem should be handled. Most papers with fabricated references contained only one or two of them. But 27 papers contained five or more fabricated references, and those 27 papers alone accounted for more than a third of all fabricated references identified in the audit.

Topaz argues that this distinction should shape editorial responses. “A paper with one unverifiable reference and a paper with five or more are not the same thing, and editors should not treat them the same way,” he says. Extrapolating from the measured rates, the authors estimate the wider footprint of the problem. Applying each field’s rate to the approximately 7 million papers published in these disciplines over the same period, they calculate that about 8,500 published papers carry at least one fabricated reference. At the time of the audit, two of the 619 affected papers had been retracted. The authors recommend that indexing services and research-integrity databases record fabricated references as a distinct category, separate from plagiarism or data falsification, so that the problem can be tracked and acted on.

The new study does not stand alone. It is part of a body of research the Columbia-led team has released this year. The first was a research letter in the Lancet in May, which identified 4,046 fabricated citations in 2,810 of 2.5 million medical papers. In July, a paper in the Journal of Internal Medicine reported that fabricated references were about twice as common in medRxiv preprints as in peer-reviewed articles, at 37.2 versus 16.4 per 10,000. In September, an arXiv preprint from the team, which has not yet been peer reviewed, found that 26 large language models fabricated 55 percent of the medical references they were asked to supply. Taken together, these studies trace the problem across the publication pipeline, from preprints that have not undergone review to published articles in peer-reviewed journals, and across both the biomedical and the physical and computational sciences.

The role of large language models in generating nonexistent citations has been a prominent concern in discussions of reference integrity, and the team’s September preprint offers a measured estimate of how often the models tested invented sources when asked to provide them. The new PNAS study, however, documents the problem in the published record itself. What the audit establishes is the presence and frequency of unverifiable references in the literature, not their causes, and the authors’ recommendations follow from that measurement. The study’s scope is also bounded by its method: it examined references that carried DOIs, and its automated classification, while validated on a random sample, cannot capture every case in which a cited title is difficult to locate online.

Those recommendations center on automation. The authors call for routine automated reference checks at the point of submission, so that citations can be verified against online databases before a paper is published, together with retroactive screening of the already-published literature to identify fabricated references in existing articles. “The encouraging part is that this is one of the few integrity problems that can be checked automatically, before a paper is ever published,” Topaz says. The audit’s own workflow demonstrates the approach: an automated tool screened 21.6 million DOI-bearing references across half a million papers, and expert review of a random sample then measured how often the automated flags were correct.

The team’s next project aims to scale this approach to the biomedical literature and to move from detection toward response. The researchers are screening up to 12 million medical papers published since 2020 and will have experts grade how much several hundred of the fabricated references affect the papers that cite them. Working with journal editors and research integrity officers, they plan to use the findings to develop screening protocols, policies, and trainings for journal submissions. The project, “Fabricated Citations in the Biomedical Literature: Measuring Prevalence, Triaging Evidentiary Impact, and Reducing Response Burden,” is supported by award ORIIR260088 from the Office of the Assistant Secretary for Health, U.S. Department of Health and Human Services, as part of a financial assistance award totaling $249,999 with 100 percent funded by OASH/HHS. The authors note that the contents are solely their responsibility and do not necessarily represent the official views of, nor an endorsement by, OASH, HHS, or the U.S. Government.

Other authors of the PNAS study include Pallavi Gupta, PhD, and Zhihong Zhang, PhD, of Columbia University School of Nursing, and Laura-Maria Peltonen, PhD, of the Department of Health and Social Management at the University of Eastern Finland, the Wellbeing Services County of North Savo, the Wellbeing Services County of Southwest Finland, and the Department of Nursing Science at the University of Turku. For now, the PNAS audit establishes a measured baseline for fields that had not previously been examined on this scale: fabricated references were identified in every scientific field studied, their prevalence rose over the three-year window, and the automated tools needed to detect them before publication already exist and have been validated against expert review.

Subject of Research: Prevalence of fabricated references in science, engineering, and computing journals

Article Title: Fake references are not only a medical problem: Columbia Nursing audit finds them across physics, engineering, and computer science journals

Article References: Fake references are not only a medical problem: Columbia Nursing audit finds them across physics, engineering, and computer science journals. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: fabricated references, citation integrity, PNAS, Columbia University School of Nursing, OpenAlex, peer review, research integrity, large language models, physics, computer science, scientific publishing, DOI verification