NYU AI tools flag citation hallucinations and locate lab protocols

Computer scientists and students at New York University have built a pair of web-hosted artificial intelligence systems aimed at two persistent pain points in modern research: citations that misrepresent the papers they reference, and the laborious hunt for suitable experimental protocols. The tools, described in two articles in the journal MDPI Information, were led by Dennis Shasha, a professor of computer science at NYU’s Courant Institute School of Mathematics, Computing, and Data Science. One system, HallucinationNerd, checks whether the claims a document makes about its citations are actually backed up by the cited works. The other, ProtocolNerd, helps bench scientists find experimental protocols matched to their research questions.

HallucinationNerd grades each claim-citation pair it examines, assigning one of six labels: “Supported,” “Partially Supported,” “Not Supported,” “Contradicted,” “Cited article does not exist,” or “Could not access.” According to Shasha, checking all the claims an article makes about other articles helps both readers and authors determine whether the citing article fairly represents the state of the art, and accurately and efficiently evaluating previous research can only improve subsequent, related studies. The tool is designed for natural scientists and, eventually, social scientists who want to be sure that papers they write and depend on fairly represent the literature they cite.

ProtocolNerd targets the other end of the research workflow. Many hundreds of thousands of experimental protocols exist, and sorting through them is labor-intensive and error-prone, Shasha observes. The system is illustrated with a molecular biology scenario: a biologist seeking ways to make rice more drought tolerant might ask, in plain English, for protocols that test for drought tolerance in rice. ProtocolNerd would first ask clarifying questions, such as the growth stage of the rice plant, and would explain why it is asking. Once the scientist answers, the system would provide multiple protocols and give reasons why each might be helpful—or, alternatively, might not be.

The researchers validated both models against research databases using testing frameworks they compare to randomized control trials. For ProtocolNerd, the benchmark drew on scientific articles from PubMed and Europe PMC and tested whether the system, alongside other state-of-the-art systems, could identify the protocol used by an article based solely on its title and abstract. For HallucinationNerd, the validation used arXiv, the open-access repository of research papers, gauging the tool’s ability to separate hallucinations from supported claims by testing its outputs against the contents of papers in the repository and comparing its performance against existing systems. Shasha says the frameworks were designed to resemble randomized control trials for the same reason bench scientists consider such trials the gold standard: to eliminate bias and be able to test for statistical significance.

Shasha notes that ProtocolNerd is currently aimed at molecular biology and chemistry but can be extended by a domain expert to other disciplines, such as psychology and economics, even if that domain expert is not a programmer. If an experimentalist wishes to answer a new scientific question and needs a protocol to address it, finding the right protocol quickly is of great use, he says. The work was supported by NYU Wireless, an academic research center at NYU’s Tandon School of Engineering that backs advances in wireless communications, sensing, networking, and devices, and by the US Department of Energy’s Advanced Research Projects Agency-Energy under award DE-AR0002025 M0002. Shasha adds that the two tools will be most useful to natural scientists who wish to design new experiments based on the most relevant existing protocols and want to verify that the papers they depend on fairly represent the literature they cite.

Subject of Research: AI tools for citation verification and experimental protocol discovery

Article Title: Computer scientists build innovative tech tools to find experiments, detect citation “hallucinations”

Article References: Computer scientists build innovative tech tools to find experiments, detect citation “hallucinations”. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: Computer, scientists, build, innovative, tech, tools, find, experiments, detect, citation, hallucinations, scientific research