Retrieval-augmented LLM framework automates PRISMA-aligned systematic reviews

Systematic reviews and meta-analyses underpin scientific progress, but the work of screening thousands of papers and extracting data from them remains slow, labor-intensive and prone to human error. A team of Italian researchers now reports a framework that weaves large language models and retrieval-augmented generation directly into the PRISMA 2020 workflow, the reporting standard that governs how systematic reviews are planned, screened and documented. The study, published in Neural Computing and Applications, describes a methodological contribution rather than a new algorithm: the authors assembled established, openly available components into a single reproducible, auditable pipeline that prioritizes transparency and re-execution over customized elements.

The framework addresses each stage of the review process with a dedicated mechanism. Structured query templates generate reproducible bibliographic searches, so that the identification of candidate literature does not depend on ad hoc search strings. An LLM-driven pipeline then performs title and abstract screening, while a retrieval-augmented generation module carries out full-text eligibility assessment. Both automated stages rely on structured prompt templates that encode the inclusion and exclusion criteria of a review as machine-interpretable instructions, meaning the same criteria a human reviewer would apply are written into the prompts the models receive.

Selection is only part of the task. Beyond identifying relevant papers, the system extracts specific information from documents, such as application details or datasets, and harmonizes it into a standardized metadata schema. This harmonization step is what turns a pile of heterogeneous full texts into structured records that can be compared, pooled and reanalyzed, the core promise of any systematic review.

To test the approach, the researchers ran an experimental validation against an existing systematic literature review on Human Action Quality Assessment, a field that studies how well people perform movements. The framework achieved an extraction recall of 85.11 percent for domain-specific datasets, meaning it recovered the large majority of the dataset information that the original review had identified. Notably, it also identified 25 additional data sources that the original review had not reported, suggesting the automated pipeline can surface material that manual screening misses.

Detection was less complete. The pipeline achieved an overall article detection rate of 71.28 percent, leaving a substantial share of relevant articles unidentified. The authors attribute this shortfall primarily to the strict, dataset-oriented Boolean query adopted in the identification phase rather than to failures in the automated LLM or RAG components. In other words, the bottleneck lay in the deliberately narrow search strategy, not in the language models themselves, a distinction that matters for anyone considering where to invest effort when adapting the framework to a new field.

Reliability across repeated runs is a persistent concern for LLM-based tools, since stochastic generation can produce different answers to the same question. Supplementary analyses reported in the study confirmed near-deterministic behavior across repeated runs, indicating that the pipeline returns consistent results when executed multiple times. The analyses also confirmed successful transfer to a different LLM family at negligible computational cost, evidence that the framework is not locked to a single model provider and can be re-based on alternative models without significant expense.

These results suggest the framework can effectively assist in identifying relevant studies and extracting information while maintaining alignment with PRISMA 2020 guidelines. The authors emphasize that the approach is structured to facilitate adaptation to other research areas by declaring new query and prompt templates, so moving the system from action quality assessment to another domain is framed as a configuration task rather than a rebuild.

The contribution lands amid growing interest in automating the most mechanical parts of evidence synthesis. By operationalizing PRISMA stages as a single pipeline built from openly available components, the framework makes each step re-executable and auditable, properties that matter because systematic reviews are expected to be transparent and reproducible in a way that ordinary narrative reviews are not. The structured prompt templates serve a documentation role as well as a functional one: the inclusion and exclusion criteria are written down in machine-interpretable form, so another team can inspect exactly what rules the automated screener applied.

Limitations remain visible in the reported numbers. An extraction recall of 85.11 percent for domain-specific datasets is strong but not exhaustive, and an article detection rate of 71.28 percent means the strict Boolean query sacrificed recall at the identification stage. The validation was conducted on a single reference review in one domain, so performance in other fields has not yet been demonstrated by this study. The framework is positioned as an assistant to the review process rather than a replacement for expert judgment, with the automated stages handling screening and extraction while the underlying review design, criteria and interpretation remain with the researchers who define them.

Subject of Research: A retrieval-augmented large language model framework for automating PRISMA-aligned systematic reviews, validated on a human action quality assessment literature review.

Article Title: A retrieval-augmented LLM framework for PRISMA-aligned systematic reviews: a case study on action quality assessment

Article References: Silvestri, R., Pincheira, M., Vecchio, M., & Antonelli, F. (2026). A retrieval-augmented LLM framework for PRISMA-aligned systematic reviews: a case study on action quality assessment. Neural Computing and Applications, 38(21), Article 869. https://doi.org/10.1007/s00521-026-12546-6

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12546-6

Keywords: systematic reviews, large language models, retrieval-augmented generation, PRISMA 2020, action quality assessment, literature screening, data extraction, reproducibility, research automation, Neural Computing and Applications, retrieval-augmented, framework