The motivation behind the study is rooted in a long-documented inequity in global science. Authors from low- and middle-income countries and from regions where English is not the primary language face systemic disadvantages when preparing manuscripts for international journals. Previous studies cited by the team have shown that geographic bias shapes peer reviewer selection, that English-language dominance disadvantages researchers in places like Colombia, and that historically excluded groups continue to encounter barriers in the peer review process. Rejection and resubmission cycles impose real costs on early-career researchers everywhere, but the burden falls hardest on those without access to experienced mentors or institutional support.

The Duke-led team set out to test whether an accessible AI tool could help level that playing field. They designed a controlled comparison of three progressively refined AI approaches, all reachable through an ordinary ChatGPT account, meaning no specialized software, no programming expertise, and no additional cost beyond a standard subscription. The first tool was a baseline deployment of GPT-5 with no special configuration. The second was a Custom GPT augmented with a knowledge base of peer review guidance. The third and most sophisticated version layered on iterative optimization: enhanced system instructions developed through meta-prompting, a technique in which the model itself is used to help draft and refine the instructions given to it, combined with journal-specific guidelines drawn from the field of plastic surgery.

To evaluate these tools rigorously, the researchers assembled sixty manuscript evaluations covering fifteen plastic surgery articles, deliberately spread across five clinical studies, five clinical trials, and five review articles. Each AI-generated evaluation was compared against human peer reviews obtained from journals that practice open peer review, where reviewer reports are published alongside the paper. Two blinded raters then scored every evaluation—human and machine alike—using the Review Quality Instrument, a validated assessment tool developed in 1999 that measures review quality on a scale with a maximum score of 40. The raters did not know which reviews came from humans and which came from AI.

The results were striking. The iteratively optimized Custom GPT, Tool 3, achieved a mean Review Quality Instrument score of 28.5 with a standard deviation of 3.8, significantly higher than every other review type in the study. Human peer reviews averaged 25.2 plus or minus 4.9, the knowledge-base Custom GPT averaged 24.3 plus or minus 5.0, and the baseline GPT-5 averaged 24.0 plus or minus 1.8. The differences were statistically significant at p less than 0.001. In other words, the carefully engineered AI tool did not merely match human reviewers—it outperformed them on average, while the unoptimized baseline model sat slightly below human performance.

Digging into the subcomponents of review quality revealed where the optimized tool excelled most. Using Kendall’s W, a statistical measure of effect size for non-parametric comparisons, the team found that Tool 3 showed its largest advantages in methodological assessment, evidence provision, and interpretation evaluation, with effect sizes of 0.69, 0.59, and 0.63 respectively. These are precisely the dimensions that separate a superficial review from a genuinely useful one: whether the reviewer critically appraises the study design, whether claims are backed by specific evidence from the manuscript, and whether the reviewer interprets the findings in context rather than offering generic praise or criticism.

Equally notable was the tool’s consistency. Across the three manuscript types—clinical studies, clinical trials, and reviews—the optimized AI’s scores ranged only from 28.0 to 29.1, a narrow band suggesting stable performance regardless of article genre. Human reviews, by contrast, showed considerably greater variability, with mean scores ranging from 22.1 to 27.0 across manuscript types. That variability mirrors a well-known reality of peer review: quality depends heavily on the individual reviewer, their time constraints, and their familiarity with the subject matter. A tool that delivers reliably structured feedback could smooth out some of that randomness for authors seeking pre-submission guidance.

The study builds on a growing body of work examining whether large language models can provide useful feedback on research papers. Earlier analyses, including a large-scale empirical study published in NEJM AI, found that models like GPT-4 could identify weaknesses in papers that human reviewers sometimes missed, though concerns about hallucinated criticisms and superficiality persisted. Subsequent studies in hand surgery and transplantation research have compared AI-generated and human peer reviews with mixed results. What distinguishes the new work is its focus on accessibility and optimization: rather than asking whether AI can review manuscripts in the abstract, the team asked how much better a freely accessible tool can become through systematic instruction refinement and domain-specific knowledge integration.

The implications for global surgical research are considerable. A researcher in a resource-limited setting could, in principle, paste a draft manuscript into the optimized tool and receive structured, actionable feedback on methodology, evidence, and interpretation—feedback that might otherwise require a mentor at a well-funded institution. The authors emphasize that the tool requires no technical expertise and no additional cost, which addresses two of the most common barriers to AI implementation in healthcare identified in prior literature: control and cost. Because the tool is designed for pre-submission refinement rather than replacing journal peer review, it also sidesteps some of the thornier ethical questions about AI’s role in editorial decision-making, though the broader debate about generative AI in scientific publishing remains unresolved, as a 2024 Lancet commentary highlighted.

Cautions remain. The study evaluated reviews of plastic surgery manuscripts scored by two raters using a single validated instrument, and the human comparison reviews came from open peer review journals, which may not represent the full spectrum of review quality across all journals. The Review Quality Instrument measures the construct of a review report, not the ultimate accuracy of an AI’s scientific judgments, and large language models can still produce confident errors. The authors report no external funding and no conflicts of interest, and they have made their evaluation scores available in supplementary materials for scrutiny. Still, the central result stands as a provocative data point: with thoughtful meta-prompting and domain knowledge, a tool anyone can access from a standard ChatGPT account produced manuscript evaluations that blinded raters judged better than those written by human experts. For the millions of researchers excluded from the informal networks of academic mentorship, that could be a genuinely democratizing development.

Subject of Research: Development and evaluation of an accessible AI-powered tool for manuscript peer review in plastic surgery research

Article Title: Development and assessment of an accessible AI-powered tool for manuscript evaluation through iterative optimization in plastic surgery research

Article References: Goyal, R., Liang, J., Kulkarni, M. S., Nguyen, P., Gillipelli, S., & Patel, A. (2026). Development and assessment of an accessible AI-powered tool for manuscript evaluation through iterative optimization in plastic surgery research. Global Surgical Education – Journal of the Association for Surgical Education, 5(1), Article 188. https://doi.org/10.1007/s44186-026-00599-z

Image Credits: AI Generated

DOI: 10.1007/s44186-026-00599-z

Keywords: artificial intelligence, large language models, ChatGPT, peer review, manuscript evaluation, plastic surgery, global surgery, meta-prompting, Review Quality Instrument, research equity, low- and middle-income countries, scientific publishing