The tool at the center of the study is the C-BeDevel-I, the Chinese version of the Behavior Development Screening for Toddlers administered as an interview. Unlike brief parent checklists such as the M-CHAT, which rely on yes-or-no questionnaire items, this is a second-level screening instrument: a physician sits down with a parent or primary caregiver and conducts a structured but conversational interview about the child’s development, social communication, and restricted and repetitive behaviors. The richness of that format is both its strength and its bottleneck. Narrative answers capture nuances that checkboxes miss, but scoring them demands trained professionals, time, and consistency. The researchers asked a deceptively simple question: could a large language model read the transcript of such an interview and produce a score that a clinician would trust?

To answer it, the team assembled three retrospective analytical datasets. The first, Cohort A, included 518 children aged 9 to 42 months and was used to establish the internal consistency and diagnostic validity of the C-BeDevel-I itself as a second-level screening parent interview for autism spectrum disorder. The second, Cohort B, was a carefully selected case-control sample of 256 distinct children, 130 with autism spectrum disorder and 126 with typical development, whose interview recordings were transcribed and then scored twice by three different AI systems: Gemini-2.5, GPT-5-mini, and Qwen-Plus. The third, Cohort C, served as an independent external test. It comprised 60 children, 30 with autism and 30 with typical development, enrolled at the Department of Child Health of Lianyungang First People’s Hospital, a separate center where the clinical diagnosis had been established before any AI scoring took place.

The technical pipeline behind the experiment is as important as the results. Interview audio was converted to text using automated speech recognition, and the resulting transcripts were passed to the language models through an application programming interface. The workflow incorporated retrieval-augmented generation, a technique that grounds the model’s judgments in reference material, such as the scoring manual, retrieved at the moment of inference rather than relying solely on what the model absorbed during training. Chain-of-thought prompting was used to encourage the models to reason step by step through each interview item before committing to a score, mirroring the way a trained clinician works through a rating scale. Each model produced a total score for every interview, and the first complete output was treated as the primary result, while the second, independent call assessed how stable the model’s judgments were when the same transcript was scored again.

The headline numbers are striking. In Cohort B, the area under the receiver operating characteristic curve, a standard measure of how well a test separates cases from controls, reached 0.9961 for Gemini-2.5, with a 95 percent stratified-bootstrap confidence interval of 0.9916 to 0.9991. GPT-5-mini achieved 0.9861 and Qwen-Plus 0.9770. After Holm correction of paired DeLong tests, only the difference between Gemini-2.5 and Qwen-Plus remained statistically significant, an AUC difference of 0.0191 with an adjusted p-value of 0.012, while the other pairwise comparisons fell just short of significance. In practical terms, all three models separated autistic children from typically developing children in this selected case-control sample with near-ceiling discrimination, with Gemini-2.5 statistically ahead of only one competitor.

Discrimination alone, however, is not the same as agreement with human experts, and the researchers measured both. Gemini-2.5’s scores showed a concordance correlation coefficient of 0.9631 with expert total scores, a mean absolute error of just 0.824 points, and a mean AI-minus-expert difference of +0.184 points, indicating negligible systematic bias. The limits of agreement ranged from −2.235 to 2.602 points on the scale, meaning that in the vast majority of cases the machine landed within a few points of the clinician. When the same transcript was scored twice, the intraclass correlation coefficient for repeated outputs was 0.9875, an unusually high figure for a generative system whose outputs are not deterministic. Categorized against the diagnostic cutoff, the model achieved a sensitivity of 0.9692 and a specificity of 0.9206, correctly flagging nearly all autistic children while wrongly labeling fewer than one in twelve typically developing children as at risk.

The external validation in Cohort C provided the most sobering and arguably most informative test. At the independent hospital, Gemini-2.5 correctly classified 53 of 60 children, yielding a sensitivity of 0.9333, a specificity of 0.8333, and an overall accuracy of 0.8833. Score concordance with experts remained high at 0.9387, and the mean absolute error rose only slightly to 0.875 points. Specificity, the ability to correctly clear children who do not have autism, dropped compared with the internal cohort, a pattern consistent with the well-known performance gap between selected case-control samples and messier real-world populations. The authors are explicit that these findings demonstrate feasibility and support further clinician-supervised evaluation, not yet real-world clinical effectiveness.

That caution matters because of how the study was designed. The cohorts were retrospective and case-control by construction, deliberately balancing autistic children against typically developing peers. Real screening clinics see a far more heterogeneous stream of children, including many with developmental delay, language disorders, ADHD, and other neurodevelopmental conditions that can mimic or co-occur with autism. A screening tool that performs brilliantly at distinguishing autism from typical development may perform less impressively at distinguishing autism from these look-alike conditions, and the authors themselves call for future prospective studies using larger, consecutive, and clinically heterogeneous cohorts with neurodevelopmental comparators. The diagnostic reference in each cohort was a consensus DSM-5 clinical assessment, which is a strong standard, but the children whose interviews fed the AI were not a random sample of the screening population.

Even with those caveats, the implications are considerable. Second-level autism screening is a bottleneck in many health systems: trained clinicians are scarce, waitlists are long, and the earlier a child is identified, the earlier intervention can begin, with well-documented benefits for developmental outcomes. A workflow in which a physician conducts the interview and an AI drafts the scoring, subject to clinician review, could compress the time between interview and actionable result from days or weeks to minutes. The study also addressed the ethical infrastructure that such a workflow demands: it was conducted under the Declaration of Helsinki with central ethics approval from Nanjing Brain Hospital, written informed consent from all parents and guardians explicitly covered audio recording, automated transcription, and the transmission of de-identified transcript text to commercial AI providers strictly for research evaluation, and the authors declared no competing interests. The work was funded by the National Natural Science Foundation of China and the Nanjing Science and Technology Development Plan.

The deeper significance of the study may lie less in the specific scores than in what it demonstrates about the maturing relationship between language models and clinical measurement. Scoring a narrative interview is precisely the kind of task that defeated earlier generations of machine learning, which needed rigid input formats and labeled features. Modern large language models, augmented with retrieval and structured reasoning prompts, can now operate on the raw substance of a clinical conversation, and they can do so with quantified repeatability, agreement statistics, and confidence intervals rather than anecdotes. The path from a retrospective case-control demonstration to a deployed screening aid runs through prospective trials, regulatory scrutiny, and careful attention to bias across languages, cultures, and clinical populations. But the study offers a concrete, statistically disciplined proof of concept that the AI can meet the expert halfway, and for families waiting on an autism assessment, that halfway point may one day mark the difference between early answers and long silence.

Subject of Research: Large language model-assisted scoring of parent interviews for early autism screening in young children

Article Title: Large language model–assisted scoring of physician-administered parent interviews for autism screening in young children: a retrospective diagnostic case-control and agreement study

Article References: Zhu, J., Yue, Z., Han, Y., Yoo, H. J., Bong, G., Chen, H., Dai, Y., Qiu, N., Yin, S., Ma, Y., Wang, S., & Ke, X. (2026). Large language model–assisted scoring of physician-administered parent interviews for autism screening in young children: a retrospective diagnostic case-control and agreement study. BMC Psychiatry. https://doi.org/10.1186/s12888-026-08686-7

Image Credits: AI Generated

DOI: 10.1186/s12888-026-08686-7

Keywords: autism spectrum disorder, large language models, artificial intelligence, autism screening, C-BeDevel-I, diagnostic accuracy, retrieval-augmented generation, automated scoring, early childhood development, BMC Psychiatry, clinical agreement, case-control study