In a cross-sectional comparative evaluation of conversational artificial intelligence platforms, researchers determined that ChatGPT generated significantly more accurate and reliable answers to layperson orthodontic queries than Google Gemini or Microsoft Copilot. The investigation assessed how effectively these widely accessible chatbots address common questions regarding retainers, mini-implants, and lateral cephalograms, identifying measurable differences in reliability across models and topic domains.
Artificial intelligence systems based on large language models have rapidly become prominent information sources in medical and dental contexts, yet their factual dependability for orthodontic inquiries has remained uncertain. To evaluate their performance under standardized conditions, investigators examined how three free conversational tools answered questions typical of prospective dental patients seeking foundational knowledge online.
The experimental protocol utilized a dataset of 61 orthodontic questions gathered directly from frequently asked queries in the Google search bar. Between May 10 and May 15, 2026, identical standardized prompts formulated for a lay audience were entered into the free tiers of ChatGPT using GPT-4o, Gemini running version 1.5 Flash, and Microsoft Copilot across three primary orthodontic categories: retainers, mini-implants, and lateral cephalograms.
To evaluate the outputs, five board-certified orthodontists independently reviewed each AI-generated response using a validated five-point scoring rubric ranging from 1, representing completely inaccurate or poor content, to 5, designating highly accurate or excellent explanations. Inter-examiner agreement among the five clinical evaluators was excellent, demonstrating a two-way random-effects intraclass correlation coefficient of 0.87 with a 95 percent confidence interval spanning from 0.81 to 0.92.
Statistical evaluation revealed significant overall performance disparities among the three platforms, evidenced by a Friedman test statistic of 48.32 with a p-value below 0.001 and a Kendall’s W effect size of 0.396. Across all 61 evaluated prompts, ChatGPT achieved the highest overall mean score of 4.31 with a 95 percent confidence interval from 4.20 to 4.42, demonstrating superior educational quality across the tested domains.
Gemini achieved the second-highest overall mean rating at 3.94 with a 95 percent confidence interval from 3.82 to 4.06, while Microsoft Copilot recorded an overall mean score of 3.67 with a 95 percent confidence interval between 3.53 and 3.81. Pairwise post-hoc Wilcoxon signed-rank tests applying a Bonferroni correction threshold confirmed that ChatGPT significantly outperformed both Gemini and Copilot, with p-values falling below 0.001 for both comparisons.
Topic-specific consistency varied notably across the different orthodontic domains analyzed in the trial.
The study concluded that while AI-driven conversational agents present promising potential as supplementary educational instruments in orthodontics, substantial variability in scientific reliability and clinical consistency remains. The authors emphasized that limitations surrounding dynamic model updates and evaluator subjectivity necessitate careful professional supervision, underscoring that AI outputs should complement rather than replace professional clinical consultations.
Beyond general factual correctness, the study framework established four distinct evaluative criteria: reliability, accuracy, completeness, and clinical applicability. By evaluating questions across retainers, mini-implants, and lateral cephalograms against these multidimensional benchmarks, the protocol determined whether freely available AI platforms can generate sufficiently complete and dependable guidance for laypersons seeking basic orthodontic knowledge.
In analyzing the resulting scores, the investigators selected statistical methods suited to the paired structure of the experimental dataset. Because all three conversational platforms answered the identical 61 questions, the data points were paired per query rather than treated as independent samples. This dependent structure informed the non-parametric paired comparisons conducted via the Friedman test, which evaluated rank differences across two degrees of freedom and quantified the overall effect size using Kendall’s W. For post-hoc pairwise comparisons between individual AI models, the researchers applied Wilcoxon signed-rank tests adjusted by a Bonferroni correction threshold of alpha equals 0.017 to maintain statistical rigor. Furthermore, inter-examiner reliability was specifically calculated using a single-measures formulation, denoted as ICC (2,1), to assess the consistency of the scoring rubric across independent raters.
Subject of Research: Comparative accuracy and reliability of AI chatbots in answering basic orthodontic queries
Article Title: Assessing the reliability of artificial intelligence tools for laypersons seeking basic orthodontic knowledge: focus on retainers, mini-implants, and lateral cephalograms
Article References: Rizwan, S., Nasir, L., Ahmed, I., Sattar, A., Tabassum, H., & Anis, B. (2026). Assessing the reliability of artificial intelligence tools for laypersons seeking basic orthodontic knowledge: focus on retainers, mini-implants, and lateral cephalograms. BMC Oral Health. https://doi.org/10.1186/s12903-026-10439-7
Image Credits: AI Generated
DOI: 10.1186/s12903-026-10439-7
Keywords: orthodontics, artificial intelligence, ChatGPT, GPT-4o, Google Gemini, Gemini 1.5 Flash, Microsoft Copilot, dental education, retainers, mini-implants, lateral cephalograms, inter-examiner reliability, Friedman test, BMC Oral Health, clinical consultations, patient education
Cite Scienmag News
Ophelia Keating. (November 19, 2026). Study Finds ChatGPT Outperforms Gemini and Copilot on Basic Orthodontic Questions. Scienmag. https://scienmag.com/study-finds-chatgpt-outperforms-gemini-and-copilot-on-basic-orthodontic-questions/
Ophelia Keating. "Study Finds ChatGPT Outperforms Gemini and Copilot on Basic Orthodontic Questions." Scienmag, 19 November 2026, https://scienmag.com/study-finds-chatgpt-outperforms-gemini-and-copilot-on-basic-orthodontic-questions/. Accessed 19 November 2026.
Ophelia Keating. "Study Finds ChatGPT Outperforms Gemini and Copilot on Basic Orthodontic Questions." Scienmag. November 19, 2026. https://scienmag.com/study-finds-chatgpt-outperforms-gemini-and-copilot-on-basic-orthodontic-questions/








