AI chatbots differ in reliability, usefulness and readability for dental implant questions

Four artificial-intelligence chatbots produced broadly reliable answers to 99 public questions about dental implants, but their performance diverged in readability, usefulness and expert-rated quality. In the comparative study, DeepSeek V3.2 received the highest reliability and quality scores, while ChatGPT-5 generated the most useful responses. Microsoft Copilot and DeepSeek produced comparatively accessible text, whereas Gemini 3 Pro generated the most difficult-to-read answers. The results suggest that these systems can supplement patient education about implants, but the choice of model changes how information is delivered, and none was clinically validated for individual patient care.

Researchers began with an exploratory search for questions commonly asked about dental implants, using three online question-suggestion platforms: Also Asked, Answer the Public and Google’s “People Also Ask” feature. Searches covered terms including dental implants, implant pain, complications, cost and longevity. The search generated 176 questions. Two reviewers removed duplicates and semantically identical items, resolving disagreements by discussion and, when necessary, consultation with a third senior reviewer. The final set contained 99 unique, grammatically reviewed questions divided into four domains: biological and surgical aspects, with 16 questions; prosthetic and material science, with 22; clinical outcomes and complications, with 26; and patient-centered and economic considerations, with 35. The unequal distribution reflected the topics represented in the public searches rather than an attempt to create equal comparison groups.

Each question was submitted separately to ChatGPT-5, Gemini 3 Pro, DeepSeek V3.2 and Microsoft Copilot on December 4, 2025. The investigators used the free consumer versions of the platforms, retained their default settings, accessed them in incognito windows and disabled retrieval tools. Every question began in a new conversation, and pages were refreshed before the next submission to prevent context from carrying over. The procedure also excluded use of the “regenerate response” option. Responses were collected by one investigator and transferred into anonymized documents so that the two assessors could score them without knowing the assigned model. This reduced contextual and identification bias, although it tested only one response from each model on one particular day.

Two board-certified specialists in fixed prosthodontics evaluated the answers independently. They used the Global Quality Scale, or GQS, to assess overall quality, coherence, flow and relevance; a modified four-point usefulness scale; and the CLEAR reliability framework, which rates completeness, absence of false information, evidence support, appropriateness and relevance. Each CLEAR element was scored from one to five, producing a total from five to 25. Totals of 19–25 were classified as “very good,” 12–18 as average and 5–11 as poor. Readability was assessed with the Flesch–Kincaid Grade Level and Simple Measure of Gobledygook indices. Before the main evaluation, both specialists scored a separate pilot set of 40 question-answer pairs that was excluded from analysis, discussed discrepancies and standardized their scoring with a third senior specialist.

All 396 chatbot responses fell into the study’s highest CLEAR category, with a median total score of 25 for DeepSeek and medians of 24 for the other three systems. The investigators reported statistically significant differences among models in reliability, quality, usefulness and readability, with p values below 0.001 for the overall comparisons. DeepSeek had a mean CLEAR score of 24.68, compared with 23.31 for ChatGPT, 23.44 for Gemini and 23.65 for Copilot, and it had a median of 25 with an interquartile range of zero. It also produced the largest proportion of responses rated “excellent” on the GQS: 83 of 99, or 83.8%. By contrast, 14.1% of ChatGPT answers, 24.2% of Gemini answers and 10.1% of Copilot answers received that rating. These high scores concern the experts’ judgments under this protocol; they do not establish that the outputs are safe, traceable to evidence or suitable for every patient.

Readability produced one of the clearest contrasts. DeepSeek had the lowest mean Flesch–Kincaid Grade Level score, 8.67, followed by Copilot at 9.02, ChatGPT at 9.42 and Gemini at 11.07. A higher FKG score indicates more complex text, so the ordering placed DeepSeek closest to an eighth-grade reading level and Gemini highest. On the SMOG index, Copilot ranked as easiest at a mean of 11.11, followed by DeepSeek at 11.39, ChatGPT at 11.64 and Gemini at 13.18. A mixed-design analysis found that model choice accounted for 39% of the variance in FKG scores and 42% of the variance in SMOG scores. Gemini’s responses were significantly harder to read than those from the other three systems, while ChatGPT was harder to read than DeepSeek by the FKG measure and Copilot by the SMOG measure. Question category did not significantly affect either readability score, nor was there a significant interaction between topic and model.

Under the study’s reversed usefulness scale, lower scores represented more useful answers. ChatGPT performed best in this dimension: one response was rated “very useful,” 42 were rated “useful,” 51 were classified as “partially useful” and five were “not useful.” Gemini and Copilot each had 27 “useful” responses, while DeepSeek had only one answer in that category; 39 of its 99 responses were judged “not useful.” ChatGPT’s overall usefulness advantage was statistically significant after adjustment for multiple comparisons. Yet the pattern reveals a trade-off: DeepSeek’s answers were rated more reliable and higher in overall quality, but they were less useful according to the patient-oriented rating. The authors interpret this as evidence that greater informational depth does not necessarily translate into more accessible or user-friendly communication.

The study also distinguished reliability, quality and usefulness as separate constructs. GQS and CLEAR assessed characteristics such as coherence, completeness, support, absence of false information, appropriateness and relevance, whereas usefulness was intended to capture how well an answer met a patient’s informational needs. The distinction matters because a technically strong response may still be difficult to use. Indeed, higher readability scores were associated with lower GQS scores for Gemini, lower usefulness scores for ChatGPT and lower CLEAR scores for Gemini and Copilot. Question category had no significant effect on usefulness or GQS scores, although patient-centered and economic questions showed a modest reliability advantage over prosthetic and material science and clinical-outcome questions in the CLEAR analysis. The authors emphasized that this effect was small and had limited practical impact.

Despite the reassuring reliability results, the system produced no directly linked references that would allow each clinical statement to be independently traced, and the clear ceiling effect reduced the ability of the CLEAR tool to distinguish among systems. The authors also note that the framework itself was validated by a small number of healthcare professionals and that multiple assessment tools are therefore needed. More broadly, the findings apply only to English-language, open-ended questions answered in a controlled consumer setting. Model outputs may change as software is updated; repeated queries were not tested, patient comprehension was not measured, and no clinical outcomes were evaluated. Readability indices quantify sentence and word complexity rather than actual understanding, which can vary with health literacy, education, prior knowledge and familiarity with terms such as osseointegration or peri-implantitis.

For people seeking information about dental implants, the results offer a qualified snapshot rather than a clinical verdict. DeepSeek led this test in expert-rated reliability and response quality, ChatGPT led in usefulness, and DeepSeek and Copilot were easier to read by the measures used. All four systems received favorable reliability scores, but even an answer judged “very good” can omit patient-specific considerations such as bone quality, systemic health or radiographic findings. The researchers therefore position these tools as adjuncts within a clinician-supervised, human-in-the-loop approach, not substitutes for professional consultation. Their conclusion calls for studies involving actual patients, clinically validated outcomes, multilingual responses, repeated queries and newer model versions to determine whether differences in generated text translate into better understanding, confidence or anxiety.

Subject of Research: Comparison of four AI chatbots answering public questions about dental implants

Article Title: The use of different artificial intelligence-powered chatbots to answer questions related to dental implants

Article References: Alamoush, R. A., Atrash, M., Abusidu, Z., Sabrah, A. H. A., Alraheam, I. A., Alnsour, M. M., & AL-Omiri, M. K. (2026). The use of different artificial intelligence-powered chatbots to answer questions related to dental implants. Frontiers in Oral Health, 7, Article 1975960. https://doi.org/10.3389/froh.2026.1975960

Image Credits: AI Generated

DOI: 10.3389/froh.2026.1975960

Keywords: artificial intelligence, chatbots, dental implants, digital dentistry, large language models, patient education, health literacy, response reliability, response quality, readability, different, artificial