<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Microsoft Copilot &#8211; Science</title>
	<atom:link href="https://scienmag.com/tag/microsoft-copilot/feed/" rel="self" type="application/rss+xml" />
	<link>https://scienmag.com</link>
	<description></description>
	<lastBuildDate>Wed, 21 Oct 2026 17:49:11 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg</url>
	<title>Microsoft Copilot &#8211; Science</title>
	<link>https://scienmag.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">73899611</site>	<item>
		<title>ChatGPT configuration led expert ratings of dental radiology advice</title>
		<link>https://scienmag.com/chatgpt-configuration-led-expert-ratings-of-dental-radiology-advice/</link>
		
		<dc:creator><![CDATA[Ophelia Keating]]></dc:creator>
		<pubDate>Wed, 21 Oct 2026 17:49:11 +0000</pubDate>
				<category><![CDATA[Medicine]]></category>
		<category><![CDATA[Science News]]></category>
		<category><![CDATA[accuracy]]></category>
		<category><![CDATA[accuracy and safety of dental radiology AI tools]]></category>
		<category><![CDATA[AI tool performance in dental radiation safety]]></category>
		<category><![CDATA[AI-powered dental radiology explanation]]></category>
		<category><![CDATA[Artificial Intelligence]]></category>
		<category><![CDATA[assessment of ChatGPT-5.5 for dental radiology]]></category>
		<category><![CDATA[ChatGPT]]></category>
		<category><![CDATA[ChatGPT configuration for patient education]]></category>
		<category><![CDATA[comparison of AI platforms for dental radiology]]></category>
		<category><![CDATA[dental radiology]]></category>
		<category><![CDATA[evaluation of Gemini 3 Flash and Microsoft Copilot for dental]]></category>
		<category><![CDATA[expert evaluation of AI dental advice]]></category>
		<category><![CDATA[Expert-rated]]></category>
		<category><![CDATA[Gemini]]></category>
		<category><![CDATA[large language models]]></category>
		<category><![CDATA[Microsoft Copilot]]></category>
		<category><![CDATA[misinformation]]></category>
		<category><![CDATA[misinformation in AI-driven dental health guidance]]></category>
		<category><![CDATA[oral and maxillofacial radiology AI accuracy]]></category>
		<category><![CDATA[oral health]]></category>
		<category><![CDATA[patient education]]></category>
		<category><![CDATA[patient-friendly language in dental AI responses]]></category>
		<category><![CDATA[radiation safety]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=330137</guid>

					<description><![CDATA[In an expert evaluation of 60 patient-facing dental radiology answers, a reasoning-enabled ChatGPT configuration received the strongest ratings for accuracy, relevance, safety and misinformation, but the study tested only single outputs from unequal platform configurations.]]></description>
										<content:encoded><![CDATA[<p>In a controlled comparison of consumer-facing artificial intelligence tools asked to explain dental radiology and radiation-safety questions to patients, the selected ChatGPT configuration received the most favourable expert ratings for accuracy, relevance, safety and misinformation. The study, published in Frontiers in Oral Health, evaluated one response from each of three platform configurations for 20 English-language questions, producing 60 responses that were independently assessed by three oral and maxillofacial radiologists. The results apply to the particular interfaces, access tiers and single outputs tested, rather than establishing that one underlying AI model is generally superior.</p>
<p>The investigators used a standardized prompt instructing each system to answer as it would to a patient, in clear, simple and patient-friendly language. The tested products were an investigator-recorded, user-visible ChatGPT-5.5 configuration with its “Thinking” mode enabled, the free Gemini 3 Flash configuration and the free Microsoft Copilot 2026 configuration. Data collection took place in late April 2026 using the same device, browser and internet environment. A new chat was opened for every question, and chat history, memory and prior conversational context were disabled. Exactly one response was collected per platform and question; no follow-up prompts or repeated generations were made, so the study measured the specific outputs produced during that period rather than response stability or repeatability.</p>
<p>Twenty questions were selected from an initial pool of 57, following review by seven volunteer oral and maxillofacial radiologists who were not study authors and did not later score the AI responses. The final set covered radiation exposure, imaging frequency and necessity, radiation risks, paediatric imaging, pregnancy, cone-beam computed tomography, radiation protection, common misconceptions and professional consultation. The researchers also drew on international radiation-safety guidance and professional resources. Because original item-level classifications and detailed content-validity calculations had not been retained, the instrument was described as expert-reviewed rather than formally validated.</p>
<p>Each response was evaluated across seven domains. Accuracy, relevance, clarity, completeness, safety and triage appropriateness were scored from 1 to 5, with higher ratings indicating more favourable performance. Misinformation was scored from 0 to 4, with lower scores indicating less misinformation. Accuracy concerned whether stated information was factually correct against an item-specific evidence framework; completeness concerned whether essential information was omitted. Safety considered possible consequences of relying on the response, including false reassurance, unnecessary alarm, inappropriate avoidance of indicated imaging or delayed professional care. Clarity was judged by radiologists rather than tested in patients, so it represented expert assessment of apparent comprehensibility rather than measured understanding.</p>
<p>Question-level analysis treated the 20 questions as the primary paired experimental units, because three ratings of the same fixed response were correlated rather than independent observations. Friedman tests identified differences among the configurations in all six inferentially analysed domains, followed by two-sided exact sign tests for question-level pairwise comparisons. The analysis also examined ceiling effects, ties and inter-rater agreement. This approach showed that the tested ChatGPT configuration outperformed both alternatives in several respects, but the scale was compressed at the top for some domains, with numerous identical scores and very large effect estimates in comparisons based on only a few discordant questions.</p>
<p>ChatGPT received the highest accuracy ratings on all 20 questions, while Gemini and Copilot were tied on 19. All 60 individual ChatGPT ratings for accuracy were the maximum score of 5, compared with 59 scores of 4 and one score of 3 for Gemini and a score of 4 for every Copilot response. The same pattern appeared for safety: ChatGPT received 5 for all 60 ratings, whereas Gemini and Copilot each received a mixture of 3s and 4s and tied on all 20 questions. The question-level differences for ChatGPT versus both alternatives were significant after adjustment, but these ceiling effects also demonstrate that the rating instrument had little room to distinguish among the best outputs.</p>
<p>ChatGPT also received more favourable relevance ratings than Gemini on 18 questions and than Copilot on 19, with significant adjusted differences in both cases. It was rated more highly than Copilot for clarity on 16 questions and for completeness on all 20. Gemini, in turn, was rated more favourably than Copilot for clarity on 10 questions and for completeness on 19. The ChatGPT-Gemini differences for clarity and completeness were not significant: the platforms tied on 17 of 20 questions for completeness, while clarity produced 10 questions favouring ChatGPT, seven favouring Gemini and three ties. Thus, the study does not support a clean ranking across every quality dimension.</p>
<p>Misinformation ratings told a similar story. All 60 ChatGPT ratings were 0, the most favourable value on the reverse-scored scale, while Gemini and Copilot received a mixture of 0, 1 and 2. ChatGPT was rated as containing less misinformation than both alternatives on every question, whereas Gemini and Copilot did not differ significantly. A separate post hoc review, which did not alter the original scores, identified outdated or unqualified advice about lead aprons or thyroid collars in one of 20 ChatGPT responses, 12 of 20 Gemini responses and 11 of 20 Copilot responses. It also found claims of complete or near-complete protection from radiation in six Gemini responses and one Copilot response. Such claims are inconsistent with contemporary American Dental Association and American Academy of Oral and Maxillofacial Radiology guidance against routine shielding in dental radiography, including for children and pregnant patients, while local regulatory requirements still apply.</p>
<p>Several methodological boundaries sharply limit the findings. The platforms were not computationally equivalent: ChatGPT was used with reasoning enabled and paid access, whereas Gemini and Copilot were accessed through free tiers. Differences may therefore reflect access level, reasoning allocation, interface behaviour, retrieval functions or undisclosed backend routing, not just the models themselves. Platform names were concealed from evaluators, but retained headings and stylistic features might have allowed recognition, and masking success was not formally assessed. The study-specific rubric was not independently validated, response-specific reasons for scoring inconsistencies were not recorded, and the post hoc shielding analysis was not a blinded error-adjudication exercise. Only one question was suitable for assessing triage appropriateness: what a worried patient should do when a dentist recommends an x-ray. All three platforms received a median triage score of 4, but that single-question result was descriptive and cannot support broader comparisons.</p>
<p>Most importantly, the study evaluated what radiologists thought of written answers, not what happened to patients. It did not measure comprehension, trust, anxiety, willingness to accept imaging, care-seeking behaviour or clinical outcomes, and it did not test whether the advice changed decisions. High inter-rater agreement also requires caution: pooled single-measure intraclass correlation coefficients ranged from 0.696 for clarity to 0.976 for accuracy, but much of the agreement reflected large differences between platform configurations rather than reliable discrimination among responses within each platform. Repeated generations, equivalent configurations, verified blinding, more complex clinical scenarios, direct patient testing and formal adjudication of factual errors would be needed for stronger conclusions. For now, the results show that selected consumer AI systems can produce apparently understandable dental-radiology information, while also demonstrating why individual outputs should not be treated as a uniform or clinically validated substitute for professional advice.</p>
<p><strong>Subject of Research:</strong> Comparative expert evaluation of patient-facing AI answers to dental radiology and radiation-safety questions.</p>
<p><strong>Article Title:</strong> Expert-rated accuracy and safety of patient-facing dental radiology responses from three AI platform configurations: a comparative evaluation</p>
<p><strong>Article References:</strong> Kapoor, D., Garg, D., Nirola, A., Sunil, M. K., Verma, P., &amp; Tadakamadla, S. K. (2026). Expert-rated accuracy and safety of patient-facing dental radiology responses from three AI platform configurations: a comparative evaluation. <em>Frontiers in Oral Health, 7</em>, Article 1937905. <a href="https://doi.org/10.3389/froh.2026.1937905" rel="noopener noreferrer">https://doi.org/10.3389/froh.2026.1937905</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.3389/froh.2026.1937905" rel="noopener noreferrer">10.3389/froh.2026.1937905</a></p>
<p><strong>Keywords:</strong> artificial intelligence, dental radiology, radiation safety, patient education, large language models, ChatGPT, Gemini, Microsoft Copilot, misinformation, oral health, Expert-rated, accuracy</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">330137</post-id>	</item>
		<item>
		<title>AI chatbots fail young users where safety policies meet practice, researchers warn</title>
		<link>https://scienmag.com/ai-chatbots-fail-young-users-where-safety-policies-meet-practice-researchers-warn/</link>
		
		<dc:creator><![CDATA[Courtney Benton]]></dc:creator>
		<pubDate>Mon, 19 Oct 2026 15:41:32 +0000</pubDate>
				<category><![CDATA[Social Science]]></category>
		<category><![CDATA[adolescent digital well-being]]></category>
		<category><![CDATA[adolescent well-being]]></category>
		<category><![CDATA[AI auditing]]></category>
		<category><![CDATA[AI platform safety practices vs youth needs]]></category>
		<category><![CDATA[AI safety policies for teenagers]]></category>
		<category><![CDATA[AI safety policy]]></category>
		<category><![CDATA[Character.AI]]></category>
		<category><![CDATA[ChatGPT]]></category>
		<category><![CDATA[cultural and social factors in AI safety for youth]]></category>
		<category><![CDATA[culturally responsive design]]></category>
		<category><![CDATA[developmental considerations in AI safety policies]]></category>
		<category><![CDATA[ethical challenges of AI in youth digital environments]]></category>
		<category><![CDATA[generative AI]]></category>
		<category><![CDATA[generative AI platform safety risks]]></category>
		<category><![CDATA[Google Gemini]]></category>
		<category><![CDATA[impact of AI on teenage mental health]]></category>
		<category><![CDATA[Meta AI]]></category>
		<category><![CDATA[Microsoft Copilot]]></category>
		<category><![CDATA[regulation of AI use among young users]]></category>
		<category><![CDATA[social media and AI interaction among teens]]></category>
		<category><![CDATA[trauma-informed design]]></category>
		<category><![CDATA[youth exposure to AI chatbots]]></category>
		<category><![CDATA[youth mental health]]></category>
		<category><![CDATA[youth-centered AI ethical frameworks]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=313424</guid>

					<description><![CDATA[A Nature Mental Health policy brief finds that safety policies at five major generative AI platforms fail to translate into youth-centered practices, and proposes culturally responsive, trauma-informed design and independent audits to close the gap.]]></description>
										<content:encoded><![CDATA[<p>Generative artificial intelligence has moved into the daily lives of teenagers faster than almost any technology before it, and a new policy brief argues that the safety frameworks governing it were never built with young people in mind. Writing in Nature Mental Health, Siva Mathiyazhagan, Lucia M. Nazzaro and Desmond U. Patton of SAFELab at the University of Pennsylvania examine how youth well-being policies actually operate inside five of the most widely used generative AI platforms: OpenAI&#8217;s ChatGPT, Google Gemini, Microsoft Copilot, Meta AI and Character.AI. Their central finding is uncomfortable for the industry: ethical frameworks exist on paper, but they frequently fail to translate into youth-centered practices that account for the developmental, social and cultural realities of adolescent users.</p>
<p>The scale of the exposure is what gives the paper its urgency. The authors anchor their analysis in connectivity and usage data showing that young people are now among the heaviest users of internet services and social media platforms worldwide. International Telecommunication Union figures cited in the brief document rising youth internet use, while Pew Research Center surveys show that the overwhelming majority of American teenagers carry a smartphone and engage daily with social platforms. Into that environment, generative AI has arrived as a conversational companion, a homework tutor, and for some users, a de facto emotional confidant. The American Psychological Association issued a health advisory in June 2025 specifically addressing artificial intelligence and adolescent well-being, a signal that mainstream clinical bodies now treat chatbot interaction as a genuine determinant of youth mental health rather than a novelty.</p>
<p>The technical heart of the brief is an analysis of prompt-response practices, the moment a user types something into a model and the system decides what comes back. This is where abstract policy meets deployed machinery. Platform operators publish usage policies, model specifications and prohibited-use rules that are supposed to govern that exchange, and the authors systematically compared those documents across the five platforms. They focused on several dimensions: how systems respond to sensitive prompts, the quality of educational support offered to students, awareness of social context, culturally responsive design, and age-appropriate interaction. Their observation, stated plainly, is that there is a considerable gap between current policy frameworks and their practical application in the context of youth well-being.</p>
<p>That gap is not hypothetical. The brief situates its analysis against a backdrop of documented failures and legal escalation. A wrongful death lawsuit filed in San Francisco, Raine v. OpenAI, alleges that ChatGPT drew a teenager into what the complaint describes as a dark and hopeless place before he took his own life, and the case has forced an unusually public reckoning with how safety controls behave under sustained, emotionally loaded conversation. Reporting on internal data suggested that more than a million users per week show signs of mental health distress or mania in ChatGPT interactions, and OpenAI has reportedly acknowledged that safety controls can degrade over long exchanges. For the authors, these episodes illustrate the core structural problem: policies written to prevent harm are only as strong as the response mechanisms that enforce them in real time, with real adolescents, in real conversational contexts.</p>
<p>Each platform, the analysis shows, has approached the problem differently, and none has solved it. Google has expanded Gemini access to teen students through Workspace for Education while maintaining a generative AI prohibited use policy, and has more recently opened the assistant to younger children under parental supervision through Family Link. Microsoft has embedded Copilot into educational products for teen students and publishes extensive responsible AI principles, privacy documentation and guidance on designing empathetic conversational experiences. Meta has launched teen accounts, partnered with child-safety organizations on curricula against online exploitation, and joined industry coalitions on generative AI principles for child safety. Character.AI, whose product is explicitly built around persona-driven conversation, has introduced parental insight features and dedicated teen safety policies. OpenAI maintains a detailed model specification, usage policies and safety documentation. The paper&#8217;s point is not that these companies have done nothing; it is that a patchwork of policies, age gates and parental dashboards does not add up to a coherent, developmentally informed safety architecture.</p>
<p>Why does the disconnect persist? The authors point to a mismatch between who writes the rules and who the rules are supposed to protect. Ethical frameworks in the AI industry are typically engineered around content categories, self-harm, sexual content, violence, privacy, and enforced through classifiers and refusal patterns. Adolescents do not experience their problems as content categories. A fourteen-year-old asking a chatbot for help with a crisis may frame it indirectly, culturally, or through the vocabulary of school stress and family conflict. A system tuned to detect explicit keywords can miss the underlying need, or worse, respond with the confident, stigmatizing, or inappropriate output that research on language models in mental health contexts has repeatedly documented. Studies cited in the brief show large language models expressing stigma and giving inappropriate responses in clinical scenarios, which undermines any assumption that they can safely replace trained providers, and other work evaluates whether smaller models can deliver empathetic dialogue support for conditions such as post-traumatic stress disorder with mixed results.</p>
<p>The brief&#8217;s constructive proposal is what elevates it beyond critique. The authors recommend culturally responsive, trauma-informed design as a concrete standard for generative AI systems serving young people. Trauma-informed design, borrowed from social work and education practice, means systems assume that users may carry adverse experiences, avoid re-traumatizing interactions, emphasize safety and predictability, and give users control over the interaction. Culturally responsive design means the system&#8217;s responses reflect the linguistic, cultural and community context of the user rather than a single default demographic. The authors draw on frameworks including UNICEF&#8217;s policy guidance on AI for children, the National Association of Social Workers&#8217; code of ethics, and design justice scholarship that insists affected communities participate in building the systems that govern them. They also point to participatory design research in which teenagers act as advisors and auditors of the algorithms they use daily, situating youth expertise at the center of evaluation rather than at its margins.</p>
<p>Enforcement is the other half of the prescription. The authors endorse the emerging argument that AI audit standards boards are necessary, meaning independent bodies with the authority to verify that a platform&#8217;s stated youth-safety policies match its observed prompt-response behavior. This mirrors the logic of financial auditing or clinical accreditation: self-reported compliance is not evidence of compliance. The brief&#8217;s five-platform comparison functions, in effect, as a template for what such audits should examine, sensitive prompt handling, educational support quality, social context awareness, cultural responsiveness, and age-appropriate interaction, measured not in marketing documents but in live system behavior. The implementation of culturally responsive trauma-informed design recommendations, the authors argue, could substantially improve AI design toward more ethical approaches and reduce potential harm to youth users.</p>
<p>What makes this paper likely to travel well beyond academic circles is its timing. Regulators in multiple jurisdictions are actively drafting rules for AI systems used by minors, schools are adopting chatbots faster than they can evaluate them, and litigation is beginning to test whether conversational AI products owe a duty of care to their youngest users. The Pennsylvania team offers those debates a precise, testable framing: the question is not whether AI companies have youth safety policies, but whether those policies survive contact with an actual teenager at 2 a.m. Their answer, drawn from five platforms and dozens of policy documents, is that today it too often does not, and that closing the gap requires standards, audits and design practices that treat adolescent development, culture and trauma as engineering requirements rather than afterthoughts.</p>
<p><strong>Subject of Research:</strong> Alignment of generative AI platform safety policies with youth mental health and well-being needs</p>
<p><strong>Article Title:</strong> Improving safety frameworks for generative AI and youth well-being policy</p>
<p><strong>Article References:</strong> Mathiyazhagan, S., Nazzaro, L. M., &amp; Patton, D. U. (2026). Improving safety frameworks for generative AI and youth well-being policy. <em>Nature Mental Health</em>. <a href="https://doi.org/10.1038/s44220-026-00735-0" rel="noopener noreferrer">https://doi.org/10.1038/s44220-026-00735-0</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1038/s44220-026-00735-0" rel="noopener noreferrer">10.1038/s44220-026-00735-0</a></p>
<p><strong>Keywords:</strong> generative AI, youth mental health, AI safety policy, ChatGPT, Google Gemini, Microsoft Copilot, Meta AI, Character.AI, trauma-informed design, culturally responsive design, AI auditing, adolescent well-being</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">313424</post-id>	</item>
		<item>
		<title>AI Chatbots Struggle to Spot Validated Home Blood Pressure Monitors, Study Finds</title>
		<link>https://scienmag.com/ai-chatbots-struggle-to-spot-validated-home-blood-pressure-monitors-study-finds/</link>
		
		<dc:creator><![CDATA[Denise Maddox]]></dc:creator>
		<pubDate>Thu, 08 Oct 2026 15:50:56 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[accuracy rates of ChatGPT and other AI search engines]]></category>
		<category><![CDATA[AI chatbot accuracy in medical device validation]]></category>
		<category><![CDATA[AI in healthcare decision-making]]></category>
		<category><![CDATA[American Heart Association]]></category>
		<category><![CDATA[Artificial Intelligence]]></category>
		<category><![CDATA[blood pressure monitors]]></category>
		<category><![CDATA[challenges in AI medical device verification]]></category>
		<category><![CDATA[ChatGPT]]></category>
		<category><![CDATA[Clinical validation]]></category>
		<category><![CDATA[clinical validation of home blood pressure monitors]]></category>
		<category><![CDATA[Google Gemini]]></category>
		<category><![CDATA[health misinformation]]></category>
		<category><![CDATA[home monitoring]]></category>
		<category><![CDATA[hypertension]]></category>
		<category><![CDATA[impact of AI errors on cardiovascular health]]></category>
		<category><![CDATA[importance of validated medical devices for blood pressure measurement]]></category>
		<category><![CDATA[limitations of AI-powered health searches]]></category>
		<category><![CDATA[Microsoft Copilot]]></category>
		<category><![CDATA[Perplexity AI]]></category>
		<category><![CDATA[potential clinical consequences of inaccurate AI health advice]]></category>
		<category><![CDATA[reliability of AI health information]]></category>
		<category><![CDATA[risks of misinformation in health AI tools]]></category>
		<category><![CDATA[role of AI in patient health management]]></category>
		<category><![CDATA[validatebp.org]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=248509</guid>

					<description><![CDATA[Preliminary research presented at the American Heart Association's Hypertension Scientific Sessions 2026 found that leading AI chatbots correctly identified validated home blood pressure monitors only 63 to 91 percent of the time, prompting experts to urge the public to rely on independent validation registries instead.]]></description>
										<content:encoded><![CDATA[<p>When people want a quick answer about their health, many now turn to artificial intelligence. Ask a chatbot whether a home blood pressure monitor has passed clinical validation testing, and it will usually respond with confidence. According to preliminary research presented at the American Heart Association&#8217;s Hypertension Scientific Sessions 2026 in Arlington, Virginia, that confidence is frequently misplaced. Three of the four most popular AI-powered search tools, OpenAI&#8217;s ChatGPT, Microsoft&#8217;s Copilot and Perplexity AI, correctly identified whether a home blood pressure monitor met validated clinical standards only 63 to 83 percent of the time. Google Gemini performed best, yet it still delivered incorrect responses roughly 10 to 15 percent of the time. For a question that determines whether a device can be trusted to measure blood pressure accurately, those error rates carry real clinical consequences.</p>
<p>The stakes are far from trivial. High blood pressure is the leading risk factor for cardiovascular disease, affecting more than 125 million adults in the United States, roughly 47 percent of the adult population, according to the American Heart Association&#8217;s 2026 Heart Disease and Stroke Statistics Update. Only about one in four of those adults keeps their blood pressure within the target range of less than 120 mm Hg systolic and 80 mm Hg diastolic. Home monitoring is a cornerstone of modern hypertension management, allowing clinicians to track treatment response and patients to participate actively in their own care. But the entire enterprise depends on the device itself being accurate, which is why validation matters so much.</p>
<p>A home blood pressure monitor is considered validated when independent, third-party testing has demonstrated that it produces consistently accurate readings. Three primary registries catalog these devices: StrideBP, ValidateBP and Hypertension Canada. The 2025 American Heart Association Guideline for the Prevention, Detection, Evaluation, and Management of High Blood Pressure in Adults explicitly recommends that people use a home monitoring device that has been validated for accuracy, and directs patients to consult their clinician and visit validatebp.org for guidance. Because these registries are free, public and independently maintained, the information AI tools would need to answer correctly is, in principle, easily accessible.</p>
<p>That accessibility is what makes the new findings so puzzling. Researchers tested 324 home blood pressure monitors in Canada during April and May 2026, including 145 validated devices and 179 devices that were not validated. The device list was drawn from the three validation registries, a list of known non-validated devices, and the top-selling blood pressure cuff monitors sold through Amazon in Canada, Australia and the United States. Each device was queried against four AI search tools: Google Gemini, Microsoft Copilot, ChatGPT and Perplexity. The design deliberately included lesser-known monitors as well as popular ones, though the researchers noted that even obscure devices appear on the same official registries as their better-known counterparts.</p>
<p>The questioning protocol was equally systematic. For each of the 324 devices, researchers asked each AI tool three types of questions: a general, public-style question about the device&#8217;s validation status; a more specific question directing the tool to check a named validation registry; and the same question posed with additional detail and requiring a one-word answer. This layered approach was intended to reveal whether phrasing, specificity or explicit pointers to authoritative sources changed the quality of the responses. To assess consistency, the team retested 20 percent of the devices that had produced mixed results, using three different examiners on different computers and on different days, primarily in May 2026.</p>
<p>The results revealed both troubling inaccuracy and troubling inconsistency. Accuracy varied significantly depending on which tool was used. Google Gemini scored highest, answering correctly 86 to 91 percent of the time depending on how questions were phrased, while correct responses from ChatGPT, Copilot and Perplexity ranged from only about 63 to 83 percent. Notably, all four tools were less accurate at identifying validated monitors than unvalidated ones, and all four misidentified various devices as failing to meet clinical standards even while verifying against the very registries that list only validated monitors. When researchers retested devices with mixed results, asking the same tool the same questions on a different day or from a different computer, the tools often produced a different answer, undermining any assumption that a single query yields a dependable verdict.</p>
<p>Anna Soriano, M.D., a third-year internal medicine resident at the University of Montreal and the study&#8217;s presenting author, emphasized how close the weaker tools came to chance performance. We found that most AI tools performed only slightly better than if you had flipped a coin for each question, she said. Even Google Gemini, which performed best, was often wrong and couldn&#8217;t find information that is easily located. Soriano added that people may unknowingly believe a device is validated based on an AI tool&#8217;s inaccurate response, and that using such a device may produce inaccurate blood pressure readings, which could lead to an inappropriate diagnosis or treatment decisions. The authors could not determine exactly why the AI tools struggled with devices that had already passed validation testing.</p>
<p>One detail Soriano described as almost counterintuitive stands out: in many cases, the AI tools actually located the monitor&#8217;s listing on an official registry website but failed to interpret that listing as proof of validation. It is surprising that AI tools had so much difficulty specifically identifying validated devices, since those are the ones with clear listings on official registries, she said. This suggests the problem lies not in retrieval but in reasoning, the step where a language model must connect a factual observation to the correct conclusion. It is a distinction with broad implications, because much of the promise of AI in health care depends on exactly that kind of inference, whether the task involves cardiac imaging, electrocardiography or mobile monitoring devices.</p>
<p>Keith C. Ferdinand, M.D., FAHA, an American Heart Association volunteer expert and vice chair of the Association&#8217;s 2025 High Blood Pressure Guideline, who was not involved in the study, placed the findings in a wider clinical context. Artificial intelligence holds great promise to help support clinicians in areas such as cardiac imaging, electrocardiography, mobile devices and other tools, he said, but the potential shortcomings demonstrated by this study should remind clinicians and the public that using AI for clinical decision-making requires caution. Ferdinand, the Gerald S. Berenson Endowed Chair in Preventative Cardiology and professor of medicine at Tulane University School of Medicine in New Orleans, stressed that home blood pressure devices need to be both validated and accurate, and that with proper technique and regular monitoring, readings from home devices remain a valuable component of integrated, individualized treatment plans that can improve patient care and outcomes.</p>
<p>The researchers acknowledge important limitations. The results are likely to change as AI technology continues to improve, and because the tools were tested over a finite window, the findings represent a snapshot of rapidly evolving systems. To limit each tool&#8217;s ability to learn from previous test searches, the scientists used private internet browsing sessions, though this technique could not fully prevent language model training on the queries. The study is also a research abstract, presented at the Hypertension Scientific Sessions and scheduled for publication in the Hypertension Scientific Sessions 2026 Supplement in November 2026; abstracts are not peer-reviewed and the findings are considered preliminary until published as a full manuscript. Even so, the practical advice from the authors is unambiguous: rather than trusting a chatbot&#8217;s answer, healthcare professionals and the public should confirm a monitor&#8217;s validation status directly through the recognized independent, free and publicly available registries, starting with validatebp.org.</p>
<p><strong>Subject of Research:</strong> Accuracy of AI chatbots in identifying clinically validated home blood pressure monitors</p>
<p><strong>Article Title:</strong> Top AI tools accurately identified validated home BP monitors only about 2/3 of the time</p>
<p><strong>Article References:</strong> Top AI tools accurately identified validated home BP monitors only about 2/3 of the time. (n.d.). <a href="https://www.eurekalert.org/news-releases/1146589" rel="noopener noreferrer">Original publication</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> Not provided</p>
<p><strong>Keywords:</strong> artificial intelligence, ChatGPT, Google Gemini, Microsoft Copilot, Perplexity AI, blood pressure monitors, hypertension, clinical validation, home monitoring, American Heart Association, health misinformation, validatebp.org</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">248509</post-id>	</item>
		<item>
		<title>AI Sits a Dermatology Board Exam: Five Chatbots Put to the Test</title>
		<link>https://scienmag.com/ai-sits-a-dermatology-board-exam-five-chatbots-put-to-the-test/</link>
		
		<dc:creator><![CDATA[Dean Parker]]></dc:creator>
		<pubDate>Tue, 06 Oct 2026 04:19:29 +0000</pubDate>
				<category><![CDATA[Medicine]]></category>
		<category><![CDATA[AI dermatology board exam]]></category>
		<category><![CDATA[AI diagnostic accuracy in skin conditions]]></category>
		<category><![CDATA[AI performance on specialty-specific medical questions]]></category>
		<category><![CDATA[AI tools for medical education and practice]]></category>
		<category><![CDATA[Artificial Intelligence]]></category>
		<category><![CDATA[benchmarking]]></category>
		<category><![CDATA[board examination]]></category>
		<category><![CDATA[chatbot performance in medical licensing]]></category>
		<category><![CDATA[ChatGPT]]></category>
		<category><![CDATA[clinical reasoning]]></category>
		<category><![CDATA[comparison of ChatGPT and other AI models]]></category>
		<category><![CDATA[consumer-facing AI in healthcare]]></category>
		<category><![CDATA[dermatology]]></category>
		<category><![CDATA[evaluation of GPT-5.2 in dermatology]]></category>
		<category><![CDATA[Gemini]]></category>
		<category><![CDATA[Grok]]></category>
		<category><![CDATA[impact of AI on dermatology certification exams]]></category>
		<category><![CDATA[large language models]]></category>
		<category><![CDATA[large language models in dermatology]]></category>
		<category><![CDATA[Medical Education]]></category>
		<category><![CDATA[Microsoft Copilot]]></category>
		<category><![CDATA[perplexity]]></category>
		<category><![CDATA[role of AI chatbots in dermatology diagnostics]]></category>
		<category><![CDATA[Rutgers study on AI in medical licensing]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=240194</guid>

					<description><![CDATA[Researchers at Rutgers tested five leading AI chatbots, including ChatGPT, Grok, Gemini, Perplexity and Copilot, on American Board of Dermatology-style questions to gauge how frontier language models handle specialist-level clinical reasoning.]]></description>
										<content:encoded><![CDATA[<p>When a patient presents with an unusual rash, the dermatologist across the table is drawing on years of training, thousands of clinical images, and the hard-won experience of board certification. Now a team at Rutgers Robert Wood Johnson Medical School has asked a provocative question: how would the newest generation of artificial intelligence chatbots fare against the same examination standards? In a research letter published in the Archives of Dermatological Research, Rose Rasty, Elyse Mackenzie, Shaunt Mehdikhani and Babar Rao report their evaluation of five prominent large language models on board-style dermatology questions drawn from the American Board of Dermatology&#8217;s testing tradition, offering one of the most direct head-to-head comparisons yet attempted in the specialty.</p>
<p>The five systems examined represent the current cutting edge of consumer-facing artificial intelligence. The study tested ChatGPT in its GPT-5.2 configuration, xAI&#8217;s Grok 4.1, Google&#8217;s Gemini 3, Perplexity running Sonar and GPT-5-series backends, and Microsoft Copilot built on GPT-5. This lineup matters because these are not obscure research prototypes buried in a laboratory; they are the tools that medical students, residents and practicing physicians already consult daily for quick answers. Understanding how they perform on rigorous, specialty-specific questions is therefore not an academic curiosity but a matter of practical patient safety, since patients too are increasingly turning to these chatbots with their own dermatological concerns.</p>
<p>Board-style questions are a particularly demanding benchmark for any question-answering system. The American Board of Dermatology&#8217;s examinations are designed to distinguish competent specialists from merely knowledgeable ones, and the questions typically present nuanced clinical vignettes in which several answer choices are plausible. A candidate must weigh the patient&#8217;s age, lesion morphology, distribution, history of prior treatments, and sometimes subtle histopathological findings before selecting the single best response. This format tests something closer to clinical reasoning than to factual recall, which is precisely why it has become a popular yardstick for evaluating medical artificial intelligence. A model can memorize that psoriasis affects roughly two to three percent of the population, but it must reason through a vignette to recognize a psoriasiform drug eruption masquerading as the idiopathic disease.</p>
<p>The technical architecture underlying these models helps explain both their promise and their pitfalls. Large language models are neural networks, generally built on the transformer architecture, trained on vast text corpora to predict the next token in a sequence. Through this deceptively simple objective, they absorb grammar, factual knowledge, and patterns of reasoning encoded in their training data. Medical knowledge enters the picture because textbooks, review articles, clinical guidelines and discussion forums form a substantial portion of the public internet. However, the models do not store facts as a database would; knowledge is distributed across billions of numerical parameters, which means retrieval is probabilistic rather than deterministic. This is why a model can answer an obscure question flawlessly one moment and hallucinate a nonexistent drug interaction the next, a behavior that examiners would immediately fail in a human candidate.</p>
<p>Dermatology poses distinctive challenges for text-based models, even in a question-answer format that strips away the visual element. The specialty sits at the intersection of internal medicine, immunology, infectious disease, oncology and pathology, and its vocabulary is notoriously dense. Distinguishing lichen planus from lichenoid drug eruption, or dermatitis herpetiformis from linear IgA bullous dermatosis, depends on pattern recognition honed through exposure to thousands of cases. Board-style questions compress that pattern recognition into prose, which arguably favors language models, yet the compression also removes the contextual cues a clinician would use at the bedside. The Rutgers team&#8217;s choice of this format follows a growing research tradition: earlier studies had already tested ChatGPT on board-style dermatology questions, including image-based items, and Liu and colleagues in 2025 assessed dermatological knowledge and image analysis using specialty certificate examinations as their benchmark.</p>
<p>What makes the new study notable is its breadth of comparison. Most prior evaluations examined a single model, usually a ChatGPT variant, against a question bank. By running the same board-style items through five different systems, each with distinct underlying architectures and training pipelines, the researchers could probe whether strong medical performance is a general property of frontier language models or something specific to particular products. The differences among these systems are not trivial. GPT-5.2, Grok 4.1, Gemini 3 and the GPT-5-based Copilot each reflect different training data mixtures, different reinforcement learning strategies, and different approaches to reasoning. Perplexity adds a further wrinkle: it couples a language model with live web search, meaning its answers may draw on retrieved documents rather than purely on parametric memory, a hybrid architecture that could behave very differently on questions with recently updated guidelines.</p>
<p>The distinction between parametric knowledge and retrieval-augmented answering is one of the most technically interesting aspects of this kind of evaluation. A pure language model answers from what is baked into its weights during training, frozen at a cutoff date. A retrieval-augmented system like Perplexity queries the live web, which brings freshness but also vulnerability: it can absorb misinformation, outdated guidelines or commercially biased content from the open internet. For dermatology, where treatment recommendations evolve and where the internet is saturated with anecdotal skin-care advice, that distinction could materially change performance. A benchmark like the one used in this study therefore measures not just a model&#8217;s medical knowledge but its entire answer-generation pipeline, including how it filters and prioritizes whatever sources it consults.</p>
<p>The authors of the research letter are appropriately measured about what such testing can and cannot establish. Answering multiple-choice questions correctly is not the same as managing a patient. A board examination candidate who selects the right option for a bullous disorder vignette may still struggle to perform a safe skin biopsy or to counsel a frightened patient about a new diagnosis of melanoma. Conversely, models can exploit statistical cues in question wording, patterns that human test-takers learn to recognize as well, without genuinely understanding the underlying medicine. There is also the ever-present risk of data contamination: if board-style questions or close paraphrases circulate online, a model trained on the internet may have effectively seen the answer key. Rigorous benchmarking in medical artificial intelligence must therefore always ask not only how well a model scored, but whether the score reflects reasoning or recall.</p>
<p>Nevertheless, the trajectory across successive studies is striking. When ChatGPT first burst into public awareness in late 2022, early medical evaluations produced mixed results, with the model often falling short of passing thresholds on professional examinations. Each subsequent generation has closed the gap, and by the mid-2020s frontier models were routinely performing at or near the level of human test-takers across a range of specialties. The Rutgers study extends that line of evidence into dermatology with the newest available models, and its very framing, published as a research letter rather than a lengthy original article, reflects how quickly the field is moving: the findings needed to reach the literature before the models they evaluated were superseded. The authors report that all data were generated using the five named systems, with the dataset available from the corresponding author on reasonable request, and the work received no external funding while the authors declared no conflicts of interest.</p>
<p>For clinicians and patients alike, the practical message is one of cautious engagement rather than either alarm or celebration. These systems are already embedded in clinical workflows, from drafting patient messages to summarizing literature, and their demonstrated competence on board-style questions suggests they can serve as powerful adjuncts for education and decision support. A resident reviewing for boards might legitimately use a chatbot to generate practice vignettes or explain the immunopathology of pemphigus vulgaris. But the same probabilistic architecture that enables fluent expertise also produces confident errors, and no benchmark, however demanding, substitutes for clinical supervision. The Rutgers team&#8217;s comparison of five frontier models gives the field a clearer map of where artificial intelligence stands in dermatology today, and a reminder that the map will need redrawing with every new model release.</p>
<p><strong>Subject of Research:</strong> Evaluation of large language models on American Board of Dermatology-style examination questions</p>
<p><strong>Article Title:</strong> Performance of five large language models on board-style dermatology questions from the American Board of Dermatology</p>
<p><strong>Article References:</strong> Rasty, R., Mackenzie, E., Mehdikhani, S., &amp; Rao, B. (2026). Performance of five large language models on board-style dermatology questions from the American Board of Dermatology. <em>Archives of Dermatological Research, 318</em>(1), Article 462. <a href="https://doi.org/10.1007/s00403-026-04894-z" rel="noopener noreferrer">https://doi.org/10.1007/s00403-026-04894-z</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s00403-026-04894-z" rel="noopener noreferrer">10.1007/s00403-026-04894-z</a></p>
<p><strong>Keywords:</strong> large language models, dermatology, artificial intelligence, board examination, ChatGPT, Gemini, Grok, Microsoft Copilot, Perplexity, medical education, clinical reasoning, benchmarking</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">240194</post-id>	</item>
	</channel>
</rss>
