Conceptual framework links generative-AI health safeguards to five decision functions

A conceptual analysis proposes a multilevel governance framework for reducing the risk that conversational generative artificial intelligence will lead people to unsafe health decisions. The framework, called generative-AI health information resilience, or GenAI-HIR, combines five possible protective functions for users and health workers with controls operated by health organizations, platforms, regulators and public-health agencies. Its central premise is that fluent, personalized and apparently authoritative AI answers require safeguards proportionate to the consequences of acting on them. The article reports a theory-based synthesis rather than a clinical trial or validated measurement system, and the authors explicitly say that the model must be tested before its competency configuration can support scoring, certification or causal conclusions.

The immediate concern is a shift from retrieving ranked documents to conversing with systems that synthesize, translate and apparently tailor information. Such responses may shape how people interpret symptoms, consider prevention, choose medications, relate to institutions and decide when to seek care. Their apparent fluency can also make errors persuasive: previous literature identified unsupported claims, invented references, omitted warnings, biased performance, inadequate disclosure and outdated guidance as relevant risks. The proposed “authority substitution” mechanism suggests that people may mistake conversational quality, rapid responses, empathy cues and confident personalization for evidence of professional competence. Earlier findings on anthropomorphism, health-information quality and automation bias support this conceptual link, but the authors note that much of the automation research predates large language models and does not directly establish their effects.

GenAI-HIR divides potential protection into five conceptually distinct functions: recognize, interrogate, verify, contextualize and escalate. Recognition means noticing that an answer was generated by AI, identifying its stakes and detecting confident wording that is unsupported by evidence. Interrogation involves probing assumptions, uncertainty, source competence and missing context. Verification requires consequential claims to be checked against current, independent and decision-relevant evidence rather than the model’s own self-critique. Contextualization asks whether general guidance fits the person’s clinical situation, language, culture, finances, disability, location and available services. Escalation means reaching qualified human care when uncertainty, urgency or potential harm exceeds safe self-management. These functions are presented as possible points of protection, not proven psychological dimensions; future task data may instead support a collapsed model, a formative configuration or a non-compensatory profile.

The framework places these functions within four distinct analytical levels. Individual users and health workers exercise proximal judgment and professional protection. Organizational capacity supplies approved sources, use boundaries, accessible human routing, incident monitoring, corrective action and subgroup surveillance. Platform and regulatory conditions provide provenance, auditability, uncertainty design, reporting duties and enforceable boundaries. Population outcomes include harm, delayed care, calibrated trust, access, equity and repeated incidents. This separation challenges the idea that a disclaimer can transfer responsibility to the least-resourced user. Strong personal competencies cannot compensate fully for missing verification infrastructure, while limited competencies may receive partial protection from strong defaults, source controls and clear routes to human assistance. The authors identify the combination of weak competencies and weak infrastructure as the highest-risk configuration.

Risk is classified by first identifying the type of output. Informational answers are generally low severity, reversible and minimally context-dependent, so basic disclosure and an authoritative source may suffice. Navigational answers depend on location, timing and service availability, requiring current jurisdiction-specific routes and accessible alternatives. Interpretive answers apply information to a person and may require missing-context prompts, explicit uncertainty, independent verification and qualified review. Directive answers recommend actions whose severity, urgency or irreversibility may be high, making validated scope, restricted current sources, safety triggers and immediate human or emergency routing appropriate proposed controls. Any output involving severe or time-critical warning signs normally warrants urgent controls unless a tested mitigation reduces the residual risk. The classification, combination rule and thresholds are the authors’ synthesis and remain unvalidated rather than established regulatory standards.

Organizational capacity is more specific than general responsible-AI governance. Proposed audit evidence includes accountable ownership, approved uses, prohibited-use lists, curated sources with dates and jurisdictions, audit logs, tested and accessible referral pathways, subgroup monitoring, incident taxonomies, root-cause review, documented corrective actions and recurrence surveillance. Each domain must be assessed separately: an organization may have standardized source control without a functioning human-escalation route, so the domains should not automatically be added into a maturity score. Governance should begin before deployment by defining supported use cases, affected populations, prohibited uses, privacy controls, approved sources and validated escalation. Organizations can then progress through limited use, piloting, evaluation, scaling and adaptive management, with decisions informed by dangerous omissions, subgroup performance, escalation completion and correction time.

Equity functions as both a design constraint and an outcome rather than an individual skill. The synthesis highlights unequal access to digital services, language differences, disability barriers, affordability assumptions and limited service availability. It cites an Arabic–English comparison in which Arabic answers were less accurate for one in three matched cirrhosis questions, while also emphasizing that evidence about disabled users of consumer AI health tools remains scarce. Average accuracy can therefore conceal unequal protection. Before deployment, systems should be tested across languages and dialects, disability access, cultural expression and affordability assumptions, and subgroups should be monitored from exposure through action and outcome. Accessibility and routing supports should be individualized: neither automatic exclusion nor automatic access to the most resource-intensive control is sufficient.

The framework also proposes falsifiable research rather than treating the conceptual synthesis as proof. One proposition asks whether the five-function configuration predicts better confidence calibration and safer actions than established health literacy, digital health literacy, AI literacy and misinformation-resistance measures. Others test whether provenance and uncertainty cues strengthen verification, whether detecting red flags mediates timely care selection, whether inaccessible services weaken protection, whether completed corrective actions reduce recurrent incidents, and whether confident anthropomorphic cues increase reliance on inaccurate answers. These relationships are explicitly tentative arrows rather than demonstrated causal effects. Planned validation would combine cognitive interviews with scored behavioral vignettes, compare perceived capability with observed choices, and examine reliability, measurement invariance and incremental prediction before any domain or total score could be used.

A worked bilingual English–Arabic symptom-information chatbot illustrates the model’s operational logic. The hypothetical deployment permits general education but prohibits diagnosis, autonomous triage and medication changes. Chest pain triggers a mechanism that does not analyze symptoms or display uncertain reassurance, but provides an urgent, location-specific route in the selected language and accessible format. A failure in translation, routing or completion enters an incident log, prompts correction, and is checked for recurrence by language and disability status. For resource-limited organizations, the authors suggest a minimum viable control set: narrow the supported use case, use a small verified source collection, establish one tested human-care route, monitor a limited set of incidents and use shared regional verification services where available. The framework is designed for conversational health information, not autonomous diagnostic devices or multimodal clinical systems, and its value now rests on governance utility plus a testable research agenda rather than demonstrated reductions in harm.

Subject of Research: Conceptual multilevel governance for safe use of conversational generative AI in health information

Article Title: Managing generative-AI health information resilience: human resource capabilities and multilevel public health governance

Article References: Awad, A., El-Naggar, M. A., Al Nashily, D. H., Abdulrahim, H., Bansal, R., & Ghonim, A. (2026). Managing generative-AI health information resilience: human resource capabilities and multilevel public health governance. Frontiers in Public Health, 14, Article 1941787. https://doi.org/10.3389/fpubh.2026.1941787

Image Credits: AI Generated

DOI: 10.3389/fpubh.2026.1941787

Keywords: generative AI, public health, health information, AI governance, health literacy, patient safety, automation bias, health equity, misinformation, implementation science, Managing, generative-AI