A living scoping review published in Implementation Science reveals that the application of artificial intelligence across implementation science and practice remains in a nascent, methodologically narrow phase of development. Led by Guillaume Fontaine and an international team of researchers, the investigation represents the first cycle of the AI Methods for Implementation Science program, established to map how computational tools are currently deployed and evaluated to support evidence translation in health systems. Across forty included evidence sources, the investigators identified a marked imbalance in the scientific literature: while experimental tools and research prototypes have proliferated rapidly across data-rich administrative and analytical tasks, formal evaluations of their real-world reliability, clinical safety, sociotechnical disruption, workforce impact, and algorithmic fairness remain almost entirely absent. The authors emphasize that the current expansion of artificial intelligence in healthcare implementation is actively outpacing the field’s methodological capacity to characterize, validate, or govern the underlying systems.
To capture the emerging body of research, the team conducted a systematic search across six major bibliographic databases, comprising MEDLINE, Embase, CINAHL, PsycINFO, IEEE Xplore, and Web of Science, covering records indexed between January 1, 2010, and April 6, 2026. Applying methodological guidance from JBI and the Cochrane Collaboration, alongside reporting extensions from PRISMA-ScR and PRISMA-LSR, the systematic search identified 7,203 initial records. After deduplicating citations, two independent reviewers screened 6,604 titles and abstracts, ultimately examining 184 full-text publications. The review deliberately excluded adjacent literatures where artificial intelligence served solely as a direct clinical intervention, such as diagnostic classification or risk prediction, as a generic clinical scribe, or as automated systematic review screening software. The final synthesis retained 40 eligible sources focused directly on supporting implementation processes, 34 of which were published between 2021 and 2026, including 14 published in 2025 alone. Geographic distribution was heavily skewed toward North America, with 28 studies originating from the United States and three from Canada, alongside two from Australia, two from the United Kingdom, and single contributions from Switzerland, Ireland, Germany, Singapore, and China.
The methodological composition of the 40 included sources reflected an uneven, early-stage literature comprised of 22 primary research articles, eight conceptual, methodological, or narrative review papers, three prospectively published study protocols, three conference abstracts, two framework development articles, one comparative case study report, and one prototype development publication. Among the 22 primary research investigations, 12 utilized quantitative observational or experimental designs, eight employed mixed-methods approaches combining computational analytics with qualitative stakeholder assessments, and two relied on qualitative methodologies. Thirty-one of the 40 sources met the authors’ criteria for application-focused sources, defined as publications that directly described, developed, tested, or evaluated a specific computational tool or software prototype. The remaining nine sources contributed conceptual discussions, implementation frameworks, or theoretical perspectives outlining anticipated applications and systemic hazards without deploying or evaluating a concrete algorithmic system.
An analysis of primary computational architectures across the 31 application-focused sources demonstrated that conventional, non-deep machine learning models represented the most frequent analytical approach, appearing in 10 studies. Generative artificial intelligence powered by large language models appeared in seven studies, while multicomponent systems combining several analytical families or directly comparing divergent machine learning methods accounted for five studies. Non-generative deep learning architectures accounted for three studies, while non-learning natural language processing, symbolic or rules-based algorithms, and tools with unspecified computational mechanisms each accounted for two studies. In terms of operational maturity, 18 of the 31 specific systems were classified as early research prototypes undergoing development, four were evaluated within real-world pilot demonstrations, four achieved full deployment within clinical trials or routine healthcare operations, and five represented commercial off-the-shelf generative software packages utilized by investigators as secondary analytical aids for qualitative text analysis, manuscript preparation, or framework synthesis. More autonomous architectures, such as multi-agent collaborative systems, custom fine-tuned assistant platforms, multimodal vision-language models, and reinforcement learning pipelines, were entirely absent from the analyzed implementation literature.
Mapping included investigations onto the Knowledge-to-Action model demonstrated that computational tools cluster heavily within discrete, structured stages of the implementation life cycle, averaging 3.3 distinct process steps per source. The most frequent operational targets across all 40 publications were evaluating implementation impact, documented in 25 sources, followed by selecting and tailoring implementation strategies in 23 sources, assessing barriers and facilitators in 21 sources, and continuously monitoring intervention execution in 20 sources. In contrast, earlier-stage, relational, and value-laden activities across the implementation cycle were rarely augmented by algorithmic systems: sustainment and scale-up were examined in 13 sources, synthesizing implementation-relevant evidence in 11 sources, and problem prioritization and contextual adaptation were addressed in only nine sources each. The authors observed that artificial intelligence is currently applied almost exclusively to implementation challenges that can be operationalized as discrete classifications, numerical risk scores, thematic text extractions, or structured summaries, leaving deliberative consensus building and community negotiation unaddressed.
Concrete empirical applications highlighted throughout the scoping review illustrated how researchers have attempted to formalize complex implementation activities. In community pharmacy settings, investigators deployed random forest models to match facilitation strategies to identified contextual barriers. In educational healthcare initiatives, supervised text classification was used to separate operational intervention components from implementation strategies within school health action plans, supporting contextual tailoring. In psychiatric hospital environments, natural language processing and aspect-based sentiment analysis were deployed to process healthcare staff discourse surrounding the adoption of new information technologies, quantifying implementation climate from open-ended communication channels. In quality improvement infrastructure, investigators integrated topic modeling, aspect sentiment extraction, and named-entity recognition into automated pipelines that parsed unstructured patient feedback into actionable organizational targets. Similarly, broader multi-component platforms such as ImpleMATE and the Human Behaviour-Change Project combined supervised machine learning, natural language processing pipelines, and formal intervention ontologies to systematically code behavioral mechanisms, extract published implementation evidence, and inform longitudinal practice tailoring.
In monitoring and evaluation workflows, computational systems were deployed to audit clinical records and provider adherence. Several investigations analyzed electronic health record audit logs and provider communication transcripts to track workflow shifts following the launch of clinical decision support alerts, or to categorize organizational sites across developmental stages based on implementation broker correspondence. In a large pragmatic clinical trial, researchers deployed a fine-tuned BioClinicalBERT pipeline to screen narrative electronic health record passages for documented goals-of-care discussions, routing algorithmically flagged passages to human adjudicators to measure trial outcomes. Other teams leveraged large language models to conduct rapid qualitative analyses of implementation consultation transcripts and institutional initiatives. To forecast longitudinal program maintenance, researchers employed self-organizing neural networks to model organizational sustainability patterns and predict the risk of project abandonment. However, these monitoring systems primarily targeted proximal technical classifications rather than demonstrating improved long-term clinical or health service endpoints.
Theoretical grounding varied across the reviewed literature, with 18 of the 40 sources explicitly incorporating one or more implementation science theories, models, or frameworks across 21 distinct conceptual schemas. The Consolidated Framework for Implementation Research emerged as the most frequently cited model, appearing in five studies to structure contextual data extraction, guide reflexive thematic coding of exit interviews, and categorize implementation barriers prior to predictive modeling. The Expert Recommendations for Implementing Change taxonomy appeared in four studies to classify strategy selection and contextualize neural network sustainability predictions. The RE-AIM framework also appeared in four studies, serving to organize natural language processing survey metrics, frame machine learning integration across implementation phases, and structure equity assessments within capabilities models. In parallel, five sources incorporated specialized data science or computational ontologies alongside implementation theories, utilizing the Knowledge Discovery in Databases process, the Linguistic Inquiry and Word Count lexicon, the Behaviour Change Intervention Ontology, the Learning Implementation System ontology, and the Fairness Metrics Ontology to structure computational reasoning and knowledge representation.
The scoping review identified pervasive methodological limitations in how artificial intelligence systems are evaluated within the implementation domain. While 23 of the 31 application-focused sources reported or prospectively planned some form of system validation, only 19 reported quantifiable technical accuracy or model performance metrics. Implementation outcomes represented the most common evaluation domain, documented in 25 studies, but these were overwhelmingly restricted to proximal operational indicators such as classification accuracy, workflow shift, or fidelity scores. Human-centered outcomes, including usability, cognitive burden, and user trust, were evaluated or planned in 12 studies, efficiency or time metrics in 10 studies, and clinical or health system outcomes in nine studies. In contrast, resource and financial costs were assessed in only two studies, and equity-related outcomes were examined in only three. Crucially, across all 31 application-focused investigations, only a single study conducted external validation on an independent dataset, none performed a formal algorithmic fairness analysis, none evaluated model drift or temporal robustness, and no study formally evaluated the overall appropriateness of utilizing artificial intelligence as an independent evaluative construct.
Potential risks and unintended consequences were discussed in 22 of the 40 sources, but these were largely framed as speculative narrative limitations rather than prospectively measured empirical safety outcomes. AI-specific risks, such as hallucinated outputs, training bias, classification error, and opacity, were cited in 17 sources, workflow disruptions in 11, and user overreliance, automation bias, or deskilling in nine. Two empirical studies documented explicit negative effects: one demonstrated that commercial large language models produced inaccurate citations and lacked necessary nuance when summarizing panel discussions, requiring heavy human correction, while a trial of artificial intelligence-generated fidelity feedback in motivational interviewing observed a longitudinal decline in therapeutic quality, suggesting algorithmic feedback encouraged a transactional mindset that undermined relational care. The authors conclude that artificial intelligence cannot currently be recommended for autonomous implementation decisions, urging health systems to mandate rigorous local validation, continuous drift monitoring, algorithmic fairness audits, and multidisciplinary governance before scaling computational tools into routine operational environments.
Subject of Research: Scoping review of artificial intelligence applications and evaluations across implementation science and practice
Article Title: Artificial intelligence in implementation science and practice: a living scoping review
Article References: Fontaine, G., Lalla, O. D., Laritz, R., Durran, J., Zhang, C., Chan, J., Crump, L., Matra Putra, A. D., Abbasgholizadeh-Rahimi, S., An, R., Beidas, R. S., Fahim, C., Geng, E., Graham, I. D., Hastings, J., Lambert, S. D., Légaré, F., Michie, S., Powell, B. J., … Taylor, N. (2026). Artificial intelligence in implementation science and practice: a living scoping review. Implementation Science, 21(1), Article 66. https://doi.org/10.1186/s13012-026-01535-x
Image Credits: AI Generated
DOI: 10.1186/s13012-026-01535-x
Keywords: implementation science, artificial intelligence, living scoping review, machine learning, generative AI, Knowledge-to-Action model, CFIR framework, health services research, implementation monitoring, fidelity assessment, strategy tailoring, natural language processing, clinical decision support, BioClinicalBERT, algorithmic bias, AIM-IS program, electronic health records
Cite Scienmag News
Blake Davidson. (November 18, 2026). Implementation Science Scoping Review Finds Artificial Intelligence Applications Remain Methodologically Underdeveloped. Scienmag. https://scienmag.com/implementation-science-scoping-review-finds-artificial-intelligence-applications-remain-methodologically-underdeveloped/
Blake Davidson. "Implementation Science Scoping Review Finds Artificial Intelligence Applications Remain Methodologically Underdeveloped." Scienmag, 18 November 2026, https://scienmag.com/implementation-science-scoping-review-finds-artificial-intelligence-applications-remain-methodologically-underdeveloped/. Accessed 18 November 2026.
Blake Davidson. "Implementation Science Scoping Review Finds Artificial Intelligence Applications Remain Methodologically Underdeveloped." Scienmag. November 18, 2026. https://scienmag.com/implementation-science-scoping-review-finds-artificial-intelligence-applications-remain-methodologically-underdeveloped/








