AI Risk Alerts Modestly Boost Flu Vaccination in Trials of 90,000 Patients

Telling patients that a machine learning algorithm has flagged them as being at high risk for influenza complications nudges a meaningful share of them to get their flu shot, according to one of the largest real-world tests of algorithm-assisted health communication conducted to date. In three preregistered randomized controlled trials involving more than 90,000 patients of a large US healthcare system, personalized risk messages increased vaccination above and beyond simple reminders, and the disclosure that an algorithm had been involved in determining that risk made no measurable difference to how patients responded. The findings, published in Nature Human Behaviour, offer a rare field-based answer to a question that has largely been debated in laboratory settings: do people actually object to medical AI when it touches their own care?

The research team, led by Gail M. Rosenbaum, Amir Goren, Maheen Shermohammed, Michelle N. Meyer and Christopher F. Chabris of Geisinger’s Behavioral Insights Team, together with collaborators at Geisinger’s Department of Laboratory Medicine, MIT and the National Bureau of Economic Research, set out to test two intertwined ideas. The first was whether communicating personalized risk could serve as an effective behavior-change intervention for vaccination. The second was whether patients react differently when the risk assessment is attributed to an algorithm, or when the algorithm’s prediction is accompanied by personalized reasons, a form of what researchers call explainable AI. Both questions matter because healthcare systems are increasingly deploying machine learning models to identify high-risk patients, yet most evidence about patient attitudes comes from surveys and hypothetical vignettes rather than actual clinical behavior.

The intervention rested on a previously validated machine learning model, developed with the company Medial EarlySign, that predicts which patients face elevated risk of influenza and its complications, including cardiovascular events and worsening of chronic conditions such as diabetes. Influenza is not a trivial illness for vulnerable groups: it can trigger heart attacks, hospitalizations and death, and vaccination remains the most effective preventive measure. The model mined electronic health record data to assign each patient a risk score, and for some messages the system also generated the specific medical reasons, drawn from the patient’s own record, that contributed to the high-risk classification.

Across the three trials, registered on ClinicalTrials.gov under identifiers NCT04323137, NCT05009251 and NCT05509283, patients identified by the algorithm as high risk were randomly assigned either to receive no message at all, to receive a generic reminder encouraging flu vaccination, or to receive one of several messages informing them that they were at high risk for flu and its complications. Some of those high-risk messages mentioned that an algorithm had been used to determine the risk; others cited the personalized medical reasons behind the prediction; and still others did both or neither. The primary outcome in every trial was the same and unambiguous: whether the patient actually received a flu vaccine, verified through the health system’s records.

The results showed a consistent, if modest, behavioral effect. Among patients told they were at high risk, vaccination rates were 1.1 to 1.4 percentage points higher, a relative increase of roughly 3.3 to 5.4 percent, compared with patients who received only a simple reminder to get vaccinated. Compared with patients who received no message whatsoever, the high-risk messages raised vaccination by 1.7 to 3.5 percentage points, relative gains ranging from 3.3 to 14.7 percent depending on the trial. In a population where baseline vaccination rates varied considerably across the studies, from about 51 percent down to under 24 percent in the passive control groups, these effects are small in absolute terms but meaningful at the scale of an entire health system, where even a single percentage point represents thousands of additional vaccinated patients.

The more striking finding concerned transparency. Vaccination rates were statistically indistinguishable across message arms that disclosed algorithmic involvement and those that did not, and adding personalized explanations for the algorithm’s prediction did not change outcomes either. In other words, patients were neither averse to nor appreciative of the fact that a machine had helped determine their risk. This null result carries considerable weight because laboratory studies of so-called algorithm aversion and algorithm appreciation have produced conflicting conclusions, with some experiments showing people distrust algorithmic judgment after seeing it err, others showing people prefer it to human judgment, and research on medical AI specifically suggesting patients may resist it, particularly for subjective or identity-relevant judgments.

Why the discrepancy between the lab and the field? The authors’ design offers some clues. In a realistic healthcare context, the message patients received was embedded in an ongoing relationship with a trusted health system, and the actionable recommendation, getting a flu shot, was low-cost, familiar and clearly beneficial. Attitudes measured in hypothetical scenarios may simply fail to predict behavior when the stakes are concrete and the decision is easy. The team also examined whether the number of personalized risk reasons included in a message mattered, and whether patients’ actual underlying risk level, not just the risk level communicated to them, moderated the effect, analyses reported in extensive supplementary materials covering dozens of tables and figures.

The trials were methodologically rigorous by the standards of the nudge literature. All three were preregistered, with power analyses conducted in advance; the studies were designed to detect effects of around two percentage points, consistent with effect sizes typical of large-scale vaccination nudges. The research team posted de-identified data and analysis code publicly on the Open Science Framework, and the study protocols and statistical analysis plans are available through the trial registrations. A fourth study was also conducted and is described in the supplementary materials. The work was supported in part by the National Institute on Aging of the National Institutes of Health under award number P30AG034532, though the funders had no role in study design, analysis or the decision to publish.

The findings arrive amid a broader wave of large-scale field experiments on vaccination messaging, including megastudies of text-message nudges in pharmacies and health systems, electronic nudges delivered through national registries in Denmark and Finland, and trials of financial incentives for COVID-19 vaccination. Those studies have shown that simple, well-timed reminders can reliably move vaccination rates, but also that effects vary across contexts and populations, and that some nudges fail when scaled. The new results add a distinctive twist: the content that moves people is not the technology behind the message but the personal information it conveys. What mattered was telling patients something true and relevant about their own bodies, not whether that information came from a human clinician or a statistical model.

For health systems weighing how to deploy predictive algorithms, the practical implications are twofold. First, algorithmically identified high-risk patients represent a tractable target for outreach, and informing them of their elevated risk produces a small but reliable increase in vaccination beyond standard reminders. Second, at least for this kind of low-stakes, clearly beneficial intervention, fears that patients will recoil from algorithmic involvement appear overstated, and investing in elaborate explainability features may not be necessary to preserve patient trust. The authors caution that the results come from one US health system and one preventive behavior, so generalization to more consequential or contested medical decisions remains an open question. But as a demonstration that personalized, algorithm-assisted risk communication can change real health behavior at scale, without triggering the aversion that laboratory studies predicted, the study marks an important step toward evidence-based deployment of medical AI in everyday care.

Subject of Research: Algorithm-assisted personalized risk communication to increase influenza vaccination

Article Title: Algorithm-assisted personalized risk communication to encourage flu vaccination in the USA: three randomized field trials

Article References: Rosenbaum, G. M., Goren, A., Shermohammed, M., Wolk, D. M., Tice, A. M., Doyle, J. J., Jr., Meyer, M. N., & Chabris, C. F. (2026). Algorithm-assisted personalized risk communication to encourage flu vaccination in the USA: three randomized field trials. Nature Human Behaviour. https://doi.org/10.1038/s41562-026-02603-4

Image Credits: AI Generated

DOI: 10.1038/s41562-026-02603-4

Keywords: artificial intelligence, influenza vaccination, randomized controlled trials, risk communication, algorithm aversion, explainable AI, health behavior, nudges, machine learning, preventive medicine, vaccine hesitancy, Nature Human Behaviour