AI Chatbots Match Google Search for Accurate Political Knowledge, Study Finds

As artificial intelligence chatbots weave themselves into the daily habits of millions, one question has loomed over democracies everywhere: what happens to voters when they start asking large language models, rather than search engines or journalists, about the political issues of the day? A new peer-reviewed study published in PNAS Nexus offers a surprisingly reassuring answer. In a large-scale experiment conducted around the 2024 United Kingdom general election, researchers found that people who used AI chatbots to research political claims gained accurate political knowledge just as effectively as those who used traditional Google search — and they got there noticeably faster.

The study, led by Lennart Luettgau, Hannah Kirk, Kobi Hackenburg, and colleagues, set out to test one of the most persistent fears about conversational AI: that hallucinations, biases, and confident-sounding errors could quietly distort what citizens believe to be true. Concerns about AI’s influence on democratic processes have intensified as large language models have become a common first stop for information seekers. Yet the empirical evidence about what actually happens when ordinary people use these tools to research politics has remained thin. This experiment was designed to fill that gap with controlled, measurable comparisons.

The researchers recruited UK residents online in two waves, drawing 1,711 participants in the first and 1,147 in the second. Each participant was asked to research true or false claims on two of four politically charged topics: climate change, immigration, criminal justice, and COVID-19 policy. The two topics a participant did not research served as within-subject controls, allowing the team to separate the effects of doing research from the effects of simply answering questions twice. This design gave the study a level of internal rigor that surveys and observational analyses of AI use often lack.

Participants were randomly assigned to one of four research tools: the chatbots GPT-4o, Claude-3.5, or Mistral, or conventional Google search. Before and after the research phase, everyone answered knowledge questions covering all four topics. By comparing belief in accurate statements and in misinformation across the before-and-after measurements, and across the researched and control topics, the researchers could quantify precisely how much genuine learning occurred under each condition — and whether any of the tools pushed people toward false beliefs.

The headline result is striking in its symmetry. Research increased belief in true information and decreased belief in misinformation regardless of which method participants used. Chatbot users did not walk away more misinformed than search users, and search users did not outperform chatbot users in factual accuracy. For a technology frequently accused of poisoning the information environment, the finding suggests that, at least for structured research tasks, large language models perform on par with the self-directed internet searching that has been the default for two decades.

There was one measurable difference between the modalities, and it favored the machines. The research process was 6 to 10 percent faster when participants used chatbots rather than Google search. That efficiency gain may sound modest, but multiplied across millions of information-seeking interactions, it hints at why conversational AI is displacing search as a research tool: it compresses the time between a question and a usable answer without, in this study, exacting a price in accuracy.

Just as notable is what did not change. Working with chatbots or with Google search did not alter participants’ trust in institutions, experts, the media, or technology itself. Critics have worried that AI tools might either inflate blind faith in expertise or erode it further; the data showed no such movement. The research phase left the landscape of institutional trust essentially untouched, suggesting that a single structured research session, however conducted, is not enough to shift these deeper attitudes.

The study did detect one shift that will draw scrutiny from across the political spectrum. Both chatbot and search conditions increased agreement with progressive views and decreased agreement with conservative views among all participants, regardless of the participants’ own political leaning. Because this pattern appeared in both modalities, it cannot be attributed to AI bias alone; something about actively researching claims on these topics moved people in the same ideological direction whether they consulted a language model or a search engine. The authors flag this as an important observation, but one that requires careful interpretation rather than alarm about chatbots specifically.

The researchers are candid about the limits of what their experiment can claim. They did not test models known to carry political biases, so the results cannot be generalized to every system on the market. Neither the chatbot condition nor the search condition reflected today’s AI-enhanced, personalized search experiences, which blend generative summaries into results pages and tailor output to individual users. And the study examined structured research tasks — verifying specific true or false claims — rather than the open-ended political conversations people increasingly have with AI assistants, where sycophancy, framing effects, and conversational drift could behave very differently.

Even with those caveats, the findings land at a consequential moment. Debates about AI governance and democratic integrity are unfolding in legislatures and standards bodies around the world, often driven by worst-case assumptions about what chatbots do to political knowledge. This study suggests that for one of the most common use cases — a citizen researching political claims before forming a judgment — conversational AI may perform on par with self-directed Google search, with no apparent cost to factual political knowledge. If that pattern holds as models and search experiences evolve, the policy conversation may need to shift from whether people should use AI for political research to how the tools should be designed, audited, and monitored as they become an ordinary part of democratic life.

Subject of Research: Effects of AI chatbot use on accurate political knowledge and beliefs compared with internet search

Article Title: AI chatbots may increase accurate political knowledge

Article References: AI chatbots may increase accurate political knowledge. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: AI chatbots, large language models, political knowledge, misinformation, GPT-4o, Claude-3.5, Mistral, Google search, UK election 2024, democracy, PNAS Nexus, AI governance