AI ‘digital twins’ fail to mirror human economic preferences across 74 countries

The dream of replacing expensive, slow, globe-spanning surveys with cheap synthetic respondents generated by large language models has just received its most rigorous stress test yet — and the results are sobering. In a sweeping audit published in Nature Human Behaviour, researchers simulated the Global Preference Survey (GPS), one of the most comprehensive cross-national measurements of economic preferences ever conducted, using twenty different configurations of large language models. The verdict: while AI-generated answers show a faint but systematic echo of human responses, they explain only a sliver of the real variation in how people around the world value risk, time, fairness and trust. For anyone hoping that artificial intelligence could stand in for human survey participants in economics or policy research, the message is clear — not yet, and perhaps not soon.

The original GPS, conducted in 2012 as part of the Gallup World Poll, gathered responses from more than 80,000 people across 76 countries representing roughly 90 percent of the world’s population and economic output. It measures six experimentally validated preference domains that form the backbone of modern economic theory: risk-taking, patience, positive reciprocity, negative reciprocity, altruism and trust. These are not abstract curiosities. Patience predicts saving behaviour, educational attainment and entrepreneurial entry; trust correlates with financial development and growth; reciprocity underpins contract enforcement and workplace cooperation. If a language model could reproduce this preference landscape reliably, it could in principle generate counterfactual data for policy analysis, easing one of the deepest problems in causal inference — that we can never observe what would have happened under conditions that never occurred.

The research team, led by Fadong Chen and Qijun Wang of Zhejiang University together with Sudeep Bhatia of the University of Pennsylvania, took an unusually careful approach. Rather than asking models to answer surveys in the abstract, they built one-to-one ‘digital twins’: for each of 67,201 matched human respondents across 74 countries, they constructed a persona embedding that person’s age, gender, cognitive ability and country, then prompted each model to answer the identical GPS questionnaire. Crucially, the prompts were delivered in the respondents’ original survey languages, with payoffs denominated in local currencies — a Chinese respondent’s donation question, for instance, was framed in RMB. Twelve main model configurations, spanning OpenAI’s GPT family, Google’s Gemini models and DeepSeek-V3, generated roughly 27.8 million answers across about 1.39 million persona completions.

The headline finding is one of weak but genuine alignment. Pooling all countries, 63 of 72 correlations between model-predicted and human preferences — that is, 87.5 percent of the six preference domains across twelve configurations — were significantly positive. But the magnitude tells a different story. After normalizing both human and synthetic preference scores, the mean squared error between them averaged 1.827, nearly twice the variance of the human benchmark itself. Regression analyses showed that LLM-predicted preferences explained just 1.3 percent of the variance in individual human responses and, even after aggregating to country averages, only 4.2 percent of country-level variance. In plain terms, knowing what the AI ‘thinks’ a person would say tells you almost nothing about what that person actually said.

Perhaps the most striking pattern is that the models agree with each other far more than they agree with humans. Between-model mean squared errors were substantially lower than model-to-human errors at both the individual and country levels. This convergence suggests that today’s language models share a common output regularity — a kind of statistical accent — that systematically departs from the human benchmark. Notably, newer and larger models did not outperform older ones: GPT-5.4 showed no measurable advantage over GPT-3.5-turbo, and Gemini-2.5-Flash did not beat Gemini-2.0-Flash. Scaling alone, at least across these generations, does not appear to close the human-machine gap in economic preferences.

The audit then probed whether models reproduce the well-documented demographic gradients in preferences — differences by gender, age and cognitive ability. Here the results were more encouraging. Using a confidence-interval-based consistency criterion, models recovered 77.8 percent of the demographic patterns found in the GPS data, with cognitive-ability gradients matched in 84.7 percent of cases and gender gradients in 81.9 percent. Risk-taking and patience gradients were recovered perfectly across all twelve configurations. But cultural variation proved far harder. When the team tested whether models captured differences along fifteen cultural dimensions drawn from the World Values Survey, Hofstede’s framework and Schwartz’s value orientations, the consistency ratio collapsed to 48.1 percent — statistically indistinguishable from a coin flip. Only three of twelve models recovered the link between Hofstede’s long-term orientation and patience, and just one recovered the relationship between uncertainty avoidance and risk-taking.

Why do models handle demographics but stumble on culture? Ablation experiments, in which pieces of the persona prompt were systematically removed, provided a diagnostic answer. Stripping demographic information degraded alignment substantially, while removing the country label barely changed the results at all — individual-level error was 1.838 for full personas versus 1.961 when only country information was retained. The models, it seems, condition effectively on traits like age and gender but largely ignore the country tag, treating a persona in Nairobi much like one in Oslo. Demographic effect sizes in the ablation conditions fell from an average of 0.303 to nearly zero when demographic cues were absent, confirming that the recovered gradients depend on explicit prompt information rather than deep cultural knowledge.

The residual errors were not randomly distributed, either. The team constructed an ‘inconsistency index’ — the sampling-weighted within-country mean of individual prediction errors — and correlated it with national characteristics. In many models, prediction errors were significantly larger in countries with lower GDP per capita and weaker digital infrastructure, as measured by the ICT Development Index, while population size and internet-user counts showed no systematic relationship. The authors suggest two mechanisms: languages and content from wealthier countries receive disproportionate emphasis in model development, and digitized economies generate more of the high-quality structured data on which models train. The equity implications are uncomfortable — AI tools may simply work worse for the populations that stand to benefit most from careful policy analysis.

Practical replication tests drove the point home. When the researchers attempted to reproduce 35 published country-level findings from the GPS literature — such as the association between patience and GDP per capita, or gender gaps in preferences and gender equality — the best-performing model reproduced only 16, or 45.7 percent, with the same direction and statistical significance. Two additional biases emerged: women’s prosocial preferences were systematically overestimated across virtually all models, with the distortion largest in countries with stronger gender stereotypes, and model responses were far more concentrated than human ones, a homogenization that raising the sampling temperature only partially repaired.

The authors are careful not to declare LLM simulation useless. Synthetic respondents may still serve as tools for early-stage hypothesis generation, study-design diagnostics, robustness checks and ethically sensitive counterfactual exploration before human experiments begin, and the one-to-one digital-twin protocol itself offers a reusable benchmark for tracking how cultural alignment evolves across model generations. But the core conclusion stands as a warning against a seductive shortcut. Language models do not yet encode the cultural texture of human economic life, and their failures are patterned along lines of global inequality. Until that changes, understanding what real people want — in Nairobi, Oslo or anywhere else — still requires asking them.

Subject of Research: Validity of large language model-simulated survey responses as proxies for human economic preferences across 74 countries

Article Title: Simulated responses from large language models are weak proxies for human economic preferences across 74 countries

Article References: Simulated responses from large language models are weak proxies for human economic preferences across 74 countries. (n.d.). https://doi.org/10.1038/s41562-026-02611-4

Image Credits: AI Generated

DOI: 10.1038/s41562-026-02611-4

Keywords: large language models, economic preferences, Global Preference Survey, survey methodology, cross-cultural alignment, digital twins, synthetic data, behavioral economics, AI bias, Nature Human Behaviour, policy evaluation, cultural diversity