AI edges closer to passing Turing-like tests for language and vision

Artificial intelligence systems are closing in on one of the most enduring benchmarks of machine behaviour: the ability to pass as human. A large-scale study published in Nature Human Behaviour reports that current AI models are approaching the point of convincingly impersonating people in both language and vision tasks, while even simple computational judges can outperform humans at telling machine output apart from genuine human responses. The research, led by Mengmi Zhang, Elisa Pavarino and Gabriel Kreiman together with colleagues, systematically benchmarked the imitation abilities of 37 AI agents against responses collected from 636 human participants across six everyday tasks.

The team designed three language tasks — image captioning, word association and conversation — and three vision tasks: colour estimation, object detection and attention prediction. In each case, human participants and AI models produced answers to the same stimuli, and an independent pool of judges then attempted to classify each answer as human or machine. In total, the researchers conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges. The authors deliberately describe their procedure as Turing-like rather than a true Turing test, noting that their implementation differs from Alan Turing’s original conception, which never specified operational details.

The language results reveal how narrow the gap has become. In the image captioning task, human judges correctly identified human captions as human 67 per cent of the time, but recognised AI captions as machine-generated only 46 per cent of the time — barely above the 50 per cent chance level, and in the wrong direction. Word association showed a similar pattern: human answers were spotted 58 per cent of the time, while machine answers were detected at just 47 per cent, statistically indistinguishable from chance. In the conversation task, judges labelled human exchanges as human 66 per cent of the time but caught machine exchanges only 47 per cent of the time. Summarised as an overall imitation detectability score, where 0.5 represents a perfect imitator and 1.0 an easily detected one, the tasks yielded 0.57 for captioning, 0.53 for word association and 0.57 for conversation.

Performance varied considerably between models. Among captioning systems, the proportion of AI captions flagged as artificial ranged from 37 per cent for BLIP to 60 per cent for ClipCap. The researchers observed that the strongest imitators — GIT, OFA and BLIP — were generative transformer frameworks trained on multiple tasks, suggesting that multi-task training helps models produce more human-like descriptions. In the word association task, GPT3 with a davinci prompt achieved the best imitation performance, while Word2Vec was the easiest to detect. In conversations, Blenderbot stood out: in AI-to-AI dialogues it was judged human 67 per cent of the time, statistically comparable to the 64 per cent rate at which genuine human-to-human exchanges were classified as human. ChatGPT-4o was a slightly weaker imitator but still outperformed most competing models.

Several control analyses strengthened the findings. Detection performance did not change significantly with conversation length, remaining stable across exchanges from 3 up to 24, and extended experiments with 48-exchange dialogues produced consistent results. Imitation detectability varied modestly across conversation topics, with technology yielding 60 per cent detectability compared with 53 per cent for music. Judges’ gender and education level showed no significant effect on results, and explicitly informing participants that half the responses were machine-generated did not alter the outcomes. However, AI experts were more accurate than non-experts at recognising machine-generated captions, and the difference was statistically significant in the image captioning task, though not in the conversation task.

A striking divergence emerged when the researchers moved from passive evaluation to an interactive format closer to Turing’s original vision. Using a mobile application, each conversation involved one human judge, one human agent and one AI agent, with the judge actively probing both to determine their identities. In stark contrast to the offline results, human judges achieved 100 per cent accuracy in distinguishing human from AI agents, even when human agents were instructed to deliberately imitate machine behaviour. The authors attribute this to several factors: active judges can pose captious questions, probe personal experiences that AI models struggle to express convincingly, and exploit meta-clues such as response timing and length. Notably, when the team repeated the experiment with a LLAMA-3-70B-Instruct model whose safety safeguards were disabled, judges no longer achieved perfect accuracy, and their ability to detect machine responses increased with the number of conversational exchanges.

The study also demonstrated that algorithms themselves can serve as effective judges. Simple support vector machine classifiers, trained on text embeddings, consistently outperformed human judges at identifying machine output. In the captioning task, an SVM judge that never saw the images correctly identified human captions 72 per cent of the time and machine captions 83 per cent of the time. In the conversation task, an SVM evaluating single sentences detected machine answers 67 per cent of the time, surpassing human judges who had access to entire dialogues. Analyses of basic text statistics — parts-of-speech frequency, punctuation, word and sentence length — found no single feature that reliably distinguished humans from AI across tasks, suggesting the classifiers exploit subtler statistical signatures. In the vision tasks, AI judges again outperformed humans in object detection and attention prediction, although human judges retained the advantage in colour estimation.

The vision results mirrored the language findings. In colour estimation, human judges identified human answers 55 per cent of the time and AI answers 61 per cent of the time; even a simple median-cut colour quantisation algorithm fooled judges in 34 per cent of cases. In object detection, machine answers were caught 63 per cent of the time, with Facebook’s Detectron agent proving the most human-like, fooling judges 52 per cent of the time — significantly better than the Google, Azure and Rekognition APIs. In the attention prediction task, where judges watched animated sequences of eye movements overlaid on images, human scanpaths were recognised 63 per cent of the time while AI-generated scanpaths were detected at chance level, with the IVSN model fooling judges 57 per cent of the time. Ablation experiments showed that removing top-down target guidance from the attention model made its outputs easy to detect, underscoring the importance of both bottom-up and top-down mechanisms for human-like gaze behaviour.

Perhaps the most consequential finding concerns the relationship between imitation and conventional performance metrics. The researchers compared imitation detectability against standard image captioning benchmarks such as BLEU and CIDEr scores and found that human-likeness and traditional benchmark performance constitute distinct concepts. Some models with comparable imitation scores differed substantially in captioning accuracy, while others with higher benchmark scores proved worse at passing as human. Large multimodal foundation models, including Flamingo, ChatGPT and LLaVA, yielded imitation detectability on par with or lower than classical captioning models trained specifically for the task. The authors illustrate the distinction with an example in which GPT4 produced an accurate, articulate description of an unusual image, yet failed to capture the perceptual oddity that human observers immediately remarked upon — demonstrating that being correct and being human-like are not the same thing.

A single universal agent rounded out the picture. When ChatGPT-4o was used in a zero-shot manner across all six tasks, its imitation detectability remained around 0.5 throughout, ranging from 0.44 in object detection to 0.56 in attention prediction — an indication that a unified multimodal model can generate responses largely indistinguishable from human output without any task-specific training. The authors emphasise that their work is not intended as a measure of intelligence, but as a practical evaluation of imitation, with implications both beneficial, such as aligning machine decisions with human expertise, and harmful, such as disinformation and phishing. They note that even simple AI detectors outperforming human judges points toward a continuing arms race between better imitators and better detectors, that no gold-standard response exists that only humans could produce, and that as machines grow more human-like, society may need to educate human judges and reconsider what it means to prove one’s humanity.

Subject of Research: Turing-like tests benchmarking the ability of AI systems to imitate human responses in language and vision tasks

Article Title: Turing-like tests for language and vision demonstrate a narrowing gap between humans and AI

Article References: Zhang, M., Pavarino, E., Liu, X., Dellaferrera, G., Sikarwar, A., Chen, C., Armendariz, M., Mudrik, N., Agrawal, P., Madan, S., Shetty, M., Barbu, A., Yang, H., Kumar, T., Han, S., Singh, A. R., Sadwani, M., Dellaferrera, S., Pizzochero, M., … Kreiman, G. (2026). Turing-like tests for language and vision demonstrate a narrowing gap between humans and AI. Nature Human Behaviour. https://doi.org/10.1038/s41562-026-02590-6

Image Credits: AI Generated

DOI: 10.1038/s41562-026-02590-6

Keywords: Turing test, artificial intelligence, large language models, image captioning, computer vision, human-AI discrimination, imitation, attention prediction, benchmarking, Nature Human Behaviour, AI judges, conversational AI