The core problem the researchers set out to solve is what they describe as two-sided “goal–action ambiguity.” Conventional AI systems infer human intent from externally observable information — speech, gestures, actions — but the same observable behavior can reflect very different goals, and the same goal can be pursued through very different actions. When a person picks up a cup, an observer cannot know whether they intend to drink from it or to hand it to someone else. Conversely, if the goal is simply to quench a thirst, that goal could be satisfied by reaching for the cup, grabbing a water bottle, or asking another person to bring a drink. Because of this ambiguity, when an AI misreads a human’s intent, the user is typically forced to correct it through additional commands or demonstrations, a slow and often frustrating loop that undermines the promise of natural collaboration between people and machines.

The KAIST and MSRA team’s insight was to bypass observable behavior entirely and tap into the brain’s own error-detection machinery. When a person encounters an unexpected situation, the brain produces rapid, unconscious “prediction error” signals — an instantaneous internal alarm that something does not match expectations. In the context of human–AI interaction, these signals amount to the brain’s immediate “that’s not what I meant” response when an AI performs the wrong action or pursues the wrong goal. Crucially, the researchers found that these signals are not monolithic. They distinguished two distinct types of brain responses, each carrying different information about what exactly went wrong in the collaboration.

The first type is reward prediction error, or RPE, which appears when the AI misunderstands the person’s ultimate goal — when the machine has grasped the wrong endpoint entirely. The second is state prediction error, or SPE, which appears when the goal is correct but the process or method of action differs from what the person expected. This distinction matters enormously for how an AI should respond. If the brain is broadcasting an SPE signal, the appropriate correction is to keep the goal and change the strategy. If it is broadcasting an RPE signal, the AI must discard its assumption about what the person wants and search again for the true objective. By separating these two error classes, the technology gives AI systems a structured way to diagnose the nature of their own misunderstanding rather than simply registering that something is amiss.

To capture these signals, the team measured real-time electroencephalography (EEG) from people as they observed AI systems performing tasks. The recordings revealed that the brain produced measurably different patterns depending on whether the AI had misunderstood the goal itself or had chosen the wrong method while pursuing the correct goal. The researchers also identified distinctive brainwave signatures that emerged when both types of errors occurred simultaneously — a doubly ambiguous situation in which neither the destination nor the route matched the person’s expectations. These findings establish, at the level of neural measurement, that the human brain encodes goal-level and method-level disagreement with an AI as separable, decodable phenomena.

The decoding itself was accomplished with deep learning. By training neural networks on the recorded EEG signals, the team developed a system that can determine, from brain activity alone, how a person is interpreting the AI’s behavior at any given moment. In practical terms, even without a person saying “that’s wrong,” the AI can recognize whether the human brain is signaling that “the goal is wrong” or “the method is wrong.” The team then wrapped this decoding capability into a Neural Value Alignment–based human–AI synergy algorithm, which feeds the decoded brain signals back to the AI in real time. When the AI detects an SPE signal, it interprets the situation as “the desired goal is correct, but the method is wrong” and adjusts its action strategy accordingly. When it detects an RPE signal, it understands that “the goal itself was misunderstood” and re-searches for what the person truly intended. In both cases, the correction loop closes without a single spoken word.

Simulation results offered early evidence that the approach works under realistic stress. The proposed method adapted more quickly than existing approaches even in uncertain conditions, such as when a person’s goal changed suddenly mid-task or when some of the human neural feedback was missing — a scenario that reflects the noisy, incomplete data typical of real-world brain–computer interfaces. This robustness matters because laboratory EEG is far cleaner than what wearable sensors will deliver in homes, factories, or vehicles. The researchers emphasize that the significance of the work lies in demonstrating that AI systems can correct themselves by reading a person’s unconscious “that’s not what I meant” brain response, without requiring the user to repeatedly say “do it this way” or “that’s not right.”

The potential applications stretch across the emerging landscape of embodied and interactive AI. As the technology matures, the researchers say, it could be applied to physical AI robots in homes and industrial settings, allowing them to understand user intent more naturally and adjust their actions accordingly. Autonomous vehicles could quickly reflect driver judgment in situations where a split-second correction matters. Medical and rehabilitation robots could serve patients who have difficulty speaking or moving, for whom conventional command-based interaction is impractical and whose needs are most acute. Educational AI systems could adapt to a student’s cognitive state in real time, sensing confusion or disagreement that the student never articulates. In each case, the common thread is a tighter, faster coupling between human judgment and machine behavior than explicit interfaces allow.

Professor Sang Wan Lee of KAIST’s Department of Brain and Cognitive Sciences, who directs the Center for Neuroscience-Inspired Artificial Intelligence and led the international collaboration, framed the work as a shift in the source of intent information. “This research is meaningful because it shows that AI can move beyond inferring human intent only from visible behavioral outcomes and instead directly use cognitive signals generated in the brain during collaboration with AI,” he said. He added that the technology can be expanded to a wide range of fields where human judgment and AI behavior must be closely connected, including physical AI, brain–computer interfaces, autonomous driving, precision personalized education, medical robotics, and human–computer interaction. From MSRA, Miran Lee, Director of the Microsoft Research Accelerator, described the achievement as the result of the ongoing international collaboration between KAIST and Microsoft Research Asia and expressed the intention to continue the partnership to develop world-class BCI technologies that enable humans and AI to communicate and collaborate more naturally.

The study’s first author is Xin Xu, a Ph.D. student in KAIST’s Department of Brain and Cognitive Sciences; researchers from Microsoft Research Asia, including Yansen Wang, Dongqi Han, and Dongsheng Li, also participated. The paper, titled “Neural Value Alignment: Human–AI Collaboration Under Goal–Action Ambiguity,” appeared in IEEE Transactions on Cybernetics under DOI 10.1109/TCYB.2026.3722605. The work builds on a broader research program: a companion KAIST–MSRA study on helping AI adapt rapidly to continuously changing environments, led by first author Niklas Koeppe, was presented in June at ICML 2026 under the title “Mitigating Plasticity Loss through Architectural Design in Continual Learning.” The research was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP), funded by the Ministry of Science and ICT. Taken together, the results sketch a future in which the most important interface between humans and machines is not the keyboard, the touchscreen, or even the voice assistant, but the silent, instantaneous feedback of the brain itself — a future in which AI finally hears the corrections we never say out loud.

Subject of Research: Brainwave-based AI technology that detects human–AI cognitive mismatch and realigns AI behavior with human intent

Article Title: KAIST develops AI that can sense “that’s not what I meant” without being told

Article References: KAIST develops AI that can sense “that’s not what I meant” without being told. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: KAIST, Microsoft Research Asia, Neural Value Alignment, brain–computer interface, EEG, reward prediction error, state prediction error, human–AI collaboration, deep learning, intent inference, IEEE Transactions on Cybernetics, physical AI