Electroencephalography, or EEG, has long promised a faster and more objective way to detect brain disorders. By recording the brain’s electrical activity through sensors placed on the scalp, researchers can capture patterns that may differ between people with schizophrenia and those without the condition. In recent years, artificial-intelligence systems have been trained to recognize those patterns, sometimes producing remarkably high classification scores. But a new analysis argues that some of those impressive results may reflect a hidden problem in the research process rather than a genuine diagnostic breakthrough: information leakage.
Information leakage occurs when data from the test set, directly or indirectly, influences the model during training. In machine learning, the test set is supposed to remain untouched until the final evaluation. It acts as a simulation of completely new patients. If information from those patients enters the training process, even in subtle ways, the model may appear more accurate than it really is. The study by Ferenc S. Racz and Gergely Csukly, published in Translational Psychiatry, examines how this problem can affect EEG-based schizophrenia detection and reviews evidence from both the scientific literature and empirical analyses.
The issue is particularly serious for EEG because the recordings contain multiple layers of information. Alongside neural activity, signals may encode differences in equipment, recording locations, electrode placement, session conditions, medication status, age, movement, muscle activity and other technical factors. A machine-learning algorithm does not automatically understand which features are biologically meaningful. Unless researchers carefully control the experiment, a model may learn to identify the hospital where a recording was made, the person who contributed it or the characteristics of a particular recording session instead of learning a reproducible signature of schizophrenia.
One common route to leakage begins before the algorithm is trained. Researchers may combine recordings, remove noise, normalize signals or select informative features across the entire dataset and only afterward divide the data into training and test groups. Even though the final model never directly sees the test labels, the preprocessing step has already allowed information from the test participants to shape the representation used for learning. In large datasets, this can make the test cases resemble the training cases in ways that would not occur when the system encounters patients from a new clinic.
Another danger arises when recordings from the same individual appear in both the training and test sets. EEG signals have person-specific characteristics, including stable anatomical and physiological patterns. If one recording from a participant is used for training and another is reserved for testing, the algorithm may recognize the individual rather than the disorder. The resulting score can look like successful diagnosis, but the system may fail when asked to analyze a completely new person. For clinical applications, where the goal is generalization across patients and institutions, this distinction is fundamental.
The researchers’ analysis places these concerns within the wider literature on automated schizophrenia classification. High accuracy is often presented as evidence that neural signals contain a clear and detectable disease signature. Yet performance estimates depend heavily on how data are divided, how preprocessing is performed and whether the evaluation reflects the intended clinical setting. A model tested on recordings collected under conditions similar to its training data may perform well, while the same model can lose accuracy when exposed to new equipment, new populations or different recording protocols.
The technical lesson is not that EEG-based artificial intelligence is futile. Rather, the study highlights the need for stricter experimental design. Data should be divided at the participant level, ensuring that no individual contributes recordings to more than one partition. Operations such as normalization, feature selection and dimensionality reduction should be fitted exclusively on the training data and then applied unchanged to validation or test data. Researchers should also distinguish between internal cross-validation and genuinely independent external validation, in which the algorithm is evaluated on data from a separate cohort or institution.
The findings have implications beyond academic benchmarks. Schizophrenia is a complex psychiatric disorder with substantial variation in symptoms, treatment histories and biological characteristics. A model that performs well under controlled research conditions may not provide reliable support for clinicians unless it has been tested across diverse populations. Inflated performance can create false confidence, encourage premature deployment and make later failures harder to interpret. Transparent reporting of participant splits, preprocessing pipelines, sample sizes and external validation is therefore as important as the choice of neural network or signal-processing method.
The new work arrives as the field confronts a broader challenge in medical artificial intelligence: the difference between recognizing patterns and proving clinical usefulness. EEG may eventually contribute to earlier detection, patient stratification or treatment planning, but trustworthy systems must learn disease-related signals rather than shortcuts hidden in the data. By drawing attention to information leakage and overestimation, Racz and Csukly’s study urges researchers to replace spectacular but fragile accuracy claims with evaluations designed to survive contact with real-world patients. In brain science, the most convincing algorithm may not be the one with the highest score, but the one that still works when everything familiar about the data has changed.
Subject of Research: EEG-based schizophrenia detection, information leakage, and machine-learning performance evaluation
Article Title: Information leakage and performance overestimation in EEG-based schizophrenia detection: evidence from literature and empirical analyses
Article References: Racz, F.S., Csukly, G. Information leakage and performance overestimation in EEG-based schizophrenia detection: evidence from literature and empirical analyses. Transl Psychiatry (2026). https://doi.org/10.1038/s41398-026-04315-9
Image Credits: AI Generated
DOI: https://doi.org/10.1038/s41398-026-04315-9
Keywords: EEG, schizophrenia, machine learning, artificial intelligence, information leakage, data leakage, diagnostic biomarkers, model validation, clinical neuroscience, performance overestimation
