The research, led by Nuril Kusumawardani Soeprapto Putri of Institut Teknologi Sepuluh Nopember and Bina Nusantara University, together with Umi Laili Yuhana, Daniel Oranova Siahaan, and Istiani, addresses a problem familiar to anyone who has worked with large-scale assessment data. Universities routinely collect performance measures at admission, but these numbers are rarely translated into actionable profiles that instructors can use. The authors argue that before artificial intelligence can meaningfully stratify learners, the analytics pipeline itself must be reproducible, leakage-free, and honest about what a statistical cluster does and does not represent. Their framework is explicitly designed so that clusters are treated as operational descriptions of performance patterns, not as naturally discrete or rank-ordered psychological categories.

At the foundation of the approach lies multidimensional item response theory, a psychometric technique that estimates latent abilities from how students answer individual test items rather than relying on raw totals. The researchers extracted ability estimates in three domains: logical reasoning, abstraction, and verbal reasoning. This multidimensional representation matters because two students can arrive at the same overall score through very different routes. One might excel in verbal tasks while struggling with logic, while another shows the reverse pattern. A single aggregate number collapses those differences; a three-dimensional ability vector preserves them and makes richer profiling possible.

To guard against data leakage, one of the most insidious pitfalls in applied machine learning, the cohort was split into a training set of 29,449 students and a held-out test set of 7,364 students. Every transformation, every cluster boundary, and every profile label was derived exclusively from the training partition and then transferred, unchanged, to the test partition. This discipline ensures that the reported performance reflects how the system would behave on genuinely new students rather than on data it has already seen. Many published clustering studies skip this step entirely, fitting and evaluating on the same data and thereby inflating apparent quality. Here, the split is the backbone of the entire validation strategy.

The first level of the framework applied k-means clustering with the number of clusters fixed at four, a choice justified by a composite ranking of three standard internal validation indices: the Silhouette score, which measures how well each point fits its own cluster compared to neighbors; the Davies-Bouldin index, which rewards compact, well-separated clusters; and the Calinski-Harabasz index, which compares between-cluster to within-cluster dispersion. Using multiple indices rather than a single criterion reduces the risk of selecting a cluster count that flatters one metric but fails on others. The resulting four profiles were characterized not by how much ability students had, but by the shape of their ability centroids: a uniformly lower profile, a uniformly higher profile, a logic-dominant profile, and a verbal-abstraction dominant profile.

One of the most striking findings comes from the relationship between those two middle profiles. The logic-dominant and verbal-abstraction dominant groups crossed each other across the three ability dimensions, meaning that neither could be described as simply better than the other. If cognitive performance lay along a single ordinal continuum from low to high, such crossing would be impossible. Its presence demonstrates, in the study’s own data, that student cognition cannot be compressed onto one severity scale. Students are not merely higher or lower performers; they exhibit distinct patterns of relative strengths, and those patterns carry information that intervention designers can exploit.

The second level of the pipeline drilled into the uniformly lower profile, subdividing it into four domain-specific sub-profiles. This stratification is arguably the study’s most practically valuable contribution. Students in the lowest overall group are not interchangeable; some struggle more with logic, others with abstraction or verbal reasoning. By separating these sub-groups, the framework gives educators a way to tailor remediation, for example by directing logic-focused support to one sub-profile and verbal or comprehension-focused support to another, rather than applying a generic intervention to everyone below a cutoff. Differentiated support of this kind has long been an aspiration in higher education; this work offers a scalable computational route toward it.

The statistical evidence for the clustering’s strength was substantial. A PERMANOVA test, a non-parametric analysis of variance suited to multivariate distance data, yielded a pseudo-F statistic of 10,798 and an R-squared of 0.524, indicating that the profile assignment accounted for more than half of the multivariate variance in the ability estimates. Among the three cognitive domains, logical reasoning contributed most strongly to differentiating the profiles, with an effect size of eta-squared equal to 0.81. These are large effects by any convention in the behavioral sciences, and they indicate that the four-way partition captures genuine structure in the data rather than an arbitrary slicing of a homogeneous cloud.

The authors are notably careful about what the final validation step proves. They used hierarchical label recovery to reconstruct profile assignments on the held-out test data, achieving approximately 98 percent accuracy, which demonstrates that the assignment rule transfers reliably to unseen students. Yet they also report that a simple baseline model achieved comparable performance, and they interpret this honestly: the result demonstrates reproducibility of the assignment rule under leakage control, not psychological validation of the clusters as true categories. This distinction matters. The clusters are useful operational summaries, but the framework does not claim that four cognitive types exist in nature. That kind of epistemic restraint is rare and welcome in a field often tempted to overinterpret unsupervised structure.

Broader implications extend beyond the specific dataset. The study received institutional support from the INNOGEN Lab of Bina Nusantara University and was conducted as a retrospective analysis of fully anonymized secondary data, with ethical clearance from the university’s research ethics committee. Because the pipeline is computationally scalable and the labels are interpretable, the approach could in principle be adopted by institutions that already administer multidimensional admission or placement assessments. The framework offers an interpretable and efficient way to convert raw assessment data into profiles that support differentiated educational interventions, while its emphasis on held-out validation and modest claims sets a methodological example for the growing field of learning analytics. As universities increasingly turn to data-driven tools to support struggling students, this work suggests that the most important algorithmic ingredient may be rigor: get the measurement right, split the data honestly, and never mistake a convenient cluster for a psychological fact.

Subject of Research: A reproducible two-level machine learning framework for cognitive profiling and stratification of low-performing students in higher education

Article Title: A reproducible two level clustering framework for profiling cognitive performance and stratifying the lowest performing group in higher education

Article References: Putri, N. K. S., Yuhana, U. L., Siahaan, D. O., & Istiani (2026). A reproducible two level clustering framework for profiling cognitive performance and stratifying the lowest performing group in higher education. Discover Psychology. https://doi.org/10.1007/s44202-026-00888-0

Image Credits: AI Generated

DOI: 10.1007/s44202-026-00888-0

Keywords: cognitive profiling, item response theory, k-means clustering, cluster validation, reproducibility, higher education, learning analytics, psychometrics, data leakage, PERMANOVA, educational intervention, machine learning