Ensemble Clustering Offers Psychologists a Fix for Unstable Subgroups

Cluster analysis has quietly shaped psychology and psychiatry for more than eight decades, promising to sort messy human data into meaningful subgroups without the distortions of human bias. The technique dates back to 1939, when R.C. Tryon introduced the term to identify clusters of variables, and it has since grown into a cornerstone of research on symptom profiles, disease classification, treatment response, and biological markers. More than 60,000 clustering articles have been published to date, with over 5,000 appearing in 2024 alone. Yet a new open-access tutorial in Behavior Research Methods, led by Caroline X. Gao of the University of Melbourne and Orygen together with an international team, argues that the field’s most widely used clustering practices are quietly failing, and it offers a practical, code-heavy roadmap for a more reliable alternative: ensemble clustering.

The core problem, the authors explain, is that traditional single-run clustering methods lack stability, robustness, and generalisability. Stability refers to whether results remain consistent across repeated runs; robustness describes resilience to noise; generalisability concerns whether findings hold in different datasets. All three are chronically weak in conventional approaches. This is not simply a matter of sloppy implementation. Kleinberg’s impossibility theorem shows that no clustering method can simultaneously satisfy all desirable properties, and the optimisation problem underlying clustering has no exact solution. Algorithms therefore rely on assumptions and trade-offs, and their stochastic optimisers mean that minor perturbations of the data, different starting points, or a different choice of algorithm can produce substantially different subgroups from the same dataset.

The downstream consequence is a replication crisis for subgroup research. Identified subtypes often vary dramatically across studies, and when broadly similar patterns do emerge, they frequently reflect arbitrary cut-offs along a single severity dimension rather than genuine heterogeneity. In mental health research, the stakes are particularly high: despite decades of effort, current evidence does not support reproducible, biologically valid, and clinically agreed-upon neuroimaging subtypes for mental illnesses. By contrast, fields such as oncology have made real translational progress with tumour subtyping, largely by adopting ensemble clustering methods that combine solutions from different data distributions, algorithms, and optimisation runs into a single consensus.

Ensemble clustering works much like ensemble methods in supervised machine learning, where predictions from many models are pooled to reduce the weaknesses of any single model. The clustering version follows a two-stage framework. The first stage generates many base clusterings, or BCs, by running different algorithms, random seeds, parameter settings, or resampled subsets of observations and variables. The second stage aggregates these partitions into a consensus, defined not as complete agreement but as the stable grouping pattern that recurs across solutions. The tutorial stresses that good BCs should be both diverse and high quality, a difficult balance because aggressive sub-sampling increases diversity while degrading individual partition quality, and optimising quality tends to drive all BCs toward the same answer.

The consensus stage is where the methodological richness lies. The simplest approach is majority voting, but because clustering algorithms return arbitrary cluster labels, BCs must first be relabelled and aligned, and voting fails entirely when BCs are highly diverse. More sophisticated techniques summarise the BCs before running a second-stage clustering model. Cluster labels can be treated as categorical variables and clustered with finite mixture models or k-modes. They can be binary-encoded into a cluster association matrix for k-means-based consensus clustering, or converted into bipartite graphs for graph partitioning methods such as the hypergraph-partitioning algorithm. Alternatively, a co-association matrix records how often each pair of observations is grouped together across BCs, serving directly as a distance matrix for hierarchical clustering in evidence accumulation clustering, or transformed into weighted graphs for algorithms such as the cluster-based similarity partitioning algorithm. Newer options include the link-based clustering ensemble, the meta-clustering algorithm, adaptive clustering ensemble, and even deep learning approaches.

Choosing the right components depends on the data. K-means is the most common BC algorithm because it is computationally cheap, though it assumes spherical clusters and is sensitive to outliers; model-based methods such as latent class analysis may better suit psychological data where subgroups plausibly arise from latent generative processes. The co-association matrix remains the most popular consensus tool because heatmaps of it visually display cluster stability and help select the number of clusters. Ensemble size matters too: performance generally improves with more BCs, but benchmarking studies suggest quality plateaus between roughly 50 and 100 base clusterings. A sampling proportion of about 80 percent is typically recommended, capturing most data characteristics while preserving variation among BCs.

A major practical payoff of the ensemble approach is a principled way to choose the number of clusters. Instead of relying on internal validity indices such as Silhouette, Calinski-Harabasz, Dunn, or Gap statistics, which often disagree with one another, researchers can inspect the consensus matrix. When the chosen number of clusters matches the true structure, the matrix displays clear square-shaped blocks along the antidiagonal, indicating pairs of observations that are consistently grouped together or apart. This visual pattern can be quantified through the cumulative distribution function of consensus indices, the delta area plot, the cophenetic correlation coefficient, and the proportion of ambiguous clustering score, though the authors caution that PAC carries an inherent bias toward larger cluster counts because it ignores null reference distributions.

To make the method concrete, the tutorial walks through a full R implementation using packages such as diceR, ConsensusClusterPlus, and clue, with step-by-step code provided in supplementary materials. Using a simulated dataset of 600 observations with 12 variables generated from three multivariate normal clusters, the authors demonstrate the dice function, combining k-means, Gaussian mixture models, and Ward’s hierarchical clustering with the link-based consensus function across 100 BCs. They also cover pre-registration of analysis plans, data assessment, dimensionality reduction via PCA or alternatives such as UMAP and NMF, imputation for missing data introduced by resampling, trimming and reweighting of poor-quality BCs, permutation tests to confirm solutions did not arise by chance, and cross-validation frameworks for tuning hyper-parameters in the absence of ground truth.

The most striking evidence comes from simulations built on three real-world psychological datasets covering quality of life, alexithymic traits, and cannabis expectancies. The team perturbed each dataset by randomly removing between 0.25 and 20 percent of the data, refilling gaps with multiple imputation, and repeated the process 100 times per condition. Even tiny perturbations of 0.25 percent produced substantially different results under single-run methods; hierarchical clustering in one case performed barely better than random assignment. Ensemble clustering consistently showed the slowest decay in stability, measured by the Rand index, and much narrower variability across perturbed dataset pairs, particularly for higher-dimensional data.

The authors are careful about limits. Ensemble methods inherit the assumptions of their base algorithms, so stable solutions can still miss non-convex or arbitrary cluster shapes; stability and internal validity do not guarantee external validity, and even valid, generalisable solutions may lack practical utility if they merely partition severity. Ordinal Likert-scale items and latent constructs can distort distance matrices, and longitudinal data require specialised Bayesian ensemble pipelines. Computational cost is real, though manageable with binary association matrices and high-performance computing. Still, the tutorial’s message is unambiguous: minor perturbations of psychological data can upend single-run clustering, ensemble methods tame that instability, and with mature R tooling now available, the authors advocate widespread adoption to finally turn heterogeneous psychological data into subgroups researchers can trust.

Subject of Research: Ensemble clustering methods for improving the stability and reproducibility of subgroup identification in psychological and psychiatric research

Article Title: Ensemble clustering: A practical tutorial

Article References: Gao, C. X., Wang, S., Zhu, Y., Ziou, M., Teo, S. M., Smith, C. L., Chiu, D., Talhouk, A., Wang, M., Yu, W., Cotton, S. M., & Dwyer, D. (2026). Ensemble clustering: A practical tutorial. Behavior Research Methods, 58(10), Article 286. https://doi.org/10.3758/s13428-026-03158-y

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03158-y

Keywords: ensemble clustering, cluster analysis, consensus clustering, psychiatry, psychology, machine learning, unsupervised learning, R programming, replication crisis, latent class analysis, stability, Behavior Research Methods