The problem addressed by the method is a mismatch between the standard federated-learning model and the data held by its participants. In FedAvg, every client trains a common model on local records, after which the server combines the resulting parameters, ordinarily in proportion to each client’s number of samples. That procedure does not account for differences in the balance of positive and negative examples among clients. Such heterogeneity can produce slower convergence, unstable updates and poorer performance than a single global model can achieve on clients with sharply different distributions. The study examined eight distributions ranging from an identical, 50–50 split to extreme label imbalance. Sentiment140 was divided into 100 training clients and 50 test clients, each containing 800 examples, while the IMDB replication used 100 training clients with 360 reviews each and 50 unseen test clients with 140 reviews each.

Client clustering offers a potential remedy because clients with similar label mixtures can train models suited to similar conditions. However, several earlier approaches determine a new client’s cluster by using its labels, whether through loss, accuracy, gradients or local model updates. Such a procedure technically allows test information to influence model selection, and it does not help when a genuinely new participant arrives without labels. The study instead trained the same autoencoder architecture separately on every client, including held-out test clients. Each client’s 384-dimensional sentence embeddings were reconstructed through a 16-neuron bottleneck using two transformer blocks in the encoder and two in the decoder. The autoencoder contained 236,928 trainable parameters, was optimized for 50 epochs with Adam and mean-squared-error loss, and reached reconstruction losses of approximately 10⁻⁵ in the evaluated setting.

The learned autoencoder parameters served as a fixed-dimensional fingerprint of each client’s local data. After an optional principal component analysis step retained the first 100 components, K-means++ grouped clients into two to seven clusters. A test client was assigned to the cluster whose center had the smallest Euclidean distance from its autoencoder representation, without access to its sentiment labels. The researchers also tested encoder-only weights, 16-dimensional latent representations and a parameter-matched multilayer perceptron autoencoder. Full autoencoder weights produced better-separated clusters than the latent representations, while the transformer autoencoder achieved higher silhouette scores than the nearly equal-sized multilayer perceptron alternative across the examined values of K. The authors nevertheless cautioned that autoencoder parameters are not uniquely identifiable, so similarity between them is an empirical client representation rather than a guaranteed measurement of distributional similarity.

Clustering was only the first stage. Within each cluster, the researchers trained a transformer classifier with 1,190,518 parameters on locally held data and combined client updates at the server. Three proximity-aware alternatives to ordinary FedAvg were tested. Negative Softmax assigned smaller coefficients to clients farther from the cluster center but did not renormalize the selected clients’ scores in every communication round. Negative Softmax with normalization preserved that ordering while forcing the participating clients’ scores to sum to one. A one-minus-min-max approach also emphasized central clients, although it could assign a score of zero to a client at the greatest observed distance. The normalized negative-softmax variant was the most consistent overall, particularly in moderately skewed or nearly balanced client populations, and was therefore used in the main comparison.

On Sentiment140, clustered learning with autoencoder representations, PCA, K-means++ and normalized negative softmax beat the unclustered FedAvg baseline in every scenario. In the balanced 50–50 condition, accuracy ranged from 76.91% to 78.36%, compared with 68.92% for FedAvg. With the 5–95 label split, clustered accuracy reached 94.71% to 95.00%, versus 65.92% without clustering. In the extreme Dirichlet condition, labelled D 0.01, each client effectively contained only one sentiment category: clustered accuracy was 100.00%, compared with 50.00% for the baseline. That result primarily reflects correct label-free assignment of single-class clients and cannot be interpreted as general two-class performance because the other class was absent from each client. A separate D 0.1 condition produced 97.74% clustered accuracy and a 66.10% FedAvg baseline. The more informative 5–95 condition offered macro-F1 scores of 73.79% to 74.77% and balanced accuracy of 75.30% to 76.88%, indicating that its high overall accuracy was not simply the result of predicting one dominant class.

The IMDB evaluation showed the same broad separation from the non-clustered baseline. With two to seven clusters, accuracy ranged from 82.93% to 83.60% in the 5–95 scenario, compared with 72.73% for FedAvg, while the balanced 50–50 condition produced 78.87% to 79.21%, compared with 76.83%. The extremely skewed D 0.01 condition reached 91.49% to 92.34% with clustering but only 57.05% without it; as on Sentiment140, that result chiefly reflects successful assignment of clients dominated by a single class. Performance changed only slightly as the number of clusters varied from two to seven. A related Friedman analysis found no significant accuracy differences among those K values in any Sentiment140 scenario, with all p-values above 0.39. Repeated tests with three random client-data assignments revealed statistically detectable seed differences in the 5–95 and D 0.1 conditions, but their between-realization standard deviations remained below 1%, indicating that the practical differences were small. The K values in the box plots consequently represent sensitivity to clustering configuration rather than confidence intervals or independent repeated-run uncertainty.

The analysis also probed the reasoning behind proximity weighting. The authors assumed that clients farther from a cluster center would generally show larger discrepancies between their local gradients and the cluster-level objective, then used the rearrangement inequality to argue that giving such clients smaller weights would not increase a weighted second-moment measure of gradient discrepancy compared with equal weighting. This was a conditional analytical result, and the study explicitly presented its extension to unequal client sizes as intuition rather than a general proof. Empirically, the relationship between center distance and gradient discrepancy was positive and statistically significant in four representative clusters, though its strength varied from weak to moderate. The analysis therefore supports the weighting rule only under the tested tendency, not as a universal property of federated clustering. The reported benefit of normalization was also chiefly stability: clustered FedAvg and the weighted variants often produced similar final accuracy, while normalized weighting reduced variation across cluster configurations and converged faster in the reported analysis.

Several limitations prevent the results from establishing deployment readiness. The primary and cross-dataset studies both partitioned data from the same collections into artificial clients according to controlled label proportions, so they did not test natural user, organization, temporal or domain shifts. The test clients were excluded from model training and their labels were not used to assign them, but they still came from the same datasets and partitioning procedure as the training population. Other practical factors—including unequal client sizes, clients entering or leaving the system, communication overhead, latency, energy use and end-to-end cost—were not explicitly modeled. The study also compared the method chiefly with conventional non-clustered FedAvg and internal ablations rather than with the full range of personalized or clustered federated-learning systems, many of which were excluded because their assignment procedures rely on labels or assumed distributional relationships. The evidence therefore supports label-free cluster assignment and improved controlled sentiment classification, while leaving performance on naturally heterogeneous federated populations unresolved.

Subject of Research: Label-free clustered federated learning for sentiment analysis under non-IID client data

Article Title: Autoencoder-based clustered federated learning with importance-aware aggregation for sentiment analysis on non-IID data

Article References: Gholamiangonabadi, D., Danish, M. U., Chen, Z., & Grolinger, K. (2026). Autoencoder-based clustered federated learning with importance-aware aggregation for sentiment analysis on non-IID data. Machine Learning with Applications, 26, Article 101046. https://doi.org/10.1016/j.mlwa.2026.101046

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101046

Keywords: federated learning, sentiment analysis, non-IID data, client clustering, autoencoders, transformer models, label skew, privacy-preserving AI, model aggregation, IMDB, Sentiment140, Autoencoder-based