Long-horizon local optimization and regularized knowledge improve personalized federated recommendation

Recommender systems trained across users' own devices promise to keep private interaction histories off central servers, but a persistent tension has limited their usefulness: the more a model is personalized to an individual, the more memory, computation, and communication it tends to demand. A new study published in the Journal of Intelligent Information Systems argues that much of this tension is self-inflicted by the way federated recommenders synchronize, and that a simple change of perspective—treating the length of local training as the primary lever of personalization—can deliver better accuracy with a lighter footprint.

The work, by Jaehyung Lim, Wonbin Kweon, Woojoo Kim, Junyoung Kim, Dongha Kim, and Hwanjo Yu, introduces FedRKG, short for Federated Recommendation with Regularized Knowledge Guidance. The framework is model-agnostic, meaning it can wrap around existing federated recommendation architectures rather than requiring a new model design. Its central claim, backed by both empirical analysis and a formal argument, is that user-specific item representations can form naturally when clients are allowed to train locally for long stretches of time, and that global knowledge should then be injected softly and intermittently rather than written over the local model.

Federated recommendation, or FedRec, emerged from federated learning, the training paradigm introduced by McMahan and colleagues in which a server coordinates models trained on distributed devices without collecting raw data. Privacy regulations such as the GDPR and CCPA have made this approach attractive for services that rely on sensitive behavioral records. But recommendation differs from generic federated tasks in an important way: the same item can mean different things to different users, and each client's interaction history is heavily non-IID. A single shared item embedding table, the kind a centralized matrix factorization model learns, forces one representation to reconcile conflicting preference signals across the whole population.

Early FedRec systems simply ported centralized models like matrix factorization and neural collaborative filtering into the federated setting. Later work split into two camps. Single-knowledge methods keep one persistent item embedding per client and try to add personalization through server-side machinery—user-relation graphs, preference-based grouping, item-similarity exchange, or personalized aggregation schemes. Dual-knowledge methods instead maintain separate local and global item representations on each device, doubling the item-embedding parameters from md to 2md, where m is the number of items and d the embedding dimension, and incurring further overhead when the split is implemented at the module level, as in variational autoencoder architectures. The authors note that both camps carry costs: server-side relational signals can indirectly reveal how clients or items are related, while persistent dual representations burden resource-constrained devices.

What neither camp examined closely, the team argues, is the local optimization horizon itself—how many rounds a client trains before the server intervenes. In most prior FedRec studies, synchronization happens after only one, two, or ten local rounds. The researchers' motivational experiments, run on the Amazon-Video and LastFM-2K benchmarks with a federated matrix factorization backbone, show why this matters. As the synchronization interval grows, hit rate at rank ten improves during local training, indicating that client-side item embeddings gradually capture user-specific preference structures. But abrupt performance drops appear around synchronization rounds: replacing locally optimized embeddings with the server's global model destroys the personalization that has accumulated.

Visualization made the point concrete. Using t-SNE projections of item embeddings for individual LastFM-2K users, the authors compared frequent synchronization with an interval of one round against a longer horizon of fifty rounds. Under frequent synchronization, a user's interacted items were scattered and hard to distinguish from non-interacted ones; with the longer horizon, interacted items formed compact, separable clusters. Personalized structure, in other words, is something local training can build on its own—no server-side graph construction required.

A second set of experiments compared four synchronization strategies: full replacement with frequent aggregation, full replacement with intermittent aggregation, purely local training, and knowledge guidance. Full replacement performed poorly under both schedules, confirming that overwriting local embeddings is destructive. Local-only training preserved personalization but eventually plateaued, since it never benefits from shared collaborative trends. Knowledge guidance—softly pulling local embeddings toward a common global reference—achieved the best final performance and showed step-wise gains at each guidance round. A third analysis swept the guidance interval from one to two hundred rounds and found a sweet spot: performance improved up to moderate intervals around one hundred rounds, but very long intervals hurt, as clients over-specialized to local interactions while receiving stale global information.

These observations shaped FedRKG's design. In ordinary rounds, each client trains its personalized item embedding matrix, a private user embedding, and, if needed, a user-specific scoring function, minimizing a binary cross-entropy loss over positive interactions and sampled negatives. The user embedding and scoring function never leave the device; only item embeddings are exchanged. At every T_int rounds—more than fifty in practice, and one hundred in the main experiments—the server aggregates client item embeddings into a common global matrix using plain FedAvg-style averaging, with no client-specific personalization on the server side.

The synchronization itself is cast as a regularized optimization problem. Rather than replacing the local embedding, the client solves for a new embedding that minimizes a weighted sum of two discrepancy terms: distance from its own personalized state and distance from the global reference, balanced by a coefficient lambda. The global embedding thus acts as a soft anchor rather than a reset. The authors instantiate the discrepancy as squared Frobenius distance, which makes the solution a convex combination of the local and global matrices.

Because a single global anchor may not suit every user–item interaction equally, FedRKG adds Adaptive Guidance. A lightweight per-client gating network takes the local item embedding, the global embedding, and their difference, and outputs a gate value for each item. The gate scales the global anchor item by item: a neutral gate of 0.5 recovers the original global vector, while smaller or larger values attenuate or amplify it. The gate parameters are trained through truncated bi-level optimization. At a guidance round, the recommender's states are frozen via stop-gradient copies, and only the gate is optimized for a few inner steps to minimize the recommendation loss that would result after applying the guidance operator. The optimized gate is then used once to update the actual, unfrozen local embedding, which seeds the next long-horizon training phase. The temporary guided embedding used in the inner loop is never stored as the client's representation.

The paper also offers a theoretical rationale. Under mild assumptions—a nonnegative discrepancy measure and a local objective that is L-smooth and near-stationary after long local training—the authors prove that the regularized guidance target is no farther from the personalized local state than full replacement is, and no farther from the global reference than leaving the local state unchanged. A corollary quantifies the shock: with squared Frobenius discrepancy, replacement perturbs the local gradient by the full distance between local and global states, while guidance perturbs it by only a fraction lambda over one plus lambda of that distance. This formalizes why abrupt degradation follows naive synchronization even after long local training, and why guidance behaves as a bounded correction.

Evaluation covered four widely used benchmarks—Amazon-Video, FilmTrust, LastFM-2K, and MovieLens-1M—preprocessed with the same filtering thresholds as prior work (at least twenty interactions per user for ML-1M, ten for the others) and converted to implicit feedback. Interactions were split chronologically, with the most recent item for testing, the second most recent for validation, and the rest for training. Notably, the team ranked each test item against the entire set of unobserved items rather than the 99 sampled negatives used in much prior work, a stricter protocol that avoids inflating scores against an easy candidate pool. All methods shared an embedding dimension of 32, a batch size of 256, SGD updates, four sampled negatives per positive, and early stopping with patience of one hundred epochs. Results were averaged over five runs.

FedRKG outperformed centralized MF and NeuMF, a gap the authors attribute not to federated training being inherently better but to personalization capacity: client-specific item tables let the same item carry different meanings for different users. It also beat most single-knowledge baselines, including relation-graph, grouping, and similarity-exchange methods, and outperformed dual-knowledge models such as FedRAP, FedIAR, and the VAE-based FedAe in most cases—despite never persistently storing separate local and global embeddings. Ablations on Amazon-Video and LastFM-2K showed that removing long-horizon local optimization caused the largest degradation, with normalized discounted cumulative gain at rank five collapsing from 14.85 to under one when guidance operated without well-formed local structures. Setting lambda to infinity—full replacement—performed poorly, while lambda of zero, local-only training, was better but still inferior to the full framework. Dropping adaptive guidance left the model competitive but consistently worse.

Efficiency analysis reinforces the practical appeal. In typical rounds, each client stores (m+4)d+1 parameters—essentially one item embedding table plus small private vectors—roughly half the persistent footprint of dual-knowledge methods and far below FedCIA, which exchanges item-similarity matrices with quadratic O(m²) storage. Peak storage during a guidance round rises to (2m+4)d+1, still linear in items. Communication cost falls by a factor of T_int compared with methods that exchange embeddings every round, since transfers occur only once per guidance interval. Under partial participation, the authors show FedRKG is more communication-efficient than per-round baselines unless the client sampling ratio falls below roughly one over T_int—under two percent for typical intervals, an uncommon operating point.

Further analyses probed the guidance mechanism's behavior. Splitting Amazon-Video users into "followers," who interact heavily with popular items, and "distinct" users with idiosyncratic tastes, the authors measured performance changes immediately after guidance rounds. Distinct users sometimes dipped slightly under plain knowledge guidance, while followers consistently improved; adaptive guidance shrank the dips for distinct users and amplified gains for followers, and a larger fraction of users improved overall. The guidance paradigm also proved model-agnostic in practice, consistently improving three different backbones—FedMF, FedNCF, and PFedRec—when integrated into them, supporting the claim that the mechanism is not tied to any particular architecture.

The authors address residual privacy concerns directly. Although raw interactions never leave devices, transmitted item-embedding updates can still encode preference information; a curious server that knows the previous embedding, the global reference, and lambda could reconstruct a client's accumulated local update. To counter this, they show FedRKG can be combined with local differential privacy: updates are clipped to a bounded norm and perturbed with Gaussian noise calibrated to the sensitivity and a target (epsilon, delta) budget, and the server aggregates only the privatized updates, with aggregation acting as post-processing that preserves the guarantee. Empirical results confirm utility and convergence under this mechanism.