New Algorithm Uses LLMs to Improve Cross-Domain Recommendations

Published in the journal Data Science and Engineering, the study presents a method that aims to better understand user preferences by separating them from specific, potentially redundant item features. The team, led by Yuxi Zhang and Ji Zhang, proposes that existing methods often fail to generalize across different domains because they rely too heavily on specific item attributes rather than broader user interests. This new approach seeks to create a universal framework that can adapt to various recommendation scenarios, from dual-domain to multi-domain environments, without requiring extensive pre-training.

The core of the proposed method is the Preference Prototype-Aware (PPA) learning framework, which introduces a mix-encoder and a prototype-aware decoder. The mix-encoder is designed to learn general representations of items that users have interacted with, capturing the intrinsic relationships between items across different domains. By using learnable guidance based on item scores, the mix-encoder prioritizes relevant and informative aspects of user-item interactions. This allows the model to move beyond simple multiplication of user and item embeddings, instead generating a more nuanced representation that reflects the true nature of user preferences.

The prototype-aware decoder addresses the issue of disturbances caused by item features from the source domain. Instead of directly using item features to represent preferences, the decoder employs a learnable prototype matching mechanism. It partitions positive and negative preferences into adaptable prototypes, allowing the model to quantitatively perceive user preferences. This approach helps to filter out noise and redundant features, ensuring that the recommendation system focuses on the essential aspects of user interests. The use of prototypes enables the model to maintain a stable and interpretable representation of preferences, which is crucial for accurate recommendations in diverse domains.

To further improve the performance of the PPA framework, the researchers developed PPA++ by incorporating two additional components: a LoRA-based extractor and a domain cross-attention module. The LoRA-based extractor utilizes a frozen BERT model with Low-Rank Adaptation to extract more reliable semantic representations from item words. This technique reduces resource consumption while maintaining high accuracy, allowing the model to better handle scenarios with sparse item categories. By leveraging the pre-trained knowledge of large language models, the extractor can capture the original features of words, thereby enhancing the semantic understanding of items.

The domain cross-attention module is designed to address the imbalance in weights caused by different user-item distributions. In many recommendation scenarios, there is a significant disparity in the number of users and items, which can lead to unstable multiplication operations in traditional models. The cross-attention mechanism dynamically calibrates the attention entropy between user-space and item-space embeddings, rebalancing the weight distribution. This adaptive approach ensures that the model can effectively handle data imbalance, improving its generalizability and performance in complex environments where user and item distributions vary significantly.

The study evaluated the effectiveness of PPA and PPA++ across four different cross-domain recommendation scenarios using the Amazon benchmark. These scenarios included dual-intra-domain, dual-inter-domain, multi-intra-domain, and multi-inter-domain recommendations. The datasets used were preprocessed by previous studies, ensuring a fair comparison with existing state-of-the-art methods. The evaluation metrics included Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG), which are standard measures for assessing recommendation performance. The results demonstrated that PPA++ consistently outperformed other methods in all four scenarios, highlighting its robustness and versatility.

PPA++ further improved upon PPA by an average of 0.92%, indicating the added value of the LLM-based extractor and cross-attention module. In the dual-inter-domain scenario, PPA exceeded the state-of-the-art method by 2.57% and 1.3% in HR and NDCG on the Game&Video dataset. PPA++ continued to show superior performance, outperforming PPA by an average margin of 0.52%. These results underscore the importance of preference quantification in simple scenarios, where the disturbance from source-domain features can significantly impact recommendation accuracy.

For the multi-domain scenarios, the study found that PPA++ surpassed PPA by an average of 0.724% in the item-overlapped multi-intra-domain scenario. Although PPA++ achieved the highest NDCG, it ranked second in most HR metrics, with the M3rec method obtaining higher HR due to its specialized design for multi-target domains. In the user-overlapped scenario, PPA outperformed other models in both HR and NDCG, with PPA++ further improving performance by an average of 1.02%. These findings suggest that PPA and PPA++ are effective universal methods that can handle both dual-domain and multi-domain tasks, demonstrating their adaptability and generalization capabilities.

Ablation studies were conducted to assess the contribution of individual components in PPA and PPA++. Removing the prototype-aware decoder from PPA led to the lowest performance, confirming the importance of the prototype matching mechanism in filtering out redundant item features. Replacing the mix-encoder with a simple dot product also resulted in performance degradation, highlighting the benefit of the mix-network in capturing prior differences between domains and categories. In PPA++, the addition of the domain cross-attention module and the LoRA-based extractor both contributed to improved performance, with the extractor showing particular efficacy in scenarios with a small number of items.

Sensitivity analysis of hyperparameters revealed that the loss factor λ_pl and the number of prototypes K significantly impact model performance. The study found that setting λ_pl to around 1.0 and K to 4 provided a balance between capturing diverse user preferences and maintaining a manageable prototype space. The LoRA rank r was also analyzed, with r=8 offering the best trade-off between effectiveness and efficiency. Higher ranks led to diminishing returns and increased computational costs, while lower ranks resulted in insufficient semantic extraction. These insights provide practical guidance for tuning the model in different recommendation settings.

The visualization analysis of the learned prototypes using t-SNE showed that positive and negative prototypes maintained their distance and exhibited similar trends across different domains. This indicates that the proposed method can successfully learn distinct sparse preferences for different domain classes, effectively extracting user potential preferences from irrelevant source-domain features. The ability to learn sparse representations allows PPA to focus on the most informative and discriminative features, reducing the impact of redundant or noisy information. This generalizable module enables the model to handle more multi-domain problems, improving its performance in various scenarios.

The study concludes that PPA and PPA++ offer a unified end-to-end training framework that demonstrates the reliability of user preferences represented by prototypes across different scenarios. By leveraging large language models and domain cross-attention, PPA++ achieves better performance than PPA and outperforms existing universal cross-domain recommendation methods. The researchers suggest that future work could explore making the number of prototypes a learnable parameter and investigating the interpretability of the learned preferences. This would provide users with insights into their own preferences and the factors influencing them, further enhancing the utility of recommendation systems.

Beyond the primary recommendation objective, the optimization framework relies on specialized constraints and an auxiliary task to structure the learned preference space. The prototype learning loss incorporates a classification cross-entropy loss alongside three geometric regularizers: a cluster cost, a separation cost, and a diversity loss. The cluster cost pulls interacted items from the source domain closer to the prototype that reflects their preference category, whereas the separation cost forces encoded features away from non-corresponding prototypes. Simultaneously, the diversity loss penalizes prototypes that lie too close to one another based on a cosine similarity threshold, ensuring distinct representations across preference types. To further regularize the latent space between positive and negative interactions, the architecture integrates a contrastive learning auxiliary task that compares representations of interacted items against randomly sampled uninterested items.

To handle settings where the total number of items is extremely large, the authors addressed the computational complexity of the domain cross-attention mechanism, which otherwise requires operations proportional to the product of users, items, and dimensions. To mitigate unacceptable computational overhead, the framework adopts a top-k sparse approximation strategy that first retrieves candidate items using approximate nearest neighbor search before computing attention over those candidates. Testing demonstrated that retrieving one thousand candidates reduced memory usage to one-hundredth of the original footprint, while the resulting decline in hit ratio and normalized discounted cumulative gain remained under 0.3 percent.

The models were evaluated under standardized implementation conditions to maintain rigorous comparison across single-domain, intra-domain, inter-domain, and universal baselines. Training ran for one hundred epochs with a learning rate of 0.001, a batch size of 1024, and a hidden dimension of 128. Within the objective formulation, the prototype constraint weights for clustering, separation, and diversity were set to 0.1, 0.05, and 0.001, respectively. Evaluation involved ranking one positive test item against 999 negative items, measuring top-ten performance across both hit ratio and normalized discounted cumulative gain.

Subject of Research: Development of PPA++, a preference prototype-aware learning method using large language models for universal cross-domain recommendation.

Article Title: PPA++: Preference Prototype-Aware Learning with Large Language Model for Universal Cross-Domain Recommendation

Article References: Zhang, Y., Zhang, J., Xu, F., Chen, L., Li, B., Wang, N., Tu, H., Guo, L., & Yin, H. (2026). PPA++: Preference Prototype-Aware Learning with Large Language Model for Universal Cross-Domain Recommendation. Data Science and Engineering, 11(1), 213-229. https://doi.org/10.1007/s41019-025-00322-w

Image Credits: AI Generated

DOI: 10.1007/s41019-025-00322-w

Keywords: cross-domain recommendation, large language models, preference learning, prototype learning, LoRA, domain cross-attention, recommendation systems, user preferences, data science, machine learning, Preference, Prototype-Aware