Emotion-aware multi-agent reinforcement learning boosts customer service dialogue performance in new study

Researchers have developed a customer service dialogue system that combines multi-agent reinforcement learning with a bio-inspired optimization algorithm to recognize customer emotions and generate adaptive responses. The new framework, called MASAC-GBFIO, was described in an open-access research article published in Discover Computing on 28 October 2026. In experiments on an emotion-annotated dialogue dataset, the system achieved a response time of 1.96 seconds, a BLEU-4 score of 42.45, and a ROUGE-L score of 64.59, outperforming comparison models including ESDP and several GPT-2 variants on the reported metrics.

The motivation behind the work lies in the limitations of conventional dialogue systems. According to the authors, most existing customer service bots are built on rule-based strategies or single-agent reinforcement learning, approaches that struggle with extended conversations, fluctuating emotional states, and coordinated decision-making across multiple interacting components. In real service environments, customers expect responses that are not only informative but also attentive to their emotional condition, whether that condition is frustration, satisfaction, confusion, or urgency. The team argues that emotion perception and collaborative response generation should therefore be treated as core elements of next-generation intelligent customer service platforms rather than optional add-ons.

The proposed architecture distributes the dialogue task across four specialized agents responsible for emotion analysis, policy formulation, knowledge retrieval, and escalation handling. At the heart of the system is Multi-Agent Soft Actor-Critic, or MASAC, an extension of the Soft Actor-Critic reinforcement learning algorithm to settings with several decision-makers. MASAC follows a centralized training and decentralized execution scheme: during training, a centralized twin-critic network evaluates joint state-action pairs so that agents can learn coordinated policies, while during operation each agent acts on its own using shared state information. The Soft Actor-Critic objective jointly optimizes expected reward and policy entropy, which encourages balanced exploration and helps the system adapt to varying customer emotions and conversational contexts.

The second component, Grizzly Bear Fat Increase Optimization (GBFIO), operates as a meta-optimization layer that tunes critical hyperparameters of the reinforcement learning process, including the learning rate, entropy coefficient, and reward scaling factors. The algorithm borrows its structure from bear foraging behavior and proceeds through three phases. A searching phase performs global exploration of the hyperparameter space to identify candidate configurations; a hunting phase exploits high-performing regions to refine cooperative strategies and improve convergence stability; and a fishing phase carries out fine-grained local refinement while preserving population diversity to prevent premature convergence. The authors selected GBFIO over traditional optimizers such as Particle Swarm Optimization because of its balance between global exploration and local exploitation, which they argue suits the non-stationary conditions of multi-agent reinforcement learning.

Feature extraction relies on Sentence-BERT embeddings, which convert dialogue text into fixed-length sentence representations capturing semantic meaning, contextual relationships, and emotional characteristics. The SBERT model was used as a zero-shot pre-trained embedding model and was not fine-tuned on the target dataset; the resulting embeddings served as direct inputs to the MASAC agents. Before embedding, the dialogue data underwent standard preprocessing: missing values were handled through imputation or deletion, numerical features such as sentiment scores were scaled with min-max normalization to a range of zero to one, and text was tokenized into words or sub-word units so that emotional cues could be processed effectively.

The experiments used the Emotion-Aware Multi-Agent Dialogue Dataset, a publicly available collection of 6,500 conversation records drawn from multiple customer service domains including banking, e-commerce, technical support, billing, and account management. Each record captures user interactions, system responses, context information, and decision-related signals across multi-turn dialogues, along with emotion categories, satisfaction levels, resolution outcomes, escalation decisions, and reward signals. Emotional annotations covered five categories: satisfaction, frustration, confusion, urgency, and neutrality. The data were split with 70 percent used for training and 30 percent for testing, and the system was implemented in Python using deep learning libraries.

The reported results show consistent advantages for the combined framework. In convergence experiments, MASAC-GBFIO reached a final cumulative reward of 0.963, compared with 0.953 for MASAC alone and 0.903 for SAC, with highlighted 95 percent confidence intervals indicating consistent learning behavior. In an ablation study, the full model recorded a response time of 1.96 seconds against 2.05 seconds for MASAC and 2.10 seconds for GBFIO alone, and its BLEU-4 score of 42.45 far exceeded the 30.75 of MASAC and 28.20 of GBFIO, while its ROUGE-L score of 64.59 surpassed 51.32 and 48.40 respectively. Against GPT-2 variants evaluated on a dataset of 10,000 records, the proposed model’s BLEU-4 and ROUGE-L scores exceeded those of GPT2-Small, which scored 20.31 and 35.67, with the larger GPT-2 versions improving but still trailing. Compared with the previously published AETGCN-NGOA-CSS approach, the new model reported an error rate of 0.159 percent and a computation time of 92 seconds, against 0.2 percent and 94 seconds for the earlier method.

Statistical comparisons between MASAC and MASAC-GBFIO supported the differences in error rate, computation time, BLEU-4, and ROUGE-L, with p-values below 0.001 and corresponding confidence intervals reported. When recent intelligent customer service approaches were re-implemented and retrained under the same experimental protocol, preprocessing pipeline, SBERT-based feature representation, and evaluation settings, the proposed framework achieved the highest precision at 95.84 percent, recall at 95.62 percent, and F1-score at 95.73 percent. Qualitative analysis of representative dialogue samples from domains such as healthcare, education, and telecom illustrated how the four agents interact: the emotion agent detects user affect from intensity and sentiment information, the policy agent selects a dialogue strategy, the knowledge agent retrieves relevant information, and the escalation agent decides whether to resolve automatically or transfer to a human expert.

The authors also acknowledge meaningful limitations. The system relies on predefined responses and limited query diversity, which can reduce its applicability in real-world environments with highly dynamic, ambiguous, or previously unseen user interactions, and its generalization to open-domain conversation remains limited. Because SBERT embeddings were not fine-tuned on the target data, domain-specific terminology in areas such as banking or healthcare may not be fully captured, and the authors note that explicit domain-specific fine-tuning was not carried out. The study is also restricted to a single dataset, which the authors say can restrict generalization across diverse real-world scenarios. They suggest that future work should evaluate the model on multiple large-scale, domain-diverse datasets, incorporate continuous learning from real-time customer feedback, and explore more sophisticated natural language understanding for complex query processing.

Subject of Research: Emotion-aware multi-agent reinforcement learning for customer service dialogue systems

Article Title: Reinforcement learning and multi-agent systems-driven optimization of emotional perception and collaborative response mechanisms in intelligent customer service systems

Article References: Li, X., Wu, S., Li, Y., Zhuang, S., Zhang, P., Zhang, Y., Chen, L., Xu, Q., & Feng, Q. (2026). Reinforcement learning and multi-agent systems-driven optimization of emotional perception and collaborative response mechanisms in intelligent customer service systems. Discover Computing, 29(1), Article 816. https://doi.org/10.1007/s10791-026-10552-9

Image Credits: AI Generated

DOI: 10.1007/s10791-026-10552-9

Keywords: reinforcement learning, multi-agent systems, customer service, emotion recognition, dialogue systems, Soft Actor-Critic, bio-inspired optimization, SBERT, natural language processing, BLEU, ROUGE, chatbots