DeBERTa-v3 and Aspect Attention Decode Mixed Sentiment in Amazon Reviews

A single Amazon review can praise a product, condemn its packaging, and grumble about the setup manual all at once, yet the star rating at the top collapses that tangle of opinions into one number. A new open-access study in Discover Analytics tackles this problem with a dual-track framework that reads USA-based Amazon customer reviews at two levels simultaneously: a deep learning pipeline that assigns sentiment separately to three aspects of the customer experience, and a structured machine learning pipeline that predicts overall satisfaction from review metadata rather than text. The work, published on 13 July 2026, reports that its best text model reaches 0.737 accuracy and 0.757 macro-F1 on a held-out test set, while a gradient-boosting model on structured features reaches 82.37 percent accuracy.

The text-driven track centers on DeBERTa-v3, a transformer encoder that produces contextual embeddings for every token in a review. The researchers concatenate the review headline and body into one input, tokenize it with the model’s SentencePiece tokenizer at a maximum length of 256 tokens, and pass it through the encoder with no layers frozen, so all parameters adapt end-to-end to the informal, noisy language of marketplace reviews. On top of these embeddings sits a bidirectional recurrent layer that refines the sequence, and then an aspect-attention module that learns a separate attention distribution over token positions for each of three aspect categories: Product & Value, Delivery & Fulfillment, and Service & Usability.

The attention design is the conceptual heart of the paper. Because each aspect has its own trainable attention parameters, the same review can be read three different ways: a clause about a crushed box attracts weight in the Delivery & Fulfillment channel without dominating the Product & Value channel, and complaints about a confusing remote register under Service & Usability rather than dragging down a positive product verdict. Each aspect-specific representation then feeds an independent classification head producing a three-way label of Negative, Neutral, or Positive. The authors argue that placing attention after the recurrent refinement stage, rather than directly on transformer outputs, lets the model weight tokens that are both contextually rich and sequentially organized.

To test whether the recurrent refinement matters, the team trained three variants under an otherwise identical template: bidirectional GRU, bidirectional LSTM, and bidirectional RNN. The Bi-GRU configuration won on every aggregate measure, with precision 0.773, recall 0.734, F1 0.757, and accuracy 0.737. Bi-LSTM followed with accuracy 0.714 and macro-F1 0.722, and Bi-RNN trailed at accuracy 0.707 with macro-F1 0.726. The authors report that pairwise statistical comparisons favor Bi-GRU with p-values at or below 0.002 and Cohen’s d above 2.8, and they attribute the GRU’s edge to a favorable precision-recall balance across aspects rather than dominance on any single class.

An ablation study isolates the contribution of each component. An encoder-only baseline using CLS pooling and linear heads reached 0.697 accuracy and 0.715 macro-F1. Adding Bi-GRU refinement lifted accuracy to 0.715, inserting shared self-attention raised it to 0.725, and replacing shared attention with aspect-specific additive attention produced the full model’s 0.737 accuracy, 0.757 macro-F1, and 0.760 weighted-F1. The stepwise gains, including a 0.012 accuracy improvement over generic attention, support the paper’s central architectural claim that aspect-conditioned attention reduces cross-aspect interference when several experience dimensions share one review.

The confusion matrices reveal where the model excels and where it struggles. For Product & Value it correctly identified 644 Negative, 511 Neutral, and 2318 Positive instances; for Delivery & Fulfillment, 607 Negative, 819 Neutral, and 1455 Positive, with Neutral recall strongest at 819 of 900; and for Service & Usability, 382 Negative, 983 Neutral, and 2241 Positive. Positive sentiment proved easiest to detect, particularly where evaluative language is explicit, while Neutral dominated Delivery & Fulfillment, where many statements are factual or event-based rather than polarized.

Error analysis uncovered a recurring weakness the authors call neutral drift: borderline reviews with clear intent get absorbed into the Neutral class.

A review reading “Paid way too much for the quality… sounds like a low-end item” is a true negative that the model labeled neutral, and the authors trace similar failures across all three aspects. “Works as expected… seems reliable,” a true positive, was also absorbed into Neutral, as was “Box was crushed in mailing; looked cheap” under Delivery & Fulfillment and “Great sound quality… but remote setup was not easy” under Service & Usability. The pattern suggests the model under-weights strong complaints phrased as subjective assessments and understated praise phrased as minimal confirmation, both of which resemble neutral informational language. The authors attribute this neutral drift to hedged wording, implicit sentiment, and multi-clause constructions that blend evaluative and descriptive content, and they note it is especially acute in short texts where Neutral often serves as a catch-all for mixed or weak polarity.

Aspect-level sentiment labels were produced through a manual, guideline-based annotation process involving three trained annotators who reached a Cohen’s kappa agreement of 0.86, a level the authors characterize as strong inter-annotator reliability. The protocol included a pilot phase, consensus-based conflict resolution, and iterative quality control. The data were divided into 70 percent training, 15 percent validation, and 15 percent held-out test sets, with all preprocessing transformations learned exclusively from the training partition to prevent leakage. Across all three aspect categories, positive sentiment dominates the class distribution, which the authors note reflects real-world review tendencies while leaving negative and neutral signals critical for analysis.

The structured track operates on an entirely different label space and feature set. Rather than three aspects, it predicts satisfaction on five levels mapped directly from the star rating: Very Unsatisfied through Very Satisfied, one star through five. Its predictors are platform-native metadata rather than text: verified purchase status, product category, and helpfulness signals derived from helpful votes and total votes. Categorical variables are one-hot encoded, numeric signals are standardized, and a quantile transformation stabilizes the heavy-tailed distributions typical of vote counts. Missing numeric helpfulness fields are set to zero and missing categorical values map to a dedicated unknown category, so no records are discarded over metadata gaps. Under this pipeline, LightGBM led the gradient-boosting comparison with 82.37 percent accuracy and precision, recall, and F1-score each at 0.82. XGBoost followed at 77.48 percent accuracy with an F1 of 0.75, and CatBoost reached 76.83 percent accuracy with an F1 of 0.77, precision 0.74, and recall 0.78. The authors report statistically significant accuracy gains for LightGBM, up to 0.0554, with very large effect sizes.

A deliberate separation underpins this design. Aspect sentiment predictions from the deep model are not fed into the structured models as features, and review text is reserved for the deep learning pipeline rather than injected into the structured track. The authors frame the two outputs as complementary rather than cascaded: satisfaction prediction offers a star-aligned global indicator suitable for longitudinal monitoring and segmentation by category or verification status, while aspect sentiment supplies diagnostic attribution that can be routed to operational owners, with delivery negatives directed to logistics teams, service and usability negatives to support and UX staff, and product and value negatives to quality owners.

Training details for the best text model are documented for reproducibility. The encoder is DeBERTa-v3-base with its associated SentencePiece tokenizer and a 256-token maximum input length. Optimization used AdamW with a learning rate of 2 × 10−5, batch size 16, weight decay 0.01, and a maximum of 10 epochs, with early stopping at a patience of two epochs based on validation cross-entropy loss. No warm-up schedule or learning-rate scheduler was applied, and the training objective sums three categorical cross-entropy losses, one per aspect head. Notably, the authors state that no class rebalancing techniques such as reweighting or resampling were reported or claimed, and no imbalance mitigation was applied in the satisfaction stream either.

The authors are careful to bound their comparative claims. A positioning table places the framework alongside literature-reported aspect-based sentiment analysis methods, including domain-adapted language models, BERT-based classifiers, sentence-pair models, instruction-tuned systems, and prompt-based LLM approaches, but they explicitly caution that those numbers come from different datasets, aspect definitions, domains, and splits and should not be read as direct numerical comparisons. The directly comparable evidence is limited to the variants reproduced on the same held-out Amazon test set, where the Bi-GRU configuration performed best.

The limitations section is candid about external validity. Because the evidence is conditioned on Amazon’s platform-specific review ecosystem, its product mix, moderation practices, regional dynamics, and logistics policies, learned patterns may not transfer to other marketplaces or remain stable over time, and performance may vary by product category, marketplace country, and time window. Star ratings are an imperfect proxy for satisfaction, annotation of short hedged reviews is inherently ambiguous, and evaluation itself faces leakage risk from near-duplicate text and overlapping product or customer identifiers unless group-aware splitting and deduplication audits are enforced. The authors also treat attention patterns and SHAP attributions as decision support rather than causality, noting such explanations can be unstable or confounded, and they flag open questions around latency, calibration near Neutral and adjacent star levels, drift monitoring, and privacy constraints in production use.

Subject of Research: Multi-aspect sentiment classification of Amazon customer reviews using DeBERTa-v3 encoders with aspect attention and structured-feature satisfaction prediction

Article Title: Multi-aspect sentiment classification on USA-based Amazon customer reviews with DeBERTa-v3 encoders and aspect attention

Article References: Moushi, S. S., Bhuiyan, M. S., Akter, N., Chakraborty, U., Akther, S., Islam, R., Shahiduzzaman, M., Kabtia, M., & Keneni, R. (2026). Multi-aspect sentiment classification on USA-based Amazon customer reviews with DeBERTa-v3 encoders and aspect attention. Discover Analytics, 4(1), Article 31. https://doi.org/10.1007/s44257-026-00077-2

Image Credits: AI Generated

DOI: 10.1007/s44257-026-00077-2

Keywords: aspect-based sentiment analysis, DeBERTa-v3, Amazon reviews, attention mechanism, Bi-GRU, LightGBM, natural language processing, e-commerce analytics, customer satisfaction prediction, machine learning, transformer encoders, sentiment classification