DistilBERT Outperforms Perceiver in the Hunt for Fake Hotel Reviews

Fake hotel reviews have become one of the quiet epidemics of the digital travel age. Before booking a room, more than ninety percent of travelers consult platforms such as TripAdvisor, Booking.com, or Google Reviews, and a single wave of fabricated praise or coordinated sabotage can distort a hotel’s reputation, mislead consumers, and generate unjustified competitive advantages. A new study from the University of Oradea in Romania, published in Frontiers in Big Data, puts two prominent artificial intelligence architectures head to head to see which one can reliably separate genuine guest feedback from deliberate deception, and the results carry a sobering lesson about how easily impressive-looking metrics can hide a model that has learned nothing at all.

The research team, led by Ioana Ximena Remeş together with Felicia Mirabela Costea and Cornelia Aurora Gyorödi, compared two fundamentally different deep learning pipelines on the same task: binary classification of hotel reviews as genuine or deceptive. The first contender was DistilBERT, a compressed version of the famous BERT Transformer model created through knowledge distillation, in which a smaller student network is trained to mimic the output probabilities of a larger teacher. DistilBERT retains roughly forty percent fewer parameters than BERT-base while preserving most of its language understanding, making it attractive for real-world deployment where computing budgets are tight. Crucially, the researchers fine-tuned DistilBERT end to end, meaning the model’s internal word representations were actively adapted to the deception-detection task during training.

The second contender was the Perceiver, an architecture built around a fixed-size latent array that processes inputs through cross-attention rather than conventional self-attention over tokens. The Perceiver was designed to handle text, images, audio, and structured data within a single framework, projecting any input into a compact latent space where subsequent processing occurs. In this study, however, the Perceiver did not read raw review text. Instead, each review was first converted into a single 384-dimensional vector by a frozen, pre-trained sentence encoder, to which the numerical star rating and a binary sentiment indicator were appended, producing a 386-dimensional input. The text encoder was never adapted to the task, and the Perceiver operated on one pooled embedding rather than a sequence of words, a design asymmetry that would prove decisive.

The experimental corpus consisted of 3,322 English-language hotel reviews assembled from two sources: the publicly available Myleott benchmark, containing roughly 800 truthful and 800 deceptive reviews for twenty Chicago hotels, and a Kaggle collection of approximately 1,900 verified genuine reviews of the Ritz-Carlton New York. After language filtering, deduplication, and removal of very short reviews, the merged corpus held about 2,520 genuine and 800 deceptive reviews, an imbalance in which roughly 76 percent of samples belonged to the truthful majority class. The data were split into stratified 80/20 train-test partitions, yielding 2,657 training and 665 test reviews per split, and every evaluation was repeated across three independent random seeds to measure stability rather than relying on a single lucky partition.

The headline numbers favored DistilBERT decisively. Averaged across the three splits, the full DistilBERT pipeline, which concatenated the pooled textual representation with the rating and sentiment features before its classification head, achieved an accuracy of 0.9348 with a standard deviation of only 0.0014, an F1-score of 0.9569, and a Matthews correlation coefficient of 0.8240. The Perceiver, by contrast, plateaued at an accuracy of 0.7594 from the earliest epochs onward, with its metrics essentially flat across all ten training epochs. On the surface, its recall of exactly 1.0000 looked spectacular, suggesting it never missed a single genuine review. But the confusion matrices told a very different story: the model classified every single test instance as genuine, correctly identifying all 505 truthful reviews while catching precisely zero of the 160 deceptive ones.

This is the study’s most instructive finding, a textbook demonstration of majority-class collapse. When a dataset is skewed, a classifier can score respectable accuracy and perfect recall simply by always predicting the most common label, learning nothing about the actual distinction between classes. The imbalance-aware metrics exposed this immediately. The Perceiver’s balanced accuracy was exactly 0.5000, the value expected from random guessing between classes, and its Matthews correlation coefficient was effectively zero, indicating no correlation whatsoever between its predictions and the truth. Its specificity, the ability to recognize the minority deceptive class, was 0.0000. Notably, the model’s ROC-AUC of about 0.788 showed that the frozen embeddings still contained some ranking information capable of separating the classes; the learned decision threshold simply never exploited it.

The researchers then tested whether a standard imbalance correction could rescue the Perceiver. Retraining with a class-weighted binary cross-entropy loss, weighting the minority class by a factor of approximately 3.15, did lift the model out of degeneracy: specificity rose to 0.6771 and the MCC to 0.3748, meaning the weighted model now identified a majority of deceptive reviews. Yet the correction was incomplete and unstable, with accuracy dropping slightly, calibration worsening, and the standard deviation of specificity approaching 0.12 across splits, indicating erratic behavior from one partition to the next. The weighted Perceiver remained far below both DistilBERT and even the classical baselines, supporting the interpretation that the frozen sentence embedding, rather than the imbalance alone, fundamentally constrained what this pipeline could learn.

Perhaps the most humbling result came from the traditional baselines. A simple pipeline combining TF-IDF word weighting with logistic regression, trained directly on the raw review text, reached an accuracy of 0.8957 without any class weighting, and a class-balanced variant climbed to 0.9258 with an MCC of 0.8137. DistilBERT’s margin over this well-configured lexical model amounted to roughly one percentage point of accuracy and one hundredth of an MCC point. Where the Transformer genuinely pulled ahead was in probability calibration, with a lower log loss of 0.2516 versus 0.3174, and in consistency, its accuracy standard deviation being about a quarter of the balanced baseline’s. On corpora of this size, the study suggests, carefully tuned classical methods remain genuinely competitive, and claims of deep learning superiority deserve scrutiny against strong simple references.

The ablation analysis added a final nuance. Removing the rating and sentiment features from DistilBERT cost only about 1.3 percentage points of accuracy, and the text-only variant actually matched or slightly exceeded the full model on several metrics within the observed variation. The clearest effect of the auxiliary features was stability: the full model’s standard deviations for accuracy and log loss were several times smaller than the text-only configuration’s. In other words, the predictive signal lives almost entirely in the review text itself, while the structured attributes act as a stabilizer that reduces sensitivity to how the data happen to be partitioned, rather than as an independent source of discriminative power.

The authors are careful to frame their conclusions appropriately. The comparison evaluates two complete processing pipelines rather than the intrinsic capabilities of two architectures, and the Perceiver’s failure reflects its specific configuration, operating on frozen embeddings, not proof that the architecture is inherently unsuitable for the task. Future work could feed token sequences directly to the Perceiver, apply statistical significance testing, and validate on larger, multilingual datasets. For platform operators and researchers alike, the practical takeaways are clear: always report corpus composition, class imbalance, and test-set size; always include imbalance-robust metrics such as specificity, balanced accuracy, and the Matthews correlation coefficient alongside accuracy; and never trust a perfect recall score without checking what the model is actually doing. In the fight to keep online reputation systems honest, the fine-tuned DistilBERT pipeline emerged as the most robust option tested, but the study’s deeper contribution is a methodological warning: in imbalanced data, a model that looks like a champion may simply be echoing the majority.

Subject of Research: Comparative evaluation of DistilBERT and Perceiver deep learning pipelines for detecting deceptive hotel reviews

Article Title: A comparative study of DistilBERT and Perceiver for deceptive review detection in hospitality review data

Article References: Remeş, I. X., Costea, F. M., & Gyorödi, C. A. (2026). A comparative study of DistilBERT and Perceiver for deceptive review detection in hospitality review data. Frontiers in Big Data, 9, Article 1918383. https://doi.org/10.3389/fdata.2026.1918383

Image Credits: AI Generated

DOI: 10.3389/fdata.2026.1918383

Keywords: deceptive review detection, DistilBERT, Perceiver, hospitality, natural language processing, transformer models, class imbalance, Matthews correlation coefficient, TF-IDF, logistic regression, hotel reviews, machine learning