New Loss Function Helps AI Learn From Flawed Labels

Deep learning has conquered… (8-10 paragraphs, 1400-1600 words total)

The practical significance of the RECE approach becomes clearer when viewed against the broader economics of dataset construction. Annotating large image collections is a labor-intensive process that often involves multiple annotators working independently, and even carefully managed projects accumulate labeling disagreements that manifest as noise once a single ground-truth label is imposed. In domains such as medical imaging, where the endoscopy benchmark in this study sits, expert annotation time is scarce and expensive, so the proportion of imperfect labels can be substantial. A loss function that tolerates a moderate fraction of corrupted labels without collapsing therefore has direct value for teams that cannot afford exhaustive re-verification of every annotation.

The distinction between symmetric and asymmetric noise, which the experimental design explicitly separates, reflects two very different real-world failure modes. Symmetric noise flips a label to any other class with equal probability, which approximates random annotation errors such as accidental clicks in a labeling interface. Asymmetric noise, by contrast, systematically confuses certain class pairs, mirroring the way human annotators genuinely mistake visually similar categories, for example confusing one endoscopic finding with another that shares its appearance. Asymmetric noise is widely regarded as the harder case because the errors are correlated with the input features themselves, so a model cannot easily separate signal from corruption. The observation that the optimal reward weight depends on both the noise type and the noise rate suggests that practitioners deploying this method should treat the scaling parameter as a tunable diagnostic rather than a fixed constant, adjusting it based on what is known or suspected about the annotation process that produced the training data.

The reward terms at the heart of the method draw on properties that deep networks are known to exhibit early in training. Empirical studies of the learning dynamics of neural networks trained on corrupted labels have repeatedly shown that networks first fit the patterns that generalize before beginning to memorize label errors, a phenomenon sometimes described as the early-learning phase. During this window, predictions on correctly labeled samples tend to be confident and consistent, while predictions on mislabeled samples are often less coherent. Confidence, similarity, and diversity measured across training batches can exploit this asymmetry: samples whose predictions align with the emerging consensus of the batch receive relatively more weight, while samples that conflict with it are down-weighted, which is precisely the behavior needed to reduce the influence of erroneous labels without discarding the informative ones.

Computing these rewards on batches rather than on individual examples is a design choice with practical consequences. Batch-level statistics act as a form of implicit ensemble, since the aggregated behavior of many samples provides a more stable reference than any single prediction. This also means the method integrates naturally into standard mini-batch training pipelines without requiring a second model, a separate noise-estimation stage, or a preliminary filtering pass over the dataset. Approaches that rely on two networks trained in parallel, on small-loss sample selection, or on iterative label correction each add engineering complexity and computational cost; a modified loss function preserves the simplicity of the standard training loop, which matters for teams scaling training to larger datasets or tighter compute budgets.

The baselines chosen for comparison span a meaningful slice of the noisy-label loss literature. Mean absolute error is theoretically noise-tolerant under certain conditions but is notorious for slow convergence and poor performance on hard datasets, because its gradients do not grow with the size of the error. Generalized cross entropy interpolates between cross entropy and mean absolute error through a single parameter, trading off the fast convergence of the former against the noise tolerance of the latter. Normalized cross entropy rescales the loss to counteract the tendency of cross entropy to keep pushing already-confident predictions. By outperforming both the plain cross entropy baseline and these established robust alternatives, particularly at moderate-to-high noise rates, the reported results place RECE within a competitive landscape rather than against strawmen, though the comparison remains confined to parameter-light loss modifications rather than the heavier algorithmic machinery of co-teaching, mentor networks, or label-correction pipelines.

The choice of benchmarks also deserves attention. MNIST and Fashion-MNIST are small, low-resolution datasets where even simple architectures achieve high accuracy on clean data, so they primarily serve to illustrate behavior under controlled corruption rather than to predict performance on production workloads. CIFAR-10, with its greater intra-class variability, is a more demanding and more representative test. The inclusion of an endoscopy dataset is arguably the most consequential, because it grounds the evaluation in a domain where label noise is not a laboratory artifact but a persistent operational reality, and where the cost of a model that has memorized erroneous annotations shows up in clinical decision support. Robustness results that hold across this heterogeneous set, spanning toy, standard, and domain-specific data, carry more weight than results on any single benchmark.

The unified scaling parameter also connects the method to a recurring theme in the robust-learning literature: the trade-off between fitting the training signal and resisting corruption. When the reward weight is small, the objective behaves close to ordinary cross entropy, retaining its strong discriminative drive but little protection against noise. When it is large, the reward terms dominate, and the model risks underfitting the genuinely informative part of the data. The finding that the best setting shifts with the noise regime is consistent with theoretical analyses showing that no single loss is uniformly noise-tolerant across all noise models; robustness must be purchased with some loss of fitting power, and the exchange rate depends on how much corruption is present. This framing suggests a practical workflow in which a small validation set with verified clean labels is used to tune the parameter, which is a far cheaper requirement than cleaning the entire training set.

It is worth noting what the loss-function approach does and does not address. Label noise in deployed systems is not always homogeneous: some errors are random flips, some are systematic confusions, and some arise from ambiguous instances that even experts would label differently. Loss modification operates at the level of gradient weighting, gently steering optimization away from samples whose labels conflict with the model’s learned structure, but it cannot recover the true label of a corrupted example, nor can it resolve ambiguity in the underlying data. Complementary techniques such as noise-transition matrices, sample filtering, and label correction attack the problem from other angles, and hybrid systems that combine a robust loss with a lightweight correction stage are a natural direction for future work. The evidence here establishes that reward-augmented cross entropy is a strong and stable foundation for such systems rather than a complete solution on its own.

Finally, the emphasis on stability across noise levels addresses a failure mode that practitioners know well: methods that excel at one noise rate sometimes degrade sharply at another, or interact badly with standard hyperparameters such as learning rate schedules and augmentation strength. A loss that degrades gracefully, maintaining solid performance from clean data through moderate and high corruption, reduces the risk of silent failure when the true noise characteristics of a dataset are unknown. Combined with the modest implementation overhead of adding reward terms and one scaling parameter to an existing cross entropy objective, this positions the approach as a pragmatic default for teams training classifiers on imperfectly labeled data, particularly in applied fields where annotation quality cannot be guaranteed in advance.

Subject of Research: …

Article Title: RECE Loss: A Reward-Enhanced Cross Entropy Loss Function for Learning with Noisy Labels

Article References: Nguyen, T. T. H., Nguyen, D. T., Nguyen, T. T., & Vu, H. (2026). RECE Loss: A Reward-Enhanced Cross Entropy Loss Function for Learning with Noisy Labels. Neural Computing and Applications, 38(20), Article 810. https://doi.org/10.1007/s00521-026-12500-6

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12500-6

Keywords: machine learning, noisy labels, loss functions, deep learning, cross entropy, robustness, neural networks, label noise, RECE loss, classification, RECE, Loss