The distinction between symmetric and asymmetric noise, which the experimental design explicitly separates, reflects two very different real-world failure modes. Symmetric noise flips a label to any other class with equal probability, which approximates random annotation errors such as accidental clicks in a labeling interface. Asymmetric noise, by contrast, systematically confuses certain class pairs, mirroring the way human annotators genuinely mistake visually similar categories, for example confusing one endoscopic finding with another that shares its appearance. Asymmetric noise is widely regarded as the harder case because the errors are correlated with the input features themselves, so a model cannot easily separate signal from corruption. The observation that the optimal reward weight depends on both the noise type and the noise rate suggests that practitioners deploying this method should treat the scaling parameter as a tunable diagnostic rather than a fixed constant, adjusting it based on what is known or suspected about the annotation process that produced the training data.

The reward terms at the heart of the method draw on properties that deep networks are known to exhibit early in training. Empirical studies of the learning dynamics of neural networks trained on corrupted labels have repeatedly shown that networks first fit the patterns that generalize before beginning to memorize label errors, a phenomenon sometimes described as the early-learning phase. During this window, predictions on correctly labeled samples tend to be confident and consistent, while predictions on mislabeled samples are often less coherent. Confidence, similarity, and diversity measured across training batches can exploit this asymmetry: samples whose predictions align with the emerging consensus of the batch receive relatively more weight, while samples that conflict with it are down-weighted, which is precisely the behavior needed to reduce the influence of erroneous labels without discarding the informative ones.

Computing these rewards on batches rather than on individual examples is a design choice with practical consequences. Batch-level statistics act as a form of implicit ensemble, since the aggregated behavior of many samples provides a more stable reference than any single prediction. This also means the method integrates naturally into standard mini-batch training pipelines without requiring a second model, a separate noise-estimation stage, or a preliminary filtering pass over the dataset. Approaches that rely on two networks trained in parallel, on small-loss sample selection, or on iterative label correction each add engineering complexity and computational cost; a modified loss function preserves the simplicity of the standard training loop, which matters for teams scaling training to larger datasets or tighter compute budgets.

The baselines chosen for comparison span a meaningful slice of the noisy-label loss literature. Mean absolute error is theoretically noise-tolerant under certain conditions but is notorious for slow convergence and poor performance on hard datasets, because its gradients do not grow with the size of the error. Generalized cross entropy interpolates between cross entropy and mean absolute error through a single parameter, trading off the fast convergence of the former against the noise tolerance of the latter. Normalized cross entropy rescales the loss to counteract the tendency of cross entropy to keep pushing already-confident predictions. By outperforming both the plain cross entropy baseline and these established robust alternatives, particularly at moderate-to-high noise rates, the reported results place RECE within a competitive landscape rather than against strawmen, though the comparison remains confined to parameter-light loss modifications rather than the heavier algorithmic machinery of co-teaching, mentor networks, or label-correction pipelines.

The choice of benchmarks also deserves attention. MNIST and Fashion-MNIST are small, low-resolution datasets where even simple architectures achieve high accuracy on clean data, so they primarily serve to illustrate behavior under controlled corruption rather than to predict performance on production workloads. CIFAR-10, with its greater intra-class variability, is a more demanding and more representative test. The inclusion of an endoscopy dataset is arguably the most consequential, because it grounds the evaluation in a domain where label noise is not a laboratory artifact but a persistent operational reality, and where the cost of a model that has memorized erroneous annotations shows up in clinical decision support. Robustness results that hold across this heterogeneous set, spanning toy, standard, and domain-specific data, carry more weight than results on any single benchmark.

The unified scaling parameter also connects the method to a recurring theme in the robust-learning literature: the trade-off between fitting the training signal and resisting corruption. When the reward weight is small, the objective behaves close to ordinary cross entropy, retaining its strong discriminative drive but little protection against noise. When it is large, the reward terms dominate, and the model risks underfitting the genuinely informative part of the data. The finding that the best setting shifts with the noise regime is consistent with theoretical analyses showing that no single loss is uniformly noise-tolerant across all noise models; robustness must be purchased with some loss of fitting power, and the exchange rate depends on how much corruption is present. This framing suggests a practical workflow in which a small validation set with verified clean labels is used to tune the parameter, which is a far cheaper requirement than cleaning the entire training set.

It is worth noting what the loss-function approach does and does not address. Label noise in deployed systems is not always homogeneous: some errors are random flips, some are systematic confusions, and some arise from ambiguous instances that even experts would label differently. Loss modification operates at the level of gradient weighting, gently steering optimization away from samples whose labels conflict with the model’s learned structure, but it cannot recover the true label of a corrupted example, nor can it resolve ambiguity in the underlying data. Complementary techniques such as noise-transition matrices, sample filtering, and label correction attack the problem from other angles, and hybrid systems that combine a robust loss with a lightweight correction stage are a natural direction for future work. The evidence here establishes that reward-augmented cross entropy is a strong and stable foundation for such systems rather than a complete solution on its own.

Finally, the emphasis on stability across noise levels addresses a failure mode that practitioners know well: methods that excel at one noise rate sometimes degrade sharply at another, or interact badly with standard hyperparameters such as learning rate schedules and augmentation strength. A loss that degrades gracefully, maintaining solid performance from clean data through moderate and high corruption, reduces the risk of silent failure when the true noise characteristics of a dataset are unknown. Combined with the modest implementation overhead of adding reward terms and one scaling parameter to an existing cross entropy objective, this positions the approach as a pragmatic default for teams training classifiers on imperfectly labeled data, particularly in applied fields where annotation quality cannot be guaranteed in advance.

Subject of Research: RECE Loss: A Reward-Enhanced Cross Entropy Loss Function for Learning with Noisy Labels

Article Title: RECE Loss: A Reward-Enhanced Cross Entropy Loss Function for Learning with Noisy Labels

Article References: Nguyen, T. T. H., Nguyen, D. T., Nguyen, T. T., & Vu, H. (2026). RECE Loss: A Reward-Enhanced Cross Entropy Loss Function for Learning with Noisy Labels. Neural Computing and Applications, 38(20), Article 810. https://doi.org/10.1007/s00521-026-12500-6

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12500-6

Keywords: machine learning, noisy labels, loss functions, deep learning, cross entropy, robustness, neural networks, label noise, RECE loss, classification, RECE, Loss

The theoretical core of the paper begins with a definition that reaches back a century. Regularization, the authors argue, is best understood as the solution of ill-posed problems, a concept introduced by the French mathematician Jacques Hadamard in 1923. A problem is well-posed when a solution exists, is unique, and depends continuously on the input data. Learning from finite, noisy datasets fails all three tests at once: many different functions can fit the same training examples, tiny perturbations in the data can swing the fitted model wildly, and nothing guarantees a stable answer. From this vantage point, regularization is not an afterthought bolted onto training but the very mechanism that converts an impossible inverse problem into a tractable one.

To make this concrete, the survey examines the effectiveness of Tikhonov regularization in linear systems, the setting where everything can be worked out exactly. Tikhonov regularization, introduced by the Russian mathematician Andrey Tikhonov and familiar to statisticians as ridge regression through the 1970 work of Hoerl and Kennard, adds a penalty on the squared size of the solution to the fitting objective. Using the singular value decomposition, the authors show how the penalty suppresses the directions in which the data are least informative, damping the tiny singular values that would otherwise amplify noise into enormous errors. The analysis makes precise, in a setting free of neural network mysteries, exactly what a penalty term buys: a controlled trade of bias for stability.

The most consequential theoretical contribution is a consistency principle for the design of regularization methods, proposed to hold in both linear and nonlinear systems. The principle addresses a question that has quietly haunted the field: when a penalty helps in one architecture or task, when can we trust it to help in another? Rather than treating each regularizer as an isolated empirical trick, the survey argues that sound regularization should behave consistently with the underlying structure of the learning problem, so that the guiding principles established rigorously for linear systems carry over, in appropriately adapted form, to the nonlinear models that dominate modern practice. This reframing gives researchers a design criterion, not merely a menu of options.

With that foundation laid, the second half of the paper delivers what may become its most-used artifact: a comprehensive taxonomy of recent loss-based regularizers, the family of methods that modify the training loss itself rather than the architecture, the data pipeline, or the optimization algorithm. The taxonomy summarizes the prominent features of these regularizers, and it draws together strands of work that rarely appear in the same review. Classical sparsity penalties such as the lasso of Tibshirani and its nonconvex successors, including the minimax concave penalty and transformed l1 approaches, sit alongside group and exclusive sparsity formulations, low-rank and nuclear norm methods, and total-variation style penalties inherited from image processing.

The survey also gives careful attention to regularizers designed specifically for the deep learning era, where the classical capacity-control story falters. It covers penalty terms targeting the Lipschitz constant of networks, Jacobian and gradient-norm penalties that smooth the model’s response to its inputs, and sharpness-aware minimization, the influential technique that steers optimization toward wide, flat basins of the loss landscape. Work on flat minima stretching back to Hochreiter and Schmidhuber in 1997, and the finding of Dinh and colleagues that sharp minima can nonetheless generalize, illustrates the kind of unresolved tension the taxonomy is built to expose. Adversarial perturbation regularizers, persistent-homology-based topological penalties, mixup-style data interpolation losses and dropout all find their place in the same organized structure.

What makes this organization more than bibliographic housekeeping is the diagnosis embedded in the abstract: despite the numerous methods developed in a fast-growing literature, there is a lack of theoretical understanding and rigorous analysis of regularization. The survey is candid that the explosion of techniques has outpaced the theory explaining why they work and when they fail. By anchoring every entry in the taxonomy to the ill-posed-problems formulation and the consistency principle, the authors offer researchers a shared vocabulary for comparing methods that were previously justified by unrelated intuitions, from Bayesian priors to geometric smoothness to robustness against adversarial attack.

The paper does not rest on analysis alone. The authors report results of benchmark tests that confirm the effectiveness of representative loss-based regularizers, providing an empirical check that the theoretical framework identifies methods which genuinely deliver generalization gains in practice. The numerical experiments and the data supporting the manuscript are available from the authors upon request, in line with the journal’s transparency expectations. Co-first authors Huang and Zhang divided the labor, with Huang conducting the literature search, summarizing each surveyed method and producing the tables and figures, while Zhang carried out the mathematical proofs. Xu led conceptualization and the taxonomy, and Qinghai Zhang contributed the mathematical reasoning and manuscript writing, with support from the Fundamental Research Funds for the Central Universities.

The timing of the survey, received in 2023, revised in March 2026 and published in October 2026 after more than two and a half years of review and refinement, reflects a field in urgent need of synthesis. Modern models are trained on massive datasets with billions of parameters, and the celebrated generalization of such overparameterized networks famously defies classical bias-variance intuition, a puzzle highlighted by Belkin, Neyshabur, Zhang and their colleagues in influential analyses cited in the survey. Understanding which penalties push a model toward solutions that transfer, and which merely reshape the training landscape, has become a question with enormous economic weight, affecting everything from medical imaging systems to the reliability of language models deployed to hundreds of millions of users.

For the research community, the payoff of this work is likely to be felt in two ways. First, the rigorously analyzed linear case provides a template: students and researchers can now trace, step by step, how a penalty reshapes a solution through the singular spectrum, and carry that intuition into nonlinear settings with a stated principle rather than a vague analogy. Second, the taxonomy turns the design of new regularizers from an art into an engineering discipline with a consistency criterion to satisfy. If the authors are right that regularization should be treated as the principled resolution of ill-posedness, then the next generation of methods will not be judged solely by leaderboard margins on benchmark datasets, but by whether their penalties encode the right structural assumptions about the problem. In a discipline where the gap between memorization and understanding is the central mystery, bringing mathematical order to that gap may prove the most valuable regularization of all.

Subject of Research: Loss-based regularization methods for improving model generalization in machine learning

Article Title: A Survey on Loss-based Regularization for Model Generalization

Article References: Huang, Y., Zhang, C., Xu, R., & Zhang, Q. (2026). A Survey on Loss-based Regularization for Model Generalization. Machine Learning, 115(10), Article 244. https://doi.org/10.1007/s10994-026-07139-2

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07139-2

Keywords: regularization, model generalization, machine learning, Tikhonov regularization, ill-posed problems, loss functions, sparsity, deep learning, consistency principle, empirical risk minimization, taxonomy, benchmarking

The scaffolds at the heart of the study are made of hydroxyapatite, a mineral chemically similar to natural bone, arranged as an orthogonal lattice of rods with pores between 100 and 400 micrometers wide. Implanted into mandibular bone defects in Yucatan minipigs, these structures invite new bone to grow into their macropores, and quantifying how much bone has infiltrated each scaffold is the key to comparing different lattice designs and biological additives. The problem is that bone and hydroxyapatite share nearly identical mineral composition, so their X-ray attenuation values overlap almost completely in micro-CT scans. Ingrown bone also overlaps in brightness with the surrounding soft tissue and noise. Manual labeling of the roughly 1,000 images per scaffold is prohibitively time consuming across studies involving tens of scaffolds, which is exactly where the U-Net convolutional neural network enters the picture.

The U-Net architecture, first introduced in 2015, remains a favorite in biomedical imaging because it delivers strong segmentation from small training sets. Its U-shaped design works like an autoencoder with a memory: an encoding path progressively shrinks the image while doubling its feature channels to detect patterns, a bottleneck connects the two halves, and a decoding path restores the original dimensions, with skip connections threading fine spatial detail from encoder to decoder at every level. The Illinois team used a four-level network starting with eight feature channels and reaching 128 at the bottleneck, ultimately classifying every pixel in a 512-by-512 patch as scaffold, bone, or background. Rather than inventing a new architecture, the researchers held the network constant and varied only the loss function, the mathematical rule that tells the model how wrong its predictions are during training.

Three loss functions were pitted against each other under identical conditions. Cross-entropy loss, the workhorse of classification, evaluates every pixel individually regardless of class balance. Soft Dice loss instead optimizes the overlap between prediction and ground truth, a strategy favored when one class, like bone, occupies only a small fraction of the image. A third, less common option called Accuracy loss directly rewards correct classification of the entire patch. The team trained each model for 1,000 epochs on 225 patches derived from 31 manually labeled images, using four NVIDIA V100 GPUs and confirming that all three losses converged without overfitting. A second expert annotator independently relabeled a subset to gauge human variability, finding a bone Dice score of 0.78 between annotators and an average difference in bone growth fraction of just 1.2 percentage points.

Then came the surprise. Standard evaluation metrics, the numbers researchers typically report to demonstrate that a segmentation model works, produced results that seemed catastrophic but were actually meaningless. Some patches scored a Dice coefficient of zero and a Precision of zero, values usually interpreted as total failure, yet visual inspection showed those patches were segmented perfectly well. The catch: those patches contained no bone in the ground truth at all. When the model predicted a handful of spurious bone pixels in a bone-free image, the Dice and Precision formulas collapsed to zero even though the biological error was negligible. Similar distortions plagued Recall, which tanked on patches where a few missed pixels represented a large fraction of a tiny bone area. Under extreme class imbalance, the familiar metrics were measuring the imbalance, not the model.

To escape this trap, the researchers invented an application-specific endpoint they call delta BGF: simply the difference between the bone growth fraction predicted by the model and the bone growth fraction in the ground truth. This is the number that actually matters for the science, because the entire purpose of segmenting the scans is to quantify bone regeneration and compare treatment groups. Bland-Altman analysis showed the systematic bias between predicted and true bone growth fraction was less than 1.5 percentage points for all three models, comparable to the disagreement between two human experts. When the team correlated the standard metrics against the absolute value of delta BGF, Accuracy emerged as the clear winner, with correlation coefficients reaching negative 0.8 for the Accuracy-loss model and negative 0.78 for the Dice model. A Steiger test confirmed Accuracy’s correlation was significantly stronger than those of Dice, Precision, and Recall in eight of nine comparisons, and bootstrap resampling across 10,000 samples showed the relationship was stable, with confidence intervals lying entirely below zero.

The loss function also shaped where the errors landed, which matters because different regions of the image carry different scientific weight. The Accuracy-loss model tended to overestimate bone, scattering false positives inside the scaffold boundary, precisely where the ingrown bone quantification happens. The cross-entropy and Dice models instead missed bone in the dense ring surrounding the scaffold, errors that are conveniently located outside the region of interest and can be corrected automatically, since scaffold pixels cannot legitimately exist beyond the scaffold boundary. Given this geography of mistakes, the cross-entropy model emerged as the recommended choice for future in vivo studies quantifying spatial variation in bone growth, combining clean class edges within the scaffold with correctable peripheral errors.

Robustness testing added another layer of confidence. The team created augmented test sets by rotating patches 15, 30, and 45 degrees, shifting contrast in both directions, and degrading resolution by factors of 0.66 and 0.5, simulating the natural variation in micro-CT scanning parameters and scaffold orientation. None of these perturbations produced a statistically significant change in Accuracy, Dice, Precision, Recall, or delta BGF. The systematic bias remained below 2.6 percentage points across all augmentations, with contrast changes producing the widest limits of agreement and rotation the narrowest, suggesting that intensity shifts stress the model more than orientation changes. For a field where scanning protocols differ between labs and even between experiments, this resilience means the trained models can plausibly be applied to new datasets without retraining from scratch.

Perhaps the most consequential result is biological rather than computational. Comparing against the group’s earlier in vivo studies, the smallest treatment differences those studies could detect were on the order of 8 to 20 percentage points in bone fraction, vastly larger than the 1.5 percentage point error of the AI segmentation. In other words, the model’s mistakes are too small to obscure genuine biological differences between scaffold designs, meaning future comparisons of bone regeneration should remain statistically sound. The study’s limitations are acknowledged candidly: the 54 test patches derive from only six images, so patch-level statistics treat clustered data as independent, and the authors frame their correlation findings as descriptive rather than confirmatory. Still, the message radiates well beyond bone scaffolds. Whenever deep learning is deployed to extract a quantitative measurement rather than a pretty picture, the evaluation metric should be the measurement itself. A model that aces the standard report card can still flunk the science, and a model that appears to fail it may be quietly doing excellent work.

Subject of Research: Evaluating U-Net deep learning segmentation of bone ingrowth in micro-CT images of hydroxyapatite scaffolds using loss functions and application-specific metrics

Article Title: Segmenting scaffolds with ingrown bone in micro-CT images: considerations of evaluation metrics and loss functions

Article References: Aboutaleb, S., Haug, N., Keni-McCray, P., McCray, A. R. C., Phillips, H., Sharping, S., Cohen, D., Norato, J., & Wagoner Johnson, A. (2026). Segmenting scaffolds with ingrown bone in micro-CT images: considerations of evaluation metrics and loss functions. Medical & Biological Engineering & Computing. https://doi.org/10.1007/s11517-026-03686-x

Image Credits: AI Generated

DOI: 10.1007/s11517-026-03686-x

Keywords: U-Net, semantic segmentation, micro-CT, bone regeneration, hydroxyapatite scaffold, loss functions, evaluation metrics, Dice coefficient, cross-entropy loss, Bland-Altman analysis, deep learning, biomedical imaging