The work, led by Yacine Khaldi of the Mathematics Department at École Normale Supérieure de Ouargla in Algeria, together with Hanane Azzaoui and Mohamed Ben Bezziane of Kasdi Merbah University and Amir Benzaoui of University 20 August 1955 in Skikda, tackles a specific branch of continual learning known as Class-Incremental Learning, or CIL. In this paradigm, a model is confronted with a sequence of tasks, each introducing attack classes it has never seen before, and it must integrate the new knowledge while preserving its performance on previously learned classes. The difficulty is that, unlike humans, neural networks do not naturally compartmentalize knowledge. When gradients from the new task flow through the network, they reshape the very feature representations that encoded the old attacks, and accuracy on the old classes collapses. For an intrusion detection system, this collapse is not an academic inconvenience. A model that forgets how to recognize a denial-of-service flood while learning to spot a SQL injection attempt leaves the network exposed to an attack it was once able to catch.

The most widely used remedy in CIL is exemplar replay. The idea is elegantly simple: the model keeps a small memory buffer containing a handful of stored examples from each previously learned class, and during each incremental training step it replays these stored exemplars alongside the new data. The replayed examples act as anchors, pulling the network’s representations back toward the regions of feature space where the old classes live. But replay memory is strictly constrained. In a security appliance, storage and compute budgets are tight, and the buffer can hold only a tiny fraction of the training data. This constraint transforms exemplar selection from a minor implementation detail into a decisive design choice. Which specific examples, out of thousands of recorded network flows, should occupy the precious slots in the buffer? The answer determines how well the model’s memory of past attacks survives the continual arrival of new ones.

Conventional selection strategies answer this question in two familiar ways. The first is random selection, which simply picks exemplars uniformly at random from each class. The second is uncertainty-based selection, which favors examples near the model’s decision boundary, on the theory that these hard cases carry the most information. Both approaches, however, share a blind spot: they ignore the internal structure of each class. Network traffic for a single attack type is rarely homogeneous. A port-scan attack, for instance, may manifest as a wide spectrum of flow patterns depending on the tools used, the timing, and the network context. A buffer filled with random or boundary-focused exemplars may cover only a narrow slice of that spectrum, leaving large portions of the class distribution unrepresented in memory. When the model later relies on the buffer to reconstruct its knowledge, it reconstructs a distorted, incomplete picture of the attack, and performance on the old class degrades.

DARE-IDS answers this blind spot with a diversity-aware exemplar selection strategy. At its core, the framework selects exemplars that maximize feature diversity within each class. Rather than choosing examples that are representative in a narrow statistical sense, it chooses examples that are spread apart in the model’s learned feature space, ensuring that the buffer covers as much of each class’s distribution as possible under strict memory budgets. The intuition is that a diverse set of anchors constrains the network’s representations more completely: wherever the gradient from new training data tries to distort the feature space, a well-spread set of replayed examples pushes back from many directions at once. This is related in spirit to the core-set approach developed for active learning, where a carefully chosen subset of data can approximate the information content of the full dataset, but DARE-IDS adapts the principle to the specific demands of incremental intrusion detection, where class balance and coverage of rare attack variants matter enormously.

The exemplar selection strategy is only one component of the hybrid framework. DARE-IDS combines replay with knowledge distillation, a technique popularized by Hinton and colleagues in which the outputs of the previous model version serve as soft targets for the new one, preserving the old model’s functional behavior even on inputs that are not in the buffer. It also employs focal loss, a loss function originally introduced for dense object detection, which down-weights the contribution of easy, well-classified examples and concentrates training signal on the hard cases. This is particularly valuable in intrusion detection, where class imbalance is severe: benign traffic vastly outnumbers attack traffic, and within the attack classes themselves, some categories are extremely rare. Finally, the framework applies balanced fine-tuning, a rebalancing stage that corrects the classifier’s bias toward recently learned classes, a well-documented pathology in incremental learning where the last-seen classes dominate the decision layer.

The evaluation protocol is deliberately demanding. The researchers tested DARE-IDS on two benchmark datasets, UNSW-NB15 and CIC-IDS2017, under a five-step class-incremental protocol. This means the attack classes were split into five sequential tasks, and the model had to learn them one after another, with no access to the full training data at any point after the corresponding task had passed. The results are striking. DARE-IDS achieved a final accuracy of 88.2 percent on UNSW-NB15 and 96.8 percent on CIC-IDS2017, consistently outperforming both random and uncertainty-based exemplar selection baselines. Beyond raw accuracy, the framework yielded the highest weighted F1 score across both datasets, a metric that accounts for class imbalance and therefore reflects performance on the attack categories that matter most for security.

Perhaps the most important findings concern rare attack classes and the shape of the accuracy trajectory over the incremental steps. DARE-IDS preserved performance on rare attack classes better than the baselines, which is a critical property in cybersecurity, where the attacks that are hardest to collect data for are often the ones that cause the most damage when they strike. The framework also produced smoother accuracy trajectories with reduced catastrophic forgetting, meaning that as new tasks arrived, the model’s performance on old tasks declined less abruptly and recovered more reliably. In practical terms, this robustness translates into a security model whose behavior is predictable across deployment cycles, rather than one that oscillates between competence and amnesia as the threat landscape evolves.

The implications reach beyond the two datasets tested. The researchers have made their code publicly available on GitHub, and the datasets they analyzed are publicly accessible, which means the framework can be scrutinized, reproduced, and extended by the broader community. The work builds on a growing body of research applying continual learning to intrusion detection, including exemplar-based approaches, few-shot class-incremental methods, and federated learning frameworks, but its specific contribution, the diversity-aware selection of replay exemplars, addresses a gap that prior work had left open. As networks grow more complex, with Internet of Things devices, cloud infrastructure, and increasingly sophisticated adversaries, the ability of a detection system to learn continually without forgetting is shifting from a research aspiration to an operational requirement. DARE-IDS demonstrates that the design of the memory buffer, the humble set of stored examples that anchor a model’s past, may be the difference between a security system that adapts to the threat landscape and one that repeatedly relearns what it once knew.

For the cybersecurity industry, the message is clear: the next generation of intrusion detection systems will not be trained once and deployed forever. They will be living models, updated as new attack classes emerge, and their effectiveness will depend on how well they manage the tension between plasticity and stability. DARE-IDS shows that this tension can be managed with careful engineering, and that the diversity of remembered examples, not their quantity, is the key to a model that learns new attacks while refusing to forget the old ones.

Subject of Research: A hybrid class-incremental learning framework that combines diversity-aware exemplar replay, knowledge distillation, focal loss, and balanced fine-tuning to enable intrusion detection systems to learn new attack classes while preserving prior knowledge.

Article Title: DARE-IDS: diversity-aware replay exemplars for intrusion detection systems

Article References: Khaldi, Y., Azzaoui, H., Benzaoui, A., & Ben Bezziane, M. (2026). DARE-IDS: diversity-aware replay exemplars for intrusion detection systems. Cluster Computing, 29(15), Article 860. https://doi.org/10.1007/s10586-026-06696-4

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06696-4

Keywords: intrusion detection systems, class-incremental learning, catastrophic forgetting, exemplar replay, machine learning, cybersecurity, UNSW-NB15, CIC-IDS2017, DARE-IDS, diversity-aware, replay, exemplars

The obvious defense is a strong morphing attack detection system, or MAD, trained on large and varied collections of morphed faces. Yet this is precisely where privacy law becomes the bottleneck. Facial images are personal data, and regulations severely restrict their acquisition, storage, and sharing across institutions. Research laboratories are often forced to develop MAD solutions using only internally sourced data, which is limited in quantity and diversity. The result is a familiar pattern in modern machine learning: deep learning-based detectors perform well on their own private data but generalize poorly to new, unseen morphing techniques and image qualities. Public benchmarking platforms such as NIST FRVT MORPH and FVC-onGoing help by testing algorithms on sequestered datasets, but they evaluate finished models rather than strengthen the training procedure itself.

A team of Italian researchers at the University of Bologna and the University of Modena and Reggio Emilia, led by Lorenzo Pellegrini, Guido Borghi, Annalisa Franco, and Davide Maltoni, has now proposed a way out of this impasse. Writing in the journal Frontiers in Imaging, they introduce Adaptive-LwF, a continual learning algorithm that allows a single MAD model to be trained incrementally across multiple sites without any personal data ever being transferred. The core idea is elegant: instead of moving data to the model, the model moves from site to site, updating its weights each time a new chunk of data becomes available, then deleting or leaving behind the data it just learned from. A model trained this way can accumulate knowledge from airport gates, where images are available only temporarily, and from private laboratory datasets, without violating privacy constraints.

Continual learning, sometimes called lifelong learning, is the machine learning paradigm designed for exactly this kind of sequential training. Its central enemy is catastrophic forgetting, the tendency of a neural network to abruptly overwrite previous knowledge when it is updated on new data. The Bologna team’s detector follows a state-of-the-art differential MAD pipeline: a ResNet backbone trained with the ArcFace loss extracts two 512-dimensional identity embeddings, one from the trusted live acquisition and one from the possibly morphed document photo. The embeddings are combined by subtraction and fed into a multi-layer perceptron classifier that outputs the probability that the pair represents a morphing attack. The classifier is a deep network rather than the traditional support vector machine, which makes it compatible with continual learning algorithms.

In a preliminary comparison of replay-free continual learning methods, the team evaluated Synaptic Intelligence, Elastic Weight Consolidation, Deep Streaming Linear Discriminant Analysis, and Learning without Forgetting, or LwF, under scenarios where the size of each incoming data chunk varies. LwF emerged as the clear winner, achieving the lowest error relative to the ideal joint training upper bound in both small-chunk and large-chunk settings. LwF works through knowledge distillation: an old model, which embodies everything learned so far, acts as a teacher for a current model learning from new samples. Crucially, LwF requires no replay memory, meaning no past data is stored, which is what makes it suitable for privacy-critical biometric applications where replay-based methods are excluded by regulation.

But LwF has a hidden weakness that most continual learning research ignores: its performance depends critically on a single hyperparameter, the weight of the distillation loss, usually denoted lambda. Higher values of lambda make the model conservative, preserving past knowledge at the expense of slower adaptation to new data. Lower values allow faster learning but risk overwriting essential features. The team showed experimentally that choosing the wrong lambda can degrade detection performance by roughly five percentage points in equal error rate and up to fifteen points in the stringent BPCER at APCER 0.001, the operating point typical of real-world border control systems. Worse, the optimal lambda varies with the size of the incoming data chunk, and in practical deployments nobody can predict in advance how many new samples will arrive.

The breakthrough of Adaptive-LwF is a mathematical formula that eliminates this tuning problem entirely. By fitting a regression over experimentally observed optimal lambda values across different chunk sizes, the researchers derived an exponential scaling law: lambda equals a times x to the power of negative b, where a is 17,403, b is 0.636, and x is the input chunk size. The pattern behind the formula is intuitive: smaller chunks of new data deserve a larger distillation weight, so the model leans harder on its old knowledge when there is little new evidence to learn from, while larger chunks justify a lighter touch. Using this formula, the distillation loss is set automatically for every learning experience, removing the need for repeated grid searches or manual hyperparameter selection that would multiply computational costs, especially when chunk sizes are unpredictable.

To validate the approach, the team designed five novel benchmarks simulating realistic data arrival patterns: fixed-size chunks, the traditional continual learning setting; variable chunks corresponding to whole new datasets; and three scenarios where chunk sizes are drawn from uniform or Zipf-like distributions, favoring either small or large arrivals. Training data came from public morphing collections built with OpenCV, FaceMorpher, and StyleGAN algorithms, while the fixed test set was MorphDB, a challenging collection of manually retouched, high-quality morphs. Performance was averaged over all 24 possible orderings of the four training datasets, and new metrics, including a Temporal Area Under the Curve and a Borda Ranking over Time, were introduced to assess how methods perform across the entire incremental training process rather than at a single snapshot.

The results confirm that Adaptive-LwF consistently outperforms both naive incremental training and LwF with any fixed lambda value, whether small or large. In the preliminary investigation, the naive approach left a consistent gap behind the joint training upper bound, particularly in equal error rate and BPCER at the strictest operating point, because it lacks any forgetting-contrasting mechanism. Adaptive-LwF closes much of that gap, achieving a final error closer to, and in some cases better than, the joint method, while respecting privacy by design. The authors are candid about limitations: performance may still fall short of traditional batch learning on the full dataset, the exponential scaling function’s generalization to other scenarios requires further assessment, and the approach depends on the quality of pre-trained ResNet embeddings. Robustness to heavily imbalanced or noisy data streams also remains untested.

The implications reach beyond morphing attack detection. The continual learning paradigm proves valuable wherever data cannot be transferred between sites due to privacy concerns, enabling model transfer instead of data transfer. For border security agencies, it points toward MAD systems that continuously improve as new images are acquired at automated gates, learning to detect novel morphing techniques without forgetting previous ones and without storing a single face. For the continual learning community, the work delivers one of the first systematic investigations of how the distillation loss parameter in LwF should relate to the size of incoming training data, a question the literature had left largely unaddressed. As face morphing attacks grow easier to create and harder to spot, detectors that can learn everywhere and forget nothing may become the indispensable tool that keeps automated border controls one step ahead.

Subject of Research: Adaptive-LwF: continual training of morphing attack detector without forgetting

Article Title: Adaptive-LwF: continual training of morphing attack detector without forgetting

Article References: Pellegrini, L., Borghi, G., Franco, A., & Maltoni, D. (2026). Adaptive-LwF: continual training of morphing attack detector without forgetting. Frontiers in Imaging, 5, Article 1817515. https://doi.org/10.3389/fimag.2026.1817515

Image Credits: AI Generated

DOI: 10.3389/fimag.2026.1817515

Keywords: continual learning, face recognition, morphing attack, morphing attack detection, catastrophic forgetting, biometrics, privacy, machine learning, security, artificial intelligence, Adaptive-LwF, continual

The article, authored by independent researcher Nan Chen of Chengdu, China, maps ten components of a previously proposed dual-architecture AGI system onto the biological timeline of brain maturation. The central claim is deliberately provocative: machine intelligence may need to retrace the milestones of biological intelligence, acquiring physical tokens before social ones, building redundant capacity before compressing it, and pausing periodically to consolidate memory. Crucially, the author frames the thesis as falsifiable. If an AGI system could reach human-level physical and social reasoning while ignoring the proposed ordering, the core claim would collapse.

The developmental story begins at birth, with what the paper calls phase zero. The only fully myelinated structures in a newborn brain are the brainstem, spinal reflex arcs, and portions of thalamic and basal ganglia circuits. These are, in effect, factory pre-installed: respiration, grasp reflexes, and startle responses operate immediately without learning, at millisecond speeds. In the proposed AGI design, this maps onto a frozen reflex layer, a pretrained vision-language model with fixed weights that delivers predictions in a single forward pass. Meanwhile the prefrontal cortex, the seat of human intelligence, is almost entirely unmyelinated at birth. The most sophisticated part of the brain is built after contact with the world, not before, a fact that corresponds directly to the design principle that the AGI continuous learning layer starts with near-random weights.

From birth to age three, the brain runs two massive programs. First, synaptic overproduction: by ages two to three, cortical neurons average around 15,000 synapses each, roughly double the adult density, creating an enormous hypothesis space that experience will later prune. Second, myelination sweeps posterior to anterior, from visual cortex through somatosensory and motor regions to language areas and finally the prefrontal cortex. This sequence mirrors behavior: see first, then touch, then move, then speak, then plan. Two mechanisms launch during this window that the paper considers essential for AGI. Dopamine neurons fire not for reward itself but for prediction error, the gap between expected and actual outcomes, which grounds the architecture’s curiosity module. And hippocampal place-cell sequences replayed during sleep at roughly twenty times speed consolidate fragile memories into stable cortical storage, the biological rationale for a replay buffer that prevents catastrophic forgetting in artificial networks.

Between ages three and four, the system locks. Large-scale synaptic pruning begins, eliminating unused connections under a use-it-or-lose-it rule until roughly half of childhood synapses are gone by adolescence. The paper identifies this as the biological counterpart of token migration: patterns verified across enough interactions, exceeding an accuracy threshold and a minimum interaction count, are promoted and frozen into the reflex layer, while unverified candidates are discarded. Infantile amnesia ends at the same time, plausibly because declining hippocampal neurogenesis stops overwriting memory traces. In the AGI framework, experiences become retrievable knowledge only once abstracted into named tokens and written into a structural constraint graph; before that, they exist only implicitly as weight adjustments. The author illustrates the mechanism concretely: a robot that learned the physical token of force from a thousand box-pushing interactions can, when observing human arguments, use a learned transformation matrix to initialize a social token of persistence, reusing the causal structure of physical transfer as a representational prior rather than a metaphor.

The longest phase stretches from age three to about twenty-five, as the prefrontal cortex, responsible for planning, inhibition, and theory of mind, slowly myelinates last of all brain regions. The paper argues this ordering is a dependency, not a delay: social cognition is built on physical cognition. One cannot ground the social concept of a promise without first internalizing, through physical experience, the notion that actions constrain future outcomes, an intuition rooted in gravity, friction, and collision. In the architecture, this translates into a hard sequencing rule: social token acquisition presupposes that physical tokens are already internalized in the reflex layer. Self-modeling compounds the dependency, since mirror self-recognition in humans emerges at 18 to 24 months only after basic sensorimotor coordination, and a self-token in the framework must be composed from lower-level physical and social tokens.

A common objection is that the brain’s schedule is dictated by carbon, not computation: energy metabolism, molecular signaling, and cranial volume. The paper counters that carbon and silicon face deeply isomorphic optimization problems. Myelination boosts conduction speed from about one meter per second to roughly one hundred and cuts energy use, the same trade-off as migrating frequently used tokens from a slow, gradient-updated layer into a fast, single-pass layer. Synaptic overproduction followed by pruning echoes the lottery ticket hypothesis in machine learning, where large over-parameterized networks contain sparse subnetworks that generalize better after pruning than compact models trained directly. And the necessity of sensorimotor calibration before abstract reasoning parallels the collapse that occurs when a high-level classifier is trained on unconverged low-level features. The brain develops in sequence, the argument runs, not because it is flesh, but because it is a hierarchical learning system bootstrapping abstractions from raw data.

The author backed individual mechanisms with experiments on the MuJoCo physics engine, all run on a consumer MacBook CPU with no GPU. In a three-task continual learning benchmark on the Ant-v5 quadruped, elastic weight consolidation retained 93.2 percent of baseline performance after two task switches, synaptic intelligence 102.3 percent, and memory-aware synapses 82.7 percent, all above the 80 percent target, while unprotected training was erratic and, in a separate run, retained only 71.7 percent. The paper candidly notes that a 27 percent baseline variance limits the precision of those comparisons. A curiosity experiment showed that intrinsic prediction-error reward drove exploration roughly seven times deeper into novel territory than extrinsic reward alone. In a multimodal task on HalfCheetah-v5, temporally aligned vision, force, and audio signals predicted one another across all tested directions, while shifting the signals by just twenty steps drove performance below chance, direct evidence that shared representations are built on temporal synchrony.

Perhaps the most striking experimental finding concerns negative transfer. When a pretrained reflex layer was fine-tuned on genuinely novel physics, high-gravity environments, warm-starting with pretrained weights actively hurt: the frozen model scored better than any fine-tuned variant. A systematic gradient across five gravity levels identified a crossover threshold between roughly 1.25 and 1.5 times Earth gravity, beyond which training from scratch beats incremental adaptation. Weight-space analysis showed why: the pretrained solution sat too far from the new optimum. The revised routing strategy mirrors biology, where reflex arcs do not change with experience: default to frozen reflex predictions, train an independent learning layer from random initialization in the background, and switch when its confidence surpasses the reflex layer’s. The paper also argues that AGI may need explicit sleep-like cycles, offline consolidation phases without new input, to avoid overwriting old knowledge, and that the 25-year human timeline conflates clock time with experience time. If humans need on the order of 100 million effective physical interactions and simulators reach tens of thousands per hour, the equivalent developmental span could compress to about a year without skipping a single stage.

The author is careful about limits. The correspondence may be analogy rather than isomorphism, the experiments cover only six mechanisms at modest complexity, adult neuroplasticity suggests developmental stages are more flexible than the framework assumes, and the central falsification condition, an AGI reaching human-level reasoning without the ordering, cannot yet be tested because no such system exists. Still, the engineering paradigm on offer is a genuine inversion of the dominant scaling philosophy. Childhood, in this view, is not a bug but a feature: build an excessively redundant system, let experience prune it, freeze what works, and only then attempt abstraction. Nature took 3.5 billion years to find the sequence. Silicon need not take 25 years, but the paper bets it must still take them in the right order, physical before social, redundant before compressed, interactive before consolidated.

Subject of Research: Mapping human neurodevelopmental stages onto a dual-architecture artificial general intelligence framework

Article Title: Neurodevelopment as blueprint: biological validation of the dual-architecture AGI framework

Article References: Chen, N. (2026). Neurodevelopment as blueprint: biological validation of the dual-architecture AGI framework. Frontiers in Robotics and AI, 13, Article 1902952. https://doi.org/10.3389/frobt.2026.1902952

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1902952

Keywords: artificial general intelligence, neurodevelopment, dual architecture, catastrophic forgetting, token migration, continual learning, curiosity-driven exploration, multimodal alignment, synaptic pruning, myelination, negative transfer, MuJoCo

The research, led by Vishwanatha M. Rao, Emma Chen and Agustina Saenz at Harvard Medical School under the supervision of Pranav Rajpurkar, assembled 30 neonatal intensive care units across 20 countries and territories into one of the most geographically diverse imaging cohorts ever gathered for neonatal AI development. Their target task was the assessment of endotracheal tubes on chest radiographs, a deliberately chosen proof of concept. Several AI tools have already received FDA clearance for checking tube position in adult intensive care, but no commercial solution exists for newborns, despite the fact that misplaced tubes in neonates carry severe consequences: tubes inserted too deeply can ventilate a single lung and cause collapse, while false alarms can trigger unnecessary re-intubations that raise procedural risk and haemodynamic instability.

The team’s method is called continual learning, and it works like a relay race for model weights. A deep learning model originally trained on adult ICU radiographs was passed sequentially from hospital to hospital. Each site retrained the model on just 50 locally curated neonatal chest X-rays inside its own computing environment, then forwarded only the updated parameters to the next institution. No patient image ever left a hospital, no large dataset had to be assembled, and no specialized AI team was required at any site. Because only model weights travel, the approach sidesteps the data transfer agreements, ethics approvals across jurisdictions and regulatory hurdles that make centralized data pooling so difficult, while avoiding the synchronized compute infrastructure and multi-round communication that federated learning demands.

Technically, the underlying model was CarinaNet, a RetinaNet-based object detector with a ResNet-50 backbone and a feature pyramid network, pretrained on natural images and then adapted to 1,890 adult ICU radiographs. The model detects two structures, the tip of the endotracheal tube and the carina, the cartilaginous ridge where the trachea splits into the bronchi, and outputs bounding boxes whose centre points serve as predicted landmark locations. Its training loss combines focal loss for classification with smooth L1 regression, weighted asymmetrically to prioritize the carina, the anatomical reference point for judging tube depth. Images were histogram-equalized to standardize contrast across scanners, resized to 640 by 640 pixels with aspect-ratio-preserving padding, and augmented with small random rotations and vertical crops.

The results were decisive. For endotracheal tube tip localization, median error fell from 2.35 centimetres when the adult model was applied to neonatal images without adaptation, to 0.94 centimetres with hospital-specific fine-tuning, and down to 0.40 centimetres with continual learning. Carina localization improved from 1.62 centimetres under direct prediction and 0.27 centimetres under fine-tuning to 0.20 centimetres. Combined, the continual learning models achieved a mean landmark error of 0.67 centimetres, compared with 1.29 centimetres for fine-tuning and 4.30 centimetres for the unadapted adult model, improving localization at 29 of 30 sites, with statistical significance at 27. The study cohort comprised 2,965 radiographs from 2,960 newborns, with a median age at imaging of just two days, a mean gestational age of 32.9 weeks and a mean birth weight of 2,195 grams, reflecting a predominantly preterm population.

The clinical stakes of those millimetres are considerable. The researchers defined extreme errors as combined localization mistakes exceeding five centimetres, a threshold informed by anatomical measurements showing that the total tracheal length in neonates ranges from roughly 2.2 to 6.4 centimetres depending on gestational age. A five-centimetre error would therefore approximate or exceed the entire trachea of most newborns. Continual learning cut the rate of such catastrophic errors to 2.6 per cent of cases on average, against 8.2 per cent for fine-tuning and a startling 37.4 per cent for direct prediction. When the models were asked to triage, ranking the fifteen most concerning scans per hospital for radiologist review, continual learning achieved significantly higher precision and recall for detecting abnormally low tube placements than fine-tuning at 18 of 30 hospitals, with positive improvements at 27, and precision exceeding 80 per cent at the best-performing sites.

A crucial question for any sequential training scheme is catastrophic forgetting, the tendency of a model to lose earlier skills as it absorbs new data. The team addressed this with a backward evaluation: across 100 random hospital orderings, they compared each hospital’s performance immediately after the model trained on its data with performance after the full sequence completed. Far from degrading, the final model consistently outperformed the intermediate one at hospitals encountered early in the sequence, with a mean improvement of 0.48 centimetres in combined error at the first position. Knowledge, in other words, accumulated rather than eroded, and the gains grew steadily with each additional contributing hospital across the entire 29-hospital training sequence, with no evidence of a plateau.

The study also mapped the conditions under which the paradigm succeeds. Initializing the sequence with the adult-pretrained model beat starting from a generic object detector never exposed to radiographs, improving localization at 25 of 30 sites and triage metrics at 27, confirming that features learned from adult chests transfer usefully to newborns despite profound anatomical differences. But data quality mattered sharply: one contributing hospital supplied notably low-resolution radiographs, and including those scans in training raised the combined error from 0.67 to 0.75 centimetres and reduced the number of sites with positive triage improvements from 27 to 20, suggesting that quality thresholds for participating sites may be warranted. Encouragingly, performance remained consistent across demographic subgroups of race, sex, age, birth weight and gestational age, with no significant pairwise differences.

The researchers are candid about the limits. Pixel-to-centimetre conversions relied on the known width of the endotracheal tube as an internal calibration reference, an imperfect proxy that probably explains weaker correlations between predicted distances and clinical labels for high tube placements, which are properly defined relative to the first thoracic vertebra rather than the carina. The retrospective design simulated decentralized training centrally, and true distributed execution, prospective trials measuring time-to-confirmation and re-intubation rates, and reconciliation with regulatory oversight of continuously evolving models all remain ahead. Yet the broader implication is hard to overstate. Any paediatric imaging task for which adult AI models exist but paediatric solutions do not could, in principle, be reached the same way: a modest locally annotated dataset at each hospital, a self-contained training container, and a travelling model that grows wiser with every stop. In a field where children have been systematically left out of the AI revolution, thirty hospitals across twenty countries have just shown how to bring them in, one small, private dataset at a time.

Subject of Research: Privacy-preserving continual learning to adapt adult AI radiograph models for neonatal endotracheal tube assessment

Article Title: Repurposing adult AI radiograph models for neonatal care through continual learning: an international multicentre study

Article References: Rao, V. M., Chen, E., Saenz, A., Banerjee, O., Marklund, H., Bijnens, L., Kim, S. E., Zucker, E., Shelmerdine, S., Palacio, J., Kleinschmidt, K. R., Suryadevara, S., Jaliparthi, S. S. D., Lu, J., Akula, R., Scala, M., Niedermair, J. F., Stranger, N., Tschauner, S., … Rajpurkar, P. (2026). Repurposing adult AI radiograph models for neonatal care through continual learning: an international multicentre study. Nature Biomedical Engineering. https://doi.org/10.1038/s41551-026-01812-6

Image Credits: AI Generated

DOI: 10.1038/s41551-026-01812-6

Keywords: continual learning, neonatal intensive care, chest radiographs, endotracheal tube placement, medical AI, privacy-preserving machine learning, catastrophic forgetting, paediatric radiology, multicentre study, transfer learning, deep learning, Nature Biomedical Engineering

A new study published in Earth Science Informatics by Gang Hu of Anshan Normal University, Yifei Song of Universiti Utara Malaysia, and Mohamad Farhan Mohamad Mohsin of the same institution’s School of Computing tackles this problem head-on. The researchers have developed a model called AMCUATR, short for Attention-based Meta-learning and Class-Incremental Underwater Acoustic Target Recognition. The framework is designed to do two things that conventional deep learning systems struggle to combine: learn to recognize new vessel classes from only a handful of labeled examples, and absorb those new classes without erasing what the model has already learned about previous ones. According to the paper, the model achieved a recognition accuracy of 80.01 percent under a demanding 4-way 5-shot testing scenario on the widely used ShipsEar dataset, meaning it correctly identified ship types after seeing just five examples of each category.

The challenge the researchers address is twofold. First, there is the few-shot learning problem: in real deployments, oceanographers and naval operators constantly encounter vessel types that were never represented in the training data, and gathering hundreds or thousands of labeled recordings for each new class is rarely feasible. Second, there is the problem of catastrophic forgetting, a well-documented failure mode in neural networks in which a model trained on new categories abruptly degrades on the categories it previously mastered. Standard fine-tuning approaches, when applied to a trained network with new data, tend to overwrite the internal weight configurations that encoded the old knowledge, effectively wiping the model’s memory of earlier targets.

To overcome these obstacles, AMCUATR combines three interlocking technical components. The first is a dual-branch time-frequency feature extraction module with adaptive channel calibration. Underwater acoustic signals carry information both in how their energy evolves over time and in how it is distributed across frequencies, so the model processes two complementary representations in parallel. One branch focuses on temporal structure while the other captures spectral characteristics, and a channel attention mechanism then learns to weight the most informative feature channels more heavily. This adaptive calibration allows the network to suppress noisy or redundant dimensions and amplify the subtle cues that distinguish one vessel’s radiated noise from another’s, a critical capability when working with the low signal-to-noise recordings typical of real ocean environments.

The second component is a hierarchically embedded Swin Transformer, a vision architecture adapted here for acoustic data. Unlike conventional convolutional networks that process local neighborhoods of pixels or spectrogram cells, transformer architectures use self-attention to model relationships across an entire input. The Swin variant computes attention within hierarchical windows and shifts those windows between layers, allowing the model to capture dependencies at multiple scales without the prohibitive computational cost of full global attention. In AMCUATR, this module serves to build a richer global context around the features extracted by the dual-branch front end, improving the discrimination of the embedding space in which different vessel classes must be separated. The authors report that this hierarchical global modeling improves feature quality precisely where few-shot classification needs it most: when only a few examples define each class, the geometry of the embedding space determines whether those examples cluster cleanly or blur together.

The third component, and the conceptual heart of the system, is a prototype-based class-incremental learning mechanism. The approach draws on ideas from prototypical networks, a meta-learning method in which each class is represented by a prototype vector, typically the average of the embedding vectors of its few labeled examples. Classification then reduces to measuring which prototype an unknown sample lies closest to. Because class knowledge is stored in these prototype summaries rather than only in the network’s weights, new categories can be added by computing new prototypes from a handful of samples, while the prototypes of previously learned classes remain available to anchor the model’s old knowledge. Combined with meta-learning, in which the model is trained across many simulated few-shot tasks so that it internalizes how to adapt quickly rather than memorizing specific classes, this design gives AMCUATR its ability to keep learning over time.

The experimental evaluation was conducted on two public benchmark datasets that have become standard proving grounds for this field. ShipsEar, first described in Applied Acoustics in 2016, is a database of underwater vessel noise recordings covering multiple ship categories along with environmental background noise. DeepShip, introduced in Expert Systems with Applications in 2021, offers a larger collection of ship-radiated noise organized by vessel type and operating condition. Testing on both datasets allowed the researchers to assess how well the framework generalizes beyond a single recording campaign, an important consideration given that acoustic conditions vary with sea state, water depth, and sensor placement.

The incremental learning results are particularly notable. In the few-shot class-incremental setting, where the model had to learn novel vessel categories from only five labeled examples each, AMCUATR achieved a new-class recognition accuracy of 83.14 percent. More importantly for the forgetting problem, it maintained an old-class retention accuracy of 79.05 percent with a forgetting rate of just 1.20 percent, yielding an average incremental accuracy of 81.10 percent across the evaluated setting. Those numbers indicate that the model acquired new knowledge at only a modest cost to its existing capabilities, which is precisely the balance that practical, continuously deployed recognition systems require. A monitoring network that must be updated each time a new vessel class appears cannot afford to retrain from scratch, nor can it tolerate a model that forgets yesterday’s targets every time it learns today’s.

The significance of this work extends beyond the specific accuracy figures. Underwater acoustic monitoring is becoming central to a range of scientific and operational missions, from tracking the impact of shipping noise on marine mammals to detecting natural gas seeps with autonomous underwater vehicles, a task explored in recent control engineering research. Passive acoustic sensors are relatively inexpensive and can operate continuously in remote locations, but their value depends on automated analysis, since human analysts cannot listen to years of recordings. Systems like AMCUATR point toward recognition platforms that can be deployed with modest initial training data and then grow their knowledge base incrementally as they encounter new sounds in the field, whether novel vessel designs, unfamiliar marine species, or anomalous signals worth flagging for investigation.

The study also situates itself within a rapidly evolving research landscape. Recent years have seen transformer-based networks, self-supervised pretraining with masked hierarchical tokens, graph embeddings built on mel-spectrograms, and physics-informed Bayesian graph networks all applied to underwater acoustic recognition, while other groups have pursued data augmentation through generative adversarial networks and diffusion models to stretch limited datasets further. The contribution of Hu and colleagues is to fuse attention-based feature extraction, meta-learning, and prototype-driven incremental learning into a single coherent pipeline and to demonstrate quantitatively that the combination holds up under the joint pressures of scarce labels and evolving class sets. The authors note that the datasets analyzed in the study are publicly available, and the research received no external funding. As ocean observation networks expand and the demand for autonomous acoustic intelligence grows, frameworks that can learn continuously from little data may prove to be not just a convenience but a necessity for keeping pace with the ever-changing soundscape beneath the waves.

Subject of Research: Few-shot class-incremental deep learning for underwater acoustic target recognition of ship-radiated noise

Article Title: A few-shot UATR Model integrating attention mechanism-based meta-learning with class incremental learning approaches

Article References: Hu, G., Song, Y., & Mohamad Mohsin, M. F. (2026). A few-shot UATR Model integrating attention mechanism-based meta-learning with class incremental learning approaches. Earth Science Informatics, 19(11), Article 198. https://doi.org/10.1007/s12145-026-02247-y

Image Credits: AI Generated

DOI: 10.1007/s12145-026-02247-y

Keywords: underwater acoustics, deep learning, meta-learning, few-shot learning, class-incremental learning, attention mechanism, Swin Transformer, ship-radiated noise, ShipsEar, DeepShip, prototype networks, catastrophic forgetting

Federated learning is a training framework built around a simple but powerful idea: instead of pooling sensitive medical records on a central server, each participating institution keeps its data locally and trains a shared model on its own machines. In each communication round, a central server distributes the current global model to every client, each client performs local training using its own data, and the server then aggregates the updated parameters to produce a new global model. This approach, known as FedAvg, has attracted intense interest in medical imaging because it promises the statistical benefits of large, diverse datasets while respecting privacy regulations and institutional boundaries that often make data sharing impossible.

The research team, led by Yukihiro Nomura of Chiba University and The University of Tokyo Hospital, tested this idea using two distinct computer-aided detection systems. The first was a three-dimensional U-Net model designed to detect cerebral aneurysms in magnetic resonance angiography images, a task where missing a bulging vessel wall can be catastrophic. The second was a single-shot multibox detector for spotting brain metastases in contrast-enhanced T1-weighted MR images, a critical aid for planning stereotactic radiosurgery. For each task, the team assembled four heterogeneous datasets differing in scanner vendor or institution, totaling 423 MRA images for aneurysm detection and 398 contrast-enhanced T1-weighted images for metastasis detection, with voxel-level annotations available throughout.

The central question was deceptively simple: if a hospital network starts with two institutions and then adds a third and a fourth one at a time, does the detection software actually get better? The researchers hypothesized that it would, but the answer turned out to depend heavily on how the model is retrained when a newcomer arrives. They compared three strategies. From-scratch training reinitialized all model parameters and trained anew on the combined data of all participating clients. Full fine-tuning started from the previously trained global model and updated every parameter. Partial fine-tuning also started from the existing model but froze most layers, updating only a carefully chosen subset, such as the decoder layers of the U-Net or the upper and head portions of the single-shot detector.

The results revealed a clear pattern. For both detection tasks, sequential institution addition combined with fine-tuning generally produced higher median performance gains than training from scratch. Performance improvements tended to accumulate as institutions were added one by one, whereas adding two institutions simultaneously, a condition the team called At-once addition, yielded less consistent improvements. This distinction matters because the simultaneous addition of multiple clients introduces a larger distributional shift in one step, forcing the model to absorb more change at once. When the researchers extended the At-once stage to 200 communication rounds, gains improved in some settings but remained inconsistent across tasks and strategies, suggesting the problem is not simply a matter of insufficient training time.

Perhaps the most striking finding concerns parameter efficiency. In the aneurysm task, full fine-tuning and from-scratch training each updated roughly 0.80 million parameters, while the partial fine-tuning strategies updated only about 0.31 million and 0.08 million parameters respectively. In the metastasis task, the corresponding numbers were approximately 7.95 million for full updates versus 2.66 million and 0.53 million for the two partial strategies. Despite touching a fraction of the network, partial fine-tuning achieved performance changes comparable to full fine-tuning across both tasks. For real-world deployments, this efficiency translates directly into lower computational load and reduced communication overhead, both of which are significant practical constraints when hospitals with varying IT resources participate in a federated network.

The study also confronted an uncomfortable trade-off known in machine learning circles as catastrophic forgetting. When a model adapts to newly added institutions, its performance on the original institutions’ data can decline, which would be unacceptable if the software were already deployed and trusted at those sites. The researchers evaluated this by measuring performance changes on test sets from the initial clients separately from test sets spanning all four institutions. For cerebral aneurysm detection, performance decreases on the initial clients’ test sets were less frequent with full and partial fine-tuning than with from-scratch or centralized training. For brain metastasis detection, however, no strategy showed a clearly distinct pattern, underscoring that the forgetting problem remains task-dependent and unresolved.

Statistical caution tempers the enthusiasm. The team measured performance using the competition performance metric, which averages sensitivity across seven false-positive operating points, and quantified changes as differences from the corresponding initial model. Confidence intervals estimated by paired bootstrap resampling with 1,000 iterations included zero in every evaluated setting on the all-clients test sets, and direct pairwise comparisons among the four training strategies found no statistically significant differences after adjustment for multiple testing. The researchers also ran additional analyses with the FedProx aggregation method, designed for heterogeneous non-IID data, and found broadly similar patterns, suggesting the trends were not an artifact of the FedAvg baseline. Still, the honest conclusion is that the observed benefits are promising tendencies rather than proven guarantees.

The authors frame their setting as a domain-incremental learning problem, a concept from continual learning research in which models must accommodate new data distributions while retaining performance on previously learned ones. This lens highlights both the promise and the limits of the work. The study did not address scenarios such as institutions withdrawing from a network or participating intermittently, nor did it incorporate privacy-enhancing technologies like differential privacy or homomorphic encryption, which would likely be required in production deployments. The clients were also defined by acquisition site or scanner vendor rather than random patient-level splits, capturing realistic heterogeneity in devices and protocols but not the full breadth of differences in patient populations, clinical practice, and annotation policies that large international networks would encounter.

Even with these limitations, the study offers a roadmap for a future in which hospital AI systems improve continuously throughout their operational life. Regulatory frameworks have already approved commercially available detection software bearing CE marks and FDA clearances, and guidelines increasingly recommend multi-institutional data for building robust medical AI. What this research adds is evidence that the network can grow organically, hospital by hospital, with fine-tuned updates that are computationally light and privacy-preserving. As federated learning infrastructure matures, the modest but consistent gains documented here suggest that the collective intelligence of a medical network may indeed be greater than the sum of its parts, provided each new member is welcomed with the right training strategy.

Subject of Research: Federated learning strategies for improving computer-aided detection of cerebral aneurysms and brain metastases as institutions sequentially join a collaborative training network

Article Title: Performance changes in automated lesion detection under federated learning with sequential institution addition

Article References: Nomura, Y., Hanaoka, S., Yamada, A., Takenaga, T., Nakao, T., Nakaguchi, T., Yoshikawa, T., & Abe, O. (2026). Performance changes in automated lesion detection under federated learning with sequential institution addition. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03795-w

Image Credits: AI Generated

DOI: 10.1007/s11548-026-03795-w

Keywords: federated learning, computer-aided detection, cerebral aneurysm, brain metastasis, magnetic resonance imaging, fine-tuning, deep learning, FedAvg, catastrophic forgetting, medical imaging, multi-institutional data, privacy-preserving AI

The setting the researchers tackled, known as Online Continual Graph Learning, or OCGL, is among the harshest in machine learning. Nodes of a graph, which might represent Bitcoin transactions, scientific papers, social media posts or products on a shopping site, arrive one at a time in a continuous stream. Each new node brings its own attributes and connections to previously seen nodes, and the distribution of the data can shift at any moment, for instance when new classes of objects begin to appear. The model must make accurate predictions at every instant, adapt on the fly, and retain what it learned about earlier data, all under strict limits on memory and computation. Crucially, the model sees each node only once, so there is no possibility of the repeated offline training passes that standard deep learning relies on.

This streaming regime makes the phenomenon known as catastrophic forgetting especially damaging. When neural networks are updated sequentially on new data, the parameter changes that help with the new information often overwrite the representations that supported old tasks, erasing previously acquired knowledge. Graph data add a second, structural source of forgetting: because graph neural networks aggregate information from neighbors, the embedding of a node changes as new nodes attach to it, so even a perfectly stable classifier faces inputs that drift over time. Existing state-of-the-art methods typically fight forgetting with replay buffers that store examples of past data, or with regularization schemes that penalize changes to important weights, and both strategies carry substantial memory and computational costs in the graph setting.

The new approach, by contrast, decouples representation from prediction in a strikingly simple way. The authors split the model into two parts: a feature extractor that converts each node and its sampled neighborhood into a fixed-length embedding vector, and a lightweight classifier that maps embeddings to predictions. The feature extractor is a graph neural network whose weights are randomly initialized from an appropriate distribution and then frozen forever. Because the encoder never changes, the representations it produces cannot drift, and one of the two major sources of forgetting, the drift of backbone parameters, is eliminated by construction. Only the classifier is trained, and even that can be done without gradient descent using a streaming statistical method.

The researchers tested two flavors of randomized encoder. The first, called UGCN, is an untrained graph convolutional network with two layers of 1024 units each, whose layer outputs are concatenated into a 2048-dimensional node embedding that captures neighborhood information at multiple resolutions. The second, Graph Random Neural Features or GRNF, draws on theory showing that random samples from a universal family of graph neural networks can approximate kernel functions on graphs, and can provably separate any two non-isomorphic graphs. Both encoders operate on sparsified two-hop neighborhoods sampled around each node, keeping the computational footprint constant even as the graph grows denser. On top of either encoder, the team placed a Streaming Linear Discriminant Analysis classifier, which maintains running class means and a shared covariance matrix updated incrementally with each new example, requiring no memory buffer of past data at all.

The theoretical intuition behind the method is elegant. The authors formally decompose forgetting risk into three components: structural drift arising from the evolving graph itself, which no model can remove; backbone parameter drift, which the frozen encoder eliminates entirely; and classifier parameter drift, which for the streaming classifier shrinks as more examples of each class accumulate. Stability alone is not enough, of course, since a model must also remain plastic enough to absorb new classes and shifted distributions. That is where the over-parameterized random encoders earn their keep. Results from randomized network theory guarantee that, given a sufficiently large embedding dimension, randomly initialized features are expressive enough to make the downstream classification problem linearly separable, so a simple linear readout suffices.

The experimental results across seven benchmarks are remarkable. On six node-classification datasets, including the citation networks CoraFull and Arxiv, the co-purchase graph Amazon Computer, the Reddit post network, the heterophilic Roman Empire Wikipedia graph, and the Elliptic Bitcoin transaction network, the combination of randomized features and the streaming classifier generally beat every alternative considered, including experience replay, A-GEM, EWC, LwF and MAS applied to a conventionally trained graph network, as well as recent graph-specific replay methods such as PDGNN, SSM and TWP. In class-incremental streams, where new classes arrive in blocks, the approach approached the upper bound of joint offline training on the complete final graph, a ceiling that no online method can normally reach. Notably, the method needs no memory buffer, whereas replay-based competitors must store a substantial fraction of past nodes tailored to graph topology.

The robustness of the result is underlined by several control experiments. Even with as few as 64 random features, performance on most benchmarks matched or exceeded state-of-the-art trained methods, and gains had not saturated even at 4096 features. A frozen graph network pre-trained in a supervised way on part of the stream did not outperform the purely random encoder on most datasets, indicating that the advantage stems from the stability and richness of random representations rather than from any task-specific preparation. On the heterophilic Roman Empire dataset, where standard graph convolutions smooth neighboring features together, the more expressive GRNF encoder proved clearly superior. A hybrid extractor mixing features from both encoders delivered consistently robust performance across benchmarks. In memory terms, the fixed backbone plus streaming statistics occupied less space than the replay buffers of competing methods while sitting on the efficient frontier of the accuracy-versus-memory tradeoff.

The practical implications reach well beyond academic benchmarks. The Elliptic experiments, conducted on real Bitcoin transaction data with genuine timestamps in a time-incremental stream, demonstrate that the method works on realistic financial fraud detection problems where new transaction patterns emerge continuously. The same properties make the approach attractive for intrusion detection in Internet of Things networks, healthcare monitoring on temporal patient graphs, and recommendation systems, all applications the authors cite as motivation. Because the classifier updates are cheap streaming statistics and the encoder requires no training whatsoever, the method is naturally suited to deployment where latency is critical and predictions must be available at any moment. The authors caution that their conclusions are specific to the online setting, that a full theoretical analysis of forgetting remains future work, and that extensions to graph-level and edge-level tasks, regression and anomaly detection are still open. But the central message is already provocative: sometimes the most effective way for an artificial intelligence to remember is to stop changing its mind.

Subject of Research: Randomized fixed representations for mitigating catastrophic forgetting in online continual graph learning

Article Title: The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learning

Article References: Donghi, G., Zambon, D., Pasa, L., Alippi, C., & Navarin, N. (2026). The Unreasonable Effectiveness of Randomized Representations in Online Continual Graph Learning. Machine Learning, 115(9), Article 220. https://doi.org/10.1007/s10994-026-07128-5

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07128-5

Keywords: continual learning, graph neural networks, catastrophic forgetting, online learning, randomized representations, streaming linear discriminant analysis, node classification, experience replay, graph representation learning, concept drift, machine learning, Bitcoin fraud detection

The research, led by Sayanta Adhikari, Vishnuprasadh Kumaravelu, and P. K. Srijith of the Bayesian Reasoning and Inference Lab at the Indian Institute of Technology Hyderabad, introduces a framework called UnCLe, short for a Hypernetwork Framework for Data-Free Unlearning and Continual Learning. The core insight is that machine unlearning—the deliberate removal of a task’s influence from a trained model—has almost always been designed with offline training in mind, where engineers retain full access to the original dataset. In continual learning, that assumption collapses. Data arrives task by task and is typically discarded after use, so when an unlearning request arrives, the original examples may simply no longer exist.

The team identified two failure modes that emerge when conventional unlearning is naively applied to continual learning environments. The first is catastrophic forgetting of retained tasks, a well-known pathology in which updating a network to remove one capability wipes out unrelated capabilities it was supposed to keep. The second is subtler and, in some ways, more troubling: catastrophic remembering, in which tasks that were supposedly unlearned resurface when the model later absorbs new information. A model that appears to have forgotten sensitive data can effectively relapse, undermining the very privacy guarantees that unlearning is meant to provide.

UnCLe attacks both problems by restructuring how the model’s parameters are produced in the first place. Instead of training a single monolithic network, the framework employs a hypernetwork—a network that generates the weights of another network—conditioned on compact task embeddings. Each task the system encounters is represented by its own embedding vector, and the hypernetwork maps that embedding to a full set of task-specific parameters. Learning a new task therefore means learning or refining an embedding, while the shared hypernetwork machinery remains stable across the entire sequence of operations.

Unlearning under this scheme becomes elegantly simple at the task level. To remove a task, the framework optimizes the hypernetwork so that, for that task’s embedding, it generates parameters that behave like noise. The generated network produces uniform, maximum-entropy outputs on the forgotten task—in other words, the model becomes maximally uncertain, exactly as if it had never seen the task at all. Crucially, this procedure does not require the original training data, which is precisely what makes it data-free. The optimization is guided by a mean squared error objective that pulls the generated parameters toward freshly sampled Gaussian noise, combined with a regularization term that anchors the hypernetwork’s outputs for all previously retained tasks, preventing collateral damage.

The choice of a noise-matching objective, rather than a direct norm penalty, turns out to matter a great deal. The authors show mathematically that averaging the squared distance to random Gaussian samples converges to the squared L2 norm of the parameters plus a constant, meaning the MSE objective implicitly drives the forgotten task’s parameters toward zero and its outputs toward a uniform distribution. But applying the L2 norm directly would, over repeated unlearning operations, drag the hypernetwork’s own shared weights toward zero and destabilize the whole system. By contrast, sampling a fresh noise target at each optimization step constrains the forget task only in distribution, acting as an implicit regularizer that preserves the shared representation. The researchers compared alternatives—including fixed noise targets, pure norm reduction, and simply discarding the task embedding—and found their approach achieved the best balance between erasing the target task and protecting retained performance.

Scaling a hypernetwork to generate all the weights of a modern convolutional backbone such as ResNet18 or ResNet50 presents its own engineering challenge, since the hypernetwork’s output layer would otherwise balloon to an impractical size. The team’s solution is chunked generation: the main network’s parameters are partitioned into roughly 200 chunks, each produced by a dedicated head of the hypernetwork conditioned on a unique chunk embedding concatenated with the task embedding. These chunk embeddings are frozen after the first task to guard against forgetting, and the final layer is split into specialized heads for weights, batch normalization parameters, and residual connection parameters, reducing redundancy and computational overhead.

The practical consequences of this design are striking. Because unlearning operates entirely in parameter space, its computational cost is dominated by the hypernetwork’s forward and backward passes and is essentially independent of dataset size and class count. The only term that grows over a sequence is the regularization over retained tasks, and it grows linearly in the number of tasks, not data points. In conventional replay-based unlearning adapted to continual settings, by contrast, the cost scales with a replay buffer whose size is difficult to budget in advance. The researchers also introduced an annealing strategy that shrinks the burn-in phase of each unlearning operation by ten percent per operation, exploiting forward transfer to cut unlearning time without degrading quality.

Empirical evaluations across sequential vision benchmarks—including Permuted-MNIST, a five-dataset suite combining MNIST, Fashion-MNIST, KMNIST, notMNIST and SVHN, CIFAR-100, and TinyImageNet—showed that UnCLe can perform long interleaved sequences of learning and unlearning requests, up to 30 operations on TinyImageNet, with minimal disruption to previously acquired knowledge. Measured against baselines including fine-tuning, retraining from scratch, and hypernetwork variants that rely on natural catastrophic forgetting, UnCLe performed on par or better across metrics, and strictly better on three of five measures with a ResNet-18 backbone. It also achieved membership inference attack accuracy closest to the ideal fifty percent, a key indicator that forgotten data is genuinely indistinguishable from never-seen data—a central concern for privacy.

Perhaps most importantly for real-world deployment, UnCLe prevented the relapse phenomenon that plagues conventional approaches: tasks unlearned by prior methods tend to creep back once new learning occurs, whereas UnCLe’s unlearned tasks stayed forgotten even as subsequent tasks were absorbed. The authors argue this has broad implications for responsible AI governance, from honoring the right to be forgotten under data protection law to stripping biased or harmful behaviors from deployed models without full retraining. At the same time, they caution that the very possibility of relapse under weaker methods underscores the need for robust verification mechanisms. With code released publicly, the framework offers a template for AI systems that can keep learning throughout their operational life while remaining accountable to demands that they forget on command.

Subject of Research: A hypernetwork framework enabling data-free machine unlearning within continual learning settings

Article Title: A Hypernetwork Framework for Data-Free Unlearning and Continual Learning

Article References: Adhikari, S., Kumaravelu, V., & Srijith, P. K. (2026). A Hypernetwork Framework for Data-Free Unlearning and Continual Learning. Machine Learning, 115(9), Article 219. https://doi.org/10.1007/s10994-026-07125-8

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07125-8

Keywords: machine unlearning, continual learning, hypernetworks, data privacy, catastrophic forgetting, task embeddings, neural networks, AI safety, right to be forgotten, ResNet, membership inference, parameter generation

The research, conducted by Fuli Qiao and Mehrdad Mahdavi of the Department of Computer Science and Engineering at Penn State, builds on Low-Rank Adaptation, or LoRA, one of the most widely used parameter-efficient fine-tuning techniques in modern artificial intelligence. Instead of retraining the billions of weights inside a large language model, LoRA freezes the original weights and injects small pairs of low-rank matrices into each layer. Only these small matrices are updated during training, which slashes memory and computation costs by orders of magnitude. The catch, the authors note, is that conventional LoRA fixes the rank to the same value across every layer and every task, a one-size-fits-all choice that leaves a crucial question unexplored: how much capacity does each layer actually need for each new task?

Qiao and Mahdavi’s answer is a method that treats rank as a dynamic resource rather than a fixed hyperparameter. Their approach, which the team calls CL-Rank, relies on a subspace similarity metric to measure how orthogonal — that is, how non-overlapping — the low-rank subspaces occupied by different tasks are within a given layer. When a new task arrives, the algorithm evaluates the geometry of the learned subspaces and adaptively increases the layerwise rank for the new task where overlap threatens to interfere with previously learned representations. By steering new knowledge into directions of parameter space that are roughly orthogonal to old knowledge, the method minimizes interference while still giving each new task enough expressive capacity to generalize well.

The underlying intuition echoes a classical idea from neuroscience and machine learning known as the stability–plasticity dilemma. A learning system must be plastic enough to absorb new information but stable enough to retain old skills. Since the pioneering work on catastrophic interference in connectionist networks in the late 1980s and the advent of modern continual learning benchmarks, researchers have tried an arsenal of remedies: replaying stored examples, regularizing important weights, growing new network branches, or constraining gradient updates to null spaces of prior tasks. Many of these approaches are either memory-hungry, brittle to task order, or impractical for models with billions of parameters. What distinguishes the new work is that it tackles the problem entirely within the cheap, low-rank adapter framework, requiring no access to old data and no freezing of the base model.

Technically, the method tracks the principal subspaces spanned by the low-rank updates in each layer and computes a similarity score between the subspace of an incoming task and those of earlier tasks. If the new task’s gradient updates are projected into directions that overlap heavily with prior subspaces, the rank of the adapter in that layer is expanded, giving the optimizer room to find solutions that spare old representations. Layers whose subspaces remain naturally disjoint need no expansion, so the parameter budget is spent only where it matters. The paper’s supplementary analyses reveal that the learned rank distributions differ markedly across layers and modules — for example, the variation in ranks among the value projection modules of encoder layers is larger than among query projections, while encoder layers as a whole show more consistent rank allocations than decoder layers. This suggests that different parts of a transformer genuinely serve distinct roles, and that a uniform rank silently wastes capacity in some layers while starving others.

To test the idea, the researchers ran experiments on T5 and Llama-2-7b language models, using GPU servers with DeepSpeed for efficient training. For T5 experiments they employed four NVIDIA A6000 GPUs with a learning rate of 1e-3 and a batch size of 32, while the larger Llama-2-7b runs used four NVIDIA A100 GPUs at a learning rate of 1e-4. The evaluation spanned standard natural language processing continual learning benchmarks built from 15 datasets, including the classic text classification benchmark of Zhang and colleagues, the GLUE and SuperGLUE suites, and the IMDB movie review corpus, with six different task sequence orders to control for ordering effects. The team also pushed into harder territory with challenging mathematical reasoning benchmarks, GSM8K and MATH, each split into sequential tasks and tested in both orders to probe how task sequencing shapes forgetting.

The results show that the adaptive-rank method matches or beats strong baselines on three fronts at once: it mitigates forgetting of earlier tasks, improves accuracy on the tasks being learned, and preserves strong generalization to unseen data — all in a memory-efficient manner. On the math benchmarks, where sequential training on GSM8K and MATH typically causes steep declines in earlier-task accuracy, the method with orthogonal projection consistently outperformed the O-LoRA baseline in final average testing accuracy, forgetting less on the first task while generalizing better on the second. Notably, the study observes that fully fine-tuning a T5-large model on MATH yields only about 3.0 percent accuracy, and about 4.2 percent on GSM8K, consistent with prior work — underscoring how difficult these mathematical tasks are and how much room remains for continual learning approaches that squeeze more capability out of small adapters.

Why should this matter beyond the benchmark tables? Continual learning is arguably the missing ingredient between today’s static AI assistants and the adaptive, lifelong learning systems that both researchers and companies envision. Every time a deployed model needs a new skill — a new language, a new domain, a new tool — retraining it from scratch or even fully fine-tuning it is prohibitively expensive. Adapter-based continual learning offers a path to modular, on-demand skill acquisition, but only if adding skills does not corrupt existing ones. By showing that the humble LoRA rank, tuned per layer and per task, is a powerful lever for balancing stability and plasticity, the Penn State work gives practitioners a new, inexpensive knob to turn. Because the method requires no stored examples of past tasks, it also sidesteps the privacy and storage concerns that plague replay-based approaches.

The findings also carry a broader scientific message: the internal geography of large models is far from uniform, and adaptation schemes that respect that heterogeneity can outperform uniform ones. The observation that encoder and decoder layers, and even the query and value modules within them, prefer different rank allocations suggests that future parameter-efficient methods may benefit from treating adapters as structured, layer-aware components rather than interchangeable plug-ins. The authors have released their code publicly, and all datasets used in the experiments are openly available, making the results straightforward for other groups to verify and extend. The work was partially supported by the National Science Foundation under award EFMA-2318101, with experiments designed and conducted by Fuli Qiao under the supervision of Mehrdad Mahdavi.

As large language models continue to spread through science, medicine, and industry, the question of how to teach them new things without breaking old ones is shifting from academic curiosity to engineering necessity. This study does not solve continual learning outright — no single method does — but it demonstrates that a careful, geometry-aware allocation of low-rank capacity can deliver competitive anti-forgetting performance at a fraction of the cost of full retraining. In a field where progress often comes from scaling up, it is a reminder that sometimes the most powerful improvements come from scaling the right things in the right places, one layer at a time.

Subject of Research: Adaptive layerwise LoRA ranks for continual learning in large language models

Article Title: Learning Without Forgetting for Continual Learning in LLMs Through Adaptive LoRA Ranks

Article References: Qiao, F., & Mahdavi, M. (2026). Learning Without Forgetting for Continual Learning in LLMs Through Adaptive LoRA Ranks. Machine Learning, 115(9), Article 212. https://doi.org/10.1007/s10994-026-07145-4

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07145-4

Keywords: continual learning, large language models, LoRA, catastrophic forgetting, parameter-efficient fine-tuning, adaptive ranks, subspace orthogonality, stability-plasticity, GLUE, GSM8K, MATH, T5

The work, published in the journal Neural Computing and Applications by Parvati Bhurani, Satyendra Singh Chouhan and Namita Mittal, addresses one of the most stubborn problems in applied machine learning, known formally as catastrophic forgetting. First documented in the late 1980s by psychologists studying connectionist networks, the phenomenon describes what happens when a neural network trained sequentially on multiple tasks loses proficiency on earlier tasks as it absorbs new ones. In the context of network security, this is not an academic curiosity. An intrusion detection system that forgets how to recognize a denial-of-service flood because it has just been taught to spot a novel botnet signature is a system that has become a liability, not a safeguard.

The framework the Indian team proposes falls under an emerging learning paradigm known as few-shot class-incremental learning, or FSCIL. The idea is to structure the learning problem so that a model first learns a broad set of base classes from a fully labeled dataset, and then progressively incorporates novel classes from just a few labeled samples per class, all without revisiting the original training data. This mirrors the operational reality of cybersecurity. Organizations typically possess abundant examples of well-known attacks, but when a new exploit appears in the wild, security teams may have only a few confirmed instances of it before the next wave of probes arrives. A detection model suited to this environment must therefore extract maximum information from minimal new evidence while keeping its existing knowledge intact.

BiPC-IFS achieves this balance through two core components: a fixed feature extractor and a prototype calibration module. The feature extractor is trained only during the base session, on the well-populated set of established attack classes, and is then frozen for the remainder of the system’s operational life. Although this might seem restrictive, the researchers found that the frozen extractor still captures meaningful similarity relationships between the base classes and the novel classes that arrive later. Because the extractor encodes the geometry of network traffic in a stable feature space, new attack types can be located within that space even when only a handful of examples exist, simply by measuring where their feature representations fall relative to everything the model already knows.

The second component, prototype calibration, is where the approach earns its distinctive name. In prototype-based classification, each class is represented by a single representative vector, or prototype, typically computed as the mean of the feature vectors of its training samples. With only a few samples, these novel-class prototypes are biased, pulled away from their true class centers by sampling noise and by the tendency of a model trained on base classes to interpret everything through the lens of what it already knows. Calibration corrects this bias by adjusting the prototypes before classification. The crucial design question, the authors note, is determining how much to adjust: a calibration factor that is too high can distort the original representation of the novel class, effectively overcorrecting and making the system worse than it would have been with no calibration at all.

What sets BiPC-IFS apart from earlier calibration techniques is the way it computes that correction. Rather than relying solely on distances in feature space, the proposed calibrated class prototype aggregates both feature-based similarity and semantic similarity among different classes. In practical terms, this means the system considers not only how close a novel attack’s samples sit to the prototypes of known attacks in the learned feature space, but also how conceptually related the classes are. Two attack types that share characteristics, for example variants of the same malware family, can inform each other’s prototypes in a way that purely geometric calibration cannot achieve. This dual-source aggregation allows the model to draw richer inferences from the sparse evidence available in each incremental session, producing prototypes that better represent the true structure of the new classes.

To test whether these design choices translate into real-world performance, the researchers evaluated BiPC-IFS on two of the most widely used benchmark datasets in intrusion detection research: UNSW-NB15 and CICIDS2017. The UNSW-NB15 dataset, created at the Australian Centre for Cyber Security, combines real normal traffic with nine categories of synthesized modern attacks, including backdoors, exploits, and reconnaissance activity. CICIDS2017, produced by the Canadian Institute for Cybersecurity, captures several days of benign and attack traffic covering brute-force assaults, heartbleed exploits, botnets, denial-of-service attacks, web attacks, and infiltration attempts. Together, these benchmarks provide a demanding testbed, with realistic class distributions and attack behaviors that differ substantially across categories, exactly the conditions under which incremental learning systems tend to falter.

The results were striking. BiPC-IFS surpassed the baseline methods it was compared against and achieved the strongest performance metrics for novel classes across both datasets. The system recorded an average accuracy of 94.91 percent across all incremental sessions, a novel class accuracy of 74.25 percent, and a performance drop, measured as the decline in accuracy over the course of learning new classes, of just 8.67 percent. That final figure is the one that matters most to security practitioners, because it quantifies how much the system forgets as it learns. A small drop means that the model’s knowledge of old attacks remains largely intact even as it absorbs new ones, which is precisely the property that conventional retraining pipelines fail to deliver.

The implications extend well beyond one laboratory result. Networks today face an adversary that evolves continuously, probing for unpatched vulnerabilities and mutating attack tooling faster than human analysts can label large datasets. Systems like BiPC-IFS point toward a generation of defenses that can be updated on the fly, in operational settings, without the downtime and cost of full retraining and without the silent erosion of previously learned protections. Because the feature extractor remains frozen, the computational cost of incorporating a new attack class is minimal, and the approach avoids the need to store sensitive raw traffic data from past sessions. The researchers also note that the datasets used in the study are publicly available, which should make it straightforward for other teams to reproduce the results and build on them.

There remain, of course, open questions. The frozen feature extractor, though shown to capture useful base-novel similarity, was never trained to see the novel classes, and future work may explore how well this holds as threat landscapes diverge further from historical attack patterns. The authors’ own framing acknowledges the delicate trade-off at the center of the method: the calibration factor must be chosen carefully, since too aggressive a correction distorts the very representations it is meant to refine. Even so, the demonstration that biased prototype calibration, informed jointly by feature and semantic similarity, can push novel-class detection to over 74 percent accuracy from just a few examples marks a meaningful advance. As artificial intelligence becomes the front line of network defense, techniques that let models learn like analysts do, quickly, from limited evidence, and without forgetting hard-won lessons, may prove indispensable.

Subject of Research: A biased prototype calibration framework for incremental few-shot learning in network intrusion detection systems.

Article Title: BiPC-IFS: Biased Prototype Calibration For Incremental Few Shot Intrusion Detection

Article References: Bhurani, P., Chouhan, S. S., & Mittal, N. (2026). BiPC-IFS: Biased Prototype Calibration For Incremental Few Shot Intrusion Detection. Neural Computing and Applications, 38(17), Article 736. https://doi.org/10.1007/s00521-026-12455-8

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12455-8

Keywords: intrusion detection, few-shot class-incremental learning, catastrophic forgetting, prototype calibration, machine learning, cybersecurity, network security, semantic similarity, UNSW-NB15, CICIDS2017, neural networks, continual learning