<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>model poisoning &#8211; Science</title>
	<atom:link href="https://scienmag.com/tag/model-poisoning/feed/" rel="self" type="application/rss+xml" />
	<link>https://scienmag.com</link>
	<description></description>
	<lastBuildDate>Mon, 12 Oct 2026 18:27:00 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg</url>
	<title>model poisoning &#8211; Science</title>
	<link>https://scienmag.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">73899611</site>	<item>
		<title>New Training Trick Lets Federated AI Forget a Client Instantly and Perform Better</title>
		<link>https://scienmag.com/new-training-trick-lets-federated-ai-forget-a-client-instantly-and-perform-better/</link>
		
		<dc:creator><![CDATA[Veronica Carney]]></dc:creator>
		<pubDate>Mon, 12 Oct 2026 18:27:00 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[Bi-Models and Multi-Models Training methods]]></category>
		<category><![CDATA[client data removal in federated AI]]></category>
		<category><![CDATA[collaborative machine learning without data sharing]]></category>
		<category><![CDATA[Data Privacy]]></category>
		<category><![CDATA[enhancing federated learning security]]></category>
		<category><![CDATA[exact unlearning]]></category>
		<category><![CDATA[federated learning]]></category>
		<category><![CDATA[federated learning privacy preservation]]></category>
		<category><![CDATA[federated protocols]]></category>
		<category><![CDATA[GDPR]]></category>
		<category><![CDATA[Huffman coding]]></category>
		<category><![CDATA[improving federated model accuracy after forgetting]]></category>
		<category><![CDATA[influence trees]]></category>
		<category><![CDATA[instant client influence removal]]></category>
		<category><![CDATA[legal data erasure in AI systems]]></category>
		<category><![CDATA[LLaMA 3.2]]></category>
		<category><![CDATA[Machine Learning journal]]></category>
		<category><![CDATA[machine unlearning]]></category>
		<category><![CDATA[model forgetting algorithms in federated systems]]></category>
		<category><![CDATA[model poisoning]]></category>
		<category><![CDATA[model update aggregation and influence control]]></category>
		<category><![CDATA[post-unlearning performance]]></category>
		<category><![CDATA[preventing data poisoning in federated learning]]></category>
		<category><![CDATA[privacy regulations and AI compliance]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=265623</guid>

					<description><![CDATA[Researchers have developed two federated learning algorithms, Bi-Models Training and Multi-Models Training, that remove a client's influence from a shared model instantly and provably while improving post-unlearning accuracy by more than twofold.]]></description>
										<content:encoded><![CDATA[<p>Federated learning has become one of the most widely deployed paradigms for collaborative machine learning, allowing hospitals, banks, and phone manufacturers to train a shared model without ever moving raw data off their premises. Clients exchange only model updates such as gradients or parameters with a central server, which aggregates them into a global model. Yet this privacy-preserving design creates a stubborn problem: once a client&#8217;s updates have been woven into the shared model over many communication rounds, that client&#8217;s influence is baked in. Removing it, whether because of a legal erasure request or a discovered poisoning attack, has traditionally meant retraining everything from scratch. A new study published in the journal Machine Learning by Ze Yu Zhang, Nhung Bui, Arun Verma, Bolin Ding, and Bryan Kian Hsiang Low proposes an elegant escape from this dilemma, introducing two algorithms called Bi-Models Training and Multi-Models Training that promise exact forgetting by design, without sacrificing the model&#8217;s accuracy afterward.</p>
<p>The stakes are higher than they might first appear. Privacy regulations such as the European Union&#8217;s General Data Protection Regulation grant individuals a Right to be Forgotten, and similar obligations exist under the California Consumer Privacy Act. When a participant in a federated system demands erasure, the operator cannot simply delete the client&#8217;s local files; the fingerprints of that data persist in the aggregated global model. Membership inference attacks and gradient reconstruction attacks have demonstrated that such residual traces can leak sensitive information. In security-sensitive settings the problem is even more acute: a single malicious or poisoned client can corrupt a jointly trained fraud detection model, and eliminating its influence becomes essential to restoring the model&#8217;s reliability. In both scenarios, the removal must be complete, not approximate.</p>
<p>The gold-standard solution, retraining from scratch with the remaining clients, does achieve exact unlearning, because a model trained only on the surviving data is provably independent of the departed client&#8217;s dataset. But the cost is prohibitive. Federated training typically runs for hundreds to a thousand communication rounds, and restarting the process introduces significant deployment delays. Worse, the random initialization that begins a fresh training run can lead to severe performance degradation, particularly when the remaining client population is small or heterogeneous. For time-sensitive applications, from fraud detection to medical imaging, waiting days for a retrained model is simply not an option. This gap between the guarantees of retraining and the practical need for immediacy is precisely where the new work positions itself.</p>
<p>The first of the two proposed algorithms, Bi-Models Training, takes a surprisingly simple approach. During the normal federated training process, each client maintains an isolated local copy of the model that is trained only on its own data, alongside its participation in the shared global training. These local copies are never mixed with anyone else&#8217;s data, so their influence sets contain exactly one client each. When a client later requests to be forgotten, the server simply discards every stored state influenced by that client and uses the remaining clients&#8217; isolated local models to provide a strong initialization for the post-unlearning global model. Instead of starting from random weights, the recovered model starts from weights that already encode useful knowledge learned by the surviving participants, dramatically shortening the recovery phase and improving final accuracy.</p>
<p>Bi-Models Training, however, treats each client in isolation, which means it cannot capture how the influence of multiple clients is jointly entangled in the global model. This is where Multi-Models Training, the paper&#8217;s centerpiece, departs from convention. Rather than training a single global model, the server orchestrates a carefully constructed set of sub-federated models, each trained on a disjoint subset of clients. Because the subsets do not overlap, no client&#8217;s influence spreads across multiple sub-models, and the influence structure of the entire system can be represented as a tree whose leaves are the individual clients. When a client must be removed, the algorithm identifies the most influential models that remain unaffected by that client and aggregates them to reinitialize the global model, guaranteeing that the departed client&#8217;s data plays no role in the result.</p>
<p>One of the most striking theoretical contributions of the paper is the connection it draws between the design of these influence trees and a classic result from information theory. The authors prove that the expected number of models that must be aggregated after an unlearning event, a quantity they call the influence-dependent storage cost, is minimized when the tree of sub-models is constructed according to Huffman coding, the same greedy algorithm used to build optimal prefix-free compression codes. In other words, the optimal way to partition clients into disjoint training groups mirrors the optimal way to assign short binary codewords to frequent symbols. This unexpected bridge between compression theory and federated unlearning gives the method a principled foundation rather than a heuristic one, and it allows the system to exploit known probabilities about which clients are likely to request erasure.</p>
<p>The guarantees of exactness are established formally. The authors model training as a composition of deterministic primitives, including initialization, copying, optimizer resets, local training on a dataset, and server aggregation. They define the influence set of any model state as the set of clients whose data could have affected it, and they prove that a model state&#8217;s value depends on the training data only through this influence set. The unlearning procedure then discards every stored state whose influence set contains the departing client, aggregates the retained states, and continues training only on the remaining clients&#8217; data. Because the departing client is never reintroduced into any influence set, the final deployed model is provably invariant to any replacement of that client&#8217;s dataset, which is precisely the formal definition of exact unlearning. Notably, the framework is compatible with arbitrary federated learning protocols, so it does not require modifying the underlying aggregation rules.</p>
<p>Memory efficiency is another decisive advantage. An earlier exact approach, Exact-Fun, stores auxiliary models for every communication round, and with typical federated training spanning one hundred to one thousand rounds this becomes extremely memory-inefficient, and in the authors&#8217; experiments with Llama-3.2 language models, outright infeasible. By contrast, the storage cost of both Bi-Models Training and Multi-Models Training depends only on the number of clients, making them well suited to cross-silo deployments involving ten to one hundred participants, such as consortia of hospitals or financial institutions. The authors even arranged their evaluation order to disadvantage their own method, ensuring that the global model could not simply be replaced by a single sub-model and had to be reconstructed through aggregation, a choice that strengthens the credibility of their reported gains.</p>
<p>The empirical results span both vision and language tasks. On image classification benchmarks including CIFAR-10, Fashion-MNIST, and datasets evaluated with VGG-style convolutional networks, as well as language tasks including language identification and multilingual sentiment classification drawn from public Hugging Face datasets and experiments fine-tuning Llama-3.2, Bi-Models Training consistently outperformed existing exact unlearning baselines. Multi-Models Training went further, achieving more than a twofold improvement in post-unlearning performance while maintaining the exact unlearning guarantee. The comparison baselines included approaches such as FedEraser, Exact-Fun, and federated unlearning via weight negation, alongside the retraining-from-scratch reference point. The consistent pattern across tasks suggests that the benefit stems from the initialization strategy itself rather than from any quirk of a particular dataset or architecture.</p>
<p>The implications reach well beyond the immediate technical contribution. As federated learning spreads through healthcare, open banking, mobile keyboards, and recommendation systems, the ability to honor erasure requests instantly, and to surgically excise poisoned participants from security-critical models, is becoming a baseline requirement rather than a luxury. The work also highlights a subtle vulnerability: unlearning systems themselves can be targets, with recent research demonstrating data reconstruction attacks specifically aimed at federated unlearning procedures. By making exactness a structural property of the training process rather than a post-hoc repair, the new algorithms close a gap that approximate methods leave open. The research, supported by Singapore&#8217;s National Research Foundation and the Singapore-MIT Alliance for Research and Technology, suggests a future in which the right to be forgotten and the demand for high-performing collaborative models are no longer in tension, but are satisfied simultaneously by design.</p>
<p><strong>Subject of Research:</strong> Exact federated unlearning algorithms that remove a client&#x27;s influence from collaboratively trained models while improving post-unlearning performance</p>
<p><strong>Article Title:</strong> MMT: Achieving Exact Federated Unlearning with Improved Post-unlearning Performance</p>
<p><strong>Article References:</strong> MMT: Achieving Exact Federated Unlearning with Improved Post-unlearning Performance. (n.d.). <a href="https://doi.org/10.1007/s10994-026-07117-8" rel="noopener noreferrer">https://doi.org/10.1007/s10994-026-07117-8</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s10994-026-07117-8" rel="noopener noreferrer">10.1007/s10994-026-07117-8</a></p>
<p><strong>Keywords:</strong> federated learning, machine unlearning, exact unlearning, data privacy, GDPR, Huffman coding, model poisoning, Llama-3.2, influence trees, Machine Learning journal, federated protocols, post-unlearning performance</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">265623</post-id>	</item>
		<item>
		<title>New Defense System Shields Federated Learning From Hidden Backdoor Attacks</title>
		<link>https://scienmag.com/new-defense-system-shields-federated-learning-from-hidden-backdoor-attacks/</link>
		
		<dc:creator><![CDATA[Veronica Carney]]></dc:creator>
		<pubDate>Sun, 11 Oct 2026 02:18:30 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[adversarial robustness]]></category>
		<category><![CDATA[AI model integrity in privacy-focused systems]]></category>
		<category><![CDATA[anomaly detection]]></category>
		<category><![CDATA[backdoor attack prevention in AI]]></category>
		<category><![CDATA[backdoor attacks]]></category>
		<category><![CDATA[Centered Kernel Alignment]]></category>
		<category><![CDATA[cybersecurity]]></category>
		<category><![CDATA[cybersecurity in distributed learning]]></category>
		<category><![CDATA[Data Privacy]]></category>
		<category><![CDATA[defenses against backdoor triggers in machine learning]]></category>
		<category><![CDATA[distributed AI system vulnerability mitigation]]></category>
		<category><![CDATA[federated learning]]></category>
		<category><![CDATA[federated learning security]]></category>
		<category><![CDATA[hidden backdoor detection in AI models]]></category>
		<category><![CDATA[isolation forest]]></category>
		<category><![CDATA[Machine learning]]></category>
		<category><![CDATA[malicious participant detection in federated training]]></category>
		<category><![CDATA[model poisoning]]></category>
		<category><![CDATA[non-IID data]]></category>
		<category><![CDATA[privacy-preserving machine learning security]]></category>
		<category><![CDATA[RFLDefense for federated systems]]></category>
		<category><![CDATA[robust aggregation]]></category>
		<category><![CDATA[safeguarding federated learning from sabotage]]></category>
		<category><![CDATA[secure federated learning frameworks]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=260846</guid>

					<description><![CDATA[Researchers at Donghua University have developed RFLDefense, a layer-wise defense framework that suppresses backdoor attacks in federated learning even when up to half of the participants are malicious.]]></description>
										<content:encoded><![CDATA[<p>Federated learning has become one of the most promising ways to train artificial intelligence without sacrificing privacy. Instead of pooling sensitive data on a central server, the technique lets hospitals, banks, and smartphone makers train a shared model by exchanging only mathematical updates. But the very feature that makes federated learning attractive—its distributed nature—also makes it dangerously vulnerable. A new study published in the journal Cybersecurity by researchers at Donghua University in Shanghai introduces a defense framework called RFLDefense that promises to keep these systems safe even when up to half of the participants are secretly working against it.</p>
<p>The threat the researchers set out to counter is known as a backdoor attack, a particularly insidious form of sabotage. Rather than trying to wreck a model outright, an attacker implants a hidden trigger into the training process. The model continues to perform normally on everyday inputs, so its accuracy statistics look healthy, but when it later encounters the specific trigger pattern, it produces whatever output the attacker designed. In a medical diagnosis system, this could mean a corrupted image is misclassified as healthy; in a financial risk model, a fraudulent profile could slip through undetected. Because the model&#8217;s overall performance barely changes, backdoors are notoriously difficult to catch with conventional monitoring.</p>
<p>The challenge is compounded by the reality of real-world data. In practical federated learning deployments, no two participants hold identical data—a condition researchers call non-IID, meaning the data are not independently and identically distributed. A hospital specializing in cardiac care sees very different patient records from one focused on oncology, so their model updates naturally diverge. Existing defenses that flag suspicious updates by looking for statistical outliers struggle in this environment, because benign updates from data-skewed clients can look just as unusual as malicious ones. Some defenses also depend on a clean validation dataset held by the server, an assumption that is often unrealistic and, worse, could itself be exploited by attackers.</p>
<p>RFLDefense takes a fundamentally different approach by examining updates at the level of individual neural network layers rather than treating each client&#8217;s contribution as a single monolithic block. The framework rests on three pillars: hybrid similarity calculation, sign consistency verification, and robust layer aggregation. The first pillar uses a technique called Centered Kernel Alignment, or CKA, to measure how closely each client&#8217;s penultimate-layer parameters align with the previous round&#8217;s global model. The authors chose CKA over simpler cosine similarity because cosine measures are insensitive to scaling—exactly the manipulation attackers use when they amplify or shrink their updates to evade detection. By comparing parameter matrices in a normalized way, CKA exposes updates whose direction deviates suspiciously from the global consensus.</p>
<p>The second pillar digs even deeper into the geometry of the updates. For each layer, the system identifies the most important parameters using a TopK selection, aggregates their signs across all clients to establish a main sign direction, and then checks how well each individual client&#8217;s parameters match that consensus. Clients whose sign patterns deviate too far are flagged using a Modified Z-score, a robust statistical measure built on the median and median absolute deviation rather than the mean and standard deviation, making it resistant to distortion by extreme values. Crucially, the method also analyzes absolute similarity among same-layer updates from different clients, because colluding attackers who share a common objective tend to produce suspiciously similar parameters—a fingerprint that honest, independently trained updates should never display.</p>
<p>The final pillar addresses a subtle problem the authors call the toxin coupling effect: even after filtering, a small residue of malicious parameters can survive and accumulate across training rounds. RFLDefense counters this with an Isolation Forest, an unsupervised anomaly detection algorithm that isolates outliers through random partitioning. Anomaly scores are mapped to adaptive aggregation weights using Hampel thresholding, so highly anomalous layer updates are smoothly down-weighted rather than abruptly discarded. The system then normalizes each weighted update to preserve only its direction, averages the directions, and restores magnitude using the median of the layer&#8217;s parameter norms. This non-discarding strategy means that even a partially suspicious client can still contribute useful information for the main task, preserving accuracy while bleeding off poison.</p>
<p>The experimental evaluation was extensive. The team tested RFLDefense on three widely used benchmarks—CIFAR-10, Fashion-MNIST, and EMNIST—splitting training data among 500 simulated clients with Dirichlet-distributed partitions to enforce realistic non-IID conditions. They pitted the defense against five state-of-the-art backdoor attacks, including label flipping, model replacement, the distributed backdoor attack that splits a trigger across multiple malicious clients, the edge-case attack that targets rare inputs, and Neurotoxin, which hides adversarial updates in subspaces that benign updates rarely touch. In the harshest scenario, attackers controlled 50 percent of participants in adversarial rounds. The results showed that RFLDefense kept main-task accuracy essentially identical to undefended training while driving the true attack contribution—measured as the increase in attack success rate over a benign baseline—close to zero.</p>
<p>One of the study&#8217;s most telling comparisons involved rival defenses under escalating attack strength. AlignIns, a recent method relying on sign consistency and cosine similarity, performed well when malicious clients made up 20 to 30 percent of the pool but collapsed near 50 percent, because its median-based statistics became contaminated by the attackers themselves. RoseAgg, which extracts a principal direction from clustered updates, was actively hijacked by Neurotoxin: the attackers&#8217; highly coordinated updates in an underused subspace were mistaken for a global consensus and amplified. RFLDefense, by contrast, held the attack success rate steady at around one percent across the entire tested range, a stability the authors attribute to the multi-stage design in which no single statistical signal can be fully spoofed.</p>
<p>The framework also proved remarkably fair to honest participants. In false positive rate tests on CIFAR-10, RFLDefense incorrectly flagged only about 5.8 percent of benign clients in attack-free training and roughly 2 to 3 percent under attack, while competing methods such as AlignIns and FLAME misclassified between 38 and 49 percent. Ablation experiments confirmed that hybrid similarity was the key ingredient for breaking attacker consensus, while sign consistency and robust aggregation added complementary stability. The authors acknowledge limits: a fully adaptive white-box adversary that preserves high CKA similarity and mimics benign sign patterns could still evade the system, and beyond 50 percent malicious participation the population statistics become unreliable. Yet within its stated threat model, the method requires no clean validation data, no client identity tracking, and no knowledge of the attack type—operating entirely server-side and attack-agnostic.</p>
<p>The implications reach well beyond the laboratory. Federated learning is already deployed in keyboard prediction on mobile devices, cross-institution medical research, and financial risk modeling, and every one of those deployments is a potential target for backdoor injection. A defense that tolerates data heterogeneity, resists collusion, and avoids punishing honest but unusual clients addresses the three most common failure modes of current protections. The computational cost is moderate: a lightweight variant of the framework averaged about 15.8 seconds per training round in wall-clock tests, compared with roughly 10 seconds for AlignIns and 13.4 seconds for RoseAgg—a price the authors argue is justified by the substantial gains in robustness. As machine learning continues to spread into privacy-sensitive domains, techniques like RFLDefense suggest that the trade-off between collaboration and security may not be as unavoidable as it once seemed.</p>
<p><strong>Subject of Research:</strong> Defending federated learning systems against backdoor and targeted poisoning attacks</p>
<p><strong>Article Title:</strong> RFLDefense: robust federated learning defense against targeted attacks</p>
<p><strong>Article References:</strong> Shi, X., Gong, J., &amp; He, Y. (2026). RFLDefense: robust federated learning defense against targeted attacks. <em>Cybersecurity, 9</em>(1), Article 234. <a href="https://doi.org/10.1186/s42400-026-00679-z" rel="noopener noreferrer">https://doi.org/10.1186/s42400-026-00679-z</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1186/s42400-026-00679-z" rel="noopener noreferrer">10.1186/s42400-026-00679-z</a></p>
<p><strong>Keywords:</strong> federated learning, backdoor attacks, model poisoning, robust aggregation, cybersecurity, machine learning, non-IID data, anomaly detection, Centered Kernel Alignment, Isolation Forest, data privacy, adversarial robustness</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">260846</post-id>	</item>
		<item>
		<title>Poison-Proof AI: New Study Shows How to Keep IoT Intrusion Detectors Honest Under Attack</title>
		<link>https://scienmag.com/poison-proof-ai-new-study-shows-how-to-keep-iot-intrusion-detectors-honest-under-attack/</link>
		
		<dc:creator><![CDATA[Hailey Crawford]]></dc:creator>
		<pubDate>Fri, 09 Oct 2026 20:01:11 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[Byzantine robustness]]></category>
		<category><![CDATA[CICIoT2023]]></category>
		<category><![CDATA[decentralized machine learning]]></category>
		<category><![CDATA[differential privacy]]></category>
		<category><![CDATA[distributed AI for IoT]]></category>
		<category><![CDATA[edge computing]]></category>
		<category><![CDATA[edge device cybersecurity]]></category>
		<category><![CDATA[ESP32]]></category>
		<category><![CDATA[federated learning]]></category>
		<category><![CDATA[federated learning for intrusion detection]]></category>
		<category><![CDATA[federated learning in cybersecurity]]></category>
		<category><![CDATA[FLTrust]]></category>
		<category><![CDATA[Internet of Things]]></category>
		<category><![CDATA[intrusion detection]]></category>
		<category><![CDATA[IoT attack surface]]></category>
		<category><![CDATA[IoT botnet mitigation]]></category>
		<category><![CDATA[IoT security]]></category>
		<category><![CDATA[microcontroller-based intrusion detection]]></category>
		<category><![CDATA[model poisoning]]></category>
		<category><![CDATA[multi-Krum]]></category>
		<category><![CDATA[non-IID data]]></category>
		<category><![CDATA[privacy-preserving network security]]></category>
		<category><![CDATA[protecting IoT networks from sabotage]]></category>
		<category><![CDATA[security challenges of Internet of Things]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=255666</guid>

					<description><![CDATA[A massive 641-run study shows that trust-weighted federated learning can keep a tiny on-device IoT intrusion detector accurate under non-IID traffic and Byzantine attack, while revealing that robustness carries a clean-data premium and client-level differential privacy fails outright at this scale.]]></description>
										<content:encoded><![CDATA[<p>The Internet of Things has quietly become the largest attack surface on the planet. More than seventeen billion connected devices now sit at the edges of our networks, many of them running on microcontrollers with a few hundred kilobytes of memory and no room for conventional security software. Botnets, volumetric floods and reconnaissance scans increasingly originate at this edge rather than the network core, which is exactly where defensive visibility has always been weakest. A new open-access study in Discover Artificial Intelligence tackles a deceptively simple question: can a fleet of tiny, untrusted devices jointly learn to spot intrusions without ever shipping their raw traffic anywhere, and without being sabotaged by the very participants doing the learning?</p>
<p>The answer, according to researchers Rahul Nayak of VIT-AP University and Gondhi Navabharat Reddy of Symbiosis Institute of Technology, is a qualified yes, delivered with unusual statistical honesty. Their approach builds on federated learning, a training scheme in which each device trains a shared model on its own local data and transmits only parameter updates, never packets. That solves the bandwidth and privacy problems of centralised training, and it removes the single point of failure that a central model represents. But federated learning has two weaknesses that line up badly with the IoT setting. First, the canonical aggregation rule, FedAvg, assumes all clients draw data from a common distribution, while a camera, a thermostat and a gateway see utterly different traffic. Second, FedAvg takes an unweighted average of client updates with no scrutiny at all, so a handful of Byzantine participants, whether faulty, hijacked or openly adversarial, can quietly steer the shared model wherever they like.</p>
<p>The architecture at the heart of the study is a compact CNN–BiLSTM detector, a small hybrid of convolutional and bidirectional recurrent layers that treats each network flow&#8217;s feature vector as a short sequence. The edge configuration uses just sixteen convolutional channels and a sixteen-unit recurrent layer, amounting to roughly 6,300 parameters, about 24.7 kilobytes in 32-bit precision. The researchers trained this detector federally across twenty clients whose data was deliberately skewed using a Dirichlet partition, the standard way to simulate the non-IID reality of edge networks, and then pitted five aggregation rules against one another under three poisoning strategies: sign-flipping, in which adversaries return scaled, sign-reversed honest updates; label-flipping, in which benign and attack labels are swapped locally; and model-replacement, in which adversaries attempt to overwrite the global model with a crafted update.</p>
<p>The scale of the evaluation is what sets the work apart. Across 641 completed training runs, spanning 187 distinct configurations with five random seeds per headline configuration and three for the sweep and privacy tiers, every result is reported as a mean with a 95 percent confidence interval, and every claim of superiority is tested with a paired t-test corrected by the Holm procedure across all 64 comparisons in the family. The Matthews correlation coefficient, rather than F1 score, serves as the primary metric, and the reason is telling. A poisoned model collapses to a constant prediction, and on a balanced task that constant can be all-benign, scoring F1 of zero, or all-malicious, scoring F1 of 0.667. Two failures of identical severity would earn radically different F1 scores; MCC assigns both a deserved zero.</p>
<p>The headline result is stark. On the CICIoT2023 benchmark, every aggregation rule reaches an attack-free F1 between 0.989 and 0.990, confirming that neither the compact model nor the skewed federation is a bottleneck. But introduce a 30 percent sign-flipping adversary and plain FedAvg is annihilated: its Matthews correlation collapses from 0.979 to exactly 0.000 as the global model degenerates into a constant prediction. Three of five seeds produced a model that calls everything benign; two produced one that calls everything malicious. By contrast, trust-weighted aggregation, known as FLTrust, and the multi-Krum rule held Matthews correlations of 0.981 and 0.980 respectively, statistically indistinguishable from each other and from their own attack-free performance. The choice of aggregation rule, the authors conclude, is a security decision, not a tuning detail.</p>
<p>Perhaps the most practically valuable contribution is a sweep of the adversarial fraction from 5 to 45 percent, which locates each rule&#8217;s empirical breakdown point, defined as the smallest adversarial fraction at which mean F1 falls below 95 percent of its attack-free value. FedAvg fails at the smallest fraction tested. The coordinate-wise rules, median and trimmed-mean, both break at 20 percent, well inside the range their theoretical tolerance conditions permit, because those conditions are per-round, per-coordinate statements that say nothing about how residual bias accumulates over forty rounds. Multi-Krum breaks at 0.45, precisely the theoretical bound of (K−2)/2K for twenty clients, a rare case of practice landing exactly on theory. FLTrust showed no breakdown anywhere in the tested range, an observation the authors carefully qualify rather than inflate into an unrestricted guarantee.</p>
<p>Not every finding flatters the field. On the harder TON_IoT and Edge-IIoTset benchmarks, the strongest clean configuration was undefended FedAvg itself, at Matthews correlations of 0.838 and 0.802, while every robust rule paid a measurable accuracy premium. Robustness, the authors write, is insurance, and insurance has a premium that only pays off when an adversary actually appears. The study also corrects its own earlier version: a sampling flaw had drawn a contiguous prefix of the source archives, yielding a 91.4 percent attack-heavy sample on which a majority-class predictor scored an apparently excellent F1 while operating at a false-positive rate of 1.0. The corrected class-aware reservoir sampler scanned over 4.2 million rows to build a genuinely balanced 200,000-row sample, and the earlier perfect scores vanished.</p>
<p>The privacy results are equally uncomfortable. A formally accounted, client-level differentially private variant, composed in the Rényi differential privacy framework over forty rounds, failed at every meaningful budget. With twenty clients and full participation there is no privacy amplification by subsampling, and a closed-form noise-to-signal argument explains why: the injected noise has expected norm proportional to z times the square root of the 6,322-dimensional parameter space, roughly 79.5 times the clipped signal norm, so the update survives only at noise levels so small that the composed privacy budget becomes numerically vacuous. Every operating point measured sat at a Matthews correlation at or near zero. The authors also note a subtle interaction: FLTrust rescales updates to the root magnitude, which amplifies rather than attenuates differential-privacy noise, making the most robust aggregator the worst partner for a privacy layer.</p>
<p>The deployment claim, unusually, is a measurement rather than an extrapolation. The researchers ported the trained detector to a physical ESP32 microcontroller, bypassing a microcontroller interpreter that choked on the bidirectional recurrent layer, and instead implemented the forward pass directly in portable C, exploiting the fact that temporal average pooling means recurrent states need only be accumulated, never stored. The result runs at 23.5 milliseconds per inference, about 42.5 flows per second on a single core, using 24.7 kilobytes of flash for weights and just 5.0 kilobytes of scratch RAM, reproducing the host logits to within six parts in a hundred million. That is a per-flow rather than per-packet budget, suited to gateway-side classification, but it demonstrates that a Byzantine-resilient, federated intrusion detector can genuinely live on a two-dollar chip. The full training and evaluation pipeline, including per-round checkpointing so that multi-hour sweeps survive interruption, has been released for reproduction, and the authors point toward adaptive adversaries, partial participation and energy measurement as the next frontiers.</p>
<p><strong>Subject of Research:</strong> Byzantine-resilient federated learning for intrusion detection on resource-constrained IoT edge devices under non-IID data conditions</p>
<p><strong>Article Title:</strong> Byzantine-resilient federated edge intrusion detection for heterogeneous IoT networks under non-IID conditions</p>
<p><strong>Article References:</strong> Nayak, R., &amp; Reddy, G. N. (2026). Byzantine-resilient federated edge intrusion detection for heterogeneous IoT networks under non-IID conditions. <em>Discover Artificial Intelligence, 6</em>(1), Article 1409. <a href="https://doi.org/10.1007/s44163-026-02405-7" rel="noopener noreferrer">https://doi.org/10.1007/s44163-026-02405-7</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44163-026-02405-7" rel="noopener noreferrer">10.1007/s44163-026-02405-7</a></p>
<p><strong>Keywords:</strong> federated learning, intrusion detection, Internet of Things, Byzantine robustness, non-IID data, edge computing, differential privacy, FLTrust, multi-Krum, ESP32, model poisoning, CICIoT2023</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">255666</post-id>	</item>
		<item>
		<title>Hidden Switches: New Attack Plants Undetectable Backdoors in Vision Transformers</title>
		<link>https://scienmag.com/hidden-switches-new-attack-plants-undetectable-backdoors-in-vision-transformers/</link>
		
		<dc:creator><![CDATA[Blake Davidson]]></dc:creator>
		<pubDate>Wed, 30 Sep 2026 23:11:11 +0000</pubDate>
				<category><![CDATA[Technology and Engineering]]></category>
		<category><![CDATA[adversarial attacks on computer vision models]]></category>
		<category><![CDATA[adversarial machine learning]]></category>
		<category><![CDATA[AI safety]]></category>
		<category><![CDATA[backdoor attack]]></category>
		<category><![CDATA[backdoor attack in machine learning]]></category>
		<category><![CDATA[backdoor detection in vision transformers]]></category>
		<category><![CDATA[covert backdoor triggers in deep learning]]></category>
		<category><![CDATA[cybersecurity]]></category>
		<category><![CDATA[cybersecurity threats in AI]]></category>
		<category><![CDATA[deep learning]]></category>
		<category><![CDATA[deep learning model robustness]]></category>
		<category><![CDATA[hidden switches in neural networks]]></category>
		<category><![CDATA[high-stakes AI application vulnerabilities]]></category>
		<category><![CDATA[image classification]]></category>
		<category><![CDATA[machine learning security]]></category>
		<category><![CDATA[medical imaging model security]]></category>
		<category><![CDATA[model integrity in autonomous systems]]></category>
		<category><![CDATA[model poisoning]]></category>
		<category><![CDATA[prompt tuning]]></category>
		<category><![CDATA[self-attention]]></category>
		<category><![CDATA[self-attention mechanism exploitation]]></category>
		<category><![CDATA[supply chain security]]></category>
		<category><![CDATA[vision transformer security vulnerabilities]]></category>
		<category><![CDATA[Vision Transformers]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=219986</guid>

					<description><![CDATA[Researchers have demonstrated a switchable backdoor attack that injects dual tokens through the layers of Vision Transformers, achieving up to 99 percent attack success while leaving clean accuracy intact.]]></description>
										<content:encoded><![CDATA[<p>Vision Transformers have quietly become the backbone of modern computer vision, powering everything from image classifiers and object detectors to medical imaging pipelines and autonomous driving prototypes. Their self-attention mechanism, which lets the model weigh relationships between every patch of an image, has displaced convolutional networks in many high-stakes applications. But a new study from researchers at the University of Information Technology, Ho Chi Minh City, and Vietnam National University Ho Chi Minh City suggests that the very architecture making these models so powerful may also make them dangerously easy to compromise. In a paper published in the International Journal of Machine Learning and Cybernetics, Dung Minh Do and Khang Nguyen describe a backdoor attack that achieves attack success rates of up to 99 percent while leaving the model&#8217;s performance on clean, unmodified images essentially untouched.</p>
<p>Backdoor attacks are among the most insidious threats in machine learning security. Unlike adversarial examples, which exploit a model at inference time by perturbing individual inputs, a backdoor is baked into the model itself during training or fine-tuning. A model with a backdoor behaves perfectly normally on ordinary data, matching the accuracy and reliability its developers expect. But when a specific trigger, a subtle pattern chosen by the attacker, appears in an input, the model&#8217;s behavior flips, producing whatever output the attacker has designated. For a deployed system, this means an adversary could, for example, cause a traffic sign classifier to misread a stop sign or a security screening system to wave through a prohibited item, all without any visible degradation in everyday performance that might raise alarms.</p>
<p>What makes the new attack notable is where it operates. Rather than modifying the weights of the transformer or poisoning the training labels in conventional ways, the method injects two specially designed tokens into the model&#8217;s token stream, progressively working from the shallowest layers to the deepest ones. Vision Transformers process images by splitting them into patches, embedding each patch as a token, and then passing those tokens through a stack of transformer blocks in which self-attention layers let tokens exchange information. By inserting crafted tokens at multiple depths, the attack ensures that the malicious signal is reinforced and refined as it travels through the network, rather than being diluted or overwritten by the model&#8217;s normal processing.</p>
<p>The progressive, layer-by-layer nature of the injection is central to the attack&#8217;s stealthiness. Defenses that inspect a single layer, or that look for anomalous attention patterns at one point in the network, can miss a signal that is distributed across the entire depth of the model. Because the injected tokens interact with the image tokens through the standard attention mechanism, they can steer the model&#8217;s internal representations toward the attacker&#8217;s chosen target class only when the trigger is present. On clean inputs, the tokens remain effectively inert, which is why the authors report that the method maintains competitive clean accuracy even as it delivers near-perfect attack success rates across multiple visual datasets.</p>
<p>The word switchable in the study&#8217;s title points to another dimension of the threat. The attack draws on a growing line of research into switchable backdoors against pre-trained vision transformers, in which a single compromised model can carry multiple behaviors that an attacker can toggle. This means a defender who discovers one trigger and filters it out cannot assume the model is safe; another trigger may remain dormant, waiting to be activated. The authors position their work as an analysis and exploitation of ViT vulnerabilities, and the breadth of prior work they survey, from BadNets and weight-poisoning attacks to attention hijacking and prompt-based backdoors, underscores how rapidly this attack surface has expanded as transformers have spread through computer vision.</p>
<p>The connection to prompt tuning is particularly significant for the current state of the field. Prompt-based methods, in which small learnable tokens are prepended to a model&#8217;s input or inserted into its layers, have become a popular way to adapt large pre-trained models to new tasks without expensive full fine-tuning. Visual prompt tuning and related techniques are widely used because they are efficient and effective. But the same mechanism that makes prompts useful, namely the ability to inject learned tokens that influence the model&#8217;s attention and representations, is exactly what this attack weaponizes. A malicious actor with access to a fine-tuning pipeline, or able to distribute a poisoned adapter or checkpoint, could embed a backdoor that looks indistinguishable from a legitimate prompt-based adaptation.</p>
<p>The supply chain implications are sobering. Modern machine learning practice relies heavily on pre-trained models downloaded from public hubs, fine-tuned adapters shared between teams, and third-party datasets. Each of these channels is a potential delivery mechanism for a backdoor. Earlier research has shown that weight poisoning attacks on pre-trained models can survive downstream fine-tuning, and that backdoors can be hidden in ways that evade standard inspection. The new study adds to this picture by showing that the token-based machinery of vision transformers offers attackers a particularly clean injection point, one that does not require the crude modifications of earlier attacks that made them easier to detect.</p>
<p>The authors evaluated their method on multiple visual datasets, measuring not only attack success rate and clean accuracy but also stealthiness and robustness. The headline figure, an attack success rate of up to 99 percent, is alarming enough, but the more troubling result is the combination of that success with preserved clean performance, since it means conventional accuracy-based validation would reveal nothing amiss. The paper also situates the work against existing defenses, which include fine-pruning approaches that remove rarely activated neurons, neural attention distillation that tries to erase trigger-related attention patterns, and input-level detection methods that look for inconsistencies in a model&#8217;s predictions under image transformations. Because the new attack distributes its signal across shallow and deep layers through dual tokens, many of these defenses, which were designed with convolutional networks or single-point injections in mind, face a harder problem.</p>
<p>The researchers are explicit about the warning their findings carry for real-world deployment. Vision Transformers are increasingly used in settings where a silent failure mode could have serious consequences, including medical diagnosis support, surveillance, industrial inspection, and safety-critical perception systems. A backdoored model in any of these contexts could be remotely triggered by an input crafted to contain the attacker&#8217;s pattern, and the compromise would be invisible in routine testing. The authors make their code publicly available, which serves the defensive side of the field as well: reproducible attack implementations are essential for developing and benchmarking countermeasures, and the history of adversarial machine learning shows that security research advances fastest when attacks are fully documented.</p>
<p>For the broader community, the study is a reminder that architectural progress and security progress have been badly out of step. The references in the paper trace a decade of deep learning breakthroughs, from early convolutional networks through EfficientNet and the original Vision Transformer, alongside a parallel literature of backdoor learning surveys, prompt injection analyses, and defense proposals. Yet the authors note that security threats to ViTs, particularly backdoor attacks, have not received research attention commensurate with the architecture&#8217;s adoption. Closing that gap will likely require defenses designed specifically for token-based architectures: methods that audit injected tokens, verify the provenance of fine-tuned checkpoints, test models against a family of triggers rather than a single known pattern, and treat the entire depth of the network, not just its input layer, as a potential attack surface. Until such defenses mature, the near-perfect stealth and effectiveness demonstrated by progressive dual-token injection stands as a stark warning that the models powering tomorrow&#8217;s vision systems may harbor switches that only their attackers know how to flip.</p>
<p><strong>Subject of Research:</strong> Backdoor attack vulnerabilities in Vision Transformers via progressive dual-token injection</p>
<p><strong>Article Title:</strong> Switchable backdoor attack in vision transformers via progressive dual-token injection from shallow to deep layers</p>
<p><strong>Article References:</strong> Do, D. M., &amp; Nguyen, K. (2026). Switchable backdoor attack in vision transformers via progressive dual-token injection from shallow to deep layers. <em>International Journal of Machine Learning and Cybernetics, 17</em>(10), Article 489. <a href="https://doi.org/10.1007/s13042-026-03326-8" rel="noopener noreferrer">https://doi.org/10.1007/s13042-026-03326-8</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s13042-026-03326-8" rel="noopener noreferrer">10.1007/s13042-026-03326-8</a></p>
<p><strong>Keywords:</strong> Vision Transformers, backdoor attack, machine learning security, self-attention, prompt tuning, adversarial machine learning, model poisoning, supply chain security, deep learning, cybersecurity, image classification, AI safety</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">219986</post-id>	</item>
	</channel>
</rss>
