AI Cybersecurity Defenses Look Great on Paper but Falter in the Real World

Deep learning has become the poster child of modern network defense. Neural networks now sit inside intrusion detection systems that guard Internet of Things devices, industrial control networks, software-defined networking infrastructure, and next-generation internet backbones, and the published results look dazzling: accuracy figures of 98, 99, even 99.9 percent appear routinely in the literature. But a new mini review published in Frontiers in Communications and Networks argues that these headline numbers conceal a troubling truth. When researchers at Multimedia University in Malaysia examined 29 empirical studies published between 2024 and 2026, they found that not a single one evaluated all three capabilities that matter most for real-world deployment: generalizing to new datasets, recognizing entirely unseen attacks, and remaining statistically reliable under uncertainty. The finding strikes at the heart of how the cybersecurity community judges whether an AI defender is actually ready for the field.

The core problem, according to the review by Kai Liang Lew, Kok Swee Sim, and Hao Xian Cheong, is what researchers call closed-set evaluation. Most intrusion detection models are trained and tested on the same dataset, under the assumption that every network flow they encounter will belong to one of the attack categories seen during training. That assumption collapses the moment a model meets live traffic. Real networks experience distribution shifts as devices come and go, protocols evolve, and traffic patterns change with the seasons. Worse, attackers constantly invent new techniques, meaning a deployed system will inevitably face attack families that were never in its training data. A model that cannot tolerate these conditions may look invincible in the lab and fail silently in production.

The first deployment gap the reviewers examined is cross-dataset generalization, the ability of a model trained on one collection of traffic to perform on another. The evidence here is sobering. One study reported that macro-F1 scores, which balance precision and recall across attack classes, fell from a range of 0.84 to 0.96 under in-domain testing to just 0.69 to 0.78 when the model was tested on a harmonized future internet IoT dataset, accompanied by a benign false-positive rate of roughly 0.30. That last figure deserves particular attention: because benign flows vastly outnumber malicious ones in real traffic, even a modest false-positive rate on normal traffic can flood security analysts with more false alerts than genuine detections. Another study found that while 9 of 16 within-dataset experiments achieved Matthews correlation coefficients above 99.6 percent, the average MCC plummeted from 94.63 percent to 29.35 percent under cross-dataset testing, a near-total loss of discriminative power.

Why do models that excel on their home turf fall apart elsewhere? The review identifies three culprits. First, the tools used to extract network features such as timeouts and packet counts apply different rules in different environments, so the feature data from two networks may look radically different even when the underlying behavior is similar. Second, deep networks are prone to shortcut learning: instead of learning generalizable signatures of malicious behavior, they latch onto benign traffic footprints specific to a dataset, including IP subnets, operating system artifacts, and active ports. Third, the alignment methods designed to bridge different data sources can trigger negative transfer, actively degrading accuracy when the source and target domains are too dissimilar. Researchers have tried to close these gaps with techniques like Wasserstein distance-guided feature alignment, few-shot multidomain fusion, and class-incremental adaptation, and these approaches do shrink the performance drop, but residual sensitivity to domain shift persists across the literature.

The second gap is open-set recognition, and it may be the most dangerous. A closed-set classifier is forced to assign every flow to one of its known labels, which means a genuinely novel attack, say a zero-day exploit, gets confidently mislabeled as something familiar or dismissed as benign. Some studies have begun addressing this. Souza and colleagues built an energy-based flow classifier with a dedicated suspicious class rather than forcing every flow into a known attack label, achieving an area under the receiver operating characteristic curve of 0.859 on a reproduced CICIDS2017 open-set benchmark. Others have combined unknown-attack handling with cross-dataset evaluation, testing whether rejection mechanisms survive when both the labels and the traffic distribution change simultaneously. Yet the review found that most open-set work still relies on single-dataset holdout protocols, leaving open the question of whether an open-set detector remains trustworthy when the network itself shifts underneath it.

The third gap is reliability, and it is perhaps the most counterintuitive. High accuracy does not guarantee that a model’s confidence is meaningful. A calibrated model’s stated probability should match its actual likelihood of being correct; an overconfident model that says 99 percent when it is right only 70 percent of the time is a liability, especially when automated responses hang on those numbers. The reviewed studies deployed an arsenal of reliability techniques: Bayesian deep learning replaces fixed weights with distributions so predictions carry uncertainty, Monte Carlo dropout runs dropout at inference time to estimate variability, and abstention methods allow a model to refuse a prediction when confidence is too low. One study using Bayesian autoencoder uncertainty achieved an area under the rejection curve of 0.986 on UNSW-NB15, while another’s Wisdomnet approach drove accepted-sample error to near zero at rejection rates of roughly 5 to 7 percent. But here again, the reviewers found a pattern: calibration and abstention were typically measured only on in-distribution traffic, and calibration can degrade badly when data shifts or new attacks appear, leaving the model overconfident precisely when it matters most.

The comparative analysis reveals a fragmented landscape. Of the 29 empirical works, some tackled cross-dataset transfer, some open-set rejection, some calibration and uncertainty, and some practical deployment concerns like federated learning, edge computing, and poisoning robustness. A handful combined two of the three core concerns: one Transformer-based intrusion detection system paired cross-dataset evaluation with calibration analysis, reporting AUC improvements of roughly 10 to 20 percent and F1 gains of more than 10 points over baselines, while open-set domain adaptation studies linked unknown-attack handling with cross-domain variation. But no study delivered all three under a single unified protocol. The reviewers also graded study quality, flagging issues like test-informed threshold selection, unclear data splits, missing repeated-run uncertainty, and inconsistent reporting, and their confidence ratings ranged from higher to limited depending on protocol clarity and leakage control.

Reproducibility adds another layer of concern. Training-testing splits, label harmonization rules, and held-out attack designs vary inconsistently across the corpus, making direct comparison between studies nearly impossible. Dataset coverage is narrow as well: most results rest on a small set of well-worn benchmarks such as CIC-IDS2017, UNSW-NB15, and NSL-KDD, while newer internet traffic remains underused. The reviewers caution that their reported numbers should be read as within-study evidence rather than rankings, since differences in feature harmonization, source-target pairing, openness regimes, and threshold selection confound any cross-study comparison. In other words, the field’s benchmark culture, optimized for chasing leaderboard accuracy, is structurally ill-suited to answering the question that actually matters: will this model protect a real network?

To fix this, the authors propose a joint deployment-oriented evaluation protocol built on a four-step pipeline. First, a model undergoes offline training and post hoc calibration to establish baseline confidence curves. Second, it runs inline at network gateways, predicting labels and emitting calibrated confidence scores for incoming flows. Third, dual-path decision routing sends high-confidence predictions to automation while routing low-confidence cases and open-set detections to human analysts or deep-packet inspection tools. Fourth, flagged anomalies are periodically labeled and fed back through class-incremental updates and recalibration cycles. The protocol also specifies fixed source-target dataset pairs, the withholding of at least one attack family from training with explicit reporting of openness ratios, and calibration and abstention metrics such as expected calibration error, Brier score, and risk-coverage curves computed on shifted and unknown-attack cases, not just familiar traffic. All thresholds would be selected using only training or validation data before held-out testing.

The message for the cybersecurity industry is blunt but constructive. The limitation of current AI intrusion detection is not weak models but weak evaluation: without cross-dataset testing, dataset-specific overfitting cannot be ruled out; without open-set testing, novel zero-day attacks get forced into known labels, creating security blind spots; and without calibration, models make overconfident, incorrect calls on shifted traffic. The three checks together form the minimum evidence base for deployment readiness, though even they do not capture everything, since operational readiness also involves latency, throughput, resource consumption, temporal drift, adversarial robustness, and analyst workload. Until the field adopts joint protocols that stress-test generalization, novelty, and reliability simultaneously, the eye-popping accuracy figures filling the literature will remain what the Malaysian team suggests they are: impressive numbers about a world that no deployed system will ever actually inhabit.

Subject of Research: Deployment-oriented evaluation of deep learning network intrusion detection systems

Article Title: Deployment-oriented evaluation of deep learning network intrusion detection systems

Article References: Deployment-oriented evaluation of deep learning network intrusion detection systems. (n.d.). https://doi.org/10.3389/frcmn.2026.1863367

Image Credits: AI Generated

DOI: 10.3389/frcmn.2026.1863367

Keywords: network intrusion detection, deep learning, cross-dataset generalization, open-set recognition, calibration, uncertainty quantification, IoT security, IIoT, software-defined networking, domain shift, zero-day attacks, cybersecurity evaluation