Fuzzy autoencoder improves containment of adversarial malware

A fuzzy autoencoder designed to identify previously unseen portable-executable malware achieved a safety-aware accuracy of 97.08% in evaluations that included synthetic adversarial samples, according to the research. The system replaces a conventional autoencoder’s single reconstruction-error cutoff with overlapping fuzzy membership functions that distinguish legitimate, suspicious, and malicious behavior. In the final test of 53,271 samples, eight of 28,170 malware samples and none of 10,000 adversarially generated samples were classified as legitimate.

The results are laboratory findings based on public malware datasets and a generator trained to imitate benign executable features, rather than a trial of the system in a live security network. The study examined five one-class machine-learning approaches—One-Class Support Vector Machine, Isolation Forest, Local Outlier Factor, Elliptic Envelope, and an autoencoder—for detecting zero-day malware without using malicious examples during training. Seven public malware datasets were analyzed, with models trained and tested independently on each one.

For every dataset, 80% of benign samples were used for training, while the remaining 20% of benign samples and 20% of malware samples formed the evaluation set. This one-class setup models the characteristics of legitimate software and treats substantial deviation from that model as anomalous. The feature set was fixed before model training and included 15 portable-executable attributes drawn from core headers, sections, and resources, such as characteristics, subsystem version, section entropy, resource entropy, image base, checksum, and version-information size.

Before model training, the researchers evaluated seven feature-selection methods—SelectKBest, Extra Trees Classifier, Recursive Feature Elimination, SelectFromModel, Mutual Information, L1-based selection, and Variance Threshold—and used majority voting to identify the feature subset. Manual and statistical comparisons of feature distributions in benign and malware samples were then used to examine the selected variables. Class labels were not used to train any of the one-class models, including the fuzzy autoencoder.

Among the five baseline classifiers, the autoencoder showed the highest overall detection performance and was therefore selected for deeper analysis. The researchers also compared simple, variational, one-dimensional convolutional, and two-dimensional convolutional autoencoders, and calculated 95% confidence intervals for their performance using 1,000 bootstrap resamples of the test set. The baseline autoencoder encoded and reconstructed 15 inputs through hidden layers containing 5, 10, 15, 10, and 5 units with ReLU activations.

It minimized mean squared reconstruction error with the Adam optimizer and initially used a fixed error threshold of 0.04 to separate legitimate from malicious samples. This configuration produced 99.64% accuracy in the reported threshold experiment, but the analysis showed that performance was highly sensitive to small changes in the cutoff. A threshold near 0.06 allowed more malware to pass as legitimate, whereas a lower threshold could flag benign files as malicious.

Across thresholds from 0.01 to 0.10, the rate at which inputs passed as legitimate rose from 90.80% to 97.30%. Increasing input perturbations also reduced accuracy, indicating that small modifications could degrade reconstruction quality and create opportunities for evasion. The researchers tested the threshold weakness in two additional ways.

First, they generated statistically plausible benign-like data using kernel density estimates of legitimate feature distributions. At the selected threshold of 0.04, about 21% of these synthetic samples bypassed the autoencoder. The passing rate rose to 49.1% at a threshold of 0.05, 85.5% at 0.07, and 98.8% at 0.10.

Second, the team trained a generative adversarial network whose fully connected generator was designed to mimic the feature distribution of genuine portable executables. Dense layers expanded from 64 to 2,048 units, with batch normalization, Adam optimization, a batch size of 64, and binary cross-entropy losses. At the 0.04 threshold, 3.20% of samples from a generator trained for 8,500 epochs passed the autoencoder, while 7.14% passed for a model trained for 8,000 epochs.

These generated samples were excluded from training and used only to assess robustness. To reduce reliance on a rigid cutoff, the proposed fuzzy autoencoder converted reconstruction error into a normalized deviation score. The score represented the difference between a sample’s mean squared reconstruction error and the mean error of legitimate test samples, divided by the legitimate standard deviation.

Triangular, overlapping membership functions modeled low, medium, and high deviation, with their parameters derived from legitimate reconstruction-error statistics. The overlap preserved uncertainty instead of imposing an abrupt boundary, while the highest-membership rule produced a nine-class space combining low, medium, and high error with weak, mild, and strong confidence. Samples assigned to weak legitimate categories received further evaluation in the second-stage fuzzy severity analysis.

The strongest low-deviation cases could be retained as legitimate, borderline cases remained suspicious, and strongly deviant inputs were assigned to the malware category. The detailed evaluation used 53,271 samples from the Meraz 18 dataset: 15,101 legitimate test samples, 28,170 malware samples, and 10,000 synthetic adversarial samples. In the initial fuzzy classification, 14,896 legitimate samples were accepted, 202 were marked suspicious, and three were classified as malware.

Of the malware samples, 27,058 entered the suspicious region, 1,104 were classified as malware, and eight were mistakenly accepted as legitimate. All 10,000 adversarial samples were placed in either the suspicious or malware regions, with none classified as legitimate. The second-stage severity analysis retained 13,556 legitimate samples, marked 1,542 as suspicious, and classified three as malware.

It also moved 24,500 malware samples and 9,999 adversarial samples into the malware category, leaving 3,662 malware samples and one adversarial sample suspicious. Using the study’s error definition, the final results yielded 1,553 critical errors, comprising 1,545 legitimate samples not retained as legitimate and eight malware samples misclassified as legitimate. This produced the reported safety-aware accuracy of 97.08%.

A separate malware safety rate counted only genuine malware wrongly accepted as legitimate, producing 99.97% because eight of 28,170 malware samples escaped the legitimate category. The adversarial result was absolute within the test: none of the 10,000 synthetic samples entered the legitimate region. An ablation study attributed 85.89% accuracy to core portable-executable header features alone, while the combined feature set reached 99.09%, indicating that section and resource attributes added information beyond the basic header fields.

Mann–Whitney U, Kolmogorov–Smirnov, and independent-samples t-tests each returned p-values below 0.05 for the reported performance differences. Cross-dataset testing of the baseline autoencoder produced accuracy above 92%, while the authors also argued that the fuzzy stage should add little deployment overhead because it follows a single neural-network forward pass with lightweight membership evaluation. The model’s complexity was expressed as the cost of network propagation plus a constant fuzzy-inference term.

Nevertheless, several limitations remain. The adversarial evaluation used one GAN-based generation strategy rather than gradient-based or malware-specific attacks, and the detailed confusion analysis centered on one dataset. The system also remained trained and tested on portable-executable files, not network traffic, system-call sequences, or behavioral logs.

In addition, the large number of samples left in the suspicious category could increase sandbox-analysis workloads in security operations centers. The researchers proposed future work on adaptive membership functions, confidence calibration, incremental refinement, broader attack models, and live deployment tests before drawing conclusions about operational effectiveness.

Subject of Research: Laboratory evaluation of a fuzzy autoencoder for zero-day and adversarial portable-executable malware detection

Article Title: A hybrid Fuzzy-Autoencoder based approach for resilient zero-day malware detection under adversarial conditions

Article References: Mehta, S., Gadhavi, L., Roy, M., & Mishra, A. (2026). A hybrid Fuzzy-Autoencoder based approach for resilient zero-day malware detection under adversarial conditions. Discover Artificial Intelligence, 6(1), Article 1103. https://doi.org/10.1007/s44163-026-02127-w

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02127-w

Keywords: zero-day malware, adversarial malware, fuzzy autoencoder, one-class learning, generative adversarial network, portable executables, reconstruction error, cyber security, threat detection, security operations centers, hybrid, Fuzzy-Autoencoder