Wavelet Features Help Deepfake Detectors Spot Unseen Forgeries, But at a Cost

Deepfakes have become one of the most unsettling byproducts of the generative AI revolution. From the notorious video of Barack Obama appearing to say words he never spoke to an AI-driven voice scam that defrauded a company of US$243,000, synthetic media has repeatedly demonstrated its power to deceive. As generative models have evolved from autoencoders and generative adversarial networks to today’s diffusion models, the manipulated content they produce has grown increasingly indistinguishable from authentic media. Now, a team of Italian researchers has taken a deliberately simple but revealing step toward detectors that generalize better: they replaced the standard RGB pixel input of a deep neural network with a wavelet-domain representation, and carefully measured what changed.

The study, published in Frontiers in Imaging by Niccolò Marini and Stefano Berretti of the University of Florence and Roberto Caldelli of the National Interuniversity Consortium for Telecommunications and Universitas Mercatorum, addresses one of the central open problems in multimedia forensics. Most current deepfake detectors operate directly in the RGB domain, learning discriminative cues from spatial appearance alone. While such methods perform impressively when the test forgeries match the training distribution, their robustness often collapses when evaluated on unseen manipulation methods, different compression levels, or varying acquisition conditions. This limited cross-forgery generalization remains one of the main challenges in the field, and it is precisely the setting that matters in the real world, where new generative pipelines appear constantly.

The researchers’ motivation comes from a complementary perspective on images: frequency analysis. Image manipulation pipelines, whether they involve blending a swapped face, synthesizing textures, or resampling content, often introduce subtle local inconsistencies that are less evident in the pixel domain. Among frequency-based representations, the Discrete Wavelet Transform, or DWT, is particularly appealing because it provides a multi-scale decomposition of an image while preserving spatial locality. Unlike the Fourier transform, which represents a signal through globally supported sine and cosine functions, the DWT relies on localized basis functions called wavelets that decay rapidly. This makes it possible to separately analyze coarse structural information and directional high-frequency details, exposing forensic cues that are difficult to perceive directly in RGB.

Technically, the approach is intentionally lightweight. Given an input video frame, the team first extracts the facial region using a pre-trained MTCNN face detector, enlarges the bounding box by a factor of 1.3 to preserve useful context around facial boundaries and blending regions, converts the crop to a square, and resizes it to 224 by 224 pixels. They then apply a one-level Haar wavelet decomposition independently to each of the three RGB channels. Each channel is passed through low-pass and high-pass filters along both horizontal and vertical directions, producing four sub-bands: the LL approximation band, which captures the coarse low-frequency structure of the image, and three directional detail bands encoding vertical, horizontal, and diagonal high-frequency information. Stacking the four sub-bands from each of the three channels yields a 12-channel tensor of size 112 by 112, which becomes the network input.

A crucial subtlety is that this wavelet tensor is not a reduction of information. Because the DWT is critically sampled, the four sub-bands from each channel preserve the same total number of scalar coefficients as the original 224 by 224 channel. The 112 by 112 by 12 wavelet input therefore contains exactly the same 150,528 values as the 224 by 224 by 3 RGB tensor; the difference lies purely in how the same face crop is organized into spatial-frequency components. The researchers fed this representation into a modified ResNet-50 architecture in which only the first convolutional layer was adapted to accept 12 channels instead of three, leaving all remaining layers unchanged. Because the native 12-channel input is incompatible with standard ImageNet pretraining, the wavelet model was trained from scratch, while the RGB baselines used pretrained weights. The team also trained an RGB-112 control at reduced spatial resolution to isolate the effect of resolution from the effect of representation.

The evaluation protocol was rigorous. Using the widely adopted FaceForensics++ benchmark, each binary detector was trained on pristine samples and one manipulation type, then tested both on the same manipulation type, called in-forgery or diagonal evaluation, and on the remaining manipulation types, called cross-forgery or off-diagonal evaluation. The benchmark covers five manipulation methods, including Deepfakes, Face2Face, FaceSwap, NeuralTextures, and FaceShifter, at two compression levels, c23 and c40. Beyond the standard true positive and true negative rates, the researchers reported accuracy, precision, F1 score, and AUC, computed aggregate confidence intervals by bootstrapping over train-target pairs, and applied paired sign-flip tests over train-target metric differences. All models were trained on a single NVIDIA GeForce RTX 2080 Ti GPU, underscoring the accessibility of the approach.

The results reveal a nuanced picture rather than a uniform improvement. On the diagonal, where training and testing use the same forgery type, RGB remains the stronger representation: at the c23 compression level, RGB achieves 96.61 percent accuracy and 95.66 percent true positive rate on matched forgeries, compared with 92.65 percent and 94.33 percent for the wavelet model. RGB also preserves higher true negative rates, meaning it is more conservative on pristine samples and produces fewer false positives. But the behavior changes dramatically in the off-diagonal cross-forgery setting. On c23, off-diagonal accuracy changes only marginally from 53.20 to 54.14 percent, yet the true positive rate for fake samples nearly doubles from 8.83 to 17.32 percent, and the F1 score rises from 15.37 to 24.81 percent. At the same time, the true negative rate drops from 97.57 to 90.97 percent. The same pattern holds at c40, where off-diagonal true positive rate increases from 24.50 to 32.90 percent and F1 from 31.48 to 40.29 percent, while true negative rate falls from 83.97 to 77.30 percent.

In other words, the wavelet representation shifts the operating point of the detector toward higher sensitivity to fake samples and lower specificity on real ones. AUC, a threshold-independent measure of separability, does not show a consistent wavelet advantage, further confirming that the benefit is not a blanket improvement in detection capability. The cross-compression experiments reinforce the hypothesis: when models trained on c23 were tested on the more heavily compressed c40 data, the wavelet-based model consistently outperformed the pixel-domain method in cross-forgery scenarios, achieving higher true positive rates across manipulation techniques, this time also in the in-forgery cases. For manipulations such as Deepfakes and FaceShifter, the true positive rate improvements exceeded 10 to 15 percent compared with the pixel-domain method, suggesting that wavelet sub-bands capture forensic traces that are less sensitive to specific forgery techniques or compression artifacts.

What makes these findings scientifically valuable is the controlled framing. Unlike recent wavelet or frequency-aware works that introduce specialized architectures, multi-stage designs, or pretrained vision-language components such as Wavelet-CLIP, this study asks a narrower question: what changes when the input to the same ResNet-50 backbone is replaced by a native first-level wavelet representation? The answer is that wavelet sub-bands expose manipulation cues, such as edge distortions, resampling traces, and unnatural textures introduced by synthesis and blending operations, that are useful for detecting unseen fake classes, but they also make the detector less conservative on pristine samples. The authors are explicit that the wavelet formulation should be interpreted as a lightweight and complementary representation for cross-forgery detection, not as a universal replacement for RGB.

The practical implications depend on the deployment scenario. If the priority is to maximize fake recall on unseen manipulations, as in screening pipelines where missing a fake is the costlier error, the wavelet representation is advantageous in many off-diagonal cases. If the priority is to minimize false positives on real content, as in systems where wrongly flagging authentic media carries reputational or legal risk, RGB remains preferable. The researchers outline several directions for future work: extending the evaluation beyond FaceForensics++ to benchmarks such as Celeb-DF, DFDC, and DeeperForensics, testing newer generative pipelines including diffusion-based and recent identity-swap methods, investigating video-level aggregation strategies and temporal consistency cues, and exploring hybrid RGB-wavelet systems that preserve the strong specificity of pixel-domain models while exploiting the cross-forgery sensitivity of wavelet-domain features. As deepfakes continue to evolve, this study offers a sober lesson: generalizable detection will come not from a single image domain alone, but from carefully understanding which cues actually transfer across forgery families.

Subject of Research: Artificial Intelligence

Article Title: Wavelet-based features to improve cross-forgery generalization in deepfake detection

Article References: Marini, N., Berretti, S., & Caldelli, R. (2026). Wavelet-based features to improve cross-forgery generalization in deepfake detection. Frontiers in Imaging, 5, Article 1845480. https://doi.org/10.3389/fimag.2026.1845480

Image Credits: AI Generated

DOI: 10.3389/fimag.2026.1845480

Keywords: deepfake detection, discrete wavelet transform, machine learning, computer vision, digital forensics, Wavelet-based, features, improve, cross-forgery, generalization, deepfake, detection