Diffusion Models Get a Forensic Upgrade: Two-Stage AI Pinpoints Doctored Pixels in Photos

Every day, millions of images circulate through social media, news outlets, and courtrooms, and a growing share of them have been quietly altered. A cloned patch of sky, a spliced-in face, an airbrushed-out bystander — these edits are often invisible to the human eye, yet they can shape elections, damage reputations, and even sway legal verdicts. Researchers at the National Institute of Technology Goa have now proposed a fresh way to catch such forgeries, described in the journal Multimedia Tools and Applications, that pairs a specialized forensic classifier with the most talked-about technology in modern artificial intelligence: diffusion models.

The new framework, developed by Mohammad Zohaib Hamdule and Venkatanareshbabu Kuppili, tackles a task known as image manipulation localization, or IML. Detection alone is not enough for most real-world applications; investigators need to know exactly which pixels in a photograph were tampered with. That is a far harder problem, because manipulation traces are subtle, varied, and constantly evolving as editing tools improve. Conventional deep learning approaches, the authors note, often struggle to generalize beyond the specific forgery techniques they were trained on, faltering when confronted with new manipulation schemes.

The team’s answer is a two-stage system that splits the problem in two. Rather than asking a single network to simultaneously figure out whether an image is fake, what kind of fakery was used, and where it happened, the framework first classifies and then localizes. This modular design mirrors the way a human forensic analyst works: identify the type of edit first, then apply the right analytical lens to trace its boundaries. Modularity also brings a practical bonus — each stage can be upgraded independently as new techniques emerge.

The first stage is a Dual-Stream Manipulation Classifier, and its architecture reveals a deep understanding of how digital forgeries leave fingerprints. One stream processes the image in its ordinary RGB form, capturing semantic content — textures, objects, edges. The second stream is more forensic in spirit: it passes the image through Steganalysis Rich Model filters, a family of high-pass filters borrowed from the field of steganalysis, where researchers have long used them to expose hidden data embedded in images. These SRM filters suppress the natural content of the photograph and amplify low-level noise artifacts — the microscopic inconsistencies left behind whenever pixels are copied, spliced, erased, or enhanced.

Both streams feed into a ResNet-style four-stage backbone, the workhorse convolutional architecture that has underpinned computer vision for nearly a decade. By fusing standard visual features with these noise residuals, the classifier learns to recognize four of the most common manipulation categories: Copy-Move, where a region is duplicated and pasted elsewhere in the same image; Splicing, where content from one photograph is inserted into another; Removal, also called inpainting, where an object is erased and the hole filled in; and Enhancement, where attributes such as color, brightness, or fine detail are adjusted to deceive. On the DF2023 dataset, a benchmark for digital forensics, this classifier reached an accuracy of 89 percent — a strong result given how visually different the four categories can be.

Once the manipulation type is known, the image is routed to the second stage: a set of specialized Conditional Diffusion Models, one for each manipulation class. Diffusion models, the same family of generative networks behind today’s most impressive text-to-image systems, work by learning to reverse a gradual noising process. In this framework they are repurposed for an entirely different goal: instead of generating photorealistic pictures, they generate masks — binary maps that paint the manipulated region white and the untouched background black. The localization task is thereby reframed as an image-to-mask generation problem, with the suspect photograph serving as the conditioning input that guides the denoising process toward the correct answer.

Training such generative models for precise localization demanded a technical innovation of its own. The standard training objective for image generation, mean squared error, treats every pixel equally and tends to wash out small targets. A tiny spliced region or a narrow inpainted stroke occupies only a handful of pixels, and a model trained purely on squared error can learn to predict a blank mask and still score decently. The researchers therefore modified the loss function to combine mean squared error with Intersection over Union, the standard overlap metric in segmentation. This hybrid objective pushes the model to reproduce not just approximate shading but the exact spatial structure of the manipulation, with particular benefit for smaller masks that would otherwise be smoothed away.

The numbers back up the design. Across the DF2023 dataset, the localization diffusion models achieved an average Intersection over Union of 0.70 and an F1 score of 0.77 — metrics that balance precision and recall when judging how faithfully the predicted mask matches the true tampered region. The system also demonstrated competitive performance on well-established benchmark datasets including IMD2020, CoMoFoD, CASIA, and COVERAGE, which collectively span realistic splices, copy-move forgeries, and controlled manipulation scenarios. Consistency across these heterogeneous collections suggests the approach is not merely memorizing the quirks of one dataset, a persistent weakness in the field.

What makes the work especially timely is the central paradox it highlights: generative models now create the forgeries, and generative models can also expose them. Earlier attempts to bring generative machinery to forensics leaned on generative adversarial networks, which produce output in a single pass and can be unstable to train. Diffusion models, by contrast, refine their predictions over many iterative denoising steps, an approach that has recently proven effective in segmentation tasks from medical imaging to remote sensing. The Goa team’s results add image forensics to that growing list, joining related efforts that use diffusion-based models for inpainting localization and forgery localization more broadly.

The implications extend well beyond the laboratory. Investigators and prosecutors increasingly rely on digital images as evidence, and studies have shown that people are surprisingly poor at spotting manipulated photos of real-world scenes. A tool that can automatically classify the type of forgery and trace its pixel-level boundaries could strengthen fact-checking workflows, support media authentication desks, and give courts a more rigorous basis for judging photographic evidence. The modular architecture also offers a pragmatic path forward: as AI-generated and AI-edited imagery grows more sophisticated, individual components of the pipeline — new filters, new classifiers, new generative backbones — can be swapped in without rebuilding the entire system. For now, the framework’s 89 percent classification accuracy and 0.70 average IoU represent a meaningful step toward forensic tools that can keep pace with the editing software they are built to catch, confirming that a modular, generative strategy has real promise in the escalating contest between image manipulation and image verification.

Subject of Research: Image manipulation localization using dual-stream classification and conditional diffusion models

Article Title: A modular image manipulation localization framework using a dual-stream classifier and conditional diffusion models

Article References: Hamdule, M. Z., & Kuppili, V. (2026). A modular image manipulation localization framework using a dual-stream classifier and conditional diffusion models. Multimedia Tools and Applications, 85(10), Article 778. https://doi.org/10.1007/s11042-026-21940-0

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21940-0

Keywords: image forensics, image manipulation localization, conditional diffusion models, dual-stream classifier, deep learning, steganalysis rich model, copy-move forgery, splicing detection, inpainting localization, DF2023 dataset, digital forensics, generative models