The research, led by Qiqi Bao and Jiaoling Li of Zhejiang University of Science and Technology together with colleagues at Harbin Institute of Technology Shenzhen, Zhejiang University, and Université Paris Cité, demonstrates that concepts which appear thoroughly erased under standard text-based testing can reappear with alarming ease once the model is combined with the structural control tools that define modern image generation. The team calls the attack Structure-Guided Concept Reappearance, or SGCR, and its central insight is deceptively simple: safety evaluations focus almost exclusively on the text pathway, while real-world diffusion systems increasingly generate images through a second, non-textual route that defenses never touch.

To understand why this matters, it helps to look at how contemporary diffusion pipelines actually work. A latent diffusion model consists of an encoder that compresses images into a compact latent representation, a decoder that reconstructs them, and a denoising network that gradually transforms random noise into a coherent image. Text prompts influence this process mainly through cross-attention modules that bind semantic content to spatial locations. Concept erasure methods, whether they edit model weights as ESD, UCE, MACE, and TRCE do, or intervene at inference time as Negative Prompting, Safe Latent Diffusion, and AdaVD do, all share the same underlying objective: they weaken the association between a textual trigger and its corresponding visual concept. The implicit assumption has always been that if the text route is blocked, the concept is effectively gone.

That assumption collapses when a structural controller enters the picture. Tools such as ControlNet have made it routine for users to steer generation with Canny edge maps, depth maps, or semantic segmentation maps derived from reference images, all without modifying the backbone model. These structural representations discard color, texture, and most appearance information, which is precisely why they were assumed to be semantically inert. The researchers show otherwise. Using a zero-shot CLIP probe, they demonstrated that edge, depth, and segmentation maps extracted from target-concept reference images retain enough category-related geometry, spatial layout, and instance-specific cues to reliably distinguish target conditions from benign ones. A line drawing stripped of every pixel of explicit content still carries the silhouette of what it depicts.

The SGCR attack exploits this residual information under a deliberately practical grey-box threat model. The attacker needs no access to training data, no gradients from the defended model, and no ability to modify its parameters. Instead, the attacker simply knows the deployed backbone-controller configuration, supplies a completely benign text prompt containing no target semantics, and attaches a pre-trained structural controller fed with a condition extracted from a publicly available reference image of the target concept. The controller injects geometry-consistent residual features into the U-Net through zero-convolution connections, and these structurally modulated representations are progressively integrated across downstream blocks during denoising, steering the latent trajectory toward spatial configurations that conform to the supplied scaffold.

The experimental results are stark. Across twelve target concepts spanning explicit content from the I2P benchmark, copyrighted artistic styles including Van Gogh, Picasso, and Kelly McKernan, and generic objects drawn from CIFAR-10, SGCR markedly increased attack success rates against representative defenses from all three major paradigms. MACE, which reduced the text-only attack success rate for nudity to zero percent, saw that figure climb to 72 percent under Canny guidance, while its success rate on Picasso reached a full 100 percent. UCE’s nudity rate rose from 22 to 49 percent, and its Cat success rate jumped from 2 to 74 percent. Notably, the Fréchet Inception Distance of MACE’s nudity outputs fell from 48.61 to 7.37, meaning the attacked images were not degraded artifacts but high-quality generations closely matching the original model’s output distribution.

The team went to considerable lengths to rule out alternative explanations. In a controlled prompt-structure analysis with seven distinct conditions, they showed that adding target-consistent structure while holding the text fixed raised success rates by 20 to 51 percentage points across all eight models tested, whereas pairing the benign prompt with a non-target human structure produced far weaker effects. This establishes that the reappearance is driven by the semantic content of the structural condition itself, not merely by the activation of a controller. They also found that exact prompt-reference pairing was unnecessary: cyclically shifting target-category structures across prompts yielded statistically indistinguishable results, indicating category-level transfer across reference instances. A multi-reference, multi-seed extension covering three concepts and four defenses confirmed that the behavior persists across 100 distinct reference images and three random seeds, with success rates ranging from 58 to 84.3 percent.

Mechanistic probes added further texture to the picture. When the researchers replaced attention maps with those from paired benign generations, swapping self-attention produced larger drops in attack success than swapping cross-attention, suggesting that image-side feature interactions play a greater role than the text-conditioning pathway in propagating structural information. Activation-energy visualizations showed self-attention responses aligning with the supplied structural contours even as text-related cross-attention remained weak, and intermediate denoising trajectories revealed target-consistent spatial organization emerging progressively in the mid-to-late sampling stages. An ablation over guidance strength found the attack most effective in an intermediate range around a scale of 1.0, with effects saturating beyond that point. The authors are careful to frame all of this as a system-level property of the composed backbone-controller pipeline rather than proof that the erased backbone secretly harbors the concept in a single recoverable module.

The practical implications are scoped precisely but seriously. The demonstrated threat applies to local modular systems and to services that expose compatible structural-control interfaces, not to closed text-only APIs, since an attacker must be able to attach a controller and supply a structural condition. Within that scope, the consequences extend beyond regenerating a single reference image: structural conditions can be reused with different benign prompts and seeds to produce endless target-consistent variants, potentially defeating exact-match and duplicate-detection safeguards. The authors propose three directions for future work: evaluation protocols that include controller-augmented generation, defenses that jointly suppress text-conditioned generation, image-side feature interactions, and attached controllers, and safety alignment studied at the system level. The parallel they draw is instructive: just as visual inputs have jailbroken aligned large language models, multimodal deployments of diffusion models can expose gaps that no text-focused evaluation will ever catch. For an industry racing to certify the safety of generative systems, the message is uncomfortable but clear: erasing a concept from what a model hears does not mean erasing it from what a model can be shown.

Subject of Research: Evasion attacks on concept erasure safeguards in text-to-image diffusion models using structural conditioning

Article Title: Evasion attacks on generative safeguards: target-concept reappearance under structural control in grey-box settings

Article References: Bao, Q., Li, J., Zhang, Y., Qian, Y., Gu, Z., Ji, S., Wang, B., & Naït-Abdesselam, F. (2026). Evasion attacks on generative safeguards: target-concept reappearance under structural control in grey-box settings. Cybersecurity, 9(1), Article 227. https://doi.org/10.1186/s42400-026-00664-6

Image Credits: AI Generated

DOI: 10.1186/s42400-026-00664-6

Keywords: diffusion models, concept erasure, AI safety, ControlNet, adversarial attacks, text-to-image generation, structural conditioning, machine unlearning, cybersecurity, generative AI, grey-box attack, model robustness