The research, published in Machine Learning, addresses a growing weakness in the safety alignment of generative AI systems. Conventional approaches such as supervised fine-tuning and preference optimization teach models to refuse harmful requests and follow safety rules, but they are often evaluated primarily on isolated prompts. Multi-turn attacks exploit a different failure mode. An attacker may begin with harmless questions, establish a cooperative role-playing scenario, introduce apparently legitimate context, and only later make a request that would have been rejected if presented at the beginning. By then, the earlier turns may have altered the model’s conversational trajectory, making the final instruction appear consistent with the interaction rather than suspicious.

PIARS is built around the idea that this trajectory can be measured. Its central quantity, called the induction degree, estimates how strongly each turn contributes to the accumulation of harmful intent. The researchers divide the score into two components: an inducement measure, which captures how a turn prepares or pressures the model toward a later unsafe response, and a harmfulness measure, which reflects the risk contained in the turn itself. These scores are assigned on a turn-by-turn basis using an external language-model judge. The resulting values are then incorporated into the training objective, allowing the optimization process to give greater attention to dialogue states in which risk is intensifying.

This is technically different from simply labeling a final answer as safe or unsafe. In a conventional training example, the model may receive a penalty only after it produces a harmful response. PIARS instead treats the preceding conversation as part of the safety problem. If a sequence gradually transforms a benign-looking exchange into a jailbreak, the method attempts to identify the earlier representations associated with that drift. The model is trained to separate safe and unsafe dialogue states in its hidden-state space, creating a clearer geometric distinction between contexts that should support assistance and contexts that should trigger refusal or redirection.

Representation separation has become an increasingly prominent direction in AI safety research. Neural language models encode information in high-dimensional activation patterns distributed across many layers, and these patterns influence how the model interprets prompts and selects responses. PIARS uses hidden states from selected layers—in the reported experiments, the 10th and 20th layers—to impose separation between safe, harmful, and borderline examples. The intended result is not a separate safety classifier operating at inference time, but an adapted model whose internal processing is more resistant to contextual manipulation.

The framework also attempts to avoid a familiar trade-off in safety training: improving refusal behavior at the cost of making the model unhelpful. In addition to its multi-turn safety objective, PIARS includes auxiliary constraints for refining safety boundaries and preserving general utility. The boundary component is trained on minimal pairs, such as a potentially harmful request alongside a safe version, so that the model learns to distinguish genuinely dangerous content from legitimate discussion of sensitive subjects. A utility-preservation component uses helpful instructional conversations to reduce the risk that the model will respond with blanket refusals.

A further feature is the progressive adjustment of the objectives during training. Rather than assigning fixed weights to safety, boundary refinement, and utility preservation from the first optimization step, PIARS changes those weights over time. The researchers describe this as a curriculum-style schedule, using linear interpolation to balance competing goals. Early training can emphasize the most direct safety signals, while later stages give greater attention to maintaining useful behavior and accurately handling borderline cases. The implementation uses low-rank adaptation, or LoRA, which updates a relatively small number of trainable parameters instead of retraining the full model.

To test the approach, the team evaluated PIARS on several instruction-tuned systems, including Llama 3 8B Instruct, Qwen2.5 7B Instruct, and Mistral 7B Instruct. The training data combined helpful dialogues from UltraChat, harmful examples derived from Circuit Breaker, safety-boundary pairs based on OR-Bench, and a custom multi-turn jailbreak collection. The custom conversations were generated using an attack strategy associated with ActorAttack and filtered so that their final responses were confirmed as harmful by a HarmBench classifier. Evaluation included multi-turn attack datasets and tests of over-refusal, as well as general capability benchmarks such as MMLU, GSM8K, and HumanEval.

According to the reported results, PIARS reduced attack success rates by more than 50 percent on average compared with strong baseline methods, while also lowering over-refusal and preserving model utility. The authors compare their method with supervised fine-tuning, direct preference optimization, Circuit Breakers, gradient-based alignment, and X-Boundary, using baseline results attributed to the X-Boundary study under what they describe as matched experimental conditions. Additional ablation experiments suggest that the induction-aware objective, representation separation, safety-boundary constraints, and progressive scheduling each contribute to performance, with the combined framework producing the strongest safety–utility balance.

The researchers also report that the induction-degree scoring process was reasonably stable across evaluators and prompt variations. In one comparison involving 50 dialogues, scores from GPT-4o and Qwen2.5-72B-Instruct showed a Pearson correlation of 0.9958, although the reported Spearman rank correlation was lower at 0.9000, indicating that numerical agreement was stronger than perfect ordering agreement. Repeated scoring runs showed limited variance, but the reliance on an external language model remains an important methodological consideration. A judge can introduce its own biases, misclassify ambiguous content, or reflect safety assumptions that do not perfectly match those of the model being trained.

The most practical appeal of PIARS is that its added machinery is used during training rather than deployment. Once the LoRA adaptation is complete, the resulting model operates without judge calls, induction-degree calculations, or extra inference modules. The authors therefore report no additional latency compared with baseline models. In qualitative examples, the method withstands conversations modeled on the Crescendo attack, in which a model is gradually steered toward producing a phishing email, and ActorAttack-style dialogues that attempt to induce targeted political misinformation through role-playing and analytical framing.

The findings arrive as the AI safety community increasingly shifts attention from isolated prompts to interactive, adaptive attacks. A model may pass a static safety benchmark yet fail when an adversary can exploit memory, accumulated context, social pressure, or a carefully maintained persona. This makes multi-turn robustness a moving target: attackers can adjust their strategy according to the model’s previous answers, while defenders must determine not only whether a request is harmful, but how the conversation has changed its meaning over time.

PIARS does not eliminate that challenge, and its results should be interpreted within the limits of the reported datasets, judge-based scoring, and selected model sizes. A defense trained on generated jailbreak conversations may still encounter attack strategies that differ substantially from those used during training. Likewise, a clean separation in a visualization such as t-SNE does not by itself prove that the model’s safety reasoning is causally localized or permanently robust. Nevertheless, the work highlights a potentially influential principle for future alignment systems: safety may depend less on recognizing a single forbidden sentence than on understanding the direction in which an entire conversation is moving.

As conversational AI becomes embedded in education, customer service, coding, research, and personal assistance, that distinction could become central. Harmful intent is often concealed not in one dramatic instruction but in a sequence of individually plausible steps. By attempting to quantify that progression and encode it into the model’s internal decision-making, PIARS offers a new approach to defending against jailbreaks that unfold gradually. Its broader message is that safe dialogue systems must learn to monitor context as a dynamic process—preserving helpfulness when a conversation is benign, while recognizing when cooperation itself is being used to push the system beyond its safety boundaries.

Subject of Research: Multi-turn jailbreak defense and safety alignment for large language models.

Article Title: “Progressive Induction-Aware Optimization for LLMs Safety Under Multi-turn Jailbreaks”

Article References: Jing, S., Cao, H., Wang, B. et al. “Progressive Induction-Aware Optimization for LLMs Safety Under Multi-turn Jailbreaks.” Machine Learning, volume 115, article 151, 2026. Related works include HarmBench, ActorAttack, Crescendo, X-Boundary, Circuit Breakers, OR-Bench, and SafeDialBench.

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07067-1

Keywords: Large language models; AI safety; multi-turn jailbreaks; safety alignment; representation separation; induction degree; LoRA; over-refusal; adversarial robustness.

At the core of the problem is reinforcement-learning–style prompt generation, which tends to optimize for reward rather than diversity. When the system repeatedly gravitates toward a narrow band of high-scoring prompts, it misses other vulnerability pathways. That limitation matters because broader coverage increases the odds that developers can harden defenses before deployment.

S-GFN builds on Generative Flow Networks (GFlowNets), a class of methods trained so that outputs are sampled in proportion to their rewards. While promising for producing diverse, high-reward candidates, prior GFlowNet-based red-teaming can be computationally unstable. Even worse, noisy reward signals may accidentally award high scores to meaningless text, steering training toward failure patterns that are not truly informative.

To make the process both robust and diverse, the team introduced three stabilizing techniques. Contrastive Trajectory Balance (CTB) compares pairs of generated attack trajectories to reduce computational burden and improve training stability. Noise Gradient Pruning (NGP) filters away small reward fluctuations, helping the model learn from meaningful signals rather than stochastic artifacts. Finally, the Min-K Fluency Stabilizer (MKS) biases generation toward prompts that resemble fluent, user-like language, reducing the likelihood of degenerate “gibberish” attacks.

The results are striking: Stable-GFlowNet produced 134 unique attack types—about seven times more than a previous GFlowNet-based approach—while maintaining a 92% attack success rate. Just as importantly, defense models trained using S-GFN-generated attacks generalized well, succeeding not only on known threats but also in cross-attack evaluations that use attack methods different from those seen during training.

Beyond AI safety, the researchers report that CTB and NGP deliver faster, more stable performance in other distribution-matching tasks, including molecular generation relevant to drug discovery. In other words, the same training stability principles may benefit broader generative pipelines.

The work, led by Ph.D. candidate Minchan Kwon under Professor Junmo Kim, was presented as a Spotlight paper (top 2.2% of submissions) at ICML 2026. The project also received support from Korea’s Ministry of Science and ICT through IITP’s SW Star Lab program, emphasizing its goal of practical, reliable safety foundations for next-generation generative systems.

Subject of Research: LLM safety verification / LLM red-teaming using Stable-GFlowNet
Article Title: Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
News Publication Date: 30 July (KAIST announcement); Article Publication Date: 6-Jul-2026
Web References: http://dx.doi.org/10.48550/arXiv.2605.00553
References: arXiv:2605.00553
Image Credits: Credit: KAIST

Keywords

Stable-GFlowNet, Generative Flow Networks, LLM red-teaming, Contrastive Trajectory Balance, Noise Gradient Pruning, Min-K Fluency Stabilizer, AI safety verification, adversarial prompt generation