The starting point of the work is a simple observation about how modern language models are made safe. Commercial and open-source models alike undergo a sophisticated alignment process, typically involving reinforcement learning from human feedback and supervised instruction tuning, that teaches them to refuse harmful requests ranging from malware generation to disinformation. Previous attacks on these safeguards have mostly adopted a generate-and-test strategy: attackers craft malicious prompts, observe whether the model complies, and iterate. Techniques such as the Greedy Coordinate Gradient attack, AutoDAN, PAIR and MASTERKEY all operate at the input level, appending adversarial suffixes or evolving jailbreak prompts through optimization or genetic search. These methods can be effective, but they are computationally expensive, often requiring hundreds of optimization iterations per prompt, and they treat the model as an opaque target rather than a system whose defenses can be mapped.

The researchers instead assumed a white-box threat model in which the adversary has full access to a censored model and can obtain or construct an uncensored counterpart from the same architectural family. This assumption is less restrictive than it might sound. The authors note that as of June 2026, the Hugging Face platform hosts more than five thousand repositories explicitly labeled as uncensored variants across the LLaMA, Qwen, Gemma and Mistral families, some of which have been downloaded thousands of times. Where no such variant exists, prior research has shown that safety alignment can be removed through fine-tuning with only a modest number of curated examples, allowing an adversary to synthesize a functional uncensored twin for comparison purposes.

With both a censored model and its uncensored counterpart in hand, the first stage of XBreaking is a comparative profiling exercise. For every layer of the transformer stack, the researchers computed the mean activation value and the mean attention score in response to harmful inputs, then normalized these statistics so that layers could be ranked by how much the two models diverged. The results were striking. In the censored Llama 3.2 1B model, activations peaked at the first layer and then dropped sharply in later layers, particularly the final one, while the uncensored version continued to activate normally all the way to output. Attention patterns told a similar story: the censored model showed layer-specific attention drops and spikes at deeper layers, which the authors interpret as a strategic suppression of context propagation designed to prevent harmful content from reaching the generation stage.

To move from visual inspection to a systematic method, the team framed layer identification as a binary classification problem. For each layer, they built feature vectors from the mean activation and mean attention values of both models, then used a univariate statistical technique called SelectKBest, scored with the ANOVA F-test, to find the smallest set of layers that best distinguished censored from uncensored behavior. A knee-point analysis determined the optimal number of layers for each model. The results varied revealingly by architecture. Llama models required four and eight layers respectively, consistent with alignment tuning concentrated in the final transformer blocks. Qwen models needed three and nine layers, suggesting safety filters distributed across the depth of the network. Gemma 7B and Mistral 7B required only two and one layers, which the authors attribute to aggressive weight sharing and compressed representations that concentrate alignment differences in a few high-variance components. Fingerprinting accuracy exceeded 90 percent for most models and reached 100 percent for Gemma 7B, with Mistral at 82.5 percent.

The attack itself is remarkably simple once the targets are identified. Instead of modifying the identified layers directly, XBreaking injects noise into the layer immediately preceding each optimal target, altering the input that the safety-critical component receives. Specifically, the perturbation is applied to the weight vector of the layer normalization module that follows the self-attention mechanism, which controls the scale of post-attention activations. The researchers found that in censored models, self-attention and activations are suppressed in certain layers, and that counteracting this suppression allows harmful signals to propagate. They deliberately chose the LayerNorm weight over alternatives such as the query projection, because perturbing query projections disrupts attention globally and degrades functionality rather than specifically targeting the suppression mechanism.

Calibrating the noise required careful analysis of the models’ internal weight distributions. Most LayerNorm weights lie in a narrow band between minus one and one, so the team selected noise values ranging from minus 0.75 to 0.75, large enough to shift the model’s internal state but small enough to avoid overwriting original features and destroying generation quality. Values beyond roughly one in either direction damaged the models’ ability to produce coherent text. The evaluation used the JBB-Behaviors dataset of 100 harmful behaviors across ten categories aligned with OpenAI’s usage policies, with every response manually annotated by two domain experts who achieved a Cohen’s Kappa agreement of 0.85. The optimal-balance attack success rate reached 95.35 percent for Llama 3.2 1B, 96 percent for Llama 3.1 8B, 90.24 percent for Gemma 2B and 82.76 percent for Qwen 2.5 3B, with even the smallest Qwen model reaching 77.78 percent. Mistral 7B proved most resistant at 35.56 percent, likely because its sliding-window and multi-query attention designs stabilize activations against perturbation.

Crucially, the attack preserved benign functionality. When evaluated on the HellaSwag, TruthfulQA and MMLU benchmarks, the modified models retained substantial capability, with cosine similarity to base model outputs exceeding 75 percent for several architectures and MMLU agreement rates above 68 percent for the Llama models and 92 percent for Mistral. The authors also demonstrated that the layer fingerprints generalize beyond the dataset used to discover them. When the same frozen layer selections were applied to the held-out AdvBench and HarmBench benchmarks, XBreaking achieved attack success rates with lower bounds exceeding 70 percent on Llama and Qwen models, peaking at 84.5 percent on Llama 3.1 8B, while GCG and AutoDAN reached at most 59.5 and 54 percent respectively at far greater computational cost. Most provocatively, layers identified on small models could be transferred to production-scale systems: proportionally mapped onto Llama 3.1 70B and Qwen 2.5 72B, the attack achieved success rates of up to 77.7 and 67.7 percent respectively, suggesting that safety mechanisms occupy structurally analogous positions within each model family.

The authors are explicit about the limits of their work. XBreaking is fundamentally a white-box attack that requires access to model internals, so it does not apply to commercial systems such as GPT-4, Claude or Gemini, which are exposed only as black-box APIs. All experiments were conducted offline using publicly available weights, and no external services were targeted. The team argues that full disclosure is justified because the techniques are straightforward to implement, build on established principles, and would be discoverable by any determined adversary, particularly given the widespread availability of uncensored model variants. They propose mitigations including internal activation monitoring, activation obfuscation and architectural randomization to break the deterministic layer projection that makes cross-scale transfer possible.

The broader significance of the study lies in what it reveals about the nature of safety alignment itself. Rather than being a diffuse property spread across the entire network, censorship in these models appears as a localized, fingerprintable pattern concentrated in a small number of layers, in some cases just one. That makes alignment both more legible and more fragile than many assumed. From a defensive standpoint, the same explainability tools that enable the attack could be operationalized as automated security testing, allowing developers to probe for failure modes continuously across the software lifecycle before deployment. The researchers hope their findings will push future work toward alignment techniques that are robust to weight-space modification, and toward a deeper understanding of how safety behavior is encoded in the architecture of large language models.

Subject of Research: Explainable AI analysis of security alignment vulnerabilities in large language models

Article Title: XBreaking: understanding how LLMs security alignment can be broken

Article References: Arazzi, M., Kembu, V. K., Nocera, A., & P., V. (2026). XBreaking: understanding how LLMs security alignment can be broken. Neural Computing and Applications, 38(19), Article 794. https://doi.org/10.1007/s00521-026-12511-3

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12511-3

Keywords: large language models, XBreaking, explainable AI, jailbreaking, safety alignment, adversarial attacks, transformer layers, noise injection, model fingerprinting, LLM security, white-box attack, mechanistic interpretability

The problem the team identifies sits in the middle of the evaluation spectrum. At one end are well-bounded, closed-form tasks, such as multiple-choice medical exams or legal benchmark suites like MEDQA and LEXGLUE, where correctness is specified in advance and scored at scale. At the other end are highly professionalized expert domains, where validation rests on formal standards and regulatory norms. Between these poles lies a growing class of intermediate, situational, and interdisciplinary queries that are neither reducible to fixed answers nor stabilized by consensus authority. Recent polling shows that activities such as targeted information searches and idea generation rank among the top uses of generative AI, and categories like medical advice and legal document drafting have expanded sharply between 2024 and 2025, meaning models are increasingly treated as entry points into specialized knowledge once mediated by professionals.

To illustrate the gap, the authors contrast two kinds of questions. A prompt asking a model to diagnose a 45-year-old man with chest pain and ST-segment elevation can be scored against an established clinical benchmark. But asking how traditional Ayurvedic practices should inform post-heart-attack rehabilitation in rural India demands cultural nuance, historical knowledge, and interdisciplinary synthesis that no fixed test set can capture. Similarly, predicting wheat yields under a two-degree temperature rise can be checked against a crop model, while designing a community heat-action plan for an oasis town facing desertification, with local cultural practices and resource constraints in mind, requires open-ended, scenario-based judgment. Climate adaptation, the study argues, is precisely such a domain: inherently interdisciplinary, regionally contingent, and short on the codified corpora that make medical and legal benchmarking feasible.

The team’s answer draws on the traditions of qualitative data analysis. SQEs are streamlined, rubric-based assessments of limited sets of open-ended responses, conducted by trained human coders. The word qualitative here does not mean unstructured or subjective; evaluative categories are developed through engagement with research literature and theoretical frameworks, and responses are scored against transparent, analytically grounded criteria. The word small refers to a design principle borrowed from the qualitative methodologist Johnny Saldaña’s notion of code parsimony: rubric dimensions should be added only when they designate a salient area of performance that matters to the stated use case. Sparse code sets prioritize interpretability, inter-rater reliability, and transparency, allowing small teams without dedicated evaluation infrastructure to produce defensible assessments.

As a demonstration, the researchers built the Desert Language Model Evaluation Framework, or DLEF, a rubric for evaluating open-ended guidance on extreme heat adaptation, developed from materials produced by Arizona State University’s Knowledge Exchange for Resilience. The framework consists of 35 prompts derived from seven core heat-related questions, each with four location-specific variations across Scottsdale, Phoenix, Guadalupe, and Flagstaff, Arizona, covering topics such as heat stroke diagnosis, home weatherization, air conditioning failures, outdoor worker safety, and public transportation. Each prompt was submitted to GPT-4o ten times, yielding 350 location-based responses and 100 medical responses per condition, collected with automated Selenium scripts in November 2024. Textual consistency was measured using TF-IDF vectorization and cosine similarity, and response quality was scored across five dimensions: Place-Based Fit, General Fit and Feasibility, Executive Path, Uncertainty and Intellectual Humility, and Affect and Emotional Intelligence.

The human validation results were revealing. Two trained coders independently scored a sample of 70 items, and inter-rater reliability, measured with quadratic-weighted Cohen’s kappa, was strongest for the adaptation-relevant dimensions: Place-Based Fit reached 0.91, Executive Path 0.66, and General Fit and Feasibility 0.56, while Uncertainty and Humility (0.24) and Affect and Emotional Intelligence (0.28) fell well short of acceptable thresholds. Mixed-effects models reinforced the pattern, with Place-Based Fit showing the strongest systematic variation tied to prompt characteristics. Substantively, GPT-4o showed surface-level sensitivity to place, often naming a city when given, but limited ability to translate location into genuinely tailored advice. Prompts anchored in low-desert localities such as Scottsdale, Phoenix, and Guadalupe scored higher than those for high-desert Flagstaff, and prompts with no location specified performed worst, a pattern consistent with the distribution of the model’s training data. Apartment prompts received lower feasibility ratings than single-family or mobile homes, reflecting limited attention to the user’s actual control over infrastructure.

The study then asked whether a human-validated SQE could be replicated by automated LLM-as-judge scoring. Four open-weight models, Llama 3.1 8B, Qwen 8B, GPT-OSS 20B, and Mistral 26B, were deployed locally at temperature zero and asked to score the same 70 items under eight prompt variants that systematically varied expert versus nonexpert framing and the type of anchor examples provided. No single configuration achieved acceptable agreement, defined as kappa of at least 0.60, across all five dimensions. Place-Based Fit was the sole dimension approaching reliable replication, with GPT-OSS 20B under the nonexpert condition with human-grounded anchors reaching a mean human-agent kappa of 0.772, equivalent to 85 percent of the human-human baseline. Executive Path never exceeded 30 percent of human agreement under any model or variant. Crucially, the correlation between human inter-rater reliability and best automated agreement across dimensions was significant (Pearson r = 0.686, p = 0.005), suggesting that how consistently humans can apply a rubric predicts whether machines can replicate it.

To understand why, the team turned to mechanistic interpretability. Using the National Deep Inference Fabric and the NNsight library, they extracted last-layer hidden states from Llama 3.1 8B and 70B at the token immediately preceding score generation and trained linear probes to predict human consensus scores. For Place-Based Fit, where human agreement was high, probe accuracy was also high, and principal component analysis revealed a visible score-level gradient along the first principal component in the 70B model, indicating that the construct was geometrically organized in the model’s representations in a way consistent with human judgment. Strikingly, at the 8B scale the probe kappa of 0.713 exceeded the model’s own output agreement with human raters, suggesting the model internally represented place-sensitivity better than its scores reflected, a distinction that matters because it points to prompt design or calibration fixes rather than a need for a bigger model.

The Uncertainty and Humility dimension told a cautionary counter-story. Probe kappa exceeded the human-human baseline under all conditions, reaching 0.588 for the 70B model against a human kappa of just 0.244. Yet per-rater analysis showed that this apparent alignment was partly an artifact of averaging divergent rater scores toward the middle of the scale, a phenomenon the authors link to what recent work calls the consistency-bias paradox: high internal consistency in an AI judge does not guarantee valid measurement. When a model produces more consistent scores than humans on a dimension where humans disagree, it has more likely converged on its own interpretation of the construct than faithfully operationalized the codebook’s intent. High automated agreement on a poorly operationalized dimension, the authors argue, is a validity concern rather than evidence of scalability.

The broader implication is that human inter-rater reliability should function not merely as a quality check on a rubric but as a prior estimate of whether automated replication is feasible at all. Dimensions defined by observable, surface-detectable features of text are candidates for automation at sufficient model scale, while dimensions requiring inferential judgment about tone, structure, or epistemic stance remain dependent on human coding. The authors propose a five-stage workflow of rubric development, human calibration, automation screening, scaled deployment, and periodic revalidation, and they caution that prompt configurations validated for one model family may not transfer to another even at the same parameter count. SQEs are not a replacement for large-scale benchmarks or professional panel adjudication, but they offer a principled, repeatable strategy for the dynamic, contested, and interdisciplinary settings where AI systems are increasingly deployed, and where, as the foundation paradox makes clear, the consequences of untested guidance may be greatest.

Subject of Research: Evaluation of large language model performance in open-ended, interdisciplinary domains using small qualitative, human-in-the-loop assessment frameworks

Article Title: Performance within the foundation paradox: the case for small qualitative evaluations

Article References: Simeone, M., Hyatt, J., Bienenstock, E. J., Babu, R., & Solis, P. (2026). Performance within the foundation paradox: the case for small qualitative evaluations. Neural Computing and Applications, 38(19), Article 777. https://doi.org/10.1007/s00521-026-12328-0

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12328-0

Keywords: large language models, model evaluation, small qualitative evaluations, inter-rater reliability, LLM-as-judge, mechanistic interpretability, climate adaptation, extreme heat, qualitative data analysis, foundation models, benchmarking, GPT-4o

The central idea of the framework is deceptively simple: a prompt is not merely text but a functional transformation. When a user appends an instruction such as “answer step by step” or “respond as an expert,” the resulting embedding of that text inside the model’s semantic space is shifted, scaled, or rotated in measurable ways. Prompt Algebra formalizes these shifts as operators, each characterized by an operator norm that quantifies how strongly the prompt distorts the semantic representation of the task. By defining entropy bounds, Jacobian-based invertibility conditions, and semantic distance metrics, the framework provides a vocabulary for asking precise questions: How much does a given prompt compress the space of possible meanings? Can its effect be reversed? Does combining two prompts produce the same result regardless of the order in which they are applied?

One of the most striking theoretical contributions of the paper is the introduction of the Prompt Transform Group, or PTG, an algebraic structure that captures how prompt operators compose. The study establishes that this structure is compositional, meaning that complex prompting strategies can be decomposed into simpler building blocks whose individual effects combine in principled ways. Crucially, the group is non-commutative: applying prompt A and then prompt B generally produces a different semantic transformation than applying B and then A. Anyone who has noticed that reordering instructions in a prompt changes the quality of a model’s output has observed this non-commutativity in practice. The framework turns that intuition into a theorem, explaining why the order of instructions, examples, and constraints matters so deeply in real-world prompting.

The framework also addresses entropy dynamics, tracking how different classes of prompts systematically compress, expand, or preserve the informational richness of semantic embeddings. According to the study, some prompts act like contraction operators, squeezing diverse meanings into a narrow region of the latent space, which may improve consistency but risk collapsing nuance. Others act as expansions, spreading representations outward and potentially enabling more creative or varied outputs. Still others behave like rotations, reorienting meaning along different semantic axes without changing its magnitude. These categories give prompt designers a diagnostic language: rather than asking vaguely whether a prompt “works,” they can ask which class of operator it instantiates and what that implies for the behavior of the model.

To move beyond pure theory, the author conducted empirical validation using two widely available tools: Sentence-BERT, a popular model for producing sentence embeddings, and Phi-3-mini, a compact large language model from Microsoft. The experiments measured how embeddings changed when different classes of prompts were applied, and the results were visualized using two standard dimensionality-reduction techniques, principal component analysis and t-distributed stochastic neighbor embedding. The visualizations revealed that prompts belonging to different functional classes produced distinct, systematic geometric signatures in the embedding space. Compression operators visibly clustered representations together, expansion operators spread them apart, and rotations produced coherent reorientations. The consistency of these patterns across prompt classes suggests that the algebraic picture is not merely a metaphor but reflects measurable structure in how language models process instructions.

The paper goes further by providing pseudocode for estimating prompt operators, offering a practical recipe for researchers who want to compute the norms, entropy changes, and invertibility properties of their own prompts. All of the code, prompts, and figures used in the experiments have been released publicly in a GitHub repository, allowing independent verification and reuse. This open approach matters because the field of prompt engineering has been dominated by folklore: collections of tips, templates, and anecdotal best practices shared across blogs and documentation, with little theoretical grounding. By supplying executable methods for measuring the geometric effect of a prompt, the framework invites the community to replace folklore with experiment.

Prompt Algebra is positioned explicitly against two existing research traditions. The first is categorical semantics, exemplified by the compositional distributional models of meaning developed by Coecke, Sadrzadeh, and Clark, which use category theory to describe how word meanings combine. The second is mechanistic interpretability, the effort to reverse-engineer the internal circuits of neural networks, associated with researchers such as Chris Olah. The new framework borrows the compositional spirit of the former and the operator-level precision of the latter, but applies them at the level of the prompt itself, treating instructions as first-class mathematical objects rather than as external inputs to be studied only through model internals. The study also engages with the extensive empirical literature on prompting, including chain-of-thought prompting, least-to-most prompting, graph-of-thoughts reasoning, automatic prompt generation methods such as AutoPrompt, and parameter-efficient prompt tuning techniques, framing these disparate methods as instances of underlying operator classes.

The practical implications could be significant for the rapidly growing industry built around large language models. If prompts can be characterized by norms, entropy bounds, and invertibility conditions, then prompt design becomes an optimization problem with well-defined objectives rather than an intuition-driven guessing game. The framework suggests possibilities for certifying prompts before deployment, predicting how chained prompts in multi-stage pipelines will interact, and detecting adversarial or jailbreak prompts by identifying operators with pathological properties such as extreme norms or unstable Jacobians. It could also inform tools that automatically compose prompts with predictable effects, extending recent programmatic approaches such as DSPy and AI chains with a formal calculus of composition. For safety researchers, the ability to reason mathematically about how a prompt transforms a model’s semantic landscape offers a new layer of analysis complementary to gradient-based detection methods.

The research was funded by Taibah University in Madinah, Kingdom of Saudi Arabia, under grant number 448-15-1203, and the author declares no conflict of interest. Like any ambitious theoretical proposal, Prompt Algebra will need to prove itself through wider adoption, replication across larger and more diverse models, and demonstration that its algebraic predictions translate into reliably better prompt design in production settings. The experiments reported rely on relatively compact models, and extending the operator estimates to frontier-scale systems remains an open challenge. Nevertheless, the paper marks a conceptual shift worth taking seriously: it argues that the words we write to instruct machines are not loose incantations but operators in a mathematical space, subject to norms, entropies, and group structure. If that view holds up, the era of prompt engineering as craft may give way to an era of prompt engineering as applied mathematics, with designers composing transformations deliberately the way engineers compose circuits, and with the behavior of language models becoming not just observable but predictable.

Subject of Research: An algebraic framework modeling prompts as composable operators in the latent semantic space of large language models

Article Title: Toward an algebraic framework for operator-based prompt engineering in large language models

Article References: Olaymi, S. E. Z. (2026). Toward an algebraic framework for operator-based prompt engineering in large language models. Neural Computing and Applications, 38(17), Article 732. https://doi.org/10.1007/s00521-026-12434-z

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12434-z

Keywords: prompt engineering, large language models, Prompt Algebra, operator theory, semantic embeddings, Sentence-BERT, Phi-3-mini, entropy dynamics, Prompt Transform Group, composability, mechanistic interpretability, generative AI