The stakes of this conceptual muddle are not academic. Artificial intelligence is increasingly deployed in critical domains such as law and medicine, where decisions about bail, sentencing, diagnosis, and treatment can reshape human lives. The European Union’s AI Act, the world’s first comprehensive legal framework for artificial intelligence, explicitly demands transparency and explainability for high-risk systems. Yet the new paper observes that when lawmakers and technologists invoke explainability, they often talk past one another because they are not explaining the same thing. A lawyer asking why an algorithm flagged a defendant as high-risk is asking a different question from an engineer asking how the model’s weights were trained, and different again from an ethicist asking whether the system’s development process respected human rights. Without a shared vocabulary that distinguishes these questions, compliance analysis risks becoming a box-ticking exercise rather than genuine accountability.

The framework at the heart of the study decomposes explainability along two perpendicular axes, which the authors call horizontal and vertical levels. The horizontal axis addresses what can be explained in a system. The authors distinguish three such levels: first, the problem itself, meaning the task the AI system is intended to solve and the context in which it operates; second, the development of the system, encompassing the choices made in building it; and third, the system itself, the deployed artifact. This decomposition is deliberately fine-grained. The development level is further split between a static perspective, which considers the system’s design and documentation at a fixed point in time, and a dynamic perspective, which follows the evolution of the system across its lifecycle, including retraining and updating. The system level, in turn, is divided into explaining the input the system receives, the operations it performs, and the output it produces.

The vertical axis addresses the levels at which explanation can occur, and here the authors apply two criteria: the level of abstraction and the degree of formalisation of knowledge. An explanation pitched at a high level of abstraction might tell a hospital administrator that a diagnostic model “weighs imaging features alongside patient history,” while a low-abstraction explanation might enumerate the exact activation patterns inside a neural network. Similarly, knowledge can be formalised with mathematical precision or conveyed informally in natural language. The point, the authors argue, is that neither extreme is inherently better. The adequacy of an explanation depends on the audience, the purpose, and the regulatory context. A single AI system therefore admits a lattice of possible explanations, and conflating them produces exactly the kind of confusion that has plagued both public debate and legal compliance.

To demonstrate that the framework is more than an abstract taxonomy, the authors test it against three use cases drawn from contemporary controversy. The first maps Article 13 of the AI Act, which imposes transparency obligations on providers of high-risk AI systems. When the obligations in that article are parsed through the framework, it becomes clear that different clauses target different horizontal levels: some provisions demand explanation of the system’s operations, others of its development process, and still others of the problem definition and intended use. This mapping reveals both the strengths and the gaps of the legislation, showing regulators precisely where their demands sit on the conceptual grid and where important explanatory duties remain unaddressed.

The second use case revisits one of the most infamous episodes in the history of algorithmic justice: the debate surrounding COMPAS, the recidivism-prediction software whose alleged racial bias sparked years of dispute after a 2016 investigation. The authors show that much of the controversy can be reframed as a clash between explanations offered at different levels. One side analysed the system’s outputs statistically, asking whether error rates differed across racial groups; the other examined the system’s operations and inputs, asking whether race was used directly in the model. Because the combatants were explaining different things, the debate could not converge, and the framework makes visible exactly where the fault lines ran. The lesson is that clarifying the level of explanation upfront could defuse similar controversies before they harden into entrenched positions.

The third use case turns to the technology of the moment: large language model based AI chatbots such as conversational assistants. LLM systems are often described as especially opaque, but the framework allows a more precise diagnosis. The authors analyse opacity at each horizontal level, showing that while the low-level operations of a trained transformer network are effectively inaccessible to human inspection, explanations of the system’s inputs and outputs remain feasible, and explanations of the development process and the problem the system is meant to solve are available in principle. This stratified view suggests that declaring LLMs “black boxes” writ large is unhelpful; instead, developers and regulators can target specific levels where explanation is achievable, and be honest about levels where it is not.

Underlying the framework is a broader philosophical claim with technical consequences: AI systems are not unidimensional artifacts. A modern machine learning system comprises a problem formulation, a training pipeline, a data corpus, a model architecture, an inference mechanism, and an interface to users. Each of these components can be explained, but each demands different explanatory resources. Explaining the input might involve data provenance analysis and bias audits. Explaining operations might require interpretability techniques such as saliency maps, feature attribution, or mechanistic analysis. Explaining outputs might involve confidence calibration and counterfactual reasoning. Explaining development might mean documenting design decisions, hyperparameter choices, and evaluation protocols. The framework does not prescribe a single technique; it prescribes order. First decide what is being explained, then choose the method appropriate to that target.

The authors position their contribution explicitly as a theoretical and conceptual study, and they are careful about what it does and does not deliver. It does not provide new algorithms for generating explanations, nor does it offer empirical measurements of explanation quality. What it provides is conceptual clarity, a precondition for effective compliance analysis. In an era when organizations must certify the trustworthiness of AI systems against legal and ethical standards, a checklist that says “provide explanations” is vacuous unless it specifies which levels of explanation are required and at what abstraction and formalisation they should be pitched. The framework gives auditors, developers, and regulators a shared grid on which to locate every transparency requirement, making gaps and overlaps visible and enabling more systematic verification.

The research was conducted within a project funded by the European Union’s Horizon Europe Research and Innovation Programme under Grant Agreement number 101120406, with additional support from the DigiWorld Priority Research Area under the Strategic Programme Excellence Initiative at Jagiellonian University, and support for Andrzej Porębski from the Foundation for Polish Science. The work received on 2 May 2025 was accepted on 15 June 2026 and published on 7 September 2026, appearing as an open-access article that carries a permanent digital object identifier. The authors report no conflicts of interest relevant to the content of the study, and the article reflects only the authors’ views, with the European Commission bearing no responsibility for any use that may be made of the information it contains.

As artificial intelligence systems grow more capable and more embedded in consequential decisions, the pressure to make them accountable will only intensify. This study suggests that the path forward runs not through a single magic technique that will suddenly render neural networks transparent, but through disciplined thinking about the many different things we might mean when we ask a machine to explain itself. By separating the problem from the development process from the deployed system, and by distinguishing abstraction from formalisation in the explanations themselves, the Kraków team has offered regulators, engineers, and scholars a common map of the explanatory terrain. Whether that map becomes a standard tool of AI compliance will depend on uptake, but its central message is already clear: before demanding explanations from AI, we must first agree on what, exactly, we are asking to know.

The core problem is deceptively simple to state. Machine learning systems are updated for many reasons: new training data becomes available, better architectures are developed, or the systems must satisfy new or revised technical and ethical requirements. Conventionally, engineers evaluate whether an update is worthwhile by looking at aggregate performance measures such as accuracy. If average accuracy goes up, the update is deemed a success. But averages conceal what happens to individual samples. The researchers focus on a phenomenon known as negative flips: cases in which the updated model produces an error that the previous model handled correctly. A user whose loan application was approved under the old model may be rejected by the new one, even though the new model scores better overall. Each such instance constitutes a backward incompatibility between versions of the system.

The authors push this analysis one step further, into the domain of algorithmic fairness. If the negative flips introduced by an update are not distributed evenly across the population but are biased with respect to sensitive attributes such as sex or race, then the updated model is not merely backward incompatible; it is also unfair in a very specific and previously under-studied sense. A new model might, for example, improve overall accuracy while systematically breaking predictions for a particular demographic group, effectively shifting the burden of progress onto those least able to bear it. The new paper formalizes this situation, defining Fair Backward Compatibility as a property that simultaneously constrains both the number and the demographic distribution of negative flips.

Formally, the framework treats the previous model as a reference point against which the updated model is measured sample by sample. For any instance on which the old model was correct, the new model should ideally remain correct; deviations are negative flips. Fair Backward Compatibility then requires that these deviations, whether measured in aggregate or within groups defined by sensitive attributes, remain within tolerable bounds. By unifying the previously separate literatures on backward compatibility and fairness-aware learning, the definition allows researchers and practitioners to reason about both concerns with a single set of tools, rather than trading one against the other in an ad hoc fashion.

Building on this conceptual foundation, the team proposes Fair Backward-Compatible Empirical Risk Minimization, or FBC-ERM, a learning framework that embeds fairness-aware backward compatibility directly into the training objective of essentially any modern machine learning algorithm. In classical empirical risk minimization, a model is trained by minimizing the average loss over the training data. FBC-ERM augments this objective with additional terms that penalize negative flips relative to a reference model, as well as disparities in how those flips are distributed across sensitive groups. The practical beauty of the approach, the authors emphasize, is that it requires only simple modifications of the cost function in standard models, using relaxations of the underlying constraints that are convex or at least differentiable. This means the framework can be plugged into familiar training pipelines, from shallow classifiers to deep neural networks, without redesigning the learning architecture from scratch.

A crucial theoretical contribution of the paper is a proof of statistical consistency. In machine learning, a learning procedure is statistically consistent if, as the amount of training data grows, the model learned from the sample converges to the best possible model for the underlying population. The authors show that both the empirical risk and the empirical Fair Backward Compatibility of the model learned by their framework converge to their population-level counterparts at the same rate as in classical empirical risk minimization. In other words, adding fairness-aware backward-compatibility constraints does not slow down or destabilize the fundamental statistical behavior of learning. This is a nontrivial result, because constrained objectives often complicate generalization guarantees, and it provides the mathematical assurance needed to trust the framework beyond toy problems.

The theoretical guarantees, however, would be of limited use without a practical way to tune and evaluate models that must balance two competing demands: minimizing risk and maintaining fair backward compatibility. To address this, the researchers introduce a new procedure for tuning and evaluating the performance of models with respect to both criteria simultaneously. Because a model that is perfectly backward compatible may sacrifice accuracy, and a maximally accurate model may generate many unfair flips, practitioners need a principled way to navigate this trade-off. The proposed evaluation procedure allows system designers to characterize this trade-off explicitly and to select the operating point that best fits their deployment context, whether that is a high-stakes domain where stability of predictions is paramount or a fast-moving application where accuracy gains justify some churn.

To validate the framework empirically, the team conducted experiments on real-world datasets using both shallow and deep architectures. The results confirm that Fair Backward-Compatible Empirical Risk Minimization can substantially reduce negative flips and their demographic disparities while retaining competitive predictive performance relative to conventional training. The experiments span different model families and different relaxations of the fairness and compatibility constraints, demonstrating the flexibility of the approach across convex and nonconvex settings. While the authors note that reducing negative flips typically involves some cost in aggregate accuracy, the framework makes that cost explicit and controllable rather than leaving it hidden in the noise of a model update.

The significance of this work extends well beyond the technical details. Machine learning systems are now updated continuously in production environments, from recommendation engines and speech recognizers to credit scoring and medical decision support. Users develop expectations, sometimes implicit, about how a system will behave; negative flips violate those expectations in tangible ways. When those violations cluster along lines of sex, race or other sensitive attributes, they raise concerns of discrimination and erode trust in automated decision-making. Regulatory frameworks, including emerging AI governance rules that emphasize transparency and non-discrimination, increasingly demand that system providers understand and justify the effects of their updates. A framework that quantifies and constrains the demographic distribution of update-induced errors offers a concrete technical instrument for meeting such expectations.

The paper also reframes a debate that has largely treated fairness and backward compatibility as separate objectives. Prior research on backward compatibility concentrated on bounding the total number of negative flips between model versions, while the fairness literature focused on equalizing error rates across groups in a single model. The generalized definition introduced here shows that these are complementary facets of a single question: how the burden of model improvement is distributed, both across instances and across groups of people. By proving that a single risk-minimization-style framework can handle both facets with classical statistical guarantees, the authors provide a theoretical bridge that future work on model updating, fairness auditing and responsible deployment can build upon.

The article, published open access under a Creative Commons license, arrived as a peer-reviewed, accepted manuscript shared early to provide faster access to the research, and it carries a permanent digital object identifier ensuring citability ahead of the final version of record. The authors report no conflicts of interest. As machine learning models continue to be retrained and redeployed at an accelerating pace across industry and public services, the question of who bears the cost of each update is no longer a philosophical aside; it is a measurable, optimizable quantity. With this work, the Genoa and Cagliari researchers have given the field both the language to describe the problem precisely and the machinery to do something about it, suggesting that the next generation of AI updates may be judged not only by how much better they are on average, but by how fairly they treat the people who were already being served well.

In recent years, the rapid evolution of AI has revolutionized scientific research methodologies. AI-driven models now facilitate the design of novel molecular syntheses, anticipate drug toxicity prior to clinical trials, and assist in orchestrating complex experimental procedures. These capabilities are transforming research paradigms by enhancing efficiency and enabling discoveries that were previously unattainable. However, the same AI innovations that accelerate beneficial scientific progress also harbor the potential for malevolent exploitation. Advanced AI systems like LLMs can inadvertently or deliberately generate detailed instructions for constructing hazardous chemical agents, posing real threats to public health and security.

The research team points out that the agentic nature of LLMs—which encompasses autonomous planning, multi-step reasoning, and the invocation of external data and tools—exacerbates these challenges. Traditional prompt-based AI interactions are no longer simple; instead, LLMs can actively strategize and execute complex tasks. This means that malicious users may craft prompts designed to circumvent naive safety measures, obtaining dangerous information concealed behind seemingly innocuous queries. Therefore, safeguarding scientific AI systems necessitates a more sophisticated approach than conventional content filtering or static rule enforcement.

To address these concerns, the scientists behind SciGuard sought to build a dynamic, LLM-powered agent that serves as an intelligent gatekeeper for AI-driven chemistry applications. Rather than modifying or restricting the foundational AI models—which might degrade performance or limit research flexibility—SciGuard operates as an independent, overlaying system. Upon receiving any user query, whether it involves molecular analysis or synthesis proposal, SciGuard interprets the request’s intent meticulously, cross-references scientific and regulatory guidelines, consults external databases encompassing hazardous chemicals and toxicological data, and applies relevant legal and ethical principals to determine whether a safe and responsible response can be provided.

This multi-layered assessment capability allows SciGuard to differentiate with remarkable precision between beneficial, legitimate scientific inquiries and potentially dangerous ones. For example, any request that could facilitate the production of a lethal nerve agent or prohibited chemical weapon is categorically denied. Conversely, genuine scientific questions—such as safe handling procedures for solvents or experimental protocols—are met with comprehensive, accurate, and scientifically justified responses drawn from curated databases, cutting-edge scientific models, and regulatory texts. This dual commitment to safety and utility is a hallmark of SciGuard’s design philosophy.

At the technological core, SciGuard functions as an orchestrator, employing LLM-driven planning combined with iterative reasoning and active tool usage. It not only retrieves pertinent laws and toxicology datasets but also performs hypothesis testing through integrated scientific models. This continuous feedback loop enables SciGuard to refine its plan according to intermediate findings, ensuring that final outputs are both secure and informative. Importantly, this dynamic adaptability sets SciGuard apart from more static or brittle content moderation techniques.

One of the most significant achievements of the SciGuard team lies in striking a delicate balance: enhancing AI safety without undermining scientific creativity or accessibility. To rigorously evaluate this balance, the researchers created a specialized benchmark named SciMT (Scientific Multi-Task), designed to challenge AI systems across a spectrum of scenarios encompassing safety-critical red-team queries, scientific knowledge validation, legal and ethical considerations, and resilience to jailbreak attempts. SciMT facilitates a comprehensive understanding of how models perform when navigating real-world tensions between openness and caution.

In systematic tests using SciMT, SciGuard consistently refused to output hazardous or unethical information while maintaining high levels of accuracy and usefulness in legitimate scientific dialogue. This equilibrium is vital, as overly restrictive safeguards risk stifling AI’s transformative contributions to research, whereas inadequate controls could allow disastrous misuse. By validating SciGuard against a diverse, realistic set of challenges, the team evidences a practical path forward for integrating intelligent safety frameworks into scientific AI applications.

While SciGuard’s initial implementation focuses on chemical sciences, the researchers emphasize the framework’s extensibility to other critical fields including biology, materials science, and potentially beyond. Recognizing the global nature of AI risks and the need for collective responsibility, the team has made SciMT publicly available to encourage collaborative efforts in research, policy development, and industry-driven safety initiatives. This openness aims to foster a shared ecosystem where innovation and security advance hand in hand.

The emergence of SciGuard arrives at a critical juncture when policymakers, scientists, and the broader public are increasingly concerned about the responsible deployment of AI technologies. In the realm of science, misuse carries direct consequences for public health and international security. SciGuard offers a preventive mechanism that not only blocks malicious exploitation but also builds trust by aligning AI systems with established human values and regulatory standards. This contribution sends a powerful message: safety and scientific excellence are not mutually exclusive but can be harmonized through thoughtful design.

Reflecting on the broader implications, the developers of SciGuard underscore that responsible AI goes beyond mere technical fixes; it is fundamentally about fostering trust between humans and technology. As AI systems grow more powerful and autonomous in scientific domains, maintaining this trust is essential for sustainable progress. SciGuard’s agent-based approach exemplifies how embedding ethics and safety into AI workflow can prepare the scientific community for an era where AI plays a central research role.

The findings and framework of SciGuard have been recently published in the international interdisciplinary journal AI for Science, an outlet dedicated to showcasing transformative AI applications that propel scientific innovation forward. By marrying rigorous safety protocols with state-of-the-art AI technologies, this work charts a promising course for future efforts to harness AI responsibly while amplifying its potential to accelerate discovery.

Reference: Jiyan He et al. 2025 AI Sci. 1 015002

Subject of Research: Safeguarding AI Utilization in Chemical Sciences using Agent-Based Frameworks
Article Title: AI Scientist Shielded: Introducing SciGuard to Secure AI in Chemistry
News Publication Date: 2025
Web References: https://mediasvc.eurekalert.org/Api/v1/Multimedia/cf53a160-07eb-4786-b664-acafa48c1431/Rendition/low-res/Content/Public
References: Jiyan He et al., 2025, AI Sci., 1: 015002
Image Credits: Overview of AI risks and SciGuard framework, courtesy of Jiyan He and Haoxiang Guan, University of Science and Technology of China.

Keywords

Artificial intelligence, chemical science, AI safety, large language models, agent-based safeguards, scientific AI, responsible AI, SciGuard, SciMT benchmark, AI misuse prevention, scientific innovation, computational chemistry