The problem the researchers set out to solve is known as Automatic Exploit Generation, or AEG: the automated construction of functional payloads that turn a software vulnerability into a working attack. The field is more than a decade old, but early systems built on symbolic execution struggled to scale, and later template-driven approaches generalized poorly across architectures and mitigation schemes. Meanwhile, the pressure to automate has never been greater. Disclosed software vulnerabilities hit a record 49,000 in 2025, a 20.8 percent jump over the previous year, and manual exploit development remains a slow, cognitively punishing craft that demands deep fluency in program logic, compiler behavior, and low-level defenses such as address space layout randomization and stack canaries.

Large language models have recently shown flashes of offensive capability, from autonomously hacking websites to exploiting zero-day vulnerabilities, but the authors of the new study argue that a fundamental obstacle remains: what they call the Logical-Physical Gap. A language model works purely in text. It reasons about a program’s static, decompiled representation, yet it cannot see the actual, non-deterministic state of memory at runtime. Modern compilers and operating systems deliberately complicate that picture, introducing stack alignment padding, randomized addresses, and metadata that can silently invalidate an exploit built from static assumptions alone. The team illustrates the problem with a deceptively simple challenge in which a decompiled view suggests an overwrite offset of 116 bytes to reach a saved return address, while the compiled binary’s alignment instructions shift the true physical offset to 132. An agent that trusts the static view crashes the target and learns nothing.

PwnAgent’s answer is to treat exploit generation as a closed loop spanning three functions the authors label Sense, Model, and Diagnose. Rather than asking a single monolithic model to do everything — an approach that overloads context windows and invites hallucination — the system splits the work across specialized agents. A central MainOrchestrator maintains a global state and plans strategy, while three subordinate agents handle distinct domains: a ProgramAnalyzer that performs static semantic analysis using IDA Pro and binary metadata tools, a MeasurementExpert that probes live memory through the GDB debugger, and an ExploitCrafter that translates abstract plans into executable Python scripts built on the pwntools framework. The sub-agents are deliberately stateless and isolated from one another, communicating only through structured JSON payloads, a design the authors say prevents recursive reasoning loops and keeps the orchestrator’s context free of implementation noise.

A second pillar of the system is a hierarchical knowledge base distilled from 384 curated technical documents, including public capture-the-flag write-ups from non-benchmark challenges, CVE advisories, and exploitation tutorials. A long-context language model converted this raw material into a four-layer ontology: foundational principles covering calling conventions and protection mechanisms, thirty-two structured chain-of-thought reasoning rules for tactical planning, a dozen reference trajectories for few-shot guidance, and twenty documented pitfalls paired with corrective advice — for instance, recognizing that a crash on a 64-bit movaps instruction usually signals stack misalignment and calls for an alignment gadget. A human security expert audited the resulting knowledge base, and crucially, the team excluded any material corresponding to the benchmark tasks used in evaluation, freezing the knowledge base before testing to guard against leakage.

The knowledge is distributed through what the authors call cognitive sharding: each agent’s system prompt is pre-loaded only with the knowledge slice relevant to its role, roughly 33,000 tokens of strategy and trajectory material for the orchestrator versus about 5,400 tokens of measurement and diagnostic rules for the MeasurementExpert. The researchers chose static sharding over retrieval-augmented generation because the workflow is deterministic and each role’s recurring knowledge needs are known in advance, sidestepping retrieval noise and the well-documented tendency of language models to lose track of information buried in long contexts.

The operational workflow unfolds in five stages. The ProgramAnalyzer first extracts vulnerability primitives, interaction flows, and constraints from the binary alone. The orchestrator then selects an exploit strategy by activating rule chains whose preconditions match the observed state — for example, rejecting direct shellcode execution when the NX defense is enabled, or prepending an information-leakage sub-goal when PIE randomization is active — and decomposes the chosen path into a directed acyclic graph of atomic sub-tasks. The MeasurementExpert next resolves the graph’s unknowns with byte-level precision, calibrating stack offsets with cyclic patterns and inspecting heap layouts in GDB. The ExploitCrafter synthesizes the final script under those hard constraints, and a stratified feedback loop closes the cycle: if execution fails, the MeasurementExpert reconstructs the crash context, maps the symptom to a root cause, and the orchestrator routes the repair to the appropriate stage, re-measuring parameters, regenerating code, or abandoning the strategy entirely for a fresh plan.

To evaluate the system rigorously, the team built a 66-task benchmark of Linux ELF binaries drawn from established platforms, graded by three independent experts into easy, medium, and hard tiers, with success counted only when a generated exploit obtained an interactive shell or read the flag. The benchmark is deliberately capability-oriented rather than statistically representative: 46 of the tasks are stack overflows, with smaller subsets of format-string, heap, integer-overflow, ARM, and MIPS challenges serving as limited probes. Under the same recent Kimi-K2.6 language model backend, PwnAgent achieved a 62.12 percent end-to-end success rate, compared with 31.82 percent for the PwnGPT baseline — a gain of 30.30 percentage points. The advantage held across other backends as well, with PwnAgent reaching 33.33 percent under DeepSeek-V3.1-Terminus and 34.85 percent under GLM-4.6, against 19.70 and 18.18 percent respectively for the baseline. On the public 19-task PwnGPT benchmark, PwnAgent solved 18 of 19 tasks under Kimi-K2.6.

Ablation experiments sharpened the picture. Removing the knowledge base dropped success to 33.33 percent under Kimi-K2.6, while removing dynamic analysis and feedback repair dropped it to 42.42 percent, and removing both collapsed it to 24.24 percent. The knowledge base contributed the larger share, primarily by preventing strategic errors such as choosing techniques incompatible with active defenses, while dynamic measurement rescued the implementation-level failures — offset deviations, format-string index errors, misaligned ROP chains — that plague purely static approaches. The gains were concentrated in the stack-overflow subset and scaled with difficulty: under Kimi-K2.6, PwnAgent solved every easy task, 60 percent of medium tasks, and about a third of hard ones.

The authors are candid about the limits. The benchmark is x86- and stack-heavy, the non-x86 and heap subsets are too small to support broad claims, and public capture-the-flag material may overlap with language model training data in ways that task-level exclusion cannot eliminate. Failure analysis identified two persistent weaknesses: long-horizon strategy hallucination on complex multi-stage chains, and precision loss when translating sound plans into byte-exact scripts under ambiguous crash signals. Still, the study’s central lesson stands: structured domain knowledge, execution-grounded measurement, and diagnosis-driven repair can roughly double an AI system’s ability to autonomously exploit vulnerable binaries — a result that will energize both defensive teams racing to find flaws before adversaries do, and a broader debate about what increasingly capable offensive AI means for the future of software security.

Subject of Research: LLM-driven multi-agent systems for automatic exploit generation from binary programs

Article Title: Pwnagent: a knowledge-guided multi-agent system for automatic exploit generation

Article References: Wei, C., Geng, Y., Wang, Y., Wu, Q., Huang, J., Wu, Q., & Wei, Q. (2026). Pwnagent: a knowledge-guided multi-agent system for automatic exploit generation. Cybersecurity, 9(1), Article 224. https://doi.org/10.1186/s42400-026-00649-5

Image Credits: AI Generated

DOI: 10.1186/s42400-026-00649-5

Keywords: automatic exploit generation, large language models, multi-agent systems, binary exploitation, cybersecurity, capture the flag, GDB debugging, knowledge base, vulnerability analysis, PwnAgent, chain-of-thought reasoning, memory corruption