AI Assistants Forget Your Rules Because Similarity Search Is the Wrong Tool, Study Finds

Imagine telling your AI coding assistant on Monday, “never use pandas in this project; use polars instead.” The assistant acknowledges. All week the conversation drifts through logging, refactoring, and columnar storage formats. On Friday you ask for a quick utility to deduplicate some rows, and the assistant cheerfully writes code that imports pandas. The instruction was stored. It was indexed. It simply was not retrieved. A new study argues that this everyday failure is not a glitch or a hallucination but a structural flaw baked into the way nearly every conversational AI system handles long-term memory.

The research, published as an open-access paper in the International Journal of Data Science and Analytics by Oghenenefe Abeke, Rasheed Mohammad, and Haitham Mahmoud of Birmingham City University, draws a sharp line between two kinds of information an assistant accumulates. Episodic facts, such as a bug discussed last Tuesday or a code snippet shared in passing, are exactly what similarity-based retrieval was designed for: you want the items most relevant to the current question. But the authors identify a second class they call governing facts, including constraints, preferences, identity statements, commitments, and assigned roles. These items do not answer a query; they bind every query. Their force is constitutive rather than topical, and that distinction, the paper shows, is where the dominant architecture breaks down.

The core of the argument is a small but rigorous impossibility result. The authors prove that any retriever which ranks stored items by a type-blind similarity score and returns only a bounded top-k set can be defeated by an adversarial sequence of topically related noise. Because similarity scores cannot distinguish a binding rule from ordinary conversation, an attacker, or simply a chatty user, can always generate enough topically adjacent material to push the rule out of the retrieval window. The paper complements this worst-case theorem with an expected-case degradation law: under random topical noise, the probability that a governing fact survives in the top-k follows a binomial tail that falls below one-half at a predictable threshold roughly equal to the retrieval budget divided by the per-item outranking probability. In plain terms, the longer the conversation and the more the topic drifts, the more certain it becomes that your rules vanish.

The empirical evidence is striking. Across 5,040 retrieval measurements spanning three scenarios, five governing-fact types, eight noise levels, and seven retrieval substrates, every similarity-ranked retriever tested collapsed. Sparse methods such as TF-IDF and BM25, along with hybrid reciprocal-rank-fusion combinations, held perfect fidelity at zero noise but dropped below 0.10 governance fidelity by 100 adversarial noise pairs, with a phase transition between roughly 25 and 50 noise pairs. Dense bi-encoders fared even worse in the small-encoder panel: bge-small-en-v1.5 had already fallen to about 33 percent fidelity at just six noise pairs, and e5-small-v2 reached a floor of exactly zero from 50 pairs onward. The very property that makes dense retrievers powerful for open-domain question answering, recognising topical similarity beyond lexical overlap, is precisely what the adversarial noise exploits.

A natural objection is that bigger, better encoders might solve the problem. A follow-up sweep tested bge-large and e5-large, each roughly ten times larger than the small models, plus a cross-encoder reranker applied to an enlarged candidate shortlist. The verdict: scale and reranking postpone the collapse but do not prevent it. The large encoders still fell steeply, reaching 0.067 and 0.200 fidelity by 200 noise pairs, and the reranker, which lasted longest, still dropped to 0.140 at 500 pairs. Importantly, the same degradation appeared under random topical noise rather than adversarial construction, ruling out an artefact of the benchmark design. The collapse, the authors conclude, is intrinsic to similarity ranking as such, not to any particular encoder or its size.

The remedy the authors propose is disarmingly simple in principle: type-conditional retrieval. Instead of ranking governing facts against noise, retrieve them unconditionally by their type and scope. They instantiate this as Stratified Context Reconstruction (SCR) in Satchel, a typed graph memory where each node carries a type drawn from constraint, preference, identity, commitment, role, or episodic, along with polarity, a target predicate, and a scope expression. At read time, the governing stage applies a type-and-scope filter with no similarity scoring at all, while the episodic stage runs whatever retriever the practitioner prefers. Governing facts reach the language model along a path that never passes through a similarity ranker, so governance fidelity is 1.0 by construction whenever write-time typing is correct. In the benchmark, SCR held fidelity at 1.0 across every noise level, with a Cohen’s d of about 1.06 against the similarity retrievers and a Bonferroni-corrected p value near 2.6 times ten to the minus twenty-fifth.

But the story has a second act. Governing facts get revised. A user forbids pandas on Monday, permits it in a legacy module on Wednesday, and reverses the exception on Friday. Naive SCR, which returns every in-scope governing fact unconditionally, now hands the model a contradictory rule set containing all three statements. The authors prove a second impossibility: no conflict-blind retriever, one that ignores supersession relations and validity intervals, can guarantee governance consistency, and the error grows linearly in revision depth. Their revision-depth benchmark confirms the prediction. Naive SCR returned up to fifteen stale superseded facts per query, and its governance minimality, the precision of the governing block, decayed to 0.25. The extended remedy, SCR-T, consults validity intervals and supersession edges to return exactly the currently binding set, and it is the only strategy that holds fidelity, consistency, and minimality jointly at 1.0 across every revision depth. Even production-style baselines such as pinned rule blocks and metadata-filtered retrieval replicated the temporal failure, accumulating eight stale facts per query at the deepest revision level.

Does the retrieval-layer property matter downstream? The authors tested four small open-weight language models and found that SCR improved constraint adherence over a TF-IDF retrieval-augmented baseline by 17.59 percentage points, with a 95 percent confidence interval of 13.19 to 21.76 and p below 0.001. A further controlled four-arm experiment under the paper’s own adversarial noise, crossing retrieval strategy with prompt formatting, showed that the advantage is a high-noise phenomenon: at zero and mild noise the two arms were indistinguishable, but as noise grew the RAG baseline’s constraint recall fell monotonically while SCR’s held steady, widening the gap to about 0.15 at 500 noise pairs. Because the advantage survived when both strategies were rendered in identical formatting, the effect is attributable to what SCR retrieves rather than how the block looks.

The authors are candid about limitations. The guarantee depends on write-time classification: deterministic templates captured about 78 percent of governing facts in their deployment, with a small LLM-as-judge classifier handling the rest at roughly 86 percent accuracy, putting realistic deployed fidelity in the 0.95 to 0.99 range. SCR-T adds a second write-time dependency, inferring supersession edges, though the authors stress that both problems are moved to a place where they are checkable and auditable in a typed graph, unlike an implicit recency policy buried in a vector store. The scenarios are scripted and English-only, the taxonomy of five governing-fact types may not be exhaustive, and no full system-to-system comparison against frameworks such as Mem0, Zep, or A-MEM was run, though the authors note those systems share the similarity-ranked substrate whose failure the paper formalises.

The broader implication is a challenge to how the field evaluates conversational memory. The authors propose three governance invariants, fidelity, consistency, and minimality, reported jointly alongside standard recall, plus traceability as a structural property of typed memory. Standard recall captures episodic competence; the invariants capture whether a system structurally honours the rules users entrust to it, completely, exclusively, and economically. On cost, the approach is practical: the governing block averaged 612 tokens, a 3 to 30 percent overhead on typical episodic budgets, and unlike budget-enlargement fixes, whose required retrieval window scales linearly with conversation length, the governing stage does not grow as sessions lengthen. As AI assistants are asked to remember more, for longer, the message of this paper is blunt: a rule that is not retrieved cannot be followed, and no amount of embedding quality will change that.

Subject of Research: Structural limitations of similarity-ranked retrieval for governing facts in long-term conversational memory of language model systems

Article Title: Beyond similarity retrieval: governance fidelity, consistency, and minimality in long-term conversational memory

Article References: Abeke, O., Mohammad, R., & Mahmoud, H. (2026). Beyond similarity retrieval: governance fidelity, consistency, and minimality in long-term conversational memory. International Journal of Data Science and Analytics, 22(1), Article 328. https://doi.org/10.1007/s41060-026-01290-8

Image Credits: AI Generated

DOI: 10.1007/s41060-026-01290-8

Keywords: conversational memory, retrieval-augmented generation, large language models, governance fidelity, similarity retrieval, typed graph memory, knowledge graphs, constraint adherence, supersession, evaluation benchmark, dense bi-encoders, BM25