How Your Database Architecture Shapes the Way Machines Tell Records Apart

Every time a hospital merges two patient files, a bank links a customer across three branches, or an online retailer recognizes that two product listings describe the same washing machine, an invisible computational process called entity resolution is at work. The task sounds deceptively simple: decide which records in one or more databases refer to the same real-world object, whether that object is a person, a company, a place, or a product. In practice, it is one of the hardest and most consequential problems in data management, and a new comprehensive review published in Knowledge and Information Systems argues that the field has been solving it with one eye closed. According to the authors, led by Nour Mekki of Djillali Liabes University in Algeria, together with Djamel Berrabah, Abdelhamid Malki, and Faouzi Boufarès of Université Sorbonne Paris Nord, most existing surveys of entity resolution focus on algorithms while largely ignoring the thing that ultimately determines whether those algorithms succeed or fail: the database management system they run on.

The review, published on 12 October 2026, makes the case that the choice between SQL and NoSQL architectures is not a mere implementation detail. It fundamentally shapes which entity resolution techniques are feasible, how they must be optimized, and how well they perform at scale. To capture this, the authors construct a three-dimensional classification framework that organizes the sprawling literature along three axes: the characteristics of the underlying data model, the system architecture on which resolution runs, and the target workload the technique is designed to serve. This database-centric lens, they contend, bridges a persistent gap between algorithmic innovation in research labs and practical deployment in real data infrastructures.

The intellectual roots of entity resolution stretch back decades. A landmark statistical theory of record linkage published by Ivan Fellegi and Alan Sunter in 1969 established the probabilistic foundations still used today, framing the matching decision as a formal hypothesis test between the possibilities that two records are a match or a non-match. Early systems such as the merge/purge work of Hernández and Stolfo in 1995 tackled the problem on relational databases, where rigid schemas, structured tables, and declarative query languages offered a relatively tidy environment. In that world, rule-based methods flourished: engineers wrote matching dependencies and similarity thresholds over well-defined attributes, and techniques such as sorted neighborhood indexing and canopy clustering kept the quadratic explosion of record comparisons under control through blocking, the art of cheaply grouping likely matches before expensive comparisons are made.

Relational databases, however, assume that data arrives clean, typed, and schema-conformant. The explosive growth of web-scale, heterogeneous, and semi-structured data shattered that assumption. NoSQL systems—document stores, key-value stores, column-family databases, and graph databases—emerged precisely to handle flexible schemas, horizontal scaling, and high-velocity ingestion. But that flexibility is a double-edged sword for entity resolution. When the same customer may appear as a nested JSON document in one store and a flat row in another, when attributes are optional, inconsistently named, or embedded in free text, the tidy rule-based machinery of the relational era strains. The review documents how techniques adapted: schema extraction methods, some using BERT embeddings to infer structure from schema-less document stores, attempt to recover the semantic scaffolding that relational systems get for free, while schema-agnostic approaches such as the MinoanER framework resolve web entities without any schema knowledge at all.

Scale is the second axis of the framework, and here the review traces a dramatic architectural migration. As datasets outgrew single machines, entity resolution moved to distributed paradigms. MapReduce-based systems such as Dedoop demonstrated how deduplication could be parallelized across Hadoop clusters, and later Spark-based approaches, including a probabilistic entity resolution pipeline published in this same journal, pushed throughput further. Parallel and distributed NoSQL platforms changed the calculus of blocking as well: probabilistic signatures, meta-blocking strategies that organize candidate pairs into a block graph, and progressive blocking techniques that surface the most promising comparisons first all reflect the need to make resolution tractable when billions of pairs are in play. The review also highlights real-time and on-demand variants, from inverted-index approaches for streaming resolution to systems that resolve entities lazily, only when a query actually needs them.

The third axis, workload, distinguishes batch deduplication from streaming, interactive, and progressive scenarios. Interactive deduplication using active learning, first explored in the early 2000s, lets a human oracle adjudicate the most ambiguous cases, and later work on online entity resolution formalized how to query that oracle efficiently. Incremental record linkage addresses the reality that databases change continuously, updating resolution results without recomputing from scratch. These workload distinctions matter enormously in practice: a nightly batch job consolidating a data warehouse faces very different latency and consistency constraints than a fraud-detection system that must flag a duplicate account the moment it is created.

Perhaps the most vivid thread in the review is the rise of deep learning. A 2018 design-space exploration for deep entity matching marked a turning point, and the field has since raced through sequence-to-sequence models for heterogeneous records, BERT-based matchers optimized for efficiency, transformer architectures for data integration, graph convolutional networks that exploit relational structure, and hierarchical graph attention networks. Pre-trained language models and their embeddings have become a lingua franca for comparing messy textual attributes, with experimental analyses cataloguing which embeddings work best for which matching problems. The frontier has now reached large language models: systems such as Booster use them to enhance resolution pipelines, in-context learning offers cost-effective matching without task-specific training, and ChatGPT has even been evaluated directly as an entity matcher. Zero-ER demonstrated matching with zero labeled examples, and ensemble methods squeeze further accuracy from transformer stacks. In a striking sign of the times, the review also covers hybrid quantum neural networks, which recent experiments suggest can dramatically reduce the need for hand-tuned free parameters in entity matching.

Graph databases receive particular attention as both a challenge and an opportunity. Because graph stores encode relationships as first-class citizens, they invite collective entity resolution, where decisions about one entity inform decisions about its neighbors. Techniques range from latent Dirichlet models and collective resolution in familial and multi-relational networks to graph differential dependencies and knowledge-graph completion pipelines that perform incremental multi-source resolution. The review shows how the data model itself becomes an algorithmic resource: in a property graph, the structure around a node is evidence about its identity, something a flat relational table cannot offer. Conversely, graph workloads introduce their own costs, since traversals and neighborhood computations can be expensive at scale, motivating fast on-demand resolution methods for property graphs.

The review does not shy away from the harder constraints now bearing down on the field. Privacy-aware deep entity resolution and federated resolution address the reality that sensitive records often cannot be pooled centrally, with encrypted-data deduplication pushing matching into the cryptographic realm. Vector databases, a rapidly maturing class of systems built for similarity search over embeddings, are emerging as a natural substrate for entity resolution at scale, and the review surveys their management systems and their use for identity deduplication. Knowledge bases and ontologies add another layer, with formal frameworks now combining entity resolution with query answering so that cleaning and consumption happen in a single coherent pipeline.

What elevates this review beyond a catalogue is its insistence on actionable guidance. By mapping each family of techniques—rule-based, probabilistic, machine learning, deep learning, and LLM-based—onto the data models, architectures, and workloads where it thrives, the authors give practitioners a decision framework rather than a leaderboard. A team running a strictly relational warehouse with stable schemas and moderate volumes may still be best served by declarative rules and sorted neighborhood indexing; an organization ingesting heterogeneous documents across a distributed NoSQL estate will likely need schema-agnostic blocking, embedding-based similarity, and Spark-scale parallelism; a graph-centric knowledge base opens the door to collective, structure-aware resolution. The authors, who received no external funding for the work and declare no competing interests, frame their contribution as closing the loop between algorithmic research and database-aware deployment. In an era when data is routinely called the new oil, their message is blunt: the refinery matters as much as the crude. Entity resolution is not one problem but a family of problems, and the database underneath is not passive plumbing—it is an active determinant of what can be matched, how fast, and how well.

Subject of Research: Entity resolution techniques across SQL and NoSQL database management systems

Article Title: Entity resolution across SQL and NoSQL database management systems: a comprehensive review

Article References: Mekki, N., Berrabah, D., Malki, A., & Boufarès, F. (2026). Entity resolution across SQL and NoSQL database management systems: a comprehensive review. Knowledge and Information Systems, 68(1), Article 281. https://doi.org/10.1007/s10115-026-02878-z

Image Credits: AI Generated

DOI: 10.1007/s10115-026-02878-z

Keywords: entity resolution, entity matching, SQL databases, NoSQL databases, data integration, data quality, deep learning, large language models, blocking techniques, graph databases, record linkage, database management systems