The work, authored by Chaofan Shen, Mingyu Wang, and Jing Zhang of Southeast University’s School of Cyber Science and Engineering in Nanjing, addresses a gap that has frustrated researchers since machine unlearning emerged as a field. Full retraining from scratch after every deletion request is the gold standard for forgetting, but it is prohibitively expensive for large graphs and models that may need to serve thousands of erasure requests. Approximate unlearning methods promise speed, yet existing approaches fall into camps with distinct weaknesses. Some partition the graph so that deleting a shard leaves the rest of the model untouched, but this disrupts the very structure that makes graph learning powerful. Others approximate removal directly in parameter space, editing weights with influence functions or closed-form corrections, but they can leave residual traces of deleted data. Still others rely on fixed neighborhoods when updating predictions, which cannot capture the fact that some nodes are far more sensitive to a deletion than others.

SURGE’s central insight is conceptual: a deletion request is fundamentally a prediction-level structural perturbation. When an edge disappears from a citation network, the predictions for nodes near that edge shift in ways that depend on how information propagates through the graph. Rather than guessing which parameters to edit, SURGE asks how the model’s outputs would change if the requested data were genuinely removed, and then repairs the model to match that ideal. The framework unfolds in three stages that the authors describe as Sense, Repair, and Distill.

In the sensing stage, SURGE estimates a continuous response field over all nodes in the graph. Instead of treating every node identically, the response field quantifies how strongly each node’s prediction is affected by the structural change introduced by the deletion. Nodes directly connected to removed data register strong responses; distant nodes register weak ones. This continuous, per-node characterization is what allows the method to express heterogeneous node sensitivity, something the authors argue fixed-neighborhood schemes cannot do. The response field effectively becomes a map of where forgetting must be deep and where it can be gentle.

The repair stage then solves for corrected logits, the raw pre-softmax outputs of the classifier, across the graph. Crucially, this is formulated as a graph-regularized optimization problem that is solved without backpropagation. Rather than running gradient descent through the network, SURGE draws on techniques from numerical linear algebra for sparse systems, computing corrections that respect both the deletion constraints and the smoothness structure of the graph. The graph regularizer ensures that corrected predictions remain coherent with their neighbors, preventing the kind of fragmented, inconsistent outputs that can arise when unlearning methods patch predictions locally. Because no gradients flow through the deep network, the repair step avoids the cost and instability of fine-tuning while still producing outputs that approximate what full retraining would yield.

The final stage, distillation, transfers the corrected knowledge back into the model. Knowledge distillation, a technique originally popularized for compressing large teacher networks into smaller students, is repurposed here as a forgetting mechanism. But SURGE adapts it: the distillation is response-adaptive, meaning the influence of the corrected teacher predictions on each node is weighted by that node’s position in the response field. Nodes that were strongly perturbed by the deletion receive aggressive correction, while nodes barely affected are nudged only lightly, preserving their original utility. This residual distillation is what gives the method its name and its balance between two competing goals: erasing the influence of deleted data and retaining accuracy on everything that remains.

The empirical case for SURGE rests on an unusually rigorous evaluation design. The authors tested the framework across seven standard benchmarks, including widely used citation corpora such as Cora, CiteSeer, and PubMed alongside larger graphs, and across multiple graph neural network backbones, including graph convolutional networks and graph attention networks. They evaluated three distinct deletion types: node unlearning, edge unlearning, and feature unlearning, covering the full range of erasure requests a deployed system might receive. To eliminate luck from the comparison, they ran a matched 18-cell evaluation spanning deletion types, datasets, backbones, and ten random seeds per configuration, comparing SURGE against three established baselines: ETR, an erase-then-rectify parameter editing approach; MEGU, a mutual-evolution unlearning method; and IDEA, a framework for certified graph unlearning.

The results are striking. Across the matched evaluation, SURGE achieved the highest mean micro-F1 score of 0.8632, indicating that models unlearned with SURGE retained the most predictive power on retained data. It also achieved the best retraining faithfulness, measured as a test Jensen-Shannon divergence of just 0.0243 against models retrained from scratch, meaning its post-deletion predictions stayed closest to the gold standard of full retraining. In other words, SURGE did not trade forgetting quality for utility or vice versa; it led on both axes simultaneously. The Jensen-Shannon divergence metric, rooted in Shannon entropy theory, provides a symmetric measure of how two probability distributions diverge, making it a natural yardstick for how faithfully an unlearned model mimics its fully retrained counterpart. An additional ablation study on ogbn-arxiv, a large-scale graph from the Open Graph Benchmark with millions of nodes, further supports the method’s scalability, suggesting the approach is not confined to small academic datasets.

The broader significance of this work lies at the intersection of privacy regulation and the economics of deployed AI. Membership inference attacks, demonstrated in influential security research over the past decade, can often detect whether a specific record was part of a model’s training data, turning residual traces into genuine privacy liabilities. As graph neural networks increasingly underpin systems that process personal relationships, transactions, and social connections, the ability to provably and efficiently remove an individual’s data becomes a compliance necessity rather than an academic curiosity. SURGE’s response-field formulation offers a template that could generalize: sense how a data change propagates, repair predictions to match the ideal, and distill the repair back into the model, all without the expense of retraining.

There remain open questions, as with any approximate unlearning method. The framework’s guarantees are empirical rather than cryptographic, and the field continues to debate what level of assurance suffices for regulatory compliance. Certified unlearning approaches offer stronger theoretical promises but often at greater cost or with more restrictive assumptions. Still, the combination of leading utility, near-retraining faithfulness, backbone-agnostic design, and demonstrated scalability on a graph the size of ogbn-arxiv positions SURGE as one of the most complete graph unlearning solutions reported to date. Funded by the Basic Research Program of Jiangsu, the study signals that the era of treating forgetting as an afterthought in graph machine learning is coming to an end. As deletion requests multiply and regulators sharpen their scrutiny, methods that let models forget precisely, quickly, and faithfully may become as fundamental to trustworthy AI as the training algorithms themselves.

Subject of Research: Efficient removal of nodes, edges, and features from trained graph neural networks without full retraining

Article Title: SURGE: Structural Perturbation Response Field Guided Residual Distillation for Graph Unlearning

Article References: Shen, C., Wang, M., & Zhang, J. (2026). SURGE: Structural Perturbation Response Field Guided Residual Distillation for Graph Unlearning. Machine Learning, 115(10), Article 239. https://doi.org/10.1007/s10994-026-07176-x

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07176-x

Keywords: graph unlearning, graph neural networks, machine unlearning, knowledge distillation, privacy, GDPR, data deletion, response field, Jensen-Shannon divergence, ogbn-arxiv, membership inference, Machine Learning journal

The study, led by Marko Pranjić and Boshko Koloski of the Jožef Stefan Institute together with Nada Lavrač, Senja Pollak, and Marko Robnik-Šikonja of the University of Ljubljana, quantifies just how badly things can go wrong. Working with the well-known 20 Newsgroups collection of roughly 20,000 documents, the researchers encoded the test split of 7,532 documents into 384-dimensional vectors using the SentenceTransformer model all-MiniLM-L12-v2 and measured cosine distance between the embeddings. When they built epsilon-threshold graphs, connecting every pair of documents closer than a given distance, they found that even the minimal distance required to keep the graph connected demanded around one million edges, and increasing that distance by only five percent inflated the edge count by more than sixty percent. Sparser is better for memory and speed, but sparser also means disconnected.

The k-nearest-neighbor approach fared somewhat better, since every document is linked to its k most similar documents, and tuning k to obtain a connected graph proved easier than tuning a distance threshold. But connectedness is never guaranteed. Theoretical work cited in the study shows that a k-NN graph is only asymptotically connected with high probability when k exceeds roughly 5.1774 times the natural logarithm of the number of points, which for as few as 300 data points already means k above 30, far beyond the small values used in real pipelines. In the experiments, some dataset partitions within the TwentyNewsgroups benchmark exhibited disconnected components even at k equals 15, and a Reddit sentence-clustering dataset remained disconnected at k equals 20. The high-dimensional geometry of embeddings makes matters worse: a phenomenon known as hubness means a few points appear in a disproportionate number of neighbor lists while others become isolated anti-hubs, skewing degree distributions and aggravating disconnection.

Why does disconnection matter so much? In spectral clustering, each connected component of the graph can only be assigned to a single cluster. When the number of components equals or exceeds the number of desired clusters, the clustering becomes trivial, and no similarity-based criterion can rescue it. The same fragility propagates beyond clustering: in item-based recommender systems, unreachable items simply cannot be recommended, and in label propagation, an isolated component of documents can never receive a propagated label. Connectivity, in other words, determines reachability, and reachability determines whether the downstream algorithm can function at all.

The researchers’ solution is elegantly minimal. Instead of letting every node search for neighbors across the entire dataset simultaneously, their incremental construction inserts documents one at a time, and each new node is linked to its k nearest neighbors among the nodes already present in the graph. That single restriction, searching only among previously inserted nodes, guarantees a connected graph for any value of k. The proof is a short induction: the first insertion creates a single connected component, and every subsequent node attaches to k existing members of that component, extending it without ever splitting it. Because the first node of a new region can only attach to earlier nodes, the graph remains one piece by construction, for every k and every insertion order.

The modification also confers a practical bonus that is increasingly relevant in the era of streaming data. In a standard k-NN graph, adding a single new document can trigger a large reconfiguration, since many existing nodes might count the newcomer among their nearest neighbors, effectively forcing a rebuild. The incremental construction induces only local changes, restricted to the row and column of the newly added node in the adjacency matrix. New documents can therefore be inserted without reconstructing the existing graph, and the last added nodes can even be deleted, properties the authors argue could be exploited in applications where data arrives continuously or becomes invalidated over time.

To test whether the guaranteed connectivity translates into better clustering, the team evaluated spectral clustering with Laplacian eigenmaps across eleven sentence- and paragraph-level clustering tasks drawn from six dataset sources in the Massive Text Embedding Benchmark, covering 182 distinct clustering problems in total. Performance was measured with V-measure, the harmonic combination of homogeneity and completeness. The results show a clear pattern: in the low-k regime, where standard k-NN graphs are most prone to fragmentation, the incremental approach consistently outperformed the standard construction, achieving near-top scores already at k equals 3 on many datasets, while matching the standard graph at larger k where both methods operate on connected structures. The largest gains appeared on TwentyNewsgroups, precisely where disconnection was most prevalent.

The team then compared their method against the most obvious alternative: repairing the standard k-NN graph by augmenting it with a minimum spanning tree, a global structure that stitches disconnected pieces together. Under matched MST augmentation, the incremental graph still held the edge in the sparse regime, improving on k-NN plus MST by an average of 2.5 V-measure points for k between 1 and 3, and by 3.8 points at k equals 1. A Bayesian signed-rank analysis with a region of practical equivalence confirmed that the MST is practically beneficial only in the sparsest regime and practically irrelevant from k equals 6 onward. Crucially, the exact MST computation requires forming the dense N-by-N distance matrix, an O(N-squared) step. The incremental construction avoids that matrix entirely because it attains connectivity through the insertion order rather than through added global edges, and it proved between 23 and 44 times faster to build than the MST-augmented alternatives, with peak memory dropping from as much as 20.2 gigabytes to as little as 1.2 gigabytes on the largest evaluated partitions, run on a CPU node of the Vega EuroHPC system.

Two natural objections receive careful treatment in the paper. First, because the incremental graph depends on the order in which nodes are inserted, is the method stable? Across ten randomized orderings per dataset, the standard deviation of clustering performance rarely exceeded one percent and was often below half a percent, and the structural statistics of the resulting graphs, including transitivity, assortativity, and label homophily, varied only in the third decimal place. Even adversarial orderings, in which the first inserted nodes were deliberately concentrated near the embedding centroid or drawn entirely from a single class, changed V-measure by at most 0.4 points relative to random ordering. Second, does the approach depend on the specific embedding model? Tests with larger models such as bge-base-en-v1.5 and all-mpnet-base-v2 showed that bigger encoders consistently improve results, though the benefit of denser graphs and larger models proved dataset-dependent, with Reddit clustering the notable exception.

The authors are candid about limitations. The five largest Reddit paragraph-level partitions exceeded the 32-bit index range of the sparse eigensolver and were excluded from some recomputed comparisons, and the robustness study covers concentrated initial seeds but not fully temporally ordered document streams, since the benchmark snapshots carry no timestamps. Future directions include replacing exact nearest-neighbor search with approximate methods, coupling the incremental graph with efficient eigenvector update techniques, and deploying the construction in temporal community detection. Still, the message is striking: a one-line change to how documents are inserted into a similarity graph eliminates a failure mode that has quietly lurked in spectral text clustering, delivering guaranteed connectivity, faster builds, lower memory, and better low-k performance, all without needing a proper metric, global graph features, or any repair step at all.

Subject of Research: Incremental k-nearest-neighbor graph construction for robust spectral clustering of text embeddings

Article Title: Incremental Graph Construction Enables Robust Spectral Clustering of Texts

Article References: Incremental Graph Construction Enables Robust Spectral Clustering of Texts. (n.d.). https://doi.org/10.1007/s10994-026-07158-z

Image Credits: AI Generated

DOI: 10.1007/s10994-026-07158-z

Keywords: spectral clustering, incremental graph construction, k-nearest neighbor graphs, text embeddings, Laplacian eigenmaps, graph connectivity, minimum spanning tree, Machine Learning journal, SentenceTransformers, MTEB, document clustering, high-dimensional data