The clinical motivation behind the work is straightforward and consequential. Colorectal cancer remains one of the most common and deadly malignancies worldwide, and most of these cancers arise from polyps, small growths protruding from the lining of the colon that can be removed during routine colonoscopy before they turn malignant. Detecting and delineating those polyps accurately is therefore a direct determinant of whether a lesion is excised in time. Dermoscopy, meanwhile, is the dermatologist’s principal tool for the early detection of melanoma and other skin cancers, and the precise boundary of a lesion carries decisive diagnostic weight, informing decisions about biopsy, excision margins and follow-up. In both settings, clinicians today rely on manual or semi-manual outlining of lesions in images, a process that is slow, subjective and prone to inter-observer variability. An algorithm that could reliably draw those contours automatically would not only save expert time but could also flag subtle lesions that tired human eyes might miss.

Getting a computer to draw those contours, however, has proven remarkably stubborn. Polyps in endoscopic images vary enormously in morphology, size and appearance; some are flat and barely distinguishable in color from the surrounding mucosa, others sit at the edge of the frame, partially obscured by specular highlights, bubbles, or motion blur. Skin lesions present a parallel problem, with irregular, sometimes fading boundaries that blend into healthy skin. Early deep learning solutions, most famously the U-Net family of convolutional networks, achieved strong results by repeatedly downsampling and upsampling images and stitching fine local texture onto coarse semantic context. But convolutional kernels see only a small window at a time, so these networks struggle to connect distant parts of an image into a coherent global understanding, and they often produce ragged or leaking boundaries around small or low-contrast objects. Transformers, which use self-attention to let every pixel consult every other pixel, solved the long-range dependency problem but introduced a new one: the computational cost of global attention grows quadratically with the number of image tokens, making naive transformer segmentation expensive and sometimes imprecise at fine scales.

NAHFormer’s answer begins at the front of the network. The framework employs a pyramid-structured Mix Transformer, or MiT, encoder, a hierarchical backbone in which the image is progressively broken into larger patches through successive stages, producing a ladder of feature maps that descend in spatial resolution while ascending in semantic richness. Early stages capture delicate texture and fine edges, essential for tracing the exact rim of a polyp or lesion, while later stages encode the broader context needed to say, with confidence, that a given blob is a lesion at all. This multi-scale representation, the authors argue, is what underpins the model’s generalization capability, allowing it to cope with the extraordinary diversity of lesion appearances it encounters across different patients, cameras, imaging conditions and anatomical sites rather than overfitting to the quirks of any single dataset.

The first bespoke innovation sits above that encoder: a Cross-Resolution Semantic Perception, or CRSP, module. Its job is to make different layers of the pyramid talk to each other in a structured way, integrating semantic information across multiple resolutions so that the coarse layers’ understanding of what a lesion is can guide the fine layers’ placement of its boundary. The module performs this cross-resolution conversation through neighborhood attention, an attention mechanism in which each token attends not to the entire image but only to a small window of its nearest neighbors. That locality restriction, borrowed from the neighborhood attention transformer introduced by computer vision researchers in 2023, dramatically reduces computation compared with global self-attention while retaining most of the discriminative power, because in dense prediction tasks the pixels most relevant to resolving a given boundary are typically its immediate surroundings. The practical payoff, according to the team, is precise delineation of lesion contours and boundaries, precisely the capability where earlier transformer models have tended to stumble.

The second innovation, a Hierarchical Feature Fusion, or HFF, module, tackles a quieter but equally corrosive problem: redundancy. In multi-scale architectures, as features from different levels are combined, the same information is often carried forward repeatedly, and low-level detail can be swamped or diluted by what are effectively duplicate signals from coarser scales. The HFF module progressively aggregates the multi-scale feature hierarchy in a stepwise fashion, deliberately suppressing redundant information as it fuses, so that the final segmentation head receives a cleaner, more informative summary of the image. This redundancy-aware aggregation, combined with the boundary-sensitive attention of the CRSP module, is what the authors describe as the framework’s distinctive pairing of redundancy-aware and boundary-aware capabilities, a combination intended to raise accuracy without inflating the network into an unwieldy behemoth.

The experimental evidence spans the field’s most widely used public benchmarks. On the polyp side, the team evaluated NAHFormer on five colonoscopy datasets: Kvasir, CVC-ClinicDB, CVC-ColonDB, EndoScene and ETIS. These sets differ substantially in acquisition conditions and difficulty; CVC-ClinicDB contains relatively clean and well-lit frames, while ETIS is widely regarded as the sternest test, composed of small, poorly contrasted polyps that routinely drag down the scores of models that excel elsewhere. On the dermatology side, the framework was tested on the ISIC 2017 and ISIC 2018 skin lesion archives, curated through the International Skin Imaging Collaboration’s annual challenges for melanoma detection. The pattern of results is telling: NAHFormer did not merely top the leaderboard on the easy datasets but held its advantage on the hard ones, achieving its reported mean Dice score of 0.821 on ETIS and 0.909 on ISIC 2018, indicating that its architectural choices translate into genuine robustness rather than dataset-specific tuning.

The Dice coefficient, the metric at the center of these comparisons, measures the overlap between the algorithm’s predicted segmentation and the ground truth annotated by experts, ranging from zero for no overlap to one for perfect agreement, so the difference between a middling model and a strong one often comes down to whether the predicted boundary clips a sliver off the lesion or lets background tissue bleed in. Success on ETIS, where polyps are small and faintly demarcated, suggests that the neighborhood attention scheme is doing exactly what it was designed to do, pinning down delicate boundaries without the blurring that coarse feature fusion can introduce. The strong ISIC result reinforces the point from the opposite direction, since skin lesion boundaries are often biologically diffuse, fading gradually into surrounding skin, and demand a different kind of perceptual finesse. A single architecture scoring highly across both regimes hints at a level of adaptability that previous task-specific models, whether convolutional designs like PraNet or hybrid transformer convolutional systems such as TransFuse, Swin-UNet, MissFormer and H2Former, have generally had to trade off against one another.

The broader significance of the work lies in what it suggests about the trajectory of clinical AI. Rather than choosing between the precision of convolutional networks and the global reasoning of transformers, the Chongqing team’s design stitches the two virtues together, using a hierarchical transformer backbone, locality-restricted attention and progressive fusion to keep both computation and accuracy in a clinically usable range. All of the datasets used in the study are publicly available, which means other groups can immediately stress-test the claims and build on the architecture, and the work was supported in part by the National Natural Science Foundation of China under Grant 61701060 and by a Chongqing Graduate Scientific Research Innovation Project under Grant CYS25470. The researchers frame the system as a step toward segmentation tools that could eventually assist physicians in real time, flagging and outlining lesions as images are captured. For now, the result stands as a demonstration that rethinking where an attention mechanism should look, and what it should ignore, can push the frontier of medical image analysis forward on the benchmarks that matter.

Subject of Research: A transformer-based deep learning framework for segmenting colonoscopic polyps and dermoscopic skin lesions in medical images.

Article Title: NAHFormer: Neighborhood self-attention and hierarchical feature fusion transformer for image medical segmentation

Article References: Yin, X., Li, H., Hou, T., & Tang, C. (2026). NAHFormer: Neighborhood self-attention and hierarchical feature fusion transformer for image medical segmentation. Applied Intelligence, 56(15), Article 430. https://doi.org/10.1007/s10489-026-07474-w

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07474-w

Keywords: medical image segmentation, transformer, neighborhood attention, polyp segmentation, skin lesion segmentation, colonoscopy, dermoscopy, deep learning, computer vision, multi-scale feature fusion, Dice score, Applied Intelligence

The work, published in the journal Complex & Intelligent Systems, comes from an international team spanning SRM Institute of Science and Technology in India, the National Institute of Technology Rourkela, China University of Mining and Technology, Innopolis University in Russia, and St. Petersburg Electrotechnical University “LETI.” The team — M. R. Indresh, Soumyajit Gayen, Dmitrii Minenkov, Dmitrii Kaplun and Ram Sarkar — describes FAED, a Fuzzy Attention-aided Encoder-Decoder architecture, which swaps out the rigid binary logic of standard attention mechanisms for a soft, probabilistic notion of relevance inspired by fuzzy set theory. The results are striking not only for their accuracy but for the architecture’s remarkable frugality: with just 2.4 million parameters and roughly 4 GFLOPs of computation, FAED posts Dice scores that put it at the top tier of contemporary segmentation models while running at inference speeds measured in milliseconds.

The clinical stakes of this problem are easy to underestimate. Early-stage melanoma is highly curable, but the first line of defense is visual inspection of pigmented lesions, typically through dermoscopy — the imaging of skin through a magnifying device that reveals subsurface structures. Automated segmentation of dermoscopy images, the task of drawing an accurate boundary around a lesion, underpins every downstream measurement clinicians and computer-aided diagnosis systems rely on, including the asymmetry, border irregularity and color variation criteria used in melanoma risk scoring. Yet the task is plagued by low contrast between lesion and healthy skin, hair occlusions, specular reflections, and most fundamentally, ambiguous boundaries where the lesion fades gradually into its surroundings.

For years, the dominant tool for this job has been U-Net, a convolutional encoder-decoder architecture in which a contracting path extracts increasingly abstract features and an expanding path reconstructs a pixel-level prediction. The critical link between the two halves is a set of skip connections that pass fine-grained spatial detail from early encoder layers directly to the decoder. Most modern variants bolt attention modules onto these skip connections: the network learns to “gate” which features to pass through. But those gates are typically binary — a feature channel or spatial position is either kept or discarded. The FAED authors argue that this all-or-nothing logic is fundamentally mismatched to the nature of skin lesions, where the transition between sick and healthy tissue is gradual, not sharp. A binary gate discards exactly the soft, intermediate evidence that defines an ambiguous boundary.

FAED’s central innovation is its Boundary-conditioned Soft Fuzzy Attention (BSFA) module, which replaces standard skip connections altogether. Rather than multiplying features by a learned 0-or-1 mask, BSFA evaluates feature relevance using learnable Gaussian membership functions — mathematical constructs from fuzzy logic that assign each feature a continuous degree of membership, modeled as a probability-like value between zero and one. In practice, this means the network can express that a feature is “somewhat relevant” or “mostly relevant,” preserving graded boundary information that binary attention would crush. The Gaussian membership functions are themselves learnable parameters, so the network discovers its own notions of partial relevance during training rather than having them imposed by a fixed rule.

The architecture adds two further refinements that the authors show are individually and jointly important. The first is an Adaptive Fuzzy Mixture-based aggregation scheme. Features extracted at different depths of the network vary enormously in scale and semantics — shallow layers carry edge textures, deep layers carry abstract shape information — and fusing them well is a persistent headache in segmentation design. The fuzzy mixture approach treats each feature source as contributing to a soft ensemble, weighting its contribution according to a learned similarity-based membership rather than simple concatenation. The second refinement is an explicit Boundary Cue, a signal fed into the attention mechanism that modulates its focus along lesion perimeters. Where the fuzzy membership decides “how relevant” a feature is, the boundary cue tells the attention “where to look,” sharpening the model’s sensitivity precisely at the lesion border where errors are most costly.

The authors validated FAED on the four most widely used benchmarks in the field: the ISIC2016, ISIC2017 and ISIC2018 dermoscopy datasets from the International Skin Imaging Collaboration, and the smaller PH² dataset of melanocytic lesion images. The segmentation quality was measured with the Dice score, a standard metric that quantifies the overlap between the predicted lesion mask and the ground truth, where a score of 1.0 means perfect agreement. FAED achieved a Dice score of 0.9140 on ISIC2016, 0.9135 on PH², 0.8781 on ISIC2018, and 0.8615 on ISIC2017 — competitive-to-leading figures given the architecture’s size. Notably, the ISIC2016 and PH² results hover around the 0.91 mark, a level of overlap that corresponds to clinically meaningful boundary fidelity.

Just as important as the headline numbers is the efficiency profile, which the team documented with careful empirical measurements on an NVIDIA Tesla T4 GPU. FAED performs inference in 10.05 milliseconds per image at batch size 1, and 6.76 milliseconds per image when batched at 8 — throughput fast enough for real-time clinical workflows. Peak GPU memory during inference is similarly modest: 505 MB at batch size 1 and 948 MB at batch size 8. For context, many state-of-the-art segmentation models rely on heavyweight transformer backbones or large convolutional stacks with parameter counts in the tens of millions, demanding memory and compute budgets that make deployment on hospital hardware, edge devices or low-resource settings difficult. FAED’s 2.4 million parameters and roughly 4 GFLOPs place it in a different class entirely, suggesting that careful architectural design — rather than brute-force scale — can carry segmentation performance a long way.

To verify that each component of the design actually earns its place, the researchers conducted ablation studies, the standard experimental practice of removing parts of a system one at a time and measuring the drop in performance. These studies confirmed that both the prototype-based fuzzy aggregation and the boundary-conditioned modulation of attention contribute measurably to the observed improvements. In other words, the gains are not an artifact of added capacity or incidental tuning: the soft membership modeling and the explicit boundary guidance are doing real, distinguishable work. That finding matters for the broader field, because it offers evidence that how features are fused — treating fusion as a soft similarity-based membership problem — can be as consequential as how features are extracted.

The philosophical shift at the heart of FAED is worth dwelling on. Classical computer vision and early deep learning systems were built on crisp logic: a pixel belongs to a class, a feature passes a gate, a decision is yes or no. Fuzzy logic, introduced decades ago as a formal way of reasoning with degrees of truth, has long been touted as a natural fit for medical imaging, where human experts themselves reason in gradients — “this border looks slightly irregular,” “this region is probably part of the lesion.” What has changed recently is that learnable fuzzy components, such as Gaussian membership functions optimized end-to-end by gradient descent, can now be embedded inside deep networks so that the fuzzy rules themselves are discovered from data. FAED is a concrete demonstration that this marriage of classical soft-computing theory and modern deep learning can outperform hard-gated alternatives on a real clinical task, without any increase in architectural complexity.

The implications for melanoma screening are potentially significant, particularly for parts of the world where dermatologists are scarce and mobile screening programs depend on lightweight, fast and reliable algorithms. A model that runs in a few milliseconds on an entry-level GPU, fits comfortably in under a gigabyte of memory, and still achieves over 91 percent overlap with expert-annotated boundaries on benchmark datasets is precisely the kind of tool that can be embedded into telemedicine pipelines or portable dermoscope accessories. The authors caution, as all careful researchers do, that benchmark performance is a step toward clinical deployment, not the deployment itself — prospective validation on diverse skin tones, imaging devices and real-world lesion appearances remains an essential next stage for any segmentation technology destined for the clinic.

The article was published open access under a Creative Commons license, making the full technical description freely available to researchers and clinicians worldwide. The study was supported by the Ministry of Economic Development of the Russian Federation. As peer-reviewed, citable research made available early for faster dissemination, it joins a growing body of work arguing that the future of medical AI lies not only in ever-larger models, but in smarter ones — systems that, like the physicians they assist, know how to say “maybe.” FAED’s fuzzy attention may be an early but compelling example of that principle turned into working code.