Explainable AI pipeline spots metastasising cancer cells in microscopy images

Researchers have built an explainable deep learning pipeline that can segment individual cells in large-scale fluorescence microscopy images and classify them as normal or metastasising, with model decisions explained through biologically meaningful visual evidence. The work, published in BMC Bioinformatics as an open access research article, is aimed at supporting the development of new cancer therapies, particularly for uncommon cancers such as those of mesenchymal origin, which typically receive less research attention but still require sufficient analysis.

The motivation, according to the authors, is that developing novel therapeutic approaches to cancers demands significant time and labour. A trustworthy analysis of tissue samples to identify cancer progression and cell response to candidate therapies is a key task in developing treatments and drugs. The pipeline is designed to deliver accurate, high-throughput and autonomous recognition of metastasising cells at low cost and with high trustworthiness, so that AI-assisted testing of how cancer cells respond to various treatments and drugs can facilitate the development of effective and bespoke cancer treatments.

Technically, the pipeline combines three stages. First, it uses pre-trained and domain-generalised segmentation models to identify single cells within microscopy images. Second, it trains the latest self-supervised transformers, specifically DINO-based models, to perform the normal-versus-metastasising classification. Third, it applies explainable AI methods to produce biologically meaningful model explanations, which the authors argue is crucial for the trustworthiness of model predictions in future clinical applications.

The headline performance results centre on the self-supervised DINO family of models. When only the classifier was trained on top of the pre-trained representations, DINO achieved the best predictability among the models tested, reaching an F1-weighted score above 97 percent, and showed the most efficient downstream adaptation. When the entire model was fine-tuned, all DINO-based models classified successfully, and DINOv3 achieved the highest F1-weighted score of the study at 98.35 percent. Compared against supervised learning-based models, the self-supervised approach came out ahead on these benchmarks.

Beyond the main classification test, the team ran an additional cross-magnification test and a comparison against self-supervised and weakly supervised models. These experiments were used to discuss the generalisation ability of the DINO-based models across imaging conditions and to argue for their superiority over alternative approaches.

Interpretability was addressed through heatmaps that highlight which parts of a cell drove the model’s decision. These explanations were analysed with both quantitative metrics and qualitative evaluation. The authors report that the important cell regions identified in the model's decision-making process align well with biological consensus.

One of the study’s notable findings is that biologically meaningful cytoskeletal features can be detected and exploited by the developed pipeline for cancer cell classification. The authors suggest that the ability of the pipeline to detect and exploit biologically meaningful cytoskeletal features facilitates the development of cytoskeleton-based diagnosis.

The overall conclusion drawn by the team is that the pipeline is precise, efficient and trustworthy, and that it can facilitate the development of effective and bespoke cancer treatments via AI-assisted testing of the response of cancer cells to various treatments and drugs. By automating the recognition of metastasising cells across large image sets while exposing the reasoning behind each prediction, the approach aims to reduce the time and labour burden of manual analysis while keeping humans able to check what the model is looking at.

As with any computational study reported at the stage of an accepted manuscript, the results describe model performance on the evaluated microscopy data and the interpretability analyses performed by the authors; the pipeline is presented as a research tool to support therapeutic development, with clinical applications described as a future direction rather than a demonstrated outcome. The article, whose authors include Jiabang Chen, Antonija Kezić, Ameer Alwadiya, Annica K. B. Gad and George Panoutsos, was published on 21 October 2026 and is freely available as open access.

Subject of Research: An explainable deep learning pipeline for segmenting and classifying cancer cells in fluorescence microscopy images

Article Title: A new explainable AI pipeline for cancer cell segmentation and classification for microscopy images

Article References: Chen, J., Kezić, A., Alwadiya, A., Gad, A. K. B., & Panoutsos, G. (2026). A new explainable AI pipeline for cancer cell segmentation and classification for microscopy images. BMC Bioinformatics. https://doi.org/10.1186/s12859-026-06680-0

Image Credits: AI Generated

DOI: 10.1186/s12859-026-06680-0

Keywords: explainable AI, deep learning, cancer cell segmentation, fluorescence microscopy, DINO, self-supervised learning, metastasising cells, cytoskeleton, cell classification, BMC Bioinformatics, computational pathology, drug response testing