Merlin represents a new paradigm in artificial intelligence within medical imaging—unifying vast, unlabeled datasets through the application of foundation models. Unlike conventional approaches restricted to narrowly defined tasks, Merlin’s training employed an extensive and unique dataset encompassing more than 15,000 clinically annotated 3D abdominal CT scans paired with corresponding radiology reports and nearly one million diagnosis codes. This expansive trove emanates from the Stanford University School of Medicine, forming the most comprehensive abdominal CT database assembled to date, thus enabling Merlin to learn sophisticated relationships between visual imaging and textual medical knowledge.

The strength of Merlin stems from its innovative architecture which facilitates the fusion of complex three-dimensional scan data with the semantic richness of natural language reports. This integration empowers the model to undertake over 750 distinct tasks, ranging from elementary anatomical delineation to the intricate prediction of disease development years before clinical manifestation. By harnessing multi-modal inputs during training, Merlin effectively bridges the gap between raw imaging data and diagnostic interpretation, a task that conventionally requires expert human radiologists supported by multiple rounds of clinical testing and evaluation.

Merlin’s performance was rigorously evaluated by challenging the model with over 50,000 previously unseen abdominal CT scans sourced from four independent hospitals. The model exhibited extraordinary proficiency in correlating imaging findings with human-generated diagnostic labels and conclusions. For example, Merlin’s ability to predict relevant ICD codes associated with individual scans surpassed other contemporary AI tools, achieving greater than 81% accuracy across a broad suite of diagnostic labels and peaking at 90% accuracy within certain disease subsets. These results underscore Merlin’s potential as a reliable clinical assistant in routine radiological workflows.

Beyond retrospective diagnostic tasks, Merlin demonstrates a remarkable capacity for forecasting future disease trajectories. In predictive tests focusing on chronic diseases—such as diabetes, osteoporosis, and cardiovascular illnesses—the model effectively identified individuals at elevated risk years before the clinical onset of disease based solely on their abdominal CT scans. Specifically, Merlin’s predictive accuracy reached 75%, outperforming comparator models operating at 68%. This ability suggests the presence of subtle imaging biomarkers, heretofore unnoticed by human experts, which Merlin is uniquely equipped to detect and interpret.

A particularly compelling facet of Merlin’s versatility is its adaptability to imaging domains outside its initial training data. Despite being exclusively trained on abdominal CT scans, Merlin was tasked with interpreting chest CT images—a domain with divergent anatomical and pathological features. Impressively, Merlin matched or exceeded the diagnostic performance of models specifically trained on chest imaging data, further evidencing its generalizability and the power of foundational learning approaches within medical AI.

Although Merlin is a “jack-of-all-trades,” competing with specialized models tailored for individual diagnostic tasks, it consistently matched or outperformed these experts. This comprehensive capability cultivates excitement for integrating Merlin into clinical practice not merely as a supplemental tool but potentially as a primary diagnostic aid. Its ability to reduce reliance on scarce radiological expertise may alleviate burgeoning physician shortages while streamlining diagnostic workflows, thereby accelerating patient care and treatment initiation.

Despite these advances, some tasks such as drafting complete radiology reports from scratch remain challenging and require further refinement of Merlin’s learning algorithms and fine-tuning with more targeted datasets. The research team advocates for continuous model refinement through domain-specific customization, encouraging practitioners to augment Merlin with local clinical data to enhance performance tailored to specialized clinical environments or demographic variations.

At its core, Merlin epitomizes a leap forward in multi-modal artificial intelligence research—combining the raw spatial complexity of volumetric CT data with the semantic depth inherent in diagnostic narratives. This confluence enables the model to understand and predict disease with a degree of nuance unattainable by previous generation AI systems. The synergy between data scale, model design, and diverse task demands positions Merlin as a foundational tool upon which future medical imaging innovations can be built.

This research, supported by several NIH institutes under multiple grants, also marks a pivotal collaboration between AI researchers and clinical scientists. It illuminates the potential for AI-driven tools not only to automate routine image analysis but also to reveal new medical insights, transforming radiology from a solely human-driven discipline into a synergistic human-machine partnership.

As the community begins to adopt and build upon Merlin, the implications span beyond immediate clinical applications. The model’s capacity to identify subtle patterns invisible to human eyes fuels optimism about discovering novel imaging biomarkers. Such biomarkers could inaugurate new frontiers in understanding disease pathophysiology, risk stratification, and personalized medicine, reshaping the landscape of preventative healthcare.

Ultimately, Merlin heralds a future where the integration of advanced AI models streamlines clinical decision-making, enhances diagnostic accuracy, and expands the role of medical imaging in health management. As senior author Akshay Chaudhari from Stanford University aptly noted, this foundational AI model is poised to be a robust backbone for the broader medical community, and from this platform, the potential applications are bound only by the limits of innovation itself.

Subject of Research: Medical imaging and machine learning application in computed tomography (CT) scan analysis.

Article Title: Merlin: A Computed Tomography Vision Language Foundation Model and Dataset

News Publication Date: 4-Mar-2026

Web References:
https://www.nature.com/articles/s41586-026-10181-8

References:
Louis Blankemeier, Ashwin Kumar, et al. Merlin: A Computed Tomography Vision Language Foundation Model and Dataset. Nature. 2026 DOI: 10.1038/s41586-026-10181-8.

Keywords

Health and medicine, Artificial intelligence, Medical imaging, Clinical imaging

At the heart of this pioneering research lies the integration of diverse imaging modalities—color fundus photography, infrared fundus imaging, optical coherence tomography (OCT), and optical coherence tomography angiography (OCTA). These technologies provide complementary views of the retinal structure and vasculature, enabling a comprehensive visualization of ocular changes induced by AMD. By leveraging these high-resolution images, the research team developed an advanced algorithm designed to distinguish between healthy retinal regions and those compromised by lesions.

Conventionally, the evaluation of visual function in AMD patients involves microperimetry, a technique that assesses light sensitivity across the macula. Although valuable, microperimetry can be onerous for patients, demanding prolonged attention and inducing fatigue. The novel machine learning model aims to mitigate these drawbacks by focusing testing on regions identified as lesion-prone, thereby reducing test duration and enhancing patient comfort without sacrificing diagnostic rigor.

The core analytical engine underpinning the study is a gradient-boosted tree-ensemble model, a powerful machine learning algorithm well-suited for handling complex, high-dimensional datasets. The researchers trained this model on an unprecedented dataset comprising over 344,000 distinct retinal regions extracted from the various imaging modalities. Such an extensive training set empowered the algorithm to learn subtle variations indicative of lesion pathology, underpinning its remarkable detection capabilities.

Results from the study are striking, demonstrating an area under the receiver operating characteristic curve (AUC) of 0.95. This metric signifies extraordinary accuracy in discerning end-stage lesions within chronic AMD cases, underscoring the model’s potential as a reliable diagnostic adjunct. The AUC value not only reflects high sensitivity and specificity but also heralds a new benchmark in automated lesion detection.

Importantly, the multi-modal imaging approach addresses the limitations inherent in relying on a single imaging modality. For instance, color fundus photographs excel at visualizing pigmentary changes but may miss deeper structural anomalies best captured by OCT. Conversely, OCT and OCTA deliver cross-sectional and vascular insights but can benefit from the contextual information provided by fundus images. The fusion of these data streams within an intelligent computational framework represents an elegant solution to the diagnostic challenges posed by AMD.

This integrative technique offers profound implications for personalized medicine in ophthalmology. By accurately mapping lesion locations, clinicians can tailor microperimetry tests to focus on vision-threatening areas, optimizing testing efficiency and patient adherence. Moreover, early and precise lesion detection can facilitate timely therapeutic interventions, potentially slowing AMD progression and preserving vision.

Beyond clinical utility, the study paves the way for incorporating artificial intelligence (AI) into routine eye care workflows. The automation of lesion detection could streamline screening programs, especially in resource-limited settings where specialist availability is constrained. Furthermore, the model’s adaptability suggests potential applications across a spectrum of retinal diseases beyond AMD.

While the study marks a significant leap forward, it also invites further exploration into integrating additional data types, such as genetic markers or longitudinal imaging, to enhance predictive accuracy. Future research may focus on refining the model’s interpretability and investigating its performance in diverse patient populations.

The fusion of cutting-edge imaging and AI heralds a new era in ophthalmologic diagnostics, moving closer to a future where retinal diseases like AMD can be detected earlier, managed more effectively, and patient outcomes vastly improved. This research underscores the transformative potential of machine learning in revolutionizing healthcare.

As the global population ages, the burden of AMD is projected to escalate, magnifying the demand for efficient and precise diagnostic tools. By combining multi-modal imaging with robust machine learning algorithms, researchers are charting a path towards meeting this critical clinical need, offering hope to millions affected by vision loss worldwide.

In summary, this innovative approach signifies a paradigm shift in age-related macular degeneration diagnosis and management. The study not only exemplifies the synergy between technology and medicine but also sets a precedent for future interdisciplinary endeavors aimed at combating complex ocular diseases.

Subject of Research: Age-related macular degeneration lesion detection using machine learning and multi-modal imaging
Article Title: Lesion detection in age-related macular degeneration with a multi-modal imaging and machine learning approach
Article References: Yap, C.L., Tan, T.F., Tan, A.C.S. et al. Lesion detection in age-related macular degeneration with a multi-modal imaging and machine learning approach. BioMed Eng OnLine 24, 111 (2025). https://doi.org/10.1186/s12938-025-01439-9
Image Credits: AI Generated
DOI: https://doi.org/10.1186/s12938-025-01439-9

Fundamental to their approach is the challenge of bridging the intrinsic differences between 2D and 3D imaging modalities. Conventional AI models typically specialize in either plane-based images such as chest X-rays or volumetric data like CT or MRI scans. However, real-world radiological practice demands versatility: clinicians interpret a mixture of both dimensional formats depending on the diagnostic scenario. The novel foundation model presented addresses this discord by adopting a multi-scale, multi-modal learning strategy capable of effectively ingesting and synthesizing diverse data types. This enables the AI system to understand and extract information regardless of dimensionality, providing a truly generalizable tool for radiologists across specialties.

Central to the success of this endeavor is the assembly of a vast web-scale dataset that includes more than ten million annotated imaging studies sourced from various institutions and regions worldwide. Data diversity is critical because medical images are influenced by equipment variability, patient demographics, and pathology spectrum. The team’s ability to collate such a heterogeneous assembly ensures that the model avoids overfitting to specific imaging conditions or patient populations. Moreover, rigorous preprocessing pipelines standardize the incoming data, enabling the model to focus on clinically relevant features rather than extraneous variations. This systematic global aggregation offers the foundation model an opportunity to learn universal radiological knowledge beyond localized nuances.

The architecture designed by the researchers leverages cutting-edge transformer-based neural networks, which have revolutionized natural language processing and are now making formidable inroads into visual domains. Transformer models excel in capturing long-range dependencies and contextual information, a crucial trait for interpreting complex anatomical structures across multiple slices or planes. By extending transformer frameworks to embrace both 2D and 3D contexts, the foundation model comprehends not only local pixel-level anomalies but also broader spatial relationships that signify pathological or physiological patterns. This capacity for integrated spatial reasoning is poised to enhance diagnostic accuracy substantially and reduce false positives commonly encountered with conventional convolutional neural networks.

One of the most compelling aspects of this breakthrough is the model’s universal applicability, effectively erasing the traditional boundaries between specialized radiology subfields. Where AI has historically required separate models trained on dedicated datasets for chest, abdominal, neurological, or musculoskeletal imaging, this generalist foundation model excels across these domains without bespoke tuning. Such an advance minimizes the need for multiple development pipelines, streamlines clinical implementation, and fosters operational efficiency. In practice, a single AI tool can assist radiologists by providing preliminary diagnoses, highlighting regions of interest, or suggesting differential considerations regardless of the underlying imaging modality or anatomical site.

The system’s unsupervised and semi-supervised learning techniques are particularly noteworthy given the scarcity and cost of obtaining exhaustive expert annotations. While fully labeled medical datasets remain scarce due to privacy concerns and resource limitations, the model capitalizes on unannotated or partially annotated data by deriving implicit supervisory signals from the imaging structure itself. This approach enables expansive training on unlabeled datasets, therefore greatly amplifying the volume and variety of input information. Consequently, the ABI foundation model’s training regimen surpasses the scale and complexity of previous approaches, heralding a leap toward truly intelligent radiological AI.

In-depth evaluation of the model’s performance was conducted using multiple benchmark datasets and clinical challenge tasks. These assessments addressed not only diagnostic accuracy but also robustness against adversarial distortions, image artifacts, and variations stemming from different scanner vendors. Impressively, the generalist foundation model demonstrated consistent superiority over specialized AI counterparts, particularly in complex clinical scenarios involving subtle lesions or overlapping pathologies. Furthermore, the AI’s decision-making transparency was enhanced through integrated attention visualization mechanisms, allowing users to trace critical regions influencing the model’s predictions. This feature augments clinical interpretability and trust, which remain paramount for AI adoption in healthcare settings.

Beyond immediate clinical diagnostics, the implications of this research extend into the realms of medical education, research, and healthcare equity. By providing an accessible, universal AI tool capable of interpreting diverse imaging data, the model supports ongoing training for radiologists in underserved regions lacking subspecialty expertise. It also serves as a powerful means to accelerate medical research by rapidly characterizing large cohorts of images to identify novel disease biomarkers or subtle imaging phenotypes. Finally, the model’s ability to generalize across demographic and technological disparities could play a key role in reducing health inequities exacerbated by differential access to expert radiological opinions.

The engineering efforts behind this project involved meticulous attention to the computational infrastructure to support web-scale training. The model’s developers leveraged distributed computing frameworks running on high-performance GPUs and TPUs, enabling simultaneous processing of petabytes of imaging data. Innovative memory optimization and parallelism techniques allowed efficient training without compromising model fidelity or scope. Importantly, the researchers also emphasized reproducibility by open-sourcing their code, pre-trained weights, and curated datasets where permissible. This transparency fosters collaborative advancements and contributes to building a sustainable AI ecosystem in medical imaging.

Security and ethical considerations were deeply embedded in the project design. The team implemented rigorous de-identification protocols ensuring patient privacy was uncompromised during data aggregation. Additionally, they developed mechanisms to detect and mitigate algorithmic biases, a critical challenge given the model’s wide deployment potential. They advocate for continuous monitoring and feedback loops involving clinicians to ensure the AI’s outputs remain aligned with evolving medical standards and societal norms. Thus, the foundation model is envisioned not simply as a static artifact but as a living tool evolving alongside digital medicine.

The conceptual framework and technical achievements of this foundation model instigate fresh conversations about the future trajectory of AI-assisted healthcare. The unification of 2D and 3D data modalities under one learning paradigm marks a significant conceptual leap, undermining previous compartmentalized approaches. This research illustrates that large-scale, integrated AI systems can now transcend the limitations of narrowly scoped models, ushering in a new generation of tools that embody adaptability, scalability, and profound clinical relevance. The notion of “generalist” AI in medicine may soon extend beyond radiology to other specialties, stimulating innovative multimodal learning paradigms comprehensively integrating medical data sources.

As radiology stands on the cusp of this AI revolution, hospital systems and diagnostic centers face choices in deploying such foundation models. The integration into clinical workflows demands careful orchestration involving human-computer interaction design, validation across diverse populations, and regulatory endorsement. Experts anticipate that this technology will not supplant human radiologists but will instead augment their capabilities, automating routine interpretation tasks and freeing medical professionals to focus on complex diagnostic reasoning and patient care. In this cooperative model, AI acts as an indispensable partner, elevating efficiency and maintaining rigorous diagnostic standards.

Though the current model heralds success, the team acknowledges challenges ahead. Future directions include expanding data diversity further to encompass underrepresented patient cohorts and rare diseases. There is also interest in integrating temporal imaging data such as dynamic contrast-enhanced sequences or serial imaging to model disease progression. Additionally, bridging radiological findings with other clinical data such as genomics, pathology, and electronic health records will be critical for holistic patient modeling and personalized medicine applications. Hence, this foundation model is a seminal step within a broader roadmap toward intelligent, multimodal healthcare AI.

The impact of this study reverberates beyond academia and industry into global public health. Rapid, accurate, and scalable imaging interpretation reduces diagnostic delays critical in diseases such as cancer, cardiovascular disorders, and infectious diseases. Deployable in diverse clinical environments from high-resource urban hospitals to remote clinics, AI-powered tools democratize access to expert-level radiological insights. Especially in times of healthcare crises or pandemics, such adaptable AI infrastructure can serve as a frontline diagnostic augmentation, facilitating timely interventions and improving patient outcomes worldwide.

Ultimately, the research by Wu, Zhang, and colleagues constitutes a masterstroke in the ongoing journey to harness AI’s transformative potential in medicine. By demonstrating the feasibility and advantages of a unified foundation model built on the synergy of 2D and 3D imaging data, they set a new gold standard for radiological AI. The fusion of web-scale datasets, sophisticated transformer architectures, and robust evaluation frameworks illustrates a visionary synthesis of technology and clinical pragmatism. As this technology matures and integrates into practice, it promises to redefine radiology’s landscape, empower clinicians, and most importantly, enhance patient care on a global scale.

Subject of Research: Development of a generalist foundation model in radiology integrating web-scale 2D and 3D medical imaging data for enhanced diagnostic AI applications.

Article Title: Towards generalist foundation model for radiology by leveraging web-scale 2D&3D medical data.

Article References:
Wu, C., Zhang, X., Zhang, Y. et al. Towards generalist foundation model for radiology by leveraging web-scale 2D&3D medical data. Nat Commun 16, 7866 (2025). https://doi.org/10.1038/s41467-025-62385-7

Image Credits: AI Generated

Coal workers’ pneumoconiosis, often known as “black lung disease,” represents a severe health hazard for miners exposed to coal dust. Traditional diagnostic methods have heavily relied on chest X-rays, which struggle to capture the nuanced and intricate lung changes associated with CWP. This limitation has hindered timely and accurate diagnosis. To overcome this, the new research pivots towards analyzing high-resolution computed tomography images, which provide more detailed views of pulmonary structures, ensuring subtle pathological changes do not go unnoticed.

The central innovation lies in leveraging a cutting-edge deep learning model called DenseNet combined with an Efficient Channel Attention Network (ECA-Net). This hybrid model exploits the intricate patterns and spatial hierarchies in HRCT images, enabling the effective distinction between different clinical manifestations of pneumoconiosis. The model was trained using an extensive dataset gathered from 217 patients with confirmed CWP and dust-exposed workers, allowing it to learn and recognize the complex imaging signatures unique to this disease.

The research team painstakingly annotated more than 1700 regions of interest (ROIs) within the HRCT images, categorizing them into four distinct clinical imaging features. These categories include small miliary opacities, nodular opacities, interstitial changes, and emphysema, each representing different pathological patterns indicative of disease severity and progression. By incorporating a robust data augmentation strategy, the researchers enhanced the dataset’s diversity, enabling the model to generalize well and maintain high performance across varying imaging scenarios.

In rigorous testing utilizing tenfold cross-validation, the DenseNet-ECA model achieved an extraordinarily high average area under the receiver operating characteristic curve (AUC) of 0.98, demonstrating exceptional discriminatory power. Remarkably, each imaging feature was classified with an AUC exceeding 0.92, underscoring the model’s consistent precision. Nodular opacities and emphysema, in particular, were identified flawlessly with AUCs of 1.0, reflecting zero classification errors in these categories.

This performance marks a significant leap forward in the computational analysis of pulmonary diseases. The integration of attention mechanisms via ECA-Net allowed the model to focus selectively on the most informative channels in the image data, thereby improving feature representation without the computational overhead typically associated with attention models. DenseNet’s densely connected pathways further enhanced gradient flow and feature reuse, which bolstered the training efficiency and accuracy of this deep learning framework.

Beyond technical prowess, the practical implications for clinical radiology are profound. Automated, reliable classification of HRCT images can provide radiologists with invaluable diagnostic support, reducing human error and diagnostic time. This is particularly critical in regions burdened with occupational lung diseases where specialist expertise may be limited. The algorithm’s ability to discern subtle imaging variations promises to improve early detection, monitor disease progression, and tailor intervention strategies more effectively.

Furthermore, the use of HRCT imaging mitigates the diagnostic ambiguity often encountered in chest X-rays, where overlapping anatomical structures obscure lung details. However, the sheer volume and complexity of HRCT images typically demand considerable time and expertise for manual interpretation. This novel algorithm addresses this bottleneck by automating classification, enhancing diagnostic throughput, and allowing clinicians to focus on patient care and treatment optimization.

While the study primarily focuses on CWP, the framework’s flexibility suggests broader applications across other interstitial lung diseases and occupational respiratory conditions. The research opens avenues for exploring similar deep learning approaches in diseases where imaging plays a pivotal role but remains challenging due to complex presentation patterns. This could spearhead a new era of AI-assisted diagnostics in pulmonary medicine.

Importantly, the researchers underline that their approach does not replace clinical judgment but rather augments it. By providing highly accurate, transparent, and reproducible assessments of HRCT features, the tool acts as a second set of eyes, enabling radiologists to confirm their impressions or identify findings that might otherwise be overlooked. This synergy between AI and human expertise exemplifies the future direction of medical imaging diagnostics.

The successful trial of this DenseNet-ECA model was registered with the Chinese Clinical Trial Registry, underscoring its methodological rigor and clinical relevance. The registration details further enhance transparency and encourage future research collaborations aimed at refining and validating the model across diverse populations and imaging modalities.

As occupational health continues to grapple with the consequences of industrial exposure, innovations such as this deep learning-based classification algorithm herald transformative progress. By harnessing artificial intelligence and high-resolution imaging, the medical community moves closer to eradicating diagnostic uncertainty, ultimately improving outcomes for countless individuals affected by coal workers’ pneumoconiosis worldwide.

This study, published in the prestigious journal BioMedical Engineering OnLine, not only exemplifies interdisciplinary collaboration between engineering and medicine but also spotlights the pivotal role of AI in addressing complex healthcare challenges. With continued development and integration into clinical workflows, such technologies promise more personalized, timely, and accurate diagnostic processes, reshaping the future landscape of pulmonary disease management.

Subject of Research: Deep learning-based classification of high-resolution computed tomography features in coal workers’ pneumoconiosis

Article Title: Deep learning-based algorithm for classifying high-resolution computed tomography features in coal workers’ pneumoconiosis

Article References:
Dong, H., Zhu, B., Kong, X. et al. Deep learning-based algorithm for classifying high-resolution computed tomography features in coal workers’ pneumoconiosis. BioMed Eng OnLine 24, 7 (2025). https://doi.org/10.1186/s12938-025-01333-4

Image Credits: Scienmag.com

DOI: https://doi.org/10.1186/s12938-025-01333-4