A recent study published in the Journal of Medical Imaging has shed new light on this problem, proposing a robust and practical framework to evaluate concordance among different AI-based anatomy segmentation models without relying on expert annotations as a gold standard. This work centers on chest CT images sourced from the National Lung Screening Trial (NLST), a widely used public dataset for cancer research, ensuring high relevance and applicability to deployed clinical and research scenarios.

The dilemma stems from the nature of public datasets like NLST, which, despite containing thousands of imaging volumes, lack comprehensive organ and bone segmentations. Manual annotations for such intricate structures are astronomically time-consuming and require highly skilled radiologists, rendering complete ground truth labeling impractical. AI models can generate these labels automatically, yet disparity arises because each model may use different terminology, boundary definitions, or anatomical inclusion criteria. Without a consensus or external standard, pinpointing the superior model has remained a vexing conundrum.

Addressing this, the investigators embraced a paradigm shift: they evaluated AI segmentation tools based on their agreement rather than absolute accuracy. The hypothesis is elegant—if independently developed models concur in labeling a structure, that concordance likely indicates a reliable and valid anatomical segmentation. Rather than seeking the elusive “correct” answer, the study quantifies where AI tools align and where they diverge on the same dataset.

Achieving direct model comparison necessitated a standardized baseline. The researchers selected six prominent open-source segmentation models, including TotalSegmentator (two versions), Auto3DSeg, MOOSE, MultiTalent, and CADS. Despite their differing original output formats and nomenclature, the team harmonized all results by converting them into an interoperable DICOM segmentation standard. Furthermore, they unified labels using the SNOMED-CT vocabulary—a widely accepted medical ontology—assigning uniform color codes and identifiers to anatomical regions. This harmonization enabled side-by-side visualization of segmentations from different models on the very same scan, facilitating accurate comparison.

To enhance accessibility, the study leveraged two powerful open-source platforms widely embraced in medical imaging research: OHIF Viewer, a browser-based tool, and 3D Slicer, a robust desktop application. They extended these viewers with bespoke integrations and plugins capable of displaying multiple segmentations simultaneously in three-dimensional and orthogonal two-dimensional views. This user-friendly interface allows researchers to interactively explore congruence and discrepancies among models for individual organs and structures with unprecedented ease.

The analytic phase focused on a carefully curated subset of 18 chest CT scans from different NLST participants. After filtering out partially imaged or inconsistently detected anatomical structures, the study concentrated on 24 key regions, including lung lobes, the heart, ribs, thoracic vertebrae, and the sternum. For each structure, the authors identified a “consensus” segmentation defined as the voxel set concurrently labeled by all models recognizing that anatomical part. Subsequent comparisons measured how each model’s output overlapped with this consensus region, employing metrics quantified shape similarity and volumetric congruence.

These quantitative results were further distilled into interactive plots enabling rapid identification of outlier models or scans exhibiting problematic segmentations. Notably, the team released a publicly accessible interactive website to disseminate these findings, inviting the broader research community to examine the detailed concordance metrics and underlying imaging data themselves, fostering transparency and collaborative refinement.

Results illuminated variable performance across structures. Lung segmentation demonstrated remarkable agreement, with high overlap and nearly indistinguishable boundaries across all models. This consistency highlights the maturity of lung segmentation technologies—likely a function of abundant training data and well-defined anatomical landmarks. In contrast, heart segmentations initially showed moderate concordance owing primarily to one outlier model adopting a narrower definition of the heart. Excluding this model markedly improved overall alignment among the remainder.

Bone structures revealed greater challenges. Four of the six models manifested frequent errors in rib and thoracic vertebrae labels, including merges of adjacent bones or misidentification of vertebral levels. Conversely, two models trained on distinct datasets produced notably more consistent and anatomically comprehensive segmentations. These subtleties eluded aggregate statistics but emerged clearly through simultaneous visual scrutiny, underscoring the indispensability of combined quantitative and qualitative evaluation techniques.

This investigation underscores a crucial insight: even highly cited AI segmentation models can harbor systematic weaknesses, particularly when trained on overlapping or limited data. It also validates a novel pathway for meaningful model assessment without the prohibitive cost of manual ground truth annotation. By integrating standardized atlases, ontology-driven label harmonization, automated voxelwise comparison, and interactive visualization, this framework provides a reproducible, scalable solution for evaluating medical imaging AI tools.

Beyond its immediate findings, this work promotes a vital cultural shift in biomedical AI research—from chasing a mythical single “best” model to embracing evidence-based decision-making informed by comparative strengths and weaknesses. The open availability of software, label mappings, and sample datasets offers the community an invaluable toolkit applicable not only to chest CT anatomy but extensible to other modalities and segmentation tasks.

As AI becomes integral to clinical workflows and population-scale studies alike, transparent evaluation frameworks like this will be indispensable. They empower data scientists, clinicians, and researchers to select segmentation models thoughtfully, gauge reliability, and appreciate limitations—ultimately enhancing the trustworthiness and impact of AI in healthcare.

In a landscape increasingly reliant on AI-generated annotations, the study by L. Giebeler et al. pioneers a path that balances rigor with practicality. Their approach bridges methodological divides, nurtures collaboration, and elevates the standard of medical image analysis through collective truth-seeking, even when classical ground truths remain elusive.

Subject of Research: Not applicable
Article Title: In search of truth: evaluating concordance of AI-based anatomy segmentation models
News Publication Date: 3-Apr-2026
Web References:

• https://www.spiedigitallibrary.org/journals/journal-of-medical-imaging/volume-13/issue-06/062204/In-search-of-truth–evaluating-concordance-of-AI-based/10.1117/1.JMI.13.6.062204.full
• http://dx.doi.org/10.1117/1.JMI.13.6.062204
  References:
• Giebeler L., et al., “In search of truth: evaluating concordance of AI-based anatomy segmentation models,” Journal of Medical Imaging, 13(6), 062204 (2026).
  Image Credits: L. Giebeler et al.
  Keywords: Artificial intelligence, Medical imaging, Anatomy, Anatomy segmentation, AI evaluation, Chest CT, National Lung Screening Trial, Open-source models, DICOM segmentation, SNOMED-CT, 3D Slicer, OHIF Viewer

Artificial intelligence (AI) has the potential to revolutionize many aspects of healthcare, and radiology is at the forefront of this transformation. The volume of imaging studies generated daily is overwhelming, creating an urgent need for more efficient means to analyze and interpret this wealth of data. AI algorithms, which can process vast quantities of information far quicker and often more accurately than human practitioners, are being harnessed to assist radiologists in making more precise diagnostics. This study underscores the importance of leveraging these technologies not merely for efficiency but to enhance educational outcomes and the quality of patient care.

Key to the research is the concept of “deliberate practice,” a term that encapsulates focused, goal-oriented training aimed at improving individual performance. In the context of radiology, deliberate practice involves not just repetition of imaging interpretations but also a targeted approach to understanding specific challenges and developing strategies to overcome them. This method has been shown to yield significant improvements in skill acquisition across various disciplines, and this study posits that its systematic integration into radiology training can similarly enhance diagnostic capabilities among practitioners.

The researchers employed a mixed-methods approach, combining quantitative data from AI algorithms with qualitative insights from interviews conducted with radiology professionals. By engaging with the experiences and perceptions of those within the field, the study closely examines the barriers and facilitators in adopting AI-driven educational frameworks. This approach fosters a deeper understanding of the complex interplay between technology and human expertise, shedding light on how best to implement these advancements in practice.

Preliminary findings are promising. Participants reported a heightened sense of confidence in their diagnostic abilities when utilizing AI tools in conjunction with deliberate practice techniques. Many noted that the integration of such technologies not only made their daily workloads more manageable but also enhanced their understanding of complex cases that may have previously posed a challenge. This synergy between AI and human expertise highlights the fundamental shift the study seeks to promote: instead of viewing AI as a replacement, there is an empowering opportunity to see it as an augmentative force that enhances human capabilities.

A central theme of the research involves outlining a framework through which radiologists can gradually incorporate AI into their daily routines. This framework is designed to be adaptable and responsive to the diverse needs of practitioners, fostering an environment where continuous learning is not just encouraged but effectively facilitated. The researchers advocate for a tailored approach, recognizing that the learning curves and needs of individual professionals will vary significantly.

The implications of this study extend beyond training. As hospitals and healthcare systems integrate AI technologies into their practices, the results of the research emphasize the importance of grounding these innovations in educational principles. This creates a sustainable ecosystem wherein radiologists not only leverage AI tools but also possess the critical thinking and analytical skills required to interpret and question the results these tools provide. The study advocates for a holistic integration of AI in radiology, one that prioritizes human intelligence as an indispensable component.

Crucially, the findings also highlight the significance of fostering a culture of collaboration among radiology professionals. A cooperative environment can encourage the sharing of insights and methodologies, ultimately leading to enhanced skill development across teams. This collaborative approach is essential as radiology continues to evolve in response to technological advancements and increased demands for precision and accuracy in diagnostics.

Moreover, the research indicates the potential for AI-driven tools to support not only radiologists but also other healthcare practitioners involved in patient management. By enhancing the interoperability and communicative capabilities of AI systems, the study opens avenues for improved interdisciplinary collaboration, ensuring that all members of the healthcare team can contribute effectively to patient outcomes. The multifaceted impact of these technologies suggests a transformative effect on the entire healthcare ecosystem.

As this research paves the way for future exploration, it also raises essential questions regarding the ethical implications of AI in healthcare. The ability to automate certain tasks must be balanced with the necessity for human oversight and clinical judgement. The responsibility of ensuring patient safety and the ethical management of sensitive healthcare data must remain paramount, guiding the implementation of AI technologies in medical practice.

In conclusion, Kelly and colleagues’ research marks a significant milestone in the intersection of artificial intelligence and medical education in radiology. By advocating for a shift from volume to value and emphasizing deliberate practice, they provide a roadmap for enhancing the precision of learning among radiologists. As the healthcare landscape continues to evolve, the recommendations outlined in this study will be instrumental in equipping future practitioners with the necessary tools and skills to navigate complex medical challenges. The move towards a more integrated approach in radiology is not just a trend but a necessity in the quest for excellence in patient care.

Through meticulous research and dedicated effort, this study illuminates a path forward for radiologists seeking to harness the full potential of artificial intelligence. It invites a reconsideration of existing educational frameworks and encourages proactive engagement with innovative technologies. The proactive response from medical education institutions and professional bodies will be pivotal in shaping the future of radiology and ultimately improving patient outcomes through enhanced diagnostic practices.

Subject of Research: The integration of artificial intelligence and deliberate practice in radiology education.

Article Title: From volume to value: leveraging artificial intelligence and deliberate practice to foster precision learning in radiology.

Article References:

Kelly, B.S., Duignan, S., Booth, C. et al. From volume to value: leveraging artificial intelligence and deliberate practice to foster precision learning in radiology.
Pediatr Radiol (2025). https://doi.org/10.1007/s00247-025-06470-5

Image Credits: AI Generated

DOI: 19 November 2025

Keywords: artificial intelligence, radiology, deliberate practice, precision learning, medical education

The AI system has been tested across the extensive 12-hospital network of Northwestern Medicine. Over a span of five months in 2024, the system undertook the analysis of nearly 24,000 radiology reports, scrutinizing and improving the efficiency of report generation. The findings from this significant study indicate an impressive average increase of 15.5% in the efficiency of creating radiograph reports, with some radiologists obtaining gains nearing 40%. Remarkably, these improvements in productivity do not come at the expense of clinical accuracy, a point underscored by the creators of the technology.

Dr. Mozziyar Etemadi, a key figure in this innovation, emphasizes that this development represents a landmark achievement in healthcare technology. He highlighted its uniqueness in that it demonstrably enhances efficiency within the healthcare sector, noting that similar technologies in other industries have not come close to delivering such substantial productivity boosts. The implications of these findings could ripple through the medical field, optimizing workflow and reshaping patient care dynamics.

Unlike conventional narrow AI systems that are limited to identifying specific conditions, Northwestern’s system employs a comprehensive approach. By evaluating the entirety of the X-ray or CT scan, it can automatically generate a report that is approximately 95% complete. This personalized report assists radiologists, who can fine-tune the output to suit each patient’s unique situation. The ability of the AI to deliver tailored reports significantly lightens the workload for radiologists, allowing them to focus on critical interpretations and decisions.

The immediate clinical applications of this technology are potentially life-saving, particularly in emergency situations where timely diagnostics are crucial. The AI actively monitors reports for urgent conditions like pneumothorax, signaling the presence of dire needs before a radiologist has the opportunity to examine the images. This immediate flagging system serves not only to enhance the efficiency of the radiology department but also to ensure that patients receive the necessary care without unnecessary delays—a critical factor in life-and-death situations.

The overwhelming clinical benefits are echoed by Dr. Samir Abboud, a co-author of the study and chief of emergency radiology at Northwestern Medicine. He cites the AI technology as a powerful ally in increasing efficiency. This enhancement allows medical professionals to triage cases more effectively, identifying urgent cases that require swift action. The pressing need for such innovations grows alongside anticipated shortages in the radiology workforce, projected to reach up to 42,000 by 2033 due to rising imaging volumes and insufficient training positions.

In developing this generative AI product, the Northwestern team prioritized an in-house approach, utilizing clinical data specifically sourced from within the Northwestern Medicine network. This strategic decision allowed for the creation of a nimble AI model tailored to the nuances of radiology, distinguishing it from larger, generalized models such as ChatGPT, which lack specificity for medical applications. The team’s commitment to developing custom AI solutions promises to democratize access and foster a future where health systems are less dependent on tech giants.

This approach not only enhances functionality and accuracy but also reduces the computational resources required to implement such an AI tool. For medical institutions, the study suggests that reliance on external technologies is not necessary, advocating for the empowerment of local systems and the cultivation of their own AI capabilities. The findings illuminate a pathway for other healthcare systems to harness AI technologies efficiently and economically, paving the way for a broader adoption in the medical field.

As the radiology sector faces mounting pressure, the Northwestern AI system arrives as a beacon of potential solutions to the challenges ahead. By facilitating faster diagnostic processes and introducing innovative tools to assist collision detection, the technology allows healthcare professionals to manage broader patient care responsibilities. Moreover, it is crucial to emphasize that notwithstanding the advancements brought by AI, the expertise and judgement of trained radiologists remain irreplaceable in ensuring the perfection of patient diagnoses and treatment choices.

Indeed, while the AI system heralds a new era of technological intervention in the radiological world, it is not intended to displace human expertise but to augment it. The collaborative interplay between AI capabilities and human oversight promises to maintain a high standard of care even as technological landscapes evolve. The team is also investigating the potential of the AI model to identify instances of delayed or missed diagnoses, such as those associated with early-stage lung cancer, further sharpening the focus on safeguarding patient health.

The implications of this study, with two patents already granted and more pending, signal a series of exciting developments on the horizon as the tool inches closer to commercialization. As the healthcare industry eagerly anticipates these revelations, the enthusiasm surrounding the generative AI system exemplifies the collaborative potential of technology and medicine in redefining patient care.

With radiology positioned at the crossroads of technological innovation and patient treatment efficacy, the strides taken by Northwestern Medicine could indeed serve as a model for future healthcare advancements. As organizations worldwide grapple with similar issues, the adoption of tailored, effective AI systems could become the linchpin for modernizing medical imaging and diagnostics.

In conclusion, as hospitals and health systems seek innovative strategies to transform healthcare delivery, Northwestern’s generative AI tool stands as a testament to the power of targeted technological interventions. The merging of artificial intelligence with real-world clinical applications opens a new chapter in radiological practice—one that could ultimately save lives and reshape the future of medical diagnostics.

Subject of Research: Not provided
Article Title: Efficiency and Quality of Generative AI–Assisted Radiograph Reporting
News Publication Date: 5-Jun-2025
Web References: Not provided
References: Not provided
Image Credits: Please credit animation to Northwestern University
Keywords: /Applied sciences and engineering/Computer science/Artificial intelligence, /Health and medicine/Medical specialties/Radiology