Lung needle biopsies are among the most common and consequential procedures in interventional radiology. When a radiologist spots a suspicious nodule on a chest scan, the definitive diagnosis usually requires extracting a sliver of tissue through the chest wall, guided by computed tomography. The challenge is geometric as much as medical. The needle must reach the lesion while dodging ribs, avoiding the mediastinum and its great vessels, steering clear of pulmonary fissures, and traversing as little healthy lung parenchyma as possible. Every extra millimeter of lung crossed raises the risk of pneumothorax, the collapsed lung that occurs when air leaks through a puncture, as well as bleeding and hemoptysis. Yet despite decades of advances in imaging, the choice of trajectory still varies considerably from one radiologist to the next, a variability that AI and robotics have long been expected to tame.

The new study, led by Taylor Hoffman and Francois H. Cornelis with colleagues including Stephen B. Solomon and Debkumar Sarkar, set out to test whether a general-purpose generative AI model could shoulder part of that planning burden. The team conducted a retrospective, institutional review board-approved analysis of de-identified CT images from 30 consecutive lung biopsies performed by three interventional radiologists, each with at least a decade of experience. The cohort included 17 men and 13 women with a median age of 65 years, and the lesions ranged from 5 to 45 millimeters across, deliberately spanning technically difficult territory: five apical lesions at the top of the lung, seven subpleural nodules hugging the chest wall, three para-mediastinal lesions, and four near the lung base.

The technical setup was elegantly simple. Each axial CT image, acquired on a GE Healthcare scanner and exported in DICOM format, was cropped and magnified but not rescaled. Using the open-source image editor GIMP, the team overlaid a radial grid on the image and converted it to PNG. One author annotated the target lesion with a blue circle and outlined the sensitive structures in red, including the mediastinum and major vessels. The annotated image was then uploaded to ChatGPT-4 with a standardized prompt specifying five key parameters for an optimal entry angle: reaching the target, minimizing the distance traveled through lung tissue, and avoiding bones, fissures, and the mediastinum. The model was asked to read the radial grid and return the angle at which the needle should enter.

Before running the full analysis, the researchers validated the approach by comparing a pre-biopsy planning CT image with a procedural CT image captured during the actual biopsy, acquired at the same anatomical level and showing the needle, skin, and target in place. A separate validation dataset of 30 paired planning and procedural images was then used for data collection. Each case was processed independently in a fresh chat session, with iterations allowed whenever the model failed to incorporate all criteria. The median number of iterations needed to arrive at a complete answer was just two, with an interquartile range of one to three, suggesting the model could converge on a usable plan with minimal back-and-forth.

The results were striking. Independent reviewers, two experienced interventional radiologists who had not performed the original procedures and who assessed the AI-generated paths in blinded fashion, judged 28 of 30 paths, or 93.3 percent, safe and adequate for performing the biopsy. The two rejected paths were deemed theoretically feasible but not representative of best clinical practice: in one case the AI path crossed 20 millimeters of lung tissue, four times longer than the manual route, and in another it unnecessarily punctured lung parenchyma to reach a subpleural nodule, raising concerns about needle stability. More remarkably, the AI paths crossed a median of just 5 millimeters of lung tissue, compared with 23 millimeters for the paths the human radiologists actually chose, a difference that reached statistical significance with a p-value of 0.0063 on a Wilcoxon signed-rank test. Paths with zero lung traversal corresponded exclusively to subpleural nodules, where a direct pleural puncture enters the lesion immediately.

On the question of whether the AI simply rediscovered what the humans had already chosen, the answer was nuanced. The mean entry angle suggested by the AI was 179 degrees, against 174 degrees for the manual procedures, a difference that was not statistically significant. Using a definition of path match requiring entry angles within 10 degrees of each other and similar trajectories through tissue planes, 8 of 30 cases, or 26.7 percent, showed good concordance. The authors note that this match rate is lower than in earlier work, such as a study by Too and colleagues that used a three-dimensional convolutional neural network for lesion detection and path planning and reported 82 percent concordance with an average angular deviation of 2.30 degrees, alongside 93.5 percent sensitivity and 93.2 percent specificity in lesion detection. That earlier system, however, was purpose-built for the task rather than adapted from a general-purpose chatbot, and it defined path match more narrowly, within 5 degrees.

Why should a shorter path matter? The authors argue that reduced traversal through lung tissue could lessen tissue manipulation and lower the risk of pneumothorax and hemoptysis, potentially shortening procedure time, improving patient comfort, reducing anesthesia use, and increasing operating room efficiency. In the 30 procedures studied, no major complications occurred, and minor events were self-limiting: minimal pneumothorax in four cases, 13.3 percent, which resolved spontaneously, and mild hemoptysis in two cases, 6.7 percent. But the authors also flag a clinical caveat. With a coaxial technique, in which an outer introducer needle anchors the pathway for repeated sampling, a very short path through lung tissue might provide insufficient purchase, increasing the risk that the needle dislodges during the procedure. Shorter, in other words, is not automatically better in every scenario.

The study is candid about its limits. With only 30 cases and a retrospective design, generalizability remains unproven. The analysis was performed in two dimensions on single axial slices rather than in three dimensions, a simplification that may underestimate the true spatial complexity of biopsy planning. The prompt did not require the AI to maintain a standard 5-millimeter safety margin from the subclavian and axillary neurovascular bundles, though those structures were kept out of the imaging field because patients were scanned with their arms raised. Iterative follow-up prompts were not scripted, which limits reproducibility, and the study measured technical feasibility only, not clinical outcomes such as diagnostic yield, complication rates, or procedure time. The authors also caution that GPT-4 itself may soon be superseded, arguing that the field should move toward specialized generative models trained specifically on interventional radiology rather than adapting general-purpose systems.

Even so, the trajectory of the technology is hard to ignore. The authors envision generative AI integrated with robotic biopsy platforms to automate trajectory planning, real-time procedural guidance in which the model continuously analyzes intraoperative imaging and suggests adjustments as the patient moves or breathes, pre-procedural virtual rehearsals that let clinicians simulate multiple approaches before committing to one, and eventual three-dimensional reconstruction that would eliminate manual image annotation altogether. Combined with augmented reality, AI-recommended paths could be projected directly into the clinician’s field of view. The same logic extends beyond the lung to tumor ablation, where precise positioning determines whether the entire tumor is covered, and to vascular interventions, where AI could recommend optimal access routes based on patient-specific anatomy. For now, the message of this small feasibility study is measured but provocative: a model built to converse with humans can, with a radial grid and a well-crafted prompt, plan a needle path through the lung that is measurably less invasive than the one a seasoned specialist picked. The needle, for the moment, remains in human hands, but the map may increasingly be drawn by machines.

Subject of Research: Feasibility of using a multimodal large language model to plan entry angles for CT-guided lung needle biopsies

Article Title: Feasibility study on using multimodal large language model for CT-guided lung biopsy trajectory planning

Article References: Hoffman, T., Worthington, M., Sarkar, D., Solomon, S. B., & Cornelis, F. H. (2025). Feasibility study on using multimodal large language model for CT-guided lung biopsy trajectory planning. CVIR Oncology, 1(1), Article 5. https://doi.org/10.1007/s44343-025-00005-3

Image Credits: AI Generated

DOI: 10.1007/s44343-025-00005-3

Keywords: generative AI, ChatGPT-4, lung biopsy, CT-guided biopsy, trajectory planning, interventional radiology, pneumothorax, needle path planning, multimodal large language model, Memorial Sloan Kettering, CVIR Oncology, medical imaging

TrafficPerceiver represents a significant leap forward by integrating textual instructions with visual data within a unified multimodal Transformer architecture. Unlike conventional perception frameworks that rely on isolated, task-specific decoders for semantic comprehension and segmentation, TrafficPerceiver seamlessly aligns linguistic commands and image features. This design facilitates natural language-guided reasoning and allows the framework to generate pixel-level target segmentation based on explicit textual queries, enabling nuanced, interpretable scene analysis that reflects human intent.

At the core of TrafficPerceiver’s innovation lies the introduction of a special segmentation token within its Transformer-based model. This token acts as a cognitive bridge that directly associates textual instructions with relevant spatial regions in input imagery. By doing so, it obviates the need for adding separate task-specific segmentation heads, streamlining the architecture and enhancing computational efficiency. This token-driven alignment empowers the system to isolate individual traffic participants or infrastructural elements precisely, such as differentiating a single vehicle from surrounding pedestrians or identifying road signs amid cluttered urban environments.

Robustness in degraded visual conditions is paramount for any real-world traffic perception system. The research team addressed this by incorporating an advanced reinforcement learning strategy rooted in Group Relative Policy Optimization (GRPO). Distinct from standard absolute score maximization, GRPO evaluates the model’s responses relative to a cohort of sampled outputs within a shared group context. This relativity-focused training fosters consistent and stable adherence to natural language instructions, especially when input images suffer quality loss from rain splatter, fog, low light, or motion-induced blur, thus establishing a new benchmark for stability in adverse scenarios.

Recognizing the scarcity of datasets tailored to complex, adverse traffic environments, the researchers developed the Challenging Traffic Scene Understanding (CTSU) dataset. CTSU is meticulously curated to encompass an array of realistic traffic complexities including diverse weather phenomena, variations in illumination, occlusion instances, and regional traffic structural differences. Crucially, the dataset is enriched with paired language instructions, detailed textual responses, and pixel-accurate segmentation annotations, providing an invaluable resource for training and validating multimodal traffic perception models under stringent, real-world conditions.

Experimental evaluations on CTSU alongside well-established benchmarks demonstrate TrafficPerceiver’s superiority over existing state-of-the-art methods. The model not only excels at high-level scene understanding tasks such as descriptive narration and interactive question answering but also surpasses traditional segmentation approaches in fine-grained, target-oriented extraction. Particularly impressive is its maintained accuracy and interpretability in scenes severely affected by environmental disturbances, marking it as a robust candidate for deployment in practical autonomous driving and smart traffic management systems.

TrafficPerceiver’s architecture challenges the long-standing paradigm of segregated perception modules by illustrating the efficacy of a unified multimodal Transformer framework. This cohesion facilitates cross-modal contextual reasoning where linguistic queries dynamically inform visual attention mechanisms, thereby enhancing the system’s flexibility and user interactivity. Drivers and traffic operators could benefit from this interactive capability, querying specific scene components via natural language and receiving precise, actionable insights in real time.

Beyond technical performance, the integration of reinforcement learning via Group Relative Policy Optimization embodies a theoretical advancement that enriches model adaptability. By redefining the learning objective from absolute correctness to relative consistency within groups, GRPO addresses the inherent uncertainty and variability of real-world traffic visuals. This approach encourages a more resilient perception model that can generalize across conditions without succumbing to the brittleness exhibited by many conventional vision systems.

The CTSU dataset not only advances the scope of testing frameworks available in this domain but also fosters the growth of instruction-driven multimodal AI research in intelligent transportation. By supplying diverse, annotated examples rich with linguistic and visual references, CTSU invites researchers worldwide to push the envelope on holistic traffic perception models that marry language understanding with pixel-level precision—a critical step toward truly autonomous, context-aware vehicular systems.

TrafficPerceiver exemplifies how harmonizing large-scale language models with visual scene perception can innovate beyond incremental improvements to deliver fundamentally new functional capabilities. Its design reflects a deeper understanding of the complex interactions between textual instructions and dynamic road environments, positioning it at the frontier of AI research where autonomous systems become not only perceptive but communicative and responsive to human guidance.

Published in the prestigious journal Communications in Transportation Research, this work marks a milestone in transportation AI, setting a precedent for future research trajectories that blend instruction-driven learning, multimodal transformers, reinforcement learning, and challenging dataset construction. The study situates emerging transportation technologies at an inflection point where machine perception adapts robustly to real-world complexity, enabling safer and smarter mobility solutions globally.

As TrafficPerceiver continues to be refined and evaluated, its principles could broadly influence the design of perception systems across related domains—urban surveillance, robotics, and beyond—demonstrating the transformative power of instruction-enabled multimodal AI underpinned by reinforcement learning strategies. The path ahead points toward more interactive, reliable, and interpretable AI agents capable of navigating and understanding our world in human-centric, linguistically grounded ways.

Subject of Research: Traffic scene understanding and segmentation via multimodal large language models with reinforcement learning
Article Title: TrafficPerceiver: A Multimodal Large Language Model with Reinforcement Learning for Unified Challenge Traffic Scene Perception
News Publication Date: 31-Mar-2026
Web References: https://doi.org/10.26599/COMMTR.2026.9640008, https://www.sciopen.com/journal/2097-5023
References: Communications in Transportation Research, Volume 6 (2026)
Image Credits: Communications in Transportation Research