AI Learns to Simulate Abdominal Ultrasound Scans from Simple Anatomical Maps

Medical trainees learning to perform abdominal ultrasound face a persistent problem: they need realistic scans to practice on, but generating those scans traditionally requires detailed patient-specific CT volumes and computationally expensive physics simulations. Now, a team of researchers from the National Scientific and Technical Research Council (CONICET) and PLADEMA at Universidad Nacional del Centro de la Provincia de Buenos Aires in Argentina has developed a two-stage deep learning pipeline that can produce realistic abdominal ultrasound images directly from simple semantic labels — anatomical maps that mark where organs sit in the body. The work, published in the Annals of Biomedical Engineering, promises to make ultrasound simulation more flexible, more varied, and potentially far more accessible for training the next generation of sonographers.

The core limitation the researchers set out to address lies in how ultrasound simulators have traditionally worked. Conventional approaches rely on ray-casting through CT volumes — essentially tracing sound waves through a three-dimensional anatomical model to compute how they would scatter and reflect. This physics-based method produces convincing images, but it ties every simulation to a specific patient’s CT scan. If a trainer wants to simulate a deformed organ or introduce a pathology, they must somehow modify the underlying CT volume, a task that is technically demanding and often impractical. As a result, the range of anatomical variation and pathological scenarios available in existing simulators has remained narrow.

The Argentine team’s solution, developed by Santiago Vitale, Duilio Deangeli, Ignacio Larrabide, and José Ignacio Orlando, is elegant in its decomposition of the problem. Rather than asking a single neural network to jump from an abstract anatomical map straight to a photorealistic ultrasound image, the pipeline works in two stages. Stage I takes a semantic label map — a simplified representation where each pixel is tagged with the organ or structure it belongs to — and predicts what the researchers call a physics-based image: the output of ray-casting simulation through a CT volume. Stage II then takes this intermediate physics-based image and refines it into a realistic ultrasound scan using an anatomically guided unpaired translation model.

This intermediate representation is the key innovation. By training Stage I on pairs of semantic labels and their corresponding CT-based ray-casting outputs, the network learns the underlying physics of how sound interacts with abdominal anatomy — at least in an approximate, learned form. Once trained, the model can generate these physics-based intermediate images directly from semantic labels at inference time, without needing any patient-specific CT volume. The second stage then handles the perceptual gap between the clean, simulated physics output and the noisy, artifact-laden appearance of real ultrasound scans.

To evaluate their approach, the researchers compared two generative architectures for Stage I: Pix2Pix, a well-established conditional generative adversarial network, and a Semantic Diffusion Model (SDM), which belongs to the newer class of denoising diffusion models that have transformed image generation in recent years. For Stage II, they employed a segmentation-guided variant of CycleGAN, an unpaired translation framework that learns to convert images between domains without requiring matched pairs, using anatomical guidance to preserve structural fidelity during the translation.

The results reveal a fascinating trade-off between the two Stage I architectures. The Semantic Diffusion Model significantly outperformed Pix2Pix on morphological metrics — measures of how well the generated image preserves anatomical structure. The SDM achieved a mean absolute error of 19.85 compared to Pix2Pix’s 21.65, a structural similarity index (SSIM) of 0.28 versus 0.24, and a mean intersection-over-union (mIoU) of 0.43 versus 0.29. These numbers indicate that the diffusion model was substantially better at getting the anatomy right — at producing images whose organ boundaries and spatial relationships matched what the physics-based simulation would actually produce.

Yet Pix2Pix held its own advantage in perceptual quality. On Learned Perceptual Image Patch Similarity (LPIPS), Pix2Pix scored 0.17 against the SDM’s 0.19; on Fréchet Inception Distance (FID), it achieved 0.32 versus 0.37; and on Kernel Inception Distance (KID), 0.25 versus 0.48. These metrics measure how visually convincing images appear to both neural networks and human observers, and here the adversarial training of Pix2Pix produced images that looked more realistic, even if their underlying anatomy was less faithful. The finding underscores a persistent tension in generative modeling for medical applications: perceptual realism and anatomical accuracy do not always improve together, and the right choice depends on whether the downstream use prioritizes visual fidelity or structural correctness.

Perhaps the most compelling aspect of the framework is what it enables once trained. Because deformations and pathologies are generated simply by editing the semantic label maps — stretching a liver, inserting a lesion, altering organ boundaries — the pipeline offers controllable simulation of both healthy and pathological anatomy without any manual modification of volumetric data. A training scenario designer could, in principle, specify an anatomical configuration on the label map and receive a corresponding realistic ultrasound image, opening the door to systematically varied training curricula that expose trainees to a far broader spectrum of presentations than fixed patient datasets allow.

The researchers are candid about the framework’s limitations. While the trained pipeline does not require patient-specific CT volumes at inference time, the training of Stage I still depends on CT-derived segmentations and ray-casting simulations — meaning the approach inherits whatever biases and coverage limitations exist in the training data. The learned approximation of physics is only as good as the physics simulations it was trained on. Moreover, the relatively modest absolute values of the morphological metrics suggest that the generated images, while promising, do not yet perfectly reproduce the ground-truth physics outputs, leaving room for refinement as diffusion model architectures and training strategies continue to evolve.

The work arrives amid a broader surge of interest in generative models for medical ultrasound simulation, with recent efforts exploring diffusion-based synthesis for fetal, echocardiographic, and musculoskeletal imaging. What distinguishes this contribution is its explicit pairing of learned generation with physics-based supervision — using the outputs of traditional ray-casting simulation as the training target rather than real ultrasound images alone. This grounding in physics offers a form of anatomical accountability that purely data-driven approaches can lack. The team has made their processed dataset publicly available through Kaggle, along with reproducibility resources on GitHub, though the complete source code cannot be released due to project-related restrictions. As simulation-based training becomes an increasingly central pillar of ultrasound education worldwide, tools that can generate diverse, controllable, and anatomically plausible scans on demand may prove transformative — turning what was once a hardware- and data-bound endeavor into something a laptop and a label map can accomplish.

Subject of Research: Deep learning-based abdominal ultrasound image simulation from semantic anatomical labels using paired label-to-physics-based image translation

Article Title: Abdominal Ultrasound Simulation from Semantic Labels Using Paired Label-to-Physics-Based Image Translation

Article References: Vitale, S., Deangeli, D., Larrabide, I., & Orlando, J. I. (2026). Abdominal Ultrasound Simulation from Semantic Labels Using Paired Label-to-Physics-Based Image Translation. Annals of Biomedical Engineering. https://doi.org/10.1007/s10439-026-04411-3

Image Credits: AI Generated

DOI: 10.1007/s10439-026-04411-3

Keywords: ultrasound simulation, semantic image synthesis, diffusion models, generative adversarial networks, medical imaging, ray-casting, abdominal imaging, image translation, pathology simulation, medical education, deep learning, CT segmentation