Two AI Tools Judging Breast Positioning in Screening Barely Agree

Two commercially available software systems designed to automatically grade breast positioning quality in digital breast tomosynthesis screening have turned out to be far from interchangeable, according to a new study published in European Radiology Experimental. When both tools scored the same 3,112 mammographic views from the same screening cohort, they landed on exactly the same Perfect, Good, Moderate or Inadequate rating in fewer than half of the images, and a statistical measure of agreement was close to what would be expected by chance.

The work, part of a project called PGMI-BENCH, was carried out by researchers including Gisella Gennaro and colleagues and drew on the RIBBS personalized breast screening program in Padua, Italy. The retrospective analysis included 778 women aged around 45 years, whose examinations had been recalled for further diagnostic assessment after double reading at the first screening round between January and December 2020. Seven examinations with incomplete software outputs were excluded, leaving 778 examinations with an equal split of craniocaudal and mediolateral oblique views.

The Perfect/Good/Moderate/Inadequate classification, known as PGMI, is widely used in screening programs to judge whether a mammographic image is properly positioned, since inadequate positioning can hide tissue, produce motion artifacts and skin folds, and ultimately affect cancer detection and recall rates. Human PGMI scoring is known to suffer from substantial inter- and intra-observer variability, which is precisely why automated tools have been developed: they promise objective, reproducible and scalable image quality assessment for audits, training and continuous monitoring of radiographer performance.

The two systems compared, however, work in fundamentally different ways. Software 1, Volpara Lab v2.0.3 from Lunit International, analyzes raw tomosynthesis projection images, the DICOM data labeled for processing. Software 2, b-box plus from b-rayZ in Zurich, analyzes processed synthetic 2D reconstructions generated from the same DBT acquisition, the data labeled for presentation. Both generate PGMI scores based on criteria such as breast tissue coverage, nipple position, pectoral muscle visualization, inframammary fold depiction and artifacts, but the exact definitions, thresholds and aggregation rules are proprietary and differ between the vendors.

The headline numbers are striking. Software 2 assigned 10.9 percent of images to the Inadequate category, while Software 1 flagged only 0.7 percent as Inadequate. Software 1 was more generous at the top of the scale, rating 21.9 percent of images Perfect compared with 12.9 percent for Software 2. Exact agreement between the two systems was just 48.2 percent, and the quadratic weighted kappa, a statistic that corrects for chance agreement, was a mere 0.044 with a 95 percent confidence interval of 0.006 to 0.085, indicating poor agreement. Agreement within one PGMI category was much higher at 87.7 percent, but the authors caution that this reflects ordinal proximity near category boundaries and should not be read as clinical equivalence.

The pattern of disagreement was structured rather than random. Images judged Inadequate by Software 2 were mostly reassigned as Moderate by Software 1, while many images rated Perfect by Software 2 were downgraded to Good or Moderate by Software 1. Disagreements spanning more than one category were uncommon. Agreement was better for mediolateral oblique views, with exact agreement of 57.3 percent, than for craniocaudal views, where it fell to 39.2 percent. The authors interpret this clustering around boundary categories as evidence of systematic differences in threshold definitions and feature weighting rather than fundamentally different detection of gross positioning errors.

Stratified analyses revealed how anatomy and acquisition parameters shaped the divergence. Using Software 1-derived breast density categories as a common reference, the largest inter-software differences appeared in the highest-density group, where Software 2 assigned substantially more Inadequate ratings and fewer Perfect ratings than Software 1. When stratified by breast volume quartiles, the greatest discrepancies occurred in the lowest-volume quartiles, while the highest-volume quartile showed the best agreement, with exact agreement around 51 percent, agreement within one category around 90 percent, and no statistically significant differences in category distributions.

Quantitative modeling reinforced these patterns. In univariate analysis, higher volumetric breast density was associated with lower PGMI scores in both systems, with a weak correlation for Software 1, a Spearman rho of minus 0.121, and a moderate correlation for Software 2, rho of minus 0.355. Breast volume showed positive associations with PGMI scores, again stronger for Software 2, rho of 0.361 versus 0.154. Multivariable ordinal logistic regression confirmed that increasing volumetric breast density and breast thickness were independently associated with lower odds of higher PGMI categories in both systems, while breast volume was associated with higher scores. Compression force showed an independent positive association with PGMI only in Software 2, and compression pressure was not significant in either model. Overall, Software 2’s outputs depended more strongly on breast composition and acquisition-related parameters.

The two systems also disagreed on which quality-control criteria to blame. In craniocaudal views, Software 1 most frequently flagged nipple alignment and posterior coverage, while Software 2 predominantly flagged pectoral muscle depiction and lateral parenchymal coverage. In mediolateral oblique views, both systems most often identified inframammary fold deficiencies, but Software 1 emphasized pectoral geometry while Software 2 more frequently flagged pectoral nipple-level adequacy and parenchymal coverage. Despite targeting similar conceptual domains of positioning quality, the two algorithms prioritized different anatomical features.

The authors are careful about what these findings do and do not show. Because no reference standard was available, the study cannot say which system is more accurate; it demonstrates relative disagreement between two complete end-to-end pipelines, including their different image inputs, rather than a direct comparison of decision rules. The cohort was a single screening population of relatively young women recalled after double reading, which may have introduced case enrichment and limits generalizability, and both algorithms were closed black boxes that could not be inspected internally. The authors conclude that automated PGMI systems are not interchangeable and call for harmonized definitions and validation against expert reader assessments, planned as the second phase of PGMI-BENCH, before such tools are deployed in routine clinical quality assurance.

Subject of Research: Agreement between two automated PGMI-based software tools for assessing breast positioning quality in digital breast tomosynthesis screening

Article Title: Comparison of two automated PGMI-based software tools for breast positioning quality assessment in DBT screening

Article References: Gennaro, G., Trazzi, S., Bordon, V., & Galli, V. (2026). Comparison of two automated PGMI-based software tools for breast positioning quality assessment in DBT screening. European Radiology Experimental, 10(1), Article 156. https://doi.org/10.1186/s41747-026-00825-9

Image Credits: AI Generated

DOI: 10.1186/s41747-026-00825-9

Keywords: digital breast tomosynthesis, PGMI, breast positioning, image quality assessment, screening, automated software, inter-rater agreement, breast density, mammography quality assurance, Volpara, b-box plus, artificial intelligence