The concern is not hypothetical. AI systems learn from data, and when the data they scrape or are trained on reflect existing human stereotypes and healthcare disparities, the resulting models can form, copy, and amplify harmful patterns. In everyday consumer applications, biased AI is troubling enough, but in clinical settings the stakes rise sharply. An algorithm that performs well on average may quietly underperform for women, older patients, or members of ethnic minorities, and those failures can translate directly into missed diagnoses, delayed treatment, and worse outcomes. Because medical AI increasingly influences decisions about who gets scanned, who gets referred, and who gets treated, researchers argue that careful evaluation of potential demographic bias is not an optional refinement but a core safety requirement.

To understand how well the field is meeting that requirement, the Osaka Metropolitan University team conducted a scoping review, a form of systematic literature mapping designed to capture the breadth of research on a topic. Their goal was twofold: first, to identify studies that validated commercially available radiology AI products, and second, to document trends in how those studies reported results for demographic subgroups defined by sex, age, and ethnicity. Scoping reviews are particularly useful when a research landscape is sprawling and inconsistent, because they reveal not just what has been measured but what has been left out. In this case, what has been left out turned out to be the most important part of the safety picture.

The scale of the search was substantial. The team collected 545 studies covering 252 commercial AI products that included some reported demographic subgroup data. On its face, that number might suggest a healthy level of attention to demographics. But the picture changed dramatically when the researchers applied stricter criteria. Of those 545 studies, only 77, validating just 52 products, were found to include both demographic details and actual subgroup analysis results. In other words, fewer than one in seven of the studies they identified provided the kind of per-subgroup performance evidence needed to detect bias, and the validated products represented only a fraction of the commercial landscape.

The statistical dimension of the problem is where the findings become especially striking. The researchers applied the Wilson Confidence Interval formula, a standard method for calculating proportions and their uncertainty, to studies validating AI for tuberculosis detection. They found that 67 percent of the reported datasets were at risk of being underpowered for sex subgroup analysis. Underpowered means the dataset was too small to detect a meaningful difference in performance between subgroups even if one existed. An underpowered analysis can produce a reassuring result that is essentially meaningless, because the study simply lacked the statistical capacity to reveal a disparity. For clinicians relying on such studies, the absence of evidence of bias can be mistaken for evidence of the absence of bias.

Perhaps the most sobering conclusion is that this situation has not improved despite years of calls for better demographic reporting. The demand for improved reporting on sex, age, and ethnicity in medical AI has been voiced repeatedly by researchers, ethicists, and regulators, precisely because of the dangers that biased algorithms pose to marginalized groups. Yet the regression analysis used by the Osaka team to map trends across the collected studies revealed that performance reporting for demographic subgroups has not become more common over time. The field, in effect, has been accumulating validation studies without accumulating the evidence needed to confirm equitable performance, a pattern the authors describe as a systemic problem rather than a series of isolated oversights.

Dr. Walston characterized the landscape in stark terms. “This scoping review quantifies how fragmented the commercial validation landscape is, showing that reporting for both the demographics and per-subgroup performance is inadequate for estimating subgroup bias,” she stated. “This systemic problem requires effort from all stakeholders, from researchers to regulatory agencies, encouraging thorough reporting and commercial product validation to support physician and patient trust in medical AI products.” The statement frames the issue as one that no single actor can solve: journal reviewers, device manufacturers, hospital procurement teams, and government regulators all play a role in whether subgroup evidence is produced, demanded, and acted upon.

The fragmentation Dr. Walston describes has practical consequences for how medical AI reaches patients. When validation is scattered across many small studies, each examining a different product with different datasets and different reporting conventions, it becomes nearly impossible to compare products or to build a coherent picture of where bias risk concentrates. A hospital choosing between competing algorithms may find that one has published subgroup results and another has not, or that both report averages without revealing how performance varies by sex or ethnicity. Without standardized, transparent subgroup reporting, the market cannot reward fairness, and regulators cannot easily verify it. The result is a validation ecosystem that quantifies overall accuracy while leaving the distribution of that accuracy across patient groups largely unmeasured.

The study, published in European Radiology under the title describing the current state of demographic subgroup reporting for commercially available AI in radiology, arrives at a moment when health systems worldwide are scaling up their reliance on algorithmic tools. Tuberculosis detection, one of the applications examined in the power analysis, is a case in point: screening programs in high-burden countries increasingly depend on AI readers, and any systematic performance gap between demographic groups could quietly shape which communities receive timely diagnoses. The authors note that the findings expose the need for effective, transparent reporting to confirm the safe and unbiased performance of medical AI products across all patient subgroups, a requirement they argue underpins both clinical safety and public confidence.

The path forward, as the researchers frame it, involves collective accountability. Manufacturers would need to release demographic details and per-subgroup performance data for their products; researchers would need to design validation studies with sufficient statistical power to detect subgroup differences; and regulatory agencies would need to make such reporting a condition of approval rather than a voluntary courtesy. The authors declare a relationship with the Medical AI Promotion Institute, Inc., in their conflict of interest statement, underscoring how closely the commercial and academic sides of this field are intertwined. What the review makes clear is that trust in medical AI cannot be assumed from headline accuracy figures alone. Until subgroup evidence becomes routine, the possibility that these tools are quietly copying and amplifying healthcare bias remains an open question, one that patients, physicians, and regulators deserve to have answered with data rather than assurances.

Subject of Research: Demographic subgroup reporting in validation studies of commercially available radiology AI products

Article Title: Check for copycat bias in medical AI

Article References: Check for copycat bias in medical AI. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: medical AI, radiology, algorithmic bias, demographic subgroups, scoping review, Osaka Metropolitan University, European Radiology, tuberculosis detection, validation studies, health equity, regulatory reporting, statistical power

Tourism seasonality is not a minor scheduling nuisance. When demand concentrates around school holidays, summer weather, and headline attractions, destinations suffer overcrowding, strained infrastructure, environmental stress, degraded visitor experiences, and unstable seasonal labour markets, while hotels and restaurants sit half-empty for much of the year. Researchers have long argued that seasonality is a structural feature of destination systems rather than a marketing failure. The central challenge, as Dalir frames it, is not attracting more visitors but redistributing the attention, movement, and spending of the visitors a destination already receives across time and space.

The theoretical heart of the study lies in a distinction borrowed from Herbert Simon’s bounded rationality: the difference between the full choice set — everything objectively available to a traveller at a moment of decision — and the perceived choice set, the small subset a person actually notices and evaluates. Digital platforms appear to expand choice, but they also mediate how choice is encountered. Rankings, recommendation lists, review summaries, filters, and AI-generated itineraries determine which alternatives become cognitively accessible. An off-season festival or a charming peripheral museum may fail not because it lacks value, but because it never enters the traveller’s field of attention. AI, in this framing, is not a neutral information tool but a decision-mediating infrastructure that shapes what becomes visible before any preference is formed.

To test how different forms of algorithmic mediation play out at scale, Dalir built an agent-based model in NetLogo 6.4.0 simulating a destination ecosystem populated by heterogeneous visitor agents and supply agents — attractions, restaurants, accommodations, and souvenir shops. The model was calibrated with multiple data sources, including secondary destination data, online platform data, stakeholder input, and a survey of 400 respondents distributed through Prolific, of which 310 valid responses informed behavioural parameters such as spending, price sensitivity, and revisit probabilities. Visitors evaluate options using a bounded-rational random utility function in which congestion-adjusted quality, price, travel costs, and memory-based expectations combine, and each AI intervention adds a mechanism-specific cognitive term weighted by the visitor’s trust in AI — so effects vary across people rather than assuming uniform responsiveness.

Three AI conditions were compared against a business-as-usual baseline, each intervening at a different point in the decision process. Predictive nudging raises the salience of congestion and timing information, steering visitors toward less crowded options while leaving the choice set intact. Recommender-system filtering, the most aggressive intervention, restricts AI-adopting visitors to a top-k list of attractions ranked by predicted preference fit — directly shrinking the perceived choice set. Fairness-oriented visibility enhancement boosts the attractiveness of low-visibility, underrepresented, or off-peak alternatives without narrowing anything. The full experiment comprised 800 simulation runs, 200 per condition, generating 1,600 season-specific observations that were analysed with two-way ANOVA using intervention condition and season as factors.

The headline results are striking in both directions. All three AI interventions significantly increased spending and visitor numbers compared with business as usual, with large effect sizes. For spending, the intervention main effect was F(3, 1592) = 157.55, p < .001, with a partial eta-squared of .229. Recommender-system filtering produced the highest overall spending mean of roughly 818,793 simulation currency units, followed by predictive nudging at 809,527 and fairness-oriented visibility enhancement at 780,010. Visitor numbers rose from a baseline mean of about 338 to roughly 404–405 under all three interventions. On raw economic performance, AI looks like an unambiguous win.

But the seasonality result is where the study delivers its most important punch. The interaction between intervention and season — the statistical test of whether AI effects differ between high and low season — was not significant for either spending (p = .312) or visitor numbers (p = .655). In fact, the seasonal spending gap widened under every AI condition, growing from 94,597 under business as usual to as much as 127,297 under predictive nudging. AI lifted both seasons, but it lifted them roughly equally, leaving the structural imbalance untouched. The interventions, Dalir concludes, are performance-enhancing rather than seasonality-reducing. The reason is that seasonality is not merely an information problem; it is anchored in climate, school calendars, annual leave, work routines, event schedules, and accommodation availability — conditions that sit outside the platform interface and that no recommendation engine can rewrite.

The mechanism-level analysis reveals why the three designs behave so differently. The recommender condition showed the highest influence rate at 0.547 and cut the perceived choice set from about 25.6 options to 12.3, providing direct evidence of filtering at work. Predictive nudging had the lowest influence rate at 0.169, but every AI-influenced decision under it was attributable to congestion avoidance, confirming its value as a flow-coordination tool. Fairness-oriented visibility enhancement, with an influence rate of 0.396, kept the perceived choice set equal to the full set while spreading attention more evenly. Distributionally, the trade-offs are stark: recommender filtering concentrated demand around highly ranked attractions even as it maximised spending, while fairness-oriented enhancement activated more attractions with lower dispersion — the best profile for provider visibility and destination resilience, even though it was not the strongest earner.

The robustness analysis adds a subtle warning about design parameters. Varying the recommender choice-set size across k = 3, 5, 10, and 15 showed that influence peaks at moderate filtering levels and declines when choice sets become either too narrow or too broad — a non-linear threshold dynamic suggesting an optimal filtering range. Excessive filtering suppresses exploration and reinforces concentration; weak filtering fails to reduce cognitive burden. In other words, settings that look like technical dials are actually behavioural and governance choices with system-level consequences, and small parameter changes near critical regions can produce disproportionate effects on how demand and economic opportunity are distributed.

The broader message extends well beyond tourism. AI-enabled platforms should not be evaluated solely on accuracy, personalisation quality, engagement, or conversion, because systems that perform well by user-level metrics can still generate uneven consequences for providers, destinations, and ecosystems. For destination managers, the implication is to deploy AI with explicit objectives — revenue growth, congestion reduction, low-season stimulation, or provider resilience each demand a different design — and to track dashboards that monitor seasonal gaps and exposure concentration, not just visitor counts. For platforms and policymakers, the study argues for hybrid architectures combining filtering, nudging, and visibility mechanisms, transparency around ranking criteria, and exposure-diversity audits. AI can support more balanced tourism systems, the study concludes, but only when its decision architecture is deliberately designed to redistribute attention, visibility, and demand. Left to its own optimisation logic, it will happily make the busy season busier.

Subject of Research: Agent-based simulation of how AI-mediated decision mechanisms in tourism platforms affect visitor choice, spending, and seasonality

Article Title: Can AI Solve Eco-Economic Challenges? Simulating the Impact of AI Systems on Seasonality Using Agent-Based Modelling

Article References: Dalir, S. (2026). Can AI Solve Eco-Economic Challenges? Simulating the Impact of AI Systems on Seasonality Using Agent-Based Modelling. Information Systems Frontiers. https://doi.org/10.1007/s10796-026-10818-z

Image Credits: AI Generated

DOI: 10.1007/s10796-026-10818-z

Keywords: artificial intelligence, tourism seasonality, agent-based modelling, smart tourism, recommender systems, digital choice architecture, bounded rationality, predictive nudging, algorithmic bias, destination management, platform ecosystems, simulation

The central argument of the review is that digital health technologies have demonstrated strong clinical potential yet have not fully transcended traditional intervention paradigms. Mobile applications, wearable sensors, telemedicine platforms, and machine learning models have each shown benefits in isolated settings, but the authors contend that the real transformation will come from integrating these components into coherent systems that connect patients, clinicians, and health records. The rapid digitalisation of healthcare, accelerated in recent years, has created the infrastructure for such integration, and the review evaluates whether the clinical evidence has kept pace with the technological enthusiasm.

One of the most striking findings concerns risk prediction. For decades, cardiometabolic risk has been estimated using tools such as the Framingham Risk Score, SCORE, QRISK3, and the more recent PREVENT equations, which combine a handful of variables—age, sex, blood pressure, cholesterol, smoking status, and diabetes—into a statistical estimate. Machine learning models, by contrast, can extract high-dimensional patterns from multimodal data streams, including electronic medical records, electrocardiograms, retinal images, genomic profiles, and continuous physiological measurements from wearables. The review highlights evidence that AI approaches can outperform these traditional risk models, particularly when they incorporate data types that classical equations cannot use, such as polygenic risk scores derived from single nucleotide polymorphisms, or subtle features extracted from routine electrocardiograms that correlate with future atrial fibrillation, left ventricular dysfunction, and other silent cardiac conditions.

Technical performance in these studies is typically quantified using metrics such as the area under the receiver operating characteristic curve, or AUROC, and the area under the precision-recall curve, alongside calibration measures like the Brier score. The review notes that models reporting strong discrimination in retrospective datasets often degrade when deployed in real-world clinical environments, a phenomenon driven by differences in patient demographics, data collection practices, and measurement standards between development and deployment settings. This gap between benchmark performance and clinical utility is a recurring theme, and the authors identify rigorous, real-world validation as the first of three priorities for future investment.

Beyond prediction, the review examines how artificial intelligence is being embedded into clinical decision support systems through integration with electronic medical records. When a model can read a patient’s longitudinal record—laboratory trends such as estimated glomerular filtration rate, alanine aminotransferase, and glycated haemoglobin, alongside medication histories and clinical notes processed by natural language processing—it can generate individualised recommendations at the point of care. In nephrology, for example, AI-assisted systems have been explored for predicting kidney function decline and optimising the timing of erythropoietin stimulating agent therapy. In hepatology, machine learning models incorporating genetic variants such as PNPLA3, TM6SF2, and APOL1 have been used to stratify liver disease risk in patients with obesity and diabetes. In diabetes research, unsupervised clustering of clinical and genetic data has reproduced and refined the now widely cited subtyping of the disease into five clusters, ranging from severe autoimmune diabetes to mild age-related diabetes, each with distinct trajectories and treatment responses.

The review also documents the contribution of consumer-facing digital health tools. Photoplethysmography signals from smartwatches and fitness bands, transdermal optical imaging, and smartphone-based screening have enabled continuous, passive monitoring of heart rhythm, blood pressure trends, and physical activity. Randomised trials of app-based coaching, remote monitoring, and technology-assisted hybrid cardiac rehabilitation show improvements in patient engagement, medication adherence, and selected clinical outcomes. Telemedicine has expanded access to specialist care for patients in rural and underserved regions, a benefit that became particularly visible during recent global health disruptions. These tools successfully promote engagement, improve outcomes, and enhance access—the three achievements the review credits to the current generation of digital health innovations.

Yet the same technologies that promise to democratise care risk deepening existing inequities. The review identifies inequality in digital health access as a significant barrier to sustained implementation. Wearables and smartphones are less prevalent among older adults, low-income populations, and communities in low- and middle-income countries, precisely the groups that carry a disproportionate burden of cardiometabolic disease. Algorithmic bias compounds this problem: models trained on datasets dominated by particular ethnic or socioeconomic groups may perform poorly when applied to others, generating inaccurate risk estimates and potentially misdirecting care. Generalisability across diverse demographics is therefore not a peripheral concern but a central test of whether these systems can serve whole populations rather than privileged subsets.

Data privacy and governance present a second cluster of challenges. Health data flowing through consumer devices and cloud platforms falls under a patchwork of regulations, including the Health Insurance Portability and Accountability Act in the United States, the General Data Protection Regulation in Europe, and the Asia Pacific Economic Cooperation Privacy Framework. The review argues that robust regulatory frameworks are essential to ensure that AI tools are safe, effective, and accountable before they influence clinical decisions. A third priority is the adoption of explainable artificial intelligence, or XAI. Deep learning models often operate as opaque systems whose internal reasoning cannot be inspected by clinicians, which undermines trust and complicates error analysis. Explainable approaches that surface the features driving a prediction—a specific electrocardiographic pattern, a laboratory trend, a genetic variant—allow clinicians to verify that a model is reasoning sensibly and to identify spurious associations before they cause harm.

The authors also situate these technologies within the broader shift toward precision medicine in cardiometabolic health. Traditional categories such as metabolically healthy obesity, metabolically unhealthy normal weight, and the homoeostasis model assessment of insulin resistance are being re-examined through the lens of high-dimensional data, allowing finer stratification of patients who superficially resemble one another but follow very different disease courses. Combining polygenic risk scores with clinical and behavioural data streams could, in principle, allow prevention strategies to be tailored to an individual’s specific risk architecture rather than applied uniformly. The review suggests that this convergence of genomics, sensing technology, and machine learning is where the greatest transformative potential lies, both for prevention and for treatment.

Taken together, the review offers a measured but optimistic verdict. Artificial intelligence and digital health innovations have already demonstrated that they can promote patient engagement, improve clinical outcomes, and enhance healthcare access in cardiometabolic medicine. However, the authors conclude that realising their transformative potential will require deliberate investment in three areas: rigorous validation in real-world settings rather than curated datasets, the adoption of explainable AI to make algorithmic reasoning transparent to clinicians and patients, and robust regulatory frameworks that safeguard privacy and equity while allowing innovation to proceed. If those conditions are met, the tools described in this synthesis could move from promising prototypes to standard components of cardiometabolic prevention and care, reshaping a field that currently accounts for an enormous share of global morbidity and mortality.

Subject of Research: Artificial intelligence and digital health innovations for cardiometabolic disease prevention and management

Article Title: Reimagining cardiometabolic health through artificial intelligence and digital health innovations

Article References: Tiong, K. I. L., Abd Rahman, N. E., Lee, Y.-H., Chan, W.-K., Lim, S.-K., Boey, V. W.-F., Lui, J. N., Tae, S.-K., & Lim, L.-L. (2026). Reimagining cardiometabolic health through artificial intelligence and digital health innovations. BMC Medicine. https://doi.org/10.1186/s12916-026-05196-x

Image Credits: AI Generated

DOI: 10.1186/s12916-026-05196-x

Keywords: artificial intelligence, digital health, cardiometabolic disease, cardiovascular disease, diabetes, chronic kidney disease, MASLD, machine learning, clinical decision support, explainable AI, algorithmic bias, risk prediction

The audit was ambitious in scale. Shukla queried nine live model configurations, including GPT-5.5, Claude Opus 4.7, Claude Sonnet 4.6, three Gemini variants, two DeepSeek versions, and Gemma 4 31B, using twenty experimental phrases and five control phrases. Each model-phrase pair was tested with fifty stateless generations at temperature 1.0, producing 11,250 responses in total. Half of the experimental phrases bundled authority and resource-control content with direct, directive language, drawn from technology, business, medicine, and law. The other half bundled coordination and assistance content with reporting, deferential, or permission-seeking phrasing. Every response was parsed with deterministic regular expressions to extract the name, age, and gender the model had volunteered.

The headline numbers are striking. In the authority and directive condition, 63.58 percent of the imagined writers were labeled female, 35.89 percent male, 0.47 percent nonbinary, and 0.07 percent unparsed. In the support and deferential condition, the female share surged to 92.62 percent, while male attribution collapsed to 6.04 percent. In other words, both prompt bundles produced majority-female outputs, but the relative shift between them was enormous. The study is careful to frame this as a relative contrast within an overall female-skewed distribution rather than evidence of male dominance in authoritative roles, a nuance that distinguishes it from earlier, blunter headlines about occupational bias in language models.

Beneath the aggregate numbers lies an even more important story: the models disagreed wildly with one another. The difference in male-attribution risk between the two conditions ranged from minus 12.2 percentage points for Gemini 3.1 Pro, meaning it actually assigned more male writers to the supportive condition, to a staggering plus 93.6 points for Gemma 4 31B. Seven of the nine models showed positive differences and two showed negative ones. A Paule-Mandel random-effects synthesis across models produced an I-squared statistic of 98.6 percent on the log-odds scale, and a 95 percent prediction interval for the odds ratio stretched from 0.014 to 63,659. That interval spans the null effect and several orders of magnitude, which means no single pooled estimate can be trusted to describe any model outside the panel.

This heterogeneity is not a statistical nuisance; it is the dominant empirical result. A prediction interval that wide tells us that knowing the average behavior of these nine systems tells you almost nothing about the next model, or even the next version of the same model. The author argues, convincingly, that deployment audits should preserve model-specific estimates, exact provider routes, prompts, and collection dates rather than quoting a panel average as if it were a durable property of large language models in general. For organizations building on these systems, the practical implication is that demographic defaults must be measured on the exact endpoint they intend to use.

The study also confronts a subtle statistical trap: stateless API calls are not independent observations. Although each of the fifty generations per cell was a separate request with no conversational memory, the models often revisited the same narrow output modes. Within-cell name diversity, measured by Shannon entropy and effective name counts, ranged from roughly two to three effective names in several Gemma and Gemini 3 Flash strata to about twenty-seven to twenty-nine for DeepSeek v4 Pro. Some individual cells had a modal-name share as high as 98 percent, meaning nearly every generation in that cell produced the same name. Treating such responses as fifty independent draws would dramatically overstate the precision of any estimate.

To handle this dependence, the analysis employed four complementary techniques: a linear probability model with two-way cluster-robust covariance by phrase and model, a wild phrase-cluster bootstrap with Rademacher weights and 9,999 replications, a cross-classified block bootstrap that resampled models and phrases while keeping each fifty-response cell intact, and the random-effects synthesis. The cell-block bootstrap estimated an aggregate male-attribution gap of 29.9 percentage points with a 95 percent confidence interval of 6.0 to 55.6, while the two-way clustered model estimated 29.8 points. All approaches preserved a positive average contrast in this fixed panel, but with far wider uncertainty than naive response-level calculations would suggest.

The control conditions revealed another unsettling behavior. Five control phrases contained explicit demographic cues, and the models were asked to imagine a writer reproducing those cues. Control agreement ranged from 66.8 to 100 percent, and crucially, the mismatches were almost all explicit model outputs rather than parser failures. Some systems simply generated a gender inconsistent with the cue in the prompt, even when the task seemed to require preserving it. Excluding the two lowest-agreement models did not erase the aggregate gap; it actually widened from 29.9 to 33.5 points at an 80 percent threshold and to 36.5 points at 85 percent, showing that the contrast was not an artifact of poorly behaved endpoints.

Perhaps the most intellectually honest feature of the paper is its relabeling of its own estimand. An earlier draft described the comparison as an occupational-status effect, but inspection of the stimuli revealed that status-related role content covaried with speech act, mood, agency, and deference. Authority prompts use decisions and directives; support prompts use reports, questions, and requests for permission. The experiment therefore cannot distinguish whether models respond to the occupation, the communicative register, the degree of agency, or some combination. The published version renames the contrast a role-and-register effect and calls for a preregistered crossed design that independently varies role content and register, for example by expressing high-authority work deferentially and support work directively.

For anyone deploying language models in workplace contexts, the takeaways are concrete. Audit the exact register of your prompts, because directness and deference carry demographic associations of their own. Report full gender distributions rather than a single odds ratio, since a relative shift can obscure that both conditions share the same majority category. Measure response diversity, because repeated names signal that your apparent sample of outputs is far narrower than it looks. And treat cost and latency as separate axes from fairness: in this run, the inexpensive Gemma showed the largest positive risk difference while the pricier Gemini 3.1 Pro showed a negative one, a descriptive pattern with no causal meaning but a clear warning that fairness cannot be inferred from a price tag. As generated text increasingly drafts our examples, training materials, and simulations, these invisible demographic defaults quietly shape who gets imagined as authoritative and who gets imagined as helpful, one prompt at a time.

Subject of Research: Gender attribution defaults in large language model outputs under workplace role-and-register prompts

Article Title: Gender attribution under workplace role-and-register prompts in contemporary large language models

Article References: Shukla, R. (2026). Gender attribution under workplace role-and-register prompts in contemporary large language models. International Journal of Data Science and Analytics, 22(1), Article 333. https://doi.org/10.1007/s41060-026-01332-1

Image Credits: AI Generated

DOI: 10.1007/s41060-026-01332-1

Keywords: large language models, gender attribution, algorithmic bias, workplace language, model heterogeneity, prompt design, communicative register, response diversity, statistical audit, AI fairness, occupational stereotypes, bootstrap inference

The experiment was designed with unusual rigor. Rospocher assembled 80 recent English-language arXiv manuscripts, twenty each from computer science, mathematics, physics, and quantitative biology, deliberately chosen in short and long length buckets and posted after March 2025 to reduce the chance that the evaluator models had memorized them. Author-identifying information, acknowledgements, and repository links were stripped from the PDFs so that the text itself carried no identity clues. Each manuscript was then evaluated under a factorial design: a corresponding-author profile was appended varying three factors at two levels each, producing eight profile conditions plus a blind baseline with no author information at all. Because every manuscript was evaluated under every condition, each paper served as its own control, a repeated-measures structure that sharply isolates the effect of the metadata from the effect of the content.

The three manipulated factors were carefully chosen to represent different classes of real-world cues. The first was a name-coded identity cue, alternating between first names commonly perceived in the United States as Black female-coded, such as Lakisha and Tanisha, and White male-coded, such as Brad and Hunter, all paired with a fixed surname to limit confounding. The second was institutional prestige, drawn from the QS World University Rankings 2026, contrasting elite institutions like MIT, Oxford, and Stanford with lower-ranked ones like Cleveland State and Central Michigan. The third was a bibliometric profile: a high profile with an h-index of 32 and roughly 8,000 citations, versus a low profile with an h-index of 8 and 320 citations. These numbers matter because they are exactly the kind of externally retrievable signals that a retrieval-augmented AI assistant could surface through a routine author lookup, even when the manuscript itself is anonymous.

Six heterogeneous evaluator models scored every manuscript under every condition: GPT-4.1-mini accessed through an API, and five locally hosted open-weight models including Gemma 3, Llama 3.1, Olmo 3, Qwen3, and a review-specialized model called DeepReviewer. Each model produced integer ratings on a seven-point scale from minus three to plus three across fourteen review-style criteria, spanning writing clarity, novelty, methodological soundness, perceived importance, award-worthiness, funding potential, hiring-committee appeal, and an overall acceptance recommendation. Each manuscript-condition-model combination was run five times with near-deterministic decoding settings, and the results were averaged and pooled across models with equal weight. Uncertainty was quantified with 20,000 bootstrap resamples, sign-flip permutation tests, and false-discovery-rate correction, alongside an ordinal-direction diagnostic that used only the sign of each paired difference to avoid assuming equal intervals on the rating scale.

The headline finding is that evaluator outputs are not invariant to author metadata. The strongest and most consistent driver was the bibliometric profile: papers attributed to a high-citation author scored systematically higher on perceived prestige, impact, and acceptance recommendation than the very same papers attributed to a low-citation author. Institution tier produced a similar but smaller elevation. The name-coded identity cue had comparatively modest effects overall, though it was not absent: it produced a reliable positive shift on prestige-oriented items such as award-worthiness and hiring-committee appeal. Crucially, the effects were not distributed evenly across the evaluation instrument. Content-quality judgments, such as clarity and methodological adequacy, barely moved. The distortions concentrated on the gatekeeping layer: top-tier acceptance, awards, funding, promotion, and the final accept-or-reject recommendation.

This selective vulnerability is what makes the result so consequential. It suggests the models are not applying a uniform halo effect that inflates every judgment when a prestigious name appears. Instead, they are perturbing a specific layer of evaluation tied to anticipated status, visibility, and institutional reward. The question-level decomposition showed the largest and most robust effects on items asking whether the manuscript meets the standard of a top-tier venue, deserves a best-paper award, warrants competitive funding, would impress a hiring committee, or is likely to change how the field thinks. Items about whether the writing is clear or the evidence sufficient were largely untouched. In other words, the metadata leaks into the decisions, not the diagnosis.

The study then pushed the analysis one step further, from scores to consequences. Using the acceptance recommendation as the ranking scalar, Rospocher recomputed manuscript rankings under counterfactual metadata swaps while holding everything else fixed. The results were striking. When a high-bibliometric profile was swapped for a low one, roughly a quarter to nearly two fifths of manuscripts originally selected in top-K shortlists, across thresholds from ten to twenty-five percent of the pool, fell below the cutoff after the perturbation. Institution-tier swaps produced an intermediate disruption, and name-cue swaps a smaller but still non-trivial one. Rank-displacement distributions were asymmetric, with downward movements more common and more extreme than upward ones, consistent with the finding that high-prestige cues systematically elevate scores. Even when most manuscripts stayed near their original position, those sitting near the selection boundary could cross the threshold, changing shortlist membership outright.

The audit also revealed meaningful variation across evaluator models. Random-effects meta-analysis showed that effect directions were broadly consistent, but magnitudes varied substantially, especially for prestige-related contrasts, with a large share of total variation attributable to between-model heterogeneity rather than sampling error. Inter-evaluator agreement diagnostics reinforced the picture: single models showed only modest absolute agreement on the same manuscripts, while averaging across all six improved reliability considerably. The practical implication is that metadata sensitivity is not a quirk of one model but a property of the model class, and that two pipelines differing only in their choice of evaluator may exhibit meaningfully different bias profiles. Domain-stratified analyses, meanwhile, found the qualitative pattern held across all four sampled fields, though magnitudes varied, and interaction tests suggested the three cue channels operate approximately additively rather than through narrow cue combinations.

The author is careful about scope. The study does not include a human-reviewer baseline, so it cannot say whether this sensitivity is unique to machines or inherited from patterns in human judgments embedded in training data. The corpus is limited to 80 English-language papers from four largely hard-science arXiv domains, the name cues are U.S.-centric and name-coded rather than ground-truth demographic attributes, and the experiment disabled browsing and retrieval to isolate the effect of explicitly supplied metadata. Real-world assistants that actively look up author information may introduce additional pathways not tested here. Still, the counterfactual logic is airtight within its design: identical texts, varied only in author context, produced different verdicts.

The design implications are immediate. The paper argues that author-context metadata should be treated as a consequential design variable, not neutral input, in any system that uses large language models for scientific evaluation, ranking, or discovery. In double-blind settings, LLM-assisted review should default to content-only evaluation with identifiers and lookup pathways suppressed. In single-blind or retrieval-augmented settings, systems should structurally decouple content scoring from context-informed summaries, document which metadata fields were available to the evaluator, and treat track record as an explicit, separately disclosed criterion if it is genuinely part of the decision rule, rather than letting it silently leak into manuscript scores. Because sensitivity varies by model, deployments should be audited whenever the evaluator is changed or updated. As AI-assisted reviewing spreads through editorial pipelines, the study delivers a clear warning: an algorithmic reviewer that can see who wrote a paper may not be able to stop that knowledge from coloring its judgment of the work itself.

Subject of Research: Counterfactual auditing of author-metadata sensitivity in large language model-based scientific peer review

Article Title: Author metadata affects Large Language Model scores in scientific peer review

Article References: Rospocher, M. (2026). Author metadata affects Large Language Model scores in scientific peer review. Discover Artificial Intelligence, 6(1), Article 1404. https://doi.org/10.1007/s44163-026-02268-y

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02268-y

Keywords: large language models, peer review, algorithmic bias, bibliometrics, institutional prestige, counterfactual audit, scientific evaluation, editorial triage, LLM-as-a-judge, ranking fairness, arXiv, research integrity

The partnership behind the series is significant in itself. CRC Press, a long-established publisher of scientific, technical, and medical books, brings editorial infrastructure and academic reach, while AAAI, widely regarded as the premier scientific society in artificial intelligence, contributes deep engagement with the field and its research community. According to the publishers, that combination is intended to ensure the series delivers cutting-edge insights into current challenges and a clear roadmap for navigating the future of AI ethics. The series is edited by Francesca Rossi, an IBM Fellow and AI Ethics Global Leader based in the United States, and Virginia Dignum, a professor at Umeå University in Sweden, two figures who have spent years working at the intersection of AI research, governance, and ethical theory. Their stewardship signals that the series is not intended as a collection of cautionary essays but as a sustained, scientifically grounded examination of how AI should be built, governed, and used.

Rossi has been candid about why she believes a book series, rather than another single-volume treatment, is the right vehicle for this work. In her view, the field of AI ethics does not lack books, but it can look fragmented, with many different assumptions about what the issues actually are and what it means to address them. The series, she explains, reflects a perspective she shares with Dignum: that questions about AI ethics are empirical and evidence-based, inseparable from governance, and best approached constructively rather than purely critically. That framing matters. It positions AI ethics not as an abstract philosophical exercise detached from practice, but as a discipline that treats claims about AI’s harms and benefits as testable, measurable, and subject to the same standards of evidence as any other scientific question. It is, as Rossi notes, a view that fits AAAI’s commitment to treating AI questions scientifically, and a series format makes it possible to carry that perspective consistently across multiple volumes over time.

The commercial and institutional logic behind the series is articulated clearly by Lucy McClune, Senior Commissioning Editor for Computer Science at CRC Press. In her assessment, the ethical questions about whether and how AI should be used in different situations are arguably more important than the technological challenges still to be overcome. That is a striking claim coming from a computer science editor, and it captures a sentiment increasingly voiced across the AI community: the hardest problems are often not technical but social, legal, and moral. McClune describes the series as aiming to address those crucial questions, providing what she calls an essential resource for researchers, students, policymakers, technologists, and anyone committed to ensuring AI serves a fair, just, and equitable society. The intended audience is deliberately broad, reflecting a recognition that decisions about AI are no longer made solely by engineers but by legislators drafting regulations, school administrators adopting new tools, and citizens encountering AI systems in everyday life.

The first book in the series turns to one of the most immediate and contested arenas of AI deployment: education. Titled Ethical AI and Education: Reality, Opportunities, Governance, and Consequences of AI in the Classroom, the book is written by Clara Hawking and Darren Coxon and will be published later this month. The choice of education as the opening subject is telling. Few domains have experienced such rapid and disruptive AI adoption as schools and universities, where generative models capable of writing essays, solving problem sets, and simulating tutoring conversations arrived with little warning and even less preparation. Teachers, students, and institutions have been left to improvise policies on the fly, often without reliable evidence about what these tools do to learning, motivation, and academic honesty.

Hawking and Coxon’s book examines the realities of AI in educational settings and covers the key ethical considerations facing educators today. Its scope runs from governance frameworks and accountability structures to practical classroom applications, addressing critical issues including bias, data privacy, academic integrity, and teacher agency. Each of these topics represents a live controversy. Bias in AI systems can affect how students are assessed or sorted, particularly when algorithms are trained on data that reflects historical inequities. Data privacy raises questions about what information schools collect on children, who controls it, and how it may be used. Academic integrity has been upended by tools that can produce fluent written work on demand, forcing a re-examination of what assessment is supposed to measure. And teacher agency, the professional judgment and autonomy of educators themselves, is at stake when decisions about instruction are increasingly mediated or influenced by automated systems. The authors pair this analysis with concrete strategies for building human-centered classrooms in an increasingly AI-saturated world, a phrase that captures the series’ constructive orientation: the goal is not to expel AI from education but to ensure it remains subordinate to human purposes and human relationships.

The second volume in the series has already been announced and shifts the focus from the classroom to society at large. The End of Reality: Humans in the Age of ChatGPT, Deepfakes, and Artificial Intelligence, by Mateusz Łabuz, is scheduled for publication in December 2026. Its title signals an engagement with what many observers consider the most destabilizing consequence of generative AI: the erosion of shared epistemic ground. As synthetic text, images, audio, and video become indistinguishable from authentic material, the question of how individuals and institutions can establish what is real becomes urgent. The book’s framing, situating humans within an age defined by chatbots and deepfakes, suggests an examination of how these technologies reshape trust, communication, and perception on a civilizational scale, complementing the first volume’s more institutional and practical focus.

Together, the two announced books illustrate the editorial architecture the series appears to be building. One volume addresses a specific sector, education, where AI’s effects are concrete, measurable, and already felt by millions of students and teachers. The other addresses a diffuse but fundamental problem, the status of reality and truth in an environment saturated with synthetic content. Both fall squarely within the editors’ stated method: empirical, evidence-based, governance-oriented, and constructive. This approach distinguishes the series from much of the existing AI ethics literature, which has often oscillated between utopian boosterism and dystopian alarm. By insisting that ethical claims about AI should be treated as questions to be investigated rather than positions to be asserted, Rossi and Dignum are effectively proposing a research program, one that AAAI’s scientific community is well positioned to advance.

The timing of the initiative is notable. Public debate about AI ethics has intensified as generative models have proliferated, and governments around the world have moved to regulate the technology through frameworks that address risk, transparency, and accountability. Yet the knowledge base informing these efforts remains uneven, and practitioners on the ground, from teachers to product managers to civil servants, often lack accessible, authoritative guidance grounded in current research. A sustained series that publishes regularly, engages leading international researchers, and connects ethical theory to actionable practice could serve as a bridge between the technical community that builds AI and the societies that must live with it. Whether it succeeds will depend on the volumes that follow, but the ambition is clear: to replace a fragmented conversation with a coherent, cumulative body of work.

For researchers, students, and policymakers wondering where to begin with AI ethics, the series offers a structured entry point that begins where the stakes are most tangible. The first book’s arrival later this month will show whether the editors’ vision of constructive, evidence-based AI ethics can be translated into guidance that educators can actually use, and the December 2026 publication of the second volume will test the series’ reach beyond institutional settings into the broader human question of what it means to know what is real. What is already evident is that two of the most authoritative institutions in artificial intelligence have concluded that the field’s most pressing challenges are ethical ones, and that answering them requires the same rigor, patience, and cumulative effort that science itself demands.

Subject of Research: Ethical implications and governance of artificial intelligence, examined through a new AAAI and CRC Press book series beginning with AI in education

Article Title: Crucial questions about AI ethics explored in new book series from CRC Press and Association for the Advancement of Artificial Intelligence

Article References: Crucial questions about AI ethics explored in new book series from CRC Press and Association for the Advancement of Artificial Intelligence. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: AI ethics, AAAI, CRC Press, Francesca Rossi, Virginia Dignum, AI in education, academic integrity, data privacy, algorithmic bias, deepfakes, generative AI, AI governance

The stakes are higher than they might appear. Psychological assessment increasingly depends on language, narrative, and context, precisely the raw material that large language models and large multimodal models process best. An emerging vision sometimes called generative psychometrics proposes using these models to organize unstructured subjective data and produce structured psychological characterizations while preserving quantitative rigor. But the review stresses that a model’s ability to produce coherent, clinically plausible language says nothing about whether its outputs measure anything at all. Under established frameworks such as the Standards for Educational and Psychological Testing and the COSMIN taxonomy, validity is an evidentiary claim about how scores are interpreted and used, and that burden does not shrink because a machine wrote the items.

The authors draw a crucial terminological line by distinguishing four levels of evidence: raw data, candidate indicators, algorithmic scores, and validated psychometric measures. Digital traces from smartphones, chatbots, ecological momentary assessment, wearables, social media, and clinical notes should not be treated as psychometric measures by default, they argue. They remain raw data or candidate indicators until their construct interpretation and intended use are theoretically specified, technically verified, empirically calibrated, and validated with human data. Only then can a digital or AI-assisted system earn the label of a psychometric measure. This framing directly challenges the growing habit of treating any quantifiable signal, from heart-rate variability to keyboard dynamics, as if quantification alone conferred measurement status.

To organize the field, the review maps generative AI onto a seven-stage lifecycle of measurement development: construct definition, generation of items or signals, evaluation of content and data quality, piloting and calibration, validation, fairness and measurement invariance, and finally scoring, interpretation, and documentation. At each stage, AI can assist but no stage should be delegated entirely to it. Construct definition remains the anchor: language models can summarize literature and propose domains, but their conceptualizations are probabilistic syntheses of training data rather than theory-driven definitions, and they risk reinforcing dominant-language or culturally narrow perspectives. In mental health, where neighboring constructs such as distress, depression, burnout, and loneliness overlap semantically while differing clinically, that imprecision is dangerous.

The empirical evidence so far supports a conservative reading. Studies of AI-generated items, including ChatGPT-produced concept inventory items in physics education and machine-authored personality items, show that such material can achieve acceptable psychometric properties only after careful prompt engineering, expert review, selection, and testing with real respondents. Even then, problems of ambiguity, redundancy, and weak construct alignment persist. The review also highlights the V3 framework for biometric monitoring technologies, which separates verification, analytical validation, and clinical validation, and notes that generative AI cannot compensate for unverified devices, poorly validated algorithms, or unexamined missingness in sensor-derived data.

One of the most provocative findings concerns synthetic respondents. Researchers have begun using large language models as simulated survey participants to pre-screen items before costly piloting. In one study, six large language models were evaluated as respondents within an item response theory framework, and some model-derived item parameters correlated highly with human-calibrated ones. But the models’ ability distributions were markedly narrower than human distributions, failing to reproduce real population variability. Synthetic responses may help flag obviously poor items or prioritize candidates for pilot testing, the authors conclude, but they cannot replace calibration in human samples when the goal is estimating symptom severity, prevalence, clinical cut-offs, or treatment response.

The review also introduces a conceptual distinction that may shape the field for years: GenAI for psychometrics, in which generative models support measurement development and scoring, versus psychometrics for GenAI, in which psychometric methods are turned on the models themselves. When a large language model becomes an active component of a measurement pipeline, eliciting narratives, classifying chatbot turns, scoring open-ended responses, or acting as an automated judge, its stability, bias, prompt sensitivity, and behavioral consistency directly affect the validity of the resulting scores. Recent preprint evidence underscores the concern: an LLM-native psychometric instrument found that stable model self-reports did not reliably predict observed model behavior across 25 models, suggesting a model can appear internally consistent while still missing the construct human observers care about.

The catalog of failure modes is extensive. Construct drift can occur when generated items gradually shift the target construct while remaining superficially coherent. Face validity without structural validity arises when AI-generated items sound right but prove factorially unstable or poorly related to external criteria. Algorithmic bias can emerge because training corpora encode and reproduce biases tied to gender, race, age, language, disability, culture, and socioeconomic position, making measurement invariance and differential item functioning non-optional analyses rather than afterthoughts. Prompt sensitivity and model drift mean that outputs can change with a provider’s silent update, undermining reproducibility across time and sites. Automation bias tempts clinicians and researchers to over-trust machine outputs, and data-security failures involving sensitive mental health information threaten confidentiality, scoring integrity, and public trust.

There is also a subtler philosophical risk: the flattening of subjectivity. Mental health constructs are inherently subjective, contextual, and culturally mediated. When open-text responses, clinical narratives, or expert annotations are aggregated into single labels, consensus scores, or embeddings, meaningful disagreement and individual variability can be obscured. The authors argue that even technically sophisticated AI-assisted systems may produce psychometrically limited scores if they do not preserve, model, or report the uncertainty and plurality of human psychological experience, an emerging concern that remains far from standard practice in annotation and evaluation pipelines.

The paper closes with a five-priority research agenda: treating fairness and measurement invariance as first-order requirements; establishing longitudinal reproducibility, prompt stability, and recertification rules as models drift; defining the boundary conditions under which LLM respondents and synthetic data help or harm calibration; building psychometric benchmarks for evaluating model behavior; and clarifying validity standards for multimodal measurement combining text, speech, wearables, and clinical notes. The authors also demand radical transparency, calling for reporting of exact model names, versions, prompts, inference settings, human edits, and provenance of every AI-generated element, in line with guidelines such as TRIPOD-LLM and CONSORT-AI. Their conclusion is deliberately conservative: generative AI should augment, not replace, psychometric science. It does not reduce the need for psychometrics; it makes psychometrics more central, more demanding, and more consequential than ever.

Subject of Research: Psychometric applications and risks of generative artificial intelligence in digital mental health measurement

Article Title: Psychometric applications of generative artificial intelligence: Lifecycle, risks, and research agenda

Article References: Villarreal-Zegarra, D., Paredes-Gonzales, Y., & García-Serna, J. (2026). Psychometric applications of generative artificial intelligence: Lifecycle, risks, and research agenda. PLOS Mental Health, 3(9), e0000702. https://doi.org/10.1371/journal.pmen.0000702

Image Credits: AI Generated

DOI: 10.1371/journal.pmen.0000702

Keywords: generative AI, psychometrics, large language models, digital mental health, measurement validity, psychological assessment, algorithmic bias, measurement invariance, synthetic data, item generation, automation bias, PLOS Mental Health

The study, published in the journal AI & Society by Ivan Efreaim Gozum of the University of Santo Tomas in Manila, examines how AI-assisted triage systems allocate scarce medical resources during public health emergencies and in ordinary emergency department settings. Gozum’s central claim is provocative: contemporary triage frameworks, even those that employ sophisticated multidimensional clinical criteria beyond the widely used SOFA scores, remain ethically incomplete. They address the calculable dimensions of patient care while leaving the moral and relational dimensions of clinical judgment underdeveloped, and AI systems inherit and amplify that incompleteness.

The technical context matters here. Triage scoring systems such as the Sequential Organ Failure Assessment, or SOFA, score were developed to predict mortality in critically ill patients by quantifying dysfunction across organ systems. During the COVID-19 pandemic, such scores became central to allocation protocols for intensive care beds and ventilators, and researchers have since validated their predictive value across diverse patient populations, including patients with and without COVID-19 infection. More recently, machine learning models have been developed to predict critical outcomes in emergency departments, and systematic reviews document a rapid expansion of AI-based triage tools into clinical practice worldwide.

Yet the empirical record on these tools is sobering. A landmark study published in Science in 2019 dissected racial bias in a widely deployed algorithm used to manage the health of populations, showing how a seemingly neutral proxy variable led the system to systematically underestimate the illness of Black patients. Reviews of prediction models developed during the pandemic found that most were at high risk of bias and few were suitable for clinical use. Studies of emergency department professionals’ experiences with machine learning triage integration reveal persistent concerns about opacity, trust, and the erosion of clinical autonomy. Gozum’s analysis takes these documented problems as its starting point: algorithmic bias, opacity, and the reduction of persons to calculable outcomes are not edge cases but structural features of how triage AI is currently conceived.

What would Aquinas add to this picture? Gozum’s framework draws on the virtue ethics tradition that Aquinas inherited from Aristotle and systematized in the Summa Theologica. Virtue ethics differs fundamentally from the utilitarian and deontological frameworks that dominate mainstream bioethics. Where utilitarian approaches, which gained prominence during pandemic allocation debates, ask which policy maximizes overall benefit, and Kantian approaches ask which rules respect persons as ends in themselves, virtue ethics asks a prior question: what kind of moral agent does good care require, and what habits of character enable that agent to judge well in particular, concrete circumstances?

Gozum develops this Thomistic framework across three ethical levels. The first concerns AI system design itself. For Aquinas, justice is the virtue that gives each person their due, and Gozum argues that a triage algorithm designed without attention to justice, including fairness across social groups and the dignity of patients who cannot be captured by a score, fails at the level of its architecture. The second level concerns the moral agency of healthcare practitioners. Prudence, in the Thomistic sense, is not mere caution but the perfected ability to deliberate well about particular actions, and Gozum contends that clinicians using AI triage must exercise precisely this practical wisdom, treating the algorithm’s output as one input among many rather than as a verdict. The third level concerns transparency and the conditions for responsible human oversight, since oversight is impossible when the reasoning behind a triage recommendation cannot be inspected, questioned, or overridden.

Crucially, the framework positions AI as an assistive instrument rather than a replacement for ethical discernment. This distinction echoes arguments from the philosophy of technology about the technological imperative, the tendency to adopt and extend a technology simply because it exists and works. Gozum’s Thomistic response is that the moral worth of an action lies in the free, informed choice of the agent, not in the efficiency of the tool. A triage system that outputs rankings faster than any human committee still does not make a moral decision; it makes a statistical estimate, and the moral decision remains with the clinician and the institution that act on it. Charity, the third virtue in Gozum’s framework, grounds this relational view: care is not the distribution of resources to outcomes but a relationship between persons, and systems that obscure that relationship, however accurate their predictions, diminish the practice of medicine.

The study situates itself within ongoing bioethical debates about resource allocation that intensified during COVID-19. Influential frameworks published in the New England Journal of Medicine in 2020 proposed maximizing benefits, treating people equally, and prioritizing the worst off as core values for scarce resource allocation. Other scholars defended explicitly utilitarian approaches for the pandemic, while disability rights scholars and ethicists raised alarms about quality-of-life judgments that could systematically disadvantage disabled patients. Gozum does not reject these frameworks outright, but argues that they insufficiently address the moral and relational dimensions of clinical judgment, and that a virtue-centered account can complement them by specifying who must judge, how they must be formed, and what institutional conditions make good judgment possible.

The practical implications are concrete. At the level of governance, Gozum calls for an interdisciplinary framework integrating theology, philosophy, bioethics, and healthcare governance, an unusual combination in AI policy circles but one that reflects the breadth of existing AI ethics guidance, which a major 2019 review found to converge on principles of transparency, justice, non-maleficence, responsibility, and privacy without specifying how to enact them. Virtue ethics offers an enactment mechanism: institutions can cultivate prudence in clinicians through training that treats AI literacy as a moral skill, demand justice in procurement by auditing algorithms for disparate performance across patient groups, and require charity, in the sense of relational care, by ensuring that no patient’s care is reduced to an automated score. The approach also builds on Gozum’s earlier work connecting Catholic social teaching with AI ethics to address inequity in AI healthcare and on philosophical work applying Thomistic risk analysis to social welfare.

Whether a medieval theology of virtue can meaningfully discipline twenty-first-century machine learning remains an open question, and skeptics will note that virtue formation is slow where algorithmic deployment is fast. But the study’s timing is pointed. AI triage systems are moving from pilot studies to routine use in emergency departments, and the documented failures of biased and opaque algorithms suggest that technical fixes alone have not resolved the ethical core of the problem. Gozum’s argument is that the core was never technical in the first place. Deciding who is treated first in an emergency is an act of practical wisdom performed by moral agents accountable to the people they serve, and any system, however intelligent, that displaces that act also displaces the responsibility and the dignity that come with it. In that sense, the oldest ethical tradition in the conversation may have the newest thing to say.

Subject of Research: Thomistic virtue ethics as a framework for the ethical use of AI triage systems in healthcare emergencies

Article Title: Aquinas and the ethics of AI triage systems in healthcare emergencies

Article References: Gozum, I. E. (2026). Aquinas and the ethics of AI triage systems in healthcare emergencies. AI &amp; SOCIETY. https://doi.org/10.1007/s00146-026-03225-0

Image Credits: AI Generated

DOI: 10.1007/s00146-026-03225-0

Keywords: artificial intelligence, triage, healthcare ethics, Thomas Aquinas, virtue ethics, emergency medicine, algorithmic bias, resource allocation, medical ethics, AI & Society, human dignity, clinical decision-making

The researchers assembled a baseline of 33 U.S. testing locations known to be in use in 2019, then added six additional cities documented as automated vehicle testing sites in 2025. Comparing the two snapshots allowed them to track how the geography of testing has evolved over a period when the industry matured dramatically. Using U.S. Census Bureau QuickFacts data and other public sources, they characterized each site by racial and ethnic composition, disability prevalence, poverty rates, age structure, gender balance, bicycling rates, and economic context. The goal was not simply to map where testing happens but to quantify what kinds of people and conditions the vehicles encounter during the millions of miles that feed their development.

Several trends emerged from the comparison. The share of residents identifying as White alone declined across many testing cities, a shift the authors note is consistent with changes in how the 2020 Census measured and processed race data, including a marked increase in multiracial reporting. Disability shares rose modestly in the tested locations, reflecting both demographic reality and improved measurement. Poverty rates declined modestly in many of the cities studied. None of these shifts, however, altered the study’s central finding: the sites where autonomous vehicles gain public-road experience remain overwhelmingly urban, even though roughly one in five Americans lives in a rural area and rural roads carry a disproportionate share of fatal crashes.

That urban bias matters for reasons that go beyond fairness optics. Rural driving presents conditions that city miles rarely supply: high-speed two-lane roads with narrow or absent shoulders, unmarked lanes, gravel surfaces, wildlife crossings, infrequent lighting, and long stretches with no cellular coverage. Emergency response times are longer, and the consequences of a perception failure can be more severe. The study’s reference literature documents these challenges in detail, including reviews of autonomous vehicle deployment in rural areas and federal initiatives such as the U.S. Department of Transportation’s Rural Autonomous Vehicle research program launched in April 2025. A vehicle trained and validated almost exclusively on dense urban grids may simply lack the exposure needed to handle a deer at dusk on a farm road.

The demographic dimension is equally consequential. Modern perception systems are built on machine learning, and machine learning inherits the biases of its training data. The study cites a growing body of fairness research showing that object detection systems can exhibit predictive inequity across demographic groups, and that pedestrian detection performance can degrade depending on pose, occlusion, and skin tone. Pedestrians who use wheelchairs face documented disparities in road crash mortality, yet they remain rare in typical training and testing exposure. If a vehicle’s sensors and decision algorithms are tuned primarily on the populations of a handful of tech-friendly cities, the safety case for deployment elsewhere rests on assumptions the data may not support.

What distinguishes this analysis is its insistence that diversity in testing must extend beyond geography and demographics into behavioral and data-pathway domains. Behavioral diversity means ensuring that vehicles encounter the full range of road user conduct: bicyclists commuting in large numbers, pedestrians with varied gait patterns and mobility aids, drivers who behave in ways that differ from the norms of San Francisco or Phoenix. Data-pathway diversity addresses a subtler problem: it is not enough for a vehicle to physically pass through a community. The system must actually record and learn from its interactions with under-represented groups, so that those encounters flow into the development pipeline rather than vanishing as unlogged miles.

The authors also introduce the idea of public-exposure diversity, arguing that trust in autonomous vehicles is built through visible, repeated, equitable contact with the technology. Communities that never see a self-driving car during its formative testing years may be asked to accept a system they had no role in shaping, a dynamic the study links to research on public perception and adoption. Perception studies cited in the paper show that people’s comfort with automated vehicles is influenced by demographic and contextual factors, and that automation itself can alter how pedestrians judge safety. Exposure, in other words, is not just a data collection strategy; it is a social contract that determines who benefits from and who trusts the technology.

To make the framework actionable, the researchers propose practical criteria for diversified pilot programs, spanning rural share of vehicle miles traveled, racial and ethnic representation, disability visibility, gender balance, bicycling prevalence, and economic context. These metrics give operators, regulators, and municipalities a common vocabulary for evaluating whether a testing program is genuinely broadening its exposure or simply accumulating more miles in familiar territory. The framework is explicitly complementary to simulator-based and dataset-driven testing, which can scale virtually but cannot guarantee that the underlying data reflects the full population. Where and how exposure occurs, the authors argue, provides a population-level foundation for safety, trust, and equity that no simulation alone can supply.

The study’s own data caveats are worth noting. The apparent decline in White alone shares across testing sites partly reflects Census measurement and processing changes rather than dramatic demographic turnover, and the modest trends in disability and poverty shares are sensitive to how those categories are defined and reported. The authors are transparent that their location list, drawn from public documentation including the Bloomberg Philanthropies AVs in Cities guide and Census QuickFacts, captures documented testing rather than every mile driven by every operator. Still, the pattern is robust enough to support the central claim: the industry’s real-world learning environment is narrower than the population it intends to serve.

The stakes of this gap are not hypothetical. The 2018 fatality involving a self-driving Uber test vehicle in Arizona remains a defining moment for the field, and subsequent research on crash injury trends has emphasized how occupant and road user characteristics shape outcomes. As autonomous vehicles move from pilots toward commercial deployment, the question of whose streets, whose bodies, and whose behaviors are represented in their training and validation becomes a matter of public health and social justice, not merely engineering. This analysis offers a measurable way to hold the industry accountable, arguing that the road to trustworthy autonomy must be paved with testing that looks like America in all its demographic, geographic, and behavioral variety, and that the miles a self-driving car logs should count only if the people it encounters are the people it will one day carry and protect.

Subject of Research: Diversity of autonomous vehicle testing sites in the United States

Article Title: An analysis of autonomous vehicle testing site diversity

Article References: Gettel, O., Gates, C. W., Gettel, C. D., Huggins, J., & Peters, D. (2026). An analysis of autonomous vehicle testing site diversity. AI &amp; SOCIETY. https://doi.org/10.1007/s00146-026-03269-2

Image Credits: AI Generated

DOI: 10.1007/s00146-026-03269-2

Keywords: autonomous vehicles, testing sites, diversity, rural roads, algorithmic bias, pedestrian detection, public trust, equity, Census data, vehicle miles traveled, machine learning, transportation safety

The technology itself has traveled an extraordinary distance since its origins in the 1960s, when Woodrow Wilson Bledsoe’s early systems required researchers to manually plot the coordinates of facial features on photographs. Decades of advances in pattern recognition and computing transformed those laborious beginnings into today’s artificial intelligence-powered systems, which can automatically detect, encode, and match faces across massive image collections in real time. For law enforcement, the appeal is obvious: retrospective identification of suspects, generation of investigative leads, location of missing persons, verification of arrested individuals who refuse to identify themselves, and rapid screening of crowds at airports, stadiums, and border crossings. Under ideal conditions, top-performing systems achieve impressive accuracy. But real-world deployments rarely offer ideal conditions, and the study emphasizes that poor lighting, obscured views, and low-quality imagery can significantly degrade performance precisely in the situations where police are most likely to rely on the technology.

The technical weaknesses are not evenly distributed across the population, and this is where the ethical stakes become acute. Testing conducted by the U.S. National Institute for Standards and Technology in late 2019, covering nearly 200 facial recognition algorithms, found that Asians, African Americans, and American Indians were more likely to be misidentified than white individuals, that women were misidentified more often than men, and that children and the elderly were misidentified more often than middle-aged adults. Researchers attribute these discrepancies largely to unrepresentative training data, and some newer algorithms show no detectable bias, suggesting the problem is fixable through better development practices. In the meantime, however, the consequences are real: the study notes six documented wrongful arrests of Black individuals in the United States connected to facial recognition use, a figure that likely understates the problem because no comprehensive data exists on how extensively American law enforcement actually deploys these systems.

The United States exemplifies regulatory fragmentation. There is no federal law governing facial recognition, leaving states and cities to experiment on their own. San Francisco became the first American city to ban the technology for public agencies in 2019, followed by Oakland, California, and Somerville, Massachusetts, and more than twenty-two local governments have adopted surveillance oversight rules modeled on an American Civil Liberties Union template. At the same time, private actors have operated at extraordinary scale with little constraint. Clearview AI scraped tens of billions of facial images from the public internet, including from social media platforms, without the knowledge or consent of the people pictured, prompting the ACLU to sue under Illinois’ Biometric Information Privacy Act. The company was barred from selling its system to most private businesses, yet it continues to sell access to state, local, and federal law enforcement agencies across the country.

Canada’s experience illustrates a different failure mode: adoption outpacing oversight. The Royal Canadian Mounted Police used Clearview AI until public outcry halted the practice in 2020, and reporting revealed the force had also employed other tools, including Amazon’s Rekognition for combating human trafficking of minors and other platforms for counterterrorism work that were later abandoned over privacy concerns. In February 2021, the Office of the Privacy Commissioner of Canada determined that Clearview AI’s app was illegal under Canadian privacy law, which requires consent for the use of personal data. By 2022, the Commission issued guidance for police use of facial recognition while candidly acknowledging that no specific legal framework exists in Canada, only a patchwork of federal and provincial privacy statutes, police powers legislation, and Charter jurisprudence. Unlike the United States, however, Canada benefits from a national privacy commission alongside provincial and territorial commissioners, such as the Ontario Information and Privacy Commissioner, which has issued detailed recommendations emphasizing accountability and transparency for police use of the technology.

Germany stands out as the most restrictive of the five cases, a stance rooted in both history and law. The European Union’s General Data Protection Regulation, particularly Article 9, prohibits the processing of biometric data unless justified by public interest, law enforcement needs, or explicit consent, and Germany’s Federal Data Protection Act reinforces these protections for both private businesses and public agencies. Concepts such as the right to be forgotten make deploying facial recognition in public spaces exceptionally difficult, effectively confining the technology to law enforcement applications subject to judicial oversight. German politicians have pushed for bans on facial recognition in public spaces and have resisted provisions of the EU’s Artificial Intelligence Act that would permit warrantless use by police. A telling test came in 2017, when authorities ran a prototype project at Berlin Südkreuz railway station attempting to identify consenting volunteers at known times; the accuracy results were described as underwhelming. Despite continued advocacy by public agencies to expand facial recognition for public safety, surveys consistently show German citizens are less supportive of the technology than their peers in other nations.

Italy and France, both unitary states with uniform national policies, have taken sharply divergent paths. Italy deployed thermal facial recognition during the COVID-19 pandemic to monitor travelers at airports, schools, and train stations, but in the pandemic’s wake the country prohibited biometric facial recognition except for judicial investigations and crime fighting. After the Clearview AI data leak scandal, Italy’s data protection authority, the Garante, banned use of the technology on limited legal grounds, and a moratorium on facial recognition video surveillance in public spaces remains in place through 2025, with fines imposed for violations. France, by contrast, has embraced the technology, most visibly through the mass surveillance apparatus deployed for the 2024 Paris Olympic Games under public safety exceptions. Critics, including civil society organizations, argue that French deployments often occur without meaningful explicit consent and that the government has prioritized security and efficiency over individual privacy, raising fears of a drift toward a surveillance state within an EU framework nominally designed to prevent exactly that.

The comparative analysis, organized around three dimensions—legal compliance, ethical alignment, and effectiveness of implementation—shows that political structure shapes regulatory outcomes. Federal systems like the United States and Canada produce fragmented rules that vary by state or province, which enables policy experimentation but creates inconsistencies that, combined with the technology’s demographic accuracy gaps, can produce unequal policing outcomes across communities. Germany’s federalism is less fragmented because national policy sets a restrictive floor. Unitary systems can impose uniform rules quickly, but that uniformity cuts both ways: France could rapidly extend facial recognition nationwide for the Olympics, while Italy could just as rapidly impose a nationwide moratorium. On consent, the five countries are largely similar in one troubling respect: explicit consent is generally not required for facial recognition use in the criminal justice system, even though it is typically required in other settings, with Canada allowing warrantless-style use only where an explicit need exists.

The authors argue that no existing AI ethics framework specifically addresses facial recognition in criminal justice, and they call for one built on privacy, accountability, and consent, incorporating mechanisms such as the GDPR’s privacy by design and privacy by default principles and mandatory Data Protection Impact Assessments for high-risk applications. Embedding these safeguards at the design stage, they contend, would surface technical risks before systems reach the street. The study also warns that diminished public trust in governance breeds skepticism even toward well-intentioned deployments, and that the chaotic rollout of facial recognition in many countries has deepened that distrust. Global institutions such as the United Nations will need to grapple with ethical principles for facial recognition and artificial intelligence more broadly, while domestic policymakers balance international guidance against the demands of their own citizens—no small task given how much public attitudes toward privacy and security vary across nations.

The research has limits the authors acknowledge freely: it relies on secondary sources, covers only five developed democracies, and excludes authoritarian regimes where facial recognition poses the gravest human rights risks, such as the European Court of Human Rights ruling against Russia for using the technology to arrest political protestors. Future work, they write, should gather primary data from policymakers, legal experts, and the public, extend the comparison to less developed democracies and autocracies, and flesh out an ethical framework adaptable to different political, social, and cultural contexts. What the current analysis makes unmistakably clear, however, is that the technology is already embedded in democratic policing, its error rates fall hardest on marginalized groups, and the absence of clear, tailored legislation has left citizens’ rights protected unevenly at best. As cameras multiply and algorithms improve, the window for democracies to regulate deliberately rather than reactively is narrowing—and the five countries studied here offer both a warning and a menu of possible paths forward.

Subject of Research: Comparative regulation of facial recognition technology in criminal justice arrests across five democracies

Article Title: Global perspectives on regulating facial recognition technology utilization for criminal justice arrests

Article References: Robles, P., Mallinson, D. J., Best, E., Devaney, C., & Azevedo, L. (2025). Global perspectives on regulating facial recognition technology utilization for criminal justice arrests. Global Public Policy and Governance, 5(2), 186-204. https://doi.org/10.1007/s43508-025-00117-9

Image Credits: AI Generated

DOI: 10.1007/s43508-025-00117-9

Keywords: facial recognition, criminal justice, privacy, GDPR, algorithmic bias, law enforcement, surveillance, AI ethics, Clearview AI, data protection, civil liberties, comparative policy