The scale of the underlying dataset is one of the study’s most striking features. The team examined primary and secondary care costs for the financial year 2018/19 among 10,119,490 adult patients registered with general practices contributing to the Clinical Practice Research Datalink and linked to Hospital Episode Statistics, a cohort representing roughly 23 percent of the adult population of England. Because the data capture both general practice contacts and hospital activity, the researchers could compute a genuinely comprehensive picture of each patient’s annual cost to the system, valued in 2018 prices using established unit cost references. High-cost patients were defined as those falling in the top one percent of total care costs, a threshold widely used in health economics because spending is so heavily concentrated at the extreme of the distribution.

The numbers illustrate that concentration vividly. High-cost patients had average annual costs of £17,485, with a median of £14,609 and an interquartile range running from £12,028 to £19,633. The rest of the population, by contrast, averaged just £653 per year, with a median of only £103 and an interquartile range of £14 to £352. In other words, the typical high-cost patient cost the system well over a hundred times as much as the typical patient, and even the cheapest quarter of the high-cost group outspent the most expensive quarter of everyone else by a wide margin. Understanding who these patients are, the authors argue, is therefore central to any serious attempt at resource allocation, case management and health system planning.

Methodologically, the study is a careful exercise in unsupervised learning, the branch of machine learning that searches for structure in data without any predefined labels. The researchers clustered patients on the basis of diagnoses, age, sex, deprivation measured by the Index of Multiple Deprivation, and ethnicity. They compared multiple algorithms, including approaches built on multiple correspondence analysis and t-distributed stochastic neighbour embedding for dimensionality reduction, before settling on hierarchical clustering as the optimal method. That choice was not arbitrary: the team evaluated candidate solutions using established validity and stability indices, including the Davies–Bouldin Index, the Calinski–Harabasz index and the Adjusted Rand index, which reward clusters whose members are internally similar, well separated from other clusters, and reproducible under resampling.

The winning solution produced nine clusters with clearly differentiated multimorbidity and sociodemographic profiles. Mean age across the clusters ranged from 56 to 79 years, a reminder that high costs are not confined to the very old. Mean annual costs per cluster ranged between £15,792 and £19,107, with confidence intervals indicating that these differences were estimated precisely. The spread of clinical pictures was equally wide. Several core profiles recurred across different clustering approaches and across subgroup analyses, suggesting they represent genuine and robust archetypes within the high-cost population rather than statistical artefacts of a single algorithm.

Among those recurring archetypes, four stand out. Younger people with liver disease and mental health conditions formed one recognisable group, a combination that clinicians will recognise as associated with complex social circumstances and frequent emergency attendance. Patients with nodal metastases, meaning cancers that have spread to lymph nodes, formed another, as did patients with prostate cancer complicated by nodal metastases. The fourth recurring profile comprised older people with cardiovascular disease and dementia, a group whose costs typically accumulate through repeated admissions, long hospital stays and heavy reliance on both primary and secondary care. That the same profiles emerged repeatedly, including in analyses restricted to in-year decedents and to patients with the highest unplanned care costs, strengthens the case that these are stable, clinically meaningful segments.

The study also stratified its analyses by age, examining age-stratified subgroups alongside the whole population. This matters because the drivers of high cost plausibly differ across the life course: a 58-year-old with alcoholic liver disease and depression uses services very differently from an 80-year-old with heart failure and cognitive decline. By confirming that the core clinical profiles persisted across these stratifications, the researchers provide evidence that segmentation is not simply a proxy for age. The consistency of the clusters across methods and subgroups was assessed formally, and the authors report that the hierarchical solution performed best on the validity indices they applied, giving the nine-group taxonomy a firmer empirical footing than ad hoc clinical categorisations.

Why does this matter for policy? Population health management, the approach increasingly adopted by integrated care systems in England, depends on being able to identify groups of patients whose needs are similar enough to be addressed by a common service model. If the top one percent of patients were homogeneous, a single intensive case management programme might suffice. The new analysis shows the opposite: a younger cohort dominated by liver disease and mental illness needs addiction, psychiatric and social care integration, while an older cohort with dementia and cardiovascular disease needs falls prevention, medication review and coordinated discharge planning. Designing services for a blended average of these groups risks serving none of them well. Cluster-based segmentation, the authors suggest, could inform the targeting of case management programmes, the allocation of budgets and longer-term workforce and capacity planning.

The research also carries implications for how health systems think about equity. By incorporating deprivation and ethnicity into the clustering alongside diagnoses, the analysis acknowledges that cost is shaped by social context, not just by disease burden. Patients in deprived areas often present later, have more multimorbidity at younger ages, and generate higher unplanned care costs, so a segmentation that ignores socioeconomic characteristics would systematically misdescribe the population. The study was approved by the Clinical Practice Research Datalink’s independent scientific advisory committee and used anonymised patient data under ethical approval from the Health Research Authority, and the authors declare no competing interests. Funding came from the Health Foundation and the National Institute for Health and Care Research, with the funders having no role in study design, analysis or the decision to publish.

Limitations and next steps remain. The analysis reflects a single financial year, 2018/19, before the disruptions of the pandemic reshaped waiting lists and care patterns, and cost data capture NHS spending rather than the full social care picture. Unsupervised clustering, however carefully validated, identifies statistical structure rather than causal mechanisms, so the clusters describe who high-cost patients are rather than why their costs are high. Even so, the study offers one of the most comprehensive segmentations of a national high-cost population attempted anywhere, drawing on a cohort of over ten million linked primary and secondary care records. Its central message is likely to travel well beyond England: the costliest patients are a heterogeneous collection of recognisable clinical archetypes, and health systems that learn to see them as distinct groups, rather than as an undifferentiated burden, will be better placed to design care that is both more effective and more humane.

Subject of Research: Machine learning segmentation of high-cost patients in the English NHS

Article Title: Segmentation of high-cost patients in England: insights from a machine learning analysis

Article References: Segmentation of high-cost patients in England: insights from a machine learning analysis. (n.d.). https://doi.org/10.1186/s12916-026-05233-9

Image Credits: AI Generated

DOI: 10.1186/s12916-026-05233-9

Keywords: high-cost patients, machine learning, hierarchical clustering, multimorbidity, NHS, health economics, population health management, CPRD, Hospital Episode Statistics, patient segmentation, healthcare costs, England

Patient segmentation, the practice of dividing a patient population into groups with similar needs, has become a cornerstone of modern health service planning. Most current approaches rely on register data: diagnoses, medication purchases, hospital admissions, and cost histories crunched by statistical or machine-learning models. These methods are powerful, but they see only the administrative shadow of a patient’s life. What they cannot easily capture is how a person actually copes day to day — whether they feel enabled to manage their own health, whether their values and functional ability align with the care they receive, and how they experience their own health status. Navigator, developed in Finland, was designed to fill precisely that gap by asking patients and the professionals who care for them directly.

The service works through paired questionnaires. Patients answer questions covering their individual values, mood, and ability to function in everyday life, while nurses and other professionals assess the patient’s health status and how well current care is fulfilling the patient’s needs. Because Navigator draws on this human-reported information, its developers argue it can supplement data-driven segmentation with a richer, more personal picture of each individual. Until now, however, a fundamental question hung over the tool: did it actually measure what it claimed to measure? That question — the issue of validity — is what the research team led by Riikka Riihimies of Tampere University set out to answer.

The study took place at Valkeakoski Health Center in Finland, where sixteen nurses used Navigator with 304 diabetic patients receiving primary care. Diabetes was a deliberate choice of study population: people living with diabetes typically have multiple interacting health concerns, long-term relationships with primary care services, and wide variation in how well they cope with daily self-management, making them an ideal group for testing whether a segmentation tool can distinguish meaningful differences between individuals. The study received ethical approval from Tampere University Hospital’s Ethics Committee, and all participants gave written informed consent.

The researchers ran two complementary tests. The first, construct validity, asked whether Navigator’s internal structure matched its theoretical design. Using exploratory factor analysis, a statistical technique that searches for hidden patterns underlying responses to a set of questions, the team examined whether the questionnaire items clustered into the dimensions Navigator was built around. The second test, concurrent validity, asked whether Navigator’s scores moved in step with established, well-validated instruments measuring similar things. If Navigator’s functional ability dimension correlated strongly with a recognized disability measure, for example, that would be evidence the tool is capturing something real.

The factor analyses delivered encouraging news for the tool’s designers. In the patient questionnaire, the analysis identified two coherent factors, which the researchers labeled the patient’s mood and the patient’s enablement — the sense of being equipped and empowered to manage one’s own situation. In the professional questionnaire, two further factors emerged: fulfilling care, reflecting how well the patient’s needs are being met by current services, and the patient’s health status. These four factors lined up neatly with the dimensions Navigator was intended to measure, providing statistical confirmation that the questionnaires hang together in a theoretically sensible way rather than producing noise.

The construct validity checks extended beyond internal structure. The team derived factor scores and tested specific hypotheses about how different groups should respond. They predicted that men and women would show only small differences in their Navigator responses, and that patients aged 75 and older would differ markedly from younger patients on dimensions tied to enablement, fulfilling care, and health status. Both predictions were largely borne out: gender differences were modest, while age-group differences were significant on most factor scores. A tool that behaves exactly as theory says it should, in the directions theory says it should, gains credibility as a genuine measuring instrument rather than an arbitrary scoring exercise.

The concurrent validity results were equally substantive. Navigator’s functional ability dimension correlated strongly and significantly with four established patient-reported measures: the World Health Organization Disability Assessment Schedule 2.0, a comprehensive disability instrument; self-rated health; the EuroQol Visual Analog Scale, which captures health-related quality of life on a single scale; and a twelve-item well-being questionnaire. All of these correlations were highly significant. Meanwhile, the professional-assessed health status dimension correlated significantly with objective clinical indicators, including the number of chronic conditions the patient reported, the total amount of medication used, glycated hemoglobin reflecting blood sugar control, the urine albumin-to-creatinine ratio signaling kidney involvement, smoking status, and body-mass index. In other words, what patients and nurses reported through Navigator tracked closely with both validated subjective measures and hard clinical data.

For clinicians and health service managers, the significance of these findings lies in what Navigator could add to existing segmentation practice. Data-based methods excel at identifying high-cost, high-need patients from administrative records, but they are blind to the personal experience of illness. A patient whose records look moderate on paper may in fact be struggling badly with daily life, while another with an intimidating diagnosis list may be coping confidently and needs little extra support. By adding structured information about mood, enablement, functional ability, and how well care is fulfilling individual needs, Navigator offers a way to refine the groups produced by register-based algorithms and to direct resources with finer granularity. The authors conclude that the tool appears valid for measuring patients’ coping in everyday life and health status, and may therefore supplement data-based segmentation by deepening understanding of the individuals within patient groups.

Cautions remain, as they do with any validation study. The research was cross-sectional, capturing a single moment in time rather than following patients over months and years, and it involved a single Finnish health center and a specific population of diabetic patients. Whether Navigator performs equally well in other patient groups, other regions, and other health system contexts will require further study, as will evidence that using the tool actually improves outcomes or reduces costs in practice. Still, the study marks an important step for a tool built on a simple premise: that the best way to understand how patients are coping is to ask them, in a structured and scientifically validated way, and to pair their answers with the observations of the professionals who know their care. As health systems everywhere search for ways to personalize care without exploding budgets, instruments like Navigator — now with evidence behind them — may prove that the human voice belongs at the heart of population health analytics.

Subject of Research: Validation of the Navigator patient segmentation service for diabetic patients in Finnish primary care

Article Title: Construct and concurrent validity of the Navigator patient segmentation service: cross-sectional study among diabetic patients in Finnish primary care

Article References: Riihimies, R., Tolvanen, E., Helminen, M., Kosunen, E., & Koskela, T. (2026). Construct and concurrent validity of the Navigator patient segmentation service: cross-sectional study among diabetic patients in Finnish primary care. BMC Health Services Research. https://doi.org/10.1186/s12913-026-15684-8

Image Credits: AI Generated

DOI: 10.1186/s12913-026-15684-8

Keywords: patient segmentation, Navigator, diabetes, primary care, construct validity, concurrent validity, exploratory factor analysis, Finland, health services research, functional ability, questionnaires, eHealth