Systematic Review Finds Weak Qualitative Methods in Diagnostic Test Evaluations

A systematic review published in Diagnostic and Prognostic Research has found that when researchers evaluate new diagnostic tests using qualitative or mixed methods, the studies often fall short on methodological transparency, consistent terminology and patient involvement.

The evaluation of new diagnostic tests has traditionally centred on technical and diagnostic accuracy, the test’s place in the clinical pathway, effectiveness and cost-effectiveness. But the review’s authors point out that other considerations matter for whether a test is ultimately adopted, including how usable and acceptable tests are to clinicians and patients, and how well they fit particular settings and contexts. Questions like these, the authors argue, are best explored through qualitative and mixed methods research, which can capture the social, psychological, organisational, ethical and legal impacts that quantitative designs alone cannot fully address.

To map how such methods are actually being used, the team searched Medline, Embase and CINAHL from inception to 26 September 2025. The search yielded 11,331 results. After removing 6,435 duplicates, 4,896 articles went to title and abstract screening, 4,544 were excluded as irrelevant, and 352 full-text papers were assessed for eligibility. Of these, 324 were excluded, leaving 28 studies in the final review. The review was designed as a ‘knowledge support’ rather than a ‘decision support’ review, meaning it assessed methodological aspects only and was therefore ineligible for PROSPERO registration because it included no outcomes of direct patient or clinical relevance.

Every one of the 28 included studies evaluated point of care tests or technologies. The studies covered seven different medical conditions across 18 countries. Respiratory tract infections dominated, accounting for 12 studies, followed by malaria with three and dengue with two. Single studies addressed SARS-CoV-2, cutaneous leishmaniasis, gonorrhoea and urinary tract infection, while seven studies investigated point of care tests or point of care ultrasound as general investigations for conditions with non-specific symptoms. Over half of the tests, 16 in total, required blood samples. Just over half the studies, 15, were conducted in the UK or Europe, seven in Africa, and the remainder in North America, Southeast Asia, New Zealand and Australia. More than half, 17 studies, took place in primary care, general practice or pre-hospital care settings.

Eighteen of the studies used a qualitative-only design and ten used mixed methods. Semi-structured interviews with healthcare professionals were the sole or principal data collection method in nearly all the studies; all but three featured them, sometimes combined with focus groups or observation and field notes. In 23 studies, participants were healthcare professionals only, while five studies included both patients and healthcare professionals. Thirteen studies used thematic analysis, with grounded theory, constant comparison, content analysis and a framework approach appearing less often. In four studies the analytic approach was not stated or was otherwise unclear.

The quality appraisal, using an adapted version of Spencer and colleagues’ 2003 framework for qualitative studies and an abridged Mixed Methods Appraisal Tool for mixed methods studies, revealed recurring weaknesses. Ten of the eighteen qualitative studies failed to reference or sufficiently describe the analytic steps performed. Only five qualitative studies gave an explicit argument for why qualitative methods suited their aims; the rest relied on implicit justification. Among the mixed methods studies, only three explicitly justified combining quantitative and qualitative approaches, and none provided an explicit justification for the qualitative component. Reporting of qualitative analysis was incomplete in all ten mixed methods studies, with frequent omissions of analytic references and failures to state who performed the analysis.

Integration, a core strength of mixed methods design, was also inconsistent. In four mixed methods studies there was no integration of quantitative and qualitative data at all, three showed explicit integration, and two only implicit. Just two studies summarised how integration was handled in the methods section, and in one of those the stated integration plan did not match the presentation of results, where no evidence of explicit integration appeared. The reviewers also noted that individual interviews and focus groups did not always appear to be treated as distinct ways of generating data, and they argue that focus groups should be recognised as a distinct method suited to discussion-based research questions relevant to real-life implementation.

Terminology emerged as another problem area. Terms such as ‘acceptability’, ‘acceptance’, ‘usability’, ‘satisfaction’ and ‘feasibility’ appeared frequently across the studies but only four studies defined the evaluative terms they used. Three of those drew on the Health Technology Adoption Framework, adapting its pre-defined terms to their contexts, which produced different definitions of the same terms across studies. For example, the term ‘learnability’ was used in two studies with different meanings in each, relating in one to confidence, training and the ease of the learning process and in the other to the ability of the healthcare worker to use the test. The authors note this mirrors earlier work by Sekhon and colleagues, who called for research on ‘acceptability’ to be more robust in defining and assessing the concept.

Patient and public involvement was largely absent. Nineteen of the included studies did not mention involving patients, the public or other stakeholders in study development, and three confirmed patients were not involved. Only six studies involved patients or the public in some way, whether through a patient and public involvement group informing design and dissemination, consultation with stakeholders on proposed research, or patient and community involvement in verifying analysis. The authors suggest possible explanations, including the additional ethics approvals, time and resources that exploring patient views may require, and the perception among some researchers that diagnostic testing concerns clinicians more than patients. They counter that patient care is always likely to be affected by new diagnostic tests, making investment in capturing patient perspectives worthwhile.

The review’s authors acknowledge limitations: because their inclusion criteria focused on evaluations of particular tests using qualitative or mixed methods, the included studies all concerned point of care diagnostics, leaving other types of diagnostic technology unexplored, and the team did not contact study authors for clarification of unreported details. Nevertheless, they argue that as diagnostic technologies proliferate, higher quality and more standardised qualitative and mixed methods approaches are needed so that evidence of wider impacts can count in formal assessment programmes such as the National Institute for Health and Care Excellence’s Diagnostics Assessment Programme and in regulatory processes. They call for consolidated guidance, developed using consensus methods, covering choice and justification of method, definitions of terminology, and improved reporting of conduct and analysis, potentially including minimum essential design and reporting requirements alongside desirable characteristics, to supplement existing guidance for reporting quantitative diagnostic accuracy studies.

Subject of Research: The use and quality of qualitative and mixed methods research in the evaluation and implementation of diagnostic health technologies

Article Title: Use of qualitative and mixed methods research in diagnostic technology evaluation and implementation: insights from a systematic review

Article References: Roleston, C., Ukwatte, U., Roberts, N., Hayward, G. N., Turner, P. J., & Glogowska, M. (2026). Use of qualitative and mixed methods research in diagnostic technology evaluation and implementation: insights from a systematic review. Diagnostic and Prognostic Research, 10(1), Article 26. https://doi.org/10.1186/s41512-026-00240-z

Image Credits: AI Generated

DOI: 10.1186/s41512-026-00240-z

Keywords: systematic review, diagnostic tests, qualitative research, mixed methods, point of care testing, diagnostic technology evaluation, implementation science, methodological quality, patient involvement, health technology assessment, semi-structured interviews, thematic analysis