Strategy for prioritizing health recommendations in Colombia’s system

In a sweeping audit of Colombia’s evidence-based health guidance, researchers have found that more than half of the country’s prioritized clinical recommendations remain stuck waiting for changes that only the health system itself can deliver — shifts in governance, financing, or the way care is actually organized and delivered. The study, published in Health Research Policy and Systems by a team led by Marcela Vélez of the University of Antioquia in collaboration with St. Michael’s Hospital in Toronto and the Alliance for Health Policy and Systems Research at the World Health Organization, offers one of the most comprehensive attempts yet to answer a deceptively simple question: which evidence-based recommendations are pending implementation in a national health system, and which of them cannot move forward without structural reform?

The research team undertook a systematic document analysis of every national clinical practice guideline developed by the Colombian Ministry of Health — 58 in total, produced between 2008 and 2016 — together with all 14 health systems guidance documents published by the World Health Organization. Searches were conducted between December 2019 and April 2020, with two team members working independently to retrieve the most current versions of each document. From the Colombian guidelines, which collectively contain roughly 3,500 recommendations, the researchers focused on those explicitly flagged by their development groups as key, priority, or tracer recommendations — those judged to carry the greatest potential impact on patient outcomes or on the health system itself. Where no such prioritization existed, the team contacted the guideline development group leaders directly to obtain it. All 363 recommendations from the WHO health systems guidance were included, on the rationale that supranational guidance inherently contains priority recommendations and that Colombia’s government has formally committed to receiving WHO advice.

The unit of analysis throughout was the individual recommendation, and the extraction protocol was deliberately granular. For each recommendation, the team recorded the parent guideline’s characteristics, the recommendation’s full text, its strength as graded by the original development group — whether through letter grades A through D or the GRADE system, which assesses the certainty of the underlying evidence and the likelihood that all informed patients would choose the intervention — and its direction, meaning whether it favored or argued against an intervention. Recommendations graded A, B, or strong were pooled as strong; those graded C, D, or weak were treated as weak; conditional recommendations formed a third category. The team also logged the type of technology or service addressed — medication, diagnostic test, device, procedure, counseling, or education — and the currency of the evidence, classifying recommendations as updated if published or revised within the previous four years, close to outdated in their fifth year, and outdated beyond that. A pilot extraction of 50 recommendations was used to calibrate the team before the full dataset was compiled, with all extraction performed independently in duplicate and discrepancies resolved by consensus or third-party adjudication.

The numbers that emerged are striking. Across the 71 guidelines analyzed, 958 prioritized recommendations were identified: 595 from the Colombian guidelines and 363 from the WHO documents. Of these, only 393 — 41 percent — were current, while 425 (44.4 percent) were outright outdated and another 140 (14.6 percent) were close to becoming so. Among the Colombian priority recommendations specifically, 56.8 percent had aged past the point where their evidence base could be considered reliable, and the proportion was even higher — 62.5 percent — among the WHO guidance documents. In terms of strength, 540 recommendations (56.4 percent) were strong, 189 (19.7 percent) were weak, 173 (18.1 percent) were conditional on specific contexts, and 37 represented good-practice statements grounded in expert consensus. The overwhelming majority — 861 recommendations, or 89.9 percent — called in favor of an intervention, implying that something new must be done or added to clinical practice; only 97 recommended against a technology or service. The therapeutic landscape they covered was broad: medications were the most common target (228 recommendations), followed by therapeutic procedures (155), counseling and education (138), and diagnostic tests or procedures (119). Maternal and child health, chronic noncommunicable diseases, cancer care, and sexual and reproductive health dominated the subject matter.

The heart of the study, however, lies in its second analytical layer: determining which recommendations cannot be implemented without changing the architecture of the health system itself. Two researchers independently assessed every recommendation against the Cochrane Effective Practice and Organisation of Care (EPOC) taxonomy, a comprehensive classification framework originally developed for systematic reviews that distinguishes interventions across four domains — delivery arrangements, financial arrangements, governance arrangements, and implementation strategies. The team deliberately excluded the implementation-strategies domain, which addresses individual and institutional behavior change, and focused on the three system-level dimensions. For medications and devices, the researchers cross-checked two Colombian government databases to determine whether the product was approved by the national regulatory agency INVIMA and whether it was publicly funded. For WHO recommendations, experienced assessors judged whether each had been implemented completely, partially, or not at all in Colombia, based on review of existing national plans and programs. The classification was verified by additional team members and finalized through consensus meetings.

The verdict: 518 of the 958 recommendations — 54.1 percent — require a change in health system arrangements before they can be put into practice. The dominant category was delivery arrangements, implicated in 503 recommendations (52.5 percent). The most frequent specific need was availability of care, cited for 391 recommendations, followed by staff training (209), role expansion and extension of the healthcare workforce (155), and timely access to services (114). The pattern reveals a systemic paradox: many recommended technologies and services are formally approved and publicly funded on paper, yet remain physically inaccessible to large segments of the population. Diagnostic tests may be covered by insurance but offered only in capital cities with robust healthcare infrastructure, or provided by some insurance companies and not others. Seventy recommendations additionally required that care be delivered in a culturally appropriate manner, 56 of them in sexual and reproductive health — a finding heavily driven by WHO guidance on health worker roles in providing safe abortion care and post-abortion contraception.

Governance arrangements — the rules and processes governing authority and accountability — were required for 259 recommendations (27 percent). The most common governance change involved training, licensure, and scope-of-practice policies, accounting for 159 recommendations: for instance, allowing general practitioners to prescribe medications currently restricted to specialists. Another 89 recommendations hinged on completing regulatory registration with INVIMA before the relevant products could be authorized, and 22 required altering who decides what public insurance covers and what must be provided. Financial arrangements were needed for 102 recommendations (10.6 percent), and within this group, 92 involved modifying the list of covered services and products; 86 recommendations required adding an intervention to the statutory benefits plan or clarifying and communicating its public funding status, with medications alone accounting for 58 of these. Notably, nearly two-thirds of the recommendations requiring benefit-plan inclusion had been graded strong, offering policymakers a ready-made shortlist for cost-effectiveness analysis.

The study also surfaced a tier of recommendations that exceed the decision-making authority of the health ministry altogether. Forty-six WHO recommendations — drawn from guidance on safe abortion care, post-abortion contraception, self-care interventions for sexual and reproductive health, and national pharmaceutical pricing policies — would require legislative action, changes in other ministries, or navigation of constraints imposed by Colombia’s bilateral free trade agreement with the United States. Some of the abortion-related recommendations presuppose that abortion is legal in Colombia, a matter for lawmakers rather than health regulators. Equally sobering is the currency problem: among WHO recommendations, 244 of 363 — 67.2 percent — had not been implemented at all in Colombia, and more than half of all prioritized recommendations were past their evidence shelf life. The researchers flag this as an opportunity to redirect resources toward updating the guidelines most in need, noting that guideline development in Colombia mobilized substantial public investment and the research capacity of universities, hospitals, professional societies, and patient organizations.

The work responds to a well-documented global problem in knowledge translation. Prior research has shown that even high awareness and agreement among clinicians does not translate into guideline adherence, and that it can take up to 17 years for merely 14 percent of published research to reach everyday practice. What distinguishes the Colombian approach is its comprehensiveness — previous studies had examined individual guidelines or sampled guidelines across countries, but none had systematically consolidated every prioritized recommendation available for a single national health system and mapped the structural changes each demands. The team’s methodology has acknowledged limitations: the guidelines were not developed with uniform methodology, with some using GRADE and others relying on older grading schemes that measure different things, making harmonization imperfect. The researchers also chose not to appraise the quality of the guidelines themselves, reasoning that their unit of analysis was the recommendation and that guideline quality does not necessarily predict recommendation quality.

To make the findings actionable, the team built a publicly available interactive visualization tool, published in Spanish, that links the datasets of guidelines, recommendations, and extracted characteristics, allowing policymakers and stakeholders to filter recommendations by condition, age group, disease relevance, and required system change. The researchers frame this as the first step in a larger research program on evidence-informed implementation: identifying key recommendations is the prerequisite for prioritizing them and for diagnosing the health system challenges that must be resolved. The Colombian guidelines requiring the most structural changes were those for sexually transmitted infections (35 changes), heart failure (21), and juvenile idiopathic arthritis (18); on the WHO side, the guidance on health worker roles in key maternal services topped the list with 168 required changes. The next phase, the authors say, will explore how the database can support real-world prioritization and will incorporate the preferences of patients and health professionals — building on work such as Lynch and colleagues’ consumer-inclusive prioritization process for Australian stroke guidelines — to determine which of these 518 blocked recommendations should move first.