Tracing and Assessing the Evolution of Clinical Guidelines.

See Article DuBose-Briski et al.

For decades, the American Heart Association (AHA) and the American College of Cardiology (ACC) have collaborated to develop practice guidelines that align scientific evidence with recommendations for the care of patients with cardiovascular disease. This work began in the early 1980s when US federal authorities requested that the AHA and ACC jointly develop appropriate practice guidelines for permanent implantation of pacemakers. The first guideline was published in 1984.1 Underlying the effort to create guidelines is a strong commitment to improving the quality of cardiovascular care in the United States. As an example, adherence to hospital American College of Cardiology Foundation/AHA heart failure guidelines resulted in stepwise improvements in in‐hospital mortality from the lowest to the highest quartile of guideline adherence in the CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes with Early Implementation of the ACC/AHA Guidelines) study.2

Individual recommendations in practice guidelines are categorized along 2 independent dimensions: class of recommendation (COR) and level of evidence (LOE). CORs provide a rating of the magnitude of benefit versus risk for any recommendation; they, in effect, indicate the strength of a recommendation. CORs range from Class I to Class III, with Class I indicating a procedure or intervention that is effective for most patients and should be used. Class III is the weakest COR and indicates that a procedure or intervention is unlikely to benefit or may harm most patients. Class II has been divided into 2 levels (Class IIa and IIb) where procedures or interventions are probably recommended and may be appropriate for some patients; IIa indicates moderate benefit while IIb suggests benefit only marginally exceeds risk. The latter (COR IIb) suggests careful consideration of individual patient risk factors and expertise available for implementation of the interventional recommendations. LOEs provide a rating of the quality of the evidence supporting a recommendation. Current LOEs range from Level A to Level C. Level A evidence provides the greatest certainty, with high‐quality supporting data from multiple randomized clinical trials and/or meta‐analyses. Level C evidence provides the least certainty, with supporting data derived from nonrandomized or observational studies or reflecting a consensus of expert opinion. Level B evidence is intermediary.

Critically, over the years, the methods used to formulate and justify guidelines have evolved and the LOE and COR categories have not remained static (Table).3, 4, 5 In 2012, the American College of Cardiology Foundation/AHA convened a summit to update clinical practice guideline methodology.6 As a result, the methodology of the guidelines evolved to align more closely with recommendations from the National Academy of Medicine (formerly the Institute of Medicine). This major effort was executed by 5 working groups (organized around the following themes: developing guidelines deemed trustworthy, initiating a systematic review, finding and assessing individual studies, synthesizing evidence, and reporting systematic reviews). The major outcomes of the methodology summit were that all guideline writing groups include patients, conduct a formal systematic review, articulate a detailed and specific relationship with industry policy, balance representation across race/ethnicity and sex (among others), and expand the review process (ie, expanded beyond subject matter experts only to important stakeholder groups, including governmental representatives). In 2016, the AHA/ACC made efforts to create guidelines that are parsimonious and focus on high‐quality evidence in order to improve the utility of the guideline to the healthcare provider because historically a large proportion of recommendations were based on LOE C and a modest proportion on LOE A.5

Table 1

Comparison of Levels of Evidence and Classes of Recommendation Criteria in Practice Guidelines in 2008 and 2016

Criterion	2008a	2016b
Class of recommendation
	Class I	Class 1 (STRONG)
	Class IIa	Class IIa (MODERATE)
	Class IIb	Class IIb (WEAK)
	Class III	Class III: no benefit (MODERATE)
		Class III: harm (STRONG)
Level of evidence
	Level A	Level A
	Level B	Level B‐R
		Level B‐NR
	Level C	Level C‐LD
		Level C‐EO

EO indicates expert opinion; LD, limited data; NR, nonrandomized; R, randomized.

Criterion from3 Note that in the Methodology Manual and Policies From the ACC/AHA Task Force on Practice Guidelines (2010),4 the Class III class of recommendation was split into “no benefit” and “harm” subcategories.

Criterion from5

Investigators have periodically assessed how changes in the standards of guideline creation are reflected in actual guidelines.7 In this issue of the Journal of the American Heart Association (JAHA), DuBose‐Briski and colleagues8 evaluate the trends in 2‐ to 5‐year time windows (2008–2012 and 2013–2017) in clinical practice guidelines produced by the AHA/ACC in 5 areas: arrhythmias, prevention, acute and stable ischemia, heart failure, valvular heart disease, and vascular medicine. They evaluated changes in the AHA/ACC designated CORs as well as changes in the LOE. They found that between the first and second 5‐year time window, the median number of recommendations decreased from 281 to 247. The pattern of the CORs changed across the periods as well, where the percentage of COR II recommendations increased by 6.5% while that of COR I recommendations dropped by 5% and COR III recommendations dropped by 1.3%. In terms of LOE, there was a comparable increase in “B” level evidence (from 41.0% to 52.8%) and a decrease in “A” level evidence (15.7–14.1%) and “C” level evidence (from 46.9% to 32.5%). DuBose‐Briski and colleagues8 conclude that AHA/ACC guideline recommendations have decreased in number, and that this decrease is because of the removal of recommendations based on lower‐quality evidence.

The DuBose‐Briski et al study has a number of strengths. First, their overall objective is an important one. Strengthening the standards for creating clinical guidelines is a necessary but not sufficient disciplinary objective. These changing standards must be reflected in the evolving guidelines upon which both clinicians and their patients depend. Periodic assessment of the impact of the evolution of the guidelines is necessary. Second, they examined (separately) a wide range of topic areas representing a broad swath of cardiovascular medicine, from prevention to heart failure. Third, the 2 time periods chosen for their comparison likely capture significant evolutionary changes in the methods, policies, and COR and LOE criteria used for guideline creation.

The utility of the analysis by DuBose‐Briski et al analysis was, however, dampened by a number of shortcomings of their study design. For example, where the authors state “guidelines and focused updates published between 2008 and 2017 were separated into two time periods, 2008–2012 and 2013–2017,” guidelines from 2003, 2006, and 2007 were used in the analysis (see DuBose‐Briski et al table 28). The justification for their inclusion is unclear. The authors state that when a “focused” update was identified, the abstractor went back and “merged” the focused update with the prior full guidelines to obtain the counts of recommendations. However, the 2003, 2006, and 2007 guidelines did not meet the “focused” update criterion. It is likely that had these 3 studies been excluded (in alignment with the stated methods), the results would be quite different since the analysis relies on simple counts of recommendations by COR and LOE over the 2 time periods. Since the latter time window was a true 5‐year period, the number of recommendations may appear artificially low compared with the 10‐year time window from the first period.

The authors also chose not to distinguish Class IIa and Class IIb recommendations; instead they grouped both into a single Class II category. They note that IIa procedures or treatments are “reasonable” while those rated IIb “may/might be reasonable.” According to the AAC/AHA guidelines rubric, Class IIa recommendations offer much greater benefit than risk while those rated as Class IIb may offer greater benefit than risk or offer benefits only equal to risk. By either definition, it is clear that Classes IIa and IIb are very different. Given that the IIa‐IIb distinction was instituted before 2008, the decision to combine them in their analysis also seems unwarranted. The authors chose to use a single abstractor to tally recommendations and their COR and LOE designations. Using 2 or more abstractors and adjudicating interindividual differences in the collected data would have been ideal. This seems especially important given that data from focused updates were sometimes used to replace previously published recommendations. The authors acknowledge that “some guidelines differed between the two time periods which limits our ability to make direct comparisons.”8 In fact, the inventory of guideline documents in some topic areas changed profoundly between the 2 time periods. For example, in the area of “prevention,” the earlier time period included secondary prevention, cardiovascular prevention in women, perioperative evaluation, and cardiovascular risk for guidelines for asymptomatic patients. The later time period included high blood pressure, perioperative cardiovascular evaluation, cardiovascular risk, blood cholesterol, overweight/obese, and lifestyle management. These changes represent significant shifts in this area of cardiovascular practice; while the summary data of DuBose‐Briski et al are no doubt correct, they do little to illuminate or assess the impact of these fundamental changes in guidelines.

Despite its shortcomings, the analysis by DuBose‐Briski et al suggests something important about assessments of clinical guidelines: Simple counts and longitudinal comparisons of changes in a large sample of recommendations and their associated CORs and LOEs do not provide a nuanced view of the evolutionary trends in clinical guidelines. Instead, a close reading of the guidelines and a careful dissection of the changes that have occurred over time would be most informative. Given the messiness of the history of guideline changes in any particular clinical domain, this would not be a simple task. But this messiness is precisely why the approach is necessary. This level of understanding of the evolution of guidelines combined with more and better assessments of the effects of changing guidelines on the quality of cardiovascular care will have the greatest impact on ultimate beneficiaries of clinical guidelines—patients.

Disclosures

None.