Suggestions and Commentary on Creating, Interpreting and Using Clinical Practice Guidelines.

How to cite this article: Farhad Kapadia. Suggestions and Commentary on Creating, Interpreting and Using Clinical Practice Guidelines. Indian J Crit Care Med 2020;24(Suppl 1):S2-S5.

INTRODUCTION

The current issue covers a new and comprehensive set of guidelines developed by personnel from ISCCM, for its members and for other clinicians. This accompanying comment is an evidence-free opinion suggesting how a practicing clinician could evaluate and use these guidelines. The clinician should review the strengths and weaknesses of all guidelines by using a simple 4-point pragmatic approach.

Trust but verify section

Obsessively follow pragmatic protocols to enhance safety

Largely follow other guidelines based on high-quality evidence

Resist slavishly following other guidelines based on poor-quality evidence.

TRUST BUT VERIFY

The clinician should countercheck at least two aspects of each and every guideline, regardless of the quoted strength of evidence or the level of recommendation. The first, is the guideline based on a clinical outcome; and, the second, is the guideline supported by data from adequately powered randomized controlled trials (RCTs). The clinician should also verify the process of development of the guidelines.

The clinician should first check whether the recommendation is based on a surrogate or physiological outcome alone or it has demonstrated clinical benefit. Box 1 shows the differences between these two outcomes. Those with demonstrated clinical benefit are more reliable and beneficial to patients. Those with only physiological or surrogate benefit should be seen as conditional recommendations. In conditional, one awaits future data on clinical benefit. If a recommendation is based on a proven physiological benefit but has been demonstrated as not to improve clinical outcomes, the guideline should be assumed to be of questionable value. Example of such guidelines include the use of recruitment maneuvers in ARDS (improve oxygenation) or the use of erythropoietin or colony-stimulating factors (improve hemoglobin and WBC), but these interventions do not have further improvement in the clinical outcomes listed in Box 1. Use of such guidelines should be individualized to the specific clinical situation or patient. A couple of examples, in the deep venous thrombosis (DVT) guidelines, most of the studies focus on detection of DVT by venous Doppler and simply assume that this translates into clinical benefit. In the central venous catheter guidelines, the guidelines concentrate on mechanical complications, infection and thrombosis, and almost none evaluates the impact of these on clinical outcomes.

The second aspect, the clinician should check, is if the level of evidence is strong. The most reliable are adequately powered RCTs with clinical outcomes.

Interpretation of data and trials requires some effort and training. Box 2 gives a suggested approach to evaluation of an RCT. It is a mistake on the part of the reader to assume that strong recommendations and high levels of evidence imply that the recommendation has proven clinical benefit. There are many common errors in interpreting data. Box 3 gives extreme examples of common pitfalls in interpreting data and coming to erroneous recommendations. The common errors are mistaking an association or a correlation for causation, mistaking a hypothesis for a conclusion and basing a conclusion on post-hoc or subgroup analysis of data.

The process of developing guidelines is difficult and generates significant ambiguity. A series of guidelines have been proposed (Box 4). The reader should check if the creation of the guidelines complies with the standards of the Institute of Medicine of the National Academies.[1] These guidelines are demanding and surveys have shown that most guidelines development groups do not meet the required standards.[2]

OBSESSIVELY FOLLOW PRAGMATIC PROTOCOLS TO ENHANCE SAFETY

The primary role of ICUs is to stabilize vulnerable patients while the disease process is reversed. These patients are vulnerable to multiple adverse events, and safety standards to prevent them are paramount. Guidelines have been instrumental in framing these safety standards. The early guidelines by the pediatricians, the medical societies and the anesthetists laid the foundation for the safety standards that now exist, and it is these safety standards that minimize inadvertent and preventable patient harm.[3-5]

In intensive care, guidelines for safe conduct of procedures, for preventions of drug or fluid administration errors, or for infection prevention, etc. should be obsessively followed. It is inappropriate to demand high-quality clinical outcome data for these practices. For example, checking the drugs labels and doses before they are administered, checking that laryngoscopes are working before starting an intubation, and handwashing prior to touching any indwelling device, are obvious and self-explanatory enough to be blindly followed without scientific backing. All ICUs should develop and document their pragmatic safety procedures and the compliance to these requirements should be close to 100%.

Physiological Outcomes

Vital parameters

Hemodynamic parameters

Ventilatory parameters

Laboratory investigations

– Hematology

– Biochemistry

– Microbiology

– Imaging

– Scoring systems

Clinical Outcomes

Lower Mortality (all cause)

– ICU/Hospital

– Time point

Less Severity (in ICU)

– Less invasive intervention

– Less duration

Less distress or discomfort

Better Recovery (after ICU discharge)

– Shorter duration for recovery

– Full return of function

Step 1: Understand the hypothesis

Patients, Intervention, Comparator, Outcome

Time points for outcomes

Statistics used

Step 2: Quick checklist*: Are methods sound?

Step 3: Go straight to the Table & Figures

Is quality of data adequate?*

Step 4: Make your own conclusion

Primary outcomes and secondary outcomes

Magnitude of benefit or risk

Pre-hoc subgroup analysis

Post-hoc subgroup analysis

Step 5: Read rest of the paper

Compare your conclusion with that of the authors

Compare with existing literature

*Checklist for methods and quality of data

Did the study ask a clearly focused question?

Were participants appropriately randomized?

Were all the participants accounted for at its conclusion?

Were patients analyzed in the groups to which they were randomized?

Were participants, staff, and study personnel blinded?

Were the groups similar at the start of the study?

Except for the experimental interventions, were participants in both groups treated in the same way?

Did the study have enough participants?

How precise are the results?

Were all important outcomes considered?

Can the results be applied by you locally?

Example 1

Observations:

Smokers develop polycythemia and emphysema.

Large lung volumes and high hemoglobin benefit athletes.

Erroneous recommendation: Smoking will improve exercise performance.

Example 2

Observations: EPIC 2 database.

In India, infection rates and mortality were ~40% and 18% respectively.

In the UK, infection rates and mortality were ~55% and 27% respectively.

Erroneous recommendation: Transport patients from the UK to India to decrease infections and mortality.

Example 3

Observations:

The mortality in an ICU is 10%.

Mortality in the general indoor beds is 1%.

Erroneous recommendation: Admit high-risk patients to the general beds in preference to the ICU.

The error in the above three examples is that association or correlation is assumed to be cause and effect. Hypothesis generating data are misinterpreted as a conclusion.

Example 4

Observation: Trump won the Electoral College and Clinton won the popular vote.

Erroneous recommendation: Clinton should become the President.

The error in the above example is that a post-hoc analysis is changing the interpretation and conclusion of the election

LARGELY FOLLOW OTHER GUIDELINES BASED ON HIGH-QUALITY EVIDENCE

We are now in an era where many of our practices have been tested in large RCTs with clinical outcomes. Unfortunately, the majority of these trials have been negative.[6] Recommendations that are based on such trials should be widely implemented by practitioners in ICUs. Examples include low tidal volume ventilation in acute respiratory distress syndrome (ARDS) and prone-position ventilation in severe ARDS, restrictive use of packed red blood cells (PRBCs), and mean arterial blood pressure targets of 65 mmHg among many others. Clinicians should be aware of these major trials and following the critical care sections of the large journals is the simplest way of staying up to date. Compliance of recommendations made on the basis of these RCTs should be high, ~ 90-100%, but the clinician should individualize their implementation according to the specifics of the patient. Some individual patients may not resemble the population of patients studied in the original RCT. A quick perusal of the critical care section of the New England Journal of Medicine (NEJM) online[7] or others like the Journal of American Medical Association (JAMA) or Lancet, helps the clinician cover the vast majority of these important clinical trials. Ideally, each guideline should have a table that shows the major RCTs on which their recommendations are based.

Clinical Practice Guidelines We Can Trust: Standards for Developing Trustworthy Clinical Practice Guidelines (CPGs)

Standard 1: Establishing transparency

1.1 The processes by which a CPG is developed and funded should be detailed explicitly and publicly accessible.

Standard 2: Management of conflict of interest (COI)

2.1 Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG.

Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG.

2.2 Disclosure of COIs within GDG: All COIs of each GDG member should be reported and discussed by the prospective development group prior to the onset of their work. Each panel member should explain how their COI could influence the CPG development process or specific recommendations.

2.3 Divestment: Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations.

2.4 Exclusions: Whenever possible, the GDG members should not have COI. In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical specialists who receive a substantial portion of their income from services pertinent to the CPG. Members with COIs should represent not more than a minority of the GDG. The chair or co-chairs should not be a person(s) with COI. Funders should have no role in CPG development.

Standard 3: Guideline development group composition

3.1 The GDG should be multidisciplinary and balanced, comprising a variety of methodological experts and clinicians, and populations expected to be affected by the CPG.

3.2 Patient and public involvement should be facilitated by including (at least at the time of clinical question formulation and draft CPG review) a current or former patient and a patient advocate or patient/consumer organization representative in the GDG.

3.3 Strategies to increase effective participation of patient and consumer representatives, including training in appraisal of evidence, should be adopted by GDGs.

Standard 4: Clinical practice guideline—systematic review intersection

4.1 CPG developers should use systematic reviews that meet standards set by the Institute of Medicine's Committee on Standards for Systematic Reviews of Comparative Effectiveness Research.

4.2 When systematic reviews are conducted, specifically to inform about particular guidelines, the GDG and systematic review team should interact regarding the scope, approach, and output of both processes.

Standard 5: Establishing evidence foundations for and rating strength of recommendations

5.1 For each recommendation, the following should be provided: An explanation of the reasoning underlying the recommendation, including: A clear description of potential benefits and harms. A summary of relevant available evidence (and evidentiary gaps), description of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence. An explanation of the part played by values, opinion, theory, and clinical experience, in deriving the recommendation. A rating of the level of confidence in (regarding certainty) the evidence underpinning the recommendation. A rating of the strength of the recommendation in light of the preceding bullets. A description and explanation of any differences of opinion regarding the recommendation.

Standard 6: Articulation of recommendations

6.1 Recommendations should be articulated in a standardized form detailing precisely what the recommended action is and under what circumstances it should be performed.

6.2 Strong recommendations should be worded so that compliance with the recommendation (s) can be evaluated.

Standard 7: External review

7.1 External reviewers should comprise a full spectrum of relevant stakeholders, including scientific and clinical experts, organizations (e.g., health care, specialty societies), agencies (e.g., federal government), patients, and representatives of the public.

7.2 The authorship of external reviews submitted by individuals and/or organizations should be kept confidential unless the reviewer (s) has waived that protection.

7.3 The GDG should consider all external reviewers’ comments and keep a written record of the rationale for modifying or not modifying a CPG in response to reviewers’ comments.

7.4 A draft of the CPG at the external review stage or immediately following it (i.e., prior to the final draft) should be made available to the general public for comment. Reasonable notice of impending publication should be provided to interested public stakeholders.

Standard 8: Updating

8.1 The CPG publication date, date of pertinent systematic evidence review, and proposed date for future CPG review should be documented in the CPG.

8.2 Literature should be monitored regularly following CPG publication to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.

8.3 CPGs should be updated when new evidence suggests the need for modification of clinically important recommendations. For example, a CPG should be updated if new evidence shows that a recommended intervention causes previously unknown substantial harm; that a new intervention is significantly superior to a previously recommended intervention from an efficacy or harms perspective; or that a recommendation can be applied to new populations.

RESIST SLAVISHLY FOLLOWING OTHER GUIDELINES BASED ON POOR-QUALITY EVIDENCE

This recommendation is probably going to meet with a fair bit of resistance from clinicians and guideline developers and, therefore, warrants some explanation. As explained in the “trust but verify” section, a lot of guidelines are based on potentially erroneous interpretation of data. In the absence of robust data of clinical outcomes from RCTs, guideline developers feel obliged to give some form of recommendation. This may be accompanied by a statement that the recommendation is weak and the level of evidence is poor, but it invariably gets translated into something that must be followed. This leads to two problems. The potential for harm, and pressurizing clinicians to follow practices they do not believe. An unfortunate reality of most guidelines is that they are based on educated and biased guesses of experts based on poor-quality data. Many of these are simply backed by tradition and habit. As the saying goes, “the chains of habit are too small to be noticed, until they become too strong to be broken”. A guideline based on tradition and habit can simply end up reinforcing that practice, independent of its scientific validity. They also lead to clinicians ignoring the lack of scientific validation by citing “the guidelines” to justify their intervention or confirmation biases.

Harm has been demonstrated in the blind use of guidelines in ICU patients[8] and in perioperative patients.[9] It is widely accepted that the opiate crisis in USA was amplified by well-meaning but flawed clinical practice guidelines that overemphasized benefit and downplayed harm. This bias in guidelines has been repeatedly demonstrated earlier too.[2] When using guidelines based on poor evidence or on surrogate endpoints, a compliance approaching 100% could suggest failure of the part of clinician in individualizing care to the specific patient. Such a clinician is at risk of becoming a slave to the guidelines—an unfortunate and unintended consequence of the contemporary culture of guideline-based medicine. We have discussed the limitation of guidelines in more detail elsewhere [10]

CONCLUSION

This opinion piece attempts to navigate the evolving world of guidelines and suggests a four-pronged approach to their use. First, trust but verify by referring to the studies behind the recommendation. Second, obsessively follow safety guidelines. Third, largely follow guidelines based on adequately powered RCTs with clinical outcomes. Lastly, individualize the use of all other guidelines to each specific patient, rather than blindly following them in all patients.