Grading the quality of evidence and strength of recommendations: the GRADE approach to improving dental clinical guidelines.

Substantial progress has been made since 1979 when the Canadian Task Force on the Periodic Health Examination set about generating ‘statements of evidence’ for ranking the validity of evidence about the effectiveness of preventative measures[14]

And, although their goal was to provide a way of rating the strength and quality of studies and link these to ‘grades of recommendations’, these ratings were based to a large extent on the type of study design with the randomised controlled trial as the highest level. Significantly, these grades of recommendation did not consider uncertainty around the results nor did they incorporate explicit judgements, such as for example the weighting of trade-offs between harms and benefits, and how these might correlate with the strength of recommendations.

Clinical practice guidelines

Clinical guidelines have been defined as, “systematically developed statements designed to help practitioners and patients decide on appropriate healthcare for specific clinical conditions and circumstances”[7].

The process of guideline development should be transparent, reproducible and robust and aim to provide clinicians and other health care decision makers with comprehensive, critical and well balanced information on the benefits and limitations of a range of diagnostic and therapeutic interventions[2,13].

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group was established in 2000 and since that time has sought to develop a “common, sensible and transparent approach to grading quality of evidence and strength of recommendations”[9]. The process of development began with a review of existing grading systems and, to ensure transparency and accountability has involved continuous input from researchers, health professionals and methodologists.

International acceptance of the GRADE system is increasing steadily and now includes a wide range of professional bodies, medical journals and healthcare regulatory authorities. Although clinical dentistry has been a little slower than clinical medicine in moving towards a wider acceptance of GRADE, a number of publishers of dental guidelines and related organizations such as for example NICE, AHRQ and BMJ Clinical Evidence have already adopted and are using the GRADE system. The World Workshop of Oral Medicine Committee recently used GRADE in the development of one of their guidelines for the management of oral submucous fibrosis[12].

One of the reasons suggested for the relative absence of any grading of the quality of evidence and strengths of recommendations in clinical dentistry, was the comparative absence of high quality evidence for the effectiveness of many dental interventions[4]. However this situation is hardly unique to dentistry and is in fact not uncommon in many other disciplines of health care. Undoubtedly the pace of change and general acceptance of the use of evidence in clinical decision-making has been somewhat slower in dentistry than in medicine but perhaps some reassurance can be taken from the fact that “evidence-based medicine is so far down the track it has created a powerful vacuum, pulling evidence-based dentistry right along with it”[3].

The GRADE approach

Step number one of the process involves the formulating of important clinical questions that address the problems and questions that dental clinicians and their patients face in everyday practice. Key elements in the approach are clarity in defining of the population, the treatment strategies including all alternative management strategies, in addition to identifying all of the outcomes which are of importance to patients[11]. Defining how broad or narrow the question should be can be quite challenging e.g. a broader question might compare resorbable plating systems with titanium ones for fixation after mandibular fractures or osteotomies[1] versus a more narrowly focused topic covering fixation of parts of the facial skeleton in elective orthognathic surgery[5].

The relative importance of each individual outcome needs to be defined by the guideline group and in doing so it is essential that not only are clinicians involved but that they also take patient preferences and values into consideration when rating the importance of these outcomes. The GRADE approach then requires that all of these important outcomes are categorised as either; critical for making a recommendation; important but not critical; or not important from the patient’s perspective. So for example self-assessment of oral halitosis is of far more importance as an outcome measure to a patient as compared to a clinician-based assessment with a halimeter or via gas chromatography[6].

The most appropriate way of addressing any clinical question is through a systematic review of the relevant evidence. The guideline group may be able to identify an existing high quality up to date systematic review or if there is a gap in the evidence may suggest that a review should be conducted. The GRADE working group recommends that the resulting systematic review should be summarized in a specific structured format to demonstrate transparently the best estimates of the benefit or adverse consequences of the health interventions being addressed in the clinical question and this should include the extent of our confidence in these estimates[10].

Grading the quality of evidence

The GRADE system recommends five categories that can be used in defining our confidence in the estimate of evidence to make the grading of the evidence more transparent for its users.

Study limitations: design and implementation of available studies which are suggestive of a high likelihood of risk of bias;

Inconsistency of results: unexplained heterogeneity in the results;

Indirectness of evidence: indirect population, intervention, control or outcomes;

Imprecision of results: wide confidence intervals;

Reporting bias: failure to report studies that show no effect (publication bias), outcomes that were harmful or for which no effect was observed (selective outcome reporting).

These factors can be used to guide categorizing of the evidence as ‘High Quality’ i.e. situations in which the current evidence base leads us to the assumption that it is unlikely that further research would change our confidence in the estimate of effect. ‘Moderate Quality’, when we can assume that further research may well change the estimate of effect and our confidence in it. ‘Low Quality’, when we consider that it is very likely that further research would change our estimate of effect or our confidence in it. ‘Very Low Quality’, the current evidence leads us to be very uncertain in any estimate of effect.

Strength of recommendation

Quality of evidence is one of the major factors in defining the strength of clinical recommendations. Guideline developers not only need to consider the quality of evidence but also the balance between the desirable and undesirable effects of the intervention for the patient. In order to further increase transparency, the factors that affect the strength of evidence have been categorized into four groups: the quality of the evidence; any uncertainty about the balance between desirable and undesirable effects; uncertainty or variability in the values and preferences and resource utilisation[10]. So for example, implant dentistry might not represent a feasible intervention in situations where resources are limited, as indeed the use of atraumatic restorative treatment (ART) may be a suitable alternative to the use of amalgam as a restorative material in similar resource-poor settings[8].

There are currently a very limited number of dental clinical guidelines that have explicitly adopted the GRADE approach. We recommend that guideline groups now consider adopting GRADE which will ensure better transparency and accountability in the process and that the end product is not only evidence-based but also takes in to account patient relevant outcomes.

We would like to take this opportunity to encourage our dental colleagues who are in the process of developing clinical guideline to join the GRADE Working Group so that they can share information and participate in further methodological development in this field.

The GRADE Working Group can be accessed through their website www.gradeworkinggroup.org.

The Working Group has published a series of articles which can help clinicians, methodologists and guideline groups to get a better understanding of the GRADE approach, these are available at http://www.gradeworkinggroup.org/publications/index.htm