Adapting clinical guidelines in low-resources countries: a study on the guideline on the management and prevention of type 2 diabetes mellitus in Indonesia.

RATIONALE, AIMS AND
OBJECTIVES: Most of the clinical guidelines in low-resource countries are adaptations from preexisting international guidelines. This adaptation can be problematic when those international guidelines are not based on current evidence or original evidence-based international guidelines are not followed. This study aims to evaluate the quality of an Indonesian type 2 diabetes mellitus guideline adapted from selected international guidelines.
METHODS: The "Consensus on the Management and Prevention of type 2 Diabetes in Indonesia 2011" is a guideline by the Indonesian Society of Endocrinology (Perkeni). Four parent guidelines identified from its list of references were from the International Diabetes Federation (IDF), American Association of Clinical Endocrinologist (AACE), American Diabetes Association (ADA), and one jointly released by ADA and European Association for the Study of Diabetes (EASD). Two reviewers independently assessed its quality using the Appraisal of Guidelines, Research and Evaluation Collaboration (AGREE II) instrument. Six recommendations were compared: (1) screening for diabetes; (2) diagnosis; (3) control of hyperglycemia; (4) target blood glucose; (5) target blood pressure; and (6) treatment of dyslipidemia.
RESULTS: Perkeni's guideline satisfied 55% of the AGREE II items, while its parent guidelines satisfied 59% to 74%. Perkeni's shows low score on "rigor of development" and "applicability" and the lowest score in the "scope and purpose" domain. Differences were found in 4 recommendations: the screening of diabetes, control of hyperglycemia, target blood glucose, and treatment of dyslipidemia. In 3 of 4, Perkeni followed the ADA's recommendation.
CONCLUSION: Derivation of recommendations from parent guidelines and their adaptation to the context of Indonesian health care lacks transparency. When guidelines are either derived from other guidelines or adapted for use in different context, evidence-based practice principles should be followed and adhered to.

INTRODUCTION

A clinical practice guideline is defined as “systematically developed statements to assist practitioner and patient decisions on appropriate health care for specific clinical circumstances.”1 It is seen as a way to translate evidence from research to clinical practice, and its production and utilization are remarkably increased during the past few decades. One of the many benefits of guidelines is to improve the consistency of care. However, guidelines developed by various institutions for similar health problems may result in conflicting recommendations.2

To ensure the quality of the guidelines, transparency on the development process is considered crucial, in particular a rigorous approach to the development is needed, and various skills and experts should be involved.3, 4 For some institutions, especially those in developing countries, the availability of such resources is often limited.5, 6 A recent systematic review on diabetes guidelines in non‐western countries found that 79% of the guidelines were based on recommendations from other national or international guidelines.7 Nevertheless, an adaptation of a guideline produced in one cultural and organizational setting for use in another setting (trans‐contextual adaptation8) needs to ensure that the resulting and final recommendations could still preserve its validity.

The overall aim of adaptation is to take advantage of existing guidelines to enhance the efficient production and use of high‐quality adapted guidelines.

Several approaches to adoption and adaptation of guidelines to local situation have been proposed and endorsed, such as the ADAPTE collaboration9 and the “Systematic Guidelines Review method.”10 Basically, the approaches should involve systematic search and selection of guidelines, a quality assessment of the guidelines, and a transparent approach during recommendation formulation, plus an external peer review and a formal endorsement procedure. While this approach involves relatively complex processes and certain expertise, these are scarce sources in low‐resources countries.

In Indonesia, the adoption and adaptation of international guidelines has also been chosen as a pragmatic and practical approach to guideline development. Currently the number of clinical practice guidelines in Indonesia is less than 20. Although no data are available, observation by the author revealed that all of the guidelines were developed using that approach. One such clinical guideline is the so‐called Consensus on the Management and Prevention of type 2 Diabetes in Indonesia of the Indonesian Society of Endocrinology (Perkeni guideline). The guideline was first released in 1993 and has been updated 5 times in the last 10 years.11 Using the Indonesian type 2 diabetes mellitus guideline as a case study, this study aims to analyze a guideline of a national body from a low‐resource country to assess if the guideline has been developed appropriately and has recommended appropriate conclusions.

METHODS

Retrieval of guidelines

For this case study, we used the fifth edition of the Indonesian type 2 diabetes mellitus guideline by the Indonesian Society of Endocrinology (published in 2011). Recommendations adapted from the parent guidelines were included in the Perkeni guideline based on a consensus from the members of the Indonesian Society of Endocrinology.

Four guidelines were listed as the parent guidelines: the Global Guideline for type 2 Diabetes by the International Diabetes Federation 2005,12 the Medical Guidelines for Clinical Practice for Developing a Diabetes Mellitus Comprehensive Care Plan 2007 by the American Association of Clinical Endocrinologist,13 the Medical Management of Hyperglycemia in Type 2 Diabetes: A Consensus Algorithm for the Initiation and Adjustment of Therapy of the American Diabetes Association and the European Association for the Study of Diabetes,14 and the Standards of Medical Care in Diabetes—2010 by the American Diabetes Association.15 We retrieved the original full version of the parent guidelines from the website of the issuing institute or society.

Quality appraisal of the guidelines

Two reviewers (I.S. and G.W.) who were not involved in the development of any of the guidelines independently assessed the quality of the guidelines using the modified version of the instrument developed by the Appraisal of Guidelines, Research and Evaluation Collaboration (AGREE II).4, 16 The AGREE II instrument contains 23 key items organized in 6 methodological domains: scope and purpose (items 1‐3), stakeholder involvement (items 4‐6), rigor of development (items 7‐14), clarity of recommendations (items 15‐18), applicability (items 19‐21), and editorial independence (items 22‐23). The AGREE instrument is sensitive for differences in important aspects of guidelines, can be used consistently and easily by a wide range of professionals from different backgrounds, and has acceptable reliability for most domains. The instrument uses a 7‐point response scale (strongly agree [7] to strongly disagree [1] for each item). The assessors then compared their individual scores for each item and came to consensus on discrepant scores (defined as scores varying by 3 points or more on the 7‐point AGREE II scale). If the 2 assessors were unable to reach consensus, opinion from a third person (GvdH) was sought and opted as the final decision. If the 2 assessors' scores differed by 2 points, they were averaged; if they differed by 1 point, the lower score was kept. Standardized domain scores (expressed on a scale of 0‐100) were calculated using the approach of AGREE II ([obtained score−minimum possible score] divided by [maximum possible score−minimum possible score]).17

Interview with the person responsible for the development of the current Perkeni guideline was conducted by the principal investigator (I.S.) to obtain more insight into the guideline development process.

Comparison of adopted guideline with its parental guidelines

We identified 6 major clinically relevant recommendations from the Perkeni guideline: (1) the screening of diabetes; (2) the diagnosis of type 2 diabetes mellitus; (3) the control of hyperglycemia; (4) the target blood glucose; (5) the target blood pressure; (6) the treatment of dyslipidemia. I.S. and G.W. extracted the major clinically relevant recommendation from the guidelines.

I.S. and G.W. compared the similarity of each recommendation statement with the 4 parent guidelines. For each major recommendation in Perkeni, we also assessed which parent guidelines was followed (ie, which recommendation has been adopted). In addition, we identified and checked citations of the original research used as the source for each recommendation in the parental guidelines. We identified the highest quality of study design among the references as the representative level of evidence for each recommendation.

RESULTS

The guidelines

Three of the parent guidelines (IDF, AACE, and ADA) made general recommendations on the medical treatment and early identification of complications and comorbidities of diabetes. Meanwhile, the joint consensus of the ADA and EASD focused on the pharmacologic intervention for hyperglycemia (see Table 1).

Table 1

Characteristics of the parent guidelines of the Indonesian type 2 diabetes mellitus guideline

Title	Publisher	Country, Language	Publication Date	Guidelines Scope	Basis for the Recommendation
Global guidelines for type 2 diabetes mellitus12	IDF	International, English	2005	Diagnosis (of diabetes and complications), Therapy (of hyperglycemia, complications, and comorbid)	RCTs and other primary studies; systematic reviews or other reviews; guidelines
Medical guidelines for clinical practice for developing a type 2 diabetes mellitus comprehensive care plan13	AACE	United States, English	May 2007	Diagnosis (of diabetes and complications), Therapy (of hyperglycemia, complications and comorbid)	RCTs and other primary studies; systematic reviews or other reviews; guidelines; expert opinion/consensus
Medical management of hyperglycemia in type 2 diabetes mellitus14	ADA and EASD	United States and European Union, English	January 2009	Therapy (of hyperglycemia; focus on pharmacotherapy)	RCTs and other primary studies; systematic reviews or other reviews; guidelines, clinical judgment
Standards of medical care in diabetes15	ADA	United States, English	January 2010	Diagnosis (of diabetes and complications), Therapy (of hyperglycemia, complications and comorbid)	RCTs and other primary studies; systematic reviews or other reviews; guidelines; clinical judgment

Abbreviations: AACE, American Association of Clinical Endocrinologist; ADA, American Diabetes Association; EASD, European Association for the Study of Diabetes; IDF, International Diabetes Federation; RCT, randomized clinical trial.

The recommendations of the ADA, AACE, and ADA‐EASD guidelines are based on a combination of expert opinion and literature reviews. IDF guideline is the only guideline that had no explicit reference to expert opinions/consensus or clinical judgment.

The IDF, ADA‐EASD, and Perkeni guidelines did not provide their method for assessment and rating of evidence. The Perkeni, IDF, and ADA‐EASD guidelines did not grade the recommendations. ADA classified their grade of recommendations into 5 groups with “A” being the highest grade, which incorporates clear evidence from well‐conducted RCTs or meta‐analysis with quality ratings, and “E” being the lowest as it is based on expert consensus or clinical experience. On the other hand, AACE have 4 categories of recommendations where grade “A” recommendation is the one supported by homogenous evidence from ≥1 RCTs or meta‐analysis with quality ratings and grade “D” when no conclusive studies are available to support the recommendation.

Quality of the guidelines

Table 2 shows the overall wide variation in the fulfillment of the AGREE II items for all of the guidelines. Still, all guidelines attained scores higher than 80% in the “clarity of presentation” domain. Yet in all other domains, the scores varied considerably, and in the “rigor of development” domain, only IDF guideline obtained a score higher than 50% while in the “applicability” domain all guidelines obtained scores lower than 40%. Compared with the other guidelines, Perkeni guideline has the lowest quality in all AGREE II domains except for “scope and purpose.”

Table 2

Score achievement (%) of the type 2 diabetes mellitus guidelines based on Appraisal of Guidelines, Research and Evaluation Collaboration (AGREE) II items

Domain	IDF	AACE	EASD‐ADA	ADA	Perkeni
Scope and purpose	47	67	56	39	72
Stakeholder involvement	53	64	53	50	33
Rigour of development	57	42	28	46	0
Clarity of presentation	97	89	100	97	92
Applicability	38	0	10	13	17
Editorial independence	88	25	25	33	0

Abbreviations: AACE, American Association of Clinical Endocrinologist; ADA, American Diabetes Association; EASD, European Association for the Study of Diabetes; IDF, International Diabetes Federation; RCT, randomized clinical trial.

Data presented are AGREE II scores (0‐100; low scores reflect poor quality). Each item was rated on a 7‐point Likert scale that measured the extent to which an item was fulfilled: strongly disagree (1) to strongly agree (7). Scores were standardized within domains by dividing the difference between the consensus score and the minimum possible score by the difference between the maximum and minimum possible scores.

During the interview, the process of developing the guideline by Perkeni was usually started with 1 small team consisted of 1 or 2 experts supported by 1 or 2 technical team members developing the first draft. Rigorous and systematic searching on the identification of the source guidelines was lacking, while the appraisal of the quality of the original research that was used as the source for each recommendation in the parental guidelines. The draft of the guideline was then presented several times in society meetings to gain consensus. Meanwhile, the guideline development team searched for evidence to support the agreed recommendations.

Comparison of the recommendations with PERKENI guideline

Table S3 shows the source of the Indonesian guideline recommendations and the highest level of the study design used to build the recommendation in each parent guideline. Recommendations for the management of type 2 diabetes mellitus were rather similar across the parent guidelines, but the detail varied.

Perkeni and all parent guidelines have similar criteria to diagnose diabetes (based on the presence of classic diabetes symptoms and blood glucose or HbA1c measurement) and recommend that blood pressure should be lowered below of 130/80 mm Hg.

Differences between the parent guidelines were found in 4 areas: screening for diabetes, control of hyperglycemia, blood glucose target, and dyslipidemia management.

Screening for diabetes

All parent guidelines agreed that the screening of asymptomatic patients for diabetes should be targeted to high‐risk adults. Differences existed in defining those at risk especially in terms of age group and nutrition status. In the AACE guideline, an individual aged above 30 years should be screened for any diabetes' risk factors. According to the ADA guideline, age older than 45 years and nutritional status are preconditions for screening while the IDF provides no information. Perkeni adopted ADA screening recommendation for individuals who are overweight/obese or aged older than 45 years.

Control of hyperglycemia

ADA, AACE, and ADA‐EASD guidelines recommend initiation of pharmacologic intervention (metformin) for the control of hyperglycemia simultaneously with lifestyle modification. Perkeni adopted a recommendation from IDF, which recommends pharmacologic intervention for control of hyperglycemia when target blood glucose is not achieved. However, the IDF recommendation did not mention a period for this target.

Blood glucose target

Perkeni recommends a somewhat lenient blood glucose target of HbA1c < 7 compared to HbA1c < 6.5%. This is an adoption of the ADA and ADA‐EASD recommendations.

Dyslipidemia management

All guidelines recommend statin as a preferable treatment, but for different specific indications. IDF and ADA guidelines recommend statin prescription based on age group (above 40 years) and the presence of CVD or CVD risk factors, regardless of baseline lipid levels. The AACE recommends taking baseline lipid levels and prescribing statins when needed to achieve certain target lipid levels. Perkeni recommendations were adopted from IDF and ADA guidelines.

Comparison of linked citations

Citations of similar recommendations

The recommendation on the diagnostic criteria in ADA guideline was derived from their own expert committee report from 1997. IDF cited the 2003 version of ADA expert committee report and a WHO report in 1999, while AACE made their recommendation based on a 2006 joint report of WHO and IDF.

The recommendation on the blood pressure target was made based on guidelines from various institutions. The citation that was cited by 3 parent guidelines (AACE, ADA, and IDF) was the seventh report of the Joint National Committee on the Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7).18 Both ADA and IDF also cited similar trials by Hanson et al (Hypertension Optimal Treatment study)19 and UK Prospective Diabetes Study (UKPDS) 38.20

Citations of different recommendations

The recommendation from ADA and AACE on the importance of screening of high‐risk individuals for diabetes was made based on their own independent literature review. Only IDF cited primary studies such as the UKPDS21 and a population study by Harris et al,22 in addition to WHO consultation report.23 However, no further references could be traced from the 3 guidelines on the risk factors which warrant screening.

Three guidelines (AACE, ADA, and ADA‐EASD) were in agreement about the use of pharmacologic treatment simultaneously with lifestyle modification on newly diagnosed diabetic patients. In this case, ADA merely cited the latest and previous ADA‐EASD recommendation while AACE based their recommendation on a trial by Esposito et al24 and the Diabetes Control and Complications Trial (DCCT), which was conducted on type 1 diabetes mellitus.25 ADA‐EASD agrees that metformin therapy should be initiated concurrently with lifestyle intervention at diagnosis based on the clinical judgment that for most individuals with type 2 diabetes mellitus; lifestyle interventions fail to achieve or maintain the metabolic goals either because of failure to lose weight, weight regain, progressive disease, or a combination of factors. AACE also included the report from Diabetes Prevention Program Research Group,26 which described superior effectiveness of lifestyle to metformin. Despite all the clinical trials results, AACE recommendation was in the end made based on clinical judgment. IDF recommendation that pharmacologic intervention should be given when target blood glucose is not achieved by lifestyle modification was adopted from several other guidelines27, 28, 29, 30 and the UKPDS trial.31

ADA recommendation on blood glucose target of HbA1c < 7 was made based on several trials including the ACCORD trial, which demonstrated no benefit of intensive glycemic control on CVD outcomes.32 The source of recommendation on blood glucose target from the latest ADA‐EASD was the 2008 version of the ADA guideline.33 Between the 2 guidelines (AACE and IDF), which have recommended both lower HbA1c target (<6.5), the only common source being used is the prospective observational UKPDS 35 study.34 While AACE cited several other trials and observational studies in their 2006 Consensus Conference Report35 for this recommendation, IDF cited systematic review of prospective observational studies by Laakso et al36 and Selvin et al,37 together with several guidelines including the 1999 IDF guideline.38 Both AACE and IDF did not include the ACCORD trial while the study was published after both guidelines have been released.

Several trials were cited by 3 parent guidelines (AACE, ADA, and IDF) to recommend that statins are the pharmacologic treatment of choice for lipid management of diabetic patients; the commonly cited reference were the Heart Protection Study (HPS)39 and the Collaborative Atorvastatin Diabetes Study (CARDS).40 Several other guidelines were also cited by the AACE and IDF on their recommendation on statins.

DISCUSSION

The quality of the Indonesian diabetes guidelines is poor based on its low AGREE II score, especially in 4 areas: stakeholder involvement, rigour of development, applicability, and editorial independence. The reporting on the approach to its development lacked transparency. This in particular pertained to the derivation of the principal recommendations from existing guidelines and their adaptation to the context of the Indonesian health care. The interview with the Perkeni guideline developers confirmed important shortcomings in the approach. Hence, we report the rigor of the development as poor.

Discrepancies were found in 4 clinical recommendations: the screening of diabetes, the control of hyperglycemia, the target blood glucose, and treatment of dyslipidemia. Most Perkeni guideline recommendations were derived from the ADA, which was the latest guideline published among the 4 parent guidelines although the AGREE II score was not the highest. Hence, adherence to evidence‐based practice principles during its development can be questioned.

Our finding on the low AGREE II scores in each parent guideline, especially in the “Rigor of development” domain, means that generally those parent guidelines failed to show that they have conducted a systematic review on the best available evidences.41 A previous systematic review that assessed the quality of 24 CPGs on diabetes management reported similar findings.42 Another study that examines the quality of CPGs that included recommendations on pharmacotherapy for glycemic control in type 2 diabetes mellitus indicated several guidelines that achieve higher score on the “rigor of development area” such as those developed by the UK National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, and the American College of Physicians.42 Careful appraisal and selection of the source guidelines is clearly paramount before adapting recommendations from 1 guideline to another.43

Several studies revealed that there are considerable variations and even conflicting recommendations concerning type 2 diabetes mellitus management from different guidelines.44, 45 Variation was believed to be due to insufficient evidence, differing interpretations of evidence, unsystematic guideline development methods, the influence of professional bodies, cultural factors such as differing expectations of apparent risks and benefits, socioeconomic factors, or the characteristics of the health care systems.46 Our study revealed that even though all the source guidelines cited the same studies, yet they can come up with different recommendations. There is a higher chance that the (clinical) judgment of the guideline developer plays a dominant role in the final recommendations.

As expected, each of the guideline used different sources. In the era where evidence‐based clinical practice guidelines are reinforced, systematic searching of the evidence is considered a vital process in the guideline development. While sources included for recommendations in Perkeni have been taken from parent guidelines, transparency on and justification of their appropriateness is lacking. This was also a finding of Aarts et al in their study on Obstructive Sleep Apnea‐Hypopnea Syndrome guidelines.47

Although a wide range of diabetes guidelines existed, the most cited are guidelines from ADA, IDF, EASD, and AACE.7 This might explain the use of these 4 guidelines by Perkeni. In both ADAPTE collaboration9 and the “Systematic Guidelines Review method,”10 the systematic search and selection of the guidelines, the quality assessment of the guidelines, and the transparent approach on the formulation of the recommendation are considered crucial steps in the guidelines adaptation process. However, we found no statement in the guideline that shows that this approach has been followed by Perkeni. This was confirmed during the interview with the Perkeni guideline developers.

Engaging potential end users in the process of evaluating and adapting existing guidelines may help improve the uptake and utilization of the guideline.48 This process has also been overlooked in the Indonesian type 2 diabetes mellitus guideline development; hence, we found in our previous study that the adherence to the recommendations on the Indonesian type 2 diabetes mellitus guidelines is very low.49

As far as we know, this is a first study that examines how the recommendations from different guidelines were being adapted to develop a local diabetes guideline. Previous studies compared the quality and recommendations from different diabetes guidelines from different countries or different institutions.42, 44, 45, 50, 51, 52 While our findings only concern the Perkeni diabetes guideline, they may hold true for other guidelines developed under similar conditions.

In this study, we minimized the observer bias during the assessment of the guideline quality through independent extraction and quality assessment by 2 researchers. While findings reported are mainly based on the literal or statements from the guidelines, we only interviewed the developers of the Perkeni guideline.

Implementing evidence‐based practice principles in guideline adaptation will help the efforts in low‐resource countries to improve their quality care practice through the use of high‐quality practice guidelines. In addition, these countries should aim to improve their capacity in assessing and selecting the guidelines as part of the adaptation process. In the future, the guideline could gain strength and quality by improving transparency in the process of guideline adaptation and by selecting guidelines that fulfill the AGREE II criteria at a high level to be adapted.

CONCLUSION

In view of the potential impact of CPGs on health care delivery and patient outcomes, it is crucial that clinical guidelines should be of optimal quality. The process underlying the Indonesian type 2 diabetes mellitus guideline development is curtailed because of being under‐resourced, and the use of the cited suboptimal source guidelines might risk the validity of the recommendations it contains. Implementation of evidence‐based practice principles such as those proposed by ADAPTE collaboration should be adhered to when guideline is derived from other guidelines to be used in other than its original context or circumstances.

Supporting info item

Click here for additional data file.