Clinical practice guidelines for hypertension in China: a systematic review of the methodological quality.

OBJECTIVE: Clinical practice guidelines (CPGs) provide clinicians with specific recommendations for practice, but due to the increasing number of CPGs developed by diverse organisations over the past few years, there are concerns about the quality of some CPGs. This paper proposes a systematic review of the methodological quality of the CPGs for hypertension that were developed in China.
DESIGN: A systematic review of CPGs for the management of hypertension in adult patients in China. DATA RESOURCES: Chinese electronic databases, Chinese guideline websites and Google Scholar were searched, and the reference lists of relevant publications were also screened for additional information. CPGs for the management of hypertension in adult patients were identified. The main characteristics of the CPGs were extracted, and the scaled Appraisal of Guidelines, REsearch and Evaluation II (AGREE II) domain percentages were independently evaluated by two reviewers.
RESULTS: A total of 17 CPGs, with publication dates ranging from 2001 to 2011, were identified. There was considerable variation in the quality of the CPGs across the AGREE II domains. Overall, the domains of 'rigor of development' and 'editorial independence' were poorly addressed, with an average score of 18% and 16%, respectively. Also less well addressed were the 'stakeholder involvement' and 'applicability' domains, for which the average domain scores were 28% and 20%, respectively. The CPGs performance was less problematic in the domains of 'scope and purpose' and 'clarity and presentation', with a median of 41% for both. After considering the domain scores, 8 CPGs could be recommended with modification for use.
CONCLUSIONS: There is considerable room for improvement of the methodological quality of CPGs for hypertension in China. Greater efforts should to be devoted to ensure the explicit and transparent reporting of potential conflicts of interest of stakeholders, and to consider the quality of the evidence and grade recommendations in the CPG development process. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This is the first attempt to evaluate the methodological quality of clinical practice guidelines (CPGs) for the management of hypertension in China using the Appraisal of Guidelines, REsearch and Evaluation II (AGREE II) instrument.

The results highlight the strengths and weaknesses of the existing CPGs, providing suggestions on how improvements can be made in the future.

The main limitation of this study is that the methodological quality assessment was based on the information about the assessment items the individual CPGs reported, which may not actually reflect the construction process.

Introduction

Hypertension is an important public health challenge, and is the leading cause of death and disability in China. It has been estimated that one in six Chinese adults have hypertension but that only one-quarter are aware of their condition.1 Hypertension is the most powerful risk factor for cardiovascular disease.2 In addition, increased blood pressure is the leading preventable risk factor for premature mortality in the general population in China.3 Despite increased rates of blood pressure-lowering treatment, only a few patients have their hypertension effectively controlled.4 The economic burden of hypertension is heavy due to both direct healthcare costs and indirect costs from the cardiocerebral vascular complications.

Clinical practice guidelines (CPGs) for hypertension have been developed to help optimise the management of the condition and are thought to be capable of improving the quality, appropriateness and cost-effectiveness of patient care.5 The intention of CPGs is to reduce the gap between research and practice, and to provide professionals with recommendations based on the best currently available evidence on how to manage health conditions.6 7 The use of CPGs is recognised as an important component of hypertension management, including support for the use of interventions that are of proven benefit and enhanced awareness of ineffective methods. However, a large number of practice guidelines have been produced by numerous organisations in China over the past 10 years, and the increasing interest in the development of CGPs for hypertension has been accompanied by growing concerns about the variations in the recommendations and qualities among the guidelines.8–10 The methodological quality is of great importance, and it is thus critical to assess CPGs for hypertension using a well-defined process. A high methodological quality CPG development process is more likely to yield a CPG that contains relevant and appropriate recommendations.11

Several guideline appraisal tools have been developed to assess the methodological quality of guidelines.12 The Appraisal of Guidelines for REsearch & Evaluation (AGREE) Instrument, developed by an international group of researchers from 13 countries, has been validated and accepted to address the issue of variability and to appraise the methodological quality of CPGs.13–15 The original AGREE Instrument, which was released in 2003, has been refined to improve the usability and methodological properties, which resulted in the AGREE II reported in 2009.16 The AGREE II instrument is valid and reliable, with a 23-item tool comprising six quality domains, followed by 2 final overall assessment items that require the appraiser to make overall judgments of the CPGs and to reflect on the development process. The aim of the present systematic review was to evaluate the methodological quality of CPGs for hypertension, developed in China, using the AGREE II instrument.

Methods

Search strategy

According to a prespecified search strategy, relevant guidelines were identified through searches of the CBM (Chinese Biomedical Literature Database), Wanfang Database, CNKI (Chinese National Knowledge Infrastructure) and CGC (China Guideline Clearinghouse) websites up to March 2014. Reference lists of all relevant guidelines were manually scanned and, in addition, Google Scholar was searched for additional information.

We employed a combined search of MeSH terms and free-text words, and the following search terms, “hypertension” or “high blood pressure” and “guideline” or “recommendation” or “consensus”, were used. Two investigators independently screened the titles and abstracts of the literature for potentially relevant guidelines.

Inclusion and exclusion criteria

All CPGs developed in China about the management of hypertension in adult patients were eligible for inclusion in the present study. Single-author overviews, secondary or multiple publications, editorials, translations and short summaries, were specifically excluded. The latest version of the guideline was identified for assessment and, if several publication forms of one guideline existed, only the form that included the greatest detail on the methodology used for the guideline development was assessed.

Quality assessment

The methodological quality of existing CPGs for hypertension was evaluated using the AGREE II instrument, which consists of 23 items organised within six domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability and editorial independence.

When using this instrument, each item is ranked on a 7-point scale ranging from 1 (strongly disagree) to 7 (strongly agree). A score of 1 is given when there is no information on that item or if it is poorly reported. A score of 7 is given if the quality of reporting is excellent and when full criteria have been met.

The score for each domain is obtained by adding together all of the scores of the individual items in a domain and then standardising them as follows:

Although using a single quality score is not encouraged by the AGREE II, an overall assessment is included. The final component of the AGREE II instrument involves a recommendation regarding the use of the guidelines in practice as “Yes, to recommend this guideline (Y)”; “Yes, to recommend but with modification (Y, but)” and “No, not to recommend (N)”. For each guideline that was given a recommendation of (Y), the overall domain scores were ≥60% for all six domains. For guidelines that were given a recommendation of (N), the overall domain scores were <30% for at least three domains. For the guidelines that were given a recommendation of (Y, but), the overall domain scores were ≥30% for at least three domains, while at least one domain had a score of <60%.17 18

Two investigators with a full understanding of the AGREE II manual independently assessed the identified CPGs. Prior to evaluating the guidelines included in the review, each investigator rated a superseded guideline, and then the rates were compared among reviewers, discrepancies were discussed, and a consensus was reached about the interpretation of each question.19 The κ score was adopted as a measure of the agreement between the two appraisers’ ordinal item assessment (strongly agree/agree vs strongly disagree/disagree). Although there are no absolute cut-offs for interpreting κ coefficients, a κ>0.8 is generally considered to be acceptable.20 21

Results

Search and description of studies

A total of 7845 articles were identified, 101 of which were considered to be potentially relevant and 17 of which proved eligible for final evaluation, as summarised in figure 1. The general characteristics of the assessed guidelines are shown in table 1.22–38 The publication dates ranged from 2001 to 2011. The number of references cited ranged from 0 to 218, with five guidelines not reporting any references,26 31 34 35 38 four of which were developed on the Chinese mainland.31 34 35 38 The majority of the reviewed CPGs (13 guidelines) were published in Chinese, while four guidelines were published in English,26–28 37 two of which were developed in Hong Kong26 37 and two in Taiwan.27 28 A total of four CPGs23 29 33 36 stated that they received drug company sponsorship, but only one of these guidelines declared that the views of the funding bodies did not influence the recommendations, and a declaration of the conflicts of interest of the guideline developers was not provided for 13 CPGs.

Figure 1

Flow diagram of guideline selection (CBM, Chinese Biomedical Literature Database; CGC, China Guideline Clearinghouse; CNKI, Chinese National Knowledge Infrastructure).

Table 1

General characteristics of the included guidelines

Serial number	Guidelines	Publish date	Number of pages	Number of authors	Number of references	Subject	Region	Organisation
1	Diuretics for hypertension: Chinese experts’ consensus22	2011	9	46	65	Drug intervention	Chinese mainland	The Subspecialty Group of Hypertension, Society of Cardiology, Chinese Medical Association
2	2010 Chinese guidelines for the management of hypertension23	2011	38	43	218	Comprehensive management	Chinese mainland	Writing Group of 2010 Chinese Guidelines for the Management of Hypertension
3	Expert consensus on the clinical application of levamlodipine besylate24	2010	3	4	19	Drug intervention	Chinese mainland	Cardiovascular Medicine Branch of China Medical Doctor Association; Cardio-cerebrovascular Disease Branch of Chinese Geriatrics Society
4	Expert consensus on diagnostic and treatment of Chinese emergency hypertension25	2010	12	U	35	Emergency hypertension	Chinese mainland	Emergency Physicians Branch of Chinese Medical Doctor Association
5	Hong Kong reference framework for hypertension care for adults in primary care settings26	2010	40	82	0	Comprehensive management	Hong Kong, China	Task Force on Conceptual Model and Preventive Protocols Working Group on Primary Care; Food and Health Bureau
6	2010 Guidelines of the Taiwan Society of Cardiology for the management of hypertension27	2010	34	12	208	Comprehensive management	Taiwan, China	Hypertension Committee of the Taiwan Society of Cardiology
7	Guidelines for the management of hypertension. Taiwan stroke association28	2010	8	15	U	Comprehensive management	Taiwan, China	Taiwan Stroke Association
8	β-Blocker in the management of cardiovascular diseases: an expert announcement29	2009	23	18	70	Drug intervention	Chinese mainland	Chinese Society of Cardiology of Chinese Medical Association; Editorial Board of Chinese Journal of Cardiology
9	Expert consensus on the β-blocker for the treatment of hypertension30	2008	4	23	70	Drug intervention	Chinese mainland	The Chinese Medical Doctor Association, Evidence-Based Medicine Professional Committee
10	Uyghur diagnosis and treatment guidelines for hypertension31	2008	1	5	0	Ethno medicine	Chinese mainland	Uyghur Medical Hospital
11	Traditional Chinese medicine for hypertension (first draft)32	2008	3	1	33	Traditional Chinese medicine	Chinese mainland	Cardiology Branch of China Association of Chinese Traditional Medicine
12	Expert consensus on hypertension in the elderly in China33	2008	9	77	52	Elderly people	Chinese mainland	Chinese Elderly Hypertension Treatment Consensus Committee
13	Expert consensus on long-term two hydrogen pyridine calcium channel blockers for chronic renal hypertension34	2008	1	U	0	Drug intervention	Chinese mainland	Kidney Diseases Branch of the Chinese Medical Association
14	Expert consensus on the clinical application of the new fixed-dose combinations of ARB/HCTZ35	2007	3	18	0	Drug intervention	Chinese mainland	“The new fixed-dose combinations of ARB/HCTZ” Expert Group
15	Angiotensin converting enzyme inhibitor in the management of cardiovascular diseases: a Chinese expert announcement36	2007	10	42	65	Drug intervention	Chinese mainland	Chinese Society of Cardiology of Chinese Medical Association; Editorial Board of Chinese Journal of Cardiology
16	Hypertension—a guide to clinical practice37	2004	20	13	20	Comprehensive management	Hong Kong, China	Hong Kong Medical Association
17	Hypertension family medication guideline38	2001	2	1	0	Home medication	Chinese mainland	U

ARB, angiotensin II receptor blockers; HCTZ, hydrochlorothiazide; U, unclear.

Six of the 17 CPGs assessed covered the management of hypertension in general,23 25–28 37 with 1 focused on prevention28 and 1 focused on emergency hypertension.25 The others were specific to the pharmaceutical treatment of hypertension, one of which concerned Traditional Chinese medicine,32 one ethnomedicine,31 one the elderly,33 and one, home medication.38

The majority of CPGs did not describe the level of evidence they cited or grade the recommendations. Two guidelines developed on the Chinese mainland coded the level of evidence (I, IIa/IIb, III) on the basis of the study design, without linking these codes to the recommendations (A, B, C).29 36 Only one guideline developed in Hong Kong classified the level of evidence and explicitly linked the evidence to the recommendations using the Scottish Intercollegiate Guidelines Network (SIGN) classification.26

Guideline quality scores

The scaled AGREE II domain percentages for all 17 CPGs are presented in table 2. Concerning the ‘scope and purpose’ and ‘clarity of presentation’ of the guidelines, the mean score for both was 41%, with a range of 6–78% and 6–67%, respectively. These two domains were the most adequately addressed, with scores above 60% in 4 of the 17 clinical guidelines.

Table 2

AGREE scores of the included guidelines

	Score (%)						Overall assessment
Serial number of guidelines	Scope and purpose	Stakeholder involvement	Rigour of development	Clarity of presentation	Applicability	Editorial independence
1	14	25	20	44	6	13	N
2	50	56	25	64	69	46	Y, but
3	19	22	15	42	10	17	N
4	67	22	19	45	21	13	Y, but
5	61	67	35	42	42	13	Y, but
6	28	39	36	36	17	17	N
7	58	36	28	67	17	33	Y, but
8	50	14	34	64	25	29	Y, but
9	39	36	11	61	17	13	N
10	6	3	1	6	0	0	N
11	11	11	7	8	4	0	N
12	44	28	11	28	38	21	N
13	33	8	7	22	15	0	N
14	64	19	5	42	17	0	N
15	78	36	30	56	17	17	Y, but
16	58	47	27	56	27	33	Y, but
17	11	3	2	17	4	0	N
Total	41±23	28±18	18±12	41±19	20±17	16±14	Y, but (8)
							N (9)

Serial number of guidelines are shown in table 1.

AGREE, Appraisal of Guidelines, REsearch and Evaluation; Y, but, yes, to recommend but with modification; N, no, not to recommend.

Less well addressed was the ‘stakeholder involvement’ domain, for which the average domain score was 28%, with the extremes being 3–67%, and only 1 of the 17 guidelines scored above 60%. CPGs performed similarly in the domain of ‘applicability’, with one scoring above 60%, while the average score was 20% and varied from 0% to 69%, including one CPG that scored 0%.

The last two domains were the least well addressed. On average, the score for the ‘rigor of development’ domain was 18%, with scores ranging between 1% and 36%. Finally, the scores for the ‘editorial independence’ domain ranged from 0% to 46%, with an average domain score of 16%, including five CPGs that scored 0%, making this domain the least well addressed.

The final evaluation of the guidelines summarises the overall opinion of whether or not the guideline should be recommended for clinical use. Eight out of the 17 CPGs can be recommended, with modification, for use based on the AGREE II instrument.

Discussion

There has been an increase in the development of CPGs in the field of hypertension; however, to the best of the authors’ knowledge, the present systematic review is the first attempt to evaluate the methodological quality of CPGs for the management of hypertension in China using the AGREE II instrument. It should be noted that the AGREE II instrument reflects how well the development process is reported but does not assess the content of the CPG or the quality of evidence used to formulate the recommendations, which is a common deficit of all existing appraisal tools.12

We found that there was considerable variation in the assessed CPGs in terms of the number of pages, authors and references. It was found that some CPGs comprised just one page, without any references or authors’ names, and many CPGs did not provide a statement about any potential conflicts of interest. It has been stated that if a conflict of interest is unavoidable, it should be disclosed. For these guidelines with no declaration of the conflicts of interest, there may have been no conflicts, but because the authors did not state that there was ‘no conflict of interest’, the potential for conflict is unclear. A few CPGs described the level of evidence, but without linking it to the recommendations. The differences and shortcomings in the classification and grading system were confusing.

Our results demonstrated that the methodological quality of CPGs for hypertension is undesirable in general and varies from guideline to guideline according to the different domains, with the ‘clarity of presentation’ and ‘scope and purpose’ receiving the highest scores, the ‘rigor of development’ and ‘editorial independence’ scoring the lowest, and ‘stakeholder involvement’ and ‘applicability’ being between these domains. The assessed guidelines performed best in the domains of ‘clarity of presentation’ and ‘scope and purpose’, which was consistent with the findings of a systematic review for recent hypertension CPGs,39 five priority diseases’ CPGs within Southern Africa40 and endocrine CPGs in North America.41 Despite the fact that these two domains scored highest among the six, the scores were still much lower than the global average scores, and great improvements that include clearly summarising and focusing on the issues that are most related to the patients and physicians are needed in the future in China.42

The CPGs for hypertension developed by the Writing Group of 2010, Chinese Guidelines for the Management of Hypertension in 2010, received the highest scores on average, but these were generally still less than 60%. Further assessment indicated that it failed to provide sufficient details about how the final recommendations were formulated.

The most poorly undertaken domain, ‘editorial independence’, is meant to assess the possible conflicts of interest of the author(s) and whether the guideline was developed independently of the funding source. European guidelines on a range of topics published from 2000 to 2007 similarly found that most guidelines scored low on ‘editorial independence’,43 while hypertension CPGs published from 2006 to 2011 confirmed that great progress had been made in the ‘editorial independence’ domain.39 Poor scores for hypertension CPGs developed in China highlight the need to improve the development process under an explicit conflict of interest. Insight into the ‘editorial independence’ increases the transparency of the guideline development process. The cited reasons for the low scores included poor reporting on whether a conflict of interest was assessed during the process of development of the guidelines or, if it was assessed, how it was addressed.44 However, low scores can also be explained by poor reporting. When no information is provided about the assessment item, the corresponding score will be low, which may not mean that the guideline was inappropriately developed, but that there was inadequate information provided.

All of the guidelines performed poorly with respect to ‘rigor of development’, which is considered to be crucial for the guideline quality by ensuring that a rigorous process was used to judge the underlying quality of evidence on which the guidelines were developed. A further analysis found that seven CPGs with the name of ‘expert consensus’ scored 12% (ranging from 5% to 20%), which is less than the average level, indicating that the scores for the domain of ‘rigor of development’ could be considered a useful reference to differentiate consensus from guideline. This step can play a key role in determining whether the recommendations are truly based on the evidence and in understanding how the evidence is synthesised. There is an explicit link between the best evidence available and the recommendations made, including clarifying which systems were used to evaluate the quality of the evidence and to grade recommendations.45

The less well addressed domain, ‘applicability’, produced disagreement, with the findings of the most recent published studies reporting that ‘applicability’ domain scored (42%) lower than the other five AGREE domains among 137 guidelines developed in USA, Canada, UK and an international group.46 Coming to the hypertension CPGs, ‘applicability’ scored low (38%) relative to all domains but ‘rigor of development’ (30%).39 It was shown that the type of developer (disease-specific foundation, non-profit healthcare system) was associated with applicability score, and the majority of hypertension CPGs included in the present study was developed by hypertension-specific association and non-profit healthcare system, which may be explained by the applicability scores achieved. Although the reported ‘applicability’ domain performed poorly, the scores are much higher than the score in the present study. The poor ‘applicability’ scores reflected that the implementation of guidelines, such as organisation barriers, the cost and the criteria used to monitor the local implementation or adaption, were not clearly addressed.47 48 It is worth considering that the process of defining facilitators and barriers to application should be integrated early in the guideline development process, including professional implementation strategies.49

The assessed guidelines achieved a moderate mean score in the ‘stakeholder involvement’ domain, which assessed the degree to which the guideline represents the views of its intended audience.50 It could be noted that patient preferences and experiences should be factored into decisions regarding clinical care, especially hypertension, the management of which can impact quality of life. The guidelines we reviewed would have benefited from ensuring that all guideline committees had patient representatives and that literature reviews specifically addressed the quality of life (when available).

The main limitation of this study is that the methodological quality assessment was based on what the CPGs developers actually reported, which might not truly reflect the construction process. Although two independent trained appraisers conducted the assessment using a standardised instrument, the involvement of subjective bias in the evaluation was unavoidable. The AGREE collaboration recommends that each guideline be assessed by at least two appraisers, but that without proper training, adding appraisers may increase the rating bias.51

Conclusion

The overall methodological quality of the CPGs for hypertension in China was generally low throughout the appraisal process as determined using the AGREE II instrument. Considerable variability remained between guidelines, and strategies should be implemented to enhance the clarification of the subject of CPGs, the application of evidence-based systematic methods, and the transparency of CPGs. In addition, multidisciplinary groups associated with professional organisations in China, especially the methodological experts in the field for which the CPG is being drawn up, should be consulted.