Reporting quality of chronic kidney disease practice guidelines according to the RIGHT statement: a systematic analysis.

AIM: The aim of this study was to evaluate the reporting quality of chronic kidney disease (CKD) guidelines.
METHODS: PubMed, EMBASE, and guideline-related websites were searched from 2008 to 2019. The CKD guidelines were included. Two reviewers used the RIGHT (Reporting Items for Practice Guidelines in Healthcare) checklist to assess the quality of guidelines and calculate the reporting proportion of each guideline.
RESULTS: We included 13 guidelines, of which 30.8% (4/13) were developed in Europe and about two-thirds (8/13) were published on their own website. The average quality of the 13 guidelines was 68.57%. The reporting proportion of the seven domains (i.e. basic information; background; evidence; recommendations; review and quality assurance; funding and declaration and management of interests; other information) were 65.39%, 81.73%, 63.08%, 69.23%, 53.85%, 63.46%, and 61.54%, respectively.
CONCLUSION: CKD guidelines had moderate reporting quality in some domains, but guideline developers should increase the reporting items in basic information, guideline evidence, and recommendations. The RIGHT checklist would be a useful tool to improve the reporting quality of guidelines.

Introduction

Chronic kidney disease (CKD) is defined as the presence of kidney damage (usually detected as urinary albumin excretion of ⩾30 mg/day or equivalent) or decreased kidney function [defined as estimated glomerular filtration rate (eGFR) <60 ml/min/1.73 m2] for 3 or more months, irrespective of cause.[1] Research has shown that CKD has a high global prevalence, with a consistent estimated global CKD prevalence of between 11% and 13%, with the majority at stage 3.[2] In assessments of disability-adjusted life year (DALY), CKD was ranked among the top 10 diseases that severely impact DALY in 27 of the 188 countries examined.[3] CKD has increasingly become a public health issue in the last several years.[4]

Clinical practice guidelines (CPGs) are statements that include recommendations intended to optimize patient care that are informed by systematic review of evidence and an assessment of the benefits and harms of alternative care options.[5] As guiding documents for clinical practice, high-quality guidelines can effectively regulate the medical treatment behavior of medical staff, improve the quality of medical services, and reduce medical costs.[6] However, CKD guidelines proved to be poor quality in the development process.[7] In this study, we will use the RIGHT (Reporting Items for Practice Guidelines in Healthcare) checklist to evaluate the reporting quality of CKD CPGs to identify problems in this field and find relevant solutions.[8]

Method

Inclusion and exclusion criteria

We included CPGs that met the following criterion: (1) based on evidence and related to CKD; (2) published between 2008 and 2019; (3) published in English; and (4) published in peer-reviewed journals or publicly available websites.

The following types of CPGs were excluded: (1) interpretation of guidelines; (2) guidelines related to other comorbidities or focused on only one symptom; and (3) older version of guidelines if an updated version was available.

Literature search

We searched Medline via PubMed, EMBASE, National Institute for Health and Clinical Excellence (NICE), Scottish Intercollegiate Guidelines Network (SIGN), Guidelines International Network (GIN), and international nephrology societies or associations from 1 January 2008 to 1 June 2019. Google Scholar was used to supplement the search. We also contacted any author whose guideline we could not acquire. The search formula used in PubMed is detailed in Appendix 1.

RIGHT checklist

The RIGHT checklist provides users and evaluators with a clear, explicit description of guideline developing processes and procedures, and the evidence used to formulate each recommendation. It contains 22 items (35 sub-items) grouped in seven domains: basic information (items 1–4), background (items 5–9), evidence (items 10–12), recommendations (items 13–15), review and quality assurance (items 16 and 17), funding and declaration and management of interests (items 18 and 19), and other information (items 20–22) (Appendix 2). Each item was scored as fully reported, partially reported, or not reported. In addition, we defined the reporting quality as (number of fully reported + number of partially reported)/35.

Data collection

One reviewer extracted the following information from included guidelines: title, published year, published country, developer, type of based evidence, grading system, and published journal or websites. When any of these data were not available, we contacted the authors of the articles to ask for the missing information. Two reviewers performed the evaluation of reporting quality of included guidelines, and the third reviewer checked the evaluation results.

Data analysis

Descriptive statistics were applied using Microsoft Excel 2019. The overall reporting rate, each domain reporting rate, and each item reporting rate were calculated. We performed subgroup analysis based on the guideline published in journals or on websites.

Results

The literature search yielded 1537 studies, of which 974 were considered relevant for our study. After screening the title and abstract, 946 articles were excluded. After reading the full text, 15 guidelines were excluded due to non-compliance with the inclusion criteria, and 13 guidelines were eventually included (Figure 1).

Figure 1.

Flowchart of guidelines screening.

CKD, chronic kidney disease.

Characteristics of included guidelines

A total of 13 guidelines were included, of which 30.8% (4/13) guidelines were developed in Europe; the remaining 70% were from the United states (US; n = 2), Canada (n = 2), Malaysia (n = 1), India (n = 1), Japan (n = 1), Australia (n = 1) and global (n = 1). The majority of guidelines were developed based on systematic reviews, and about two-thirds of guidelines (8/13) were published only on their own website. In terms of the grading system, most guidelines (76.9%) performed grading of evidence quality and recommendation strength, of which, 60% (6/10) guidelines used the GRADE system (Table 1).

Table 1.

Basic characteristics of included guidelines.

No.	Guidelines	Published year	Published country	Developer	Type of evidence	Grading system	Published journal or websites
1	Chronic Kidney Disease in Adults-Identification, Evaluation and Management[9]	2019	Canada	Ministry of Health (British Columbia)	Guideline; SR; others	Unclear	British Columbia website
2	Management of chronic kidney disease in adults [Second Edition][10]	2018	Malaysia	MHM	SR	GRADE; CTFPHC	MHM website
3	Northern Ireland Guidelines for the Management of Chronic Kidney Disease (CKD)[11]	2015	United Kingdom	GAIN & NINF	Guideline; SR;	None	GAIN website
4	Chronic kidney disease in adults: assessment and management[12]	2014	United Kingdom	NICE	SR	NICE	NICE website
5	Management of Chronic Kidney Disease (CKD) in Primary Care (2014)[13]	2014	United States	VA/DoD	SR; clinical and epidemiological evidence	Modified GRADE; USPSTF	VA/DoD website
6	Evidence-based Clinical Practice Guideline for CKD 2013[14]	2014	Japan	Japanese Society of Nephrology	SR; clinical and epidemiological evidence	Unclear	Clin Exp Nephrol
7	KHA-CARI Guideline: Early chronic kidney disease: Detection, prevention and management[15]	2013	Australia	KHA-CARI	SR	GRADE	Nephrology
8	Screening, Monitoring, and Treatment of Stage 1–3 Chronic Kidney Disease: A Clinical Practice Guideline from the American College of Physicians[16]	2013	United States	ACP	SR	ACP	Annals of Internal Medicine
9	Indian Chronic Kidney Disease Guidelines[17]	2013	India	ISN	Guideline; SR; others	GRADE	ISN website
10	KDIGO 2012 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease[1]	2013	Global	KDIGO	SR	GRADE	KDIGO website
11	Renal Association Clinical Practice Guideline on Detection, Monitoring and Management of Patients with CKD[18]	2011	England	The Renal Association	Guideline; SR	Modified GRADE	Nephron Clin Pract
12	Guidelines for the management of chronic kidney disease[19]	2008	Canada	Canadian Society of Nephrology	Guideline; SR	CHEP	CMAJ
13	Diagnosis and Management of Chronic Kidney Disease. A National Clinical Guideline[20]	2008	Scotland	SIGN	SR	SIGN	SIGN website

ACP, American College of Physicians; CHEP, Canadian Hypertension Education Program; CMAJ, Canadian Medical Association Journal; CTFPHC, US/Canadian Preventive Services Task Force; GAIN, Guidelines and Audit Implementation Network; GRADE, Grading of Recommendations, Assessment, Development and Evaluation; ISN, Indian Society of Nephrology; KDIGO, Kidney Disease | Improving Global Outcome; KHA-CARI, Kidney Health Australia-Caring for Australasians with Renal Impairment; MHM, Ministry of Health Malaysia; NICE, The National Institute for Health and Care Excellence; NINF, Northern Ireland Nephrology Forum; SIGN, Scottish Intercollegiate Guidelines Network; SR, Systematic Review; USPSTF, US Preventative Service Task Force; VA/DoD, Department of Veterans Affairs and the Department of Defense.

Overall reporting quality of included guidelines

The reporting rate of the 13 guidelines ranged from 28.57% to 97.14%, and the mean reporting rate was 68.57%. Two guidelines scored more than 90%, seven guidelines (53.85%) more than 60%, and two guidelines lower than 30% (Table 2).

Table 2.

Reporting quality of included guidelines.

Domain	Items	Guidelines													Reporting proportion (%)
		1	2	3	4	5	6	7	8	9	10	11	12	13
Basic information	1a	+	+	+	−	+	+	+	+	+	+	+	+	+	92.31
	1b	−	+	−	−	−	+	−	−	−	+	−	−	−	23.08
	1c	+	+	+	+	+	−	+	+	−	+	+	+	+	84.62
	2	+	+	−	+	+	+–	+–	+	−	+–	+	+	−	76.92
	3	+–	+	+	+	+	+	−	−	−	+	−	−	+	61.54
	4	−	−	+	−	−	+	+	+	−	+	+	+	−	53.85
Reporting proportion (%)		66.67	83.33	66.67	50.00	66.67	83.33	66.67	66.67	16.67	100.00	66.67	66.67	50.00	65.39
Background	5	+	+	+	+	+	−	+–	+	+	+	+–	+	+	92.31
	6	−	+	+	+	+	−	+	+	+	+	+–	+	+	84.62
	7a	+	+	+	+	+	−	+	+	+	+	+	+	+	92.31
	7b	−	+	+	+	−	−	−	+	−	+	+	+–	+	61.54
	8a	−	+	+−	+	+	−	+	+	+	+	+−	+	+	84.62
	8b	−	+	+	+	+	−	+−	+−	+−	+	+	+	+	84.62
	9a	−	+	+	+	+	−	+−	+	+	+	−	+	+	76.92
	9b	−	+	+	+	+−	−	−	+	+	+	+−	+	+	76.92
Reporting proportion (%)		25.00	100.00	100.00	100.00	87.50	0.00	75.00	100.00	87.50	100.00	87.50	100.00	100.00	81.73
Evidence	10a	+–	+–	−	+	+	+	+	+	+	+	+	+	+	92.31
	10b	−	−	−	+	−	−	−	−	−	+	−	−	+	23.08
	11a	−	+	+	+	+	−	−	+	−	+	−	+	+	61.54
	11b	−	+	+	+	+–	−	−	+	−	+	−	+–	+	61.54
	12	−	+	−	+	+	−	+	+	+–	+	+	+	+	76.92
Reporting proportion (%)		20.00	80.00	40.00	100.00	80.00	20.00	40.00	80.00	40.00	100.00	40.00	80.00	100.00	63.08
Recommendations	13a	+	+	+–	+	+	+	+	+	+	+	+	+	+	100.00
	13b	+	+	+	+	+	+	+	+	+	+	+	+	+	100.00
	13c	−	+	−	+	+	+	+	+	+	+	+	+	+	84.62
	14a	−	+–	+–	+	+	−	−	−	+	+	−	+–	+–	61.54
	14b	−	+	−	+	+	+–	−	+	+	+	+	−	+–	69.23
	14c	−	+	−	+	+	−	−	−	−	−	−	−	+–	30.77
	15	−	+	+	+	+	−	−	−	−	−	−	−	+	38.46
Reporting proportion (%)		28.57	100.00	57.14	100.00	100.00	57.14	42.86	57.14	71.43	71.43	57.14	57.14	100.00	69.23
Review and quality assurance	16	−	+	+	+	+	−	−	+	−	+	−	+	+	61.54
	17	−	+	−	+	−	−	−	+–	−	+	−	+	+	46.15
Reporting proportion (%)		0.00	100.00	50.00	100.00	50.00	0.00	0.00	100.00	0.00	100.00	0.00	100.00	100.00	53.85
Funding, declaration and management of interests	18a	−	+	+	+	−	−	−	+–	−	+	+	+	+	61.54
	18b	−	+	+	+	−	−	−	+–	−	+	−	−	+–	46.15
	19a	−	+	+	+	+	−	+	+	−	+	+	+	+	76.92
	19b	−	+	+	+	+	−	+–	+	−	+	−	+	+–	69.23
Reporting proportion (%)		0.00	100.00	100.00	100.00	50.00	0.00	50.00	100.00	0.00	100.00	50.00	75.00	100.00	63.46
Other information	20	+	+	+	+	+	−	+	+	+	+	+	+	+	92.31
	21	−	−	−	−	+	−	−	+	+	+	−	+	−	38.46
	22	−	+	−	+	+	−	−	+	+	+	−	−	+	53.85
Reporting proportion (%)		33.33	66.67	33.33	66.67	100.00	0.00	33.33	100.00	100.00	100.00	33.33	66.67	66.67	61.54
Total reporting proportion		28.57	91.43	68.57	88.57	80.00	28.57	51.43	82.86	51.43	97.14	57.14	77.14	88.57	68.57

Fully reported, +; Partially reported, +–; Not reported, –.

Reporting quality of each domain

The reporting quality of the background domain was highest (81.73%), the reporting quality in domain 5 (review and quality assurance) was lowest (53.85%), and the reporting quality in domain 1 (basic information), domain 3 (evidence), domain 4 (recommendations), domain 6 (funding, declaration and management of interests), and domain 7 (other information) was 65.39%, 63.08%, 69.23%, 63.46%, and 61.54%, respectively (Table 2).

Reporting quality of each item

The RIGHT checklist contains 35 sub-items, of which item 13a (recommendations) and 13b (recommendations) were fully reported (100%); however, the reporting rate of item 1b (title/subtitle) and10b (health care questions) were lowest (23.08%). For seven sub-items (17, 18b, 15, 21, 14c, 1b, and 10b), the reporting quality was less than 50% (Figure 2).

Figure 2.

Percentage distribution of scores for each item in the RIGHT checklist (n = 13).

RIGHT, reporting items for practice guidelines in healthcare.

Subgroup analysis

We used Student’s t tests for independent samples to compare the reporting quality of journal published or websites published guidelines. The results showed that the reporting quality of guidelines was not associated with the place of publication [mean difference (MD) = 0.15, 95% confidence interval (CI) (–0.10, 0.40), p = 0.24].

Discussion

We included 13 CKD guidelines to evaluate reporting quality; the mean reporting score was 68.57%, which was a little higher than for the guidelines in other fields.[21,22] However, the reporting quality in some domains was still poor. Reasons, suggestions, and comments should be given to CKD guideline developers to improve the quality of guidelines.

First, in the “basic information” domain, item 1b (describe the year of publication of the guideline) and item 4 (corresponding developer) were poorly reported. The reason for the poor reporting quality of item 1b might be because guideline developers did not attach importance to the time in the title of their guideline. But the year of publication is an indicator as to whether the guideline is up to date, and, thus, whether the guideline’s reader needs to look elsewhere for the latest guideline. For item 4, most CKD guidelines published on websites did not report the corresponding developer, because the reader could contact the person in charge of the website to obtain relevant guideline information directly.

Second, in the “evidence” domain, item 10b (indicate how the outcomes were selected and sorted) had a reporting score of only 23.08%, that is, most CKD guidelines did not report how they select outcomes, mainly because the outcomes of the kidney disease guidelines were relatively simple. However, the choice of outcome is critical in the development of PICO (patients, intervention, control and outcome) questions because it impacts the balance of benefits and harms upon which recommendations are based, and readers need to know how and why outcomes are selected.

Third, in the “recommendations” domain, item 14c (other factors considered in the formulation of recommendations) and 15 (evidence to decision processes) had a reporting score of less than 40%. As we know, recommendations are the core components of guidelines and presented with clear, specific, and actionable statements.[23] When forming a recommendation, we should consider not only the quality of the evidence, but also other relevant factors, and we should report it in the guideline full text in detail, which is also consistent with the AGREE II requirements.[24] In general, the processes used by the guideline panel to formulate the recommendations and to make other group decisions should be clearly described in the final guideline to ensure transparency of the development process. To promote guideline quality in CKD, we should reference the RIGHT checklist to report and present recommendations.

Finally, in the “other information” domain, the CKD guidelines only poorly reported suggestions for further research (item 21). However, guideline developers should describe gaps in the evidence and/or provide suggestions for future research according to the RIGHT checklist, which will help developers to highlight future research needs and better fill the existing gaps. In our study, about two-thirds of CKD guidelines did not report the section of discussion, with the absence of other information as well.

In the “background, review and quality assurance and funding, declaration and management of interests” domains, the reporting quality of CKD guidelines was over 60%, which is consistent with the research of Daza et al. regarding the methodological quality CKD guidelines.[7] It also suggests that CKD guideline development should continue to maintain the quality of reporting in related domains as well as improve the quality in poor reporting domains.

The RIGHT statement was developed in 2017 by an international group of stakeholders. Of the 13 CKD guidelines we included, 11 were published before 2017. Out of 11 guidelines, 4 had a report quality of less than 60%. However, at that time, the RIGHT statement had not yet been developed. Guideline developers failed to follow the reporting guidelines. Thus, when we use these guidelines, we need to consider the bias caused by publication year.

To the best of the authors’ knowledge, this is the first study to use the RIGHT checklist to evaluate the reporting quality for CKD guidelines. We carried out strict quality control during the evaluation process. However, our study also has some weaknesses. First, we did not include guidelines published in other languages, which maybe introduce publication bias; second, we excluded guidelines on CKD comorbidities.

In conclusion, CDK guidelines had moderate reporting quality in some domains. But guideline developers still should increase reporting items in basic information, guideline evidence and recommendations, and the RIGHT checklist would be a better way to improve the quality of guidelines.

Click here for additional data file.

Supplemental material, APPENDIX_1 for Reporting quality of chronic kidney disease practice guidelines according to the RIGHT statement: a systematic analysis by Yang Zhao, Yanyan Li, Junwei Li, Weijuan Song, Jun Zhao, Yan Xu, Yongxia Zhai and Shuaimin Xu in Therapeutic Advances in Chronic Disease