Literature DB >> 32858405

Quality and reporting of clinical guidelines for breast cancer treatment: A systematic review.

Marta Maes-Carballo¹, Luciano Mignini², Manuel Martín-Díaz³, Aurora Bueno-Cavanillas⁴, Khalid Saeed Khan⁵.

Abstract

BACKGROUND: High-quality, well-reported clinical practice guidelines (CPGs) and consensus statements (CSs) underpinned by systematic reviews are needed. We appraised the quality and reporting of CPGs and CSs for breast cancer (BC) treatment.
METHODS: Following protocol registration (Prospero no: CRD42020164801), CPGs and CSs on BC treatment were identiﬁed, without language restrictions, through a systematic search of bibliographic databases (MEDLINE, EMBASE, Web of Science, Scopus, CDSR) and online sources (12 guideline databases and 51 professional society websites) from January 2017 to June 2020. Data were extracted in duplicate assessing overall quality using AGREE II (% of maximum score) and reporting compliance using RIGHT (% of total 35 items); reviewer agreement was 98% and 96% respectively.
RESULTS: There were 59 relevant guidance documents (43 CPGs, 16 CSs), of which 20 used systematic reviews for evidence synthesis. The median overall quality was 54.0% (IQR 35.9-74.3) and the median overall reporting compliance was 60.9% (IQR 44.5-84.4). The correlation between quality and reporting was 0.9. Compared to CSs, CPGs had better quality (55.4% vs 44.2%; p = 0.032) and reporting (67.18% vs 44.5%; p = 0.005). Compared to subjective methods of evidence analysis, guidance documents that used systematic reviews had better quality (76.3% vs 51.4%; p = 0.001) and reporting (87.1% vs 59.4%; p = 0.001).
CONCLUSION: The quality and reporting of CPGs and CSs in BC treatment were moderately strong. Systematic reviews should be used to improve the quality and reporting of CPGs and CSs.

Entities: Chemical Disease Gene Species

Keywords: Breast cancer”; “AGREE II”; “Appraisal instruments”; “Clinical practice guidelines”; “Consensus”; “Guidelines”; “Quality of guidelines”; “RIGHT”; “Treatment”

Mesh：

Year: 2020 PMID： 32858405 PMCID： PMC7473996 DOI： 10.1016/j.breast.2020.07.011

Source DB: PubMed Journal: Breast ISSN： 0960-9776 Impact factor: 4.380

Introduction

Breast cancer (BC) is the most frequent cancer in women (about 2 million new cases annually) accounting for 15% of global cancer deaths (about 670,000 annually) [[1], [2], [3]]. Recent advances have shown the potential to decrease morbidity and mortality [[4], [5], [6]], but treatment success varies by region and type of hospital [7]. Clinical practice guidelines (CPGs) and consensus statements (CSs) are being promoted to harmonize the provision of effective health care [[8], [9], [10], [11]]. Rigorously developed CPGs and CSs should be well-reported, deploying objective approaches for evidence analysis to underpin the recommendations [10,12]. Previous evaluations of guidance in BC treatment have shown that their quality can be heterogeneous [[13], [14], [15]]. However, these reviews are non-recent, covering CPGs and CSs published between 2009 and 2017. They were limited in their searches and applied languages restrictions to English only [[13], [14], [15]]. They have not had the benefit of recent developments in the assessment of CPGs and CSs [16,17]. It has been highlighted that quality and reporting are two distinct aspects that need to be examined separately. The former deals with issues of validity of the recommendations made while that latter examines the thoroughness of the presentation of the document prepared. In this regard, the thoroughness and transparency of evidence synthesis is a key guideline feature [18]. As there is a requirement for periodic revisions, an updated and comprehensive evaluation of recently published guidance documents is required [7]. In a systematic review, we exhaustively searched for recent CPGs and CSs for BC treatment and appraised their quality and reporting using validated tools, paying special attention to the method used for evidence analysis.

Methods

Following prospective registration (Prospero no: CRD42020164801) a protocol-driven systematic review was performed using currently recommended methods for search and assessment of guidelines and reported using PRISMA statement (see Appendix 1) [[19], [20]].

Data sources and searches

The initial search from 2017 onwards was conducted on April 4th, 2020. A search update was undertaken on June 15th, 2020. We looked for online databases and guideline-specific databases without language restrictions associating MeSH terms “breast cancer”, “breast neoplasms”, “practice guidelines”, “guidelines”, “consensus” and including word alternatives, covering the period January 2017 to June 2020. We have also checked the specific professional society’s websites looking for updated guidelines. We decided to look for CPGs and CSs from 2017 onwards. The main reason for focusing on this 3-year time window was that a systematic review of literature stated that most of the guidance methodological handbooks for updating CPGs determined that the time between updates should be two or three years [21]. By excluding older guidance documents in which new knowledge for good CPG methods has not been incorporated we were able to review the most up-to-date literature. We looked for online databases (MEDLINE, CDSR, Web of Science, EMBASE, Scopus, etc.), 51 websites of important professional societies, and 12 guidance-specific databases (see Appendix 2). The main criterion for searching the websites of professional societies was the contribution of their country of origin to global breast cancer’s scientific production. We included professional societies in countries that produce at least 0.5% of the documents appearing in Scopus about Breast Cancer and Health Care (23,748 document results at July 10th, 2020). Finally, we searched the bibliographies of well-known publications and the World Wide Web to include other important documents in the review.

Study selection and data extraction

CPGs and CSs about BC management produced by national or international professional organizations and societies or governmental agencies were included. Randomized controlled trials (RCTs) and observational studies, narrative reviews, scientific reports, discussion papers, conference abstracts and posters, CPGs and CSs about screening and diagnosis, obsolete guidelines replaced by updates from the same organization, and CPG and CSs for education and information purpose only were excluded. The eligibility of each of the abstracts and titles from the citations was considered independently by two reviewers (MMC and LM), both breast cancer specialists. Full-text versions of potentially relevant citations were obtained to confirm eligibility. A third reviewer (MMD) helped to solve disagreements by consensus or arbitration. Duplicate articles were identified and removed. Where multiple versions were retrieved the most updated version of the guidelines was included. Data were extracted from selected CPGs and CSs in duplicate, independently.

Assessment of quality and reporting

Two reviewers (MMC and LM) extracted data on a piloted proforma to assess the quality and reporting of CPGs and CSs using two validated appraisal tools, the AGREE II instrument and the RIGHT statement (Appendix 3).16 [17] According to AGREE II quality was the “reliability that potential development biases have been appropriately addressed and recommendations are internally and externally valid” [22]. Data were extracted for its 23 items according to predefined criteria divided into six domains: scope and purpose (items 1 to 3), stakeholder involvement (items 4 to 6), the rigor of development (items 7 to 14), clarity and presentation (items 15 to 17), applicability (items 18 to 21) and editorial independence (items 22 and 23). A 7-point scale was used to score each item (anchored between 1 or strongly disagree, i.e. when there was no relevant information concerning the item, to 7 or strongly agree, i.e. when the quality of reporting was exceptional, and the criteria were fully met). The domain quality scores (0–100%) were calculated by summing up reviewers’ individual scores and scaling as a percentage of the maximum possible score according to the formula provided in the AGREE II manual averaging the scores of the two reviewers [22]. To avoid major deviations in reviewers’ assessments, we deployed discussion to reach consensus. In addition, an overall guideline assessment was calculated using the mean scores of the 6 standardized domain and a recommendation made: a CPG or CS was “recommended” if the score >80% [23], “recommended with modifications” if it was 50–80%, and “not recommended” if <49% [24]. For reporting assessment data were extracted for the RIGHT [17] statement’s 35 items divided into 7 domains: basic information (items 1 to 4), background (items 5 to 9), evidence (items 10 to 12), recommendations (items 13 to 15), review and quality assurance (items 16 and 17), funding and declaration and management of interests (items 18 and 19), and other information (items 20 to 22). A numeric score of 1 (reported), 0.5 (partially reported), or 0 (unreported) was assigned to each item. Disagreements between two reviewers in the score were discussed and unresolved matters were addressed by an arbitrator (MMD). A percentage of the total was calculated to obtain an overall reporting assessment and guidance documents were classified as “well-reported” if the score was >80%, “moderate-reported” if it was 50–80%, and “low-reported” if <50% [24].

Data analysis

Consistency between reviewers in data extraction was assessed using the intraclass correlation coefficient (ICC), where excellent reliability level was >0.90 [25]. A descriptive statistical analysis was conducted for domains and overall scores. Kruskal-Wallis test was used to compare scores and to evaluate factors that might affect the quality and reporting of CPGs and CSs. All analyses were performed using Stata 16. A value of p < 0.05 denoted statistical significance.

Results

Study selection

Of the 7430 potential citations identified, 7334 were from online databases (MEDLINE, EMBASE, SCOPUS, Web of Science, Trip database) and 96 were from additional sources (guideline specific databases, professional societies, and the Word Wide Web). Of them, 168 publications were found duplicated and 7205 did not meet the selection criteria. A total of 59 documents (43 CPGs [[26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68]] and 16 CSs 42–57 [[69], [70], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83], [84]]) were identified for final evaluation (Table 1). The flow diagram detailing the study selection process is provided in Fig. 1. ICC for reviewer agreement was 0.98 in AGREE II and 0.96 in RIGHT. The correlation between AGREE II and RIGHT scores was r = 0.90 (Appendix 4).

Table 1

Description of the CPGs and CSs (n = 167) selected for the systematic review.

	Name of the CPG	Abbreviated name	Entity	Country	Year	Publication in a Journal	Version	Evidence analysis	Quality tool referral
1	Chinese guidelines for diagnosis and treatment of breast cancer 2018⁽²⁶⁾	Chinese BC diagnosis treatment⁽²⁶⁾	NHCPRC	China	2018	CJCRCN	1	Not reported	Not reported
2	Early breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up⁽²⁷⁾	ESMO BC 2019⁽²⁷⁾	ESMO	Europe	2019	Annals of Oncology	3	Review	Not reported
3	ESO-ESMO 4th international consensus guidelines for breast cancer in young women (BCY4) (²⁸⁾	BCY4⁽²⁸⁾	ESMO, ESO, EUSOMA	Europe	2020	The Breast	3	Consensus method; review	Not reported
4	AGO Recommendations for the Diagnosis and Treatment of Patients with Early Breast Cancer: Update 2019⁽²⁹⁾	AGO Early BC 2019⁽²⁹⁾	AGO	Germany	2019	Breast Care	5	Review	Not reported
5	AGO Recommendations for the Diagnosis and Treatment of Patients with Advanced and Metastatic Breast Cancer: Update 2018⁽³⁰⁾	AGO Advanced BC 2018⁽³⁰⁾	AGO	Germany	2018	Breast Care	5	Review	Not reported
6	Dutch breast reconstruction guideline⁽³¹⁾	Dutch BCR⁽³¹⁾	DPRS	Netherlands	2017	JPRAS	1	Systematic review	AGREE II
7	Cáncer de mama/Breast Cancer⁽³²⁾	Fisterra BC⁽³²⁾	Fisterra	Spain	2017	Not published	3	Not reported	Not reported
8	SEOM clinical guidelines in early-stage breast cancer⁽³³⁾	SEOM early-stage⁽³³⁾	SEOM	Spain	2018	CTO	2	Consensus method, not specified technique	Not reported
9	SEOM clinical guidelines in advanced and recurrent breast cancer⁽³⁴⁾	SEOM advanced BC⁽³⁴⁾	SEOM	Spain	2018	CTO	3	Consensus method, not specified technique	Not reported
10	Abemaciclib with fulvestrant for treating hormone receptor-positive, HER2-negative advanced breast cancer after endocrine the therapy⁽³⁵⁾	NICE Abemaciclib⁽³⁵⁾	NICE	UK	2019	Not published	1	Systematic review	Not reported
11	Ribociclib with fulvestrant for treating hormone receptor-positive, HER2-negativHER2-negative, advanced breast cancer⁽³⁶⁾	NICE Ribociclib⁽³⁶⁾	NICE	UK	2019	Not published	1	Systematic review	Not reported
12	Early and locally advanced breast cancer: diagnosis and management⁽³⁷⁾	Early and locally advanced BC ⁽³⁷⁾	NICE	UK	2018	Not published	1	Systematic review	Not reported
13	Breast reconstruction following prophylactic or therapeutic mastectomy for breast cancer⁽³⁸⁾	AHS reconstruction BC⁽³⁸⁾	AHS	Canada	2017	Not published	2	Consensus method; review	Not reported
14	Adjuvant systemic therapy for early stage (lymph node negative and lymph node positive) breast cancer⁽³⁹⁾	AHS early BC⁽³⁹⁾	AHS	Canada	2018	Not published	4	Consensus method; review	Not reported
15	Performance and Practice Guidelines for the Use of Neoadjuvant Systemic Therapy in the Management of Breast Cancer⁽⁴⁰⁾	ABSB Neoadjuvance BC ⁽⁴⁰⁾	ASBS	USA	2017	Not published	1	Consensus method; review	Not reported
16	Evidence-Based Clinical Practice Guideline: Autologous Breast Reconstruction with DIEP or Pedicled TRAM Abdominal Flaps⁽⁴¹⁾	ASPS DIEP & TRAM⁽⁴¹⁾	ASPS	USA	2017	PRS	2	Review	Not reported
17	Use of Endocrine Therapy for Breast Cancer Risk Reduction: ASCO Clinical Practice Guideline Update⁽⁴²⁾	ASCO Endocrine therapy risk BC⁽⁴²⁾	ASCO	USA	2019	JCO	2	Systematic review	Not reported
18	Postmastectomy Radiotherapy: An American Society of Clinical Oncology, American Society for Radiation Oncology, and Society of Surgical Oncology Focused Guideline Update⁽⁴³⁾	ASCO postmastectomy RT⁽⁴³⁾	ASCO	USA	2017	JCO	2	Systematic review	Not reported
19	Selection of Optimal Adjuvant Chemotherapy and Targeted Therapy for Early Breast Cancer: ASCO Clinical Practice Guideline Focused Update⁽⁴⁴⁾	ASCO treatment for early BC⁽⁴⁴⁾	ASCO	USA	2018	JCO	2	Systematic review	Not reported
20	Systemic Therapy for Patients With Advanced Human Epidermal Growth Factor Receptor 2–Positive Breast Cancer: ASCO Clinical Practice Guideline Update⁽⁴⁵⁾	ASCO systemic therapy EGR2 BC⁽⁴⁵⁾	ASCO	USA	2018	JCO	2	Systematic review	Not reported
21	Recommendations on Disease Management for Patients With Advanced Human Epidermal Growth Factor Receptor 2–Positive Breast Cancer and Brain Metastases: ASCO Clinical Practice Guideline Update⁽⁴⁶⁾	ASCO EGRF2 MBC⁽⁴⁶⁾	ASCO	USA	2018	JCO	2	Systematic review	Not reported
22	Integrative Therapies During and After Breast Cancer Treatment: ASCO Endorsement of the SIO Clinical Practice Guideline⁽⁴⁷⁾	ASCO BC treatment⁽⁴⁷⁾	ASCO	USA	2018	JCO	2	Systematic review	Not reported
23	Role of Bone-Modifying Agents in Metastatic Breast Cancer: An American Society of Clinical Oncology–Cancer Care Ontario Focused Guideline Update⁽⁴⁸⁾	ASCO bone-mod agents MBC⁽⁴⁸⁾	ASCO	USA	2017	JCO	2	Systematic review	Not reported
24	Role of Patient and Disease Factors in Adjuvant Systemic Therapy Decision Making for Early-Stage, Operable Breast Cancer: American Society of Clinical Oncology Endorsement of Cancer Care Ontario Guideline Recommendations⁽⁴⁹⁾	ASCO factors in early BC⁽⁴⁹⁾	ASCO	USA	2019	JCO	2	Systematic review	Not reported
25	Use of Adjuvant Bisphosphonates and Other Bone-Modifying Agents in Breast Cancer: A Cancer Care Ontario and American Society of Clinical Oncology Clinical Practice Guideline⁽⁵⁰⁾	ASCO use bone-mod agents BC⁽⁵⁰⁾	ASCO	USA	2017	JCO	1	Systematic review	Not reported
26	Use of Biomarkers to Guide Decisions on Adjuvant Systemic Therapy for Women With Early-Stage Invasive Breast Cancer: American Society of Clinical Oncology Clinical Practice Guideline Focused Update⁽⁵¹⁾	ASCO biomarkers in early BC⁽⁵¹⁾	ASCO	USA	2019	JCO	2	Review	Not reported
27	Use of Biomarkers to Guide Decisions on Systemic Therapy for Women With Metastatic Breast Cancer: American Society of Clinical Oncology Clinical Practice Guideline⁽⁵²⁾	ASCO biomarkers in MBC⁽⁵²⁾	ASCO	USA	2019	JCO	2	Systematic review	Not reported
28	Adjuvant Endocrine Therapy for Women With Hormone Receptor-Positive Breast Cancer: ASCO Clinical Practice Guideline Focused Update⁽⁵³⁾	ASCO endocrine treatment Her2 BC⁽⁵³⁾	ASCO	USA	2019	JCO	2	Systematic review	Not reported
29	Optimal margins for breast-conserving surgery with whole-breast irradiation in ductal carcinoma in situ: Results of the ASTRO, ASCO, and SSO consensus guideline⁽⁵⁴⁾	ASCO, ASTRO, SSO CID⁽⁵⁴⁾	ASCO, ASTRO, SSO	USA	2017	Annals of Surgery	1	Consensus method; review	Not reported
30	Radiation therapy for the whole breast: Executive summary of an American Society for Radiation Oncology (ASTRO) evidence-based guideline⁽⁵⁵⁾	ASTRO RT for whole breast⁽⁵⁵⁾	ASTRO	USA	2018	PRO	2	Systematic review	Not reported
31	Breast Cancer. Version 3.2019⁽⁵⁶⁾	NCCN BC⁽⁵⁶⁾	NCCN	USA	2019	JNCCN	4	Review	Not reported
32	Influencing best practice in breast cancer⁽⁵⁷⁾	Australia BC⁽⁵⁷⁾	AG	Australia	2017	Not published	1	Systematic review	Not reported
33	Guía de práctica clínica (GPC) para la detección temprana, tratamiento integral, seguimiento y rehabilitación del cáncer de mama⁽⁵⁸⁾	GPC Colombia⁽⁵⁸⁾	INC	Colombia	2017	Not published	2	Systematic review	Not reported
34	Guía de Práctica Clínica para el Tratamiento del Cáncer de Mama⁽⁵⁹⁾	GPC Perú⁽⁵⁹⁾	IETSI	Perú	2017	Not published	1	Systematic review	AGREE II
35	The Screening, Diagnosis, Treatment, and Follow-Up of Breast Cancer⁽⁶⁰⁾	Würzburg BC⁽⁶⁰⁾	UHW	Germany	2018	DAI	1	Systematic review	Not reported
36	Cirugía de la Mama⁽⁶¹⁾	AEC BC⁽⁶¹⁾	AEC	Spain	2017	Not published	2	Not reported	Not reported
37	Manual de Práctica Clínica en Senología. 4^a Edición. 2019 ⁽⁶²⁾	SESPM ⁽⁶²⁾	SESPM	Spain	2019	Not published	2	Not reported	Not reported
38	Linee guida: Neoplasie della mammela ⁽⁶³⁾	CIS Neoplasia mammella ⁽⁶³⁾	CIS	Italy	2019	Not published	1	Not reported	Not reported
39	La radioterapia nel carcinoma della mammella. Indicazioni e tecniche ⁽⁶⁴⁾	CIS RT mammella ⁽⁶⁴⁾	CIS	Italy	2018	Not published	1	Not reported	Not reported
40	Recommandations du GEFPICS pour la prise en charge des prélèvements dans le cadre du traitement néoadjuvant du cancer du sein ⁽⁶⁵⁾	GEFPICS Cancer du sein ⁽⁶⁵⁾	GEFPICS	France	2019	Annals of Pathologie	1	Not reported	Not reported
41	Breast Cancer Clinical Guidelines⁽⁶⁶⁾	NCA BC⁽⁶⁶⁾	NCA	UK	2019	Not published	1	Review	Not reported
42	The Japanese Breast Cancer Society Clinical Practice Guidelines for systemic treatment of breast cancer, 2018 edition⁽⁶⁷⁾	Japanese systemic BC⁽⁶⁷⁾	JBCS	Japan	2020	Breast Cancer	2	Systematic review	Not reported
43	The Japanese Breast Cancer Society Clinical Practice Guidelines, 2018 edition: the tool for shared decision making between doctor and patient⁽⁶⁸⁾	Japanese SDM BC⁽⁶⁸⁾	JBCS	Japan	2020	Breast Cancer	1	Systematic review	Not reported
44	Consenso Mexicano sobre diagnóstico y tratamiento del cáncer mamario ⁽⁶⁹⁾	GPC México⁽⁶⁹⁾	SSM	México	2019	Not published	7	Nominal group technique	Not reported
45	Indian Solutions for Indian Problems—Association of Breast Surgeons of India (ABSI) Practical Consensus Statement, Recommendations, and Guidelines for the Treatment of Breast Cancer in India⁽⁷⁰⁾	Indian ICMR CS⁽⁷⁰⁾	ABSI	India	2017	IJS	2	Delphy modified technique	Not reported
46	4th ESO–ESMO International Consensus Guidelines for Advanced Breast Cancer (ABC 4) (⁷¹⁾	ABC4⁽⁷¹⁾	ESMO	Europe	2018	Annals of Oncology	4	Nominal group technique	Not reported
47	St. Gallen/Vienna 2019: A Brief Summary of the Consensus Discussion about Escalation and De-Escalation of Primary Breast Cancer Treatment⁽⁷²⁾	St. Gallen 2019⁽⁷²⁾	St. Gallen	Europe	2019	Breast Care	4	Nominal group technique	Not reported
48	Biomarkers in breast cancer: A consensus statement by the Spanish Society of Medical Oncology and the Spanish Society of Pathology⁽⁷³⁾	SEOM & SEAP⁽⁷³⁾	SEOM & SEAP	Spain	2017	CTO	1	Not reported	Not reported
49	Provincial consensus recommendations for adjuvant systemic therapy for breast cancer⁽⁷⁴⁾	CCM 2017⁽⁷⁴⁾	CCM	Canada	2017	Not published	1	Systematic review	AGREE II
50	Consensus Guideline on Accelerated Partial Breast Irradiation⁽⁷⁵⁾	ASBS RT⁽⁷⁵⁾	ASBS	USA	2018	Not published	1	Review	Not reported
51	Consensus Guideline on the Use of Transcutaneous and Percutaneous Ablation for the Treatment of Benign and Malignant Tumors of the Breast⁽⁷⁶⁾	ASBS ablation⁽⁷⁶⁾	ASBS	USA	2018	Not published	1	Review	Not reported
52	Consensus Guideline on the Management of the Axilla in Patients With Invasive/In-Situ Breast Cancer⁽⁷⁷⁾	ASBS axilla⁽⁷⁷⁾	ASBS	USA	2019	Not published	1	Review	Not reported
53	Consensus Guideline on Breast Cancer Lumpectomy Margins⁽⁷⁸⁾	ASBS margins⁽⁷⁸⁾	ASBS	USA	2017	Not published	1	Review	Not reported
54	The American Brachytherapy Society consensus statement on intraoperative radiation therapy⁽⁷⁹⁾	AB intraoperative RT⁽⁷⁹⁾	AB	USA	2017	Brachytherapy	1	Nominal group technique	Not reported
55	ESTRO-ACROP guideline: Interstitial multi-catheter breast brachytherapy as Accelerated Partial Breast Irradiation alone or as boost - GEC-ESTRO Breast Cancer Working Group practical recommendations⁽⁸⁰⁾	ESTRO-ACROP RT⁽⁸⁰⁾	ESTRO	Europe	2018	RO	1	Consensus method; review	Not reported
56	ESTRO ACROP consensus guideline for target volume delineation in the setting of postmastectomy radiation therapy after implant-based immediate reconstruction for early stage breast cancer⁽⁸¹⁾	ESTRO-ACROP postmactectomy⁽⁸¹⁾	ESTRO	Europe	2019	RO	1	Consensus method; review	Not reported
57	Recommendations for hypofractionated whole-breast irradiation⁽⁸²⁾	SBRT RT⁽⁸²⁾	SBRT	Brazil	2018	RO	1	Consensus method, not specified technique	Not reported
58	Treating HR+/HER2- breast cancer in premenopausal Asian women: Asian Breast Cancer Cooperative Group 2019 Consensus and position on ovarian suppression⁽⁸³⁾	ABCCG BC⁽⁸³⁾	ABCCG	Asia	2018	BCRT	1	Consensus method; review	Not reported
59	International multidisciplinary expert panel consensus on breast reconstruction and radiotherapy⁽⁸⁴⁾	IMEP BR and RT⁽⁸⁴⁾	IMEP	Europe	2019	BJS	1	Consensus method; review	Not reported

Fig. 1

The flow diagram detailing the study selection.

Description of the CPGs and CSs (n = 167) selected for the systematic review. The flow diagram detailing the study selection.

Quality assessment

The analysis of the documents with the AGREE II instrument showed a wide overall score range (16–92%) (Fig. 2 and Appendix 5). The median overall quality was 54.0% (IQR 35.9–74.3). Only 13 (22%) of the CPGs or CSs were “recommended” as presented; the rest were not (19 (32%) “not recommended”, 27 (46%) “recommended with modifications”). Quality was heterogeneous in the domains (Appendix 5). In Domains 1 (scope and purpose) and 4 (clarity of presentation) 39 (66%) and 30 (51%) CPGs and CSs respectively scored >75%. In domain 5 (applicability) only 1 (2%) CPG scored >75%. Domain 6 (Editorial independence) related to the bias linked to conflict of interest, scored >75% in 34 (58%) CPGs but it was 0% or almost 0% in five CPGs [26,40,57,63,64,66] and four CSs [72,75,76,78,82]. The ASCO [[43], [44], [45], [46], [47], [48], [49], [50],52,53], Dutch [31] and Colombian [58] CPGs had the highest quality scores (Fig. 2, Appendix 6). For a better understanding of NICE guidelines, we studied the “Developing NICE guidelines: the manual” [85]. This led to a slight increase in the NICE CPGs scores, although it would be better if the relevant manual content were included in each NICE CPG itself. It is noteworthy that no specific methods are explained in the manual and this made it difficult to analyze the quality of the guidances.

Fig. 2

AGREE II overall score of BC CPGs and CSs.

Reporting assessment

CPGs and CSs reporting was heterogeneous and had a wide overall score range (16–89%) using the RIGHT statement (Fig. 3 and Appendix 7). The median overall reporting compliance was 62.5% (IQR 44.5–84.4). Only 5 (8%) of the CPGs and CSs were “well-reported”, 31 (53%) were “moderate-reported” and 23 (39%) were “low-reported”. Fig. 3 showed that reporting in domains was heterogeneous. The median of the domain scores was 67% (17–100%) for domain 1 (basic information), 63% (0–100%) for domain 2 (background), 60% (0–100%) for domain 3 (evidence), 50% (0–86%) for domain 4 (recommendations), 25% (0–75%) for domain 5 (review and quality assurance), 0 (0–19%) for domain 6 (funding and declaration and management of interests) and 50% (0–100%) for domain 7 (other information). The ASCO [46,[48], [49], [50]] and Dutch [31] CPGs had the highest reporting compliance (Appendix 8).

Fig. 3

RIGHT overall score of BC CPGs and CSs.

Variables related to quality and reporting

As shown in Table 2 CPGs scored better than CSs regarding quality (p = 0.032) and reporting (p = 0.005). CPGs from the USA had a better score than Europe and the rest of the world (AGREE II 75.7% vs 45.1% vs 55.1, p = 0.003; RIGHT 87.1% vs 55.5% vs 59.4, p = 0.015). The year of publication did not affect the quality (p = 0.791) or reporting (p = 0.718). Compared to consecutive updates of the CPG or CS, the second version when published within the review period had better quality (p = 0.001) and reporting (p = 0.002). Compared to subjective methods of evidence analysis, guidance documents that used systematic reviews had better quality than consensus (76.3% vs 51.4%; p = 0.001) and reporting (87.1% vs 59.4%; p = 0.001). CPGs and CSs published in a journal showed better quality (66.5% vs 42.0%; p = 0.001) and reporting (65.6 vs 50.4; p = 0.001) than those unpublished.

Table 2

Variables related to quality and reporting of CPGs and CSs.

	AGREE II			RIGHT
Variable	Median	IGQ Range	p value	Median	IGQ Range	p value
Type of document
CPGs	55.4%	44.6–76.5		67.18%	50.7–88.2
CSs	44.2%	32.2–60.9	p = 0.032	44.5%	30.1–63.7	p = 0.005
Country
USA	75.7%	48.9–76.8	p = 0.003	87.1%	59.3–93.0	p = 0.015
Europe	45.1%	34.4–53.9		55.5%	43.0–66.8
Other countries	55.1%	45.3–64.5		59.4%	44.5–76.5
Publication Year
2017	60.5%	46.4–75.4		71.9%	44.5–90.6
2018	48.3%	31.9–68.8		60.9%	35.9–76.6
2019	49.3%	37.0–75.2		58.2%	48.4–83.2
2020	53.9%	51.8–55.0	p = 0.791	60.9%	59.4–65.6	p = 0.718
Publication in a journal
Yes	66.5%	48.9–76.5		68.8%	59.4–89.8
No	42.0%	27.9–52.9	p = 0.001	46.8%	37.5–52.4	p = 0.001
Versión number
1	45.1%	32.1–60.8		50.4%	30.1–64.8
2	76.0%	55.8–76.8		87.1%	62.5–91.8
3 or more	45.3%	33.3–68.8	p = 0.001	65.6%	46.9–70.3	p = 0.002
Evidence analysis
Consensus	51.4%	35.9–56.5		59.4%	42.2–67.2
Not reported	38.0%	15.9–45.6		50.0%	28.9–52.3
Review	42.0%	27.9–72.5		60.9%	30.5–78.1
Systematic review	76.3%	69.7–77.2	p = 0.001	87.1%	75.0–92.9	p = 0.001
Quality tool referral
Reported	70.3%	69.2–89.5		83.6%	76.6–97.7
Not reported	52.5%	35.7–73.7	p = 0.073	60.9%	43.4–82.4	p = 0.065

Variables related to quality and reporting of CPGs and CSs.

Discussion

Main findings

The median overall quality and reporting of CPGs and CSs in BC treatment were poor. Around two-thirds of all guidance documents could not be recommended as written. Over three-quarters of all guidance documents were not well-reported. Compared to CSs, CPGs had better quality and reporting. Compared to subjective methods of evidence analysis, CPGs and CSs using systematic reviews and those published in a journal showed better quality and reporting. Compared to updates, the first iteration CPGs and CSs published within the review period had better quality and reporting.

Strengths and weaknesses

Our review had a global perspective with a reasonable number of CPGs and CSs identified using a comprehensive search without language restrictions. English and Spanish are the most widely spoken languages [86] and many of the Societies [[32], [33], [34]] present versions in both English and Spanish. One strength of this review is that the authors had command of both languages. We had a prospective protocol using two well-developed assessment tools, AGREE II instrument [16] and RIGHT statement [17], for as complete an assessment as possible. To our knowledge, an evaluation of guidance documents for BC treatment, using both AGREE II and RIGHT tools, has not been reported previously. While AGREE II instrument addresses different aspects of quality and RIGHT statement is a reporting tool, some items partially overlap. Our results suggest that reporting and quality are correlated. So reporting CPGs or CSs according to the RIGHT recommendations can lead to an increase in the AGREE scores, thus increasing the quality of the guidances. One presumed limitation of this review could be the subjective nature of data extraction concerning quality and reporting items. We minimized this issue by using two experienced BC specialist clinicians who studied the assessment tool manuals to create a mutual understanding of the scoring procedures before duplicate data extraction. Where concerns about major deviations arose, we used reviewer consensus backed by independent arbitration. It was reassuring to note that the reviewer agreement was excellent, with the ICC >95%. Our main findings have some provisos in that the overall assessments made might be limited because of the lack of clear rules about the weighting of domains and items in the quality and reporting scoring manuals [87]. Although RIGHT statement [17] recommends against deriving a score from the checklist (the items may not be equally weighted, and scores have been shown to be problematic in research synthesis), we found it useful for comparing CPGs and CSs. It also facilitated the comparison of quality with reporting. The AGREE II Consortium [16] and RIGHT team [17] have not preset the thresholds to differentiate between high, moderate, and poor quality and reporting. We used previously reported limits [23,24] to set the cut-offs for our analyses a priori. We are, therefore, confident that our main findings concerning poverty of guideline quality and reporting, and the negative impact of lack of systematic review for evidence synthesis are robust. These deficiencies merit urgent attention. We studied articles published from 2017 onwards. So, we are aware that guidance documents outside our time range from reputable organizations would have been excluded. There was heterogenicity amongst the guidelines included in the review. We only included those guidelines that fulfilled the inclusion criteria. This formal demonstration of heterogeneity in our review is in itself an important observation that merits consideration as a limitation of the existing guidances. However, this type of heterogeneity may be unavoidable as the guidances differ in their development, structure, context, endpoint definitions, etc. according to target users, both patients and clinicians [88].

Implications

Our review and analysis highlighted that the quality and reporting of the guidance documents in BC treatment has a wide space for improvement. This is especially obvious in domains concerning applicability and rigor of development in AGREE II. To increase the general quality of CPGs and CSs, there is a necessity of improvement in considering the potential resource implications of applying the recommendations, presenting monitoring and/or auditing criterion, and providing a procedure for updating the guideline (Appendix 9). In reporting using RIGHT, the domains in need of closer attention are basic information, background, the contrast of evidence of recommendations, and the declaration of interest and funders. There is a need of amelioration in adding new or key terms, a list of abbreviations and acronyms, in indicating whether the draft guideline underwent independent review or whether the guideline was subjected to a quality assurance process (Appendix 10). CPGs scored higher than CSs due to the fact their methods were better developed, and they more often deployed systematic reviews. Although the terms CPGs and CSs are often used interchangeably, they have differences that need to be highlighted. A clinical practice guideline produces statements that are informed by a systematic review of the evidence and an assessment of the benefits and harms of alternative options. A consensus statement is developed by an independent panel of experts, usually multidisciplinary, convened to review the research literature in an evidence-based manner for the purpose of advancing the understanding of an issuing procedure or method [89]. CSs are more likely to be sponsored by a pharmaceutical company and to endorse a specific product [89]. Unfortunately, transparency of document development was generally poor in both types of documents, and there was infrequent documentation of conflicts of interests, sources of funding, how guideline groups were established and who comprised their guideline development team. CSs are known to score lower than CPGs for scores of the rigor of development and editorial independence [89]. It is also necessary to highlight that CSs are intended for controversial areas of breast management (where the evidence is still incomplete), and the recommendations are based on experts’ perspectives. This brings in the notion of lower quality and broader risks of bias [89], which is relevant for the guidance based on consensus.[27,28,81] It is interesting that only 2 CPGs referred to AGREE II in the development of recommendations. The publication in a journal was associated with better quality and reporting. This could be due to reverse causality; however, every guidance should be submitted for publication in a peer-review journal. Our observations are that there is room for improvement that applies even to CPGs and CSs with high scores as all have some deficiencies. There remains a debate about cut-offs for defining acceptable scores and weighting of the items and domains. These issues should be subject to future research. In the current climate of formality and transparency, it should not be admissible that some CPGs or CSs do not even meet the basic quality and reporting criteria. These flaws will inevitably reduce the possibility of providing the best care to patients.

Conclusions

This systematic review found that CPGs and CSs for BC treatment insufficiently followed quality and reporting assessment tools. In the future, CPGs and CSs should take AGREE II and RIGHT into account to produce high-quality guidance documents underpinned by systematic reviews to ensure that recommendations are trustworthy. Focus on rigor in guidance development and practical advice concerning the application of recommendations in clinical setting is required for the implementation of evidence-based medicine to improve health outcomes.

Funding

Contributors

Each author certifies that he/she has made a direct and substantial contribution to the conception and design of the review, development of the search strategy, the establishment of the inclusion and exclusion criteria, data extraction, analysis, and interpretation. MMC was involved in the design of the review, literature search, data collection and analysis, quality appraisal, and writing. LM was involved in the development of data extraction, analysis, and writing. MMD was involved in the analysis of data. ABC was involved in the design of this review and provided critical revision of the paper. KSK was involved in the design of this review, conducted the quality appraisal, in the writing, and provided critical revision of the paper. All authors read and provided the final approval of the version to be published.

Financial support and sponsorship

None.

Data sharing statement

All the supplementary materials can be accessed upon request via email to the corresponding authors of this review.

Declaration of competing interest

The review was conducted in the University of Granada, Spain. There are no conflicts of interest.

63 in total

Review 1. [GEFPICS' guidelines for management of breast cancer tissue samples in the neoadjuvant setting].

Authors: Aurélie Maran-Gonzalez; Camille Franchet; Raphaëlle Duprez-Paumier; Martine Antoine; Catherine Barlier; Véronique Becette; Anca Berghian; Cécile Blanc-Fournier; Eva Brabencova; Emmanuelle Charafe-Jauffret; Marie-Pierre Chenard; Marie-Mélanie Dauplat; Paul Delrée; Clémence Fleury; Christian Garbar; Jean-Pierre Ghnassia; Juliette Haudebourg; Gaëtan MacGrogan; Marie-Christine Mathieu; Patrick Michenet; Frédérique Penault-Llorca; Bruno Poulet; Yves Robin; Pascal Roger; Elisabeth Russ; Isabelle Treilleux; Alexander Valent; Véronique Verriele; Anne Vincent-Salomon; Laurent Arnould; Magali Lacroix-Triki
Journal: Ann Pathol Date: 2019-06-27 Impact factor: 0.407

2. Integrative Therapies During and After Breast Cancer Treatment: ASCO Endorsement of the SIO Clinical Practice Guideline Summary.

Authors: Gary H Lyman; Kari Bohlke; Lorenzo Cohen
Journal: J Oncol Pract Date: 2018-08 Impact factor: 3.840

3. Evidence-Based Clinical Practice Guideline: Autologous Breast Reconstruction with DIEP or Pedicled TRAM Abdominal Flaps.

Authors: Bernard T Lee; Jayant P Agarwal; Jeffrey A Ascherman; Stephanie A Caterson; Diedra D Gray; Scott T Hollenbeck; Seema A Khan; Lauren D Loeding; Raman C Mahabir; Archibald S Miller; Galen Perdikis; Jaime S Schwartz; Beth A Sieling; Achilles Thoma; Judith A Wolfman; Jean L Wright
Journal: Plast Reconstr Surg Date: 2017-11 Impact factor: 4.730

4. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations.

Authors: J M Grimshaw; I T Russell
Journal: Lancet Date: 1993-11-27 Impact factor: 79.321

5. The American Brachytherapy Society consensus statement on intraoperative radiation therapy.

Authors: Martin C Tom; Nikhil Joshi; Frank Vicini; Albert J Chang; Theodore S Hong; Timothy N Showalter; Samuel T Chao; Suzanne Wolden; Abraham J Wu; Douglas Martin; Zain Husain; Shahed N Badiyan; Matthew Kolar; Tracy Sherertz; Firas Mourtada; Gilad N Cohen; Chirag Shah
Journal: Brachytherapy Date: 2019 May - Jun Impact factor: 2.362

6. ESTRO-ACROP guideline: Interstitial multi-catheter breast brachytherapy as Accelerated Partial Breast Irradiation alone or as boost - GEC-ESTRO Breast Cancer Working Group practical recommendations.

Authors: Vratislav Strnad; Tibor Major; Csaba Polgar; Michael Lotter; Jose-Luis Guinot; Cristina Gutierrez-Miguelez; Razvan Galalae; Erik Van Limbergen; Benjamin Guix; Peter Niehoff; Kristina Lössl; Jean-Michel Hannoun-Levi
Journal: Radiother Oncol Date: 2018-04-21 Impact factor: 6.280

Review 7. Evaluation of guidelines regarding surgical treatment of breast cancer using the AGREE Instrument: a systematic review.

Authors: Xin Lei; Fengtao Liu; Shuying Luo; Ya Sun; Liling Zhu; Fengxi Su; Kai Chen; Shunrong Li
Journal: BMJ Open Date: 2017-11-14 Impact factor: 2.692

Review 8. Novel Strategies on Personalized Medicine for Breast Cancer Treatment: An Update.

Authors: Carmen W H Chan; Bernard M H Law; Winnie K W So; Ka Ming Chow; Mary M Y Waye
Journal: Int J Mol Sci Date: 2017-11-15 Impact factor: 5.923

Review 9. Breast Cancer Treatment: A Review.

Authors: Adrienne G Waks; Eric P Winer
Journal: JAMA Date: 2019-01-22 Impact factor: 56.272

10. The Japanese Breast Cancer Society Clinical Practice Guidelines, 2018 edition: the tool for shared decision making between doctor and patient.

Authors: Hiroji Iwata; Shigehira Saji; Masahiko Ikeda; Masashi Inokuchi; Takayoshi Uematsu; Tatsuya Toyama; Rie Horii; Chikako Yamauchi
Journal: Breast Cancer Date: 2019-11-22 Impact factor: 4.239

5 in total

Review 1. Self-Managed Non-Pharmacological Interventions for Breast Cancer Survivors: Systematic Quality Appraisal and Content Analysis of Clinical Practice Guidelines.

Authors: Jing-Yu Benjamin Tan; Jianxia Zhai; Tao Wang; Hong-Juan Zhou; Isabella Zhao; Xian-Liang Liu
Journal: Front Oncol Date: 2022-05-30 Impact factor: 5.738

2. Quality and consistency of clinical practice guidelines for treating children with COVID-19.

Authors: Qinyuan Li; Qi Zhou; Yangqin Xun; Hui Liu; Qianling Shi; Zijun Wang; Siya Zhao; Xiao Liu; Enmei Liu; Zhou Fu; Yaolong Chen; Zhengxiu Luo
Journal: Ann Transl Med Date: 2021-04

3. International clinical practice guidelines for gender minority/trans people: systematic review and quality assessment.

Authors: Sara Dahlen; Dean Connolly; Isra Arif; Muhammad Hyder Junejo; Susan Bewley; Catherine Meads
Journal: BMJ Open Date: 2021-04-29 Impact factor: 2.692

4. A quality evaluation of the clinical practice guidelines on breast cancer using the RIGHT checklist.

Authors: Hanqiong Zhou; Haiyang Chen; Cheng Cheng; Xuan Wu; Yanfang Ma; Jing Han; Ding Li; Geok Hoon Lim; Warren M Rozen; Naohiro Ishii; Pankaj G Roy; Qiming Wang
Journal: Ann Transl Med Date: 2021-07

5. Quality assessment of clinical practice guidelines on psychological distress of cancer patients using the AGREE II instrument.

Authors: Ran Hao; Haoyu Jin; Jinfan Zuo; Rumeng Zhao; Jie Hu; Yixin Qi
Journal: Front Oncol Date: 2022-08-09 Impact factor: 5.738

5 in total