Quality of clinical practice guidelines about red blood cell transfusion.

BACKGROUND: Red blood cell (RBC) transfusions are essential in health care. The quality of recommendations included in clinical practice guidelines (CPG), regarding this intervention, has not been systematically evaluated. This paper systematically assessed CPGs for RBC-transfusion, to appraise their methodological quality, to explore changes in quality over time, and to assess the consistency of the hemoglobin threshold (HT) recommendations.
METHODS: We searched for CPGs that included recommendations of RBC-transfusion in generic databases, compiler entities, registries, clearinghouses and guideline developers. Three reviewers extracted data on CPGs characteristics and HT recommendations, independently appraised the quality of the studies using AGREE II and resolved disagreements by consensus.
RESULTS: We examined 16 CPGs. Mean scores (mean ± SD) were: scope and purpose (59.4% ± 19.8%), stakeholder involvement (43.2% ± 22.6%), rigor of development (50% ± 25%), clarity of presentation (74.4% ± 12.6%), applicability (19.4% ± 18.8%), and editorial independence (41% ± 30%). Seven CPGs recommended a restrictive strategy for RBC transfusion; four CPGs gave a guarded statement considering an HT of 7 g/dL, as safe to prescribe an RBC transfusion. Eight CPGs did not provide an HT stating that RBC transfusions should not be prescribed by HT alone.
CONCLUSIONS: Only 3 out of the 16 evaluated CPGs were "recommended" by the independent evaluators. Four domains "stakeholder involvement," "rigor of development," applicability," and "editorial independence" had serious shortcomings. Recommendations about the use of an HT for RBC-transfusion were heterogeneous among guidelines. Greater efforts are needed to provide high-quality CPGs in the RBC-transfusion practice.

INTRODUCTION

Blood transfusion is the infusion of both soluble and cell‐associated forms like RBCs, white blood cells, and platelets into a recipient.1 A blood transfusion is an acute intervention, implemented to solve life and health‐threatening conditions on a short‐term basis.2, 3 However, RBCs and other blood components therapies have been associated with several adverse clinical events, and require physicians to be fully informed of the risks and benefits.4, 5 Several strategies for preventing adverse events caused by RBC‐transfusions have been studied; however, their clinical effectiveness has not yet sufficiently demonstrated.2, 3, 6, 7, 8, 9, 10, 11

About 85 million people are transfused annually, with considerable variation in the use of RBC‐transfusion practices worldwide.6 In spite of the efforts to standardize transfusion practice, as the publication of clinical practice guidelines, this variability in transfusion practices has persisted. For instance, while some CPGs have included recommendations focused on hemoglobin concentration to guide RBC‐transfusion, other CPGs emphasize that transfusions should be provided in the presence of anemia symptoms and should not be based on hemoglobin concentration only.2, 5, 8, 9

CPGs are defined as systematically developed statements to assist practitioner and patient decisions about appropriate health care.12 International organizations have introduced and promote standards for the development of CPG, such as the Institute of Medicine (IOM),13 World Health Organization (WHO),14 National Institute for Health and Clinical Excellence (NICE),15 Scottish Intercollegiate Guideline Network (SIGN),16 and Guidelines International Network (G‐I‐N).17 All these efforts provide resources to assist guideline developers in producing high‐quality recommendations.  Despite these initiatives, the quality of the CPGs and the adherence to methodological guidelines has been improved only lightly in the last decade.18, 19, 20, 21

In the field of RBC‐transfusion, a large body of clinical evidence has been generated; resulting in the publication of many CPGs.22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40 These CPGs face with inconsistent recommendations that potentially result in confusion among clinicians, and the quality of the guidelines could be put to question. For these reasons, there is a need to assess the methodological quality of the CPGs in this field, to explain the variability of the recommendations. We conducted a systematic assessment of CPGs for RBC‐transfusion, to appraisal their methodological quality using AGREE II tool, and to explore changes in quality over time, and to evaluate the consistency of hemoglobin concentration recommendations to guide transfusion.

METHODS

Data search

We searched for CPGs that included recommendations of RBC‐transfusion in generic databases, compiler entities, registries, clearinghouses and guideline developers. We used free terms such as red blood cell transfusion, blood transfusion, anemia, and erythrocyte cells for these searches. For the MEDLINE search, via PubMed, we combined MeSH terms (“blood transfusion,” “erythrocytes,” “Erythrocyte Transfusion,” “blood component transfusion,” “anemia”) and free terms (transfus* [tiab], transfusion requirements, RBC, RBCs, transfusion strategy, blood loss, blood conservation, transfusion of RBCs, red cell transfusion, management of anemia). Additionally, we used a series of terms related to guidelines as: “practice guideline,” “consensus,” “development conference,” and “guideline.” The search strategy and sources are listed in Table 1.

Table 1

Searched sites for the identification of CPGs

Generic databases	Websites
MEDLINE (PubMed)	http://www.ncbi.nlm.nih.gov/pubmed/
TRIP database	http://www.tripdatabase.com
Excelencia Clínica	http://www.excelenciaclinica.net/
Compiler Entities, Registries, or Clearinghouses	Websites
National Guidelines Clearinghouse	http://www.guideline.gov/
Agency for Healthcare Research and Quality	http://www.ahrq.gov/
Biblioteca de Guías de Práctica Clínica del Sistema Nacional de Salud	http://www.guiasalud.es
Canadian Medical Association Infobase: Clinical Practice Guidelines	http://www.cma.ca
Guidelines Developers	Websites
National Institute for Health and Care Excellence	http://www.nice.org.uk
Scottish Intercollegiate Guidelines Network	http://www.sign.ac.uk
New Zealand Guidelines Group	http://www.nzgg.org.nz
Institute for Clinical Systems Improvement	http://www.icsi.org
American College of Physicians	http://www.acponline.org
International Society of Blood Transfusion	http://www.isbtweb.org/
Asian Association of Transfusion Medicine (AATM)	http://saatm.org/
Australian and New Zeland Society of Blood Transfusion	http://www.anzsbt.org.au/
British Blood Transfusion Society	https://www.bbts.org.uk/
American Red Cross	http://www.redcross.org/

Inclusion and exclusion criteria

We included (1) CPGs with recommendations related to hemoglobin concentration to guide the RBC‐transfusion; (2) CPGs that performed a search in at least one database; and (3) CPGs published from 2006 until October 2017, in English or Spanish. We excluded (1) secondary publications like systematic reviews or meta‐analyses and (2) CPGs with recommendations about pediatric patients (<15 years) and neonates.

Data collection

Two reviewers independently screened abstracts using the inclusion criteria stated above. If the inclusion criteria met, we retrieved the full‐text article and screened it to determine their eligibility. Two reviewers independently extracted the following data from each CPG: title, year, organization that developed the guideline, country of origin, and source of funding. In the case of disagreement, a third reviewer was consulted. One reviewer extracted the recommendation about hemoglobin threshold to guide transfusion, and the individual studies used to support the recommendation.

Quality assessment

We used the AGREE instrument to evaluate the quality of the included CPGs.41, 42, 43, 44 This was developed primarily for guideline developers and researchers, to outline and measure the core elements of guideline development and implementation. The AGREE instrument (initially AGREE I, now AGREE II) contains 23 items,41 spread over six domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence, in addition to a final general item that evaluates the extent to which the guideline can be recommended for use in practice. To evaluate the items within the six domains, a 7‐point Likert scale was used, ranging from “strongly disagree” to “strongly agree.” For the overall assessment, we used a 3‐point scale ranging from “not recommended” to “strongly recommended.” Three independent reviewers, with experience in CGs assessment, applied the AGREE II instrument. In the case of disagreement, an agreement was reached by consensus. In the event of persistent disagreement, a fourth evaluator was consulted.

Statistical analysis

We performed a descriptive analysis of the CPGs according to the country of origin, the type of organization that developed them, the year of publication and the language of the CPGs. To establish the quality of each CG, the standardized score was calculated as a percentage; this was obtained by adding all the individual points from the items of a domain, and standardizing the total as a percentage of the maximum possible score from that area: (score obtained − minimum possible score)/(maximum possible score − minimum possible score) × 100. Once the quality of each CG was established, it was compared to the aforementioned descriptive variables. The degree of agreement between the reviewers was assessed using an intraclass correlation coefficient (ICC) with a 95% confidence interval (CI). Student's t‐test compared the scores between different variables (date of publication and restrictive recommendations). For the analysis of the change in the global score over time, the date of publication was categorized into two periods (2006–2011 and 2012–2015). We used the statistical package IBM SPSS (version 22).

RESULTS

Guideline characteristics

The search strategy provided 615 references after eliminating duplicates. A review of the titles and abstracts identified 47 potentially eligible CPGs. From the 47 examined CPGs, only 16 fulfilled the eligibility criteria and were included (Table 2).22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40 One of these guidelines included four chapters that give the hemoglobin threshold recommendation for different settings.34, 35, 36, 37 Included guidelines were published from 2008 to 2016. Six CPGs were from the United States,23, 28, 29, 33, 40 four from the United Kingdom,22, 31, 38, 39 one with four chapters from Australia,34, 35, 36, 37 one from Canada,24 Finland,27 the Netherlands,26 Singapore,25 and Spain.30 Twelve documents were developed by scientific societies,22, 23, 24, 27, 28, 29, 30, 31, 32, 33, 39, 40 and seven CPGs were developed by government agencies.25, 26, 34, 35, 36, 37, 38 Five of 16 included CPGs, focused solely on RBC‐transfusion,22, 23, 24, 31, 40 while there remaining 11 gave recommendations on blood products in general.25, 26, 27, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 39

Table 2

CPG characteristics and hemoglobin threshold recommendations

Guideline	Organization	Year	Hemoglobin threshold recommendation	Country and language	Methods used to assess the quality and strength of the evidence
Blood transfusion and the anesthetists. Red cell transfusion22	The Association of Anaesthetists of Great Britain and Ireland	2008	“The decision to transfuse should always be made on an individual patient basis. Patients should not normally be transfused if the hemoglobin concentration is >10 g/dL. A strong indication for transfusion is a hemoglobin concentration <7 g/dL″ (Perioperative patients)	UK/IR, English	Grading system not stated
Clinical practice guideline: red blood cell transfusion in adult trauma and critical care23	The Eastern Association for Surgery of Trauma Practice Management Workgroup	2009	“The use of only Hb level as a ‘trigger’ for transfusion should be avoided. A ‘restrictive’ strategy of RBC transfusion (transfuse when Hb 7 g/dL) is as effective as a “liberal” transfusion strategy (transfusion when Hb 10 g/dL) in critically ill patients with hemodynamically stable anemia, except possibly in patients with acute myocardial ischemia” (Critically ill patients)	USA, English	Canadian and US Preventative Task Force grading system
Guidelines for red blood cell and plasma transfusion for adults and children. updated24	Guidelines for Canadian Clinical Practice Guidelines. Expert Panel	2009	“Red blood cell transfusion should not be dictated by a single hemoglobin trigger but should be based on a complete evaluation of the patient including volume status, tissue perfusion and comorbid disease” (General medical patients)	Canada, English	Modified version of the Canadian Task Force on the Periodic Health Examination grading system
Clinical blood transfusion25	Singapore Ministry of Health	2011	“When hemoglobin >10 g/dL, there is usually very little indication for red cell transfusion. When hemoglobin <7 g/dL, red cells transfusion may be beneficial particularly in symptomatic patients or ongoing blood loss is expected” (General medical patients)	Singapore, English	Own rating scheme used to assess the quality of the evidence
Blood transfusion guideline26	Dutch Institute for Healthcare Improvement	2011	“The indication for administering erythrocytes is based on medical factors and is aimed at treating or preventing the symptoms of a lack of oxygen transport capacity by the blood. Consider a transfusion if the following occurs at a Hb < 4 mmol/L: acute blood loss in a healthy individual (ASA I) < 60 years, normovolemic, blood loss at 1 location” (General medical patients)	The Netherlands, English‐Dutch	Own rating scheme used to assess the quality of the evidence
Blood transfusion: indications, administration, and adverse reactions24	Finnish Medical Society Duodecim	2011	“It is not possible to give single hemoglobin (Hb) value as a trigger for red cell transfusion since the requirement for a transfusion is based on anemia symptoms, the patient's age, and the underlying diseases (chronic or slowly developing anemia)” (General medical patients)	Finland, English	Rating scheme modified of GRADE 2011 by the EBM Guidelines Editorial Team
2011 update to The Society of Thoracic Surgeons and the Society of Cardiovascular Anesthesiologists Blood Conservation Clinical Practice Guidelines28	The Society of Thoracic Surgeons and the Society of Cardiovascular Anesthesiologists	2011	“With hemoglobin levels below 6 g/dL, red blood cell transfusion is reasonable since this can be life‐saving. Transfusion is reasonable in most postoperative patients whose hemoglobin is less than 7 g/dL, but no high‐level evidence supports this recommendation” (Perioperative patients)	USA, English	The assessment was conducted according to the level of evidence recommended by the AHA/ACCF Task Force on Practice Guidelines
KDIGO Clinical Practice Guideline for anemia in chronic kidney disease29	Kidney Disease: Improving Global Outcomes (KDIGO)	2012	“We suggest that the decision to transfuse a CKD patient with nonacute anemia should not be based on any arbitrary Hb threshold, but should be determined by the occurrence of symptoms caused by anemia” (Chronic kidney diseases patients)	USA, English	KDIGO grading system
The “Seville” document on consensus on the alternatives to allogenic blood transfusion30	The Spanish Societies of Anesthesiology, Critical Care Medicine and Coronary Units, Hematology and Hemotherapy, Blood Transfusion and Thrombosis and Hemostasis	2013	“The majority of trauma, critical and surgical patients can tolerate hemoglobin levels of 70 g/L. However, if they present acute cardiological and/or central nervous system involvement, hemoglobin levels of at least 80 g/L may be required. In any case, the decision to transfuse should be individualized for each patient” (General medical patients)	Spain, Spanish	GRADE 2008
Guidelines on the management of anemia and red cell transfusion in adult critically ill patients31	British Committee for Standards in Haematology	2013	“A transfusion threshold of 70 g/L or below, with a target Hb range of 70–90 g/L, should be the default for all critically ill patients, unless specific comorbidities or acute illness‐related factors modify clinical decision‐making” (Critically ill patients)	UK, English	GRADE 2008
Treatment of anemia in patients with heart disease: a clinical practice guideline from the American College of Physicians32	American College of Physicians	2013	“ACP recommends using a restrictive red blood cell transfusion strategy (trigger hemoglobin threshold of 7 to 8 g/dL compared with higher hemoglobin levels) in hospitalized patients with coronary heart disease”(Patients with heart disease)	USA, English	ACP's clinical practice guidelines grading system
Practice guidelines for perioperative blood management—an updated report by the American Society of Anesthesiologists Task Force on Perioperative Blood Management33	American Society of Anesthesiologists	2014	“A restrictive red blood cell transfusion strategy may be safely used to reduce transfusion administration” (Perioperative patients)	USA, English	ASA grading system
The National Blood Authority's Patient Blood Management Guideline: Module 2—Perioperative34	National Blood Authority Australia	2012	“RBC transfusion should not be dictated by a hemoglobin ‘trigger’ alone but should be based on an assessment of the patient's clinical status” (Perioperative patients)	Australia, English	National Blood Authority scheme
The National Blood Authority's Patient Blood Management Guideline: Module 3—Medical35	National Blood Authority Australia	2012	“RBC transfusion should not be dictated by a Hb concentration alone, but should also be based on an assessment of the patient's clinical status. Direct evidence is not available in general medical patients” (General medical patients)	Australia, English	National Blood Authority scheme
The National Blood Authority's Patient Blood Management Guideline: Module 4—Critical Care36	National Blood Authority Australia	2012	“In critically ill patients, a restrictive transfusion strategy should be employed” (Critically ill patients)	Australia, English	National Blood Authority scheme
The National Blood Authority's Patient Blood Management Guideline: Module 5—Obstetrics and Maternity37	National Blood Authority Australia	2015	“In maternity patients who are not actively bleeding, RBC transfusion should not be dictated by a Hb concentration alone, but should also be based on an assessment of the patient's clinical status (eg, the risk of further hemorrhage)” (Obstetric patients)	Australia, English	National Blood Authority scheme
Blood Transfusion NICE guideline38	National Institute for Health and Care Excellence	2015	“Use restrictive red blood cell transfusion thresholds for patients who need red blood cell transfusions and who do not: have major hemorrhage, or have the acute coronary syndrome, or need regular blood transfusions for chronic anemia” (General medical patients)	UK, English	GRADE
Blood transfusion in obstetrics39	Royal College of Obstetricians & Gynaecologists	2015	“There are no firm criteria for initiating red cell transfusion. The decision to provide blood transfusion should be made on clinical and hematological grounds” (Obstetric patients)	UK, English	Scheme using Royal College of Obstetricians and Gynaecologists for grading recommendations
Clinical practice guidelines from the AABB: red blood cell transfusion thresholds and storage40	American Association of Blood Banks	2016	“The AABB recommends a restrictive RBC transfusion threshold in which the transfusion is not indicated until the hemoglobin level is 7 g/dL for hospitalized adult patients who are hemodynamically stable, including critically ill patients, rather than a liberal threshold when the hemoglobin level is 10 g/dL” (General medical patients)	USA, English	GRADE

Eight guidelines gave recommendations for general medical patients.24, 25, 26, 27, 30, 35, 38, 40 The other 11 CPGs focused on specific populations: four on perioperative patients22, 28, 33, 34; three on critically ill patients23, 31, 36; two on obstetric patients37, 39; one on patients with heart disease32; and one on chronic kidney diseases patients.29

For the analysis of recommendations, each chapter of one of the CPGs included,34, 35, 36, 37 were considered separately due to differences in hemoglobin threshold recommendation. Seven guidelines recommended a restrictive strategy for RBC transfusion,30, 31, 32, 33, 36, 38, 40 defined as the administration of blood transfusion when the hemoglobin level falls below 7 g/dL. Four CPGs had a guarded statement considering a hemoglobin threshold of 7 g/dL, as safe to indicate RBC transfusion.22, 25, 26, 28 The remaining eight CPGs avoid giving a hemoglobin threshold23, 24, 27, 29, 34, 35, 37, 39 and state that RBC transfusion should not be dictated by hemoglobin concentration alone (Table 2).

Overall, 39 clinical trials supported these recommendations. The references of included studies to base the recommendations were not possible to obtain in two guidelines.24, 27 Another two CPGs supported their recommendations in previously published guidelines37, 39 (see supplementary material for information about the evidence supporting recommendations). Only four guidelines used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology to evaluate the quality of evidence and to grade the strength of recommendations.30, 31, 38, 40 One guideline used a modified version of GRADE,27 13 guidelines applied other methods to determine the quality of the evidence,23, 24, 25, 26, 28, 29, 32, 33, 34, 35, 36, 37, 39 and 1 guideline did not explain the methodology used to assess the quality of evidence.22

The agreement between the three reviewers was high, with an ICC of 0.90 (95% CI: 0.81‐0.96). Table 2 shows the standardized score of the AGREE II tool by domain and by guideline, as well as the overall evaluation. Additionally, Figure 1 shows the statistical summarized analysis of the total standardized score by domain.

Figure 1

Distribution of the standardized domain scores for 16 CPGs. The top and bottom of the box represent the 75th (Q3) and 25th percentile (Q1), respectively, and the band near the middle of the box indicates the 50th percentile (median). The upper and lower ends of the whisker represent Q3 + 1.5× (interquartile range), and Q1‐1.5× (interquartile range), respectively

Domain 1: scope and purpose

This domain focuses on the general goal of the CPGs, considering the health condition, and the specific population for applying the guideline. The average score was 59.4% (median = 62% and a range from 22.2% to 87%; Figure 1). Five CPGs (31%) scored above 70%.23, 30, 32, 34, 38 See Table 2 for details about Domain 1.

Domain 2: stakeholder involvement

This domain assesses the working group that developed the CPGs, the involvement of stakeholders, and potential users. The average score was 43.2% (median = 40% and a range from 13% to 78%; Figure 1). Only three CPGs (18.7%) scored more than 70 on this domain.26, 29, 38 See Table 3 for details about Domain 2.

Table 3

AGREE II domain‐standardized scores

Guideline	Scope and purpose	Stakeholder involvement	Rigor of development	Clarity of presentation	Applicability	Editorial independence	Overall recommendation
Blood transfusion and the anesthetists. Red cell transfusion22	35.2	24.1	9	53.7	8.3	0	Not recommended
Clinical practice guideline: red blood cell transfusion in adult trauma and critical care23	83.3	33.3	54.2	92.6	11.1	55.6	Recommended with modifications
Guidelines for red blood cell and plasma transfusion for adults and children. updated24	22.2	13	9	51.9	1.4	5.6	Not recommended
Clinical blood transfusion25	55.6	42.6	36.8	87	22.2	0	Not recommended
Blood transfusion guideline26	64.8	77.8	79.2	75.9	52.8	36.1	Recommended with modifications
Blood transfusion: indications, administration and adverse reactions27	35.2	14.8	16	68.5	0	8.3	Not recommended
2011 update to The Society of Thoracic Surgeons and the Society of Cardiovascular Anesthesiologists Blood Conservation Clinical Practice guidelines28	27.8	50	45.1	64.8	9.7	33.3	Recommended with modifications
KDIGO Clinical Practice Guideline for anemia in chronic kidney disease29	66.7	74.1	80.6	87	34.7	72.2	Recommended
The “Seville” document on consensus on the alternatives to allogenic blood transfusion30	72.2	37	37.5	75.9	13.9	41.7	Recommended with modifications
Guidelines on the management of anemia and red cell transfusion in adult critically ill patients31	61.1	24.1	36.8	74.1	18.1	8.3	Not recommended
Treatment of anemia in patients with heart disease: a clinical practice guideline from the American College of Physicians32	77.8	35.2	61.8	92.6	0	75	Recommended with modifications
Practice guidelines for perioperative blood management—an updated report by the American Society of Anesthesiologists Task Force on Perioperative Blood Management33	63	48.1	52.1	68.5	1.4	55.6	Not recommended
The National Blood Authority's patient blood management guideline34 – 37	79.6	31.1	81.9	81.5	54.2	86.1	Recommended
Blood transfusion NICE guideline38	87	74.1	86.8	85.2	51.4	83.3	Recommended
Blood transfusion in obstetrics39	59.3	13	54.2	63	18.1	38.9	Not recommended
Clinical practice guidelines from the AABB: red blood cell transfusion thresholds and storage40	31.1	68.5	59	68.5	13.9	55.6	Recommended with modifications
Mean score (SD)	59.5 (±19.9)	43.1 (±22)	50 (±25.1)	74.4 (±12.7)	19.4 (±18.8)	41(±30)
Median (range)	62 (22.2–87)	39.8 (13–77.8)	53 (9–86.8)	75 (52–92.6)	13.8 (0–54.2)	40.2 (0–86.1)

Domain 3: rigor of development

This domain addresses the process used to identify and summarize the evidence, the methodology to formulate recommendations, and their updates. The average score was 50% (median = 53% and a range from 9% to 87%; Figure 1). Four CPGs (25%) scored above 70% on this domain.26, 29, 34, 38 See Table 3 for details about Domain 3.

Domain 4: clarity and presentation

This domain focuses on the wording, the structure, and the general format of the CPGs. The average score was 74.4% (median = 75% and a range from 51% to 92.6%; Figure 1). Nine CPGs (56.2%) scored above 70% on this domain.23, 25, 26, 29, 30, 31, 32, 34, 38 This domain scored the highest among the six domains included in the AGREE II instrument. See Table 3 for details about Domain 4.

Domain 5: applicability

This domain considers the barriers and facilitators for the implementation of the CPGs, including aspects of resources and adherence to the recommendations. The average score was 19.4% (median = 14% and a range from 0% to 54.2%; Figure 1). This was the lowest evaluated domain for all the CPGs, and none of the included CPGs scored above 70% on this domain. See Table 3 for details about Domain 5.

Domain 6: editorial independence

This domain assesses if funding sources influenced recommendations. The average score was 41% (median = 40% and a range from 0% to 86%; Figure 1). Four CPGs (25%) scored above 70% on this domain.29, 32, 34, 38 See Table 3 for details about Domain 6.

Overall assessment

Three out of the 16 evaluated CPGs (18.7%) were “recommended” by the independent evaluators,29, 34, 38 6 CPGs (37.5%) were “recommended with modifications,”23,26,28,30,32,40 and 7 CPGs (43.7%) were “not recommended” (see Table 3).22, 24, 25, 27, 31, 33, 39 The three “recommended” CPGs scored ≥ 70% in the “rigor of development” domain. The seven CPGs (18.7%) “not recommended”22,24,25,27,31,33,39 by evaluators had scores below 70% in five of the six reported domains (see Table 3).

We did not find statistically significant differences in the AGREE II global score between CPGs published in 2006‐2011 and those published in the period 2012–2015 (P = 0.49). Additionally, those CPGs recommending restrictive strategies scored similarly in the rigor of development domain, as those that did not recommend a specific threshold (P = 0.92).

DISCUSSION

Summary of the main finding

In our review, we found 16 CPGs that met the eligibility criteria.22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40 In the overall CPGs’ assessment, only 3 out of the 16 evaluated CPGs (18.7%) were “recommended” by the independent evaluators,29, 34, 38 6 CPGs (37.5%) were “recommended with modifications,”23, 26, 28, 30, 32, 40 and 7 CPGs (43.7%) were “not recommended.”22,24,25,27,31,33,39

Most of the CPGs did not describe the literature search and selection methods, and they were ambiguous regarding how the evidence was appraized and whether or not the recommendations were truly evidence‐based. The domains with the highest scores were “clarity and presentation” and “scope and purpose,” and the domains with the lowest scores were “applicability” and “editorial independence” (see Table 3 and Figure 1). Only four CGs26, 29, 34, 38 scored ≥ 70% in the domain “rigor of development,” which was considered one of the most critical domains, as it refers to methodological aspects concerning how the recommendations were developed.

In the analyzed CPGs, the use of a hemoglobin threshold for RBC‐transfusion was variable. Some guidelines recommended restrictive strategies, and other CPGs avoided using a hemoglobin threshold, on the basis that RBC‐transfusion should not be dictated by hemoglobin concentration alone. However, when the score in the rigor of development domain, of the CPGs recommending restrictive strategies, as compared with the CPGs that avoid giving a hemoglobin threshold, we did not find statistically significant differences. Therefore, the variability in recommendations cannot be explained by differences in this domain (P = 0.92).

Finally, our study could not demonstrate statistical differences over time in the global score of CPGs quality (published in 2006–2011 versus 2012‐2016; P = 0.49). However, we believe that the low number of included CPGs did not allow an adequate evaluation of the variability in the quality of RBC‐transfusion CGs over time.

The context of this review with other literature

This review represents the first systematic assessment of the quality of clinical practice guidelines focused on red blood cell transfusion recommendations. Consistently with previous CPG evaluations in other clinical areas,45, 46, 47, 48 the domains with the highest scores were “clarity of presentation” and “scope and purpose,” whereas the domains with the lowest scores were “stakeholder involvement,” “editorial independence,” and “applicability.” The lowest scores related to the “applicability” domain can be related to the belief that the activity of formulating recommendations was separated from the implementation processes. Our results for the domains were similar to those of previous systematic assessment done by our group, that included the evaluation of 626 CPGs.19 Specifically, in the “rigor of development” domain our review found low quality, with an average of 46.3% compared to 68% in other similar reviews.19

Strengths and limitations

Our systematic assessment has some limitations. First, although a robust set of search criteria was formulated and tested prior to full guideline identification, some CPGs might not have been adequately indexed as they were only used for institutional purposes, so we failed in their identification. We think that the quality of the CPGs not indexed in biomedical databases is probably lower compared to those indexed. Second, there is also a potential risk of selection bias because we included only studies that had been published in English or Spanish. To this extent, our assessment could be overestimating the quality of CPGs in RBC‐transfusion. Third, the AGREE II41, 42, 43, 44 instrument has undergone some revisions since the development of the original AGREE instrument.41 A 7‐point scale is used instead of a 4‐point scale for evaluating the items in the domains. This may have been a limitation in assessing the quality of the CPGs because the only well‐defined points in the scale are 1 and 7. We found that the evaluators had difficulty in distinguishing between 3, 4, and 5 Likert values, which may have introduced a potential risk of reporting bias. However, the agreement among reviewers using the AGREE II instrument was high, with an ICC of 0.97.

On the other hand, we recognize some strengths of this systematic assessment. First, we are the first to assess the quality of development of clinical practice guidelines focused on red blood cell transfusion recommendations using methodological instruments that are widely recognized and accepted. Second, the uses of extensive search strategies, covering both indexed and gray literature and the use of expert appraisers who completed training and calibration to assess the quality of CPGs.

In conclusion, our findings show that much remains to be done to reach excellence in the area of CPGs on RBC‐transfusion. Only three out of the 16 evaluated CPGs were “recommended” by the independent evaluators. Four domains (“stakeholder involvement,” “rigor of development,” “applicability,” and “editorial independence”) had serious shortcomings. The domains: “scope and purpose” and “clarity of presentation” were the more precisely reported.

Moreover, our study could not demonstrate statistical differences over time in the global score of CPGs quality (published in 2006‐2011 versus 2012‐2016; P = 0.49). Also, the recommendations about hemoglobin threshold for RBC‐transfusion was variable among the CPGs analyzed.

Clinical practice guidelines users should be aware of the low quality reported in this study. Meanwhile, developers should adhere to rigorous methods, like those provided in handbooks from institutions, such as the Scottish Intercollegiate Guidelines Network (SIGN) or the National Institute for Health and Care Excellence (NICE).15 Additionally, guideline developers should use checklists to optimize methods for the development and reporting of CPG, such as the GIN‐McMaster Guideline Development Checklist (GDC)49, 50 and AGREE II,42, 43, 44 respectively. Moreover, to improve communication among users and developers, GRADE methodology is highly recommended, as it is a widely implemented rigorous system.

Regarding research strategies, additional efforts should be made to develop and consolidate networks, to improve the evaluation and synthesis of the available evidence in the RBC‐transfusion field. Researchers, who wish to identify knowledge gaps, and policy‐makers, looking to endorse adequate CPG development, should work together to ensure the adherence to recommendations related to RBC‐transfusion, and minimize the heterogeneity in clinical practice.

CONFLICTS OF INTEREST

None