Assessing the Quality of Existing Clinical Practice Guidelines for Chemotherapy Drug Extravasation by Appraisal of Guidelines for Research and Evaluation II.

BACKGROUND: Extravasation is a potentially hazardous event that may occur during chemotherapy. The aim of this study is to assess the quality of existing Clinical Practice Guidelines (CPGs) for chemotherapy drug extravasation by Appraisal of Guidelines for Research and Evaluation II (AGREE II).
MATERIALS AND METHODS: Valid electronic databases and CPGs from 2007 to August 2018 were searched by keywords of CPGs, extravasation, chemotherapy, and cancer. CPGs were evaluated independently by five experts through AGREE II tool, and the consensus among evaluators was calculated by ICC (Intra-class Correlation Coefficient).
RESULTS: Five of the 111 CPGs matched the inclusion criteria. The methodological quality of CPGs in domains of "scope and purpose," "stakeholder involvement," "clarity of presentation," and "applicability" were good, in the domain of "rigor of development," was acceptable, and in "editorial independence" domain, it needed more attention of developers of CPGs. The range of assessors' consensus was within a range of moderate to very good (0.55--0.93).
CONCLUSIONS: The methodological quality of existing CPGs of chemotherapy drugs extravasation assessed by AGREE II tool is appropriate. Four CPGs had high level while one had moderate level of quality. Therefore, their use is recommended in the clinic to reduce the risk of chemotherapy extravasation to the entire treatment team and the nurses working in the oncology departments. Copyright:

Introduction

Extravasation is a serious and important issue in chemotherapy, occurring in 1 to 7% of cases.[1] This complication may expose not only the patients but also the nurses and caregivers to its harmful effects.[2] As in the success of managing chemotherapy and preventing complications, especially extravasation, nurses' knowledge and competence play a vital role, they should have adequate knowledge in this regard and follow evidence-based recommendations and Clinical Practice Guidelines (CPGs).[345] Organizations, institutions, associations, and cancer groups have developed several CPGs in this regard. CPGs play an important role in clinical practice; they offer valuable recommendations, based on the highest level of evidence.[6] However, CPGs may vary widely in quality.[7] The quality of CPGs refers to being sure that the probable bias in the process of development of the guidelines is well prevented, the internal and external validity of the recommendations are provided, and recommendations are applicable in clinical setting.[8] Expanding the number, complexity, and heterogeneity of CPGs and the concerns about their quality have led to an increase in the need for development of internationally recognized criteria to ensure the quality of CPGs[910] because it is important to evaluate the methods on which a guideline is developed in order to be confident of its recommendations.[7] To do this, different quality appraisal instruments have been developed for evaluating guidelines. Among these, Appraisal of Guidelines for Research and Evaluation II (AGREE II) is reported to be a reliable, internationally used and validated tool.[11]

The AGREE II instrument has been approved by several organizations including the World Health Organization.[12131415] Comprehensive guidelines of high quality are expected to increase adherence among health workers, reduce individual decisions lacking evidence, and improve the provision of care.[16] So far, no study has been conducted to assess the quality of CPGs for chemotherapy drugs extravasation. However, studies have been conducted to evaluate the quality of CPGs in other sectors.[16171819202122232425] Therefore, the aim of this study was to assess the quality of CPGs for chemotherapy drug extravasation by use of AGREE II to be applied in cancer patients' clinical setting to improve evidence-based decision-making and reduce the risk of extravasation among the patients and the treatment team.

Materials and Methods

We conducted a review of published CPGs for the management of cancer therapy-induced extravasation. Selection of CPGs was conducted through a systematic search on valid databases such as Scopus, PubMed, Proquest, Cochrane Library, MEDLINE, the web of science, and CPGs editorial bases including Guidelines International Network (GIN), National Institute for Clinical Excellence (NICE), The Cancer Care Ontario (CCO), the National Guidelines Clearinghouse (NGC), and the Scottish Intercollegiate Guidelines Network (SIGN) from 2007 to August 2018 (professional practice guidelines published during the last 10 years as up to date knowledge was considered necessary)[26272829] by a combination of Mesh keywords including CPGs, extravasation, chemotherapy, and cancer. In this systematic search, the word OR was used to replace the keywords, and the word AND, to combine different keywords together [Table 1]. Inclusion criteria for CPGs covered all guidelines for chemotherapy extravasation, in English, developed by institutes, associations, and cancer groups, based on systematic and evidence-based review (quality of evidence was reviewed). The latest updated version of the guideline for assessment was also selected and for the guidelines, published in several forms, the one with the most details about the development methodology was selected. The CPGs whose access to their full text was impossible and other cases such as service packs, care plans, systematic reviews, patients' guides, and the books were excluded.

Table 1

Search keywords and their combinations

Keywords	Combination
Guideline, Extravasation, chemotherapy, cancer	“practice guidelines” OR “clinical practice guideline” OR “Best Practices” OR “Best Practice” OR guideline?	Extravasation of Diagnostic and Therapeutic Materials OR “Injection Site Reactions” OR “Injection Site Event” OR “Injection Site Events” OR “ Injection Site Adverse Event” OR “Infusion Site Reaction” OR “Infusion Site Reactions” OR “Infusion Site Adverse Reaction” OR “Infusion Site Adverse Event”	“Drug therapy” OR “Therapy Drug” OR “Drug Therapies” OR “Therapies Drug” OR Chemotherapy OR Chemotherapies OR Pharmacotherapy OR Pharmacotherapies OR Neoplasms OR Neoplasia OR Neoplasias OR Neoplasm OR Tumors OR Tumor OR Cancer OR Cancers OR “Malignant Neoplasms” OR “Malignant Neoplasm” OR “Neoplasm Malignant” OR “Neoplasms Malignant” OR Malignancy OR Malignancies OR “Benign Neoplasms” OR “Neoplasms Benign” OR “Benign Neoplasm” OR “Neoplasm Benign”

After systematic search, the repetitive titles of the searched CPGs were removed and guidelines titles, introductions, and full texts were reviewed by the research team while the inclusion-- exclusion criteria were also considered. In the meantime, the research team agreed on a re-negotiation in cases of controversy until reaching a consensus.

The quality of the existing CPGs for managing extravasation of chemotherapy drugs was independently and separately assessed by the experts through AGREE II. According to the recommendations in AGREE II manual, each guideline is assessed by at least two and preferably four appraisers, as increasing the number of appraisers will increase the reliability of the assessment.[8] Therefore, in this study, each guideline was appraised by five healthcare workers from different disciplines in order to improve the reliability of the assessment. The assessors consisted of three oncology specialists and two nursing staff with work experience in chemotherapy department.

AGREE II is a general tool with 23 items in six domains and has two overall checking items. Table 2 shows a detailed description of all AGREE II items. Each item is scored on seven-point scale from one (strongly opposed) to seven (fully agree). The score of each of six sections is calculated independently and through the standardization of scores, obtained in each domain. In the final assessment section, the overall quality of the CPGs and the final recommendation of the assessors for the application of the guidance in the clinical setting are expressed.[11] The assessors' agreement among was calculated by ICC.[30] According to previous studies, the researchers calculated the overall quality of each CPG (by a threshold of 60%).[3132] Recommendations for using the CPGs in clinic were expressed as “I recommend, I recommend with modifications, and I do not recommend.“ The scores of the domains were categorized as good (80%), acceptable (60--79%), low (40--59%), and very low (40%>), according to previous similar articles.[1925] The reliability and validity of AGREE have been investigated in various studies. Terrace (2003), studied accreditation of AGREE, as an international assessment tool for assessing the quality of CPGs, and reported that to be 95% useful, claimed by of the assessors. In addition, the reliability of the parts of the tool was acceptable with a score of 88--64%.[31] In Iran, Rashidianet al. (2012) translated AGREE tool to Persian, whose validity was approved by Tehran University of Medical Sciences and the Ministry of Health and Medical Education joint group. In addition, comparison of the Persian and English versions reliability showed no significant difference.[33]

Table 2

Summary of Guidelines for Research and Evaluation II (AGREE II) structure and detailed list of items within each domain

Domain 1. Scope and PurposeItem 1 The overall objective (s) of the guideline is (are) specifically describedItem 2 The health question (s) covered by the guideline is (are) specifically describedItem 3 The population (patients, public, etc.) to whom the guideline is meant to apply is specifically describedDomain 2: Stakeholder InvolvementItem 4 The guideline development group includes individuals from all the relevant professional groupsItem 5 The views and preferences of the target population (patients, public, etc.) have been soughtItem 6 The target users of the guideline are clearly definedDomain 3: Rigor of DevelopmentItem 7 Systematic methods were used to search for evidenceItem 8 The criteria for selecting the evidence are clearly describedItem 9 The strengths and limitations of the body of evidence are clearly describedItem 10 The methods for formulating the recommendations are clearly describedItem 11 The health benefits, side effects and risks have been considered in formulating the recommendationsItem 12 There is an explicit link between the recommendations and the supporting evidenceItem 13 The guideline has been externally reviewed by experts prior to its publicationItem 14 A procedure for updating the guideline is providedDomain 4: Clarity of PresentationItem 15 The recommendations are specific and unambiguousItem 16 The different options for management of the condition or health issue are clearly presentedItem 17 Key recommendations are easily identifiableDomain 5: ApplicabilityItem 18 The guideline describes facilitators and barriers to its applicationItem 19 The guideline provides advice and/or tools on how the recommendations can be put into practiceItem 20 The potential resource implications of applying the recommendations have been consideredItem 21 The guideline presents monitoring and/or auditing criteriaDomain 6: Editorial IndependenceItem 22 The views of the funding body have not influenced the content of the guidelineItem 23 Competing interests of guideline development group members have been recorded and addressed

Ethical consideration

Review by a Human Ethics Review Committee was not required as this research involved only review of published work and did not involve any data collection from humans.

Results

After a systematic search, 111 CPGs, related to search keywords were found whose selection flowchart is shown in Figure 1. Selected CPGs were evidence based, in English, with number of references in the range between 10 and 18 references, and published between 2012 and 2017. All guidelines, in addition to extravasation management, also focused on prevention and education of signs and symptoms of extravasation to patients.[3435363738] Only two CPGs provided the level of evidence-based recommendations.[3536] In the majority of CPGs, the conflict of interest was not expressed or was absent. Updated versions of three CPGs were not available,[343536] one CPG would be updated in 2019,[37] and the other one in 2020.[38] Only one CPG provided the source of its funding.[38] Summary of CPGs details has been presented in Table 3. Five CPGs were assessed, criticized by five experts and analyzed by standardizing the scores, obtained in each domain, and the overall mean score for each domain was calculated. In addition, the scores of the domains of each separate CPGs, and the general level of quality of CPGs, as well as the final assessors' recommendation on application of the guideline in clinic setting are presented in Table 4. Four of the CPGs had a high level of quality, with a minimum score of their five domains higher than 60%. Among them was the CPG for the Management of Extravasation of Cytotoxic Drugs in Adults with the highest score (82.46%).[38] Comparison of the overall mean score in each domain showed that the domain of stakeholder involvement gained the highest score (92.89%) while the editorial independence domain had the lowest (54%).

Figure 1

Flow diagram of CPGs selection

Table 3

Characteristics of clinical practice guidelines

No	Guideline title	Year of publication	Institution	Conflicts of interest	Evidence-based guideline	Update	Funding	Size of complete guideline (pages)	Number of references
1	Chemotherapy extravasation guideline	2012	WOSCAN Cancer Nursing and Pharmacy Group	Not available	Yes	Not available	Not disclosed	24	14
2	Guideline for the management of extravasation of a cytotoxic agentor a monoclonal antibody used in the treatment of malignantdisease	2015	Pan Birmingham Cancer Network Governance Committee	Not available	Yes	Not available	Not disclosed	29	17
3	Guideline for the Management of Extravasation	2014	Pan Birmingham Cancer Network Governance Committee	Not available	Yes	Not available	Not disclosed	21	12
4	Clinical Guideline For The Management Of Extravasation Of Cytotoxic Drugs In Adults	2017	Pan Birmingham Cancer Network Governance Committee	No conflicts of interest	Yes	December 2020	The Royal Cornwall Hospitals NHS Trust	21	10
5	Guidelines for the Management ofExtravasation of a Systemic Anti-Cancer Therapy including Cytotoxic Agents	2017	West Midlands Expert Advisory Group for Systemic Anti-cancer Therapy (SACT)	Not available	Yes	July 2019	Not disclosed	29	18

Table 4

Summary of mean domain scores of Guidelines according to AGREE II

NO	Guideline title	Domain 1 (scope and purpose)	Domain 2 (stakeholder involvement)	Domain 3 (rigor of development)	Domain 4 (clarity of presentation)	Domain 5 (applicability)	Domain 6 (editorial independence)	Total score mean	Overall quality	Overall recommendation (to use guideline)
1	Chemotherapy extravasation guideline	78.89%*(acceptable)	94.44%*(good)	78.33%*(acceptable)	94.44%*(good)	80%*(good)	40%(low)	76.96%	High	Yes (3 reviewers)Yes with modifications(2 reviewers)
2	Guideline for the management of extravasation of a cytotoxic agentor a monoclonal antibody used in the treatment of malignantdisease	78.89%*(acceptable)	94.44%*(good)	86.25%*(good)	98.89%*(good)	85%*(good)	36.67%(very low)	81.16%	High	Yes (3 reviewers)Yes with modifications(2 reviewers)
3	Guideline for the Management of Extravasation	72.22%*(acceptable)	94.44%*(good)	80.83%*(good)	74.44%*(acceptable)	95%*(good)	45%(low)	76.96%	High	Yes (2 reviewers)Yes with modifications(3 reviewers)
4	Clinical Guideline For The Management Of Extravasation Of Cytotoxic Drugs In Adults	96.67%*(good)	95.56%*(good)	80.42%*(good)	95.56%*(good)	92.50%*(good)	100%*(good)	82.46%	High	Yes (4 reviewers)Yes with modifications(1 reviewer)
5	Guidelines for the Management ofExtravasation of a Systemic Anti-Cancer Therapy including Cytotoxic Agents	90%*(good)	85.56%*(acceptable)	56.67%(low)	74.44%*(acceptable)	50.83%(acceptable)	48.33%(low)	62.03%	Moderate	Yes (1 reviewer)Yes with modifications(4 reviewers)
Total domain score; mean		83.33%(good)	92.89%(good)	76.50%(acceptable)	87.56%(good)

* = Total score of domain>60%

The scores of CPGs showed that the CPG “Guideline for the management of extravasation of a cytotoxic agent, or a monoclonal antibody used in the treatment of malignant disease“ had the highest scores in terms of clarity of presentation and the rigor of development, while the lowest score was in the field of editorial independence.[36] The guideline for the Management of Extravasation CPG had the highest score in terms of applicability and the lowest scores in terms of scope and purpose and clarity of presentation.[35] The CPG for the Management of the Extravasation of Cytotoxic Drugs in Adults received the highest levels of success in the domains of scope and purpose, stakeholder involvement, and editorial independence.[38] “Guidelines for the Management of Extravasation of a Systemic Anti-Cancer Therapy including Cytotoxic Agents“ had a moderate overall quality in the domains of stakeholder involvement and rigor of development, while the domains of clarity of presentation and applicability obtained the lowest scores.[37] It should be noted that the lowest score in the domain of clarity of presentation (acceptable = 74.44%) was shared in two CPGs.[3537] The highest consensus among the assessors was in domain of stakeholder involvement (very good = 0.93) while the lowest was in editorial independence (moderate = 0.55).

Discussion

The findings of this study showed that assessment of existing CPGs for chemotherapy drugs extravasation by AGREE II tool is appropriate, and the overall quality of four CPGs out of five is high. However, the results, with regard to the scores within the domains, varied from good to very low. Meanwhile, the editorial independence domain had the maximum variability in all guidelines, which means that this domain is not well described in most of the assessed guidelines. Among the domains, the least attention was paid to editorial independence, while the only exception was the CPG for the Management of Extravasation of Cytotoxic Drugs in Adults,[38] which earned a score of 100% in this domain. This finding suggests that either CPGs developers may have not understood the importance of expressing conflicts of interest or they have not explicitly stated them. Evidence suggests that the conflict of interest of the guideline developers may affect the guideline recommendations.[39] Financial conflicts are also common in CPGs.[404142] Therefore, in the development process of the guideline, financial information and conflict of interest should be reported and should not affect the development of the guidelines. The stakeholder involvement domain obtained the highest score. Its score was more than 90%. Such high scores are due to the detailed description of the relevant specialized groups, associated with the guideline editorial team, and the clear identification of the main users of the guideline. The development methodology is the main determinant of the quality of CPGs. However, the rigor of development obtained a reasonable overall mean score, and only two CPGs provided levels of evidence-based recommendations,[3940] and updated versions of three CPGs were not available.[383940] These relatively lower scores are mainly due to inadequate description of the methodology of the development process of the CPGs, lack of external review and updating, lack or shortage of information on the systematic review method, the inclusion and exclusion criteria, and the quality of evidence. High-quality CPGs have systematic reviews and a valid international system for rating the quality of evidence and recommendations.[434445]

A limitation of the present study is that we may have lost some of the CPGs and updates. The other limitation of this study was the assessment of English-only guidelines due to the lack of the research team members' other languages ability. The strength points of this study were the use of the AGREE II evaluation tool, which is highly reliable and valid, teaching the assessors how to use it, and assessing the guidelines by them all independently.

Conclusion

The application and implementation of CPGs depends on the staff's trust on their quality and credibility. Since nurses are responsible for prescribing drugs and play a major role in chemotherapy and due to the high quality of the current CPGs and chemotherapy drugs extravasation, their use is recommended in the clinic to prevent the serious side effects of chemotherapy drugs extravasation to the entire treatment team and the nurses working in the oncology departments.

Financial support and sponsorship

Isfahan University of Medical Sciences

Conflicts of interest

Nothing to declare.