Quality appraisal of clinical guidelines for venous thromboembolism prophylaxis in patients undergoing hip and knee arthroplasty: a systematic review.

INTRODUCTION: Venous thromboembolism (VTE) occurs in up to 40%-80% of patients after hip and knee arthroplasty. Clinical decision-making aided by guidelines is the most effective strategy to reduce the burden of VTE. However, the quality of guidelines is dependent on the strength of their evidence base. The objective of this article is to critically evaluate the quality of VTE prevention guidelines and the strength of their recommendations in VTE prophylaxis in patients undergoing hip and knee arthroplasty.
METHODS: Relevant literature up to 16 March 2020 was systematically searched. We searched databases such as Web of Science, PubMed, EMBASE, Cumulative Index of Nursing and Allied Health Literature, China National Knowledge Infrastructure and WanFang and nine guidelines repositories. The identified guidelines were appraised by two reviewers using the Appraisal of Guidelines for Research and Evaluation II and appraised the strength of their recommendations independently. Following quality assessment, a predesigned data collection form was used to extract the characteristics of the included guideline.
RESULTS: We finally included 15 guidelines. Ten of the included guidelines were rated as 'recommended' or 'recommended with modifications'. The standardised scores were relatively high in the domains of Clarity of Presentation, and Scope and Purpose. The lowest average standardised scores were observed in the domains of Applicability and Stakeholder Involvement. In reference to the domains of Rigour of Development and Editorial Independence, the standardised scores varied greatly between the guidelines. The agreement between the two appraisers is almost perfect (intraclass correlation coefficients higher than 0.80). A considerable proportion of the recommendations is based on low-quality or very-low-quality evidence or is even based on working group expert opinion.
CONCLUSIONS: In summary, the majority of the recommendations are based on low-quality evidence, and further confirmation is needed. Furthermore, guideline developers should pay more attention to methodological quality, especially in the Stakeholder Involvement domain and the Applicability domain. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Our research critically evaluated the quality of guidelines for prevention of venous thromboembolism (VTE) in patients undergoing elective hip and knee arthroplasty and the strength of their recommendations in VTE prophylaxis.

Two appraisers used Appraisal of Guidelines for Research and Evaluation II, an assessment with methodological rigour and reliability, to appraise the quality of included guidelines and resolved any discrepancies by discussion.

Our search strategy was also reproducible; however, because of language or publication restrictions, there may be a language barrier.

Introduction

Total knee arthroplasty (TKA) and total hip arthroplasty (THA) are widely regarded as effective treatment options for patients with joint failure, which can help alleviate pain and improve function.1–3 Despite considerable advances in surgical and anaesthetic techniques, patients undergoing TKA and THA are at high risk of venous thromboembolism (VTE), manifesting as deep vein thrombosis or pulmonary thromboembolism.4 VTE is a severe postoperative complication, which commonly occurs in 40%–80% of patients undergoing THA and TKA.5 VTE is a potentially preventable medical condition that can prolong hospital stays and increase mortality.6 Despite the cost-effectiveness of THA and TKA, in-hospital cost and rehabilitation cost associated with hospital-acquired VTE place significant burdens on global healthcare systems.7

Using evidence-based VTE programmes can improve practice outcomes while reducing the physical, psychological, social and economic burden on individuals, families and countries. Clinical practice guidelines (CPGs) enable health professionals and patients to make the best decisions about treatment or care for a particular condition or situation and reduce waste. However, the quality of a CPG is dependent on the strength of its evidence base.8 As such, there is a need to evaluate CPGs to assess their quality. Therefore, we undertook this systematic review to evaluate the quality of the CPGs and the strength of their recommendations in VTE prophylaxis.

Methods

Objectives

The purpose of this systematic review is to critically appraise the quality of VTE prevention guidelines specific to the patients after THA and TKA. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool was used. We wrote this study following Preferred Reporting Items for Systematic Review and Meta-Analysis 2009 statement9 (see online supplemental table 1).

Data sources and search strategy

Academic databases, including Web of Science, PubMed, EMBASE, Cumulative Index of Nursing and Allied Health Literature, and Chinese databases (China National Knowledge Infrastructure and WanFang), were searched from inception until 16 March 2020. The search strategy was tailored to the requirements of each database. Searching of reference lists from identified papers was carried out along with forwarding citation searching using Google Scholar. All searches were saved in each database and imported into EndNote (V.X9; Clarivate Analytics), where duplicates were removed. To supplement our database searches, we also searched guidelines repositories, including CPG Infobase: Clinical Practice Guidelines (Canadian Medical Association), the Guidelines International Network, the National Health and Medical Research Council—Australian Clinical Practice Guidelines, the National Institute for Health and Care Excellence (NICE), the National Guideline Clearinghouse, Scottish Intercollegiate Guideline Network, New Zealand Guidelines Group, BMJ Best Practice and Chinese guidelines repository (YiMaiTong). Details of the searches are provided in online supplemental appendix 1.

Eligibility criteria

A complete list of inclusion and exclusion criteria is detailed in table 1.

Table 1

Inclusion and exclusion criteria

No.	Items
Inclusion criteria
1	Published international and national guidelines on the management and/or prevention of VTE after THA or TKA
2	Published as full text
3	Guidelines published in Chinese or English
4	Most recent complete guideline (from a single working group, ie, ACCP) and any partial revisions for the guideline published thereafter
5	Include an explicit statement identifying the document as a ‘guideline’
Exclusion criteria
1	Guidelines under development
2	Guidelines were specific to one institution
3	Complete guidelines with publication dates that have been superseded by more recent complete guidelines
4	Guidelines that only cover one aspect of VTE prevention (ie, anticoagulant prophylaxis)
5	Clinical practice standards, defined as a statement reached through consensus, which identifies the desired outcome. Usually used in audit as a measure of success47 48
6	Guidelines inclusive of only one phase of care, for example, Ginzburg et al49 (ie, during rehabilitative therapy)

ACCP, American College of Chest Physicians; THA, total hip arthroplasty; TKA, total knee arthroplasty; VTE, venous thromboembolism.

Data screening and extraction

Two reviewers used prespecified eligibility criteria to screen titles and abstracts. Articles that met the above inclusion and exclusion criteria were included for a second full-text screen. Conflicts were resolved through discussion or the involvement of a third reviewer. Reasons for exclusion were documented in a tabular format (online supplemental appendix 2). Data extraction was then performed independently using a standardised data extraction form developed based on AGREE II.10

Quality assessment of CPGs

To evaluate the quality of pre-existing guidelines selected for guideline adaptation, two reviewers graded each guideline according to AGREE II.11 This instrument consists of 23 items organised into six domains. AGREE II also includes two overall assessment items for overall judgements of the practice guideline. Online supplemental appendix 3 provides a brief description of each domain.

The 23-item AGREE II tool uses a seven-point agreement scale from 1 (strongly disagree) to 7 (strongly agree).10 Standardised scores for each domain were computed as (X/Y) ×100%, where X = obtained score−minimum possible score and Y = maximum possible score−minimum possible score.10 As defined by AGREE II, we considered a CPG as ‘recommended’ if it scored above 50% on ≥4 domains, as ‘recommended with modifications’ if it scored above 50% on 3 domains and as ‘not recommended’ if it scored less than 50% on ≥4 domains.

Before the quality appraisal using AGREE II, two reviewers completed an Online Training Tool12 and performed calibration exercises to clarify the eligibility criteria. Following training, the two reviewers independently applied AGREE II criteria to eligible CPGs using the My AGREE PLUS online platform.13 Our team met regularly to resolve any discrepancies in the quality appraisal. We used intraclass correlation coefficients (ICCs) to measure the agreement between the two assessors’ assessment of quality (AGREE II) of included CPGs. The results were interpreted as follows: 0.00, poor agreement; 0.00–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–1.00, almost perfect agreement.14

Results

The electronic database search retrieved 4808 citations. We retrieved and assessed the full texts of 42 promising reports, and among these, we excluded 32 (figure 1). The guidelines repositories search retrieved 327 citations, of which 317 full texts were excluded (figure 2). In total, 15 guidelines were included in the final analysis, and the detailed characteristics are shown in table 2. These CPGs were published between 2006 and 2019. Most of the CPGs were developed in the USA (n=3),15–17 with the remaining coming from China (n=1),18 the UK (n=1),19 France (n=1),20 Poland (n=1),21 Malaysia (n=1),22 Korea (n=1),23 Italy (n=1),24 Scotland (n=1)25 and Southern Africa (n=1),26 or from Asia (n=1),27 Europe (n=1)28 or International (n=1).29 Information sources regarding where CPGs were obtained are shown in online supplemental appendix 4.

Figure 1

Search strategy for library databases (final search undertaken on 16 March 2020). CPGs, clinical practice guidelines; CINAHL, Cumulative Index of Nursing and Allied Health Literature; WOS, Web of Science; CNKI, China National Knowledge Infrastructure.

Figure 2

Search strategy for guideline repositories (final search undertaken on 16 March 2020). CPGs, clinical practice guidelines.

Table 2

Characteristics of CPGs regarding VTE prevention in patients undergoing THA or TKA

	AAOS 2011	ACCP 2012	ASH 2019	Asian VTE CPG 2017	COA 2016
Original CPG title	Guideline on preventing VTE disease in patients undergoing elective hip and knee arthroplasty	Prevention of VTE in orthopedic surgery patients. Antithrombotic therapy and prevention of thrombosis	Prevention of VTE in surgical hospitalized patients	Asian VTE guidelines: updated recommendations for the prevention of VTE	Guidelines for the prevention of VTE in major orthopedic surgery in China (in Chinese)
Date published	2011	2012	2019	2017	2016
Country of origin	USA	USA	USA	Asia	China
Objective of CPG	Guide VTE prevention in patients undergoing THA and TKA	Guide VTE prevention in orthopaedic surgery patients	Guide VTE prevention in surgical hospitalised patients	Guide VTE prevention specific for the Asian population	Guide VTE prevention in patients undergoing THA, TKA, and HFS
Methods used to collect/select the evidence	A targeted systematic review using 4 databases	Identify critical priorities using PICO; Systematic reviews of topic areas	A targeted systematic review using 3 databases	Not stated	Not stated
Methods used to analyse the evidence	The hierarchical system used to grade levels of evidence	The hierarchical system used to grade levels of evidence	The hierarchical system used to grade levels of evidence	Not stated	Not stated
Ranking scheme to determine the strength of the evidence and recommendation	High, moderate, low, very low	1A, 1B, 1C, 2A, 2B, 2C	High, moderate, low, very low	Not stated	Not stated
Methods used to formulate the recommendations	Expert consensus	Expert consensus	Expert consensus	Expert consensus	Expert consensus
Number of recommendations	15	16	12	8	19
Method of CPG validation	External and internal peer review	External and internal peer review	External and internal peer review	Not stated	External and internal peer review
Intended users	Orthopaedic surgeons and all qualified clinicians	Healthcare providers in both primary and specialty care	Patients, surgeons, intensivists, internists, haematologists, general practitioners, hospitalists, other clinicians, pharmacists, and decision-makers	Not stated	Not stated
Composition of CPG working group	6 groups: Workgroup The external peer review group AAOS Guidelines oversight committee AAOS evidence-based practice committee AAOS council on research and quality AAOS board of directors	3 groups: The topic panel The entire ACCP AT9 Executive Committee The external peer review group	3 groups: 15 Panel members 16 researchers from McMaster GRADE centre The external peer review group	A working group of clinicians of various specialties and subspecialties from China, China Hong Kong, India, South Korea, Malaysia, Philippines, Singapore, and Thailand	46-panel members from COA
Number of documents included in the appraisal	2CPG (861 pages); review comments and AAOS responses (136 pages)	3CPG (48 pages); methodology for the development of antithrombotic therapy and prevention of thrombosis guidelines (13 pages); online data supplement (84 pages)	2CPG (47 pages); online data supplement (210 pages)	1CPG (20 pages)	1CPG (7 pages)

AAOS, American Academy of OrthoPaedic Surgeons; ACCP, American College of Chest Physicians; ASH, American Society of Hematology; AT9, Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines; COA, Chinese Orthopaedic Association; CPGs, clinical practice guidelines; ESA, European Society of Anaesthesiology; FSAIC, French Society for Anaesthesiology and Intensive Care; GRADE, Grading of Recommendations, Assessment, Development, and Evaluation; HFS, hip fractures surgery; ICS, International Consensus Statement; IICS, Italian intersociety consensus statement; KSTH, Korean Society of Thrombosis and Hemostasis; MHM, Ministry of Health Malaysia; NICE, National lnstitute for Health and Care Excellence; PCS, Polish Consensus Statement; PICO, population, interventions, comparisons, outcomes; SFSTH, Southern African Society of Thrombosis and Haemostasis; SIGN, Scottish Intercollegiate Guidelines Network; THA, total hip arthroplasty; TKA, total knee arthroplasty; VTE, venous thromboembolism.

A steering committee

8 working groups

Guideline development group

Review committee

External reviewer group

A guideline committee

5 obstetric subgroup members

7 orthaopedic subgroup members

13 NGC technical team members

3 co-opted expert advisers

2 peer reviewers

Guideline development group

SIGN executive

13 specialist reviewers

SIGN editorial group

Two assessors appraised each CPG. The AGREE II domain scores of each guideline are presented in table 3. Detailed scoring of each AGREE II item under each domain is presented in online supplemental appendix 5. Online supplemental figure 1 shows a radar chart of the results of the guideline appraisal. The quality of the evaluated guidelines showed significant variability. The standardised scores ranged from 50% to 100% in the Scope and Purpose domain, and all CPGs scored above 50%. The standardised scores in the Stakeholder Involvement domain ranged from 3% to 89%, with 6 of 15 CPGs scoring above 50%. The standardised scores in the Rigour of Development domain ranged from 16% to 98%, with 8 of 15 CPGs scoring above 50%. The standardised scores in the Clarity of Presentation domain ranged from 42% to 100%, with only one CPG scoring below 50%. The standardised scores in the Applicability domain ranged from 4% to 94%, with only 2 of 15 CPGs scoring above 50%. The standardised scores in the Editorial Independence domain ranged from 0% to 92%, with 8 of 15 CPGs scoring above 50%. Per the quality assessment tool used in this review, 6 of the 15 included CPGs were judged to be ‘recommended’. There is an almost perfect agreement between two appraisers, with the ICC ranging from 0.875 to 0.955.

Table 3

AGREE II scaled domain scores of CPGs for VTE prevention in patients undergoing THA or TKA

	AAOS 2011	ACCP 2012	ASH 2019	Asian VTE CPG 2017	COA 2016	ESA 2017	FSAIC 2006	ICS 2013	IICS 2011	KSTH 2014	MHM 2013	NICE 2019	PCS 2017	SIGN 2014	SFSTH 2013
1. Scope and Purpose	86%	92%	89%	50%	67%	86%	92%	81%	81%	72%	94%	100%	83%	97%	69%
2. Stakeholder Involvement	61%	67%	89%	25%	3%	25%	28%	22%	17%	47%	53%	89%	22%	83%	19%
3. Rigour of Development	98%	72%	83%	20%	21%	41%	50%	65%%	22%	22%	55%	91%	21%	81%	16%
4. Clarity of Presentation	78%	100%	92%	69%	69%	75%	75%	81%	61%	42%	78%	92%	72%	86%	53%
5. Applicability	10%	79%	21%	29%	4%	13%	13%	40%	25%	10%	23%	94%	15%	42%	17%
6. Editorial Independence	92%	92%	1%	67%	0%	71%	0%	42%	0%	50%	83%	54%	0%	75%	0%
Recommended use of this CPG	Yes	Yes	Yes	Yes*	No	Yes*	Yes*	Yes*	No	No	Yes	Yes	No	Yes	No
ICC (including overall CPG score)	0.945	0.916	0.907	0.929	0.945	0.892	0.918	0.875	0.906	0.878	0.882	0.932	0.948	0.950	0.897

*Recommended with modifications.

AAOS, American Academy of Orthopaedic Surgeons; ACCP, American College of Chest Physicians; AGREE II, Appraisal of Guidelines for Research and Evaluation II; ASH, American Society of Hematology; COA, Chinese Orthopaedic Association; CPGs, clinical practice guidelines; ESA, European Society of Anaesthesiology; FSAIC, French Society for Anaesthesiology and Intensive Care; ICC, intraclass correlation coefficeint; ICS, International Consensus Statement; IICS, Italian Intersociety Consensus Statement; KSTH, Korean Society of Thrombosis and Hemostasis; MHM, Ministry of Health Malaysia; NICE, National lnstitute for Health and Care Excellence; PCS, Polish Consensus Statement; SFSTH, Southern African Society of Thrombosis and Haemostasis; SIGN, Scottish Intercollegiate Guidelines Network; THA, total hip arthroplasty; TKA, total knee arthroplasty; VTE, venous thromboembolism.

Table 4 shows the levels of evidence for recommendations of VTE prevention in patients undergoing THA or TKA, as reported in the included CPGs. There are four CPGs developed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system to rank recommendations.15 16 23 28 Comparatively, four CPGs were developed based on expert opinion.18 26 27 29 Despite unanimous agreement in the recommendations for providing pharmacological and/or mechanical prophylaxis, early or delayed prophylaxis, and extended duration of prophylaxis, details disagree on the pharmacological and mechanical prophylaxis choice, time of early or delayed prophylaxis, and duration of prophylaxis. The American College of Chest Physicians (ACCP) 2012 guidelines,16 European Society of Anaesthesiology (ESA) 2017 guidelines28 and French Society for Anaesthesiology and Intensive Care (FSAIC) 2006 guidelines20 recommended low-molecular-weight heparin (LMWH) as a preference pharmacological prophylaxis choice, whereas direct oral anticoagulants (DOACs) were recommended in the American Society of Hematology (ASH) 2019 guidelines.15 An extended duration of thromboprophylaxis of 35 days in patients undergoing THA and 14 days in patients undergoing TKA seemed to be the primary choice.16 18 19 21 26 In terms of improving CPG implementation, patient/family education, type of anaesthesia, risk assessment and bridging therapy, we observed little recommendations with very low quality. The recommendations from each CPG that are informed in table 4 are detailed in online supplemental appendix 6. Online supplemental appendix 7 shows an explanation of the different evidence levels used across included CPGs.

Table 4

Levels of evidence for recommendations of VTE prevention in patients undergoing THA or TKA as reported in included CPGs

Recommendations*	AAOS 2011	ACCP 2012	ASH 2019	Asian VTE CPG 2017	COA 2016
1. Against routine postoperative VTE screening	Low–High	Grade 1B	–	–	–
2.VTE risk assessment	VTE history (Low, Moderate) Other factors (Very Low–Moderate)	–	–	NR about primary thrombophilia (WG)	–
3. Bleeding risk assessment	Bleeding disorders and active liver disease (Very Low) Other factors (Very Low, Low)	–	–	–	Assess risk factors (WG)
4. Bridging therapy	Discontinuation of antiplatelet preoperative (Moderate–High)	–	–	–	Discontinuation of antiplatelet preoperative (WG)
5. Stopping oestrogen–containing oral contraceptives or hormone replacement therapy	–	–	–	–	–
6. Provide pharmacologic and/or mechanical prophylaxis	WG, Moderate–High	pharmacologic and IPCD (Grade 2C)	Very Low, Low	WG	WG
7 Thromboprophylaxis for patients with bleeding risk	Mechanical prophylaxis (WG)	IPCD or no prophylaxis as a preference (Grade 2C)	Mechanical prophylaxis as a preference (Very Low)	IPCD (WG)	GCS, IPCD, and FIT (WG)
8. Pharmacological prophylaxis preference choice†	NR (WG)	LMWH (Grade 2B, 2C)	DOACs (Low, Moderate)	–	–
9. Mechanical prophylaxis preference choice†	NR (WG)	–	IPCD (Very Low)	–	–
10. Evaluation of pharmacological prophylaxis contraindications	–	–	–	–	WG
11. Evaluation of mechanical prophylaxis contraindications	–	–	–	–	WG
12. Use the fitted/correct size of GCS	–	–	–	–	–
13. Correct use of mechanical prophylaxis	–	–	–	–	–
14. Early or delayed prophylaxis	–	12 h preoperative or 12 h postoperative (Grade 1B)	12 h preoperative or 12 h postoperative (Very Low)	–	WG (Time depending on the adopted regimen)
15. Duration of prophylaxis	NR (WG)	A minimum of 10 to 14 days (Grade 1B–1C) Up to 35 days (Grade 2B)	19–42 days (Very Low)	–	A minimum of 10 to 14 days, up to 35 days for THA (WG)
16. General measures of thromboprophylaxis
Early mobilization	(Low, Moderate)	–	–	WG	WG
Hydration	–	–	–	–	WG
17. Adverse effects monitoring	–	–	–	–	WG
18. Euraxial anesthesia	Moderate, High	–	–	–	–
19. Against the use of IVC	Very Low, Low	Grade 2C	Very Low	–	WG
20. Improve CPGs implementation
Multidisciplinary collaboration	–	–	–	WG	–
Continuous education	–	–	–	WG	–
Implement an integrated Care pathway	–	–	–	WG	–
Create a personalized shared folder	–	–	–	–	–
Identify a lead	–	–	–	–	–
Carry out a baseline assessment	–	–	–	–	–
Think about what data you need to measure improvement	–	–	–	–	–
Implement the action plan with oversight	–	–	–	–	–
Review and monitor	–	–	–	–	–
Adopt approaches to increase CPG compliance	–	–	–	–	–
Develop local prophylaxis guidelines	–	–	–	–	–
21. Patient/family education
Reasons and importance of prevention	–	–	–	–	–
Symptoms/recognizing/reporting VTE	–	–	–	–	–
Correct use of/possible side effects of VTE prophylaxis	–	–	–	–	WG
Early rehabilitation exercise	–	–	–	–	WG
Discharge planning	–	–	–	–	–

*Refer to Appendix 6 for the recommendations from each CPG that informed in Table 4 and Appendix 7 for an explanation of the different evidence levels.

†For more prophylaxis choice please refer to Appendix 6–Specific recommendations across all CPGs that informed in Table 4.

–, Not reported; AAOS, American Academy of Orthopaedic Surgeons; ACCP, American College of Chest Physicians; ASH, American Society of Hematology; COA, Chinese Orthopaedic Association; CPG, clinical practice guideline; CPGs, clinical practice guidelines; DOACs, direct oral anticoagulants; ECS, elastic compression; ESA, European Society of Anaesthesiology; FIT, foot impulse technology; FSAIC, French Society for Anaesthesiology and Intensive Care; GCS, graduated compression stockings; ICS, International Consensus Statement; IICS, Italian intersociety consensus statement; IPCD, intermittent pneumatic compression device; IVC, inferior vena cava; KSTH, Korean Society of Thrombosis and Hemostasis; LMWH, low-molecular-weight heparin; MHM, Ministry of Health Malaysia; NICE, National lnstitute for Health and Care Excellence; NR, No recommendation/unresolved issue; PCS, Polish Consensus Statement; SFSTH, Southern African Society of Thrombosis and Haemostasis; SIGN, Scottish Intercollegiate Guidelines Network; THA, total hip arthroplasty; TKA, total knee arthroplasty; VTE, venous thromboembolism; WG, Working group expert opinion.

Discussion

To our knowledge, this is the first systematic quality appraisal of CPGs for VTE prevention in patients undergoing THA and TKA. Finally, 15 guidelines were recognised. Generally, the quality of 67% (10/15) of included guidelines was acceptable and evaluated as ‘recommended’ or ‘recommended with modifications’. The included CPGs were consistent in the recommendations, whereas they used different classification systems in indicating the levels of evidence. The data availability of trials and the timing of approval by regulatory agencies may also explain some differences in the preferred pharmacological prophylaxis (such as LMWH or DOACs). It is worth noting that a considerable proportion of the recommendations is based on low-quality or very-low-quality evidence or is even based on working group expert opinion, representing uncertain clinical significance. Therefore, high-quality randomised controlled trials are needed to support the evidence and potentially improve the cost-effectiveness of treatment.30 Notably, in terms of patient/family education and improving CPG implementation, very few strong recommendations were identified, indicating a lack of robust evidence. These findings would explain why VTE prophylaxis is still not routinely administered as guideline recommended in most hospitals.31 32

The standardised scores varied between different domains. In the Scope and Purpose domain and the Clarity of Presentation domain, the standardised scores were relatively high. In reference to the Rigour of Development domain and Editorial Independence domain, the standardised scores varied considerably between the CPGs. Our results are consistent with the results of other CPG quality appraisal focusing on different clinical topics.33 34 Marked improvements in CPG development methodology over the past decade may have a role in explaining the variance scores. Moreover, guideline development should be carried out according to the formulated plan, such as the WHO Guideline Development Handbook.35 It is also recommended to report methodological details for clinical guideline development based on AGREE II.36

We found that the domains of Stakeholder Involvement and Applicability were marked with the lowest standardised scores, which may be factors influencing implementation. Stakeholder involvement focuses on gaining support from a strong collaborative multidisciplinary network and obtaining the needs of all the potential users.37 Indeed, a multidisciplinary approach to VTE prevention involving key stakeholders is essential for putting recommendations into practice.19 However, only three CPGs included patients and their representatives in guideline development.15 19 25 Evidence-based medicine highlights the importance of patient-centred communication.38 Patient values and preferences should be taken into account, and the pros and cons of these options should be discussed with the patient.39 Therefore, guideline developers should consider the involvement and engagement of patients and the public in future CPG updates.

Guideline applicability is exceptionally critical for implementation. However, there is a lack of consensus on how CPG should be done in practice. Only two CPGs appraise the barriers and facilitators to guideline implementation and provide strategies to improve guideline uptake.16 19 Putting recommendations into practice is always challenging. Examples of multiple evidence-based implementation strategies for preventing VTE have been evaluated, such as computerised reminder systems, education, audit and feedback, and distribution of guidelines.40–44 Two published Cochrane systematic reviews have reported the interventions for implementing thromboprophylaxis in hospitalised patients at risk of VTE.45 46 We call researchers to add the Improve CPG Implementation domain as one of the pillars in guideline development.

This review has some strengths and weaknesses. First, our search strategy was developed with an experienced senior librarian. Our search strategy was also reproducible, as required by systematic reviews of published work. However, because of language or publication restrictions, we may miss some CPGs. Second, the CPGs we included range from 2006 (FSAIC) to 2019 (ASH and NICE). CPGs that are ‘recommended’ based on the AGREE II scoring could be obsolete if the CPGs are derived from outdated evidence. Therefore, some caution is warranted here. Finally, two appraisers used AGREE II, an assessment with methodological rigour and reliability, to appraise the quality of included guidelines and resolved any discrepancies by discussion. Although the appraisers were inexperienced in guideline evaluation, all had completed the AGREE II online training. Besides, the team members met weekly online to discuss progress and problems. And six of our group members have attended the Joanna Briggs Institute (JBI) evidence-based medicine training courses.

Conclusions

In summary, the majority of the recommendations are based on inadequate evidence quality, and further confirmation is needed. Furthermore, guideline developers should pay more attention to methodological quality, especially in the Stakeholder Involvement domain and the Applicability domain. Finally, improving CPG implementation and sustainability should also be carefully considered in CPG development.