What is the quality of reporting on guideline, protocol or algorithm implementation in adult trauma centres? Protocol for a systematic review.

INTRODUCTION: Quality improvement (QI) is mandatory in trauma centres but there is no prescription for doing successful QI. Considerable variation in implementation strategies and inconsistent use of evidence-based protocols therefore exist across centres. The quality of reporting on these strategies may limit the transferability of successful initiatives across centres. This systematic review will assess the quality of reporting on guideline, protocol or algorithm implementation within a trauma centre in terms of the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0). METHODS AND ANALYSIS: We will search for English language articles published after 2010 in EMBASE, MEDLINE, CINAHL electronic databases and the Cochrane Central Register of Controlled Trials. The database search will be supplemented by searching trial registries and grey literature online. Included studies will evaluate the effectiveness of guideline implementation in terms of change in clinical practice or improvement in patient outcomes. The primary outcome will be a global score reporting the proportion of studies respecting at least 80% of the SQUIRE 2.0 criteria and will be obtained based on the 18-items identified in the SQUIRE 2.0 guidelines. Secondary outcome will be the risk of bias assessed with the Risk Of Bias In Non-randomised Studies- of Interventions tool for observational cohort studies and with the Cochrane Collaboration tool for randomised controlled trials. Meta-analyses will be conducted in randomised controlled trials to estimate the effectiveness of guideline implementation if studies are not heterogeneous. If meta-analyses are conducted, we will combine studies according to the risk of bias (low, moderate or high/unclear) in subgroup analyses. All study titles, abstracts and full-text screening will be completed independently and in duplicate by the review team members. Data extraction and risk of bias assessment will also be done independently and in duplicate. ETHICS AND DISSEMINATION: Results will be disseminated through scientific publication and conferences. PROSPERO REGISTRATION NUMBER: CRD42018084273. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The research team comprises methodological and content experts in the fields of knowledge synthesis, trauma and quality improvement (QI).

A rigorous literature search and systematic review methodology will be used to identify relevant guideline implementation studies in the trauma care context.

Indexing of QI studies in the electronic databases is poor and inconsistent which may result in some studies not being captured in this review.

Introduction

Trauma centres are state or regionally designated hospitals that are resourced to provide specialised care for severely injured patients. In addition to being designated as such, trauma centres may also go through the process of accreditation or verification by an external body; the verification process uses standard criteria to ensure that trauma centres are suitably equipped to provide the highest quality trauma care.1 Centre verification criteria include engagement in quality improvement (QI) activity as part of a Performance Improvement and Patient Safety (PIPS) programme.1 A PIPS programme is designed to monitor centre performance and outcomes over time, with continuous improvement as the ultimate objective.

Despite the mandated existence of PIPS programmes, patient outcomes across accredited trauma centres continue to be highly variable.2–4 Differences in structures and processes of care across centres are hypothesised to contribute to these persistent variations in outcomes.5 Trauma centres also report inconsistent use of evidence-based protocols, which may contribute to differences in quality of care across institutions.6–8 Finally, implementation of PIPS programmes and QI strategies has varied considerably across centres; inadequacy of a centre’s PIPS programme is the most frequent reason for failing verification review.9 Identifying ways to support trauma centres in developing and implementing successful QI strategies is therefore critical.

One mechanism to support QI in trauma centres is the American College of Surgeons’ Trauma Quality Improvement Program (ACS TQIP).10 Launched in 2010, TQIP provides performance data reports to enrolled trauma centres on their processes of care and patient outcomes relative to their peers using risk-adjusted benchmarking, as well as evidence-based guidelines. TQIP has also developed best practice guidelines in the areas of geriatric trauma management, massive transfusion in trauma, traumatic brain injury management, management of orthopaedic trauma and palliative care.11 Each year, more than 1000 representatives of TQIP enrolled trauma centres meet to share successes and challenges in their QI efforts. Some centres have also published these successes in peer-reviewed journals, for example, reporting on reduction in rates of venous thromboembolism and urinary tract infection.12 13

While successful QI strategies are increasingly published in the scholarly and grey literatures, the quality of that reporting may be playing an important role in the observed variation in the success of QI initiatives across centres. At this time, we do not know if QI reporting in trauma is of sufficient detail or of high enough quality to enable replication or transferability to other centres. Even successful QI strategies, if inadequately reported, may result in failed initiatives at other centres wishing to implement them. The Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) guidelines were developed to ensure the utility and quality of QI reporting in healthcare and to establish a common ground to share QI outcomes in the scholarly literature.14 SQUIRE 2.0 is a checklist that guides reporting on QI methods and interventions (see online supplementary appendix I). The objective of our study is to assess the quality of reporting on trauma QI studies with reference to SQUIRE 2.0. Our research question is: In trauma centres, what is the quality of reporting on guideline, protocol or algorithm implementation within a hospital setting in terms of the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0)?

Methods and analysis

This protocol was drafted using Preferred Reporting Items for Systematic Reviews and Meta-analysis Protocols (PRISMA-P) 2015.15 (see online supplementary appendix II). Registration of this protocol was completed in the International Prospective Register of Systematic Reviews (PROSPERO) on 15 January 2018, registration number CRD42018084273. If protocol amendments are made, we will date and elaborate on the rationale and details of such amendments in the final report.

Eligibility criteria

Studies

We will include quantitative studies such as experimental studies (randomised clinical trials (RCTs), quasi-RCTs, non-RCTs); quasi-experimental studies (controlled before and after studies, interrupted time series); and observational studies (cohort, case–control, registry studies). Only studies evaluating the effectiveness of guideline implementation in terms of change in clinical practice (eg, adherence to guideline) or improvement in patient outcomes (eg, mortality, morbidity, resource utilisation). Studies will be excluded if they describe a guideline, protocol or algorithm implementation without evaluating its effects in a trauma setting. Qualitative studies, conference abstracts, proceedings, editorials and commentaries will be excluded.

Participants

Articles will be included in the review if the reported QI study is exclusively oriented to the care of injured adults (>18 years) and focused on change involving healthcare practitioners within a hospital setting (ie, trauma centre).

Healthcare practitioners include the following:

Surgeon (trauma, orthopaedic, neurosurgery, plastics, vascular, urology)

Physician (emergency medicine, anaesthesiologist, radiologist, transfusion medicine, geriatrician, intensive care)

House staff or trainee in any of the previously listed medical or surgical specialties

Nurse or nurse practitioner

Health professional student

Physiotherapist

Occupational therapist

Speech therapist

Pharmacist or pharmacy technician

Social worker

Intervention: guideline, protocol or algorithm

Using the American College of Surgeons’ Resources for Optimal Care of the Injured (2014) as a guide, we have defined a guideline, protocol or algorithm as any effort to reduce unnecessary variation in care through the development of a standardised tool and/or statement derived from evidence-based validated sources or best available literature and clinical experience.1 Guidelines and protocols may be systematically developed consensus statements that are designed to assist in clinical decision-making within an institution, and consist of a step-by-step explanation of procedures for problem solving or achieving a desired outcome.16 Protocols are often displayed in an algorithm format and implemented as clinical pathways using provider education and/or computerised clinical decision support tools in the form of order sets or standardised consultation requests.

Outcomes

The primary outcome will be a global score reporting the proportion of studies respecting at least 80% of the SQUIRE 2.0 criteria and will be obtained based on the 18 items identified in the SQUIRE 2.0 guidelines. Secondary outcome will be the risk of bias in each study assessed with the Risk Of Bias In Non-randomised Studies—of Interventions tool for observational cohort studies17 and with the Cochrane Collaboration tool for randomised controlled trials.18 Meta-analyses will be conducted in randomised controlled trials to estimate the effectiveness of guideline implementation if studies are not heterogeneous. If meta-analyses are conducted, we will combine studies according to the risk of bias (low, moderate or high/unclear) in subgroup analyses.

Setting

We will include reports of QI studies undertaken in any adult trauma centre (levels I, II or III). Clinical areas may include but are not limited to the trauma ward, trauma bay, emergency department, operating room, intensive care unit and angioembolisation suite.

Information sources: search strategy

The databases searched will include EMBASE, MEDLINE, CINAHL and the Cochrane Central Register of Controlled Trials from 2010. The year 2010 was selected to coincide with the launch of TQIP which has made evidence-based guidelines widely available to trauma centres. The database search will be supplemented by searching trial registries (eg, clinicaltrials.gov) and grey literature online (eg, American College of Surgeons Trauma Quality Improvement Program (ACS TQIP); Victorian State Trauma Outcomes Registry Monitoring Group; the Trauma Audit and Research Network; Agency for Healthcare Research and Quality). Only English articles will be considered due to limited language resources. There are no restrictions by country of study. If necessary, we will contact study authors for data clarification and to identify additional studies. Lastly, we will hand search publications known for publishing QI in trauma namely the Journal of Trauma and Acute Care Surgery and Journal of Trauma Nursing. We will also hand search the references of included studies. An information specialist (LP) with expertise developing searches for QI systematic reviews will develop and implement the search (see online supplementary appendix III— Search strategy).

To verify the sensitivity of our search strategy, we identified three articles that would be included in the review. We subsequently ran our search strategy to ensure these articles were captured. We assessed the specificity of the search which resulted in 2259 articles in MEDLINE which we determined to be feasible for review.

Data management

A web-based software such as Covidence (https://www.covidence.org/) will be used for data management.

Study selection

A pilot test of 75 randomly selected citations will be conducted by all authors to verify the inclusion/exclusion criteria. Subsequently, all study titles and abstracts will be reviewed by two reviewers independently (level I screening). Full texts of studies considered appropriate or uncertain for inclusion will be retrieved. Full-text articles will be reviewed by two reviewers independently (level II screening). All discrepancies will be resolved by discussion or a third reviewer. Study selection process and reasons for exclusion will be reported.

Data collection

Data from included studies will be abstracted independently by two reviewers using a standardised data collection form to address the primary and secondary outcomes. Data to address the primary outcome will be obtained based on the 18 items identified in the SQUIRE 2.0 guidelines (see online supplementary appendix II—SQUIRE 2.0 guideline data collection form). Additional data will be collected to address the secondary outcomes, and will include information on study design, guideline type, population, control, outcomes and other necessary information for assessment of risk of bias or meta-analyses, such as effect estimates on guideline implementation outcomes. The data collection form will be pilot tested on a randomly selected 10% sample of included studies to ensure high inter-rater agreement between reviewers. Discrepancies will be resolved by discussion or with a third reviewer.

Data synthesis

The flow of the screening process will be presented in a PRISMA flow diagram. A table in PICO format will present the characteristics of the included studies. Other important information will be included in the tables as needed. The primary outcome, that is, the proportion of studies respecting at least 80% of the SQUIRE 2.0 criteria, will be presented descriptively by narrative summaries and as percentage scores for each guideline item across included studies. These data will be extracted by way of their absence or presence in the QI report (ie, yes, no or unclear). Narrative summaries of subgroups will also be provided, for example, of guideline type, targeted deficiencies or providers (eg, physicians, nurses, etc).

Risk of bias assessment will be performed independently in duplicate for each included study. Any disagreement will be resolved through discussion and consensus. For RCTs, we will use the Cochrane Collaboration’s tool,18 which assesses bias in domains of sequence generation, allocation concealment, blinding of outcomes, incomplete outcome data, selective outcome reporting and baseline imbalances. For cohort studies, risk of bias will be evaluated on the selection of the exposed and non-exposed cohorts, the comparability of the cohorts, the assessment of outcomes and the adequacy of follow-up, using the ROBINS-I tool.17

The outcomes of each intervention (the effects of the guideline implementation on processes and/or patients) will be synthesised in a table. Studies will be combined in meta-analyses if sufficient clinical, methodological and statistical homogeneity is found. Clinical and methodological heterogeneity across the studies will be assessed by examining the details of the subjects, the baseline data, the interventions and the outcomes to determine whether the studies are sufficiently similar or not. Statistical heterogeneity will be determined using the I2 statistic and the Χ2 test. Pooling of overall estimates of effect will be performed using generic inverse variance weighting methods. Using these methods, each study estimate of the relative treatment will be given a weight that is equal to the inverse of the variance of the effect estimate (ie, one divided by the SE squared).

We will use computer software (RevMan 5.3, The Nordic Cochrane Centre, The Cochrane Collaboration, 2015 or similar) to carry out a quantitative analysis. If performed, meta-analyses will be conducted using a mixed-effect model if heterogeneity is high. Reporting bias will be assessed with a funnel plot of the studies included in the review.

Ethics and dissemination

QI is required in trauma centres but needs to be effectively designed and implemented in order to achieve improvements in targeted outcomes. Trauma centres have variable success with QI that may be modifiable by enhancing the quality of QI reporting and thus the transferability of findings. It is therefore timely to review the quality of reporting by trauma centres that are implementing guidelines and protocols with a view to describe current gaps and opportunities. There are no ethical or safety concerns with this study. We will disseminate the results of this review through scientific publication and conferences. We expect our results to have important implications for trauma centre leaders and practitioners when planning future guideline implementations.