Effectiveness of trauma centre verification: a systematic review and meta-analysis.

Background: There is a growing trend toward verification of trauma centres, but its impact remains unclear. This systematic review aimed to synthesize available evidence on the effectiveness of trauma centre verification.
Methods: We conducted a systematic search of the CINAHL, Embase, HealthStar, MEDLINE and ProQuest databases, as well as the websites of key injury organizations for grey literature, from inception to June 2019, without language restrictions. Our population consisted of injured patients treated at trauma centres. The intervention was trauma centre verification. Comparison groups comprised nonverified trauma centres, or the same centre before it was first verified or re-verified. The primary outcome was in-hospital mortality; secondary outcomes included adverse events, resource use and processes of care. We computed pooled summary estimates using random-effects meta-analysis.
Results: Of 5125 citations identified, 29, all conducted in the United States, satisfied our inclusion criteria. Mortality was the most frequently investigated outcome (n = 20), followed by processes of care (n = 12), resource use (n = 12) and adverse events (n = 7). The risk of bias was serious to critical in 22 studies. We observed an imprecise association between verification and decreased mortality (relative risk 0.74, 95% confidence interval 0.52 to 1.06) in severely injured patients.
Conclusion: Our review showed mixed and inconsistent associations between verification and processes of care or patient outcomes. The validity of the published literature is limited by the lack of robust controls, as well as any evidence from outside the US, which precludes extrapolation to other health care jurisdictions. Quasiexperimental studies are needed to assess the impact of trauma centre verification. Systematic reviews registration: PROSPERO no. CRD42018107083.

The introduction of trauma systems, defined as an organized and multidisciplinary response to injury from prehospital care to rehabilitation and community integration, has led to important reductions in injury burdens in many high-income countries.1,2 Essential to the development of a trauma system is the designation of trauma centres according to levels of care (levels I–V for adults, and I or II for pediatric centres), which is commonly the role of states or provinces.3 Trauma centres are acute care hospitals where resources are prioritized to ensure that injured patients receive appropriate and timely care.4,5 Injury organizations, including the American College of Surgeons, have established trauma facility standard guidelines.3 These guidelines have been used to develop trauma centre verification or accreditation processes, aimed to determine whether trauma centres are fulfilling the criteria for optimal care. “Accreditation” and “verification” of trauma centres refer to the same process; hereafter we use the term verification to refer to both.5,6

The terms “verification” and “designation” are sometimes used interchangeably despite having different meanings.7–10 Designation is conducted by regional health authorities at the local or state stage, where centres are categorized in levels (I–V for adults, and I or II for pediatric centres), whereas verification (or accreditation) is generally an optional program to verify that a facility is performing as a trauma centre and meets the criteria for its designation level.5,9,11 For example, verification is offered by the American College of Surgeons in the United States3,12,13 and accreditation by Accreditation Canada;6 these organizations are not responsible for designation.11 A centre can be designated at a particular level without having received verification.9,10 In some US states or Canadian provinces, regulatory agencies may require regular verification for a trauma centre to maintain designation within their systems. Verification allows for standardization of personnel and equipment and a facility’s commitment to trauma care.9 Perceived advantages of verification include commitment to trauma care, and identification of opportunities and priorities for improvement.14 Verification is, however, an expensive and resource-intensive process.15,16 It generally requires a centre to submit a prereview questionnaire and have an on-site visit by an experienced peer review team.3 A summary of verification modalities in different countries is presented in Table 1.

Table 1

Examples of verification agencies

Jurisdiction	Agency	Certificate duration, yr	First verification
United States*	American College of Surgeons	3	1987 (ongoing)
Canada (except Quebec)	Trauma Association of Canada	5	1995 (to 2014)
	Accreditation Canada	4	2014 (ongoing)
Quebec†	Institut national d’excellence en santé et services sociaux	~6	1995 (ongoing)
Australia	Royal Australasian College of Surgeons	3	2000 (ongoing)

Some states (e.g., Pennsylvania) have their own verification agencies.

Verification is mandatory in Quebec.

Although verification has become a common practice,14,17 the evidence of its effectiveness on patient outcomes has not been systematically assessed and synthesized. It is essential to know whether the allocation of financial and human resources used in the verification process has its intended effect.17,18 This systematic review aims to synthesize available evidence on trauma centre verification to evaluate whether verification reduces in-hospital mortality, adverse events and resource use and improves processes of care.

Methods

The protocol of this review was registered in the PROSPERO database (record CRD42018107083) and published.19 The review was conducted in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.20 We received ethics approval for this project from the McGill University Faculty of Medicine Research Ethics Committee.

Literature search and study selection

We conducted a systematic search of the CINAHL, Embase, HealthStar, MEDLINE and ProQuest databases, as well as the websites of key injury organizations for grey literature from inception to June 2019, without language restrictions. Manual searches for additional eligible studies were performed by reviewing the reference lists of included studies. The search strategy is available in Appendix 1 (available at canjsurg.ca/016219-a1). Conference abstracts were included unless they were subsequently published as full articles.

Study population and intervention

Our study population consisted of injured patients treated at trauma centres. The intervention under evaluation was trauma centre verification. Comparison groups consisted of nonverified centres, or the same centre before it was first verified or re-verified. We considered all study designs; however, narrative studies without a quantitative estimate of the association between verification and the investigated outcomes were excluded.

Outcomes

Our primary outcome was in-hospital mortality. Secondary outcomes included population-based injury-related mortality, adverse events (e.g., complications), resource use (e.g., length of stay [LOS] and costs) and adherence to evidence-based processes of care (e.g., nonsurgical management of splenic injuries).

Data collection and extraction

After duplicates were removed from the search results,21 titles and abstracts were independently screened by 2 authors (B.B. and M.C.) using a Web and mobile app for systematic reviews.22 In case of disagreement or uncertainty, full papers were retrieved and discussed with a third author (L.M.). Full texts of selected studies were retrieved and examined to determine eligibility by 2 authors (B.B. and M.C.), who also independently extracted the data using standardized forms. When available, data recorded included country of the study, the number of centres, study design, patient demographic characteristics and outcome. Efforts were made to contact the corresponding author for further information when needed. Descriptive statistics and measures of associations were extracted directly from the studies or computed if enough information was provided.23–25

We assessed the risk of bias using the Risk Of Bias In Non-randomised Studies – of Interventions (ROBINS-I) assessment tool.26 We evaluated the quality of the collective evidence and strength of recommendations using Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group methodology.27

Statistical analysis

We summarized included studies descriptively. Owing to the diverse types of measures of association used and missing standard errors (SEs) or confidence intervals (CIs), some studies were not included in the meta-analysis. These studies were summarized narratively.

For studies included in the meta-analysis, we calculated the overall summary estimates, including relative risks (RRs), odds ratios (ORs) and weighted mean difference using random-effects meta-analysis.28 We quantified heterogeneity with the I2 statistic.29 We also produced funnel plots to examine the potential for publication bias. Sensitivity analysis according to the risk of bias was planned but could not be done owing to the small number of studies included. All analyses were performed with admetan and metafunnel packages in Stata 15 (Statacorp).30

Results

A total of 5125 citations were initially identified by the search strategy after de-duplication. Among them, 102 articles were selected for full-text review, of which 29 satisfied our inclusion criteria (Fig. 1 and Appendix 1, Table S1).

Fig. 1

Flow diagram showing study selection. *Including 3 conference abstracts and 1 thesis.

All included studies assessed American College of Surgeons verification in the US and were observational, including 18 cross-sectional,7,9,10,14,16,31–43 10 pre–post8,44–52 and 1 time-series53 design. Mortality was the most commonly investigated outcome (n = 20), followed by processes of care (n = 12), resource use (n = 12) and adverse events (n = 7). A summary of study characteristics is presented in Table 2. It was not possible to compute CIs for the measure of associations in 7 studies (24%). Almost half of included studies (13 [45%]) did not adjust for centre or patient case mix characteristics,8,35,38,39,40,44,46–52 and only one-third (6/18 [33%]) of multicentre studies considered the clustered nature of the data in their analyses. The risk of bias was serious to critical in 22 studies, and moderate in 7 (Table 3).

Table 2

Summary of characteristics of included studies

Design; study	Outcome(s) investigated	Population/age, yr	No. of centres (patients)	Data collection period	Summary of results
Cross-sectional
Osler et al.,31 2001	Mortality	Pediatrics (all injuries)/< 18	53 (49 428)	1985–1996	Survival rates among children at verified centres were higher than at nonverified centres (OR 0.75, 95% CI 0.58 to 0.97). However, authors could not conclude that it was the process of verification itself that improved outcomes. It is possible that only trauma centres with better results pursue verification. If so, verification would recognize, but not contribute to, improved outcomes.
Hesdorffer et al.,32 2002	Processes of care	Adults (TBI)/no details	411 (no details)	1999–2000	Authors surveyed all designated US trauma centres caring for adults with severe TBI to determine degree of guideline compliance and identify predictors. Full compliance occurred more commonly among hospitals with level I designation, a neurosurgery residency program, treatment protocols, a neurologic ICU and ACS verification (23% v. 15%).
Demetriades et al.,33 2006	Mortality	Adults (all severe injuries [ISS > 15])/> 14	256 (130 154)	1994–2003	Authors compared verified centres and nonverified centres and found that adjusted mortality in nonverified centres was higher than in verified level I centres (OR 1.09, 95% CI 1.05 to 1.13). However, they highlighted that this finding needs cautious interpretation because the group of nonverified centres included facilities that were only state-designated and those with no trauma centre designation.
Kim,34 2006	Mortality, processes of care, resources	All patients (head injuries)/≤ 89	16 (487)	2002–2003	12 verified centres and 4 state-designated centres. No associations between verification and outcomes investigated (mortality, LOS, home discharge disposition and time to surgery).
Hesdorffer et al.,35 2007	Processes of care	Adults (TBI)/no details	413 (no details)	2006	Web-based survey conducted in 413 designated trauma centres admitting patients with severe TBI. Good adherence was defined as adherence to median number of guidelines. Higher rate of good adherence was found in verified trauma centres (70.6%, n = 153) than in state-designated centres (60.8%, n = 232).
Horton et al.,36 2008	Processes of care, resources	All patients (all injuries)/no details	156 (no details)	2005	Authors surveyed 435 trauma centres (level I and II) throughout the US. 156 surveys were returned. ACS verification and trauma level I designation were independent predictors of recombinant factor VIIa use.
Smith et al.,10 2011	Mortality, adverse events	All patients (all injuries)/> 16	No details (519 402)	2002–2006	Authors compared verified level I facilities to state-designated centres. Overall, no adjusted survival advantage. However, among patients with acute respiratory distress syndrome, mortality rate was lower after admission to ACS-verified centres than to state-designated centres (20.3% v. 27.1%).
Theologis et al.,38 2012	Processes of care	All patients (spine injuries)/no details	No details	No details	Authors contacted trauma managers in all level I trauma centres in US to analyze institutions’ official cervical spine clearance protocols, if applicable. Response rate was 83%. Two-thirds of respondents had an official cervical spine clearance protocol. More ACS-verified centres than nonverified centres had protocols (75% v. 54%).
Notrica et al.,37 2012	Mortality	Pediatrics (all injuries)/< 18	NA	2008	Population-based study of pediatric injury mortality rates per 100 000. Authors determined availability of verified PTCs and verified ATCs in each state and compared mortality rates. Findings highlight protective association between pediatric injury mortality rates and presence of verified level I PTCs.
Brown et al.,16 2013	Mortality	Adults (all injuries)/> 15	374 (900 274)	2007–2008	Retrospective analysis of 246 verified and 128 state-designated centres. Verified level I centres had lower median SMR than state-designated centres (0.95 [IQR 0.82–1.05] v. 1.02 [IQR 0.87–1.15]); no difference in level II centres. State-designated level II centres had higher SMR outliers than ACS-verified level II centres.
Russell et al.,40 2015	Resources	Pediatrics (all injuries)/no details	102 (no details)	No details	Authors conducted structured telephone survey of emergency departments registered with National Association of Children’s Hospitals and ACS-verified PTCs. Bedside ultrasonography has become largely ubiquitous for care of children at designated PTCs; no significant difference between verified PTCs (97% [56/58]) and designated trauma centres (89% [39/44]).
Alarhayem et al.,39 2015	Processes of care	Pediatrics (splenic injuries)/< 17	No details (2342)	2012	Majority of children with splenic injuries were treated in nonverified PTCs. Verified level I PTCs had highest success with nonoperative management of high-grade splenic injuries (6%), followed by ACS-verified level II PTCs (10%) and nonverified PTCs (13%).
Bogumil et al.,41 2017	Processes of care	Pediatrics (all injuries)/< 18	No details (475 527)	2007–2014	Authors used National Trauma Data Bank to compare prevalence of nonaccidental trauma between ACS-verified PTCs and. They calculated crude and ISS-adjusted prevalence ratio estimates. Prevalence of nonaccidental trauma was higher at ACS-verified PTCs than at non-ACS-verified PTCs.
Grossman et al.,14 2017	Mortality, adverse events	All patients (all injuries)/all ages	94 (392 997)	2012	Authors analyzed national representative sample of 94 trauma centres (72 verified and 22 nonverified). Measurable benefits in complications were observed only among cases of major trauma (ISS > 25) in all age groups.
Roubik et al.,42 2017	Mortality	All patients (ground-level falls)/> 15	794 (812 051)	2007–2014	Retrospective analyses comparing 335 verified and 459 state-designated centres. SMR was lowest for verified level III/IV centres (0.97, 95% CI 0.97 to 0.98) and highest for state-designated level III/IV centres (1.04, 95% CI 1.04 to 1.04).
Agrawal et al.,7 2018	Mortality, adverse events, resources	All patients (all injuries)/< 16	109 (1 504 848)	2002–20092013–2014	After risk adjustment, authors observed lower mean ICU LOS (−0.2 d [SD 0.02 d]), hospital LOS (−0.3 d [SD 0.019 d]), mortality (OR 0.94, 95% CI 0.92 to 0.96) and number of patients who developed complications at verified centres than at state-designated centres.
Schubert et al.,9 2019	Mortality, adverse events	Adults (all injuries)/> 17	863 (4 044 449)	2010–2015	Overall, patients admitted to verified v. state-designated centres had similar adjusted mortality risk (RR 1.00, 95% CI 0.91 to 1.03) and unplanned return to operating room (RR 1.10, 95% CI 0.92 to 1.31), but higher unplanned intubation (RR 1.30, 95% CI 1.11 to 1.52). However, verified level III and IV facilities had lower adjusted mortality risk, with much lower mortality risk in ACS-verified level IV facilities.
Jenkins et al.,43 2019	Mortality	Adults (all injuries)/> 16	155 (94 655)	2010–2011	Authors examined association between national surgery conferences and in-hospital trauma mortality. Mortality increased significantly during meetings among patients admitted to hospitals that lacked ACS trauma verification; association was particularly pronounced among patients with penetrating injuries.
Pre–post
Richardson et al.,44 1997	Mortality, processes of care	All patients (all injuries)/no details	2 (381)	1988 and 1995	Two level III facilities; 1 had received level III verification, and the other had changes that lessened general surgeons’ involvement in initial evaluation and treatment. Verified centre had increase in patients transferred to level I hospital and in patient acuity. More operations were performed locally, and care was delivered more efficiently. The other hospital had large increase in transfers and decreased admissions locally as general surgical involvement decreased.
Piontek et al.,45 2003	Mortality, adverse events, resources	All patients (all injuries)/no details	1 (7811)	1993–2001	Pre–post study with internal and external negative control outcomes of level II centre. Results suggest that efforts and resources consumed in achieving ACS level II trauma centre verification result in decreased LOS (by 10%), reduced in-hospital mortality (severity-adjusted mortality observed/ expected ratio 0.81 before v. 0.59 after) and reduced costs (by 5%).
Ehrlich et al.,46 2005	Processes of care	Pediatrics (all injuries)/< 16	1 (no details)	1997–2002	Verification process at a level I ATC seeking level I PTC verification affected patient care through changes in care indicators. Mortality and ISS distributions remained unaltered. Patient evaluation, including radiology and time to discharge from emergency department (< 120 min), improved. PICU duration of stay increased, and prehospital and emergency department fluid monitoring remained unsatisfactory.
Maggio et al.,47 2009	Mortality, resources	All patients (all injuries)/no details	1 (3891)	2001 and 2007	Commitment to ACS verification resulted in increased admissions and interfacility transfers. Despite admission of more seriously injured patients, there was sustained reduction in mortality (by 47% in patients with ISS > 24) and trend toward decreased ICU LOS. Authors also found 78% increase in revenue and sustained increase in hospital profitability.
Norwood et al.,48 2011	Mortality, resources	All patients (major torso vascular injuries)/no details	1 (274)	1992–2008	Centre was verified level II facility before obtaining level I verification in 1998. Commitment of hospital resources required to achieve level I verification in community hospital improved survival, particularly in patients with blunt or penetrating thoracic injuries (73% before v. 30% after).
Murphy et al.,49 2015	Mortality, processes of care, resources	Pediatrics (splenic injuries)/< 16	No details (231)	1998–2012	Addition of verified PTC within inclusive trauma system was associated with significant reduction in proportion of patients undergoing splenectomy. However, results regarding mortality were inconclusive (RR 0.70, 95% CI 0.12 to 4.09).
Choi et al.,8 2016	Mortality, adverse events, resources	Pediatrics (all injuries)/no details	1 (4353)	2009–2010 and 2012–2014	Retrospective review of state-designated level I PTC, comparing 2 years before and 2 years after verification. Overall, no differences in mean age or ISS. Hospital and PICU LOS, ventilator days and mortality were also unchanged. Proportion of PICU admissions decreased from 17.2% to 13.7%. Adverse events in form of hospital-acquired conditions also decreased following verification, most notably through reduction in pneumonia.
Schlegel et al.,51 2018	Mortality, resources	Pediatrics (all injuries)/< 18	1 (1190)	2004–2016	Retrospective analysis divided into 3 chronologic treatment eras: early ATC, PTC and late PTC after ACS verification. Decrease in intensive care admissions was identified during late PTC compared to early PTC and ATC (51% v. 62.4% v. 67%), but overall mortality was unchanged.
Alexander et al.,50 2019	Mortality, adverse events, processes of care, resources	Pediatrics (splenic injuries)/< 18	1 (126)	2005–2017	Comparison of verified level I ATC after PTC verification. Lower rate of splenic intervention under PTC than ATC verification (7.1% v. 19.6%). Primary driver of decrease was reduction in operative rates (14.3% under ATC v. 4.3% under PTC). Average hospital LOS (7.4 d v. 6.5 d) and average ICU LOS (2.7 d v. 2.3 d) were similar. No change in in-hospital mortality.
Abd El-Shafy et al.,52 2019	Processes of care, resources	Pediatrics (all injuries)/no details	1 (1293)	2011 and 2016	Process changes associated with ACS level I pediatric verification and reductions in nonsurgical admission rate (29% in 2011 v. 5% in 2016) were also marked by reduction in hospital LOS (3.78 d v. 3 d).
Time-series
Notrica et al.,53 2018	Mortality	Pediatrics (all injuries)/15–17	NA	1999–2015	Authors collected prospective data on motor vehicle fatalities, crash characteristics, state driving laws and verified trauma centres for all 50 US states. Verified PTCs were associated with 12% decrease in rate of change in crude adolescent fatalities.

ACS = American College of Surgeons; ATC = adult trauma centre; CI = confidence interval; ICU = intensive care unit; IQR = interquartile range; ISS = Injury Severity Score; LOS = length of stay; NA = not applicable; OR = odds ratio; PICU = pediatric intensive care unit; PTC = pediatric trauma centre; RR = relative risk; SMR = standardized mortality ratio; SD = standard deviation; TBI = traumatic brain injury.

Table 3

Risk of bias as assessed with the Risk Of Bias In Non-randomised Studies – of Interventions assessment tool

Study	Confounding	Selection of participants into study	Classification of interventions	Deviations from intended of interventions	Missing data	Measurement of outcomes	Selection of reported result	Overall bias	Direction
Osler et al.,31 2011	Serious	Serious	Serious	Serious	Serious	Low	Low	Serious	Unpredictable
Hesdorffer et al.,32 2002	Serious	Serious	Serious	No information	Serious	Moderate	Serious	Serious	Unpredictable
Demetriades et al.,33 2006	Critical	Serious	Serious	No information	No information	Low	Moderate	Serious	Favours experimental
Kim,34 2006	Moderate	Serious	Moderate	Moderate	Critical	Low	Low	Moderate	Unpredictable
Hesdorffer et al.,35 2007	Critical	Serious	Critical	Low	Serious	Moderate	Serious	Serious	Unpredictable
Horton et al.,36 2008	Critical	Critical	Moderate	No information	Critical	Critical	Moderate	Critical	Unpredictable
Smith et al.,10 2011	Serious	Serious	Moderate	Moderate	No information	Low	Serious	Serious	Unpredictable
Theologis et al.,38 2012	Critical	Serious	Serious	No information	No information	Moderate	No information	Critical	Unpredictable
Notrica et al.,37 2012	Critical	Serious	Moderate	No information	No information	Moderate	Moderate	Serious	Unpredictable
Brown et al.,16 2013	Moderate	Serious	Moderate	Moderate	Moderate	Low	Moderate	Moderate	Toward the null
Russell et al.,40 2015	Critical	Critical	Serious	No information	No information	Critical	Moderate	Critical	Unpredictable
Alarhayem et al.,39 2015	Critical	Critical	Moderate	No information	No information	Low	Serious	Critical	Unpredictable
Bogumil et al.,41 2017	Critical	Moderate	Moderate	No information	Serious	Moderate	Low	Serious	Unpredictable
Grossman et al.,14 2017	Serious	Serious	Serious	No information	No information	Low	Moderate	Serious	Unpredictable
Roubik et al.,42 2017	Moderate	Serious	Moderate	Serious	Critical	Low	Moderate	Serious	Unpredictable
Agrawal et al.,7 2018	Serious	Moderate	Low	No information	Serious	Moderate	Serious	Moderate	Unpredictable
Schubert et al.,9 2019	Moderate	Moderate	Moderate	Low	No information	Low	Moderate	Moderate	Unpredictable
Jenkins et al.,43 2019	Low	Moderate	Low	No information	Serious	Moderate	Serious	Serious	Unpredictable
Richardson et al.,44 1997	Critical	Low	Low	Moderate	No information	Moderate	Serious	Serious	Unpredictable
Piontek et al.,45 2003	Moderate	Low	Low	Low	No information	Moderate	Moderate	Moderate	Unpredictable
Ehrlich et al.,46 2005	Critical	Low	Low	Serious	No information	Moderate	Low	Serious	Unpredictable
Maggio et al.,47 2009	Critical	Moderate	Moderate	Moderate	No information	Serious	Serious	Serious	Favours experimental
Norwood et al.,48 2011	Critical	Low	Low	Moderate	No information	Low	Moderate	Serious	Favours experimental
Murphy et al.,49 2015	Critical	Low	Low	Moderate	No information	Moderate	Moderate	Serious	Unpredictable
Choi et al.,8 2016	Critical	Low	Moderate	Moderate	No information	Low	Moderate	Serious	Favours Comparator
Schlegel et al.,51 2018‡	Critical	Low	Low	Moderate	No information	Moderate	Moderate	Serious	Favours experimental
Alexander et al.,50 2019	Serious	Low	Low	Low	No information	Moderate	Serious	Moderate	Favours experimental
Abd El-Shafy et al.,52 2019	Critical	Low	Low	Moderate	No information	Moderate	Moderate	Serious	Unpredictable
Notrica et al.,53 2018	Moderate	Moderate	Moderate	No information	Serious	Low	Moderate	Moderate	Unpredictable

Mortality

Twenty studies7–10,14,16,31,33,34,37,42–45,47–51,53 investigated mortality, of which 18 looked at in-hospital mortality. Seven of the 20 (35%) focused on pediatric injured patients, 4 (20%) on adults, and 9 (45%) on both. Most studies had cross-sectional (11) or pre–post (8) designs, with only 1 time-series study. Half of the studies presented only crude estimates.

We excluded 5 studies from the meta-analysis owing to the different effect measure scales used, as well as missing SEs (Table 2). Jenkins and colleagues43 found that mortality increased during surgery conferences compared to non-conference dates in trauma centres lacking verification (OR 1.2, 95% CI 1.1 to 1.4). However, among verified trauma centres, no association was observed. Piontek and colleagues45 found a 22% reduction in standardized mortality ratio following verification of a level II trauma centre. The study by Schubert and colleagues,9 one of the few studies that accounted for time-varying verification status during the study period, showed a protective association in lower-level centres (RR 0.84, 95% CI 0.72 to 0.99 for level III and RR 0.25, 95% CI 0.12 to 0.54 for level IV). Notrica and colleagues37 found that population-based pediatric injury mortality rates in US states with verified level I pediatric trauma centres were 37% lower than those in states without a verified pediatric trauma centre. The only time-series analysis showed that the lagged crude rate of verified level I pediatric trauma centres was protective and contributed to a decline (12%, 95% CI 4% to 18%) in the rate of change in adolescent injury mortality in the US between 1999 and 2015.53 A similar, but smaller, protective effect was observed for combined adult and pediatric verified level I trauma centres.

We included 15 studies in the meta-analysis of mortality.7,8,10,14,16,31,33,34,42,44,47–51 Analysis of crude RR (n = 11) showed that verification was generally associated with decreased mortality (Fig. 2A). This association was also observed in analysis of adjusted estimates (n = 7), except among severely injured patients (Injury Severity Score > 24) (OR 1.1, 95% CI 1.05 to 1.30) (Fig. 2B).

Fig 2

(A) Meta-analysis of crude association between trauma centre verification and in-hospital mortality. (B) Meta-analysis of risk-adjusted association between trauma centre verification and in-hospital mortality. Odds ratios (ORs) are presented instead of relative risks (RRs) because it was the effect measure reported by studies with adjusted analyses, and they did not provide enough details to compute adjusted RRs. ARDS = acquired respiratory distress syndrome; BAI = blunt abdominal injury; BCI = blunt cardiovascular injury; CI = confidence interval; ISS = Injury Severity Score; PAI = penetrating abdominal injury; PCI = penetrating cardiovascular injury; TBI = traumatic brain injury.*From random-effects model.

Funnel plots indicated a certain degree of asymmetry, which was more pronounced among studies providing crude estimates (Fig. 3A and Fig. 3B), which suggests publication bias. These figures also showed substantial variability among studies with larger samples. The GRADE results suggest that the quality of evidence is very low (Appendix 1, Supplemental Table S2).

Fig. 3

Funnel plots with pseudo 95% confidence limits (CLs) of studies showing the crude association between trauma centre verification and in-hospital mortality (A) and of studies showing the adjusted association between trauma centre verification and in-hospital mortality (B). OR = odds ratio; RR = relative risk; SE = standard error.

Resource use

Length of stay, including overall and intensive care unit, was the most studied outcome in the resource use category (10 of 12 studies).7,8,34,45,47–52 Other outcomes examined were blood products transfused, hospital charges, mechanical ventilation, bedside use of ultrasonography and use of recombinant factor VIIa (rFVIIa).36,40 Six of 12 studies focused on injured pediatric patients. Only 4 studies adjusted for at least 1 potential confounder.7,34,36,45

We conducted a meta-analysis of 7 studies assessing LOS. Three studies described the distribution of LOS using the median and interquartile range,49–51 and 4 presented the mean and standard deviation.8,34,45–48 We used a well-established technique to combine results reported on log-transformed or raw scales24,25 to conduct the analysis. Because of the skewed distribution of LOS, we computed only weighted mean differences of the log-transformed LOS (which can be interpreted as the geometric mean ratio when exponentiated). Our results suggest that American College of Surgeons verification was associated with longer intensive care unit LOS but not hospital LOS (Fig. 4).

Fig. 4

Meta-analysis of studies reporting an association between trauma centre verification and length of stay (log scale). Overall subpopulation = total length of stay estimate in a subgroup. Exponentiate of weighted mean differences (WMDs) can be interpreted as geometric mean ratio. BCI = blunt cardiovascular injury; CI = confidence interval; ICU = intensive care unit; ISS = Injury Severity Score; PAI = penetrating abdominal injury.

Funnel plots displayed asymmetry in favour of studies with increased LOS among verified centres (Fig. 5). Our GRADE assessment suggested that the evidence was of very low quality (Appendix 1, Table S2).

Fig. 5

Funnel plot with pseudo 95% confidence limits (CLs) of studies reporting the association between trauma centre verification and length of stay (LOS) (log scale). Overall subpopulation = total length of stay estimate in a subgroup. ICU = intensive care unit; SE = standard error. SE of effect size

Studies excluded from meta-analysis showed mixed and inconsistent results concerning the association between verification and use of various resources. For instance, Alexander and colleagues50 found that pediatric verification was associated with a decrease in the average number of blood products transfused per patient (7.2 units v. 2.4 units, 95% CI −10.1 to 0.6) and in professional charges (US–$16 171, 95% CI −$30 898 to −$1362). However, Piontek and colleagues45 reported that, after verification of a level II trauma centre, there was an increase in ventilation use (RR 1.30, 95% CI 1.12 to 1.51). Horton and colleagues36 surveyed level I and II trauma centres and found that American College of Surgeons verification was a predictor of rFVIIa use (OR 3.74, 95% CI 1.53 to 9.09) (Table 2).

Adverse events

Of the 7 studies that reported adverse events,7–10,14,45,50 3 adjusted for potential confounders.7,9,14 Four studies were cross-sectional, and 3 were pre–post. Two studies focused on injured pediatric patients, 1 study focused on adults, and 1 study focused on both pediatric and adult patients. Investigated outcomes included a wide range of complications such as pneumonia, pulmonary emboli, unplanned intubation, unplanned return to the operating room and unplanned readmission.

We did not conduct a meta-analysis of results for adverse events owing to the diversity of outcomes investigated; rather, we report them narratively. After risk adjustment, Agrawal and colleagues7 found lower odds of complications in verified centres than in state-designated centres (OR 0.88, 95% CI 0.87 to 0.90). Schubert and colleagues9 found a positive association between verified centres and unplanned intubation, especially among level I trauma centres (RR 1.53, 95% CI 1.11 to 1.65), after adjusting for hospital and patient characteristics. They did not observe an association between verification and unplanned return to the operating room. Piontek and colleagues45 found low evidence for changes in the incidence of complications (RR 1.27, 95% CI 0.86 to 1.89) or unplanned 30-day readmission (RR 0.91, 95% CI 0.77 to 1.08) after verification of a level II trauma centre. Likewise, Alexander and colleagues50 did not find an association between re-verification of a pediatric trauma centre and unplanned 30-day readmission at an already verified adult level I centre. The small number of patients (126) and readmissions (2), however, limit the interpretation of their findings. Conversely, Choi and colleagues8 found a decrease in unplanned hospital readmissions (RR 0.36, 95% CI 0.15 to 0.87) and hospital-acquired pneumonia (RR 0.41, 95% CI 0.17 to 0.99) 2 years after verification of a level I trauma centre. Grossman and colleagues,14 using a representative sample (n = 94) of US trauma centres, found that verified centres had a lower incidence of major complications (based on the National Trauma Data Bank definition54) than non-verified centres. This association was higher among older adults (OR 0.40, 95% CI 0.27 to 0.60) and children with an Injury Severity Score greater than 24 (OR 0.23, 95% CI 0.12 to 0.47). Finally, Smith and colleagues10 observed fewer cases of acute respiratory distress syndrome in verified level I trauma centres than in state-designated centres (RR 0.91, 95% CI 0.84 to 0.99) (Table 2).

Processes of care

In 4 of the 12 included studies, the authors adjusted for at least 1 potential confounder.34,35,41,52 Six studies focused on pediatric patients,39,41,46,49,50,52 2 on adults,34,37 and 4 on both pediatric and adult patients.34,36,38,44 There were 7 cross-sectional studies and 5 pre–post designs.

We did not conduct a meta-analysis owing to the diversity of outcomes investigated. In the pediatric population, a reduction in the incidence of splenectomy was found by Murphy and colleagues49 (2.7% among verified trauma centres v. 11% among nonverified centres) and Alarhayem and colleagues39 (6% among verified trauma centres v. 13% among nonverified centres) following verification of a pediatric level 1 centre. Alexander and colleagues50 observed a decrease in splenic interventions (i.e., splenectomy, splenorrhaphy or embolization) among children with blunt splenic injuries following pediatric verification (RR 0.36, 95% CI 0.13 to 0.99). Ehrlich and colleagues46 reported an improvement in pediatric trauma patient evaluation (including radiology) and time to emergency department discharge (< 120 min) following verification of a combined adult/pediatric level I trauma centre. Finally, after adjusting for Injury Severity Score, Bogumil and colleagues41 observed a higher prevalence of nonaccidental trauma in verified pediatric centres than in nonverified centres (prevalence ratio 1.81, 95% CI 1.73 to 1.90). This association was higher in level I centres (prevalence ratio 1.89, 95% CI 1.80 to 1.98) than level II centres (prevalence ratio 1.62, 95% CI 1.51 to 1.75).

Surveys of all state-designated US trauma centres in 2000 and 2006 showed that verified centres had a higher likelihood of full compliance with published guidelines for the management of severe traumatic brain injury for both 2000 (OR 5.1, 95% CI 1.1 to 23) and 2006 (OR 1.55, 95% CI 1.00 to 2.40).32,35 Similarly, Theologis and colleagues38 reported that verified level I trauma centres had a higher proportion of compliance with cervical spine clearance protocols than nonverified centres (75% v. 54%). Kim34 did not find any association between verification and time to surgery in patients with head injuries. Finally, Richardson and colleagues44 found that verification of a level III trauma centre was associated with an increase in the proportion of admissions of transferred patients into a referent level I centre (RR 1.19, 95% CI 1.03 to 1.36) (Table 2).

Discussion

In this systematic review and meta-analysis, we found mixed and inconsistent results for the association between trauma centre verification by the American College of Surgeons and all outcomes studied (in-hospital mortality, adverse events, resource use and processes of care). Nonetheless, verification was imprecisely associated with decreased mortality and longer LOS. Some evidence pointed to positive associations between verification and some processes of care, including adherence to published guidelines and reductions in the occurrence of complications. These findings, however, should be interpreted with caution given serious methodologic concerns about the quality of the empirical evidence.

First, inference of the obtained estimates is limited by the unclear nature of the control group in each study. For instance, in cross-sectional studies (18/29), it was not possible to distinguish those that failed during the verification process from those that never applied among nonverified centres. In addition, a quarter of multicentre studies (5/18) combined centres that had no trauma designation and state-designated centres as nonverified centres. If we placed ourselves in a trial framework, the results obtained from these studies would be neither intention-to-treat nor per-protocol estimates. This issue leads to selection and healthy-user biases, in the sense that high-performing centres may be more willing to seek verification than low-performing centres. This may affect the validity and generalizability of the observed associations. It is important to note that not all included studies assessed the impact of verification as the primary objective.

Although pre–post studies are less vulnerable to the biases mentioned above, they cannot account for the underlying trend in the measured outcomes before verification,55 which can bias estimates in either direction. An interesting alternative to assess verification benefits would be the use of quasiexperimental designs such as difference-in-differences and interrupted time-series, which are frequently used to assess the impact of policy and other population-level interventions in health research. These methods can account for unobservable or unmeasured variables that are fixed over time, and for secular trends in outcomes.56,57

Second, preparation for verification visits may lead to improvements in measured outcomes and therefore bias estimates of associations. Only 3 studies8,46,51 accounted for this, by removing the period just before verification in the analysis or via stratification. Of the 73 articles excluded from our review, 6 were excluded because the authors assessed only the preparation for verification visits.58–63

Finally, issues related to analytical methods may have biased the results. For instance, 45% of studies did not adjust for centre-level or patient-level risk factors. The latter is necessary to account for the changing epidemiologic features of trauma populations (e.g., due to population aging and possible change in referral patterns generally attributed to increased marketability).8 In addition, several papers reported ORs as a measure of association, but ORs are known to overestimate RRs, especially when the outcome is common.64 Only one-third (6/18) of multicentre studies accounted for this in their analysis, which may have led to type I errors and CIs that were too narrow.31 Also, the competing risk of death was not considered when LOS was assessed, and missing data were rarely handled appropriately (Table 3).

Our findings are similar to those of previous systematic reviews assessing verification in other health care fields65–69 that showed that many of the studies were heterogeneous and highly vulnerable to confounding, and added little clarity or guidance. They also highlighted major methodologic challenges such as self-selection and lack of robust controls, which limit their inference.

Limitations

Although the uptake of trauma centre verification is rising worldwide, all included articles were from the US. Pediatric patients were overrepresented. The inclusion of multiple study designs provided a more comprehensive assessment of the relevant literature; however, it introduces substantial heterogeneity, which, in turn, affects the robustness of meta-analysis estimates. Our choice of random-effects meta-analysis was based on the assumption that there might not be a common RR or OR applicable to all trauma populations.70 The small number of studies included in our meta-analysis made it difficult to properly summarize estimates and interpret funnel plots. Nonetheless, publication bias seems to be more likely in crude than in adjusted analyses. We also noted that several large studies fell outside the projected lines of the funnel plots, which indicates substantial variability among studies with small SEs.71 Since trauma-verification standards have evolved with time, we were unable to stratify our results by time, which may have introduced a bias.3 Studies were excluded from meta-analyses because of missing CIs or SEs and the scale of effect measure used, despite our efforts to compute desired statistics when raw data were available. Finally, the quality and strength of the cumulative evidence (as assessed with the GRADE framework) was very low.72

Conclusion

Our review illustrates the inability to extrapolate or infer causality on the effectiveness of trauma centre verification from the published literature owing to significant methodologic challenges, such as the lack of robust controls and the concentration of all the available studies in the US. Considering the prevalence and spread of trauma verification globally, this systematic review and meta-analysis underscores the need for quasiexperimental studies that assess the impact of trauma centre verification on changes in clinical processes of care and outcomes. Such studies may provide solid evidence to guide policy-making and individual hospitals’ decisions to seek verification.