BACKGROUND: Currently, trauma center quality benchmarking is based on risk adjusted observed-expected (O/E) mortality ratios. However, failure to account for number of patients has been recently shown to produce unreliable mortality estimates, especially for low-volume centers. This study explores the effect of reliability adjustment (RA), a statistical technique developed to eliminate bias introduced by low volume on risk-adjusted trauma center benchmarking. METHODS: Analysis of the National Trauma Data Bank 2010 was performed. Patients 16 years or older with blunt or penetrating trauma and an Injury Severity Score (ISS) of 9 or greater were included. Based on the statistically accepted standards of the Trauma Quality Improvement Program methodology, risk-adjusted mortality rates were generated for each center and used to rank them accordingly. Hierarchical logistic regression modeling was then performed to adjust these rates for reliability using an empiric Bayes approach. The impact of RA was examined by (1) recalculating interfacility variations in adjusted mortality rates and (2) comparing adjusted hospital mortality quintile rankings before and after RA. RESULTS: A total of 557 facilities (with 278,558 patients) were included. RA significantly reduced the variation in risk-adjusted mortality rates between centers from 14-fold (0.7-9.8%) to only 2-fold (4.4-9.6%) after RA. This reduction in variation was most profound for smaller centers. A total of 68 "best" hospitals and 18 "worst" hospitals based on current risk adjustment methods were reclassified after performing RA. CONCLUSION: "Reliability adjustment" dramatically reduces variations in risk-adjusted mortality arising from statistical noise, especially for lower volume centers. Moreover, the absence of RA had a profound impact on hospital performance assessment, suggesting that nearly one of every six hospitals in National Trauma Data Bank would have been inappropriately placed among the very best or very worst quintile of rankings. RA should be considered while benchmarking trauma centers based on mortality.
BACKGROUND: Currently, trauma center quality benchmarking is based on risk adjusted observed-expected (O/E) mortality ratios. However, failure to account for number of patients has been recently shown to produce unreliable mortality estimates, especially for low-volume centers. This study explores the effect of reliability adjustment (RA), a statistical technique developed to eliminate bias introduced by low volume on risk-adjusted trauma center benchmarking. METHODS: Analysis of the National Trauma Data Bank 2010 was performed. Patients 16 years or older with blunt or penetrating trauma and an Injury Severity Score (ISS) of 9 or greater were included. Based on the statistically accepted standards of the Trauma Quality Improvement Program methodology, risk-adjusted mortality rates were generated for each center and used to rank them accordingly. Hierarchical logistic regression modeling was then performed to adjust these rates for reliability using an empiric Bayes approach. The impact of RA was examined by (1) recalculating interfacility variations in adjusted mortality rates and (2) comparing adjusted hospital mortality quintile rankings before and after RA. RESULTS: A total of 557 facilities (with 278,558 patients) were included. RA significantly reduced the variation in risk-adjusted mortality rates between centers from 14-fold (0.7-9.8%) to only 2-fold (4.4-9.6%) after RA. This reduction in variation was most profound for smaller centers. A total of 68 "best" hospitals and 18 "worst" hospitals based on current risk adjustment methods were reclassified after performing RA. CONCLUSION: "Reliability adjustment" dramatically reduces variations in risk-adjusted mortality arising from statistical noise, especially for lower volume centers. Moreover, the absence of RA had a profound impact on hospital performance assessment, suggesting that nearly one of every six hospitals in National Trauma Data Bank would have been inappropriately placed among the very best or very worst quintile of rankings. RA should be considered while benchmarking trauma centers based on mortality.
Authors: D M Shahian; S L Normand; D F Torchiana; S M Lewis; J O Pastore; R E Kuntz; P I Dreyer Journal: Ann Thorac Surg Date: 2001-12 Impact factor: 4.330
Authors: Tolulope A Oyetunji; Joseph G Crompton; Imudia D Ehanire; Kent A Stevens; David T Efron; Elliott R Haut; David C Chang; Edward E Cornwell; Marie L Crandall; Adil H Haider Journal: J Surg Res Date: 2010-10-16 Impact factor: 2.192
Authors: G Barmparas; E J Ley; M J Martin; A Ko; M Harada; D Weigmann; K R Catchpole; B L Gewertz Journal: Eur J Trauma Emerg Surg Date: 2017-03-22 Impact factor: 3.693
Authors: Peter C Jenkins; Lava Timsina; Patrick Murphy; Christopher Tignanelli; Daniel N Holena; Mark R Hemmila; Craig Newgard Journal: Ann Surg Date: 2022-02-01 Impact factor: 13.787
Authors: Adil H Haider; Zain G Hashmi; Sonia Gupta; Syed Nabeel Zafar; Jean-Stephane David; David T Efron; Kent A Stevens; Hasnain Zafar; Eric B Schneider; Eric Voiglio; Raul Coimbra; Elliott R Haut Journal: World J Surg Date: 2014-08 Impact factor: 3.352
Authors: Andrew A Gonzalez; Micah E Girotti; Terry Shih; Thomas W Wakefield; Justin B Dimick Journal: J Vasc Surg Date: 2014-03-12 Impact factor: 4.268
Authors: Ryan P Barbaro; Robert H Bartlett; Rachel L Chapman; Matthew L Paden; Lloyd A Roberts; Achamyeleh Gebremariam; Gail M Annich; Matthew M Davis Journal: J Pediatr Date: 2016-03-19 Impact factor: 4.406
Authors: Paul G Barnett; Josephine C Jacobs; Jeffrey G Jarvik; Roger Chou; Derek Boothroyd; Jeanie Lo; Andrea Nevedal Journal: JAMA Netw Open Date: 2020-07-01