OBJECTIVE: We sought to determine the reliability of surgeon-specific postoperative complication rates after colectomy. BACKGROUND: Conventional measures of surgeon-specific performance fail to acknowledge variation attributed to statistical noise, risking unreliable assessment of quality. METHODS: We examined all patients who underwent segmental colectomy with anastomosis from 2008 through 2010 participating in the Michigan Surgical Quality Collaborative Colectomy Project. Surgeon-specific complication rates were risk-adjusted according to patient characteristics with multiple logistic regression. Hierarchical modeling techniques were used to determine the reliability of surgeon-specific risk-adjusted complication rates. We then adjusted these rates for reliability. To evaluate the extent to which surgeon-level variation was reduced, surgeons were placed into quartiles based on performance and complication rates were compared before and after reliability adjustment. RESULTS: A total of 5033 patients (n = 345 surgeons) undergoing partial colectomy reported a risk-adjusted complication rate of 24.5%. Approximately 86% of the variability of complication rates across surgeons was explained by measurement noise, whereas the remaining 14% represented true signal. Risk-adjusted complication rates varied from 0% to 55.1% across quartiles before adjusting for reliability. Reliability adjustment greatly diminished this variation, generating a 1.2-fold difference (21.4%-25.6%). A caseload of 168 colectomies across 3 years was required to achieve a reliability of more than 0.7, which is considered a proficient level. Only 1 surgeon surpassed this volume threshold. CONCLUSIONS: The vast majority of surgeons do not perform enough colectomies to generate a reliable surgeon-specific complication rate. Risk-adjusted complication rates should be viewed with caution when evaluating surgeons with low operative volume, as statistical noise is a large determinant in estimating their surgeon-specific complication rates.
OBJECTIVE: We sought to determine the reliability of surgeon-specific postoperative complication rates after colectomy. BACKGROUND: Conventional measures of surgeon-specific performance fail to acknowledge variation attributed to statistical noise, risking unreliable assessment of quality. METHODS: We examined all patients who underwent segmental colectomy with anastomosis from 2008 through 2010 participating in the Michigan Surgical Quality Collaborative Colectomy Project. Surgeon-specific complication rates were risk-adjusted according to patient characteristics with multiple logistic regression. Hierarchical modeling techniques were used to determine the reliability of surgeon-specific risk-adjusted complication rates. We then adjusted these rates for reliability. To evaluate the extent to which surgeon-level variation was reduced, surgeons were placed into quartiles based on performance and complication rates were compared before and after reliability adjustment. RESULTS: A total of 5033 patients (n = 345 surgeons) undergoing partial colectomy reported a risk-adjusted complication rate of 24.5%. Approximately 86% of the variability of complication rates across surgeons was explained by measurement noise, whereas the remaining 14% represented true signal. Risk-adjusted complication rates varied from 0% to 55.1% across quartiles before adjusting for reliability. Reliability adjustment greatly diminished this variation, generating a 1.2-fold difference (21.4%-25.6%). A caseload of 168 colectomies across 3 years was required to achieve a reliability of more than 0.7, which is considered a proficient level. Only 1 surgeon surpassed this volume threshold. CONCLUSIONS: The vast majority of surgeons do not perform enough colectomies to generate a reliable surgeon-specific complication rate. Risk-adjusted complication rates should be viewed with caution when evaluating surgeons with low operative volume, as statistical noise is a large determinant in estimating their surgeon-specific complication rates.
Authors: Stephanie G Yi; Nelda P Wray; Stephen L Jones; Barbara L Bass; Jeanne Nishioka; Sarah Brann; Carol M Ashton Journal: J Am Coll Surg Date: 2013-06-29 Impact factor: 6.113
Authors: Bruce L Hall; Barton H Hamilton; Karen Richards; Karl Y Bilimoria; Mark E Cohen; Clifford Y Ko Journal: Ann Surg Date: 2009-09 Impact factor: 12.969
Authors: Michael J Englesbe; Linda Brooks; James Kubus; Martin Luchtefeld; James Lynch; Anthony Senagore; John C Eggenberger; Vic Velanovich; Darrell A Campbell Journal: Ann Surg Date: 2010-09 Impact factor: 12.969
Authors: Aristithes G Doumouras; Fady Saleh; Sama Anvari; Scott Gmora; Mehran Anvari; Dennis Hong Journal: Surg Endosc Date: 2017-04-13 Impact factor: 4.584
Authors: Vahagn C Nikolian; Neil S Kamdar; Scott E Regenbogen; Arden M Morris; John C Byrn; Pasithorn A Suwanabol; Darrell A Campbell; Samantha Hendren Journal: Surgery Date: 2017-02-21 Impact factor: 3.982
Authors: J A Pereira; A Bravo-Salva; B Montcusí; S Pérez-Farre; L Fresno de Prado; M López-Cano Journal: BMC Surg Date: 2019-08-07 Impact factor: 2.102