Sara K Pasquali1, Amelia S Wallace2, J William Gaynor3, Marshall L Jacobs4, Sean M O'Brien2, Kevin D Hill2, Michael G Gaies5, Jennifer C Romano6, David M Shahian7, John E Mayer8, Jeffrey P Jacobs4. 1. Department of Pediatrics and Communicable Diseases, C.S. Mott Children's Hospital, Ann Arbor, Michigan. Electronic address: pasquali@med.umich.edu. 2. Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina. 3. Department of Surgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania. 4. Division of Cardiac Surgery, Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland; Division of Cardiovascular Surgery, Department of Surgery, Johns Hopkins All Children's Heart Institute, All Children's Hospital and Florida Hospital for Children, St. Petersburg, Tampa, and Orlando, Florida. 5. Department of Pediatrics and Communicable Diseases, C.S. Mott Children's Hospital, Ann Arbor, Michigan. 6. Department of Cardiac Surgery, University of Michigan Medical School, Ann Arbor, Michigan. 7. Department of Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts. 8. Department of Cardiovascular Surgery, Boston Children's Hospital, Boston, Massachusetts.
Abstract
BACKGROUND: Performance assessment in congenital heart surgery is challenging due to the wide heterogeneity of disease. We describe current case mix across centers, evaluate methodology inclusive of all cardiac operations versus the more homogeneous subset of Society of Thoracic Surgeons benchmark operations, and describe implications regarding performance assessment. METHODS: Centers (n = 119) participating in the Society of Thoracic Surgeons Congenital Heart Surgery Database (2010 through 2014) were included. Index operation type and frequency across centers were described. Center performance (risk-adjusted operative mortality) was evaluated and classified when including the benchmark versus all eligible operations. RESULTS: Overall, 207 types of operations were performed during the study period (112,140 total cases). Few operations were performed across all centers; only 25% were performed at least once by 75% or more of centers. There was 7.9-fold variation across centers in the proportion of total cases comprising high-complexity cases (STAT 5). In contrast, the benchmark operations made up 36% of cases, and all but 2 were performed by at least 90% of centers. When evaluating performance based on benchmark versus all operations, 15% of centers changed performance classification; 85% remained unchanged. Benchmark versus all operation methodology was associated with lower power, with 35% versus 78% of centers meeting sample size thresholds. CONCLUSIONS: There is wide variation in congenital heart surgery case mix across centers. Metrics based on benchmark versus all operations are associated with strengths (less heterogeneity) and weaknesses (lower power), and lead to differing performance classification for some centers. These findings have implications for ongoing efforts to optimize performance assessment, including choice of target population and appropriate interpretation of reported metrics.
BACKGROUND: Performance assessment in congenital heart surgery is challenging due to the wide heterogeneity of disease. We describe current case mix across centers, evaluate methodology inclusive of all cardiac operations versus the more homogeneous subset of Society of Thoracic Surgeons benchmark operations, and describe implications regarding performance assessment. METHODS: Centers (n = 119) participating in the Society of Thoracic Surgeons Congenital Heart Surgery Database (2010 through 2014) were included. Index operation type and frequency across centers were described. Center performance (risk-adjusted operative mortality) was evaluated and classified when including the benchmark versus all eligible operations. RESULTS: Overall, 207 types of operations were performed during the study period (112,140 total cases). Few operations were performed across all centers; only 25% were performed at least once by 75% or more of centers. There was 7.9-fold variation across centers in the proportion of total cases comprising high-complexity cases (STAT 5). In contrast, the benchmark operations made up 36% of cases, and all but 2 were performed by at least 90% of centers. When evaluating performance based on benchmark versus all operations, 15% of centers changed performance classification; 85% remained unchanged. Benchmark versus all operation methodology was associated with lower power, with 35% versus 78% of centers meeting sample size thresholds. CONCLUSIONS: There is wide variation in congenital heart surgery case mix across centers. Metrics based on benchmark versus all operations are associated with strengths (less heterogeneity) and weaknesses (lower power), and lead to differing performance classification for some centers. These findings have implications for ongoing efforts to optimize performance assessment, including choice of target population and appropriate interpretation of reported metrics.
Authors: David M Shahian; Treacy Silverstein; Ann F Lovett; Robert E Wolf; Sharon-Lise T Normand Journal: Circulation Date: 2007-03-12 Impact factor: 29.690
Authors: David M Shahian; Xia He; Jeffrey P Jacobs; J Scott Rankin; Eric D Peterson; Karl F Welke; Giovanni Filardo; Cynthia M Shewan; Sean M O'Brien Journal: Ann Thorac Surg Date: 2013-06-29 Impact factor: 4.330
Authors: Jeffrey Phillip Jacobs; Sean M O'Brien; Sara K Pasquali; Marshall Lewis Jacobs; Francois G Lacour-Gayet; Christo I Tchervenkov; Erle H Austin; Christian Pizarro; Kamal K Pourmoghadam; Frank G Scholl; Karl F Welke; Constantine Mavroudis Journal: Ann Thorac Surg Date: 2011-12 Impact factor: 4.330
Authors: Sean M O'Brien; David R Clarke; Jeffrey P Jacobs; Marshall L Jacobs; Francois G Lacour-Gayet; Christian Pizarro; Karl F Welke; Bohdan Maruszewski; Zdzislaw Tobota; Weldon J Miller; Leslie Hamilton; Eric D Peterson; Constantine Mavroudis; Fred H Edwards Journal: J Thorac Cardiovasc Surg Date: 2009-11 Impact factor: 5.209
Authors: Jeffrey P Jacobs; David M Shahian; Richard L Prager; Fred H Edwards; Donna McDonald; Jane M Han; Richard S D'Agostino; Marshall L Jacobs; Benjamin D Kozower; Vinay Badhwar; Vinod H Thourani; Henning A Gaissert; Felix G Fernandez; Cam Wright; James I Fann; Gaetano Paone; Juan A Sanchez; Joseph C Cleveland; J Matthew Brennan; Rachel S Dokholyan; Sean M O'Brien; Eric D Peterson; Frederick L Grover; G Alexander Patterson Journal: Ann Thorac Surg Date: 2015-10-31 Impact factor: 4.330
Authors: Mark J Peters; Andrew Argent; Marino Festa; Stéphane Leteurtre; Jefferson Piva; Ann Thompson; Douglas Willson; Pierre Tissières; Marisa Tucci; Jacques Lacroix Journal: Intensive Care Med Date: 2017-03-17 Impact factor: 17.440
Authors: Brian D Benneyworth; Christopher W Mastropietro; Eric M Graham; Darren Klugman; John M Costello; Wenying Zhang; Michael Gaies Journal: J Thorac Cardiovasc Surg Date: 2017-02-04 Impact factor: 5.209
Authors: Dayanand N Bagdure; Jason W Custer; Cortney B Foster; William C Blackwelder; Vladimir Mishcherkin; Allison Portenoy; Adnan Bhutta Journal: J Pediatr Intensive Care Date: 2020-08-10
Authors: Sarah Tabbutt; Jennifer Schuette; Wenying Zhang; Jeffrey Alten; Janet Donohue; J William Gaynor; Nancy Ghanayem; Jeffrey Jacobs; Sara K Pasquali; Ravi Thiagarajan; Justin B Dimick; Mousumi Banerjee; David Cooper; Michael Gaies Journal: Pediatr Crit Care Med Date: 2019-02 Impact factor: 3.624
Authors: Sean M O'Brien; Jeffrey P Jacobs; David M Shahian; Marshall L Jacobs; J William Gaynor; Jennifer C Romano; Michael G Gaies; Kevin D Hill; John E Mayer; Sara K Pasquali Journal: Ann Thorac Surg Date: 2018-09-15 Impact factor: 4.330
Authors: Sara K Pasquali; David M Shahian; Sean M O'Brien; Marshall L Jacobs; J William Gaynor; Jennifer C Romano; Michael G Gaies; Kevin D Hill; John E Mayer; Jeffrey P Jacobs Journal: Ann Thorac Surg Date: 2018-09-15 Impact factor: 4.330