BACKGROUND AND OBJECTIVES: Reporting of standardized patient and graft survival rates by the Scientific Registry of Transplant Recipients (SRTR) aims to influence transplant centers to improve their performance. The methodology currently used is based on calculating observed-to-expected (OE) ratios for every center. Its accuracy has not been evaluated. Here, we compare the accuracy of standardized rates across centers with the OE method to an alternative generalized mixed-effect (ME) method. We also examine the association between public reporting and center outcome improvement. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: Accuracy was measured as the root mean square error (RMSE) of the difference between standardized rates from one time period to standardized rates from a future time period. Data from the United States Renal Data System on all kidney transplants between January 1, 1996, and September 30, 2009 were analyzed. RESULTS: The ME method had a 0.5 to 4.5% smaller RMSE than the OE method. It also had a smaller range between the 5(th) and 95(th) percentile centers' standardized rates: 7.5% versus 10.5% for 3-year graft survival and 4.7% versus 7.9% for 3-year patient survival. The range did not change after the introduction of public reporting in 2001. In addition, 33% of all deaths and 29% of all graft failures in the 3 years after transplant could be attributed to differences across centers. CONCLUSIONS: The ME method can improve the accuracy of public reports on center outcomes. An examination of the reasons why public reports have not reduced differences across centers is necessary.
BACKGROUND AND OBJECTIVES: Reporting of standardized patient and graft survival rates by the Scientific Registry of Transplant Recipients (SRTR) aims to influence transplant centers to improve their performance. The methodology currently used is based on calculating observed-to-expected (OE) ratios for every center. Its accuracy has not been evaluated. Here, we compare the accuracy of standardized rates across centers with the OE method to an alternative generalized mixed-effect (ME) method. We also examine the association between public reporting and center outcome improvement. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: Accuracy was measured as the root mean square error (RMSE) of the difference between standardized rates from one time period to standardized rates from a future time period. Data from the United States Renal Data System on all kidney transplants between January 1, 1996, and September 30, 2009 were analyzed. RESULTS: The ME method had a 0.5 to 4.5% smaller RMSE than the OE method. It also had a smaller range between the 5(th) and 95(th) percentile centers' standardized rates: 7.5% versus 10.5% for 3-year graft survival and 4.7% versus 7.9% for 3-year patient survival. The range did not change after the introduction of public reporting in 2001. In addition, 33% of all deaths and 29% of all graft failures in the 3 years after transplant could be attributed to differences across centers. CONCLUSIONS: The ME method can improve the accuracy of public reports on center outcomes. An examination of the reasons why public reports have not reduced differences across centers is necessary.
Authors: D M Shahian; S L Normand; D F Torchiana; S M Lewis; J O Pastore; R E Kuntz; P I Dreyer Journal: Ann Thorac Surg Date: 2001-12 Impact factor: 4.330
Authors: Eric D Peterson; Matthew T Roe; Jyotsna Mulgund; Elizabeth R DeLong; Barbara L Lytle; Ralph G Brindis; Sidney C Smith; Charles V Pollack; L Kristin Newby; Robert A Harrington; W Brian Gibler; E Magnus Ohman Journal: JAMA Date: 2006-04-26 Impact factor: 56.272
Authors: P D McGrath; D E Wennberg; J D Dickens; A E Siewers; F L Lucas; D J Malenka; M A Kellett; T J Ryan Journal: JAMA Date: 2000-12-27 Impact factor: 56.272
Authors: Jesse D Schold; Laura D Buccini; David A Goldfarb; Stuart M Flechner; Emilio D Poggio; Ashwini R Sehgal Journal: Clin J Am Soc Nephrol Date: 2014-09-18 Impact factor: 8.237
Authors: J D Schold; L D Buccini; E L G Heaphy; D A Goldfarb; A R Sehgal; J Fung; E D Poggio; M W Kattan Journal: Am J Transplant Date: 2013-05-24 Impact factor: 8.086
Authors: Anne Tsampalieros; Gregory A Knoll; Nicholas Fergusson; Alexandria Bennett; Monica Taljaard; Dean Fergusson Journal: Can J Kidney Health Dis Date: 2017-10-19