Risk-adjusted outcomes in Medicare inpatient nephrectomy patients.

Without risk-adjusted outcomes of surgical care across both the inpatient and postacute period of time, hospitals and surgeons cannot evaluate the effectiveness of current performance in nephrectomy and other operations, and will not have objective metrics to gauge improvements from care redesign efforts.We compared risk-adjusted hospital outcomes following elective total and partial nephrectomy to demonstrate differences that can be used to improve care. We used the Medicare Limited Dataset for 2010 to 2012 for total and partial nephrectomy for benign and malignant neoplasms to create prediction models for the adverse outcomes (AOs) of inpatient deaths, prolonged length-of-stay outliers, 90-day postdischarge deaths without readmission, and 90-day relevant readmissions. From the 4 prediction models, total predicted adverse outcomes were determined for each hospital in the dataset that met a minimum of 25 evaluable cases for the study period. Standard deviations (SDs) for each hospital were used to identify specific z-scores. Risk-adjusted adverse outcomes rates were computed to permit benchmarking each hospital's performance against the national standard. Differences between best and suboptimal performing hospitals defined the potential margin of preventable adverse outcomes for this operation.A total of 449 hospitals with 23,477 patients were evaluated. Overall AO rate was 20.8%; 17 hospitals had risk-adjusted AO rates that were 2 SDs poorer than predicted and 8 were 2 SDs better. The top performing decile of hospitals had a risk-adjusted AO rate of 10.2% while the lowest performing decile had 32.1%. With a minimum of 25 cases for each study hospital, no statistically valid improvement in outcomes was seen with increased case volume.Inpatient and 90-day postdischarge risk-adjusted adverse outcomes demonstrated marked variability among study hospitals and illustrate the opportunities for care improvement. This analytic design is applicable for comparing provider performance across a wide array of different inpatient episodes.

Introduction

Nephrectomy is a common operation that is performed for neoplasms, trauma, living organ donation, and other disease conditions that make retention of the organ undesirable. In 2013, estimates of 45,610 unilateral total nephrectomies and 19,610 partial nephrectomies were performed in the United States.[ The outcomes of nephrectomy care are reported primarily for inpatient results, with complication rates being highly variable. Only a limited number of reports have examined postdischarge outcomes.

The lack of consistent and reproducible metrics for measuring outcomes in inpatient surgical care has led our group to adopt 4 objective measurements for the surgical adverse outcome (AO): inpatient mortality, risk-adjusted postoperative length of stay, 90-day postdischarge death without readmission, and 90-day readmission to an acute care hospital. Each of these 4 components can be independently risk-adjusted.[ By using the Medicare data set, the full scope of postdischarge events can be captured in the analysis. The Medicare population only constitutes one-third of the total/partial nephrectomies annually in the United States, but it provides a high-risk population for definition of important comorbid conditions and provides a benchmark for care redesign and improvement. Thus, this study will report a risk-adjusted national prediction model for nephrectomy surgery across the full continuum of inpatient and 90-days of postdischarge outcomes. This design can be used for comparing provider performance and measurement of care improvement efforts within hospitals across a wide array of different inpatient episodes of care.

Methods

Database development

The Centers for Medicare and Medicaid Services (CMS) inpatient limited data set for 2010 to 2012 was used to identify all elective nephrectomy patients with an International Classification of Diseases 9th Revision-Clinical Modification (ICD-9) procedure code of 55.4 (partial nephrectomy) or 55.51 (nephroureterectomy). This research was not approved by any Institutional Review Board. The use of the CMS inpatient limited data set for research does not require Institutional Review Board review, but the authors are expected to comply with federal policy of not reporting data cells of less than 11 observations to preserve patient confidentiality. Nephroureterectomy is an all-inclusive term used in ICD-9 coding to identify all total nephrectomy operations regardless of the length of ureter that may have been resected. A code of 17.41 to 17.43 identified the case as robotic-assisted. No ICD-9 codes are available to differentiate open nephrectomy from laparoscopic assisted procedures. All qualifying cases were required to have an ICD-9 principal diagnosis of 189.0 to 189.2, 223.0, or 223.1. Procedures were only included if performed on days 0, 1, or 2 of hospitalization and patients were 65 years of age or older. Missing data, transfers from another acute care hospital, and all discharges against medical advice were excluded.

Two separate but overlapping databases were used in this study. The developmental database was used for the patient-level design of risk-factors. It included all cases meeting the above criteria that were from hospitals with 20 or more qualifying cases for the study period, but only for those hospitals meeting accurate coding criteria that we have previously developed.[ A minimum of 20 cases for each hospital is required for the control chart methods detailed below, and the use of good coding hospitals for model development is to optimize accuracy in final prediction equations. Final predictive models were then applied to all hospitals and patients in the study database independent of coding accuracy, but only for hospitals that had 4.5 predicted AOs to avoid excessively small numbers in statistical evaluation.

Model development

An extensive group of candidate risk factors including medical comorbidities identified as present-on-admission, and specific diagnoses (e.g., cancer) were used in model development. Similar low-volume variables with potential univariate significance were aggregated together. Hospital dummy variables were used in each predictive model to account for hospital effects.[ In predictive modeling, hospital effects can have a large influence on the coefficients of final risk factors. Hospital dummy variables eliminate hospital effects so that variables in final models are not affected by extreme hospital performances and were then removed to avoid hospital influences and bias on final risk factor coefficients.

Stepwise logistic regression was used to develop final prediction models for each of the 4 AOs (Fig. 1). First, a model was developed for inpatient deaths (IpD). Second, all live discharges were evaluated for risk-adjusted prolonged length-of stay (prLOS). In surgical care standardized definitions, surveillance, and reporting for complications of surgical care are not present.[ Because specific coded complications are numerous and inconsistent following major operations with many having no measurable impact on ultimate outcomes, we have developed prLOS as a surrogate marker for severe inpatient complications.[ This necessitates only a single prediction model as a composite representation of significant inpatient morbidity among live discharges. The prLOS patients are identified by developing a linear prediction model for length-of-stay among patients without coded complications, identification of excess lengths of stay against prediction values in patients without coded complications, and identification of those cases where observed minus predict values exceed the upper control limit by 3-σ using a moving-range control chart. The 3-σ outliers have major complications, have dramatic increases in costs of care, and are strong predictors of postdischarge AOs.[ Once prLOS patients are identified, they become the dependent variable in a logistic risk equation to predict this surrogate event as a marker of severe inpatient complications.

Figure 1

A schematic representation of how each segment of the nephrectomy population of study patients was used in the development of each of the final 4 prediction models used in this study.

The third model is for 90-day deaths following discharge of nephrectomy patients that were not readmitted to an acute care hospital (PD-90). The fourth prediction model is for 90-day readmissions (RA-90) among patients who survived the entire 90-day postdischarge period of time. Candidate risk factors for the postdischarge events were the same as for inpatient adverse outcomes except for the addition of prLOS from the index hospitalization. Readmissions for medical diagnostic categories (MDCs) 2 (eye diseases), 17 (myeloproliferative diseases), 22/24 (burns/major trauma), and all medicare severity-diagnosis related groups (MS-DRGs) related to the management of trauma or cancer regardless of MDC were excluded as readmissions not associated with the index hospitalization.

The 90-day postdischarge window has been selected since it is consistent with Model 2 of the CMS bundled payment for care improvement initiative that is currently being conducted by Medicare [ and it is consistent with the comprehensive care of joint replacement program that is being launched by Medicare,[ which makes 90-days the likely interval for accountability in pending Medicare bundled payment initiatives. Our prior studies in other major surgical areas have identified over 40% of readmissions that are associated with the index hospitalization occur between 30 and 90 days after discharge.[

Final models contained only variables with P <0.001. Schwarz criterion was used to prevent over-fitting final models.[ Final models were evaluated by C-statistics.[ All analyses were performed with SAS software (Version 9.4, SAS Institute, Cary, NC).

Hospital performance

Only hospitals with 25 or more cases in the study dataset met criteria of ≥4.5 predicted AOs and were used to measure comparative performance. The final dataset had total predicted AOs set equal to total observed AOs to account for the 1.4% difference from the application of the developmental dataset. Total observed AOs for each hospital were determined by only identifying the first qualifying AO for each patient to avoid double-counting. Total predicted AOs were identified by respective prediction models using only those cases that were eligible at each stage. The standard deviation (SD) for predicted cases in each hospital was computed by the formula: , where P is the probability of an AO and N is the number of total cases within each hospital. The Z-score = (observed AOs minus predicted AOs)/SD was then computed. Negative Z-scores indicated performance that was better than predicted and positive Z-scores indicated suboptimal performance. The risk-adjusted AO rate for each hospital was computed by multiplication of the overall population AO rate by the observed/predicted AO rate of that specific hospital.[ The risk-adjusted AO rate expresses the overall outcome of the hospital on a linear scale that would be expected in the entire population of patients. Performance that demonstrates the observed-to-predicted ratio of 1.0 for an overall population AO rate of 15% would have a risk-adjusted rate of 15%. If the observed-to-predicted ratio was 2.0, then the risk-adjusted rate would be 15% multiplied by 2.0, or 30%. If the observed to predicted ratio was 0.5, then the risk-adjusted rate would be 7.5%. The method adjusts all hospital performance to that which would be expected by multiplying each hospitals ratio of observe-to-predicted rates against the performance of the national performance. All hospitals were then grouped by decile of performance for comparative evaluation. Regression analysis was performed by hospital volume of cases against hospital AO rates to assess the influence of case load upon outcome results.

Results

There were a total of 26,417 patients in the total dataset. A total of 23,193 patients (87.7%) were in the developmental database for IPD model design that came from good coding hospitals. There were 175 inpatient deaths (0.8%). There were 10 significant risk factors in the IPD model, with a c-statistic of 0.790. For prLOS, there were 1471 length-of-stay outliers (6.3%), 25 risk factors, and a c-statistic of 0.696 in the final model. There were 206 (0.9% of total cases) deaths in PD-90, the risk model had 7 risk factors, and the c-statistic was 0.811. A total of 3400 patients (14.7% of total cases) in the developmental database were 90-day readmissions following exclusion of nonassociated MS-DRGs. There were 22 significant risk factors in the RA-90 model with a c-statistic of 0.660. The odds ratios of significant risk factors in prediction models are identified in Table 1. PrLOS for the index hospitalization was a significant predictor of both PD-90 and RA-90. A total of 5123 patients (22.1%) underwent robotic-assisted nephrectomy or partial nephrectomy. Robotic-assisted procedures were associated with significantly lower rates of prLOS and RA-90, but no benefits in mortality rates.

Table 1

The risk factors and odds ratios (± standard error) for predictive models in nephrectomy.

The MS-DRGs of readmissions in the developmental database are presented in Table 2. A total of 3399 patients (14.7%) were readmitted 4382 times during the 90 days following discharge. A total of 2339 readmissions (53.4%) were in the first 30 days, 1130 (25.8%) were between days 31 and 60, and 913 (20.8%) were from day 61 to 90. Readmissions were from cardiac, infectious, gastrointestinal, kidney/urinary tract, and other causes.

Table 2

Major causes of 90-day readmissions following nephrectomy.

There were 23,477 patients from 439 hospitals in the study database that had 25 or more cases and predicted AOs of 4.5 or more. The addition of patients from all hospitals in the study database was then reduced by nearly an equivalent amount by the increased in the threshold for hospital evaluation to 25 instead of 20 cases. A total of 176 patients (0.75%) died during inpatient care and 1561 (6.7%) were live discharges with prLOS. An additional 307 patients (1.3%) died within 90 days following discharge without readmission, and 3458 live discharged patients (14.7%) were readmitted 1 or more times. Among readmitted patients, another 208 patients died within 90 days for a total mortality rate including inpatient and 90-day postdischarge deaths of 2.9% (691 patients). There were 4891 patients (20.8%) who had 1 or more AOs including inpatient and the 90-day postdischarge period.

The Z-scores of the 439 hospitals are in Fig. 2. A total of 8 hospitals (1.8%) had outcomes that were better than 2 SDs from the average and 17 hospitals (3.9 %) were 2 SDs poorer than average. In Fig. 3, the risk-adjusted median AO rates of hospitals are presented by decile of performance. The error brackets define the interquartile range within each decile. The best performing decile had a median risk-adjusted AO rate of 10.2% while the poorest performing decile of hospitals had a risk-adjusted AO rate of 32.1%.

Figure 2

The variability of hospital adverse outcomes z-scores in this study population.

Figure 3

Risk-adjusted adverse outcomes by decile of hospital performance.

Hospital case volume did not have a significant influence upon AOs. Hospitals in the smallest decile had 25 to 27 cases for the study period. The hospitals in the largest volume decile had more than 105 cases. A weak correlation coefficient of 0.08 favored smaller hospitals with better outcomes, but no statistical significance was seen (P = 0.16) with analysis of variance.

Discussion

We have identified a dramatic difference in performance by hospitals using the 4 risk-adjusted metrics in nephrectomy. Best performing hospitals had adverse outcome rates approaching 10% while poorest performing hospitals exceeded 30%. This dramatic difference indicates that an opportunity for major improvement is present. Two-thirds of AOs following nephrectomy occurred after discharge. It is likely that hospitals and urologic surgeons are not completely aware of postdischarge AOs among their patients given the high rates of readmissions to other facilities. Improvement of outcomes must begin with better methods for tracking patient outcomes than has previously existed. This will result in better care for patients and reduce penalties from Medicare for excessive readmission rates.[ Predictive modeling for readmissions has been difficult[ and likely relates to socioeconomic, geographic, and patient compliance issues being major factors.[ Innovative strategies for reducing readmissions will require more than management of medical issues.

The prediction models of Table 1 identify those risk factors that are associated with the 4 categories of AOs. These predictive models can be of use in describing the patient conditions that require increased attention by clinicians caring for nephrectomy patients. Many such as advance age are obvious. Cardiac, pulmonary, and biliary disease when present predict adverse outcomes across the inpatient and postdischarge period and require focused attention for prevention. Better postdischarge follow up and improved patient contact after discharge can improve readmission rates. Of particular interest, patients who have major inpatient complications as identified by prolonged length-of-stay predict postdischarge death and readmission in nephrectomy patients and in surgical patients in general.[ Special follow-up and surveillance strategies are necessary to improve outcomes for this population in particular. The trends in hospitalized surgical care over recent decades have resulted in shorter lengths-of-inpatient care. Patients are rapidly discharged and the consequences have been that more complications of care are not identified until after discharge. Outcomes of nephrectomy and other operations cannot be accurately made without inclusion of postdischarge events. Postdischarge deaths and readmissions may not be captured by quality review initiatives or even by the operating surgeon. A total of 20% to 40% of discharged surgical cases is readmitted to hospitals other than the primary institution.[ The use of Medicare administrative claims data permits tracking the patients over time and accurate identification of all AOs across the continuum of care, including postdischarge deaths. While the Medicare database may be criticized for having only the elderly nephrectomy patient, alternative patient samples across all age groups (e.g., National Inpatient Sample from the Healthcare Cost and Utilization Project) do not have encrypted patient identifiers that permit identification of postdischarge deaths and readmissions.

While most clinicians recognize that postdischarge events need to be included in outcome assessments, the duration of postdischarge time included in this measurement remains controversial. Traditional mortality and complication rates that are reported for urology and other surgical care have been inclusive of 30-days following the procedure.[ Many have used this 30-day interval for reporting postdischarge readmissions.[ However, as is illustrated in Table 2, significant readmissions occur from days 31 to 90. The comprehensive care for joint replacement initiative by Medicare will be inclusive of 90-days following discharge,12 and it can be expected that other “bundled” payment programs by CMS in the future including nephrectomy will follow this prototype.8 It is also likely that private payers will follow the Medicare bundled payment model. Understanding why patients are readmitted as is illustrated in Table 2, and developing care redesign strategies to avoid unnecessary readmissions (and emergency department visits) will be essential.

Discussion has surrounded whether better outcomes are a function of hospital volume for specific operations. Some have argued that better outcomes follow larger volumes,[ while others have not identified that relationship.[ In this study, large volume and small volume centers had comparable risk-adjusted results. It does not appear that better results are associated with larger volumes of cases particularly when postdischarge adverse outcomes are included.

Other studies of nephrectomy outcomes including postdischarge events have been limited. Gore et al[ studied all-payer radical nephrectomy over 5 years in the State of Washington. They identified highly variable rates of prolonged length of stay (≥75th percentile) among hospitals using methodology different from our study. Overall mortality rates were similar to our report, but 30-day readmission rates demonstrated little variability among hospitals as opposed to the highly variable rates that we identified. They too identified no benefit with increased hospital case volume. They concluded that hospital variation was a major issue in urologic care including radical cystectomy, prostatectomy, and nephrectomy. Similarly, Hwang et al[ demonstrated hospital length-of-stay for radical nephrectomy was a significant predictor of deaths and readmissions. Individual hospital performance was not evaluated. Leow et al[ identified increased 30-day readmissions following inpatient complications from the operation, which is an observation that is similar to our prolonged length of stay in the index hospitalization being a predictor of 90-day postdischarge deaths and readmissions. Our study identified a favorable effect of robotic surgery on prolonged length-of-stay outliers and upon 90-day readmissions. Unfortunately, ICD-9 coding convention does not permit the identification of laparoscopic-assisted nephrectomy. Consistent with our findings, Schmid et al[ found improved 30-day readmissions with minimally invasive approaches to nephrectomy. The analytic methods used in the current study underscore that major new technology can be evaluated for its impact on outcomes by including a risk factor among candidate variables for new methods that are employed. Favorable or unfavorable influences of the new treatment approach can then be identified.

Our study has limitations. The quality of risk adjustment is always an issue in predictive modeling. Administrative data have limitations in terms of completeness and often in accuracy. We have used screens to identify only quality coding hospitals for model development, but appreciate that comprehensive clinical abstraction of cases is potentially best. Clinical data would permit identification of the stage of cancer and would also identify those cases with extension of the primary cancers into the vena cava. Knowledge of the stage of primary renal cell carcinoma will affect the selection of minimally invasive surgical decisions versus open nephrectomy. Medical centers receiving more advanced stage cancers will have a risk profile that is not currently captured, and may partially explain the absence of enhanced outcomes in large volume facilities in this study.

However, clinical data is self-reported, expensive to gather, and is often deficient in accurate postdischarge information. Enhancement of administrative data with readily available clinical information, such as admission laboratory data from electronic medical records, has been shown to enhance discrimination of administrative data mortality models and should be useful for the future.[ Enhancement of administrative data with numerical laboratory data at admission has improved model performance for risk-adjusted prolonged length of stay in cardiac inpatient episodes of care.[ Admission laboratory data did not improve model performance for 90-day readmission models.[ It can only be hoped that the electronic medical record will enhance the retrieval of pertinent clinical information (e.g., stage of cancer or laparoscopic-assisted procedures) for model development without requiring full chart abstraction.

Our study was limited by employing only Medicare patients. This was necessary because all-payer databases other than Medicare are generally not available. State-based, all-payer databases that have encrypted patient identification will permit studies that evaluate outcomes for all patients and provide the necessary postdischarge information about readmissions. These databases with patient identifiers are present in only about 10% of States at present. It should be emphasized that bundled payment strategies by Medicare will be using the same database that we have used in this study.

A final limitation is that additional exclusions need to be defined for the evaluation of hospital performance. We have excluded ophthalmology, cancer, and trauma readmissions. We have identified that the overwhelming majority of readmissions of Table 2 are linked either to the index hospitalization or represent decompensation of underlying medical illnesses. A small number of additional exclusions need to be made to bring our readmission criteria in compliance with that being used by Medicare. In conclusion, the risk-adjusted AO rates in nephrectomy and partial nephrectomy are highly variable among acute care hospitals. Observed AO rates that can be compared with risk-adjusted predicted results allow individual hospitals and clinicians to benchmark their performance. Thus, the methods for outcome modeling and measurement are applicable for other surgical conditions and more comprehensive populations of patients. In an era of increased public reporting of outcomes and value-based purchasing of healthcare services, it is very important for hospitals and surgeons to know what are their results of care that are inclusive of the postdischarge period of time. Risk-adjusted results permit the evaluation of specific areas that need improvement (e.g., infections), provide focus for care redesign, and serve as a method of actually tracking of improved outcomes from efforts to modify hospital processes and clinician practice patterns.