Validation of a pretransplant risk score for new-onset diabetes after kidney transplantation.

OBJECTIVE: Identification of patients at high risk for new-onset diabetes after kidney transplantation (NODAT) will facilitate clinical trials for its prevention. RESEARCH DESIGN AND METHODS: We previously described a pretransplant predictive risk model for NODAT using seven pretransplant variables (age, planned use of maintenance corticosteroids, prescription for gout medicine, BMI, fasting glucose, fasting triglycerides, and family history of diabetes). We have now applied the initial model to a cohort of 474 transplant recipients from another center for validation. We performed two analyses in the validation cohort. The first was a standard model with variables derived from the original study. The second was a summary score model, in which the sum of dichotomized variables (all the variables dichotomized at clinically relevant cut points) was used to categorize, individuals into low (0-1), intermediate (2, 3), or high (4-7) risk groups. We also conducted a combined database analyses, merging the initial and validation cohorts (n=792) to obtain better estimates for a prediction equation.
RESULTS: Although the frequency of several risk factors differed significantly between the two cohorts, the models performed similarly in each cohort. Using the summary score model, incidences of NODAT in low-risk, medium-risk, and high-risk groups in the initial cohort were 12, 29, and 56%, and in the validation cohort incidences were 11, 29, and 51%.
CONCLUSIONS: A pretransplant model for NODAT, including many type 2 diabetes risk factors, predicted NODAT in the validation cohort.

New-onset diabetes after kidney transplantation (NODAT) affects 20–30% of kidney transplant recipients in the first year posttransplantation and has many negative effects on allograft and patient survival, quality of life, and health care costs (1–5). If development of NODAT could be prevented or delayed, then health outcomes after transplantation might be improved.

There is a fivefold to sixfold higher annual incidence of new-onset diabetes in the first year after transplantation than in subsequent years. Interestingly, the majority of new cases occurs within the first few months after transplantation (2), and the rapidity with which NODAT develops suggests that risk factors for diabetes are present even before surgery. Identification of pretransplant risk factors may help to explain why NODAT develops in only some individuals, even though all are exposed to similar transplant immunosuppression, many of which (calcineurin inhibitors, mTOR inhibitors, and glucocorticoids) (6–8) are also diabetogenic. We recently reported a pretransplant predictive risk model for NODAT using seven pretransplant risk factors (9). Our models were developed from a cohort of nondiabetic recipients of a first kidney transplant in a single center. Using univariate regression, we identified seven significant variables. Then, we used them in the three different multivariate models for predicting NODAT. One model, the standard model, used continuous and discrete variables weighted with β-coefficients that maximized their predictive power. A second model used continuous variables dichotomized to 0 or 1 at clinically relevant cut points and weighted to maximize prediction. The third model, the summary score, was simply the unweighted sum of the dichotomized variables. Surprisingly, there were no statistically significant differences in the predictive abilities of the three models. Areas under the receiver operating curve for predicting NODAT were 0.72, 0.71, and 0.70, respectively, and were not significantly different from each other (9). The seven pretransplant variables that were most predictive of NODAT were as follows: age 50 years or older; planned use of maintenance corticosteroids; use of gout medicine; BMI ≥30 kg/m2; fasting glucose ≥100 mg/dL; fasting triglycerides ≥200 mg/dL; and family history of type 2 diabetes. We conducted the current study to validate our predictive pretransplant risk models in a second cohort.

RESEARCH DESIGN AND METHODS

Validation cohort

Our validation cohort included all adult nondiabetic patients undergoing a first kidney transplantation at Mayo Clinic in Florida between March 2001 and July 2010. All patients had at least 1 year of follow-up posttransplantation. After Institutional Review Board approval, we identified the study cohort by systematic retrospective chart review. Absence of diabetes before transplantation was documented in the form submitted to United Network for Organ Sharing, with the information obtained from documentation provided by medical care providers before transplantation. Additionally, all patients had a fasting plasma glucose <126 mg/dL and HbA1c <6.5% (<48 mmol/mol) at pretransplant testing. All the methods and entry criteria were the same in the initial and the validation cohorts (9).

Immunosuppression after kidney transplantation

Steroid-based maintenance immunosuppression was used before 2005; afterward, rapid steroid withdrawal maintenance immunosuppression was adopted, except in patients who required prednisone for nontransplant indications or who were at high risk for rejection for immunologic reasons (positive cross-match, second transplant panel reactive antibody test >20%). Thus, the cohort included patients prescribed or not prescribed maintenance prednisone. Induction therapy with rabbit antithymocyte immunoglobulin or basiliximab was used in rapid steroid withdrawal patients and in some patients who were continued on maintenance prednisone for the indications noted. All patients received a 5-day tapering course of glucocorticoids (methylprednisolone intravenously 500 mg on day 1, 250 mg on day 2, 125 mg on day 3, oral prednisone 60 mg on day 4, and 30 mg on day 5; it was then discontinued if the patient was in the rapid steroid withdrawal group). Patients requiring ongoing steroid therapy received the same initial 5-day corticosteroid treatment with tapering of prednisone over 8–12 weeks to maintenance with 5 mg prednisone daily. Tacrolimus was initiated when serum creatinine decreased by >30%; the day of tacrolimus initiation differed among patients. All patients, including those with delayed graft function, began tacrolimus before discharge. Mycophenolate mofetil and tacrolimus were the maintenance immunosuppressants for all patients, including those who did and did not require ongoing steroid therapy.

Definition of NODAT

We used the following composite diagnostic criteria for NODAT: HbA1c ≥6.5% (≥48 mmol/mol); fasting venous plasma glucose ≥7 mmol/L; or prescribed diet or medical therapy for diabetes between 1 month and 1 year posttransplantation (9). We evaluated subjects between 1 month and 1 year posttransplantation to exclude patients who only developed transient hyperglycemia in the immediate posttransplantation period and because the highest incidence of NODAT is within the first year posttransplantation (4,5).

Data analyses

The characteristics of the two cohorts were described by summary statistics and compared using the χ2 test, two-sample t test, or Wilcoxon rank sum test. We constructed two analyses. We validated two of the initial predictive models, the standard model and the summary score, developed in the previous article using the validation cohort (9). Additionally, we created a new cohort by merging both cohorts (initial and validation cohorts), and the β-coefficients for the models were estimated again using the combined dataset.

Validation of the predictive models.

The predictive probabilities of NODAT in the validation cohort were estimated by applying the β-coefficients from the initial predictive model to the validation cohort. The two models included the standard model, in which both continuous and discrete variables were included and weighted according to the β-coefficients in the multivariate logistic model, and the summary score model, which was the sum of variables dichotomized at clinically relevant cut points. The following measures were then computed:

Brier score (10), which provides a global assessment of the performance of the models. The Brier score is the mean square difference between predictive probability and outcome. For each patient, the score ranges from 0 to 1, and a score of 0.25 indicates that the model has poor performance. For instance, if the predictive probability of a patient is 0.5 and the outcome is either 0 or 1, then the Brier score for this patient is 0.25. In other words, the predictive probability of the outcome is the same as flipping a coin, and thus it may not have any advantage of using a predictive model.

Areas under receiver operating characteristic (ROC) curves (AUC) and their corresponding 95% CIs for discrimination analyses. This measured how well the model discriminated patients with and without NODAT.

Hosmer-Lemeshow test statistic for calibration analyses evaluating the performance of the prediction models (11). The Hosmer-Lemeshow evaluated the goodness of fit for each model and compared the observed and expected numbers of events in each decile group (based on the predictive probability). A lack of goodness of fit implied the deviation of the observed and expected counts of events. To perform calibration analysis, the observed and expected events of NODAT were compared and the Hosmer-Lemeshow test statistic was calculated based on the observed and expected events. A calibration plot was given by graphing the observed events against the expected events of NODAT.

Analyses of the combined dataset (initial and validation cohort): refining model estimates and subsequent internal validation for this combined dataset model.

Data from the initial cohort and the validation cohort were combined to refine the β-coefficients. Using the variables from the standard model and the indicator for the study cohort (initial/validation cohort), the model with the variables and interaction terms of the cohort indicator was examined first. An interaction term or the indicator for the study cohort was removed if it was not significant. Internal validation of this model was performed using the bootstrap method (12). A bootstrap method was chosen over data-splitting or cross-validation method because of better efficiency (13,14). This method randomly draws the sample to create a replacement of the same size from the combined database. The estimated β-coefficients from the bootstrap samples were then applied to the original combined database. For both bootstrap and combined databases, AUC and Brier scores were computed. The indices computed from the bootstrap sample represented the apparent performance, and those computed from the combined database using the β-coefficients from the bootstrap sample represented the test performance. The estimated optimism was the difference of the indices between the bootstrap and combined databases. The procedure was repeated 100 times, and the average estimated optimism for each index was calculated. Finally, the adjusted performance of the model was estimated by subtracting the average estimated optimism from the index computed using the original combined sample.

Statistical significance was set at two-sided P < 0.05. Statistical analyses were conducted with SAS version 9.2 (SAS Institute, Cary, NC).

RESULTS

From March 2001 through July 2010, 474 nondiabetic patients underwent kidney transplantation at Mayo Clinic in Florida. Patient characteristics of the initial and validation cohorts are summarized and compared in Table 1. BMI, planned use of maintenance corticosteroids, fasting glucose and triglycerides, and family history of type 2 diabetes differed significantly between the initial and validation cohorts. Associations of risk variables with NODAT incidence are described in Table 2. In the validation cohort, maintenance immunosuppression was predominantly tacrolimus and mycophenolate mofetil; 91% and 88% were prescribed tacrolimus at 4 and 12 months posttransplantation, respectively, and 51% were prescribed ongoing maintenance corticosteroids posttransplantation.

Table 1

Clinical characteristics in the initial and validation cohorts

Table 2

Individual risk factors in the initial and validation cohorts

Validation analyses

The 1-year incidence of NODAT in the validation cohort was 27% (128 out of 474). The β-coefficients developed in the previous article (9) and the indices of model performance using the validation cohort are presented in Table 3. The Brier score for the standard model was slightly lower than that for the summary score model (0.183 vs. 0.184). The AUCs of the ROC curves for predicting NODAT using the standard model were 0.72 (95% CI, 0.66–0.79) and 0.67 (0.61–0.72) for the initial and validation cohorts, respectively, and were significantly different from 0.5 (both P < 0.0001). From the Hosmer-Lemeshow test, there was no evidence of lack of fit for the standard model (P = 0.96), but the summary score model may not fit the validation cohort (P = 0.05). Although the Brier scores and AUCs were similar between two models, the results from the Hosmer-Lemeshow test were quite different. This may be because there were only six different values of predictive probability estimated from the summary score model and a large deviation occurred in the patients with moderate predictive probability of NODAT ∼0.2 when comparing the observed and expected counts. Based on the measures of the performance, the standard model appeared to be better than the summary score model to predict NODAT. Risk of NODAT development was similar in the initial and the validation models (Fig. 1).

Table 3

Regression model for the standard model and performance measures

Figure 1

Comparison of pretransplant risk score with development of NODAT in the initial cohort and the validation cohort. A: The simple risk score model (low risk, medium risk, and high risk are based on risk score). B: The standard model (low risk, medium risk, and high risk are based on the tertiles).

From the results using the validation cohort, the probability of NODAT, P(X), can be computed using the following equation: logit(P(X)) = −6.9148 + [age at transplant (in years) ⋅ 0.0328] + [pretransplant glucose (mmol/L) ⋅ 0.3914] + [pretransplant BMI ⋅ 0.0553] + [family history of diabetes (1 if yes, 0 if no) ⋅ 0.6751] + [planned use of maintenance corticosteroids (1 if yes, 0 if no) ⋅ 0.4089] + [log2 triglyceride (mmol/L) ⋅ 0.1703] + [use of gout medication before transplant (1 if yes, 0 if no) ⋅ 0.4000].

Analyses of the merged dataset: initial and validation cohorts

The merged dataset (from combining the initial and validation cohorts) was modeled to update the β-coefficients. The total sample of this dataset was 792. The model using the seven variables from the standard model and the interaction term of the cohort indicator (initial/validation model) were evaluated. All the interaction terms and the cohort indicator were not significant and thus were excluded from the model. Because results using the summary score model with the validation cohort showed evidence of lack of fit at marginal significance (P = 0.05), and because the AUC (0.65) was worse than it was using the standard model, we did not pursue the summary score with the merged cohort.

From the results using the combined dataset, the probability of NODAT, P(X), can be computed using the following equation: logit(P(X)) = −6.8855 + [age at transplantation (in years) ⋅ 0.0289] + [pretransplantation glucose (mmol/L) ⋅ 0.4744] + [pretransplantation BMI ⋅ 0.0434] + [family history of diabetes (1 if yes, 0 if no) ⋅ 0.6048] + [planned use of maintenance corticosteroids (1 if yes, 0 if no) ⋅ 0.5291] + [log2 triglyceride (mmol/L) ⋅ 0.2849] + [use of gout medication before transplantation (1 if yes, 0 if no) ⋅ 0.5103].

Results of the updated model and coefficients and results from bootstrap internal validation are shown in Table 3. The corrected Brier score and AUC were improved slightly using the merged dataset. The calibration slope was close to 1, implying the updated model had a good fit with the merged dataset.

CONCLUSIONS

This study confirmed that the seven previously identified pretransplant variables predict NODAT in the replication cohort in a manner similar to the initial model, although the AUCs were higher in the initial model (Table 3). The seven pretransplant risk factors are similar to those identified as risk factors for type 2 diabetes in the nontransplant population, suggesting that NODAT and type 2 diabetes share a similar pathophysiology. This idea is further supported by a recent study that evaluated the performance of two other risk scores for predicting type 2 diabetes (San Antonio Diabetes Prediction Model and Framingham Offspring Study–Diabetes Mellitus) in a cohort of kidney transplant patients and demonstrated ROC curves AUCs of 0.76 and 0.81, respectively (15). Furthermore, markers of obesity and insulin resistance (plasma adiponectin, triglycerides, and insulin), when measured pretransplantation, predict NODAT (16,17).

One interpretation of our results, and the results of many other studies, is that NODAT represents the progression of type 2 diabetes risk factors after kidney transplantation. Kidney disease suppresses appetite, and the catabolic effects of end-stage renal disease coupled with the decreased clearance of insulin may delay progression to type 2 diabetes in obese patients who would otherwise be at high risk for type 2 diabetes. After transplantation, when kidney function is restored, appetite returns, patients gain weight, and, in a considerable fraction, NODAT follows.

When risk factors for type 2 diabetes are present pretransplantation, the development of a pretransplant predictive model to identify patients at highest risk for NODAT will enable development of clinical interventions structured to reduce risk. The success of the Diabetes Prevention Program, in which the incidence of type 2 diabetes in a high-risk group was reduced by 58% by a lifestyle weight loss intervention (18), suggests that similar interventions may help to reduce the incidence of NODAT. Our center at Mayo Clinic Arizona is performing a pilot study of a Diabetes Prevention Program–type behavioral lifestyle intervention to see if the incidence of NODAT can be diminished. Posttransplantation interventions also might be of benefit in prevention of NODAT. Belatacept is a selective inhibitor of T-cell activation that replaces calcineurin inhibitors. Studies suggest that transplant recipients who receive belatacept have a better metabolic profile and a lower incidence of NODAT compared with those who receive calcineurin inhibitors (19). Another posttransplantation strategy for prevention of NODAT is use of basal insulin in the immediate posttransplantation period. Surgery, high-dose corticosteroids, and initiation of calcineurin inhibitors all stress β-cells, and it is thought that administration of exogenous insulin decreases that stress (20).

One limitation of our study is our composite definition of NODAT rather than the American Diabetes Association diagnostic criteria that require an oral glucose tolerance test; however, our definition was clinically available and has been used previously (9,21,22). Some of our patients would have been excluded because of having pretransplant diabetes, but others might have met diagnostic criteria for NODAT had we used an oral glucose tolerance test. Thus, the overall effect on our results had we included glucose tolerance testing is unknown.

Another limitation of our study is the predominance of white transplant recipients in both the initial and validation cohorts. In the United States, risk factors for type 2 diabetes are known to be more prevalent among nonwhites (23), and this may need to be explored in transplant recipients as well. Univariate analyses, performed on the initial cohort, did not support race as a variable significantly predictive of NODAT. A post hoc univariate analysis performed on the merged cohort, in which 28% of subjects were nonwhite, showed that nonwhite race was not significantly associated (P = 0.13) with future development of NODAT.

This risk model for NODAT was developed for a specific population of kidney transplant recipients, and its generalizability to recipients of other solid organs will need to be tested. NODAT is a significant problem after liver and heart transplantation, with reported incidence rates of 20–40% (24,25). Because end-stage heart failure and end-stage liver disease are also catabolic processes and often occur in obese patients, it will be important to determine if risk factors for diabetes after transplantation of other solid organs are similar to those for type 2 diabetes, as we have described here.

In conclusion, the many advantages of kidney transplantation are severely undermined by development of NODAT. Pretransplant risk factors for NODAT are similar to those for type 2 diabetes, and a risk calculator allows identification of patients at highest risk for clinical trials of intervention strategies that have already been proven effective in prevention of type 2 diabetes.