Predicting Kidney Failure, Cardiovascular Disease and Death in Advanced CKD Patients.

Risk prediction equations (RPEs) are ubiquitous in medicine. Nephrology is not immune to the phenomenon. For example, a recent systematic review found 42 articles mentioning 1 or more novel RPEs in the nephrology literature since 1986. The Kidney Disease Improving Global Outcomes consortium gave a rather tepid endorsement of the use of RPEs in their 2013 chronic kidney disease (CKD) management guidelines but more robust support in a 2019 consensus conference regarding timing of vascular access placement in patients with severe and/or progressive CKD. A widely cited RPE is the Kidney Failure Risk Equation (KFRE), published in 2011, which currently has 620 citations listed in the Web of Science (Clarivate, London, UK). The KFRE predicts the absolute risk of kidney failure with replacement therapy (KFRT) at 1, 2, and 5 years among patients with stage G3 to G5 CKD. The 4-variable, 2-year KFRE includes age, sex, estimated glomerular filtration rate ([eGFR], using the CKD-Epidemiology Collaboration 2009 creatinine equation), and log-transformed urine albumin-to-creatinine ratio as predictors. A convenient on-line calculator is available.

The KFRE has endured some criticism since publication. It was derived and externally validated in 2 Canadian CKD cohorts, which raised concerns regarding generalizability to the general population and to other jurisdictions. Nevertheless, a large, international, and extended validation study among 31 general populations and CKD cohorts dispersed around the globe with 721,357 participants demonstrated that the KFRE had uniformly excellent discrimination but required an addition of a calibration factor for non-North-American cohorts. Another concern is that the KFRE estimates risk using predictor values obtained at a single point in time and does not consider their prior, longitudinal trajectory. Nevertheless, a study conducted by the KFRE investigators compared a dynamic RPE that did account for changes in predictors over time versus the original static KFRE using a single eGFR value and found only a slight improvement in prediction performance metrics.

Perhaps the biggest concern with the KFRE is that it was developed using standard Cox regression methods which did not account for the competing risk of death. This effect has been shown to result in an over-estimation of KFRT risk, particularly among older CKD patients. This issue is somewhat nuanced because clinicians caring for older CKD patients cannot know their true future risk of KFRT at the time decisions are made but rather rely on predicted risk. The latter is systematically lower among older patients with CKD for 2 reasons. The first reason is that elderly patients tend to be sarcopenic resulting in creatinine-based eGFR values that overestimate measured GFR and subsequently result in lower predicted KFRT risk. This issue may be exacerbated among non-Black, older patients if new creatinine-based GFR estimating equations that do not include race as a covariate come into widespread use because they further overestimate GFR compared to the 2009 CKD-Epidemiology Collaboration creatinine equation. The second reason is that the hazard ratio for age in the KFRE is less than 1, leading to even lower predicted risks for older patients (Figure 1).

Figure 1

The contour plots for men (panel a) and women (panel b), demonstrate combinations of eGFR and urine ACR that yield predicted two-year risks of 10% or greater via the Kidney Failure Risk Equation for different ages. Regions above and to the left of a given contour line represents combinations of eGFR and ACR that yield risks above 10%. The color-coding of the contour lines for different ages is given to the right of each panel. ACR, albumin-to-creatinine ratio; eGFR, estimated glomerular filtration rate.

Systematically lower predicted KFRT risk among older individuals may lead to unanticipated consequences when CKD management policy is based on KFRE thresholds. For example, in Ontario, Canada on January 1, 2016, the criterion for reimbursement of multidisciplinary team (MDT) clinics shifted from eGFR ≤ 33 ml/min to either eGFR ≤ 15 ml/min or a 4-variable, 2-year KFRE predicted risk ≥10%. The rationale is that the purpose of MDT clinics is for KFRT preparation. Therefore, patients at low predicted risk of this outcome derive no benefit from MDT care and should be excluded. This rather narrow view does not consider other goals of MDT care such as, among many others, diagnosis and treatment of cardiovascular disease, which may contribute to the proven benefit of MDT clinics.,

The newer Grams RPE published in 2018 was a welcome addition to the CKD clinician’s toolkit because it provides 2 and 4 year predictions for a broader range of outcomes among patients with stage G4 and G5 CKD, including KFRT, cardiovascular disease and death, and also predicts all 9 possible orderings (trajectories) of these events. Predictors include age, sex, race, history of cardiovascular disease, current smoking status, systolic blood pressure, diabetes status, eGFR (using the CKD-Epidemiology Collaboration 2009 creatinine equation), and urine albumin-to-creatinine ratio. The Grams RPE employed a complex derivation process. Using Fine and Gray competing risk regression, subdistribution hazard ratios were meta-analyzed across 29 CKD prognosis consortium cohorts comprising 264,296 individuals for first and second events. Fixing the meta-analyzed subdistribution hazard ratios, allowed the baseline subdistribution hazard function to be estimated within each cohort and then averaged across cohorts. A Weibull survival model was then fit to the average baseline subdistribution hazards. These and the meta-analyzed subdistribution hazard ratios were then employed to estimate time-varying transition intensities in a 5-state Markov process. The process was run for 24 or 48 months with every possible combination of covariate values to generate a large set of simulated data. Finally, a multinomial logistic meta-model was fit to the simulated data in order to produce a web-based tool. Given a set of covariate values, the probabilities of each of the 9 mutually exclusive outcome trajectories are provided at each time point such that the predicted probabilities sum to one.

In this issue of Kidney International Reports, Ramspeck et al. make a valuable contribution by externally validating the Grams RPE among 1517 participating patients in the European Quality Study. Starting in 2012, The European Quality Study recruited patients 65 years or older from nephrology clinics in 6 European countries with eGFR values at baseline between 10 and 30 ml/min. Patients were followed between 4 and 8 years or until kidney transplantation. The outcomes were combinations of the trajectories considered by the Grams RPE, including “any KFRT,” “any CVD,” death, and no event at 2 and 4 years. Predicted outcomes at these time points were calculated using the Grams multinomial logistic equations by summing the predicted probabilities for trajectories that were included in a given combination. Observed probabilities were estimated using cumulative incidence functions. Performance was assessed via discrimination using a method that takes competing risks into account as well as calibration using calibration-in-the-large (predicted vs. observed probabilities) and smoothed calibration plots. The c-statistics for the Grams RPE varied according to the combined outcome being predicted. For KFRT, the c-statistics were 0.76 and 0.74 for 2 and 4 years, respectively. There was generally good calibration to observed risks with the 4-year version outperforming the 2-year RPE.

The complex derivation of the Grams RPE presents a challenge to external validation. The data simulation process produced an arbitrarily large set of hypothetical individuals who could not be censored and therefore the timing and occurrence of events could be determined precisely at 2 and 4 years, hence the multinomial logistic structure of the meta-model. Of course, no real CKD cohort is free of censored observations. In the analysis by Ramspeck et al., the investigators sensibly use cumulative incidence functions to estimate observed 2-year and 4-year observed risks but it should be pointed out that these are not the same as multinomial probabilities. This may account for some of the difference between predicted and observed risks. The Grams RPE predicts 8 possible outcome trajectories and 1 nonevent trajectory. Assessing discrimination and calibration with a large number of possible outcomes is difficult. Understandably, the investigators have combined disease trajectories into the 4 groups listed above in order to have sufficient numbers of patients per group. These groupings are no longer mutually exclusive but this would not be expected to meaningfully impair the ability to externally validate the Grams RPE.

The investigators employ decision curve analysis to consider the clinical effect of decision-making with respect to initiation of KFRT preparation using the Grams and KFRE RPEs and a simpler rule based solely on eGFR. Decision curve analysis is a validated approach but some readers may be unfamiliar with it. The basic idea is to consider positive tests (i.e., predicted KFRT risks above a given threshold). A lower threshold would increase the proportion of true positives ([TP], individuals above the threshold who did experience KFRT) but also the proportion of false positives ([FP], individuals above the threshold who did not experience KFRT), whereas a higher threshold would reduce both TP and false positive. Decision curve analysis attempts to “convert” false positives into TP-equivalents using a harm-to-benefit ratio (the harm of a false positive divided by the benefit of a TP). The net benefit is then calculated as difference between the proportion of TPs and the TP-equivalents that are “lost” because of the diagnostic test characteristics and threshold. Because a harm-to-benefit ratio is difficult to estimate, it is more useful to examine net benefit across a range of ratios. The investigators show that at low ratios, using the Grams predicted 2-year KFRT risk above 20%, and at higher ratios a Grams predicted risk above 40% yielded higher net benefits than an eGFR threshold of ≤ 15 ml/min. Net benefits for KFRE-derived thresholds were similar to Grams thresholds.

Ramspeck et al. have taken on the difficult but important task of externally validating a potentially more useful RPE for patients with stage G4 and G5 CKD. Decision-making, with respect to access to MDT care and initiation of planning for KFRT, are fraught in this population making the insights from RPEs potentially useful. Work remains to be done with respect to the development and validation of RPEs in CKD as changes to measurement and estimation evolve such as the wider use of cystatin-C as a filtration marker and wider use of the CKD-EPI 2021 equation that does not consider race as a covariate. Ultimately, the utility of RPEs will need to be proven in cluster randomized trials. Given their wide diffusion, however, that horse may have already left the barn.

Disclosure

The author declared no competing interests.