Literature DB >> 35243307

Potential Surrogate Outcomes for Kidney Failure in Advanced CKD: Evaluation of Power and Predictive Ability in CKDopps.

Jarcy Zee1,2, Daniel Muenz3, Keith P McCullough3, Brian Bieber3, Marie Metzger4, Natalia Alencar de Pinho4, Antonio A Lopes5, Danilo Fliser6, Bruce M Robinson3, Eric Young3, Ronald L Pisoni3, Bénédicte Stengel4, Roberto Pecoits-Filho3,7.   

Abstract

RATIONALE &
OBJECTIVE: Potential surrogate end points for kidney failure have been proposed in chronic kidney disease (CKD); however, they must be evaluated to ensure accurate, powerful, and harmonized research, particularly among patients with advanced CKD. The aim of the current study was to investigate the power and predictive ability of surrogate kidney failure end points in a population with moderate-to-advanced CKD. STUDY
DESIGN: Analysis of longitudinal data of a large multinational CKD observational study (Chronic Kidney Disease Outcomes and Practice Patterns Study). SETTING & PARTICIPANTS: CKD stage 3-5 patients from Brazil, France, Germany, and the United States. OUTCOMES: Reaching an estimated glomerular filtration rate (eGFR) < 15 mL/min/1.73 m2 or eGFR decline of ≥40%, and composite end points of these individual end points. ANALYTICAL APPROACH: Each end point was used as a time-varying indicator in the Cox model to predict the time to kidney replacement therapy (KRT; dialysis or transplant) and was compared by the number of events and prediction accuracy.
RESULTS: 8,211 patients had a median baseline eGFR of 27 mL/min/1.73 m2 (interquartile range, 21-36 mL/min/1.73 m2) and 1,448 KRT events over a median follow-up of 2.7 years (interquartile range, 1.2-3.0 years). Among CKD stage 4 patients, the eGFR < 15 mL/min/1.73 m2 end point had higher prognostic ability than 40% eGFR decline, but the end points were similar for CKD stage 3 patients. The combination of eGFR < 15 mL/min/1.73 m2 and 40% eGFR decline had the highest prognostic ability for predicting KRT, regardless of the CKD stage. Including KRT in the composite can increase the number of events and, therefore, the power. LIMITATIONS: Variable visit frequency resulted in variable eGFR measurement frequency.
CONCLUSIONS: The composite end point can be useful for CKD progression studies among patients with advanced CKD. Harmonized use of this approach has the potential to accelerate the translation of new discoveries to clinical practice by identifying risk factors and treatments for kidney failure.
© 2021 The Authors.

Entities:  

Keywords:  Chronic kidney disease progression; composite outcomes; estimated glomerular filtration rate; kidney failure; surrogate end points

Year:  2021        PMID: 35243307      PMCID: PMC8861958          DOI: 10.1016/j.xkme.2021.10.008

Source DB:  PubMed          Journal:  Kidney Med        ISSN: 2590-0595


Kidney failure is an important outcome for monitoring and investigating the progression of chronic kidney disease, but this outcome is not always available in many chronic kidney disease research studies. Several surrogate end points for kidney failure have been proposed based on a low or declining estimated glomerular filtration rate (eGFR). This study conducted a systematic comparison of eGFR-based end points among a multinational, advanced chronic kidney disease cohort. End points were compared based on their ability to predict kidney failure and their statistical power. A composite end point that includes a low eGFR, declining eGFR, and kidney replacement therapy was optimal. Harmonized use of this approach has the potential to accelerate the translation of new discoveries to clinical practice by identifying risk factors and treatments for kidney failure. Kidney failure is a clinical and patient-centered outcome of interest for monitoring and investigating the progression of chronic kidney disease (CKD). A recent Kidney Disease: Improving Global Outcomes (KDIGO) Consensus Conference emphasized the importance of explicitly defining kidney failure in terms of kidney function, duration, symptoms, and/or treatment. The International Society of Nephrology thus proposed that in the setting of clinical trials, kidney failure outcomes be comprised of a composite including kidney transplantation, maintenance dialysis, and death from kidney failure. In observational and clinical settings, understanding and identifying patients with CKD at high risk of kidney failure can accelerate the interpretation of data and treatment decisions, for example, to mitigate CKD progression or prepare patients for kidney replacement therapy (KRT). Surrogate end points are often needed in clinical research because a small sample size or short follow-up time implies a potentially low number of KRT events. The International Society of Nephrology consensus suggested that a sustained low glomerular filtration rate (GFR) and a sustained percent decline in GFR could be incorporated into a kidney failure outcome, as the former is concordant with KDIGO guideline definitions for kidney failure and the latter has been extensively studied as an acceptable surrogate for kidney failure by researchers and regulatory agencies.2, 3, 4, 5, 6, 7, 8 These surrogate end points are advantageous not only in clinical trials but also in observational studies because kidney disease progression can be slow in some patients, even among those with advanced CKD and especially among those with CKD etiologies such as glomerulonephritis, polycystic kidney disease, or interstitial disease, who may be enrolled in clinical studies at higher estimated GFRs (eGFRs) or have slower progression. In clinical care, the eGFR-based surrogate end points can be useful for monitoring patients before they reach kidney failure. New treatment strategies and targets from recent trials—for example, blood pressure control in SPRINT, SGLT2i in CREDENCE, and finerinone in FIDELIO-DKD—have triggered recent changes in practice guidelines and will soon be introduced in clinical practice.10, 11, 12, 13, 14 Demonstrating earlier treatment effects on validated surrogate end points in the real world, especially among high-risk populations (eg, patients with CKD with low eGFR and high albuminuria), would allow for quicker analysis of practice patterns related to best outcomes and the earlier introduction of therapies that could prevent or postpone KRT. If proven efficient, this harmonized application of surrogates across clinical research applications may have a strong impact in improving kidney failure outcomes for patients. A systematic evaluation of potential eGFR-based surrogate end points for kidney failure would ensure accurate research that maximizes power and facilitates harmonization across studies. Although previous studies have established eGFR percentage declines as valid surrogates for kidney failure, none have directly compared them to low eGFR thresholds using real-world data.4, 5, 6, 7, 8 Yang et al showed that associations of risk factors for KRT were largely similar to those for halving of eGFR among patients in the Chronic Renal Insufficiency Cohort study. However, the Chronic Renal Insufficiency Cohort study population includes many patients with only mild CKD and only those from US centers, and the frequency of eGFR evaluation was established by the study protocol. Several other CKD cohorts from around the world include a greater proportion of patients with more advanced CKD and older age, but the performance of potential eGFR-based surrogate end points in multinational cohorts of advanced CKD with diverse populations and practices is unknown., In this study, we compared the potential individual and composite surrogate end points for the number of events and prognosis of time to kidney failure in the Chronic Kidney Disease Outcomes and Practice Patterns Study (CKDopps), a multinational study involving patients with advanced CKD. Prentice established 2 operational criteria for surrogate end points: first, that the true outcome is independent of treatment after conditioning on the surrogate, and second, that the surrogate end point has some prognostic implication for the true outcome. Given our focus on observational studies and real-world data, we evaluated surrogate end points in terms of fulfilling the second criterion. We included time-to-event end points based on a low eGFR and a percentage decline in eGFR. Given the cohort of patients with advanced CKD, we hypothesized that many would progress to low eGFR quicker than to a percentage decline in eGFR, especially those starting with CKD stage 4 at study enrollment. We also hypothesized that composite end points would provide greater power without sacrificing the prognostic ability or accuracy of exposure effect estimates.

Methods

Study Sample and Data Collection

CKDopps is an ongoing, international, prospective cohort study of nondialysis patients with advanced CKD under nephrology care. The current study sample includes patients from Brazil, France, Germany, and the United States who were enrolled in 2013 and followed until 2019 in France and Germany and until 2017 in Brazil and the United States. CKDopps was approved by national and/or local ethics committees (approval numbers available on request), and all patients provided written informed consent as required by local ethics regulations. Samples of nephrologist-run CKD clinics were randomly selected after stratification by geographic region and key clinical characteristics (ie, size and public versus private). Participants aged >18 years, receiving care for CKD at the clinic, and having an eGFR < 60 mL/min/1.73 m2 at screening were selected from each clinic. Although an approximately equal number of CKD stage 3 and 4 patients were enrolled in France, CKD stage 4 patients were oversampled in other countries. Furthermore, CKD stage 5 patients were excluded from enrollment in France and Germany. Individuals with a prior kidney transplant or those receiving maintenance dialysis (having >2 treatments/week without the expectation of kidney function recovery) were excluded. Patients with no follow-up were also excluded. Demographics (age, sex, and Black race), comorbid conditions (diabetes, hypertension, and heart failure), and clinical data were transcribed from medical records in Brazil, France, and the United States and abstracted from electronic health records in Germany (except race was not reported in Germany). Routine laboratory data based on the usual care were collected longitudinally, up to monthly frequency. In France, a standard set of urine and blood tests (including albuminuria) were also requested at study enrollment and annually thereafter. In the current study, GFR was estimated using the CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) formula based on serum creatinine level. Albuminuria was categorized on the basis of KDIGO guidelines using, in descending order of priority, spot urine albumin-creatinine ratio, spot urine protein-creatinine ratio, 24-hour timed urine albumin, 24-hour timed urine protein, or protein reagent strips. The “severely increased” albuminuria category was also split to capture “nephrotic syndrome” albuminuria (albumin-creatinine ratio > 2,200 mg/g and protein-creatinine ratio > 3,500 mg/g).

Outcome and Potential Surrogate End Points

The primary outcome was time from study enrollment to KRT, defined by the start of maintenance dialysis or preemptive kidney transplant. The primary potential surrogate end points included the time from study enrollment to an eGFR < 15 mL/min/1.73 m2, time from study enrollment to a 40% eGFR decline, or a composite defined by the earliest of these 2 eGFR-based end points. For the eGFR < 15 mL/min/1.73 m2 end point and the composite, only patients who were enrolled in the study with an eGFR of at least 15 mL/min/1.73 m2 were eligible for inclusion. Event times for a 40% eGFR decline were determined on the basis of repeated linear regression models for each participant: at each eGFR measurement, all historic eGFRs were used in a linear regression model, and the intercept and slope of the regression line were used to determine whether the participant reached the 40% eGFR decline event. For patients who did not reach eGFR-based end points during their follow-up, event times were censored at the time of the patient’s last available eGFR measurement. The eGFR-based end points are often combined with KRT into composite outcomes for use in practice. Since KRT is the outcome of interest, adding KRT would not change predictive accuracy but could increase the number of outcome events. Therefore, we also evaluated the impact of including KRT with eGFR-based potential surrogate end points. Secondary end points used a lower eGFR threshold of eGFR < 10 mL/min/1.73 m2 and both 30% and 50% eGFR declines. Finally, we conducted sensitivity analyses with sustained low eGFR end points. For example, an eGFR < 15 mL/min/1.73 m2 was considered sustained if there was another confirmatory eGFR < 15 more than 4 weeks after the initial eGFR < 15 mL/min/1.73 m2 value; if no subsequent eGFR values were available, it was considered sustained if there was a KRT event or death event after the initial eGFR < 15 mL/min/1.73 m2 value.

Statistical Analysis

Descriptive statistics were used to describe the study sample and counts of KRT events and potential surrogate end point events. Cause-specific Cox models were used to assess the prognostic ability of each potential surrogate end point for the true outcome of KRT, treating competing death events as censored. Each potential surrogate end point was used in a separate model as a time-varying binary indicator, with value 0 from study enrollment until the end point was observed and 1 after it was observed. We used unadjusted models for each end point after separating the sample by CKD stage at enrollment (stage 3 versus stages 4 or 5). We also used additional models separated further by age tertiles, sex, diabetes comorbid condition status, and country. Prognostic ability was estimated using the methods of O’Quigley and Flandre, who developed the measure ρ2 to represent the proportion of randomness explained by a time-dependent Cox model. The measure is analogous to R2 for linear models but can be applied to quantify the criteria established by Prentice for surrogacy for time-to-event surrogate end points of time-to-event true outcomes. Cause-specific Cox models were used to compare hazard ratios of each end point or outcome by patient demographics and baseline clinical characteristics previously identified as risk factors for KRT. Models treated competing death events as censored and were stratified by country and CKD stage at enrollment. Missing data for patient demographic and baseline clinical characteristics were multiply imputed by chained equations, and results from 20 imputed data sets were combined using the formula by Rubin., All eGFR-based end points required ≥2 eGFR measurements, and end points with low eGFRs were limited to patients starting the study above the eGFR threshold. Although primary analyses included any eligible patient in each model, we also conducted sensitivity analyses to compare across end points using the same subsample of patients. The subsample used in sensitivity analyses was therefore based on the overlapping sample of patients: that is, patients with ≥2 eGFRs who started the study above the low eGFR threshold. Statistical analyses were conducted using R software, version 4.0.2 (R Development Core Team).

Results

The study sample included 8,211 participants from Brazil, France, Germany, and the United States (Table 1). The overall median age was 71 years (interquartile range, 62-78 years), with country-specific medians between 67 and 75 years, and over half of participants (n=4,807; 59%) were men. Of the participants, 3,747 (46% overall; between 43% and 55% across countries) had diabetes, 7,148 (89% overall; between 84% and 92% across countries) had hypertension, and 1,135 (14% overall; between 13% and 16% across countries) had heart failure. The median eGFR was 27 mL/min/1.73 m2 (interquartile range, 21-36 mL/min/1.73 m2; median, 24-31 mL/min/1.73 m2 across countries). Consistent with sampling protocols, the majority of participants were with CKD stage 4 or 5 at study enrollment in Brazil, Germany, and the United States (70%-76%) whereas participants were more evenly distributed across CKD stage 3 versus stages 4 or 5 in France. Albuminuria data were missing in 30%-38% of patients in Brazil, Germany, and the United States and in 328 (11%) patients in France. Among those with albuminuria data, 34%-47% had severely increased or nephrotic-range albuminuria. Follow-up times in Brazil and the United States (medians, 1.4 and 1.2 years, respectively) were shorter than those in France and Germany (medians, 3.0 and 2.9 years, respectively; Table 1). The KRT event rates varied from 5.3 to 10.3 events per 100 person-years across countries.
Table 1

Descriptive Table of Study Sample by Country

BrazilFranceGermanyUnited States
At study enrollment
 N9122,9692,7791,551
 Age, y67 (56-76)69 (61-77)75 (67-81)70 (61-78)
 Male481 (53%)1,943 (65%)1,580 (57%)803 (52%)
 Black race252 (28%)73 (2%)-320 (21%)
 Diabetes411 (45%)1,274 (43%)1,209 (45%)853 (55%)
 Hypertension804 (92%)2,694 (91%)2,233 (84%)1,417 (92%)
 Heart failure141 (16%)390 (13%)356 (13%)248 (16%)
 eGFR, mL/min/1.73 m224 (18-32)31 (23-40)26 (21-30)24 (18-31)
 CKD stage 3278 (30%)1,606 (54%)679 (24%)445 (29%)
 CKD stage 4 or 5634 (70%)1,363 (46%)2,100 (76%)1,106 (71%)
 Albuminuriaa
 Normal to mildly increased287 (31%)715 (24%)567 (20%)309 (20%)
 Moderately increased127 (14%)831 (28%)554 (20%)207 (13%)
 Severely increased151 (17%)880 (30%)401 (14%)299 (19%)
 Nephrotic syndrome71 (8%)215 (7%)189 (7%)154 (10%)
 Unknown276 (30%)328 (11%)1,068 (38%)582 (38%)
During follow-up
 Follow-up time, y1.4 (0.5-2.0)3.0 (2.9-3.0)2.9 (1.3-5.0)1.2 (0.5-2.2)
 No. of eGFR measurements3.0 (1.0-5.0)8.0 (6.0-12.0)9.0 (4.0-16.0)3.0 (1.0-6.0)
 Has >1 eGFR measurement601 (66%)2,893 (97%)2,568 (92%)1,068 (69%)
 Months between eGFR measurements3.3 (2.3-4.6)3.9 (2.6-5.8)3.3 (2.1-5.3)3.4 (2.2-5.2)
 KRT events100 (11%)412 (14%)706 (26%)230 (15%)
 KRT event rate, per 100 person-years7.775.278.0310.26
 Pre-KRT deaths53 (6%)251 (8%)458 (17%)182 (12%)
 Pre-KRT death rate, per 100 person-years4.123.215.218.11

Note: All values are shown as the median (interquartile range) or frequency (%). N missing (Brazil, France, Germany, and United States): diabetes (0, 7, 99, and 0, respectively), hypertension (36, 6, 123, and 18, respectively), heart failure (37, 7, 123, and 24, respectively), and KRT or pre-KRT deaths (11, 0, 13, and 27, respectively). Race is not reported in Germany.

Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Percentages in albuminuria categories in Germany do not add up to 100% due to rounding.

Descriptive Table of Study Sample by Country Note: All values are shown as the median (interquartile range) or frequency (%). N missing (Brazil, France, Germany, and United States): diabetes (0, 7, 99, and 0, respectively), hypertension (36, 6, 123, and 18, respectively), heart failure (37, 7, 123, and 24, respectively), and KRT or pre-KRT deaths (11, 0, 13, and 27, respectively). Race is not reported in Germany. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. Percentages in albuminuria categories in Germany do not add up to 100% due to rounding. The diagonal in Table 2 shows the number of KRT events (n=1,448) and the number of events (between 1,361 and 1,882) for each primary potential surrogate end point, with and without KRT included. The composite end point that combined KRT, an eGFR < 15 mL/min/1.73 m2, and a 40% eGFR decline had the highest number of events (n=1,882), followed closely by the composite that combined an eGFR < 15 mL/min/1.73 m2 and a 40% eGFR decline without KRT (n=1,876), then the composite that combined KRT with an eGFR < 15 mL/min/1.73 m2 (n=1,499). The percentages show that only a fraction (ie, between 45% and 100%) of events were typically shared between each pair of outcomes.
Table 2

Number of Events Shared Across Outcomes

Number (Row %; Column %) of Shared Events
KRT(1) eGFR < 15 mL/min/1.73 m2(2) 40% eGFR Decline(3) eGFR < 15 mL/min/1.73 m2 or 40% eGFR Decline(4) KRT or eGFR < 15 mL/min/1.73 m2(5) KRT or 40% eGFR Decline(6) KRT or eGFR < 15 mL/min/1.73 m2 or 40% eGFR Decline
KRT1,448749 (51.7%; 50.4%)654 (45.2%; 48.1%)845 (58.4%; 45.0%)763 (52.7%; 50.9%)682 (47.1%; 49.1%)851 (58.8%; 45.2%)
(1)1,485909 (61.2%; 66.8%)1,485 (100%; 79.2%)1,485 (100%; 99.1%)916 (61.7%; 65.9%)1,485 (100%; 78.9%)
(2)1,3611,300 (95.5%; 69.3%)917 (67.4%; 61.2%)1,361 (100%; 98.0%)1,300 (95.5%; 69.1%)
(3)1,8761,493 (79.6%; 99.6%)1,307 (69.7%; 94.1%)1,876 (100%; 99.7%)
(4)1,499930 (62.0%; 67.0%)1,499 (100%; 79.6%)
(5)1,3891,313 (94.5%; 69.8%)
(6)1,882
No. of patients8,1606,7357,0746,7356,7357,0746,735

Note: Row % and column % represent the proportions of the corresponding row and column counts of events on the diagonal. For example, of 1,448 participants with KRT events, 749 (51.7%) also had eGFR < 15 events, and of 1,485 participants with eGFR < 15 events, 749 (50.4%) also had KRT events. All countries are combined. KRT is defined by transplant or dialysis. The eGFR < 15 mL/min/1.73 m2 end points are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. N values along the diagonal are the number of events for each outcome.

Abbreviations: eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Number of Events Shared Across Outcomes Note: Row % and column % represent the proportions of the corresponding row and column counts of events on the diagonal. For example, of 1,448 participants with KRT events, 749 (51.7%) also had eGFR < 15 events, and of 1,485 participants with eGFR < 15 events, 749 (50.4%) also had KRT events. All countries are combined. KRT is defined by transplant or dialysis. The eGFR < 15 mL/min/1.73 m2 end points are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. N values along the diagonal are the number of events for each outcome. Abbreviations: eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Predictive Discrimination of Potential Surrogate End Points

Primary Potential Surrogates

Figure 1 shows the number of events and prognostic ability of each primary potential surrogate end point for predicting KRT, separated by CKD stage at enrollment. Among participants with CKD stage 3 at enrollment, all potential surrogate end points had similar prognostic ability (ρ2 between 0.95 and 0.96). However, among participants with CKD stage 4 or 5 at enrollment, the eGFR < 15 mL/min/1.73 m2 end point had higher predictive discrimination (ρ2 of 0.72) than the 40% eGFR decline end point (ρ2 of 0.65). The combination of an eGFR < 15 mL/min/1.73 m2 and a 40% eGFR decline had the highest prognostic ability among CKD stage 4 or 5 participants (ρ2 of 0.77). Including KRT made almost no difference in the number of events among CKD stage 3 participants and slightly increased the number of events among CKD stage 4 or 5 participants. The composite end point defined by the earliest of either KRT, an eGFR < 15 mL/min/1.73 m2, or an eGFR decline of 40% had the highest number of events and the highest predictive discrimination.
Figure 1

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. After separating further by age tertiles (Fig 2), we found that among participants aged 18-66 years or 76-98 years with CKD stage 3 at enrollment, the 40% eGFR decline end point slightly outperformed the eGFR < 15 mL/min/1.73 m2 end point. Otherwise, we observed mostly similar patterns as our main results but with smaller differences in prognostic ability between end points among patients at younger ages with CKD stage 4 or 5. Among women (Fig 3) and participants with diabetes (Fig 4) with CKD stage 3, the 40% eGFR decline end point also slightly outperformed the eGFR < 15 mL/min/1.73 m2 end point. Among CKD stage 4 or 5 participants, there were also smaller differences in prognostic ability between end points in women than men (Fig 3) and in participants with no diabetes than those with diabetes (Fig 4). Results by country showed similarities with our main results in France, Germany, and the United States; however, the eGFR < 15 mL/min/1.73 m2 end point performed poorest among CKD stage 3 and CKD stage 4 or 5 participants in Brazil (Fig 5). Consistently, the composite end point that included both a low eGFR and an eGFR decline had the highest prognostic ability.
Figure 2

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and age tertiles. Participants of age 66 are included in the interval [18, 66] and not in the interval (66, 76], and participants of age 76 are included in the interval (66, 76] and not in the interval (76, 98] as indicated by the parentheses and square brackets. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Figure 3

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and sex. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Figure 4

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and diabetes status. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Figure 5

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and country. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and age tertiles. Participants of age 66 are included in the interval [18, 66] and not in the interval (66, 76], and participants of age 76 are included in the interval (66, 76] and not in the interval (76, 98] as indicated by the parentheses and square brackets. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and sex. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and diabetes status. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. Scatterplots showing number of events versus ρ2 for each primary potential surrogate end point, separated by CKD stage at study enrollment and country. ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. A higher number of events implies higher statistical power. Therefore, points closer to the top right are generally optimal. ∗Potential surrogate end points that include eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. One set of sensitivity analyses included the 6,735 patients with at least 2 eGFR measurements and who started the study with an eGFR of at least 15 mL/min/1.73 m2. The second set of sensitivity analyses required a confirmatory eGFR, a KRT event, or death after the initial eGFR < 15 mL/min/1.73 m2 event to define a sustained low eGFR. Results from both these analyses were very similar to the main analysis results.

Comparisons with Other Potential Surrogates

The prognostic ability was similar or slightly higher for end points with an eGFR < 15 mL/min/1.73 m2 compared with end points with an eGFR < 10 mL/min/1.73 m2 (Table 3). The 30% eGFR decline end point had the highest prognostic ability among CKD stage 4 or 5 patients, but the lowest prognostic ability among CKD stage 3 patients. When combined with an eGFR < 15 mL/min/1.73 m2, the 40% and 50% eGFR decline end points had similar or higher prognostic abilities than the 30% eGFR decline end point. Differences in prognostic ability across different thresholds were less apparent with CKD stage 3 compared with CKD stage 4 or 5. Therefore, the poorer performance using an eGFR < 10 mL/min/1.73 m2 or a 50% eGFR decline may be explained by the lower starting eGFRs resulting in fewer end points observed before KRT events. Similarly, lower ρ2 values for CKD stage 4 patients versus CKD stage 3 patients are likely driven by fewer end points observed before KRT events.
Table 3

Number of Events and ρ2 for Each Potential Surrogate End Point, Overall and Separated by CKD Stage at Study Enrollment

Overall
CKD Stage 3
CKD Stage 4 or 5
N Eventsρ2N Eventsρ2N Eventsρ2
eGFR < 105420.640420.7515000.545
eGFR < 15a1,4850.8441290.9521,3560.724
30% eGFR decline2,2620.7426990.9141,5630.684
40% eGFR declinea1,3610.7283840.9649770.651
50% eGFR decline7610.6432180.9665430.551
eGFR < 10, 30% eGFR decline2,2690.7626990.9141,5700.707
eGFR < 10, 40% eGFR decline1,4290.7733850.9641,0440.700
eGFR < 10, 50% eGFR decline9350.7452210.9667140.656
eGFR < 15, 30% eGFR decline2,4420.8216990.9131,7430.751
eGFR < 15, 40% eGFR declinea1,8760.8653910.9631,4850.774
eGFR < 15, 50% eGFR decline1,6330.8662370.9671,3960.758
Including KRT
 KRT, eGFR < 105780.640440.7515340.545
 KRT, eGFR < 15a1,4990.8441310.9521,3680.724
 KRT, 30% eGFR decline2,2750.7426990.9141,5760.684
 KRT, 40% eGFR declinea1,3890.7283850.9641,0040.651
 KRT, 50% eGFR decline8020.6432200.9665820.551
 KRT, eGFR < 10, 30% eGFR decline2,2760.7626990.9141,5770.707
 KRT, eGFR < 10, 40% eGFR decline1,4460.7733860.9641,0600.700
 KRT, eGFR < 10, 50% eGFR decline9580.7452230.9667350.656
 KRT, eGFR < 15, 30% eGFR decline2,4450.8216990.9131,7460.751
 KRT, eGFR < 15, 40% eGFR declinea1,8820.8653920.9631,4900.774
 KRT, eGFR < 15, 50% eGFR decline1,6400.8662390.9671,4010.758

Note: ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. End points that include eGFR < 10 mL/min/1.73 m2 or eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of at least 10 or 15 mL/min/1.73 m2, respectively.

Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Primary potential surrogate end points, which are also already illustrated in Fig 1. End points including KRT have the same ρ2 as the end points without KRT but are displayed to demonstrate the increase in number of events.

Number of Events and ρ2 for Each Potential Surrogate End Point, Overall and Separated by CKD Stage at Study Enrollment Note: ρ2 represents the proportion of variability in KRT that can be explained by the potential surrogate, with 0 representing no prognostic value of the potential surrogate and 1.0 representing a perfect prediction. End points that include eGFR < 10 mL/min/1.73 m2 or eGFR < 15 mL/min/1.73 m2 are limited to those with enrollment eGFR of at least 10 or 15 mL/min/1.73 m2, respectively. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy. Primary potential surrogate end points, which are also already illustrated in Fig 1. End points including KRT have the same ρ2 as the end points without KRT but are displayed to demonstrate the increase in number of events.

Effects of KRT Risk Factors Across Outcomes

Older age and a higher eGFR were associated with lower risks of KRT, whereas male sex, Black race, diabetes, heart failure, and higher albuminuria were associated with higher risks of KRT (Table 4). Potential surrogate end points had similar effect estimates to KRT for age. However, they tended to have attenuated effects for sex, diabetes, and heart failure and larger effect estimates for albuminuria. These results were similar in sensitivity analyses.
Table 4

Adjusted Hazard Ratios (95% Confidence Intervals) of Time-to-Event Outcomes by Patient Demographic and Baseline Clinical Characteristics

KRT(1) eGFR < 15 mL/min/1.73 m2(2) 40% eGFR Decline(3) eGFR < 15 mL/min/1.73 m2 or 40% eGFR Decline(4) KRT or eGFR < 15 mL/min/1.73 m2(5) KRT or 40% eGFR Decline(6) KRT or eGFR < 15 mL/min/1.73 m2 or 40% eGFR Decline
N events1,4481,4851,3611,8761,4991,3891,882
Age, per 5 y0.93 (0.91-0.95)0.92 (0.90-0.94)0.90 (0.88-0.92)0.92 (0.90-0.94)0.92 (0.90-0.94)0.90 (0.88-0.92)0.92 (0.90-0.94)
Male sex1.46 (1.29-1.64)1.01 (0.90-1.12)0.98 (0.88-1.10)0.99 (0.90-1.09)1.00 (0.90-1.12)0.98 (0.87-1.10)0.99 (0.90-1.09)
Black race1.41 (1.09-1.82)1.35 (1.04-1.74)1.25 (0.96-1.63)1.30 (1.03-1.63)1.36 (1.06-1.74)1.24 (0.95-1.62)1.30 (1.03-1.63)
Diabetes1.27 (1.12-1.43)1.08 (0.97-1.22)1.07 (0.95-1.20)1.10 (0.99-1.21)1.09 (0.97-1.22)1.07 (0.95-1.20)1.10 (0.99-1.21)
Hypertension1.14 (0.95-1.37)1.01 (0.86-1.20)1.12 (0.94-1.33)1.03 (0.89-1.20)1.00 (0.85-1.18)1.12 (0.94-1.34)1.03 (0.89-1.20)
Heart failure1.45 (1.23-1.71)1.07 (0.90-1.27)1.03 (0.86-1.24)1.14 (0.98-1.32)1.08 (0.91-1.28)1.07 (0.89-1.29)1.15 (0.99-1.33)
Baseline eGFR, per mL/min/1.73 m20.87 (0.86-0.88)0.83 (0.82-0.84)0.98 (0.97-0.99)0.91 (0.90-0.92)0.83 (0.82-0.84)0.98 (0.97-0.99)0.91 (0.90-0.92)
Baseline albuminuria
 Normal to mildly increased1.00 (ref)1.00 (ref)1.00 (ref)1.00 (ref)1.00 (ref)1.00 (ref)1.00 (ref)
 Moderately increased1.25 (1.00-1.56)1.41 (1.16-1.70)1.71 (1.37-2.13)1.48 (1.25-1.76)1.41 (1.17-1.70)1.70 (1.37-2.11)1.47 (1.24-1.74)
 Severely increased2.22 (1.80-2.74)2.31 (1.92-2.78)3.30 (2.71-4.02)2.52 (2.14-2.96)2.28 (1.90-2.75)3.29 (2.71-4.00)2.50 (2.13-2.94)
 Nephrotic syndrome4.95 (3.95-6.22)5.07 (4.16-6.17)8.47 (6.86-10.44)5.92 (5.00-7.02)5.09 (4.19-6.20)8.46 (6.87-10.42)5.90 (4.99-6.99)

Note: Each column represents 1 multivariable cause-specific hazard model that includes all listed risk factors, stratified by country and CKD stage at study enrollment. Outcomes with eGFR < 15 mL/min/1.73 m2 were limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater.

Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Adjusted Hazard Ratios (95% Confidence Intervals) of Time-to-Event Outcomes by Patient Demographic and Baseline Clinical Characteristics Note: Each column represents 1 multivariable cause-specific hazard model that includes all listed risk factors, stratified by country and CKD stage at study enrollment. Outcomes with eGFR < 15 mL/min/1.73 m2 were limited to those with enrollment eGFR of 15 mL/min/1.73 m2 or greater. Abbreviations: CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Discussion

In this systematic evaluation of potential surrogate end points for kidney failure among multinational patients with moderate to advanced CKD, we found some different results in surrogate end point performance for different subgroups of patients. An eGFR < 15 mL/min/1.73 m2 had higher prognostic ability than a 40% eGFR decline in the subgroup of patients with CKD stage 4 or 5, whereas the prognostic ability was similar for most patients with CKD stage 3. The combination of the 2 eGFR-based end points had the highest prognostic ability for predicting KRT, regardless of the CKD stage at study enrollment, age, diabetes status, or country. Including KRT into composite end points increased the number of events and, therefore, power. With further evaluation—that is, of the first Prentice criterion for surrogacy—the proposed end point that includes KRT and both a low eGFR and a percentage eGFR decline can therefore be appropriate for many studies to assess CKD progression among patients with advanced CKD in diverse clinical settings. This end point is consistent with recommendations from the International Society of Nephrology for clinical trials and would be useful for future trials to evaluate treatment effects for new interventions that aim to slow CKD progression in a similar population. Based on our findings, observational studies could similarly benefit by increased power and/or a shorter follow-up time required to evaluate new biomarkers, practice patterns, or risk factors for CKD progression. This surrogate end point can also be helpful to quickly interpret the application of trial results or other exposure effects in the real world, allowing for earlier introduction of therapies that could prevent or postpone KRT. Finally, the development of risk scores incorporating this surrogate could facilitate more timely assessments of high- versus low-risk patients, which can facilitate a precision medicine approach to CKD and/or prioritization of resources. In the developing world, low availability of specialists, kidney allografts and immunosuppressive medications, dialysis supplies, and other expensive treatments present barriers to care and necessitate finely tuned resource allocation strategies. We initially considered counterintuitive the finding that an eGFR decline of 50% had the lowest predictive discrimination compared with eGFR declines of 40% or 30% among CKD stage 4 or 5 patients. Similarly, an eGFR < 10 mL/min/1.73 m2 had lower predictive discrimination than an eGFR < 15 mL/min/1.73 m2. However, the less apparent differences within CKD stage 3 indicate that these findings were likely because of the fact that patients starting at lower eGFRs—especially CKD stage 4 patients—often reached KRT before experiencing the 50% eGFR decline or eGFR < 10 mL/min/1.73 m2 end points. Although the focus of this study was not to determine an optimal threshold for a low eGFR or a percentage eGFR decline, these results demonstrate unique considerations in the advanced CKD population. The choice of threshold may be driven more by improvements in power through greater numbers of end point events. There are also situations in which the composite end point with KRT, reaching a low eGFR, and a percentage eGFR decline may not be the most appropriate outcome. First, end points that only include KRT may be more suitable for studies specifically interested in delaying the need for dialysis: that is, studies on low-protein diets, acidosis treatment, or anemia management for prolonging the tolerability of a low eGFR without the need for KRT. Second, including a low eGFR in the composite end point implies that those who already have a low eGFR at study initiation are excluded, which can reduce power and decrease the generalizability of results for those with a low eGFR. For studies among nondialysis CKD stage 5 patients, for example, an end point that only includes KRT may be preferred. Third, recent work has shown that an eGFR slope is a valid surrogate for clinical trials and more useful than time-to-event end points for study populations with higher baseline eGFRs, shorter follow-ups, and interventions with uniform treatment effects. Other kidney-related end points are also important to consider but are challenging in observational studies. Unfortunately, we were unable to clearly identify patients managing kidney failure without replacement therapy (ie, with comprehensive conservative care and/or because of limited dialysis access). Particularly for older patients and others with a lower life expectancy, conservative management is a popular choice and should be considered in future studies as a treatment modality for kidney failure. Kidney-related death is also challenging to capture, and our study was not designed to do so. Rather than include kidney-related death as a hard clinical outcome in our evaluation, we treated all-cause death as a competing event. Future studies that can adequately ascertain causes of death could be useful to explore surrogates of kidney-related death, although this may be very difficult to achieve given practical limitations, especially in the real-world setting. One limitation of our study was that variable visit frequency resulted in variable eGFR measurement frequency. Future studies using protocol-based eGFR measurements could assess whether results differ with more frequent and/or standardized timings of eGFR measurements, while simulation studies that artificially remove observed eGFRs could mimic settings with less frequent eGFR measurements. Nonetheless, our eGFR-based end point definitions and study findings should closely resemble those expected in the real world. Furthermore, a study on human immunodeficiency virus mortality biomarkers showed little difference in the predictive value of biomarkers for different biomarker measurement frequencies. Strengths of our study include the large, multinational cohort of patients with advanced CKD (median eGFR of 27 mL/min/1.73 m2) that is representative of real-world practice and a comprehensive statistical methods comparison of potential surrogate end points. This study provides empirical evidence that a potential surrogate end point that combines KRT, a low eGFR, and a percentage eGFR decline can accurately represent KRT while increasing the number of outcome events among patients with advanced CKD and in a multinational, real-world setting. Further research is needed to confirm the end point’s suitability as a surrogate by evaluating its ability to fully capture treatment effects on KRT. The harmonized use of this approach for patient-centered kidney failure outcomes in clinical research will accelerate the translation of new discoveries to clinical practice, such as the identification of modifiable risk factors of kidney failure and effective treatments to prevent kidney failure.
  23 in total

Review 1.  Kidney failure: aims for the next 10 years and barriers to success.

Authors:  Giuseppe Remuzzi; Ariela Benigni; Fredric O Finkelstein; Jean-Pierre Grunfeld; Dominique Joly; Ivor Katz; Zhi-Hong Liu; Toshio Miyata; Norberto Perico; Bernardo Rodriguez-Iturbe; Luca Antiga; Franz Schaefer; Arrigo Schieppati; Robert W Schrier; Marcello Tonelli
Journal:  Lancet       Date:  2013-05-31       Impact factor: 79.321

Review 2.  GFR decline as an end point for clinical trials in CKD: a scientific workshop sponsored by the National Kidney Foundation and the US Food and Drug Administration.

Authors:  Andrew S Levey; Lesley A Inker; Kunihiro Matsushita; Tom Greene; Kerry Willis; Edmund Lewis; Dick de Zeeuw; Alfred K Cheung; Josef Coresh
Journal:  Am J Kidney Dis       Date:  2014-10-16       Impact factor: 8.860

3.  Executive summary of the KDIGO Controversies Conference on Supportive Care in Chronic Kidney Disease: developing a roadmap to improving quality care.

Authors:  Sara N Davison; Adeera Levin; Alvin H Moss; Vivekanand Jha; Edwina A Brown; Frank Brennan; Fliss E M Murtagh; Saraladevi Naicker; Michael J Germain; Donal J O'Donoghue; Rachael L Morton; Gregorio T Obrador
Journal:  Kidney Int       Date:  2015-04-29       Impact factor: 10.612

4.  Nomenclature for kidney function and disease: report of a Kidney Disease: Improving Global Outcomes (KDIGO) Consensus Conference.

Authors:  Andrew S Levey; Kai-Uwe Eckardt; Nijsje M Dorman; Stacy L Christiansen; Ewout J Hoorn; Julie R Ingelfinger; Lesley A Inker; Adeera Levin; Rajnish Mehrotra; Paul M Palevsky; Mark A Perazella; Allison Tong; Susan J Allison; Detlef Bockenhauer; Josephine P Briggs; Jonathan S Bromberg; Andrew Davenport; Harold I Feldman; Denis Fouque; Ron T Gansevoort; John S Gill; Eddie L Greene; Brenda R Hemmelgarn; Matthias Kretzler; Mark Lambie; Pascale H Lane; Joseph Laycock; Shari E Leventhal; Michael Mittelman; Patricia Morrissey; Marlies Ostermann; Lesley Rees; Pierre Ronco; Franz Schaefer; Jennifer St Clair Russell; Caroline Vinck; Stephen B Walsh; Daniel E Weiner; Michael Cheung; Michel Jadoul; Wolfgang C Winkelmayer
Journal:  Kidney Int       Date:  2020-03-09       Impact factor: 10.612

5.  GFR decline and subsequent risk of established kidney outcomes: a meta-analysis of 37 randomized controlled trials.

Authors:  Hiddo J Lambers Heerspink; Hocine Tighiouart; Yingying Sang; Shoshana Ballew; Hasi Mondal; Kunihiro Matsushita; Josef Coresh; Andrew S Levey; Lesley A Inker
Journal:  Am J Kidney Dis       Date:  2014-10-16       Impact factor: 8.860

6.  Evaluation of human immunodeficiency virus biomarkers: inferences from interval and clinical cohort studies.

Authors:  Bryan Lau; Stephen J Gange; Gregory D Kirk; Richard D Moore
Journal:  Epidemiology       Date:  2009-09       Impact factor: 4.822

7.  The CKD Outcomes and Practice Patterns Study (CKDopps): Rationale and Methods.

Authors:  Laura Mariani; Bénédicte Stengel; Christian Combe; Ziad A Massy; Helmut Reichel; Danilo Fliser; Roberto Pecoits-Filho; Antonio A Lopes; Kunihiro Yamagata; Takashi Wada; Michelle M Y Wong; Elodie Speyer; Friedrich K Port; Ronald L Pisoni; Bruce M Robinson
Journal:  Am J Kidney Dis       Date:  2016-04-22       Impact factor: 8.860

8.  Using All Longitudinal Data to Define Time to Specified Percentages of Estimated GFR Decline: A Simulation Study.

Authors:  Jarcy Zee; Sarah Mansfield; Laura H Mariani; Brenda W Gillespie
Journal:  Am J Kidney Dis       Date:  2018-09-21       Impact factor: 8.860

9.  KDIGO 2020 Clinical Practice Guideline for Diabetes Management in Chronic Kidney Disease.

Authors: 
Journal:  Kidney Int       Date:  2020-10       Impact factor: 10.612

10.  Decline in estimated glomerular filtration rate and subsequent risk of end-stage renal disease and mortality.

Authors:  Josef Coresh; Tanvir Chowdhury Turin; Ron T Gansevoort; Andrew S Levey; Kunihiro Matsushita; Yingying Sang; Shoshana H Ballew; Lawrence J Appel; Hisatomi Arima; Steven J Chadban; Massimo Cirillo; Ognjenka Djurdjev; Jamie A Green; Gunnar H Heine; Lesley A Inker; Fujiko Irie; Areef Ishani; Joachim H Ix; Csaba P Kovesdy; Angharad Marks; Takayoshi Ohkubo; Varda Shalev; Anoop Shankar; Chi Pang Wen; Paul E de Jong; Kunitoshi Iseki; Benedicte Stengel
Journal:  JAMA       Date:  2014-06-25       Impact factor: 56.272

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.