Prediction models used in the progression of chronic kidney disease: A scoping review.

OBJECTIVE: To provide a review of prediction models that have been used to measure clinical or pathological progression of chronic kidney disease (CKD).
DESIGN: Scoping review. DATA SOURCES: Medline, EMBASE, CINAHL and Scopus from the year 2011 to 17th February 2022. STUDY SELECTION: All English written studies that are published in peer-reviewed journals in any country, that developed at least a statistical or computational model that predicted the risk of CKD progression. DATA EXTRACTION: Eligible studies for full text review were assessed on the methods that were used to predict the progression of CKD. The type of information extracted included: the author(s), title of article, year of publication, study dates, study location, number of participants, study design, predicted outcomes, type of prediction model, prediction variables used, validation assessment, limitations and implications.
RESULTS: From 516 studies, 33 were included for full-text review. A qualitative analysis of the articles was compared following the extracted information. The study populations across the studies were heterogenous and data acquired by the studies were sourced from different levels and locations of healthcare systems. 31 studies implemented supervised models, and 2 studies included unsupervised models. Regardless of the model used, the predicted outcome included measurement of risk of progression towards end-stage kidney disease (ESKD) of related definitions, over given time intervals. However, there is a lack of reporting consistency on details of the development of their prediction models.
CONCLUSIONS: Researchers are working towards producing an effective model to provide key insights into the progression of CKD. This review found that cox regression modelling was predominantly used among the small number of studies in the review. This made it difficult to perform a comparison between ML algorithms, more so when different validation methods were used in different cohort types. There needs to be increased investment in a more consistent and reproducible approach for future studies looking to develop risk prediction models for CKD progression.

Introduction

Chronic Kidney Disease (CKD) is a global health burden with an estimated 5 to 10 million annual deaths worldwide due to kidney disease [1, 2]. Current data predict CKD will be the fifth leading cause of death worldwide by the year 2040 [3]. CKD is characterised by a gradual loss of the kidney’s ability to remove wastes from the blood, and the severity of the disease is determined by the individual’s estimated glomerular filtration rate (eGFR) [4]. CKD is arbitrarily categorised into five progressive stages with stage five often referred as end-stage kidney disease (ESKD), and its progression often leads to multiple overlapping complications [5, 6]. There is a spectrum of pathological, hereditary, and sociodemographic factors known to contribute to a decline in kidney function [5-11]. These factors include age (≥60 years), smoking, low socioeconomic status, diabetes, hypertension, cardiovascular disease, body mass index (≥30 kg/m2), family history of kidney disease and use of pain-reliving medications [9-11].

The global nephrology community recognises that current models of care are insufficient to curb the growing CKD burden and that new care models are required to improve patient outcomes [12-14]. It has been suggested that the management framework for CKD needs to consider the disease across the entire life course of each individual [13]. New care models also need to consider improvements in areas such as disease surveillance, mitigation of risk factors, expanding research knowledge, and developing novel clinical interventions to slow the progression of CKD [13]. Despite having identified a number of risk factors associated with the onset of CKD, gaps remain in the methods for predicting the risk of CKD progression and interventions to slow CKD progression [13, 15, 16]. In addition, a large number of patients with CKD remain undetected through health systems [16] and clinicians have the challenge of managing the growing number of cases with limited tools for triaging patients.

Predictive modelling techniques

Predictive modelling techniques applied to the growing number of clinical datasets have shown promise in accurately predicting the progression of chronic disease in the population [17-23]. Previous attempts have employed a wide range of prediction models, from well-established generalised linear models to more recent Machine Learning (ML) techniques [17-23]. Renal clinicians and researchers recognise the significant potential in developing risk prediction models that can improve our ability to identify individuals at risk, in addition to potentially improving our understanding of the natural history of disease progression and contribute to the clinical management of CKD [22, 24, 25]. The application of ML models provides capacity to tap into the information contained in large and complex datasets and exploit the complex non-linear dependencies [18, 21, 23, 26–28]. The application of these analytical techniques promises to improve our understanding of CKD progression and inform key interventions to help slow progression and reduce the burden of CKD [11, 29–31]. Moreover, it can help inform clinicians with regards to treatment options by increasing confidence in the patient’s likely prognostic course [32, 33].

Whilst the use of predictive modelling is gaining traction in CKD research, efforts are beset by the lack of a uniform approach to the reporting of important methodological advancements and developments of prediction models for CKD progression [23–25, 34]. This lack of consistent reporting of key characteristics and the evaluation of model performance has likely impeded uptake and support of prediction models by clinicians, while undermining reproducibility of research and clinical utility [24]. An example of a standardised reporting guidelines can be seen with the Equator Network who published the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement that consists of a checklist considered vital by healthcare professionals, methodologists and journal editors, for the transparent reporting of multivariable prediction model studies [35, 36]. By implementing such a checklist, reporting can be standardised and reproducibility improved while facilitating progress towards cross-validation between different health settings and populations globally.

With inconsistency in the advancements of predictive modelling used in CKD progression analysis, this paper provides timely evidence from a scoping review about prediction models used in the progression of CKD. The review aims to 1) Identify and outline existing models used in predicting CKD progression; 2) To understand what measured outcome(s) and selected significant variables were chosen when building a prediction model for CKD progression. Its results will help inform clinical and scientific developments in this area and provide a better understanding of CKD progression.

Classification of predictive models

Predictive modelling techniques can be generally classified into four broad categories; supervised, unsupervised, semi-supervised and reinforcement learning; with supervised and unsupervised being the most commonly applied in the medical field [18, 22, 25, 33]. This was also reflected in this scoping review where only supervised and unsupervised techniques were found in the studies that were assessed for full-text and will be discussed in later sections.

Supervised techniques can be further divided by the type of outcome they predict, with the two major groupings including continuous outcomes and categorical outcomes [37]. The regression technique is utilised when output variables are continuous data, such as values for weight or height [37]. On the other hand, classification techniques are commonly used for simpler data such as nominal or categorical data, where a simple binary outcome or a few predetermined categorical responses are required [37]. Supervised techniques have their own challenges and require sufficiently large volumes of correctly labelled data initially to perform accurately [26]. Some examples of commonly used supervised machine learning algorithms are linear or logistic regression, artificial neural networks, decision trees, k-nearest neighbours (KNN), random forest for classification, gradient boosting and support vector machines (SVM) [37, 38].

Unsupervised techniques can be further grouped into 2 types, clustering or association. Clustering is the process of segregating data into groups according to similar characteristics, whereas association is the process of identifying newer relationships within datasets based on certain selected attributes of the data. Additionally, unsupervised algorithms do not need manual labelling of datasets, as they can group data into clusters or identifying associations by themselves [26, 38]. The end result of these methods is to provide a simplified interpretation of a complex dataset, and often to sort observations into groups [38]. These groups can then be inspected for their ability to predict the outcome of interest. Some common examples of unsupervised ML algorithms include K-means clustering, mixture models, distribution models, dimensionality reduction, independent component analysis and principal component analysis.

Methods

A scoping review was selected as it allows identification and mapping of existing evidence and to investigate and determine the knowledge gaps surrounding the topic [39]. This method is suitable for examining emerging evidence across a broad field of study and was guided by the PRISMA extension for Scoping Reviews (PRISMA-ScR), following a standardised approach to search, screen, and report articles [40].

Data sources and searches

This scoping review was performed in the context of a larger study that investigates improving chronic kidney disease outcomes using linked routine records. With this context in mind, an initial concept grid was developed to address the objectives of the scoping review, together with the subsequent search histories that can be found in S1 Appendix. The review included studies in the past 10 years that developed or utilised any type of predictive modelling to predict the progression of CKD towards more severe stage of the disease. Articles included were published in peer-reviewed journals from any country, in the English language, between 1st January 2011 to 17th February 2022 inclusive. The which addresses the objectives of the scoping review. Four electronic databases, Medline, EMBASE, CINAHL, and Scopus were chosen for their bibliographic peer-reviewed publications that covers a broad range of medical life sciences, allied health, nursing and healthcare. Fig 1 illustrates the overall flow diagram of the literature review.

Fig 1

PRISMA flow diagram.

Study selection and search

Four main overarching concepts, as described in the concept grid, were selected for the development of the search strategy, they were: kidney disease; disease progression; techniques; outcomes. The initial search strategy was developed for use in Medline and subsequently adapted for the other databases- keywords and sub headers were amended to reflect search terms used in each respective database. The steps used in Medline are as follows:

(chronic kidney disease* or chronic renal disease* or CKD or kidney disease* or kidney failure).ti,ab.

Renal Insufficiency, Chronic/ or Kidney Failure, Chronic/ or Diabetic Nephropathies/

1 or 2

(progress* adj7 (CKD or disease)).ti,ab.

Disease Progression/

4 or 5

(deep learning or machine learning or artificial intelligence or algorithms or prediction model* or statistic* model*).ti,ab.

Artificial Intelligence/ or Big data/ or machine learning/ or algorithms/ or models, statistical/

7 or 8

(End stage renal disease or ESRD or Transplant* or Hemodialysis or Hospitali?ation or Mortality or Morbidity or Heart failure or Stroke).ti,ab.

Dialysis/ or Peritoneal Dialysis/ or Renal Dialysis/ or Kidney Transplantation/ or Cardiovascular Diseases/ or Hypertension/ or Coronary Artery Disease/ or Coronary Disease/ or Hospitalization/ or Heart failure/ or Stroke/

10 or 11

3 and 6 and 9 and 12

limit 13 to (english language and yr = "2011 -Current")

The first key concept for kidney disease included keywords and MeSH terms used in steps 1 and 2, to capture different types of chronic kidney diseases, such as diabetic nephropathies or similar diseases, since it is chronic disease with multiple overlapping manifestations with associated comorbidities and risk factors [39]. The type of model used was not limited and included either statistical or ML algorithms used to predict CKD progression towards a wide range of clinical outcomes. A clear distinction was made that the study should examine prediction models for CKD progression, rather than models that predicted the onset of CKD.

Title and abstract screening

All articles were exported into EndNote and duplicate articles were removed. Two independent reviewers performed title and abstract screening by applying inclusion and exclusion criteria. Studies included in the review were based on inclusion criteria, which included an implementation of a predictive model that was developed through analysis of health records; and they also had to include a reported outcome on the progression of CKD. The list of exclusion criteria can be found in Table 1.

Table 1

Exclusion criteria during title and abstract screening.

• Animal studies

• Was not in the English language

• The study’s primary focus was not on the progression of CKD

• The article was a commentary, conference paper, editorial, a review, an opinion piece, a supplementary abstract.

• The study did not consider determining progression of CKD from data records.

• Study that looked at risk factors, specific markers, case study

• Interventional studies

The authors recognised that CKD is a very broad topic and did not place restrictions on the type of predictive model that was developed, the population of interest, the source of health data records, the predictive variables that were used, or a specific outcome. If there were any disagreements to the exclusion of articles, it was resolved through a discussion between the two reviewers—if required, a third reviewer for adjudication.

Data extraction and quality assessment

The researchers wanted to better understand the significant considerations taken into account when developing a prediction model for CKD progression, and to explore how these studies measured CKD progression [35]. The information extracted followed the items listed on the TRIPOD statement such as the article’s title, author(s), publication year, year of study period, study locations and population size, study design (retrospective or prospective), predicted outcome(s), type of prediction model, predictors in the model, validation assessment, limitations, implications, eGFR formula and data balancing. Corresponding authors were contacted by email if full text was not available and were excluded if unobtainable.

Results

The initial search had a combined total of 516 articles across Medline, EMBASE, CINAHL and Scopus, of which 188 duplicates were removed. 328 articles were then screened for their title and abstract, of which 245 articles were excluded based on exclusion criteria. 83 articles were then assessed for full-text eligibility by inclusion criteria, and subsequently 33 articles remained and were included in final qualitative review. Table 2 summarises the final articles that were included for full-text review.

Table 2

Summary of full-text review.

Author(s), Title of article	Year of publication (study dates)	Study location (n = size of cohort)	Study design (retrospective or prospective)	Predicted Outcome(s)	Type of prediction model	Predictors in the model
						Modifiable	Non-modifiable
Supervised technique–cox regression
Akbari et al., Prediction of Progression in Polycystic Kidney Disease Using the Kidney Failure Risk Equation and Ultrasound Parameters [41]	2020 (Jan 2010 –Jun 2017)	Eastern Ontario, Canada (n = 340)	Retrospective	A composite of 1) eGFR decline ≥30% from baseline and/or 2) the need for KRT (initiation of dialysis or pre-emptive transplantation).	Cox proportional hazards	Co-morbidities (cardiac disease, cancer, diabetes, hypertension, hyperlipidaemia), longitudinal biochemistry (proteinuria, eGFR by CKD-epi), kidney failure risk equation (KFRE), systolic blood pressure (SBP), and total kidney volume (TKV) were modelled as continuous predictors.	Age, sex.
Chang et al., A predictive model for progression of CKD [42]	2019 (2006–2013)	Taiwan (n = 1549)	Retrospective	Kidney failure; dialysis.	Cox proportion hazard model survival analysis was used to investigate the risks of CKD progression to dialysis	Primary disease category, risk factors, co-morbidities (hypertension, hyperlipidaemia, hyperglycaemia, proteinuria, hypoproteinemia), and biochemical test values.	Age, sex, family medical history.
Cornec-Le Gall et al, The PROPKD Score: A New Algorithm to Predict Renal Survival in Autosomal Dominant Polycystic Kidney Disease [43]	2016 (2009–2015)	Brittany, France (n = 1341)	Retrospective	PROPKD score: low, intermediate, and high risk for progression to ESKD.	Multivariate cox regression	Need for antihypertensive therapy before 35 years of age (referred hereinafter as age at hypertension onset, occurrence of the first urologic event before 35 years of age, and genetic status.	Age, sex.
Crnogorac et al., Clinical, serological and histological determinants of patient and renal outcome in ANCA-associated vasculitis (AAV) with renal involvement: an analysis from a referral centre [44]	2017 (Jan 2003 –Dec 2013)	University Hospitals Dubrava and Merkur, Zagreb, Croatia (n = 83)	Retrospective	Primary outcome was combined endpoint patient death or progression to ESKD. Secondary outcomes were patient survival and progression to ESKD (kidney survival) singularly and disease relapse.	Univariate and multivariate cox proportional hazards regression analysis for each outcome was done. Multivariate Cox proportional hazards regression was done using backward stepwise analysis. Time to outcomes survival analysis was made using Kaplan–Meier estimates and categories were compared using log-rank test.	eGFR, Proteinuria, CRP, renal syndrome, pathohistological phenotype (normal/crescentic/sclerotic glomeruli, IFTA, fibrinoid necrosis)	Age, gender, time to diagnosis (months).
Dai et al., A predictive model for progression of chronic kidney disease to kidney failure using a large administrative claims database [45]	2021 (2015–2017)	United States (n = 74,114)	Retrospective	From CKD stages 3 or 4 who were at high risk for progression to kidney failure	Logistic regression model.	CKD stage, hypertension (HTN), diabetes mellitus (DM), congestive heart failure, peripheral vascular disease, anaemia, hyperkalaemia (HK), prospective episode risk group score, and poor adherence to renin-angiotensin-aldosterone system inhibitors. The strongest predictors of progression to kidney failure were CKD stage (4 vs 3), HTN, DM, and HK.	Age, sex.
Dunkler et al., Risk Prediction for Early CKD in Type 2 Diabetes [46]	2015 (2001–2008, 2003–2011)	The ONTARGET (n = 25,620) and the ORIGIN Trial (n = 12,537)–over 40 countries	Prospective	The outcome states after 5.5 years of follow-up were defined as alive without CKD, alive with CKD, or dead.	Two prediction models were developed: a laboratory model, containing laboratory markers of kidney function, sex and age, and a clinical model, containing the same markers and some clinical variables. Multinomial logistic regression was applied to develop prediction models for the three outcome states.	Baseline albuminuria, eGFR, UACR (urinary albumin-creatinine ratio), eGFR, albuminuria stage (normo- or microalbuminuria)	Age, sex.
Halbesma et al., Development and validation of a general population renal risk score [47]	2011 (1997–1998)	City of Groningen, Netherlands (n = 6,809)	Prospective	A risk score identifies patients at risk for progressive CKD,	Backward logistic regression analysis	Hypertension, smoking, BMI, baseline eGFR and eGFR2, urea & electrolytes (U&E), C-reactive protein (CRP), SBP, plasma total cholesterol, glucose, triglycerides, urinary albumin exretion, and known HTN.	Age, sex, family history for CVD/CKD.
Hasegawa et al., Clinical prediction models for progression of chronic kidney disease to end stage kidney failure under pre-dialysis nephrology care: Results from the chronic kidney disease Japan cohort study [48]	2018 (2007–2008)	CKD-JAC study—Japan (n = 2034)	Retrospective	ESKD onset, defined as the need for dialysis or pre-emptive kidney transplantation at 3 years	Cox proportional hazard regression	Physical examination findings, including body mass index (BMI) and systolic blood pressure (SBP); comorbid conditions (diabetes and hypertension), laboratory variables (eGFR, the urinary albumin-creatinine ratio (UACR), serum creatinine, serum sodium, serum albumin (ALB), haemoglobin (Hb), serum calcium, serum phosphorus, intact parathormone (iPTH), and FGF-23).	Age, sex.
Kang et al., An independent validation of the kidney failure risk equation in an Asian population [49]	2020 (Jan 2001 –Dec 2016)	Korea (n = 38,905)	Retrospective	2- and 5-year risk of ESKD	Cox proportional hazards models were fit using the variables included in each of the original equations, and baseline hazard was analysed.	eGFR, UACR, serum calcium, serum phosphorus, serum ALB, serum total CO2, diabetes mellitus, and hypertension, were obtained to calculate the KFREs. Because bicarbonate is not checked routinely, total CO2 value was used as a bicarbonate value.	Age, sex.
Kataoka et al., Time series changes in pseudo-R2 values regarding maximum glomerular diameter and the Oxford MEST-C score in patients with IgA nephropathy: A long-term follow-up study [50]	2020 (1993–2017)	Kameda General Hospital, Japan (n = 43)	Prospective	Primary outcome was kidney disease progression, defined as ≥ 50% eGFR decline from baseline, or the development of ESKD requiring dialysis.	Kidney prognostic factors were also evaluated in cox regression analyses, and the Kaplan-Meier method was used for survival analyses. The prognostic variables for the kidney outcomes were assessed using univariate and multivariate cox proportional hazards models.	BMI, eGFR, laboratory results (urea and electrolytes, triglycerides, immunoglobulins, proteinuria), comorbidities, concomitant drugs, initial treatments, histological findings.	Age, sex.
Kim et al., Systolic blood pressure and chronic kidney disease progression in patients with primary glomerular disease [51]	2021 (2005–2017)	Korea (n = 157)	Retrospective	A composite including ≥ 50% decrease in eGFR from the baseline (in at least two consecutive measurements), and ESKD (Initiation of maintenance dialysis or kidney transplantation).	A time-varying Cox model	BMI, smoking status, comorbid disease, glomerular disease type, laboratory measurements (eGFR, UPCR, total cholesterol, phosphorus, and ALB), medications (renin–angiotensin–aldosterone system (RAAS) blockers, diuretics, statins, immunosuppressive drugs), and remission status	Age, sex.
Li et al., Dynamic Prediction of Renal Failure Using Longitudinal Biomarkers in a Cohort Study of Chronic Kidney Disease [52]	2017	African American Study of Kidney Disease and Hypertension (AASK) (n = 1094)	Prospective	Survival regression models relating the predictor variables measured at or prior to the time of prediction to the time gap from the prediction time to the outcome event of interest (ESKD).	The Landmark Model and Predicted Probabilities. This is a variant of the Cox model.	Any hospitalization in the history window, the most recent log urine protein-to-creatinine ratio (Up/Cr) in the history window, the eGFR at the time of prediction, and the eGFR slope in the history window.	Age at the time of prediction
Maziarz et al, Homelessness and Risk of End-stage Renal Disease [53]	2014 (Jan 1996 –Feb 2008)	Department of Public Health of the City and County of San Francisco (n = 16,656)	Retrospective	Risk of ESKD within 1, 3 and 5 years.	Linked with the national ESKD registry (United States Renal Data System) files based on patient last name, first name, date of birth, and Social Security Number. Four proportional hazards models each building on the previous, stratified by housing status.	eGFR, dipstick proteinuria, health insurance coverage, comorbidities (diabetes mellitus, CVD, hypertension, substance abuse, and chronic viral disease), and additional laboratory variables (serum ALB, serum calcium, serum cholesterol, and haemoglobin)	Age, sex, race-ethnicity
Palant et al., The association of serum creatinine variability and progression to CKD [54]	2015 (1999–2005)	United States of America (n = 342,086)	Retrospective	Probability of entry into stage 4 CKD (30 mL/min/1.73 m2) over a continuous timeline.	Logistic regression model. Time-to-event analysis was also used Kaplan-Meier and Cox regression	Initial eGFR, serum creatinine (SCr) variability, SCr slope, number of months with SCr readings, and comorbidities (DM, CAD, PNE, MI, angina, AKI, COPD, CHF).	Age, sex, race
Park et al., Predicted risk of renal replacement therapy at arteriovenous fistula referral in chronic kidney disease [55]	2020 (May 2013 –May 2018)	Kaiser Permanente Northwest, Oregon and Washington, USA (n = 205)	Prospective	2-year risk of KRT (following stage 4 CKD patients with 2-year observation period)	Cox regression model outlined by Schroeder et al.	eGFR (calculated from Chronic Kidney Disease Epidemiology Collaboration equation), haemoglobin, presence of proteinuria or albuminuria, systolic blood pressure, antihypertensive use, and Diabetes Complications Severity Index (The index was based on the International Statistical Classification of Diseases and Related Health Problems, Ninth Edition (ICD)-9 and, Tenth Edition 10 codes)	Age, sex
Schroeder et al., Predicting 5-year risk of RRT in stage 3 or 4 CKD: Development and external validation [56]	2017 (Jan 2002 –Dec 2013)	Kaiser Permanente Northwest, USA (n = 22,460)	Retrospective cohort	Risk score for predicting the 5-year KRT risk for patients in stage 3 and 4 CKD.	A cox regression model using statistical methods described by Harrell and Steyerberg and endorsed by the Prognosis Research Strategy (PROGRESS) Group (26–28) and outlined in the TRIPOD guidelines. To avoid over-fitting the model, it required 20 KRT events per degree of freedom.	eGFR, hypertension, diabetes, and anaemia, proteinuria/albuminuria, body mass index (BMI), anti-hypertensive medication use, and prescription nonsteroidal anti-inflammatory drugs [NSAID] use. ICD-9 codes, counting complications such as: retinopathy, nephropathy, neuropathy, cerebrovascular disease, cardiovascular disease, peripheral vascular disease, and metabolic complications such as diabetic ketoacidosis.	Age, sex
Sun et al., Development and validation of a predictive model for end-stage renal disease risk in patients with diabetic nephropathy confirmed by renal biopsy [57]	2020 (Feb 2012 –Dec 2018)	First Affiliated Hospital of Zhengzhou, China. (n = 968)	Retrospective	Primary outcome was a fatal or nonfatal ESKD event (peritoneal dialysis or haemodialysis for ESKD, kidney transplantation, or death due to chronic kidney failure or ESKD). ESKD was defined as 1) death due to diabetes with kidney manifestations or kidney failure; 2) hospitalization due to nonfatal kidney failure; and 3) an estimated GFR <15 mL/min/1.73 m2 (National Kidney Foundation, 2002)	Multivariable logistic regression to identify baseline predictors for model development.	History of DM and HTN; laboratory parameters, including pathological grade (Class I, II a, II b, III, and IV represented as 1, 2, 3, 4, and 5 respectively), haemoglobin (Hb) levels, ALB levels, haemoglobin A1c (HbA1c) levels, blood urea nitrogen (BUN) levels, SCr levels, uric acid (UA) levels, cystatin C (CysC) levels, the estimated glomerular filtration rate (eGFR), 24-h urine protein levels, point total protein (TCr) levels, UACR, total cholesterol levels, triglyceride (TG) levels, HDL levels, LDL levels, serum lipid (HDL/total cholesterol ratio) levels; and inflammatory indicators such as PCT, ESR and CRP, creatine kinase isoenzyme (CKmb), B-type natriuretic peptide, and renin-angiotensin system blocker use.	Age, sex
Tangri et al., A Dynamic Predictive Model for Progression of CKD [58]	2016 (Apr 2001—Dec 2009)	Outpatient CKD clinic of Sunnybrook Hospital in Toronto, Canada (n = 3004)	Prospective	Treated kidney failure, defined by initiation of dialysis therapy or kidney transplantation.	Cox proportional hazards models for time to kidney failure	Urinary albumin-creatinine ratio at baseline, eGFR, serum albumin, phosphorus, calcium, and bicarbonate values as time-dependent predictors.	Age, sex
Tangri et al, A predictive model for progression of chronic kidney disease to kidney failure [59]	2011 (Apr 2001—Dec 2008)	Sunnybrook Hospital, Canada (n = 3449 and n = 4942)	Prospective	Risk categories (low, intermediate, high) of kidney failure at 1, 4, and 5 years—defined as initiation of dialysis or kidney transplantation and censored for mortality before kidney failure. Outcomes were ascertained by reviewing clinic records as well as through a matching algorithm with the Toronto Regional Dialysis Registry. Outcomes such as dialysis, death, and transplantation are all captured in the database, which matches all kidney failure outcomes with provincial and national registry	Developed sequentially using Cox proportional hazards regression methods.	Demographic variables, including; physical examination variables, including blood pressure and weight; comorbid conditions, including diabetes, hypertension, and aetiology of kidney disease; and laboratory variables from serum and urine collected at the initial nephrology visit. All predictor variables were obtained at baseline from the nephrology clinic EHR in the development data set	Age and sex
Xie et al., Risk prediction to inform surveillance of chronic kidney disease in the US Healthcare Safety Net: a cohort study [60]	2016 (1996–2009)	Western United States (n = 28,779)	Retrospective	Risk of progression to ESKD (at years 1,3,5 and 7) and death, defined as having a first service date for maintenance dialysis or kidney transplantation.	Linkage to United States Renal Data System (USRDS). Calculated unadjusted incidence rates of ESKD for the full cohort, and for clinical subgroups defined by diabetes mellitus, hypertension, chronic viral diseases (HBV, HCV and/or HIV) and severe CKD (<30 mL/min/1.73m2). We focused on these four subgroups because they represent common conditions frequently targeted by our Chronic Disease Management programs. Tested three proportional hazards regression models to predict progression to ESKD in each subgroup.	eGFR, dipstick proteinuria.	Age, sex, race
Xu et al., An easy-to-operate web-based calculator for predicting the progression of chronic kidney disease [61]	2021 (Oct 2010 –Dec 2011)	Tokyo, Japan (n = 1,045)	Retrospective	1-, 2-, and 3-year progression-free survival	Univariate and multiple Cox proportional hazard models	Aetiology (diabetes, nephrosclerosis, and Glomerulonephritis), haemoglobin level, creatinine level, proteinuria, and urinary protein/creatinine ratio	Age, sex
Supervised technique–other non-ML
Diggle et al., Real-time monitoring of progression towards renal failure in primary care patients [62]	2014 (Mar 1997—Mar 2007)	Salford Royal Hospital Foundation Trust (SRFT), Greater Manchester, UK (n = 22,910)	Retrospective	The predictive probability that they meet the clinical guideline for referral to secondary care. A person who is losing kidney function at a relative rate of at least 5% per year	The time-course of a person’s underlying kidney function through a combination of explanatory variables, a random intercept and a continuous-time, non-stationary stochastic process.	eGFR, co-morbidities, and other baseline information.	Age, sex
Furlano et al., Autosomal Dominant Polycystic Kidney Disease: Clinical Assessment of Rapid Progression [63]	2018 (Jan 2016 –Jun 2017)	Outpatient clinic in Spain (n = 305)	Retrospective	Rapid progression of disease according to their algorithm, including ultrasound, MRI measurements of kidney volume plus genetic testing historical eGFR.	ERA-EDTA WGIKD/ERBP algorithm (European Renal Association-European Dialysis and Transplant Association (ERA—EDTA) Working Groups of Inherited Kidney Disorders and European Renal Best Practice (WGIKD/ERBP),	Historical eGFR decline, historical TKV growth, age and height adjusted TKV, kidney length, PROPKD score,	Age, sex, family history
Lennartz et al., External Validation of the Kidney Failure Risk Equation and Re-Calibration with Addition of Ultrasound Parameters [64]	2016 (CARE FOR HOMe study: 2008–2012—over 6 years & Hannover cohort: 1995–1999	Saarland University hospital, Germany (n = 444)	Prospective	Risk of ESKD at 3 years following recruitment to validate KFRE	KFRE	eGFR (per 5 ml/min per 1.73 m2, according to the MDRD formula), and urine albumin-to-creatinine ratio (ACR). eGFR and ACR were assessed as reported earlier. The KFRE prediction model formula with hazard ratios.	Age (per 10 years), sex
Nastasa et al., Risk prediction for death and end-stage renal disease does not parallel real-life trajectory of older patients with advanced chronic kidney disease-a Romanian center experience [65]	2020 (Oct 2016—Oct 2018)	Romanian Outpatient Nephrology Department (n = 958)	Retrospective	Bansal score and KFRE give an estimate of mortality and progression to ESKD over five years	Individual risk for mortality was predicted using Bansal score, a nine-variable equation model developed in a US cohort of 828 participants aged ≥65 years with an eGFR For estimating the risk for progression to ESKD at 5 years, we used the 4-variable KFRE, according to the algorithm proposed by the ERBP guideline.	eGFR, clinical and biochemical variables	A set of demographic variables, not specific.
Zachasrias et al., A Novel Metabolic Signature To Predict the Requirement of Dialysis or Renal Transplantation in Patients with Chronic Kidney Disease [66]	2019 (2010 –ongoing)	German Chronic Kidney Disease (GCKD) study (n = 4640)	Prospective	The Tangri score	Three proportional hazards models	eGFR, UACR, 24 NMR features (proton nuclear magnetic resonance (NMR) spectroscopy of blood plasma), creatinine, high-density lipoprotein, valine, acetyl groups of glycoproteins, and Ca2+-EDTA carried the highest weights.	Age, sex.
Supervised technique—ML
Cheng et al., Applying the Temporal Abstraction Technique to the Prediction of Chronic Kidney Disease Progression [9]	2017 (Jan 2004 –Dec 2013)	Taiwan (n = 2066)	Retrospective	Predicting stage 4 CKD eGFR level decreasing to less than 15 ml/min/1.73 m2 (ESKD) 6 months after collecting their final laboratory test information by evaluating time-related features	Several common supervised learning techniques, including C4.5, CART, and SVM.	TA-related variables (Temporal abstraction related variables), diabetes, blood pressure, drinking, smoking, heart disease, Variables exerting the greatest impact are consistent with those reported in previous studies, indicating that kidney function, BP, and blood haematocrit, were all vital indicators.	Age, sex. (Sex was the most critical factor affecting the deterioration of CKD among the first 25 variables that exerted the greatest impact.)
Makino et al., Artificial intelligence predicts the progression of diabetic kidney disease using big data machine learning [67]	2019 (2005–2016)	Fujita Health University Hospital, Japan (n = 64,059)	Retrospective	Progression of type 2 diabetic kidney disease after 180 days (6 months)	Processing natural language and longitudinal data with big data machine learning. Applied logistic regression using the Python code with scikit-learn library for model solving. Among many machine learning packages including R, SPSS, Matlab, SAS, Weka and other, scikit-learn was chosen due to feature extraction processes written in Python. Due to the large number of explanation variables, L2-regularisation was used to avoid overfitting.	36 features, where 12 sources (e.g., Urine protein, albuminuria and eGFR) were selectively chosen by extraction from known literature and 3 types of values (mean, latest, SD).	Past history of diseases.
Zhao et al., Predicting outcomes of chronic kidney disease from EMR data based on Random Forest Regression [68]	2019 (2009–2017)	United States, Sioux Falls (n = 120,495)	Retrospective	The estimation of future eGFR value from the past eGFR values adjusted by clinical covariates, at year 1, 2 and 3.	Random Forest regression	eGFR, age, gender, body mass index (BMI), obesity, hypertension, and diabetes, which achieved a mean coefficient of determination of 0.95.	Age, sex.
Zhou et al., Use of disease embedding technique to predict the risk of progression to end-stage renal disease [69]	2020 (Jan 2003 –Dec 2011)	California, United States (n = 35,844,800)	Retrospective	Progression of CKD to ESKD	Disease2disease (D2D)	Word2vec, comorbidities, ICD-9 or ICD-10 coding, five lab parameters: bicarbonate, calcium, protein, PTH, and urine protein/creatinine ratio, 25-OH vitamin, haematocrit, potassium, sodium and triglyceride.	Age, sex.
Supervised and unsupervised techniques
Dovgan et al., Using machine learning models to predict the initiation of renal replacement therapy among chronic kidney disease patients [70]	2020 (1998–2011)	Taiwan’s national health insurance research database (NHIRD) (n = 23,948),	Retrospective	The onset of KRT at the time of CKD diagnosis—at 3, 6, and 12 months	Evaluated 10 ML algorithms that are implemented in the Python packages Scikit-learn and XGBboost: Decision Tree, Bagging Decision Trees, Random Forest, XGBoost, SVMs, Simple Gradient Descendent, Nearest Neighbours, Gaussian Naive Bayes, Logistic Regression, and Neural Network. Logistic Regression in combination with time features and data balancing, and without feature selection, filtering, or dimensionality reduction.	eGFR, albumin, haemoglobin, phosphorus, potassium, Correlations between diagnoses; diagnoses that are related to CKD, i.e., diabetes, HTN, hypertensive heart disease, glomerulonephritis, polycystic kidney, renal calculus, vesicoureteral reflux, kidney infections.	Age, sex,
Norouzi et al., Predicting Renal Failure Progression in Chronic Kidney Disease Using Integrated Intelligent Fuzzy Expert System [71]	2016 (Oct 2002—Oct 2011)	Clinic of Nephrology, Imam Khomeini Hospital (Tehran, Iran) (n = 465)	Retrospective	Either GFR value less than 15 mL/kg/min/1.73 m2, start of KRT or patient death, at 6, 12, or 18 months.	Adaptive neuro-fuzzy inference system (ANFIS)	Weight, underlying diseases, diastolic blood pressure, creatinine, calcium, phosphorus, uric acid, and GFR.	Age, sex,

Predicted outcomes

It was generally observed that regardless of the model used, the predicted outcome included measurement of risk towards ESKD which were defined as [41, 43, 44, 48–50, 53, 56–60, 64, 69, 70, 72]:

when the eGFR value is <15 mL/kg/min/1.73 m2 and / or

the initiation of kidney replacement therapies (KRTs) such as dialysis or kidney transplantation.

This risk of ESKD was generally predicted for specified time intervals of 1, 2, 3, and 5 years for supervised models, and shorter time intervals of 3, 6, 12 and 18 months for unsupervised models. There were very few studies that had predicted outcomes such as progression from an earlier stage to a more severe stage of CKD, for example from stage 1 to stages 3 or 4 [54, 56], and other predicted endpoints of stated percentage decline in eGFR levels [9, 41]. Depending on the quality of the available dataset [59, 60], the predicted outcome could also be combined with other variables such death, comorbidities, the type of dialysis and the time of diagnosis [46, 59, 68]. Some examples of outcomes that integrated these additional variables include, predicting the chances of future KRT at the time of CKD diagnosis [70]; a ≥ 50% decline in the eGFR from baseline [50] or an eGFR decline ≥30% from baseline [41]; the 5-year risk of KRT in CKD stage 3 and 4 [56]; the mortality and progression to ESKD over five years [65].

Type of predictive model

Fig 1 shows that 31 studies implemented supervised models, and only 2 studies included unsupervised models with 1 of these 2 studies being a comparison study between supervised and unsupervised models. Of the studies that used supervised models, 21 studies implemented cox proportional hazards regression [41-61]. Seven studies used machine learning (ML) methods [9, 67–71], and one compared the performance among a number of ML techniques [70]. One study developed a model using Random Forest regression [68], and another study implemented a disease2disease model by first learning the International Classification of Diseases and then clustering the data into groups by considering the variables within the dataset [69]. A multistate marginal structural model (MS-MSM) was also developed in one study that considers an estimated effect of time-dependent variables towards the predicted outcome [72]. Other ML algorithms that were tested include, neural networks, decision tree, random forest, XGBoost, Gaussian Naïve Bayes and logistic regression [57, 70, 71].

Three studies [49, 64, 65] performed an evaluation of the Kidney Failure Risk Equation (KFRE), and three other studies developed their own unique scoring algorithm that predicted ESKD [55, 63, 66].

Significant variables in the model

Common predictors used in studies included age, sex, eGFR, urinary albumin to creatinine ratio (ACR), serum creatinine (SCr), diabetes, cardiovascular disease, body mass index (BMI), and high blood pressure. Each predictive model was unique and incorporated different combinations of variables, and slightly different definitions of variables, such as high blood pressure. A recent paper by Xu et al. [61] published in 2021 highlighted that there are currently no robust biomarkers to predict progressive CKD, but rather relied on multiple longitudinal kidney measurements, such as eGFR and proteinuria.

The eGFR formula was also not consistent across studies, 13 studies used the CKD Epidemiology Collaboration (CKD-EPI) equation [9, 41, 42, 44, 46, 51, 54–56, 58, 63, 65, 66] and 9 studies used the Modification of Diet in Renal Disease (MDRD) equation [43, 45, 47, 49, 53, 60, 62, 64, 71]. Two studies used unique equations customised for their specific cohort [48, 69]. There were also 9 studies that did not specify the formula that they used to calculate the eGFR.

Study population

The smallest study [50] had 43 participants and the largest study included over 300,000 patient records [54]. Included study populations were from the United States [45, 52–56, 60, 68, 69, 72], Canada [41, 46, 58, 59], Taiwan [9, 42, 70], Germany [64, 66], Japan [48, 50, 61, 67], France [43], Croatia [44], Korea [49, 51], United Kingdom [62], Iran [71], Romania [65], Spain [63], Netherlands [47] and China [57].

These studies that investigated on CKD progression used data records that were sourced from all levels of healthcare. Data records ranged from single medical facilities at a local level [58], to tertiary hospitals [50, 58, 59, 62, 67, 71], and to databases that were linked nationwide [42, 60, 69, 72]. The populations were also selected based on a particular comorbidity of interest, for example, polycystic kidney disease [41, 43, 63], ANCA associated vasculitis (AAV) [44], diabetes [46] or other cardiovascular conditions [53, 56, 67].

Validation assessment

30 papers reported on the performance of their respective predictive models (regardless of the type of prediction model used) with 25 studies assessing the performance of their model by measuring the Area Under the Curve (AUC) [9, 41, 43, 45–54, 56–62, 64, 66–68, 70]. Both supervised and unsupervised techniques were shown to have used AUC to validate their prediction model, each having a relatively high value that indicated that their model was reliable in predicting their defined outcome. Relative performance of the prediction model was indicated using a variety of methods including sensitivity analysis, specificity, discrimination index and a goodness of fit analysis. However, only three studies were externally validated on an external population dataset [46, 56, 64].

Four studies explored the KFRE [41, 49, 64, 65] as a variable to try and improve the performance of their prediction model. Only one study reported using the F-score with confidence intervals [67], and there were a range of alternative measures that were used including the mean square error, mean absolute error, normalised mean square error, positive predictive values, negative predicted values, Harrell bootstrap resampling method, D-statistic and various confusion matrices [56, 68, 71].

Missing data & imbalanced data

The most common limitation reported was missing or limited data, potentially due to the quality and availability of the data collected. Studies tried to overcome this issue by filling in the missing data using imputation techniques and internal validation techniques to help justify the dataset [42, 60]. There were also studies that reported having unbalanced data and outlined the methods applied to re-balance the data before initiating model development [9, 42, 45, 62, 70]. Knowing these limitations, studies recognised that their prediction models would only be applicable to their own given study population and would require external validation to allow generalisation of their model to other populations [41, 44–46, 49, 50, 53, 56, 59, 61, 64–66, 69, 73, 74].

Discussion

The arrival of big data and data science techniques have supported better analytics using data from a variety of sources. However, many healthcare systems around the world are yet to fully utilise healthcare data for research purposes. Many of the data challenges within health relate to missing data, inconsistencies in recorded data and privacy concerns for linking data across organisations [75]. Despite these challenges, the application of health data is critical to support clinical decision making [31, 76, 77].

The success of disease management for conditions like CKD is dependent upon a clinician’s ability to identify the risk of disease progression and poor outcomes. By utilising big data analytics, healthcare professionals may be able to predict disease progression in a timely manner, allowing the potential for better treatment for patients and reduced health costs.

Our review identified studies that had developed models to predict patient outcomes for CKD that measured the risk of progression towards ESKD over given time intervals. There was no single gold standard model identified, with each study producing its own unique prediction model, dependent on cohort’s characteristics and quality of the available data. While Cox regression modelling was the predominant method; the burgeoning research on the use of ML techniques to improve the prediction of CKD progressing towards ESKD [23]. However, the decision to use a particular modelling technique should depend on finding the most suitable model based on the type of data available, size and dimensionality [19].

The application of both traditional and ML techniques have been explored as a way of determining the most significant variables or features for inclusion in the model [56, 70]. Studies that combined the use of both regression and ML techniques, first identified significant variables through regression prior to their inclusion into the development of a risk prediction model [56, 70, 78]. However, the practicality of determining significant features can be highly dependent on the availability and the quality of data. It is clear that the performance of a model is degraded if there is a lack of significant variables or if it includes irrelevant features [78-80]. Therefore, it is also recommended that future studies attempt to obtain whole population datasets that can help reduce the risk of missing data within the dataset and overcome the limitation of small study populations that are not generalisable to whole populations.

The study by Norouzi et al. demonstrated that an unsupervised adaptive neuro-fuzzy inference system (ANFIS), a type of neural network, was able to accurately predict GFR at sequential 6, 12 and 18-month intervals [71]. Other supervised non-ML models such as the KFRE and the ERBP algorithms, also produced results with high accuracy [63-65].

The comparison study by Dovgan et al. also demonstrated that features which correlated with a time approach produced the best results [70]. While the study did not include pathology results when developing their model, it produced the highest AUC via logistic regression, with XGBoost and Simple Gradient Descendent as a close second.

The MS-MSMs developed by Stephens-Shields et al. [72] developed a model that accounts for varying windows of time associated with different states while describing the effect of different exposures have on between states or endpoints. This is particularly applicable to the slow progression of CKD patients who enter the health system at different points in time and at various stages of the disease. In addition, each patient will have acquired different comorbidities and medical histories at different stages of their life.

Since the application of unsupervised and ML models are still in their exploratory stages, further research is required to investigate how these less explainable models manipulate very large and complex datasets that contain multi-dimensional and continuous variables [37, 78] and reflecting their application to predict CKD progression.

The review revealed a lack of consistent reporting of the methodology used for development and validation of prediction models. This often led to under reporting of model development, which hinders the ability of researchers to do a true comparison and externally validate their predictive models against existing models. This was emphasised when almost one third of studies reviewed did not report on the eGFR formula used, and is a significant limitation towards the development of this area of research. The development of a standardised reporting statement has yet to be widely implemented among CKD progression research which may be due to its relative novelty in the area of predictive modelling and statistical research [35].

Few studies explained how they attempted to re-balance their data, and methods differed for each study including log transformations, data resampling techniques, running simulation studies, and applying inversely proportional weights to class frequencies [9, 42, 62, 70]. The predictive models that have been developed are often difficult to implement locally as they lack information that allows clinicians to validate them. Limitations on data linkage within and between health organisations also contribute to the challenge of implementing this research, where siloed datasets are unlikely to be representative of whole populations. It is also recommended that future studies should include clear reporting of model development including any balancing of skewed datasets, steps to validate the model, and a description of how significant variables were chosen, which should theoretically at least include age, sex, eGFR (using a formula that provides reliable estimates for the study population), details on the population’s characteristics, ACR, BMI and time-related variables if available.

A reliable risk prediction model for CKD progression would not only provide clinicians with earlier identification of CKD patients at greatest risk of progression, it would also enhance consultations and help clinicians determine suitable treatment options to improve patient outcomes [81, 82].

Conclusions

Nephrology researchers are working towards producing an effective model to assist the detection of the risk of chronic kidney disease progression. The review highlights that supervised techniques, and more specifically, cox regression is the predominant model that is used to predict the progression of CKD. There were only a small number of studies in the review that used unsupervised and ML models, with the limited numbers making it very difficult to perform a comparison between these models. A more consistent and reproducible approach is required for future studies looking to develop risk prediction models for CKD progression. This would improve international collaborations and build upon the existing research to overcome the challenges to improve the effectiveness and reliability of these prediction models. Subsequently, this would also translate into enhanced health system planning, allocation of resources and improved health outcomes for CKD patients.

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist.

(DOCX)

Click here for additional data file.

Detailed search strategy per database.

Concept grids and search histories for Medline, EMBASE, CINAHL and Scopus.

(DOCX)

Click here for additional data file.

30 May 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Pierre Delanaye

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating the following in the Acknowledgments Section of your manuscript:

"This project is supported by the DHCRC and is part of a larger 4-year collaborative partnership with Curtin University, La Trobe University, WA Department of Health, WA Country Health Service, WA Primary Health Alliance, and the DHCRC. All authors declare no conflict of interest and received no additional funding towards this manuscript. The DHCRC is funded under the Commonwealth's Cooperative Research Centres (CRC) Program (DHCRC-0073). All authors also consent for publication and the supplementary data that supports the findings of this review will be available upon submission and publication."

We note that you have provided funding information. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

"This research is Digital Health CRC Limited (DHCRC) and is part of a larger 4-year collaborative partnership between Curtin University, La Trobe University, WA Department of Health, WA Country Health Services, WA Primary Health Alliance, and the DHCRC. The DHCRC is funded under the Commonwealth's Cooperative Research Centres (CRC) Program, project ID DHCRC-0073. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Additional Editor Comments:

The authors should better justify the way some studies have been included or not. They should also better define outcomes and variables included in the models.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for giving me the opportunity to review the manuscript by David Lim et al entitled: “Prediction models used in the progression of chronic kidney disease: a scoping review”.

In this manuscript, the authors propose a review of the different models that have been proposed in order to predict the risk of progression of CKD. This subject is of great importance, as the authors explain it in their introduction, from a clinician point of view but also from a public health point of view.

The manuscript is well written and easy to read. The review was built following the PRISMA statement, which is a pledge of quality.

A bullet point from this review is the lack of consistency regarding the definitions of the outcomes, the definitions of the variables included in the models

I only have few comments:

- I find it useful to have described the four categories of modelling techniques in the introduction.

- I would have liked to get more insight into the explanatory variables included in the different models, separating the modifiable variables from the non-modifiable variables.

- Could the authors add some details in the text about the different source populations. Some studies tried to explore the risk of progression of CKD in the general population, whereas in some others studies, it was in very specific populations: ANCA vasculitis, ADPKD…

- The last proposal of the results part: “Additionally, a more precise estimation of …. versus end-of-life care” is a different subject, so I would propose to remove this sentence and to insist on the lines 19 to 23 : “Future studies should… variables if available” which is the really important point nephrologists should focus on, based on the results of this review.

- Thank you for table S2!

- Given the aim of the article, to review the models used in the progression of CKD, could the authors add in the discussion part some commentaries about the best adapted models for further works, according to the variables used or the population studied?

Reviewer #2: Dear Editorial Board

David KE Lim et al present a scoping review about Prediction modelling on CKD progression.

A major comments is about context method and selection of article.

These is already some recent review publication about this topic. The authors needs to refer to these articles in introduction and justify why they don't find the same articles. If they want to study prediction model of CKD progression ESRD must be an outcome in research equation.

An important score of progression is the prediction model of Grams et al 2018 but was not cited in references. 2 authors did review on this topic and found several article not found in this scoping review ( Ramspeck CL & Prouvot J).

Grams ME, Sang Y, Ballew SH, Carrero JJ, Djurdjev O, Heerspink HJL, Ho K, Ito S, Marks A,

Naimark D, Nash DM, Navaneethan SD, Sarnak M, Stengel B, Visseren FLJ, Wang AY-M, Köttgen

A, Levey AS, Woodward M, Eckardt K-U, Hemmelgarn B, Coresh J: Predicting timing of clinical

outcomes in patients with chronic kidney disease and severely decreased glomerular filtration

rate. Kidney International 93: 1442–1451, 2018

Ramspek CL, de Jong Y, Dekker FW, van Diepen M: Towards the best kidney failure prediction

tool: a systematic review and selection aid. Nephrol Dial Transplant [Internet] Available from:

https://academic.oup.com/ndt/advance-article/doi/10.1093/ndt/gfz018/5369192

Low performance of prognostic tools for predicting dialysis in elderly people with advanced CKD.

Prouvot J, Pambrun E, Couchoud C, Vigneau C, Roche S, Allot V, Potier J, Francois M, Babici D, Prelipcean C, Moranne O; PSPA investigators. J Nephrol. 2021 Aug;34(4):1201-1213. doi: 10.1007/s40620-020-00919-6.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Antoine Lanot

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

28 Jun 2022

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?i d=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response to reviewer:

Please find the revised manuscript with all the necessary changes made to meet PLOS ONE’s style requirements, including file naming rules.

________________________________________

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

Response to reviewer:

We have now ensured that the grant award number matches with the funding information section, it should read that the Award Number: DHCRC-0073, by the Digital Health CRC Ltd.

________________________________________

3. Thank you for stating the following in the Acknowledgments Section of your manuscript:

"This project is supported by the DHCRC and is part of a larger 4-year collaborative partnership with Curtin University, La Trobe University, WA Department of Health, WA Country Health Service, WA Primary Health Alliance, and the DHCRC. All authors declare no conflict of interest and received no additional funding towards this manuscript. The DHCRC is funded under the Commonwealth's Cooperative Research Centres (CRC) Program (DHCRC-0073). All authors also consent for publication and the supplementary data that supports the findings of this review will be available upon submission and publication."

We note that you have provided funding information. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

"This research is Digital Health CRC Limited (DHCRC) and is part of a larger 4-year collaborative partnership between Curtin University, La Trobe University, WA Department of Health, WA Country Health Services, WA Primary Health Alliance, and the DHCRC. The DHCRC is funded under the Commonwealth's Cooperative Research Centres (CRC) Program, project ID DHCRC-0073. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

Response to reviewer:

We have removed funding information from the Acknowledgements section in the manuscript, and it now reads:

“This project is supported by the DHCRC and is part of a larger 4-year collaborative partnership with Curtin University, La Trobe University, WA Department of Health, WA Country Health Service, WA Primary Health Alliance, and the DHCRC. All authors declare no conflict of interest and received no additional funding towards this manuscript. All authors also consent for publication and the supplementary data that supports the findings of this review will be available upon submission and publication.”

________________________________________

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Response to reviewer:

We have added the “Supporting information captions” section on the last page of the manuscript:

“Supporting information captions

S1 Appendix. Detailed search strategy per database. The concept grid and search history for Medline, EMBASE, CINAHL and Scopus.”

Tables 1, 2, and Fig 1 have all been added appropriately into the manuscript body, they are no longer “supporting information”.

________________________________________

Additional Editor Comments:

The authors should better justify the way some studies have been included or not. They should also better define outcomes and variables included in the models.

Response to reviewer:

Thanks for the comment. This scoping review was done in context of a larger project that looks at improving chronic kidney disease outcomes through the use of linked routine pathology records and hospital records. With that context in mind, one of the project’s future objective is to explore how other studies have tried to understand the progression of CKD. The four main areas that would cover this future objective are as described in the concept grid found in S1 Appendix:

1) chronic kidney disease; and

2) Disease progression; and

3) different statistical / machine modelling techniques that were developed/used to understand the progression of CKD; and

4) A range of outcomes that are commonly associated with ESRD, with ESRD included as one of several other outcomes of the search strategy.

Additional clarity has been added to the method section:

“Methods

A scoping review was selected as it allows identification and mapping of existing evidence and to investigate and determine the knowledge gaps surrounding the topic [39]. This method is suitable for examining emerging evidence across a broad field of study and was guided by the PRISMA extension for Scoping Reviews (PRISMA-ScR), following a standardised approach to search, screen, and report articles.

Data Sources and Searches

This scoping review was performed in the context of a larger study that investigates improving chronic kidney disease outcomes using linked routine records. With this context in mind, an initial concept grid was developed to address the objectives of the scoping review, together with the subsequent search histories that can be found in S1 Appendix.”

Outcomes from each study have been included in Table 2’s column “Predicted Outcome(s)”, with the types of variables distinguished as modifiable and non-modifiable.

________________________________________

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Response to reviewer: Not required.

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

Response to reviewer: Not required.

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Response to reviewer: Not required.

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Response to reviewer: Not required.

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for giving me the opportunity to review the manuscript by David Lim et al entitled: “Prediction models used in the progression of chronic kidney disease: a scoping review”.

In this manuscript, the authors propose a review of the different models that have been proposed in order to predict the risk of progression of CKD. This subject is of great importance, as the authors explain it in their introduction, from a clinician point of view but also from a public health point of view.

The manuscript is well written and easy to read. The review was built following the PRISMA statement, which is a pledge of quality.

A bullet point from this review is the lack of consistency regarding the definitions of the outcomes, the definitions of the variables included in the models

I only have few comments:

*** I find it useful to have described the four categories of modelling techniques in the introduction.

Response to reviewer:

Thanks for letting us know that you found it useful, and have provided more clarity to have the description of the categories of the modelling techniques in the introduction under the subheading “Classification of predictive models”:

“Classification of predictive models

Predictive modelling techniques can be generally classified into four broad categories; supervised, unsupervised, semi-supervised and reinforcement learning; with supervised and unsupervised being the most commonly applied in the medical field. This was also reflected in this scoping review where only supervised and unsupervised techniques were found in the studies that were assessed for full-text and will be discussed later in later sections.”

We have also provided more clarity when describing these categories of modelling techniques within Results section by adding a subheading “Type of predictive model”:

“Type of predictive model

Fig 1 shows that 31 studies implemented supervised models, and only 2 studies included unsupervised models with 1 of these 2 studies being a comparison study between supervised and unsupervised models….”

________________________________________

*** I would have liked to get more insight into the explanatory variables included in the different models, separating the modifiable variables from the non-modifiable variables.

Response to reviewer:

We have now included in Table 2 an extra split in the column for “Predictors in the model”, this has been done by further dividing this column into two columns namely “modifiable” and “non-modifiable”.

________________________________________

*** Could the authors add some details in the text about the different source populations. Some studies tried to explore the risk of progression of CKD in the general population, whereas in some others studies, it was in very specific populations: ANCA vasculitis, ADPKD…

Response to reviewer:

We have included additional clarification about the different population sources within the results section, under “Study population”, which now reads:

“Study population

The smallest study had 43 participants and the largest study included over 300,000 patient records. Included study populations were from the United States, Canada, Taiwan, Germany, Japan, France, Croatia, Korea, United Kingdom, Iran, Romania, Spain, Netherlands and China.

These studies that investigated on CKD progression used data records that were sourced from all levels of healthcare. Data records ranged from single medical facilities at a local level, to tertiary hospitals, and to databases that were linked nationwide. The populations were also selected based on a particular comorbidity of interest, for example, polycystic kidney disease, ANCA associated vasculitis (AAV), diabetes or other cardiovascular conditions.”

________________________________________

*** The last proposal of the results part: “Additionally, a more precise estimation of …. versus end-of-life care” is a different subject, so I would propose to remove this sentence and to insist on the lines 19 to 23 : “Future studies should… variables if available” which is the really important point nephrologists should focus on, based on the results of this review.

Response to reviewer:

Thanks for the suggestion, we have removed the sentence, “Additionally, a more precise estimation of a patient’s risk of death after starting dialysis would also enable clinicians to better assist patients to consider the benefits of treatment versus end-of-life care”.

________________________________________

*** Thank you for table S2!

Response to reviewer: Not required.

________________________________________

*** Given the aim of the article, to review the models used in the progression of CKD, could the authors add in the discussion part some commentaries about the best adapted models for further works, according to the variables used or the population studied?

Response to reviewer:

We have added in more clarity within the discussion section which highlights the importance of trying to capture the whole population with a standardised reporting method, which improves the modelling process:

“The application of both traditional and ML techniques have been explored as a way of determining the most significant variables or features for inclusion in the model [56, 70]. Studies that combined the use of both regression and ML techniques, first identified significant variables through regression prior to their inclusion into the development of a risk prediction model [56, 70, 78]. However, the practicality of determining significant features can be highly dependent on the availability and the quality of data. It is clear that the performance of a model is degraded if there is a lack of significant variables or if it includes irrelevant features [78-80]. Therefore, it is also recommended that future studies attempt to obtain whole population datasets that can help reduce the risk of missing data within the dataset and overcome the limitation of small study populations that are not generalisable to whole populations…

… Limitations on data linkage within and between health organisations also contribute to the challenge of implementing this research, where siloed datasets are unlikely to be representative of whole populations. It is also recommended that future studies should include clear reporting of model development including any balancing of skewed datasets, steps to validate the model, and a description of how significant variables were chosen, which should theoretically at least include age, sex, eGFR (using a formula that provides reliable estimates for the study population), details on the population’s characteristics, ACR, BMI and time-related variables if available.”

Reviewer #2: Dear Editorial Board

David KE Lim et al present a scoping review about Prediction modelling on CKD progression.

A major comments is about context method and selection of article.

These is already some recent review publication about this topic. The authors needs to refer to these articles in introduction and justify why they don't find the same articles. If they want to study prediction model of CKD progression ESRD must be an outcome in research equation.

An important score of progression is the prediction model of Grams et al 2018 but was not cited in references. 2 authors did review on this topic and found several article not found in this scoping review ( Ramspeck CL & Prouvot J).

Grams ME, Sang Y, Ballew SH, Carrero JJ, Djurdjev O, Heerspink HJL, Ho K, Ito S, Marks A,

Naimark D, Nash DM, Navaneethan SD, Sarnak M, Stengel B, Visseren FLJ, Wang AY-M, Köttgen

A, Levey AS, Woodward M, Eckardt K-U, Hemmelgarn B, Coresh J: Predicting timing of clinical

outcomes in patients with chronic kidney disease and severely decreased glomerular filtration

rate. Kidney International 93: 1442–1451, 2018 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5967981/)

Ramspek CL, de Jong Y, Dekker FW, van Diepen M: Towards the best kidney failure prediction

tool: a systematic review and selection aid. Nephrol Dial Transplant [Internet] Available from:

https://academic.oup.com/ndt/advance-article/doi/10.1093/ndt/gfz018/5369192

Low performance of prognostic tools for predicting dialysis in elderly people with advanced CKD.

Prouvot J, Pambrun E, Couchoud C, Vigneau C, Roche S, Allot V, Potier J, Francois M, Babici D, Prelipcean C, Moranne O; PSPA investigators. J Nephrol. 2021 Aug;34(4):1201-1213. doi: 10.1007/s40620-020-00919-6.

Response to reviewer:

Thanks for the comment. This scoping review was done in context of a larger project that looks at improving chronic kidney disease outcomes through the use of linked routine pathology records and hospital records. With that context in mind, one of the project’s future objective is to explore how other studies have tried to understand the progression of CKD. The four main areas that would cover this future objective are as described in the concept grid found in S1 Appendix, through the inclusion of keywords and subject terms with the help of the University’s Health Sciences faculty librarian:

1) chronic kidney disease; and

2) Disease progression; and

3) different statistical / machine modelling techniques that were developed/used to understand the progression of CKD; and

4) A range of outcomes that are commonly associated with ESRD, with ESRD included as one of several other outcomes of the search strategy.

After reviewing the Medline search strategy as described in S1 Appendix, page 2, it was found that the papers by Grams et al. and Prouvot et al., were excluded at step 9 of the search history. This meant that these papers did not include the listed keywords or MeSH terms of modelling techniques found in the concept grid column “Techniques” within their title or abstract.

For the study by Ramspek et al., while this paper included kidney failure and prediction of disease progression, it does not report any specific outcomes in its title or abstract. More specifically, it was excluded at step 12 of the search history, which meant that the paper did not include the listed keywords or MeSH terms of any of the range of outcomes that are commonly associated with ESRD, within its title or abstract.

________________________________________

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Antoine Lanot

Reviewer #2: No

Response to reviewer: Not required.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

5 Jul 2022

Prediction Models Used in the Progression of Chronic Kidney Disease: A Scoping Review

PONE-D-22-10875R1

Dear Dr. Lim,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Pierre Delanaye

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

No further comments.

Reviewers' comments:

15 Jul 2022

PONE-D-22-10875R1

Prediction Models Used in the Progression of Chronic Kidney Disease: A Scoping Review

Dear Dr. Lim:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Professor Pierre Delanaye

Academic Editor

PLOS ONE