Development and external validation of an acute kidney injury risk score for use in the general population.

BACKGROUND: Improving recognition of patients at increased risk of acute kidney injury (AKI) in the community may facilitate earlier detection and implementation of proactive prevention measures that mitigate the impact of AKI. The aim of this study was to develop and externally validate a practical risk score to predict the risk of AKI in either hospital or community settings using routinely collected data.
METHODS: Routinely collected linked datasets from Tayside, Scotland, were used to develop the risk score and datasets from Kent in the UK and Alberta in Canada were used to externally validate it. AKI was defined using the Kidney Disease: Improving Global Outcomes serum creatinine-based criteria. Multivariable logistic regression analysis was performed with occurrence of AKI within 1 year as the dependent variable. Model performance was determined by assessing discrimination (C-statistic) and calibration.
RESULTS: The risk score was developed in 273 450 patients from the Tayside region of Scotland and externally validated into two populations: 218 091 individuals from Kent, UK and 1 173 607 individuals from Alberta, Canada. Four variables were independent predictors for AKI by logistic regression: older age, lower baseline estimated glomerular filtration rate, diabetes and heart failure. A risk score including these four variables had good predictive performance, with a C-statistic of 0.80 [95% confidence interval (CI) 0.80-0.81] in the development cohort and 0.71 (95% CI 0.70-0.72) in the Kent, UK external validation cohort and 0.76 (95% CI 0.75-0.76) in the Canadian validation cohort.
CONCLUSION: We have devised and externally validated a simple risk score from routinely collected data that can aid both primary and secondary care physicians in identifying patients at high risk of AKI.

INTRODUCTION

Acute kidney injury (AKI) affects ∼15% of all hospitalized patients in developed countries, with a significant proportion originating in the community [1, 2]. Even small changes in kidney function are associated with adverse outcomes, including increased mortality in patients with (Stage 1) AKI, defined as an increase in serum creatinine of at least 26 μmol/L or 1.5 times the baseline [3, 4]. It has been suggested that up to 30% of AKI episodes may be preventable [5]. Recognition of individuals at risk of AKI is a critical first step in implementing strategies to prevent AKI. Improving recognition of patients at increased risk of AKI in the community may facilitate earlier detection and implementation of proactive prevention measures that mitigate the impact of AKI.

Current validated risk scores for AKI were developed to predict hospital-acquired AKI and most focus on prediction of post-operative AKI [3, 6–9] in selected populations within short time periods during post-operative care. However, a recent population-based study from Scotland showed that 39% of AKI episodes in this geographic region in fact originated in the community, with 23% admitted to hospital and 16% not admitted to hospital [6-11]. Much less is known about how to predict the risk of AKI in the general population, including the risk of community-acquired AKI (CA-AKI), where most existing AKI risk scores were not developed for use [10]. No externally validated risk scores currently exist for predicting AKI in the general population.

The aim of this study was to develop and externally validate a practical risk score that could be used in the general population to predict the risk of AKI, occurring either in hospital or in the community, based on routinely collected demographic, comorbidity and biochemistry data.

MATERIALS AND METHODS

Reporting and methods of the study adhered to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement [11].

Study population and data sources

We used routinely collected, linked datasets from the Tayside region, Scotland, UK to develop the risk score and from Kent (UK) and Alberta, Canada to externally validate the score.

The dataset used for derivation consisted of linked healthcare information on every person ≥18 years of age with at least one creatinine measurement between 1 January 2004 and 31 December 2012. This dataset covered ∼80% of the entire adult population of Tayside (∼330 000 people), an area with limited geographical mobility, a wide range of deprivation and a mix of urban and rural environments. The population of Tayside is >99% white ethnicity. Approximately 20% of the Tayside population were not included in this dataset, as they had not had a creatinine measurement during this time period. Data were collected by the Health Informatics Centre, University of Dundee [13]. The Health Informatics Centre enables anonymized health record linkage from the population of Tayside using a unique identifying Community Health Index number. Data were linked between the following datasets: Scottish Morbidity Record of hospital admissions (SMR01); laboratory results, medicines dispensed by community pharmacies, Scottish Index of Multiple Deprivation, the Scottish Care Initiative–Diabetes Collaboration (SCI-DC), the Scottish Renal Registry and Scottish death registry data held by the Scottish General Records Office. SMR01 provided information on age, sex, postcode and admission and discharge dates. The SCI-DC provided information on diabetes type. This system is used to record all diabetes clinical care in Scotland, thereby capturing >99% of patients with diabetes [14]. Patients receiving chronic dialysis or following renal transplant were identified using the Scottish Renal Registry.

East Kent Hospitals catchment area (∼700 000 people) is a mainly rural area with a large coastal community. The population is mainly of white ethnicity (93.7%) with a significant mix of areas of social deprivation and affluence. Data for the Kent validation cohort were extracted from the data warehouse as part of the Kent Predicted Patient Outcomes Database (KePPOD); laboratory results were extracted from regional laboratory system and patients on haemodialysis were excluded. Comorbidities were extracted from East Kent Hospitals’ patient administration system. The Kent external validation cohort included all residents ≥18 years of age with at least one creatinine measurement between 1 April 2013 and 1 April 2016.

The external validation cohort from Alberta, Canada was formed from the Alberta Kidney Disease Network (AKDN), a population-based repository comprised of provincial administrative health and laboratory data that includes serum creatinine measurements from >1.8 million adults obtained from acute care and outpatient settings for all residents of the province of Alberta (2009 Alberta provincial population 3.5 million) [15]. The Alberta external validation cohort included all Alberta residents ≥18 years of age with at least one creatinine measurement between 1 January 2007 and 31 December 2012.

The AKDN database includes patient-level data from all outpatient and inpatient clinical laboratories as well as the provincial administrative health sources of Alberta Health. Patient comorbidities and procedures were characterized using theInternational Classification of Diseases, 10th revision (ICD-10) diagnosis and Canadian Classification of Heath Intervention procedure codes applied to hospital discharge abstracts and physicians’ claims databases. Patients receiving chronic dialysis or kidney transplant were identified from physician claims and the repositories of the northern and southern Alberta renal programmes as previously described [16].

For external validation of the models in the Alberta external validation cohort, those without end-stage kidney disease who were alive on 1 January 2012 were included in the cohort, with this date used as the index date for calculating the risk score. Comorbidities including diabetes mellitus and heart failure were identified using validated coding approaches based on ICD-10 coding algorithms applied to the Alberta hospital discharge abstract and physicians’ claims databases using a lookback period extending back 6 years prior to the index date [17, 18].

All estimated glomerular filtration rates (eGFRs) were based on creatinine measurements traceable to isotope dilution mass spectrometry for all cohorts.

Ethical statement

Anonymized record linkage was conducted according to the Health Informatics Centre, University of Dundee [13] standard operating procedure. The Tayside Research Ethics Committee does not require submission of individual studies that follow this standard operating procedure which is approved by the local Data Protection Officer (Caldicott Guardian). Ethical approval for the Kent (UK) validation dataset was sought through the KePPOD database ethics (Oxford C 18/SC/0158), IRAS (Integrated Research Application System) project ID 227612. Approval for the Alberta cohort validation study was obtained from the Conjoint Health Research Ethics board of the University of Calgary, Alberta, Canada.

Development of risk score

We started development by exploring a large number of potential predictors in a development cohort described in the Supplementary data, File S1. To derive a simple score that would be generalizable for use at a population level, e.g. in primary care, we selected variables from the development cohort with a P-value <0.05 using binary logistic regression. We then excluded uncommon predictors defined as affecting <1% of the population and those variables not readily available from routinely collected datasets.

Medications were not included in the parsimonious model, as obtaining medication information in an easily analysable format from routine data is challenging and may adversely impact the ability to apply the model in practice. In addition, medications are likely to be a key target for future interventions and so would potentially affect the predictive properties of the model.

Furthermore, previous AKI was not included, as deriving this variable from routine datasets may not be practical. Urological surgery and renal artery stenting were not included despite relatively high odds ratios, because of the small number of affected individuals; such predictors added very little to the predictive power of the score at a population level. Although we included sex in the model, little difference in risk was seen between the sexes, therefore sex was not included in the final score.

We therefore selected age, eGFR category, a previous discharge diagnosis of heart failure using ICD-10 code I50 (Supplementary data, Table S1) and a diagnosis of diabetes mellitus as predictors for a parsimonious model. We then ran the binary logistic regression analysis using only these variables in a forced entry model and derived weighted scores based on the beta values from the model.

Outcome

AKI was defined using the National Health Service (NHS) England algorithm based on the Kidney Disease: Improving Global Outcomes (KDIGO) serum creatinine–based criteria [4, 19]. We distinguish between any degree of AKI (Stages 1–3) or more severe AKI (Stages 2 and 3), regardless of community or hospital-onset of AKI. The change in creatinine from baseline was calculated for each index creatinine measurement according to the AKI detection algorithm used by NHS England [19]. This algorithm has been shown to perform well compared with nephrologist diagnosis across a number of settings [20]. Baseline was taken as the median creatinine level in the period between 8 and 365 days prior to the index creatinine measurement or, if not available, the lowest level in the period 0–7 days prior to the index measurement. In the absence of any baseline measure, an increase of >26 µmol/L in the creatinine level in a 48-h window was also labelled as AKI Stage 1. For the Dundee cohort, AKI was subcategorized into hospital acquired (>24 h after the date of admission to hospital) or community acquired, to allow for subgroup analyses.

Statistical analyses

The regression coefficients from the final logistic regression models in the derivation sample were fixed and the fitted models were applied to the validation cohorts to assess their predictive performance based on discrimination and calibration. The integer values were derived by dividing the beta weights for each category of each variable by the lowest beta weight in the model (and subtracting 1 from each integer).

RESULTS

The Tayside development cohort included 273 450 individuals. The mean age of the cohort was 52 years [standard deviation (SD) 18.6]. In total, 4761 (1.7%) individuals had an episode of AKI during the calendar year 2010, of which 581 were hospital acquired. Of these, 1619 were Stage 2 or 3 AKI, of which 433 were hospital acquired. Table 1 presents the baseline details for the development cohort along with the results of the regression analysis for all AKI and Stages 2 and 3 AKI. In total, 16 785/18 510 (90.6%) people with diabetes had Type 2 diabetes; therefore the type of diabetes was not subdivided in any analyses. The Kent (UK) external validation cohort included 218 019 individuals, while the Alberta external validation cohort comprised of 1 173 607 individuals. The baseline characteristics of the Tayside development cohort and external validation cohorts (Kent and Alberta) are presented in Table 2. The Kent validation cohort was older with a mean age of 61.7 years (SD 17.8) compared with the Dundee development cohort [52 years (SD 18.6)] and Alberta validation cohort [52.9 years (SD17.9)]. The proportion of patients with CKD defined as an eGFR <60 mL/min was also higher in Kent (16.3%), compared with 9.75% in Alberta and 7.5% in Dundee. The AKI incidence was higher in the validation cohorts compared with the development cohort: Alberta 3.31%, Kent 2.93% and Dundee 1.74%.

Table 1.

Baseline data and multivariable regression analysis for development cohort: all AKI and AKI Stage 2 or 3

		Any AKI		AKI Stage 2 or 3
Covariate	n (%)		OR (95% CI)	β	OR (95% CI)
Age (years) (<20 as index)	4780 (1.7)	–	1	–	1
20–29	35 994 (13.2)	0.213	1.24 (0.77–2.00)	−0.657	0.52 (0.25–1.06)
30–39	36 247 (13.3)	0.438	1.55 (0.97–2.49)	−0.234	0.79 (0.40–1.57)
40–49	48 411 (17.7)	0.445	1.56 (0.98–2.49)	0.110	1.12 (0.58–2.16)
50–59	47 340 (17.3)	0.680	1.97 (1.24–3.14)	0.120	1.13 (0.59–2.17)
60–69	45 887 (16.8)	0.935	2.55 (1.61–4.04)	0.232	1.26 (0.66–2.41)
70–79	33 656 (12.3)	1.059	2.89 (1.82–4.57)	0.278	1.32 (0.69–2.52)
80–89	18 118 (6.6)	1.068	2.91 (1.83–4.62)	0.225	1.25 (0.66–2.40)
>90	3107 (1.1)	1.021	2.78 (1.71–4.50)	−0.107	0.90 (0.45–1.79)
Female sex	1 51 826 (55.5)	–	1	–	1
Male sex	1 21 714 (44.5)	0.099	1.10 (1.04–1.19)	0.148	1.16 (1.05–1.29)
eGFR ≥60 mL/min/1.73 m2	2 53 233 (92.6)	–	1	–	1
eGFR 45–59 mL/min/1.73 m2	13 500 (4.9)	0.535	1.71 (1.55–1.88)	−0.088	0.92 (0.78–1.07)
eGFR 30–44 mL/min/1.73 m2	5286 (1.9)	0.678	1.97 (1.76–2.20)	−0.159	0.85 (0.71–1.03)
eGFR <30 mL/min/1.73 m2	1521 (0.6)	1.749	5.75 (5.03–6.58)	1.236	3.44 (2.91–4.08)
Previous episode AKI	17 138 (6.3)	2.826	16.87 (15.70–18.13)	4.652	104.78 (88.43–124.15)
Diabetes mellitus	18 510 (6.8)	0.433	1.54 (1.42–1.67)	NI	NI
Liver disease	18 887 (6.9)	0.262	1.30 (1.18–1.43)	NI	NI
Heart failure	4860 (1.8)	0.153	1.17 (1.04–1.31)	NI	NI
Myocardial infarction	9274 (3.4)	0.230	1.26 (1.14–1.39)	NI	NI
Stroke	4039 (1.5)	0.262	1.30 (1.10–1.54)	NI	NI
Neurological disease	3537 (1.3)	0.234	1.26 (1.11–1.44)	NI	NI
AAA repair	394 (0.1)	0.692	2.00 (1.45–2.75)	−0.723	0.49 (0.24–1.00)
Aldosterone antagonist	2219 (0.8)	NI	NI	NI	NI
Loop diuretic	11 767 (4.3)	0.211	1.24 (1.13–1.35)	0.398	1.49 (1.31–1.69)
Non-loop diuretic	17 339 (6.3)	−0.165	0.85 (0.77–0.94)	NI	NI
NSAID	23 218 (8.5)	NI	NI	0.259	1.30 (1.07–1.57)

OR, odds ratio; AAA, abdominal aortic aneurysm; NSAID, non-steroiodal anti-inflammatory drug; NI, Not included in the model as it failed to reach significance.

Table 2.

Baseline characteristics of participants in the development (Dundee) cohort and Alberta and Kent validation cohorts

		Dundee					Alberta						Kent (UK)
Variable		All		Any AKI		AKI Stage 2 or 3	All		Any AKI		AKI Stage 2 or 3		All	Any AKI	AKI Stage 2 or 3
		(N = 2 73 540)		(n = 4761)		(n = 1619)	(N = 1  173 607)		(n = 38 868)		(n = 7911)		(N = 2 18 091)	(n = 6397)	(n = 1632)
Age (years), mean (SD)		52.0 (18.6)		67.9 (17.3)		68.7 (16.4)	52.9 (17.9)		66.3 (18.3)		67.5 (17.1)		61.7 (17.8)	72.6 (16.6)	72.9 (15.5)
Age group (years), n (%)
18–19	4780 (1.75)		19 (0.40)		10 (0.62)		9599 (0.82)	88 (0.23)		11 (0.14)		706 (0.3)		14 (0.2)	2 (0.1)
20–29	35 994 (13.16)		171 (3.59)		37 (2.29)		131 018 (11.16)	1733 (4.46)		250 (3.16)		12764 (5.9)		162 (2.5)	23 (1.4)
30–39	36 247 (13.25)		231 (4.85)		62 (3.83)		1 72 279 (14.68)	2451 (6.31)		352 (4.45)		15 679 (7.2)		193 (3.0)	43 (2.6)
40–49	48 411 (17.70)		327 (6.87)		119 (7.35)		2 02 285 (17.24)	3019 (7.77)		613 (7.75)		23 889 (11.0)		289 (4.5)	78 (4.8)
50–59	47 340 (17.31)		496 (10.42)		170 (10.50)		2 48 259 (21.15)	5777 (14.86)		1249 (15.79)		36 452 (16.7)		535 (8.4)	148 (9.1)
60–69	45 887 (16.78)		875 (18.38)		305 (18.84)		1 93 078 (16.45)	7390 (19.01)		1621 (20.49)		44 909 (20.6)		977 (15.3)	272 (16.7)
70–79	33 656 (12.30)		1278 (26.84)		454 (28.04)		1 24 417 (10.6)	7850 (20.2)		1644 (20.78)		49 079 (22.5)		1639 (25.6)	416 (25.5)
80–89	18 118 (6.62)		1131 (23.76)		396 (24.46)		74 760 (6.37)	7913 (20.36)		1605 (20.29)		27 491 (12.6)		1818 (28.4)	464 (28.4)
>90	3107 (1.14)		233 (4.89)		66 (4.08)		17 912 (1.53)	2647 (6.81)		566 (7.15)		7122 (3.3)		770 (12.0)	186 (11.4)
Sex, n (%)
Female	1 51 826 (55.50)		2528 (53.10)		840 (51.88)		6 52 295 (55.58)	21 071 (54.21)		4174 (52.76)		1 19 732 (54.9)		3311 (51.8)	787 (48.2)
Male	1 21 714 (44.50)		2233 (46.90)		779 (48.12)		5 21 312 (44.42)	17 797 (45.79)		3737 (47.24)		98 359 (45.1)		3086 (48.2)	845 (51.8)
eGFR stage (mL/min/1.73 m2), n (%)
≥60		2 53 233 (92.58)		2746 (57.68)		984 (60.78)	1 059 208 (90.25)	24 731 (63.63)		4788 (60.52)		1 82 495 (83.7)		3986 (62.3)	971 (59.5)
45–59		13 500 (4.94)		818 (17.18)		224 (13.84)	72 509 (6.18)	5590 (14.38)		1229 (15.54)		24 255 (11.1)		1151 (18.0)	260 (15.9)
30–44		5286 (1.93)		640 (13.44)		159 (9.82)	30 645 (2.61)	4828 (12.42)		1098 (13.88)		8184 (3.8)		752 (11.8)	155 (9.5)
<30		1521 (0.56)		557 (11.70)		252 (15.57)	11 245 (0.96)	3719 (9.57)		796 (10.06)		2547 (1.2)		482 (7.5)	239 (14.6)
Diabetes mellitus, n (%)		18 510 (6.77)		1142 (23.99)		354 (21.87)	58 238 (4.96)	6838 (17.59)		1512 (19.11)		15 112 (6.9)		1162 (18.2)	323 (19.8)
Liver disease, n (%)		18 887 (6.90)		663 (13.93)		226 (13.96)	11 914 (1.02)	2282 (5.87)		774 (9.78)		–		–	–
Heart failure, n (%)		4860 (1.78)		626 (13.15)		205 (12.66)	26 417 (2.25)	6678 (17.18)		1410 (17.82)		1753 (0.8)		253 (4)	62 (3.8)
Myocardial infarction, n (%)		9274 (3.39)		762 (16.01)		243 (15.01)	31 049 (2.65)	5127 (13.19)		1127 (14.25)		–		–	–
Stroke, n (%)		4039 (1.48)		338 (7.10)		113 (6.98)	13 776 (1.17)	1752 (4.51)		379 (4.79)		–		–	–

Predictive model

Table 3 presents the results of the regression analysis and risk score weights derived for the four-variable score. Table 4 presents the sensitivity, specificity and positive and negative predictive values for a range of cut-off scores in the Dundee cohort.

Table 3.

Regression analysis for forced entry development model

	AKI Stages 1–3 (4761 events)				AKI Stages 2–3 (1619 events)
Covariate	OR (95% CI)		β	Points	OR (95% CI)		β
Age (years) (<20 as index)	1		–	0	1	–	–
20–29	1.20 (0.74–1.92)		0.178	0	0.49 (0.24–0.99)		−0.713
30–39	1.56 (0.98–2.49)		0.445	2	0.80 (0.41–1.56)		−0.225
40–49	1.59 (1.00–2.52)		0.462	2	1.12 (0.58–2.13)		0.109
50–59	2.28 (1.44–3.61)		0.824	4	1.54 (0.82–2.93)		0.435
60–69	3.51 (2.22–5.53)		1.254	6	2.53 (1.35–4.76)		0.928
70–79	4.89 (3.10–7.71)		1.586	8	3.85 (2.05–7.23)		1.347
80–89	5.74 (3.63–9.07)		1.747	9	4.50 (2.38–8.48)		1.503
>90	5.76 (3.58–9.28)		1.751	9	3.46 (1.76–6.81)		1.241
eGFR ≥60 mL/min/1.73 m2	1	–	–	0	1	–	–
eGFR 45–59 mL/min/1.73 m2	2.67 (2.44–2.92)		0.983	5	1.87 (1.60–2.20)		0.628
eGFR 30–44 mL/min/1.73 m2	4.66 (4.20–5.17)		1.539	8	2.87 (2.37–3.46)		1.053
eGFR <30 mL/min/1.73 m2	20.78 (18.37–23.50)		3.034	16	18.93 (15.95–22.47)		2.941
Diabetes mellitus	2.15 (1.99–2.32)		0.766	3	1.73 (1.53–1.97)		0.550
Heart failure	2.48 (2.24–2.75)		0.910	4	2.10 (1.78–2.49)		0.744
Constant	–	–	−5.546	–	–	–	−6.183

OR, odds ratio.

Table 4.

Sensitivity, specificity and positive and negative predictive values for different score cut-offs in the Dundee cohort

	Any AKI						AKI Stage 2 or 3
Risk score	No AKI	Any AKI	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	No AKI 2/3	Any AKI 2/3	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
0	40 072	152	96.8	14.9	2.0	99.6	40193	31	98.1	14.8	0.7	99.9
≥1	2 28 707	4609					231728	1588
≤4	1 64 429	829	82.6	61.2	3.6	99.5	165011	247	84.7	60.7	1.3	99.9
≥5	1 04 350	3932					106910	1372
≤9	2 45 430	2422	49.1	91.3	9.1	99.0	246989	863	46.7	90.8	2.9	99.7
≥10	23 349	2339					24932	756
≤14	2 61 077	3329	30.1	97.1	15.7	98.7	263271	1135	29.9	96.8	5.3	99.6
≥15	7702	1432					8650	484

PPV, positive predictive value; NPV, negative predictive value.

Sensitivity analyses

Supplementary data, Tables S2 and S3, show the inclusion of male sex and previous AKI into the model. Inclusion of previous AKI further improved the model performance for both any AKI {C-statistic 0.88 [95% confidence interval (CI) 0.87–0.88]} and severe AKI [C-statistic 0.84 (95% CI 0.83–0.85)].

Model performance

The C-statistics of the model for predicting any stage of AKI were 0.80 (95% CI 0.80–0.81) in the development cohort, 0.71 (0.70–0.72) in the Kent (UK) external validation cohort and 0.76 (0.75–0.76) in the Canadian external validation cohort. Discrimination for predicting Stage 2 or 3 AKI was slightly better, with C-statistics of 0.81 (95% CI 0.80–0.82) in the development cohort, 0.74 (0.73–0.75) in the Kent (UK) external validation cohort and 0.78 (0.77–0.78) in the Canadian external validation cohort (Table 5). The observed risk of AKI at each risk score in all cohorts is shown in Figure 1. For the Dundee cohort, the discrimination was further assessed separately for hospital-acquired AKI and CA-AKI. For any AKI stage, the C-statistic was 0.82 (95% CI 0.80–0.84) for hospital-acquired AKI and 0.81 (0.80–0.82) for CA-AKI. For AKI Stage 2 or 3, the C-statistic was 0.81 (95% CI 0.79–0.83) for hospital-acquired AKI and 0.81 (0.80–0.82) for CA-AKI.

Table 5.

Performance of prediction models

	Development (Dundee)		Validation (UK-Kent)		Validation (Canada)		Validation (Canada), recalibrated
Cohort	AKI Stages 1–3	AKI Stage 2 or 3	AKI Stages 1–3	AKI Stage 2 or 3	AKI Stages 1–3	AKI Stage 2 or 3	AKI Stages 1–3	AKI Stage 2 or 3
Discrimination C-statistic (95% CI)	0.80 (0.80–0.81)	0.81 (0.80–0.82)	0.71 (0.70–0.72)	0.74 (0.73–0.75)	0.76 (0.75–0.76)	0.78 (0.77–0.78)	0.76 (0.75–0.76)	0.78 (0.77–0.78)
Calibration
Calibration-in-the-large (H0: intercept = 0)	<0.001	<0.001	<0.001	<0.001	0.0787	−0.801	−0.0043	0.0025
Calibration slope (H0: slope = 1)	1.000	1.000	1.000	1.000	0.8436	0.810	0.9984	0.9997

Prediction equation: Risk = 1/(1 + e β0 + β1*age + β2*eGFR + β3*diabetes mellitus + β4*heart failure), where β0–β4 are the respective regression coefficients of each predictor variable as presented in Table 3.

FIGURE 1

Observed risk of AKI at each risk score in all cohorts.

The calibration plots for the development and external validation cohorts are shown in Figures 2 and 3. The model predicted AKI accurately at lower risks of AKI but overestimated risk at higher risks (>20% predicted probability) in all three cohorts. This improved following recalibration in the Canadian cohort. For prediction of Stage 2 or 3 AKI, the model performed reasonably well at lower risk but overestimated risk at higher risks; AKI observed risk remained stable at ~10% for predicted risk values of >10%. Calibration plots for the Dundee development model and Kent UK validation model are shown in Figure 2. Calibration plots for the Canadian external validation model and recalibrated model are shown in Figure 3.

FIGURE 2

Calibration plots for Tayside and Kent data: observed versus predicted risk.

FIGURE 3

Calibration and recalibration plots for Alberta data: observed versus predicted risk.

DISCUSSION

We have developed and externally validated a simple risk score comprised of four variables—age, eGFR category, diabetes and heart failure—that can be used to predict the risk of AKI in the general population using routinely collected data. Unlike other prediction models for AKI in acute care settings, our externally validated risk score for AKI can be used for prediction of AKI events in community or hospital settings. This risk score could enable identification of patients at either high risk (who might be targeted for AKI prevention intervention) or low risk (and hence could avoid unnecessary intervention) across broad community-based and clinical settings.

Approaches to identifying patients at risk of CA-AKI are lacking. Most of the available data examine the incidence of CA-AKI in those admitted to hospital [1, 21, 22]. Sawhney et al. [23] applied the NHS England algorithm for AKI to a large Scottish cohort of >50 000 patients. They found the overall incidence of AKI was 9%, with a CA-AKI incidence of 3.5%. Talabani et al. [24] examined the incidence of CA-AKI from all outpatient creatinine measurements sent to a laboratory over a 1-month period. Their incidence of CA-AKI was 0.7%. However, there are few studies examining predictors of CA-AKI. Der Mesropian et al. [25] compared patients with CA-AKI, hospital-acquired AKI and no AKI from a cohort of hospitalized patients. In keeping with our findings, they found increasing age, diabetes, CKD, coronary artery disease and heart failure were associated with AKI []. Similarly, risk factors for CA-AKI in a large cohort of Taiwanese adults were diabetes, rheumatological diseases, CKD, liver disease, chronic pulmonary disease, cerebrovascular disease, heart failure, peptic ulcer disease and malignancy [26].

There is a need for better risk stratification for AKI. Most research to date has focused on patients undergoing cardiac surgery. In this context, several externally validated risk scores have been produced, although many include intraoperative variables [27-30]. There are fewer non-cardiac surgery scores [7, 9, 31, 32]. A risk score derived and validated in two large Scottish cohorts of patients undergoing orthopaedic surgery identified several predictors similar to those in our score: age, sex, diabetes, number of prescribed drugs, lower eGFR rate, use of angiotensin-converting enzyme inhibitors and American Society of Anaesthesiologists grade [3]. However, a risk score that is also useful for identifying high-risk patients in community settings remains lacking. Future approaches might incorporate machine learning, but the utility and additional value of such an approach in widespread clinical practice remain to be shown.

The strength of our study is the use of multiple, large cohorts, providing the ability to evaluate several candidate predictor variables and to perform external validation in geographically distinct cohorts. Other strengths include the use of the KDIGO creatinine-based criteria to identify AKI, allowing for development and validation of a score to predict any stage of AKI and more severe (KDIGO Stage 2 or 3) AKI. A potential limitation is that the risk score was developed and validated in retrospectively collected routine datasets and so diagnosis of AKI was dependent on creatinine testing, thereby potentially introducing ascertainment bias. However, by developing a risk score using routinely collected data, this score can be used for predicting AKI risk in the general population without gathering extra information, making it well suited for implementation into primary care software systems. A further limitation was that there was insufficient proteinuria data to allow its inclusion into the model. Inclusion of proteinuria may have improved the predictive performance of the model due to the well-recognized association between albuminuria and AKI [33]. It is also important to note that we did not include previous AKI in the risk score, due to inconsistent coding, even though it was a strong predictor. This may be a useful predictor in the future once uniform coding and notation in medical records are established. In addition, we did not have primary care data for comorbidities such as heart failure, which may have led to an underestimation. Conversely, the ICD-10 code for heart failure lacks specificity, leading to potential false positives. Furthermore, our risk score was developed and validated in predominantly Caucasian populations in high-income countries that provide universal access to healthcare, therefore further work is required to establish whether it is applicable to other ethnicities, geographic regions and health systems.

It is surprising that the discriminant ability for AKI Stages 2 and 3 was no higher than for AKI 1 in the development cohort. This may reflect the fact that the factors included in the risk score are associated with occurrence, but other non-baseline factors may be associated more closely with severity. For instance, the severity of intercurrent illness that precipitates AKI may align more closely with whether a patient develops mild or severe AKI. However, such information cannot form part of a predictive score, as it becomes known only at the point of kidney injury. The incidence of AKI was lower in the Dundee cohort compared with both validation cohorts, despite the use of a consistent approach for AKI ascertainment based on the NHS AKI detection algorithm. Possible explanations include the older mean age and higher rates of baseline CKD in the Kent cohort compared with the Dundee cohort. Also, measured or unmeasured comorbidities, medication use or differences in clinician behaviour driving different rates of biochemistry requesting may have contributed. Despite these differences in AKI incidence between the cohorts, discrimination of our risk score was good in both the external validation datasets with C-statistics >0.7 and was better for predicting more severe AKI. However, the risk score overestimated the predicted risk for any stage of AKI and AKI Stages 2 and 3 for individuals at high predicted risk. Recalibration to the incidence of AKI within a region is a recognized method of improving differences in observed versus predicted risks [34, 35]. Recalibration of the risk score in the Canadian cohort reduced the overestimation of predicted risk of any stage of AKI, however, less so for AKI Stages 2 and 3. This difference in performance is likely to be due to the risk score weights being derived from the regression model using all AKI as a predictor. While it would be possible to calculate a separate risk score for AKI Stages 2 and 3, this would complicate the implementation of the score into clinical practice. At a population level, the overestimation of risk at high risk levels may not preclude clinical utility of the implementation of the risk score to guide decisions based on certain risk thresholds; the score could still be valuable to identify those at high risk (e.g. >10% risk per year) and risk-stratify them distinctly from those at low risk (e.g. <5% risk per year). However, it remains unclear at what level of risk is it worth intervening. Conversely, our findings suggest caution against using our risk score to distinguish those at a high risk of AKI Stages 2 and 3 (e.g. 10–20% risk per year) from a very high risk of AKI Stages 2 and 3 (e.g. >20% risk per year).

Our study has several important clinical implications. The four variables forming the risk score are readily available in the UK and North American primary care clinical software systems, allowing the potential for the score to be used to flag those at high risk. The simplicity of both the variables used and the scoring makes the score practical to apply across many healthcare settings, facilitating implementation studies on AKI risk assessment. This contrasts with other AKI risk stratification that relies on proprietary algorithms, machine-learning approaches or the use of prognostic biomarker tests, which are less amenable to broad uptake. Our score could potentially be used for identification of patients requiring closer monitoring during acute illnesses, with earlier checking of kidney function and the use of AKI prevention strategies that could mitigate progression to more severe stages of AKI and potentially avoidance of hospital admission. General practitioners (GPs) working in the community have limited access to acute biochemical and haematological tests and regularly need to make decisions about whether unwell patients require hospital admission. Our risk score could provide useful additional information to aid GPs with these difficult decisions, but it is important to note that risk may be overestimated with accompanying additional workload. It is important to evaluate this additional workload against the potential benefits. Furthermore, this score could potentially be utilized at the point of hospitalization. However, implementation studies and clinical impact analysis are required to establish the clinical value of such approaches based on AKI risk stratification.

There remains a lack of evidence on the optimal ways of preventing AKI (as opposed to mitigating established AKI). Effective interventions in primary care will need to be simple, inexpensive and carry minimal risk of harm; interventions must be able to be deployed at scale without causing a major burden on practitioners or patients [36, 37]. Potential interventions include ensuring adequate hydration, identifying and treating infections, avoidance of nephrotoxins and monitoring renal function. An important step in finding effective preventative interventions is to identify those patients at increased risk of AKI who might benefit most from such targeted interventions. This model may help to achieve this step. Conversely, it is important to identify those at very low risk of AKI, as this group can then be protected from the additional time, costs and potential adverse consequences that result from trade-offs of applying preventative interventions.

CONCLUSION

Identification of patients in the community at high risk for AKI is a key to early identification and prevention of AKI. We have derived and externally validated a simple risk score from routinely collected data that can aid both primary and secondary care physicians in identifying these patients within the general population.

Click here for additional data file.