Integrating electronic health data records to develop and validate a predictive model of hospital-acquired acute kidney injury in non-critically ill patients.

BACKGROUND: Models developed to predict hospital-acquired acute kidney injury (HA-AKI) in non-critically ill patients have a low sensitivity, do not include dynamic changes of risk factors and do not allow the establishment of a time relationship between exposure to risk factors and AKI. We developed and externally validated a predictive model of HA-AKI integrating electronic health databases and recording the exposure to risk factors prior to the detection of AKI.
METHODS: The study set was 36 852 non-critically ill hospitalized patients admitted from January to December 2017. Using stepwise logistic analyses, including demography, chronic comorbidities and exposure to risk factors prior to AKI detection, we developed a multivariate model to predict HA-AKI. This model was then externally validated in 21 545 non-critical patients admitted to the validation centre in the period from June 2017 to December 2018.
RESULTS: The incidence of AKI in the study set was 3.9%. Among chronic comorbidities, the highest odds ratios (ORs) were conferred by chronic kidney disease, urologic disease and liver disease. Among acute complications, the highest ORs were associated with acute respiratory failure, anaemia, systemic inflammatory response syndrome, circulatory shock and major surgery. The model showed an area under the curve (AUC) of 0.907 [95% confidence interval (CI) 0.902-0.908), a sensitivity of 82.7 (95% CI 80.7-84.6) and a specificity of 84.2 (95% CI 83.9-84.6) to predict HA-AKI, with an adequate goodness-of-fit for all risk categories (χ2 = 6.02, P = 0.64). In the validation set, the prevalence of AKI was 3.2%. The model showed an AUC of 0.905 (95% CI 0.904-0.910), a sensitivity of 81.2 (95% CI 79.2-83.1) and a specificity of 82.5 (95% CI 82.2-83) to predict HA-AKI and had an adequate goodness-of-fit for all risk categories (χ2 = 4.2, P = 0.83). An online tool (predaki.amalfianalytics.com) is available to calculate the risk of AKI in other hospital environments.
CONCLUSIONS: By using electronic health data records, our study provides a model that can be used in clinical practice to obtain an accurate dynamic and updated assessment of the individual risk of HA-AKI during the hospital admission period in non-critically ill patients.

INTRODUCTION

Acute kidney injury (AKI) is a frequent and serious complication in hospitalized patients [1-3]. In addition, AKI has been associated with long-term morbidity and mortality after hospital discharge [4, 5].

Most cases of AKI in hospitalized patients are caused by ischaemia or nephrotoxicity [6-8]. The risk of developing AKI depends on the characteristics of the patient in terms of age, presence of previous kidney disease and number and types of comorbidities [9]. Since a large part of the AKI episodes are due to potentially avoidable causes, knowing as accurately as possible the individual risk at any time of the hospital stay could help decision making and implementation of preventive measures to reduce the incidence of hospital AKI [10]. The Kidney Disease: Improving Global Outcomes (KDIGO) guidelines recommend that patients be stratified for risk of AKI at admission and managed according to their susceptibilities and exposures to reduce the risk of AKI [11]. The incidence and risk factors associated with AKI in patients admitted to intensive care units (ICUs) have been extensively analysed [12, 13]. However, these models are difficult to extrapolate to non-critically ill patients since they have been developed for patients that are under the influence of a cluster of risk factors related to haemodynamic instability, use of vasoactive drugs, low tissue oxygenation, inflammatory response and invasive procedures such as mechanical ventilation that are unique to this environment [14]. The few studies analysing the epidemiology and risk factors associated with AKI in non-critically ill patients have two main limitations to identify accurately the risk factors associated with AKI. First, all of them are based on demographic characteristics and comorbidities that have been registered retrospectively from the discharge administrative codes and therefore are subject to a potential bias in the collection of coded information [15-20]. Second, they do not allow us to know whether the exposure to risk factors preceded the detection of the AKI episode [21].

The aim of our study was to develop and validate a predictive model of hospital-acquired AKI (HA-AKI) in non-critically ill patients in which risk factors are automatically obtained by integrating electronic health databases, it is ensured that the exposure to risk factors precedes the detection of the AKI episode and AKI episodes are automatically detected through electronic systems based on the calculation of differences in creatinine levels.

MATERIALS AND METHODS

This prospective study was performed at two different hospital centres. The first centre developed the predictive model (study set) and the second centre performed the external validation of the predictive model (validation set).

Study set

The study set included patients admitted to the Vall d’Hebron University Hospital from January to December 2017. Vall d’Hebron is a tertiary, high-complexity hospital that provides assistance to a population of 500 000 habitants in Barcelona, Spain and provides all kinds of medical and surgical procedures, including neurosurgery, cardiac surgery, endovascular catheter-guided procedures and lung, liver, kidney and bone marrow transplantation programmes. We included all patients >18 years of age who were admitted to the hospital during this period and did not meet any of the following exclusion criteria: admission for community-acquired AKI; hospital stay <24 h; admission for elective heart surgery; direct admission from the emergency room to the ICU; admission as a recipient of a renal, lung, liver or bone marrow transplant; absence of serum creatinine measurements done at least 12 months after hospital admission; chronic haemodialysis treatment and denial of written consent to participate in the study. Community-acquired AKI was diagnosed whenever patients met the AKI criteria within the first 24 h of hospital admission. Patients initially admitted to conventional hospitalization wards who afterwards required admission to the ICU were only included if the AKI episode was detected while they were admitted in non-critically ill wards prior to their admission to the ICU.

Baseline kidney function

Our patient care system integrates the laboratory databases of the hospital and primary care registers, thus allowing historical data to be obtained for all patients who are hospitalized, provided that these data appear in those registers. Baseline kidney function was obtained from the electronic laboratory data records of primary healthcare and defined as the most recent glomerular filtration rate, estimated by the Chronic Kidney Disease Epidemiology Collaboration equation, within the 12 months prior to hospital admission.

Definition of AKI

AKI was defined and classified in severity stages according to the KDIGO criteria [11]. HA-AKI was defined as an increase in serum creatinine ≥0.3 mg/dL or >50% over the baseline occurring from the first 24 h to any time within the hospital admission.

AKI detection

The software integrated into the electronic laboratory database was used to perform repeated comparisons among all serum creatinine levels available for each patient during the hospital stay and generated an identification code, assigning 1 when the AKI criteria were met and 0 when not. It also assigned a level of AKI severity according to the maximum differences in serum creatinine detected. The date of AKI detection was also recorded. The number of the admission episode was used as a filter so that patients with more than one AKI episode during the hospital stay were entered into the database only once, corresponding with the more severe episode of AKI.

Clinical evaluation at hospital admission and during hospital stay

At hospital admission, a team of 10 trained nurses and 4 nephrologists examined the medical data and interviewed all patients to record age, gender, ethnic group and the presence of the following chronic comorbidities: diabetes mellitus, hypertension, ischaemic heart disease (IHD), ischaemic cerebrovascular disease (ICD), ischaemic peripheral vascular disease (PVD), chronic digestive disease, chronic liver disease, chronic congestive heart failure (CCHF), malnutrition (MN), chronic obstructive pulmonary disease (COPD), malignancy, dementia, rheumatologic disease, acquired immunodeficiency syndrome (AIDS)/human immunodeficiency virus (HIV), urologic disease or chronic kidney disease (CKD). All these variables were recorded in the general study database according to the criteria detailed in the Supplementary methods operational definitions. Nutritional status was assessed using theNutritional Risk Screening 2002 test [22]. The allocation of comorbidity codes to each patient was carried out by consensus among clinical researchers. All patients were followed up until hospital discharge. During the hospital stay, the data of six electronic health databases, i.e. vital signs, laboratory, pharmacy prescription, interventional radiology, interventional cardiology and surgery, were integrated together using the number of the admission episode, which is unique for each patient and common to all these databases. Overall, the information extracted from these six databases included haemoglobin levels, leucocyte count, oxygen saturation, body temperature, blood pressure, heart rate and respiratory rate, as well as a complete list of nephrotoxic drugs (detailed in Supplementary data, Table S1) and exposure to contrast dyes or major surgery. Every 24 h, updated information from all these data was dumped into the general study database, which also contained the comorbidity data and all available values of serum creatinine for each patient. From these data, software generated classification codes for anaemia, hypoxaemic acute respiratory failure, systemic inflammatory response syndrome (SIRS), shock, exposure to nephrotoxic drugs, contrast dyes and major surgery. Using these codes, the exposure to all these risk factors was classified as positive (= 1), when the system detected at least one exposure during the hospital stay, or negative (= 0), when no exposure was detected. In all cases, the system recorded the data of exposure to each of these variables as well as the number of exposures to them. In patients with a code of AKI = 1, the exposure to these risk factors only was classified as equal to 1 when it occurred within a maximum period of time prior to AKI detection (48 h for anaemia, SIRS and shock, 72 h for contrast dyes and surgery and 7 days for nephrotoxic drugs). Figure 1 shows a schematic view of the interrelation process among the different electronic databases carried out to obtain the information on the clinical variables during the hospital stay.

FIGURE 1:

Schematic representation of the interrelation between electronic databases performed to obtain updated clinical information during the hospital stay.

At hospital admission (A), chronic comorbidities are checked by the research team according to explicit criteria and recorded in the general database. During the hospital stay (B), the data of five different electronic health databases are integrated using the admission episode number and all of them dump the requested information into the general study database. The laboratory database performs repeated comparisons among all serum creatinine levels and generates an identification code, assigning a 1 when the AKI criteria are met and a 0 when not. It also assigns a level of AKI severity according to the maximum differences in serum creatinine detected. The date of AKI detection is also recorded. The admission episode number is used as a filter so that patients with more than one AKI episode during the hospital stay are entered into the system only once, corresponding with the more severe episode of AKI. The follow-up of haemoglobin levels is used to generate a classification code of anaemia. The level of oxygen saturation is used to generate a code of hypoxaemic acute failure. Information on blood leucocyte levels, together with temperature, heart and respiratory rate, are integrated to generate a code for SIRS and information on blood pressure, together with the prescription of vasoactive drugs, is used to generate a code for shock. A complete list of direct nephrotoxic drugs is introduced in the pharmacy prescription database, which generates a code of exposure every time the prescription list contains any of these drugs. The databases of radiology, angioradiology and interventional cardiology provide information about the exposure to contrast dyes and the database of surgery provides information about major surgery and anaesthesia. In all cases, the system records the data for exposure to each one of these factors. In patients with a code of AKI = 1, the exposure to risk factors is classified as equal to 1 only when it occurs within a maximum period of time prior to AKI detection (48 h for anaemia, SIRS and shock, 72 h for contrast dyes and surgery and 7 days for nephrotoxic drugs). In patients with a code of AKI  = 0, the exposure to risk factors is classified as positive (= 1), when the system detects at least one exposure during the hospital stay, or negative (= 0), when none is detected. In both cases (AKI and no AKI), the number of exposures to each risk factor is also recorded.

Unlike the haemoglobin level, arterial oxygen saturation, heart rate, respiratory rate and blood pressure level, being numerical variables that can be directly transferred into the general database, both circulatory shock and SIRS are complex variables that, to be automatically detected using a software-guided detection code, require the integration of data from various electronic records and the definition of classification algorithms. In both cases, before using them in statistical analyses, we analysed the accuracy of the automatic detection systems in a sample of 3426 patients. To do this, using data blindly obtained by two independent clinical investigators, we performed a concordance analysis between the identification of cases using electronic detection systems and the diagnosis made by the investigators using clinical criteria, as well as an analysis of interobserver agreement for both clinical diagnoses. The results of these analyses are summarized in Supplementary data, Table S2.

Validation set

The predictive model obtained at the Vall d’Hebron Hospital was externally validated in patients admitted to the Arnau de Vilanova Hospital of Lleida between June 2017 and December 2018. Arnau de Vilanova Hospital is a high-complexity teaching centre that provides assistance to 490 000 habitants. This centre provides similar activities as the Hospital Vall d’Hebron with the exceptions of transplant programmes and cardiac surgery. The selection of patients and the study procedures were done according to the same criteria stated for the study set. The external validation study was performed by an independent research team that did not participate in the development of the predictive model and it was tested in the hospital electronic health record only.

The ethics committee of the Arnau de Vilanova Hospital was consulted and they decided that informed consent was not necessary for the validation of the model, given that no type of intervention was carried out on the patients.

Statistics

The incidence and prevalence calculations included the total number of admissions. For patients who developed more than one AKI episode during the hospital admission, only the most severe episode was included in the study. Patients were considered to be at risk each time they were admitted to the hospital and therefore patients who were admitted two or more times during the study period were included in the calculations on each admission, except when readmission occurred within 30 days after hospital discharge. Results are given as the mean ± standard deviation (SD) or median and [interquartile range (IQR), 25th percentile–75th percentile]. Differences in risk factors between groups were calculated by the Student’s unpaired t-test oranalysis of variance test. Qualitative variables were compared using the chi-squared test. Concordance analyses between qualitative variables were done by the kappa coefficient. P-values ˂0.05 were considered statistically significant. To determine which variables were independently associated with AKI, we carried out a univariate analysis comparing patients with and without AKI. All the variables with a P-value ˂0.1 in the univariate analysis were entered into stepwise multiple logistic regression analysis with a forward selection method based on changes in the likelihood ratio (LR). Odds ratios (ORs) were calculated from the regression coefficients as an approximation of the relative risk. The predictive value of the logistic model was evaluated using the C statistic, Cox and Snell R2 and Nagelgerke R2. Model overfitting was prevented using the Akaike information criterion (AIC) [23, 24]. The Hosmer–Lemeshow test [25] was used as well to calculate the discrimination power and goodness-of-fit of the logistic model. Results are presented according to theTransparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines for risk prediction models [26, 27]. Once obtained in the study set, the predictive logistic model was blindly tested on the external validation set by an independent group of researchers who did not participate in the development of the predictive model. Statistical analyses were performed with the Statistical Package for the Social Sciences for Windows version 20.0 (IBM, Armonk, NY, USA).

RESULTS

During the study period there were 42 449 hospital discharges. Figure 2 shows the flow chart for patient selection. The final study group comprised 36 852 patients. Of this cohort, 1453 (3.9%) developed AKI, with an incidence of 39 AKI episodes/1000 hospital admissions. Distribution by AKI stages was Stage 1, n = 1069 (73.5%); Stage 2, n = 258 (17.8%) and Stage 3, n = 126 (8.7%).

FIGURE 2:

Flow chart for patient selection.

Table 1 summarizes the demographic characteristics, comorbidities, clinical events and procedures during the hospital stay in the study group, classified according to the presence of AKI. AKI patients were older and more frequently male than non-AKI patients. Comorbidities including IHD, ICD, ischaemic PVD, chronic liver disease, CCHF, MN, COPD, malignancy, urologic disease and CKD stages were also more frequent in AKI patients. The AKI risk increased linearly as glomerular filtration rate decreased. Patients with AKI also showed significantly higher rates of urgent admission, anaemia, acute respiratory failure, SIRS, shock, major surgery and exposure to contrast dyes and nephrotoxic drugs.

Table 1.

Demographic characteristics, chronic comorbidities, clinical events and procedures during the hospital admission and univariate logistic analysis of variables associated with HA-AKI in the study group

Variables	Total	AKI	Non-AKI	OR (95 % CI)
Patients, n (%)	36 852 (100)	1453 (3.9)	35 399 (96)
Gender (male), n (%)	16 782 (45.5)	879 (60.5)	15 903 (44.9)	1.76 (1.59–1.96)
Age (years), mean (SD)	54.9 (20.6)	73 (15.0)	54 (20.5)	1.065 (1.061–1.070)
Chronic comorbidities, n (%)
Diabetes	6837 (18.6)	574 (39.5)	6263 (17.7)	3.04 (2.73–3.39)
Hypertension	14 507 (39.4)	990 (68.1)	13 517 (38.2)	3.46 (3.09–3.87)
IHD	2728 (7.4)	194 (13.4)	2534 (7.2)	2.00 (1.71–2.34)
ICD	2560 (6.9)	181 (12.5)	2379 (6.7)	1.98 (1.68–2.32)
Ischaemic PVD	1924 (5.2)	138 (9.5)	1786 (5.0)	1.98(1.65–2.37)
Chronic digestive disease	2132 (5.8)	70 (4.8)	2062 (5.8)	0.82 (0.64–1.05)
Chronic liver disease	1277 (3.5)	123 (8.5)	1154 (3.3)	2.74 (2.26–3.33)
CCHF	2988 (8.1)	225 (15.5)	2763 (7.8)	2.16 (1.87–2.51)
MN	8524 (23.1)	766 (52.7)	7758 (21.9)	3.97 (3.57–4.42)
COPD	5383 (14.6)	537 (37.0)	4846 (13.7)	3.7 (3.10–4.30)
Malignancy	5278 (14.3)	496 (34.1)	4782 (13.5)	3.32 (2.97–3.71)
Dementia	332 (0.9)	14 (1.0)	318 (0.9)	1.07 (0.63–1.84)
Rheumatologic disease	1543 (4.2)	58 (4.0)	1486 (4.2)	0.95 (0.73–1.24)
AIDS/HIV	293 (0.8)	28 (1.9)	265 (0.7)	2.61 (0.76–3.86)
Urologic disease	2731 (7.4)	172 (11.8)	2559 (7.2)	1.72 (1.46–2.07)
CKD stages, n (%)
0 + 1	30 260 (82.1)	879 (60.5)	29 381 (83.0)	Reference
2	3654 (9.9)	192 (13.2)	3462 (9.8)	1.85 (1.58–2.18)
3	2171(5.9)	231 (15.9)	1940 (5.5)	3.98 (3.42–4.63)
4	767 (2.1)	151 (10.4)	616 (1.7)	8.19 (6.77–9.91)
Clinical variables during hospital admission, n (%)
Urgent admission	24 441 (66.3)	1282 (88.2)	23 159 (65.4)	3.96 (3.37–4.66)
Anaemia	5417 (14.7)	528 (36.3)	4889 (13.8)	3.56 (3.19–3.98)
Acute respiratory failure	1827 (5.0)	286 (19.7)	1541 (4.4)	5.39 (4.69–6.19)
SIRS	658 (1.8)	271 (18.7)	387 (1.1)	20.74 (17.58–24.48)
Circulatory shock	650 (1.8)	300 (20.6)	350 (1.0)	26 (22.09–30.73)
Major surgery	12 127 (32.9)	594 (40.9)	11 533 (32.6)	1.43 (1.29–1.59)
Exposure to contrast media	3353 (9.1)	303 (20.9)	3050 (8.6)	2.80 (2.45–3.19)
Exposure to nephrotoxic drugs	19 145 (52.0)	1011 (69.6)	18 134 (51.2)	2.18 (1.94–2.44)

The results of the logistic model to predict AKI are summarized in Table 2. The highest ORs were associated with advanced stages of CKD, shock, acute respiratory failure and SIRS. The model showed an AUC of 0.907 (95% CI 0.902–0.908), with a sensitivity of 82.7 (95% CI 80.7–84.6) and a specificity of 84.2 (95% CI 83.9–84.6) to predict HA-AKI and showed an adequate goodness-of-fit for all risk categories (Table 3; χ2 = 6.02, P = 0.64).

Table 2.

Final multivariate model selected by forward stepwise logistic regression to predict HA-AKI

Variables	β	Standard error	Wald	OR (95% CI)	P-value
Gender (male), n (%)	0.21	0.069	9.63	1.29 (1.08–1.42)	0.002
Age (years), mean (SD)	0.05	0.003	218.25	1.05 (1.04–1.05)	<0.001
Chronic comorbidities
Diabetes	0.48	0.085	31.67	1.61 (1.36–1.90)	<0.001
Hypertension	0.17	0.073	5.76	1.19 (1.03–1.37)	0.016
Ischaemic heart disease	0.32	0.101	10.02	1.38 (1.13–1.67)	0.002
Ischaemic peripheral vascular disease	0.41	0.123	11.31	1.51 (1.19–1.93)	0.001
Chronic liver disease	1.04	0.126	68.95	2.84 (2.22–3.63)	<0.001
CCHF	0.48	0.079	37.05	1.61 (1.38–1.88)	<0.001
MN	0.25	0.078	10.23	1.29 (1.10–1.50)	0.001
COPD	0.32	0.085	14.03	1.37 (1.16–1.62)	<0.001
Malignancy	0.59	0.089	40.27	1.76 (1.48–2.10)	<0.001
Chronic urologic disease	0.96	0.117	67.02	2.60 (2.07–3.27)	<0.001
CKD stages
0 + 1					Reference
2	0.89	0.096	84.42	2.42 (2.01–2.93)	<0.001
3	1.38	0.098	198.25	3.98 (3.28–4.82)	<0.001
4	2.04	0.125	265.63	7.67 (5.99–9.78)	<0.001
Clinical variables during hospital admission
Urgent admission	0.79	0.097	66.42	2.21 (1.83–2.67)	<0.001
Anaemia	0.78	0.069	125.61	2.18 (1.90–2.49)	<0.001
Acute respiratory failure	1.26	0.097	169.11	3.53 (2.92–4.27)	<0.001
Acute heart failure	0.69	0.095	53.24	2.00 (1.66–2.41)	<0.001
SIRS	1.25	0.129	94.88	3.50 (2.72–4.5)	<0.001
Circulatory shock	1.82	0.127	205.48	6.16 (4.80–7.89)	<0.001
Major surgery	0.99	0.076	169.43	2.70 (2.32–3.13)	<0.001
Exposure to contrast media	0.52	0.087	36.52	1.69 (1.43–2.00)	<0.001
Exposure to nephrotoxic drugs	0.57	0.070	67.04	1.77 (1.54–2.03)	<0.001

Table 3.

Hosmer–Lemeshow’s goodness-of-fit of the logistic predictive model in the study group

Risk deciles	AKI = 0		AKI = 1		Total
	Observed	Expected	Observed	Expected
<0.0008504	3684	3683.9	2	2.1	3686
0.0085041–0.0016950	3678	3680.7	8	5.3	3686
0.0016951–0.0031772	3675	3675.8	11	10.1	3686
0.0031773–0.0053733	3663	3660.6	15	17.3	3678
0.0053733–0.0087714	3655	3656.7	30	28.2	3685
0.0087715–0.0140660	3644	3641.7	42	44.2	3686
0.0140661–0.0228354	3628	3615.7	56	68.2	3684
0.0228355–0.0394850	3586	3575.0	100	110.9	3686
0.0394851–0.0850955	3471	3477.2	214	207.7	3685
>0.0850955	2715	2731.0	974	957.9	3689

χ2 = 6.01. P = 0.645.

Supplementary data, Table S3 summarizes the results of the stepwise forward procedures done to develop the final logistic model, including changes in the LRs, Cox and Snell R2, Nagelkerke R2 and AIC.

The demographic characteristics, comorbidities and clinical parameters of the study and external validation cohorts are summarized in Table 4.

Table 4.

Comparison of demographic characteristics, comorbidities and clinical variables between the study set and the external validation set

Variables	Study set	Validation set	P-value
Patients, n	36 852	21 545	–
Gender (male), n (%)	16 782 (45.5)	9932 (46)	0.19
Age (years), mean (SD)	549 (20.6)	60.1 ± 19.7	0.38
Chronic comorbidities, n (%)
Diabetes	6837 (18.6)	3942 (18.3)	0.44
Hypertension	14 507 (39.4)	8389 (38.9)	0.27
IHD	2728 (7.4)	1573 (7.3)	0.66
ICD	2560 (6.9)	1486 (6.9)	0.83
Ischaemic PVD	1924 (5.2)	1163 (5.4)	0.36
Chronic digestive disease	2132 (5.8)	1228 (5.7)	0.68
Chronic liver disease	1277 (3.5)	775 (3.6)	0.41
CCHF	2988 (8.1)	1659 (7.7)	0.08
MN	8524 (23.1)	4869 (22.6)	0.14
COPD	5383 (14.6)	3102 (14.4)	0.49
Malignancy	5278 (14.3)	3038 (14.1)	0.47
Dementia	332 (0.9)	172 (0.8)	0.1
Rheumatologic disease	1543 (4.2)	851 (4)	0.56
AIDS/HIV	293 (0.8)	86 (0.4)	<0.0001
Urologic disease	2731 (7.4)	1573 (7.3)	0.63
CKD stages, n (%)
0 + 1	30 260 (82.1)	17 731 (82.3)	0.015
2	3654 (9.9)	2198 (10.2)	–
3	2171 (5.9)	1142 (5.3)	–
4	767 (2.1)	474 (2.2)	–
Clinical variables during the hospital admission, n (%)
Urgent admission	24 441 (66.3)	14 422 (66.9)	0.13
Anaemia	5417 (14.7)	3189 (14.8)	0.74
Acute respiratory failure	1827 (5)	1120 (0.5)	0.19
SIRS	658 (1.8)	383 (1.8)	0.97
Circulatory shock	650 (1.8)	370 (1.72)	0.68
Major surgery	12 127 (32.9)	6753 (31.3)	<0.0001
Exposure to contrast media	3353 (9.1)	2068 (9.6)	0.6
Exposure to nephrotoxic drugs	19 145 (52)	11 144 (51.7)	0.59

When compared with the study set, patients of the validation set showed significantly lower prevalences of major surgery and patients with AIDS/HIV. There was as well a significant difference in the distribution of CKD stages between the two centres. In the validation set, 807/21 545 (3.7%) developed HA-AKI, with an incidence of 37.4 AKI episodes/1000 hospital admissions. Distribution by AKI stages was Stage 1, n = 605 (75%); Stage 2, n = 129 (16%) and Stage 3, n = 736 (9%), with no significant differences between the study set and validation set. When the predictive model was tested in the validation set, it showed an AUC of 0.905 (95% CI 0.904–0.910), with a sensitivity of 81.2 (95% CI 79.2–83.1) and a specificity of 82.5 (95% CI 82.2–83) to predict HA-AKI and an adequate goodness-of-fit for all risk categories (χ2 = 4.2,P = 0.83; Table 5).

Table 5.

Hosmer–Lemeshow’s goodness-of-fit of the logistic predictive model in the validation group

Deciles of risk	AKI = 0		AKI = 1		Total
	Observed	Expected	Observed	Expected
<0.0009026	2154	2153.6	1	1.4	2155
0.0009027–0.0018699	2151	2149.9	2	3.1	2153
0.0018700–0.0032362	2150	2149.5	5	5.4	2155
0.0032363–0.0054175	2143	2145.2	11	8.7	2154
0.0054176–0.0085382	2144	2140.2	10	13.8	2154
0.0085383–0.0127563	2134	2131.9	20	22.1	2154
0.0127564–0.0214063	2121	2118.3	33	35.6	2154
0.0214064–0.0391575	2093	2092.6	61	61.3	2154
0.0391575–0.0874214	2034	2035.0	120	118.9	2154
>0.0874214	1612	1623.3	546	533.6	2158

χ2 = 4.2. P = 0.836.

There were no significant differences between the AUC obtained in the study set and that obtained in the validation set (Figures 3 and 4).

FIGURE 3:

Flow chart for patient selection.

FIGURE 4:

Comparison of AUCs obtained in the study set and in the validation set.

DISCUSSION

In this study we integrated the information of six electronic health databases commonly used in clinical practice to develop and externally validate a predictive dynamic model that allows one to accurately estimate the individual likelihood of suffering AKI at any time during a hospital stay in non-critically ill patients. In the study group, the final logistic model identified two sets of risk factors. The first set included the demographic data and the patient’s chronic comorbidities. The second included a set of risk factors related to the patient’s clinical status and to the exposure to major surgery, contrast media or nephrotoxic drugs during the hospital stay. This model showed a high sensitivity and specificity to predict hospital AKI and showed an adequate calibration for all risk categories, both in the study group and in the validation group. When compared with previously published risk models, our model differs at various points. First, unlike previous studies, our study provides a model that allows estimating the risk of HA-AKI tailored to patients admitted to non-critical hospital wards. Moreover, in order to obtain a predictive model that could be exportable to hospitals with different characteristics and complexities, patients who were admitted for programmes and/or procedures such as cardiac surgery or solid organ or bone marrow transplantation that are not commonly available at all hospital centres were deliberately excluded. This potential scalability to less complex centres could be demonstrated as the model had the same performance in the validation set as in the study set. Second, in our study, comorbidities were not obtained through the administrative discharge codes but were checked case by case. Moreover, the classification of comorbidities was performed using explicit and objective definition criteria. In this way, biases related to discrepancies in assigning administrative codes to clinical conditions or to the lack of coding of certain comorbidities were minimized as much as possible. As proof of this, the prevalences of certain comorbidities observed in our cohort of patients are higher than those reported from administrative discharge codes in previous studies [28, 29]. Additionally, in our cohort, the prevalence of comorbidities such as MN, which are barely recorded in the diagnostic codes of discharge, showed similar figures than those described in studies specifically designed to analyse its prevalence [30]. Overall, these differences are in agreement with previously published data that demonstrate the variability and limitations of administrative data to define co-morbidities and clinical conditions [31]. Third, comorbidities were considered separately, which allowed assigning a risk to each of them and identifying independent predictors of AKI risk such as MN, which are not described in previous models. In addition, our model allowed stratifying the risk associated with renal function in greater detail than that provided by the dichotomous classification, depending on the presence or absence of chronic renal failure. Fourth, the main novelty of our study is the prospective monitoring of the evolution of the clinical data of the patients through integration and cross-talk between the different electronic databases containing all these data. This procedure allowed us to analyse the dynamic exposure to risk factors related to the clinical status of patients during the hospital stay, such as hypoxaemia, haemoglobin level, blood pressure changes, contrast dyes or nephrotoxic drugs, prior to the detection of the AKI episode. This integration allowed as well to perform an accurate transformation of single variables such as blood pressure, heart rate, arterial oxygen saturation, prescription of vasoactive drugs or blood leucocyte counts into more complex variables defining specific syndromes such as SIRS and circulatory shock. Electronic records also allowed us to record the exposure to the same variables and risk factors in patients who did not develop AKI during the hospital admission. This approach made it possible to estimate the individual risk, based on the actual exposure to each risk factor. Lastly, since our predictive model was developed from the values of risk factors assessed prior to AKI detection, our model allows one perform dynamic monitoring of risk and even to predict the changes in the individual risk that are expected to happen every time the values of different predictive risk factors change.

Our group recently performed external validation of one of the most recent predictive models of acute renal failure, the Madrid Acute Kidney Injury Prediction Score (MAKIPS) [32]. This model can be calculated automatically from electronic medical records and could be easily implemented in clinical practice. With our validation, we conclude that the MAKIPS can be a useful tool, easily obtainable from electronic records data, to predict AKI in hospitals of different complexity. However, this model, as well as many others described, has the main limitation that it does not include dynamic factors. The inclusion of dynamic changes of possible acute precipitants in the models is technically complex and constitutes a challenge for future research. It could lead to a significant improvement in the discrimination of predictive models and could also generate dynamic predictive models capable of detecting changes in the risk profile of patients throughout the hospital stay.

Our model has some limitations that affect neither its predictive capacity nor its calibration but must be highlighted. First, the record of clinical variables such as blood pressure, heart rate, respiratory rate or oxygen saturation were automatically dumped into the study database; however, these values are not without potential error related to the variability in the manual introduction of these variables into their corresponding databases. Second, although the model allows AKI to be accurately predicted, it does not predict its severity stage. Third, our data indicate that integrating data from different electronic databases make it possible to obtain a reliable prediction of the risk of AKI. However, the model obtained in our study is not the only one that can be obtained with the combination of these data. As exposure to each of the acute complications or nephrotoxic agents can occur at different times after hospital admission, in order to relate the exposure to them with the development of AKI it was necessary to define a maximum period of time between exposure and detection of AKI. In our study, the duration of this period of time was defined by consensus of the research group using pathophysiological criteria. The definition of other periods of time, based on alternative criteria, would modify the prevalence of exposure to these risk factors and consequently the magnitude of the associations found between these variables and AKI.

In conclusion, our study provides an externally validated model based on demographic data, specific comorbidities, acute clinical conditions and procedures that can be used in clinical practice to obtain an accurate dynamic assessment of the individual risk of suffering AKI during the entire hospital stay period in patients admitted into non-critical hospitalization wards. This model is highly versatile and allows for performing repeated manual risk estimation, using the prediction algorithm, to provide an automatic risk measure updated in real time in those centres where it is possible to carry out a complete integration of the healthcare databases containing the necessary information.

SUPPLEMENTARY DATA

Supplementary data are available at ckj online.

FUNDING

This study was funded by grants from Amgen S.A. and Menarini S.A, but these companies did not participate in the study design, data collection, analyses or writing of the study.

CONFLICT OF INTEREST STATEMENT

None declared.

Click here for additional data file.