Predicting and Validating 30-day Hospital Readmission in Adults With Diabetes Whose Index Admission Is Diabetes-related.

OBJECTIVE: The primary objective is to develop a prediction model of 30-day hospital readmission among adults with diabetes mellitus (DM) whose index admission was DM-related. The secondary aims are to internally and externally validate the prediction model and compare its performance with 2 existing models. RESEARCH DESIGN AND
SETTING: Data of inpatients aged ≥ 18 years from 2008 to 2015 were extracted from the electronic medical record system of the National University Hospital, Singapore. Unplanned readmission within 30 days was calculated from the discharge date of the index hospitalization. Multivariable logistic regression and 10-fold cross-validation were performed. For external validation, simulations based on prevalence of 30-day readmission, and the regression coefficients provided by referenced papers were conducted.
RESULTS: Eleven percent of 2355 patients reported 30-day readmission. The prediction model included 4 predictors: length of stay, ischemic heart disease, peripheral vascular disease, and number of drugs. C-statistics for the prediction model and 10-fold cross-validation were 0.68 (95% CI 0.66, 0.70) and 0.67 (95% CI 0.63 to 0.70), respectively. Those for the 3 simulated external validation data sets ranged from 0.64 to 0.68.
CONCLUSION: The prediction model performs well with good internal and external validity for identifying patients with DM at risk of unplanned 30-day readmission.

Readmission is defined as unplanned hospital admission within a prespecified period (eg, 30 days) (1, 2). Thirty-day hospital readmission rates are increasingly used for both quality improvement and cost control (3). Adult patients with diabetes mellitus (DM) represent 10% to 25% of all 30-day unplanned hospital readmissions (4). DM-related readmission is common in patients with DM (5, 6); in addition, patients with DM-related index admission diagnosis also have a higher readmission rate compared to those with other diagnoses (7). However, predictors of readmission among patients with DM whose index admission is DM-related are not well-studied; 30-day readmission prediction models (5, 8, 9) often included other types of index admission diagnoses (4, 5). Demographic, socioeconomic, inpatient factors, and comorbidities may be different in patients whose index admission diagnosis is DM-related (4, 5). Poorly controlled DM is commonly associated with ischemic heart disease (IHD) (10), peripheral vascular disease (PVD) (11), and renal failure (12). It may be appropriate to determine the individual effect of each comorbidity on 30-day readmission among DM patients with index admission diagnosis related to DM; a previous study has attempted to predict readmission based on consolidated Charlson Comorbidity Index (13) (CCI).

Routine inpatient medical records can be used to identify patients at risk of hospital readmission and develop prediction models (14, 15). Although prediction models can objectively support healthcare professionals to make clinical decisions and interventions (16), their performance should be evaluated using independent data (17). When independent data are not available, simulated data may be generated to compare the results with the prediction model (18). LACE (13) and PCi (19) models included 4 [length of stay (LOS), acuity of the hospital admission (emergency vs nonemergency), CCI score, and number of visits to emergency department in the past 6 months] and 2 (polypharmacy and CCI score) variables, respectively: neither model has been externally validated in DM patients whose index admission is DM-related.

The primary objective of this study is to develop a prediction model of 30-day unplanned hospital readmission among adult patients with DM with a DM-related index admission. Secondary aims are to (1) internally validate the prediction model using 10-fold cross-validation; (2) externally validate the prediction model by simulating 30-day readmission rates and patient characteristics based on results of published studies (8, 13, 20); and (3) compare the performance of the prediction model with LACE (13) and PCi (19).

Method

Administrative data of the National University Hospital, Singapore, from January 2008 to December 2015 were retrospectively extracted from the registration system, electronic health records, and pharmacy database. Index admission was defined as the first admission during the study period. The sample consisted of inpatients aged ≥ 18 years at the time of index admission with hospital stay more than 24 hours and survived to hospital discharge. Patients were included if the primary or secondary diagnosis of index admission was DM-related based on the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM, from 2007 to 2011) and International Classification of Diseases, Tenth Revision (ICD-10AM, 2012 onward). These included but are not limited to diabetic ketoacidosis, hypoglycemia, impaired glucose regulation, abnormal glucose tolerance with previous history of DM, elevated blood glucose level with previous history of DM, and preexisting DM in pregnancy. Exclusion criteria included DM arising during pregnancy, diabetes insipidus, death during index admission, and missing data on comorbidities and number of drugs administered.

Dependent Variable

Being discharged and readmitted within 30 days without a planned second admission was the outcome of interest. Readmission within 30 days was calculated from the discharge date of index hospitalization.

Independent Variables

Demographic characteristics included sex, age, race, and residential status. Medical history included number of surgical operations, emergency department visits in the past 12 months prior to the index admission, and number of drugs on discharge. Admission and discharge information included LOS (1-3 days vs ≥4 days) in the hospital, LOS in the intensive care unit, type of hospital admission (emergency or elective), hospital ward subsidy (as a marker of socioeconomic status), discharge to home or stepdown care facilities, and comorbidities listed in the CCI score (21).

Predictors in LACE and PCi Models

In the LACE (13) model, logarithmic and square root transformations were performed for LOS and number of visits to emergency department during the previous 6 months, respectively. In the PCi model (19), polypharmacy was defined as having ≥6 medications vs 1 to 5 medications, and the CCI score was coded as a binary variable (≥5 vs 0-4).

Statistical Analysis

Chi-square tests were used to compare 30-day readmission status for categorical variables. Mann-Whitney U tests were used to compare 30-day readmission status for CCI and number of drugs, which were not normally distributed. The initial multivariable logistic model included all significant variables in the bivariate analysis after considerations of collinearity and linearity. Nested models were compared using the likelihood ratio test, and variable selection of the final model was based on the principle of parsimony. The Hosmer-Lemeshow test was used to determine the goodness-of-fit of the prediction model. The area under the receiver operating characteristic curve (AUROC) based on 10-fold (22) cross-validation was generated to internally validate the prediction model (17).

External validation of the prediction model involved generating simulated data with 20 000 observations for 3 different scenarios mimicking real-life data. The first external validation data were simulated from patient characteristics and regression coefficients from the prediction model. LOS was simulated using a zero-truncated negative binomial distribution with parameter 4.0 and probability 0.5 to give a median LOS of 4. Number of drugs was simulated assuming a Poisson distribution with mean 8.7 and dispersion parameter 2.0. Binary IHD and PVD were generated assuming binomial distributions, both with probability of 0.05. The second external validation data were simulated assuming the prevalence of readmission, patient characteristics, and regression coefficients reported by Enomoto et al (20). As information on the number of drugs was not available from the published study, its distribution was simulated as in the prediction model. The third external validation data were simulated using a combination results from 2 studies—the distribution for 3 of the predictors were simulated based on LACE (13) while that for PVD was simulated from the information provided by Eby et al (8). Readmission rates for the 3 external validation data sets 1, 2, and 3 (Table 1) were 11.8%, 20.4%, and 10.3%, respectively, corresponding to those of the prediction data by Enomoto et al (20) and Eby et al (8).

Table 1.

Characteristics of predictors from source and simulated data

Model	Median LOS	% IHD	% PVD	Mean number of drugs	30-day readmission rate
External validation data 1	4 (4)	4.1 (4.9)	6.0 (5.0)	8.8 (8.7)	11.0 (11.8)
External validation data 2	4.8a (5.7)	24.1 (24.2)	8.7 (8.6)	8.8 (8.7b)	18.0 (20.4)
External validation data 3	5c (6)	9.0c (8.9)	2.1d (2.2)	4c (5.5)	10.0$ (10.3)

Data are the summary statistics from the selected studies, while those of the simulated data are presented in parenthesis.

Abbreviations: IHD, ischemic heart disease; LOS, length of stay; PVD, peripheral vascular disease.

aEnomoto et al (20) summarized LOS in terms of mean. In the simulation, a mean LOS of 5.64 was assumed.

bThis is simulated based on number of drugs from the prediction model, whereas the other variables are simulated based on results of Enomoto et al (20).

cSimulated based on the results of LACE (13).

dSimulated based on results of Eby et al (8).

Calibration plots of observed vs predicted probability were generated with points shown at the deciles of predicted probability for each plot; the corresponding observed probability was calculated as the proportion readmitted at each decile cutoff (23).

The calibration-in-the-large statistic was presented to determine whether the average predicted probability overestimated or underestimated the average observed probability (17). The calibration slope was estimated to determine whether the predicted probabilities showed the same variation as the observed probabilities (17). C-statistics of the simulated data were estimated to determine whether observations who were readmitted had higher predicted probabilities than those who were not readmitted (17).

The distribution of the predicted risks of the prediction model was also compared with those of the 3 simulated validation data by generating membership regression models with dependent variable coded 1 to denote individual participants from the prediction model, and 0 to denote individual participants from the simulated validation data (17). The ratio of SD of the linear predictor (LP) as a measure of discriminative ability and difference in mean LP as a measure of the difference in predicted outcome frequency derived from the membership models were compared. In addition, the C-statistics of the membership models were presented to distinguish the prediction model from the individual simulated data.

The performance of the prediction model was compared with LACE (13) and PCi (19) using AUROC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR−), and F-score.

All statistical analyses were performed using Stata version 16 with the exception of the zero-truncated negative binomial distribution which was simulated using R. The level of significance was set at 0.05 assuming a 2-sided test. The study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement, a 22-item checklist for reporting (24).

Results

Descriptive Statistics of Patient Characteristics

Demographic characteristics of the study cohort involving 2355 patients (Fig. 1) were summarized by 30-day readmission status (Table 2). Overall, 11.0% of the patients had unplanned readmission within 30 days. Malay (25.5%) and Indian (15.4%) patients were overrepresented in the study cohort compared to the local adult population census, where the percentages were 13.5% and 9.0%, respectively, in 2020 (25). Ethnic distributions were similar in not readmitted vs readmitted patients. Most patients were Singapore residents, with at least 1 emergency department visit in the past 12 months and had stayed in government-subsidized wards. Their most common comorbidities were diabetes-related chronic complications (67.2%), followed by renal disease (23.3%) (Table 3).

Figure 1.

Selection of study cohort.

Table 2.

Characteristics of 2355 participants by 30-day readmission status

Demographic characteristics	Total (n = 2355, 100%)	Not readmitted (n = 2095, 89.0%)	Readmitted (n = 260, 11.0%)	P-value
Sex				0.623
Female	1116 (47.4)	989 (47.2)	127 (48.9)
Male	1239 (52.6)	1106 (52.8)	133 (51.1)
Age, years				<0.007
≤64	1529 (64.9)	1380 (65.8)	149 (57.3)
≥65	826 (35.1)	715 (34.2)	111 (42.7)
Ethnicity				0.484
Chinese	1171 (49.7)	1034 (49.3)	137 (52.7)
Malay	601 (25.5)	545 (26.00)	56 (21.5)
Indian	362 (15.4)	320 (15.3)	42 (16.2)
Others	221 (9.4)	196 (9.4)	25 (9.6)
Residential status				0.712
Nonresident	214 (9.1)	192 (9.2)	22 (8.5)
Resident	2141 (90.9)	1903 (90.8)	238 (91.5)
Medical history
Surgical operation, n				<0.001
0	1755 (74.5)	1593 (76.1)	162 (62.3)
≥1	600 (25.5)	502 (23.9)	98 (37.7)
Emergency department visits in the past 12 months, n				0.248
0	255 (10.9)	223 (10.7)	32 (12.3)
1	1808 (76.7)	1619 (77.2)	189 (72.7)
≥2	292 (12.4)	253 (12.1)	39 (15.0)
Number of drugs, median (IQR)	8 (5, 11)	8 (5, 11)	11 (8, 14)	<0.001
Admission and discharge information
Length of stay, days				<0.001
1-3	1141 (48.4)	1059 (50.5)	82 (31.5)
≥4	1214 (51.6)	1036 (49.5)	178 (68.5)
ICU length of stay, days				0.003
0	2255 (95.8)	2015 (96.2)	240 (92.3)
≥1	100 (4.2)	80 (3.8)	20 (7.7)
Type of hospital admission				0.880
Nonemergency	149 (6.3)	132 (6.3)	17 (6.5)
Emergency	2206 (93.7)	1963 (93.7)	243 (93.5)
Type of wards accommodation				0.322
Private	357 (15.2)	323 (15.4)	34 (13.1)
Subsidized	1998 (84.8)	1772 (84.6)	226 (86.9)
Discharge type				0.281
Discharged home/discharged to home with day rehab or medical appointment	2,25 (94.4)	1984 (94.6)	241 (92.7)
Discharged to other hospitals or nursing homes/discharged against medical advice/absconded	130 (5.6)	111 (5.4)	19 (7.3)

Unless otherwise indicated, data are given as n (%).

Abbreviations: ICU, intensive care unit; IQR, interquartile range.

Table 3.

Comorbidities of participants by 30-day readmission status

Comorbidity	Total n (%)	Not readmitted (n = 2095, 89.0%)	Readmitted (n = 260, 11.0%)	P-value
CCI, median (IQR)	3 (1, 4 )	3 (1, 4)	3 (2, 4)	<0.001
Diabetes chronic complication	1582 (67.2)	1379 (65.8)	203 (78.1)	<0.001
Renal disease	550 (23.3)	464 (22.1)	86 (33.1)	<0.001
Heart failure	162 (6.9)	140 (6.7)	22 (8.5)	0.299
Peripheral vascular disease	142 (6.0)	112 (5.3)	30 (11.5)	<0.001
Ischemic heart disease	97 (4.1)	67 (3.2)	30 (11.5)	<0.001
Liver disease	58 (2.5)	50 (2.4)	8 (3.1)	0.497
Dementia	34 (1.4)	28 (1.3)	6 (2.3)	0.215
Chronic obstructive pulmonary disease	21 (0.9)	18 (0.9)	3 (1.1)	0.633
Peptic ulcer disease	21 (0.9)	15 (0.7)	6 (2.3)	0.010
Any tumor	19 (0.8)	15 (0.7)	4 (1.50)	0.149
Cerebrovascular disease hemiplegia	12 (0.5)	11 (0.5)	1 (0.4)	1.000
Metastatic tumour	6 (0.30)	4 (0.20)	2 (0.8)	0.135
Connective tissue disease	5 (0.2)	5 (0.2)	0 (0.0)	1.000
Acquired immune deficiency syndrome	2 (0.1)	2 (0.1)	0 (0.0)	1.000

Unless otherwise indicated, data are given as n (%).

Abbreviations: CCI, Charlson comorbidity index; IQR, interquartile range.

Bivariate analysis identified nine significant risk factors (age, LOS, number of surgical operations, number of drugs, diabetes chronic complications, IHD, PVD, peptic ulcer disease, and renal disease) of 30-day unplanned hospital readmission. These were included in the initial logistic regression model.

Prediction Model

The final prediction model (LIPiD) included 4 significant predictors: LOS, IHD, PVD, and number of drugs (Table 4) after removing the insignificant predictors in multivariable regression. Those with LOS ≥ 4 days had a 45% increase in odds of 30-day readmission [odds ratio (OR) 1.45 (95% CI 1.07, 1.95)] as compared to patients with LOS 1 to 3 days. Patients who had IHD had more than twice the odds [OR 2.31 (95% CI 1.43, 3.72)] of being readmitted than those without IHD. Patients with PVD had 58% (OR 1.58 (95% CI 1.01, 2.48)] higher odds of being readmitted as compared to those without PVD. Number of drugs was also positively associated with readmission [OR 1.09 (95% CI 1.06, 1.12)]. The Hosmer-Lemeshow test for LIPiD suggested a good fit (P = 0.37). The 10-fold cross-validation (mean AUROC: 0.67, bootstrap bias corrected 95% CI 0.63, 0.70) demonstrated a relatively reasonable performance (26) of the prediction model (Fig. 2).

Table 4.

Significant risk factors of 30-day unplanned hospital readmission in the bivariate and multivariable analyses

Predictors	Bivariate analysis			Multivariable LIPiD model (n = 2355)
	OR	95% CI	P-value	OR	95% CI	P-value
Age (years)
≤64	Reference
≥65	1.44	1.11, 1.87	0.007
Surgical operations, n
0	Reference
≥1	1.92	1.47, 2.52	<0.001
Number of drugs, median (IQR)	1.12	1.09, 1.15	<0.001	1.09	1.06, 1.12	0.001
Length of stay, days
1-3	Reference			Reference
≥4	2.22	1.69, 2.92	<0.001	1.45	1.07, 1.96	0.016
Diabetes chronic complication	1.85	1.36, 2.51	<0.001
Renal disease	1.73	1.32, 2.29	<0.001
Peripheral vascular disease	2.31	1.51, 3.53	<0.001	1.58	1.01, 2.47	0.042
Ischemic heart disease	3.94	2.51, 6.20	0.001	2.31	1.43, 3.72	0.001
Peptic ulcer disease	3.28	1.26, 8.52	0.015

Abbreviation: IQR, interquartile range.

Figure 2.

Area under the receiver operating characteristic curve of LIPiD model based on 10-fold cross-validation. Mean Cross-Validated Area Under the ROC Curve (cvAUC) (solid curve) and k-fold receiver operating characteristic curves (dashed curves).

Results on External Validation of LIPiD Using Simulated Data

The estimated regression coefficients from external validation data 1 were similar to those of the prediction model although the 95% CIs were narrower due to the increased simulated sample size (Table 5). For external validation data 1, LOS ≥ 4 days [log-odds (95% CI 0.37 0.28, 0.46)], IHD [log-odds 1.29 (95% CI 1.10, 1.47)], PVD [log-odds 0.37 (95% CI 0.20, 0.55)], and number of drugs [log-odds 0.17 (95% CI 0.16, 0.18)] were significantly associated with 30-day readmission. The 4 predictors of external validation data 2 were also significant. However, as expected, the estimated regression coefficients of IHD [0.10 (95% CI 0.02, 0.18)] and PVD [0.14 (95% CI 0.02, 0.26)] were weaker as compared to those in our prediction model, since external validation data 2 was simulated based on the findings of Enomoto et al (20), which also reported lower ORs for both IHD and PVD but higher incidence of 30-day readmission. The regression coefficient for IHD was also lower for external validation data 3 where we simulated its prevalence based on information provided for acute coronary syndrome and atrial fibrillation in the supplementary materials (Appendix A) of the LACE (13) study. Interestingly, neither acute coronary syndrome nor atrial fibrillation were significant predictors of readmission in the LACE study (13). Thus, this suggest that the prediction model was robust in a variety of clinical settings, including instances where the prevalence rates of IHD (20) or 30-day readmission (20) were higher than the source data as well as when the PVD rate was lower (8).

Table 5.

Estimated coefficients and 95% CIs of the prediction and external validation data

Predictors	Prediction model (95% CI)	External validation data 1 (95% CI)	External validation data 2 (95% CI)	External validation data 3 (95% CI)
Length of stay	0.37 (0.06, 0.67)	0.37 (0.28, 0.46)	0.40 (0.33, 0.47)	0.37 (0.28, 0.47)
Ischemic heart disease	0.83 (0.35, 1.31)	1.29 (1.10, 1.47)	0.10 (0.02, 0.18)	0.19 (0.04, 0.35)
Peripheral vascular disease	0.46 (0.15, 0.90)	0.37 (0.20, 0.55)	0.14 (0.02, 0.26)	0.40 (0.12, 0.68)
Number of drugs	0.08 (0.05, 0.11)	0.17 (0.16, 0.18)	0.09 (0.08, 0.10)	0.15 (0.14, 0.16)
Constant (model intercept)	−3.25 (−3.58, −2.92)	−3.96 (−4.11, −3.81)	−2.51 (−2.61, −2.41)	−3.40 (−3.51, −3.28)

Points on the validation plots were relatively close to the origin, indicating low observed and predicted probabilities of readmission in most instances (Fig. 3). When the LIPiD model was applied to the prediction sample, several observed frequencies per decile of predicted probabilities (as indicated by circles) were on the line of equality indicating good agreement (23). However, there were a few points below the line of equality suggesting the predictions of 30-day readmission were slightly higher at the extreme (23). The observed and predicted probabilities for external validation data model 1, external validation data model 2 [data simulated from Enomoto et al (20)], and external validation data model 3 [data simulated from published results of LACE (13) and Eby et al (8)] were also in agreement, and hence close to line of equality especially for earlier deciles, while those for the later deciles tended to have higher predictive probabilities. The 3 plots suggest a relatively good performance of the LIPiD model when validated against a variety of real-life clinical settings.

Figure 3.

Observed vs predicted probabilities of 30-day readmission of the prediction model and 3 external validation data models. a = calibration-in-the-large, b = calibration slope, and c = c-statistic. Prediction model (LIPiD), a = −0.21, b = 0.98, and c = 0.68. External validation 1: a = −0.24, b = 0.97, and c = 0.68. External validation 2: a = −0.14, b = 0.97, and c = 0.64. External validation 3: a = −0.31, b = 0.93, and c = 0.68.

The calibration-in-the-large statistic (Figure 3) was −0.21 for the prediction model, indicating that LIPiD slightly overestimated 30-day readmission (17). This was also true for the 3 external validation data models. The calibration slope for the prediction model and all the external validation data models were close to 1, suggesting that the predicted risks were proportionally accurate (17). The C-statistics of the prediction model and external validation data 1 and 3 were 0.68 and slightly higher than those for external validation data 2 (Fig. 3).

The SD of the LP for external validation data model 2 showed slight variations from that of the prediction model while those for external validation data models 1 and 3 were more similar (Fig. 4A). This suggested external validation models 1 and 3 had the same discriminating ability as the prediction model (17). The mean LP for external validation models 1 and 3 were also similar to that of the prediction model (Fig. 4B), suggesting similar model performance in terms of predicted outcome frequency (17). The C-statistic of the membership model implied that the samples for prediction and external validation data models 1 were similar (17).

Figure 4.

(A) Relative difference in standard deviation of the linear predictor comparing validation vs prediction sample. (B) Difference in mean of the linear predictor comparing validation vs prediction sample.

Model Comparison: LIPiD, LACE, and PCi Using Prediction Data

The discrimination performance of LIPiD [AUROC 0.68 (95% CI 0.66, 0.70)] was significantly better than LACE [AUROC 0.65 (95% CI 0.63, 0.67), P = 0.041] and PCi [AUROC 0.60 (95% CI 0.58, 0.62), P < 0.001], respectively (Table 6). At a readmission rate of 11%, the sensitivity and specificity of LIPiD were 62% (95% CI 55%, 67%) and 65% (95% CI 63%, 67%), respectively. LIPiD yielded higher values than PCi in terms of specificity, PPV, LR+, LR−, and F-score. Although the sensitivity of LIPiD was lower than PCi, its specificity was more than twice that of PCi. The 3 models did not yield a high PPV owing to the low prevalence of readmission.

Table 6.

Performance indicators of prediction models at 30-day readmission rate of 11%

Model	AUROC (95%CI)	Sensitivity, % (95%CI)	Specificity, % (95%CI)	PPV, % (95%CI)	NPV, % (95%CI)	LR+	LR−	F-score
LIPiD	0.68 (0.66, 0.70)	62 (55, 67)	65 (63, 67)	18 (16, 19)	93 (92, 94)	1.81	0.58	0.28
LACE	0.65 (0.63, 0.67)	57 (51to 63)	64 (61, 65)	16 (14, 18)	92 (91, 93)	1.59	0.66	0.25
PCi	0.60 (0.58, 0.62)	89 (84, 92)	28 (26, 30)	13 (12, 14)	95 (93, 96)	1.24	0.38	0.23

Abbreviations: AUROC, area under the receiver operating characteristic curve; LR+, positive predictive value; LR−, negative predictive value; NPV, negative predictive value; PPV, positive predictive value

Discussion

The LIPiD prediction model identified 4 predictors of unplanned 30-day readmission among DM patients with DM-related index admission: LOS, IHD, PVD, and number of drugs. The finding that LOS was a significant predictor corroborates with that observed from 2 prediction models for privately insured (8) and Medicare patients with DM (9). The 2 publications included all patients with type 2 DM regardless of diagnoses at index admission. Thus, LOS appeared to be an important risk factor for 30-day readmission in DM patients irrespective of the actual diagnosis at initial admission, case-mix, or type of DM. It is also 2 of the 4 predictors of the LACE model (13).

Consistent with the reviews by Robbins et al (27) and Soh et al (4), who reported numerous comorbidities as independent predictors of 30-day unplanned hospital readmission in patients with DM, we found IHD and PVD to be important predictors. The combination of hyperglycemia, insulin resistance, and free fatty acid excess can possibly lead to the development of IHD (28). Thus, it is necessary to ensure patients with DM and IHD adequately control their glycemia and lipids. The UK Prospective Diabetes Study had shown that hyperglycemia (as indicated by hemoglobin A1c) was independently associated with an increased risk for PVD (29), thus connecting overall glycemic control to the development of PVD. A further study reported that PVD increased the risk of 30-day unplanned hospital readmission among patients with DM and was associated with prolonged LOS (20). Consistent glucose monitoring was associated with improvements in glycemic status (30), which may thus prevent the development of PVD. The odds of readmission were found to increase with number of drugs in this study, which may point to the overall disease burden. A local study demonstrated the number of medications and poor compliance to be predictors of readmission (31). Patients on multiple drugs who are noncompliant may not receive the desired therapeutic benefits resulting in disease progression and readmission. Number of drugs is also a predictor in the PCi model (19), although the variable was classified differently from the LIPiD model.

The internal validity of the LIPiD model was acceptable and reproducible. Our external validation used 3 sets of simulated data reflecting a variety of real-life clinical scenarios. The 4 predictors of 30-day unplanned hospital readmission identified by LIPiD remained significant in all models with the C-statistic ranging from 0.64 to 0.68. LIPiD performs well not only for the simulated data with readmission rate of around 10% (external validation data 1 and 3) but also when the readmission rate was doubled in external validation data 2, as well as with varying ranges of IHD from 9% to 24% and PVD from 2% to 9%.

We considered individual comorbidity as possible predictor of 30-day readmission rather than CCI as in the LACE and PCi models. In the derivation of CCI, more weight is allocated to more severe comorbidities such as AIDS and metastatic tumor, which are not known predictors of 30-day readmission among patients with DM, although these have been demonstrated to be predictors of 10-year survival (21). Conversely, CCI underweights IHD and PVD, which are associated with DM. Hence, the use of CCI may be less relevant for identifying 30-day unplanned hospital readmission among DM patients whose index admission was attributed to DM. In addition, local studies by Low et al highlighted the limitation of LACE in predicting 30-day readmission (32, 33). Both studies yielded higher AUROCs than the LACE model after including blood test results (32) and markers of hospitalization severity (33).

Electronic hospital records may provide a rich source of secondary data for research but may not contain all the information needed for a particular study. We acknowledge the following limitations in our study. First, data on insulin therapy, a possible predictor of 30-day unplanned hospital readmission (4), were not available for analysis. Second, the diagnosis at admission was too diverse for more meaningful subgroup analysis. Other limitations of this study included the lack of information on medication compliance and blood test results such as hemoglobin A1c prior to index admission, which are known predictors of 30-day readmission (34, 35). Other relevant risk factors such as psychosocial characteristics (36) including history of depression or anxiety and social support were also unavailable, as were data on continuity of care after discharge (37). In addition, the prediction model did not yield a high PPV possibly due to the low prevalence of readmission (11%). Thirty-day readmission rate among patients with DM varies from country to country and usually hovers between 10% and 20% (8, 38). Thus, the model may be applicable to diverse clinical settings as demonstrated by our external validation, which assumed 30-day readmission rates of between 10% and 20% and prevalences of IHD and PVD ranging from 5% to 24% and 2% to 9%, respectively. Future studies may be conducted to explore the clinical utility of the prediction model for readmission > 20%.

Conclusion

A validated prediction model, LIPiD, demonstrated that unplanned 30-day readmission among patients with DM whose index diagnosis was DM-related was associated with LOS, IHD, PVD, and number of drugs administered. The model has good internal and reasonable external validity, and the findings may help healthcare providers identify patients at high risk of readmission.