Development of a prediction model for 30-day acute readmissions among older medical patients: the influence of social factors along with other patient-specific and organisational factors.

Background: Readmission rate is one way to measure quality of care for older patients. Knowledge is sparse on how different social factors can contribute to predict readmission. We aimed to develop and internally validate a comprehensive model for prediction of acute 30-day readmission among older medical patients using various social factors along with demographic, organisational and health-related factors.
Methods: We performed an observational prospective study based on a group of 770 medical patients aged 65 years or older, who were consecutively screened for readmission risk factors at an acute care university hospital during the period from February to September 2012. Data on outcome and candidate predictors were obtained from clinical screening and administrative registers. We used multiple logistic regression analyses with backward selection of predictors. Measures of model performance and performed internal validation were calculated.
Results: Twenty percent of patients were readmitted within 30 days from index discharge. The final model showed that low educational level, along with male gender, contact with emergency doctor, specific diagnosis, higher Charlson Comorbidity Index score, longer hospital stay, cognitive problems, and medical treatment for thyroid disease, acid-related disorders, and glaucoma, predicted acute 30-day readmission. Area under the receiver operating characteristic curve (0.70) indicated acceptable discriminative ability of the model. Calibration slope was 0.98 and calibration intercept was 0.01. In internal validation analysis, both discrimination and calibration measures were stable. Conclusions: We developed a model for prediction of readmission among older medical patients. The model showed that social factors in the form of educational level along with demographic, organisational and health-related factors contributed to prediction of acute 30-day readmissions among older medical patients.

Introduction

Acute readmission within 30 days from hospital discharge is a major concern in quality of healthcare for older patients, with consequences for both patient and healthcare financing.1 In addition, readmissions are associated with increased risk of in-hospital complications, such as pressure ulcers and delirium.2 Several countries report consistently high rates of readmission among older patients.3 A Danish study of older patients with comorbidities reported a 30-day readmission rate of approximately 25%.4 When older patients are transferred from hospital to home, they are in a state of increased vulnerability due to impaired health and the influence of multiple stressors. Besides recovering from the specific condition that led to hospitalisation, newly discharged patients are often influenced by the physiological distress of experiencing illness and hospitalisation.5

One study has shown that the ability to manage disease is associated with socioeconomic factors such as income, wealth and educational level,6 and numerous studies on the association of specific risk factors with 30-day readmission have showed that both health-related and social factors are risk factors for acute readmissions among older patients.7 8 For older patients, readmission diagnosis often differs from the previous admission, indicating a high level of comorbidities, vulnerability related to transfer of care and to the fact that the majority of older patients do not fit into a single disease category.5 9 A number of studies have focused on prediction of readmission among non-selected older medical patients who are 65 years or older.10–14 All studies had relatively poor discriminative performance (c-statistics between 0.55 and 0.66) indicating that developing a prediction model for acute 30-day readmission among older medical patients is complex and should include more predictors than can be obtained from most administrative hospital databases.10–14 One study including more than a million older patients in Hong Kong focused on admissions of both inpatient and outpatient clinics,15 showing excellent discriminative ability based on predictors concerning sociodemographic factors, healthcare utilisation and diagnosis.15 Similarly, a prediction study targeting older patients found that ethnicity and healthcare insurance contributed to prediction of readmission.16 However, among patients in specific disease groups, other social measures such as educational level and income have additionally influenced readmission.17 18 The present study was conducted in Denmark, which has high-quality register information about social factors like civil status, children, income and education,19–22 which permits the investigation of the role of a broad range of social factors.

This study aims to develop a comprehensive model for predicting 30-day readmission among older medical patients. It tests the predictive use of social, demographic, organisational and health-related factors. Further, we will internally validate the model.

Method

This study was informed by the Prognosis Research Strategy framework23 and reported in line with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.24 The study was designed as a prospective observational study.

Study population

Patients were sampled consecutively from the Department of Medicine at Holbæk University Hospital, Denmark, which has acute care capacity. We included patients aged 65 years or older, who were discharged from the Department of Medicine, including the medical specialities of internal medicine, nephrology, cardiology, respiratory medicine and neurology. Further, data included patients who lived in one of three surrounding municipalities and who were consecutively screened for readmission risk factors during the period from February to September 2012 (figure 1). Patient screening was obtained as part of a previous randomised controlled intervention study using risk of readmission as main outcome.4 This previous study had found no difference in readmission rate between intervention and control groups.4 In our study, we included all patients screened for the previous study; the group included both patients who were categorised as eligible and patients categorised as non-eligible in the original randomised study. We included all screened patients, but excluded patients if the screening questionnaire was incomplete (figure 1).

Figure 1

Patient flow diagram illustrating how patients were included or excluded in this study

Screening data from the previous study were supplemented with administrative register data to obtain additional information about potential demographic, social, organisational and health-related candidate predictors, as well as readmission data. Data were obtained from several data sources as described below.25

Outcome measure

The outcome measure was acute all-cause readmission within 30 days to any Danish hospital obtained from the Danish National Patient Register.26

Candidate predictors

Social factors

We included patients’ educational level, obtained from the Danish Education Register20; income, obtained from the Danish Income Statistics Register27; whether patients were living alone or with a spouse, obtained from the Civil Registration System21; and the Charlson Comorbidity Index of the spouse (see section about health factors). We obtained information about children, from the Danish Fertility Database and the Danish Adoption Register,19 22 and we obtained information regarding the children’s residence from the Civil Registration System.21 We categorised patients as having children living close by if at least one adult child was living in the same or in a neighbouring municipality as the older patient.

Demographic factors

Information regarding age and gender was obtained from the Civil Registration System,21 and size of city was provided by Statistics Denmark.

Organisational factors

We included the following: contact with general practitioners and the after-hours emergency service doctor 1 month before indexed admission, obtained from the Danish National Health Service Register28; and prior admission within 3 months of indexed admission, obtained from the Danish National Patient Register.26 Distance between patient home and general practitioner clinic was based on information from the Danish Civil Registration System and the Danish National Health Service Register.21 28 Municipal home service was obtained from municipal reporting on service indicators,29 and was divided into personal and practical assistance. Personal assistance includes assistance with personal hygiene, whereas practical assistance covers cleaning or other practical healthcare needs.

Health-related factors

We included 21 different potential predictors related to patient health. Among these were diagnosis, previous admissions and length of stay obtained from the Danish National Patient Register26; medication obtained from the Danish National Prescription Registry30; and clinically assessed cognitive problems and loss of activities of daily living (ADL) obtained from clinical screening. Cognitive problems was derived from clinical assessment of “does the patient’s behaviour indicate cognitive problems (not diagnosed dementia)?” Loss of ADL was derived from the clinical assessment of “has the patient experienced greater ADL impairment compared with prior to admission?” Diagnosis was categorised according to the main groups of ICD-10. The three most frequent primary diagnoses were included in the analysis: diagnoses related to atypical symptoms, the respiratory system or the circulatory system. Patients were categorised according to whether their specific diagnosis related to the specific group or not. The patients’ Charlson Comorbidity Scores were calculated using information about primary and secondary diagnoses from all hospital contacts up to 10 years before index admission. Information on prescribed medication, found using Anatomical Therapeutic Chemical (ATC) codes,31 was used to identify 22 chronic conditions. Polypharmacy was defined as the use of five or more different prescribed drugs within a period of 3 months prior to admission, defined as the number of unique drug ATC codes purchased.

Analysis

We included only candidate predictors with a sufficient number of observations, that is, at least 10 readmissions for each predictor category. Consequently, we included 33 candidate predictors in the model search. The analysis was divided into three steps: (1) descriptive univariate logistic regression analyses exploring the crude associations between each predictor and outcome; (2) backward stepwise logistic regression model selection including only those predictors showing significance on a 10% significance level determined by a likelihood-ratio test32; (3) performance evaluation of the model developed in step 2 in terms of its discriminative ability and calibration (see next section).

Missing values

We expected a low level of missing observations due to the high coverage of Danish registers. However, we encountered missing values for distance between general practitioner (12%) and patient address, and for educational level (7%). Thus, in order to gain statistical strength, we made imputations before performing multiple logistic regression analysis. For distance between general practitioner and patient address, we calculated the mean distance stratified by city size and used this value for missing values, while for educational level, missing values were imputed as primary school completed since more than half of the Danish population in the included age range had no further education after primary school.

Non-participants

We have no way of verifying whether all potential eligible patients who were discharged from the Medical Department at Holbæk University Hospital in the study period were screened. Based on data obtained from the Danish National Patient Register, we identified patients who were discharged from the Medical Department at Holbæk University Hospital in the study period. We performed descriptive analysis of basic characteristics on the total cohort of discharged patients (n=2026 patients) compared with the cases where screening were intended though not completed (n=336 patients) and the cases where screening were completed (n=770).

Performance measures

We assessed the overall model performance by calculating the Brier score, with a range between 0 for a perfect prediction model and 0.25 for a non-informative model.33 In addition, we assessed the discriminative ability of the model by calculating the area under the receiver operating characteristic curve (AUC).33 AUC can be interpreted as the probability that a readmitted patient is assigned a higher probability of readmission than a randomly chosen patient who is not readmitted.34 Generally, an AUC of <0.7 represents poor discrimination, 0.7–0.8 acceptable discrimination, 0.8–0.9 excellent discrimination and 0.9–1.0 outstanding discrimination.35 Calibration refers to the agreement between the observed and the predicted frequencies of a given outcome.34 We assessed calibration by constructing calibration plots that illustrated observed proportions of readmission in four groups separated by quantiles of risk versus predicted risks produced from the prediction model. We also derived the calibration slope and intercept. In a well-calibrated model, the intercept will be 0 and the slope will be 1.34

To assess model optimism, we performed bootstrapping and compared the Brier score, discrimination and calibration measures between the original model and bootstrapped estimates.35 Bootstrap samples were created by drawing random samples with replacement from the study population (1000 replications) and then evaluating model performance in the newly created sample.

Patient and public involvement

Prior research shows that lower levels of patient satisfaction are associated with a higher risk of 30-day readmission, and readmission can be regarded as a relevant outcome for both patients and healthcare providers.36 In this study, we investigated clinical and register-based predictors; hence, patient participation was not part of the design.

Results

Seven hundred and seventy patients were clinically screened for specific readmission risk factors and included in this study. Of these, 151 patients (20%) were acutely readmitted within 30 days from index discharge (figure 1). The overall group of patients (n=770) had a median age of 78 years and an equal number of men and women. The majority of patients had primary school as highest level of education (50%), children living close by (62%) and were living alone (55%). The patients’ health was characterised by high comorbidity; 60% had a Charlson Comorbidity Score of 2+. A majority of patients were medically treated for cardiovascular disease (80%), and 30% of the patients had a main diagnosis related to the circulatory system. For additional data, see online supplementary tables 1 and 2.

Univariate analyses showed that gender, education, contact with general practitioner and emergency service doctor the previous month, diagnosis related to atypical symptoms, diagnosis related to the respiratory system, Charlson Comorbidity Index, length of hospital stay, previous hospital admissions, polypharmacy and being medically treated for respiratory disease, glaucoma, acid disorders or pain were associated with readmission (results not shown). Two-way tables of potential predictors and readmission are available as online supplementary tables 1 and 2.

The final prediction model presented in table 1 shows 11 predictors of demographic, social, organisational and health-related characteristics. Both in univariate analysis and in the final multiple regression model, the clinical assessment of ‘cognitive problems’ seemed to be a strong predictor based on OR. Additionally, being male, having primary school as highest level of education, contact with emergency service doctor, primary diagnosis related to the respiratory system, Charlson Comorbidity Score of 4 or more, patients with five or more days at hospital at index admission, and patients who received medication for glaucoma, acid disorders or thyroid disease were all associated with a greater likelihood of acute readmission in the final model (table 1).

Table 1

Prediction model for 770 patients discharged from medical department, Holbæk University Hospital, Denmark, between February and September 2012

Variables		OR (95% CI)	P value
Social
Educational level	Higher education	–
	Vocational education	1.09 (0.55 to 2.19)	0.81
	Primary school	1.79 (0.94 to 3.38)	0.08
Demographic
Gender	Female	–
	Male	1.71 (1.15 to 2.54)	0.01
Organisational
Contact with emergency service doctor within 1 month before index admission	No	–
	Yes	1.43 (0.99 to 2.09)	0.06
Health related
Primary diagnosis related to the respiratory system	No	–
	Yes	1.57 (0.93 to 2.65)	0.09
Primary diagnosis related to atypical symptoms	No	–
	Yes	0.64 (0.39 to 1.03)	0.07
Charlson Comorbidity Index score	0–1	–
	2–3	1.37 (0.87 to 2.16)	0.18
	4+	1.88 (1.63 to 3.04)	0.01
Length of hospital stay	≤5 days	–
	6+ days	1.47 (1.01 to 2.15)	0.05
Clinical assessment of cognitive problems (not dementia)	No	–
	Yes	2.19 (1.10 to 4.36)	0.03
Medication for glaucoma	No	–
	Yes	2.20 (1.02 to 4.78)	0.05
Medication for acid disorders	No	–
	Yes	1.89 (1.29 to 2.77)	0.001
Medication for thyroid disease	No	–
	Yes	1.83 (0.94 to 3.57)	0.08

The table shows predictors, ORs and p values for the final prediction model.

Model fit

The overall model fit, as indicated by the Brier score, was 0.15 (table 2). The model’s ability to discriminate patients was acceptable based on AUC (0.70), please see table 2 and online supplementary figure 1. The calibration plot showed a reasonable fit with a minor outlier for the third quantile risk group (online supplementary figure 2) and calibration plot for 10 groups of predicted probabilities did not change this result (online supplementary figure 3). The calibration intercept was close to 0 (intercept=0.01), and the slope close to 1 (slope=0.98), indicating good calibration. The results from bootstrapping were similar to the model’s development, yet showed slightly higher median AUC (0.72).

Table 2

Summary performance measures for the model and in internal validation (bootstrapping)

	Measure	Final prediction model	Bootstrap analysisMedian value (range)
Overall performance	Brier score	0.15	0.14 (0.13–0.16)
Discrimination	AUC	0.70	0.72 (0.67–0.76)
Calibration	Calibration intercept	0.01	0.00 (−0.01 to 0.01)
	Calibration slope	0.98	0.98 (0.93–1.04)

AUC, area under the receiver operating characteristic curve.

Analysis of non-participants

A total of 2,026 patients above the age of 65 with residence in the relevant municipalities were discharged from Holbæk Medical Department during the study period. Only a smaller group of 1,136 patients had completed or partly completed screening data (figure 1), and it appears that 890 patients were not screened during the data collection period. A descriptive comparison between the total cohort of discharged patient (n=2026) versus partly screened (n=366) or screened patients (n=770) showed no systematic differences in terms of age, gender, municipality or educational level (online supplementary table 3).

Discussion

In this study, we used comprehensive clinical and registry data to develop a model for predicting readmission among older patients. The model showed good calibration and higher discrimination (AUC=0.70) than comparable prediction models related to non-selected older medical patients,10–14 and AUC just reached acceptable level of discrimination. Performance measures were consistent in the bootstrap analysis and thus did not indicate data overfitting. The discriminative capacity of the model is crucial since the model aimed to distinguish precisely those patients with a higher readmission risk.37 In the initial literature search, only one study showed better discriminative ability (AUC=0.82) in prediction of acute all-cause 30-day readmission among older medical patients.15 This previous model included predictors regarding sociodemographic and health-related factors, and focused on a group of patients discharged from both in-hospital and specialist outpatient clinics. However, prediction of readmission among the diverse group of older medical patients is complex, and previous prediction models with exclusive focus on older patients discharged from hospital were based on administrative hospital data and tested only a few social factors, such as use of healthcare security and education.10 12–14 In the current study, we have included various candidate predictors related to social factors, such as educational level, income, children living close by, and having a spouse with or without comorbidity. However, the final model in this study revealed that educational level was the strongest social candidate predictor of readmission. There is no consensus on how to measure the impact of social factors on health among older patient groups.38 Some studies include different social factors like healthcare insurance and ethnicity, which might be less relevant in a setting with universal tax-paid healthcare,39 and relatively low ethnic diversity,40 as in Denmark. Still, this study indicated social factors, in terms of educational level, have a crucial influence on risk of readmission, which should be acknowledged both in future studies of readmission risk prediction and in the design of readmission prevention interventions for older patients. Previous research indicates that differences in healthcare utilisation between educational groups can be associated with differences in the applied acute care and differences in patient ability to manage care.6 41 Yet, more knowledge is needed. Further, results showed that demographic, organisational and health-related factors as well as social factor need to be emphasised in clinical practice when identifying older medical patients at risk of readmission.

In addition, we have found that clinical screening based on one simple question related to the potential for cognitive problems (“does the patient’s behaviour indicate cognitive problems (not diagnosed dementia)?”) strongly contributed to the prediction of readmission; hence, clinically assessed cognitive problems not related to dementia more than doubled the risk of readmission. This clinical observation seemed to have an even stronger influence than previous studies using diagnosed cognitive problems as a predictor, thus indicating the importance of bedside clinical assessment. This result might reflect both the importance of clinical observations related to prediction of risk and that cognitive problems can be associated with a number of conditions including delirium. However, our data does not allow for further investigation into this predictor.

Studies have highlighted that readmission risk is influenced by both comorbidities treated outside hospital in the primary healthcare sector and organisational factors.8 42 43 To our knowledge, prediction studies have not previously included chronic condition information from non-hospital data sources. We found that three different disease categories based on prescribed medications predicted readmission. Older patients who suffer from glaucoma potentially struggle with impaired vision;44 yet, previous studies show inconsistent results on whether impaired vision increases risk of readmission.27 28 Potentially, the impact of glaucoma is related to other factors.44 We have not been able to find literature on the general outcomes of thyroid disease or thyroid disease therapy; yet, a meta-analysis has shown that the thyroid disease hypothyroidism is associated with increased all-cause mortality and hospitalisation of patients with heart failure.45 Likewise, studies have reported that acid-related disorders are associated with specific comorbidities and that medication is in itself associated with increased risk of readmission.

Predictors of readmission among older medical patients should be acknowledged when planning discharge and post-discharge care. Various interventions have been described and tested to prevent readmissions and increase quality of care in the transition from hospital to home.46

Study strengths and limitations

From our view, a strength of this study is that the clinical data were obtained and provided information not covered by the administrative hospital data. Further, our dataset included comprehensive register data that allowed us to investigate demographic, social and organisational candidate predictors, and it provided a low level of missing observations. Additionally, we performed internal validation in the form of bootstrapping in order to account for model overfitting or uncertainty.

The population group was not very large (n=770 patients), and due to an incomplete list of general practitioners, we had 12% missing observations for distance between general practitioner clinic and patient address. Likewise, in 7% of the included patients, we lacked information about their educational level. As previously stated, the screening data included in this analysis were obtained as part of a randomised controlled intervention study using risk of readmission as main outcome.4 The study did not show any effect of the tested intervention on readmissions; however, we cannot rule out that the risk of readmissions was changed for some of the 149 patients who received the intervention. We did not publish a study protocol prior to the data analysis phase, as recommended by TRIPOD, and despite bootstrap analysis showing good internal validity, the model should be tested on an external population of older medical patients.

Conclusion

In this study, we developed a model to predict acute readmission within 30 days after discharge among older medical patients. The model has an acceptable discriminative ability and good calibration. It succeeded in elucidating the influence of a social predictor, in terms of educational level as well as demographic, organisational and health-related predictors. Hence, it adds important new knowledge to prediction of readmissions among older patients. In predicting readmission risk, administrative hospital data should not and need not stand alone. By supplementing administrative hospital data with additional types of data, more accurate and useful models of readmission risk can be developed.