Comparison of Three Comorbidity Measures for Predicting In-Hospital Death through a Clinical Administrative Nacional Database.

BACKGROUND: Various authors have validated scales to measure comorbidity. However, the prognosis capacity variation according to the comorbidity measurement index used needs to be determined in order to identify which is the best predictor. AIMS: To quantify the differences between the Charlson (CCI), Elixhauser (ECI) and van Walraven (WCI) comorbidity indices as prognostic factors for in-hospital mortality and to identify the best comorbidity measure predictor.
METHODS: A retrospective observational study that included all hospitalizations of patients over 18 years of age, discharged between 2017 and 2021 in the hospital, using the Minimum Basic Data Set (MBDS). We calculated CCI, ECI, WCI according to ICD-10 coding algorithms. The correlation and concordance between the three indices were evaluated by Spearman's rho and Intraclass Correlation Coefficient (ICC), respectively. The logistic regression model for each index was built for predicting in-hospital mortality. Finally, we used the receiver operating characteristic (ROC) curve for comparing the performance of each index in predicting in-hospital mortality, and the Delong method was employed to test the statistical significance of differences.
RESULTS: We studied 79,425 admission episodes. The 54.29% were men. The median age was 72 years (interquartile range [IQR]: 56-80) and in-hospital mortality rate was 4.47%. The median of ECI was = 2 (IQR: 1-4), ICW was 4 (IQR: 0-12) and ICC was 1 (IQR: 0-3). The correlation was moderate: ECI vs. WCI rho = 0.645, p < 0.001; ECI vs. CCI rho = 0.721, p < 0.001; and CCI vs. WCI rho = 0.704, p < 0.001; and the concordance was fair to good: ECI vs. WCI Intraclass Correlation Coefficient type A (ICCA) = 0.675 (CI 95% 0.665-0.684) p < 0.001; ECI vs. CCI ICCA = 0.797 (CI 95% 0.780-0.812), p < 0.001; and CCI vs. WCI ICCA = 0.731 (CI 95% 0.667-0.779), p < 0.001. The multivariate regression analysis demonstrated that comorbidity increased the risk of in-hospital mortality, with differences depending on the comorbidity measurement scale: odds ratio [OR] = 2.10 (95% confidence interval [95% CI] 2.00-2.20) p > |z| < 0 using ECI; OR = 2.31 (CI 95% 2.21-2.41) p > |z| < 0 for WCI; and OR = 2.53 (CI 95% 2.40-2.67) p > |z| < 0 employing CCI. The area under the curve [AUC] = 0.714 (CI 95% 0.706-0.721) using as a predictor of in-hospital mortality CCI, AUC = 0.729 (CI 95% 0.721-0.737) for ECI and AUC = 0.750 (CI 95% 0.743-0.758) using WCI, with statistical significance (p < 0.001).
CONCLUSION: Comorbidity plays an important role as a predictor of in-hospital mortality, with differences depending on the measurement scale used, the van Walraven comorbidity index being the best predictor of in-hospital mortality.

1. Introduction

The association of two or more distinct diseases in the same individual at a rate higher than expected by chance is known as comorbidity [1]. It significantly influences different outcomes of hospital care, such as length of stay, development of complications, surgical outcomes, mortality and hospital readmissions [2].

Clinical administrative databases are the most interesting sources to study the results of effectiveness and efficiency of medical care. For this purpose, the adjustment of risk is an essential aspect of its use for this purpose, given that the patients are not randomly assigned to the comparison groups, eliminating the confounding effect caused by the different level of severity of the patients. This bias can be reduced measuring comorbidity, and that is the objective of the different comorbidity indices that have been developed [2].

Three popular comorbidity measurement tools developed by Charlson et al. [3], Elixhauser et al. [4] and van Walraven et al. [5] are used widely to measure the burden of disease with administrative data. Charlson et al. developed a composite score summarized by the weighted combination of 17 comorbidities using clinical conditions recorded in charts [3], Elixhauser et al. defined 30 comorbidities [4] and van Walraven et al. developed a modification of Elixhauser index, weighting the different comorbidities according to the risk of an in-hospital mortality risk of each one [5].

The minimum basic data set (MBDS) is the largest administrative database maintained in Spain with standardized clinical data of hospitalized patients, as well as the main source of information on treated morbidity. MBDS contains, for each hospitalization, the coding of the main diagnosis, up to 19 secondary diagnoses, and 22 procedures, according to the International Classification of Disease, 10th revision (ICD-10) [6,7]. The structure and content of the MBDS is regulated by the Ministry of Health (Royal Decree 69/2015 of February 6, which regulates the Register of Specialized Health Care Activity). When the provision of healthcare provided to a patient ends (due to discharge, transfer to another hospital or death) the hospital, using as a reference the public coding manual (based on ICD-10), transfers all the information about the patient’s admission to the MBDS. The MBDS is reported on a monthly basis, including the main diagnosis, which refers to that which lead to hospital admission, and secondary diagnoses are diseases that coexist at the time of admission or which emerged during the hospital stay and influence the duration of treatment [6]. Therefore, MBDS can be used to research patient comorbidity.

Quan et al. developed ICD-10 coding algorithms for Charlson and Elixhauser comorbidities, adapting codes validated by other authors in International Classification of Disease, 9th Revision (ICD-9) [8].

Although the comorbidity indices share some characteristics, they differ in the concept of comorbidity they measure and in their weights. Some of these indices consider only chronic conditions, while others include any accompanying diagnosis, with or without the inclusion of lifestyle habits. These differences may be due to the different comorbidities and their weights as a death predictor: the Charlson index chose 17 comorbidities to classify the prognosis of death at one year attributed to the comorbidity [2], Elixhauser improve measures of comorbidity for use with administrative inpatient databases con 30 variables [4], and van Walraven weighted Elixhauser comorbidities to improve mortality prediction [5]. The differences may also derive from the characteristics of the patients on whom the index was built. The first assumption could imply limitations to generalize an index built on a specific result to results of another type, while the second would imply limitations of generalization to other types of patients [2].

Some authors have already made comparisons between comorbidity indices in different populations. However, further research is needed to determine whether these comorbidity indices are valid elsewhere to accurately predict the risk of in-hospital death and to use the best predictor in each population.

The aim of this study was to quantify the differences between the Charlson (CCI), Elixhauser (ECI) and van Walraven (WCI) comorbidity indices as prognostic factors for in-hospital mortality and to identify the best comorbidity measure predictor in our population.

2. Methods

This is a retrospective observational study that included all hospitalizations of patients over 18 years of age discharged between the 2017 and 2021 fiscal year in a 401-bed public hospital of the Spanish National Healthcare System, located in the Region of Madrid. We used Spanish administrative database MBDS. When the patient’s health care episode ends, the hospital’s coding unit transfers the medical information from the discharge report to MBDS. To code the MBDS, this group of professionals uses a complex coding manual based on ICD-10. Similarly to studies by Charlson et al. [3], Elixhauser et al. [4] and van Walraven et al. [5] we excluded pediatric and obstetrical admissions (pregnancy/birth/puerperium), because these patient populations have very few chronic comorbidities and a very small chance of dying in hospital.

The individual comorbidities for each patient episode of hospitalization were measured using Elixhauser and Charlson ICD-10 coding algorithms and then, we calculated CCI, ECI and WCI.

The correlation and concordance between the three indices were evaluated by Spearman’s rho and Intraclass Correlation Coefficient (ICC), respectively.

Correlation is defined as a relation existing between phenomena or things or between mathematical or statistical variables that tend to vary, be associated, or occur together in a way not expected by chance alone. The interpretation of the Spearman’s correlation coefficient values ranges from 0 (absence of correlation) to 1 (full correlation). In medicine, the following categories are accepted (in absolute numbers): 0 none, 0.1–0.2 poor, 0.3–0.5 fair, 0.6–0.7 moderate, 0.8–0.9 very strong and 1 perfect [9].

The ICC is a reliability test that allows us to measure the concordance between two or more continuous quantitative evaluations obtained with different measuring instruments or evaluators. Values range from 0 (absence of concordance) to 1 (absolute agreement). We use the absolute agreement (ICC type A) that takes the following categories proposed by Fleiss in 1986: ICC < 0.40 poor, ICC 0.41–0.75 fair to good and ICC > 0.75 excellent [10].

The logistic regression model, fit by age, the used Elixhauser, van Walraven or Charlson comorbidity index and age as independent variables for predicting in-hospital mortality. For model construction, each comorbidity index was categorized into 4 quartiles depending on the outcome (strata) and age was transformed into binomial variable using the population median as the cut-off point.

We used the receiver operating characteristic (ROC) curve for comparing the performance of each index in predicting in-hospital mortality. The area under the ROC curve (AUC) or C-statistic measures how well the model can discriminate between observations, with possible values of 0.5 (no predictive ability), 0.7 to 0.8 (acceptable), 0.8 to 0.9 (excellent), 0.9 to 1.0 (outstanding), and 1 (perfect discrimination) [11]. According this, in our study C-statistic is a measure of a model’s ability to discriminate who die vs. who do not die in hospital. A nonparametric method described by Delong et al. [12] was employed to test the statistical significance of differences in the AUC between the Elixhauser, van Walraven or Charlson comorbidity index. Statistical analysis was performed using Stata/MP v16.

3. Results

3.1. Coding Algorithms and Comorbidity Frequencies

ICD-10 coding algorithms for Charlson and Elixhauser comorbidities [8] and the weight of each defined by authors [3,4,5], are presented in Appendix A Table A1 and Table A2.

Table A1

Comparison of ICD-10 coding algorithms for Elixhauser/van Walraven and Charlson comorbidities.

Comorbidities Elixhauser and van Walraven	ICD-10 Coding Algorithms Comorbidity	Comorbidities Charlson
Congestive heart failure	I09.9, I11.0, I13.0, I13.2, I25.5, I42.0, 142.5–I42.9, I43.x, I50.x, P29.0	Congestive heart failure
Peripheral vascular disorders	I70.x, I71.x, I73.1, I73.8, I73.9, I77.1, I79.0, I79.2, K55.1, K55.8, K55.9, Z95.8, Z95.9	Peripheral vascular disorders
Paralysis	G04.1, G11.4, G80.1, G80.2, G81.x, G82.x, G83.0–G83.4, G83.9	Hemiplegia or paraplegia
Chronic pulmonary disease	I27.8, 127.9, J40.x–J47.x, J60.x–J67.x, J68.4, J70.1, J70.3	Chronic pulmonary disease
AIDS/H1V	B20.x–B22.x, B24.x	AIDS/H1V
Metastatic cancer	C77.x–C80.x	Metastatic solid tumor
Cardiac arrhythmias	I44.1–I44.3, I45.6, I45.9, I47.x–I49.x, R00.0, R00.1, R00.8, T82.1, Z45.0, Z95.0
Valvular disease	A52.0, I05.x–I08.x, I09.1, I09.8, I34.x–I39.x, Q23.0–Q23.3, Z95.2, Z95.4
Pulmonary circulation disorders	I26.x, I27.x, I28.0, I28.8, I28.9
Hypertension, uncomplicated	I10.x
Hypertension, complicated	I11.x–I13.x, I15.x
Other neurological disorders	G10.x–G13.x, G20.x–G22.x, G25.4, G25.5, G31.2, G31.8, G31.9, G32.x, G35.x–G37.x, G40.x, G41.x, G93.1, G93.4, R47.0, R56.x
Diabetes, uncomplicated	E10.0, E10.1, E10.9, E11.0, E11.1, E11.9, E12.0, E12.1, E12.9, E13.0, E13.1, E13.9, E14.0, E14.1, E14.9
Diabetes, complicated	E10.2–E10.8, E11.2–E11.8, E12.2–E12.8, E13.2–E13.8, E14.2–E14.8
Hypothyroidism	E00.x–E03.x, E89.0
Liver disease	B18.x, I85.x, I86.4, I98.2, K70.x, K71.1, K71.3–K71.5, K71.7, K72.x–K74.x, K76.0, K76.2–K76.9. Z94.4
Renal failure	I12.0, I13.1, N18.x, NI9.x, N25.0, Z49.0–Z49.2, Z94.0, Z99.2
Peptic ulcer disease excluding bleeding	K25.7, K25.9, K26.7, K26.9, K27.7, K27.9, K28.7, K28.9
Lymphoma	C81.x–C85.x, C88.x, C90.0, C90.2, C96.x
Solid tumor without metastasis	C00.x–C26.x, C30.x–C34.x, C37.x–C41.x, C43.x, C45.x–C58.x, C60.x–C76.x, C97.x
Rheumatoid arthritis/collagen vascular diseases	L94.0, L94.1, L94.3, M05.x, M06.x, M08.x, M12.0, M12.3, M30.x, M31.0–M31.3, M32.x–M35.x, M45.x, M46.1, M46.8, M46.9
Coagulopathy	D65–D68.x, D69.1, D69.3–D69.6
Obesity	E66.x
Weight loss	E40.x–E46.x, R63.4, R64
Fluid and electrolyte disorders	E22.2, E86.x, E87.x
Blood loss anemia	D50.0
Deficiency anemia	D50.8, D50.9, D51.x–D53.x
Alcohol abuse	F10, E52, G62.1, I42.6, K29.2, K70.0, K70.3, K70.9, T51.x, Z50.2, Z71.4, Z72.1
Drug abuse	F11.x–F16.x, F18.x, F19.x, Z71.5. Z72.2
Psychoses	F20.x, F22.x–F25.x, F28.x, F29.x, F30.2, F31.2, F31.5
Depression	F20.4, F31.3–F31.5, F32.x, F33.x, F34.1, F41.2, F43.2
	I21.x, I22.x, I25.2	Myocardial infarction
	G45.x, G46.x, H34.0, I60.x–I69.x	Cerebrovascular disease
	F00.x–F03.x, F05.1, G30.x, G31.1	Dementia
	E10.0, E10.1, E10.6, E10.8, E10.9, E11.0, E11.1, E11.6, E11.8, E11.9, E12.0, E12.1, E12.6, E12.8, E12.9, E13.0, E13.1, E13.6, E13.8, E13.9, E14.0, E14.1, E14.6, E14.8, E14.9	Diabetes without chronic complication
	E10.2–E10.5, E10.7, E11.2–E11.5, E11.7, E12.2–E12.5, E12.7, E13.2–E13.5, E13.7, E14.2–E14.5, E14.7	Diabetes with chronic complication
	B18.x, K70.0–K70.3, K70.9, K71.3–K71.5, K71.7, K73.x, K74.x, K76.0, K76.2–K76.4, K76.8, K76.9, Z94.4	Mild liver disease
	I85.0, I85.9, I86.4, I98.2, K70.4, K71.1, K72.1, K72.9, K76.5, K76.6, K76.7	Moderate or severe liver disease
	I12.0, I13.1, N03.2–N03.7, N05.2–N05.7, N18.x, N19.x, N25.0, Z49.0–Z49.2, Z94.0, Z99.2	Renal disease
	K25.x–K28.x	Peptic ulcer disease
	C00.x–C26.x, C30.x–C34.x, C37.x–C41.x, C43.x, C45.x–C58.x, C60.x–C76.x, C81.x–C85.x, C88.x, C90.x–C97.x	Any malignancy, including lymphoma and leukemia, except malignant neoplasm of skin
	M05.x, M06.x, M31.5, M32.x–M34.x,M35.1, M35.3, M36.0	Rheumatic disease

Note: “x” applies to all codes within the range.

Table A2

Elixhauser, Charlson and van Walraven comorbidities weight.

Comorbidities Elixhauser/van Walraven	Comorbidities Charlson	Elixhauser Points	Charlson Points	van Walraven Points
Congestive heart failure	Congestive heart failure	1	1	7
Peripheral vascular disorders	Peripheral vascular disorders	1	1	2
Paralysis	Hemiplegia or paraplegia	1	2	7
Chronic pulmonary disease	Chronic pulmonary disease	1	1	3
AIDS/H1V *	AIDS/H1V *	1	6	0
Metastatic cancer *	Metastatic solid tumor *	1	6	12
Cardiac arrhythmias		1		5
Valvular disease		1		−1
Pulmonary circulation disorders		1		4
Hypertension, uncomplicated		1		0
Hypertension, complicated		1		0
Other neurological disorders		1		6
Diabetes, uncomplicated *		1		0
Diabetes, complicated *		1		0
Hypothyroidism		1		0
Liver disease		1		11
Renal failure		1		5
Peptic ulcer disease excluding bleeding		1		0
Lymphoma		1		9
Solid tumor without metastasis *		1		4
Rheumatoid arthritis/collagen vascular diseases		1		0
Coagulopathy		1		3
Obesity		1		−4
Weight loss		1		6
Fluid and electrolyte disorders		1		5
Blood loss anemia		1		−2
Deficiency anemia		1		−2
Alcohol abuse		1		0
Drug abuse		1		−7
Psychoses		1		0
Depression		1		−3
	Myocardial infarction		1
	Cerebrovascular disease		1
	Dementia		1
	Diabetes without chronic complication *		1
	Diabetes with chronic complication *		2
	Mild liver disease*		1
	Moderate or severe liver disease *		3
	Renal disease		2
	Peptic ulcer disease		1
	Any malignancy, including lymphoma and leukemia, except malignant neoplasm of skin *		2
	Rheumatic disease		1

Note: * A hierarchy was established between the following pairs of comorbidities, so that only the more severe comorbidity was counted. For example: If both uncomplicated diabetes and complicated diabetes are present, count only complicated diabetes.

We studied 79,425 admission episodes between the 2017 and 2021 fiscal year in our hospital. Of these, 54.29% were men. The median age was 72 years (interquartile range, [IQR]: 56–80) and the median of length of hospital stay was 4 days (IQR: 2–8). The in-hospital mortality rate was 4.47%. The demographic and clinical characteristics of the study population are defined in Table 1. In this table, the international classification by Main Diagnostic Categories (MDC) associates all possible reasons for admission into 25 groups, according to the main apparatus or body systems.

Table 1

Demographic and clinical characteristics of hospitalized adults.

Variable	Hospitalizations
Age, years
Median (IQR)	72 (57–81)
Sex, no (%)
Male	43,520 (54.79)
Female	35,905 (45.21)
Length in stay, days
Median (IQR)	4 (2–8)
Death, no (%)	3551 (4.47)
MDC, no (%)
Circulatory system	11,202 (14.10)
Digestive system	9656 (12.16)
Kidney and urinary tract	7413 (9.33)
Nervous system	5978 (7.53)
Infectious and parasitic diseases and disorders	2557 (3.22)
Mental diseases and disorders	2016 (2.54)
Skin, subcutaneous tissue and breast	1788 (2.25)
Ear, nose, mouth and throat	1702 (2.14)
Male reproductive system	1543 (1.94)
Female reproductive system	1358 (1.71)
Eye	1149 (1.45)
Endocrine, nutritional and metabolic system	1142 (1.44)
Musculoskeletal system and connective tissue	9660 (12.16)
Myeloproliferative diseases and disorders	924 (1.16)
Factors influencing health status	886 (1.12)
Blood and blood forming organs and immunological disorders	719 (0.91)
Injuries, poison and toxic effect of drugs	714 (0.90)
Alcohol/drug use or induced mental disorders	332 (0.42)
Multiple significant trauma	126 (0.16)
Human immunodeficiency virus (HIV) infection	54 (0.07)
Respiratory system	13,530 (17.03)
Hepatobiliary system and pancreas	4976 (6.26)

Note: MDC, Major Diagnostic Categories, formed by dividing all possible principal diagnoses (reason for admission) into to 25 mutually exclusive diagnosis areas that correspond to a single organ system or etiology; IQR, interquartile range.

The most frequent Elixhauser’s comorbidities present were hypertension uncomplicated (36.34%), cardiac arrhythmias (20.3%), diabetes uncomplicated (17.03%), chronic pulmonary disease (16.53%), obesity (13.17%), renal failure (12.96%), solid tumor without metastasis (12.95%) and congestive heart failure (12.94%). The complete list is shown in Table 2.

Table 2

Comorbidities of Elixhauser/van Walraven frequencies.

Comorbidities Elixhauser/van Walraven	Frequency (%)
Congestive heart failure	12.94
Peripheral vascular disorders	6.27
Paralysis	1.43
Chronic pulmonary disease	16.53
AIDS/H1V	0.46
Metastatic cancer	6.88
Cardiac arrhythmias	20.30
Valvular disease	11.46
Pulmonary circulation disorders	7.22
Hypertension, uncomplicated	36.34
Hypertension, complicated	12.23
Other neurological disorders	7.98
Diabetes, uncomplicated	17.03
Diabetes, complicated	8.92
Hypothyroidism	8.35
Liver disease	7.68
Renal failure	12.96
Peptic ulcer disease excluding bleeding	0.53
Lymphoma	1.48
Solid tumor without metastasis	12.95
Rheumatoid arthritis/collagen vascular diseases	3.41
Coagulopathy	3.50
Obesity	13.17
Weight loss	1.50
Fluid and electrolyte disorders	8.75
Blood loss anemia	1.19
Deficiency anemia	5.27
Alcohol abuse	7.15
Drug abuse	2.40
Psychoses	1.57
Depression	6.30

The most frequent Charlson’s comorbidities present were diabetes without chronic complication (21.39%) and chronic pulmonary disease (16.53%). Any malignancy included lymphoma and leukemia, except for malignant neoplasm of skin (15.25%), renal disease (13.00%) and congestive heart failure (12.94%). The complete list is shown in Table 3.

Table 3

Comorbidities of Charlson frequencies.

Comorbidities Charlson	Frequency (%)
Congestive heart failure	12.94
Peripheral vascular disorders	6.27
Hemiplegia or paraplegia	1.43
Chronic pulmonary disease	16.53
AIDS/H1V	0.46
Metastatic solid tumor	6.88
Myocardial infarction	7.73
Cerebrovascular disease	7.50
Dementia	5.69
Diabetes without chronic complication	21.39
Diabetes with chronic complication	8.48
Mild liver disease	0.84
Moderate or severe liver disease	1.79
Renal disease	13.00
Peptic ulcer disease	0.83
Any malignancy, including lymphoma and leukemia, except malignant neoplasm of skin	15.25
Rheumatic disease	2.38

According to Table 1 and Table 2, for the six common comorbidities (congestive heart failure, peripheral vascular disorders, hemiplegia or paraplegia, chronic pulmonary disease, AIDS/H1V and metastatic solid tumor) the frequencies were the same. Different frequencies were obtained for unrelated comorbidities.

The frequencies 3/7 comorbidities that may have a clinical relationship between the Elixhauser and Charlson definitions were different [Elixhauser vs. Charlson]: diabetes uncomplicated 17.03% vs. diabetes without chronic complication 21.39%, liver disease 7.68% vs. mild liver disease 0.84% plus moderate or severe liver disease 1.79% and rheumatoid arthritis/collagen vascular diseases 3.41% vs. rheumatic disease 2.38%. For 4/7 related clinical comorbidities, the frequencies were similar [Elixhauser vs. Charlson]: renal failure 12.96% vs. renal disease 13.00% and solid tumor without metastasis 12.95% plus lymphoma 1.48% vs. any malignancy including lymphoma and leukemia except malignant neoplasm of skin 15.25%, diabetes complicated 8.92% vs. diabetes with chronic complication 8.48% and peptic ulcer disease excluding bleeding 0.53% vs. 0.83%.

The median of ECI was 2 (IQR: 1–4), ICW was 4 (IQR: 0–12) and ICC was 1 (IQR: 0–3). The score intervals of each strata comorbidity index and the frequencies represented, are detailed in Table 4.

Table 4

Elixhauser, van Walraven and Charlson score interval and frequency of each strata.

Comorbidity Index	Strata 1	Strata 2	Strata 3	Strata 4
Elixhauser
Score interval	0	1	2 to 3	4 to 13
no (%)	13,322 (16.77)	14,047 (17.69)	27,283 (34.35)	24,773 (31.19)
van Walraven
Score interval	−14 to −1	0 to 3	4 to 11	12 to 52
no (%)	7812 (9.84)	30,822 (38.81)	19,694 (24.80)	21,097 (26.56)
Charlson
Score interval	0	1	2 to 3	4 to 18
no (%)	27,740 (34.93)	15,569 (19.60)	20,044 (25.24)	16,072 (20.24)

3.2. Correlation, Concordance and In-Hospital Mortality Prediction

The correlation between the comorbidity indices (by strata) was moderate: ECI vs. WCI rho = 0.645, p < 0.001; ECI vs. CCI rho = 0.721, p < 0.001); and CCI vs. WCI rho = 0.704, p < 0.001).

The concordance between the comorbidity indices (by strata) was fair to good: ECI vs. WCI CCIA = 0.675 (Confidence Interval CI 95% 0.665–0.684) p < 0.001; ECI vs. CCI ICCA = 0.797 (CI 95% 0.780–0.812), p < 0.001; and ICCA vs. WCI ICCA = 0.731 (CI 95% 0.667–0.779, p < 0.001).

The multivariate regression analysis demonstrated that comorbidity increased the risk of in-hospital mortality, with differences depending on the comorbidity measurement scale (Table 5).

Table 5

Analysis adjusted for mortality by covariates: age and Elixhauser, van Walraven or Charlson score.

Covariate	Adjusted OR	CI 95%	p > |z|
Elixhauser score (all strata)	2.10	2.00–2.20	0.000
Elixhauser strata 1	1
Elixhauser strata 2	3.24	2.44–4.31	0.000
Elixhauser strata 3	6.84	5.26–8.90	0.000
Elixhauser srata 4	13.33	10.27–17.32	0.000
Age > 72	2.20	2.02–2.38	0.000
van Walraven score (all strata)	2.31	2.21–2.41	0.000
van Walraven strata 1	1
van Walraven strata 2	1.28	1.01–1.63	0.039
van Walraven strata 3	3.82	3.04–4.81	0.000
van Walraven strata 4	8.10	6.47–10.15	0.000
Age > 72	2.40	2.22–2.60	0.000
Charlson score (all strata)	2.53	2.40–2.67	0.000
Charlson strata 1	1
Charlson strata 2	2.73	2.40–3.11	0.000
Charlson strata 3	4.66	4.40–5.11	0.000
Charlson strata 4	6.68	5.90–7.56	0.000
Age > 72	2.44	2.25–2.64	0.000

Note: OR: Odds Ratio, CI 95%: Confidence Interval 95%.

3.3. Prediction Performance

The ROC-statistic (Figure 1) showed an area under the curve [AUC] = 0.714 (CI 95% 0.706–0.721) using as a predictor of in-hospital mortality CCI; AUC = 0.729 (CI 95% 0.721–0.737) for ECI and AUC = 0.750 (CI 95% 0.743–0.758) using WCI; Nonparametric tests suggest that AUC differences between the three comorbidity indices are statistically significant (p < 0.001).

Figure 1

ROC curve for predicting in-hospital mortality by Elixhauser, Walraven and Charlson comorbidity indices.

4. Discussion

To our knowledge, this is the first published study that compares the prediction of in-hospital mortality in the general population, using administrative data of three of the most used comorbidity indices: Charlson, Elixhauser and van Walraven. All possess an acceptable death prediction, but we aimed that the best comorbidity measure, as prognostic factor for intra-hospital mortality, was the van Walraven comorbidity index, followed by the Elixhasuer comorbidity index.

We found that the three most used comorbidity indices in administrative databases are unrelated and therefore, measure comorbidity differently. Although we have found a moderate correlation, since the main utility of comorbidity indices is the prediction of mortality, small differences have a great clinical implication, since the prediction will be different depending on the index used. Therefore, we assume that the indices are unrelated. We have found that differences in the validated ICD-10 coding algorithms that define clinically related comorbidities in comorbidity indices translate into variations in the frequency of these comorbidities. This, together with the different comorbidities measured and the different associated weights in the construction of each index, means that the concept of comorbidity measured is not the same, translating into differences in the prediction of mortality. This makes it necessary to search for the best predictor in the study population, so that future authors adequately control the potential confounding factor that comorbidity supposes in their studies.

In 1987, Charlson et al. validated 17 mortality predictive comorbidities [3]. Subsequently, in 1992 Deyo et al. adapted the Charlson comorbidity index for use with ICD-9 administrative databases [8]. In 1998 Elixhauser et al. tried to improve the CCI prediction by validating 30 comorbidities, of which 6 were identical to proposed by Charlson and 7 have some clinical similarity; however, the ICD-9 coding algorithms that define them include small variations. They eliminated 3 comorbidities present in CCI (myocardial infarction, cerebrovascular disease and dementia) and defined 17 new comorbidities. However, unlike the ICC, the ECI construction assigns equal weight to all variables [4]. In 2005, Quan et al. translated and validated Charlson and Elixhauser comorbidities ICD-9 codes to the new ICD-10 classification [8]. Finally, van Walraven condensed the 30 binary variables of the original Elixhauser comorbidity system into a single numerical score that summarizes the burden of disease and modified the ECI through a scoring system that reflected the strength of association of each Elixhauser comorbidity with in-hospital death, using Quan et al. ICD-9 and ICD-10 codes [5].

The main comorbidities described in our study confirm that arterial hypertension is the most frequent, in almost 1/3 of the patients, and is followed by diabetes without chronic complication, cardiac arrhythmias and chronic pulmonary disease. These results are in relation to other studies that have used the general population through administrative databases such as Elixhauser el al. [4] in the United States, where the most frequent comorbidities among all discharges were hypertension (17.9% of cases), fluid and electrolyte disorders (13.3%), chronic pulmonary disease (9.9%), diabetes (7.8%), deficiency anemias (7.3%), and cardiac arrhythmias (6.8%); or van Walraven el at. [5] in Canada, where the most common comorbidities were hypertension (20.2%), fluid and electrolyte disorders (12.4%), solid tumor without metastases (11.7%), cardiac arrhythmias (11.2%) and uncomplicated diabetes (9.5%).

In our study, the frequencies for some comorbidities of Elixhauser and Charlson definitions were different. This, together with the different comorbidities measured by both indices, estimate a moderate correlation and regular agreement between the measurement of comorbidity by ICE, ICC or the weighted version of ICE (WCI).

The differences in the construction of the indices and the different concept of comorbidity that they measure mean that the prediction of in-hospital mortality is different for each one. That is the reason why the OR range (Table 5) varies from 2.10 to 2.53 depending on the index used for measuring comorbidity, which implies important differences in the real prediction of in-hospital death. The mortality is zero (OR = 1) for the low index score (strata 1) and increases as the index score increases (strata 2–4). In all cases, age (in our case above the median age) also increases in-hospital mortality.

According to the ROC-statistic, in our population the van Walraven model performed slightly better in predicting in-hospital mortality than Elixhauser and Charlson models. The ECI prediction was also superior to CCI. One possible explanation is the assignment of weights to comorbidities by van Walraven versus Elixhauser, based on the risk of death they represent, as well as the greater number of comorbidities measured by van Walraven and Elixhauser versus Charlson.

Not many direct comparisons have been made through correlation and concordance of comorbidity indices. Ou et al. compared the performance of comorbidity indices in predicting health care-related behaviors in type 2 diabetes patients, though an American administrative database, obtained a correlation of fair-moderate between CCI and ECI (Spearman’s rho = 0.56) [13].

Various authors have explored the prediction of mortality from comorbidity indices in different populations and compared CCI and ECI prediction. Tsai et al. made a comparison of Elixhauser and Charlson methods for discriminative performance in-mortality risk in patients with schizophrenic disorders using a Taiwan administrative database, whose C-statistic was better than the ECI (0.856 vs. 0.854, respectively) [11]. A similar study conducted by Menéndez et al. in Canada concluded that ECI outperforms CCI in predicting inpatient death after orthopedic surgery (AUC = 0.86 vs. 0.83, respectively) [14].

No studies have been found with direct comparisons for the three indices in the same population nor comparisons between WCI with the rest of the comorbidity indices. In the validation of the index in a Canadian population through an administrative database, van Walraven et al. demonstrated that WCI (AUC = 0.763) was a similar discriminative for death in hospital to ECI (AUC = 0.760) and exceeded discrimination when comorbidity was expressed using CCI (0.745).

Our study has several limitations. Firstly, it relies on administrative data, which are never complete or detailed enough to provide a clinically precise method for identifying comorbidities, due to lack of control in the registry (under-registration) with a tendency to register the most important and acute events, to the detriment of those that are milder or chronic [15,16]. Secondly, the validity of the coding algorithms, and specifically, their sensitivity and specificity relative to a standard criterion (e.g., chart review data) [8]. Furthermore, our population refers to a single hospital and the administrative database used (MBDS), which registers the reason for admission (MDC) but does not include the death reason.

However, the study also presents some strengths: we used the most widely used coding algorithms and calculated the frequencies of the comorbidities to closely observe the differences in the measurement of comorbidity. Moreover, the database used in our study (MBDS) allows one main diagnosis and up to nineteen secondary ones. In this way, it minimizes the problem of under-registration of chronic comorbidity in other administrative databases, especially studies prior to 2010, that only allow one main diagnosis and 4–9 secondary. In our study, in contrast to other studies, the annotation of the chronic diagnosis codes is subject to the non-existence of other acute diagnoses; thus, the chronic comorbidity collected by the indices in these databases will be annotated when no other complications or acute comorbidity coexists [2,16].

The authors who developed the comorbidity indices advise that they should be validated by other colleagues in their populations. The original indices are validated in the general population, but in most of the publications the authors compare the indices in specific populations, which increases the sensitivity and specificity of the prediction, but does not allow other authors to decide which index to choose if they do not have population-specific studies. Like the original authors, we have tested the most used comorbidities indices in the general population of Spain, providing researchers evidence on which index choose to control the confounding effect of comorbidity in their future research.

5. Conclusions

Comorbidity plays an important role as a predictor of in-hospital mortality, but the differences in the construction of the comorbidity indices imply differences in this prediction. In our population, the van Walraven comorbidity index was the best predictor of in-hospital mortality.