Development and Validation of a Nordic Multimorbidity Index Based on Hospital Diagnoses and Filled Prescriptions.

Purpose: To develop the Nordic Multimorbidity Index (NMI), a multimorbidity measure specifically suited to the Nordic health and administrative registry data based on current diagnosis, treatment, and coding practices.
Methods: The NMI was developed to predict 5-year mortality in a population-based cohort of randomly sampled Danish residents aged ≥40 years (n = 425,087) followed from 2013 to 2018. Included predictors were selected from hospital diagnoses and filled drug prescriptions based on a combination of subject matter knowledge and a data-driven approach using backwards elimination. The performance of the NMI was assessed in a temporal validation cohort of Danish residents followed from 2007 to 2012 and in six cohorts of new users of selected drugs. The discriminative performance of the NMI, Charlson Comorbidity Index (CCI) and the Elixhauser Comorbidity Index (ECI) was assessed using the c-statistic from logistic regression models with 5-year mortality as dependent variable and the multimorbidity index score, age, and sex as independent variables.
Results: The NMI included 50 predictors. In the temporal validation cohort, the c-statistic of the NMI (0.887, 95% CI 0.883-0.890) exceeded that of the CCI (0.871, 95% CI 0.868-0.874) and ECI (0.866, 95% CI 0.863-0.870). In all new user cohorts, the NMI outperformed the other indices with c-statistics ranging from 0.781 (95% CI 0.779-0.784) to 0.838 (95% CI 0.834-0.842).
Conclusion: The NMI predicted 5-year mortality in a general Danish population and six cohorts of new users of selected drugs and was superior to the CCI and ECI. The NMI could be preferred over these indices to quantify the level of multimorbidity for, eg, descriptive purposes or confounding control. The NMI should be validated in other patient populations and other Nordic countries.

Introduction

Routinely collected administrative and healthcare data have been used for decades to assess safety of drugs and other medical interventions and play an increasing role in the evaluation of drug efficacy and treatment.1,2 However, in non-randomized studies, confounding has to be accounted for. While routinely collected healthcare data often provide detailed information on diagnoses and prescriptions, measures of the general physical condition and health of a given individual are rarely available. Comorbidity or multimorbidity indices are often used as proxies hereof by aggregating a range of conditions into a single numerical variable. They are easy to apply and provide a standardized summary of a range of health-related conditions. The most commonly used indices are the Charlson Comorbidity Index (CCI) developed in 1987 to predict 1-year mortality in hospitalized patients and the Elixhauser Comorbidity Index (ECI) developed in 1998 to predict in-hospital mortality, length of hospital stay, and hospital charges.3,4 The CCI included 19 weighted medical conditions and has since been adapted to use with administrative data by assigning ICD-9 and ICD-10 codes to the conditions and combining leukemia and lymphoma with any malignancy resulting in 17 conditions.5–9 The ECI originally included 30 comorbidities as dichotomous variables describing the presence of a given condition. In 2009, van Walraven et al assigned weights to 21 of the Elixhauser conditions based on their association with in-hospital mortality.10 There are several reasons that these and other indices may not perform optimally in summarizing morbidity in the general population of the Nordic countries. The Nordic countries provide a unique data source for health research with individual level data on, among others, hospital diagnoses and prescription drug use.11 These data are not necessarily identical to data collected by manual chart review since the underlying data infrastructure is different, eg, hospital diagnoses are likely influenced by geographical coding practices. Further, the CCI and ECI were developed on hospitalized patients in North America, limiting the generalizability of these indices to a general population in the Nordic countries. Lastly, since the development of the CCI and ECI, diagnosis, treatment, and prognosis of many conditions have changed substantially. For example, survival in HIV-positive individuals dramatically changed upon introduction of reverse transcriptase-inhibitors in 1996,12 the prognosis of many cancers has improved,13 and mortality rates have decreased in patients with diabetes.14

A general multimorbidity index designed specifically for use in the Nordic countries does not exist. The Nordic countries provide a unique setting for pharmacoepidemiologic research due to their nationwide individual-level registries and for providing government-funded universal taxed-based health care.11 With the increasing use of Nordic registry data to evaluate safety and effectiveness of drugs and other medical interventions,15 we aimed to develop a multimorbidity index that reflects current clinical practice and data infrastructure in the Nordic countries.

Materials and Methods

We developed the Nordic Multimorbidity Index (NMI) based on the ability to predict 5-year mortality in the Danish population aged 40 years and older. A list of candidate variables was identified by clinical review of all hospital diagnoses and prescription drugs with a prevalence above 1 in 1000 individuals and, from these, the final predictors were selected using backwards elimination. A total of 50 predictors were included, and the NMI was calculated as a single numeric score by summing the weights of each predictor. The performance of the NMI was assessed in a temporal validation cohort and in six cohorts of new users of selected drugs and compared to the performance of the CCI and ECI.

Data Sources

In Denmark, health care services are provided through a tax-funded health care system with coverage for all residents. All residents in Denmark have been assigned a unique personal identification number since 1968 allowing for individual level linkage across the Danish health and administrative registries.16 The Danish National Patient Register contains information on all in-patient visits since 1978 and all outpatient visits, emergency room visits, and psychiatric contacts since 1995. From 1994 and onwards, diseases have been classified according to the International Classification of Diseases (ICD)-10 system.17 The Danish National Prescription Registry contains data on all prescription drugs dispensed at outpatient pharmacies since 1995.18 Drug substances were classified according to the 2020 WHO Anatomical Therapeutic Chemical (ATC) Classification System in this study.19

Population

The study population consisted of a random 20% sample of all Danish individuals born in 1977 or before. This population was split randomly assigning 75% of the individuals to a development cohort (2013–2018) and the remaining 25% to a temporal validation cohort (2007–2012).

The development cohort was created by assigning a random index date between Jan 1, 2013 to Dec 31, 2013 to everyone and including individuals alive and aged 40 years or older at the index date. To allow for complete covariate assessment, we required continuous Danish residency 5 years prior to the index date. Individuals were followed from the index date until death, migration, or end of follow-up 5 years after the index date, whichever occurred first. Individuals who migrated during follow-up were excluded.

The temporal validation cohort was created by assigning a random index date to everyone between Jan 1, 2007 and Dec 31, 2007 applying the same eligibility criteria as the development cohort. Thus, the validation cohort was separated in time, serving as a validation of the model across calendar time and assessment of transportability of the index over time.

Next, we constructed six validation cohorts of new users of selected drugs aiming to imitate typical study populations in pharmacoepidemiologic studies. The source population for the new user cohorts were all Danish individuals born in 1953 or before. We identified new users of six drug classes: bisphosphonates (ATC code: M05BA, M05BB), long-acting muscarinic antagonists (LAMA, ATC code: R03BB04–R03BB07), low-dose oral methotrexate (ATC code: L04AX03), statins (ATC code: C10AA), urate lowering drugs (ATC code: M04AA), and warfarin (ATC code: B01AA03). New users during 2004 to 2013 were included, and cohort entry was defined by the first ever prescription fill. To allow for complete covariate assessment, individuals who did not reside in Denmark continuously 5 years prior to cohort entry were excluded. Individuals were followed until death, migration, or end of follow-up 5 years after cohort entry. Individuals who migrated during follow-up were excluded.

Outcome

We developed the NMI to predict 5-year all-cause mortality. This time span was chosen to capture conditions reflecting mild-to-moderate multimorbidity that will not necessarily influence short-term mortality and because a non-hospitalized, general population has a long life expectancy despite of multimorbidity.

Candidate Predictors

Potential predictors included filled prescriptions at community pharmacies and primary and secondary diagnoses from in- and outpatient hospital visits. Prescription fills were recorded in the 6-months period prior to the index date, while diagnoses were recorded 5 years prior to the index date. The presence of a diagnosis or prescription fill was defined as one or more diagnoses/prescription fills during the respective assessment windows. A 5-year assessment window for diagnoses was chosen to avoid overlap between development and validation periods and to ensure identical look-back periods for each cohort. We applied a shorter assessment window of 6 months for prescriptions to capture current drug use, as former use may represent conditions that are no longer present.

To assemble a list of clinically relevant candidate predictors with face validity, all codes with a prevalence above 1 in 1000 in the development cohort were assessed by three medical doctors (DPH, MR, and KBK). For this purpose, prescriptions were truncated to the fourth ATC level which depicts the chemical subgroup of the drug, eg, C10AA: HMG CoA reductase inhibitors (statins), and ICD-10 codes were truncated to the first three characters designating the category of the diagnosis, eg, I10: Essential (primary) hypertension. We did not consider the ICD chapters: O00-O99; Pregnancy, childbirth, and the puerperium, R00-R99; Symptoms, signs, and abnormal clinical and laboratory findings, not elsewhere classified, S00-T99; Injury, poisoning, and certain other consequences of external causes, and Z00-Z99; Factors influencing health status and contact with health services. ATC and ICD-10 codes were not included as candidate predictors if they were imprecisely defined, did not reflect diseases or chronic conditions, were not expected to meaningfully predict mortality, or were otherwise thought to have low face validity, eg, ICD-10 code K92: Other diseases of digestive system and ATC code R01AX: Other nasal preparations. Selected ICD-10 codes were combined to a single group, eg, J12-J18 were combined to the group “Pneumonia”. The grouping of ICD-10 codes was guided by existing suggested or validated coding algorithms for the Danish registries.17,20 We allowed conditions with a prevalence below 1 in 1000 to be included in groups when these were deemed to belong naturally to that group, eg, I72: Other aneurysm and dissection was included in the group “Aneurysm and dissection of aorta and other arteries” even though its prevalence was below 1 in 1000. We used ICD-10 codes with four digits when relevant, eg, K221: Ulcer of esophagus was included with K25-K28 in the group “Peptic ulcers”. Likewise, selected ATC codes were combined by either truncating to the third level of the ATC code or by combining relevant codes, eg, R03AC02–R03AC05 were combined to the group “Short-acting beta agonists”.

To explore the discrimination of each of the candidate predictors and their strength of association with age, we plotted the c-statistic from a logistic regression model with the predictor and age as independent variables and 5-year mortality as dependent variable according to the strength of correlation between the predictor and age expressed by Pearson’s correlation coefficient.

Development of the NMI

To construct a multimorbidity index that was easy to communicate and apply in research, we used a combination of subject matter knowledge and a data-driven approach for predictor selection. First, candidate predictors were selected and grouped based on review by three medical doctors as described above. We then used a backwards elimination approach, where age (numeric), sex, and the identified candidate predictors were included as independent variables in a logistic regression model with death within 5-years as dependent variable. The predictor with the highest p-value was eliminated whereafter the model was refitted with the remaining predictors. This process was repeated until 50 predictors (in addition to age, sex, and the intercept) remained. This number of predictors was chosen as a trade-off between a simple index that is feasible to implement while still achieving a high predictive performance. The c-statistic to discriminate 5-year mortality was plotted against the number of predictors in the model to visualize loss of discrimination as the number of predictors was reduced.

To ease communication and use of the NMI, we assigned weights to each of the selected 50 predictors by multiplying the beta-coefficient by ten and rounding to the nearest integer.21 The NMI score was then calculated for each individual as the sum of the weights for all predictors that were present before the index date (6 months for prescriptions and 5 years for diagnoses).

Performance

We assessed the performance of the model in the development cohort, the temporal validation cohort and the six new user cohorts by fitting a logistic regression model with 5-year mortality as dependent variable and the NMI (numeric), age (numeric), and sex as independent variables and estimating the predicted probability of death within 5 years for each individual. The performance of ECI and CCI was similarly estimated by fitting a logistic regression model with 5-year mortality as dependent variable and the CCI/ECI (numeric), age (numeric), and sex as independent variables and estimating the predicted probability of death for each individual from this model. The same covariate assessment window was used to define the CCI/ECI conditions, ie, diagnoses within 5 years before the index date. The CCI was defined according to ICD-10 coding definitions based on Quan et al 2005 (), and the original weights for the CCI comorbidities were used.3,8 In a sensitivity analysis, we examined whether the updated CCI weights developed by Quan et al in 2011 performed better than the original weights with regard to discrimination ().9 We fitted logistic regression models with age, sex and the CCI scores with the original and the updated weights and then used the CCI weighting algorithm with the highest concordance (c)-statistic for comparison to the NMI. The Elixhauser comorbidities were defined according to ICD-10 coding definitions based on Quan et al 2005 using the van Walraven weighting algorithm ().8,10

Discrimination, the ability to discriminate individuals with an outcome from those without the outcome, was assessed using the c-statistic that quantifies the ability to assign higher probabilities to patients who died than to patients who survived. The c-statistic is equal to the area under the receiver operating curve for binary outcomes.22 The values range from 0.5 corresponding to an uninformative model to 1 corresponding to a model with perfect concordance and, although the interpretation of the c-statistic is context dependent, as a rule of thumb values less than 0.7 can be considered poor, values in the range 0.7 to 0.8 can be considered acceptable, and values above 0.8 can be considered excellent.23 The c-statistic was the main performance measure of interest since good discrimination is usually the most relevant quality for research purposes.22 C-statistics with 95% confidence intervals were calculated using the DeLong method.24,25 We assessed discriminative performance for 5-year mortality and, secondarily, for 1- and 2-year mortality. Discriminative performance for 1- and 2-year mortality was assessed similarly by estimating predicted probabilities of death by fitting a logistic regression model with the NMI (using the original weights developed to predict 5-year mortality), age, and sex as independent variables and 1- or 2-year mortality as dependent variables.

Calibration of the NMI was assessed by comparing observed and predicted 5-year mortality by levels of the NMI score. To compare the predicted and observed mortality rates, the NMI was categorized by grouping adjacent NMI scores for each cumulative percentile of 2% from the lowest NMI to the highest. We calculated exact binomial 95% confidence intervals around the observed 5-year mortality proportion in each NMI category.26

Other

Statistical programming was conducted using Stata version 17.0, R version 4.1.2 and R Studio version 2021.09.1+372.27–29

Results

Study Cohorts

The development cohort included 425,087 individuals (). The median age was 58 years, and 48% were males. Baseline characteristics are shown in Table 1. A total of 38,301 individuals (9.0%) died during the 5-year follow-up period. The temporal validation cohort included 134,545 individuals with a median age of 58 years, 48% males, and a 5-year mortality of 9.6%. The six new user validation cohorts were older, had a higher 5-year mortality ranging from 12.9% to 34.8%, and had more diagnoses and prescription fills compared to the development cohort ( and ).

Table 1

Baseline Characteristics of the Development and Temporal Validation Cohort

Characteristic (%)	Development Cohort	Temporal Validation Cohort
	n = 425,087	n = 134,545
Age, median (p25-p75)	58 (49–69)	58 (48–68)
Male	204,848 (48.2%)	64,537 (48.0%)
1-year mortality	7505 (1.8%)	2646 (2.0%)
2-year mortality	15,123 (3.6%)	5291 (3.9%)
5-year mortality	38,301 (9.0%)	12,935 (9.6%)
Selected common diagnoses (within 5 years)
Essential (primary) hypertension	39,383 (9.3%)	9573 (7.1%)
Type 2 diabetes mellitus	14,650 (3.4%)	3897 (2.9%)
Atrial fibrillation and flutter	14,143 (3.3%)	3626 (2.7%)
Chronic ischaemic heart disease	12,529 (2.9%)	4297 (3.2%)
Heart failure	7099 (1.7%)	2296 (1.7%)
Cerebrovascular disease	11,825 (2.8%)	3660 (2.7%)
Chronic lower respiratory diseases and failure	11,822 (2.8%)	3481 (2.6%)
Pneumonia	12,455 (2.9%)	3709 (2.8%)
Osteoporosis	10,994 (2.6%)	2057 (1.5%)
Anaemia	7604 (1.8%)	1990 (1.5%)
Selected common drug fills (within 6 months)
Platelet aggregation inhibitors excl. heparin	58,600 (13.8%)	18,385 (13.7%)
HMG CoA reductase inhibitors (statins)	80,026 (18.8%)	18,338 (13.6%)
ACE inhibitors incl. combinations	53,473 (12.6%)	13,515 (10.0%)
ARBs incl. combinations	41,065 (9.7%)	9500 (7.1%)
Low-ceiling diuretics	37,330 (8.8%)	14,503 (10.8%)
High-ceiling diuretics	20,669 (4.9%)	7155 (5.3%)
Blood glucose lowering drugs, excl. insulins	24,607 (5.8%)	5026 (3.7%)
Anilides (paracetamol)	41,763 (9.8%)	11,518 (8.6%)
Benzodiazepines and related drugs	33,026 (7.8%)	16,114 (12.0%)
Antidepressives	42,605 (10.0%)	12,742 (9.5%)
Potassium	19,410 (4.6%)	6306 (4.7%)

Abbreviations: ACE, angiotensin converting enzyme; ARB, angiotensin receptor blockers.

Potential Predictors

In the development cohort, 526 diagnoses and drugs had a prevalence above 1 in 1000 individuals. After clinical review, this list was reduced and combined to 150 candidate predictors ().

In the univariate analyses of each of these candidate predictors (including adjustment for age), the highest discriminative ability was seen for high-ceiling diuretics, mental and behavioral disorders due to alcohol, chronic lower respiratory diseases and failure, and pneumonia (). All above mentioned predictors were positively associated with age except mental and behavioral disorders due to alcohol.

Included Predictors and the NMI

The 50 predictors of the NMI selected using the backwards elimination algorithm are shown in Table 2 along with their beta-coefficients and weights. The c-statistic for the model with 50 predictors (0.892) was nearly identical to the c-statistic (0.894) for the full model with all 150 candidate predictors (). The assigned weights of the 50 predictors ranged from −3 to 22. The NMI scores ranged from −5 to 94 with a mean of 3.1 in the development cohort, where approximately half of the population had a score of zero (Table 3). The distribution of the NMI was similar in the temporal validation cohort. In the new user cohorts, the mean NMI score ranged from 5.0 in the statin cohort to 11.0 in the LAMA cohort, and the proportion of individuals with a score of zero ranged from 9% in the LAMA cohort to 30% in the statin cohort.

Table 2

Included Entities, Beta-Coefficients, and Weights in the Nordic Multimorbidity Index

Name	Weighta	β	ICD-10 or ATC Codes
Secondary malignant neoplasms and malignancy of unspecified site	22	2.230	C76-C79, C80
Malignant neoplasm of bronchus and lung	19	1.902	C34
Alcoholic liver disease, liver fibrosis, cirrhosis, and failure	13	1.289	K70, K72, K74, K766-K767
Mental and behavioural disorders due to use of alcohol	12	1.177	F10
Decubitus ulcer and pressure area	11	1.115	L89
Anti-dementia drugs	11	1.112	N06D
Chronic viral hepatitis	10	1.036	B18
Dementia	9	0.857	F00-F03, G30
Leukemia	8	0.847	C91-C95
Malignant neoplasm of bladder	8	0.840	C67
Drugs for constipation	8	0.753	A06A
Tumor of brain or meninges	8	0.751	C70, C71, C751-C753, D32, D330-D332, D352-D354, D42, D430-D432, D443-D445
Multiple sclerosis	7	0.747	G35
Other interstitial pulmonary diseases	7	0.716	J84
Drugs used in opioid dependence	7	0.709	N07BC
Parkinson’s disease and other parkinsonism	7	0.707	G20-G22
Antipsychotics	7	0.696	N05A excl. N05AN
Chronic kidney disease and unspecified kidney failure	7	0.675	N18-N19
Volume depletion	6	0.577	E86
Atherosclerosis, thrombosis, embolism, and other peripheral arterial disease	5	0.536	I70, I73-I74, I77
Iron preparations	5	0.531	B03A
Antipropulsives	5	0.524	A07DA
Diseases of teeth and supporting structures	5	0.505	K02-K06, K08
High-ceiling diuretics	5	0.499	C03C, C03EB
Long-acting anti-muscarinic agents	5	0.491	R03BB04-07
Anaemia	5	0.486	D50-D59, D60-D64
Malignant neoplasm of prostate	5	0.479	C61
Epilepsy	5	0.468	G40-G41
Insulins and analogues	4	0.448	A10A
Pneumonia	4	0.431	J12-J18
Chronic lower respiratory diseases and failure	4	0.422	J41-J44, J47, J961, J969
Digitalis glycosides	4	0.410	C01AA
Malignant neoplasm of breast	4	0.404	C50
Cerebrovascular disease	4	0.398	I60-I69
Aneurysm and dissection of aorta and other arteries	4	0.376	I71-I72
Mental and behavioural disorders due to use of tobacco	4	0.375	F17
Heart failure	4	0.356	I110, I130, I132, I420, I426-I429, I50
Short-acting beta agonists	3	0.311	R03AC02-05
Aldosterone antagonists	3	0.298	C03DA
Antidepressants	3	0.273	N06A
Opioids	2	0.233	N02A
Anilides	2	0.231	N02BE
Type 2 diabetes mellitus	2	0.217	E11
Aortic and mitral valve disease	2	0.214	I05-I06, I34-I35
Glucocorticoids for systemic use	2	0.213	H02AB
Platelet aggregation inhibitors excl. heparin	2	0.155	B01AC
Benzodiazepines and related drugs	1	0.140	N05BA, N05CD, N05CF
Beta-lactam antibacterials, penicillins	1	0.106	J01C
ARBs incl. combinations	−2	−0.175	C09C, C09D
HMG CoA reductase inhibitors (statins)	−3	−0.288	C10AA

Note: aWeights were derived by multiplying the beta-coefficient by ten and rounding to the nearest integer.

Table 3

Distribution of Index Summary Scores in the Development and Validation Cohorts

	Development	Validation	Bisphosphonates	LAMA	Methotrexate	Statins	ULD	Warfarin
Nordic Multimorbidity Index
Median (p25-p75)	0 (0–3)	0 (0–3)	5 (1–13)	8 (3–16)	3 (0–8)	2 (0–7)	7 (1–17)	6 (1–13)
Mean (sd)	3.1 (7.2)	3.0 (6.8)	8.8 (11.0)	11.0 (11.3)	5.5 (7.4)	5.0 (7.5)	10.4 (11.8)	9.1 (10.6)
Index = 0 (%)a	47.1	49.3	17.0	8.9	18.5	30.4	13.8	13.2
Range	−5 to 94	−5 to 88	−5 to 90	−5 to 99	−5 to 69	−2 to 96	−5 to 105	−5 to 100
Charlson Comorbidity Index
Median (p25-p75)	0 (0–0)	0 (0–0)	0 (0–1)	1 (0–2)	1 (0–1)	0 (0–1)	1 (0–2)	1 (0–2)
Mean (sd)	0.3 (0.9)	0.3 (0.8)	0.8 (1.2)	1.1 (1.4)	0.8 (1.1)	0.6 (1.0)	1.2 (1.6)	1.1 (1.5)
Index = 0 (%)a	81.3	83.2	57.7	42.1	47.1	61.9	48.9	46.8
Range	0 to 12	0 to 11	0 to 14	0 to 13	0 to 9	0 to 13	0 to 12	0 to 12
Elixhauser Comorbidity Index
Median (p25-p75)	0 (0–0)	0 (0–0)	0 (0–4)	3 (0–5)	0 (0–0)	0 (0–0)	0 (0–7)	5 (0–8)
Mean (sd)	1.0 (3.1)	0.8 (2.7)	2.3 (4.3)	3.5 (5.0)	1.2 (3.2)	1.4 (3.4)	3.9 (5.8)	5.3 (5.3)
Index = 0 (%)a	80.7	84.1	61.9	43.2	74.0	72.6	51.2	25.3
Range	−15 to 46	−10 to 40	−11 to 44	−10 to 46	−9 to 28	−14 to 44	−12 to 53	−14 to 40

Note: aProportion of individuals with an index summary score of zero.

Abbreviations: LAMA, long-acting muscarinic antagonists; ULD, urate lowering drugs; sd, standard deviation.

The CCI score ranged from 0 to 12 with a mean of 0.3 in the development cohort where 81% had a CCI score of zero (Table 3). For the ECI score, the mean was 1.0 in the development cohort, and 81% had a score of zero.

The c-statistic for the NMI in the temporal validation cohort was 0.887 (95% CI 0.883–0.890) for 5-year mortality (Table 4). Discrimination improved with 1- and 2-year mortality as outcomes, eg, the c-statistic for 1-year mortality was 0.900 (95% CI 0.894–0.906). The NMI added to the discriminative ability compared to a model with age and sex alone where the c-statistic was 0.846, 95% CI 0.842–0.850 (Figure 1 and ). The c-statistic for the NMI exceeded that of the CCI and ECI for 1-, 2-, and 5-year mortality eg, the c-statistic for the CCI to predict 5-year mortality was 0.871 (0.868–0.874). For the new-user cohorts, the c-statistic for the NMI ranged from 0.781 (95% CI 0.779–0.784) in the LAMA cohort to 0.838 (95% CI 0.834–0.842) in the urate lowering drug cohort. In the new user cohorts, the c-statistic of the NMI exceeded that of sex and age alone as well as the CCI and ECI for 1-, 2- and 5-year mortality (Table 4).

Table 4

The c-Statistic with 95% Confidence Intervals for 1-, 2-, and 5-Year Mortality for the Base Model (Age and Sex) and the Base Model in Addition to the Charlson Comorbidity Index, the Elixhauser Comorbidity Index, and the Nordic Multimorbidity Index

	Base Model (Age and Sex)	Charlson Comorbidity Index	Elixhauser Comorbidity Index	Nordic Multimorbidity Index
Development cohort
1-year mortality	0.836 (0.831–0.840)	0.886 (0.882–0.889)	0.882 (0.878–0.886)	0.908 (0.905–0.911)
2-year mortality	0.840 (0.837–0.843)	0.879 (0.876–0.882)	0.875 (0.873–0.878)	0.901 (0.898–0.903)
5-year mortality	0.846 (0.844–0.848)	0.873 (0.871–0.875)	0.870 (0.868–0.872)	0.892 (0.890–0.894)
Temporal validation cohort
1-year mortality	0.839 (0.831–0.846)	0.879 (0.872–0.885)	0.876 (0.869–0.882)	0.900 (0.894–0.906)
2-year mortality	0.842 (0.836–0.847)	0.875 (0.871–0.880)	0.871 (0.866–0.876)	0.894 (0.890–0.899)
5-year mortality	0.846 (0.842–0.850)	0.871 (0.868–0.874)	0.866 (0.863–0.870)	0.887 (0.883–0.890)
Bisphosphonate users
1-year mortality	0.709 (0.703–0.715)	0.785 (0.780–0.790)	0.779 (0.773–0.784)	0.827 (0.823–0.832)
2-year mortality	0.714 (0.709–0.718)	0.780 (0.776–0.784)	0.775 (0.771–0.779)	0.823 (0.820–0.826)
5-year mortality	0.743 (0.740–0.746)	0.792 (0.789–0.795)	0.788 (0.785–0.791)	0.832 (0.830–0.835)
Long-acting muscarinic antagonist users
1-year mortality	0.679 (0.673–0.684)	0.752 (0.747–0.756)	0.744 (0.739–0.749)	0.786 (0.782–0.791)
2-year mortality	0.684 (0.679–0.688)	0.747 (0.743–0.751)	0.741 (0.737–0.745)	0.780 (0.777–0.784)
5-year mortality	0.702 (0.699–0.706)	0.750 (0.747–0.753)	0.745 (0.742–0.748)	0.781 (0.779–0.784)
Methotrexate users
1-year mortality	0.753 (0.733–0.773)	0.788 (0.769–0.807)	0.788 (0.769–0.807)	0.817 (0.799–0.835)
2-year mortality	0.736 (0.721–0.751)	0.770 (0.756–0.785)	0.767 (0.753–0.781)	0.796 (0.782–0.809)
5-year mortality	0.739 (0.729–0.749)	0.767 (0.758–0.776)	0.765 (0.756–0.775)	0.790 (0.781–0.798)
Statin users
1-year mortality	0.765 (0.761–0.769)	0.824 (0.820–0.827)	0.808 (0.804–0.812)	0.838 (0.834–0.841)
2-year mortality	0.756 (0.753–0.759)	0.810 (0.807–0.812)	0.794 (0.791–0.797)	0.826 (0.823–0.829)
5-year mortality	0.756 (0.754–0.758)	0.799 (0.797–0.801)	0.785 (0.783–0.787)	0.817 (0.815–0.818)
Urate lowering drug users
1-year mortality	0.726 (0.718–0.733)	0.802 (0.796–0.808)	0.787 (0.780–0.793)	0.821 (0.815–0.826)
2-year mortality	0.734 (0.728–0.740)	0.803 (0.798–0.808)	0.790 (0.785–0.795)	0.822 (0.817–0.827)
5-year mortality	0.758 (0.754–0.763)	0.818 (0.814–0.822)	0.809 (0.804–0.813)	0.838 (0.834–0.842)
Warfarin users
1-year mortality	0.628 (0.623–0.634)	0.760 (0.756–0.765)	0.718 (0.714–0.723)	0.778 (0.773–0.782)
2-year mortality	0.647 (0.643–0.651)	0.761 (0.758–0.765)	0.724 (0.721–0.728)	0.779 (0.776–0.783)
5-year mortality	0.696 (0.693–0.699)	0.773 (0.770–0.776)	0.747 (0.744–0.750)	0.792 (0.789–0.795)

Figure 1

ROC curves for 5-year mortality for the Charlson Comorbidity Index (CCI), the Elixhauser Comorbidity Index (ECI), and the Nordic Multimorbidity Index (NMI) with and without the base model (age and sex).

We examined whether discrimination of the CCI improved when updated weights were used in a sensitivity analysis. This was not the case with a c-statistic for 5-year mortality in the validation cohort of 0.869 (95% CI 0.866 to 0.873) for the updated weights compared to the c-statistic for the original weights of 0.871 (95% CI 0.868 to 0.874).

The agreement between the expected and predicted 5-year mortality rates (ie, calibration) for the NMI was high in the temporal validation cohort as well as the new user cohorts (Figures 2 and 3).

Figure 2

Calibration of the Nordic Multimorbidity Index (NMI) for predicting 5-year mortality in the development cohort and temporal validation cohort.

Figure 3

Calibration of the Nordic Multimorbidity Index (NMI) for predicting 5-year mortality in the new user validation cohorts.

Discussion

We developed a multimorbidity index, the Nordic Multimorbidity Index (NMI), specifically suited to the comprehensive and population-wide Nordic health- and administrative registries. The NMI score showed good performance in both the general population and in new users of selected drugs and was superior to the CCI and ECI with regard to discriminative ability. The c-statistics were higher for the NMI for 5-year mortality as well as 1- and 2-year mortality and even though the differences in c-statistics appear small, slight improvements in the c-statistics can substantially improve control of confounding bias.30 The NMI is as easily applied as the CCI and ECI, so this increased performance comes at no additional cost to the researcher. A statistical code example to calculate the NMI including the used ICD-10 and ATC codes is available online ().

The NMI differs from the CCI and ECI in several ways that are likely to contribute to the observed increased performance of the NMI compared to existing indices. First, the NMI was developed in the general population in Denmark rather than hospitalized patients in North America. These populations differ with regard to, among others, age, ethnicity, prevalence of chronic conditions, and mortality rate. Second, the NMI was developed in a cohort followed from 2013 to 2018, while the CCI weights were derived based on patients hospitalized in 1984 and the used ECI weights were based on patients hospitalized between 1996 and 2008.10 It is likely that changes in prognosis of conditions over time may also explain some of the observed differences in performance between the NMI, CCI, and ECI. It is also likely that the data infrastructure and coding practice differ between the Nordic countries and North America and that these have changed over time. Third, the weight derivation methods differed between the indices. The NMI weights were derived by multiplying the beta coefficients by ten and rounding to the nearest integer, while, eg, for the ECI, the weights for each condition was its coefficient divided by the coefficient with the smallest absolute value.10 Fourth, the NMI includes prescription drug use, while the CCI and ECI only include chronic conditions defined by diagnosis codes. By including prescription drug use, a wider array of conditions with potential impact on overall mortality are captured presumably increasing predictive performance. Of the 50 items in the NMI, 21 were prescription drugs. Prescription drug data are readily available in the prescription databases for all Nordic countries.15 Since the vast majority of drugs in Denmark are prescribed in the primary care sector, ie, outside of hospitals,31 the individuals that are typically included in pharmacoepidemiologic studies are probably better represented by the general population than by hospitalized patients. The patient registries of the Nordic countries do not include diagnoses from the primary care sector and by including prescription drugs in the NMI, we were able to include data on treatment and hence the presence of chronic conditions from both the primary care sector and the hospital setting. Fifth, the main outcome of 5-year mortality as opposed to in-hospital mortality may be better suited to capture the impact of less acute and milder comorbidities and serve as a more general marker of the underlying health state and multimorbidity. Finally, the distribution of the NMI, CCI and ECI scores differed notably. The proportion of individuals with a score of zero was high for the CCI and ECI, and the poorer performance in these could partly be explained by the zero-inflated distribution of these scores.

Several comorbidity and multimorbidity indices are available besides the CCI and ECI. For example, the Combined Comorbidity Index that combines Charlson and Elixhauser conditions and has been shown to outperform the CCI and ECI.32 It was outside the scope of this study to compare the performance of the NMI to all existing multimorbidity indices, and we chose the CCI and ECI for comparison due to their widespread use.33 The main rationale for developing the NMI was the lack of an index developed and validated in the Nordic setting with tax-supported universal health care and a long-standing tradition for health and administrative registries and availability of prescription drug data from primary care.11,15

The NMI was developed with the purpose to provide a summary score to adjust for confounding by multimorbidity in research and to describe the multimorbidity level of populations in a standardized way. Multimorbidity or comorbidity summary scores are extensively used in health research as they are simple to use, and a single score can substitute the individual comorbidity variables for confounder adjustment reducing the number of parameters that is included in the model allowing for adjustment of more confounding variables than otherwise possible and, in small datasets, reducing the risk of overfitting.34 The use of multimorbidity summary scores as a means of adjusting for confounding also has its limitations. If the study population differs from the population that was used to derive the multimorbidity score, the performance may be inferior compared to directly adjusting for the individual components of the score. It has been argued that study-specific weights should be estimated, ie, that coefficients for each multimorbidity variable should be estimated directly using the study sample.34 Further, effective means of confounder adjustment exist, including propensity scores and disease risk scores that should be preferred when possible. Multimorbidity scores remain useful when high-dimensional approaches are unfeasible, eg, in studies with small sample sizes, in studies with many potential confounders compared to sample size, or in studies where computational efficiency is a concern such as hypothesis-free drug-outcome screening studies. Depending on the study where the NMI is intended to be used, it may be necessary to review the individual items included in the NMI to avoid items closely related to the exposure under investigation, eg, in a study on effects of statin use, the item statins should be omitted from the NMI.

It is important to note that the derived weights of the predictors should not be interpreted causally. For example, the negative weight of −3 for use of statins should not be interpreted as a protective effect of statins towards 5-year mortality. The negative weight may be driven by the joint model (the other variables included in the model) or reflect prescribing practices where healthier patients with long life expectancy are preferably prescribed statins. Similarly, the high positive weight given to drugs for constipation should not be interpreted as an intrinsic toxicity of laxatives but likely reflects constipation as a frequent manifestation in individuals with multimorbidity. This lack of causal interpretation should not be regarded as a limitation. The overarching purpose of NMI is to capture multimorbidity and while the clinical concept of multimorbidity is somewhat elusive in register-based research, a record of, eg, laxative use might capture some of it.

We did not validate the NMI in an external cohort but performed an in-depth validation of a population separated in time and in six new user cohorts representing study populations of typical pharmacoepidemiologic studies. The NMI was developed using Danish registry data and therefore based on coding and treatment practice in Denmark. Even though the structure of registries in the Nordic countries is highly similar, national differences in ICD-10 coding practices and implementation and subsidizing of drugs are likely and should be considered when applying the model to data from other Nordic countries. Further, the NMI may perform differently in selected subpopulations. Thus, the NMI should be validated further in other specific populations and in other Nordic countries.

In conclusion, we developed a multimorbidity score based on current clinical and coding practice in Denmark. The score had a high predictive performance and was superior to the Charlson and Elixhauser comorbidity indices. The NMI can be used as a confounder summary score and to describe the multimorbidity level in epidemiological studies.