Investigation of the international comparability of population-based routine hospital data set derived comorbidity scores for patients with lung cancer.

INTRODUCTION: The International Cancer Benchmarking Partnership (ICBP) identified significant international differences in lung cancer survival. Differing levels of comorbid disease across ICBP countries has been suggested as a potential explanation of this variation but, to date, no studies have quantified its impact. This study investigated whether comparable, robust comorbidity scores can be derived from the different routine population-based cancer data sets available in the ICBP jurisdictions and, if so, use them to quantify international variation in comorbidity and determine its influence on outcome.
METHODS: Linked population-based lung cancer registry and hospital discharge data sets were acquired from nine ICBP jurisdictions in Australia, Canada, Norway and the UK providing a study population of 233 981 individuals. For each person in this cohort Charlson, Elixhauser and inpatient bed day Comorbidity Scores were derived relating to the 4-36 months prior to their lung cancer diagnosis. The scores were then compared to assess their validity and feasibility of use in international survival comparisons.
RESULTS: It was feasible to generate the three comorbidity scores for each jurisdiction, which were found to have good content, face and concurrent validity. Predictive validity was limited and there was evidence that the reliability was questionable.
CONCLUSION: The results presented here indicate that interjurisdictional comparability of recorded comorbidity was limited due to probable differences in coding and hospital admission practices in each area. Before the contribution of comorbidity on international differences in cancer survival can be investigated an internationally harmonised comorbidity index is required. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Can routine hospital discharge data sets derived comorbidity scores be used to make international comparisons to explain differences in lung cancer survival?

While it is feasible to derive comorbidity scores from international routine hospital discharge data sets, the data sets on which they are based are not uniform, making international comparisons difficult.

We carried out assessment of the feasibility, content, face, concurrent and predictive validity, and reliability of Charlson and Elixhauser Comorbidity Scores as well as bed days derived from linked cancer registry and hospital discharge data sets in nine jurisdictions of the International Cancer Benchmarking Partnership to determine the international comparability of comorbidity indices for use in cancer survival analyses.

Introduction

Module 1 of the International Cancer Benchmarking Partnership (ICBP), along with many other studies, have demonstrated large variation in 5-year survival for lung cancer.1 2 Differences are driven by a higher number of excess deaths occurring (most frequently in older people) in the first few months after diagnosis.2 To eliminate these inequalities it is important to understand what drives the differences in early mortality rates.

Many factors have been proposed as potentially explanatory including differences in the quality of the data on which the analyses are based,3–5 stage of disease at presentation,6 delays in diagnosis7 and differences in access to optimal treatments.8 While the current evidence indicates all these factors may contribute to the variations, they do not fully explain the differences.3–6 9

Another potentially important, but less well studied, factor that may affect early mortality rates are differences in levels of comorbidity. Comorbid disease can be defined as other diseases or medical conditions a person may have in addition to the primary disease of interest. Multiple studies have shown that comorbidity can influence cancer outcomes.10 11 In general, increasing severity of comorbidity or number of comorbidities tends to be associated with increasingly poor outcomes and lower rates of potentially curative treatment, and have been shown to differ between countries.10 12–14

The international cancer survival differences observed may, in part, be a result of differences in the prevalence of comorbid disease between jurisdictions. To our knowledge, no international studies have investigated the extent to which differences in comorbidity explain the observed variations in lung cancer survival. While many of the ICBP jurisdictions have used linked cancer registry and hospital discharge data to quantify comorbidity in their own population-based studies of cancer outcome none have compared comorbidity internationally. This study aimed, therefore, to investigate whether comorbidity scores derived from a linked lung cancer registry and hospital discharge data sets in each of the participating ICBP jurisdictions produced sufficiently robust scores that could be used to inform international survival comparisons.

Patients and methods

Study design and data collection

The study population was derived from nine population-based cancer registries in five countries. These included the New South Wales and Victoria cancer registries in Australia, Alberta and Ontario cancer registries in Canada, the Norwegian cancer registry and the English, Northern Irish, Scottish and Welsh registries in the UK.

Anonymised individual cancer registration records for all individuals diagnosed with a first primary lung cancer between 2009 and 2012 (or 2011 in New South Wales) coded to either the International Classification of Diseases (ICD10)15 or the third revision of the International Classification of Diseases for Oncology (ICD-O-3)16 of C34.0-C34.9 were obtained from each cancer registry. Basic information about the cancer and the individual in whom it was diagnosed were provided. This included age, sex, year and basis of diagnosis, tumour morphology and differentiation, stage of disease and information on the site and time interval between any other prior or subsequent cancer diagnoses for that person. In addition, vital status and survival time were provided.

Population-based cancer registry-derived cohorts were linked to corresponding individuals in the hospital discharge data set available for their region. This included: in New South Wales the Admitted Patient Data Collection;17 in Victoria the Victorian Admitted Episodes Dataset;18 in Canada, the Discharge Abstract Database;19 in Norway the Norwegian Patient Register;20 in England the Hospital Episode Statistics data set;21 in Northern Ireland the Patient Administration System; in Scotland the SMR-01 data set;22 in Wales the Patient Episode Data Wales data set.23 For each linked case, several comorbidity measures were extracted relating to the 4–36 months prior to their lung cancer diagnosis. This time period was chosen as it was deemed unlikely to include hospital attendances related to the lung cancer diagnosis itself but be a sufficient time period in which to detect relevant comorbid disease likely to impact on prognosis. The Norwegian patient register had no hospital discharge records available prior to 2008 so Norwegian lung cancer cases potentially had incomplete comorbidity score ascertainment.

Charlson24 25 and Elixhauser26 Indices were used. These are two commonly used measures that quantify comorbidity in epidemiological analyses27–29 and are generated by scoring attendances in hospital that relate to the specific diagnostic groups outlined in table 1. Each hospital discharge data set records the diagnostic reasons behind any hospital admission for a person and the relevant comorbidity was flagged if any of the diagnoses included in the Charlson and Elixhauser Indices occurred in one or more hospital episode for any individual in the study’s cohorts 4 months to 36 months before their lung cancer diagnosis. This posed a number of challenges based on the differing hospital discharge data sets in each jurisdiction. First, different coding systems were used to record diagnoses during attendance in hospital and so, prior to extraction, the various coding systems were aligned in extensive consultation with clinicians and coders from each participating jurisdiction to ensure comparability. The codes used are detailed in online supplementary tables A and B. In addition, the number of diagnoses recorded per episode of care was also variable ranging from six to an unlimited number across the jurisdictions. In order to minimise bias due to the level of diagnostic detail recorded in each jurisdiction, only the first six diagnostic codes in each episode of care were used as six was the minimum any hospital discharge data set would allow.

Table 1

Diagnostic groupings included in the Charlson and Elixhauser Indices

Charlson	Elixhauser
Myocardial infarction	Congestive heart failure
Congestive heart failure	Cardiac arrhythmias
Peripheral vascular disease	Valvular disease
Cerebrovascular disease	Pulmonary circulation disorders
Dementia	Peripheral vascular disorders
Chronic pulmonary disease	Hypertension uncomplicated
Rheumatic disease	Hypertension complicated
Peptic ulcer disease	Paralysis
Mild liver disease	Other neurological disorders
Diabetes without chronic complication	Chronic pulmonary disease
Diabetes with chronic complication	Diabetes uncomplicated
Hemiplegia or paraplegia	Diabetes complicated
Renal disease	Hypothyroidism
Malignancy	Renal failure
Moderate/severe liver disease	Liver disease
Metastatic solid tumour	Peptic ulcer disease excluding bleeding
AIDS/HIV	AIDS/HIV
	Lymphoma
	Metastatic cancer
	Solid tumour without metastases
	Rheumatoid arthritis/collagen vascular diseases
	Coagulopathy
	Obesity
	Weight loss
	Fluid and electrolyte disorders
	Blood loss anaemia
	Deficiency anaemia
	Alcohol abuse
	Drug abuse
	Psychoses
	Depression

The third measure considered was the total number of in-patient days (IPDs) in hospital in the 4–36-month period. This has previously been used as a crude indicator of comorbidity that is not affected by differences in diagnostic coding.30

Each jurisdiction prepared an anonymised data file for their population and submitted it to the central project team via the secure CyberArk system.

On receipt of these data sets, each was assessed to determine adherence to the project data specification. Results were discussed with the relevant registry and any amendments or revisions agreed. Once the data sets were finalised they were pooled and descriptive analyses undertaken comparing the characteristics of the lung cancer populations from the participating jurisdictions and countries.

Data analysis

Summary Charlson and Elixhauser Scores of 0, 1, 2 or ≥3 were created. The Charlson Summary Score was generated with the standard weighting algorithm25 but with information on malignant diagnoses taken from the registry information instead of the hospital discharge data sets. Likewise, a summary Elixhauser Score of 0, 1, 2 or ≥3 was created but, in this index, no weighting system was used for the different diagnostic groups, rather, a score of 1 was given to each disease. Again, information within this summary score on the solid tumour and lymphoma groupings were taken from the supplied registry information rather than the hospital discharge data set. For both summary scores, increasing values indicate higher levels of comorbid disease.

The median and inter-quartile range (IQR) of IPDs (irrespective of diagnostic reason) in the 4–36 months prior to diagnosis were also investigated and individuals grouped into those who spent 0–10 days, 11–25 days and >25 days in hospital.

To investigate the international comparability of these three comorbidity measures their feasibility, validity and reliability were assessed using the criteria outlined in a recent review.31 The first criterion was the feasibility of generating the three comorbidity measures of Charlson, Elixhauser and IPDs across the jurisdictions. The proportion of each population for which they could be determined was, therefore, calculated.

The second criterion was content and face validity, or the degree to which these indicators evaluated the construct of comorbidity they purported to measure. The distribution of the summary comorbidity measures across the populations and their association with 1-year crude survival was therefore calculated and, again, compared across jurisdictions and countries. Survival time was calculated from the date of diagnosis recorded by each registry to death or censoring on 31 December 2013, except for Norway, where patients were followed up till 31 December 2014 and England and Northern Ireland, where patients were followed up till 31 December 2015. Kaplan-Meier survival32 was used to make survival comparisons. Patients with zero survival (n=2830) were given 0.5-day survival to be included in the analysis.

Concurrent validity, or the degree to which each index correlated with the other indices measuring comorbidity across this study cohort, was then investigated. This was done by comparing Spearman’s rank correlation coefficient for each combination of the indices both overall and by jurisdiction. As none of the measures used can quantify comorbidity perfectly, a good correlation was deemed to be in the range of 0.4–0.8.33

The predictive validity, or the extent to which the indices were able to predict 1-year cancer survival, was then assessed. This was done using a rank correlation measure of goodness of fit, concordance statistic (C-statistic), which is the proportion of pairs of observations that are concordant allowing for tied observations. Four models were fitted for each jurisdiction; one for the baseline model (including age, sex and year of diagnosis) and then additional models with these baseline characteristics and each comorbidity index. C-statistics were calculated from each model and compared with the baseline models and each other. Higher C-statistics indicate a better model fit. In addition, to assess goodness of fit, the Akaike information criterion (AIC) was calculated for each of the models. The AIC uses the log likelihood and number of parameters in each model to assess model fit, where the minimum AIC value would indicate the preferred model.34

The final criterion was the reliability or the extent to which the indices measured a stable phenomenon across the populations. This was examined by calculating age-standardised distributions (standardised to an adult cancer population)35 of the incidence of hospital admissions in each of the relevant Charlson and Elixhauser diagnostic categories and comparing across jurisdictions. If extreme differences between jurisdictions were observed evidence was sought from independent sources of data (eg, the Global Burden of Disease studies36 37), and relevant epidemiological, clinical and coding experts, to determine whether these were, or were not, likely to be real differences in the prevalence of disease and, consequently, the interjurisdiction reliability and comparability of the indices assessed.

Results

Study population

A total of 249 381 individuals were diagnosed with lung cancer between 2009 and 2012 across the jurisdictions. Following the exclusions of 15 400 (6.2%) cases that had invalid lung cancer morphologies (0.03%), were death certificate only cases (4.98%) and/or were outside the age range of the study (<15 years or>99 years of age (0.04%)) there remained 233 981 individuals (93.9% of the submitted population) who formed the basis of the study population. Their characteristics are shown in table 2.

Table 2

Characteristics of the study population

Characteristic		New South Wales		Victoria		Alberta		Ontario		Norway		England		Northern Ireland		Scotland		Wales		Total
		n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%
Total		10 098	100.0	9139	100.0	7204	100.0	30 203	100.0	10 411	100.0	1 34 417	100.0	4292	100.0	19 628	100.0	8589	100.0	2 33 981
Sex	Male	6005	59.5	5414	59.2	3689	51.2	15 851	52.5	5830	56.0	74 247	55.2	2490	58.0	10 109	51.5	4759	55.4	1 28 394
	Female	4093	40.5	3725	40.8	3515	48.8	14 352	47.5	4581	44.0	60 170	44.8	1802	42.0	9519	48.5	3830	44.6	1 05 587
Age at diagnosis,  years	<51	529	5.2	455	5.0	333	4.6	1437	4.8	414	4.0	4538	3.4	167	3.9	579	2.9	244	2.8	8696
	51 to 60	1383	13.7	1237	13.5	1104	15.3	4909	16.3	1498	14.4	15 209	11.3	580	13.5	2254	11.5	958	11.2	29 132
	61 to 70	2989	29.6	2586	28.3	2125	29.5	9505	31.5	3486	33.5	38 020	28.3	1325	30.9	5633	28.7	2542	29.6	68 211
	71 to 80	3213	31.8	2990	32.7	2401	33.3	9884	32.7	3295	31.6	45 446	33.8	1469	34.2	7067	36.0	2959	34.5	78 724
	>80	1984	19.6	1871	20.5	1241	17.2	4468	14.8	1718	16.5	31 204	23.2	751	17.5	4095	20.9	1886	22.0	49 218
Tumour site	Main bronchus	861	8.5	630	6.9	367	5.1	1858	6.2	448	4.3	7708	5.7	127	3.0	1414	7.2	611	7.1	13 163
	Upper lobe	4312	42.7	3747	41.0	3777	52.4	15 438	51.1	4069	39.1	55 183	41.1	958	22.3	8626	43.9	3249	37.8	95 047
	Middle lobe	426	4.2	357	3.9	354	4.9	1375	4.6	363	3.5	4595	3.4	56	1.3	515	2.6	302	3.5	7917
	Lower lobe	2433	24.1	2059	22.5	1943	27.0	7625	25.2	2341	22.5	28 856	21.5	424	9.9	4383	22.3	1761	20.5	49 392
	Overlapping lesion	133	1.3	94	1.0	43	0.6	159	0.5	92	0.9	560	0.4	8	0.2	125	0.6	21	0.2	1102
	Unspecified	1933	19.1	2252	24.6	720	10.0	3748	12.4	3098	29.8	37 515	27.9	2719	63.4	4565	23.3	2645	30.8	57 262
Charlson	0	7818	77.4	7203	78.8	5817	80.7	25 757	85.3	8205	78.8	96 860	72.1	3174	74.0	15 294	77.9	6200	72.2	1 76 328
	1	1030	10.2	891	9.7	662	9.2	2228	7.4	1254	12.0	21 656	16.1	607	14.1	2311	11.8	1424	16.6	32 063
	2	563	5.6	469	5.1	332	4.6	1058	3.5	577	5.5	8912	6.6	301	7.0	978	5.0	577	6.7	13 767
	≥3	687	6.8	576	6.3	393	5.5	1160	3.8	375	3.6	6989	5.2	210	4.9	1045	5.3	388	4.5	11 823
Elixhauser	0	7084	70.2	6575	71.9	5508	76.5	24 901	82.4	7882	75.7	84 335	62.7	2904	67.7	14 245	72.6	5485	63.9	1 58 919
	1	1192	11.8	1035	11.3	551	7.6	2126	7.0	1173	11.3	18 779	14.0	515	12.0	2447	12.5	1199	14.0	29 017
	2	814	8.1	645	7.1	465	6.5	1562	5.2	769	7.4	14 592	10.9	394	9.2	1527	7.8	943	11.0	21 711
	≥3	1008	10.0	884	9.7	680	9.4	1614	5.3	587	5.6	16 711	12.4	479	11.2	1409	7.2	962	11.2	24 334
Inpatient bed days	0 to10	8302	82.2	7448	81.5	6301	87.5	27 261	90.3	9126	87.7	1 21 449	90.4	3680	85.7	16 398	83.5	7335	85.4	2 07 300
	11 to 25	959	9.5	889	9.7	459	6.4	1821	6.0	767	7.4	6299	4.7	297	6.9	1603	8.2	594	6.9	13 688
	>25	837	8.3	802	8.8	444	6.2	1121	3.7	518	5.0	6669	5.0	315	7.3	1627	8.3	660	7.7	12 993

Feasibility of generating the comorbidity measures

All the data sets submitted from the participating jurisdiction adhered to the study protocol and, as such, it was feasible to routinely generate these comorbidity measures. The comorbidity measures could, however, only be generated for individuals who had records in both the cancer registry and the hospital discharge data set and, overall, 1.4% of the cancer registry population did not match. This proportion ranged across jurisdictions from 0.2% in Norway to 4.6% in Victoria. Given the low proportion of cases that had to be excluded in all jurisdictions, it was deemed feasible to create these comorbidity measures.

The content and face validity of the comorbidity measures

For the Charlson Index, a total of 57 653 (24.6%) individuals in the study population had a hospital attendance with one or more of the relevant diagnoses. This proportion ranged across jurisdiction from 14.7% in Ontario to 27.9% in England (figure 1). There were also large differences in the distribution of the summary scores between jurisdictions with, for example, only 3.8% of cases in Ontario having the highest Charlson Scores of ≥3 compared with 6.8% in New South Wales. Overall, when grouped by country, the UK had the highest proportion of individuals with a Charlson Comorbidity Score of one or more and Canada the lowest (27.2% vs 15.6% with a score of ≥1, respectively).

Figure 1

Summary Charlson Comorbidity Scores across the participating jurisdictions and countries.

In all jurisdictions, higher Charlson Comorbidity Scores were associated with lower 1-year survival (figure 2). One-year survival differences between Charlson Comorbidity Scores of 0 and ≥3 ranged from an 8% difference in Northern Ireland to 17% difference in Victoria.

Figure 2

Charlson Comorbidity Scores and survival across the participating jurisdictions.

When pooled to create a summary score, the total number of individuals in the study population who had a hospital attendance with one or more of the Elixhauser diagnoses was higher than Charlson at 75 062 (32.1%) due to the higher number of diagnoses included in the index. There were large differences in the distribution of the summary scores across jurisdictions with Ontario having the lowest proportion of cases (17.6%) with a summary score of ≥1 compared with England having the greatest at 37.3% (figure 3). The overall pattern of the distribution of the proportion of the populations with comorbid disease was consistent with the Charlson Index findings, however, with the UK jurisdictions having the highest rates of ≥1 comorbidities at 35.9% and Canada the lowest at 18.7%.

Figure 3

Summary Elixhauser Comorbidity Scores across the participating jurisdictions and countries.

Again, across each of the jurisdictions and countries higher Elixhauser Comorbidity Scores were associated with lower 1-year survival (figure 4). One-year survival differences between Elixhauser Comorbidity Scores of 0 and ≥3 ranged from a 5% difference in England and Wales to 14% difference in New South Wales and Victoria.

Figure 4

Elixhauser and survival across the participating jurisdictions.

The overall median length of stay in hospital was 0 days (IQR 0–1). There were again, however, large differences across jurisdictions ranging from 1 day in both Australian jurisdictions and Scotland and Wales and 0 days in all Canadian jurisdictions (figure 5). The proportion of people with a hospital stay of 10 days or lower was greatest in Canada at 89.7%, similar in the UK at 89.2% and lowest in Australia at 81.9%.

Figure 5

Inpatient bed days category across the participating jurisdictions and countries.

Increasing time in hospital, measured by IPDs in the 4–36 months prior to diagnosis, was associated with lower 1-year survival and this measure of comorbidity provided the most discrimination between groups (figure 6).

Figure 6

Inpatient bed days and survival across the participating jurisdictions.

In summary, within these populations, all the comorbidity measures appeared to have content and face validity.

The concurrent validity of the comorbidity measures

There were strong correlations between Elixhauser and Charlson Comorbidity Scores and good correlations between these measures and IPDs (table 3).

Table 3

Spearman’s rank correlation coefficient of the different comorbidity measures in each jurisdiction and country

	New South Wales	Victoria	Alberta	Ontario	Norway	England	Northern Ireland	Scotland	Wales	Australia	Canada	Norway	UK	Total
Charlson and Elixhauser	0.82	0.80	0.86	0.83	0.82	0.82	0.84	0.80	0.83	0.81	0.84	0.82	0.82	0.82
Elixhauser and inpatient days	0.59	0.62	0.62	0.60	0.57	0.44	0.57	0.55	0.53	0.60	0.60	0.57	0.45	0.50
Charlson and inpatient days	0.53	0.53	0.58	0.57	0.55	0.42	0.56	0.52	0.50	0.53	0.57	0.55	0.43	0.46

The predictive validity of the comorbidity measures

Table 4 illustrates the C-statistics and AIC from the logistic regression models investigating the predictive validity (or the extent to which they predict 1-year mortality) of the comorbidity indices. The C-statistics show that the inclusion of any of the comorbidity measures in the baseline models did little to improve their predictive ability.

Table 4

Comparison of concordance (C) statistics and Akaike information criterion (AICs) from logistic regression models predicting death within 1 year from diagnosis for each jurisdiction and country. For ease of interpretation the lowest AIC is set to 0, with the difference between that model’s AIC and the other models’ AIC represented in the table.

	New South Wales		Victoria		Alberta		Ontario		Norway		England		Northern Ireland		Scotland		Wales
	c	AIC	c	AIC	c	AIC	c	AIC	c	AIC	c	AIC	c	AIC	c	AIC	c	AIC
Baseline model	0.62	35	0.62	26.04	0.59	12.6	0.58	19.32	0.63	2.53	0.61	110	0.60	13.7	0.60	39.8	0.61	4.9
+Charlson	0.62	16.8	0.62	0	0.59	6.47	0.58	0	0.63	0	0.61	15.9	0.60	15	0.60	30.2	0.61	2.7
+Elixhauser	0.62	6.7	0.62	4.53	0.59	10.5	0.58	6.43	0.63	6.17	0.61	0	0.60	16.5	0.60	30.2	0.61	8.2
+IPD	0.62	0	0.62	7.54	0.59	0	0.58	2.89	0.63	3.28	0.61	78	0.60	0	0.60	0	0.61	0

IPD, in-patient days.

The reliability of the comorbidity measures

Age-standardised rates per 100 000 of the lung cancer population produced for each Charlson diagnostic category across the participating jurisdictions are included in online supplementary file C. Large variation was observed in these rates, which suggested coding practices may vary substantially by jurisdiction. For example, although the difference was not statistically significant the rate of diabetes without chronic complication was high in England (5979 per 100 000 cancer population (95% CI 5849 to 6112)) and low in New South Wales (2696 per 100 000 (95% CI 2383 to 3049)) and Victoria (2493 per 100 000 (95% CI 2173 to 2856)). Conversely, when diabetes with chronic complication was examined, its prevalence was low in England (572 per 100 000 population (95% CI 532 to 615)) but significantly higher in New South Wales (5200 per 100 000 population (95% CI 4784 to 5654)) and Victoria (4831 per 100 000 population (95% CI 4416 to 5285)) populations. These findings suggest that the coding of diabetes with and without chronic complication may be inconsistent internationally.

The age-standardised prevalence per 100 000 patients with lung cancer produced for each Elixhauser diagnostic category across the participating jurisdictions are included in online supplementary file D. As with the Charlson Index, large variation in these proportions were observed that suggested coding practices may vary substantially by jurisdiction. Again, prevalence of the two Elixhauser classifications of diabetes (uncomplicated and complicated) appeared to be transposed in Australia and UK jurisdictions. For example, the hospital attendance per 100 000 cancer population for uncomplicated and complicated diabetes in New South Wales were 2378 (95% CI 2090 to 2704) and 5291 (95% CI 4871 to 5748), respectively, compared with the equivalent English figures of 5971 (95% CI 5841 to 6105) and 595 (95% CI 554 to 640). Again, these findings were consistent between all jurisdictions in the UK and Australia and suggested coding practice between the countries may be transposed.

Discussion

This is the first study to make international comparisons of commonly used measures of comorbidity derived from routine, population-based administrative health data sets. It has shown it was feasible to create the Elixhauser, Charlson and length of stay measures of comorbidity from the available data in each jurisdiction, that they appeared to be valid and that they predicted short-term survival. The study was less able, however, to determine the interjurisdictional comparability (or reliability) of these measures. For example, the Charlson Index did not appear to support interjurisdictional comparisons due to differences in coding practice and the weighting of the score. The Elixhauser and length of stay measures appeared to be more robust but the existence of differences in coding, recording or admission practice cannot be discounted. Across the indices, however, similar patterns were observed with the prevalence of comorbid disease in the lung cancer population being highest in the UK and lowest in Canada. Although the Norwegian data did not show any obvious deviation from the observed difference between the jurisdictions, the interpretation of the data from Norway needs to be treated with caution since no hospital discharge records prior to 2008 were available.

In a recent review31 it was noted that, despite the importance of comorbidity in cancer care, there is no consensus about how to define or measure it and no gold standard approach for quantifying it. While the Charlson and Elixhauser Indices scored moderately well in the review, it concluded that further work was required. This is the first study to investigate the use of these measures in international comparisons. It was shown that while it is feasible to create them, and they were broadly predictive of survival, there are limitations.

Principally, we have been unable to quantify the level of international comparability of these comorbidity measures. Each jurisdiction has its own routine administrative data sets and each of these has distinct processes and incentives for coding, capturing and collating the data. Charlson and Elixhauser Indices rely on diagnostic coding within these data sets and this study suggests there are international differences in this practice. In an effort to address this independent sources of data on international differences in mortality36 and years of life lost37 have been sought and comparisons made with the results obtained from this study. While not specific to patients with lung cancer these independent sources provide reassurance that the overall patterns of comorbid disease in each jurisdiction appear broadly similar to those detected in this study and, perhaps, confirm the fact that differences in coding practice exist. The extent of the differences cannot, however, be quantified as the majority of these independent studies used the same, or similar, data sources to derive their rates and, as a result, may suffer the same biases or measurement errors as this study. In the absence of truly independent data, such as those that would come from a high resolution study involving detailed review and standardised coding of the medical histories of population-based cohorts of individuals from each jurisdiction, it would not be possible to truly quantify any relevant differences in these data sets and, hence, the impact it may have on the comorbidity measures derived.

In addition, there may be other factors limiting comparability. For example, there may be international differences in each jurisdiction’s population preference for attending primary or secondary care or differences in propensity to keep patients in secondary care and, hence, length of stay in hospital. Again, it is difficult to determine to what extent such factors are influencing our findings and, hence, determine the comparability of the scores. As a result, other types of data sets may be more suited for international comparisons of comorbidity such as primary care prescribing data sets. These can include information on all drugs administered to each person and as such allow a more robust quantification of the type and extent of disease experienced in each population. Such data sets are not, however, routinely available or as readily accessible as the hospital discharge data used in this study so further work is required to investigate their utility and value to generating the internationally comparable comorbidity scores demanded.

A lack of comparability of key prognostic measures for cancer between jurisdictions has been previously observed. For example, international comparisons of the influence of stage of disease at diagnosis on survival have been hindered by differences in the recording of stage between jurisdictions.6 9 Likewise, there is also evidence that differences in the registration practices of cancer registries may account for some of the variability in survival observed.38 While such differences do not appear to entirely explain the international survival variation observed they remain influential. Significant further work to standardise data collection is required therefore, to enable robust interjurisdictional comparisons.

Another limitation of our study is, perhaps, the relatively short survival time of those diagnosed with lung cancer. With survival being low and early mortality high even among healthier patients the additional effect of comorbid disease over and above the cancer and its management on the number of people who die rapidly may be relatively small.39–41

This links to another concern regarding how to apply the different comorbidity indices. In this study only diagnostic reasons for attendances in hospital in the 4–36 months prior to diagnosis have been included. This may well lead to an underestimate of comorbidity levels as it excludes any comorbid conditions that may be recorded in episodes associated with the diagnosis of the cancer. However, lung cancer can be associated with a range of specific diagnoses, especially COPD and lower respiratory tract infections plus paraneoplastic syndromes that result in a variety of symptoms and signs, which make it impossible to robustly distinguish between any diagnoses that were related to the lung cancer itself and those that would be considered true comorbidities. For this reason conditions recorded within the 3 months immediately preceding the diagnosis of cancer were excluded but it is recognised this may lead to a concomitant reduction in comorbidity rates in the populations studied. Moreover, for nearly half of the Norwegian patients, record linkage did not go back as far as 36 months prior to diagnosis, which may limit comparability.

Finally, in this study we have investigated only two commonly used comorbidity measures that use diagnostic reasons for attendances in hospital, but we recognise that other systems and methods also exist. Further work is required to assess the most appropriate to use and, hence, determine the most robust approach to quantifying international comorbidity levels.

Improving cancer survival, and hence eliminating international differences in outcomes, are priorities for many health services.42 Comorbidity may have a profound impact on these survival rates having the potential to strongly influence fitness for treatment and its extent,10 side effects, costs, quality of life and, ultimately, outcome. Given the increasing levels of comorbidity in the ageing populations of each of the ICBP jurisdictions its presence is of growing importance to those who provide cancer services.43 While this study has shown it is feasible to generate crude indicators of comorbidity in each of the jurisdictions it has not been able to prove that these measures are internationally comparable. Before there can be any quantification of the contribution of comorbidity on the differing levels of survival obtained across the ICBP partnership, it is essential that more comparable measures of comorbidity are derived.