The impact of worldwide, national and sub-national severity distributions in Burden of Disease studies: A case study of cancers in Scotland.

BACKGROUND: Increasingly Burden of Disease (BOD) measures are being used to influence policy decisions because they summarise the complete effects of morbidity and mortality in an equitable manner. An important element of producing non-fatal BOD estimates are severity distributions. The Global Burden of Disease (GBD) study use the same severity distributions across countries due to a lack of available country-specific data. In the Scottish BOD (SBOD) study we developed national severity distributions for cancer types. The main aim of this study was to consider the extent to which the use of worldwide severity distributions in BOD studies are influencing cross-country comparisons, by comparing weighted-average disability weights (DW) based on GBD severity distributions with nationally derived severity distributions in Scotland for cancer types.
METHODS: We obtained individual records from the Scottish Cancer Registry for 21 cancer types and linked these to registered deaths. We estimated prevalent cancer cases for 2016 and assigned each case to sequelae using GBD 2016 study definitions. We compared the impact of using severity distributions based on GBD 2016, a Scotland-wide distribution, and distributions specific to deprivation strata in Scotland, on the weighted-average DW for each cancer type.
RESULTS: The relative difference in point estimates of weighted-average DW based on GBD 2016 worldwide severity distributions compared with Scottish national severity distributions resulted in overestimates in the majority of cancers (17 out of 21 cancer types). The largest overestimates were for gallbladder and biliary tract cancer (70.8%), oesophageal cancer (31.6%) and pancreatic cancer (31.2%). Furthermore, the use of weighted-average DW based on Scottish national severity distributions rather than sub-national Scottish severity distributions stratified by deprivation quintile overestimated weighted-average DW in the least deprived areas (16 out of 18 cancer types), and underestimated in the most deprived areas (16 out of 18 cancer types).
CONCLUSION: Our findings illustrate a bias in point estimates of weighted-average DW created using worldwide severity distributions. This bias would have led to the misrepresentation of non-fatal estimates of the burden of individual cancers, and underestimated the scale of socioeconomic inequality in this non-fatal burden. This highlights the importance of not interpreting non-fatal estimates of burden of disease too precisely, especially for sub-national estimates and those comparing populations when relying on data inputs from other countries. It is essential to ensure that any estimates are based upon country-specific data as far as possible.

Introduction

Burden of Disease (BOD) studies include both morbidity and mortality by framing them in terms of health loss suffered as a function of time [1]. Estimates of the frequency of morbidity in a population, such as prevalence, are transformed into Years Lived with Disability (YLD) using disability weights (DW) for each disease-specific sequelae. The proportional distribution of sequelae within a disease is commonly referred to as the severity distribution [2]. Recent advances in the Global Burden of Disease (GBD) study has seen the number of disease sequelae triple from 1,160 in GBD 2010 [3] to 3,484 in GBD 2017 [4]. While advances in the development of the granularity of disease and sequelae have been made, they have not been matched by the development of country-specific severity distributions [2]. The GBD study produces country-specific modelled estimates of prevalence for each non-fatal cause of disease, but the allocation of sequelae prevalence is largely defined using the same severity distributions in each country. This has prompted the development of country-specific severity distributions [5-7], although the impact of using these in favour of GBD severity distributions has not yet been evaluated.

There is a growing appetite for BOD studies to provide more granular estimates for regions within countries in both the GBD study [8-10] and independent national studies, particularly in Europe [5, 6, 11, 12], to influence local policy development. This raises further questions about the utility of worldwide severity distributions in sub-national calculations. Findings from the GBD UK study raised some concerns over the accuracy of local estimates of YLD, noting that country-specific electronic health records have a role to play in refining estimates [8]. Whilst electronic health records can inform better disease estimates, it is important to advocate for and develop better prevalence estimates at the disease sequelae level to remove the reliance of worldwide severity distributions. Disability associated with different levels of sequelae varies significantly and thus has a major impact on YLD estimates. In short, if we assume a fixed proportion of prevalence in each sequelae within a disease across countries or in sub-populations within a country, we risk introducing a systematic bias in the YLD results.

The main aim of this study was to consider the extent to which worldwide severity distributions are influencing cross-country comparisons by comparing weighted-average DW based on GBD severity distributions with nationally derived severity distributions in Scotland across a range of cancer types. A secondary aim was to investigate the impact of sub-national severity distributions compared to nationally derived severity distributions, by comparing weighted-average DW from both scenarios across a range of cancers.

Methods

Data

There were 21 cancer types included in this study. Details of the cancer types included and excluded in this study are available in the online supplementary appendix (S1 File). Cancer types were restricted to those that had four common sequelae: (i) diagnosis and primary therapy phase; (ii) controlled phase; (iii) metastatic phase; and (iv) terminal phase. This approach was chosen to avoid attribution of differences due to interpretation of the GBD 2016 model when dealing with additional specific sequelae, such as procedural [13]. For example, the GBD 2016 breast cancer model has an additional mastectomy sequelae which includes details of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes [14] used to define the procedure. However, in Scotland procedural codes are defined using Office of Population Censuses and Surveys Classification of Interventions and Procedures version 4 (OPCS-4) codes [15]. Assuming equivalence and suitability of national procedural recording systems with the GBD model may have introduced an undue bias, which for the purposes of this comparison we have chosen to avoid. DW for each of the four sequelae of cancer were derived from the GBD 2016 study [16]. Unadjusted DW were used to illustrate the theoretical scale of effect. The worldwide prevalence of the sequelae of each cancer type and corresponding 95% uncertainty intervals (UI) were sourced from GBD 2016 [17] and apportioned into severity distribution proportions.

The GBD 2016 modelling process involved calculating cancer incidence directly from cancer mortality using mortality to incidence ratios (MIR) [13]. These MIR were based on data from locations that reported both incidence and mortality data, or where high quality mortality estimates were available but not reported. Relative cancer survival was determined between a ‘best’ and ‘worst’ case scenario. The ‘best’ case was determined based on 2010 data from the Surveillance, Epidemiology, and End Results (SEER) program in the United States [18], with the worst case being determined on whichever was the lowest between 1950 US Mortality Files [19] compared to Cancer Survival in Africa, Asia, the Caribbean and Central America (SurvCan) data [20]. Estimates of cancer survival for individual countries were scaled between the ‘best’ and ‘worst’ case based on an access to care model and these scalars were used to transform incidence to prevalence.

As part of the Scottish Burden of Disease (SBOD) 2016 study, national and sub-national severity distributions were derived using individual patient records from the Scottish Cancer Registry, which holds registration records from 1980 onwards of all incident cancers diagnosed within the NHS in Scotland [21]. The disease model used to define the sequelae of each case was developed using definitions from the GBD 2016 technical appendix [13]. We calculated 10-year prevalence of the incidence cohort for each cancer type to establish prevalent cases. Prevalent cases were apportioned to each sequelae using fixed durations found in the GBD 2016 technical appendix for the diagnosis and primary therapy phase, metastatic phase and terminal phase. Cases were assigned to the controlled phase if they did not satisfy the time-based criteria of the other three sequelae.

Patients were followed up over time and the date and cause of death were obtained from the National Records of Scotland’s register of deaths [22]. Using deterministic matching [23] of a patient-identifier between the cancer registry and register of death, we could confidently classify and exclude cases in accordance with the GBD fixed duration cancer survival model definitions for each cancer type [13]. The operationalisation of the allocation to individual phases was carried out by converting the GBD 2016 fixed durations to days, and applying from the date of the incident registration on the Scottish Cancer Registry. The availability of specific death dates meant that the time to death from incident registration could be calculated in days for both exclusion and classification purposes. Due to the ability to link patient records and the nature of the registrations on Scottish Cancer Registry, we did not require persistent identification of ICD-10 codes during the follow-up period.

An area-based deprivation score was available for each patient, defined by the Scottish Index of Multiple Deprivation 2016 [24], which allowed us to create severity distributions for patients living in the most deprived fifth of areas, and for those living in the least deprived fifth of areas in Scotland. The same fixed durations were used to classify sequelae across all methods.

Analyses

Severity proportions of sequelae were calculated for each cancer type and assessed for differences with GBD 2016 95% UIs. For ease of comparison we opted to summarise the effect of severity distributions in a single point estimate. We used sequelae prevalence in conjunction with DW that were sourced from the GBD 2016 study [16] to calculate point estimates of the weighted-average DW of each cancer type, using the following formula:

This involved summing the product of the multiplication between the number of prevalent cases for each sequelae by their corresponding sequelae disability-weight, across all four sequelae, and then dividing the sum by the total number of prevalent cases across all sequelae. The resulting weighted-average DW can be interpreted as the YLD suffered for a single individual, where higher values indicate an increasing proportion of time experiencing non-fatal health loss. The weighted-average DW is a useful summary measure which incorporates the relative frequency of severity of each sequelae in a single measure. As disability weights were the same for each individual cancer, this measure is useful in summarising the average non-fatal health loss for each cancer type. Point estimates of the weighted-average DW for each cancer type were calculated based on four different severity distributions scenarios: (i) GBD 2016 worldwide; (ii) Scotland overall; (iii) the most deprived fifth of local areas in Scotland; and (iv) the least deprived fifth of local areas in Scotland. Relative and absolute differences in the point estimate of weighted-average DW were assessed between approaches (i) and (ii) as the primary outcome. Absolute differences were presented by rescaling the DW into days of health loss by multiplying the DW by 365.25 days (where the additional 0.25 of a day takes into account the occurrence of a leap year every four years).

In addition, relative and absolute differences between (ii) and (iii), and (ii) and (iv) were assessed as secondary outcomes. The secondary analyses were restricted to 18 cancer types. Three cancer types (gallbladder and biliary tract, mesothelioma and nasopharynx) were excluded in the sub-national analyses because their deprivation-stratified severity distributions were based on a total number of prevalent cases less than 100.

Data permissions and access

Formal permission to access linked National Health Service (NHS) administrative databases was granted by the Privacy Advisory Committee, NHS National Services Scotland (NSS) [PAC Reference 51/14] [25]. Patient-identifiable data extracts were extracted by NHS NSS and provided to NHS Health Scotland in the form of aggregate statistics that were subsequently used in this study. All summary data used in this study are provided in the online supplementary appendix (S1 File).

Results

Scottish national severity distributions for cancer types

The total number of prevalent cases, severity distribution across the four common cancer sequelae and weighted-average DW for 21 cancer types for Scotland in 2016 are listed in Table 1. Additional data on the number of prevalent cases for sub-national severity distributions based on the most and least deprived fifth of local areas in Scotland are outlined in the online supplementary appendix (S1 File). Across the 17 cancer types with permissible publication of severity data, the proportion of prevalent cases in the controlled phase was greater than the combination of the other three phases for 16 cancer types. The only exception to this was mesothelioma which had the highest proportion of prevalent cases in the metastatic phase (0.395) of all cancer types.

Table 1

Total number of prevalent cases, severity distribution and weighted-average disability weight by cancer type, Scotland, 2016.

Cancer type	Number of prevalent cases	Proportion of prevalent cases in each phase/sequelae a, b				Weighted-average disability weight
		Diagnosis and primary therapy phase	Controlled phase	Metastatic phase	Terminal phase
Brain and nervous system	978	0.091	0.724	0.153 b	0.032 b	0.148
Cervical	2,343	0.056 a	0.903	0.036 a	0.005 a	0.079
Gallbladder and biliary tract	359	0.114 a	0.719 b	0.123 a	0.047 a	0.148
Hodgkin lymphoma	1,202	Data not available at sequelae level				0.078
Kidney	4,592	0.078 a	0.865	0.047	0.010 b	0.091
Lip and oral cavity	2,970	0.062 a	0.853 b	0.075 a	0.010 a	0.099
Liver	927	0.124 a	0.744 b	0.086 a	0.045 a	0.136
Malignant skin melanoma	8,905	0.033 a	0.941	0.023	0.003 a	0.068
Mesothelioma	228	0.123 b	0.408 a	0.395 b	0.075 b	0.273
Multiple myeloma	1,971	0.105 b	0.616	0.264 a	0.015 a	0.187
Nasopharynx	131	Data not available at sequelae level				0.107
Non-Hodgkin’s lymphoma	5,885	0.043 a	0.902	0.048 a	0.008 a	0.082
Oesophageal	1,543	0.134 b	0.669 b	0.154 a	0.043 a	0.164
Other pharynx	1,155	0.078 a	0.828 b	0.081 a	0.013 a	0.107
Ovarian	2,715	0.033 a	0.801 b	0.154 a	0.011 a	0.125
Pancreatic	662	0.165 a	0.600 b	0.148 a	0.088 a	0.191
Stomach	1,343	0.110 a	0.754 b	0.102 a	0.034 a	0.133
Testicular	1,895	Data not available at sequelae level				0.061
Thyroid	1,811	Data not available at sequelae level				0.069
Tracheal, bronchus, and lung	7,642	0.090 b	0.714 b	0.149 a	0.047 a	0.153
Uterine	4,809	0.055 a	0.886	0.054 b	0.006 b	0.086

The number of prevalent cases were supressed for all four sequelae if one or more of the sequelae had a value of 9 or less, as per the NHS National Services Scotland disclosure protocol. All data were retained for use in intermediate calculations of weighted-average disability weights.

Shaded cells within the table indicate that the Scottish severity proportion lies outside the GBD 2016 severity proportion 95% uncertainty interval (UI).

a Scottish severity proportion lies below the lower limit of the GBD 2016 severity proportion 95% UI.

b Scottish severity proportion lies above the upper limit of the GBD 2016 severity proportion 95% UI.

The five cancer types with the highest weighted-average DW were: mesothelioma (0.273); pancreatic cancer (0.191); multiple myeloma (0.187); oesophageal cancer (0.164); and tracheal, bronchus, and tracheal, bronchus and lung cancer (0.153). Conversely the five cancer types with the lowest weighted-average DW were: testicular cancer (0.061), malignant skin melanoma (0.068); thyroid cancer (0.069); Hodgkin lymphoma (0.078); and cervical cancer (0.079).

Comparing Scottish severity proportions to the GBD 2016 95% UI for each cancer type yielded several differences. Of the 17 cancers where Scottish severity proportions could be published, the Scottish severity proportion for the diagnosis and therapy phase was not contained within the GBD 2016 95% UI in 16 cancer types. For 12 cancer types this was due to the Scottish severity proportion being lower, with it being higher for the remaining 4 cancer types. In the controlled phase the Scottish severity proportion was not contained within the GBD 2016 95% UI for 10 cancer types (1 below; 9 above). For the metastatic phase, 15 cancer types had a Scottish severity proportion that lay outside the GBD 2016 95% UI (12 below; 3 above). Finally, for the terminal phase the Scottish severity proportion was not contained within the GBD 2016 95% UI in all 17 cancer types (13 below; 4 above).

GBD 2016 worldwide compared to Scottish national severity distributions

The relative difference across cancer types between the point estimate of the weighted-average DW based on GBD 2016 worldwide severity distributions compared with Scottish national severity distributions resulted in positive values across 17 out of 21 cancer types (Fig 1). In these 17 cancer types, using GBD 2016 severity distributions would have resulted in a relative overestimate of the point estimate of the weighted-average DW. The largest relative overestimates were observed for: gallbladder and biliary tract cancer (70.8%), oesophageal cancer (31.6%) and pancreatic cancer (31.2%). There were four instances where the use of GBD 2016 severity distributions would have resulted in a relative underestimate of the point estimate of the weighted-average DW: mesothelioma (-20.9%), brain and nervous system cancer (-7.1%), uterine cancer (-4.8%) and thyroid cancer (-2.4%).

Fig 1

Relative and absolute comparison of cancer disability weights: GBD 2016 worldwide versus Scottish national.

In terms of absolute difference, the largest overestimates would have been made for gallbladder and biliary tract cancer (38.3 additional days of health loss per year, per person), pancreatic cancer (21.7 additional days of health loss per year, per person) and oesophageal cancer (18.9 additional days of health loss per year, per person). The largest absolute underestimate would have been made for mesothelioma (20.9 fewer days of health loss per year, per person).

Scottish national compared to sub-national severity distributions

When the relative difference between the point estimates of the weighted-average DW based on Scottish national severity distributions was assessed against severity distributions based on the most deprived fifth of Scottish areas, there were negative values in 16 out of 18 cancer types (Fig 2). In these 16 cancer types, using nationally derived severity distributions would have resulted in a relative underestimate of the point estimate of the weighted-average DW. The largest relative underestimates were observed for: lip and oral cavity cancer (-11.6%), oesophageal cancer (-8.4%) and other pharynx cancer (-8.1%). There were two instances where the use of Scottish national severity distributions would have resulted in a relative overestimate of the point estimate of the weighted-average DW: brain and nervous system cancer (10.3%), and Hodgkin lymphoma (0.1%).

Fig 2

Relative and absolute comparison of cancer disability weights: Scottish national versus most deprived fifth of Scottish areas.

The cancer types where the largest absolute underestimates would have been made were for oesophageal cancer (5.5 fewer days of health loss per year, per person), lip and oral cavity cancer (4.7 fewer days of health loss per year, per person) and multiple myeloma (4.6 fewer days of health loss per year, per person). The largest absolute overestimate would have been made for brain and nervous system cancer (5.0 additional days of health loss per year, per person).

The relative difference across cancer types between the point estimate of the weighted-average DW based on Scottish national severity distributions compared to severity distributions based on the least deprived fifth of Scottish areas resulted in positive values across 16 out of 18 cancer types (Fig 3). In these 16 cancer types, using nationally derived severity distributions would have resulted in an overestimate of the weighted-average DW. The largest relative overestimates were observed for: other pharynx cancer (16.9%), pancreatic cancer (9.9%) and lip and oral cavity cancer (8.7%).

Fig 3

Relative and absolute comparison of cancer disability weights: Scottish national versus least deprived fifth of Scottish areas.

The largest absolute differences in overestimation were observed for pancreatic cancer (6.3 additional days of health loss per year, per person), other pharynx cancer (5.6 additional days of health loss per year, per person) and tracheal, bronchus and lung cancer (3.9 additional days of health loss per year, per person).

The magnitude of the differences in point estimates of weighted-average DW were much larger when comparing GBD 2016 worldwide with nationally derived severity distributions, than when comparing differences between national and sub-national severity distributions.

Discussion

Summary of findings

This study is the first to evaluate the impact of using published GBD 2016 worldwide, compared to national and sub-national severity distributions on weighted-average disability weights. Our study illustrated that the majority of point-estimates of Scottish national sequelae severity proportions for cancers lay outside the uncertainty intervals given in the GBD 2016 study. By summarising the effects of severity distributions in a single weighted-average disability weight, we illustrated that GBD 2016 worldwide severity distributions would have led to overestimation in the point estimate of disability weights assigned to individual cancer types in a large majority of cases. The range of overestimation varied in size, with the largest relative overestimates being observed in gallbladder and biliary tract cancer (70.8%), oesophageal cancer (31.6%) and pancreatic cancer (31.2%). Although it was less common, there were some instances whereby using GBD 2016 worldwide severity distributions would have resulted in relative underestimation. This was observed for mesothelioma (-20.9%), brain and nervous system cancer (-7.1%) and uterine cancer (-4.8%).

Additionally we have illustrated the smaller additional benefit in considering differences in severity distributions across sub-regions of a country. Our assessment of sub-national severity distributions stratified by area-based deprivation quintiles indicated that the use of Scottish national severity distributions would have led to an underestimate of the point estimate of the weighted-average disability weight in the most deprived areas of Scotland across the majority of cancer types. Conversely, the use of Scottish national severity distributions would have led to overestimates of point estimates of weighted-average disability weights in the least deprived areas of Scotland. Considering both these findings, using national severity distributions would have understated the extent of socioeconomic inequalities in YLD associated with individual cancer types.

How this compares with existing literature

Existing studies of BOD severity distributions are limited. Those already published have been designed to develop severity distributions and have not evaluated the impact of using different severity distributions [2, 7]. A study in Korea estimated severity distributions for eight diseases using two independent national surveys [7]. Both methods resulted in similar patterns of severity across all eight diseases. The GBD severity distributions study acknowledges concerns over applying estimates of severity distributions based on data from the United States and Australia, noting that it is the only available information that they were able to use [2].

Strengths and weaknesses

A major strength of this study lies within the use of the Scottish Cancer Registry [21] with patient-linked death registrations records [22, 23]. This allowed us to precisely classify each case to an exact sequelae. The availability of a patient postcode of residence on each cancer registration record allowed us to classify each case to an area-based deprivation quintile. The transparency of the GBD 2016 models appendix [13] allowed us to utilise the definitions and allocate cases on a like-for-like basis. Whilst the use of patient-linked electronic health records is a major strength in our study, we acknowledge that for many countries it would be impractical to obtain estimates of severity for sub-strata of the population, or even nationally due to lack of data availability or access permissions. Additionally issues may arise for less frequent conditions or small population sizes [26]. Our estimates were made for a single calendar year and have not been assessed for stability over time as study permissions were not designed to report a time series. If there were significant fluctuations across time, our recommendation would be to pool data across several years and carry out sensitivity analyses of the results of moving averages.

We have chosen to produce sub-national severity distributions based on deprivation, as our secondary focus was around local area estimates with a focus on socioeconomic inequalities. We also acknowledge the limitation that deprivation is not the solely significant factor and that on a disease by disease basis, other stratifications such as ethnicity or gender may be more appropriate and display important intersectional effects [27]. In addition we also note that our study has only assessed one of the aspects of potential error and bias in estimates; severity distributions. There are other aspects such as the disability weight assigned to a particular health state that could also bias estimates. At present the same four disability weights are used across each cancer sequelae. Further research is needed to assess whether these estimates are appropriate, or whether disability weights across cancers, and indeed other causes, are context-specific and socially influenced [28]. We have applied the GBD fixed cancer survival durations across our comparisons. Given that survival times vary across and within countries, our findings on the scale of difference are a likely to be a conservative account. Survival has been shown to vary significantly over time. In Scotland, adjusted 5-year survival from colorectal cancer has improved by 5.4% from 2002–06 to 2007–11 [29]. Similar findings for the same time periods indicated a 2.4% increase in adjusted 5-year survival for lung cancer [30]. This raises the further need for the effects of temporal changes in severity to be considered.

Implications for research and policy

This study has important implications for policy and practice. In highlighting potential biases in the over and underestimation of point estimates of disability weights, we have illustrated a proportional bias that will carry through to estimates of YLD. In terms of cancer, the magnitude of bias on Disability-Adjusted Life Years (DALYs) would be minor because the majority of DALYs are generated through Years of Life Lost to premature mortality (YLL). However the bias remains for cancer YLD estimates and these concerns extend to other causes of disease, particularly leading causes of YLD and causes of YLD that exhibit wide inequalities [31-34].

Further research is required to understand the impact of severity distributions for conditions that may have larger inequalities in case fatality, or severity, than cancers to explore differences across a larger range of diseases. Caution should be taken over interpreting the applicability of our cancer severity distributions to other countries due to differences in demography, social circumstance and access to healthcare systems across countries. There are large resource requirements in undertaking BOD studies. Both independent BOD researchers and collaborators of the GBD need to know where to focus their efforts in order to obtain the maximum benefit of information. Our results have illustrated the importance of ensuring that any estimates are based upon the best available country-specific data, in the context of cancer types, with an additional benefit of using data at lower levels of granularity. BOD researchers should work in collaboration with disease experts in their country to understand sequelae and disability weight definitions to assess for equivalence within country-specific data sources, such as nationally representative surveys.

BOD studies are a means to influence policy and practice and these findings are important in highlighting a systematic bias in the point estimates that are being used to rank causes that is largely being overlooked. To a degree, the lack of sub-national severity distributions would negate each other in national results, but would mask differences between regions that have heterogeneous levels of deprivation and therefore underestimate the true extent of inequalities. Consideration should be given to the research questions and within-country context to establish how sub-national severity distributions would be best designed, for example it may be more appropriate to estimate severity distributions for fixed geographical regions. Care must be taken in interpreting YLD estimates too precisely especially with sub-national estimates and estimates comparing populations.

Online supplementary appendix.

(XLSX)

Click here for additional data file.