Estimating the effect of the 2005 change in BCG policy in England: a retrospective cohort study, 2000 to 2015.

BackgroundIn 2005 in England, universal Bacillus Calmette-Guérin (BCG) vaccination of school-age children was replaced by targeted BCG vaccination of high-risk neonates.AimEstimate the impact of the 2005 change in BCG policy on tuberculosis (TB) incidence rates in England.MethodsWe conducted an observational study by combining notifications from the Enhanced Tuberculosis Surveillance system, with demographic data from the Labour Force Survey to construct retrospective cohorts relevant to both the universal and targeted vaccination between 1 January 2000 and 31 December 2010. We then estimated incidence rates over a 5-year follow-up period and used regression modelling to estimate the impact of the change in policy on TB.ResultsIn the non-United Kingdom (UK) born, we found evidence for an association between a reduction in incidence rates and the change in BCG policy (school-age incidence rate ratio (IRR): 0.74; 95% credible interval (CrI): 0.61 to 0.88 and neonatal IRR: 0.62; 95%CrI: 0.44 to 0.88). We found some evidence that the change in policy was associated with an increase in incidence rates in the UK born school-age population (IRR: 1.08; 95%CrI: 0.97 to 1.19) and weaker evidence of an association with a reduction in incidence rates in UK born neonates (IRR: 0.96; 95%CrI: 0.82 to 1.14). Overall, we found that the change in policy was associated with directly preventing 385 (95%CrI: -105 to 881) cases.ConclusionsWithdrawing universal vaccination at school age and targeting vaccination towards high-risk neonates was associated with reduced incidence of TB. This was largely driven by reductions in the non-UK born with cases increasing in the UK born.

Introduction

In 2005, England changed its Bacillus Calmette–Guérin (BCG) vaccination policy against tuberculosis (TB) from a universal programme aimed at 13 and 14 year olds to a targeted programme aimed at high-risk neonates. High-risk babies are identified by local TB incidence and by the parents’ and grandparents’ country of origin. The change in policy was motivated by evidence of reduced TB transmission [1-3], high effectiveness of the BCG vaccine in young children [4-6] and variable effectiveness in adults [7]. Little work has been done to evaluate the impact of this change in vaccination policy.

Globally, several countries with low TB incidence have moved from universal vaccination, either of those at school age or neonates, to targeted vaccination of neonates considered at high risk of TB [7]. In Sweden, which discontinued universal vaccination of neonates in favour of targeted vaccination of those at high risk, incidence rates in Swedish-born children increased slightly after the change in policy [8]. In France, which also switched from universal vaccination of children to targeted vaccination of those at high risk, a study found that targeted vaccination may have reduced coverage in those most at risk [9].

The number of TB notifications in England increased from 6,929 in 2004 to 8,280 in 2011 but has since declined to 5,137 in 2017 [1]. A recent study found that this reduction may be linked to improved TB interventions [10]. Directly linking trends in TB incidence to transmission is complex because after an initial infection an individual may either develop active disease, or enter a latent stage, which then may later develop into active disease. Incidence in children is a proxy of TB transmission, because any active TB disease in this population is attributable to recent transmission. Using this approach it is thought that TB transmission has been falling in England since 2011, a notion supported by strain typing [1]. Nevertheless, the effect of the change in BCG policy, which is likely to have reduced incidence rates in children, has not been assessed.

Although the long-term effects of BCG vaccination such as reducing the reactivation of latent cases and decreasing onwards transmission are not readily detectable over short time scales, the direct effects of vaccination on incidence rates can be estimated in vaccinated populations, when compared with comparable unvaccinated populations [11]. Here, we aimed to estimate the impact of the 2005 change in BCG policy on incidence rates in England, in both the United Kingdom (UK) and non-UK born populations, directly affected by it.

Methods

Data sources

Data on all notifications from the Enhanced Tuberculosis Surveillance (ETS) system from 1 January 2000 to 31 December 2015 were obtained from Public Health England (PHE). The ETS is maintained by PHE and contains demographic, clinical and microbiological data on all notified cases in England. A descriptive analysis of TB epidemiology in England is published each year, which fully details data collection and cleaning [1]. Tuberculosis is highly heterogeneous in England with the majority of cases occurring in urban, non-UK born populations. The yearly PHE report contains more descriptive detail [1].

We obtained yearly population estimates from the April to June Labour Force Survey (LFS) for years 2000 to 2015. The LFS is a household study of the employment circumstances of the UK population, which provides the official measures of employment and unemployment in the UK. It is also used to study population demographics such as ethnicity, country of birth and age. Reporting practices have changed with time so the appropriate variables for age, country of origin, country of birth, and survey weight were extracted from each yearly extract, standardised and combined into a single dataset.

Constructing retrospective cohorts

We constructed retrospective cohorts of TB cases and individuals using the ETS and the LFS. Tuberculosis cases were extracted from the ETS based on date of birth and date of TB notification.

Cohort 1: individuals aged 14 years between 2000 and 2004, who were notified with TB while aged between 14 and 19 years.

Comparison cohort 1: individuals aged 14 years between 2005 and 2010, who were notified with TB while aged between 14 and 19 years.

Cohort 2: individuals born between 2005 and 2010, who were notified with TB while aged 0 to 5 years.

Comparison cohort 2: individuals born between 2000 and 2004, who were notified with TB while aged 0 to 5 years.

Cohorts were stratified by vaccination programme using age criteria and then stratified further by whether the scheme was in place during the time period they entered the study. Each cohort was further stratified by UK birth status, with both non-UK born and UK born cases assumed to have been exposed to England’s vaccination policy. Corresponding population cohorts were calculated using the LFS population estimates, resulting in eight population level cohorts, each with 5 years of follow-up (Table 1).

Table 1

Summary of relevance and eligibility criteria of cohorts studied to assess the effect of the 2005 change in BCG policy, England, 2000–2010.

Cohort	Vaccination programme	Eligible for the programmea	Birth status	Age in years at study entry	Year of study entry
Cohort 1	Universal	Yes	UK born	14	2000–2004
Comparison cohort 1	Universal	No	UK born	14	2005–2010
Cohort 1	Universal	Yes	Non-UK born	14	2000–2004
Comparison cohort 1	Universal	No	Non-UK born	14	2005–2010
Cohort 2	Targeted	Yes	UK born	Birth	2005–2010
Comparison cohort 2	Targeted	No	UK born	Birth	2000–2004
Cohort 2	Targeted	Yes	Non-UK born	Birth	2005–2010
Comparison cohort 2	Targeted	No	Non-UK born	Birth	2000–2004

BCG: Bacillus Calmette–Guérin; UK: United Kingdom.

a Eligible signifies that the cohort fit the criteria for the vaccination programme and entered the study during the time period it was in operation not that the cohort was vaccinated by the vaccination programme.

Statistical methods overview

We estimated incidence rates (with 95% confidence intervals (CI)) by year, age and place of birth as (number of cases) divided by (number of individuals of corresponding age). UK birth status was incomplete, with some evidence of a missing not at random mechanism. We imputed the missing data using a gradient boosting method (GBM; see Supplement). We then used a descriptive analysis to describe the observed trends in age-specific incidence rates over the study period, comparing incidence rates in the study populations relevant to both vaccination programmes before and after the change in BCG policy.

We calculated incidence rate ratios (IRRs) with 95% credible intervals (CrIs) for the change in incidence rates associated with the change in BCG vaccination policy (modelled as a binary breakpoint at the start of 2005) for both the UK born and non-UK born populations that were relevant for the universal programme and for the targeted programme, using a range of models. We considered the following covariates: age [1,7], incidence rates in both the UK born and non-UK born who were not in the age group of interest [1] and year of study entry (as a random intercept). We evaluated a range of models using a statistically rigorous criterion that accounted for model fit and complexity, for model selection.

Statistical modelling details

We first investigated a univariable Poisson model, followed by combinations of covariates (Supplement Table S1). We also investigated a negative binomial model, adjusting for the same covariates as in the best fitting Poisson model. The models were estimated with a Bayesian approach using Markov Chain Monte Carlo (MCMC) with default weakly informative priors (Supplement). Model fit, penalised by model complexity, was assessed using the leave one out cross validation information criterion (LOOIC) and its standard error [12]. Models were ranked by goodness of fit, using their LOOIC, with a smaller LOOIC indicating a better fit to the data after adjusting for the complexity of the model. No formal threshold for a change in the LOOIC was used, with changes in the LOOIC being evaluated in the context of their standard error (SE).

The inclusion of the change in policy in the best fitting model was tested by refitting the model excluding the change in policy and estimating the improvement in the LOOIC.

Once the best fitting model had been identified we estimated the number of cases prevented, from 2005 until 2015, for each vaccination programme in the study population relevant to that programme (Supplement).

Implementation

R 3.5.2 was used for all analyses [13]. Missing data imputation using a GBM was implemented using the h2o package [14]. Incidence rates, with 95% CIs, were calculated using the epiR package [15]. The brms package [16] and STAN [17] were used to perform MCMC. Models were run until convergence (four chains with a burn in of 10,000 and 10,000 sampled iterations each), with convergence being assessed using trace plots and the R hat diagnostic [17]. All numeric confounders were centred and scaled by their standard deviation and age was adjusted for using single year of age categories.

Ethical statement

Ethical approval was not required for this study, which used anonymised secondary data sources only.

Results

Descriptive analysis

During the study period, there were 114,820 notifications of TB in England, of which 93% (106,765/114,820) had their birth status recorded. Of notifications with a known birth status 27% (29,096/106,765) were UK born, while among notification with an imputed birth status, 33% (2,634/8,055) were UK born. Trends in incidence rates varied by age group and UK birth status (Supplement). During the study period, there were 1,729 UK born cases and 2,797 non-UK born cases in individuals relevant to the universal school scheme and 1,431 UK born cases and 238 non-UK born cases relevant to the targeted neonatal scheme, who fit our age criteria. Univariable evidence for differences between mean incidence rates before and after the change in BCG policy in the UK born was weak. In the non-UK born incidence rates were lower after the change in BCG policy in both the cohort relevant to the universal school-age scheme and the cohort relevant to the targeted neonatal scheme (Figure).

Figure

Mean incidence rates per 100,000 population, with 95% confidence intervals for each retrospective cohorta studied to assess the effect of the 2005 change in BCG policy, stratified by the vaccination policy and UK birth status, England, 2000–2015

a Cohort definitions figure in Table 1.

The top and bottom panels are on different scales in order to highlight trends in incidence rates over time.

Adjusted estimates of the effects of the change in policy on school-age children

In the UK born cohort relevant to universal vaccination there was some evidence across all models adjusting for age, that ending the scheme was associated with a modest increase in TB rates (Supplement Table S2). Using the LOOIC goodness of fit criteria, the best fitting model was found to be a negative binomial model that adjusted for the change in policy, age and incidence rates in the UK born (Table 2). In this model there was some evidence of an association between the change in policy and an increase in incidence rates in those at school age who were UK born, with an IRR of 1.08 (95%CrI: 0.97 to 1.19). Dropping the change in policy from the model resulted in a small decrease in the LOOIC (0.52; SE: 2.63) but the change was too small, with too large a SE, to conclusively state that excluding the change in policy from the model improved the quality of model fit. We found that it was important to adjust for UK born incidence rates, otherwise the impact from the change in BCG vaccination policy was overestimated.

Table 2

Incidence rate ratios of tuberculosis in the UK born and non-UK born cohorts relevant to the universal school-age BCG vaccination scheme, using the best fitting modelsa as determined by comparison of the LOOIC, England, 2000–2015

Variable	IRR (95% CrI)
	UK born	Non-UK born
Policy changeb
Pre-change	Reference	Reference
Post-change	1.08 (0.97 to  1.19)	0.74 (0.61 to  0.88)
Age in years
14	Reference	Reference
15	1.18 (0.98 to 1.42)	1.03 (0.87 to 1.22)
16	1.24 (1.03 to 1.50)	1.25 (1.07 to 1.47)
17	1.59 (1.33 to 1.91)	1.40 (1.19 to 1.63)
18	1.92 (1.60 to 2.30)	1.47 (1.26 to 1.73)
19	1.80 (1.49 to 2.17)	1.47 (1.24 to 1.73)
Incidence rate
UK born incidence rate (per standard deviation)	1.08 (1.03 to 1.14)	NA
Non-UK born incidence rate (per standard deviation)	NA	1.11 (1.03 to 1.19)
Year of study eligibility, group level
Intercept (standard deviation)	NA	1.13 (1.05 to 1.26)
Year of study eligibility, individual level
2000	NA	1.10 (0.96 to 1.29)
2001	NA	1.06 (0.93 to 1.24)
2002	NA	1.07 (0.94 to 1.25)
2003	NA	0.90 (0.76 to 1.03)
2004	NA	0.89 (0.75 to 1.02)
2005	NA	0.98 (0.85 to 1.12)
2006	NA	1.13 (0.99 to 1.33)
2007	NA	1.04 (0.91 to 1.20)
2008	NA	0.96 (0.83 to 1.09)
2009	NA	0.95 (0.81 to 1.08)
2010	NA	0.96 (0.82 to 1.11)

BCG: Bacillus Calmette–Guérin; CrI: credible interval; IRR: incidence rate ratio; LOOIC: leave one out cross validation information criterion; SE: standard error; NA: not applicable (i.e. model terms were not included in the given cohort); UK: United Kingdom.

a The best fitting model for the UK born was a negative binomial model adjusting with fixed effects for the change in policy, age and incidence rates in the UK born (Model 7; Supplement Table S1). For the non-UK born the best fitting model was a negative binomial model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age and incidence rates in the non-UK born (Model 17; Supplement Table S1).

b There was an improvement in the LOOIC score of 0.52 (SE: 2.63) from dropping the change in policy from the model in the UK born cohort and a −3.02 (SE: 3.52) improvement in the non-UK born cohort.

For the comparable non-UK born cohort who were relevant to the universal vaccination there was evidence, in the best fitting model, that ending the scheme was associated with a decrease in incidence rates (IRR: 0.74; 95%CrI: 0.61 to 0.88). The best fitting model was a negative binomial model, which adjusted for the change in policy, age, incidence rates in the non-UK born and year of eligibility as a random effect (Table 2). We found omitting change in policy from the model resulted in poorer model fit (LOOIC increase of 3.02; SE: 3.52), suggesting that the policy change was an important factor explaining changes in incidence rates, after adjusting for other covariates.

All models that adjusted for incidence rates in the UK born or non-UK born estimated similar IRRs (Supplement Table S3).

Adjusted estimates of the effect of the change in policy in those relevant to the targeted neonatal programme

For the UK born cohort relevant to the targeted neonatal vaccination programme, the evidence of an association, across all models, was mixed and CrIs were wide compared with models for the UK born cohort relevant to the universal school-age vaccination programme (Supplement Table S4). The best fitting model was a Poisson model, which adjusted for the change in policy, age, UK born incidence rates and year of study entry with a random effect (Table 3). In this model, there was weak evidence of an association between the change in BCG policy and a decrease in incidence rates in UK born neonates, with an IRR of 0.96 (95%CrI: 0.82 to 1.14). There was weak evidence to suggest that dropping the change in policy from this model improved the quality of the fit, with an improvement in the LOOIC score of 0.92 (SE: 1.07). This suggests that the change in policy was not an important factor for explaining incidence rates, after adjusting for covariates. Models, which also adjusted for non-UK born incidence rates estimated that the change in policy was associated with no change in incidence rates in the relevant cohort of neonates (Supplement).

Table 3

Incidence rate ratios of tuberculosis, in the UK born and non-UK born cohorts relevant to the targeted neonatal BCG vaccination scheme, using the best fitting modelsa as determined by comparison of the LOOIC, England, 2000–2015

Variable	IRR (95% CrI)
	UK born	Non-UK born
Policy changeb
Pre-change	Reference	Reference
Post-change	0.96 (0.82 to 1.14)	0.62 (0.44 to 0.88)
Age
0	Reference	Reference
1	1.39 (1.20 to 1.61)	0.49 (0.30 to 0.83)
2	1.24 (1.06 to 1.44)	0.49 (0.30 to 0.80)
3	1.21 (1.03 to 1.41)	0.42 (0.26 to 0.68)
4	0.90 (0.76 to 1.06)	0.41 (0.25 to 0.66)
5	0.89 (0.75 to 1.06)	0.27 (0.16 to 0.45)
Incidence
UK born incidence rate (per standard deviation)	1.12 (1.06 to 1.18)	NA
Non-UK born incidence rate (per standard deviation)	NA	1.25 (1.04 to 1.51)
Year of study eligibility, group level
Intercept (standard deviation)	1.13 (1.04 to 1.26)	NA
Year of study eligibility, individual level
2000	0.83 (0.68 to 0.99)	NA
2001	0.93 (0.79 to 1.07)	NA
2002	1.08 (0.95 to 1.28)	NA
2003	1.07 (0.93 to 1.26)	NA
2004	1.12 (0.97 to 1.32)	NA
2005	1.02 (0.89 to 1.17)	NA
2006	1.02 (0.89 to 1.17)	NA
2007	0.97 (0.83 to 1.11)	NA
2008	1.01 (0.88 to 1.15)	NA
2009	1.01 (0.88 to 1.16)	NA
2010	0.98 (0.85 to 1.13)	NA

BCG: Bacillus Calmette–Guérin; CrI: credible interval; IRR: incidence rate ratio; LOOIC: leave one out cross validation information criterion; SE: standard error; NA: not applicable (i.e. model terms were not included in the given cohort); UK: United Kingdom.

a The best fitting model for the UK born was a Poisson model with a random intercept for year of study entry, adjusting with fixed effects for the change in policy, age and incidence rates in the UK born (Model 16; Supplement Table S1). For the non-UK born a negative binomial model was the best fit, adjusting with fixed effects for the change in policy, age and incidence rates in the non-UK born (Model 8; Supplement Table S1).

b There was an improvement in the LOOIC score of 0.92 (SE: 1.07) from dropping the change in policy from the model in the UK born cohort and a −3.45 (SE: 4.63) improvement in the non-UK born cohort.

For the comparable non-UK born cohort who was relevant to the targeted neonatal vaccination programme there was evidence, across all models, that change in policy was associated with a large decrease in incidence rates (IRR: 0.62; 95%CrI: 0.44 to 0.88) in the best fitting model (Table 3). The best fitting model was a negative binomial model that adjusted for the change in policy, age, and non-UK born incidence rates (Table 3). All models for this cohort, which at least adjusted for age, estimated comparable effects of the change in policy (Supplement Table S5).

Magnitude of the estimated impact of the change in Bacillus Calmette–Guérin policy

We estimate that the change in vaccination policy was associated with preventing 385 (95%CrI:  −105 to 881) cases from 2005 until the end of the study period in the directly impacted populations after 5 years of follow-up (Table 4). The majority of the cases prevented were in the non-UK born, with cases increasing slightly overall in the UK born. This was due to cases increasing in the UK born at school age despite a decrease in UK born neonates, although both these estimates had large CrIs.

Table 4

Estimated number of cases prevented, from 2005 until 2015, for each vaccination programme in the study population relevant to that programme, using the best fitting model for each cohort, England, 2000–2015

Vaccination programme	Birth status	Cases prevented (95% CrI)	Notified cases
Universal school-age(vaccination at 14 yearsa)	All	−291 (24 to −571)	2,364
	UK born	76 (188 to −26)	969
	Non-UK born	−367 (−165 to −546)	1,395
Targeted high-risk neonates(vaccination at birth)	All	94 (−81 to 310)	906
	UK born	30 (−95 to 173)	800
	Non-UK born	65 (14 to 137)	106
Change in policyb	All	385 (−105 to 881)	3,270
	UK born	−46 (−284 to 199)	1,769
	Non-UK born	431 (179 to 682)	1,501

CrI:  credible interval; UK: United Kingdom.

a In effect, vaccination was implemented across a school year (hence in 13 and 14 year olds) but for the synthetic cohort used in this study, vaccination at 14 years of age was used as an entry criterion.

b Estimated total number of cases prevented due to the change in vaccination policy in 2005.

Discussion

In the non-UK born we found evidence of an association between the change in BCG policy and a decrease in TB incidence rates in both those at school age and neonates, after 5 years of follow-up. We found some evidence that the change in BCG policy was associated with a modest increase in incidence rates in the UK born population who were relevant to the universal school-age scheme and weaker evidence of a small decrease in incidence rates in the UK born population relevant to the targeted neonatal scheme. Overall, we found that the change in policy was associated with preventing 385 (95%CrI:  −105 to 881) cases in the study population, from 2005 until the end of the study period, with the majority of the cases prevented in the non-UK born.

We were unable to estimate the impact of the change in BCG policy after 5 years post vaccination, so both our estimates of the positive and negative consequences are likely to be underestimates of the ongoing impact. Tuberculosis is a complex disease and the BCG vaccine is known to offer imperfect protection, which has been shown to vary both spatially and with time since vaccination [18,19]. By focusing on the impact of the change in policy on the directly affected populations within a short period of time and by employing a multi-model approach we have limited the potential impact of these issues. Our study was based on a routine observational dataset (ETS), and a repeated survey (LFS) both of which may have introduced bias. While the LFS is a robust data source, widely used in academic studies [20-22], it is susceptible to sampling errors particularly in the young and in the old, which may have biased the estimated incidence rates. As the ETS is a routine surveillance system some level of missing data are inevitable. However, the UK birth status was relatively complete (93%) and we imputed missing values using an approach, which accounted for missing not at random mechanisms captured by variables included in the imputation model. We were unable to adjust for known demographic risk factors for TB, notably socioeconomic status [23,24] and ethnicity [23-25]. However, this confounding is likely to be mitigated by our use of multiple cohorts and our adjustment for incidence rates in the UK born and non-UK born. Finally, we have assumed that the effect we have estimated for the change in BCG policy is due to the changes in BCG vaccination policy as well as other associated changes in TB control policy, after adjusting for hypothesised confounders. However, there may have been additional policy changes, which we have not accounted for.

While little work has been done to assess the impact of the 2005 change in BCG vaccination, several other studies have estimated the impact of changing BCG vaccination policy, although typically only from universal vaccination of neonates to targeted vaccination of high-risk neonates. A previous study in Sweden found that incidence rates in Swedish-born children increased after high-risk neonatal vaccination was implemented in place of a universal neonatal programme. This corresponds with our finding that introducing neonatal vaccination had little impact on incidence rates in UK born neonates. Theoretical approaches have indicated that targeted vaccination of those at high risk may be optimal in low incidence settings [26]. Our study extends this work by also considering the age of those given BCG vaccination, although we were unable to estimate the impact of a universal neonatal scheme as this has never been implemented nationally in England. It has previously been shown that targeted vaccination programmes may not reach those considered most at risk [27]. Our findings may support this view as we observed only a small decrease in incidence rates in UK born neonates after the introduction of the targeted neonatal vaccination programme. Alternatively, the effectiveness of the BCG in neonates, in England, may be lower than previously thought as we only observed a small decrease in incidence rates, while a previous study estimated BCG coverage at 68% (95%CI: 65% to 71%) among those eligible for the targeted neonatal vaccination programme [28].

This study indicates that the change in England’s BCG vaccination policy was associated with a modest increase in incidence in the UK born who were relevant to the school-age vaccination programme, and with a small reduction in incidence in the UK born who were relevant to the high-risk neonatal vaccination programme, although both these estimates had wide CrIs. We found stronger evidence of an association between the change in policy and a decrease in incidence rates in the non-UK born populations relevant to both programmes. This suggests that the change of vaccination policy to target high-risk neonates may have resulted in an increased focus on high-risk non-UK born individuals who may not have been the direct targets of the vaccination programme. Further validation is required using alternative study designs, but this result should be taken into account when vaccination policy changes are being considered. Our results should be interpreted carefully, especially in the non-UK born, as we could not fully rule out the impact of other TB control measures that may have been changed at the same time as vaccination policy. The severity of TB is known to differ across age groups, with children having a higher incidence of TB meningitis, which can be severe, compared with other age groups [1]. This variation should also be considered when evaluating these results.

It is well established that interventions against infectious diseases, such as TB, should be evaluated not only for their direct effects but also for future indirect effects via ongoing transmission. Statistical approaches such as those used in this paper are not appropriate for capturing these future indirect effects, and instead dynamic disease models should be used. In addition, this study could not evaluate the impact of the neonatal programme on the high-risk population it targets, due to a lack of reliable data. Improved coverage data for the BCG programme is required to more fully evaluate its ongoing impact. As only 5 years of follow-up data were available – and BCG vaccination has been shown to provide long lasting protection in the UK – repeating this study once more data are available may alter the findings [29]. For this reason, this study has been implemented with reproducibility in mind – please see the code reference below for details. Finally, the results from this study could be combined with estimates of the impact of TB disease, stratified by age, to give an estimate of the overall impact of the change in policy that accounts for the severity of disease.