Disregarding the restrictive vial-opening policy for BCG vaccine in Guinea-Bissau: impact and cost-effectiveness for tuberculosis mortality and all-cause mortality in children aged 0-4 years.

OBJECTIVE: BCG vaccination is frequently delayed in low-income countries. Restrictive vial-opening policies, where a vial of BCG vaccine is not opened for few children, are a major reason for delay. During delays, children are unprotected against tuberculosis (TB) and deprived of non-specific effects of BCG. We assessed the potential effect and cost-effectiveness of disregarding the restrictive vial-opening policy, on TB and all-cause mortality, in children aged 0-4 years in Guinea-Bissau.
METHODS: Using static mathematical models, we estimated the absolute and percentage change in TB and all-cause deaths, in children aged 0-4 years, between the current BCG vaccine restrictive-opening policy scenario, and a non-restrictive policy scenario where all children were vaccinated in the first health-facility contact. Incremental cost-effectiveness was estimated by integration of vaccine and treatment costs.
FINDINGS: Disregarding the restrictive BCG vial-opening policy was estimated to reduce TB deaths by 11.0% (95% uncertainty range (UR):0.5%-28.8%), corresponding to 4 (UR:0-15) TB deaths averted per birth cohort in Guinea-Bissau, resulting in incremental cost-effectiveness of US$ 911 per discounted life-year gained (LYG) (UR:145-9142). For all-cause deaths, the estimated reduction was 8.1% (UR: 3.3%-12.7%) corresponding to 392 (UR:158-624) fewer all-cause deaths and an incremental cost-effectiveness of US$ 9 (UR:5-23) per discounted LYG.
CONCLUSIONS: Disregarding the restrictive BCG vial-opening policy was associated with reductions in TB deaths and all-cause deaths and low cost-effectiveness ratios. Our results suggest that it would be cost-effective to disregard the restrictive vial-opening policy. Other settings with similar practice are also likely to gain from disregarding this policy. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

BCG is the only improved vaccine against tuberculosis (TB). However, BCG vaccination is often delayed in low-income countries, and a major cause of delay in Guinea-Bissau is the restrictive vial-opening policy, where a vial of BCG is not opened unless 10–12 children are present for vaccination at the same time.

We identified one mathematical modelling study that estimated the global impact of different BCG coverage scenarios on TB mortality in children aged 0–14 years, which concluded that improving timeliness of BCG vaccination at birth was associated with reductions in TB deaths.

We found no study estimating the health economic impact of increasing timeliness, the impact of improving BCG timeliness on all-cause mortality, nor any studies assessing the impact at the country level.

In this study, we provide the first country-level estimates of the impact of change in BCG coverage and timeliness. We estimated that disregarding the restrictive vial-opening policy and vaccinating every child at the first contact with a health facility in Guinea-Bissau could reduce TB and all-cause mortality.

Our study provides the first effect and cost-effectiveness estimates for this easily implementable intervention of disregarding the restrictive vial-opening policy, and our estimates show that it is both effective and cost-effective to disregard the restrictive vial-opening policy.

Existing empirical evidence has demonstrated the importance of early BCG vaccination for TB control and that early BCG may reduce all-cause mortality. Our study strongly supports disregarding the restrictive vial-opening policy, as we find that this would reduce child mortality and be cost-effective.

Since Guinea-Bissau is not the only setting with this restrictive vial-opening policy and delays in BCG vaccination, other settings are likely to also gain by disregarding the restrictive vial-opening policy.

Introduction

The BCG vaccine was developed to protect against tuberculosis (TB), and is currently the only approved TB vaccine.1 The efficacy and effectiveness of BCG against TB varies considerably between studies and populations,2 however, neonatal BCG has consistently been associated with reduced prevalence of TB disease.2

TB is estimated to be among the top ten causes of death worldwide.3 In 2015, global paediatric all-cause mortality was estimated at 5.9 million deaths,4 and 191 000 (95% CI: 132 000 to 257 000) of these were estimated as attributable to TB.5 As paediatric TB is difficult to diagnose, prevention may be more feasible than cure, particularly in settings with high TB burden and few resources. Vaccination is part of the WHO strategy for reducing TB deaths by 95% between 2015 and 2035.1 6 Recent estimates suggested that increasing global BCG coverage from 92% final coverage to an immediate 92% coverage at birth could reduce TB deaths <15 years by 5449 per global birth cohort (95% uncertainty range (UR): 218–15 071).7

BCG may have beneficial non-specific effects (NSEs), protecting against disease unrelated to TB.8–11 In 2014, a WHO-commissioned review concluded that BCG may have beneficial NSEs reducing all-cause child mortality.12 Thus, BCG delays may be important for both TB-specific mortality and all-cause mortality.

BCG is recommended at birth, and although global coverage is estimated at 88% by 12 months,13 this does not reveal underlying vaccination delays. To meet vaccine wastage targets, local practices of not opening a vial of BCG unless a sufficient number of children are present for vaccination have arisen (restrictive vial-opening policy) and BCG is consequently delayed in many low-income countries.14–16

The restrictive vial-opening policy is imposed with the goal of using scarce resources effectively. However, this saving may come at a cost and the impact has not been examined. The objective of this study was to estimate the epidemiological impact and cost-effectiveness of disregarding the restrictive BCG vial-opening policy in Guinea-Bissau, considering both the TB-specific and all-cause mortality effects.

Methods

We developed two static mathematical models, one for TB deaths based on a model developed by Roy et al 7 and one for all-cause deaths, both in Microsoft Excel 2013 (figure 1A, B). The model by Roy assumed all-or-nothing efficacy, where a proportion (corresponding to the vaccine efficacy) of BCG-vaccinated children are assumed to be fully protected, and the remainder are assumed to be unprotected. However, the mechanism of efficacy is unknown. As it is unrealistic to assume complete protection against all-cause mortality, we assumed leaky BCG vaccine efficacy in both models, that is, all BCG-vaccinated children are assumed partly protected corresponding to the vaccine efficacy (online supplemental appendix 3.1).

Figure 1

(A) Structure of the TB model. White boxes indicate stages in the model. Grey boxes indicate the included costs. Full arrows indicate the model flow, while dashed lines indicate the flow in the real world. All children enter the model as BCG unvaccinated. Children are counted as BCG vaccinated from the registration of a BCG vaccine (restrictive scenario) or registration of first health facility contact (non-restrictive scenario). In each scenario, we estimated the number of TB deaths. Based on a case-fatality rate, we calculated the number of TB cases. Using national data on the number of identified TB cases, we assumed that all identified TB cases were admitted, we estimated the number of TB hospital admissions in each scenario. (B) Structure of the all-cause model. White boxes indicate stages in the model. Grey boxes indicate the included costs. Full arrows indicate the model flow, while dashed lines indicate the flow in the real world, but in the models, there were no direct link, as the numbers were derived and calculated separately, and we only used the data on admission to be able to include these in the cost analyses. All children enter the model as BCG unvaccinated. Children are counted as BCG vaccinated from the registration of a BCG vaccine (restrictive scenario) or registration of first health facility contact (non-restrictive scenario). In each scenario, we estimated the number of all-cause hospital admissions, and included cost of hospital admissions incurred by the healthcare system and household cost of hospital admission (time spent in the hospital). We estimated the number of all-cause deaths and calculated the cost-effectiveness of disregarding the restrictive vial-opening policy.

The primary outcomes were absolute and percentage change in TB deaths and all-cause deaths averted among children <5 years per birth cohort due to disregarding the restricted BCG vial-opening policy (vial-policy change). The current scenario with restrictive BCG vial-opening policy (baseline) was compared with the non-restrictive scenario, defined as a scenario in which every child was BCG vaccinated at the first registered health facility contact.

Data inputs and assumptions

Where data were available, we used country-specific estimates for Guinea-Bissau. Parameter values and data sources are summarised in table 1.

Table 1

Data inputs and assumptions

Variable	Estimate	Source	Distribution
Population characteristics
Population at risk—birth cohort (2017)	69 212	Joint Reporting Form data from Guinea-Bissau*	Fixed
Life expectancy at birth	57 years	World Bank 2017†	Fixed
Estimated no of TB cases in males aged 0–4 in 2016	74 (37–121)	Global Burden of Disease Results Tool‡	Log-normal
Estimated no of TB cases in females aged 0–4 in 2016	89 (47–145)	Global Burden of Disease Results Tool‡	Log-normal
Estimated no of TB deaths in males aged 0–4 in 2016	16 (8–36)	Global Burden of Disease Results Tool‡	Log-normal
Estimated no of TB deaths in females aged 0–4 in 2016	17 (5–52)	Global Burden of Disease Results Tool‡	Log-normal
Reported TB cases aged 0–4 years in Guinea-Bissau	46	Annual TB report 2017, Guinea-Bissau§	Fixed
Case-fatality rate	0.21	Estimated from Global Burden of Disease estimates(See online supplemental appendix 1.3.1)	Fixed
TB estimates used in sensitivity analyses
Estimated no of TB cases in males aged 0–4 in 2016	290 (270–310)	WHO TB data¶	Log-normal
Estimated no of TB cases in females aged 0–4 in 2016	240 (220–250)	WHO TB data¶	Log-normal
Estimated no of TB deaths in children aged 0–4 years in 2016	238 (48–518)	Dodd, Lancet Global Health, 2017**	Log-normal
Individual daily all-cause mortality risk
Day 0	0.014184 (0.012901–0.015632)	BHP HDSS routine data††	Log-normal
Day 1	0.003370 (0.002757–0.004165)	BHP HDSS routine data††	Log-normal
Day 2	0.002100 (0.001646–0.002724)	BHP HDSS routine data††	Log-normal
Day 3	0.001090 (0.000772–0.001592)	BHP HDSS routine data††	Log-normal
Day 4	0.001123 (0.000799–0.001630)	BHP HDSS routine data††	Log-normal
Day 5	0.000787 (0.000518–0.001259)	BHP HDSS routine data††	Log-normal
Day 6	0.001110 (0.000785–0.001624)	BHP HDSS routine data††	Log-normal
Day 7	0.000547 (0.000319–0.001024)	BHP HDSS routine data††	Log-normal
Day 8–28	0.000142 (0.000115–0.000179)	BHP HDSS routine data††	Log-normal
Day 29–365	0.000054 (0.000049–0.000058)	BHP HDSS routine data††	Log-normal
Day 366–1826	0.000018 (0.000017–0.000020)	BHP HDSS routine data††	Log-normal
Individual daily risk of all-cause hospital admission
Day 0–28	0.000083 (0.000065–0.000107)	BHP HDSS routine data††	Log-normal
Day 29–365	0.000080 (0.000074–0.000085)	BHP HDSS routine data††	Log-normal
Day 366–1826	0.000042 (0.000039–0.000044)	BHP HDSS routine data††	Log-normal
BCG coverage distribution in baseline scenario	figure 1	BHP HDSS routine data‡‡	Fixed
BCG coverage distribution disregarding the vial-opening policy	figure 1	BHP HDSS routine data‡‡	Fixed
Vaccine characteristics TB-specific
				Risk ratio of BCG on TB deaths	0.34 (0.12–0.92)	Abubakar, Health Technol Assess, 2013§§	Log-normal
Duration of protection	>5 years	Abubakar, Health Technol Assess, 2013§§	Fixed
Waning of protection	None	Assumption	Fixed
Prevention of transmission by BCG	None	Assumption	Fixed
Vaccine characteristics all-cause effects
				Risk ratio of BCG on all-cause deaths	0.58 (0.42–0.81)	Meta-estimate of studies from Guinea-Bissau¶¶	Log-normal
Risk ratio of BCG on all-cause hospital admissions	0.97 (0.72–1.31)	Meta-estimate of studies from Guinea-Bissau***	Log-normal
Duration of protection	>5 years	Assumption	Fixed
Waning of protection	None	Assumption	Fixed
Prevention of transmission by BCG	None	Assumption	Fixed
Cost estimates
				BCG vaccine price per dose incl. freight	0.20 USD	National department of the Expanded Programme on Immunisation*	Fixed
Injection syringe price incl. freight	0.05 USD	National department of the Expanded Programme on Immunisation*	Fixed
Mixing syringe price incl. freight	0.03 USD	National department of the Expanded Programme on Immunisation*	Fixed
Safety box	0.67 USD	National department of the Expanded Programme on Immunisation*	Fixed
Median bed day per TB hospital admission	60 days	Personal communication: Victor Gomes, National TB Programme§	Fixed
Median bed day per all-cause hospital admission	5 days	BHP HDSS hospital admission data†††	Fixed
Costs per hospital bed day incurred by health system	15.58 USD	Enemark, in preparation, 2019‡‡‡	Fixed
Household productivity costs per bed day hospital admission	2.98 USD	Knight, PNAS, 2014§§§	Fixed
Household costs of seeking BCG vaccination	1.92 USD	Thysen, Vaccine, 2019¶¶¶	Fixed
Average no of times seeking BCG vaccination—restrictive (baseline) scenario	1.26	Thysen, Vaccine, 2019¶¶¶	Fixed
Average no of times seeking BCG vaccination—non-restrictive scenario	1	Assumption	Fixed
Vaccine wastage—restrictive (baseline) scenario	50%	Assumption	Fixed
Vaccine wastage—non-restrictive scenario	95%	Assumption	Fixed

*Personal communication: Carlitos Bale, Director of the Expanded Programme on Immunisation in Guinea-Bissau.

†World Bank.17

‡Institute for Health Metrics and Evaluation.18

§Personal communication: Victor Gomes, Programmatic Manager of MDR-TB, National TB Programme.

¶WHO.31

**Dodd PJ etal 5

††online supplemental appendix 1.2.1.

‡‡online supplemental appendix 1.2.2.

§§Abubakar I et al 20

¶¶online supplemental appendix 1.2.3, figure 2.

***Schaltz-Buchholzer F et al 22

†††online supplemental appendix 1.2.6.

‡‡‡Enemark U et al 24

§§§Knight GM et al 25

¶¶¶Thysen SM etal 23

BHP, Bandim Health Project’s; HDSS, Health and Demographic Surveillance Systems; TB, tuberculosis.

Figure 2

BCG coverage estimates in Guinea-Bissau in the restrictive (baseline) scenario and when disregarding the restrictive BCG vial-opening policy.

Population estimates

We used routine 2012–2017 surveillance data from the Bandim Health Project’s (BHP) urban and rural Health and Demographic Surveillance Systems (HDSS) to estimate the daily individual risk of all-cause mortality and all-cause hospital admission in children aged 0–4 years (table 1, online supplemental appendix 1.2.1). We used WHO/UNICEF birth cohort estimates (online supplemental appendix 1.1) and World Bank estimate of 57 years life-expectancy in Guinea-Bissau in 2017.17

TB data

For TB incidence and TB mortality, we used the 2016 Guinea-Bissau Global Burden of Disease18 (GBD) estimates, as reported TB data from Guinea-Bissau are likely underestimates. Using this information, we calculated a TB case fatality rate (CFR) of 0.21 (table 1). We assumed that all reported paediatric TB19 cases were hospitalised [personal communication: Victor Gomes, Programmatic Manager of MDR-TB, National TB Programme]. We calculated the proportion of hospitalised TB cases as reported cases divided by the GBD case estimate.

BCG coverage and timeliness

Using BHP routine data, we estimated BCG coverage and timeliness for the restrictive scenario using current estimates, and in the non-restrictive scenario using first health facility contact (figure 2, online supplemental appendix 1.2.2).

BCG vaccine efficacy

We used a BCG vaccine efficacy against TB death of 66% (95%CI: 8% to 88%).20 We assumed vaccine efficacy was constant regardless of age of administration.7 BCG protection may last for up to 15 years20, we, therefore, assumed no waning of protection between the ages 0 and 4 years. As children rarely contribute to transmission,21 we assumed that a vial-policy change would not affect transmission.

We undertook a meta-analysis of seven studies from Guinea-Bissau (online supplemental appendix 1.2.3), and obtained a vaccine efficacy against all-cause death of 42% (95%CI:19% to 58%) (figure 3). For all-cause hospital admission, we used a vaccine-efficacy from a meta-analysis (3%, 95% CI:−31% to 28%).22 Vaccine efficacies were assumed constant regardless of age at administration with no waning of protection before 5 years of age.

Figure 3

Meta-analysis of the effect of BCG on all-cause mortality in Guinea-Bissau HRs below 1 indicate that BCG vaccine is associated with lower mortality. HRs above 1 indicate that BCG vaccination is associated with higher mortality. 1: Kristensen et al 9, 2: Aaby et al 34, 3: Roth et al 35, 4: Aaby et al 36, 5: Biering-Sørensen et al 37, 6: Biering-Sørensen et al 8, 7: Thysen et al.38

Costs

Costs were calculated using 2017 US dollar (US$) values. We included vaccination costs (including materials and freight), household costs of seeking BCG (US$ 1.92 per child,23 online supplemental appendix 1.2.5), costs incurred by the health system per hospital bed day (US$ 15.58,24 online supplemental appendix 1.2.6), and time spent accompanying the child to the hospital (US$ 2.98 per hospital bed day,25 online supplemental appendix 1.2.6). Hospital admissions are free for children <5 years in Guinea-Bissau,26 so we assumed no out-of-pocket payments for treatment. We assumed 50% vaccine wastage in the restrictive scenario, as a 20-dose vial is usually opened if 10 or more children are present. We assumed 95% wastage in the non-restrictive scenario, equivalent to vaccinating only one child per vial.

Mathematical model

TB deaths model

We developed a static cohort Markov model to calculate the absolute and relative difference in TB death between the restrictive and non-restrictive scenarios (online supplemental appendix 2.1). We estimated the daily risk of TB death in vaccinated and unvaccinated children in the restrictive scenario based on the total number of TB deaths (0–4 years), vaccine efficacy and the number of children BCG vaccinated and unvaccinated by age (table 1). To calculate the cumulative number of TB deaths per birth cohort in the first 5 years of life in each scenario, we applied the daily individual risk of TB among vaccinated children to the number of vaccinated children, and likewise for unvaccinated children. These risks were assumed constant between the ages 0 and 4 years. The population at risk was adjusted for all-cause deaths by day; assuming BCG had no effect on all-cause mortality.

The number of TB cases per birth cohort in the first 5 years of life in each scenario was estimated by dividing the number of TB deaths in each scenario by CFR of 0.21 (table 1). To estimate TB hospital admissions, the proportion of TB cases identified through national TB surveillance was applied to the model-estimated number of TB cases in each scenario, assuming all identified TB cases were admitted.

All-cause deaths model

We developed a static cohort Markov model to calculate the absolute and relative difference in all-cause death between the two scenarios (online supplemental appendix 2.2). We estimated daily, age-specific risk of all-cause death using the BHP routine all-cause mortality data (online supplemental appendix 1.2.1). Applying similar methods as the TB model, we used baseline data on BCG coverage by age, vaccine efficacy meta-estimate for all-cause death (online supplemental appendix 1.2.3), and the daily risk of all-cause death to calculate the daily risk of all-cause death in BCG-vaccinated children and in BCG-unvaccinated children, respectively.

We applied the estimated daily individual risk of all-cause mortality among BCG-vaccinated children to the number of BCG-vaccinated children, and likewise for unvaccinated children, and estimated daily and cumulative all-cause deaths between 0 and 4 years under the restrictive and non-restrictive scenarios (online supplemental appendix 2.2). Using a similar approach, we calculated the absolute and relative difference in all-cause hospital admissions between scenarios (online supplemental appendix 2.3).

Increasing evidence suggests that vaccines may interact.12 27 We, therefore, conducted a secondary analysis to assess outcomes only in the period prior to receipt of other vaccines (0–6 weeks of age).

Cost-effectiveness analyses

Cost-effectiveness was assessed from a societal perspective, including total population-level vaccination costs, household costs of seeking BCG vaccination, and costs of hospital admission incurred by the health system and the household (online supplemental appendix 2.4).

We added the number of deaths per year between the ages 0 and 5 years in 1-year intervals, to calculate the total number of deaths averted. Life-years gained (LYG) in the non-restrictive scenario were calculated in each model by multiplying the number of TB deaths or all-cause deaths averted per year of life by the remaining life expectancy.17 As WHO recommends,28 we discounted future costs and life years by 3%/year. We calculated incremental cost-effectiveness ratios (ICERs) of a vial-policy change. We used Wood’s purchasing power parity adjusted cost-effectiveness threshold for Guinea-Bissau of US$22–US$645.29

Urban versus rural subanalyses

Vaccination opportunities, healthcare-seeking behaviour and number of children present at health centres are likely to differ between the urban and rural population, resulting in different effects of a vial-policy change. We, therefore, calculated separate estimates for urban and rural Guinea-Bissau. We used the same approaches, but substituted risk of all-cause mortality, risk of all-cause hospital admission, BCG coverage, and birth cohort estimates with regional estimates.

Uncertainty and sensitivity analyses

We performed a probabilistic uncertainty analysis using Oracle Crystal Ball (Release 11.1.2.4.850, Oracle, USA), where a statistical distribution was set for each parameter with a reported UR (table 1). Location and scale parameters were estimated using the 2.5%, 50% and 97.5% percentiles in the ‘rriskDistributions’ R package. Parameters with fixed values were not considered uncertain. A total of 100 000 parameter sets and model outputs were generated through Monte-Carlo simulations. Median and 95% URs were calculated from the 100 000 model outputs.

In sensitivity analyses, we assessed the impact of assuming leaky BCG vaccine efficacy by adapting the TB model to all-or-nothing efficacy, as in previous models7 30 (online supplemental appendix 2.5). We evaluated the impact of using GBD estimates of TB mortality in two sensitivity analyses, using WHO TB incidence31 combined with CFR estimates from Jenkins,32 and using modelled estimates of TB mortality from Dodd.5 We also varied the assumption of perfect correlation between (1) male and female TB estimates, and (2) agewise mortality estimates, to be uncorrelated in sensitivity analyses. We assumed very high wastage (95%) in the non-restrictive scenario, and therefore conducted sensitivity analyses with wastage assumptions in the restrictive scenario from 35% to 60% with 5% intervals, and in the non-restrictive scenario from 70% to 90%.

Patient and public involvement

The communities were involved in locating households for the HDSS data collection, when the BHP HDSS was setup. No participant was involved in setting the research question or the outcome measure, nor were they involved in developing the models, or design of the study. No participant was asked to advise on interpretation or writing up the results. The results are disseminated to the national public health institute. There are no plans to disseminate the results of the research to study participants or the community.

Results

Effects of a vial-policy change on TB-specific outcomes

Disregarding the restrictive vial opening policy was estimated to reduce TB deaths, admissions and cases by 11.0% (95% UR: 0.5%–28.8%). The number of TB deaths was 33 (UR: 13–89) per birth cohort in the restrictive scenario and 29 (UR: 11–79) in the non-restrictive scenario, averting 4 (UR: 0–15) TB deaths per birth cohort in the first 5 years of life. TB cases were reduced from 162 (UR: 96–273) to 142 (UR: 82–245), and TB hospital admissions from 46 to 41 (table 2).

Table 2

Effects of disregarding the restrictive BCG vial-opening policy in Guinea-Bissau

	Restrictive (baseline) scenario	Non-restrictive scenario	Absolute change	Percentage change
	Median(95% uncertainty range)	Median(95% uncertainty range)	Median(95% uncertainty range)	Median(95% uncertainty range)
TB-specific effects
Total no of paediatric TB deaths	33 (13 to 89)	29 (11 to 79)	−4 (−15 to 0)	−11.0% (−28.8% to −0.5%)
Total no of paediatric TB cases	162 (96 to 273)	142 (82 to 245)	−18 (−54 to −1)	−11.0% (−28.8% to −0.5%)
Total no of paediatric TB hospital admissions	46*	41 (33 to 46)	−5 (−13 to 0)	−11.0% (−28.8% to −0.5%)
Sub-analysis stratifying urban and rural regions
Total no of paediatric TB deaths—urban data	8 (3 to 22)	8 (3 to 21)	0 (−1 to 0)	−2.6% (−8.3% to −0.1%)
Total no of paediatric TB deaths—rural data	26 (10 to 69)	21 (8 to 57)	−4 (−17 to 0)	−16.4% (−38.6% to −0.8%)
All-cause effects
Total no of all-cause deaths	4820 (4309 to 5425)	4429 (3920 to 5028)	−392 (−624 to −158)	−8.1% (−12.7% to −3.3%)
Total no of all-cause hospital admissions	5926 (5538 to 6346)	5940 (5532 to 6380)	18 (−125 to 133)	0.4% (−2.6% to 2.8%)
Subanalysis stratifying urban and rural regions
Total no of all-cause deaths urban data	1071 (897 to 1302)	961 (795 to 1180)	−111 (−172 to -47)	−10.4% (−15.3% to −4.5%)
Total no of all-cause deaths rural data	3787 (3303 to 4386)	3467 (2992 to 4048)	−319 (−527 to −124)	−8.4% (−13.5% to −3.3%)

*The number of paediatric TB hospital admissions is the reported number of TB cases from Guinea-Bissau in 2017 (Programa Nacional de Luta contra a Tuberculose. Relatório anualde tuberculose 2017: Ministério da Saúde Publica da Guiné-Bissau, 2018).

TB, tuberculosis.

Effects of a vial-policy change on all-cause outcomes

The vial-policy change was estimated to reduce all-cause mortality by 8.1% (UR: 3.3%–12.7%), from 4820 (UR: 4309–5425) all-cause deaths in the restrictive scenario to 4429 (UR: 3920–5028) all-cause deaths in the non-restrictive scenario, averting 392 (UR: 158–624) all-cause deaths per birth cohort in the first 5 years of life (table 2). There was an estimated 0.4% (UR:−2.6 to 2.8%) increase in all-cause hospital admissions, due to more children surviving and becoming admitted to hospital (table 2).

Costs and cost-effectiveness of a vial-policy change

In both models, the vial-policy change resulted in higher BCG-vaccination costs and lower household costs. Accounting only for the TB-specific effects, the 95 (UR: 4–397) discounted LYG resulted in an ICER of US$ 911 (UR: 145–9142) per discounted LYG and US$ 26 527 (UR :4225–2 66 291) per discounted TB death averted (table 3).

Table 3

Cost-effectiveness of disregarding the restrictive BCG vial-opening policy for the Guinea-Bissau birth cohort in 2017

	Restrictive (baseline) scenario	Non-restrictive scenario
No of children born in health facilities	33 354	33 354
No of children BCG-vaccinated at birth	13 438	33 354
No of children not BCG-vaccinated at birth	55 774	35 858
BCG coverage at 12 months of age	93 %	99 %
Total household costs of seeking BCG vaccination	US$134 631	US$68 694
Total BCG vaccine costs	US$26 221	US$279 769
Total injection supply costs	US$3909	US$6640
TB-specific effects only	Median (95% uncertainty range)	Median (95% uncertainty range)
No of paediatric TB deaths	33 (13 to 89)	29 (11 to 79)
LYG by averted TB death	–	195 (7 to 817)
LYG by averted TB death—discounted	–	95 (4 to 397)
No of paediatric TB hospital admissions	46*	41 (33 to 46)
Costs of TB hospital admissions	US$51 207	US$45 557
Costs of TB hospital admissions discounted	US$47 011	US$41 813
Incremental costs of disregarding the restrictive vial-opening policy (including vaccination costs, household costs and costs averted by TB hospital admissions)	–	US$184 692
Incremental costs of disregarding the restrictive vial-opening policy discounted	–	US$185 144
ICER per LYG (USD)	–	440 (69 to 4440)
ICER per LYG discounted (USD)	–	911 (145 to 9142)
ICER per TB death averted (USD)	–	24 269 (3827 to 2 44 869)
ICER per TB death averted discounted (USD)	–	26 527 (4225 to 2 66 291)
All-cause effects	Median (95% uncertainty range)	Median (95% uncertainty range)
No of all-cause deaths	4820 (4309 to 5425)	4429 (3920 to 5028)
LYG by averted all-cause deaths	–	22 166 (8949 to 35 288)
LYG by averted all-cause deaths—discounted	–	10 605 (4279 to 16 896)
No of all-cause hospital admissions	5926 (5538 to 6346)	5940 (5532 to 6380)
Costs of all-cause hospital admissions	US$549 815	US$551 478
Costs of all-cause hospital admissions discounted	US$508 556	US$510 076
Incremental costs of disregarding the restrictive vial-opening policy (including vaccination costs, household costs and costs averted by hospital admissions)	–	US$192 005
Incremental costs of disregarding the restrictive vial-opening policy discounted	–	US$191 863
ICER per LYG (USD)	–	4 (3 to 11)
ICER per LYG discounted (USD)	–	9 (5 to 23)
ICER per all-cause death averted (USD)	–	249 (144 to 615)
ICER per all-cause death averted discounted (USD)	–	259 (150 to 639)

*The number of paediatric TB hospital admissions is the reported number of TB cases from Guinea-Bissau in 2017 (Programa Nacional de Luta contra a Tuberculose. Relatório anualde tuberculose 2017: Ministério da Saúde Publica da Guiné-Bissau, 2018).

ICER, incremental cost-effectiveness ratio; LYG, life-year gained; USD, US dollar 2017 value.

Including all-cause effects of BCG, the number of discounted LYG were 10 605 (UR: 4279–16,896), resulting in an ICER of US$9 (UR: 5–23) per discounted LYG and US$259 (UR: 150–639) per discounted all-cause death averted (table 3).

Effects of a vial-policy change in urban and rural health regions

The vial-policy change in urban Guinea-Bissau was estimated to have a smaller effect on TB mortality, with 2.6% (0.1%–8.3%) fewer TB deaths, corresponding to less than 1 TB death per birth cohort (table 2). There was, however, a larger impact in the all-cause model with 111 (UR: 47–172) fewer deaths and 10.4% (UR: 4.5%–15.3%) estimated lower mortality, as many children had vaccination opportunities within the first days of life, where all-cause mortality is highest. Including all-cause effects, the estimated ICERs were US$10 (UR: 6–23) per discounted LYG and US$280 (UR: 173–653) per discounted all-cause death averted (online supplemental appendix 3.4).

In the rural population, the policy change was estimated to reduce TB deaths by 16.4% (0.8%–38.6%), corresponding to 4 (UR: 0–17) fewer TB deaths. The resulting ICERs were US$524 (UR: 77–5455) per discounted LYG and US$15 272 (UR: 2245–159 037) per TB death averted. The estimated reduction in all-cause mortality was 8.4% (3.3%–13.5%) corresponding to 319 (UR: 124–527) fewer all-cause deaths (table 4). The resulting ICERs were US$8 (UR: 4–20) per discounted LYG and US$221 (UR: 120–567) per discounted all-cause death averted (online supplemental appendix 3.4).

Table 4

Effects of disregarding the restrictive BCG vial-opening policy in Guinea-Bissau—sensitivity analyses

	Restrictive (baseline) scenario	Non-restrictive scenario	Absolute differences	Percentage change*
	Median (95% uncertainty range)	Median (95% uncertainty range)	Median (95% uncertainty range)	Median (95% uncertainty range)
TB-specific effects
Total no of paediatric TB deaths - main analysis	33 (13 to 89)	29 (11 to 79)	−4 (−15 to 0)	−11.0% (−28.8% to −0.5%)
Sensitivity to BCG mode of action model structure
Total no of paediatric TB deaths—all-or-nothing BCG effectiveness	33 (13 to 89)	29 (11 to 79)	−4 (−15 to 0)	−11.0% (−28.8% to −0.5%)
Sensitivity to calibration data
Total no of paediatric TB deaths—WHO† and Jenkins‡ data	231 (196 to 272)	204 (156 to 252)	−25 (−68 to −1)	−11.0% (−28.8% to −0.5%)
Total no of paediatric TB deaths—P. Dodd§ data	238 (110 to 519)	209 (94 to 463)	−25 (−94 to −1)	−11.0% (−28.8% to −0.5%)
All-cause effects
Total no of all-cause deaths day—main analysis day 0–1826	4820 (4309 to 5425)	4429 (3920 to 5028)	−392 (−624 to −158)	−8.1% (−12.7% to −3.3%)
Subanalysis limiting follow-up to scheduled age of next vaccination
Total no of all-cause deaths day 0–42	1922 (1612 to 2318)	1648 (1352 to 2028)	−277 (−415 to −120)	−14.5% (−20.7% to −6.3%)
Subanalysis excluding day 0
Total no of all-cause deaths day 1–1826	3838 (3417 to 4345)	3621 (3208 to 4112)	−216 (−373 to −80)	−5.6% (−9.5% to −2.1%)

*Percentage change comparing the non-restrictive scenario with the restrictive scenario.

†WHO.31

‡Jenkins HE et al 32

§Dodd PJ et al 5

TB, tuberculosis.

Sensitivity analyses

Structural

Adapting the TB model structure from a model assuming leaky vaccine efficacy (main analysis) to all-or-nothing efficacy did not alter results (table 4). Similarly, assuming no correlations between mortality estimates yielded similar results (online supplemental appendix 3.5). The vial-policy change was cost saving in both TB and all-cause models when vaccine wastage was 80% or less in the non-restrictive scenario (online supplemental appendix 3.6).

Source of TB data

Both the WHO TB incidence31 combined with Jenkins CFR32 and the Dodd estimates5 resulted in 25 averted TB deaths per birth cohort in the first 5 years of life. However, UR were greater when using Dodd estimates (table 4).

Prior to other routine vaccines

Limiting the analyses to the period prior to other scheduled routine vaccines, the policy change was estimated to reduce all-cause mortality by 14.5% (UR: 6.3%–20.7%) (table 4).

Discussion

Main findings

Disregarding the restrictive BCG vial-opening policy was estimated to reduce TB deaths by 11.0% (UR: 0.5%–28.8%) corresponding to 4 (UR: 0–15) TB deaths averted per birth cohort. All-cause death estimates were reduced by 8.1% (UR: 3.3%–12.7%) corresponding to 392 (UR: 158–624) all-cause deaths per birth cohort. The ICER of the vial-policy change was estimated as US$911 (UR: 145–9142) per discounted LYG by averting TB deaths. The estimated ICER considering all-cause mortality was considerably lower: US$9 (UR: 5–23) per discounted LYG. Compared with Wood’s purchasing power parity adjusted cost-effectiveness threshold for Guinea-Bissau (US$22–US$645),29 the most conservative wastage assumption (95% in the non-restrictive scenario) may not lead to the conclusion that the policy was cost effective assuming effect only on TB deaths. However, at less conservative wastage assumptions, the cost declined markedly and despite few TB deaths averted, changing the policy could be cost-effective for TB deaths, and the policy was highly cost-effective including the effect on all-cause deaths, demonstrating the potential importance of considering all-cause effects in vaccine cost-effectiveness evaluations. Importantly, in the non-restrictive scenario, increased costs were incurred by the health system, while household costs were reduced. Hence, the intervention is likely to increase equity.

The vial-policy change was associated with larger impact in rural than urban Guinea-Bissau, as vaccination delays are greater in rural Guinea-Bissau and mortality is higher.

Comparisons and perspectives

Previously, Roy estimated that increasing BCG coverage to 92% at birth globally could reduce TB deaths <15 years of age by 2.8%.7 We found a greater percentage reduction in TB deaths (11.0%, UR: 0.5%–28.8%) by improving timeliness of BCG vaccination. Greater delays in BCG vaccination in the restrictive scenario and more precise estimates of BCG coverage in Guinea-Bissau contribute to the difference.

In recent years, manufacturing problems have created BCG shortages, estimated to be associated with 7433 (UR: 320–19 477) excess TB deaths in 0–14 year olds per global birth cohort,30 but potential excess all-cause deaths were not included. Our results suggest that if all-cause deaths had been estimated, the assessed public health impact of these shortages would have been substantially higher, and emphasise the importance of including all-cause mortality to inform the full impact of a policy change.

Strengths and limitations

We present the first model assessing impact and cost-effectiveness of the vial-opening policy change on all-cause mortality, and first country-level model assessing impact and cost-effectiveness of this change on any outcome. The BHP HDSS data allowed for country-representative age-specific estimates based on individual level data, which are more accurate than the aggregated data usually available. Results were robust to sensitivity analyses conducted.

Increasing vaccination coverage is usually resource demanding, but using the first registered health-facility contact to provide BCG could easily be implemented without additional initiatives. Thus, the high coverage in the non-restrictive scenario would be realistic if enough vaccines were available.

TB surveillance in Guinea-Bissau is limited and TB is likely underdiagnosed. Due to uncertainty in estimates of TB incidence and mortality, we assessed robustness of our results to the TB-calibration data by conducting sensitivity analyses using other data sources. The main estimates of absolute TB deaths averted were likely conservative, as using other data sources resulted in more TB deaths averted. The efficacy estimates of BCG against TB have varied between studies and populations,2 20 and thus the meta-analysed estimates have wide URs. We assumed that TB mortality was constant in 0–4 years old, since age-stratified data were not available. The estimates of TB cases and hospital admissions should be interpreted with caution as they are calculated based on different data sources assuming comparability.

The vaccine efficacy on all-cause deaths was estimated from trials in low-weight neonates and observational studies from Guinea-Bissau, which provided similar mortality estimates. More data from normal birth weight children would be of value. The current level of evidence is not sufficient to conclude on the duration of NSEs.

Cost estimates were derived from Guinea-Bissau. Household costs of seeking BCG vaccination23 and health system costs per hospital bed day were included.24 Out-of-pocket payments for treatment were not included, but as hospital admission for children is free, we expect out-of-pocket payments to be few. In main analyses, we conservatively assumed only one child was vaccinated per vial in the non-restrictive scenario, therefore likely overestimating the costs of the intervention, as more than one child would likely be vaccinated per vial. Sensitivity analyses showed that the intervention would potentially be cost saving, if on average four children were vaccinated per vial. However, we did not calculate URs for sensitivity analyses on wastage assumptions, and this conclusion should therefore be interpreted with caution.

Our models only included Guinea-Bissau. However, the restrictive vial-opening policy is not limited to Guinea-Bissau.15 33 While the exact impact and cost-effectiveness estimates are not directly transferable to other countries, the conclusions of the study are likely to be generalisable. Thus, important gains are likely to be possible in other settings with a similar practice.

Conclusion

Disregarding the restrictive vial-opening policy in Guinea-Bissau was estimated to result in small reductions in TB deaths and substantial reductions in all-cause deaths. Our results support that it would be cost-effective to disregard the restrictive vial-opening policy.