COVID-19 vaccination in Sindh Province, Pakistan: A modelling study of health impact and cost-effectiveness.

BACKGROUND: Multiple Coronavirus Disease 2019 (COVID-19) vaccines appear to be safe and efficacious, but only high-income countries have the resources to procure sufficient vaccine doses for most of their eligible populations. The World Health Organization has published guidelines for vaccine prioritisation, but most vaccine impact projections have focused on high-income countries, and few incorporate economic considerations. To address this evidence gap, we projected the health and economic impact of different vaccination scenarios in Sindh Province, Pakistan (population: 48 million). METHODS AND
FINDINGS: We fitted a compartmental transmission model to COVID-19 cases and deaths in Sindh from 30 April to 15 September 2020. We then projected cases, deaths, and hospitalisation outcomes over 10 years under different vaccine scenarios. Finally, we combined these projections with a detailed economic model to estimate incremental costs (from healthcare and partial societal perspectives), disability-adjusted life years (DALYs), and incremental cost-effectiveness ratio (ICER) for each scenario. We project that 1 year of vaccine distribution, at delivery rates consistent with COVAX projections, using an infection-blocking vaccine at $3/dose with 70% efficacy and 2.5-year duration of protection is likely to avert around 0.9 (95% credible interval (CrI): 0.9, 1.0) million cases, 10.1 (95% CrI: 10.1, 10.3) thousand deaths, and 70.1 (95% CrI: 69.9, 70.6) thousand DALYs, with an ICER of $27.9 per DALY averted from the health system perspective. Under a broad range of alternative scenarios, we find that initially prioritising the older (65+) population generally prevents more deaths. However, unprioritised distribution has almost the same cost-effectiveness when considering all outcomes, and both prioritised and unprioritised programmes can be cost-effective for low per-dose costs. High vaccine prices ($10/dose), however, may not be cost-effective, depending on the specifics of vaccine performance, distribution programme, and future pandemic trends. The principal drivers of the health outcomes are the fitted values for the overall transmission scaling parameter and disease natural history parameters from other studies, particularly age-specific probabilities of infection and symptomatic disease, as well as social contact rates. Other parameters are investigated in sensitivity analyses. This study is limited by model approximations, available data, and future uncertainty. Because the model is a single-population compartmental model, detailed impacts of nonpharmaceutical interventions (NPIs) such as household isolation cannot be practically represented or evaluated in combination with vaccine programmes. Similarly, the model cannot consider prioritising groups like healthcare or other essential workers. The model is only fitted to the reported case and death data, which are incomplete and not disaggregated by, e.g., age. Finally, because the future impact and implementation cost of NPIs are uncertain, how these would interact with vaccination remains an open question.
CONCLUSIONS: COVID-19 vaccination can have a considerable health impact and is likely to be cost-effective if more optimistic vaccine scenarios apply. Preventing severe disease is an important contributor to this impact. However, the advantage of prioritising older, high-risk populations is smaller in generally younger populations. This reduction is especially true in populations with more past transmission, and if the vaccine is likely to further impede transmission rather than just disease. Those conditions are typical of many low- and middle-income countries.

Introduction

The Coronavirus Disease 2019 (COVID-19) pandemic has resulted in over 50 million cases and nearly 2 million deaths in 2020, with cases in nearly every country [1]. To reduce transmission of the causal Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) virus, many countries have imposed physical distancing measures such as closure of schools and workplaces and restrictions on public gatherings [2]. Such measures often incur socioeconomic costs that are not indefinitely sustainable, particularly in resource-poor settings [3], and, when these measures are lifted, transmission has readily resumed in most places [4].

Vaccination may provide a durable option to protect individuals. If a vaccine also reduces transmission (e.g., by preventing infection or limiting infectiousness of disease), even unvaccinated individuals would have reduced infection risk. As of January 2021, 3 vaccines have completed Phase III trials, and at least 20 other vaccine candidates were in Phase III trials, with over 250 in earlier trials or preclinical studies [5,6].

Many high-income and large middle-income countries have signed bilateral agreements with manufacturers, preordering enough vaccines to cover their populations, some multiple times [7]. The World Health Organization (WHO), Gavi, the Vaccine Alliance, and the Coalition for Epidemic Preparedness Innovations (CEPI) launched the COVAX Facility to enable many small and lower-income countries to pool their purchasing power. To date, 141 countries and territories have started the COVAX participation process, which will distribute vaccines to participating countries according to population size [8,9]. This distribution will cover a small proportion (3%) of those populations in the months following vaccine approval, aiming to expand to 20% by the end of 2021 [10]. Given that this supply may not be sufficient, countries may need to purchase additional vaccines, and to do so need to be able to quantify the costs and benefits of various vaccination programmes to compare to other health sector investments. Additionally, countries face other substantial health sector resource constraints that may require external funding including scale-up of vaccine delivery infrastructure and workforce and continued care and treatment of those with COVID-19.

WHO’s Strategic Group of Experts on Immunization (SAGE) has issued a roadmap to help countries prioritise distribution of these limited doses. This roadmap draws from work across multiple disciplines, including modelling to project the health outcomes, health sector financing, and broader economic consequences of different vaccine prioritisation strategies, but much of the available research has focused on high-income settings [11-14].

To address this gap, we assessed the health impact, economic impact, and cost-effectiveness of COVID-19 vaccination in Sindh Province, Pakistan, using a combined epidemiological and economic model. We chose a specific setting to ensure that our model could incorporate local mobility and cost data. Sindh Province initially confirmed a large number of cases, followed by declining incidence after a nationwide lockdown. Our analysis addresses vaccine prioritisation questions faced by both global (WHO) and national (Pakistan Ministry of Health) decision-makers and illustrates the decision support analysis that should be applied more broadly in low- and middle-income countries (LMICs).

Methods

Ethics

All clinical data used were obtained from publicly available sources, so no ethical approval was required for this study.

Epidemiological model

To capture the natural history and transmission of SARS-CoV-2, we used a previously published compartmental model [15-17] tailored to the population of Sindh using population data from WorldPop [18] and assumed baseline population contact rates from previously estimated national patterns for Pakistan [19].

Briefly, the model compartments are an extended SEIRS+V (Susceptible, Exposed, Infectious with multiple subcompartments, Recovered and/or Vaccinated, potentially converting either to Susceptible or if in the combined state to only Recovered or Vaccinated) system with births, deaths, and age structure. For all compartments other than Recovered and/or Vaccinated, we use event-time distributions derived from global observations (Table 1). For Recovered and/or Vaccinated, we consider multiple characteristic protection durations, given the uncertainty in these durations. For the Recovered compartment (with and without vaccination), we assume perfect protection; we address the Vaccinated compartment protection along with the vaccination programme details in the “Vaccine programme” section. The modelled loss of protection can represent a range of phenomena, from antibody waning to shifts in the circulating pathogen with time leading to immune escape.

Table 1

Summary of epidemiological, vaccine, and economic parameters used in the base case and scenario analysis.

Parameter	Base case value	Scenario range	Source
Epidemiological parameters
Latent period	Gamma(mean = 2.5, k = 5)		[41,42]
Contact rates	Age-dependent synthetic contact matrix for Pakistan		[19]
Proportion asymptomatic	age-specific		posterior from [15]
Duration of infectiousness	Gamma(mean = 5, k = 4)		[41,42]
Duration of natural immunity	2.5 years	1 year, lifelong	Assumed
Vaccine-related parameters
Duration of vaccine programme	1 year	5 and 10 years
Initial age targeting	65+	15+
Duration of vaccine-induced immunity	2.5 years	1 year, lifelong	Assumed
Initial number of courses administered per day	4,000 (based on COVAX availability; see S1 Text)	8,000, 12,000, 184,000 (short campaign only)	[25]
Number of doses per course	2	1	[25]
Efficacy	70%	30%, 90%	Assumed
Economic parameters
Vaccine procurement price per dose	$3	$6 and $10	[8]
Wastage	15% of vaccine procurement price per dose, 10% of immunisation supplies procurement price per dose		[36]; see S1 Text for details
Freight	10% of vaccine procurement price per dose		Assumed
Syringes and safety boxes	0.04		UNICEF supply division price list
Cold chain costs per dose (national level)	$0.133		[37]
Cold chain costs per dose (service level)	$0.029		Microcosting. Various sources (see S1 Text).
Human resource per dose	$0.38		Microcosting from DCP project [38,39]
Transport per dose	Transport to facility: $0.04 Transport from facility to campaign site: $0.001		Microcosting. Various sources (see S1 Text).
Social mobilisation per dose	$0.16		[40]
Health system markup	31% of cost per dose excluding procurement price per dose, immunisation supplies, wastage, and freight		[37]
Perspective	Health system	Societal
Discount rate	3% costs, 3% DALYs	3% costs, 0% DALYs	[35]

US dollars abbreviated as $ and disability-adjusted life years as DALYs.

We assumed that contact patterns changed over the course of the epidemic, and estimated these changes using Google Community Mobility indicators [20] for Sindh and school closures as reflected in the Oxford Coronavirus Government Response Tracker [2]. For projections, we assume that contact patterns return to the baseline contract matrix at the end of May 2021, and no further physical distancing interventions are imposed.

Model fitting and projections

Using Bayesian inference via Markov Chain Monte Carlo, we fit 5 elements of the model: the effective introduction date, t0, as number of days after 1 January 2020; the basic reproduction number in Sindh without any interventions, R0; a time-varying ascertainment rate for both COVID-19 deaths and cases; and the standard deviation characterising the distribution of reported data points around the model-predicted mean value. We fitted the model to the new daily cases and deaths in Sindh reported by the Government of Pakistan COVID-19 Dashboard [21] from 30 April to 14 September 2020 (Fig 1A and 1B).

Fig 1

Outcomes for fitted model ascertained outcomes compared to data.

Sample ascertained trajectories (n = 250) from the posterior of model parameters (blue) compared to observed outcomes (black). For observed cases and deaths, the solid line is the 7-day average, with points corresponding to daily reports. For the limited serological data, the crosshairs show the collection period and binomial confidence interval on the seropositivity estimates. The serial study results with expected low seropositivity are faded. Expected duration of infection-derived immunity assumed to be 2.5 years; other immunity assumptions in Fig C in S1 Text. All of the assumptions considered produce comparable fits to reported cases and deaths through September 2020.

As a validation, we compared model outputs to 3 reports of SARS-CoV-2 seroprevalence (Fig 1C), all from Karachi [22-24]. Two concerned broad population samples over an extended period [23,24]: These generally overlap with model estimates but are aggregated over time making precise comparison difficult. The third conducted repeat surveys in 2 specific regions [22], finding clear qualitative trends that match model trends—limited exposure pre-May, rapid rise through July, and subsequent plateau—but recorded lower levels. Additional figures in that study, however, indicate that the specific study sites recorded lower positivity trends generally than their broader districts, and particularly during the case surge in June, suggesting that the measured values may be lower than those in the broader population. As another out-of-sample validation, we also compared forward projections to Sindh data for 15 September 2020 to 15 January 2021.

Since the waning rate of infection-acquired immunity is unknown and the benefits of vaccination highly sensitive to this parameter, we repeated the fitting exercise with 4 assumptions for waning of infection-acquired immunity: lifelong and exponentially waning immunity with expected durations of 1, 2.5, and 5 years.

Vaccine programme

We assumed vaccination distribution consistent with the availability of doses indicated by WHO SAGE’s Working Group on COVID-19 Vaccines [25]. We assumed that vaccination required 2 doses per course, since this is true of most vaccines in the COVAX portfolio.

The number of full courses available each month was assumed to be divided among all COVAX participating countries proportional to population, and likewise for subnational regions. Additionally, we assumed that 15% of courses would be wasted for reasons such as cold chain failures, incorrect use, or failure to complete second doses (which we pessimistically assume means lack of vaccine protection). Hence, we assumed that Sindh would complete 4,000 courses/day in the first 3 months (with a 30-day delay accounting for timing of second dose) after a vaccine is approved. We assumed that courses delivered would increase to 16,000, 24,000, then 32,000 in subsequent quarters using the schedule suggested by WHO SAGE [25] modified to reflect the current vaccine landscape (see Section E in S1 Text). For sensitivity analyses, we also consider (a) a constant 4,000 courses per day and (b) a sufficient constant rate to cover the eligible population in 6 months, 184k courses per day, which is comparable to peak rates achieved in some high-income countries (around 0.4% of the population per day).

For primary vaccine scenarios, we assumed that the vaccine is infection-blocking and that protection is complete for some individuals and absent in others (i.e., “all-or-nothing” protection); we considered other vaccine models (every exposure tests the efficacy independently, i.e., “leaky” protection, and/or disease-only blocking) as sensitivity studies. Vaccine doses are distributed among individuals in the Susceptible and Recovered compartments; Susceptible individuals become Vaccinated, and Recovered individuals become Recovered and Vaccinated.

We considered different durations of protection: Once vaccinated in the model, individuals lose vaccine protection with an exponentially distributed duration. Finally, we considered different efficacy levels of protection. A wide variety of COVID-19 vaccines are available to Pakistan via COVAX, which have reported different efficacy levels [26]. Instead of modelling a particular vaccine, for our base case scenario, we assume a vaccine with 70% efficacy that protects for 2.5 years on average. As alternatives, we considered a higher efficacy (90%) or longer duration of protection (5 years). See S1 Text for more combinations.

We track vaccine impact for 10 years and assume that vaccination continues at the same rate (after initial scale-up) for 1 (base case), 5 or 10 years, or for 0.5 year for the sensitivity study of high delivery rate. For simplicity, during the time the vaccine programme is active, vaccination occurs on each day of the week, rather than excluding weekends and holidays. We assume that vaccination cost does not fundamentally change as the programme continues and coverage increases; this implicitly assumes a linear average in the costs of vaccination across groups and time.

Given the emphasis on prioritising older adults in WHO’s vaccine prioritisation roadmap [27], we considered 2 scenarios for distribution: either individuals 15+ years old for the duration or individuals 65+ years old for the first two-quarters of the first year before shifting to 15+. For all scenarios, we assume that vaccine doses are uniformly (i.e., proportional to fraction of population) distributed in the targeted populations.

Health and economic outcomes

We modelled the impact of COVID-19 vaccination on cases, deaths, and disability-adjusted life years (DALYs) compared to counterfactual scenarios with no vaccination over a 10-year time horizon. For different vaccination scenarios, the averted DALYs were combined with the costs of the vaccination programme and any reduction in COVID-19 case management costs from vaccination to calculate incremental cost-effectiveness ratios (ICERs). Our analysis followed the Consolidated Health Economic Evaluation Reporting Standards (CHEERS; for checklist, see Section J in S1 Text), and base case model parameters are listed in Table 1.

DALYs

For each scenario, we modelled the health burden in DALYs for symptomatic cases, nonfatal hospitalisations, nonfatal admissions to critical care, and premature death due to COVID-19. For the nonfatal outcomes, and in the absence of specific DALY data, we used quality-adjusted life years (QALYs) reported by Sandmann and colleagues [28] based on pandemic influenza studies treated 1 QALY gained as equivalent to 1 DALY averted.

For COVID-19 deaths, we estimated DALYs, guided by the approach presented by Briggs and colleagues [29]. We generated age at death in 5-year age bands, and then applied age-specific life expectancy at death using national life tables for Pakistan (United Nations estimates for 2015 to 2020 [30]). We adjusted years of life lost (YLLs) considering the overall level of disability for any remaining years of life using data on QoL by age band from Zimbabwe [31] since all other countries with available data were high-income. However, in our base case analysis, we did not adjust standard life tables to take into account any reduced life expectancy due to specific comorbidities associated with COVID-19. As a sensitivity analysis, since risk of severe COVID-19 is higher for people with comorbidities [32], we modelled an alternative scenario in which half of COVID-19-related deaths were assumed to occur in individuals with higher baseline mortality (standardised mortality ratio = 1.5) and 10% lower baseline QoL. We calculated the average DALYs per death using both 3% (base case) and 0% discounting (Table G in S1 Text).

Costs

We estimated annual economic costs of vaccine introduction and of diagnosis and treatment in 2020 values, using an exchange rate of 155 PKR (Pakistani Rupee) for 1 $ (US dollar) on 1 January 2020 [33] and adjusting earlier data by the gross domestic product (GDP) deflator for Pakistan [34]. Following WHO guidelines, we used a 3% discount rate for future costs and for annualising capital investments, while health outcomes are discounted at either 3% (base case) or 0% [35]. The costing was carried out from a health system (vaccination, testing, and care and treatment costs) and partial societal perspective (including household costs incurred by COVID-19 illness and case management or costs of illness, but excluding benefits of reduced nonpharmaceutical interventions (NPIs)), using a bottom-up ingredients-based approach.

Costs of COVID-19 vaccine introduction

It was assumed that all vaccine doses would be delivered through campaigns in the community. Vaccine and immunisation costs, including supplies costs per dose with freight charges and wastage, are in Table 1. Full costing details are given in Sections 6 and 7 in S1 Text.

The price of the COVID-19 vaccine itself was set at $3 per dose, at which the Serum Institute of India has capped prices for LMICs [36]. The cost per dose of expanding national and provincial level cold chain equipment was obtained from a model of the costs of delivering COVID-19 vaccines in the 92 COVAX countries developed by UNICEF [37]. The additional cold chain costs at the facility level were calculated by allocating a proportion of existing equipment and electricity costs to the COVID-19 vaccine relative to the volume of other vaccines in the immunisation programme.

We estimated vaccine delivery costs via a statewide campaign by adding together the costs of human resources, social mobilisation, and transport. We assumed that nurses and vaccinators would deliver the vaccines. We carried out a microcosting of human resource costs using data from the Disease Control Priorities project [38,39]. Social mobilisation costs were obtained from budgets from a poliovirus campaign [40]. Transportation costs of delivering vaccines to the distribution sites were obtained from the UNICEF model [39]. Transportation costs associated with campaigns originating at facilities were calculated by estimating the catchment areas for facilities and assuming daily vehicle journeys corresponding to the radius of the catchment area.

Our delivery costs did not include additional health system activities, such as planning and coordination, pharmacovigilance, and waste management. Accordingly, we added a 31% markup on the delivery costs, obtained from the UNICEF model [37].

Costs of COVID-19 diagnosis and treatment

The economic impact of COVID-19 on the health system includes diagnosis and clinical management. Costing methods and estimates are reported in full elsewhere [17,43]. Briefly, unit costs of outputs, such as bed days or outpatient visits, were sourced from a range of primary published and unpublished sources in Pakistan. These estimates represent the economic cost of all resources required to deliver health services, including staff time, capital and equipment, drugs, supplies, and overhead costs. Quantities of resources used were defined following WHO guidelines and refined based on expert advice to identify less resource–intensive activities in the area of case management that were more feasible in low- and middle-income settings. More information on unit costs calculations can be found in Sections G and H in S1 Text.

Household costs of COVID-19 diagnosis and treatment include out-of-pocket expenses for care seeking, funeral expenses, and productivity losses due to lost income from isolation of cases and were sourced from previously published work [17].

Outcome evaluation

For our scenarios, we simulate 100 matched replicates sampling from the epidemiological parameter distribution developed by the fitting process. We calculate the resulting epidemiological and economic outcomes (e.g., cumulative DALYs averted, costs and ICERs at annual increments after start of vaccination) for each intervention scenario matched to the corresponding nonintervention scenario (i.e., by draw from the parameter distribution). We then take the relevant quantiles of these simulation outcomes across the samples.

Results

Fit to data and epidemic projections without vaccination

Our transmission model is able to fit reported COVID-19 cases and deaths in Sindh for April to September 2020 for different infection-induced immunity assumptions; each gives comparable quality fits (deviance information criterion (DIC) values: no waning protection, 2,771; expected protection 5 years, 2,772; 2.5 years, 2,778; 1 year, 2,766). The model also produces seropositivity comparable to 3 serosurveys in Karachi. At the end of the fitting period, we estimate 48.1K deaths (95% credible interval (CrI): 45.3 to 49.7K) and 10.5M cases (95% CrI: 9.9 to 10.9M), with ascertainment of 5.3% (95% CrI: 4.8% to 5.8%) of deaths and 1.4% (95% CrI: 1.2% to 2.0%) of cases. Fig 1 shows our baseline assumption of 2.5 years for infection-derived immunity. When the best fitting parameters are used to project cases and deaths beyond September 2020, however, only the shorter durations of protection appear to give a reasonable fit (Fig C in S1 Text).

In forward projections of epidemics between 2022 and 2030 in the absence of vaccination, we found that the duration of immunity following infection is the major determinant of the size of epidemics, as measured by annual incidence (Fig 2). If immunity largely wanes within a year, the region will rapidly settle into recurring epidemics of comparable scale to the 2020 waves. For longer durations of protection, there will tend to be some interannual oscillation. Lifelong immunity results in transmission only at very low residual levels, though we do not consider external reintroductions. This is consistent with epidemic theory, where low immunity duration leads to a rapidly stabilising endemic disease burden, while intermediate durations lead to a series of shrinking epidemic waves settling eventually to lower endemic transmission.

Fig 2

Long-term baseline projections without vaccination for different assumptions about the duration of natural immunity.

Black line shows median simulation, and grey windows mark 50 and 95% simulation intervals.

Impact of vaccination on projected cases and deaths

In our base case scenario, vaccination averts 0.93 (95% Crl: 0.91, 1.0) million cases and 7.3 (95% CrI: 7.2, 7.4) thousand deaths over 10 years (Table 2). We found that the annual cases averted by vaccination are higher for longer duration of vaccine-induced immunity, in scenarios targeting people aged over 15 or over 65 for vaccination (Fig 3). For 70% efficacious vaccines that generated 1, 2.5, and 5 years of protection, the median cumulative cases averted was negative in 2022, and for vaccines of 1 and 2.5 years of protection, also negative in 2023. Only duration of immunity of 1 and 2.5 years showed negative deaths averted in 2022 when targeting 15+, and duration of 1 year when targeting 65+.

Table 2

Costs in US dollars ($), DALYs averted, cost-effectiveness ratio, cases averted, and deaths averted for different vaccination programme scenarios compared to a counterfactual scenario without vaccination.

No.	Description	Impact compared to no vaccination
		Cases Averted (millions)	Deaths Averted (thousands)	Difference in Cost ($ millions)	DALYs Averted (thousands)	Cost per DALY Averted ($)
Base case
1	Vaccine base case	0.9 (0.8, 0.9)	10.1 (10.1, 10.3)	2.0 (0.1, 2.9)	70.1 (69.9, 70.6)	27.9 (1.7, 40.9)
Economic assumptions
2	DALYs discounted at 0%	0.9 (0.8, 0.9)	10.1 (10.1, 10.3)	2.0 (0.1, 2.9)	97.0 (96.8, 97.3)	20.1 (1.3, 29.5)
3	DALYs based on higher comorbidities				54.9 (54.8, 55.4)	35.5 (2.2, 52.2)
4	Societal perspective			−20.2 (−22.5, −19.1)	70.1 (69.9, 70.6)	cs (cs, cs)
5	$6 price per dose			54.7 (52.8, 55.6)		780.5 (749.0, 793.0)
6	$10 price per dose			124.8 (123.0, 125.8)		1,781.9 (1,744.2, 1,795.7)
Vaccine and immunity assumptions
7	Target 15+ from outset	1.0 (0.9, 1.0)	6.4 (6.4, 6.5)	2.7 (−1.1, 4.3)	55.9 (55.1, 57.9)	48.4 (cs, 78.0)
8	5-year campaign	6.0 (6.0, 6.0)	40.7 (40.2, 41.3)	85.6 (84.5, 87.3)	344.8 (341.8, 347.7)	248.1 (243.6, 255.1)
9	10-year campaign	10.9 (10.9, 11.0)	67.0 (66.2, 67.9)	270.3 (267.9, 271.8)	560.7 (556.9, 565.5)	482.1 (473.8, 488.1)
10	Slow rollout: 4K courses per day (no scale-up) for 10 years	0.7 (0.6, 0.7)	32.5 (31.7, 33.5)	38.2 (35.8, 40.1)	154.0 (150.3, 158.5)	247.9 (225.9, 267.1)
11	Fast rollout: 184K courses per day (no scale-up) for 6 months	3.2 (3.0, 3.3)	20.1 (20.0, 20.2)	102.8 (97.2, 112.4)	181.8 (178.4, 183.2)	565.6 (530.9, 629.5)
12	1-year vaccine and natural immunity waning	0.9 (0.9, 0.9)	11.9 (11.7, 12.2)	−5.1 (−6.1, −4.7)	81.9 (81.0, 83.8)	cs (cs, cs)
13	5-year vaccine and 2.5-year natural immunity waning	1.6 (1.6, 1.7)	18.4 (18.2, 18.8)	−53.8 (−54.5, −53.4)	126.8 (125.8, 127.9)	cs (cs, cs)
14	1-dose regimen (twice rate of people vaccinated)	1.7 (1.6, 1.7)	12.9 (12.9, 13.1)	−48.1 (−53.0, −46.7)	105.8 (104.8, 108.0)	cs (cs, cs)
15	30% vaccine efficacy	0.4 (0.4, 0.4)	5.0 (5.0, 5.1)	36.4 (35.8, 36.9)	33.8 (33.7, 33.9)	1,080.7 (1,056.8, 1,091.2)
16	90% vaccine efficacy	1.1 (1.0, 1.1)	12.2 (12.1, 12.4)	−13.4 (−15.8, −12.3)	85.5 (85.3, 86.2)	cs (cs, cs)
17	Vaccine protects against disease-only	0.6 (0.6, 0.6)	9.1 (8.9, 9.3)	14.2 (13.9, 14.4)	60.1 (59.5, 61.1)	236.8 (227.3, 242.3)
18	Vaccine protection is leaky	0.6 (0.6, 0.7)	8.2 (8.2, 8.3)	18.0 (15.8, 19.3)	55.0 (54.6, 56.0)	327.4 (281.4, 353.4)

The base case vaccination scenario assumes the following: a 1-year campaign using a 2-dose vaccine regimen with 70% efficacy at a price of $3 per dose; 2.5-year duration of natural and vaccine induced immunity; and costing from a healthcare perspective.

cs, cost saving; DALY, disability-adjusted life year.

Fig 3

Cumulative cases and deaths averted by the end of each year.

For a vaccine efficacy of 70%, delivered in a 2-dose schedule over a 1-year vaccine campaign, and expected duration of infection-derived immunity assumed to be 2.5 years, the median averted disease (lines; darker ribbon 50% IQR, lighter ribbon 95% IQR) with varying vaccine protection duration (from dark to light, increasing vaccine protection duration) and initial target age group (either 15+ or 65+; after the first quarter of vaccination, 15+ is targeted in both cases); other scenarios and health outcomes in Figs D and E in S1 Text.

These temporary negative years are an outcome of delaying a wave of infections (without replacing lost immunity ongoing vaccination beyond the initial year in the base scenario), leading to offset epidemic years, which ultimately has some net reduction in cases and deaths, but in the short-term experiences an epidemic when with no intervention the infection-induced immunity would be preventing that epidemic. In general, a vaccine with low duration of immunity delayed and slowed the oscillation of epidemics but does not substantially reduce total burden because rapid waning leads to relatively low effective coverage.

Economic outcomes of vaccination strategies

The cost of delivery per dose was estimated to be $1.01, excluding vaccine price, wastage, and freight charges. Based on a vaccine price of $3 per dose, the total undiscounted cost of the vaccination programme was estimated to be $64.1 million, $496 million, and $1.04 billion for a 1-year, 5-year, and 10-year campaign, respectively.

The incremental cost, taking into account cost savings from reduced COVID-19 burden, was influenced by the duration of infection-induced immunity (Fig 4). When this duration was short (1 to 2.5 years), then annual incremental costs are likely to be cost-saving in the long run. For longer durations of infection-induced immunity, the duration of the campaign affected the annual incremental costs, with the potential for negative costs at the cessation of 5-year campaigns from a health sector perspective. If the infection-induced immunity is lifelong, then the extra protection from the vaccine is of limited benefit. The cumulative number of DALYs averted over the entire 10-year time horizon is positive for all vaccine strategies, although it is especially high for a short duration of natural immunity and long vaccine campaign (Fig 5).

Fig 4

Annual incremental costs of vaccination programme (compared to no vaccination) for different vaccination strategies and assumptions about the duration of infection-induced immunity.

Results are shown for vaccination using a 2-dose vaccine regimen with 70% efficacy and 2.5-year duration. The societal perspective includes household out-of-pocket payments and lost income but excludes wider economic impacts of the pandemic. Red lines show different vaccine prices, and the solid and dashed lines show health system costs and with societal costs, respectively.

Fig 5

Cumulative DALYs averted over the 10-year period due to potential vaccination programmes.

For vaccination using a 2-dose vaccine regimen with 70% efficacy and 2.5-year duration. DALY, disability-adjusted life year.

Cost-effectiveness of vaccination scenarios

Over 10 years, our base case vaccination scenario averts 70.1 (95% CrI: 69.9, 70.6) thousand DALYs, at an additional cost of $2.0 (95% CrI: 0.1, 2.9) million after deducting the cost of the vaccination programme (Table 2). These results are relatively stable when vaccination is not age-targeted (i.e., the entire population 15 years and older are given vaccination from the outset), DALYs are discounted at 0%, or COVID-19 patients are assumed to have a higher rate of comorbidities.

A 1-dose regimen (assuming no loss in efficacy) with twice the rate of people vaccinated results in greater health gains averting 105.8 thousand DALYs and is cost saving. Extending the length of the vaccination campaign to 5 or 10 years also substantially increases health benefits but also leads to higher costs yielding ICERs of $248.1 and $482.1, respectively.

Increasing the vaccine price to $6 or $10 per dose would dramatically increase the net costs, leading, respectively, to ICERs of $781 or $1,782 per DALY averted. On the other hand, a vaccine with higher efficacy, longer duration of vaccine protection, or using a societal perspective would make vaccination cost saving.

Discussion

Under a variety of practically relevant epidemiological and economic assumptions, this model of COVID-19 projects that a vaccine programme consistent with the rollout speed projected by COVAX can avert millions of cases and tens of thousands of deaths, and do so in a cost-effective or even cost-saving manner. The particular context considered, Sindh, is a setting with a young population, high SARS-CoV-2 transmission in the past, and limited resources. The age distributions, contact patterns, pandemic history, costs, and income levels of most LMICs are likely more similar to those used in this analysis than to those in comparable studies for high-income countries. While the specific quantities found in this analysis are unlikely to apply explicitly, the qualitative differences from high-income countries likely will.

Reiterating our base case scenario, results: assuming that SARS-CoV-2 does not produce lifelong infection-induced immunity, reaching the COVAX-proposed level of coverage (20%) of an efficacious vaccine (70% efficacy for 2.5 years) within a year (and then ceasing vaccination) may avert 900,000 cases and 10,000 deaths.

Under our base case assumptions, a single year of vaccination would cost an additional $2 million compared to no vaccination and would avoid 70,000 DALYs resulting in an ICER of $28 per DALY averted. This assumes that vaccination can be delivered at $1 a dose, in line with incremental economic cost estimates from the EPIC vaccine delivery costs catalogue, which range between $0.48 and $1.38 for new vaccines [44]. A vaccination campaign extended to 5 years or 10 years would avert substantially more DALYs, but with higher incremental costs, resulting in ICERs of $248 and $482 per DALY averted, respectively. Pakistan does not have a fixed cost-effectiveness threshold. However, a recently conducted exercise defining Pakistan’s Essential Package of Health Services found that over half of the interventions included had an ICER higher than $500 per DALY averted [38,39].

This model also indicates that these benefits are not particularly dependent on the age group targeted. We found that initially prioritising 65+ year olds would avert 60% more COVID-19 deaths compared to vaccinating everyone 15+ years, although the 2 strategies had similar cost-effectiveness since the broader strategy would prevent more cases and more deaths in younger individuals. We did not consider more complex age prioritisation strategies, such as vaccinating 5-year age bands until reaching a particular coverage and then prioritising the next lower band, because such a programme would entail more complex administration costs and need a model validated on age-specific past cases. We do not have the appropriate data to support such an analysis.

The similarity in economic benefits differs from other model-based analyses set in high-income countries [11-13,28], which find that targeting older adults initially would be much more cost-effective. Potential reasons for these differences include the younger age structure of Sindh compared to high-income countries, and the inferred seroprevalence, which was greater than 50% by September 2020. Initial epidemic waves in Sindh (and other settings) may have raised population-level immunity to a point where transmission-reducing vaccination in high-transmission subgroups (i.e., younger, working age) can indirectly protect subgroups at high risk of severe disease (i.e., older, comorbid). While the quantitative benefits decline if the vaccine only provides protection against disease, the conclusions about comparable benefits for either age prioritisation scheme hold.

However, vaccination would be substantially less cost-effective, and potentially not cost-effective from a purely healthcare payer perspective, if the vaccine could only be procured and delivered at $10 a dose or had efficacy as low as 30%. Also, even if a large-scale multiyear mass vaccination programme is cost-effective, it may nonetheless drain scarce financial and human resources from other essential health services. In addition, there are many nonfinancial constraints (e.g., trained personnel), meaning that health opportunity cost may be higher without careful delivery planning. Decisions about vaccination should also take account of other factors besides cost-effectiveness, such as the disproportionately high burden of COVID-19 and related interventions on socioeconomically marginalised groups, and the urgent need to return the economy and society to normal. To effectively inform policy decisions, analyses such as this should be combined with analyses of macroeconomic impact and data on broader societal impacts in a transparent decision framework (e.g., health technology assessment).

In general, our epidemiological projections have relatively narrow uncertainty intervals. While there remains substantial uncertainty on a daily basis, this tends to be offsetting: Cases may shift a little in time, but an annual aggregation results in fairly narrow estimates. These relatively small intervals propagate through the rest of the analysis. These narrow intervals are an accurate reflection of the model assumptions, but the model is fixing many aspects of the real world that are likely to shift unpredictably over the next several years.

As demonstrated by recent emergence of novel variants, the underlying epidemiology may shift, as will technological and social trends, including the relative prices of the inputs to the economic estimation. Variants able to escape vaccine-induced immunity may be introduced either through importation or local mutation. This process is partially addressed by considering loss of infection- and vaccine-derived immunity. For the fastest immunity loss we considered, expected protection durations of a year, a consistently efficacious vaccine (as might be produced by annual updates) can still be cost saving. If variant emergence was more rapid, revaccination with updated formulations might not be able to keep pace, corresponding to lower efficacy. Lower efficacy vaccine (30%) scenarios for rapid protection loss generally resulted in much worse costs per DALY averted (order 1,000s of $ per DALY).

We also assumed that within a particular age group, there is no association between probability of getting vaccinated and risk of disease. This may not be accurate if, e.g., vaccination targets people with comorbidities (and hence higher risk of severe COVID-19 disease), or people who are risk averse (and hence less likely to be infected) are also more likely to get vaccinated. We do not consider future NPIs beyond May 2021 or innovative coordination with vaccination. If there are substantial changes from the impacts integrated into the fitted estimates of local transmission, our projections will not reflect those. Given these core limitations and uncertainties, the intervals ought to be thought of as about our estimate of the central trend conditional on the scenarios, rather than as reflecting the total volatility in the system.

We used a range of scenarios for the duration of natural immunity, although the shortest duration (1 year) best fitted case and death data. This is because the apparent loss of natural immunity may be driven by other factors we did not consider such as behaviour change or emergence of escape variants. If natural immunity is indeed short-lived, this will further strengthen the conclusion that vaccination is likely to be cost saving.

Our findings provide an example of the type of analysis that LMICs can employ to inform vaccination strategies in terms of target populations and financing requirements. While the economic and societal impact of COVID-19 is substantial, the real resource constraints within the health sector in many LMICs mean that vaccination strategies need to balance the current emergency and the longer term needs of the health sector. The slow rate of vaccine distribution is the major impediment to a larger health impact. Administering vaccine doses in line with projected COVAX availability in a province of roughly 50 million people, it would take around 3 years to reach 60% population coverage. Such a long-term programme may not be feasible if vaccine delivery disrupts delivery of other health services, which is a possibility given that the vaccine is targeted at an age group outside the usual Expanded Program on Immunization (EPI). Hence, both short-term rapid response and longer-term consideration about how COVID-19 vaccination can be incorporated in the broader package of essential health services are important in Pakistan and beyond.

Supporting information.

Fig A. Model diagram. Fig B. Demographics in the modelled population. Fig C. Alternative infection-induced protection durations. Fig D. Trends in ICU admissions and person-days. Fig E. Trends in general ward admissions and person-days. Table A. Timeline for COVAX dose availability. Table B. Intervention description and unit costs. Table C. Quantities and price of inputs and unit costs per activity. Table D. PPE costs per general ward bed day and per ICU bed day. Table E. Hygiene costs per general ward and ICU bed day. Table F. Pathways of oxygen management. Table G. Disability life years per COVID-19 death. Fig F. Sensitivity of ICER to time horizon under different assumptions about vaccine price, costs, campaign duration, and duration of natural immunity. Fig G. Sensitivity of ICER to time horizon under different assumptions about comorbidities, discounting of DALYs, campaign duration, and duration of natural immunity.

(PDF)

Click here for additional data file.

24 Feb 2021

Dear Dr Pearson,

Thank you for submitting your manuscript entitled "Health impact and cost-effectiveness of  COVID-19 vaccination in Sindh Province, Pakistan" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff and I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by February 26, 2021.

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Beryne Odeny

Associate Editor

PLOS Medicine

16 Apr 2021

Dear Dr. Pearson,

Thank you very much for submitting your manuscript "Health impact and cost-effectiveness of  COVID-19 vaccination in Sindh Province, Pakistan" (PMEDICINE-D-21-00936R1) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by May 07 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Beryne Odeny,

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

1. Please revise your title according to PLOS Medicine's style. Your title must be nondeclarative and not a question. It should begin with main concept if possible. Please place the study design (for example, "A cost-effectiveness analysis”, “A modelling study”) in the subtitle (i.e., after a colon).

2. Abstract summary - At this stage, we ask that you reformat your non-technical Author Summary. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. The summary should be accessible to a wide audience that includes both scientists and non-scientists. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary.

3. Abstract:

a. Please structure your abstract using the PLOS Medicine headings (Background, Methods and Findings, Conclusions).

b. Please combine the Methods and Findings sections into one section, “Methods and findings”. Please ensure that all numbers presented in the abstract are present and identical to numbers presented in the main manuscript text.

c. In the abstract, please include the important parameters included in your model.

d. In the last sentence of the Abstract Methods and Findings section, please describe the main limitation(s) of the study's methodology.

4. Please avoid assertions of primacy ("Our study provides the first combined epidemiological and economic analysis ...."). Instead use the phrase, “To our knowledge, this is the first…”

5. Please use the "Vancouver" style for reference formatting, and see our website for other reference guidelines https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references.

Comments from the reviewers:

Reviewer #1: General Comments

This manuscript contributes to the evidence base on the cost-effectiveness of COVID-19 vaccination relative to no vaccination in low- and middle-income countries. Within a constrained vaccine supply environment, the authors set out to determine the most cost-effective vaccine prioritisation strategy for LMICs. To do this they combined an epidemiological and economic model to assess the health impact, economic impact and cost-effectiveness of Covid-19 vaccination in the Sindh province, Pakistan. The authors show that it is still cost-effective to vaccinate a very small proportion of the population regardless of age-related targeting strategy as long as the vaccine is reasonable priced, efficacy is high, and a reasonable period of natural- and vaccine-induced immunity. This paper provides compelling evidence for the cost-effectiveness of a small-scale COVID-19 vaccination programme irrespective of targeting strategy.

Minor Essential Revisions

* Vaccine distribution. What does a vaccine campaign of 5 or 10 years mean? Are people vaccinated continuously over the period at a rate of 4000 vaccines/day? Is it assumed that a different segment of the population is vaccinated each year? On page 18, the authors state that "Administering 4000 doses/day in a province of roughly 50 million people would need to be continued for a long time for vaccination to have a large impact". What do the authors mean by 'large impact' - greater reduction in cases and mortality, or closer to herd immunity? At a rate of 4000 doses/day, after 1 year approximately 1.5million (out of a population of 50 million people) will be vaccinated. This is a very small proportion of the population (3%). It would be interesting for the authors to show a scenario that involves a different roll-out from that predicted based on COVAX but results in the ability to vaccinate a higher proportion of the population (that would be closer to a herd immunity target) in one year. Would this still be cost-effective?

* Rate at which vaccine roll-out occurs. The authors also state that "For simplicity, vaccination occurs at the same rate on every day in the model" and that the "slow rate of vaccine distribution is the major impediment to larger health impact" (page 18). How would changing the rate change the results? How would a slower-rate of roll-out affect the results?

* Imports: It would appear that the model does not take into account imports - "Do not consider external re-introductions" (page 10). How does this affect the results? This could be a discussion point.

* Variants: How would the inclusion of variants impact the results? This could be added to the discussion - e.g. the effect of variants is likely to reduce the period of acquired immunity - possibly to less than a year? This would make vaccines more cost-effective if they are protective against variants or less cost-effective if it reduces the vaccine immunity to less than a year. Would it be worthwhile examining vaccine protection of less than a year (worst case scenario for vaccination) to determine what protection duration would no longer be cost-effective.

* One dose versus two-dose vaccine. In the sensitivity analysis in Table 2 it would appear that the adoption of a one dose over a two-dose vaccine changes the number of people vaccinated. But how does it impact vaccine immunity - e.g. what if a person receives only the first dose of a two-dose vaccine, or if there is a delay in receiving the second dose? Is the assumption that the second dose is always provided? If so, this needs to be more clearly stated.

* Methods

o The authors have conducted a very detailed in-depth costing of vaccine delivery which is a major strength of the paper. The analysis would benefit from more clearly disaggregating this costing and presenting it succinctly in Table 1 (or in the results) so that it would be possible to see the total cost per dose administered and the different components that make up this total cost. For example, it would be very interesting to see what proportion of the cost per dose the $3 price of the vaccine contributes towards. Currently, it would appear that the major cost item is this vaccine price, as this changes the results of the analysis from cost saving to no longer cost saving.

o Table 1 - what proportion of doses are assumed to be facility-based v. campaign-based? This presumably informs the range for the vaccine cost per dose?

o The authors assume that the vaccine provides protection against infection and disease? How does efficacy play out in the model? Also, the authors use the term "infection-blocking" in the abstract but in the main text describe the vaccine as providing equal protection against infection and disease.

o Table 1 only shows the base case parameter values but a range is used in the analysis (and presented in Table 2). It would be helpful to the reader to list these all here in Table 1. For example, list base case (70%) efficacy as well as low (30%) and high (90%) etc. This should be followed through to the text - for example, authors state "As alternatives, we considered a higher efficacy (90%) …" But, an efficacy of 30% is also evaluated.

o Organise Table 1 a bit better and maybe include some more of the costs that are mentioned in 'Costs of COVID-19 diagnosis and treatment' section in the table.

o Describe in the methods how the confidence intervals presented in Table 2 are calculated (especially for the ICER).

* Discussion:

o How representative is the Sindh province in Pakistan to other LMICs? How generalizable are these results to other LMICs, for example to LMICs which have lower seroprevalence/higher seroprevalence?

* Other

o Two full stops at end of second last paragraph on Page 6

o Define NPI bottom of page 7

o Define DIC bottom of page 9.

o Figure 3 - refers to Figures SY-SZ? Page 13.

Reviewer #2: This was a timely analysis looking at the cost-effectiveness of vaccination in a province in Pakistan. As COVAX begins to distribute vaccines, it's helpful to understand the expected impact and cost-effectiveness of this vaccine distribution. The authors do an excellent job looking at all different types of

Methods

1. "Leaky protection" - please describe what this means in comparison to non-leaky protection- and the rationale for choosing this approach.

2. "Given the emphasis on prioritising older adults in WHO's vaccine prioritisation roadmap (27), we considered two scenarios for distribution: either individuals 15+ years old for the duration or individuals 65+ years old for the first quarter before shifting to 15+. For all scenarios, we assume vaccine doses are uniformly (i.e., proportional to fraction of population) distributed in the targeted populations."

I recommend considering a third scenario- which may be particularly in resource limited settings- the impact and cost-effectiveness of targeting those 65+ for the entire duration of the time period.

3. "For COVID-19 deaths we estimated age-specific DALYs using the premature-death method by Briggs (29,30) which builds on standard life-table methods to estimate the discounted years of life lost adjusting for age-related quality-of-life (QoL) in the general population, and also allows for inclusion of different baseline morbidity and mortality assumptions."

Traditional DALY calculations would be difficult in COVID, particularly in LMICs where the average age of COVID death is probably greater than the average life expectancy. I had to read the Briggs paper in depth to be able to understand the approach- I think it's worth a few-sentence summary here and address how this specifically is handled in your application of the Briggs approach.

4. Scenario analysis: There were many scenarios evaluated, which were helpful. However, what if there is differential immunity for vaccine and natural immunity- like we see with influenza? E.g. what if the duration of vaccine immunity is 2.5 years and duration of natural infection immunity is 10 years? To what degree does this impact cost-effectiveness?

5. Distribution assumption: The fact that this model examines the COVAX distribution is very useful, particular for the early phase of vaccination. However, once high-income countries feel satisfied with their vaccination rates (ugh. so much for vaccine equity.), there may be more vaccine available for LMICs (or could be advocated for). How much more cost-effective would the vaccination campaigns be if you could double/triple/quadruple the number of vaccines per day (say, starting in mid 2022?)? If this is shown to be vastly (or even marginally) more cost-effective, the findings of this manuscript could be used to further advocate for more vaccines through COVAX or other distribution mechanisms.

I could also imagine a scenario in which you have a massive vaccination campaign every so-many-years (depending on the duration of vaccine-induced immunity)- and that could be substantially more cost-effective than a slow drip of immunization. (So, what if we could vaccinate most of the 65+ population every 5 years or so?) Any of these different distribution scenarios could have the power to change and advocate for vaccines for LMICs and I strongly suggest the authors pursue these scenarios.

Minor comments:

1. "For the non-fatal outcomes, and in the absence of specific DALY data, we used Quality Adjusted Life Years (QALYs) reported by Sandmann et al. (28) based on pandemic influenza studies treated one QALY as equivalent to one DALY averted."

I suggest adding the word 'gained': "… treated one QALY gained as equivalent to one DALY averted."

Reviewer #3: This study estimated COVID-19 cases and death over 10 years under various vaccine scenarios in a population of 48 million people for a Pakistani province. The simulation model is a previously published compartmental transmission model under an extended SEIRS+V structure including birth, death, and age-strata. The model is calibrated to new daily cases and deaths in the Sindh province from April to September 2020 and validated until January 2021. The authors conducted a cost-effectiveness analysis of various vaccine scenarios compared to no vaccination over 10 years. Sensitivity analyses include the length of vaccination campaign, cost per dose, natural immunity loss and duration of vaccine protection, etc. The study concluded that COVID-19 vaccination is likely to be cost-effective if the cost is low and vaccine has good protection against infection in low- and middle- income countries.

This study is well done. Long-term model projections under various natural immunity loss and vaccine protection waning scenarios are particularly insightful. I have the following comments to the authors.

Major comments:

1. Contact patterns changes were estimated using Google Community Mobility indicators and school closures were considered using government response tracker. The authors assumed there is no further social distancing measures after May 2021. This is a very strong assumption. Have the authors considered face mask use data in Sindh and its effect on transmissibility? Given recent government interventions responding to third waves of infection in Europe and face mask use recommendations in many countries, I recommend the authors at least adding a discussion on continuous use of non-pharmaceutical interventions and prolonged changes in contact patterns after 2021.

2. There are several optimistic assumptions about vaccine. First, 10% wastage was assumed. I would like to see if there is any evidence backing up this assumption which seems to be very low. Second, the authors assumed that an effective vaccine provides full protection against infection. I would recommend an additional scenario analysis that relaxes the full protection assumption or a more in-depth discussion on this point. For example, if the vaccine is disease-modifying only, or gives partial protection against infection and reduces transmissibility.

3. One page 6, could the authors clarify this vaccination prioritization strategy, "individuals 65+ years old for the first quarter before shifting to 15+"? Does this mean 65+ only in the first quarter of the first year of vaccination, or prioritized annually like a seasonal flu vaccination scenario?

4. In the base case analysis, future cost is discounted at 3% annually, and health outcomes are discounted at either 0% (base case) or 3%. Equal rate of discounting is the more common practice, though there are some debates about this practice (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5999124/). Was there a particular reason that the authors decided to not discount health outcomes in the base case?

5. The author mentioned in the Discussion that 4000 doses/day would need to be continued for a long time to have a large impact. It would be more informative to give readers some ideas on the vaccine coverage over time.

Reviewer #4: This is an interesting, well-conducted, and timely analysis on the impact of targeted vs general COVID vaccination strategies in low and middle-income countries.

My major comment is that the authors evaluated only two scenarios of vaccine distribution: 1) >65 first followed by the entire adult population, and 2) all persons 15+. Given the complex tiers of vaccine rollout in high-income settings (stratified by healthcare worker status, age, age + comorbidities, essential workers, etc) it would be useful to project the impact of other distribution strategies. For example, the authors find that vaccination of individuals >65 prevents slightly more deaths than mass vaccination, which is not surprising given the smaller proportion of older individuals in low-income settings. However, would vaccinating individuals >50 provide greater benefits? What would be the added benefit of a tiered approach with respect to age and co-morbidities? Projections of vaccine benefit by population heterogeneity (particularly co-morbidities) may be difficult due to the simpler compartmental model design, but it would be helpful to discuss the potential impacts of these strategies or their benefit over a mass vaccination approach in the discussion section.

It would be helpful for the authors to address the impact of vaccine hesitancy and the potential correlation between risk-taking behavior and refusal to uptake vaccination. In particular, vaccinations have a complex political history (ie with polio eradication efforts) and targeted violence toward vaccination workers. https://www.npr.org/sections/goatsandsoda/2021/02/24/968730972/pakistans-polio-playbook-has-lessons-for-its-covid-19-vaccine-rollout

https://gallup.com.pk/wp/wp-content/uploads/2021/01/Gallup-Covid-Opinion-Tracker-Wave-9-pdf.pdf

Given a high distrust of COVID-19 vaccination, would a country such as Pakistan avert greater illness using mass vaccination for all age groups? Perhaps the impact of vaccine refusal rates can be explored in a sensitivity analysis. Perhaps this can also be mentioned in the discussion section.

The authors assume an exponential distribution for waning immunity—this would imply a fast waning early on. It is possible that immunity declines more slowly at first. What are the implications of choosing this distribution?

Introduction:

The authors state: "For all vaccine scenarios, we assumed the vaccine provides protection against infection (not just disease) and that protection is tested with each exposure in the model (i.e. "leaky" protection).

It would be helpful to define the concept of leaky vaccine more clearly for a lay audience of policymakers and researchers.

Do the authors have data on the age distribution of comorbidities associated with COVID-19 severity in Pakistan? If so, how were these data incorporated into the model?

The population mixing assumptions would likely have a strong impact on the results, particularly for herd immunity. Can the matrix assumptions be varied? If not, what are the likely implications of misspecifying the matrix, in terms of choice of vaccination strategy?

For Table 1, it would be useful to list the range of values used in the sensitivity analyses in addition to the base case values.

Much of the discussion section provides a summary of the findings. It would be useful to also contextualize the finding in terms of the strengths and limitations of the model and inputs used. How generalizable are the findings to other countries and what are the factors that most impact generalizability? Eg age structure or mixing in the population?

Reviewer #5: This study aims to assess the health impact, economic impact, and cost-effectiveness of COVID-19 vaccination in Sindh province, Pakistan, using a combined epidemiological and economic model.

Comments:

The authors apply a previously published compartmental model, providing the relevant citation and a concise summary in brief here.

A technically appropriate methodology of Bayesian inference via Markov Chain Monte Carlo has been used to fit elements of the model, and the authors have undertaken a thorough set of out-of-sample validations.

"Vaccine doses are distributed amongst individuals in the Susceptible and Recovered compartments; Susceptible individuals become Vaccinated, while Recovered are unchanged."

Can the authors please explore and discuss whether, by leaving Recovered unchanged, it is realistic to assume that the Recovered population is the same as the Recovered and Vaccinated population within the model?

The authors appropriately acknowledge the limitation of the current model by not including the possible impact of variants.

"As demonstrated by recent emergence of novel variants, the underlying epidemiology may shift, as will technological and social trends, including the relative prices of the inputs to the economic estimation. Given that core uncertainty, the intervals ought to be thought of as on our estimate of the central trend, rather than as reflecting the volatility in the system."

The authors have suitably provided the CHEERS checklist.

Any attachments provided with reviews can be seen via the following link:

[LINK]

3 Jun 2021

Submitted filename: PLoS Med Response to Reviewers.pdf

Click here for additional data file.

22 Jun 2021

Dear Dr. Pearson,

Thank you very much for submitting your manuscript "COVID-19 vaccination in Sindh Province, Pakistan: a modelling study of health impact and cost-effectiveness" (PMEDICINE-D-21-00936R2) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Jul 13 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Beryne Odeny,

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

Thank you for addressing editorial requests.

Comments from the reviewers:

Reviewer #2:

The authors have substantially improved the manuscript in response to reviewer comments. However, upon re-read of the updated version, in light of everything we know about vaccination at this moment in time I have one major concern:

Who is asking whether or not vaccination is cost-effective (and is basing vaccine delivery strategies based on cost-effectiveness)? Across all LMICs that I've worked with throughout the pandemic, the cost of the vaccine does not seem to be the limiting factor in all of this (particularly as the cost/dose is not the $10 that the authors cite as a worst-case scenario) [https://www.who.int/docs/default-source/coronaviruse/act-accelerator/covax/costs-of-covid-19-vaccine-delivery-in-92amc_08.02.21.pdf]. The costs are low, the effect is clearly high. The real issue remains access to vaccines in the first place (while high income countries eat up the world supply) and within country distribution.

I do see some marginal utility in this analysis with regard to type of scale-up strategy (65+ versus everyone)- and the analyses were all very elegant… but ultimately, by the time more guidance is needed, particularly with the introduction of variants that may reduce the response to current vaccines (as the authors have added into the discussion)- the question on policy maker's minds may then rather (and rightly) become the cost-effectiveness of booster vaccines for some proportion of the population instead of just focusing on initial vaccines.

The authors should do a more robust job convincing the reader why this one regional analysis is important, and what it adds to the literature - or adapt the model explicitly to look to the question of variants and boosters.

Minor revisions:

Methods: Suggest changing "highly developed settings" to "high income countries"

Reviewer #3: The authors have addressed all my comments from the last round.

Reviewer #4: The authors have sufficiently addressed the reviewer comments in this revised manuscript. I have a few additional suggestions.

Abstract:

"Varying these assumptions, we generally find that prioritizing the older (65+) population

prevents more deaths, but broad distribution from the outset is economically comparable in

many scenarios, and either scheme can be cost-effective for low per-dose costs."

This sentence is a bit long and hard to follow.

"However, high vaccine prices ($10/dose) may not be cost-effective."

This statement seems very tentative. Can it be made more definitive? For example: We found vaccination was not cost-effective at high vaccine costs (10/dose).

"These projections are limited by the mechanisms present in the model."

This sentence is a bit vague and can be removed. The following sentence could start with: Limitations of this study include…

"Preventing severe disease is an important contributor to this impact, but the advantage of focusing initially on older, high-risk populations may be smaller in generally younger populations where many people have already been infected, typical of many low- and -middle income countries, as long as vaccination gives good protection against infection as well as disease."

This is a very long sentence. It would be helpful to divide it into a few shorter sentences and clarify the conclusion in more certain terms.

"What Do These Findings Mean?"

This section can be modified to incorporate the implications of the study on COVID vaccination policy.

It might be worth mentioning in the discussion section that other age stratified vaccination scenarios were not modeled because of the lack of epidemiologic data and the difficulty of conducting stratified vaccination for most health systems. Otherwise it may be somewhat intitutive that the health impact of prioritizing persons >65 years in LMICs is low because of the small number of people in that category.

Methods, costs

"Following WHO guidelines, we used a 3% discount rate for future costs and for annualising

capital investments, while health outcomes are discounted at either 0% (base case) or 3%

(35)."

Most economic guidelines recommend using the same discount rate for health benefits and costs. It would be useful for the authors to provide an explanation behind the rationale to not discount health benefits (but only discount costs) in their sensitivity analysis. It would be useful to provide a sensitivity analysis where neither costs nor health benefits are discounted.

Table 1

It would be helpful for the authors to conduct more sensitivity analyses as many of the parameters are highly uncertain, particularly COVID natural history, vaccination efficacy/duration. These could be presented in the supplemental appendix and referenced in the text. Costs could also be varied more widely. Vaccination is cost-effective at $3 per dose but not cost effective at $10 per dose—this is a wide range. The ICER for vaccination at $10 per dose is very high. At what approximate threshold does the vaccine no longer become cost effective? Similarly, a 30% vaccine efficacy is unlikely but what is the impact of a 50% efficacious vaccine?

Table 2:

It may be helpful to replace "dom" with "cs" for cost-saving as dom usually refers to dominated (a strategy that is more costly and less cost-effective).

Discussion:

"Many lower- and middle-income settings have similar age distributions and

contact patterns, pandemic history, costs, and income levels. As such, we expect these

qualitative conclusions to apply broadly, though with detailed quantitative outcomes

depending on the location-specific values for those parameters."

Could the authors be more specific about which countries these results would be generalizable to? Other countries in South Asia, or Asia? Or Asia and Africa?

"Under our base case assumptions, a single year of vaccination would cost an additional

USD 2 million compared to no vaccination, and would avoid 70,000 DALYs resulting in an

ICER of USD 28 per DALY averted."

Could these results be specified in cases and deaths averted since DALYs are harder to understand intuitively?

"This model also indicates that these benefits are not particularly dependent on the target

population."

I don't think this can be concluded without conducting an age stratified analysis and including healthcare workers, which is not done in this present study. It is unlikely that vaccinating the general population would result in higher benefits than vaccinating healthcare workers.

Reviewer #5: The authors have satisfactorily responded to each comment in turn, and appropriately adjusted the model to include an Recovered-and-vaccinated (RV) compartment.

Any attachments provided with reviews can be seen via the following link:

[LINK]

Submitted filename: PlosMed VAC response.docx

Click here for additional data file.

6 Jul 2021

Submitted filename: ResponsetoReviewers2.pdf

Click here for additional data file.

5 Aug 2021

Dear Dr. Pearson,

Thank you very much for re-submitting your manuscript "COVID-19 vaccination in Sindh Province, Pakistan: a modelling study of health impact and cost-effectiveness" (PMEDICINE-D-21-00936R3) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor and it was also seen again by three reviewers. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Please note, when your manuscript is accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you've already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosmedicine@plos.org.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Aug 12 2021 11:59PM.

Sincerely,

Beryne Odeny,

Associate Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

1. Please format your references as follows:

a. Please use the PLOS Medicine style reference call outs throughout the text, noting the absence of spaces within the square brackets, e.g., "... every country) [1,2]." https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references.

b. Please ensure that journal name abbreviations match those found in the National Center for Biotechnology Information (NCBI) databases, and are appropriately formatted and capitalized. https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references. Please ensure that weblinks are current and accessible to date.

2. Please consistently define abbreviations in tables and figures e.g., USD, DALYS

Comments from Reviewers:

Reviewer #1: The authors have addressed all my comments from the last round. I however feel that the authors still need to adjust table 1 to reflect the "15% wastage for vaccines, and the 10% wastage for immunization supplies" as indicated in the text.

Reviewer #4: The authors have sufficiently addressed the reviewer comments in this revised manuscript.

Any attachments provided with reviews can be seen via the following link:

[LINK]

23 Aug 2021

Submitted filename: Sindh Response Letter, part III.pdf

Click here for additional data file.

14 Sep 2021

Dear Dr Pearson,

On behalf of my colleagues and the Academic Editor, Dr. Brooke E Nichols, I am pleased to inform you that we have agreed to publish your manuscript "COVID-19 vaccination in Sindh Province, Pakistan: a modelling study of health impact and cost-effectiveness" (PMEDICINE-D-21-00936R4) in PLOS Medicine.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

In the meantime, please log into Editorial Manager at http://www.editorialmanager.com/pmedicine/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with medicinepress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for submitting to PLOS Medicine. We look forward to publishing your paper.

Sincerely,

Beryne Odeny

Associate Editor

PLOS Medicine