Variation in COVID-19 booster uptake in England: An ecological study.

INTRODUCTION: Variable and low uptake of the COVID-19 booster is a recognised problem, associated with individual characteristics including age, gender, ethnicity, and deprivation. Are there other relevant predictors at area level?
METHODS: Anonymous grouped data was downloaded from the UK Government Coronavirus Dashboard for Middle Super Output Areas (MSOA) in England, along with demographic, employment, and health data from public sources. Mixed models with a random intercept for Upper Tier Local Authority were analysed as quasibinomial Generalized Additive Models. The estimated random effects were then fitted with Bayesian linear mixed models using flu vaccination uptake, change in public health budgets, population proportion of vaccination sites at pharmacies, GP-led, vaccination centres, and hospital hubs, and Region.
RESULTS: Models for the MSOA-level COVID-19 first and second vaccinations and the Third Injection (including the booster), fit well. Index of Multiple Deprivation, proportion Aged 15-24 and 25-44, and ethnicity groupings Other White, Indian-Pakistani-Bangladeshi, and African-Caribbean-Other Black-Other, are highly significant predictors of lower uptake. The estimated random effects vary widely amongst local authorities, with positive impact of flu vaccine uptake and change in public health budgets, and regional impacts which are positive for London and South East (first and second doses only), and negative for North West and North East. The impact of vaccination sites did not reach 90% credibility, in general.
CONCLUSION: COVID-19 vaccination rates at each stage are very well modelled if local authority random effects are included along with non-linear terms for demographic, employment and health data. Deprivation, younger age, and Other White, South Asian, and African-Caribbean-Other ethnicities are associated with lower uptake. The estimated local effects show strong regional variation and are positively associated with flu vaccination and increasing public health budgets. One simple way to improve COVID-19 vaccine uptake in England would be to increase local public health allocations.

Introduction

The booster programme has been central to the UK government’s strategy for containing COVID-19 in England during the autumn and winter of 2021–22. The booster was introduced on 16 September 2021, and rolled out to people in eligible groups who had their second primary dose at least six months previously [1]. However, uptake of the booster remains well below the levels achieved for the first and second vaccine doses [2]. Vaccination uptake is highly dependent on age, gender, ethnicity, and deprivation, as is widely recognised both in the UK [3-7] and internationally [8, 9].

As demography varies across England, vaccination rates will also vary. For example, as the Omicron BA.1 variant swept the country on 4 January, the cumulative Third Injection (Booster or 3rd primary for persons with compromised immunity) uptake in Newham was 27.2%, in contrast with 41.1% in Liverpool and 67.4% in Gloucestershire, according to data from the UK Coronavirus Dashboard [10]. These are different populations, but does demography explain the disparity in full? If there are additional sources of variation, are they characteristics of people or of where they live?

I explore these questions by modelling cumulative COVID-19 vaccination in 6789 Middle Super Output Areas (MSOA) of England, a UK Census geographic unit comprising around 8,000 people on average and covering the whole of England. Each MSOA belongs to a single Upper Tier Local Authority (UTLA), an adminstrative and geographic unit. There are 149 UTLA, containing an average of 46 MSOA each, ranging from 5 (Rutland) to 182 (Kent). Apart from Rutland, each UTLA contains at least 12 MSOA. In the first stage of analysis, the observed vaccinations at MSOA level are fitted with a non-linear mixed model using demographic, employment and health covariates at MSOA level and a random intercept for the UTLA. The model does not estimate individual take-up of vaccinations on the basis of covariates known for individuals, but accounts for the take-up at MSOA level using MSOA covariates including the UTLA to which the MSOA belongs. This analysis is structured as a (non-Bayesian) Generalized Additive Model, whose output includes curves showing the impact of each covariate in the presence of all others, and an estimated random effect for each UTLA.

These estimated random effects are then taken as observations in a Bayesian linear mixed model, using selected predictors available at UTLA level and a random intercept for the 9 Regions which partition the 149 UTLA. The fixed effects in this second stage model are flu vaccination rate, the population proportions of various types of vaccination sites, and annual change in local authority public health budgets. Regional disparity, public experience of flu vaccination, access to COVID-19 vaccines, and changes in public health department capacity, are all plausible influences on how local authorities may differ even while controlling for other factors present in the first stage model.

An overview of the modelling is set out in the Statistical methods below, with full codes and simulations in the Supporting information. The first stage models fit the data well, and show variation in Upper Tier Local Authority random effects. The second stage models identify positive and negative predictors for the estimated random effects.

Materials and methods

Data sources

For each MSOA, the cumulative number of persons vaccinated, separately for each of the first two primary doses and the Third Injection, is available daily from the UK Coronavirus Dashboard along with the Vaccine Register Population. Data for the Third Injection comprises people given the Booster plus people over 12 years old with severely weakened immune system given a third (primary) dose. The Dashboard also provides MSOA incidence (“7 day rolling rate”) of COVID-19 cases, weekly through to 5 days before download. Dashboard data was downloaded on 4 January 2022.

For each MSOA, the Index of Multiple Deprivation (IMD), and separate indices for the Health and Education deprivation domains, were taken from the English indices of deprivation 2019 [11]. Each MSOA comprises a number of Lower Super Output Areas (LSOA), whose population and area (mid-2019 estimates) [12] were used to obtain population density at MSOA level. Population by MSOA was taken from the mid-2019 estimates [13] which include data by gender and each year of age. Ethnicity at MSOA level from the 2011 Census was downloaded from the Office for National Statistics (ONS) Nomis dataset DC2101EW—Ethnic group by sex by age [14]. Employment by industrial sector is available at MSOA level from the 2011 Census and was downloaded from the Nomis dataset DC6110EW [15]. Communal establishments data was downloaded from the Nomis dataset QS421EW [16], and Multi-occupation housing from the Nomis dataset DC1109EW [17].

Mean distance to GP, Emergency Department, and Pharmacy is available at LSOA level from the AHAH dataset [18]. GP registration mapped to LSOA is available from NHS Digital [19]. MSOA level values were obtained as population-weighted means from the LSOA values within the MSOA. A lookup from MSOA to Local Authority to Upper Tier Local Authority to Region was available from Public Health England as part of weekly surveillance. The 28 October 2020 report is online, with lookup in columns 1–8 (select all, show columns) [20].

Seasonal Flu Vaccine Uptake data for 2020/21 at UTLA level is released by Public Health England [21]. Data was extracted for GP-registered patients aged under 65. The uptake for “Leicester and Rutland” was then assigned to each area separately. COVID-19 vaccination sites as of 17 November 2021 are listed by NHS England and NHS Improvement [22], separately for pharmacies, GP-led, vaccination centres, and hospital hubs. Their postcodes were mapped to Lower Tier Local Authority codes with the ONS National Statistics Postcode Lookup [23], and then to UTLA codes via the lookup above. Public Health ring-fenced grant allocations for 2020/21 and 2021/22 are available at UTLA level from the Department of Health [24, 25].

A shapefile with the May 2021 UTLA boundaries was downloaded from the ONS Open Geography Portal [26]. ONS files are free to use without a specific application for a licence, subject to the conditions of the Open Government Licence and the Framework [27]. Source: Office for National Statistics licensed under the Open Government Licence v.3.0. Contains OS data © Crown copyright and database right [2022].

Statistical method

The first stage of modelling seeks to account for the observed vaccination rates at MSOA level using demographic, employment and health data along with random effects for the 149 UTLA.

Cumulative numbers of persons vaccinated were combined with the Vaccine Register Population to form a matrix vmat with the numbers vaccinated and unvaccinated in each of 6789 MSOA, as of 4 January 2022. For each vaccination, the corresponding matrix (vmat1 etc) is the response variable. Four models were considered, differing in the choice of explanatory covariates, and fitted separately for each vaccination (first and second primary doses, and the Third Injection).

These first stage models were structured as Generalized Additive Models (GAM), using the R package “mgcv” [28-30]. The factor utla with 149 levels denotes the UTLA for each MSOA, and is treated as a random effect in all four models. All other covariates are treated as fixed effects.

In one sense, these particular models have a simple form, because the random effect of utla is a random intercept, without interaction with the other covariates. A model of this type is the simplest of the mixed models outlined within a linear context in a review of Multilevel Analysis in Public Health Research [31] (Equation 4 in this reference would reduce to Y = γ00 + γ10 I + U0 + ϵ). Random intercept models are included in a more recent review of ecological modelling [32], in a linear or generalized linear context. However, it is overly restrictive to assume models of these types. Non-linear mixed models are included in Mixed Effects Models in S and S-PLUS [33]. GAM is an even more flexible approach, in which the transformed mean response is approximated by unknown combinations of spline functions of the covariates.

Briefly, GAM models without interaction terms have the general form: where μ = E(y) is the expectation of the r response y which is assumed to follow a specified distribution, g is the link function, X is the r row of the model matrix for the parametric terms with parameter vector θ, and the f are smooth functions of the covariates x whose value at the r observation is x.

In our case, y is the vaccine uptake vmat[r,1]/(vmat[r,1]+vmat[r,2]) and g is the standard link function for binomial data, g(μ) = log(μ/(1 − μ)). The random intercepts appear as f1(x1) where x1 is the factor utla and f1 maps each level of the factor to an unknown coefficient c whose value is estimated during the fitting. The parametric term X * θ is omitted and all levels of utla enter the model on an equal footing.

The smooth functions f2 … are splines, evaluated at the covariates for the fixed effects which were scaled or transformed before applying the spline functions. Covariates for IMD, cumulative incidence, and population density were scaled. Most other covariates were transformed to improve performance. For a covariate x, the monotonic transform Lx is defined as log(1+x/mean(x)). For example, the population proportion for a particular ethnicity or industrial sector may be very low, but its transformation has better distribution and will be more easily smoothed.

The vaccination data has greater variability than expected with a binomial distribution, so the models use the “quasibinomial” family, whose dispersion is estimated during fitting. Models use cubic regression splines with basis dimension 7.

Model uses a random intercept for utla, and smoothers for cumulative incidence (scaled), IMD (scaled), population density (scaled), five age bands (as transformed proportions), male (transformed proportion), and six ethnicities (transformed proportions). The age bands are 0–4, 5–14, 15–24, 25–44, and 45–64. The ethnicities, using the designations from the 2011 Census, are White Irish and White Gypsy (combined group); Other White; Indian, Pakistani, and Bangladeshi (combined group); Chinese and Other Asian (combined group); African, Caribbean, Other Black, and Other Ethnicity (combined group); Arab.

Model enlarges with additional terms for the transformed population proportions of 15 industrial sectors: Agriculture energy and water, Manufacturing, Construction, Wholesale and retail, Transport and storage, Accommodation and food service, Information and communication, Financial and insurance, Real estate, Professional scientific and technical, Administrative and support service, Public adminstration and defence, Education, Human health and social work, Other.

Model enlarges with additional demographic and local health covariates for transformed population proportion of Communal establishments, of Multi-occupation housing, population weighted mean distance to GP, to A&E, to Pharmacy, proportion registered with GP, Health deprivation, and Education deprivation.

Model enlarges with most of the additional terms from and , but omits six terms whose contributions showed negligible or low significance: population density, Manufacturing, Accommodation and food, Information and communication, Real estate, mean distance to GP. The included fixed effects cover cumulative incidence, IMD, age (5 bands), gender, ethnicity (6 groups), employment (11 sectors), communal establishments, multi-housing, distance to A & E, distance to pharmacy, GP registration, health deprivation, educational deprivation.

The methods used here for fitting a GAM and optimising the choice of smoothing parameters are outlined in S1 Appendix. Models were fitted separately for each of the first two doses and the Third Injection. Model fit was evaluated by residual plots, a test of convergence of the smoothness parameter optimisation, and tests of whether the basis dimensions are sufficient. Outliers were detected by Cooks distance > 0.02, a somewhat arbitrary criterion. The actual and predicted proportions vaccinated in each MSOA were plotted, along with the smoother curves for particular covariates. Models were compared by ML value (see S1 Appendix), with lower value indicating better fit. For each smoothed covariate, the fitted model shows a p-value for an approximate significance test against the hypothesis that the corresponding smoother is in fact zero [29, 30].

For each fitted model, the first 149 coefficients c are the estimated contributions of the utla factor levels. These coefficients also enable comparison of the fitted value for a particular MSOA, with the hypothetical fitted value if the same population were located in a different UTLA, replacing c with c. If g denotes the link g(μ) = log(μ/(1 − μ)) and h is the inverse link h(t) = 1/(1 + exp(−t)), the fitted value would be altered from fit1 to h(g(fit1) − c + c).

As outlined in S1 Appendix, each c has an approximate standard error se. “Caterpillar plots” were drawn using the c to show the range of estimated random effects, with approximate 95% credible intervals obtained from the se. A UTLA map of the variation across England was produced in QGIS [34], using the ONS boundary shapefile [26] with shadings determined by the c values.

In a second stage of modelling, the c were taken as observed, to be predicted using covariates available for Upper Tier Local Authorities. Region was treated as a random effect, with fixed effects for the uptake of Flu vaccination, the change in Public Health ring-fenced grant allocations from 2020–21 to 2021–22, and the number of each type of COVID-19 vaccination site, per population within the local authority.

For this stage, 4000 posterior draws from a linear mixed model were obtained with the Bayesian programme rstanarm [35], and graphical checks were carried out with shinystan [36] and bayesplot [37]. Output includes parameter estimates and 90% credible intervals. Plots of predicted and observed c were generated for each dose and the Third Injection. Using the associated package loo [38], a pointwise value of Pareto k > 0.7 was taken to indicate an outlier [39]. A Bayesian version of R2 describes overall model fit [40].

In fact the c are not observed, but are amongst the coefficients of the fitted first stage model. The coefficients have an associated covariance V (S1 Appendix). To estimate the impact of this uncertainty, simulated c* were drawn as multivariate normal with mean c (the vector with components c) and variance V[1 : 149, 1 : 149]. Second stage modelling was repeated using the simulated c* to give fresh sampling output (4000 rows), and this process was repeated 100 times to produce a combined sampling matrix with 404,000 rows (including the original output from second stage modelling of the c). The mean, 0.05 and 0.95 quantiles of its columns were taken as corrected estimates and credible intervals of the second stage parameters, taking into account the uncertainty in the c.

A guide, data, code files, simulations, summary of model and the appendix are included as Supporting information.

Results

Outputs for the 4 first stage models and 3 vaccinations are shown in Table 1. For each vaccination, has the lowest ML value and the highest Proportion of deviance explained (see S1 Appendix). The models pass tests of model fit, and spline basis dimensions were sufficient (see Statistical methods).

Table 1

ML values and proportion of deviance explained.

Model	First dose		Second Dose		Third Injection
	ML	DevExp	ML	DevExp	ML	DevExp
A	-15317.25	0.953	-15905.70	0.958	-15605.74	0.965
B	-15695.38	0.959	-16208.56	0.962	-16012.88	0.969
C	-15985.15	0.962	-16503.18	0.965	-15981.23	0.969
D	-16198.07	0.965	-16691.77	0.968	-16222.98	0.972

For each model and vaccination, the table gives the ML value of the fitted model and the proportion of deviance explained (see S1 Appendix). Lower values of ML correspond to models with a better fit, according to the criterion used here to fit the models. Higher values of DevExp correspond to models which come closer to the best possible fit using a different criterion (the deviance).

For the Third Injection with model , all predictors have highly significant impacts (p<0.001) except for distance to pharmacy (p<0.005) (see S7 File). Smoothers for each of the following covariates have p<2e-16: utla, the age bands 5–14, 15–24, and 25–44, ethnicities Other White, African Caribbean Other Black and Other (combined), South Asian (combined), IMD, Male, GP Registration, Health and Social work. explains 97.2% of the deviance, whilst a model using only the random effects term explains 51.5%, and a model using only the fixed effects explains 95.7% (percentages do not sum to 97.2% as in the absence of some terms, others compensate). Both of the reduced models have considerably worse fit than (higher ML values), and are far inferior by anova tests. In particular, the fixed effects model has ML = -15247.41, which is 975.57 above the value for , and an anova test against has p<2.2e-16, whilst the random effects term in has p<2e-16. This is strong evidence that utla has a significant impact on vaccination uptake, even when controlling for all the fixed effects.

Figures were produced for the Third Injection with model . Fig 1 shows the observed and fitted values of uptake, the outliers with Cooks distance > 0.02 and the 37 MSOA within Newham. Cooks distance is a metric combining the influence of individual points on the fitted model, and the magnitude of their deviance residual. There are six outliers as judged by Cooks distance > 0.02, and of these only Forest Heath 002 (Lakenheath) has a high residual, the others being highly influential points. Newham 013 (Olympic Park & Mill Meads) is an outlier, with uptake higher than expected. Newham 035 (Beckton Park), with the lowest uptake (0.223) in this local authority, nearly matches its predicted value (0.229). Overall, the model fits the data very well.

Fig 1

Fitted value versus uptake, model for the Third Injection.

Observed values of uptake are shown as semi-transparent blue points, and the green dashed line shows the fitted values. Outliers are shown in brown, and the Middle Super Output Areas within Newham in red. Vaccination data as of 4 January 2022. is a Generalized Additive quasibinomial mixed model, with a random effect for Upper Tier Local Authority, and fixed effects as described in Statistical methods.

Fig 2 shows selected smoothers for model . The smoothers in panels A—G have much larger impact on fitted values than subsequent covariates, and those in panels B—G have a negative impact: higher values of scaled IMD (Fig 2B) give lower fitted values, and likewise for the age bands and ethnicities shown. An increasing male proportion (Fig 2H) also has a negative impact, but smaller than that of IMD, age, or ethnicity.

Fig 2

Selected smoothers for model for the Third Injection.

The panels show the modelled impact of individual covariates on the overall fitted value (displayed on the scale of the linear predictor, before it is translated to fitted output by the inverse link function). The x-axis is limited by the 0.01 and 0.99 quantiles of the covariate. In some panels the y-axis is limited by the range of the smoother and is labelled “narrow scale”.

Fig 3 shows the “Caterpillar plot” for the estimated random effects c. Many estimated effects have 95% credible intervals which do not overlap the mean value. On the left, 51 estimated random effects are negative, whilst on the right 56 are positive. Only 42 UTLA have estimated effects whose 95% credible interval includes the mean value. The estimated random effects obtained from fitting model to the first and second doses are highly correlated (ρ = 0.99), whilst the estimates when fitting to the second dose and Third Injection are well correlated (ρ = 0.85).

Fig 3

“Caterpillar plot” for model for the Third Injection.

Each estimated random effect is displayed with a point whose y-value is c and a vertical segment showing its 95% credible interval. The points are ordered along the x-axis by increasing value of c. Points marked • also show the Upper Tier Local Authority name below. The blue line shows the mean value of c.

Fig 4 maps the geographic distribution of estimated random effects using model for the Third Injection. There is clear regional variation, with negative impact for Upper Tier Local Authorities in the North West and parts of the North East, and positive impact in parts of London, the South East and South West.

Fig 4

Geography of estimated random effects of Upper Tier Local Authorities, for the Third Injection.

A map of England is shown with Upper Tier Local Authorities, each shaded according to the estimated random effect c using model for the Third Injection. Areas with red (blue) shading have estimated random effects below (above) the mean.

Table 2 shows the results of second stage modelling of the scaled estimated random effects from each of four models for the Third Injection. This second stage modelling is then repeated with the scaled simulated random effects from each model for the Third Injection.

Table 2

Parameter estimates from 2nd stage modelling of random effects for Third Injection.

	A		B		C		D
Bayes R 2	0.203		0.293		0.353		0.369
ci estimated
	estimate	90% credible
Flu vaccination	0.23	(0.06,0.40)*	0.23	(0.06,0.40)*	0.32	(0.16,0.48)*	0.29	(0.13,0.45)*
Public Health change	0.33	(0.16,0.49)*	0.22	(0.06,0.38)*	0.30	(0.15,0.46)*	0.21	(0.06,0.37)*
Pharmacy vacc sites	-0.01	(-0.15,0.14)	-0.06	(-0.21,0.08)	-0.07	(-0.21,0.07)	-0.06	(-0.20,0.07)
GP vacc sites	0.12	(-0.02,0.25)	0.06	(-0.07,0.19)	0.11	(-0.01,0.23)	0.08	(-0.05,0.20)
Vaccination Centres	0.12	(-0.01,0.24)	0.09	(-0.04,0.21)	0.13	(0.01,0.25)*	0.10	(-0.02,0.22)
Hospital Vacc Hubs	0.05	(-0.08,0.18)	0.02	(-0.11,0.14)	0.05	(-0.08,0.17)	0.04	(-0.08,0.15)
East Midlands	0.12	(-0.18,0.53)	0.26	(-0.14,0.66)	0.27	(-0.11,0.66)	0.27	(-0.12,0.67)
East of England	-0.14	(-0.53,0.15)	-0.03	(-0.40,0.35)	0.06	(-0.30,0.44)	0.10	(-0.29,0.49)
London	0.16	(-0.10,0.49)	0.43	(0.09,0.79)*	0.30	(-0.00,0.62)	0.47	(0.14,0.80)*
North East	-0.01	(-0.38,0.34)	-0.36	(-0.85,0.07)	-0.31	(-0.78,0.09)	-0.53	(-0.99,-0.08)*
North West	-0.38	(-0.75,-0.04)*	-0.79	(-1.13,-0.47)*	-0.81	(-1.12,-0.49)*	-0.94	(-1.24,-0.63)*
South East	-0.08	(-0.39,0.18)	0.12	(-0.19,0.42)	0.16	(-0.13,0.45)	0.25	(-0.05,0.55)
South West	0.07	(-0.21,0.42)	-0.02	(-0.37,0.32)	0.20	(-0.13,0.54)	0.14	(-0.19,0.49)
West Midlands	0.16	(-0.11,0.53)	0.28	(-0.05,0.62)	0.19	(-0.14,0.53)	0.23	(-0.10,0.56)
Yorkshire & Humber	0.10	(-0.16,0.43)	0.06	(-0.27,0.38)	0.04	(-0.27,0.34)	-0.03	(-0.36,0.28)
ci simulated
	estimate	90% credible
Flu vaccination	0.23	(0.05,0.40)*	0.23	(0.05,0.41)*	0.31	(0.14,0.47)*	0.28	(0.11,0.45)*
Public Health change	0.32	(0.15,0.49)*	0.22	(0.04,0.39)*	0.30	(0.13,0.46)*	0.21	(0.04,0.37)*
Pharmacy vacc sites	-0.01	(-0.15,0.14)	-0.05	(-0.20,0.10)	-0.07	(-0.21,0.08)	-0.06	(-0.20,0.08)
GP vacc sites	0.11	(-0.03,0.25)	0.06	(-0.07,0.19)	0.11	(-0.02,0.23)	0.08	(-0.05,0.20)
Vaccination Centres	0.11	(-0.02,0.25)	0.08	(-0.05,0.21)	0.13	(0.00,0.25)*	0.10	(-0.03,0.22)
Hospital Vacc Hubs	0.04	(-0.10,0.18)	0.02	(-0.11,0.16)	0.04	(-0.09,0.17)	0.04	(-0.09,0.16)
East Midlands	0.14	(-0.18,0.52)	0.26	(-0.14,0.68)	0.27	(-0.11,0.68)	0.27	(-0.13,0.68)
East of England	-0.15	(-0.52,0.16)	-0.03	(-0.43,0.36)	0.05	(-0.32,0.43)	0.10	(-0.29,0.49)
London	0.16	(-0.11,0.49)	0.42	(0.06,0.80)*	0.30	(-0.03,0.64)	0.46	(0.11,0.81)*
North East	-0.02	(-0.38,0.33)	-0.37	(-0.85,0.08)	-0.31	(-0.76,0.11)	-0.51	(-0.99,-0.05)*
North West	-0.36	(-0.73,-0.01)*	-0.76	(-1.11,-0.41)*	-0.78	(-1.11,-0.44)*	-0.91	(-1.23,-0.59)*
South East	-0.09	(-0.38,0.18)	0.11	(-0.20,0.44)	0.16	(-0.14,0.47)	0.24	(-0.07,0.56)
South West	0.08	(-0.22,0.40)	-0.02	(-0.38,0.33)	0.19	(-0.15,0.54)	0.13	(-0.22,0.49)
West Midlands	0.18	(-0.11,0.52)	0.27	(-0.07,0.63)	0.19	(-0.14,0.53)	0.21	(-0.13,0.56)
Yorkshire & Humber	0.11	(-0.17,0.43)	0.06	(-0.28,0.40)	0.02	(-0.30,0.35)	-0.03	(-0.36,0.31)

The first line shows the Bayes R2 (see Statistical method) when second stage modelling is applied to the scaled estimated random effects for each of four models applied to the Third Injection. Below this, the upper portion shows the parameter estimates and 90% credible intervals for each covariate in the second stage model. The lower portion uses second stage modelling of scaled simulated random effects (see Statistical method). Credible intervals which exclude 0 are starred.

Bayes R2 improves when second stage modelling is applied to the estimated random effects obtained from better first stage models ( is preferred to other models by ML value and leads to higher Bayes R2). All four models lead to second stage models showing Flu vaccination and the rise in Public Health budgets as positive predictors (90% credible interval strictly positive) of estimated or simulated random effects. The population proportion of Vaccination Centres is a positive predictor only with model . North West Region is a negative predictor with all four models, and the North East is negative with model . London is a positive predictor with models and .

A comparable table for the first dose shows that Bayes R2 rises from 0.398 for modelling the estimated random effects from , to 0.464 for , 0.606 for , and 0.600 for . All four models lead to second stage models which show Flu vaccination and the rise in Public Health budgets as positive predictors of the estimated or simulated random effects, and likewise for London and South East regions, whilst North West is negative, and North East is negative for , , and . South West is positive for and . Hospital hubs are positive for .

For the second dose, Bayes R2 rises from 0.400 for modelling the estimated random effects from , to 0.462 for , 0.594 for , and 0.589 for . All four models lead to second stage models which show Flu vaccination and the rise in Public Health budgets as positive predictors of estimated or simulated random effects, and likewise for London and South East regions, whilst North West is negative, and North East is negative for , , and . South West is positive for and , and borderline for .

Fig 5 shows the scaled estimated random effects from model against the prediction from second stage modelling, for each vaccination.

Fig 5

Second stage prediction of scaled estimated random effects from model .

For each vaccination, the corresponding panel shows the scaled estimated random effects c against the median posterior predictive values from the second stage model. The dashed green line shows the predictions, and the dashed red line shows a smooth spline through the c. Points marked • are selected Upper Tier Local Authorities mentioned elsewhere in the text. For these points, the vertical dotted lines show the 90% credible interval of posterior predictive values.

Discussion

There is extensive literature on COVID-19 vaccine uptake, including evidence from individuals. Before the vaccines were deployed, a telephone and web survey of attitudes, considered by the UK government’s advisory group SAGE in December 2020 found “marked differences existed by ethnicity, with Black ethnic groups the most likely to be COVID-19 vaccine hesitant followed by the Pakistani/Bangladeshi group. Other White ethnic groups (which includes Eastern European communities) also had higher levels of COVID-19 vaccine hesitancy than White UK/White Irish ethnicity” [3]. SAGE cited previous studies to conclude “Barriers to vaccine uptake include perception of risk, low confidence in the vaccine, distrust, access barriers, inconvenience, socio-demographic context and lack of endorsement, lack of vaccine offer or lack of communication from trusted providers and community leaders.”

A UK cohort analysis by Helen Curtis et al [5] of 57.9 million patients’ primary care records, concerns the period from 8 December 2020 to 17 March 2021. Of patients aged ≥80 years not in a care home, 94.7% received a vaccine, but with substantial variation by ethnicity (White 96.2%, Black 68.3%) and deprivation (least deprived 96.6%, most deprived 90.7%).

A meta-analysis by Eric Robinson [8] of 28 nationally representative samples from 13 countries found “Being female, younger, of lower income or education level and belonging to an ethnic minority group were consistently associated with being less likely to intend to vaccinate”. A meta-analysis by Qiang Wang [9] of international studies including representative samples found that “Gender, educational level, influenza vaccination history, and trust in the government were strong predictors of COVID-19 vaccination willingness” and that healthworkers were less willing than the general public.

Recent flu vaccination history and trust in routine vaccination recommendations from health workers were amongst the positive factors in willingness-to-pay for COVID-19 vaccines in a survey by Malik Sallam et al [41] of individuals from 10 low-middle income countries in Asia, Africa, and South America.

An international review of COVID-19 vaccine hesitancy by Wardah Hassan et al [42] included a table of factors associated with acceptance or rejection in a wide range of countries, and highlighted the role of disinformation and negative religious beliefs as barriers.

Occupation also influences vaccine uptake. The Office for National Statistics Coronavirus (COVID-19) latest insights: Vaccines, (as of 31 December 2021) [43] shows survey data on how the proportion receiving 3 vaccinations varied by occupation groupings, ranging from 80.4% (for persons employed in Health) to 48.0% (Elementary trades and related occupations); and for more specific occupations; and by ethnicity, ranging from 68.4% (White British) to 33.9% (Black Caribbean) [44].

Vaccination status is an important predictor of mortality, though some ethnic contrasts persist even when controlling for it. An ONS study of ethnic contrasts in coronavirus death rates 8 December 2020 to 1 December 2021 found that “Location, measures of disadvantage, occupation, living arrangements, pre-existing health conditions and vaccination status accounted for a large proportion of the excess rate of death involving COVID-19 in most ethnic minority groups; however, the Bangladeshi ethnic group and men from the Pakistani ethnic group remained at higher risk than White British people in the third wave, even after adjusting for vaccination status” [45].

Several authors have made recommendations based on the impact of individual characteristics. Mohammed S Razai outlined a strategy to overcome vaccine hesitancy [6], highlighting Confidence (importance, safety and efficacy of vaccines); Complacency (perception of low risk and low disease severity); Convenience (access issues dependent on the context, time and specific vaccine being offered); Communications (sources of information); and Context (sociodemographic characteristics). In an article entitled “What must be done to tackle vaccine hesitancy and barriers to COVID-19 vaccination in migrants?” [7] Alison Crawshaw highlighted “mistrust of the state and health system, stemming from historical events, data sharing policies and dissatisfaction with the initial handling of the pandemic” and advocated “engaging with communities to understand their concerns or barriers to vaccination and working together to co-develop tailored approaches to encourage uptake and rebuild trust”.

Geographical factors

Individual characteristics are not the only factors influencing COVID-19 vaccine uptake. An ONS technical article [46] considered vaccination status of individuals, controlling for sex, ethnicity, age, geographical region, urban or rural classification of their address, deprivation percentile, household size, whether the household was multigenerational. The model outputs [47] include regional variation, with negative impact of residence in the North East, North West, and West Midlands amongst persons aged 18–34.

Geographical factors are also known to influence the uptake of childhood vaccinations. A Michigan study by ST Hegde of Neighborhood Influence on Diphtheria-Tetanus-Pertussis Booster Vaccination [48] considered individual data on uptake along with covariates at the level of US census tract (average 4,000 people) and the much smaller units of census block, whose socio-economic characteristics were taken as proxies for the individual household. Tract level affluence and socio-economic disadvantage both showed negative impact on child vaccination uptake.

A study of childhood Measles, Mumps, Rubella (MMR) immunisation rates in Italy by Veronica Toffolutti [49], using annual data at regional level from 2000 to 2014, found that reductions in public health budgets were a significant predictor of falling MMR rates. The declines after 2010 were concentrated in those regions which experienced the greatest reductions in the real public health expenditure. Modelling found that each 1% reduction in real public health expenditure per-capita corresponded to an 0.5% reduction in MMR coverage. However, the models do not appear to control for socio-economic variation between regions.

In the geographical analysis reported here, I initially sought to learn whether the current very low booster uptake in particular small areas of England was unexpected, or could be predicted from the national data on the UK Coronavirus Dashboard, at Middle Super Output level. The observations are the grouped data for each MSOA, and as such they contain no direct information on the individuals within the area, unlike many of the studies cited above. However, the MSOA data is comprehensive, rather than a sample. Planning relies on area data, and so it is relevant to know which characteristics at area level are associated with variation in vaccination uptake, which will in turn lead to variation in rates of serious infection and demand for healthcare. Although associations at area level do not imply the same associations for individuals, they are at least suggestive of individual impacts to be examined, and can be compared with results based on individual data.

In each MSOA, the data shows the number of people eligible for the vaccine, and the cumulative number actually vaccinated with the first and second doses, and the Third Injection. The latter covers both people receiving the booster, and those with compromised immunity who were given a third primary dose. The ratio of those vaccinated to those eligible, is the uptake. But the actual numbers vaccinated and eligible give more information, and can be modelled as a binomial variable.

The Generalized Additive Model, implemented in R with the mgcv package, is an established technique for non-linear modelling. The variance in this data exceeds what would be expected for a binomial variable, but can be estimated by relaxing the “binomial” assumption to “quasibinomial”. Using demographic, employment and local health data at MSOA-level as predictors, such models fit this data well. They are improved by including a random effect assigned to the Upper Tier Local Authority within which the MSOA is found, allowing other aspects of the UTLA to influence the prediction. This is analogous to the neighborhood influence on individual uptake of the DTP booster [48], but with MSOAs rather than individuals as the unit of analysis, and grouped by UTLAs rather than census tracts.

Four different models, for each of the three vaccinations, all gave a good fit to the data with the proportion of deviance explained ranging from 95.3% to 97.2%. Fig 1 shows how well the preferred model performs for the Third Injection. This close fit means that the observed values in a particular MSOA are almost always near the prediction from the national data. Including the random effects term significantly improves the model fit.

The covariate structure was simplified to enable models to be computable on a PC. For example, whilst population data is available for each year of age, the models use broader categories such as aged 15–24 or 25–44. Even so, has 33 smoothers, each with a smoothing parameter which is estimated during fitting, along with 341 coefficients.

The smoothers for for the Third Injection, some of which are shown in Fig 2, show the impact of specific covariates in the context of all others in the model. The most powerful fixed effects are the Index of Multiple Deprivation (Fig 2B), the population proportions aged 15–24 (Fig 2C), aged 25–44 (Fig 2D), Other White ethnicity (Fig 2E), South Asian (Indian, Pakistani and Bangladeshi combined) (Fig 2F), and the combined group of African, Caribbean, Other Black, and Other ethnicity (Fig 2G).

The negative slope of the smoother in Fig 2B shows that higher IMD leads to a lower predicted uptake if all other variables are unchanged. Education Deprivation (a specific domain within the overall index IMD) also has a negative slope (Fig 2O), although its impact is smaller and is shown with “narrow scale”. All six most powerful fixed effects have smoothers with negative slope. The dependence on age is to be expected, as the vaccine rollout targeted different age groups at different times, but Aged 15–24 is particularly negative (Fig 2C).

Ethnicity has significant impact on uptake at MSOA level, as many other studies have found for individuals. The smoother for the “Other White” group (Fig 2E) has the widest range, comparable to that for IMD (Fig 2B), showing that the “Other White” proportion of the population has a strong impact on predicted uptake.

Several other smoothers have small positive slope, such as those for the proportion employed in Public Adminstration (Fig 2I), or Health and Social Care (Fig 2J). The latter indicates that despite the well publicised vaccine hesitancy amongst a minority of NHS and care staff [50], and the international meta-analysis [9], the overall impact of employment in these sectors in England is to increase the area uptake of the Third Injection, as in the ONS survey data [43]. For the first and second doses, the smoother rises when the proportion of Health and Care staff exceeds the 20th percentile of this covariate.

The smoother for average distance to A&E (Fig 2L) also has a positive slope, suggesting that people living further from an emergency department may be more concerned to take the vaccination. Likewise, increasing Health Deprivation (Fig 2N) is associated with increasing uptake. Conversely, increased GP registration (Fig 2M) is associated with decreasing uptake. However, there is some uncertainty in the GP registration data, and NHS Digital have commented on the excess of GP registrations over ONS population estimates [51].

The smoother for male (Fig 2H) shows uptake of the Third Injection declining with increasing male proportion of the population. This differs from the international meta-analyses [8, 9], and from the ONS technical report [47]. However, the meta-analyses concern data from 2020, and the ONS report does not show a significant negative uptake for females during time periods after 5 September 2021, which cover the Third Injection. Model includes many more covariates including employment sectors, which may affect the smoother for male. In any case, whilst the smoother is highly significant (p<2e-16), its range is narrow as shown in Fig 2H.

Fitting any of these models also gives estimates for the estimated “random effect” of each of the 149 UTLA. For example “Knowsley” has a strong negative impact, and “Slough” has a strong positive impact (as shown for model in Fig 2A). These random effects are not to be confused with the actual uptake of vaccination in the local authority, which may be low or high due to deprivation or other fixed effects, and which may vary widely amongst MSOAs within the local authority. Uptake of the Third Injection varied from 32.2% to 57.6% within Knowsley, from 17.3% to 65.5% in Liverpool, and from 36.9% to 75.8% in Sefton.

The random effect alters the prediction from the fixed effects alone. If Beckton Park (in Newham) had been located in Slough with identical demography, employment, and health indicators, the predicted uptake of the Third Injection would rise by 19%, whilst if it were in Knowsley the prediction would fall by 28%.

The range in magnitude of the random effects is comparable to that of ethnicity groups. On the scale of the linear predictor, the smoothers for Other White, South Asian, and African-Caribbean-Other groups range from 0.09 to -0.70, 0.05 to -0.42, and 0.07 to -0.51 respectively, each smoother descending as the ethnicity proportion rises. The random effects range from -0.15 (Knowsley) to 0.51 (Slough). The difference between the random effects for Knowsley and Slough is greater than the maximal difference between any two MSOA due to the population proportion of South Asian or African-Caribbean-Other ethnicity, and only slightly less than the maximal difference due to the proportion of Other White.

Fig 3 shows that many UTLA have estimated random effects whose 95% credible interval does not include the mean value. Thus, as well as the entire random effects term being highly significant, particular UTLAs can be distinguished from each other and from the mean value.

As Fig 4 shows, there is a strong Regional component to the variation in random effects, with the North West containing many low values. The second stage of modelling sought to explain the estimated random effects in terms of other information at UTLA level. This stage tested the impact of flu vaccination rates, change in public health budgets, number of vaccination sites (per UTLA population) of four types, and Region, treated as a random factor. Simulated random effects generated from the first stage model were used to refine the parameter estimates and credible intervals from the second stage, but this made little difference. As Table 2 shows, the parameters and credible intervals for second stage modelling of random effects estimated from model are similar to those after the random effects are simulated, using the corrected covariance matrix from . This method is illustrated with synthetic data in the code file simutest1 (see S2 File), described in the Guide (S1 File).

The second stage model passes Bayesian checks and shows clear impacts of Region, flu vaccination rates, and change in public health budgets. Fig 5 shows the extent to which the chosen predictors actually account for the estimated random effects. None of the points are outliers (by Pareto k) and the model fits well for the first and second doses with Bayes R2 ∼ 0.6. The lower value for the Third Injection, ∼ 0.37, suggests that there may be other relevant covariates at UTLA level.

It is striking that Knowsley, Liverpool and Sefton all appear at the lower left of each panel in Fig 5 and Knowsley is conspicuously low for all three vaccinations. The Merseyside local authorities are amongst the most deprived in England, but IMD already appears in the first stage model as a fixed effect so was not expected to have any impact on the second stage model. Indeed, if IMD is averaged over the MSOA within each UTLA and then used as a predictor in second stage modelling, its 90% credible interval includes 0.

Simply being in the North West is the most powerful predictor of low uptake in this model, and in addition Knowsley had the 13th lowest increase in public health funding (and the third lowest within the North West), rising by only 0.88%. Newham benefits from being in London and from the larger increase in its public health budget (rising by 2.09%), and possibly from the vaccination centre located in Olympic Park, an outlier with positive residual (see Fig 1). Flu vaccination rates are higher in Knowsley (49.2) than in Newham (45.5), so cannot explain the disparity in random effects.

Slough benefits from being in the South East and possibly from a vaccination centre, whilst its flu vaccination rate and rise in public health budget are both close to their respective mean values. However the random effects for Slough also exceed prediction. The highest predicted value is in Lambeth, where the flu vaccination rate is below average, but which benefits from being in London with a 4.88% rise in public health budget and 3 vaccination centres. Its estimated random effect is close to prediction for each vaccination.

All such conclusions depend on the validity of the models. The first stage MSOA level modelling fits very well, and the resulting estimated random effects are similar from all four models. That is, the random effects are not simply artefacts of the model, once ethnicity is included, although they change dramatically if it is omitted. Correlations between estimated random effects when fitting the same model to different vaccinations show that UTLAs which performed better (worse) for the first two doses, generally did so for the Third Injection.

This analysis concerns data as of 4 January 2022, and cannot show how the performance of UTLAs changed over time. There may be other relevant covariates at UTLA or MSOA level not included here. For example, the Understanding Society: COVID-19 Study, 2020–2021 [52] may contain relevant covariates which can be linked to geographic information [53]. Social media use could indicate the local influence of disinformation. Data on ethnicity and occupation was taken from the 2011 Census and may be outdated, though it was the most recent available. Ideally all of the UTLA and MSOA level variables would be incorporated in a single Bayesian model. But that appeared prohibitively slow to compute, and the relevant programme gamm4 used with rstanarm does not admit quasi families, so the random effects estimates from the first stage GAM modelling were considered as observations, to be modelled in their own right and then refined by simulation. The first stage modelling used the quasibinomial family, and other possibilities were not investigated. A random intercept for UTLA was the simplest possible random effect structure, but UTLA may also interact with some of the fixed covariates. Likewise, all of the fixed effects were given independent smoothers, without interactions. However, the chosen model did give a good fit to the data.

Conclusion

The UTLA level random effects could be described as a postcode lottery, as they are not explained by the population characteristics controlled for by the fixed effects, but are associated with other geographical factors. However, a “lottery” suggests pure chance whereas Regional disparity and changes in public health budgets impact on COVID-19 vaccination rates. In turn, these factors reflect economic policy decisions.

Much of the literature has focused on “vaccine hesitancy” of specific population subgroups at individual level. The modelling here confirms the impact on area uptake, of ethnicity along with deprivation and age, but also identifies additional factors at MSOA level and others characteristic of the wider locality rather than the population living in it, as with evidence on childhood immunisations in Italy.

These models indicate that whatever other barriers exist due to deprivation and within particular ethnicities, the annual change in local authority public health budgets is also a positive predictor. Therefore, increasing local public health allocations would be one simple way to improve COVID-19 vaccine uptake.

Guide.

Guide to the files contained within the zip files for codes, data, and outputs, how to run the codes, and short simulations to illustrate second stage modelling.

(PDF)

Click here for additional data file.

Codes.

A zip file containing the codes used to model the data.

(ZIP)

Click here for additional data file.

Data1.

A zip file containing most of the data files.

(ZIP)

Click here for additional data file.

Data2.

A zip file containing large data files for GP registration and IMD.

(ZIP)

Click here for additional data file.

Data3.

A zip file containing large data files for MSOA vaccinations and AHAH Output.

(ZIP)

Click here for additional data file.

Outputs.

A zip file containing the outputs of modelling. This can be used to load the fitted models.

(ZIP)

Click here for additional data file.

Summary.

Summary of model applied to the Third Injection.

(PDF)

Click here for additional data file.

Appendix.

Brief outline of how GAM models are fitted, smoothing parameter optimisation by “Marginal Likelihood”, and the standard errors of fitted coefficients.

(PDF)

Click here for additional data file.

3 May 2022

Variation in English Covid booster uptake: Generalized Additive Model

PLOS ONE

Dear Dr. Dropkin,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 15 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Harapan Harapan, MD, PhD

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.Please amend either the abstract on the online submission form (via Edit Submission) or the abstract in the manuscript so that they are identical.

3. Please include a separate caption for each figure in your manuscript.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

5. We note that [Figure 4] in your submission contain [map/satellite] images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a. You may seek permission from the original copyright holder of Figure(s) [#] to publish the content specifically under the CC BY 4.0 license.

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an ""Other"" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/.

6. Please update your submission to use the PLOS LaTeX template. The template and more information on our requirements for LaTeX submissions can be found at http://journals.plos.org/plosone/s/latex.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: in the abstract results, what do you mean by 90% significance? Note - I think you use Bayesian techniques and therefore refer to 90% credibility intervals throughout. But I would stay away from making inference about "significance" within a Bayesian framework.

some sort of figure or flowchart might be useful to explain or diagram what middle super output areas versus upper tier local authorities are. An explanation within the text of what these mean (in terms of average population and area) also could be useful.

The other uses publicly available data about area-level variables on area-level vaccination. But some of the inferences are for an individual level. Distinguishing whether the author thinks of these as a proxxy for an individual level or if he is just studying area-level associations as an ecological association.

On that note, I personally would appreciate some connection to the existing literature on multilevel or neighborhood effects (examples similar but not limited to:

Diez-Roux A. Multilevel analysis in public health research. Annu Rev Public Health. 2000;21: 171–192.

Hegde ST, Wagner AL, Clarke PJ, Potter RC, Swanson RG, Boulton ML. Neighborhood Influence on Diphtheria-Tetanus-Pertussis Booster Vaccination. Public Health. 2019;167: 41–49. https://doi.org/10.1016/j.puhe.2018.11.009

I think the GAM is not the important part that I would emphasize in the title - maybe something like: COVID booster uptake in England: an ecological study

When did booster / 3rd dose vaccination start in England? Would be useful to include this in Intro or Methods.

I do appreciate the attention to detail in the methods. I think as is would be fine, but my recommendation would be to put much of what you have in the appendix along with your guide, and instead explain it in more vague terms which are not specific to the coding in R. Again - this coding is incredibly useful for people to replicate the results, but to me that would be more appropriate for the supplementary appendix.

I would stay away from using causal language like "effects" and instead use terms like association, or maybe even impact, since you have an observational study.

Subheadings in the discussion could be useful. It seems like you have one section related to relating the results to previous literature, one section related to explaining geographic variation with less comments - that could be helpful as an initial structure. As is it is hard to follow your thought process across paragraphs.

Reviewer #2: The author performed Generalized Additive Model analyses on the Covid vaccination data from the UK Government Coronavirus Dashboard. In general, the writing requires heavy improvement, where many technical errors (such as logical flows of the paragraph, captions of charts and tables, data presentation) could be found here and there. I have some recommendations to guide author in revising their manuscript.

SUPPLEMENTARY FILE

1. I could not find/access the supplementary or supporting files author stated in the manuscript. Probably making a special section about Data Availability at the end of the manuscript would help.

ABSTRACT

2. “Are there other relevant predictors?” So, what are other predictors? Please state in the last part of the abstract.

3. What is MSOA? All abbreviation should be defined first.

4. The “discussion” part fits better with “conclusions”

INTRODUCTION

5. “Vaccination uptake is highly dependent on age, gender, ethnicity, and deprivation, as is widely recognised 3 4 5 6 7 8 9” Are all the studies cited from UK? Or, are they from different countries? Readers may assume they are all from UK, I suggest author to breakdown each of the cited literatures.

6. “…in contrast with 41.1% in Liverpool and 67.4% in

Gloucestershire.” Where does this information come from? Please cite the literature.

7. Please define abbreviations: MSOA, UTLA, etc.

8. In the last paragraph, author is encouraged to elaborate as to why this approach is appropriate. Is it more robust in comparison with other approaches?

METHODS

9. “Data sources” and “Statistical method and models” should be under one section “Methods”

RESULTS

10. Table 1 has no border, and should be revised. Not sure what this value mean -15317.25 (0.953), I understand that author put it in the caption, but that is not enough. Furthermore, why author does not make it uniform, e.g. Dose 1, Dose 2, Dose 3 or 1st injection, 2nd injection, 3rd injection?

11. Before Table 1, the paragraph only consists of 1 sentence. I encourage the author to move more sentences into the paragraph. And this applies for every data presented (including the following charts or tables).

12. “….except for distance to pharmacy (p<0.01).” Where can I see the data? Supplementary file? Author should cite it in the text.

DISCUSSION

13. “…COVID-19 vaccine hesitant followed by the Pakistani/Bangladeshi group.” Provide citation please.

14. In the second paragraph of Discussion, consider to include these following studies.

# In a study performing willingness-to-pay analysis, flu vaccination is also among the positive predictors for COVID-19 vaccine acceptance. Citing: Sallam et al. Narra J 2022; 2(1): e74 – doi: 10.52225/narra.v2i1.74

# Side effects of COVID-19 vaccine become the major barrier for vaccination program in population from different countries. Citing: Rosiello et al. Narra J 2021; 1(3): e55- doi: 10.52225/narra.v1i3.55.

#Disinformation and certain religious narrative could discourage individuals to take COVID-19 vaccine. Citing: Hassan et al. Narra J 2021; 1(3): e 57 - doi: 10.52225/narra.v1i3.57

15. I don’t understand what is the author trying to convey in the fourth paragph of the Discussion.

16. This paragraph, “An ONS technical article…” Does author try to compare the method with previous study? I think this paragraph better fit in the introduction, rather than in discussion. Also, clarify the intention of putting this paragraph, because it seems author just arbitrarily put random paragraph in the Discussion.

17. Discussion needs a lot of re-arrangements to make it more readable and acceptable for publication. Other than re-arranging the paragraphs to optimize the logical flow, author is encourage to distinct the findings from previous studies and that of from present study, and followed by a clear comparison. At the present version, it’s very hard to get the comparative pictures of the studies.

18. Please provide limitation of the study at the end of discussion.

19. Please provide a section of “Conclusions”

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

4 Jun 2022

Thanks for all these comments, which I have tried to deal with in the amended version.

Journal requirements

1. Response: I have tried to comply with the style requirements including file names.

2. Response: I have amended the Abstract on the online submission to agree with the revised manuscript.

3. Response: Each figure and table now has a caption.

4. Response: The Supporting Information files have captions and are cited accordingly.

5. Response: Figure 4 is not a reproduction, but is an original map whose shading is derived from the modelling in this paper. To my knowledge it has not been published elsewhere, and I am unaware of any equivalent published modelling from which this map could be derived. The upper tier local authority boundaries which enclose the areas separately shaded in the map, are freely available for download from the UK Office for National Statistics, and I have amended the text and endnotes to comply with the conditions for their use, which involves no other copyright.

6. Response: I have used LaTex for the revision.

Reviewers responses to Questions

Reviewer #1: in the abstract results, what do you mean by 90% significance? Note - I think you use Bayesian techniques and therefore refer to 90% credibility intervals throughout. But I would stay away from making inference about "significance" within a Bayesian framework.

Response: Thanks, almost everything now refers to credible intervals, although those in the first stage models (and in Figure 2) are 95% credible. The first stage modelling is not Bayesian, but the intervals are now called “credible” because they are derived from standard errors obtained from the posterior covariance as now outlined in S1_Appendix. The second stage modelling is Bayesian and I report 90% credible intervals. However, the p-values computed by “mgcv” in the first stage modelling are approximate tests of significance, as they represent the approximate probability that a smooth term is actually 0. I do think it is appropriate to highlight those smooth terms which are extremely significant (p<2e-16) according to this test. I have cited the reference below Wood SN (2013b) as well as Wood’s book.

From the mgcv help line:

P-values for smooth terms are usually based on a test statistic motivated by an extension of Nychka's (1988) analysis of the frequentist properties of Bayesian confidence intervals for smooths (Marra and Wood, 2012). These have better frequentist performance (in terms of power and distribution under the null) than the alternative strictly frequentist approximation. When the Bayesian intervals have good across the function properties then the p-values have close to the correct null distribution and reasonable power (but there are no optimality results for the power). Full details are in Wood (2013b), although what is computed is actually a slight variant in which the components of the test statistic are weighted by the iterative fitting weights.

Nychka (1988) Bayesian Confidence Intervals for Smoothing Splines. Journal of the American Statistical Association 83:1134-1143.

Marra, G and S.N. Wood (2012) Coverage Properties of Confidence Intervals for Generalized Additive Model Components. Scandinavian Journal of Statistics, 39(1), 53-74.

Wood, S.N. (2013b) On p-values for smooth components of an extended generalized additive model. Biometrika 100:221-228

Reviewer #1: some sort of figure or flowchart might be useful to explain or diagram what middle super output areas versus upper tier local authorities are. An explanation within the text of what these mean (in terms of average population and area) also could be useful.

Response: I hope that the text is now clear, without a figure, but including the average MSOA population, the total number of MSOA, the fact that each MSOA belongs to a unique UTLA, the total numbers of MSOA and UTLA and the range of numbers of MSOA contained in a UTLA.

Reviewer #1: The other uses publicly available data about area-level variables on area-level vaccination. But some of the inferences are for an individual level. Distinguishing whether the author thinks of these as a proxxy for an individual level or if he is just studying area-level associations as an ecological association.

Response: I am just studying area-level associations, but I point out in the discussion that several of these area-level associations are similar to individual survey results from other papers. None of the inferences in this paper are for an individual level and I am not regarding MSOA-level covariates as a proxy for individual values.

Reviewer #1: On that note, I personally would appreciate some connection to the existing literature on multilevel or neighborhood effects (examples similar but not limited to:

Diez-Roux A. Multilevel analysis in public health research. Annu Rev Public Health. 2000;21: 171–192.

Hegde ST, Wagner AL, Clarke PJ, Potter RC, Swanson RG, Boulton ML. Neighborhood Influence on Diphtheria-Tetanus-Pertussis Booster Vaccination. Public Health. 2019;167: 41–49. https://doi.org/10.1016/j.puhe.2018.11.009

Response: Thanks, and I have mentioned them both and clarified that the random effects in this paper are “random intercept”, the simplest possible type, although they are embedded in GAMs which are more complex than the linear models in the Diez-Roux review. I have also cited another review of linear and generalized linear mixed models and a book which includes non-linear mixed models, though not using GAMs. The Hegde paper does not display the model explicitly, so I am unclear if it is linear or nonlinear in the covariates at block level which are taken as proxies for individual level, but I cite it in the discussion as it shows area (tract level) impacts on vaccination uptake.

Reviewer #1: I think the GAM is not the important part that I would emphasize in the title - maybe something like: COVID booster uptake in England: an ecological study

Response: Agree, thanks.

Reviewer #1: When did booster / 3rd dose vaccination start in England? Would be useful to include this in Intro or Methods.

Response: Thanks, now included in the Intro.

Reviewer #1: I do appreciate the attention to detail in the methods. I think as is would be fine, but my recommendation would be to put much of what you have in the appendix along with your guide, and instead explain it in more vague terms which are not specific to the coding in R. Again - this coding is incredibly useful for people to replicate the results, but to me that would be more appropriate for the supplementary appendix.

Response: Thanks, I’ve deleted any mention of R codes from the “Statistical methods” section, which I hope is now the minimum needed to see what the first and second stage models are doing. There is now a short S1_Appendix.pdf which also omits R codes but gives a brief mathematical overview. It also explains some fitted model outputs including “ML” and “Proportion of Deviance Explained” which are used in Table 1, and the standard errors used in making Figures 2 and 3. S1_File.pdf is a Guide which points to the code files.

Reviewer #1: I would stay away from using causal language like "effects" and instead use terms like association, or maybe even impact, since you have an observational study.

Response: Thanks, I am now only using “effects” in the phrases “random effects” and “fixed effects” which I think are the correct terms in these models. Elsewhere, I’ve generally used impact rather than association.

Reviewer #1: Subheadings in the discussion could be useful. It seems like you have one section related to relating the results to previous literature, one section related to explaining geographic variation with less comments - that could be helpful as an initial structure. As is it is hard to follow your thought process across paragraphs.

Response: Thanks, I hope it is clearer now with the rewrites, some reordering, and a subhead (“Geographical factors”) in the discussion.

Reviewer #2: The author performed Generalized Additive Model analyses on the Covid vaccination data from the UK Government Coronavirus Dashboard. In general, the writing requires heavy improvement, where many technical errors (such as logical flows of the paragraph, captions of charts and tables, data presentation) could be found here and there. I have some recommendations to guide author in revising their manuscript.

Response: Thanks, I hope the writing is clearer now.

Reviewer #2: SUPPLEMENTARY FILE

1. I could not find/access the supplementary or supporting files author stated in the manuscript. Probably making a special section about Data Availability at the end of the manuscript would help.

Response: The Supporting information section at the end of the revised manuscript now lists 7 files and an appendix. S1_File.pdf is a Guide with details of all the files, including codes, data, and output. It also explains how to run the code files, and outlines the simulations which illustrate the modelling technique using synthetic data. There also a summary of model D applied to the Third Injection, and a separate Appendix which outlines the method of fitting the models.

Reviewer #2: ABSTRACT

2. “Are there other relevant predictors?” So, what are other predictors? Please state in the last part of the abstract.

Response: Thanks, some of the relevant predictors for first stage modelling and all of the relevant predictors for second stage modelling are mentioned in the Results and Conclusion.

3. What is MSOA? All abbreviation should be defined first.

Response: Thanks, now defined at first mention in the Abstract.

4. The “discussion” part fits better with “conclusions”

Response: Thanks, now renamed as Conclusion.

Reviewer #2: INTRODUCTION

5. “Vaccination uptake is highly dependent on age, gender, ethnicity, and

deprivation, as is widely recognised 3 4 5 6 7 8 9” Are all the studies cited

from UK? Or, are they from different countries? Readers may assume they are all from

UK, I suggest author to breakdown each of the cited literatures.

Response: Thanks, I have separated these into UK and international studies.

6. “…in contrast with 41.1% in Liverpool and 67.4% in

Gloucestershire.” Where does this information come from? Please cite the

literature.

Response: Thanks, I have now cited the UK Government Coronavirus Dashboard as the source for the data from which these figures are derived.

7. Please define abbreviations: MSOA, UTLA, etc.

Response: Thanks, both are spelled out at first use in this section, with more detail to explain their meaning.

8. In the last paragraph, author is encouraged to elaborate as to why this approach

is appropriate. Is it more robust in comparison with other approaches?

Response: Thanks, I am not including this in the Introduction, but the Statistical methods section now refers to reviews of mixed models, and points out that it is overly restrictive to assume linearity. A comparison of GAM with other non-linear modelling techniques is beyond the scope of this paper, but as mentioned in the Discussion, “The Generalized Additive Model, implemented in R with the mgcv package, is an established technique for non-linear modelling.” For model D applied to the Third Injection, the Results show that the mixed model is superior (by anova, or the ML criterion or Proportion of Deviance Explained) to models using only the fixed effects, or only the random effect.

Reviewer #2: METHODS

9. “Data sources” and “Statistical method and models” should

be under one section “Methods”

Response: Thanks, the section is now called “Materials and methods”, and contains subsections “Data sources” and “Statistical methods”. I think it is helpful to refer to these subsections separately.

Reviewer #2: RESULTS

10. Table 1 has no border, and should be revised. Not sure what this value mean -15317.25 (0.953), I understand that author put it in the caption, but that is not enough. Furthermore, why author does not make it uniform, e.g. Dose 1, Dose 2, Dose 3 or 1st injection, 2nd injection, 3rd injection?

Response: Tables are now formatted with borders, and include a Title and a Caption. Table 1 displays the ML value and the Proportion of Deviance Explained for each of the fitted models. These outputs are now explained in the supporting information S1_Appendix. The “dose” and “Third Injection” terminology follows that of the downloads from the UK Government Coronavirus Dashboard, and is explained in the “Data sources” subsection. Dose 1 and Dose 2 are the first and second primary vaccinations, but the Third Injection data comprises persons given either the Booster, or a 3rd primary given to clinically vulnerable persons.

11. Before Table 1, the paragraph only consists of 1 sentence. I encourage the author to move more sentences into the paragraph. And this applies for every data presented (including the following charts or tables).

Response: 11. Thanks, I have rewritten with longer paragraphs, in general.

12. “….except for distance to pharmacy (p<0.01).” Where can I see the data? Supplementary file? Author should cite it in the text.

Response: This sentence describes an aspect of the fitted output of model D applied to the Third Injection, using all the covariates. A summary of the output is now shown in S7_File.pdf

Reviewer #2: DISCUSSION

13. “…COVID-19 vaccine hesitant followed by the Pakistani/Bangladeshi group.” Provide citation please.

Response: This is a quote from the SAGE report in December 2020, as cited. I have moved the citation to just after the quote.

14. In the second paragraph of Discussion, consider to include these following studies.

# In a study performing willingness-to-pay analysis, flu vaccination is also among the positive predictors for COVID-19 vaccine acceptance. Citing: Sallam et al. Narra J 2022; 2(1): e74 – doi: 10.52225/narra.v2i1.74

# Side effects of COVID-19 vaccine become the major barrier for vaccination program in population from different countries. Citing: Rosiello et al. Narra J 2021; 1(3): e55- doi: 10.52225/narra.v1i3.55.

#Disinformation and certain religious narrative could discourage individuals to take COVID-19 vaccine. Citing: Hassan et al. Narra J 2021; 1(3): e 57 - doi: 10.52225/narra.v1i3.57

Response: Thanks, the Discussion now mentions two of the suggested studies.

15. I don’t understand what is the author trying to convey in the fourth paragph of the Discussion.

Response: This is now paragraph 6 of the Discussion, and concerns the fact that occupation also influences vaccine uptake. It cites data from the Office for National Statistics on how the proportion of the population receiving 3 vaccinations varied according to their employment, and also, separately, according to their ethnicity. For example, 80.4% of people employed in the Health Sector had received 3 vaccinations, as of 31 December 2021. The citations now go to the relevant earlier releases from the ONS.

16. This paragraph, “An ONS technical article…” Does author try to compare the method with previous study? I think this paragraph better fit in the introduction, rather than in discussion. Also, clarify the intention of putting this paragraph, because it seems author just arbitrarily put random paragraph in the Discussion.

Response: Thanks, I’ve amended this and placed it as the first paragraph in the subsection “Geographical factors”. It is included because although it was based on individual survey data, it also showed a significant regional effect.

17. Discussion needs a lot of re-arrangements to make it more readable and acceptable for publication. Other than re-arranging the paragraphs to optimize the logical flow, author is encourage to distinct the findings from previous studies and that of from present study, and followed by a clear comparison. At the present version, it’s very hard to get the comparative pictures of the studies.

Response: Thanks, I’ve re-ordered the presentation of previous research so that the discussion moves from surveys of individuals, to studies which show the impact of vaccination status on mortality and make recommendations about improving uptake, to studies which include geographical factors, including studies of childhood immunisation, one of which features the impact of change in public health budgets. The discussion then explains that my article concerns area data rather than individual data, and then comments on the results, some of which are compared with the previous research cited.

18. Please provide limitation of the study at the end of discussion.

Response: Thanks, I’ve included a paragraph on this at the end of Discussion.

19. Please provide a section of “Conclusions”

Response: Thanks, the final 3 paragraphs are now headed Conclusions.

Submitted filename: Response_to_reviewers_1.docx

Click here for additional data file.

14 Jun 2022

Variation in COVID-19 booster uptake in England: an ecological study

PONE-D-22-05078R1

Dear Dr. Dropkin,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Harapan Harapan, MD, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The author adequately responded to my previous comments.

Reviewer #2: The author has responded to all my concerns. The revised version is satisfactory and can be accepted for publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

20 Jun 2022

PONE-D-22-05078R1

Variation in COVID-19 booster uptake in England: an ecological study

Dear Dr. Dropkin:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Harapan Harapan

Academic Editor

PLOS ONE