Evaluation of the role of vaccination in the COVID-19 pandemic based on the data from the 50 U.S. States.

Vaccination is considered as the ultimate weapon to end the pandemic. However, the role of vaccines in the pandemic remains controversial. To explore the impact of vaccination on the COVID-19 pandemic, we used logistic regression models to predict numbers of population-adjusted confirmed cases, deaths, intensive care unit (ICU) cases, case fatality rates and ICU admission rates of COVID-19 in the 50 U.S. states, based on 17 related variables. The logistic regression analysis showed that percentages of people vaccinated correlated inversely with the numbers of COVID-19 deaths and case fatality rates but showed no significant correlation with numbers of confirmed cases or ICU cases, or ICU admission rates. The Spearman correlation analysis showed that the percentages of people vaccinated correlated inversely with the numbers of COVID-19 deaths, ICU cases, ICU case rates, and case fatality rates but showed no significant correlation with numbers of confirmed cases. The number of deaths and mortality in the group after the vaccine usage were significantly lower than those in the group before the vaccine usage. However, after delta became the dominant strain, there were no longer significant differences in the number of deaths and the mortality rate between before and after delta became the dominant strain, although vaccines were used in both periods. Vaccination can significantly reduce COVID-19 deaths and mortality, while it cannot reduce the risk of COVID-19 infection. In addition to vaccination, other measures, such as social distancing, remain important in containing COVID-19 transmission and lower the risk of COVID-19 severe outcomes.

Introduction

The pandemic of COVID-19 has lasted for more than two and a half years since December 2019. To date, more than 585 million COVID-19 cases and 6.4 million deaths have been reported [1]. More seriously, the spread of this disease is still ongoing with the emergence of SARS-CoV-2 variants [2]. Particularly, because the Delta variant has stronger transmissibility, can result in more severe symptoms and higher mortality, as well as exhibits a high ability to cause breakthrough infection [3], [4], [5], [6], [7], it had become the main virus variant in most countries [8]. Due to the lack of effective drugs against SARS-Cov-2 infection, vaccination is hopeful to end the pandemic as the ultimate weapon. However, currently the role of vaccines in the pandemic remains controversial. Therefore, it is crucial to evaluate the impact of vaccination on the COVID-19 pandemic based on updated data.

Prior studies have revealed certain factors that have significant associations with the spread of COVID-19, such as temperature [9], humidity [10], and social distancing [11], [12]. Some other factors, such as economic inequality, per capita hospital beds, and blood types, also show associations with COVID-19 transmission and fatality [13]. Nevertheless, most previous epidemiological studies did not take the vaccination factor into account, and most vaccines-related studies were involved in clinical trials. For example, Pilishvili et al. assessed the effect of COVID-19 vaccines from BioNTech and Moderna against the transmission of COVID-19 among healthcare workers based on data from 33 sites in the United States from January to March 2021 [14]. Voysey et al. measured the safety and efficacy of a COVID-19 vaccine, namely ChAdOx1 nCoV-19, using the data from four ongoing clinical trials in three countries [15]. Syed et al. showed that the effectiveness of the mRNA vaccine decreased steadily over time, with BNT162b2 decreasing from 67.6 % to 12.5 % between months 2 and 10, and the effectiveness of mRNA-1273 decreasing from 84.5 % to 48.1 % [16]. Nordström et al. discovered that the effectiveness of a second dose of COVID-19 vaccine diminished over time [17]. Mousa et al. showed that the inactivated vaccine BBIBP-CorV and the mRNA vaccine BNT162b2 have the close effectiveness against the B.1.617.2 Delta variant [18]. Murillo-Zamora et al. showed that BNT162b2 could protect against severe COVID-19 infections [19]. Nevertheless, because most of these studies were limited to a small population or a short time span, the role of vaccination in containing the COVID-19 pandemic remains unclear.

To explore the impact of vaccination on the COVID-19 pandemic among large population in a long period of time, we used logistic regression models to explore the role of multiple variables, including vaccines, in the COVID-19 pandemic based on the data from the 50 U.S. states. We collected data related to politics, economy, mobility, weather, population, medical resources, smoking, age structure and vaccination. These data included most of the case data released by the Centers for Disease Control and Prevention (CDC) in the U.S. from January 2020 to August 2021. Based on these data, we constructed a logistic model to predict numbers of population-adjusted confirmed cases (per one million), deaths (per one million), intensive care unit (ICU) cases (per one million), as well as case fatality rates and ICU admission rates of COVID-19. This study aimed to evaluate the role of vaccines in the COVID pandemic. The major contributions of this study included: first, we measured the impact of various factors, including vaccination, on the pandemic over a long-term horizon and a large population size; second, we assessed associations of various variables with COVID-19 prevalence and severity; finally, we explored the impact of delta strains on the effectiveness of vaccination.

Methods

Datasets

This analysis defined 17 variables that have potential impact on the transmission, severity or fatality of COVID-19. These variables included population, mobility data from google (residential percent change from baseline, parks percent change from baseline, transit stations percent change from baseline, grocery and pharmacy percent change from baseline, retail and recreation percent change from baseline, and workplaces percent change from baseline), GDP per capita, smoking population, hospital beds (per 1000 population), government stringency index, temperature, dew point temperature, people vaccinated per hundred, population aged from 0 to 18 (percent), population aged from 19 to 64 (percent), and population aged over 65 (percent). The latest versions of these data were downloaded from main repositories, as shown in Table 1. We collected the data of COVID-19 cases from the U.S. CDC, termed COVID-19 Case Surveillance Public Use Data with Geography (https://data.cdc.gov/Case-Surveillance/COVID-19-Case-Surveillance-Public-Use-Data-with-Ge/n8mc-b4w4).

Table 1

A description of the variables related to COVID-19 transmission and fatality.

Variable	Description	Period	Type of variables	Data point	Source	URL
Aged from 0 to 18 (percent)	Percent of the population aged 0 to 18	2019	continuous	annual	Kaiser Family Foundation	https://www.kff.org/other/state-indicator/distribution-by-age
Aged from 19 to 64(percent)	Percent of the population aged 19 to 64	2019	continuous	annual	Kaiser Family Foundation	https://www.kff.org/other/state-indicator/distribution-by-age
Aged over 65(percent)	Percent of the population aged over 65	2019	continuous	annual	Kaiser Family Foundation	https://www.kff.org/other/state-indicator/distribution-by-age
Residential	residential percent change from baseline	2020–2021	continuous	monthly average	Google	https://www.google.com/covid19/mobility/
Park	park percent change from baseline	2020–2021	continuous	monthly average	Google	https://www.google.com/covid19/mobility/
Retail and recreation	retail and recreation percent change from baseline	2020–2021	continuous	monthly average	Google	https://www.google.com/covid19/mobility/
Transit station	transit station percent change from baseline	2020–2021	continuous	monthly average	Google	https://www.google.com/covid19/mobility/
Grocery and pharmacy	grocery and pharmacy percent change from baseline	2020–2021	continuous	monthly average	Google	https://www.google.com/covid19/mobility/
Workplace	workplace percent change from baseline	2020–2021	continuous	monthly average	Google	https://www.google.com/covid19/mobility/
Population	population of each states	2021	continuous	annual	World Population Review	https://worldpopulationreview.com/states
GDP	GDP per capita (2021)	2021	continuous	annual	Wikipedia	https://en.wikipedia.org/wiki/List_of_states_and_territories_of_the_United_States_by_GDP
People vaccinated	percent of vaccinated population	2021	continuous	monthly average	Our World in Data	https://ourworldindata.org/https://github.com/owid/covid-19-data
Temperature	temperature	2020–2021	continuous	monthly average	National Centers for Environmental Information	https://www.ncei.noaa.gov/maps/daily/https://www.ncei.noaa.gov/data/global-summary-of-the-day/archive/
Stringency index	government stringency index	2020–2021	continuous	monthly average	Blavatnik School of Government	https://www.bsg.ox.ac.uk/research/research-projects/covid-19-government-response-tracker
Smoking	percent of smoking people	2019	continuous	annual	CDC	https://chronicdata.cdc.gov/Survey-Data/Behavioral-Risk-Factor-Data-Tobacco-Use-2011-to-pr/wsas-xwh5
Dew point	dew point temperature	2020–2021	continuous	monthly average	National Centers for Environmental Information	https://www.ncei.noaa.gov/maps/daily/https://www.ncei.noaa.gov/data/global-summary-of-the-day/archive/
Hospital beds	number of hospital beds (per 1000 population)	2018	continuous	annual	Hospital review	https://www.beckershospitalreview.com/rankings-and-ratings/states-ranked-by-hospital-beds-per-1–000-population.html

Data preprocessing

Because the vaccination-associated data collected were the number of population vaccinated, and vaccination often takes effect after half a month, we used monthly data as data points in the model to reduce the impact of the time lag between injection of vaccines and vaccine working. We deleted all missing and “NA” or “unknown” values for death and ICU states in the CDC dataset and calculated the sum of their monthly values for each U.S. state. We calculated the mean values of mobility variables, government stringency index, and vaccination rates monthly, respectively. We selected the weather data from the 20 weather stations closest to the coordinates of each state to represent the weather data of each state and then calculated their monthly average temperature and dew point temperature, respectively. We calculated the total numbers of confirmed cases (per one million), deaths (per one million), ICU cases (per one million) and case fatality rate and ICU admission rate monthly.

Logistic regression model

We defined the states as having low or high number of confirmed cases (per one million), deaths (per one million), and ICU cases (per one million), case fatality rate or ICU admission rate using their median values as the cutoff. The dataset was randomly separated into a training set (70 percent of samples) and a test set (the remaining 30 percent of the samples). We first built a logistic regression model based on the training set and applied it to predict the test set. We reported the prediction accuracy in the training set (10-fold cross validation) and in the test set, respectively. Furthermore, we estimated the sensitivity and specificity of the prediction model using the area under the receiver operating characteristic curve (AUC) (Fig. 1).

Fig. 1

Schematic illustration of the model.

Statistical analysis

We grouped the data according to different time points, namely the data of 12 months before vaccination (from January 2020 to December 2020), the data of 5 months before and after vaccination (from August 2020 to May 2021), and the data of 3 months before and after the Delta variant became the main variant (from March 2021 to August 2021) [20]. We compared two groups of data using the Mann–Whitney U test. We evaluated the correlations between numbers of confirmed cases (per one million), deaths (per one million), ICU cases (per one million), and case fatality rates and ICU admission rates with vaccination rates (people vaccinated per hundred) using the Spearman correlation. The correlation coefficient (R) and P-values were reported.

Results

The effect of different factors on the spread and fatality of COVID-19 in the U.S. States

We used logistic regression models to predict numbers of population-adjusted confirmed cases (per one million), deaths (per one million), ICU cases (per one million), case fatality rates and ICU admission rates of COVID-19 in the U.S. states. Surprisingly, the percentages of all the three age populations were positive predictors of the five response variables (Fig. 2). This result appears to conflict with findings from previous studies showing that older persons infected with SARS-Cov-2 have a higher risk to develop into severe cases [21], [22]. One potential explanation for this difference could be that SARS-Cov-2 has evolved to be highly infectious and lethal to all groups of people, including young persons. Indeed, recent reports also demonstrated the increased COVID-19 cases and hospitalization rate of young people during the second wave of the pandemic [23], [24]. Smoking and hospital beds showed inconsistent correlations with the five response variables. For example, smoking was negatively correlated with numbers of confirmed cases and numbers of ICU cases, while it was positively correlated with ICU admission rates (Fig. 2). In contrast, numbers of hospital beds were positively correlated with numbers of confirmed cases and numbers of ICU cases, while they were negatively correlated with ICU admission rates (Fig. 2). The correlation between smoking and COVID-19 remains controversy. Some studies showed that smoking was a risk factor for COVID-19 progression [25], [26], while some other studies concluded that smoking has no significant association with COVID-19 [27], [28].

Fig. 2

Prediction of COVID-19 transmission and fatality in 50 U.S. states using 17 variables with logistic regression models. The β-coefficients for each variable and prediction accuracies and AUCs in the logistic models are shown. AUC: the area under the receiver operating characteristic curve. * P < 0.05, ** P < 0.01, *** P < 0.001, ns not significant.

In general, mobility showed positive correlations with numbers of confirmed cases, deaths, and ICU cases, case fatality rates and/or ICU admission rates of COVID-19, such as grocery and pharmacy, transit stations, workplaces, and residential mobility (Fig. 2). It confirms that social distancing is an important measure to contain COVID-19 transmission [29], [30], [31]. The increased mobility from grocery stores to pharmacies may indicate that more persons were likely to have symptoms of COVID-19 to purchase drugs. However, we could not exclude the possibility that panic buying occurred for worrying about an upcoming shortage or price hike of drugs. Temperature showed a significant negative correlation with numbers of COVID-19 cases, while dew point temperature had a significant positive correlation with them (Fig. 2). It is consistent with the result showed in a previous study [32]. The percentage of people vaccinated showed significant inverse correlations with numbers of COVID-19 deaths and case fatality rate (Fig. 2). It indicates that widespread vaccination can significantly reduce COVID-19 deaths and mortality. However, the percentage of people vaccinated showed no significant correlation with numbers of confirmed cases or ICU cases, or ICU admission rates. Taken together, these results indicate that increase in vaccination rates may lessen the mortality of COVID-19 but not lower the risk of its transmission. This indication appears to be in agreement with the global pandemic trend of COVID-19. Based on the 17 variables, we predicted high versus low number of confirmed cases, deaths, and ICU cases, and case fatality rate and ICU admission rate by logistic regression models, respectively. In most cases, the prediction accuracy and AUC were not less than 70 % (Fig. 2).

Associations of vaccination with COVID-19 transmission and fatality in the U.S. States

To further investigate the association between vaccination and COVID-19 vulnerability, we used the whole period data of vaccine usage (percentage of people vaccinated), confirmed COVID-19 cases (per one million), deaths (per one million), ICU cases (per one million), case fatality rate and ICU admission rate. We found that vaccine usage had significant negative correlations with numbers of COVID-19 deaths (Spearman correlation, R = -0.19; P = 5.5 × 10-3) and ICU cases (R = -0.12; P = 2.1 × 10-2), ICU admission rates (R = -0.15; P = 3.8 × 10-3), and case fatality rates (R = -0.17; P = 1.3 × 10-2) (Fig. 3). However, the correlation between vaccine usage and numbers of COVID-19 cases were not significant (R = 0.04; P = 3.7 × 10-1). Again, these results suggest that increased vaccine usage may reduce the severity of COVID-19 but not the risk of its transmission.

Fig. 3

Relationships between vaccination usage and COVID-19 transmission and fatality in the U.S. states. The Spearman correlation coefficients (R) and P-values are shown.

Relationships among vaccination, Delta variant, and the COVID-19 pandemic

We found that numbers of COVID-19 deaths and case fatality rates were significantly higher in the group before vaccine usage (5 months, from August 2020 to December 2020) than in the group after vaccine usage (5 months, from January 2021 to May 2021) (one-tailed Mann–Whitney U test, P = 3 × 10−3, 1.3 × 10−2, respectively) (Fig. 4A). However, numbers of COVID-19 deaths and case fatality rates were not significantly different between the group without vaccine usage at an earlier stage (6 months, from January 2020 to June 2020) and the group without vaccine usage at a later stage (6 months, from July 2020 to December 2020) (Fig. 4A). Also, they were not significantly different between the group before the Delta variant became the main strain (3 months, from March 2021 to May 2021) and the group after the Delta variant became the main strain (3 months, from June 2021 to August 2021) (Fig. 4A). These results again suggest that vaccine usage may reduce the severity of COVID-19. Furthermore, these results suggest that the Delta variant may weaken the effect of vaccines on lessening the severity of COVID-19, although vaccine usage remains effective in reducing the severity of COVID-19 caused by the Delta variant infection. This is consistent with the reports from a previous study [33]. In comparisons of numbers of COVID-19 cases, numbers of ICU cases, and ICU admission rates, no significant difference was observed between these groups (Fig. 4B). Again, it suggests that vaccine usage cannot lower the risk of COVID-19 transmission.

Fig. 4

Comparisons of COVID-19 transmission and fatality in the U.S. states between different groups. A. Comparisons of numbers of COVID-19 deaths and case fatality rates between different groups. B. Comparisons of numbers of COVID-19 cases, numbers of ICU cases, and ICU admission rates between different groups. The one-tailed Mann–Whitney U test’s P-values are shown.

Discussion

We used the machine learning approach to evaluate the impact of various factors on the COVID-19 pandemic. Interestingly, we found that all age groups were risk factors, indicating that different ages of persons have equal susceptibility to COVID-19 and that young persons have the same risk of developing into severe/critical cases after infection. The emergence of SARS-CoV-2 variants could enhance the susceptibility and risk of young people to COVID-19 [3], [4], [5]. Other risk factors include various mobility variables, such as grocery and pharmacy change, transit stations change, workplaces change, and residential change. These mobile patterns usually indicate narrower social distancing to result in a higher risk of COVID-19 transmission. The impact of smoking on the pandemic remains controversial. Our results showed that the increased smoking rate was associated with reduced COVID-19 cases and ICU cases but with the elevated rate of ICU admission. Thus, smoking is a risk factor for developing into severe cases. Our results indicate that high temperature may reduce COVID-19 prevalence, while it cannot lower the risk of COVID-19 developing into critical and even death cases. More importantly, our various analyses consistently support that increase in vaccination rates is able to lower the risk of developing into critical cases of COVID-19, although it cannot lower the risk of its spread. A recent study from the CDC showed that the death rate of the unvaccinated COVID-19 population was 11 times higher than that of the vaccinated COVID-19 population [34], supporting the protective role of vaccines against COVID-19 deaths.

This study is interesting for several reasons. First, we used the logistic regression model to evaluate the contributions of different variables to five COVID-19-related response variables, namely confirmed cases, deaths, ICU cases, ICU admission rate, and case fatality rate. Our method overcame the limitation of univariate analysis that ignored the interdependence between different variables. Second, this study used the data of all cases released by the US CDC, which covered as long as 18 months of pandemic period. Third, we utilized both machine learning and statistical methods to explore the role of vaccines in the pandemic and obtained consistent results. That is, vaccine usage is able to lower the risk of COVID-19 developing into critical cases or deaths instead of controlling COVID-19 infections.

This study also has several limitations. First, because the CDC data is grouped according to months, we have to use monthly averages that may rough the characteristics of variables to a certain extent. Second, this study only analyzed the data from the U.S.; thus, the conclusions may not necessarily apply to the other countries which have significantly different conditions from the U.S. Third, limited by the timely update of publicly available data, some of the data used in this study, such as number of hospital beds, may not exactly represent the data in the period we analyzed. This may have some impact on the results of the study. Finally, some datasets, such as Google’s mobile datasets, may cause certain analytical biases due to their limited representation.

Conclusions

Vaccination can significantly reduce COVID-19 deaths and mortality, while it cannot reduce the risk of COVID-19 infection. Therefore, in addition to vaccination, other measures, such as social distancing, remain important in containing COVID-19 transmission and lower the risk of COVID-19 severe outcomes.

Declarations

Ethics approval and consent to participate

This work did not require ethical approval as the analysis was based on publicly available data.

Consent for publication

All authors read and approved the publication of the final manuscript.

Availability of data and materials

The statistics of COVID‐19 were from the Centers for Disease Control and Prevention (CDC) in the U.S. (https://data.cdc.gov/Case-Surveillance/COVID-19-Case-Surveillance-Public-Use-Data-with-Ge/n8mc-b4w4). The additional datasets supporting the conclusions of this article are included in Table 1.

CRediT authorship contribution statement

Rongfang Nie: Methodology, Validation, Formal analysis, Investigation, Data curation, Visualization, Writing – original draft. Zeinab Abdelrahman: Investigation, Data curation. Zhixian Liu: Investigation, Funding acquisition, Project administration. Xiaosheng Wang: Conceptualization, Methodology, Resources, Investigation, Writing – review & editing, Supervision, Project administration, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.