A time-varying vulnerability index for COVID-19 in New Mexico, USA using generalized propensity scores.

The coronavirus disease (COVID-19) pandemic has highlighted systemic inequities in the United States and resulted in a larger burden of negative social outcomes for marginalized communities. New Mexico, a state in the southwestern US, has a unique population with a large racial minority population and a high rate of poverty that may make communities more vulnerable to negative social outcomes from COVID-19. To identify which communities may be at the highest relative risk, we created a county-level vulnerability index. After the first COVID-19 case was reported in New Mexico on March 11, 2020, we fit a generalized propensity score model that incorporates sociodemographic factors to predict county-level viral exposure and thus, the generic risk to negative social outcomes such as unemployment or mental health impacts. We used four static sociodemographic covariates important for the state of New Mexico-population, poverty, household size, and minority population-and weekly cumulative case counts to iteratively run our model each week and normalize the exposure score to create a time-varying vulnerability index. We found the relative vulnerability between counties varied in the first eight weeks from the initial COVID-19 case before stabilizing. This framework for creating a location-specific vulnerability index in response to an ongoing disaster may be used as a quick, deployable metric to inform health policy decisions such as allocating state resources to the county level.

Introduction

On March 11, 2020, Coronavirus disease 2019 (COVID-19) was declared by the World Health Organization (WHO) as a pandemic [1], [2]. In the United States, the first possible COVID-19 case from local transmission was recorded on February 26, 2020 (CDC, 2020) and as of December 31, 2020, there have been a total of 20,024,801 cases and 346,050 deaths [3]. COVID-19 has disproportionately affected racial minority and underserved groups [4], [5], [6], highlighting systemic inequalities in the burden of negative social outcomes resulting from the pandemic. While a particular burden of COVID-19 incidence in a less vulnerable community may result in few negative social outcomes, a similar burden in more vulnerable communities could have cascading negative effects, such as unemployment, increased poverty, and/or mental health effects. This underscores the need to identify the most vulnerable communities, especially within culturally diverse states.

New Mexico (population 2,100,000) is a state in the southwestern US that has a unique population relative to the rest of the country. The state has the highest percent population that identifies as Hispanic or Latino at 49% compared to the US state average of 19% [7], [8], [9], [10]. New Mexico also ranks as the second highest state by percent population identifying as American Indian (hereinafter called Native American) or Alaska Native (11%, compared to the US state average of 1%)[7], [9]. The majority of the Native American population within New Mexico, the Diné, live in the northwest corner of the state within the largest Native American reservation in the US by size and second largest by population, the Navajo Nation [7]. New Mexico is home to 23 Native American tribes, including 19 Pueblos, with most located in the northern half of the state [11]. New Mexico is one of the poorest states in the US, with approximately 19% of the population living in poverty [12].

In part due to New Mexico’s relatively high rate of poverty and large minority population, the US Centers for Disease Control and Prevention (CDC) identifies counties in New Mexico as highly vulnerable to negative outcomes of disasters, such as pandemics [13]. The CDC quantifies this vulnerability using the CDC Social Vulnerability Index (SVI), which includes four related themes—socioeconomic status, household composition, race/ethnicity/language, and housing/transportation—to calculate a relative vulnerability to a generic disaster among all US counties. Counties in New Mexico with vulnerable communities may suffer more from negative outcomes resulting from COVID-19, including stresses on human health and the economy. Indeed, the negative health outcomes from COVID-19 in New Mexico are already apparent: as of September 14, 2020, 76% of COVID-19 cases were Native American, Alaska Native, or Hispanic/Latinos, although they make up only 60% of the population, while 14% of cases were White, although they make up 37% of the population [10], [14].

The SVI is computed for the entire US to act as a comparison among all counties and is static with time, so the level of relative vulnerability does not vary in response to an ongoing, identified disaster. Since we can now define the rapidly evolving COVID-19 pandemic as an ongoing disaster, a time-varying vulnerability index that incorporates an updated measure of COVID-19 burden may provide a more accurate, real-time estimate of relative vulnerability to negative outcomes from COVID-19. Case burden may cause a positive feedback loop within a community, whereby as case burden increases, so do the resultant negative outcomes, which in turn further increases case burden. An example of this would be a case outbreak at a community business that causes the business to close; residents would lose their source of income, which results in those individuals not being able to afford to purchase personal protective equipment like face masks, resulting in higher risk of infection and an increase in community cases. Additionally, due to the unique population makeup in New Mexico, incorporating sociodemographic variables that represent disparities common among communities in New Mexico may provide a more nuanced measure of vulnerability for the state, as opposed to national-level measures that are included in the SVI.

Our goal was to compute a county-level vulnerability index for the state of New Mexico to identify which communities may be at highest risk for negative social outcomes as the COVID-19 pandemic progressed. To do so, we compiled sociodemographic information at the county-level that represents vulnerabilities specific to communities in New Mexico. Then, we used a generalized propensity score model to create a generic causal effect measure of COVID-19 case burden in a community. We normalized the causal effect measure to create the vulnerability index, representing the relative county-level vulnerability to negative social outcomes to COVID-19. As an initial evaluation of the performance of our vulnerability index, we compared the ranked levels of vulnerability against county-level cumulative death and case fatality ratio data. Our vulnerability index may be used by state health officials as a decision support tool to identify the relative vulnerability of communities across New Mexico to allocate state resources to the county-level appropriately. More broadly, this framework may be applied to other locations and contexts to identify at-risk communities in response to an ongoing disaster.

Methods

Identification of important sociodemographic covariates

We identified four important covariate themes that may make communities in New Mexico more vulnerable to negative outcomes from COVID-19. The first important covariate we identified was the total population within each county. Population metrics play a direct role in the capacity for an infectious disease to spread by community transmission [15], [16]. We included total population here, not population density, since county-level measures of population density are skewed in New Mexico due to the majority of residents living within a few large cities in each county. New Mexico has four cities with populations over 50,000 (Albuquerque: 561 k, Las Cruces: 103 k, Rio Rancho: 99 k, and Santa Fe: 85 k)[10] that are situated across four different counties (Bernalillo, Doña Ana, Sandoval, and Santa Fe Counties, respectively), while a large portion of the state is either rural or uninhabited [17]. Although large population centers have the capacity for higher COVID-19 cumulative case counts, these communities usually have more available social support and healthcare services since there are often more resources in cities versus rural communities [15]. So smaller, rural populations may be at higher risk for negative social outcomes from COVID-19.

The second important covariate we identified was the percent of population living in poverty. New Mexico ranks as the second poorest state in the US based on the percent of total population living in poverty and has the fourth lowest median household income [12]. Within the state, 20 counties (30% of total) report at least 20% of residents living below the state-adjusted poverty line [18]. People living in poverty may not have the same access to disease prevention resources such as facemasks, water, and disinfectant, placing them at higher risk for contracting the disease. Areas with lower income are more likely to not adhere to COVID-19 stay-at-home protocols, which may accelerate disease transmission among the community [19]. People living in poverty who contract COVID-19 may be more financially overwhelmed by medical costs, which could leave them vulnerable to unpaid rent and subsequent evictions or food insecurity. People holding lower-income jobs are often not able to work from home due to the nature of their work (e.g., transportation services, supermarket workers)[20] and may be more vulnerable to losing their job due to increasing COVID-19 cases in their community or having to take unpaid sick leave, which could cause financial stress and result in eviction from housing.

The third important covariate we identified was average household size. The average household size in the US is 2.63 people per household and statewide, New Mexico has a similar average of 2.64 people per household [10]. However, some cultures within New Mexico live in multigenerational household settings resulting in larger household sizes, with the county-level range in household size from 2.02 to 3.62 people per household [7]. Areas in the US with larger Hispanic or Latino and Native American populations, such as New Mexico, tend to have higher percentages of multigenerational households [21] due to variations in cultural norms, socioeconomic status, immigration, marriage patterns, and health status [22], [23], [24], [25], [26]. Increased housing density is likely a contributing factor to increased virus transmission within a community since the number of secondary COVID-19 cases generated from each primary case is likely higher and can exacerbate negative social outcomes if multiple family members are sick at the same time.

The fourth covariate we identified was the percent minority population, defined here as those not identifying in the US Census category, “White alone, not Hispanic or Latino”. New Mexico has a 63% non-White population, compared to the US average of 40% [8], [10]. Native Americans and Alaska Natives represent 11% of the population in New Mexico and Hispanic or Latinos represent 49% [7]. Minority populations have been disproportionately affected by COVID-19 [5], likely due to both biomedical factors and factors relating to health inequity [5]. Many individuals within minority groups have disproportionately higher levels of underlying health conditions such as heart disease and diabetes or suffer from substance use disorders such as alcohol, smoking, or opioid use that would not only make them at risk of complications from COVID-19, but also compounding negative social outcomes [4], [5], [6], [27]. Minority-run small businesses within a community may suffer, especially from sick workers or having to close their business to adhere to public health mandates.

COVID-19 and sociodemographic covariate data

We use a generalized propensity score model to predict the exposure of COVID-19 cases at the county-level (n = 33) for the state of New Mexico. We use cumulative case counts as a metric for exposure (Figure S1). We gathered daily, county-level COVID-19 case data from the New York Times (NYT) from the first reported case in the state on March 11, 2020 to August 3, 2020 [3]. Following Marvel et al. [28], as a preliminary examination of our index in relation to available, measurable COVID-19 outcomes, we compared the results of our vulnerability index to cumulative deaths from COVID-19 from the NYT over the same period, and the case fatality ratio, which we calculated as the daily ratio of cumulative cases to deaths (not lagged, due to variations in data across counties). We examined the lags between the vulnerability index and COVID-19 outcomes between one to four weeks. We started this analysis on April 1, 2020, when at least three counties in New Mexico had reported deaths from COVID-19.

We gathered county-level total population and three sociodemographic variables from the US Census 2014–2018 American Community Survey (ACS) 5-Year Estimates [18] (except for Rio Arriba County, which was taken from the 2013–2017 ACS 5-Year Estimates) [18]: percent population below the state-adjusted poverty line, average household size in county, and percent non-White population. We used aggregated data from recent years in order to capture data for counties with low population size, as seen in New Mexico. We calculated percent non-White population by subtracting the percent of population who identified as, “White alone, not Hispanic or Latino,” from 100%. We used the natural log of population to reduce the large right-hand skew in population sizes of New Mexico counties to create a more normalized population distribution.

Propensity score model

We used a propensity score model to quantitatively assess the relative vulnerability of counties to negative social outcomes from COVID-19. Propensity scores are frequently used in epidemiologic studies to determine the probability of receiving a non-randomized exposure or treatment (for more on propensity scores, see Supporting Information), as precursor to a causal analysis. The goal of our propensity score analysis was to create a generic causal effect measure of COVID-19 case burden in a community that can be used by future researchers or policymakers to assess how COVID-19 has caused increased negative social outcomes within communities. Our model took the form of a Poisson linear regression model with a population offset term (glm() function in R version 4.0.1; Eq. (1)), suitable for predicting count data:where the values of the regression coefficients change each week in response to the changing number of cumulative COVID-19 cases, . Since we are predicting a generalized outcome and want to measure the relative vulnerability among counties, we normalized the time-varying vulnerability index to [0,1].

The first travel-related cases of COVID-19 were reported in New Mexico on March 11, 2020 [29]. Starting a week after this first case, we fit our generalized propensity score model each week, which allowed for dynamic reweighting of variable importance. We chose to run our model using cumulative case counts once a week instead of daily to reduce the noise from rapid changes in case counts (data not shown). We performed these analyses for a total of 22 weeks, starting on March 18, 2020 and ending on August 12, 2020.

Results

Time-varying community vulnerability index

The four covariates we incorporated that may make communities in New Mexico more vulnerable to negative social outcomes from COVID-19 highlighted the sociodemographic variations among counties (Fig. 1 ). Counties with the highest population are in the northcentral part of the state, with one high-population county in the southcentral region. The counties with the highest population had low to moderate levels of poverty, percent non-White populations, and average household sizes. Counties with lower total populations tended to have the highest levels of poverty, higher average household sizes, and larger percentages of non-White population. The county with the highest percent of poverty was McKinley (Fig. S2), which lies in the Navajo Nation. Counties with large percentages of non-White population were mostly rural counties in the northern half of the state, including Cibola, Rio Arriba, McKinley, San Juan, San Miguel, and Guadalupe.

Fig. 1

We used four important sociodemographic covariates that may make communities in New Mexico counties more vulnerable to negative social outcomes from COVID-19: (a) total population, (b) percent of the county population below the state-adjusted poverty line, (c) average household size in county, and (d) percent non-White population.

As case counts became available, we calculated the generalized propensity score model using cumulative COVID-19 cases as a metric for exposure to predict negative social outcomes from COVID-19. The counties that had higher levels of average household size had some of the largest ranges in the magnitude of their vulnerability index from the first to last week in our analysis (Fig. 2 ). The relative vulnerability among the counties oscillated in the month following the first case of COVID-19 in New Mexico (Fig. 3). After the first week, all covariates were statistically significant (p < 0.01; Table S1). After approximately eight weeks, the vulnerability indices stabilized. Bernalillo County, the most populous county, was ranked as the most vulnerable county for the majority of the analysis. McKinley County began as the 15th (out of 33) most vulnerable county. Once COVID-19 cases increased in McKinley County starting in April 2020, its vulnerability index increased and then wavered between the highest and second highest for the remainder of our analysis. At the end of our analysis, the most vulnerable counties were Bernalillo (1.00), McKinley (0.94), and Doña Ana (0.93), while the least vulnerable were Harding (0.00), Catron (0.23), and De Baca (0.26; Fig. 3, Table S2). The vulnerability score for each county is available in the Supplemental Information, separated into the five health districts in the state (Figs. S3–S7).

Fig. 2

A county-level map of New Mexico with (a) the vulnerability indices calculated at the beginning of our time series analysis at week 1, (b) the vulnerability indices calculated at the end of our time series analysis at week 22, and (c) the maximum range of the vulnerability index within each county, highlighting which counties had the largest changes of magnitude in vulnerability.

Fig. 3

A time series analysis of the vulnerability index for counties in New Mexico. We plot all 33 counties and highlight nine counties that we discuss in the text for reference.

We examined the time series of regression coefficient values for our four sociodemographic covariates and found the coefficient for average household size was the most variable through time (Fig. 4 ; Table S1). Population had a relatively stable, non-zero regression coefficient that slightly decreased over time. The percent of population in poverty was not as important for determining which counties had variable vulnerability indices and was roughly inverse the effect of the percent non-White population.

Fig. 4

A time series analysis of the regression coefficients of the four sociodemographic covariates in our model. We excluded the value of the intercept since its interpretation is not meaningful in context of our model. The gray line at the zero vertical is for visual reference.

As a preliminary evaluation of our vulnerability index, we compared the relative measure of county-level vulnerability to available negative outcomes of COVID-19. We examined both county-level cumulative deaths and the case fatality ratio using Spearman correlation (Fig. S8). Though our goal was not to predict cases or deaths from COVID-19, the only measures of COVID-19 outcomes we had available at this time of this analysis early on in the pandemic to compare with our vulnerability index were cases and deaths. To ensure that our relative measures of vulnerability weren’t vastly different than the resultant deaths from the pandemic, we analyzed lags between the ranked vulnerability index and outcomes of 1–4 weeks since we assume there is a lag between exposure to COVID-19 cases and the negative outcomes. We started this analysis on April 1, 2020, after three counties had reported deaths from COVID-19. The mean correlation values between our vulnerability index and cumulative deaths were between 0.64 and 0.66 for the four different lag periods. The mean correlations were lower for case fatality ratio, between 0.34 and 0.36 (Fig. S8). For both negative outcomes, the mean rho values were similar across all lag times. For both outcomes and all lag times, the rho values at the beginning of the time series were less significant than later. This is likely due to more counties having reported deaths from COVID-19 later in the analysis. For comparison, we calculated the mean correlation values between cumulative case counts and deaths and case fatality ratio (data not shown). Between cumulative case counts and cumulative deaths, the mean correlation values were higher between 0.74 and 0.78. Between cumulative cases and case fatality ratio, the correlation values over the four time lags were all 0.35. So, while cumulative case counts would be a better predictor of deaths from COVID-19 than our vulnerability index, our index was instead designed to analyze negative social outcomes of COVID-19, but it still displayed an adequate ability to capture these COVID-19 outcomes available early on in the pandemic.

Discussion

Our generalized propensity score model allowed us to predict the exposure of COVID-19 cases in a county while accounting for the confounding effects of sociodemographic factors, using cumulative case counts as a metric for exposure. This approach allows for a quick and relatively simple decision support tool to help policy makers allocate resources to the county-level and rapidly shift their focus if relative vulnerability changes, such as allocating mobile testing units or vaccine distribution. One of the benefits of using generalized propensity scores to calculate vulnerability is that they account for confounding variables without specifying causal relationships, so that the score can be applied to a wide range of potential outcomes of interest.

Average household size was an important, time-varying determinant of county-level vulnerability. In the first week following the first COVID-19 case in New Mexico, larger household sizes were predictive of a higher number of cases. Larger households have more family members who could become infected as the primary case of COVID-19, and initial within-household transmission can cause cascading negative outcomes, especially before people were fully aware of the ease of transmission of COVID-19. One week after the first case, New Mexico shut down schools [30] and mandated stay-at-home orders [31] two weeks later, which then limited additional exposure pathways between communities. Following the first week, the regression coefficient for average household size decreased until it was negatively associated with cases for the next five weeks. This may indicate that after the initial onset of the pandemic and following stay-at-home orders, larger households were protective against the negative social outcomes from COVID-19, perhaps because only one person within a household left the residence regularly for essential services like buying food. This may have especially protected the elderly and children within multigenerational households. Two months after the first case of COVID-19 in New Mexico, around the US holiday of Memorial Day with a three-day weekend, household size again became positively correlated with COVID-19 cases. More cases may have been related to increased holiday travel, fatigue from quarantine measures, or businesses beginning to partially reopen in the state [32], allowing for increased exposure pathways within larger household sizes.

The three other covariates—population, percent of non-White residents, and percent of population in poverty—did not have as large of time-varying roles as average household size. However, there were significant correlations between the four covariates (Fig. S9), with the largest between average household size and percent non-White population (0.55). So average household size may have been accounting for multiple vulnerabilities within communities in New Mexico. We ran a sensitivity test using average household size as the only covariate in our generalized propensity score model and found the same pattern of time-varying regression coefficient as our full model (Fig. S10), indicating that average household size was indeed the main driver of the time-varying pattern of vulnerability within a county. Average household size has also been shown to be important in predicting infection rate of other respiratory diseases, such as influenza [33].

Comparison to other COVID-19 vulnerability indices

Our vulnerability index is unique compared to most other indices of risk because it is time-varying as opposed to static. For example, the CDC SVI is static and most useful to assess risk for a generalized disaster [13]. Since we have identified the target disaster as the COVID-19 pandemic, a time-varying vulnerability index using available COVID-19 case data and sociodemographic information particularly relevant to a pandemic is a way to provide a more specified, real-time measure of community vulnerability. The SVI was also combined with risk factors specific to COVID-19, including the population with underlying health conditions and health system capacity, to calculate a vulnerability score specific to COVID-19 for all counties in the US [34]. The counties that are most vulnerable according to this index, created by the Surgo Foundation, are McKinley, Guadalupe, and Cibola Counties. Our index also identifies McKinley as highly vulnerable throughout the majority of our analysis. However, our index identifies the other top two most vulnerable counties as Bernalillo and Doña Ana, whereas the Surgo Foundation vulnerability scores place Bernalillo as third to last and Doña Ana in the least vulnerable half.

Another nationwide index that is also time-varying is the NIEHS COVID-19 Pandemic Vulnerability Index [28], [35]. This model includes both static sociodemographic covariates and time-varying covariates such as COVID-19 outcomes (i.e., transmissible cases and disease spread), population mobility, and intervention measures (i.e., social distancing and testing). The NIEHS Vulnerability Index pulls some sociodemographic data used by the CDC Social Vulnerability Index. For our last analysis date on August 12, 2020, the top three counties we identify as most vulnerable to the negative outcomes from COVID-19 were Bernalillo, McKinley, and Doña Ana Counties, while the NIEHS COVID-19 Pandemic Vulnerability Index places Cibola, McKinley, and Lea counties as most vulnerable, though Doña Ana and Bernalillo are in the top 10 most vulnerable. The differences in results between the CDC and NIEHS vulnerability indices and ours may be due to the generalization of factors across the whole US that make counties more vulnerable to COVID-19, while our model incorporates covariates chosen specifically for the state of New Mexico. Regional differences in culture, policy, and state-level disease mitigation efforts would make levels of vulnerability difficult to compare across the nation, which could potentially limit the functionality of a nation-wide vulnerability index.

While our model was intended for use at the state-level for allocating resources to counties vulnerable to negative social outcomes from COVID-19 in general, our vulnerability index may be modified to assess risk to more specific outcomes such as negative health outcomes from COVID-19 [36]. Our index did not explicitly account for preexisting medical conditions that may put a person at higher risk for contracting COVID-19 or having a more complicated form of the disease, and thus be more vulnerable to negative health outcomes. As the pandemic progressed, we learned that factors such as chronic disease burden, mortality rates from diabetes, heart disease, and chronic lower respiratory disease have been strongly associated with sub county-level COVID-19 cases [37]—these factors would be important to include in a causal effect measure of COVID-19 case burden on negative health outcomes. Additionally, we learned that older populations were more susceptible to becoming infected with COVID-19 and at greater risk of severe disease [38]. We didn’t include an age driver in our model since we don’t expect an older population to be particularly predictive of negative county-level social outcomes. Some vulnerability indices have combined both social and epidemiologic factors of vulnerability to create a composite index to assess health risks within a community [39].

Applying a generalized propensity score-based vulnerability index to other locations and disasters

Our vulnerability index used sociodemographic variables that may make communities in New Mexico at higher risk of negative outcomes from COVID-19. By doing so, we created an index unique to the state. Our framework may be adapted to calculate vulnerability in other regions by using covariates relevant to that area, or it could be expanded to analyze the whole US by selecting covariates that in general put the US population at greater risk of negative health outcomes, similar to the CDC and NIEHS vulnerability indices. One way our vulnerability index could be expanded upon is by creating a fully dynamic model in which the magnitudes of the covariates also change with time, such as unemployment, homelessness, or mobility data, to capture potential exposure to COVID-19. This would allow the model to capture feedback mechanisms; for example, poverty may cause a positive feedback loop onto the level of vulnerability to negative outcomes from COVID-19.

The method of calculating a vulnerability index from generalized propensity scores is easily transferrable to other COVID-19 applications or other disasters, so long as there is an exposure variable that can be used as a proxy for the level of vulnerability to the outcome of interest. For infectious diseases like COVID-19, case counts will be biased and may reflect inconsistent reporting across counties and by days of the week, so it is important to consider at what time scale to calculate the vulnerability index to provide a stable measure of risk. For natural disasters, such as hurricanes, floods, or wildfires, the outcome variable could be a measure of displacement, disaster-related injuries or deaths, or another negative outcome.

Conclusions

We created a time-varying vulnerability index for the state of New Mexico using generalized propensity scores to identify which communities may be at highest risk for negative social outcomes from the COVID-19 pandemic. We found that the county vulnerabilities wavered for two months following the first COVID-19 case in New Mexico and then stabilized. Average household size was an important determinant of the time-varying vulnerability within each county; initially, household size was positively correlated to vulnerability, indicating larger families would place a community at higher vulnerability to negative outcomes from COVID-19. However, after a week, a larger household size was protective against vulnerability to negative outcomes from COVID-19. After more counties reported deaths from COVID-19, our vulnerability index was significantly correlated with cumulative deaths and the case fatality ratio, suggesting the index could be used to predict the relative vulnerability to negative outcomes of COVID-19. As opposed to other static vulnerability measures, our time-varying approach allows the vulnerability of a community to change in response to an ongoing disaster. Our framework of creating a region-specific vulnerability index for a known disaster can be adapted to both other locations to assess the vulnerability to COVID-19, as well as other natural disasters.

CRediT authorship contribution statement

Morgan E. Gorris: Conceptualization, Methodology, Writing – original draft. Courtney D. Shelley: Conceptualization, Methodology, Writing – original draft. Sara Y. Del Valle: Writing – review & editing, Supervision, Project administration. Carrie A. Manore: Writing – review & editing, Supervision, Project administration.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.