Unpacking the impact of the COVID-19 pandemic: identifying structural domains.

Background: The novel coronavirus-19 (COVID-19) pandemic is a collective crisis that imposed an abrupt and unprecedented impact on college students, as universities were closed with little warning. Paired with the challenges associated with physical distancing (e.g. economic stress, job loss, food insecurity, housing challenges) and the simultaneous need to balance continued and new academic demands, impact will be wide-ranging. It is critical to determine the structure of the impact of this heterogeneous stressor (e.g. health concerns, pandemic worry, financial concerns) for prevention and intervention planning. Objective: Through an existing recruitment pipeline we were in a unique position to study the wide-ranging reach of this pandemic in a cohort of students for whom their university experiences were like no other cohort in history. Method: Data were collected from students who were in their third year of college during the onset of the pandemic; of the N = 1,899 in the cohort who were invited to participate in this COVID-related survey, 897 (47.2%) completed measures of impact between May and July of 2020.
Results: A series of confirmatory and exploratory models were fit to examine the structure of the pandemic-related domains. Following estimation of a single-factor model, a correlated five factors model, as well as two second-order factor structures, the five correlated factors (exposure, worry, housing/food instability, social media, substance use) model was found to represent the data most appropriately, while producing an interpretable solution. Conclusions: These measurement model analyses set the stage for future research to examine how these correlated factors impact psychiatric, substance, and academic outcomes in this vulnerable population.

Introduction

The novel coronavirus-19 (COVID-19) is a collective crisis resulting in the loss of ~500,000 lives in the USA alone (as of 1 March 2021) (“Daily Updates of Totals by Week and State. Provisional Death Counts for Coronavirus Disease 2019 (COVID-19),” 2021), with the death toll rising daily. The rapid transmission coupled with the lack of preparedness to both prevent and treat COVID-19 have exacerbated the impact in a wide variety of domains. Indeed, the pandemic is taking an unprecedented toll on a number of societal systems, including but not limited to: health care, politics, economic, and social-welfare systems (e.g. Baker, Bloom, Davis, & Terry, 2020; Nicola et al., 2020). The effects are also evident on numerous individual-level domains (e.g. social (Killgore, Cloonan, Taylor, Miller, & Dailey, 2020), academic (Kapasia et al., 2020), professional (Organization, I. L, 2020)). Rapid research on COVID-19 related impact has demonstrated the early psychological toll that the pandemic is having on various mental health symptoms (e.g. depression, substance use; for reviews, see Brooks et al., 2020; Hossain, Sultana, & Purohit, 2020). Thus, it is critical to understand the nature of this heterogeneous collective crisis (e.g. is there an overall continuum of exposure/stress, is there a higher order structure that best explains impact, are domains of impact distinct but correlated?). To the best of our knowledge, there has been no investigation into the structure of domains that comprise the COVID-19 related ‘impact.’ Moreover, such a study has not been conducted among college students specifically who may be uniquely affected due to disruptions in university life. The lack of specificity when referring to COVID-19 ‘impact’ in the extant literature is creating a subsequent lack of clarity regarding the consequences of COVID-19.

It is well documented that large-scale traumatic events (e.g. war [for a review, see Rousseau, Jamil, Bhui, & Boudjarane, 2015)], natural disaster [for a review see (Beaglehole et al., 2018)], and terrorist attacks [for a review, see Smith, Holmes, and Burkle Jr (2019)]) have pervasive adverse impacts on mental health, with studies demonstrating increased risk for PTSD (e.g. Carmassi et al., 2020), depression (e.g. Acarturk et al., 2018), and substance use disorders (e.g. Adams et al., 2015) following exposure. Moreover, a wealth of data from research on large-scale traumatic events have demonstrated that exposure to these events varies in severity based on ‘amount’ of exposure, as well as domain of impact. For example, even though a group of people may all be ‘exposed’ to a hurricane, there is great heterogeneity in severity of exposure. That is, some lose a loved one to the disaster, and yet other individuals do not face any direct consequences because of the event (e.g. Rhodes et al., 2010; Schwartz, Rothenberg, Kerath, Liu, & Taioli, 2016). Severity of exposure is critical to measure accurately, as it predicts psychiatric outcomes. S. W. Adams et al. (2019) demonstrated that World Trade Center tower survivors with PTSD and comorbid depression were more likely to have had greater exposure to the events of 9/11 than those with depression alone. Additionally, in their study of Bosnian war survivors over an 11-year period, Comtesse and colleagues (Comtesse, Powell, Soldo, Hagl, & Rosner, 2019) found that those with higher trauma exposure during the war, compared to those with a smaller ‘dosage’ of trauma exposure during the war period measured objectively (i.e. with a checklist of war-related experiences), demonstrated higher symptoms of psychological distress.

The diathesis-stress model may help to explain the impact of event severity on subsequent outcomes, from a theoretical standpoint, positing that psychopathology develops only when an environmental stressor and/or trauma is severe enough to activate diathesis (e.g. McKeever & Huff, 2003). In the case of COVID-19, those experiencing more severe consequences related to the virus are more likely to develop symptoms of psychopathology, accounting for the observed heterogeneity in psychopathology related to COVID-19. Although there is less literature examining exposure severity as related to COVID-19 and subsequent psychopathology, studies on past pandemics (e.g. SARS, Ebola) demonstrate that those with more severe exposure to the pandemic are more likely to develop psychopathology than those with less severe exposure. For example, Park, Lee, Park, and Choi (2018) found that survivors of Middle East Respiratory Syndrome (MERS), medical personnel, and social workers were more severely impacted by MERS due to discrimination and stigmatization, which led to worsened outcomes as compared to groups without direct exposure to the virus. Taken together, there is evidence to suggest that severity and domain of impact are important variables influencing outcomes following a stressor. This evidence, coupled with the theoretical basis of the diathesis stress model, helps clarify why one individual may develop increased fear or anxiety in the context of COVID-19, and another may not. As such, the current paper aims to provide insight into the different domains of COVID-19 impact in order to inform future work focusing on links between these domains and subsequent psychopathology.

The extant literature examining psychiatric symptoms in response to COVID-19 has assumed uniform exposure to the virus, whereby every individual is presumed to be equally impacted by the pandemic given that they are currently living through this time. However, the COVID-19 pandemic has unique characteristics compared to the well-studied mass crises, including its global scope, far-reaching and widespread impact across numerous domains (e.g. personal and network infection, worry, economic impact, changes in daily routines), and unprecedented levels of media attention and public exposure (Horesh & Brown, 2020). Given the novelty of the pandemic, research is warranted in order to identify the domains that should be included when discussing COVID-19 ‘impact’ to more comprehensively understand the adverse sequelae of the pandemic.

The research that has been disseminated regarding COVID-19 and psychiatric symptoms relies on a couple of items to assess COVID-19 impact (Cao et al., 2020; Rossi et al., 2020). Broadly, these items capture depressive symptoms, PTSD symptoms, anxiety, worry, or stress, without capturing the severity of exposure. Given that the coronavirus pandemic is a novel crisis occurring on a global scale, it is imperative to assess related impact in the context of the environmental or ‘dosage’ piece.

As with global traumatic events, there is a need to soundly understand the structure of impact of COVID-19, considering the novelty of the pandemic and the demonstrated pervasive effects on a wide range of domains. Although numerous measures, such as the Coronavirus Impact Scale (CRISIS; (Merikangas et al., 2020) and the Epidemic Pandemic Impacts Inventory (EPII; (Grasso, Briggs-Gowan, Carter, Goldstein, & Ford, 2020)) have been created rapidly to measure impact, to our knowledge, studies have yet to better understand the structure of the pandemic impact. Recent work has focused on the importance of assessing COVID-related fear (Pakpour & Griffiths, 2020), and has argued that there have been four primary fear domains during COVID-19: of events that threaten physical/psychological integrity, interpersonal relationships, cognitive ability to master stressful situations, or the behavioural consequences of fear during the pandemic (Schimmenti, Billieux, & Starcevic, 2020). This work is particularly important because another study found that COVID-related psychological impacts have been moderate to severe (Wang et al., 2020). Thus, assessing the structure of COVID-19 impact, including questions around virus-related anxiety, seems important.

In addition, once the structure of COVID impact has been ascertained, it is important to investigate whether key demographic factors, such as race and sex, might impact these factors.

Specifically, there is work suggesting that there are differences in exposure to various types of trauma by race and sex (Norris, 1992; Roberts, Gilman, Breslau, Breslau, & Koenen, 2011; Tolin & Foa, 2008). Consistent with the broader trauma literature, global research in the wake of the pandemic has suggested that women are experiencing higher levels of stress and mental health symptoms compared to men (e.g. (González-Sanguino et al., 2020) [Spain]; (Mazza et al., 2020) [Italy]; (Odriozola-González, Planchuelo-Gómez, Irurtia, & de Luis-García, 2020) [Spain], (Wang et al., 2020); thus, it is critical to examine if the structure of impact differs by sex. Additionally, there is emerging evidence suggesting that the COVID pandemic is disproportionately impacting people of colour (Gold et al., 2020; Price-Haywood, Burton, Fort, & Seoane, 2020; Stokes et al., 2020), supporting the need for analyses aimed at examination of race effects on COVID domains of impact.

Thus, the primary aim of the present paper is to better understand domains of COVID-19 impact in a sample of undergraduate students. Given the heterogeneity of the pandemic, it was hypothesized that models of correlated factors would fit the data better than models in which a single continuum of impact or a higher order latent factor were present. The second aim of the study was to determine if sex or race predict these domains of impact. Given prior studies (Liu et al., 2020; Luo, Guo, Yu, Jiang, & Wang, 2020; Rothman, Gunturu, & Korenis, 2020; Xiong et al., 2020) suggesting that the pandemic may disproportionally affect women and minorities, it was hypothesized that significant effects of sex and race would be found.

Method

Larger study sample

Participants for the current project came from a larger, ongoing longitudinal study of behavioural and emotional well-being of college students at a mid-Atlantic public university. This study was approved by the Virginia Commonwealth University’s IRB (VCU IRB approval number HM13352). Baseline and follow-up data were collected on five cohorts during the fall and spring, respectively, of participants’ first year of college. Surveys were completed online through Research Electronic Data Capture (REDCap, Harris et al., 2009), a secure, web-based application designed to support data capture for research studies. The Spit for Science project began in fall 2011, and new cohorts were recruited in 2012, 2013, 2014, and 2017 (N = 12,358). Participants were on average 18.49 years old at baseline, 36.7% were male, 61.9% were female, and 1.4% declined to identify their sex. The sample reflected the population from which it was drawn: 47.9% White, 19.3% African-American, 16.6% Asian, 6.6% Hispanic/Latino, 9.6% other/multi-race/unknown/declined to respond. Those who completed the baseline survey were invited via email to complete a follow-up assessment during each subsequent spring.

Current study sample

The fifth cohort of the larger study, Spit for Science (S4S) enrolled and collected data on 2,476 students enrolled during their first year of college (2017–2018). Individuals in the fifth cohort who were still enrolled as VCU students in the spring of 2020, were recruited for a COVID-related survey in the spring/summer of 2020 (N = 1,899). Specifically, these individuals were interviewed in the fall of 2017 (year 1 fall/freshman year), spring of 2018 (year 1 spring/freshman year), spring 2019 (year 2 spring/sophomore year), and then in spring/summer 2020 (year 3 spring/junior year), following COVID being declared a pandemic in March of 2020. Of the N = 1,899 in cohort 5 who were invited to participate in this COVID-related survey, 897 (47.2%) completed it. The intention of this survey was to understand how students were experiencing and responding to COVID and its sequelae. Compared to the initial full cohort 5 (n = 2,476), there were significant differences on both sex and race/ethnicity. Those completing this survey were more likely to be female (78.6% versus 62% of those not participating), and were more likely to Asian (22.9% versus 15.3%) and were less likely to be Black (18.4% versus 22.4%) or White (40.2% versus 44%). These sex and race/ethnic differences were both small effects (Cramer’s V of .171 and .117, respectively).

Measures

Demographic predictors

Of interest in these analyses were sex and race differences in the impact of COVID. Sex assigned at birth was coded 0 = Female (79.7%), 1 = Male (20.3%). To maximize power to detect differences between groups, individuals were coded as being in one of the three largest groups: White, Black, Asian, or were coded as Other; three dummy coded variables comparing White (40.2%) to Black (18.4%), White to Asian (22.9%) and White to Other (18.4%) were created with the White group coded as all zeros serving as the reference group.

Coronavirus health impact survey (CRISIS)

The CRISIS (Nikolaidis et al., 2020) measure was developed to assess COVID-19 impact. Although developed by content experts in the field, including intramural NIMH researchers (Merikangas, 2020), due to the unexpected nature of COVID, psychometric developments had not been undertaken at the time of the development of this tool. Twenty-four of the COVID-related impact items were taken from the CRISIS, adapting where necessary to fit the needs of college student population, assessing concern for infection, changes in substance use, changes in different types of conflict with friends and family members, ability or difficulty social distancing, whether school and professional activities have been able to be transferred to virtual format, and whether use of media (TV, video games, social media) have changed since COVID. This scale has demonstrated good concurrent and predictive validity (Nikolaidis et al., 2020). See Table 1 for items included in analyses, as well as anchors/scaling.

Table 1.

Items loading on each of five factors

Factor	Number	Item wording and answer choices	Range of item in model
COVID Exposure	1	Have you been exposed to someone likely to have coronavirus/COVID-19? (1 = Yes, Someone with positive test OR Yes, someone with medical diagnosis, but no test OR Yes, someone with possible symptoms, but no diagnosis by doctor; 0 = None of these)	0–1 (not collapsed)
COVID Exposure	2	Have you been suspected of having COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
COVID Exposure	3	Count of Symptoms – Have you had any of the following: Fever, cough, shortness of breath, sore throat, fatigue, other (total of 6)	0–4
COVID Exposure	4	Has Anyone in your family and household been diagnosed with COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
COVID Worry	1	Since COVID, how worried have you felt about being infected? (0 = Not at all, 1 = Slightly, 2 = Moderately, 3 = Very, 4 = Extremely)	0–4 (not collapsed)
COVID Worry	2	Since COVID, how worried have you felt about friends/family being infected? (0 = Not at all, 1 = Slightly, 2 = Moderately, 3 = Very, 4 = Extremely)	0–4 (not collapsed)
COVID Worry	3	Since COVID, how worried have you felt about your physical health? (0 = Not at all, 1 = Slightly, 2 = Moderately, 3 = Very, 4 = Extremely)	0–4 (not collapsed)
COVID Worry	4	Since COVID, how worried have you felt about your mental/emotional health? (0 = Not at all, 1 = Slightly, 2 = Moderately, 3 = Very, 4 = Extremely)	0–4 (not collapsed)
COVID Housing/Food Concern	1	Did you have to move because of COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
COVID Housing/Food Concern	2	Since COVID, to what degree are you concerned about the stability of your living situation? (0 = Not at all, 1 = Slightly, 2 = Moderately, 3 = Very, 4 = Extremely)	0–3
COVID Housing/Food Concern	3	Since COVID, do you worry whether your food will run out because of a lack of money? (0 = No, 1 = Yes)	0–1 (not collapsed)
COVID Housing/Food Concern	4	Have your friends/family moved into your home since COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
Change in Media Use during COVID	1	Is the amount of TV you’re watching more than before COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
Change in Media Use during COVID	2	Is the amount of social media you’re using more than before COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
Change in Media Use during COVID	3	Is the amount of video games you’re playing more than before COVID? (0 = No, 1 = Yes)	0–1 (not collapsed)
Change in Media Use during COVID	4	How much are you reading or talking about COVID? (0 = Never, Rarely, or Occasionally, 2 = Often, 3 = Most of the time)	0–1 (not collapsed)
Change in Substance Use during COVID	1	Since COVID, have you noticed any changes in alcohol use? (0 = Have not used, 1 = Have been using a lot less, 2 = Have been using the same, 3 = Have been using a lot more)	0–3 (not collapsed)
Change in Substance Use during COVID	2	Since COVID, have you noticed any changes in vaping? (0 = Have not used, 1 = Have been using a lot less, 2 = Have been using the same, 3 = Have been using a lot more)	0–3 (not collapsed)
Change in Substance Use during COVID	3	Since COVID, have you noticed any changes in cigarettes or tobacco products? (0 = Have not used, 1 = Have been using a lot less, 2 = Have been using the same, 3 = Have been using a lot more)	0–3 (not collapsed)
Change in Substance Use during COVID	4	Since COVID, have you noticed any changes in marijuana use? (0 = Have not used, 1 = Have been using a lot less, 2 = Have been using the same, 3 = Have been using a lot more)	0–3 (not collapsed)

Epidemic-pandemic impacts inventory (EPII)

Like the CRISIS, the EPII was developed by content experts in the field (Grasso, Briggs-Gowan, Ford, & Carter, 2020). Twenty of the COVID-related impact items used in this survey were taken from the EPII (Grasso, Briggs-Gowan, Ford, & Carter, 2020), assessing ‘tangible impacts’ of epidemics, specifically COVID across a number of domains including social life. This measure assesses whether individuals experienced work- (e.g. being laid off), education- (e.g. had a child in home that could not go to school), home life- (e.g. family or friends having to move in), social- (e.g. unable to be with close family in critical condition), economic-(e.g. unable to pay bills), emotional health-(e.g. spent more time on screens), physical health-(e.g. more time sedentary), physical distancing-(e.g. close family member having to be quarantined), and infection-related (e.g. tested positive for COVID) difficulties. At the time of this writing, there are currently no published psychometric properties available. See Table 1 for items included in analyses, as well as anchors/scaling.

Data analytic plan

To evaluate the primary study questions, a series of latent variable models is fit to the S4S COVID item-level data. All model fitting was carried out using the Mplus Version 8 software (Muthen & Muthen, 2017) while using a limited information approach where all observed values are used in the estimation of all the pairwise associations among the variables when data were missing. The full n = 897 individuals in the online data collection sample were used to investigate the structural organization of the COVID-related items tapping different domains of impact. The WLSMV (weighted least squares mean and variance) robust estimator was used to fit all models to the item data. This estimator operationalizes categorical items as latent continuous response variables upon which thresholds are estimated to distinguish between the item response categories. Figure 1 presents a general flow chart showing the steps taken to arrive at a final model. To guide the model fitting, we used both a priori theory and, where needed, carried out data-driven steps to inform our modelling decisions.

Figure 1.

Flow chart depicting how final measurement model was generated

First, we examined individual item response data, specifically checking to make sure that all values are within expected boundaries, recording ordinal variables so that the lowest response option is 0, recoding missing values into (−99), and collapsing item categories in cases where very low endorsements of extreme response options are present. Second, a first-order common factor model was specified based on a priori theory indicating which items should be designated as the indicators of each of the theoretical domain factors, using information from the instrument developers and the existing literature. Third, as the model in step 2 yielded poor fit and/or uninterpretable results, the focus shifted towards carrying out a more exploratory data-driven approach to identifying statistically supported factor structures. Specifically, we examined inter-item correlations to determine items sets and carefully prune items showing poor measurement properties. This process resulted in taking many items and then getting down to much fewer items. Fourth, we conducted a confirmatory factor analysis (CFA) on this reduced number of items purported to measure the first-order factors. Fifth and finally, we fit alternative models to the final set of items to investigate other plausible structures for accounting for the pattern of item associations present in the COVID domain data. Specifically, we tested (in decreasing order of restrictiveness): 1) a single-factor structure for all the retained COVID items, 2) a hierarchical second-order structure imposed on the first-order factor structure, 3) a correlated first-order factor structure, and 4) a bifactor model in which all items are allowed to load on one general factor and their individual group factors (e.g. each corresponding to individual COVID impact factor). The group factors are allowed to correlate with one another, but their correlations with the one general factor is set to 0.

In order to determine which model fits the data best, we examined substantive interpretability of each model and its loadings, goodness-of-fit indexes (Comparative Fit Index (CFI): ≥ .9, Tucker Lewis Index (TLI): ≥ .9, and Root Mean Square Error of Approximation (RMSEA) ≤ .08; (Hu & Bentler, 1999)), and a ‘covariance’ nesting perspective to test whether more restricted models are nested/equivalent under less restricted ones. Specifically, we used the Nested functionality implemented in Mplus to first evaluate, pairwise, which of the four models are covariance nested. We then set up and ran adjusted chi-square difference tests in Mplus (DIFFTEST for the robust weight least square estimator) to determine whether a more restricted nested model could be retained when compared to the less restricted model. Once a measurement model was decided on using all of these pieces of information, additional modelling was conducted to test for the effects of race/ethnicity and sex on this selected factor structure.

Results

Initial factor analyses of all possible items

After carrying out Step 1 in the flow chart in Figure 1, we carried out Step 2, which specified a first-order common factors based on a priori theory dictating which items should serve as indictors of each of the factors. Specifically, this model, including all 73 items, yielded model results producing several Heywood cases (i.e. negative residual variances resulting from standardized factor loadings estimated to be greater than 1) and overall poor model fit [χ2(1937) = 4129.57, p < .001; LRT/df = 2.1; CFI: .792, TLI: .783, RMSEA: .036]. Thus, we proceeded to Step 3 and shifted to an exploratory factor analytic (EFA) approach. For the full set of 73 raw items, the EFA model optimization produced uninterpretable results with many technical failures (i.e. zero cell warnings and a correlation matrix that produced many negative eigenvalues, indicating a non-positive definite matrix).

Given that neither the initial CFA model or the EFA modelling including all 73 items yielded proper and/or interpretable results, Step 4 was carried out. Specifically, we performed a confirmatory factor analysis (CFA) on a reduced number of items. Correlations among item sets designated to be indicators of a priori determined constructs (factors) were examined. Between item correlations greater than or equal to r = .3 were retained for further investigation. This process yielded eight different item groupings with four-item indicators per set. Three of these eight items sets produced factors having correlations with other factors at or above r = .8, and/or had items with poor item information curves (IICs), which represent the probability that someone will endorse a particular response item (Yang & Kao, 2014). This indicates that one or two items were providing the dominant discriminating information for determining differences on the factor compared to the other items in the indicator set. Specifically, factors with item indicators tapping work/academic changes (e.g. Were classes disrupted?), nature/quality of relationships (e.g. Has quality of friendships changed?), and conflict with/separation from close others (e.g. Has there been an increase in physical conflicts with others in your home?) were dropped due to these poor quality measurement properties. That is, nature/quality of relationships was extremely highly correlated (r > .8) with conflict with others and housing/food stability (the latter of which was retained as one of five final factors), and all three factors were being driven by one (conflict with others, quality of relationships) or two (work/academic changes) items. Thus, the remaining five factors were retained for further examination (below). Table 1 presents the factor names, retained items and their exact wording, original answer choices with the anchors, as well as the collapsed item ranges. Specifically, we collapsed an infrequently endorsed extreme category into the one below it for one item and collapsed two very infrequently endorsed extreme values in a count of symptoms into the third most extreme value for another item. Only two of the 20 items had answer choices that were collapsed. We collapsed in this way because leaving infrequently endorsed response options can introduce zero cells into the cross-classification tables used to estimate polychoric correlations, impacting the WLSMV estimation approach we used. Of the items used in our modes, half had a 0–1 range and the other half were either 0–3 or 0–4.

Modelling of Items to be Retained

The fifth and final step listed in the data analytic procedural flow chart involved fitting alternative plausible models to evaluate the final set of items retained to index the different COVID related domains. Specifically, we estimated the following models starting with the most restrictive a) a single-factor model for all retained items, b) a hierarchical second-order factor model, c) a correlated first-order factor and finally d) the least restrictive (i.e. most estimated parameters) bifactor model (Chen, West, & Sousa, 2006; Reise, 2012). See Figure 2 for visual depiction of the four models tested.

Figure 2.

Visual depiction of models tested

The single-factor CFA, in which all 20 items were treated as indicators of one common factor, produced in a poor fit to the data [χ2(170) = 2621.483, p < .001; LRT/df = 15.4; CFI: 760, TLI: .732, RMSEA: .127]. Thus, a single common factor model solution was unable to adequately account for the patterning of 20 inter-item associations.

A hierarchical second-order factor model was fit next, in which the inter-correlations among the five first-level factors are structured by introducing a single second-order factor to account for these first-order correlations (Dunn & McCray, 2020). Specifically, within this model, each item is allowed to load on its lower/first order factor (e.g. exposure), and each of these lower order factors are regressed onto the higher/second order factor. This model returned a more reasonable fit to the data [χ2(165) = 817.278, p < .001; LRT/df = 5.0; CFI: .936, TLI: .927, RMSEA: .066] with all factor loadings on the higher and lower order factors being statistically significant (p < .001). However, as with the bifactor model, there was a Heywood case in which one item had a standardized loading greater than 1; dropping this item or attempting to set its loading resulted in other items becoming Heywood cases, suggesting that more substantial changes to this model would be needed. Heywood cases are signs of an improper solution, and without making large changes to the model, it did not appear that this second-order factor model would run without a Heywood case. Additionally, even with the Heywood case, several factor loadings for the first- and second-order factors were relatively low. Thus, this model was not considered further.

The multiple first-order factor model (i.e. correlated factor model) was then estimated, with factor inter-correlations allowed (i.e. oblique solution). This model produced reasonable fit [χ2(160) = 789.979, p < .001; LRT/df = 5.0; CFI: .938, TLI: .927, RMSEA: .066]. All factor loadings were significant at p < .001. Thus, there does seem to be evidence that these impact factors ought to be treated as separate, correlated factors. See Table 2 for the specific standardized estimated factor loadings (5 Correlated Factors Model).

Table 2.

Model fitting results for evaluating the structure of 20 S4S online COVID items

Factor	One factor (a)	Higher -order:lower (b)	Higher order-higher	Five factor (c)	Bi-factorgeneral (d)	Bi-factorgroup
Chi-square Value	2621.483	817.278	789.979	313.252
Degrees of Freedom	179	165	160	140
χ2 p-value	0.000	0.000	0.000	0.000
# of Free Parameters	66	71	76	96
CFI	0.760	0.936	0.938	0.983
TLI	0.732	0.927	0.927	0.977
RMSEA	0.127	0.066	0.066	0.037
Any COVID Exposure?	0.348***	0.633***	.269***	0.633***	0.072	.636***
Suspected of COVID	0.950***	1.007***		0.992***	0.094	1.006***
Count of COVID symptoms	0.744***	0.900***		0.914***	0.155**	.887***
Anyone in Household Dx	0.507***	0.774***		0.761***	0.235	0.722***
Worry: Being Infected	0.917***	0.941***	.808***	0.942***	0.300***	0.902***
Worry: Family/Friends Infected	0.762***	0.802***		0.803***	0.261***	0.759***
Worry: Physical Health	0.826***	0.851***		0.850***	0.399***	0.754***
Worry: Mental Health	0.597***	0.649***		0.648***	0.314***	0.566***
Moved Because of COVID	0.040	0.433***	.374***	0.721***	0.267*	1.036***
Concerned about Housing	0.308***	0.742***		0.483***	0.775***	−0.193*
Worry Food Would Run Out	0.249***	0.629***		0.444***	0.641***	−0.310***
Family/Friends Moved In	0.095*	0.460***		0.687***	0.211*	0.638***
Move TV During COVID	0.278***	0.726***	.512***	0.745***	−0.053	0.950***
Social Media During COVID	0.245***	0.669***		0.668***	0.106	0.650***
Video Games During COVID	0.146*	0.426***		0.449***	−0.023	0.492***
Reading/Talking -COVID	0.274***	0.445***		0.425***	0.381***	0.213***
Change in Alcohol Use	0.193***	0.602***	.211***	0.581***	0.014	0.561***
Change in Vaping	0.254***	0.733***		0.705***	0.046	0.706***
Change in Tobacco Products	0.264***	.808***		0.819***	−0.073	0.826***
Change in Marijuana Use	0.243***	.743***		0.777***	−0.012	0.788***

*p < .05, **p < .01, ***p < .001

The bifactor, and most highly parameterized model, in which a ‘general’ factor, as well as ‘group’ factors are specified, was also fit to these item data (Dunn & McCray, 2020). For the bifactor model, the variances of the factors are set to 1, and, by definition, the general and group factors are allowed to be correlated with one another, but their correlations with the general factor are set to 0. Each item loads on the general factor, as well as onto one of the group factors, with the former representing what is common among all the indicators and the group factors structuring the residual associations not accounted for by the general factor. While this model fit the data well [χ2(140) = 313.352, p < .001; LRT/df = 2.2; CFI: .983, TLI: .977, RMSEA: .037], there were multiple Heywood cases in which items on the group factors had standardized loadings greater than 1. Additionally, while all of the loadings on the group factors were significant (p < .05), many of the loadings on the general factor were small and not statistically significant. Thus, the general factor within in the bifactor model is not well identified, with most of the structural information being modelled by the group factors.

Nested model testing

Using the Nested functionality, as can be seen in Table 3, all six of the pairwise-nested model comparison tests returned very small fit function values (column 4) that indicated the more restricted models are covariance nested within their less restricted counterparts. Next, the results of the chi-square difference tests suggested that in all cases, the null hypotheses of no discernable differences in fit between the compared models was rejected. That is, the bifactor and five-factor models fit better than the single factor and higher order models, with the bifactor model showing marginal improvement over the five factor as well. However, the interpretation of this bifactor model is unclear. Specifically, when examining the estimated factor loadings of the Bifactor model, we see that only 10 of the 20 factor loadings on the general factor are significantly different from zero. Additionally, eight have point estimates that are essentially zero and only 2 out of the 20 exceed the .4 threshold guideline suggested as a cut-off for determining a weak versus not weak loading on a factor (Garson, 2010). Additionally, all 20-item indicators had salient loadings on their respective individual group factors with the exceptions of the 2 indicators for the housing/food group factors that were the only strong markers of the general factor. This patterning of factor loadings is contrary to what is typically found when fitting the bifactor model to data. We also note that there is a growing literature recognizing the trade-offs and compromise between model choices based exclusively on omnibus model fitting results and the substantive interpretation of estimated model parameters given the research questions (Murray & Johnson, 2013; Reise, Kim, Mansolf, & Widaman, 2016). Thus, we decided to retain the five-factor model, given that it performed objectively better than the single-factor and higher order models and is more substantively interpretable than the bifactor model.

Table 3.

Summary of nested and difference model testing results

NestedTest	H0 (Nested)(ep–df)	H1 (Compare)(ep–df)	Models nested?Fit function	H0 Fit indices	H1 Fit indices	Chi-square Difference TestGoodness-of-fit Differences
1	Single Factor(66–170)	Higher Order(71–165)	Yesε = 0.00000002	χ2 = 2621.48 (170) p < .001CFI = 0.760TLI = 0.732RMSEA = 0.127 (.123-.131)	χ2 = 817.28 (165) p < .001CFI = 0.936TLI = 0.926RMSEA = 0.066 (.062-.071)	χ2diff = 1048.23 (5) p < .001ΔCFI = 0.176ΔTLI = 0.194ΔRMSEA = – 0.061
2	Single Factor(66–170)	Corr. 5 Factor(76–160)	Yesε = 0.00000000	χ2 = 2621.48 (170) p < .001CFI = 0.760TLI = 0.732RMSEA = 0.127 (.123-.131)	χ2 = 789.98 (160) p < .001CFI = 0.938TLI = 0.927RMSEA = 0.066 (.062-.071)	χ2diff = 909.23 (10) p < .001ΔCFI = 0.178ΔTLI = 0.195ΔRMSEA = – 0.061
3	Single Factor(66–170)	Bifactor(96–140)	Yesε = 0.00000000	χ2 = 2621.48 (170) p < .001CFI = 0.760TLI = 0.732RMSEA = 0.127 (.123-.131)	χ2 = 313.36 (140) p < .001CFI = 0.983TLI = 0.977RMSEA = 0.037 (.032-.043)	χ2diff = 1465.09 (30) p < .001ΔCFI = 0.223ΔTLI = 0.245ΔRMSEA = – 0.090
4	Higher Order(71–165)	Corr. 5 Factor(76–160)	Yesε = 0.00000000	χ2 = 817.28 (165) p < .001CFI = 0.936TLI = 0.926RMSEA = 0.066 (.062-.071)	χ2 = 789.98 (160) p < .001CFI = 0.938TLI = 0.927RMSEA = 0.066 (.062-.071)	χ2diff = 38.45 (5) p < .001ΔCFI = 0.002ΔTLI = 0.001ΔRMSEA = 0.000
5	Higher Order(71–165)	Bifactor(96–140)	Yesε = 0.00000000	χ2 = 817.28 (165) p < .001CFI = 0.936TLI = 0.926RMSEA = 0.066 (.062-.071)	χ2 = 313.36 (140) p < .001CFI = 0.983TLI = 0.977RMSEA = 0.037 (.032-.043)	χ2diff = 348.46 (25) p < .001ΔCFI = 0.047ΔTLI = 0.051ΔRMSEA = – 0.029
6	Corr. 5 Factor(76–160)	Bifactor(96–140)	Yesε = 0.00000000	χ2 = 789.98 (160) p < .001CFI = 0.938TLI = 0.927RMSEA = 0.066 (.062-.071)	χ2 = 313.36 (140) p < .001CFI = 0.983TLI = 0.977RMSEA = 0.037 (.032-.043)	χ2diff = 327.82 (20) p < .001ΔCFI = 0.045ΔTLI = 0.050ΔRMSEA = – 0.029

H0 = designated nested model (restricted: fewer parameters), H1 = designated comparison model (unrestricted: more parameters). ep = number of estimated parameters, df = number of degrees of freedom, χ2 = chi-square of model fit, CFI = goodness-of-fit comparative fit index, TLI = goodness-of-fit Tucker–Lewis fit index, RMSEA = Root mean square error of approximation index, χ2diff = chi-square difference test, ΔCFI = comparative goodness-of-fit index difference (H1 – H0), ΔTLI = Tucker–Lewis goodness-of-fit index difference (H1 – H0), ΔRMSEA = Root mean square error of approximation index difference (H1 – H0).

Five factor model

In general, the inter-correlations among the first-order factors were small aside from the exposure and change in substance use (r = .77) and worry and change in media use (r = .44) factors. All but one of these inter-factor correlations were positive with most being significantly different from zero. In terms of the negative correlation, those with more housing and food-related concerns reported less substance use during COVID, compared to pre-COVID. In terms of non-significant positive effects, COVID exposure was uncorrelated with change in media consumption and only marginally significantly associated with housing and food stability (p = .055). See Table 4 for correlations among these factors. In general, these COVID domain measurement factors were relatively distinct, suggesting that in this college student sample the impacts of the COVID pandemic are playing out as domain-specific rather than global effects. This is further supported by the lack of evidence for an interpretable single common factor in both the second-order and bifactor models Table 5.

Table 4.

Correlations among five factors

	1.	2.	3.	4.
1. COVID exposure	–
2. COVID worry	.199***	–
3. COVID housing/Food concern	.115 (p = .055)	.263***	–
4. Change in media use during COVID	.084	.413***	.139*	–
5. Change in substance use during COVID	.777***	.162***	−.238***	.153*

*p < .05, **p < .01, ***p < .001.

Table 5.

Structural equation model results

	COVID Exposure		COVID Worry		Food/Housing Concern		Media Increase		Substance Use Increase
Predictors/Item Loadings	Std. Estimate	SE Est.	Std. Estimate	SE Est.	Std. Estimate	SE Est.	Std. Estimate	SE Est.	Std. Estimate	SE Est.
Item 1	.641***	.044	.942***	.009	.708***	.052	.739***	.061	.609***	.044
Item 2	.992***	.031	.803***	.014	.504***	.046	.661***	.056	.688***	.045
Item 3	.913***	.030	.848***	.012	.468***	.061	.434***	.093	.826***	.048
Item 4	.756***	.085	.653***	.022	.668***	.051	.440***	.057	.763***	.042
Sex	.027	.046	−.161***	.037	−.148***	.041	−.047	.050	.091*	.043
Race: White-Black (WB)	−.176**	.062	−.046	.041	.060	.065	−.097	.053	−.225*	.089
Race: White-Asian (WA)	−.104	.065	−.004	.042	.192***	.055	−.084	.057	−.373***	.080
Race: White-Other (WO)	−.040	.072	.081*	.037	.096	.063	.061	.058	−.064	.105

***p < .001, **p < .01, *p < .05; standardized estimates and SE of estimates pertain to factor loadings for the measurement portion of the model and for coefficients for the predictive portion of the mode. See Table 1 above for which items correspond to which item numbers. Sex is coded such that 0 = female, 1 = male. Race is coded such that for all, 0 = White, and for WB: 1 = Black, for WA: 1 = Asian, and WO: 1 = Other.

Prediction of factors from sex and race

A structural equation model in which the five correlated factors are regressed onto the covariates of sex and race demonstrated decent fit to the data, χ2(220) = 926.756, p < .001; LRT/df = 4.2; CFI: .932, TLI: .914, RMSEA: .060. See Table 4 for full findings. In terms of predictors of COVID exposure, Whites, on average, reported greater exposure than Blacks. There were no other race differences and no sex mean differences. In terms of worry, females reported more worry, and those in the Other group reported more worry than Whites. There were no other race differences. In terms of housing and food concern, females reported more concern, and Asians reported more than Whites, but there were no other race differences. In terms of change in media use, there were no significant race or sex differences. In terms of change in substance use, males reported more increase in substance use, and Whites reported more increase than Blacks or Asians, but there was no difference between Whites and Others.

Discussion

The overarching goal of this study was to investigate the structural organization of specific domains of COVID-19 related impact. To achieve this, the present study utilized a systematic model testing approach to determine what latent variable model best accounted for the item associations used to assess the impact of COVID-19. Existing rapid research studies related to COVID-19 thus far have assumed a uniform trauma exposure, or have included a limited number of items that assess severity; these approaches do not consider dosage and other environmental and contextual factors (Luo et al., 2020; Xiong et al., 2020). Our results indicate that the impact of COVID-19 is better explained by five correlated factors. This model had superior fit and interpretability to all other models tested, suggesting that there is not a general higher order factor that explains impact, nor is the pandemic impact best explained by a single continuum of severity.

The five-factor correlated model had a reasonable overall model fit, and all standardized loadings on each factor were above .4. There was no additional benefit from creating a higher order or general factor. These results indicate there is not an overarching COVID-19 impact, rather there are discrete impacts of various COVID-related factors. This is consistent with other traumatic stress research on mass crises, which indicates that there is variability among the trauma exposed due to dosage and context (Comtesse et al., 2019; Rhodes et al., 2010; Schwartz et al., 2016).

The final five-factor model was comprised of the following factors: exposure, worry, housing/food concerns, change in media, and change in substance use due to COVID. Although initial steps in the analysis process yielded factors tapping work/academic disruption, relationship quality change, and conflict/separation from close others, these constructs were redundant with other constructs and/or were being fully explained by one or two items. Any further modelling in which predictors of these factors, or outcomes from these factors were included would likely have yielded significant model estimation issues (in case of collinearity) or provide misleading results (in case of factor being driven by one or two items). It is possible that these factors might be more distinct and/or robust in other populations in which the nature of academic, social, and food/housing changes are less intertwined (i.e. college students were all told to leave school/friends/dining and residence halls and go home to their families at a specific time – disrupting all of these processes simultaneously). The inter-correlations among factors were modest, indicating these factors are relatively distinct, but related domains. The factors were generally positively correlated, with the exception of food/housing concern and substance use change during COVID. This negative correlation could be explained by those with more housing- and food-related concerns having less money to spend on substance use habits, thus using less substances compared to their pre-COVID habits. All factors were significantly correlated with the exception of COVID exposure with change in media consumption and housing/food concerns. However, both were highly significantly correlated with COVID worry. This could potentially be due to the fact that data were collected in the earlier phases of the pandemic, prior to case rate being high among the college demographic. Further, data collection occurred when the university was operating virtually. COVID cases in the area did not begin to rise among college students until the end of summer when students began physically returning to the area. Perhaps exposure was less salient than the worry of the pandemic at large for these participants. This calls for a critical future step in this line of research: the need to compare COVID’s impact in the early versus more advanced stages of the pandemic when the infection rates are significantly higher and exposure is more salient.

Finally, sex and ethnicity/race were examined as predictors of the final five-factor model. In terms of sex, women experienced significantly more COVID-related worry and more food/housing instability than men. Men experienced a significant increase in substance use compared to pre-COVID use than women did. These findings are supported by existing COVID research which has consistently found that globally, women are experiencing higher levels of stress and mental health symptomology than men ((González-Sanguino et al., 2020) [Spain]; (Mazza et al., 2020) [Italy]; (Odriozola-González et al., 2020) [Spain], (Wang et al., 2020) [China]). To our knowledge, the sex differences related to change in substance use due to COVID is a novel finding and is important to further investigate given the mixed overall reporting of changes in substance use behaviours (e.g. increased alcohol poisoning (Rostami, 2020); overall substance use decrease, frequency of alcohol and cannabis use increase (Dumas, Ellis, & Litt, 2020) and long-term nature of this pandemic.

There were relatively few significant differences predicted by race/ethnicity, however important differences did exist. White participants reported higher COVID exposure than Black participants and increased their substance use during the pandemic at a higher rate than Black and Asian participants. Asian participants had more food/housing concerns than White participants, and the ‘Other’ group comprised of individuals who were not White, Black, or Asian, experienced more COVID worry than White participants. Taken together, these results support that COVID is impacting different racial/ethnic communities in different, and often disproportionate ways (Bowleg, 2020; Selden & Berdahl, 2020).

Limitations and future directions

While this study extends the literature on measuring the discrete impacts of COVID, it is not without limitations. While this study utilized a sample that matched the racial/ethnic makeup of the broader community from which it came, it is limited to students affiliated with one Mid-Atlantic university. This is an important demographic to study, given global COVID research has shown that younger age is significantly associated with higher stress (Mazza et al., 2020), increased anxiety and depression (Odriozola-González et al., 2020) and that the 18–25 age bracket is particularly vulnerable to COVID-related mental health impacts (Ozamiz-Etxebarria, Dosil-Santamaria, Picaza-Gorrochategui, & Idoiaga-Mondragon, 2020). Yet, there may be important differences between college students and their non-college peers that limit generalizability of the present study to all young adults. Given the prolonged nature of the pandemic in the USA, the continued COVID impacts on young people’s psychological, social, and physical health should be investigated longitudinally. Future studies should test the generalizability of the resulting factor structure on a non-college aged sample as well as young adults not in college. Additionally, although outside the scope of this analytically complicated paper, future work should examine whether these correlated but distinct factors may display measurement invariance by demographic factors (e.g. race, sex), as well as examine how these domains of impact influence mental health and substance use outcomes. One final limitation that should be mentioned is that the model that fit reasonably well and substantively was the most interpretable, and that we believe represented the data most accurately, did not objectively fit the data best. Additional research is needed to attempt to replicate or extend these findings.

While race/ethnicity was collapsed into four categories to maximize power to detect differences between groups, a limitation of this study is that this coding did not allow for specific differences to be detected beyond White, Black, and Asian participants. Multi-racial, Hispanic/Latino, and other Indigenous populations were analysed as one group, thus differences between these groups and among the full sample cannot be parsed out. This is an important limitation to consider in future research, given the disproportionate impact of COVID on communities of colour (Bowleg, 2020; Selden & Berdahl, 2020). This research should be extended by utilizing invariance testing across race/ethnicity and sex to determine if the structure of COVID impact is different across these groups. Additionally, given that COVID impact may vary across time, it will be important to measurement invariance of the factor structure across the different stages of the pandemic in the full sample as well as across race/ethnicity and sex.

Conclusion

Despite these limitations, the present study is the first, to our knowledge, to examine the factor structure of items used to assess COVID-19 impact. Through a series of model testing procedures, the results of this study suggest that there is little evidence supporting a single overarching dimension of COVID-19 liability impact. Rather, our findings indicate that COVID-19 impact is better viewed as playing out within relatively distinct but correlated domains of behaviours and situations. This suggests that other researchers should analyse data on the pandemic in order to determine the domains of impact of this heterogeneous effects of the stressor. This study is the first step in determining the discrete impact of COVID factors to conceptualize the impact of dosage on traumatic stress. These findings have important implications for future research to examine how the final five-factor COVID impact structure predicts risk of mental health symptomology and other adverse sequelae of the pandemic at large.