The latent and item structure of COVID-19 fear: A comparison of four COVID-19 fear questionnaires using SEM and network analyses.

Since the outbreak of the coronavirus disease (COVID-19), several reports have shown that fear relating to COVID-19 has sharply increased. To measure fear of COVID-19, various questionnaires have been developed in parallel. However, fear concerning COVID-19 is not necessarily a uniform construct and the different questionnaires may cover diverse aspects. To examine the underlying structure of fear of COVID-19, we conducted structural equation modelling and network analyses on four scales in an online convenience sample (N = 829). Particularly, the Fear of COVID-19 Scale (Ahorsu et al., 2020), the Fear of the Coronavirus Questionnaire (Mertens et al., 2020), and the COVID Stress Scales (Taylor, Landry, Paluszek, Fergus et al., 2020, Taylor, Landry, Paluszek, Rachor et al., 2020) were included in our study, along with a new scale that also assessed socio-economic worries relating to COVID-19. We found that fear of COVID-19 was best classified into four clusters: Fear of health-related consequences, fear of supplies shortages and xenophobia, fear about socio-economic consequences, and symptoms of fear (e.g., compulsions, nightmares). We also find that a central cluster of items centered on fear of health, which likely represents the core of fear of COVID-19. These results help to characterize fear due to COVID-19 and inform future research.

Introduction

The ongoing coronavirus disease (COVID-19) pandemic is having a tremendous impact on our lives. In addition to having over 100 million confirmed cases and more than 2.5 million deaths at the time of writing (March 2021), millions of people worldwide are affected by lockdowns that interfere with daily life and employment. Logically, many researchers and clinicians have warned that the COVID-19 pandemic may lead to further mental health issues such as anxiety-related disorders, depression, obsessive-compulsive disorders, and post-traumatic stress (Asmundson & Taylor, 2020; Galea et al., 2020; Lu et al., 2020; Satici, Saricali, et al., 2020). In support of these concerns, Google searches relating to “anxiety attack” and “panic attack” substantially increased by over 50 % in the first weeks of the pandemic (Ayers et al., 2020), and the rate of people with anxiety and depressive disorders in the US more than tripled in April and May of 2020 compared to the rates in 2019 (Twenge & Joiner, 2020).

One central emotion that may explain the link between the COVID-19 pandemic and its mental health consequences is fear. Fear is a negative emotion elicited by real or perceived threat and is characterized by increased arousal, behavioral tendencies (e.g., fleeing), and negative apprehensions (e.g., worrying) (Lang, 1968). To monitor fear for COVID-19, several research groups have developed various questionnaires in parallel. The first published questionnaire was the Fear of COVID-19 Scale (FCV-19S) (Ahorsu et al., 2020). This scale has since been translated into several languages including Turkish, Italian, Arabic, Russian, Hebrew, and Bangla (Pakpour et al., 2020). Soon thereafter, several other questionnaires were published such as the COVID Stress Scales (CSS) (Taylor, Landry, Paluszek, Fergus et al., 2020) and the Fear of the Coronavirus Questionnaire (FCQ) (Mertens et al., 2020).

However, as fear can be elicited by a wide range of threats and given that COVID-19 is affecting our lives in many ways (e.g., on a social, economic, relational, and professional level), fears elicited by COVID-19 may be considerably heterogeneous. Indeed, initial validation of the factor structure of the CSS showed 5 factors of stress and worries relating to COVID-19: (1) danger and contamination fears, (2) fears about economic consequences, (3) xenophobia, (4) compulsive checking and reassurance seeking, and (5) traumatic stress symptoms (Taylor, Landry, Paluszek, Fergus et al., 2020). Furthermore, in a study conducted days after the World Health Organization declared COVID-19 a worldwide pandemic (March 2020), we found that participants freely reported a wide range of concerns and fears relating to the COVID-19 pandemic in an open-ended question (Mertens et al., 2020). Particularly, 16 different topics of worries and fear were identified that ranged from fear about one’s own health and the health of loved ones, to worries about the imposed lockdowns, shortage of supplies, travel restrictions, and fake news. These studies indicate that COVID-19 related fears and worries are considerably heterogeneous and that the various questionnaires probably measure different types of fear and worries relating to COVID-19.

Two psychometric techniques can help to uncover which constructs COVID-19 related fear questionnaires precisely measure. First, Structural Equation Modelling (SEM) is a statistical technique that can be used to identify latent factors that explain the associations between the item scores of each COVID-19 fear questionnaire. This also allows for estimating overarching higher-order latent factors that may point to the similarity or distinctiveness of the identified latent COVID-19 fear constructs. A second, more recently developed psychometric technique called network analysis can be used to investigate the network structure of the different items in the various COVID-19 fear questionnaires. According to network theory, psychological constructs can be seen as networks of symptoms that causally influence each other. Network analysis allows for examining and identifying items that play a central role in such causal networks. As such, both techniques can provide complementary information, with SEM providing information about the number of latent factors underlying the different questionnaires and network analysis providing information about the item structure and potential causal pathways between the items of the different questionnaires. In this study we make use of both psychometric techniques to investigate how the items of the various COVID-19 fear questionnaires relate to each other.

Prior work has already investigated the network structure of the CSS and several COVID-19 related constructs such as belief in conspiracy theories, avoidance behaviors, disregard for social distancing, and hygienic behaviors. The most central node in this network were worries about the dangerousness of COVID-19. Other important hubs of nodes were disregard for social distancing and compulsive checking and reassurance seeking (Taylor, Landry, Paluszek, Rachor et al., 2020). Furthermore, prior work has performed confirmatory factor analyses to establish the factor structure of single questionnaires of fear of COVID-19 (Pakpour et al., 2020; Taylor, Landry, Paluszek, Fergus et al., 2020). However, to our knowledge, no prior studies have directly compared different fear of COVID-19 questionnaires with one another. This is important, as it can inform the literature on the overlap and differences between the questionnaires.

In addition, it is interesting to examine how different types of fear of COVID-19 relate to inter-individual differences. Particularly, for treatment and prevention of mental health issues relating to fear of COVID-19, it is important to understand inter-individual differences in sensitivity to increased anxiety and worries (Asmundson & Taylor, 2020). Several psychological constructs have been related to increased risk for anxiety-related disorders. Here, we will focus on two such psychological vulnerability factors: health anxiety and intolerance of uncertainty. Health anxiety is the tendency to misidentify normal or benign physical symptoms as indicating that one has or is acquiring a serious illness, in the absence of any actual illness (Abramowitz et al., 2007; Salkovskis & Warwick, 2001). Intolerance of uncertainty refers to the tendency to find uncertain situations and lack of information intolerable, which triggers negative feelings (Carleton, 2016). It is conceivable that people scoring higher on health anxiety will be more concerned with the physical health consequences of COVID-19, whereas people scoring higher on intolerance of uncertainty may be more concerned with the uncertainty of the further development of the pandemic and its socio-economic consequences. Indeed, multiple previous studies have found a relationship between fear of COVID-19 and health anxiety (Fedorenko et al., 2021; Jungmann & Witthöft, 2020; Mertens et al., 2020; Sauer et al., 2020; Wheaton et al., 2021) and between fear of COVID-19 and intolerance of uncertainty (Bakioğlu et al., 2020; Elsharkawy & Abdelaziz, 2020; Fedorenko et al., 2021; Satici, Saricali, et al., 2020; Wheaton et al., 2021). However, it remains unclear how these two anxious personality traits relate to different types of fear of COVID-19.

Taken together, the aim of this study was to investigate the latent factor (using SEM) and item structure (using network analysis) of three existing fear of COVID-19 questionnaires (i.e., FCV-19S, CSS, and FCQ) and one novel questionnaires (i.e., a questionnaires based on the 16 COVID-19 related worries identified in Mertens et al., 2020) and to examine their relationship with intolerance of uncertainty and health anxiety. Therefore, we conducted a large online survey (N = 829) in which an unselected sample completed the different COVID-19 questionnaires, the Short Health Anxiety Inventory and the Intolerance of Uncertainty scale. It is worth noting that for reasons of brevity and feasibility, we did not include all the questionnaires that have recently been developed to measure COVID-19 related fear, such as the Coronavirus Anxiety Scale (Lee, 2020), the COVID-19 Phobia Scale (Arpaci et al., 2020), or the Multidimensional Assessment of COVID-19-Related Fears (Schimmenti et al., 2020).

Methods

Sample and sample size determination

Participants were recruited through Prolific in July 2020 (see Table 1 for a detailed overview of the demographics of our sample). Participation was on a voluntary basis and they were compensated according to the standard rate of Prolific (£7.5/h). The study was approved by the Ethics Committee of the Tilburg School of Social and Behavioral Sciences (reference code: RP216).

Table 1

Demographic information of the respondents (total N = 829).

	N	%
Age in years
16–20	141	17.0 %
21–30	384	46.3 %
31–40	187	22.6 %
41–50	67	8.1 %
51–60	31	3.7 %
61–70	15	1.8 %
71–80	4	0.5 %
Gender
Male	435	52.5 %
Female	393	47.4 %
Prefer not to say	1	0.1 %
Highest education
Less than High School	16	1.9 %
High School diploma	327	39.4 %
College degree	316	38.1 %
Master’s degree	151	18.2 %
Doctorate (PhD or equivalent)	19	2.3 %
Country of residence by region1
Asia (incl. India)	2	0.2 %
Australia/New-Zealand	14	1.6 %
Europe (incl. Russia)	783	93.2 %
Middle-East (incl. Israel)	1	0.1 %
North-America	34	4.0 %
South-America	1	0.1 %
Sub-Sahara Africa	7	0.8 %
Chronic illness
Yes	127	15.3 %
No	702	84.7 %
Infected by the coronavirus?
Yes	19	2.3 %
No	749	90.3 %
Unsure	61	7.4 %
Know someone infected?
Yes	323	39.0 %
No	485	58.5 %
Unsure	21	2.5 %

Note:1Full list of countries of residence: Australia, Austria, Azerbaijan, Belgium, Canada, Chile, Czech Republic, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Israel, Italy, Japan, Mexico, Netherlands, New-Zealand, Norway, Poland, Portugal, Slovakia, Slovenia, South-Africa, Spain, Sweden, Switzerland, United Kingdom, USA.

Our sample constitutes a voluntary convenience sample (i.e., prolific users could freely decide to participate in this study) and is not necessarily representative of any particular population. Indeed, our sample was overall quite young (94 % was ≤ 50 years old) and well educated (58.6 % had at least a college degree). Gender was fairly well balanced between men and women (see Table 1). Finally, the vast majority of our sample were residents in European (93.2 %) or North-American (4 %) countries. For these countries, the number of cases, hospitalizations, and restrictions due to COVID-19 were comparable and relatively mild, as the spread of the SARS-CoV-2 virus was relatively suppressed during the summer months in the Northern hemisphere.

The main statistical methods used in this study were confirmatory factor analysis and network modeling. Earlier research has indicated that at least 500 participants are needed to adequately estimate these models when using diagonally weighted least squares estimation to handle the ordered categorical item scores (Forero et al., 2009). The R-package Bootnet (Epskamp & Fried, 2018) was used to conduct a Monte Carlo power analysis to determine the sample size required to estimate the network structure underlying the questionnaires with sufficient precision. Supplemental Fig. 1 shows the results of this power analysis for various sample sizes, suggesting that our current sample of 829 participants likely resulted in a correlation between the estimated and true network of .92. The sensitivity to detect true network edges was estimated at .73, whereas the specificity to omit non-existing edges was estimated at .90. The correlation between the true and estimated centrality indices was approximately .70. A sample size of 1500 would have increased the sensitivity and the correlation between the true and estimated centrality indices to .80.

Careless responding by participants was checked using an attention check in which participants had to select a particular response option (i.e., “please indicate ‘somewhat disagree’”). The vast majority of participants (97.6 %) did this correctly. Twenty (2.4 %) participants selected a different response option. However, inspection of their data revealed normal response patterns on the other questionnaires and thoughtful responses to the open-ended questions. Therefore, the data of these 20 participants were retained for the final analyses.

Materials & procedure

Measures

Fear of the Coronavirus Questionnaire (FCQ) (Mertens et al., 2020)

The FCQ consists of eight items. Respondents were asked to rate their level of agreement with each statement on a 5-point Likert scale (1 = “Strongly disagree”, 5 = “Strongly agree”; possible range: 8–40). Examples of the items are: “I am very worried about the coronavirus”, “I am taking precautions to prevent infection (e.g., washing hands, avoiding contact with people, avoiding door handles)”, and “I am constantly following all news updates regarding the virus”. These items were chosen because they correspond with different fear components, such as subjective experiences (worrying), attentional biases, and avoidance behaviors (Lang, 1968). The internal consistency of the FCQ was acceptable (Cronbach’s alpha = 0.77).

Fear of COVID-19 Scale (FCV-19S) (Ahorsu et al., 2020)

The fear of COVID-19 Scale consists of seven items (e.g., “I cannot sleep because I am worried about getting COVID-19″). Each of the items are rated on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree) with a possible range from 7 to 35. The higher the score, the higher the level of fear of COVID-19. Cronbach’s alpha in the current sample was good (α = .86).

COVID stress scales (CSS) (Taylor, Landry, Paluszek, Fergus et al., 2020, Taylor, Landry, Paluszek, Rachor et al., 2020)

The COVID Stress Scales consists of 36 questions and is divided into six subscales with 6 items each: (1) COVID danger (e.g., “I am worried about catching the virus”), (2) COVID fears about economic consequences (e.g., “I am worried about grocery stores running out of food”), (3) COVID xenophobia (e.g., “I am worried that foreigners are spreading the virus because they’re not as clean as we are”), (4) COVID contamination fears (e.g., “I am worried that people around me will infect me with the virus”), (5) COVID traumatic stress symptoms (e.g., “I thought about the virus when I didn’t mean to”), and (6) COVID compulsive checking and reassurance seeking (e.g., “Searched the Internet for treatments for COVID-19”). The subscales 1–3 are rated on a 5-point Likert scale from 1 (not at all) to 5 (extremely), and the subscales 4 and 5 are rated from 1 (never) to 5 (always). Cronbach’s alpha yielded 0.95 for the total score, and .89, .92, .92, .93, .90 and .82 for subscales 1–6, respectively.

Newly developed scale (based on Mertens et al., 2020)

This scale consisted of 16 statements based on the coded open-ended answers in Mertens et al. (2020). These 16 statements are listed in Table 2 (together with items of all other questionnaires used in this study). The statement were answered with a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Horn’s parallel analysis was performed to identify the optimal number of factors underlying these 16 item scores. Based on both the scree plot and Horn’s parallel analysis a two-factor structure best described the correlations between the items (see Supplemental Fig. 2). These two factors were labeled “fear of health” and “fear of socio-economic consequences” and used in further analyses. Cronbach’s alpha yielded 0.85 and 0.80 for these two subscales, respectively.

Table 2

Overview and factor structure for all included scales.

Scale	Number of items	Sample item	Factor(s)	Factor label(s)
FCQ	8	“I am very worried about the corona virus outbreak.”	1	Fear of COVID
FCV-19S	7	“I am most afraid of coronavirus-19”	1	Fear of COVID
CSS	6	“I am worried about catching the virus”	1	Danger
CSS	6	“I am worried about grocery stores running out of food”	2	Socio-economic consequences
CSS	6	“I am worried that foreigners are spreading the virus in my country”	3	Xenophobia
CSS	6	“I am worried that people around me will infect me with the virus”	4	Contamination
CSS	6	“I had trouble sleeping because I worried about the virus”	5	Traumatic stress
CSS	6	“Checked social media posts concerning COVID-19”	6	Checking
NEW	7	“I am worried about vulnerable loved ones (e.g., parents, grandparents) becoming infected by the coronavirus”	1	Fear of health
NEW	9	“I am worried that the economy will collapse because of COVID-19”	2	Broad socio-economic consequences
IUS-12	7	“Unforeseen events upset me greatly.”	1	Prospective anxiety
IUS-12	5	“Uncertainty keeps me from living a full life.”	2	Inhibitory anxiety
SHAI	14	“Time spent worrying about health”	1	Illness likelihood
SHAI	4	“Ability to enjoy life if have an illness”	2	Illness severity

Note: FCQ = Fear of the Coronavirus Questionnaires; FCV-19S = Fear of COVID-19 Scale; CSS = COVID Stress Scales; NEW = new scale based on self-reported concerns about COVID-19 in Mertens et al. (2020); IUS-12 = short version of the Intolerance of Uncertainty Scale; SHAI = Short Health Anxiety Inventory.

Intolerance of uncertainty scale

Intolerance of uncertainty (IU) was measured using the short version of the Intolerance of Uncertainty scale (IUS-12) developed and validated by Carleton et al. (2007), which assesses an individual’s propensity to find uncertain situations unpleasant. It consists of 12 statements scored on 5-point Likert scales (1 = “Not at all characteristic of me”, 5 = “Entirely characteristic of me”). Examples of the statements are: “Unforeseen events upset me greatly”, “It frustrates me not having all the information I need”, and “Uncertainty keeps me from living a full life”. This scale is often subdivided into two subscales: prospective IU and inhibitory IU. However, it is commonly recommended to use total scoring for this scale (Huntley et al., 2020; Lauriola et al., 2016). The internal consistency in the current sample was excellent for the overall scale (Cronbach’s alpha = 0.91) and good for both subscales (prospective IU: 0.85; inhibitory IU: 0.86).

Short health anxiety inventory

The Short Health Anxiety Inventory (SHAI) was used to evaluate individuals’ tendency to worry about their health (Abramowitz et al., 2007; Salkovskis et al., 2002). It consists of 18 four-choice questions. An example item is: “1 = I do not worry about my health; 2 = I occasionally worry about my health; 3 = I spend much of my time worrying about my health; 4 = I spend most of my time worrying about my health”. Typically, this scale is divided into two subscales: Illness likelihood and illness severity (Alberts et al., 2013). The internal consistency was good for the illness likelihood subscale (Cronbach’s alpha = 0.89) and acceptable for the illness severity subscale (Cronbach’s alpha = 0.71).

Demographic information

As demographic predictors, respondents were asked to indicate the gender they identify with the most (“male”, “female”, “prefer not to say”), their age (in decade categories), their highest educational level obtained (from” less than high school degree” to “Doctorate (PhD or equivalent)”), whether they already got infected by the virus (“yes”, “no”, “unsure”), whether they knew anyone that is/was infected by the virus (“yes”, “no”, “unsure”), and their country of residence.

Survey administration

All questionnaires described above were delivered through an online survey using the Qualtrics platform (https://www.qualtrics.com/) and posted on the Prolific participants platform (https://www.prolific.co/). The complete survey took approximately 15 min to complete.

Data analysis strategy

All analyses were conducted using the freely available software R studio (Version 1.3.1073). For each questionnaire, separate confirmatory factor analyses (CFA) were first conducted to investigate whether the factor structure proposed by the questionnaire developers showed a good fit to the current data. The R-package Lavaan (Version 0.6–7; Rosseel, 2012) was used to estimate these CFAs, using diagonally weighted least squares estimation (DWLS) and polychoric threshold parameters to model the ordered categorical item scores. The fit of each CFA model was evaluated based on several fit indices (RMSEA < .06; SRMR < .08; CFI > .95; Hooper et al., 2008; Hu & Bentler, 1999). We could replicate the proposed factor structures of the included questionnaires. For brevity, the results of these analyses are provided in the Supplementary Materials.

Next, the measurement models of all questionnaires were entered simultaneously in a correlated factor model involving 14 latent factors (i.e., the latent factors of the used questionnaires previously identified in the literature; see Table 2), to estimate the latent correlation between all constructs required for the subsequent higher-order EFA. The advantage of modeling a latent correlation matrix compared to a matrix consisting of correlations between total scale scores is that total score correlations are typically attenuated due to the measurement error in the questionnaire item scores. Latent correlations are not affected by this problem and therefore paint a less biased picture of the associations between these constructs.

This latent correlation matrix was subsequently used in an exploratory factor analysis (EFA) using promax rotation to examine whether models ranging between 1 and 6 higher order latent factors adequately explained the association between the latent constructs measured by each of the questionnaires (Asparouhov & Muthén, 2009; Marsh et al., 2014). The optimal number of higher order latent factors identified by the EFA was determined based on the extended Bayesian information criterion (EBIC), with lower values indicating better model fit. For each EFA, factor loadings lower than 0.30 were omitted from the model and the resulting factor structures were modeled in six second-order CFAs. Second-order CFAs simultaneously estimate the measurement models of each latent construct, as well as the second-order factor structure that connects these constructs to the higher-order latent factors identified in the EFA. Ordered categorical item scores were again modeled using DWLS estimation and the model fit was evaluated in terms of the RMSEA, SRMR and CFI.

Network analyses were conducted and visualized using the R-package qgraph (Version 1.6.5; Epskamp et al., 2012) to explore and describe the associations between the item scores of the various questionnaires. Both correlation as well as regularized partial correlation networks were estimated based on the polychoric correlation matrix to handle the ordered categorical item scores. Partial correlation networks estimate the association (edges) between two items (network nodes) adjusted for the influence of all other items in the network. Such networks may give insight into possible mediating or causal connections between items. When estimating the partial correlation networks, the hyperparameter (gamma) controls whether the EBIC fit index should prefer sparse or dense networks. This hyperparameter was set at 0.5, thereby lowering the chance on spurious edges at the cost of not detecting true edges when they are present (Epskamp & Fried, 2018). For each item (node) in the network, we reported the item centrality indices closeness (average distance to other nodes), betweenness (how often is a node in the shortest path connecting two other nodes) and strength (average absolute connection weight).

Results

Data availability

The data files and data analysis syntax of the results reported here can be obtained through the Open Science Framework (https://osf.io/fxd7c/).

Second-order EFAs & CFAs on the latent factors of the questionnaires

Table 3 presents the results of the higher-order CFAs that were fitted to investigate whether one or more higher-order constructs adequately explained the latent correlations between the fear constructs discussed in the previous section. First, EFAs using promax rotation were applied to the latent correlation matrix with models ranging from 1 to 6 higher order factors. Factor loadings lower than 0.30 were omitted from each EFA model and the resulting factor structures were modeled in six second-order CFAs. For the EFA’s, the EBIC suggested that the model containing six higher-order factors best fitted the data, closely followed by the five higher-order factor model. However, the second-order CFA model with six higher-order factors did not converge, possibly due to the many cross-loadings identified in the EFA. The fit indices of the remaining five higher-order CFAs suggest that all models adequately fitted the data, while the five higher-order factor model showed the best fit. The five second-order factors explained 76 % of the variance in the latent lower-order constructs, considerably more than models with less higher-order factors.

Table 3

Fit indices of higher order confirmatory factor analysis models.

2nd order factors	% variance explained	Parameters	p-value (χ^2)	χ^2 / df	RMSEA (95 %CI)	EBIC	CFI	SRMR
1	49 %	487	<.001	4.70	0.067 (0.066, 0.068)	1447.9	0.97	0.071
2	64 %	482	<.001	5.03	0.07 (0.069, 0.071)	584.1	0.968	0.074
3	65 %	485	<.001	4.54	0.065 (0.064, 0.066)	212.1	0.972	0.07
4	67 %	489	<.001	4.47	0.065 (0.064, 0.066)	−48.0	0.972	0.069
5	76 %	491	<.001	4.29	0.063 (0.062, 0.064)	−99.0	0.974	0.067

Note: df = degrees of freedom.

Inspection of the factor loading structure of this best fitting model (Table 4 ) shows that higher-order factor 1 (fear of health) contained constructs related to fear of getting contaminated with the coronavirus, of infecting others and of potential health related consequences. Higher-order factor 2 (fear of supplies shortages & xenophobia) concerned scales on fear of shops running out of supplies and of foreigners spreading the virus. The third higher-order factor (psychological consequences of fear) contained scales assessing the psychological impact of the COVID-19 crisis (i.e., stress and anxiety-related symptoms such as compulsions, intrusions and nightmares). The fourth higher-order factor (socio-economic consequences) only involved a subscale of our newly developed scale, containing items measuring the broad socio-economic consequences of the pandemic. Lastly, the four SHAI and IUS-12 constructs loaded together on the fifth higher-order factor (stable fear related traits), suggesting that these constructs are distinct from those assessed by the COVID-19 fear questionnaires.

Table 4

Factor loading structure of the second-order factor model with 5 higher-order factors.

Scale	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5
	Fear of health	Fear for supplies shortages and xenophobia	Psychological consequences of fear	Fear about socio-economic consequences	Stable fear related traits
FCQ	1.05
NEW (health)	1.06
NEW (socio-economic consequences)				0.82
FCV-19S	0.39		0.66
CSS (danger)	0.80
CSS (socioeconomic)		1.01
CSS (xenophobia)		0.65
CSS (contamination)	0.63
CSS (traumatic stress)			1.16
CSS (checking)			0.55
IUS-12 (prospective)					0.97
IUS-12 (inhibitory)					1.03
SHAI (likelihood)					0.37
SHAI (severity)					0.42

Note: Factor loadings smaller than 0.3 have been omitted from the table.

Network analysis on the items of all questionnaires

The redundancy of items included in the networks was first evaluated using the goldbricker function in the R-package networktools (Jones, 2018). The analysis flagged 11 pairs of items that correlated similarly with other items in the network. Surprisingly, each items pair involved items that were both part of the same questionnaire (CSS, SHAI or IUS-12). Given that these are well validated questionnaires and the researchers developing these questionnaires have already made effort to prevent conceptual overlap in the included items, we therefore retained all original scale items in the network models.

The CFAs reported in the previous sections aimed to explain the correlational structure between the questionnaire item scores with only a small amount of latent constructs. These models assume that latent constructs explain all covariation between the items loading on a construct. A limitation of such models is that they do not shed light on possible causal pathways between items within a construct, or between items across various constructs. Network analysis is a psychometric technique that can provide a different perspective on the overlap between the scales because it models the association between individual items. This approach may also elucidate how various latent constructs are connected by identifying items that serve as a bridge in the causal pathway between two or more constructs.

Fig. 1 shows the correlational structure of COVID-19 fear questionnaire items. Each of the edges represents a correlation between two items larger than 0.1. The items are colored according to the latent factor they were supposed to measure. The figure reveals several interesting patterns. First, underlying latent constructs differ in the extent to which items cluster together. The items of most constructs measured by the CSS closely cluster together. The items measuring the contamination and danger constructs also cluster together, suggesting that these constructs are closely related (in line with the estimated latent correlations reported in Table 4 and the factor structure proposed by Taylor, Landry, Paluszek, Fergus et al., 2020, Taylor, Landry, Paluszek, Rachor et al., 2020). The FCQ and FCV-19S scales were originally proposed to measure a single latent construct, yet the correlation graph shows several small clusters of items, suggesting that there may be more constructs underlying these item scores. Another finding is that some clusters of items are located at the borders of the network plot, suggesting that these are measuring unique issues not captured by other questionnaires. Examples are the xenophobia construct of the CSS and the socio-economic consequences construct of the newly developed scale.

Fig. 1

Correlation network structure of the COVID-19 fear scales. The nodes represent the items in the different questionnaires and the edges represent the correlations larger than .1 between these items. For a list of all the items, see https://osf.io/fxd7c/.

Fig. 2 illustrates the regularized partial correlation network of all COVID-19 fear questionnaire items, as well as the SHAI and IUS-12 items. The items are colored according to the earlier identified five higher-order latent factors. An edge represents the association between two items adjusted for the influence of the other items. The figure reveals that the SHAI and IUS-12 items clustered together at the border of the graph, suggesting that these constructs are conceptually different from those assessed by the COVID-19 fear questionnaires. Although the SHAI and IUS-12 items primarily correlated with each other, there may be a possible causal pathway between the IUS-12 and some COVID-19 fear items. For instance, the path from IUS item 2 (“It frustrates me not having all the information I need”) to NEW item 16 (“I am worried that fake or inaccurate news is being spread about COVID-19″) and NEW item 4 (“I am worried that COVID-19 is causing mass panic”) could signify how people with low uncertainty tolerance may worry that fake news about COVID-19 can cause mass panic.

Fig. 2

Regularized partial correlation network of the SHAI, IUS-12 & COVID-19 fear scales with colors indicating the identified higher order five-factor structure. Partial correlations smaller than .1 are not shown to enhance interpretability. For a list of all the items, see https://osf.io/fxd7c/.

The five most central items in the partial correlation network are reported in Supplementary Table 3. The results showed that NEW item 10 (“I am worried that others will not continue to follow the rules”) had the largest average strength, betweenness, and closeness. Furthermore, to determine the stability of the estimated network models, we performed a bootstrap analysis with 1000 samples investigating the stability of (1) node strength, (2) betweenness, (3) closeness, and (4) edge weight. The results of this analysis are presented in the Supplementary Materials. Altogether, these results suggest that excellent stability for node strength and somewhat lower but still acceptable stability for betweenness. The stability of the closeness index performs worse than the typically expected threshold of 50 %, suggesting that the order of the closeness estimates in our networks should be interpreted with care.

Discussion

In this study, we examined the latent factor and item structure of four questionnaires for measuring fear of COVID-19 in a large online sample. Several findings stood out in our analyses. First, we were able to replicate the factor structure reported in the literature of the included questionnaires (i.e., FCQ, FCV-19S, CSS, IUS and SHAI). Second, for the new questionnaire included in this study based on 16 topics of worry relating to COVID-19 previously reported by Mertens et al. (2020), we found that a 2-factor structure (fear of health and fear of socio-economic consequences) fit the data best. Third, for the overall analysis of all questionnaires, we found a 5-factor structure (i.e., fear of health, fear of supplies shortages and xenophobia, the psychological consequences of fear of COVID-19, the broad socio-economic consequences of the pandemic, and stable fear-related traits). Of note is that these 5 factors differentially loaded onto items of the different questionnaires, indicating that most of the questionnaires (except the FCQ) captured several factors and that the factor structure is unlikely the result of shared method variance between items within the same questionnaire. Fourth and final, the network analyses highlighted relevant clusters, identified central items and outlines potential causal pathways in the network. In the remainder of the Discussion, we discuss the implications of these findings, consider the limitations of our work and end with our general conclusion.

Implications

These results demonstrate that fear of COVID-19 is a relatively heterogeneous construct and the distinct questionnaires do not necessarily tap into the same underlying latent factors. As such, questionnaires for measuring fear of COVID-19 should be carefully chosen. The two general questionnaires (i.e., the FCV-19S and FCQ) seem to capture two different factors of COVID-19 fear. The FCQ primarily on the first factor (fear of health), whereas the FCV-19S loads both on the first and the third factor (fear of health and psychological consequences of fear). Furthermore, the two larger questionnaires (i.e., the CSS and the new questionnaire) capture several, but not all, factors (see Table 4). Therefore, we strongly recommend considering more specifically the aim and focus of a study when investigating (different aspects of) fear of COVID-19. Furthermore, it is conceivable that the different factors of fear are related to different health outcomes, which should be a topic of research for future studies. Moreover, the stable anxiety-related traits (i.e., SHAI and IUS) did not cluster specifically with one of the identified COVID-19 factors in particular. This implies that they are conceptually different from the questionnaires assessing fear of COVID-19. As such, it appears that for measuring COVID-19 related fear, it is more appropriate to use specifically developed scales rather than general anxiety scales (see also Taylor et al., 2021).

The results of the network analyses indicate that there appears to be a central core of items from various scales closely clustering together (i.e. CSS (danger), CSS (contamination), NEW (health) and some items of the FCQ and FCV-19S). Interestingly, the second-order CFA indicated that these scales load onto the first higher-order factor (fear of health). Based on the results of both the CFAs and the network visualization, we argue that this item cluster largely represents the overlap between the various COVID-19 fear questionnaires. This result resembles the recent findings by Taylor, Landry, Paluszek, Fergus et al. (2020), Taylor, Landry, Paluszek, Rachor et al. (2020), who found that worries about the dangerousness of COVID-19 were a central cluster in a prior comprehensive network analysis of COVID-19 related worrying, avoidance and coping. Given these results, we recommend researchers to use questionnaires that capture this cluster (e.g., CSS danger and contamination or the FCQ) if they want to measure this core aspect of fear of COVID-19. Furthermore, in case researchers would want to create a brief screener, we could recommend using items from this cluster as they seem to capture a core aspect of fear of COVID-19. That said, we do want to encourage researchers to carefully consider which questionnaire is optimal for their research purposes (e.g., if researchers would want to measure psychological stress symptoms due to COVID-19, it would be better to use the FCV-19S or CSS traumatic stress).

In addition, the results of the partial correlation network allow to identify possible causal pathways between fear of COVID-19 items. For instance, one pathway that we highlighted may help explain why frustration about a lack of information (IUS item 2) and worries about fake news (NEW item 16) is related to worries about mass panic (NEW item 4). Similarly, the path from FCV-19S item 1 (“I am most afraid of coronavirus-19”) to FCV-19S item 2 (“It makes me uncomfortable to think about coronavirus-19”) and to FCV-19S item 3 (“My hands become clammy when I think about coronavirus-19”) may represent the causal pathway to physiological stress reactions. Furthermore, the partial correlation network helps clarify the role of media use in fear of COVID-19. For example, the path from FCV-19S item 5 (“When watching news and stories about coronavirus-19 on social media, I become nervous or anxious”) to FCV-19S item 7 (“My heart races or palpitates when I think about coronavirus-19”) to FCV-19S item 6 (“I cannot sleep because I'm worrying about getting coronavirus-19”) may suggest how following media updates on COVID-19 may cause difficulty falling asleep due to excessive worrying.

Taken together, our results help clarify the nature of fear of COVID-19. This is important, as COVID-19 has a profound impact on individual’s psychological well-being (Ettman et al., 2020; Hyland et al., 2020; Twenge & Joiner, 2020), and fear likely plays an important mediating role herein (Rodríguez-Hidalgo et al., 2020; Sakib et al., 2021; Satici, Gocet-Tekin, et al., 2020). Furthermore, fear of COVID-19 appears to be related to hygienic preventive behaviors and compliance with the health regulations (Breakwell et al., 2021; Harper et al., 2020; Yıldırım et al., 2021). Our results, and those of others (e.g., Taylor, Landry, Paluszek, Rachor et al., 2020), indicate that fear of COVID-19 is considerably heterogeneous and that questionnaires that have been recently developed capture different aspects of fear of COVID-19. This needs to be taken into account if we want to fully understand the relationship between fear and the different psychological and societal consequences due to COVID-19 (e.g., fear of health may be positively related to preventive behaviors, but fear about socio-economic consequences may be unrelated or even negatively related).

Limitations

Several limitations to this work can be noted. First, the sample of our study may not be representative of the entire population in several aspects. That is, the majority (>85 %) of our sample was between 18 and 40 years old and primarily (93.2 %) lived in Europe, which limits the representability of our findings. This is especially important given that the mortality of COVID-19 is especially pronounced in the 60+ age range (World Health Organization, 2020), and therefore fear of COVID-19 may also manifest more strongly in an older sample. Furthermore, this study was conducted in the last week of June 2020, during which there were relatively few new infections and death related to COVID-19 in Europe (Nisen, 2020; Winkleman & Santamaria, 2020). These aspects likely influenced how fearful our sample was at the time of the study and may have possibly influenced the results of our analyses.

Another limitation raised by our network analysis is that our approach to identify the higher-order latent fear constructs may be biased by a possible misfit in the higher-order CFA model, as shown by the partial correlations between items loading on different higher-order constructs. For example, the edge between NEW item 13 (“I am worried that there will be shortages of food or other supplies”) and CSS item 7 (“I am worried about grocery stories running out of food”) connects two higher-order constructs broad socio-economic fear and xenophobia and fear of supplies shortages. These items almost pose the same question, yet unexpectedly they loaded on distinct higher-order constructs. An explanation for this unexpected finding could be that the EFA used to investigate the higher-order factor structure was based on a correlation matrix of latent variables instead of a correlation matrix of all item scores. Indeed, we found that in an EFA on the complete set of items, these two items were clustered under the same factor (for these results, see: https://osf.io/fxd7c/).

Conclusion

We find that there are several latent factors (or: separate clusters of items) reflecting the multidimensionality of fear of COVID-19 (i.e., fear of health, fear about supplies shortages and xenophobia, psychological stress symptoms, and fear about broad socio-economic consequences). None of the questionnaires included in our study captured all these factors, though all questionnaires loaded onto the first factor. As such, studies on fear of COVID-19 should carefully select the questionnaire most suited for their aims, and future studies should look into how the different aspects of fear of COVID-19 are related to relevant outcomes (e.g., mental health).

Declaration of Competing Interest

The authors report no declarations of interest.