Psychometric Properties of an Instrument to Assess the Fear of COVID-19 in a Sample in Argentina: a Mixed Approach.

The FCV-19S was the first instrument designed to assess the severity of fear related to COVID-19 and has already been validated in many languages. The objective of this study was to evaluate the homogeneity and construct validity of the 5-item version of the FCV19S, using an online questionnaire in 599 people. The participants' age ranged from 18 to 65 years. Age, gender, marital status, educational level, employment status, and socioeconomic status were analyzed. In the evaluation process we assessed interitem correlation, item rest-correlation, confirmatory factor analysis: Root Mean Square Error of Approximation, Comparative Fix Index, Tucker-Lewis Index; internal consistency (Cronbach's alpha, McDonald's omega), and the Rasch model was assessed for learning more about the psychometric properties of the scale, which allows a detailed knowledge of the strengths and weaknesses of a scale. The FCV-5S has adequate psychometric indicators from the perspective of the Classical Theory of Items. The major limitations were using a self-reported measure and having a convenience sample not necessarily representative of the general population of Argentina.

The epidemiological behavior of COVID-19 has been characterized by a rapid global expansion since its appearance (Li et al., 2020; Tang et al., 2020). According to the WHO (March 18, 2021), the number of infected worldwide is 120,915,219 and 53,177,050 for the American continent, with total deaths of 2,674,078 worldwide and 1,279,765 for the region, to which is added the appearance of additional strains, uncertainties regarding access, and possible adverse effects of vaccines (WHO, 2021). At the time of data collection, the situation in Argentina was characterized by 7164 total number of COVID-19 positive cases and 310 number of deaths (WHO, 2020). This situation, besides the social and economic damages, produces fear in the population, which has implications for health and well-being on a population scale (Hossain et al., 2020; Lucero-Prisno et al., 2021; Murphy et al., 2021).

In this way, Ahorsu et al. (2020) designed a seven-item instrument called the Fear of COVID-19 Scale (FCV-19S); after that, the original scale was translated and adapted to several languages like English (Perz et al., 2020; Winter et al., 2020), to Spanish (Broche-Pérez et al., 2020; Cassiani-Miranda et al., 2021; Caycho-Rodríguez et al., 2021; García-Reyna et al., 2020; Huarcaya-Victoria et al., 2020; Martínez-Lorca et al., 2020; Mercado-Lara et al., 2021), to Portuguese (Andrade et al., 2020; Cavalheiro & Sticca, 2020; Giordani et al., 2020; Magano et al., 2021), and Italian (Soraci et al., 2020).

Various studies have reported unidimensional structures of the FCV-19S (Cavalheiro & Sticca, 2020; Elemo et al., 2020; Garcia-Reyna et al., 2020; Giordani et al., 2020; Haktanir et al., 2020; Kassim et al., 2020; Mahmood et al., 2020; Martinez-Lorca et al., 2020; Nguyen et al., 2020; Perz et al., 2020; Sakib et al., 2020a, 2020b; Satici et al., 2020; Soraci et al., 2020; Stănculescu, 2021; Tsipropoulou et al., 2020; Tzur Bitan et al., 2020; Wakashima et al., 2020; Winter et al., 2020); however, some researchers found that a two-dimensional structure was kept (Andrade et al., 2020; Alyami et al., 2020; Caycho-Rodriguez et al., 2021; Chi et al., 2020; Huarcaya-Victoria et al., 2020; Kassim et al., 2020; Magano et al., 2021; Masuyama et al., 2020; Reznik et al., 2020; Soraci et al., 2020; Wibowo et al., 2020).

In South America, different studies have been carried out to evaluate the psychometric properties of the FCV-19S. In Argentina, the two-factor structure of the scale showed satisfactory goodness-of-fit indices, adequate reliability, and an invariant factor structure along age groups; however, they did not report invariance analysis by gender (Caycho-Rodríguez et al., 2021). A study conducted in Peru showed adequate validity and reliability, with invariance between healthcare workers and age groups, but without invariance by gender. In Brazil, the scale exhibited a unidimensional structure with satisfactory internal consistency (Andrade et al., 2020). In Colombia, two unidimensional versions have been proposed: the first one conducted in a sample of physicians showed acceptable results of internal consistency and dimensionality after eliminating items 3 and 7 of the original scale (Mercado-Lara et al., 2021) and the other one, called Fear of COVID-5 (FCV-5S), also showed acceptable performance in the general Colombian population after eliminating items 1 and 5 (Cassiani-Miranda et al., 2021).

To advance the construct validity of the fear of FCV-19S is necessary invariance analysis among the distinct groups (Caycho, 2017; Lee & Smith, 2020). However, few studies to date have evaluated the factorial invariance of the FCV-19S, finding dissimilar results (Caycho-Rodríguez et al., 2021; Huarcaya-Victoria et al., 2020; Stănculescu, 2021). On the other hand, to date, only a few studies have evaluated the psychometric properties of the FCV-19S using item response theory (IRT) (Caycho-Rodriguez et al., 2021; Pang et al., 2020; Sakib et al., 2020a, 2020b; Stănculescu, 2021). Classical test theory (CTT) results are sample size-dependent; a person’s score may be higher if the items are easy or lower if they are difficult, which could compromise the ability of the test to reliably measure a latent construct (Muñiz, 2010). In contrast, IRT-derived analyses offer the advantage of obtaining more detailed information about the internal structure, homogeneity, and appropriateness of the use of a total score (Liu et al., 2019).

Subsequent refinement of the FCV-19S has led to a 5-item version which has demonstrated a stable unidimensional structure, acceptable homogeneity, and nomological validity (Cassiani-Miranda et al., 2021). The preference for shorter versions of measurement instruments gives greater guarantees of unidimensionality, and more reproducible psychometric indicators in different contexts and samples (Blais & Baer, 2010; Keszei et al., 2010; Smith et al., 2000). However, this short version of the COVID-19 fear scale has not been evaluated in the Argentinean population. Therefore, this study aimed to evaluate the homogeneity and construct validity of the 5-item version of the FCV19S using a combined analytic approach with the CTT and IRT in a sample of the general Argentinean population.

Methods

Participants

The target was the general Argentinian population. A total of 599 people took part in this study through snowball sampling. Inclusion criteria were to be residents in Argentina, speak Spanish, and be 18 years of age or older.

Measurements

Sociodemographic Variables

Characteristics such as age, gender, marital status, educational level, employment status, and socioeconomic status were evaluated.

The Fear of COVID-5

The Fear of COVID-19 Scale (FCV-19S) was developed by Ahorsu et al. (2020). The original scale comprised seven items that offer five levels of agreement as a response: “Totally disagree,” “disagree,” “neither agree nor disagree,” agree,” and “totally agree.” The answers are scored from 1 to 5, with scores between 7 and 35. For this study, we use the version adapted to the Colombian population, which was subjected to a complete process of translation and back-translation from English, as well as minor adjustments in the response options and phrases in the negative sense. A factor analysis yielded a 5-item version (FCV-5S) that showed adequate homogeneity and dimensionality (Cassiani-Miranda et al., 2021). The scale contains five items that explore cognitive and physiological responses to fear related to COVID-19. This scale presents four answer options: “never,” “almost never,” “almost always,” and “always” scored from zero to three, for a score ranging from o to 15. The higher the score, the greater the fear of COVID-19 (Ahorsu et al., 2020).

Procedure

After the ethics committees approved the project, the survey was disseminated on networks employing cellphones, emails, and networks (e.g., Facebook). Google Forms® administered the online survey to ensure wide reach and easy access. Participants were asked to share the survey with their personal and professional networks. However, the absolute anonymity of the participants was guaranteed.

People who expressed their interest in being part of the study were asked to provide their email, and through this means, they received the electronic form that they must fill out. Before completing the survey, each participant had to read and accept the terms of their participation through electronic informed consent, in which the objectives of the study, its potential risks, and the general conditions for their participation were explained. The survey took between 5 and 10 min, and the data were collected between June 28 and November 24, 2020.

Statistical Analysis

The analysis began with a description of the most relevant characteristics of the sample. Then, the internal consistency of the test was estimated by Cronbach’s alpha and McDonald’s omega coefficients (McDonald, 1970). The value of 0.70 was anticipated as a threshold (Dunn et al., 2015; Gliem & Gliem, 2003). The average interitem correlation and the item rest-correlation were also estimated, to establish the association of each item with the total score of the others (Zijlmans et al., 2018). The confirmatory factor analysis (CFA) was carried out with Mplus®. The following adjustment measures were taken into account: the root mean square error of approximation (RMSEA), which is expected to be less than 0.05 in a model with a good fit, with the upper limit of its 95% confidence interval below 0.8; the comparative fix index (CFI) and the Tucker-Lewis Index (TLI). In the case of these last two indices, the minimum value is 0.90 (Hu & Bentler, 1999; Garson, 2012). Also, the × 2/degrees of freedom ratio was used as an additional index of parsimonious adjustment. A value ≤ 2 indicates good fit (Schermelleh-Engel et al., 2003).

To learn more about the psychometric properties of the scale, an analysis was made with the Rasch model, which allows a detailed knowledge of the strengths and weaknesses of a scale. For the INFIT OUTFIT estimates, the criteria for analysis were > 2 distorts or degrades the measurement system; 1.5–2.0, unproductive for construction of measurements, but does not degrade it; and 0.5–1.5 productive for measurement and < 0.5 less productive for measurement, but not degrading. Descriptive statistics were analyzed using R packages, CFA using MPLUS 7.0, and Rasch model using WINSTEPS 3.8.1.

Results

The main sociodemographic characteristics of the sample are presented in Table 1. The average age was 31.4 years old (SD = 12.9) and 62% of the participants were healthcare workers.

Table 1

Social and demographic characteristics of the participants

Variable	n	%
Age group
18–29	348	63.2
30–44	100	18.1
45–64	87	15.8
65 +	16	2.9
Gender
Female	145	26.3
Male	406	73.7
Marital Status
Single, widowed, divorced	425	77.1
Married	126	22.9
Children
Yes	172	31.2
No	379	68.8
Education
Primary or secondary	146	26.5
Superior	405	73.5
Employee
Yes	317	57.5
No	234	42.5
Income
Upper	37	6.7
Medium	475	86.2
Low	39	7.1

Cronbach’s α coefficient and McDonald’s ω for the full scale were both 0.73. The average interitem correlation was 0.4. Item rest-correlation was from 0.4 to 0.6 (Table 2).

Table 2

Item rest correlation

Item	Correlation
It makes me uncomfortable to think about coronavirus-19	0.445
My hands become clammy when I think of coronavirus-19	0.479
I am afraid of losing my life because of coronavirus-19	0.492
I cannot sleep because I am worrying about getting coronavirus-19	0.587
My heart races or palpitate when I think about getting coronavirus-19	0.623

Confirmatory Factor Analysis

Considering previous experience with this instrument (Ahorsu et al., 2020; Cassiani-Miranda et al., 2021), a CFA was run with a single factor model. Standardized factor loading values and square correlations are shown in Tables 3 and 4.

Table 3

Standardized factor loadings

Observed variable	Estimate	S.E	P-value
It makes me uncomfortable to think about coronavirus-19	0.624	0.038	< 0.01
My hands become clammy when I think of coronavirus-19	0.727	0.044	< 0.01
I am afraid of losing my life because of coronavirus-19	0.659	0.038	< 0.01
I cannot sleep because I am worrying about getting coronavirus-19	0.879	0.033	< 0.01
My heart races or palpitate when I think about getting coronavirus-19	0.898	0.031	< 0.01

Table 4

Square correlations

Observed variable	Estimate	S.E	P-value	Residual variance
It makes me uncomfortable to think about coronavirus-19	0.389	0.047	< 0.01	0.611
My hands become clammy when I think of coronavirus-19	0.529	0.065	< 0.01	0.471
I am afraid of losing my life because of coronavirus-19	0.435	0.05	< 0.01	0.565
I cannot sleep because I am worrying about getting coronavirus-19	0.773	0.058	< 0.01	0.227
My heart races or palpitate when I think about getting coronavirus-19	0.807	0.057	< 0.01	0.193

Goodness of fit indicators were chi-square test of Model Fit = 10.136 (Df = 5; p value 0.0715); RMSEA = 0.043 (C.I 0.0001 to 0.082); CFI = 0.994; and TLI = 0.987. The average variance extracted was 0.73 and the composite reliability (CR) was 0.87.

A closer analysis of the data shows that men tended to express more fear than women since their average adding the five items was 2.31 (SD = 2.17) while that of women was 1.60 (SD = 2.13). However, only one woman rated that her hands were always sweaty when thinking about COVID-19.

When running the analysis of invariance by gender, it was found that the model with the restriction of the factor loadings of the items (weak invariance) showed that the measure was not invariant by gender (DIFFTEST p value = 0.010). Table 5 shows the fit values for the models in the invariance analysis.

Table 5

Goodness of fit of invariance analysis

Models	X2	df	p-val	Difftest p value(X2)	TLI	RMSEA	CFI	ΔCFI
Male	3.081	5	0.6875		0.998	0.002	0.999
Female	7.898	5	0.1619		0.990	0.038	0.995
Configural (free parameters)	5.386	10	0.864		0.998	0.002	0.999
Metric	21.292	14	0.094	0.01	0.978	0.043	0.979	0.02
Scalar	34.685	28	0.179	0.04	0.977	0.052	0.977	0.002

To learn more about the psychometric characteristics, the Rasch model was applied. The Wright Map reveals a lack of coordination between people’s degree of fear of COVID and that measured by the test. The mean of the people is nearly two standard deviations from the test (Fig. 1).

Fig. 1

Person-item Wright map

The category analysis shows that indeed the categories of the levels of zero fear have much higher frequencies than the rest. Category three (always) seems unproductive in terms of the attribute measurement.

When ordering the size of the differential functioning of each item according to its difficulty (the probability of being answered at a high level on the scale), it is observed that the scale differentially measures fear according to gender (Fig. 2).

Fig. 2

Differential item measure ordered by item difficulty

The variance explained by the measurements was 61%. The eigenvalue of the unexplained variance in the first test was 1.6. The Rasch reliability was 0.57 and the separation index was 1.15. The Zstd measure for infit and outfit shows that item three (losing life) has adjustment problems (infit Zstd = 3.5; outfit Zstd = 2.5). Something similar happens with item five (hearth) (infit Zstd =  − 1.8; outfit Zstd =  − 2.5) (Table 6).

Table 6

Categories estimators

Category observed				Obsvd sample		Infit outfit		Andrich	Category
Label	Score	Count	%	AVRGE	EXPECT	MNSQ	MNSQ	Threshold	Measure
Never	0	1882	68	-4.65	-4.60	.97	.97	NONE	(-3.80)
Almost never	1	631	23	-1.95	-2.07	.97	.73	-2.66	-1.23
Usually	2	189	7	.18	.08	1.02	1.12	.25	1.35
Always	3	53	2	1.32	2.09	1.68	1.68	2.42	3.59

Discussion

This study provides data about the homogeneity and factorial structure of the scale FCV-5S, using classical test theory and item response theory. In the present study, the results provided evidence that the FCV-5S has good internal consistency, a unidimensional structure, lack of gender invariance, and lack of coordination between the items and the degree of the latent construct of fear of COVID-19 and fit problems in items three and five.

The FCV-5S confirmed the one-dimensional structure, as did the 5-items version in Colombia (Cassiani-Miranda et al., 2021), the original version (Ahorsu et al., 2020), and subsequent adaptations (Alyami et al., 2020; Sakib et al., 2020a, 2020b; Soraci et al., 2020). The results of some studies worldwide suggest that the FCV-19 has a two-factor structure that is supported by a two-factor model with an emotional and a psychological dimension (Tzur Bitan et al., 2020; Huarcaya-Victoria et al., 2020); however, some data suggest that the two-factor structure is not as consistent as the unidimensional structure which appears to be more stable (Ahorsu et al., 2020; Alyami et al., 2020; Sakib et al., 2020a, 2020b; Satici et al., 2020; Soraci et al., 2020).

Cronbach’s α coefficient and McDonald’s ω were both 0.73 for the full scale, which indicates high internal consistency since a scale has an excellent internal consistency when these values are at least 0.70 (Campo-Arias & Oviedo, 2008; Watkins, 2017). Both, AVE and CR confirmed good reliability (Moral de la Rubia, 2019). These findings are related to previous studies (Ahorsu et al., 2020; Alyami et al., 2020; Cavalheiro & Sticca, 2020; Caycho-Rodrıguez et al., 2021; Huarcaya-Victoria et al., 2020; Sakib et al., 2020a, 2020b; Soraci et al., 2020; Stănculescu, 2021; Tzur Bitan et al., 2020; Wakashima et al., 2020) that reported Cronbach’s alpha values between 0.70 and 0.90 for FCV-19S.

The FCV-5S has adequate psychometric indicators from the perspective of the classical theory of items. However, there are indications of a lack of invariance by gender. This finding was also reported in a study in Peru, where factorial invariance was found only for those over 40 years of age, but the invariance between men and women was not met (ΔCFI = 0.02) (Huarcaya-Victoria et al., 2020). Although there is a previous study that evaluated the psychometric performance of the COVID-19 fear scale in the general population of Argentina, it was not possible to perform gender invariance analyses given that the sample was predominantly female (268 men and 1023 women) (Caycho-Rodríguez et al., 2021). This sample configuration could be not suitable for invariance analysis (Fischer & Karl, 2019). Additionally, an investigation in Romania, in a sample of 809 adults from the general population, found measurement invariance of the scale items between gender groups demonstrated by differential item functioning analysis and graded response model (GRM) (Stănculescu, 2021). It is important to keep in mind that fear is a construct that can be affected by conditions inherent to gender (Chaplin, 2015). The best option is to review the test in its entirety since issues involving emotional expressions can be affected by gender differences.

From the perspective of item response theory (IRT), the test shows strengths such as one-dimensionality. The use of the IRT model in other studies has shown that the test has adequate psychometric characteristics (Sakib et al., 2020a, 2020b; Stănculescu, 2021). On the other hand, there is evidence that the level of fear of COVID measured by the test seems to be lower than the fear that the participants experience, which is observed in the high frequency of responses in the category “never.” It may be too much to expect that people will always or almost always have sweaty hands or feel their heart beating faster due to SARS-CoV-2.

Limitations and Strengths

The large sample size, consideration of broad age group, and utilization of both classic and modern psychometric assessments are among the strengths of the present study. However, the present study has some limitations. First, fear in the present study was assessed using a self-reported measure that can be influenced by factors such as social desirability, memory recall, and other common method biases. Studies using other methodologies are recommended (e.g., in-depth qualitative interviews, diary studies). Second, the sample was a convenience sample and was not necessarily representative of the general population of Argentina. Future studies using nationally representative studies are needed to confirm the results reported here. Furthermore, the size and configuration of the sample represent a methodological limitation as the number of female participants may be small for the differential functioning analysis and the invariance analysis (Fischer & Karl, 2019). Finally, this study did not examine the stability of the FCV-5S over time. Future research should therefore incorporate test–retest reliability measures into the design of their studies.

Public Health and Research Implications

The 5-item COVID-19 fear scale can be considered to show good psychometric performance from the TCT and may be useful for measuring fear of COVID-19 in the general population of Argentina. Therefore, it can be recommended for use in large-scale epidemiological studies, studies evaluating the efficacy of psychological interventions, and to identify the presence of fear of COVID-19 in the population of Argentina. Studies proposed to evaluate analyses of invariance of this scale should adjust sample designs by sex to obtain reliable measures of the sex invariance of this instrument, especially in Latin American populations.

Conclusions

The COVID-19 fear scale has adequate psychometric indicators from the perspective of the Classical theory of items and can be used to assess fear of COVID-19 during the pandemic among the Argentina general population. However, in this study, no evidence of gender invariance was found, which should be studied further.