Fear of illness & virus evaluation (FIVE) COVID-19 scales for children-parent/caregiver-report development and validation.

OBJECTIVE: Commonly-used youth anxiety measures may not comprehensively capture fears, worries, and experiences related to the pervasive impact of the COVID-19 pandemic. This study described the development of the Fear of Illness and Virus Evaluation (FIVE) scales and validated the caregiver-report version.
METHOD: After initial development, feedback was obtained from clinicians and researchers, who provided suggestions on item content/wording, reviewed edits, and provided support for the updated FIVE's content and face validity. Factor structure, measurement invariance, and psychometric properties were analyzed using data from a multi-site, longitudinal study of COVID-19-related effects on family functioning with 1599 caregivers from the United States and Canada.
RESULTS: Confirmatory factor analyses indicated a hierarchical five-factor structure best fit the data, resulting in a 31-item measure with four lower-order subscales: (1) Fears about Contamination and Illness; (2) Fears about Social Distancing, (3) Avoidance Behaviors, and (4) Mitigation Behaviors, and a higher-order factor, (5) Total Fears, indicated by the two fear-related lower-order subscales. Measurement invariance by country of residence, child age, and child sex was found. All subscales demonstrated strong internal consistency, appropriate item-scale discrimination, and no floor or ceiling effects. The Total Fears subscale demonstrated appropriate test-retest reliability. Concurrent validity supported by strong correlation with a youth anxiety measure. DISCUSSION: The FIVE provides a psychometrically-sound measure of COVID-19-related fears and behaviors in youth in a caregiver-report format. Future research is necessary to evaluate correlates and longitudinal symptom patterns captured by the FIVE caregiver-report, as well as the validity and reliability of a youth self-report version of the FIVE.

Introduction

Children and adolescents worldwide are significantly impacted by the COVID-19 pandemic. Abrupt changes to routine, including removal from in-person schooling and changes in social and community engagement are associated with worsening mental health in children and adolescents, (de Figueiredo, Sandre, & Portugal, 2021) including elevated anxiety, fear of contagion, frustration, boredom, reduced physical activity, and difficulties with sleep and concentration (Roccella, 2020, Wang et al., 2020). One study with a sample of 583 adolescents reported moderate-to-severe symptoms of depression (55%), anxiety (48%), and posttraumatic stress (45%); 38% reported suicidal ideation and 69% reported sleep problems (Murata, Rezeppa, & Thoma, 2020).

The impact of the pandemic is most notable with pre- and post-pandemic start comparisons. For example, a greater number of adolescents reported poor quality of life (40.2%) and elevated levels of anxiety (24.1%) during the pandemic compared to the pre-pandemic control cohort group (15.3% and 14.9%, respectively) (Ravens-Sieberer et al., 2021). Another study of adolescents found significant increases in self-reported depression and anxiety symptoms and a decrease in life satisfaction between 2019 and May of 2020 (Magson et al., 2021). A separate study using established cut-offs of a validated measure of anxiety symptoms found that 18.2% of adolescents endorsed elevated panic or somatic symptoms, 40.4% endorsed elevated generalized anxiety symptoms, and 29.5% reported elevated social anxiety symptoms (Hawes, Szenczy, Klein, Hajcak, & Nelson, 2021). A sample of 407 adolescents (ages 14–17 years) surveyed both prior to and during the pandemic also reported increases in negative affect, decreases in positive affect, elevated symptoms of depression and anxiety, and increased loneliness (Rogers, Ha, & Ockey, 2021). Importantly, these studies largely utilized existing measures of psychological distress developed before the pandemic, and did not assess the specific psychological and emotional impact of COVID-19 on resultant youth behaviors, particularly in relation to their adherence to mitigation strategies.

There have been a number of efforts to address this gap and to best understand the impact of the pandemic on the lives of individuals worldwide. Researchers across the globe have developed numerous COVID-19-related measures (19 at the time of writing). These measures can be classified into three categories: (1) symptom-specific measures (e.g., the Fear of COVID-19 Scale, 9 FCV-19S), (2) measures designed to capture a comprehensive representation of the impact associated with COVID-19 (e.g., COVID Stress Scales10), and measures of specific domains (e.g., the Pandemic [COVID-19] Anxiety Travel Scale, PATS11). Notably, all of these measures were developed to capture the emotional functioning of adults and few target both COVID-19-specific anxiety and related behaviors.

Of the published measures, only one has been tested in children and adolescents–the FCV-19S, a 7-item unidimensional measure of symptoms of anxiety provoked by COVID-19 (Ahorsu et al., 2020). However, this measure was not developed specifically for use with children and has only been tested with children in three studies; only two of which examined the factor structure: one found a unidimensional structure for the Mandarin translation and another found a bifactor structure for the Japanese translation (Chi et al., 2021a, Chi et al., 2021b, Masuyama et al., 2020, Sakib et al., 2020). Although all of these COVID-19-related measures greatly contribute to the ability to measure novel stressors brought on by the worldwide pandemic, none were developed with the purpose of measuring these in children. Thus, the purpose of this study was to describe the development and initial validation of the caregiver-report version of the Fear of Illness and Virus Evaluation (FIVE) scales, which is being used widely during the pandemic to measure COVID-19 anxiety and related avoidance and mitigation behaviors in children and adolescents.

Methods

FIVE scales measurement development process

The Fear of Illness and Virus Evaluation (FIVE) Scales (adult self-report, youth self-report, caregiver-report) were developed simultaneously using the same process in March 2020 following the initial implementation of social distancing restrictions to measure fears and behaviors hypothesized to be associated with the COVID-19 pandemic and were made freely available to researchers worldwide. This study focuses on the FIVE Caregiver-report, in its original English-language format. However, the FIVE-Caregiver report has been translated16 and tested in Spanish (Gómez-Becerra, Flujas-Contreras, & Andrés, 2020). The original translators did not test the factor structure, but reported internal consistency using Cronbach’s α and McDonald’s ω (ω ranged from 0.71 to 0.93; α ranged from 0.71 to 0.91) (Flujas-Contreras, Sánchez-López, & Fernández-Torres, 2020). The Spanish translation of the FIVE Caregiver-report was then tested in Spain in a group of 972 parents of children ages 3–18 years old. This study maintained the original hypothesized subscales and also did not test the factor structure, but reported strong internal consistency using Cronbach’s α (Total Score α = 0.92) with the other subscales α ranging from 0.71 to 0.92. Additionally, the study reported that the FIVE Total Fears score significantly correlated with child emotional symptoms on the Strength and Difficulties Questionnaire (Gómez-Becerra et al., 2020).

Item development for the FIVE scales was influenced by Rachman’s multiple pathways fear-acquisition and avoidance model, which posits that children can acquire fears through conditioning, vicarious, or indirect experiences, and through acquired information, (Rachman, 1977) and that the mechanisms through which fear is acquired do not necessarily impact its association with avoidant behaviors. That is, an avoidant behavior can emerge without direct contact with feared stimuli (Cameron, Roche, Schlundt, & Dymond, 2015).

When referring to a viral infection or illness, levels of fear are related to (1) how severe the threat seems, (2), how harmful the consequences could be, and (3) how much the person believes their actions can make a difference in the outcome (Champion et al., 2004, Greening et al., 2005). Prior research has indicated that fear and anxiety play a role in compliance with health behaviors associated with that fear or anxiety (Champion et al., 2004). The influence of fear on health behaviors is believed by vary in a curvilinear manner, i.e., less engagement in health behaviors at both very low and very high levels of fear. One explanation for this curvilinear relationship posed is that low levels of fear may lead to less motivation to engage in health behaviors, while very high levels of fear may result in avoidant behaviors (Asmundson and Taylor, 2020aaa, Asmundson and Taylor, 2020bbb).

Research published on the association between COVID-19-related fears and behaviors have indicated that higher levels of fears and worries were associated with greater reported engagement in mitigation behaviors (Taylor, Landry, Paluszek, Groenewoud et al., 2020). One study found that greater reported COVID-19-related fear was the only significant predictor of better adherence to mitigation strategies (e.g., social-distancing guidelines, hygienic behaviors), whereas symptoms of depression, political orientation, moral foundation, and values were not significant predictors (Harper, Satchell, Fido, & Latzman, 2020). However, the type of fear may make a difference (e.g., fear of infection vs. fear of social exclusion). For example, in adolescents, lower levels of COVID-19-related fears and higher levels of fears of missing out or social exclusion were associated with poor adherence to social distancing guidelines (Andrews, Foulkes, & Blakemore, 2020). Thus, the FIVE scales were developed to separately assess: (1) Fears about Contamination and Illness (C&I Fears) associated with virus and illness (e.g., catching the virus, adverse outcomes for themselves or loved ones); (2) Fears about Social Distancing (SD Fears) including disruptions to prior routines, inability to see friends); (3) Behaviors related to Fears (e.g., avoidance of people, hygienic behaviors, adaptive behaviors), and (4) two impact or impairment items.

The structural validity of the FIVE caregiver-report form fear subscales and behavior items was examined in this study via a confirmatory factor analysis (CFA) of the hypothesized factor structure, iteratively modified to improve fit as indicated by the data, followed by multigroup CFA for the best fitting model to test for measurement invariance across country of residence (Canada vs. USA), child age (<10 years vs. 10–17 years) and child sex. Additional psychometric properties were also evaluated, including internal consistency, item-scale discrimination, test-retest reliability, and concurrent validity following the COSMIN guidelines for evaluating measurement properties (Terwee, Bot, & de Boer, 2007).

Preliminary analyses: interpretability, content, and face validity

Item interpretability (i.e., reading level) was assessed to determine whether any of the items required reading skills beyond an eighth-grade level, a commonly used guideline for measurement development, (Streiner, Norman, & Cairney, 2015) with the use of two methods: calculation of the Flesh-Kincaid Grade Level, a composite of a weighted average sentence length and weighted average of the number of syllables per word (Kincaid, Fishburne, Rogers, & Chissom, 1975) and the Gunning Fog Index, a weighted average of the number and length of words per sentence (Gunning, 1969). Based on these two indices, the measure items were considered to have acceptable interpretability (Flesch-Kincaid Grade Level=5.1; Gunning Fog Index=6.7). FIVE items are answered using a 4-point Likert-type scale, chosen because a 4-point Likert-type scale is reportedly less likely to contribute to error variance and less likely to inflate the parameters estimated in the analyses than a Likert-type scale with a greater number of points. Specifically, method variance due to the number of scale points represents a systematic (not random) error which can artificially inflate internal consistency (Chang, 1994).

The content and face validity of the original items were assessed using feedback solicited from clinicians and researchers with expertise in youth anxiety-related psychopathology or treatment through electronic correspondence and social media in March 2020. Changes were made to expand and clarify the item response options, add items about topics suggested through feedback, and increase the longevity and utility of the measure. For example, all references to the coronavirus were switched to a “bad illness or virus.” The item response ratings were changed to capture the frequency of the fear or worry (how often) instead of the intensity of the fear (how fearful). Three additional items were added to the C&I Fears scale (increased from 6 to 9 items), expanded to also ask about fear of others/loved ones getting sick. Additional items about hypothesized consequences of having to engage in social distancing were added to the SD Fears scale (e.g., not being able to see friends, celebrate good things). The wording of behavior scale items was changed to solely ask about frequency and remove the qualifier or justification for engaging in the behavior (i.e., to help with fears and worries). Fourteen behavior items were developed and hypothesized to fit into three factors (i.e., adaptive, avoidance, and mitigation behaviors).

Sample

Study data stem from a longitudinal study of the impact of the COVID-19 pandemic on family functioning and mental health conducted at five universities across the USA and Canada. Approval for the study was obtained at each site’s Institutional Review Boards (IRB) or Research Ethics Boards (REB). Data were collected from parents or caregivers of 1 child (5–17 years of age) living within the USA (47 states and District of Columbia) or Canada (10 provinces and territories), Participants were recruited online using IRB/REB-approved recruitment blurbs shared via multiple means (e.g., community outreach, email listservs, social media posts, research laboratory parent-directed communications and websites),through invitations sent to families with children within the appropriate age-range participating in ongoing studies, or through Amazon’s Mechanical Turk (MTurk) crowd-sourcing platform.Data were collected between 3/20/2020 and 5/30/2020 for Week 1 (W1) and Week 2 (W2) and between 10/2/2020 and 11/30/2020 for Month 6 (M6). Data quality and validity items were incorporated into the questionnaires to check for inconsistent response patterns resulting in the exclusion of participants that did not pass (W1: 2.5%, W2:1.7%, M6: 2.8%).

Measures

Demographic characteristics and COVID-19-related information

A survey was developed by the study team to collect demographic information about the caregiver and their eldest child (e.g., child race/ethnicity, age, caregiver education, household income, postal code). Parents were also asked to complete a checklist of their family’s COVID-19-related experiences, developed by the study team for the purpose of the larger family functioning study from which these data stem and their experiences with COVID-19 (e.g., direct contact with someone diagnosed with COVID-19, disruptions to medical care). The answers to the checklist were summed to calculate a total COVID-19 experiences score. Postal code data were used to document local information related to COVID-19 (e.g., number of confirmed cases and deaths at the time of survey completion).

Overall anxiety severity and impairment scale for youth(OASIS-Y)

The (Comer, Conroy, & Cornacchio, 2022) OASIS-Y is the youth version of the validated adult measure (OASIS (Campbell-Sills, Norman, & Craske, 2009)), a unidimensional scale that measures frequency and intensity of anxiety symptoms, avoidance behaviors, and functional impairment. The OASIS-Y is a parent-report of the child’s anxiety symptoms and associated impairment, including how the child’s anxiety impacts family functioning and the parent’s own functioning. The items are answered with a 5-point Likert-type scale, with higher numbers indicating higher intensity or frequency (0−4), which are summed for a total score. It has strong internal consistency (Cronbach’s α = 0.89) and confirmed unidimensional factor structure (Comer et al., 2022).

FIVE caregiver-report scoring

All FIVE items are rated using a 4-point Likert-type scale (1−4), with higher values indicating greater fear or higher frequency of the behavior. Scoring of the measure was completed by calculating a standard score for each subscale so that the lowest possible score was 0 and the highest was 100, using the following steps: (1) the sum of subscale item responses, (2) subtract 1 from each item mean so the lowest possible mean score was 0, and (3) convert value to a percentage, illustrated by this formula (x = item response, k = total number of items for each subscale: x 100).

Statistical analyses

Descriptive statistics

FIVE item-level responses were examined using response frequencies and percentages for each of the four response categories on the Likert-type scale. A standard score was calculated for each of the final FIVE subscales, as previously described. Scale score distributions were assessed for floor effects (percentage of respondents with a score of 0) and ceiling effects (percentage of respondents with a score of 100)., Data were screened for univariate outliers (z-score values >|3|), normality (skewness < 3 and/or kurtosis values <8), and missing values.

Quantitative analyses were conducted using R version 4.1.2 (R Core Team, 2021) via the RStudio user interface version 2021.09.1 (RStudio Team, 2021) using the semTools, (Jorgensen, Pornprasertmanit, Schoemann, & Rosseel, 2020) lavaan, (Rosseel, 2012) and psych (Revelle, 2019) packages. Missing data were handled via multiple imputation (MI) using the R package mice (MI by chained equations), which iteratively applies an algorithm based on Fully Conditional Specification using Gibbs sampling (Van Buuren and Groothuis-Oudshoorn, 2011, Van Buuren, 2018). Imputations were created by estimating a separate polytomous logistic regression model for each variable using all other available variables from the dataset as predictors (i.e., recruitment site, country of residence, recruitment method, number of reported COVID-related experiences, child race/ethnicity, caregiver age, child age, caregiver sex, child sex, number of languages spoken in the home, caregiver education, caregiver income, family income, local COVID-19 infection rate, local COVID-19 mortality rate, and local COVID-19 case fatality rate). Five MI iterations were conducted, consistent with the both most commonly used guideline (Rubin, 1987) and by the most recent guidelines, which recommend conducting the same number of imputations as the average percentage rate of missingness for the data (e.g., if 20% of the data were missing, use 20 iterations) (Bodner, 2008, White et al., 2011). Although the current study had an outlier item missing 72.5% of the data (item 14 was inadvertently not collected at one site), the average percentage of missing data for FIVE items in this study, after excluding item 14, was 1.63% (n = 26). The next two highest percentages of missing data were 7.63% for FIVE item 17% and 5.5% for item 19. Therefore, five iterations were considered satisfactory for this study.

Although the FIVE is a new measure, CFAs were conducted to test the factor structure of the hypothesized models since exploratory factor analyses (EFA) should only be used when there are no pre-existing hypotheses regarding the relationships among items (Terwee et al., 2007). A mean- and variance-adjusted weighted least square estimator (WLSMV) was utilized to estimate model fit parameters (Kline, 2016, White et al., 2011) as recommended for ordinal variables (Katsikatsou, Moustaki, Yang-Wallentin, & Joreskog, 2012). To assess overall model fit, chi-square () statistics, Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR) fit indices were used. Item-level fit to assigned latent variables and item discrimination were evaluated using standardized item factor loadings (pattern coefficients). Previously established cut-points were used as guides to compare model fit (Hu and Bentler, 1999, Kline, 2016, Lei and Shiverdecker, 2020, Yu and Muthen, 2002). Specifically, results that were close to the following values were considered as indicating good model fit: (1) CFI and TLI values > 0.95, (2) RMSEA values < 0.05, (3) SRMR values < 0.08, and (4) standardized factor loadings (λ) > 0.40 (Terwee et al., 2007). Results of test of model fit were expected to be statistically significant due to the large sample size (n = 1599). Thus, additional weight was given to the CFI and TLI values, as they have been reported to perform better than other indices when using multiple imputation (Shi, Lee, Fairchild, & Maydeu-Olivares, 2020).

To fit the models to multiple imputed datasets simultaneously, the semTools (Jorgensen et al., 2020) and lavaan R packages (Rosseel, 2012) were used. Additionally, due to the categorical nature of the imputed data, likelihood ratio tests (LRT) for lavaan models fitted to multiple imputed data sets were calculated by pooling the LRT statistics from each imputation, resulting in the D2 statistic, composed of a vector of statistics from each imputation and corresponding degrees of freedom (Enders, 2010, Li et al., 1991). The first two factor structures tested were based on models hypothesized a priori and subsequent models were modified iteratively based on prior model fit results.These steps were repeated with the M6 data to assess for temporal structural stability. Multigroup CFA with the best fitting model were conducted to test for measurement invariance as a function of country of residence (Canada vs. USA), child age (<10 years vs. 10–17 years) and child sex. The same steps followed in the initial CFAs were used (i.e., WLSMV estimator). Three nested models were tested: (1) configural invariance model fit to groups without parameter constraints, (2) metric invariance model with equality constraints to all factor loadings across groups, and (3) scalar invariance model with equal factor loadings and thresholds. The models were then compared using Satorra-Bentler scaled chi-square difference (Δχ2) tests and the following fit indices: RMSEA, CFI, TLI, and SRMR. In particular, a decrease in ΔCFI or ΔTLI > 0.010 and/or an increase in ΔRMSEA > 0.015 were used as indicators of a worsening of model fit and lack of measurement invariance across groups (Chen, Wu, & Garnier-Villarreal, 2020).

Results

Participants

Table 1 presents the demographic characteristics of the sample at the three timepoints, including 1599 participants at W1, 1468 at W2, and 1349 at M6. Overall, the majority of the respondents were female (68.5–72.9%), White/Caucasian (65.7–66.8%), 36 years of age or older (73.6–76.1%), and had at least a college degree (69.4–71.3%). Children of the participants were on average 10 years, 10 months old: W1 M(SD) = 10.83 (3.51); W2 M(SD) = 10.87 (3.44); M6 M(SD) = 10.85 (3.47). There were slightly more male children across the three timepoints (W1: 55%, W2: 55.5%, M6: 53.6%). No statistically significant differences were found for any of the demographic variables by recruitment type or across timepoints.

Table 1

Sample demographic characteristics across three data collection timepoints.

VariableTotal n	Week 1n = 1599	Week 2n = 1468	6-Monthn = 1349
	n (%)	n (%)	n (%)
Caregiver SexMaleFemaleNo response provided/Missing	465 (29.1)1132 (70.8)2 (0.1)	452 (30.8)1006 (68.5)10 (0.7)	364 (27.0)983 (72.9)2 (0.1)
Caregiver Age Group20–25 years26–30 years31–35 years36–40 years41–45 years46 years or olderNo response provided /Missing	9 (0.6)61 (3.8)324 (20.3)545 (34.1)386 (24.1)246 (15.4)28 (1.8)	8 (0.5)54 (3.7)282 (19.2)513 (34.9)356 (24.3)217 (14.8)38 (2.6)	9 (0.6)47 (3.5)242 (17.9)464 (34.4)356 (26.4)206 (15.3)25 (1.9)
Caregiver EducationHigh school degree or lessCompleted collegeCompleted a graduate degreeNo response provided /missing	437 (27.3)616 (38.5)511 (32.0)35 (2.2)	405 (27.6)553 (37.7)465 (31.7)45 (30.7)	354 (26.2)492 (36.5)470 (34.8)33 (2.4)
Parent Race/EthnicityWhite/CaucasianBlack/African AmericanAsian/Pacific IslanderNative American (AI/AN/IC)Hispanic/Latinx (of any race)No response provided/Missing	1051 (65.7)88 (5.5)35 (2.1)36 (2.3)139 (8.7)389 (24.3)	980 (66.8)79 (5.4)34 (2.3)35 (2.4)137 (9.3)340 (23.2)	896 (66.4)57 (4.2)29 (2.1)30 (2.2)107 (7.9)337 (25.0)
Child SexMaleFemaleNo response provided /missing	879 (55.0)718 (44.9)1 (0.06)	815 (55.5)644 (43.9)9 (0.6)	723 (53.6)625 (46.6)1 (0.07)
Household INR ClassificationLow Income (INR<2)Adequate Income (2 <INR<4)Affluent (INR>4)No response provided /missing	334 (20.9)589 (36.8)425 (26.6)251 (15.7)	316 (21.5)554 (37.7)382 (26.0)216 (14.7)	250 (18.5)520 (38.5)380 (28.2)199 (14.8)
Child Age Group< 10 years10–17 yearsNo response provided /missing	526 (32.9)870 (54.4)203 (12.7)	482 (32.8)824 (56.1)162 (11.0)	457 (33.9)745 (55.2)147 (10.9)
Child Race/EthnicityWhite/CaucasianBlack/African AmericanAsian/Pacific IslanderNative American (AI/AN/IC)Hispanic/Latinx (of any race)No response provided/Missing	1227 (76.7)122 (7.6)50 (3.1)46 (2.9)215 (13.4)154 (9.6)	1147 (78.1)109 (7.4)49 (3.3)44 (3.0)205 (14.0)119 (8.1)	1068 (79.2)90 (67)45 (3.3)42 (3.1)173 (12.8)104 (7.7)
Recruitment TypeConvenience SampleMTurk	1304 (81.6)295 (18.4)	1232 (83.9)236 (16.1)	1118 (82.9)231 (17.1)
Participant CountryCanadaUnited States	360 (22.5)1239 (77.5)	333 (22.7)1135 (77.3)	315 (23.4)1034 (76.6)
Participant Location RegionCanadaAtlantic RegionOntario & QuebecBritish ColumbiaNorthern TerritoriesPrairie ProvincesUnited StatesNew EnglandMid-AtlanticMidwestSouth AtlanticSouth CentralMountainPacificNo response provided/Missing	5 (0.3)289 (18.1)28 (1.8)1 (0.06)14 (0.9)49 (3.1)147 (9.2)169 (10.6)255 (15.9)122 (7.6)35 (2.2)226 (14.1)259 (16.2)	4 (0.3)262 (17.8)28 (1.9)1 (0.07)14 (1.0)44 (3.0)141 (9.6)152 (10.4)236 (16.1)98 (6.7)34 (2.3)228 (15.5)226 (15.4)	3 (0.2)280 (20.8)24 (1.8)0 (0.0)18 (1.3)48 (3.6)139 (10.3)196 (14.5)238 (17.6)134 (9.9)28 (2.1)180 (13.3)61 (4.5)

Note.AI: American Indian; AN: Alaska Native; IC: Indigenous Canadian; INR: Income-to-Needs Ratio (based on family income and household size)

Descriptive Statistics

All item responses were within the expected limits (minimum of 1, maximum of 4). Measures of central tendency and univariate normality are provided in Table 2. Assumption of normality was upheld for all variables, with all skewness and kurtosis values below determined cut-off. Subscale scores were within normal limits (minimum of 0, maximum of 100) and normally distributed. No notable floor (which ranged from 1.7% to 14.3%) or ceiling effects (which ranged from 0.3% to 6.9%) were observed.

Table 2

FIVE item and scale descriptive statistics.

Item #		Item Content	Mean (SD)	Skewness	Kurtosis
Scale: Contamination & Illness Fears			27.67 (24.52)	0.844	0.065
Rate how often your child felt afraid of worried about each item in the last week…
1	…of getting a bad illness or virus		1.87 (0.924)	0.784	-0.339
2	…of getting very sick		1.76 (0.937)	1.104	0.247
3	…they will have to go to the hospital		1.76 (0.946)	0.999	-0.139
4	…they might die		1.76 (0.997)	1.054	-0.162
5	…their pet might get a bad illness or virus		1.71 (0.979)	1.113	-0.055
6	…a family member might get sick or die		2.07 (0.937)	0.612	-0.469
7	…they may do something to cause someone else to get sick		1.77 (0.973)	1.032	-0.086
8	…a friend might get sick or die		1.85 (0.975)	0.867	-0.378
9	…people in the world might get sick or die		2.01 (0.918)	0.621	-0.444
Rate how true the statement is for your child in the last week…
34	Being afraid has caused my child strong emotions (anger, sadness, anxiety)		1.75 (0.856)	0.856	-0.224
Scale: Social Distancing Fears			40.59 (22.07)	0.284	-0.680
Rate how often your child felt afraid of worried about each item in the last week…
10	…they will be stuck at home		2.12 (0.986)	0.528	-0.744
11	…It will be hard to do things they like		2.24 (0.952)	0.306	-0.837
12	…they will miss a lot of school		2.17 (1.026)	0.398	-1.003
13	…they will not be able to see their friends for a long time		2.41 (0.958)	0.205	-0.895
14	…they will do poorly in school		2.30 (1.058)	0.180	-1.315
15	…they will lose my friends		1.85 (1.024)	0.878	-0.510
16	…they will be sad and lonely		2.46 (0.966)	0.106	-0.978
17	…they will not be able to celebrate good things		2.66 (0.896)	-0.158	-0.759
18	…they will not have enough food or supplies		2.22 (1.025)	0.495	-0.878
19	…their family will not have enough money		2.26 (0.988)	0.395	-0.869
Rate how true the statement is for your child in the last week…
35	Being afraid has gotten in the way of my child enjoying life		1.70	1.036	0.072
Scale: Avoidance Behaviors			31.68 (25.42)	0.767	0.039
Rate how often your child has done the following… (Items 20–33)
21	ask people if they are sick		1.70 (0.950)	1.094	-0.051
22	avoid news or information		1.94 (1.018)	0.696	-0.759
27	avoid touching things		2.05 (1.104)	0.571	-1.084
28	avoid touching people		2.41 (1.190)	0.092	-1.511
29	check the internet for updates		1.79 (1.019)	1.024	-0.237
32	stay away from people inside my house		1.82 (1.077)	0.942	-0.582
Scale: Mitigation Behaviors			44.51 (28.09)	0.407	-0.797
23	wash my hands frequently		2.72 (1.012)	-0.196	-1.093
24	wear a mask or protective gear		1.99 (1.156)	0.699	-1.052
25	use Purell/hand sanitizer		2.49 (1.082)	0.052	-1.272
26	use Clorox/cleaners to wipe down surfaces		2.14 (1.120)	0.450	-1.211
Behavior Items Removed from Final Measure
20	stay away from people		3.02 (1.07)	-0.670	-0.908
30	use social media to stay connected		2.61 (1.096)	-0.134	-1.294
31	exercise outside		2.66 (1.071)	-0.162	-1.243
33	work on my computer		3.12 (0.977)	-0.737	-0.664

SD: standard deviation; Skewness Standard Error (SE) = 0.061; Kurtosis SE = 0.122

Structural validity

The first tested model divided the FIVE items into three subscales: (1) items 1–9 for C&I Fears, (2) items 10–19 for SD Fears, and (3) items 20–33 for Behaviors. This model demonstrated poor fit: (492) = 5635.47, p < 0.001, CFI = 0.775, TLI = 0.758, RMSEA = 0.081 (90% CI 0.079–0.083), SRMR = 0.082. Standardized item factor loadings (λ) ranged from − 0.015 (item 33) to 0.893 (item 15). Although three items demonstrated very low standardized loadings (i.e., item 20 λ = 0.110; item 30 λ = −0.296; item 33 λ = −0.019), they were not removed in the second model, consistent with the a priori hypothesized factor structure. Model 1 fit results were consistent retested at the M6 timepoint, (492) = 4889.91, p < 0.001, CFI = 0.794, TLI = 0.779, RMSEA = 0.081 (90% CI 0.079–0.084), SRMR = 0.094.

The second tested model consisted of five subscales including two impairment items (items 34 and 35), that were each placed within a Fears subscale (C&I Fears, and SD Fears, respectively). The five subscales were as follows: (1) items 1–9 and 34 for C&I Fears, (2) items 10–19 and 35 for SD Fears, (3) Avoidance Behaviors (items 20–22, 27–29, 32), (4) Mitigation Behaviors (items 23–26), and (5) Adaptive Behaviors (items 30, 31, 33). This model demonstrated slightly better fit than Model 1 but was still not adequate for the data: (517) = 5436.73, p < 0.001, CFI = 0.795, TLI = 0.778, RMSEA = 0.077 (90% CI 0.075–0.079), SRMR = 0.075. Standardized item factor loadings (λ) ranged from 0.888 (item 15) to 0.024 (item 33). Consistent with results from Model 1, items 20, 30, 31, and 33 demonstrated low standardized loadings (i.e., item 20 λ = 0.111; item 30 l λ = 0.308, and item 33 λ = 0.024), with the remaining Adaptive Behaviors item also demonstrating factor loading below the recommended cut-off value (item 31 λ = −0.313). The covariances between the C&I Fears latent factor and other latent factors were: SD Fears = 0.821, Avoidance Behaviors = 0.883, Mitigation Behaviors = 0.730, and Adaptive Behaviors = 0.779. The covariances between SD Fears and Avoidance Behaviors = 0.770, with Mitigation Behaviors = 0.537, and with Adaptive Behaviors = 0.622. The Avoidance Behaviors latent factor covariance with Mitigation Behaviors was 0.892 and 1.027 with Adaptive Behaviors. Finally, the covariance between Mitigation Behaviors and Adaptive Behaviors was 0.946. Similar model fit results were found at the M6 timepoint:: (517) = 3692.80, p < 0.001, CFI = 0.825, TLI = 0.810, RMSEA = 0.068 (90% CI 0.065–0.070), SRMR = 0.088.

The third model tested was a modified version of Model 2 after the removal of four problematic items (items 20, 30, 31, 33), which included the entire Adaptive Behaviors subscale, resulting in a four-factor structure. These modifications resulted in an improved model fit on four of five fit indices: (428) = 4965.18, p < 0.001, CFI = 0.806, TLI = 0.789, RMSEA = 0.081 (90% CI 0.079–0.083), and SRMR = 0.072. Additionally, standardized item factor loadings were above the recommended cut-off of λ = 0.400, ranging from λ = 0.438 (item 17) to λ = 0.887 (item 15). Results were similar at M6: (428) = 2966.99, p < 0.001, CFI = 0.840, TLI = 0.826, RMSEA = 0.066 (90% CI 0.064–0.069), SRMR = 0.088.

Next, modifications were made to this model, which had the best fit thus far (Model 3) to create a hierarchical five-factor model. This fourth model consisted of a higher-order factor (FIVE Total Fears) indicated by two lower-order factors (C&I Fears and SD Fears), in addition to the previously tested lower-order factors (C&I Fears, SD Fears, Avoidance Behaviors, Mitigation Behaviors). Consistent with all other model fit results, the test was significant; therefore, other fit indices were analyzed to assess model fit. Fit indices for this model were very similar to the third model, without noticeable worsening or improvement of model fit: (429) = 5033.23, p < 0.001, CFI = 0.803, TLI = 0.787, RMSEA = 0.082 (90% CI 0.080–0.084), SRMR = 0.073. However, this model was more clinically useful by providing a total fears score. Therefore, this model was chosen as the final factor structure model. Similar results were found at M6:: (429) = 2958.69, p < 0.001, CFI = 0.841, TLI = 0.828, RMSEA = 0.066 (90% CI 0.064–0.068), SRMR = 0.088. Factor loadings for this model has been provided for W1 and M6 in Table 3.

Table 3

Multi-trait analysis of FIVE Caregiver-report subscales with corrected item-total correlations (r) between each item and its intended scale are shown in bold. Standardized factor loadings (λ) from final (best fitting) model at W1 and M6.

Item #	C&I Fears				SD Fears				Avoidance Behaviors				Mitigation Behaviors				Total Fears
	W1 r	W1 λ	M6 λ		W1 r	W1 λ	M6 λ		W1 r	W1 λ	M6 λ		W1 r	W1 λ	M6 λ		W1 r
1	.789a	.850	.840		.607				.617				.475				.737
2	.753a	.844	.843		.593				.591				.492				.711
3	.723a	.820	.848		.574				.598				.463				.686
4	.741a	.856	.851		.590				.635				.479				.703
5	.721a	864	.858		.602				.660				.481				.701
6	.691a	.754	.760		.588				.539				.421				.678
7	.730a	.835	.838		.611				.621				.481				.711
8	.776a	.861	.844		.630				.646				.486				.744
9	.661a	.739	.755		.584				.537				.458				.662
34	.607a	.721	.742		.613				.480				.352				.651
10	.527				.614a	.699	.721		.401				.326				.605
11	.486				.598a	.662	.682		.324				.281				.573
12	.484				.585a	.666	.681		.395				.283				.566
13	.379				.506a	.551	.576		.250				.201				.467
14	.653				.719a	.850	.776		.620				.387				.730
15	.689				.683a	.880	.845		.617				.462				.734
16	.534				.673a	.709	.792		.477				.285				.637
17	.273				.470a	.428	.633		.213				.145				.389
18	.621				.611a	.811	.924		.601				.375				.658
19	.605				.615a	.806	.891		.587				.371				.651
35	.418				.429a	.552	.540		.274				.161				.451
21	.689b				.591				.631a	.865	.844		.482				.681
22	.500				.413				.494a	.620	.564		.345				.485
27	.607				.496				.685a	.809	.774		.686				.587
28	.325				.290				.401a	.481	.513		.466b				.327
29	.613				.517				.624a	.772	.789		.521				.601
32	.599				.527				.639a	.799	.783		.475				.599
23	.241				.185				.346				.447a	.441	.402		.226
24	.559				.412				.609b				.577a	.859	.930		.517
25	.446				.339				.526				.629a	.719	.631		.417
26	.551				.425				.632				.647a	.861	.813		.519
Dropped items: Pearson’s r correlations with final subscales at W1
	C&I Fears			SD Fears				Avoidance Behaviors				Mitigation Behaviors				Total Fears
20	-0.072			-0.050				-0.032				-0.016				-0.065
30	.195			.179				.268				.275				.199
31	-0.236			-0.161				-0.249				-0.180				-0.211
33	-0.031			-0.002				-0.043				.158				-0.018

C&I: Contamination & Illness

SD: Social Distancing

bItem-scale correlation is < 0.40.

Item-scale correlation adjusted for overlap (item removed from its scale for correlation)

Item correlates more strongly with competing vs. intended scale by at least 1 standard error

Internal consistency and reliability

Next, the internal consistency of final subscales as indicated by the best fitting model was tested by calculating Cronbach’s alpha (α), McDonald’s omega (ω), and conducting a multi-trait item analysis, including corrected item-total correlations. Results indicated all four subscales demonstrated good internal consistency, with no notable improvements made by removal of items (C&I Fears α = 0.90; ω = 0.91; SD Fears α = 0.87; ω = 0.90; Avoidance Behaviors α = 0.87; ω = 0.92; Mitigation Behaviors α = 0.92; ω = 0.77).

The Total Fears subscale was composed of the C&I and SD Fears subscales, and as predicted, had a very high internal consistency (α = 0.98; ω = 0.98). Corrected item-total correlations supported the membership of an item to its corresponding subscale (See Table 3) (Li et al., 1991). Standard errors (SE) of r were calculated for each of the items as a guide for determining whether an item demonstrated a stronger relationship with subscale other than its intended subscale (i.e., if the correlation value with the competing subscale is greater than one SE of the correlation with the intended subscale) (Hays & Hayashi, 1990). Results indicated acceptable assignment of items to their intended subscales and supported the removal of items dropped from the best fitting model.

Test-retest reliability and concurrent validity

An intraclass correlation coefficient (ICC) for consistency using a two-way mixed effects model was calculated and supported test-retest reliability of the Total Fears score after a one-week period (ICC = 0.877). Concurrent validity was supported through the correlation found between the FIVE Total Fears score and parent-reported child anxiety symptoms (Y-OASIS Pearson’s r = 0.705, p < 0.001), which indicated that greater COVID-19-related fears were strongly correlated with higher elevations in anxiety symptoms.

Measurement invariance

Multigroup CFA with the best fitting model were conducted to test for measurement invariance as a function of country of residence (Canada vs. USA), child age (<10 years vs. 10–17 years), and child sex. See Table 4 for model fit results. Although all Δχ2 tests were statistically significant, this was not considered an indicator of variance across groups due to the large sample size. All models were evaluated using the fit indices indicated above (i.e., CFI, TLI, RMSEA, SRMR). None of the models tested demonstrated worsening fit as additional parameter constraints were placed (e.g., configural vs. metric vs. scalar), indicating model fit did not worsen as a function of child age or sex.

Table 4

Measurement invariance model fit statistics and indices (multigroup CFA and nested model comparison).

Model	df	WLSMV χ^2	CFI	TLI	RMSEA	SRMR
Country (Canada vs. USA)ConfiguralMetric	858886	4337.4423430.110	0.7040.784	0.6790.773	0.0710.060	0.0780.080
Child Age Group (<10 years vs. 10–17 years)ConfiguralMetric	858886	5735.874567.44	0.7610.819	0.7400.810	0.0840.072	0.0810.089
Child SexConfiguralMetric	858886	5158.7524018.082	0.7380.809	0.7160.800	0.0790.067	0.0760.082
Nested Model Comparison &Differences in Fit Indices	df	Δχ2 (p-value)	ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR
Country (Canada vs. USA)Metric -- Configural	28	32.97 (0.24)	0.080	0.094	-0.011	0.002
Child Age Group(<10 years vs. 10–17 years)Metric -- Configural	28	85.82 (<0.001)	0.059	0.07	-0.012	0.008
Child SexMetric -- Configural	28	50.804 (0.005)	0.071	0.084	-0.013	0.006
df: degrees of freedom; WLSMV: Mean- and Variance-Adjusted Weighted Least Square.CFI: Comparative Fit Index; TLI: Tucker-Lewis Index: RMSEA: Root Mean Square Error of Approximation.SRMR: Standardized Root Mean Square Residual; Δχ2: change in Satorra-Bentler scaled chi-square tested.ΔCFI: change in scaled CFI; ΔTLI: change in scaled TLI; ΔRMSEA: change in RMSEA; ΔSRMR: change in SRMR.

Discussion

Although a number of measures have been developed to assess the impact of COVID-19 on individuals and families, (Ahorsu et al., 2020, Chi et al., 2021a, Chi et al., 2021b, Masuyama et al., 2020, Sakib et al., 2020, Taylor et al., 2020a, Zenker et al., 2021) the present study examining the FIVE caregiver-report offers the first empirical evaluation of such a measure specifically designed for youth populations. Results indicated that a hierarchical five-factor model, created after removing four problematic items and adding a higher-order Total Fears factor, provided the best fit for the data. These results were replicated with the M6 timepoint. Notably, the behaviors measured by the three items on the Adaptive Behaviors subscale (i.e., child uses social media/FaceTime to stay connected to friends; child exercises or plays outside; child does schoolwork on a computer) were endorsed by the majority of respondents regardless of their responses on other items or scales. For example, 73% of caregivers reported their child did schoolwork on a computer most or all of the time. Therefore, these items did not provide meaningful information in relation to COVID-19-related fears and worries, but instead reflected behaviors that were likely already common prior to the pandemic. Thus, the final model consisted of two subscales measuring fears (i.e., Contamination and Illness Fears, Social Distancing Fears), two behavior subscales (i.e., Avoidance Behaviors, Mitigation Behaviors), and a higher-order Total Fears factor consisting of the two fear subscales. Measurement invariance was supported across country of residence, child age, and child sex. All FIVE subscales demonstrated strong internal consistency. Results of the multitrait-item analysis indicated acceptable assignment of items to their intended subscales, supported the removal of four items dropped from the best fitting model, and revealed appropriate item-scale discrimination.

Research conducted since the start of the COVID-19 pandemic indicates widespread psychological impacts on the general population, including increased elevations in symptoms of depression, anxiety, and overall psychological distress (de Figueiredo et al., 2021, Hawes et al., 2021, Magson et al., 2021, Murata et al., 2020, Ravens-Sieberer et al., 2021, Roccella, 2020, Rogers et al., 2021, Wang et al., 2020). Although numerous COVID-19-specific measures have been developed and validated, the FIVE caregiver-report measure was specifically designed for use with children and adolescents. The FIVE caregiver-report appears to be a structurally valid and reliable measure that can be used to capture child contamination and illness fears related to COVID-19, child fears associated with the impact of social distancing restrictions, while measuring associated behaviors in response to these fears. The FIVE differs from other measures of anxiety developed for COVID-19, most of which include items that measure general symptoms of anxiety and are not specific to COVID-19, in its inclusion of items specific to the pandemic (e.g., fears related to social isolation, financial impact, adequate access to supplies). Additionally, the fears and worries captured by the FIVE are not limited to the impact of COVID-19 on the individual, but also measures health or illness-related anxiety as it relates to the pandemic’s impact on loved ones and the larger society.

The FIVE has demonstrated its utility in its different versions and across different languages. Research teams across the world have translated the FIVE scales (caregiver-report, child-report, and adult versions) and tested them in several populations either in its original hypothesized structure or by analyzing the factor structure. The Arabic FIVE adult-version was tested with 509 adults in Saudi Arabia and reported strong internal consistency based on Cronbach’s α (C&I Fears α = 0.89; SD Fears α = 0.91) (Tounsi, Alammar, & Almaflehi, 2021). However, results of their factor analyses found an alternate factor structure that divided C&I Fears into two subscales: “Fears of Getting Sick” and “Fears that Others Get Sick,” while keeping the SD Fears subscale together, and testing the Behavior items separately (Tounsi et al., 2021).

The Spanish-language FIVE adult-version, tested with 163 adults, (Cottin, Hernández, & Núñez, 2021) also divided the C&I Fears into two subscales: Fears of Getting Sick (α = 0.88) and Fears that Others May Get Sick (α = 0.74). Additionally, SD Fears were also divided into two subscales: Fears of Concrete Limitation (α = 0.85), and Fears of not being able to meet Basic Needs (α = 0.79). The Behavior items (α = 0.87) and Impact items (α = 0.84) were kept separate as supplemental items (not included as part of the four-factor structure). The four fears scales were all found to significantly predict symptoms of depression and post-traumatic stress (Cottin et al., 2021). The Turkish translation of the FIVE adult-version was tested in its original hypothesized factor structure and found to strongly correlate with symptoms of anxiety (r = 0.83) and moderately correlate with depressive symptoms (r = 0.66) (Çölkesen, Kilincel, & Sozen, 2021).

A study of adolescents using the FIVE child-report version did not test the factor structure but provided internal consistency using Cronbach’s α for some of the subscales: SD Fears α = 0.86; 7-item C&I Fears subscale α = 0.84, 9-item Behaviors α = 0.84 (Trucco, Fallah-Sohy, & Hartmann, 2022). This study reported that SD Fears was mildly correlated with measures of stress and anxiety symptoms (Trucco et al., 2022). A separate study of 31 adolescents used the self-report C&I Fears (α = 0.78) and the Behaviors subscales (α = 0.70) and found that greater C&I Fears predicted lower daily physical activity; while greater engagement in behaviors predicted greater daily physical activity (Cummings, Humiston, Cohen, & Lansing, 2021). Another study using the FIVE child-report version adapted the self-report version for 7–10 year old students by adding visuals, kept the original hypothesized factor structure, but did not provide information on the psychometric properties (Bhogal, Borg, Jovanovic, & Marusak, 2021).

In addition to its utility for research related to COVID-19, the FIVE was designed to be useful in future public health crises, particularly as it refers generically to a “bad virus or illness” and would not require alteration to item content. Another strength of this measure and validation study is the use of feedback from experts to enhance face validity and the assessment of the interpretability of items using two different estimates of their reading level. The FIVE's utility as a clinical measure or one that has incremental validity in comparison to existent measures of youth anxiety and avoidance behaviors will be an important target for further investigation.

Fear and anxiety are associated with compliance to health behaviors (Cameron et al., 2015). Specifically, the likelihood that someone will engage in a specific health behavior (e.g., handwashing) can be influenced by the individual’s level of fear or anxiety related to the predicted negative outcome (e.g., getting sick). If the level of fear is very low, it may lead to less motivation to engage in health behaviors, while very high levels of fear may result in avoidant behaviors (Asmundson and Taylor, 2020aaa, Asmundson and Taylor, 2020bbb, Cameron et al., 2015). Thus, the inclusion of avoidance and mitigation behaviors in this measure may provide helpful information for the development of public health strategies to both promote mental health and to increase adherence to mitigation strategies and increase uptake of vaccinations (Asmundson and Taylor, 2020bbb, Harper et al., 2020, Taylor et al., 2020b).

This study does have some limitations. First, the study was conducted towards the beginning of the pandemic and does not capture the potential long-term impact of this type of stressor or the effect of pandemic fatigue on fears and behaviors. Long-term follow-up data will add to the robustness of our knowledge about the measure’s predictive validity and the course of COVID-19 fears and behaviors in youth over time. Moreover, the generalizability of the present sample can be called into question, and, thus, future psychometric work will be needed in more representative samples to further validate the measure.

The FIVE itself also has some limitations. For example, the behavior subscales measure the frequency of the reported behavior and do not directly assess their impact or whether the behavior may be an adaptive or helpful response. Additionally, certain behaviors may be adaptive only up to a particular level or frequency, which is not addressed by this measure. As a measure of child fears and behaviors, it is also important to acknowledge that some of the behaviors may not be completely up to the child and may be dependent on the caregiver’s choices; thus, the association between the child’s fears and such behaviors may be indicative of a caregiver’s own fears and anxiety, more than that of the child.

Although prior research indicates that caregiver-report of child symptoms are reliable and and helpful, (Lifland et al., 2018, Poulain et al., 2020) the evidence on parent/caregiver-child agreement for anxiety symptoms has been mixed. Notably, higher agreement between parent and child reports of child anxiety symptoms have been found when measuring observable symptoms (Comer & Kendall, 2004). Therefore, future steps should include the evaluation of the caregiver-version of the measure in combination with the child self-report measure. However, child-report data from the FIVE has been relatively slower to aggregate, making direct comparisons inaccessible at the present time. Future studies should also include measures of pandemic fatigue and related factors that were not yet identified at the beginning of the pandemic, when data was initially collected for this study. Pandemic fatigue is associated with a decrease in compliance with public health policies and mitigation strategies to contain the spread of COVID-19 (Lilleholt, Zettler, Betsch, & Böhm, 2021). Therefore, future studies are needed to use the FIVE to explore relationships between COVID-19-related fears and anxiety, related behaviors, and pandemic fatigue in youth.

Despite limitations, results of this study provide support for the face and content validity, structural validity, and internal consistency of the FIVE caregiver-report. All FIVE items included in the final subscales demonstrated appropriate item-total correlations with their assigned subscales, as well as appropriate item-scale discrimination. All subscales had satisfactory internal consistency. Therefore, the FIVE caregiver-report offers a rare, psychometrically robust and useful tool that was specifically developed to assess the fears, worries, and associated behaviors of children and adolescents in the context of a pandemic.

Funding

(PI: Comer), Boston University Center for Children and Families, Florida International University Mental Health Interventions and Technology (MINT) Program, University of Miami College of Arts and Sciences (PI: Ehrenreich-May), Brock University COVID-19 Related Dean’s Discretionary Fund (PI: Angela Evans).

Disclosures

Dr. Comer earns textbook royalties from Macmillan Learning for work unrelated to the present study. Dr. Peris receives royalties from Oxford University Press and consulting fees from Odin. Dr. Ehrenreich-May receives royalties from Oxford University Press and consulting fees for unrelated psychotherapy training and implementation services.