Validation of the COVID-19 Exposure and Family Impact Scales.

OBJECTIVE: The COVID-19 Exposure and Family Impact Scales (CEFIS) were developed in Spring 2020 to assess effects of the COVID-19 pandemic on families and caregivers. Initial psychometric properties were promising. The current study examined the factor structure and evaluated convergent and criterion validity of the CEFIS in a new sample.
METHODS: In October and November 2020, caregivers (N = 2,531) of youth (0-21 years) scheduled for an ambulatory care visit at Nemours Children's Hospital, Delaware completed the CEFIS and measures of convergent (PROMIS Global Mental Health Scale, Family Assessment Device) and criterion validity (PTSD Checklist-Civilian). Confirmatory factor analysis was used to examine the factor structure of the CEFIS. Bivariate correlations and logistic regression were used to examine convergent and criterion validity.
RESULTS: Factor analysis supported the original six- and three-factor structures for the Exposure and Impact scales, respectively. Second-order factor analyses supported the use of Exposure, Impact, and Distress total scores. Higher scores on the CEFIS Exposure, Impact, and Distress scales were associated with increased mental health concerns and poorer family functioning. Higher scores on all CEFIS scales were also associated with greater odds of having clinically significant posttraumatic stress symptoms.
CONCLUSIONS: The CEFIS is a psychometrically sound measure of the impact of the COVID-19 pandemic on family and caregiver functioning and may also be useful in identifying families who would benefit from psychological supports.

Introduction

The Coronavirus Disease 2019 (COVID-19) global pandemic has impacted nearly every aspect of children’s and families’ lives. Families have experienced disruptions to daily life (e.g., changes to childcare, school, and employment, increased isolation, and reduced access to resources), some of which persist. In addition, youth and caregivers have been directly or indirectly (e.g., social media, news, extended social networks) exposed to illness, hospitalization, and death due to COVID-19. Clearly, aspects of the COVID-19 pandemic can be conceptualized as potentially traumatic medical events (Price et al., 2016) that increase risk for mental health concerns (Horesh & Brown, 2020; Russell et al., 2020; Wu et al., 2020). Indeed, psychological functioning has deteriorated among youth and adults during the COVID-19 pandemic (Gassman-Pines et al., 2020; Panchal et al., 2020; Patrick et al., 2020; Westrupp et al., 2021). Given that the COVID-19 pandemic is ongoing and new variants may result in future restrictive public health measures, it is critical to investigate its evolving impact on child and family well-being.

The COVID-19 Exposure and Family Impact Scales (CEFIS) were developed using a trauma framework to help researchers and clinicians better understand the impacts of COVID-19 on families (Kazak et al., 2021). The CEFIS is a caregiver-report measure used to examine the degree to which families are exposed to potentially traumatic aspects of the COVID-19 pandemic and the perceived impact of these experiences on child, caregiver, and family functioning. The CEFIS consists of three primary scales: Exposure, Impact, and Distress. To date, 168 users in 6 countries (United States, Australia, Spain, India, Mexico, and Colombia) have registered to use the CEFIS, and the measure is available in English, Spanish, Italian, and Brazilian Portuguese. Data from a large sample recruited from across the United States supported the initial factor structure and internal consistency of the CEFIS (Kazak et al., 2021). Exploratory factor analysis (EFA) also supported six factors within the Exposure scale and two factors within the Impact scale (Kazak et al., 2021). The CEFIS has been used to examine the impact of COVID-19 on multiple pediatric chronic illness populations (Fisher et al., 2021; Forner-Puntonet et al., 2021; Stiles-Shields et al., 2021). However, the validity of the CEFIS has yet to be examined, and the factor structure has not been confirmed in an independent sample.

In the current study, we aimed to validate the factor structure and examine convergent and criterion validity of the CEFIS and explore the preliminary psychometrics of the Spanish-language version of the CEFIS. It was hypothesized that results from the confirmatory factor analysis (CFA) would support the first- and second-order factor structures reported by Kazak et al. (2021). It was also hypothesized that higher scores on the CEFIS scales would be associated with worse caregiver global mental health and general family functioning, as well as greater odds of having clinically significant posttraumatic stress symptoms.

Methods

Participants and Procedures

All procedures were approved by the Nemours IRB prior to data collection (IRB # 1613768). The recruitment process is detailed in Figure  1. Families (n = 15,000) of youth (0–21 years old) who had an ambulatory care visit scheduled at a Mid-Atlantic children’s healthcare system in April 2020 were invited to participate in a prospective cohort study examining the effects of the COVID-19 pandemic on family functioning and engagement with the healthcare system. Caregivers were excluded if they were not proficient in English or Spanish, did not have a mobile phone number listed in their child’s electronic health record (EHR), opted out of contact for research participation, or if their child was not a current patient in our healthcare system (i.e., no encounter in the past year).

Figure 1.

Study inclusion and participation.

Note. N’s refer to unique families and not individual caregivers; race and ethnicity are for the child and not the caregiver.

Data reported in this study are from a first wave of data collection that occurred between October and November 2020. At that time, all states served by Nemours Children’s Hospital, Delaware had initiated reopening plans (e.g., 25%–50% capacity dining, nonessential businesses reopening, schools open in “hybrid” models), COVID-19 testing sites were established, and plans for COVID-19 vaccine distribution were in development. Caregivers received a text message and email inviting them to participate in a study about COVID-19, family well-being, and child health. Messages included a Research Electronic Data Capture (REDCap; Harris et al., 2019) link to an electronic informed consent form and study questionnaires (detailed below). Recruitment messages, consent/assent forms, and questionnaires were available in Spanish for all caregivers whose preferred language was Spanish. Text message reminders were sent every 3 days for 4 weeks or until the survey was completed. Email invitations were sent twice, at the start of data collection and approximately 4 weeks later, to increase response rates. Caregivers of Non-Hispanic Black or Hispanic youth, who had public insurance, lived in a rural area (defined by a Rural–Urban Community Area [RUCA] code ≥ 4; U.S. Department of Agriculture, 2020), or spoke Spanish were oversampled to ensure adequate representation. A total of 2,531 caregivers from 2,403 families consented to participate.

Demographic and Clinical Data

Demographic data (child race/ethnicity, child age, language spoken at home), type of insurance, and zip code (used to calculate RUCA code) were extracted from the EHR. Caregivers reported their gender.

COVID-19 Exposure and Family Impact Scales

The CEFIS is a 37-item caregiver-report measure that assesses how the COVID-19 pandemic affects families (Kazak et al., 2021). The CEFIS was translated from English to Spanish during its development by a bilingual research team member; a certified medical interpreter independently reviewed and certified the translation (Kazak et al., 2021). The CEFIS includes three primary scales: Exposure, Impact, and Distress (Table  I). When completing the CEFIS, caregivers are asked to think about what has happened from March 2020 to the present. The Exposure scale contains 25 Yes/No items that measure whether families have been exposed to COVID-19-related events such as lockdowns, school closures, changes in employment, or the virus itself. “Yes” responses are summed to yield a total score, with higher scores indicating greater exposure to COVID-19 and related events. The Impact scale includes 10 items that measure the impact of COVID-19 on family relationships and emotional well-being using a 4-point Likert scale ranging from 1 (Made it a lot better) to 4 (Made it a lot worse). The Distress scale uses two 10-point distress scales to assess how much distress caregivers and their children have experienced due to the pandemic. Item responses are averaged within each scale to yield the Impact and Distress scale scores, respectively. In prior work, internal consistencies were excellent for the Exposure (α = .80) and Impact (α = .92) scales and good (α = .76) for the Distress scale (Kazak et al., 2021).

Table I.

CEFIS Scales and Factors

Scale	Factor	Example item
Exposure Scale—Six Factors
	COVID-19 Experience	“Someone in the family had COVID-19 symptoms”
	Accessing Essentials	“We had difficulty getting food”
	Disrupted Living Conditions	“We self-quarantined due to travel or possible exposure”
	Income Loss	“Family income decreased”
	Family Caregiving & Activities	“We had a ‘stay-at-home’ order”
	Designated Essential Worker	“Someone in s the family was an essential worker”
Impact Scale—Two Factors
	Personal Well-Being	“How has the COVID-19 pandemic affected your emotional well-being—anxiety?”
	Family Interactions	“How has the COVID-19 pandemic affected parenting?”
Distress Scale		“Overall, how much distress have you experienced related to COVID-19?”

Two EFAs described the structure of the Exposure and Impact scales. The six Exposure factors characterize different ways in which families may have been exposed to the COVID-19 pandemic: COVID-19 Experiences (degree to which family members were sick from COVID-19), Accessing Essentials (e.g., food, medication), Disrupted Living Conditions, Income Loss, Disruptions to Family Caregiving & Activities, and Designated Essential Worker. Internal consistencies varied widely for these factors (α’s = .38 to .78; Kazak et al., 2021). The Impact scale had two factors (Personal Well-Being, Family Interactions), both of which had excellent internal consistency (both α’s = .86). Of note, the Distress scale was originally part of the Impact scale. Results from the EFA suggested that these items represented their own factor. This finding combined with the difference in response scales lead to the decision to score the Distress scale separately from the other Impact items (Kazak et al., 2021).

Measures of Convergent Validity

Convergent validity was evaluated against established measures of family functioning and caregiver mental health. The six positive items of the McMaster Family Assessment Device (FAD-GF+) were used to measure family functioning (Boterhoven de Haan et al., 2015). Caregivers rated how much they agreed with items such as “We can confide in each other” using a 4-point Likert scale ranging from 1 (Strongly Agree) to 4 (Strongly Disagree). Responses are averaged to yield a total score, where higher scores indicated worse family functioning. Caregiver mental health was assessed using the two-item Patient Reported Outcome Measurement Information System (PROMIS) Global Mental Health (GMH) scale (Hays et al., 2017). Caregivers rated their mental health and social relationships using a 5-point Likert scale ranging from 1 (Poor) to 5 (Excellent). Responses are converted to T scores, where higher scores indicated better mental health. The PROMIS GMH scale and FAD-GF+ demonstrated good (α = .78) and excellent (α = .95) internal consistencies, respectively.

Measure of Criterion Validity

The six-item PTSD Checklist—Civilian (PCL-C; Lang & Stein, 2005; Lang et al., 2012; Weathers et al., 1993) was used to evaluate criterion validity. Caregivers indicated how much they were bothered by problems such as “Feeling distant or cut off from other people” in the past month using a 5-point Likert scale ranging from 1 (Not at all) to 5 (Extremely). Responses are summed to create a total score, with higher scores indicating greater posttraumatic stress disorder (PTSD) symptoms. A cutoff score of 14 (Lang & Stein, 2005) was used to identify participants experiencing clinically significant PTSD symptoms. The PCL-C had excellent internal consistency (α = .89) in the current sample.

Statistical Analysis

Factor Analysis

Statistical analyses were performed using Mplus v8.3 and IBM SPSS v27. Descriptive statistics were used to characterize the demographic characteristics, type of insurance, and variables used in validity analyses. CFA was used to evaluate the factor structure of the CEFIS reported in the initial validation paper (Kazak et al., 2021). We evaluated the fit of four models. Models 1 and 2 tested the first-order factor structures in which individual items loaded onto the six Exposure factors or three Impact factors. In Models 3 and 4, we examined second-order factor structures with the respective factors loading onto Exposure or Impact total scores. The weighted least square means and variances (WLSMV) estimation method was used for Model 1 because the exposure items are binary (Rhemtulla et al., 2012). The maximum likelihood estimation method was used for all other models. Chi-square (p < .05), root mean square error of approximation (RMSEA) (< .08), and comparative fit index (CFI) (> .90) were used to evaluate model fit (Hu & Bentler, 1999).

Convergent and Criterion Validity

Convergent validity was evaluated via Pearson’s r correlations between the three CEFIS scale scores and the PROMIS GMH and FAD-GF+. Criterion validity was evaluated using binomial logistic regression with CEFIS scale scores predicting PCL-C scores above the clinical cutoff after controlling for demographic covariates significant in bivariate analyses.

Results

Demographics and Descriptives

Demographics and descriptive data are presented in Table  II, and item-level data for the CEFIS are presented in Supplemental Table I. The sample (n = 2,531) was diverse with regards to race/ethnicity, type of insurance, and rurality and was generally representative of the broader healthcare system population (Figure  1). Based on the CEFIS Exposure score, families reported experiencing, on average, about eight unique COVID-related events. School closures (90.0%) and stay-at-home orders (83.0%) were the most commonly endorsed exposure items, whereas few caregivers endorsed events such as moving out of their home (3.8%) or losing health insurance (4.3%). Approximately 28% of caregivers reported that someone in their family was exposed to COVID-19, and about 4% reported that someone in the family was hospitalized with COVID-19, admitted to the intensive care unit, or died from COVID-19. Average CEFIS Distress and Impact scores were both greater than the midpoints of 2.5 and 5, respectively, which indicates negative valence (Kazak et al., 2021). The means and SDs of the Exposure, Impact, and Distress scores were similar to those of the initial validation paper (Kazak et al., 2021). The average score on the PROMIS GMH Scale was slightly lower than the average of 50 but still within one standard deviation, suggesting that caregiver mental health was generally in the low-average range. Scores on the PCL-C varied widely, and the average score was just below the clinical cutoff of 14; over one third of caregivers (39.7%) endorsed clinically significant PTSD symptoms.

Table II.

Demographics and Descriptives

	N	%
Child Race/Ethnicity
Hispanic	352	13.9
Non-Hispanic Black	541	21.4
Non-Hispanic White	1,323	52.3
Othera	315	12.4
Caregiver gender
Male	437	17.3
Female	2,066	81.2
Other	28	1.1
Insurance
Public	998	39.4
Private	1,463	57.8
Self-pay	70	2.8
Ruralb
No (RUCA < 4)	2,186	86.4
Yes (RUCA ≥ 4)	345	13.6
Language
English	2,407	95.1
Spanish	124	4.9
Total	2,531	100.0
	M (SD)	Min–Max
CEFIS scales
Exposure (six-factor)	8.43 (3.39)	0–24
Exposure (five-factor)	8.10 (3.19)	0–21
Impact	2.66 (0.63)	1–4
Distress	5.62 (2.21)	1–10
PROMIS—Global Mental Health	42.75 (8.47)	25.80–64.60
Family Assessment Device	1.70 (0.68)	1–4
PTSD Checklist—Civilian	13.71 (5.80)	2–30

Note. RUCA = Rural–Urban Community Area; CEFIS = COVID-19 Exposure and Family Impact Scales; PROMIS = Patient-Reported Outcomes Measurement Information System; PTSD = posttraumatic stress disorder.

Other includes Multiracial, Asian, Native Hawaiian/Other Pacific Islander, American Indian/Alaskan Native, or some other race.

Rural–Urban Commuting Area.

Exposure Scale

Fit indices suggested good model fit for six factors within the Exposure scale (i.e., CFI > .90; RMSEA < .08), and factor loadings for individual items for most Exposure factors were strong (i.e., ≥ .40; see Table  III). Results for the Exposure scale second-order factor structure suggested good model fit (Table  III). The Accessing Essential, Disrupted Living Conditions, and Income Loss factors loaded strongly onto the overall Exposure scale. The loadings for the COVID-19 Experience and Family Caregiving & Activities factors were just below .40; however, the loading for the Designated Essential Worker factors was notably low. Internal consistencies for half of the Exposure factors were good (≥ .60), while the other half were suboptimal (< .60). The internal consistency for the overall Exposure scale was good for English-speaking (α = .71) and Spanish-speaking families (α = .68). Although some of the internal consistencies with individual factors were lower, the results suggest that the six-factor structure fit the data well and support the use of a total Exposure scale, which demonstrated good reliability.

Table III.

Exposure Six-Factor Solution—First- and Second-Order Factor Structures

Factor	Item		Factor loadings (first order)	Factor loadings (second order)
COVID-19 Experience (α = 0.68)				0.34
A family member…	Exposed to COVID-19		0.72
	Had symptoms of COVID-19		0.89
	Hospitalized w/COVID-19		0.98
	In ICU for COVID-19		0.99
	Died from COVID-19		0.77
Accessing Essentials (α = 0.65)				0.49
We had difficulty getting…	Food		0.89
	Medicine		0.86
	Health care		0.64
	Other essentials		0.67
Disrupted Living Conditions (α = 0.34)				0.58
	Our family lived separately		0.43
	Someone moved into our home		0.26
	We had to move out of our home		0.42
	Family member lost their job permanently		0.73
	We lost health insurance/benefits		0.64
	We self-quarantined		0.35
Income Loss (α = 0.74)				0.50
	Our family income decreased		0.92
	Family member cut back hours at work		0.90
	Family member furloughed		0.74
Disruptions to Family Caregiving & Activities (α = 0.59)				0.30
	We had a “stay-at-home” order		0.61
	Schools/childcare centers were closed		0.81
	Our child/ren’s education was disrupted		0.75
	Unable to visit/care for a family member		0.66
	Important family event missed/cancelled		0.50
Designated Essential Worker (α = 0.51)				0.16
A family member…	Was an essential worker		0.89
	Is a healthcare provider or first responder		0.73
Model fit statistics	χ2 (df)	RMSEA (95% CI)		CFI
First order	1,262.23 (260), p < .001	.039 [.037, .041]		.96
Second order	98.86 (5), p < .001	.063 [.052, .074]		.90

Note. RMSEA = root mean square error of approximation; CFI = comparative fit index.

Due to lower factor loadings for individual items in the first-order CFA and factors in the second-order CFA, as well as suboptimal internal consistencies, EFA was used to examine alternative models. The sample was randomly split in half. The first half was used in an EFA using the WLSMV estimation method and varimax rotation. Examination of the scree plot and eigenvalues (>1) suggested four-, five-, six-, seven-, or eight-factor solutions. The four-, six-, seven-, and eight-factor solutions had multiple items that cross-loaded (i.e., > .30), and thus these solutions were rejected. The five-factor solution minimized cross-loadings, but three items from the Disrupted Living Conditions factor of the six-factor solution had low factor loadings (i.e., < .40), and its two remaining items loaded onto other factors (i.e., Income Loss and COVID-19 Experience).

First- and second-order CFAs of the five-factor solution identified in the EFA were then conducted using the second half of the sample. Fit indices indicated good model fit for the first- (Table  IV) and second-order (Table  III) factor structures. In the first-order factor structure, factor loadings were strong and internal consistencies for three of the five factors were good (≥ .60; Table  IV). In the second-order factor structure, three of the Exposure factors loaded weakly onto the overall scale (Table  III), but internal consistency for the overall scale remained good (α = .70).

Table IV.

Exposure Five-Factor Solution—First- and Second-Order Factor Structures

Factor	Item		Factor loading (first order)	Factor loading (second order)
COVID-19 Experience (α = 0.65)				0.28
	Exposed to COVID-19		0.74
	Symptoms of COVID-19		0.89
	Hospitalized w/COVID-19		0.98
	ICU for COVID-19		0.99
	Died from COVID-19		0.76
	We self-quarantined		0.46
Accessing Essentials (α = 0.65)				0.61
We had difficulty getting…	Food		0.89
	Medicine		0.85
	Healthcare		0.65
	Other essentials		0.67
Income Loss (α = 0.7)				0.49
	Family member lost their job permanently		0.69
	We lost health insurance/benefits		0.63
	Our family income decreased		0.91
	Family member cut back hours at work		0.89
	Family member furloughed		0.72
Family Caregiving & Activities (α = 0.59)				0.31
	We had a “stay-at-home” order		0.62
	Schools/childcare centers were closed		0.81
	Our child/ren’s education was disrupted		0.75
	Unable to visit/care for a family member		0.64
	Important family event missed/cancelled		0.5
Designated Essential Worker (α = 0.51)				0.15
A family member…	Was an essential worker		0.88
	Is a healthcare provider/first responder		0.73
Model fit statistics	χ2 (df)	RMSEA (95% CI)		CFI
First order	549.22 (199), p < .001	.037 [.034, .041]		.97
Second order	42.44 (4), p < .001	.054 [.040, .070]		.93

Note. RMSEA = root mean square error of approximation; CFI = comparative fit index.

Impact Scale

Fit indices also suggested good model fit for the three factor Impact scale, with strong individual item loadings (Table  V). The internal consistencies for all three Impact factors ranged from good (≥ .70) to excellent (≥ .80; Table  V). The second-order factor structure for the Impact scale was just-identified; therefore, model fit statistics could not be interpreted. The Personal Well-Being, Family Interactions, and Distress factors all loaded strongly onto the Impact Scale. The internal consistency for the Impact total scores was excellent for English-speaking families (α = .85; Table  III) and good for Spanish-speaking families (α = .77). The internal consistency for the Personal Well-Being, Family Interactions, and Distress factors followed a similar pattern.

Table V.

Impact Factors—First- and Second-Order Factor Structures

Factor	Item		Factor loading (first order)	Factor loading (second order)
Family Interactions (α = .87)				0.85
	Parenting		0.79
	Family getting alone		0.67
	Ability to care for your child with [medical condition]		0.78
	Ability to care for other children		0.80
	Ability to care for older adults/ppl with a disability		0.65
Personal Well-Being (α = .87)				0.67
	Exercise		0.57
	Eating		0.61
	Sleeping		0.73
	Anxiety		0.87
	Mood		0.89
Distress (α = .70)				0.57
	Caregiver		0.94
	Child		0.58
Model fit statistics	χ2 (df)	RMSEA (95% CI)		CFI
First order	703.46 (50), p < .001	.073 [.068, .078]		.95
Second ordera	–	–		–

Note. RMSEA = root mean square error of approximation; CFI = comparative fit index.

Saturated model and therefore cannot interpret fit statistics.

Convergent Validity

Pearson’s r correlations between CEFIS scales and the PROMIS GMH and FAD-GF+ are reported in Table  VI. As hypothesized, for both English- and Spanish-speaking caregivers, the CEFIS Exposure (five- and six-factor structures), Impact, and Distress scales had moderate, negative correlations with the PROMIS GMH scale (r’s = −.23 to −.46, p’s < .01) and small, positive correlations with the FAD-GF+ (r’s = .11 to .17, p’s < .01). Caregivers reporting greater exposure to pandemic-related events and greater perceived impact and distress due to COVID-19 also reported worse mental health and family functioning.

Table VI.

Bivariate Correlations Demonstrating Convergent Validity

	Exposure (six-factor)	Exposure (five-factor)	Impact	Distress	PROMIS
CEFIS—Exposure (Five-factor)	.99***	–
CEFIS—Impact	.25***	.26***	–
CEFIS—Distress	.37***	.38***	.50***	–
PROMIS Global Mental Health	−.23***	−.23***	−.54***	−.46***	–
Family Assessment Device (FAD)	.11***	.10***	.16***	.17***	−.18***

Note. CEFIS = COVID-19 Exposure and Family Impact Scales.

p < .001.

Criterion Validity

In Pearson chi-square analyses, a greater proportion of caregivers who spoke English had clinically significant PCL-C scores (40.7%) when compared with caregivers who spoke Spanish (17.7%; Χ2 [df = 1], = 26.19, p < .001). Therefore, caregiver language (English vs. Spanish) was entered as covariate in logistic regression analyses; no other demographic variables (rurality, child race/ethnicity, caregiver gender) were associated with clinically significant PCL-C scores (p’s > .05). For English-speaking caregivers, higher scores on the CEFIS Exposure (aOR6-Factor 1.05, 95% CI [1.02, 1.08]; aOR5-Factor 1.04, 95% CI [1.01, 1.08]), Impact (aOR 2.62, 95% CI [2.17, 3.17]), and Distress scales (aOR 1.39, 95% CI [1.32, 1.47]) were associated with greater odds of having clinically significant PCL-C scores. For Spanish-speaking caregivers, higher scores on the Distress scale were associated with greater odds of having clinically significant PCL-C scores (aOR 1.42, 95% CI [1.08, 1.87]); neither the Exposure (aOR6-Factor 1.01, 95% CI [0.85, 1.20]; aOR5-Factor 1.01, 95% CI [0.84, 1.21]) nor the Impact scale (aOR 1.41, 95% CI [0.56, 3.57]) were associated with odds of having clinically significant PCL-C scores.

Discussion

Results from a large and diverse sample of families in the first 6 months of the COVID-19 pandemic provide strong support for the validity of the CEFIS and mixed support for the factor structure and reliability. Specifically, first-order CFA results support the six- and three-factor structures of the Exposure and Impact scales, and second-order CFA results support the use of total scores for the Exposure and Impact scales. Results from an EFA suggested an alternative five-factor structure for the Exposure scale, but the improvements in psychometric properties were incremental.

The CEFIS demonstrated convergent validity with the PROMIS GMH scale and FAD, underscoring the utility of the CEFIS for measuring the impact of the COVID-19 pandemic on caregiver mental health and family functioning. Associations between the Exposure scale and caregiver mental health support the broader literature that enduring pandemic-related hardships adversely affect caregiver mental health (Gassman-Pines et al., 2020). Caregivers may be unable to effectively cope with the cumulative number of stressors, or they may have lost access to factors that are known to buffer against traumatic stress (e.g., stable employment, easy access to social support networks). Associations between the PROMIS GMH scale and the CEFIS Impact and Distress scales add to the literature and suggest that the perceived impact of COVID-19 on families and perceived distress relate to caregivers’ overall mental health. Of note, the PROMIS GMH scale had stronger associations with the Impact and Distress scales than the Exposure scale, which suggests that the impact of COVID-19 on caregiver mental health may be better explained by the perceived impact or distress related to the pandemic relative to the number of hardships experienced. Although statistically significant, correlations with the FAD were small. This is consistent with other studies demonstrating small associations between discrete, event-related stressors (e.g., natural disaster, traumatic brain injury) and global family functioning (McDermott & Cobham, 2012; Ponsford & Schönberger, 2010). Adaptive family functioning may protect against poor long-term outcomes following exposure to stressful or life-changing events, while poor family functioning may increase the risk of poor outcomes (Hocking et al., 2015; Van Schoors et al., 2016).

Associations between the English-language version CEFIS scales and presence of clinically significant PTSD symptoms provide initial evidence for the criterion validity of the CEFIS and further reinforce the conceptualization of the COVID-19 pandemic as a potentially traumatic event. Caregivers reported experiencing several potentially traumatic pandemic-related events, with over a quarter indicating that someone in the family was exposed to COVID-19 and one in six caregivers indicating that someone in the family had symptoms of COVID-19. Secondary traumatic stress may also have occurred due to frequent exposure to news stories highlighting rates of COVID-19 morbidity and mortality. Finally, public health prevention measures implemented during the pandemic may inadvertently result in additional stressors (e.g., job loss, changes in education, limited access to social supports).

Interestingly, more English-speaking caregivers endorsed clinically significant PTSD symptoms than did Spanish-speaking caregivers. In addition, on the Spanish-language version of the CEFIS, only the Distress scale was associated with having clinically significant PTSD symptoms. There are cultural differences in the types of PTSD symptoms endorsed (Hall-Clark et al., 2017), which may contribute to the observed differences in the prevalence of clinically significant PTSD symptoms. Cultural differences in the types of PTSD symptoms endorsed could also contribute to the differences in the criterion validity analyses. Future validation studies should use culturally sensitive tools to evaluate criterion validity. In addition, well-validated and culturally sensitive tools should be used to screen for psychosocial risk in caregivers impacted by the pandemic and referring to mental health services when appropriate.

While results from the current study were generally consistent with findings from the original validation paper, several differences should be noted. In general, the factor structure and internal consistencies for the Impact and Distress scales were strong. However, some second-order factor loadings for the Exposure scale were low (< .40). Relative to the Impact and Distress scales, the items on the Exposure scale were more heterogeneous, which may explain the lower second-order factor loadings. In addition, the cumulative nature of the Exposure scale could affect the internal consistencies and factor loadings as exposures are more likely as time passes. Factors on the Exposure scale also demonstrated low internal consistencies. The Designation as an essential worker and Disruptions in family caregiving and activities factors had low internal consistencies in both the current study and the initial paper (Kazak et al., 2021). However, the internal consistency for the Disruptions to living conditions factor was lower in this study than in the initial psychometric paper (Kazak et al., 2021). This result may be due to differences in the geography of the current sample (mid-Atlantic US) versus the national sample from Kazak et al. (2021) or the timing of data collection, as families’ experiences with the pandemic are likely to continue to evolve. Although the overall model demonstrated good fit, the lower factors loadings and internal consistencies suggest that future researchers may wish to continue refining the factor structure for the Exposure scale.

Opportunities for future research and optimizing clinical care for families impacted by COVID-19 were identified. We reported on the preliminary psychometrics of the Spanish-language version of the CEFIS. However, additional research is needed to fully evaluate the Spanish version of the CEFIS (e.g., measurement invariance testing, inclusion of a more nationally representative sample). The pandemic differentially impacted sociodemographic groups (Bassett et al., 2020), indicating a need to evaluate measurement invariance and compare CEFIS scores across groups. Future research should investigate whether factors known to mitigate the mental health impact of other traumatic events (e.g., social support) also mitigate the adverse effects of the COVID-19 pandemic. Future research is also needed to examine whether scores on the CEFIS, a caregiver-report instrument, relate to child mental health. Children of caregivers who experience PTSD symptoms or poor psychological functioning may also experience worse psychological functioning (Morris et al., 2012), possibly due to changes in parenting practices (Gewirtz et al., 2008). Future research may examine post-traumatic growth during the pandemic, or how cognitive appraisals of the pandemic affect psychological adjustment. Given the scope and global impact of the COVID-19 pandemic, using a screener like the CEFIS in research batteries and routine clinical practice may be prudent. Establishing clinical cutoffs would increase the clinical utility of the CEFIS. A similar Distress scale (the Distress Thermometer) has been used as a distress screener, with a cutoff of 4 on the 10-point scale, for cancer patients (Jacobsen et al., 2005) and therefore presents the most clinical utility for the CEFIS. In the current study, the average score on the Distress scale was 5.62; therefore, the cutoff of 4 may not be applicable. Future studies should identify clinical cutoffs using validated approaches (e.g., receiver operating curve analyses). For now, researchers and clinicians can use scores 1 SD above the mean to identify significantly elevated Distress scores. Finally, studies should seek to adapt the CEFIS for future pandemics or other traumatic events.

The current study should be interpreted with the following limitations in mind. The response rate for the current study (16.9%) was relatively low. However, the response rate was similar to what has been found in other studies using a similar methodology (Weigl et al., 2019). Additionally, the sample was representative with regard to demographics and responses on the CEFIS and other measures were not skewed, which reduces the concern for a selection bias. Our sample was limited to a mid-Atlantic region of the U.S. Regional differences in severity of COVID-19 outbreaks and implementation of public health measures to manage COVID-19 may affect how participants respond on the CEFIS. However, given that the initial factor structure was developed with a national sample (Kazak et al., 2021), and applied well to a diverse sample in the current study, it is likely that the CEFIS can be used across the United States. Additionally, data for this study and for the Kazak et al. (2021) study represent the first 6 months of the pandemic in 2020, which may affect generalizability. The psychometric properties of the CEFIS should continue to be examined as the pandemic evolves (e.g., availability of COVID-19 vaccines, subsequent variants). Same-method and same-reporter variance may have resulted in inflated associations for the convergent and criterion validity analyses. Studies seeking to examine the convergent validity of this measure may wish to use other methods (e.g., semi-structured interviews). Finally, the cross-sectional nature of the study precludes test-retest reliability and predictive validity.

In conclusion, the CEFIS is a psychometrically sound measure of the effect of the COVID-19 pandemic on family functioning and caregiver mental health. The CEFIS can be used in clinical or academic settings, but researchers should be mindful of the context in which the CEFIS is administered (e.g., current pandemic-related events), attending to potential changes in how caregivers may respond in the future. Use of the three primary CEFIS scales (Exposure, Impact, Distress) and scoring outlined in Kazak et al. (2021) is recommended. The original six- and three-factor structures of the Exposure and Impact scales, respectively, were supported in this validation study. Evidence of convergent and criterion validity suggests that the CEFIS is well suited for studies evaluating the psychosocial impact of COVID-19 on families, can be adapted to assess the impact of future traumatic events on families, and may be useful for identifying families impacted by the pandemic who would benefit from psychological interventions.

Author Contributions

PTE, T-LTP, MAA, and AEK conceptualized the paper. AML managed the dataset. PTE and AML analyzed the data. PTE, AKH, ES, KSC, and GV wrote the initial draft of the manuscript. All authors revised the manuscript.

Click here for additional data file.