Validating the Psychosocial Functioning during COVID-19 Questionnaire among a Sample of Informal Caregivers.

The COVID-19 pandemic has had a dramatic impact on global economies and societies. Although social distancing policies are needed to contain the spread and impact of COVID-19, they also impose a psychological and economic burden on people who are already experiencing increased distress such as caregivers. Yet, few measures have been developed and validated to measure the psychosocial impact of COVID-19. Utilizing item response theory (IRT), the purpose of this study was to develop and psychometrically validate a measure of psychosocial functioning-the Psychosocial Functioning during COVID-19 (PFC-19) Questionnaire-to assess changes in social interaction, mental health, health behavior, and global functioning among a sample of informal caregivers during the COVID-19 pandemic. The analytic sample (n = 733) was recruited from Amazon Mechanic Turk (MTurk) (69% male, 55% white). Results suggest a two-factor measure, assessing global functioning (14 items) and affective response (8 items), with strong evidence for reliability, validity, and dimensionality. Future research should replicate this factor structure in other samples.

Introduction

The COVID-19 pandemic has caused dramatic changes to daily life, including social distancing, quarantining, and the closing of public spaces (Centers for Disease Control and Prevention [CDC], 2020a, 2020b). Unemployment in the U.S. is the highest it has been in decades (U.S. Board of Labor, 2020) and schools closed then shifted to “remote learning” from home, creating a childcare crisis for employed parents (Bayham & Fenichel, 2020). Despite early successes in controlling the virus, the “reopening” of many states in the U.S. to improve the economy led to a resurgence in cases (Julie Bosman, 2020). Although precautions are necessary to contain the spread and reduce the impact of COVID-19, it may impose an additional psychological burden on people who are already experiencing increased fear and anxiety associated with infection and contagion, such as caregivers. Research on COVID-19 is nascent; however, prior global disease outbreaks (e.g., H1N1, Zika, Ebola) have been associated with increased pandemic-related anxiety (e.g., contamination concerns, health anxiety), as well as elevated symptoms of general mental distress (Wheaton et al., 2012; Wu et al., 2009; Yip et al., 2010).

Given the unprecedented nature of this outbreak in modern history, there is limited research on the psychosocial effects of the present pandemic (Rajkumar, 2020; Xiang et al., 2020), and there are no validated measures to assess pandemic-related changes in mental health among caregivers. Psychosocial impacts are the combination of psychological and social environmental factors that influence the ability to function (Mosby’s Medical Dictionary - E-Book, 2013). A synthesis of the limited research on COVID-19 and prior disease outbreaks determined that pandemics/outbreaks and the associated public health sequelae (e.g., quarantine, social isolation) are associated with symptoms of depression and anxiety, which may elevate as social distance guidelines remain in place for longer periods and persist after the pandemic is under control (Brooks et al., 2020; Chew et al., 2020; Liu et al., 2020; Torales et al., 2020). To date, evidence suggests that the COVID-19 pandemic is associated with an increase in negative emotionality and decrease in positive emotionality (Bhattacharjee & Acharya, 2020; Li et al., 2020). Only 2 to 4 months into the pandemic, mental health services saw dramatic increases in behavioral health service utilization (e.g., SAMHSA hotline; Cuningham, 2020) as more than half of Americans reported worsening mental health (KFF Health Tracking Poll, July, 2020). Moreover, recent reports suggest that a diagnosis of COVID-19 is associated with greater odds of a subsequent psychiatric diagnosis, even compared to other health conditions (KFF Health Tracking Poll, July, 2020).

The impacts of the COVID-19 pandemic, and the associated efforts to contain the virus, are associated with distress even among non-clinical samples (Bhattacharjee & Acharya, 2020). The presence of anxiety and distress in response to COVID-19 is detectable not just in essential workers, but also in the general public (Chew et al., 2020; Liu et al., 2020; Torales et al., 2020). In addition to frontline workers, some subgroups of the population might be particularly vulnerable to negative sequelae associated with the pandemic. According to the National Survey of Caregivers, approximately 17.7 million individuals in the U.S. identify as “caregivers” of an older adult, suggesting a sizable population are facing heavy COVID-19 related burden (Schulz & Eden, 2016). Under normal societal circumstances, caregivers are at increased risk for developing mental health disorders compared to non-caregiving peers, with up to one quarter currently meeting criteria for major depression (Marks et al., 2002; Pinquart & Sörensen, 2003), rates that are even higher among informal caregivers for older adults with functional declines (e.g., those with dementia) (Covinsky et al., 2003). Informal caregivers are defined as relatives, friends, or neighbors who assist health compromised and/or individuals over 50 years old in the activities of daily living. Thus, these caregivers are in consistent contact with subpopulations at particularly high risk of COVID-19-related complications (CDC, 2020c). Yet, few measures have been developed and validated to measure the broader psychosocial impact of COVID-19, including mental health and social determinants of health among caregivers.

Understanding and identifying the immediate changes in psychosocial impact among caregivers can be useful to (1) to evaluate the role of pandemic-related psychosocial functioning and distress on its own, or as a moderator of intervention effects on research outcomes in other domains; (2) in clinical practice to address normative symptoms of distress related to the impact of COVID-19 stress as opposed to measuring disorder-specific pathology; or (3) in clinics to identify the immediate changes in psychosocial impact among caregivers due to COVID-19 to identify when and if additional resources are needed. This is of paramount importance given these caregivers are responsible for the care of other individuals. Existing COVID-19 scales are either not psychometrically validated (Pandemic Stress Index) or are well-validated but designed to measure anxiety related specifically to COVID-19 (COVID Stress Scales; Coronavirus Anxiety Scale; and CORPD) (Feng et al., 2020; Harkness et al., 2020; Lee, 2020). For example, the Pandemic Stress Index is a three-item index that measures (1) activities and role shifts during COVID-19, (2) overall impact, and (3) health and well-being (Harkness et al., 2020). It has been translated into Spanish, Turkish, Mandarin, and Italian and used throughout the U.S. and internationally (Cainelli et al., 2020; Clark et al., 2020). The Pandemic Stress Index items were assessed for face validity. The items are self-administered and ask respondents to select all the items that apply and quantify stressors. This measure was initially created using a sample of Latino and sexual minority men. Although the Pandemic Stress Index measures the psychosocial impact of COVID-19, it is a checklist of specific experiences and is not measured on a scale of how much psychosocial impact they have experienced, which may be useful when determining if additional resources are needed. No information is available about its association with other measures of psychosocial functioning or if it will be psychometrically validated. Further, one of the most widely used scales—COVID Stress Scales—is valid and reliable but designed to measure clinically significant anxiety specific to COVID-19, including: (1) danger and contamination fears, (2) fears about economic consequences, (3) xenophobia, (4) compulsive checking and reassurance seeking, and (5) traumatic stress symptoms (Taylor et al., 2020). The Coronavirus Anxiety Scale (CAS) is a five-item screener to identify probable cases of dysfunctional anxiety associated with the COVID-19 crisis (Lee, 2020). Similarly, the COVID-19 related psychological distress in healthy public (CORPD) is a 14-item scale assessing two dimensions (anxiety/fear and suspicion) among uninfected healthy populations (Feng et al., 2020). Although COVID-specific anxiety and distress is important, it fails to capture other important social determinants of health and psychosocial impact. In short, existing pandemic measures assess anxiety, obsessive compulsive symptoms, traumatic stress, and changes to daily living (Harkness et al., 2020; Lee, 2020; Taylor et al., 2020), but may not be validated or account for current experiences of social isolation and changes to daily life that may lead to increased distress and decreased psychosocial functioning among otherwise healthy adults.

There is an urgent need to address how mental health consequences of COVID-19 can be mitigated, particularly among those caring for other individuals (Holmes et al., 2020). To do so, clinicians and researchers need brief, validated measures that assess both pandemic-specific psychosocial distress and multidomain functioning—changes in social interaction, mental health, health behavior, and global functioning. In contrast to other measures, the current study sought to develop the Psychosocial Functioning during COVID-19 (PFC-19) Questionnaire to be both psychometrically valid, using item response theory (IRT), and useful for highlighting service needs among caregivers that can be used for both research purposes and service delivery. We chose to test this measure in a sample of caregivers, as we hypothesized they would be vulnerable to the same processes as the general population but perhaps be more vulnerable to some of them (e.g., need to reduce contact due to concerns of acquiring and infecting care recipient; feelings of isolation). We expect this will help fill in the gap among current psychosocial assessments that do not serve to evaluate stressors specific to COVID-19 in healthy populations.

Methods

Participants and Procedure

Study participants were recruited from a larger study using Amazon’s Mechanical Turk (MTurk; Cohen, 2020; Greaney, 2020). MTurk was utilized to quickly recruit a convenient sample and offered access to participants even with COVID-19 social distancing policies (CDC, 2020a). Eligible participants where informal caregivers for individuals over age 50 at the start of the pandemic. See Table 1 for more details on the participant demographics. Participants completed this study online through Qualtrics and received $1.50 as compensation. This study was approved by the University of Rhode Island Institutional Review Board (project #1606088-2). Prior to analysis, the data were checked to ensure there was only one record per participant by checking IP addresses. If any duplicate IP addresses were detected, only the first response was retained for analysis.

Table 1.

Sample Characteristics.

Characteristic	M (SD) or N (%)
Age	34 (10)
Gender
Female	227 (31%)
Male	506 (69%)
Racial category
White	401 (55%)
Asian	117 (16%)
Black or African American	103 (14%)
American Indian or Alaska Native	43 (6%)
Asian Indian	43 (6%)
Mixed	21 (3%)
Native Hawaiian or other Pacific Islander	3 (<1%)
Chose not to respond	2 (<1%)
Ethnicity
Hispanic	329 (45%)
Non-Hispanic	404 (55%)
Education level
Less than 9th grade	5 (<1%)
9th–12th grade	12 (2%)
High school diploma, G.E.D., or equivalent	19 (3%)
Some college, no college degree	41 (6%)
Associate’s degree	24 (3%)
Bachelor’s degree	508 (69%)
Graduate or professional degree	119 (16%)
Chose not to respond	4 (<1%)
Employment status
Employed full-time	608 (83%)
Employed part-time	84 (11%)
Unemployed	22 (3%)
Other (i.e., student, retired)	17 (2%)
Chose not to respond	2 (<1%)
Relationship to care recipient
Adult child or adult child-in-law	492 (67%)
Spouse	51 (7%)
Other relationship	190 (26%)
Lives with care recipient
Yes	295 (40%)
No	421 (57%)
Chose not to respond	17 (2%)
Length of time providing care
Less than 30 days	89 (12%)
1–6 months	261 (36%)
6 months–2 years	223 (30%)
2–5 years	87 (12%)
5 or more years	65 (9%)
Chose not to respond	8 (1%)
Hours of care provided per week
Up to 9 hours per week	205 (28%)
10–19 hours per week	347 (47%)
20–39 hours per week	128 (17%)
40 or more hours per week	41 (6%)
Chose not to respond	12 (2%)

Measures

Depression, anxiety, and stress scale (DASS-21)

The DASS-21 is a 21-item measure of mental health, with three 7-item subscales assessing feelings of depression, anxiety, and stress in the past week (Antony et al., 1998; Norton, 2007). The four response options range from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time). In this sample, the depression (ω = 0.88, Bootstrap 95% Confidence Interval (CI) [0.86, 0.89]), anxiety (ω = 0.88, 95% CI [0.86, 0.89]), and stress (ω = 0.87, 95% CI [0.85, 0.88]) subscales all showed acceptable levels of reliability according to the recommended coefficient omega (ω) (Dunn et al., 2014; Trizano-Hermosilla & Alvarado, 2016). The three subscales of the DASS-21 were chosen to test for convergent validity because of the known overlaps between psychosocial functioning and anxiety (Moitra et al., 2014), depression (Cambridge et al., 2018), and stress (Alastalo et al., 2013). Thus, it was hypothesized and expected that there would be relationships between the PFC-19 and the three DASS-21 subscales, which helped inform our scale validity.

Psychosocial functioning during COVID-19 questionnaire

The PFC-19 is a 24-item measure assessing global functioning and affective response related to the COVID-19 pandemic. Each item asked respondents to rate the extent to which COVID-19 has affected areas of their lives compared to what is normal for them. Participants responded on a seven-point Likert-type scale, with responses ranging from −3 = A Lot Less to 3 = A Lot More. The full survey can be found in Appendix A.

Appendix A.

Psychosocial Functioning during COVID-19 (PFC-19) Measure.

Instructions: Below is a list of things that may have changed in your life during the pandemic. Please read each item, and then select the response that fits best.

How has COVID-19 affected the following areas of your life relative to what is normal for you?	A lot less	Somewhat less	A little less	About the same (no change)	A little more	Somewhat more	A lot more
Scoring	−3	−2	−1	0	1	2	3
1. Amount of time you spend at home?
2. Amount of time you spend physically with other people (e.g., friends, family, roommates, neighbors)
3. Amount of time you spend physically with friends?
4. Amount of time you spend physically with the people you live with?
5. Amount of time you spend physically with family?
6. Ability to get started on tasks or “get going”?
7. Ability to concentrate on tasks?
8. Amount of time you spend physically with coworkers and/or classmates?
9. Amount of time you spend physically with romantic partner(s)?
10. Amount of time you spend talking to others on the phone or computer?
11. Amount of time you spend worrying or feeling afraid?
12. Feeling sad?
13. Feeling angry or irritable?
14. Feeling lonely?
15. Amount of time you spend exercising or going for walks?
16. Are you taking prescribed medications?
17. Drinking alcohol?
18. Using marijuana or other nonprescribed recreational substances?
19. Engaging in sexual activity?*
20. Difficulty falling asleep at night?
21. Difficulty staying asleep at night?
22. Access to housing?
23. Access to water and food?
24. Seeing the “bright side” of things and focusing on the positives (i.e., spending more time with family, having time to watch shows)?

Note. Scoring instructions: Items are scored −3 = a lot less; −2 = somewhat less; −1 = a little less; 0 = about the same; 1 = a little more; 2 = somewhat more; 3 = a lot more. Higher scores are equivalent to positive psychosocial functioning and lower scores are equivalent to worse psychosocial functioning. (Note reverse score shaded items: Item 11, Item 12, Item 13, Item 14, Item 17, Item 18, Item 20, and Item 21.)

Item 19 was dropped from the measure during psychometric validation.

PFC-19 Scale Development and Data Analysis

Item development

Questions were written by practicing clinicians and researchers who have continued to see clients and conduct research throughout the pandemic. Item development was informed directly by issues that have been raised in diverse clinical and research settings as well as COVID-19-related distress reported in the literature (Rajkumar, 2020; Xiang et al., 2020). Items were written to assess current impact, but no time period was specified within the question. Items were designed to assess mental health and social determinants of health (e.g., availability of resources to meet needs, social support).

Scale development

Psychometric validation consisted of descriptive statistics, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), item response theory (IRT) using a graded response model, and exploring evidence for reliability and validity (Boateng et al., 2018; De Vet et al., 2011). Validation was done from both the classical test theory (EFA/CFA) and IRT perspectives for a robust analysis of the measure. EFA is ideal for determining the dimensionality of a measure, which can then be confirmed through the use of CFA and IRT. An advantage of IRT over CFA are the invariance properties of the intercept and threshold parameters in IRT, which do not depend on the latent trait of any individual sample (Lord, 1980). Thus, the IRT parameter estimates should be consistent in samples different than the one analyzed in this study, which improves the generalizability of these findings.

Descriptive statistics were checked for any issues of non-normality of individual items. EFA was guided by Velicer’s Minimum Average Partial (MAP) test to determine the appropriate number of factors to extract, as relying on EFA alone can lead to either over- or under-extraction (Velicer, 1976). EFA factor extraction was done using the maximum likelihood method with Promax rotation. An oblique rotation was selected due to the expectation of correlated factors. Any items that did not load onto a factor above |0.40|, or loaded above |0.40| on two or more factors, were dropped and the EFA was re-conducted (Harlow, 2014). The CFA model was built based on the EFA results, and model fit was determined by consulting the χ² test, comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean residual (SRMR) as fit indices. Best fit would be indicated by a non-significant χ² test, CFI above 0.95 (but 0.90 is acceptable) and RMSEA below 0.05 (but below 0.08 and 0.10 suggest good and acceptable fit, respectively) and SRMR below 0.08 (Hu & Bentler, 1999). The IRT model was assessed using the same fit indices, although a M² test was used instead of the χ² test as the M² statistic performs better in sparsely populated contingency tables often seen in IRT models (Cai & Hansen, 2013). Best fit would have been non-significant χ² and M² tests for the CFA and IRT models, respectively, but both tests are typically over-powered and may have a significant result when there is still reasonable model fit. Thus, a significant χ² and M² does not necessarily suggest poor fit (Cai & Hansen, 2013; Harlow, 2014).

Scale evaluation

Reliability was assessed using the recommended coefficient ω, a measure of internal consistency, which is less biased than coefficient α, and calculating bootstrapped confidence intervals. Values of 0.70 were considered the lower limit for acceptable reliability (Dunn et al., 2014). Evidence for validity was assessed by calculating Pearson’s r correlations with the three DASS-21 subscales of depression, anxiety, and stress. When conducting the analyses, the sample was randomly split, with EFA analyses being conducted on 25% of the sample (n = 183) and CFA/IRT analyses being conducted on the remaining 75% (n = 550), so the factor structure from the EFA results could be replicated in an independent sample. Reliability and validity analyses were conducted on the full sample.

Results

Analyses were conducted in R v. 4.0.2 using the psych, lavaan, and mirt packages (Chalmers, 2012; R Core Team, 2019; Revelle, 2017; Rosseel, 2012). Less than 1% of the data were missing in the initial sample (N = 835). Since the amount of missing data were trivial, complete case analysis was used since omitting the incomplete cases was unlikely to introduce any bias (Graham, 2009). Whereas the total sample size was 835, the analyses were conducted on a subset of n = 733 due to the handling of missing data described below.

The sample of n = 733 was predominantly male (n = 506, 69%; remainder identified as female), with an average age of 33.77 (SD = 9.67). Most participants (n = 401; 55%) identified as White, with smaller proportions identifying as Asian (n = 117, 16%), Black or African American (n = 103, 14%), American Indian or Alaska Native (n = 43, 6%), Asian Indian (n = 43, 6%), Mixed (n = 21, 3%), and Native Hawaiian or other Pacific Islander (n = 3, <1%), with 2 (<1%) choosing not to respond. Nearly half the sample (n = 329, 45%) identified as Hispanic. The majority of participants (508, 69%) reported having a bachelor’s degree and being employed full-time (608, 83%). A description of the sample is provided in Table 1.

The first step of the analyses examined item-level descriptive statistics. Results suggested no issues of non-normality, so analysis proceeded to the next step without any transformations (Table 2).

Table 2.

Descriptive Statistics for the COVID-19 Pandemic Psychosocial Functioning Measure.

Item	M (SD)	Skewness/Kurtosis
1. Amount of time you spend at home?	5.26 (1.39)	−0.93/0.58
2. Amount of time you spend physically with other people (e.g., friends, family, roommates, neighbors).	4.93 (1.68)	−0.80/−0.25
3. Amount of time you spend physically with friends?	4.82 (1.74)	−0.74/−0.34
4. Amount of time you spend physically with the people you live with?	5.15 (1.44)	−0.84/0.33
5. Amount of time you spend physically with family?	5.13 (1.42)	−0.80/0.20
6. Ability to get started on tasks or “get going”?	5.06 (1.45)	−0.78/0.03
7. Ability to concentrate on tasks?	5.14 (1.43)	−0.72/0.07
8. Amount of time you spend physically with coworkers and/or classmates?	4.92 (1.66)	−0.77/−0.26
9. Amount of time you spend physically with romantic partner(s)?	5.10 (1.44)	−0.82/0.18
10. Amount of time you spend talking to others on the phone or computer?	5.24 (1.33)	−0.63/−0.16
11. Amount of time you spend worrying or feeling afraid?*	3.03 (1.53)	0.75/−0.02
12. Amount of time you spend feeling sad?*	2.96 (1.54)	0.72/−0.14
13. Amount of time you spend feeling angry or irritable?*	3.08 (1.48)	0.74/0.07
14. Amount of time you spend feeling lonely?*	2.99 (1.56)	0.82/0.08
15. Amount of time you spend exercising or going for walks?	5.01 (1.45)	−0.69/−0.01
16. Are you taking prescribed medications?	4.92 (1.62)	−0.76/−0.05
17. Are you drinking alcohol?*	3.29 (1.74)	0.69/−0.43
18. Are you using marijuana or other nonprescribed recreational substances?*	3.31 (1.77)	0.70/−0.42
19. Are you engaging in sexual activity?[†]	4.83 (1.63)	−0.69/−0.26
20. Do you have difficulty falling asleep at night?*	3.17 (1.61)	0.76/−0.09
21. Do you have difficulty staying asleep at night?*	3.18 (1.60)	0.76/−0.12
22. Do you have access to housing?	5.11 (1.45)	−0.70/0.10
23. Do you have access to water and food?	5.09 (1.40)	−0.68/0.08
24. Seeing the “bright side” of things and focusing on the positives (i.e., spending more time with family, having time to watch shows)?	5.16 (1.33)	−0.70/0.26

Note. * indicates item is reverse scored.

indicates item is dropped in later analyses.

Scale Development Results

Velicer’s MAP test suggested extracting two factors in the initial EFA. Maximum-likelihood EFA with Promax rotation suggested item 19 (“Engaging in sexual activity?”) did not load above |0.40| on either factor and was dropped from subsequent analyses. The EFA was re-conducted, and all remaining items showed loadings above |0.40| on one of the two factors in this second iteration. Together, the two factors explained 46% of the variance: 25% coming from factor 1 and 21% from factor 2. There was also a strong, negative correlation between the latent factors, r = −0.75, p < .001. Items 1 to 10, 15 to 16, and 22 to 24 loaded onto factor 1 and items 11 to 14, 17 to 18, and 20 to 21 loaded onto factor 2 (see Table 3). Based on the items contained within each factor, factor 1 was labeled “Global Functioning” and factor 2 was labeled “Affective Response.”

Table 3.

Exploratory and Confirmatory Factor Analysis of the COVID-19 Pandemic Psychosocial Functioning Measure.

Item	Loadings
	EFA—global functioning (Factor 1)	EFA—affective response (Factor 2)	CFA—global functioning (Factor 1)	CFA—affective response (Factor 2)
1. Amount of time you spend at home?	0.40	0.05	0.38	—
2. Amount of time you spend physically with other people (e.g., friends, family, roommates, neighbors).	0.42	−0.26	0.67	—
3. Amount of time you spend physically with friends?	0.43	−0.33	0.68	—
4. Amount of time you spend physically with the people you live with?	0.51	−0.09	0.53	—
5. Amount of time you spend physically with family?	0.57	−0.04	0.57	—
6. Ability to get started on tasks or “get going”?	0.83	0.13	0.62	—
7. Ability to concentrate on tasks?	0.93	0.25	0.65	—
8. Amount of time you spend physically with coworkers and/or classmates?	0.62	−0.18	0.72	—
9. Amount of time you spend physically with romantic partner(s)?	0.74	0.14	0.57	—
10. Amount of time you spend talking to others on the phone or computer?	0.49	−0.11	0.61	—
15. Amount of time you spend exercising or going for walks?	0.74	0.10	0.69	—
16. Are you taking prescribed medications?	0.46	−0.34	0.70	—
22. Do you have access to housing?	0.42	−0.28	0.71	—
23. Do you have access to water and food?	0.66	−0.01	0.70	—
24. Seeing the “bright side” of things and focusing on the positives (i.e., spending more time with family, having time to watch shows)?	0.58	−0.03	0.64	—
11. Amount of time you spend worrying or feeling afraid?*	−0.08	0.56	—	0.69
12. Amount of time you spend feeling sad?*	−0.06	0.71	—	0.77
13. Amount of time you spend feeling angry or irritable?*	0.15	0.88	—	0.77
14. Amount of time you spend feeling lonely?*	−0.12	0.58	—	0.78
17. Are you drinking alcohol?*	−0.13	0.59	—	0.72
18. Are you using marijuana or other nonprescribed recreational substances?*	−0.22	0.53	—	0.74
20. Do you have difficulty falling asleep at night?*	0.18	0.91	—	0.76
21. Do you have difficulty staying asleep at night?*	0.19	0.88	—	0.76

Note. Standardized loadings shown for EFA and CFA. Extraction method-maximum likelihood; rotation method oblique (promax).

Reversed scoring. All p < .001 for standardized factor loadings. Factor 1 items are above the dashed line, Factor 2 items are below the dashed line. Loadings ≥|0.40| in bold for EFA analysis.

The CFA and IRT models were both constructed as correlated, two-factor models based on the EFA results. The CFA model demonstrated reasonable fit based on the fit statistics: RMSEA = 0.08, 90% CI [0.08, 0.09], and SRMR = 0.06, but did not achieve adequate fit based on the χ² test and CFI results, χ² (229) = 1072.70, p < .001, CFI = 0.87. A strong negative correlation among the latent factors was also found in the CFA model, r = −0.82, p < .001. The IRT model showed good fit based on the CFI and RMSEA values, CFI = 0.97, RMSEA = 0.07, 90% CI [0.07, 0.08], and SRMR = 0.07, but not the M² test, M² (132) = 531.27, p < .001. The IRT model also showed a strong, negative correlation between the latent factors, r = −0.81, p < .001. Slope and threshold parameter values are displayed for the IRT model in Table 4. Intercept parameters, also known as a slope and discrimination parameters, indicate how well an item discriminates between levels of the latent trait, θ, being measured in IRT models. Higher intercepts suggest the item does a better job at discriminating θ. Threshold parameters, also known as location and difficult parameters, are the point along θ at which endorsing two adjacent categories (e.g., “a lot less” vs. “somewhat less”) has a probability of 0.5. Since θ follows a standard normal distribution, threshold parameters can be interpreted as z-scores of θ, so a threshold 1 of −1.50 suggests participants who respond with “a lot less” are below −1.50 on the latent trait compared to participants who respond with “somewhat less.” It is ideal for k − 1 threshold parameters, where k is the number of response categories, to cover a wide range of a standard normal distribution so different levels of θ are captured based on participant responses. See Boateng et al. (2018) or De Vet et al. (2011) for further explanation of these parameters.

Table 4.

Quantitative Self-Confidence Measure Results for the COVID-19 Pandemic Psychosocial Functioning Measure.

Item	Intercept	Item response theory parameters
		Threshold 1	Threshold 2	Threshold 3	Threshold 4	Threshold 5	Threshold 6
Factor 1—global functioning
1	0.80	−1.73	0.03	1.38	2.41	3.06	4.52
2	1.65	−2.45	−0.18	1.19	2.07	3.00	4.01
3	1.50	−2.37	−0.37	0.91	1.96	2.77	3.54
4	1.23	−2.13	−0.25	1.25	2.40	3.18	4.82
5	1.35	−2.26	−0.11	1.24	2.53	3.48	5.42
6	1.57	−2.64	−0.14	1.36	2.66	3.59	5.22
7	1.58	−2.22	−0.26	1.36	2.81	3.77	6.15
8	1.84	−2.66	−0.23	1.31	2.33	3.27	4.56
9	1.30	−2.37	−0.22	1.13	2.34	3.44	4.70
10	1.63	−2.29	−0.15	1.38	2.70	4.26	6.48
15	1.85	−2.84	−0.52	1.28	2.63	3.97	5.50
16	1.77	−2.53	−0.37	1.06	2.46	3.37	4.22
22	1.95	−2.48	−0.34	1.31	2.95	4.19	5.49
23	1.81	−2.70	−0.30	1.23	2.99	4.14	5.73
24	1.64	−2.54	−0.39	1.53	3.10	4.24	6.19
Factor 2—affective response
11	1.75	−4.85	−3.37	−2.36	−0.98	0.53	2.80
12	2.24	−6.12	−4.04	−2.88	−1.26	0.23	2.78
13	2.17	−5.80	−3.95	−2.71	−1.27	0.82	3.28
14	2.26	−5.47	−3.86	−2.77	−1.26	0.42	2.89
17	1.85	−3.77	−2.87	−2.01	−0.87	0.60	2.88
18	1.99	−3.68	−2.85	−2.11	−0.78	0.70	2.91
20	2.10	−1.86	−0.26	0.55	1.46	2.16	3.00
21	2.19	−4.65	−3.19	−2.46	−1.07	0.78	3.08

Scale Evaluation Results

Reliability and validity estimates were conducted separately for each of the two identified factors. The Global Functioning (ω = 0.91, 95% CI [0.90, 0.92]) and Affective Response factors demonstrated strong reliability (ω = 0.91, 95% CI [0.89, 0.92]). Since 91 (12%) of the 733 respondents did not complete the DASS-21, their data were imputed using multiple imputation (m = 5 imputations) to keep a consistent sample size (Buuren & Groothuis-Oudshoorn, 2010). Positive associations were found between the Global Functioning factor and the DASS-21 subscales: depression (r = 0.47, 95% CI [0.41, 0.52], p < .001), anxiety (r = 0.52, 95% CI [0.47, 0.57], p < .001), and stress (r = 0.48, 95% CI [0.43, 0.54], p < .001). Negative associations were found between the Affective Response factor and the DASS-21 subscales: depression (r = −0.65, 95%CI [−0.69, −0.60], p < .001), anxiety (r = −0.64, 95% CI [−0.68, −0.59], p < .001), and stress (r = −0.64, 95% CI [−0.68, −0.59], p < .001). All items on the Affective Response factor were reverse scored. Therefore, negative associations suggest higher levels of depression, anxiety, and stress (per the DASS-21) and were associated with higher scores on the Affective Response factor.

Discussion

The Psychosocial Functioning during COVID-19 (PFC-19) Questionnaire was the first measure developed to be a valid and reliable way to measure changes in social and psychological functioning and impairment due to the pandemic, specifically among informal caregivers. Our results yielded a measure with two domains: global functioning, which measures abilities to perform activities of daily living, and affective response, which measures emotional response and coping strategies. Both factors had evidence for relatively good psychometric properties after dropping item 19. Due to the rapid acceleration of health and social consequences attributable to the pandemic, there is a need to measure how these changes are impacting those caring for older individuals, importantly their mental health functioning (Bhattacharjee & Acharya, 2020; Li et al., 2020).

Undoubtably, the circumstances of the current pandemic (e.g., social isolation, economic hardships, fear of illness, etc.), are related to distress and impair of daily functioning. Further, the mental health impact of COVID-19 may be longer lasting than the epidemic wave of infection as some of the societal and interpersonal stressors may persist (Brooks et al., 2020; Chew et al., 2020; Liu et al., 2020; Torales et al., 2020). The PFC-19 can be a useful tool to evaluate pandemic-related psychosocial functioning and distress on research outcomes or in clinical practice to identify when and if additional resources such as psychological services or additional care assistance are needed. Understanding COVID-related psychosocial impacts can inform intervention targets for providing resources and psychological interventions at the individual and community levels.

Limitations

Limitations of this study include that the convenience sample from MTurk was a younger predominantly White sample, which limits generalizability to persons of other racial, ethnic, and cultural backgrounds. Methodological limitations include not achieving ideal fit in the CFA model based on the CFI fit indices. The affective subscale assessed only negative affect (lonely, sad, etc.), and items were reverse scored, so it is possible items were more highly correlated than they would have been if worded differently. Another methodological limitation is that we did not include attention checks within the survey; however, we did screen for IP addresses to try to limit the likelihood that someone was taking it multiple times. The sample respondents were all caregivers of an individuals over 50 years old; however precise age of person being taken care of was not collected, thus differences in caregiver distress and age of person being taken care of could not be assessed. Moreover, we cannot determine whether responses captured by this measure are due directly to the virus or are attributable to the stress of being a caregiver during the pandemic or some combination; however, the measure asks respondents to compare behaviors and feelings currently to pre-COVID-19 conditions, which may mitigate confounders related to caregiving generally. This can be considered a form of a retrospective pretest design (i.e., comparing current functioning to pre-pandemic function), which is often used to measure change but can be biased (Hill & Betz, 2005; Rockwell & Kohn, 1989).

Future Directions

Given this was a convenience sample from MTurk and was predominantly White, future studies should seek to further validate this measure using a racially diverse sample and include additional recruitment strategies. Further research could examine differences in caregiver distress based on age and illness of persons under their care to determine whether these factors would function differently. Another interesting sample would be parents and primary caregivers of children under 18 years of age, as they are also a group of high concern who may add undue stress to caregivers during this time. Additionally, given this study utilized a retrospective pretest design, future research should attempt to validate this measure using other study designs to see if the same factor structure emerges.

Conclusion

As the pandemic continues to affect the health of individuals worldwide, there are a corresponding set of psychological and social consequences (Bhattacharjee & Acharya, 2020; Li et al., 2020; U.S. Board of Labor, 2020). What was once viewed as an acute and temporary stressor is now a chronic stressor with no known terminal stage. The prolongation of a stressor of this magnitude is associated with increased distress and, as such, there is an increased need to measure, monitor, and evaluate this stress. The PFC-19 measure contributes to the growing efforts to measure psychosocial functioning in caregiver populations and has important implications for future work in assessing the psychological impacts of COVID-19.