Coronavirus Pandemic Anxiety Scale (CPAS-11): development and initial validation.

The mental health impact of the COVID-19 pandemic will increase as the outbreak continues and persist even after the pandemic passes. We developed an 11-item Coronavirus Pandemic Anxiety Scale (CPAS-11) to measure symptoms of anxiety related to the COVID-19 pandemic to help identify individuals who might need mental health services. In developing the scale items, we considered previous research and theory on anxiety symptoms and symptoms reported by clinically referred cases in the Philippines. The scale was validated in a Filipino sample (N = 925). Exploratory factor analysis indicated two factors corresponding to somatic and non-somatic symptoms; confirmatory factor analysis showed good fit for the two-factor model. CPAS-11 showed good internal consistency, convergent and divergent validity, and screening accuracy. A cutoff score of 15 showed adequate sensitivity and specificity to distinguish GAD-7 screened participants. The results support the viability of CPAS-11 as a screening tool to identify individuals experiencing COVID-19-related anxiety. © Springer Science+Business Media, LLC, part of Springer Nature 2020.

Introduction

In May 2020, the World Health Organization (2020) called attention to the mental health impact of the global novel coronavirus (COVID-19) outbreak that continues to spread in many parts of the world. Research on past natural and human-instigated disasters has shown that emotional distress and other psychological symptoms tend to pervade in affected populations, and this pattern shall be the case in populations affected by the COVID-19 pandemic (Pfefferbaum & North, 2020; Torales, O’Higgins, Castaldelli-Maia, & Ventriglio, 2020; Xiang et al., 2020). These mental health concerns may appear among individuals who have been infected by the COVID-19 virus, but also among persons who experience loss of close family members, continuing risk of infection, long periods of social isolation (Xiao, Zhang, Kong, Li, & Yang, 2020), and among medical and health workers (Lu, Wang, Lin, & Li, 2020). Reports from some countries show increases in mental health issues amidst the coronavirus pandemic specifically increases in reported levels of stress, anxiety, depressive symptoms, denial, insomnia, fear, and anger (Cao et al., 2020; Roy et al., 2020; Wang et al., 2020; Zandifar & Badrfam, 2020).

In disaster mental health research, the primary focus has been on post-traumatic stress disorder (PTSD), but as yet, there is no strong indication that the psychological symptoms arising from the COVID-19 pandemic would meet diagnostic criteria for PTSD (cf. Liu et al., 2020; Pfefferbaum & North, 2020). However, other symptoms, like anxiety, will foreseeably arise, similar to what happened in previous pandemic virus outbreaks like the SARS (Cheng, Wong, Tsang, & Wong, 2004) and H1N1 (Wheaton, Abramowitz, Berman, Fabricant, & Olatunji, 2012) outbreaks. One study estimated the frequencies of individual mental health symptoms and predicted anxiety to be the most common (Rajkumar, 2020). While appropriate levels of anxiety about viral contagion are associated with adaptive behaviors like handwashing, physical distancing, and getting vaccinated (Taylor, 2019), high levels of anxiety also tend to be associated with socially disruptive behaviors (e.g., panic-buying, exaggerated interpretation of minor symptoms) (Asmundson & Taylor, 2020). High levels of anxiety also appear together with other psychopathologies in populations affected by COVID-19 (Liu et al., 2020), as was the case in previous viral pandemics (Maunder et al., 2003). Although the symptoms manifested by individuals during the COVID-19 pandemic may be similar to those in generalized anxiety, research indicated that many people display specific forms of anxiety-related distress responses during viral outbreaks (Asmundson & Taylor, 2020). For example, studies found that despite taking safety precautions and practicing social distancing, people tended to display excessive fear of becoming infected and continuously worry about family members contracting the disease or dying (Wang et al., 2020).

Mindful of the importance of anxiety in shaping behavior during the viral pandemic and of the possibility that anxiety could evolve into long-lasting mental health problems, it is crucial that mental health professionals, public health decision-makers, and other stakeholders in the protracted COVID-19 pandemic have access to data on the pervasiveness of anxiety and other adverse psychological responses to the pandemic in communities. In response to this concern, some COVID-19 related measures have been developed. These include the Fear of COVID scale (FCV-19S; Ahorsu et al., 2020), COVID Stress Scale (CSS; Taylor et al., 2020), and COVID as Traumatic Stress Scale (Kira et al., 2020). The only anxiety measure developed is the Coronavirus Anxiety Scale (CAS; S. A. Lee, 2020), a five-item measure of COVID-19-related anxiety. CAS is a brief measure that is easy to use, but its unidimensional structure might not capture the diverse qualities of anxiety symptoms related to COVID-19 that might inform the formal diagnosis and treatment of individuals.

In this study, we developed and validated a mental health screening tool to assess the severity of COVID-19-related anxiety. The scale development process considered current knowledge about anxiety symptoms and also symptoms identified in clinically referred cases in the Philippines. Although our process draws mainly from experiences in the Philippines, we believe that the items in the scale should be applicable to different countries that also experience similar widespread viral contagion, mandated social isolation, and physical distancing.

Method

Item Selection and Scale Development

Candidate Items Selection

A systematic search for existing anxiety scales was conducted using several search engines: Microsoft Academic, Psychiatry.org, and GoogleScholar using the keywords anxiety scale, anxiety instrument, and anxiety measure. The search turned up 25 scales with a total of 529 items, which were examined based on theories of anxiety and used as the base of potential items to be adapted for the scale.

Reduction of Items

Numerous duplicate items were dropped, shortening the list to 175 items. A committee of three researchers evaluated these items intending to develop a shorter list of symptoms that more closely reflect the symptoms associated with reported anxieties during the current COVID-19 outbreak in the Philippines (i.e., symptoms associated with risk of infection, being socially and physically isolated, and other insecurities during the outbreak). The result was a list of 21 items and a draft set of instructions for these items.

Expert Review

The 21 items were presented to three registered clinical psychologists for independent review. The experts had seen clients reporting anxiety-related symptoms in the weeks prior to the review and were asked to independently comment on the appropriateness and sufficiency of the symptoms, the wording of the items and instructions, and to suggest modifications. Additional discussions were conducted with two of the experts to clarify their recommendations. This process resulted in further reduction of the items to 17.

Pilot Testing

The draft 17-item scale was pilot tested with six men and six women (aged 21 to 65 years, with varying levels of education) through online interviews. Most participants were in home quarantine during the pilot study. Based on the feedback, minor modifications were made in the wording and sequence of the items and the instructions. Some participants also emphasized that some items referring to reactions towards events outside the home did not apply to them because they were quarantined in their homes.

Validation Study Measures

Coronavirus Pandemic Anxiety Scale (CPAS)

The scale included 17 items that referred to various anxiety symptoms. The instructions stated: “Think about your experiences in the past two weeks. You may have feelings, thoughts, behaviors, and experiences that are in some way related to COVID-19. Please read each item and indicate how often you experienced each item in the past two weeks.” Participants were asked to respond from 0 (not at all) to 3 (nearly every day). Four items that involved experiences outside the home were placed at the end of the survey and were not presented to participants who indicated they were home quarantined.

Generalized Anxiety Disorder (GAD-7)

The seven-item scale is used to assesses symptoms of generalized anxiety disorder (Spitzer, Kroenke, Williams, & Löwe, 2006), as indicated in the DSM-5. Participants were also asked to think about how often they experienced the symptoms in the past two weeks and responded using the same scale as the CPAS.

Patient Health Questionnaire (PHQ-9)

The nine-item scale is used to measures symptoms of depression (Kroenke & Spitzer, 2002), as indicated in the DSM-5. Participants were also asked to think about how often they experienced the symptoms in the past two weeks and responded using the same scale as the CPAS.

Mental Health Continuum-Short Form (MHC-SF)

The 14-item scale measures general well-being with subscales measuring emotional, social, and psychological well-being (Keyes, 2002). Participants were asked to think about how often they felt or experienced each item in the past month and using a scale from 1 (never) to 6 (every day).

Positive and Negative Affect Schedule (PANAS)

The ten-item scale was used to measure positive and negative emotions (Watson, Clark, & Tellegen, 1988). The participants were asked to read ten emotion words and report to what extent they felt each on over the past week using the scale from 1 (very slightly or not at all) to 5 (extremely).

Fear of COVID-19 Scale (FCV-19S)

The seven-item measure was a newly developed scale (Ahorsu et al., 2020) that measures fear related to COVID-19. The participants were asked to indicate their agreement with each item, using a scale from 1 (strongly disagree) to 5 (strongly agree).

Validation Study Participants and Procedures

An online survey was launched on the day the country relaxed its community quarantine policy (from Enhanced Community Quarantine to Modified Enhanced Community Quarantine), which allowed more movement outside the home. Participants were recruited through social media sites and were asked whether they were still home quarantined. The survey was live online for exactly one week. All the recorded responses to the online survey were found to be valid and all participants gave their informed consent, so no data from any participant was excluded from the analysis. Participants were 925 Filipino citizens (71.14% female). Participants’ ages ranged from 18 to 79 years (M = 35.26; SD = 12.55). Most (58.27%) were single, and 60.86% were employed/working during the time of the survey. The sample was randomly split into two smaller samples for the exploratory factor analysis and the confirmatory factor analysis described below.

Results

Sample Characterization

Based on suggested cutoff scores (Spitzer et al., 2006) for anxiety (GAD-7 ≥ 15), 13.84% of our participants have severe anxiety. Based on suggested cut-off scores (Kroenke & Spitzer, 2002) for depression, 21.73% have moderately severe depression (PHQ-9 = 15 to 19), and 9.08% have severe depression (PHQ-9 ≥ 20). Most participants were still home quarantined (72.43%), so we had a lot of missing data on items 14 to 17 (i.e., items involving experiences outdoors). As a consequence, the exploratory factor analyses we report in the next subsection involved only 13 of the 17 items.

Exploratory Factor Analysis

To explore the underlying structure of the items, exploratory factor analysis (EFA) with promax rotation was conducted with the randomly selected half of the sample (n = 462) using the statistics software R (R Core Team, 2016). The Kaiser-Meyer-Olkin (KMO) = .88 and Barlett’s test of sphericity χ2(78) = 2396.29, p < .001 indicated that the data were adequate for principal components analysis. In interpreting the pattern matrix, we used a cutoff loading of at least .50 in the factor and no loading greater than .15 in the other factor. Table 1 shows that five items loaded on factor 1 and six loaded on factor 2. The items in factor 1 all referred to somatic symptoms, and the items in factor 2 all involved non-somatic symptoms. The two factors accounted for a cumulative variance of 0.46 and were moderately correlated (r = .55). Based on the EFA, we reduced the scale to 11 items, which comprise the Coronavirus Pandemic Anxiety Scale-11 (CPAS-11).

Table 1

Pattern Matrix of Exploratory Factor Analysis with Promax Rotation

	Factor 1	Factor 2
1. Worried that I or someone in my family might get infected	0.01	0.67
2. Worried about my work/studies	0.33	0.19
3. Felt panic thinking about being near a hospital or clinic for whatever reason	−0.04	0.64
4. Felt panic when I or someone else around me coughs or sneezes	−0.11	0.69
5. Felt bothered that I cannot wash or sanitize my hands as frequently as I want to	−0.04	0.58
6. Felt faster heartbeat	0.70	0.06
7. Felt restless or agitated	0.87	−0.06
8. Found it difficult to fall or stay asleep	0.77	−0.09
9. Had difficulty relaxing	0.91	−0.13
10. Had difficulty breathing	0.68	0.00
11. Felt discomfort in my throat (e.g., itchiness, lump, obstruction, choking)	0.42	0.07
12. Worried about dying	0.10	0.64
13. Worried about family and/or friends dying	0.13	0.67
Proportion variance explained	0.26	0.20
Eigenvalues/Sums of Squares loadings	3.41	2.55

Confirmatory Factor Analysis

We then assessed the structural validity of the two-factor CPAS-11 using confirmatory factor analysis (CFA) again using the statistics software R (R Core Team, 2016) with Rosseel’s (2012) lavaan (latent variable analysis) package. Using the other randomly selected half of the sample (n = 463), we tested two models: a unidimensional model where all 11 items indicated one coronavirus-pandemic-anxiety factor and a two-factor model where five items indicated the somatic coronavirus-pandemic-anxiety factor and six items indicated the non-somatic coronavirus-pandemic-anxiety factor. For each CFA, all items were treated as continuous data, and maximum-likelihood was used as the estimation method. Both CFA models ran normally after 27 to 29 iterations, and the results are summarized in Table 2. The two-factor model showed better fit with the data than the one-factor model, but the modification indices of the two-factor model indicated that covarying items 6 (“Felt faster heartbeat”) and 10 (“Had difficulty breathing”) of factor 1 and items 12 (“Worried about dying”) and 13 (“Worried about family and/or friends dying”) of factor 2 would improve model fit. The CFA results with the covaried items are also shown in Table 2 and is summarized in Fig. 1. These results provide support for the structural validity of the two-factor CPAS-11.

Table 2

Summary of Goodness-of-fit Indexes for Confirmatory Factor Analysis

	Fit indices
Models	χ2	df	p	CFI	TLI	RMSEA [95% CI]	SRMR
One-factor	710.64	44	< .001	0.70	0.62	.18 [.17: .19]	.11
Two-factor	216.98	43	< .001	0.92	0.90	.09 [.08: .11]	.05
Two-factor (covaried)	159.58	42	< .001	0.95	0.93	.08 [.07: .09]	.05

Notes: CFI = comparative fit index, TLI = Tucker-Lewis index; CFI and TLI values > .90 regarded as adequate fit, values > .95 were considered as good fit. RMSEA = root-mean-square error of approximation, SRMR = standardized root mean square residual; RMSEA and SRMR values < .08 indicate good fit (Hu & Bentler, 1999)

Fig. 1

Summary of Results of Confirmatory Factor Analysis of Two-Factor CPAS-11

Internal Consistency of Scale and Subscales

As we have established the structural validity of the two-factor model of the scale, we used the total sample to examine internal consistency. The two factor subscales both demonstrated good internal consistency. For factor 1 (somatic symptoms), Cronbach α = .87 (inter-item correlations from .44 to .71), and for factor 2 (non-somatic symptoms), α = .82 (inter-item correlations from .36 to .62). For the complete scale, α = .87 (inter-item correlations from .15 to .71).

Convergent and Discriminant Validity

To provide initial evidence for the criterion validity, we examined the correlations between the CPAS-11 total score and various established scales and FCV-19S (Ahorsu et al., 2020) using the data from the complete sample. The results (see Table 3) show that CPAS-11 has strong positive correlations with GAD-7, PHQ-9, the PANAS negative affect subscale, and FCV-19S, indicating convergent validity. CPAS-11 also has small to moderate negative correlations with MHQ-SF and the PANAS positive affect subscale, indicating divergent validity.

Table 3

Descriptive Statistics and Results of Correlation Analysis with Criterion Variables

Criterion variable	Cronbach α	M	SD	Correlation with CPAS-11 (r)
GAD-7	.94	7.57	6.06	.76***
PHQ-9	.92	8.64	6.96	.70***
PANAS: Negative Affect	.83	12.89	4.65	.67***
PANAS: Positive Affect	.85	15.93	4.25	−.23***
MHQ-SF	.95	57.11	15.82	−.40***
FCV-19S	.88	20.69	6.01	.60***

***p < .001. Notes: GAD-7: Generalized Anxiety Disorder Scale; PHQ-9: Patient Health Questionnaire; PANAS: Positive and Negative Affect Schedule; MHQ-SF: Mental Health Continuum Short Form (MHC-SF); FCV-19S: Fear of COVID Scale

We undertook a further test of discriminant validity to see whether CPAS-11’s correlation with GAD-7 is stronger compared to its correlations with PHQ-9 and the PANAS negative affect subscale. This is a stringent test because variables that relate to anxiety are also known to relate to depression. We used a software (I. A. Lee & Preacher, 2013, using the equations of Steiger, 1980) to compute the z-score to test the difference between two dependent correlations with one variable in common. The results show that CPAS-11 is correlated more strongly with GAD-7 than with PHQ-9 [.76 vs. .70; z = 5.33, p < .001]. CPAS-11 was also more strongly correlated with GAD-7 than with PANAS negative subscale [.76 vs. .67; z = 6.12, p < .001]. These results suggest that CPAS-11 seems to measure anxiety-related symptoms more than depression symptoms or negative affect.

Finally, we also evaluated CPAS-11 correlations with GAD-7 and PHQ-9 compared to FCV-19S correlations with the same. Using the same approach used in the preceding analyses, we found that GAD-7 was more strongly correlated with CPAS-11 than with FCV-19S [.76 vs. .48; z = 14.15, p < .001]. PHQ-9 was also more strongly correlated with CPAS-11 than with FCV-19S [.70 vs. .34; z = 15.91, p < .001]. Therefore, compared to FCV-19S, CPAS-11 was more strongly associated with established measures for screening anxiety and depression.

Screening Accuracy

We used the easyROC software (Goksuluk, Korkmaz, Zararsiz, & Karaagaoglu, 2016) to test CPAS-11’s screening accuracy and to determine a cutoff score for identifying individuals who experienced severe anxiety using GAD-7 (≥15). A maximum balance between sensitivity (83.6%) and specificity (81.5%) was achieved at the optimal CPAS-11 cutoff score of 15 (see Table 4 for diagnostic performance measures). Fig. 2 shows ROC curves for the behavior of CPAS-11 and FCV-19S (as comparison measure) with respect to GAD-7 screened individuals. CPAS-11’s AUC = 0.90, 95% confidence interval: 0.88: 0.92, which indicates high accuracy (Fischer, Bachmann, & Jaeschke, 2003) and which is also significantly higher than FCV-19S’s AUC = 0.73, 95% confidence interval: 0.68: 0.77, z = 6.62, p < .001.

Table 4

Diagnostic Performance Measures of CPAS-11 at Optimal Cutoff Score of 15

	Value	Lower limit	Upper limit
Sensitivity	0.84	0.77	0.89
Specificity	0.82	0.79	0.84
Positive Predictive Value	0.47	0.43	0.59
Negative Predictive Value	0.96	0.94	0.97
Positive Likelihood Ratio	4.52	3.83	5.32
Negative Likelihood Ratio	0.20	0.14	0.29

Fig. 2

ROC Curve of CPAS-11 for Comparison to FCV-19S and Sensitivity and Specificity Curves for the CPAS-11 Optimal Cutoff of 15

Discussion

The CPAS-11 was developed and initially validated in a sample of adults from different regions across the Philippines. A two-factor structure was identified and confirmed, corresponding to somatic and non-somatic symptoms of anxiety related to the COVID-19 pandemic. The total scale and subscales showed good internal consistency, convergent, and discriminant validity. The results also show that CPAS-11 was also more strongly correlated with established measures of anxiety and depression compared to the recently developed FCV-19S (Ahorsu et al., 2020). CPAS-11 was also more strongly correlated with measures of anxiety than of depression and was accurate in predicting individuals identified as having severe anxiety using the GAD-7. All these results suggest that the CPAS-11 is a promising tool for screening severity and frequency of COVID-related anxiety and for identifying individuals who might need mental health services.

Using the CPAS-11 cutoff score of 15, 29.19% of our participants could be considered as experiencing severe COVID-related anxiety. As our sample was not a nationally representative one, we should not interpret that as a national prevalence rate. But the ratio of participants experience anxiety using CPAS-11 is higher than the 13.84% identified as experiencing severe anxiety using GAD-7, which has been used to measure anxiety of Filipinos during the COVID-19 pandemic in at least one previous study (Bernardo & Mendoza, 2020). The ratio is also higher than the 11.07% of the respondents experienced severe to extremely severe anxiety using the anxiety subscale of the Depression, Anxiety, and Stress Scale (DASS-21; Lovibond & Lovibond, 1995) in another study (Tee et al., 2020) involving a Filipino sample. One might infer that the CPAS-11 is picking up Filipino participants’ anxiety symptoms that were not being screened by more general measures of anxiety like GAD-7 or DASS-21. We should consider this inference with caution, but it could indicate that CPAS-11 is useful in identifying the COVID-19-pandemic anxiety symptoms that are distinct from generalized anxiety disorder symptoms.

The scale items were developed by first referring to items in existing anxiety measures, then considering known symptoms of anxiety reported by clinically referred cases in the Philippines. However, examining the 11 items suggests that they should also apply to different populations with appropriate translation and adaptation. But some of the items refer to symptoms that relate specifically to the COVID-19 outbreak (e.g., panic when someone else coughs or sneezes, anxiety related to not washing one’s hands), and so it is uncertain if they would work with other coronavirus pandemics that involve different forms of transmission and preventive behaviors. Such items could be modified or adapted for future pandemics.

As there is no clear expectation regarding when the COVID-19 pandemic will pass, anxieties related to this coronavirus pandemic are likely to persist. Indeed, such anxieties may exist even after the pandemic passes, similar to what happened in previous viral pandemics (Taylor, 2019). The availability of a reliable and valid multidimensional measure of coronavirus-pandemic-related anxiety will be very helpful in efforts to identify individuals (and groups of individuals) who are suffering from intense adverse affective reactions during the pandemic and even after the pandemic passes. CPAS-11’s two-dimensional structure provides information that could guide therapeutic interventions (i.e., focus on somatic or non-somatic symptoms). The CPAS-11 could also be used to assess whether mental health interventions are alleviating the anxiety-related symptoms over time.

While this is not the first COVID-19 related anxiety scale developed, the CPAS-11 is an improvement over the earlier CAS (S. A. Lee, 2020) because it measures and distinguishes between somatic and non-somatic anxiety symptoms. The CPAS-11 may also somewhat overlap with the CSS (Taylor et al., 2020), which is not ostensibly an anxiety scale, but the CSS subscales correlate significantly with measures of anxiety. We note that the CSS subscales’ correlations with anxiety and depression measures seem to be in a lower range (.27 to .59) compared to CPAS-11’s correlation with anxiety and depression measures (.70 to .76), but such a comparison should be interpreted with caution because different measures of anxiety and depression were used to study CSS. Future research should more directly compare the predictive validity of CPAS-11 compared to CAS and CSS. We did compare the CPAS-11 with FCV-19S and found that CPAS-11 correlated more strongly with anxiety and depression than the fear scale.

While we note the viability and promise of the CPAS-11, we also note other important limitations of the study. First, the pilot study and validation study were both conducted using online modalities, which might make the measure inaccessible to some individuals. For some individuals, answering questions online might also cause further anxieties or other adverse emotional reactions. However, given the requirement of quarantine and physical distancing, online surveys might be the only option during certain periods of a pandemic. Second, the study of the construct validity of CPAS-11 did not include a structured diagnostic assessment, which would have provided more useful information about the diagnostic usefulness of CPAS-11. Third and importantly, although sizable, the sample was not representative of the adult population and was limited to one culture only. Future studies involving more diverse adult samples from different countries would help further establish the validity and utility of CPAS-11.

Related to the last point, in our introduction we suggested that the items in the scale should be applicable to different countries that also experience similar widespread viral contagion and containment experiences, even if the items were developed from experiences of clinical psychologists in the Philippines. This assertion is a hopeful assertion that we make that cannot be validated by our data are exclusively from Filipinos. Whether or not the items in the scale will also be useful and valid in screening anxiety related to the coronavirus pandemic in other countries would have to systematically investigated in validation studies to be conducted with samples from other countries; some of these investigations will require translating the items in the scale into different local languages.

The above limitations notwithstanding, the evidence of the initial structural and construct validity for a scale developed while the COVID-19 outbreak continued to rise in the community being studied is an important contribution. The CPAS-11 is a promising tool for identifying individuals who are experiencing extreme emotional responses during a coronavirus pandemic and who might need mental health services. The tool could also be used to develop and evaluate mental health interventions and programs for individuals and for groups that can address COVID19-related anxiety. The CPAS-11 could also be used to study how anxiety relates to other important adaptive (e.g., compliance with protective health behaviors) and maladaptive (e.g., hoarding) behaviors during the pandemic. With further validation and use, we hope the CPAS-11 will lead to new empirical findings and theoretical insights on people’s emotional, cognitive, and behavioral responses to the coronavirus pandemic.