Outbreak anxiety scale: Development, validity, and reliability.

OBJECTIVE: Epidemic and pandemic outbreak periods are as stressful for psychiatric symptoms as well as the physical symptoms of the epidemic disease and can trigger and aggravate psychiatric symptoms, especially anxiety. There is no scale specific to the outbreak period and which can be used in other outbreaks. In this study, it was planned to develop a scale to evaluate the anxiety associated with epidemic disease, especially during the epidemic period.
METHODS: In this study, a Likert type scale with 15 items was developed by our research team to evaluate outbreak disease anxiety, and the scale form created was transferred to online use and the reliability of validity was evaluated by obtaining the results of 311 participants in the online environment. The internal consistency of the scale was assessed with Cronbach's Alpha coefficients. Split-half reliability was estimated using Spearman-Brown coefficients unequel length. Explanatory factor analysis, confirmatory factor analysis in AMOS, correlation analysis, and construct validity analysis (convergent validity and discriminant validity) were conducted. Beck Anxiety Inventory and Health Anxiety Inventory was used to evaluate concurrent and discriminant validity.
RESULTS: The Cronbach Alpha coefficient calculated for the evaluation of the internal consistency (homogeneity) of the outbreak anxiety scale was determined as 0.94 and this value shows that the scale has high reliability. With the results of this study, the scale's content validity and construct validity, discrimination, and criterion validity were evaluated and it was shown to have acceptable valid features in all.
CONCLUSION: Outbreak anxiety scale is a valid and reliable tool to evaluate anxiety related with outbreak of epidemic and pandemic disease. Copyright:

Epidemic and pandemic periods are as stressful for psychiatric symptoms as well as the physical symptoms of the epidemic disease and can trigger and aggravate psychiatric symptoms, especially anxiety [1, 2]. Previously, anxiety levels were found to increase during outbreaks [3].

The indication of what could happen during the outbreaks was finally revealed by the corona virus outbreak. Recently, the new human coronavirus disease (COVID-19) is life-threatening all over the world; end of the April 2020, it has become a pandemic that has been affected by more than 2 million people [4]. COVID-19 has started from China and has widely seen in our country along with many other countries in the world. Scientists are trying to deal with the COVID-19 outbreak and continue to study it in many different dimensions. In this period, as well as the physical symptoms of the disease, it has been shown to cause various psychological problems such as anxiety, fear, depression, and insomnia [5]. On this occasion, it was once again seen that psychiatric symptoms and psychological crisis intervention played an important role in the general deployment of disease control [2].

It was once again remembered on the occasion of COVID-19 that it is important to assess the mental health of patients during a life-threatening and restrictive pandemic, to create possible help [5]. In this period, different psychiatric symptoms began to appear and the need for assessment specific to the situation was exposed [6]. Considering that this epidemic is neither the first nor the last pandemic [7]. In Turkey and in the world, lack of tools to assess specific concerns occurs in such pandemics got our attention. Indeed, as there has been seen in studies in Turkey and concerns about the epidemic in the world, which is generally used to evaluate the scales of general, nonspecific symptoms.

The period of pandemic/epidemic disease is uncertain when it ends, countries engage in quarantine policies, and many unpredictable situations related to the disease as well as its economic and social dimension come into play [8]. One of the most stressful situations in life is the unpredictability of the condition and the uncertainty of when to control difficult situations such as disease and the severity of the risk. These can increase anxiety among the masses, along with some negative analysis and misinformation during outbreaks [9]. One of the most important factors that trigger anxiety is uncertainty, a sense of threat to the future and the state of the person, and such difficulties can trigger anxiety disorders and common mental disorders such as depression and hypochondriasis [10, 11]. According to experiences from similar outbreaks and pandemics, in such cases, patients may experience serious anxiety such as fear of death and feelings of loneliness and anger among quarantined people [12, 13]. It is an expected situation that anxiety and fears will emerge in a process where there are fearful experiences and uncertainties such as epidemics. However, some individuals, possibly individuals prone to anxiety disorders, can interpret this process by disaster, experience the physical sensations of panic and anxiety, and show symptoms of anxiety disorder in relation to the outbreak [14, 15]. At this point, it is important to detect anxiety and related factors that affect functionality negatively.

Highlight key points

Anxiety related to outbreak has become a public health problem with the Covid 19 pandemic.

Measuring outbreak anxiety can contribute to studies on the outbreaks.

The Outbreak Anxiety Scale is a valid and reliable tool to evaluate anxiety related with outbreak of epidemic and pandemic disease.

In the COVID-19 period, the fear of COVID-19 scale was developed by Ahorsu et al. [16] to measure fear of the epidemic, but it was observed that the scale was specific to COVID-19. Epidemic diseases will continue to cause anxiety, fear and anxiety, albeit COVID-19, and catastrophic thoughts and comments about the epidemic will trigger anxiety and related symptoms. However, in our country and in the world, there is no scale specific to the outbreak period and which can be used in other outbreaks. In this study, it was planned to develop a scale to evaluate the anxiety associated with epidemic disease, especially during the epidemic period.

METHODS

In this study, a scale was developed by our research team to evaluate outbreak disease anxiety, and the scale form created was transferred to online use and the reliability of validity was evaluated by obtaining the results of 320 participants in the online environment.

Sampling

At least graduates of primary education, no neurocognitive disorder that would prevent them from completing the study, no diagnosis of dementia, head trauma, intracranial infection and delirium were not diagnosed, and individuals aged 18–70 years were accepted to the study. The suitability of the subjects for the study was verified according to the information they provided online. In scale validity studies, the sample size should be 10 to 20 times the number of items [17].

The sample size in this study consists of 311 people, more than 20 times the number of items. Accordingly, 320 participant were accepted to study, 311 people were included in the analysis due to outliers values. The study was conducted in accordance with the Helsinki Declaration and the ethics committee approval was received from the Ethics Committee of the University (number: 71522473/050.01.04/160, date: 20.04.2020).

Application

This study consists of the following steps.

Reviewing the Existing Scales and Developing the Pandemic Disease Anxiety Scale

At this stage, a question pool of 30 items was created by examining the similar scales and literature used especially for measuring anxiety and phobia. Then, it was reviewed with the research team; a scale of 15 items was created.

Expert Opinion for Scope/Content Validity

The quality and number of experts is important in obtaining objective results in evaluations to determine the validity of the scope [18, 19]. In this study, the opinions of 12 experts, consisting of psychiatric specialists and public health and psychiatric nursing specialists, were taken. While 11 of the scale items were found appropriate by all experts, similar arrangement suggestions were received from the experts about 4 items and related items were restructured in line with the opinions of the experts.

Pilot Application and Review of the Scales

The scale was evaluated through one-on-one interviews with 15 volunteer participants from the visitors to the hospital, and feedback was received from the participants about how much the items were understood. Beck anxiety inventory and health anxiety inventory and specific phobia scale were also applied to the participants, which were planned to be completed together in the pilot application, but the participants stated that they had difficulty in establishing a relationship with the epidemic while evaluating the specific Phobia scale. On this, a data kit was developed from the sociodemographic data form and the outbreak disease anxiety scale and the sociodemographic data form that we developed together with the back anxiety scale and the epidemic anxiety scale by excluding the specific Phobia scale from the study.

Delivery and Filling of the Scale with the Beck Anxiety and Health Anxiety Inventory to the Participants

Simultaneous criterion testing was planned through correlation evaluations during the validity and reliability study of the scale. For this purpose, sociodemographic data form and beck anxiety inventory, health anxiety inventory, which are currently used to express similar symptoms, have been filled in online. With the e-mail address and telephone number confirmation, a person filled in a single scale.

MATERIALS

Sociodemographic Data Form

This form contains information such as the age, gender, marital status, and whether psychiatric treatment has been previously obtained by the participant developed for the purpose of the study.

Beck Anxiety Inventory

Developed by Beck et al. [20], it is widely used in measuring anxiety symptoms. It measures the frequency of anxiety symptoms experienced by the individual. It is a Likert-type self-rating scale consisting of 21 items and scored between 0 and 3. High score indicates that the individual has a high level of anxiety. Validity and reliability study in our country were done by Ulusoy et al. [21]. In this study, it was accepted as: 8–15 points = mild anxiety; 16–25 points = moderate anxiety; and 26–63 points = severe anxiety.

Health Anxiety Inventory

This scale was developed by Salkovskis et al. [22] to evaluate anxiety about health. The short version is a self-report scale consisting of 18 items. Fourteen items of the scale question the mental state of the individuals and consist of expressions containing quadruple answers. In the remaining four questions, the participants are asked to have an idea of how their mental state might be with the assumption of a serious illness they have and there are questions accordingly. The scale is a Likert type scale with a score of 0–3 for each item, and a high score indicates a high level of health anxiety. The factor structure of the health anxiety inventory short form used in this study consists of two dimensions. When the factor structure of the short form was examined, it was determined as an additional dimension related to the body size and negative consequences of diseases. The body size includes the first 14 items, and the additional dimension contains four questions added in relation to the negative consequences of diseases. The first validity and reliability study of the Turkish version of the health anxiety scale was first studied in patients with panic disorder by Karaer et al. [23]. Then, in the study conducted by Aydemir et al. [24], in addition to the patients with panic disorder, somatoform disorders and major depressive disorder group were added in addition to the structured psychiatric interviews, and it was concluded that the health anxiety scale was a reliable and validly available assessment tool in the evaluation of the health anxiety. In this study, 20 and above scores were accepted as high health anxiety.

Statistical Analysis

The data of the study were transferred to SPSS 21.0 program on the computer running with Windows software package and evaluated by this program. Descriptive analyzes and frequency analyzes were done first, and then, the groups were compared. When the groups are evaluated, the Student-t test was used to compare the average of the variables that fit the normal distribution of the Kolmogorov Smirnov test and the Mann–Whitney U test for variables that do not fit the normal distribution in the comparison. Kruskal Wallis test was used to compare more than two variables (marital status) that do not fit the normal distribution Pearson correlation analysis was performed for scale scores evaluations. Categorical variables were evaluated by Chi-square analysis. Significance level was accepted as p<0.05.

The internal consistency of the scale was assessed with Cronbach’s Alpha coefficients split-half reliability was estimated using Spearman–Brown coefficients unequel length.

The construct validity of the scale was evaluated by Principal Components Analysis and Explanatory Factor Analysis using Direct Oblimin Rotation method. Before making the factor analysis, the suitability of the sample to the factor analysis was evaluated by Kaiser–Meyer–Olkin (KMO) sample adequacy measure. KMO varies between 0 and 1 and is expected to approach 1. The KMO value between 0.90 and 1.00 indicates that the sample adequacy is very good [17, 25].

The Barlett–Sphericity Test was used to determine the relationship of the items of the scale [26].

Factor structure obtained by explanatory factor analysis x2, x2/df, comparative fit index (CFI), goodness of fit index (GFI), adjusted goodness of fit index (AGFI), root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), normed fit index (NFI), trucker lewis index (TLI), consistent akaike information criteria (CAIC), akaike information criteria (AIC), and expected cross-validation index (ECVI) fit indexes were evaluated using Confirmatory Factor Analysis [27].

Simultaneous criterion test was used with the health anxiety inventory and beck anxiety inventory to evaluate the concurrent validity of the scale, item discrimination power index to evaluate discrimination (sub-upper group averages difference item analysis) and previously known groups method (with health anxiety inventory and beck anxiety inventory) to evaluate discriminant validity [25, 27].

RESULTS

A total of 320 people participated in the research, 311 people were included in the analysis. The mean age of the individuals participating in the study 36.12±11.60 and 42.8% was male. Sociodemographic data of the participants are presented in Table 1.

TABLE 1

Sociodemographic characteristics of the participants

Features	%	n
Age
18–24	16.4	51
25–34	33.4	104
35–44	25.4	79
45–54	11.3	35
55–70	13.5	42
Gender
Male	42.8	133
Woman	57.2	178
City
Istanbul	23.5	73
Sakarya	23.2	72
Bursa	11.3	35
Antalya	8.3	26
Kocaeli	5.1	16
Izmir	4.2	13
Erzurum	4.2	13
Ankara	1.9	6
Other	18.3	57
Education
Primary school	3.2	10
Middle School	2.9	9
High school	21.2	66
University	52.1	162
Master and above	20.6	64
Marital status
The married	44.1	137
Single	48.6	151
Widow/Divorced	7.4	23
Child
No	54.0	168
A child	17.4	54
Two kids	22.5	70
Three or more children	6.1	19
Occupation
Occupied	66.9	208
No-occupation	33.1	103
Having chronic illness
Yes	22.5	70
No	77.5	241
Health employee
Yes	24.1	75
No	75.9	236
Close contact history with
someone diagnosed with COVID
Yes	11.3	35
No	88.7	276
Psychiatric diagnosis
Yes	17.4	54
No	82.6	257
Psychiatric treatment
Yes	7.1	22
No	92.9	289

Internal consistency

The Cronbach Alpha coefficient calculated for the evaluation of the internal consistency (homogeneity) of the outbreak anxiety scale was determined as 0.94. The contribution of the items of the scale to the consistency is presented in Table 2. Mean of inter item correlations is 0.522, min-max correlations are 0.277–0.746.

TABLE 2

Distribution of scale items and evaluation of the effects of items on internal consistency

Items		Item mean	Item standard deviation	Scale mean if item deleted	Corrected item -total correlation	Cronbach’s alpha if item deleted
1	I am concerned about the possibility of outbreak disease transmission to me and my relatives	3.14	1.195	28.84	0.619	0.940
2	I feel anxious, worried, or nervous throughout the day	2.30	1.024	29.68	0.783	0.935
3	I have trouble dealing with my anxiety	1.92	0.916	30.06	0.750	0.936
4	I find it difficult to continue my daily life because of my anxiety	1.64	0.827	30.34	0.694	0.937
5	Although the possibility of contact is low, it seems as if I and my relatives will be infected	2.38	1.006	29.60	0.704	0.937
6	I’m worried about the future	2.88	1.188	29.10	0.633	0.939
7	When I think of the outbreak, I feel heart palpitations, sweating, difficulty breathing, weakness and/or fainting	1.40	0.710	30.59	0.531	0.941
8	I’m having trouble in concentrating and focusing on other issues due to the outbreak	1.79	0.931	30.19	0.670	0.938
9	I have difficulty getting thoughts of the outbreak out of my mind	1.89	0.963	30.09	0.765	0.935
10	My sleep pattern has been disrupted due to my concerns about the outbreak	1.80	1.131	30.18	0.686	0.937
11	I am anxiously waiting for myself and one of my relatives to fall ill at any moment	2.34	1.080	29.64	0.797	0.934
12	I have visions of myself or my relatives contracting an outbreak disease	1.82	0.935	30.16	0.688	0.937
13	It feels like the outbreak disease in myself or someone close to me, even if there are no symptoms	1.77	0.909	30.21	0.698	0.937
14	I am afraid of losing my life or my relatives because of the outbreak	2.59	1.163	29.40	0.720	0.937
15	I get nervous, anxious when I hear about or talk about the outbreak	2.32	1.044	29.66	0.767	0.935

Split half reliability for internal consistency

In split-half reliability analysis, the Spearman-Brown-coefficient unequel lenght was 92.

The suitability of the sample and the scale for factor analysis

Before factor analysis, the suitability of the sample to factor analysis was evaluated by KMO sample adequacy measure and KMO value was found to be 0.943.

As a result of the Barlett–Sphericity test to evaluate the relation of the items with each other, the Chi-square value was found to be 3,106,079 and p<0.001. These values showed that the index has at least two sub-dimensions and contains correlation levels that reflect a certain structure among the items. The data used in the research were interrelated and were suitable for factor analysis. KMO and Barlett–Sphericity Test Results are shown in Table 3.

TABLE 3

Kaiser-Meyer-Olkin ve barlett spherity test results

AKaiser-Meyer-Olkin coefficient	0.943
Chi-square	3106.079
df	105
P-value	<0.001

Construct validity of the scale

1. Explanatory factor analysis

Explanatory factor analysis was performed to determine the structural validity of the index. “Principal components” analysis was chosen as the factor determination method and “direct oblimin” technique, which is one of the oblique rotation techniques, was used. Two factors were obtained, explaining 63.60% of the total variance of the index after rotation. The total variance described in the figure and table is shown. Factor 1 and Factor 2 sub-dimensions were explained 55.799%, and 7.803% of the variance, respectively (Table 4).

TABLE 4

Total variance distribution explained

Factors	Eigen value	Variance (%)	Cumulative variance (%)
1	8.370	55.799	55.799
2	1.170	7.803	63.602

The scale consisted of two factors. When the items were evaluated according to their content, it was observed that Factor 1 was related to anxiety and Factor 2 was related to difficulty in dealing with anxiety. Factor 1 was including items: 1, 2, 5, 6, 11, 12, 13, 14, and 15, and Factor 2 was including; 3,4,7,8,9, and 10 (Table 5). The internal consistency rates of the factors are given in Table 6.

TABLE 5

Factor analysis for outbreak anxiety scale

Items		Components
		1	2
12	I have visions of myself or my relatives contracting an outbreak disease	0.848
11	I am anxiously waiting for myself and one of my relatives to fall ill at any moment	0.842
14	I am afraid of losing my life or my relatives because of the outbreak	0.807
1	I am concerned about the possibility of outbreak disease transmission to me and my relatives	0.798
13	It feels like the outbreak disease in myself or someone close to me, even if there are no symptoms	0.746
5	Although the possibility of contact is low, it seems as if I and my relatives will be infected	0.740
6	I’m worried about the future	0.642
15	I get nervous, anxious when I hear about or talk about the outbreak	0.548
2	I feel anxious, worried, or nervous throughout the day	0.520
7	When I think of the outbreak, I feel heart palpitations, sweating, difficulty breathing, weakness and/or fainting		0.903
8	I’m having trouble in concentrating and focusing on other issues due to the outbreak		0.772
4	I find it difficult to continue my daily life because of my anxiety		0.741
3	I have trouble dealing with my anxiety		0.680
10	My sleep pattern has been disrupted due to my concerns about the outbreak		0.534
9	I have difficulty getting thoughts of the outbreak out of my mind		0.494

TABLE 6

Factors and internal consistency

Factors	Cronbach alpha	Number of the items	Hotelling’s T² Testi	p
1 Anxiety	0.920	9	723.280	<0.001
2 Difficulty in coping with anxiety	0.887	6	170.218	<0.001

2. Previously known groups method for discriminant validity

A statistically significant difference was found when the scores of those who received <20 points from the health anxiety scale and those who received more than 20 points from the outbreak anxiety scale were compared. The ones who got higher than 20 points in health anxiety scale (n=244) had higher scores in outbreak anxiety scale than the ones who got higher than 20 points in health anxiety inventory (n=67) (respectively 29.16±8.98 vs. 42.25±12.53 p<0.001).

The scores taken from high to low from the beck anxiety inventory were similar in the outbreak anxiety scale. There was also a significant difference between outbreak anxiety scale scores in individuals grouped according to their scores on the beck anxiety inventory (respectively n=158, mean±SD 25.70±6.71; n=84, mean±SD 34.75±8.82; n=44, mean±SD 40.25±11.55; n=25, mean±SD 47.80±12.28, p<0.001) [20].

3. Item analysis based on difference of lower-upper group means for discriminant validity

The ones with the lower values of %27 (n=86) had lower scores of the scale than the ones with the upper scores of 27% (n=84) (respectively 20.69±2.16 vs. 47.43±7.87 p<0.001).

4. Confirmatory factor analysis

In confirmatory factor analysis, χ2: 387.437 df: 89 χ2/df: It is acceptable with 4.353 and <5. Acceptable compliance with CFI: 0.902 is good indication of SRMR: 0.050. RMSEA: 0.104 GFI: 0.851, NFI: 0.878, TLI: 0.885, AGFI: 0.799 are in poor agreement. CAIC (596.37<3266.74), AIC (449.44<3195.65), ECVI (1.45<10.31) values show acceptable fit by taking smaller values from the independent model.

Criterion validity

Beck anxiety inventory and health anxiety inventory were used to evaluate “Concurrent Validity”, which is one of the criterion validity types. The outbreak anxiety scale was found to be moderately correlated with beck anxiety (r=0.691; p<0.001) and the health anxiety inventory (r=0.565; p<0.001). The Outbreak Anxiety Scale is presented in Appendix 1.

APPENDIX 1

Salgın Anksiyetesi Ölçeği

Adı Soyadı:------------------------------------------------
Tarih: -------------------------------------------------------
Aşağıdaki sorular içinde bulunulan salgın dönemi ile ilgilidir. Özellikle son 1 haftayı dikkate alarak yanıtlayınız. Ölçekteki her bir maddeyi 0’dan 4’e kadar puanlayınız. 0 hiç, katılmıyorum, en düşük puan anlamına gelirken 4 ise en şiddetli, tamamen katılıyorum anlamına gelmektedir.
	Hemen hiç	Bazen	Sıklıkla	Çoğu zaman	Hemen her zaman
1 Bana ve yakınlarıma hastalık bulaşması olasılığı ile ilgili kaygılıyım	0	1	2	3	4
2 Kendimi gün boyu kaygılı, endişeli veya gergin hissediyorum	0	1	2	3	4
3 Kaygılarımla başa çıkmakta zorlanıyorum	0	1	2	3	4
4 Kaygılarım nedeni ile gündelik hayatımı devam ettirmekte zorlanıyorum	0	1	2	3	4
5 Temas olasılığı az olsa da kendime ve yakınlarıma hastalık bulaşacakmış gibi geliyor	0	1	2	3	4
6 Gelecek hakkında endişeliyim	0	1	2	3	4
7 Salgın hastalığı düşündüğümde kalp çarpıntısı, terleme, nefes almada zorluk, güçsüzlük ve/veya baygınlık hissediyorum	0	1	2	3	4
8 Salgın nedeni ile dikkatimi toplamakta ve başka konulara odaklanmakta zorlanıyorum	0	1	2	3	4
9 Salgın ile ilgili düşünceleri zihnimden uzaklaştırmakta zorluk çekiyorum	0	1	2	3	4
10 Salgın hastalık ile ilgili kaygılarım nedeni ile uyku düzenim bozuldu	0	1	2	3	4
11 Kendim veya yakınlarımdan biri her an hastalanabilir diye kaygı ile bekliyorum	0	1	2	3	4
12 Kendim veya yakınlarımın salgın hastalığa yakalandığı hayalleri gözümün önüne geliyor	0	1	2	3	4
13 Belirtiler olmasa da kendimde veya yakınlarımda salgın hastalık varmış gibi geliyor	0	1	2	3	4
14 Salgın hastalık yüzünden hayatımı veya yakınlarımı kaybetmekten korkuyorum	0	1	2	3	4
15 Salgın hastalık ilgili haber aldığımda veya konuşulduğunda gergin, endişeli oluyorum	0	1	2	3	4

DISCUSSION

This study was conducted during the COVID-19 outbreak. This period is a period in which face-to-face interviews are risky in terms of transmission and curfews come into play, and the online environment has to be used instead of face-to-face interviews.

During the development of the scale, at the first stage, the study team created 30 items by scanning the literature and previous anxiety-related scales. Then of these, a 15-item scale was created and opinions were received from 12 experts. While 11 of the scale items were found appropriate by all experts, similar arrangement suggestions were received from the experts about 4 items and related items were restructured in line with expert opinions. With a restructured scale, it was piloted with a group of 15 people. In the face-to-face interviews held here, it was concluded that the items were understandable and responsive. As a result of the opinions obtained, the number of items was determined as 15. This stage, which is structured with expert opinion and pilot implementation, can be considered as content validity for the 15 items obtained in the first stage [25].

In the study, 15 items considered to be valid in scope were filled in by 311 people and the advanced analysis process on the scale started. In the selection of the sample, attention was paid to ensure diversity of variables such as different gender, education level, and employment status, and this diversity was provided to the greatest extent. This situation was thought to contribute positively to the validity of the scale [28, 29].

Cronbach’s Alpha value was determined to be 0.940 in the first reliability analysis made in the scale-related analyzes, and it was investigated whether the items were used to increase reliability when removed from the scale. In the analysis made, when removed from the scale, the item that would significantly increase the internal consistency of the scale was not detected and the scale was preserved as it is. In this state, the Cronbach’s Alpha value is above 0.80 which is accepted as high reliability with the value of 0.940 and it can be said that the scale reliability is quite high [25, 30]. Split-half reliability analysis also yielded good results as the Spearman-Brown-coefficient unequel length was 92 [31].

Before factor analysis to evaluate the structural validity of the scale, firstly, KMO and Barlett’s analysis, KMO adequacy coefficient was evaluated and it was concluded that the sample was sufficient and the data were suitable for further analysis [32, 33]. As a result of the Barlett–Sphericity test conducted later, it was seen that the index had at least two sub-dimensions and included correlation levels to reflect a certain structure among the items. The data used in the research were interrelated and were suitable for factor analysis [32].

Then, explanatory factor analysis was performed to determine the structural validity of the scale. “Basic components” analysis was chosen as the factor determination method and “direct oblimin” technique, which is one of the oblique rotation techniques, was used. Two factors were obtained, explaining 63.60% of the total variance of the index after rotation. According to the Eigen value (eigenvalue) criterion, Factor 1 and Factor 2 sub-dimensions were explained 55.799% and 7.803% of the variance, respectively. This rate corresponds to “more than two-thirds of the sample” defined as acceptable for such scales and has been evaluated in favor of validity [25].

When the factor loadings of the items are evaluated, the factor loads of the items change between 0.494 and 0.903 as a result of “Oblimin with Kaiser Normalization.” All these results show that the structural validity of the scale is sufficient [34].

In this study, beck anxiety inventory and health anxiety inventory were used to evaluate “Concurrent Validity,” which is one of the criterion validity types, to evaluate the criterion validity of the scale. The outbreak anxiety scale showed a significant correlation with beck anxiety inventory.

For the discriminant validity, beck anxiety inventory and health anxiety inventory were used. The scores taken from high to low from the beck anxiety inventory were similar in the outbreak anxiety scale. In addition, a statistically significant difference was found when the scores of those who received <20 points from the health anxiety scale and those who received more than 20 points from the outbreak anxiety scale were compared. These results were congruent with previously known groups method for discriminant validity [27]. Furthermore, significant results were obtained in item analysis based on difference of lower-upper group means that shows the scale is fit for discriminant validity [25, 27].

This scale is the first scale developed to evaluate epidemic/pandemic outbreak anxiety, and it is valid and reliable according to our results. The fact that most of the work has been done in the online environment is an important limitation of the study. The discrimination power of this scale should be supported by studies, in which DSM5-5 structured psychiatric interviews were conducted during the period when the epidemic environment completely recovered and face-to-face interviews were possible. However, many analyzes were conducted to evaluate the validity and reliability of the scale, and it showed that the features such as high internal consistency, factor analysis, and criterion validity and discriminant validity were obtained and a valid and reliable scale was obtained. Using this scale in scientific studies and clinical applications can help to distinguish between situations where ordinary anxiety turns into anxiety that negatively affects the functionality of the patient and to monitor the change in anxiety level of the patient. It is hoped that it will contribute to the development of interventions such as appropriate behavior and treatment information to identify relevant factors.

Limitations

The online structure of study is a limitation due to the fact that only those who have access to the internet or those who have smart phones are included in the study. Another limitation is that the education level of the study group is high, about half of the study group consists of people with university or higher education. Therefore, it may be necessary to study the validity and reliability of the scale in groups with medium and low education levels.

Strengths

The similar distribution of sociodemographic characteristics of the study group such as gender and marital status is a positive feature in terms of generalization of the scale. Participation from provinces with different epidemic density contributed positively to the evaluation of the scale. End of all, the scale provides a practical, self-report tool which can be used to evaluate anxiety due to the outbreak during terms such as epidemics and pandemics by researchers and clinicians.

Conclusion

The Cronbach alpha coefficient calculated for the evaluation of the internal consistency (homogeneity) of the outbreak anxiety scale was determined as 0.94 and this value shows that the scale has high reliability. With the results of this study, the scale’s content validity and construct validity, discrimination, and criterion validity were evaluated and it was shown to have acceptable valid features in all.