Validation of the Korean Version of the Coronavirus Reassurance-Seeking Behaviors Scale During the COVID-19 Pandemic.

OBJECTIVE: In this study, we aimed to develop a Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale (CRBS) and to explore its reliability and validity among the general population in South Korea.
METHODS: Using an online survey conducted during November 9-15, 2021, we collected the demographic data of 400 individuals and their responses to rating scales such as the CRBS, the Stress and Anxiety to Viral Epidemics-6 items (SAVE-6), the Generalized Anxiety Disorder-7 items (GAD-7), and the Patient Health Questionnaire-9 items (PHQ-9). We conducted factor analyses and utilized item response theory to confirm the validity and reliability of the Korean version of the CRBS.
RESULTS: Factor analyses revealed that the single factor model of the Korean version of the CRBS showed a good fit with the CRBS (χ2=5.475, df=5, p value=0.361, χ2/df=1.095, CFI=0.999, TLI=0.998, RMSEA=0.015). Multigroup CFA results indicated that the CRBS measures reassurance-seeking behaviors consistently across variables of sex, depression, general anxiety, and viral anxiety. The CRBS also exhibited good convergent validity with the SAVE-6 (r=0.431, p<0.001), GAD-7 (r=0.574, p<0.001), and PHQ-9 (r=0.575, p<0.001).
CONCLUSION: The CRBS is a reliable and valid rating scale that measures reassurance-seeking behavior in relation to viral epidemics.

INTRODUCTION

Since the first case of coronavirus disease (COVID-19) [1], the virus has rapidly spread all over the world. A total of 481,756,671 confirmed cases and 6,127,981 deaths have been reported globally to date [2]. In South Korea, 12,774,956 cases have been confirmed and 15,855 deaths [3] have been noted as of March 30, 2022. In this pandemic, people suffer psychological problems such as depression, anxiety, insomnia, or post-traumatic stress disorder, adding to the stress (29.6%), anxiety (31.9%), and depression (33.7%) prevalent in the general population [4]. The number of confirmed cases has rapidly increased and the South Korean government has recently announced a policy acknowledging the continued association of human life with the coronavirus [5].

In this pandemic, people tend to be anxious about being infected and incurring physical illness or mental dysfunction. Based on the Illness Anxiety Disorder model [6], fear can provoke reassurance-seeking behavior, which may relieve fear or anxiety transiently. However, repetitive reassurance-seeking behavior can lead, in turn, to increased anxiety, enhancing a preoccupation with the causes of the fear. It was reported that 24% of dental students were assessed as having a severe degree of reassurance-seeking behavior [7]. Reassurance-seeking behaviors have been associated with viral anxiety, health anxiety, and death anxiety [8] among the Indian population and health worry or fear of illness and death [9] among the general population in Iraq. In this pandemic, an individual’s viral anxiety can provoke reassurance-seeking behaviors such as washing hands, checking body temperature, monitoring physical symptoms, or collecting information on COVID-19 by searching the internet. These behaviors can reduce viral anxiety transiently, but repetitive behaviors can generate preoccupation with COVID-19. Therefore, we need to assess our degree of reassurance-seeking behaviors and develop guidelines for public mental health policy.

Previously, several rating scales measuring reassurance-seeking behaviors such as the Reassurance-Seeking Scale [10,11], the Reassurance-Seeking Questionnaire [12], and the Threat-related Reassurance-Seeking Scale [13] were developed. However, these scales were not specific to the viral epidemic. To understand people’s motives for seeking reassurance in order to reduce anxiety or fear related to COVID-19 infection, a proper assessment scale which is able to measure viral epidemic-specific, reassurance-seeking behaviors is required. The Coronavirus Reassurance-Seeking Behaviors Scale (CRBS) [14] was developed to measure how frequently individuals engaged in reassuranceseeking activities, specifically in relation to the coronavirus. In the same study, the scale was shown to have a good internal consistency (Cronbach’s α=0.90), although the structural validity of the scale was not revealed. In other research, the CRBS was studied and reported to correlate with neuroticism or hypochondriasis [15]. In this study, we aimed to develop a Korean version of the CRBS and to explore its validity and reliability among the general population in South Korea.

METHODS

Participants and procedure

We conducted an anonymous online survey on November 9–15, 2021, among the general population of South Korea. All 400 individuals participated in this study via EMBRAIN (www.embrain.com), a professional survey company. We collected participants’ information on age, sex, living region, and marital status, but we did not collect any identifiable personal information. We also gathered responses to questions on COVID-19 such as: “Did you experience being quarantined due to infection with COVID-19?,” “Did you experience being infected with COVID-19?,” “Did you get vaccinated?”

Participants’ past psychiatric history was investigated with the following question: “Have you ever experienced depression, anxiety, or insomnia and had treatment for it?” Current psychiatric distress was probed with the question: “Do you think you are depressed or anxious now and do you need help with your mood?”

After the development of an e-survey according to the Checklist for Reporting Results of Internet e-Surveys guidelines [16], investigators (S.C. and C.HK.P.) tested its usability and technical functionality before its implementation. The study protocol was approved by the Institutional Review Board (IRB) of the Asan Medical Center (2021-1490), and the obligation to obtain written informed consent was waived by the IRB.

The sample size estimation was done in terms of 30 participants per cell [17]. We allocated 30–40 samples for 10 cells (two groups of biological sex × five age groups), and the responses of 400 participants were collected from the 14 million general population panels of the survey company. The company sent emails for study enrollment to 1,000–2,000 panelists, and 400 participants among 599 panelists who accessed the survey system were finally enrolled. The company delivered the collected data to the investigators after excluding all identifiable personal information.

Measures

Coronavirus Reassurance-Seeking Behaviors Scale

The CRBS is a self-report rating scale which was developed to measure reassurance-seeking behaviors related to COVID-19 infection [14]. Based on the illness anxiety disorder (formerly known as hypochondriasis) model [6], fears of being sick may make a person anxious, thus inducing reassurance-seeking behavior. Relief from reassurance strengthens the original fear and increases preoccupation with the illness. In this COVID-19 pandemic, recurrent engagement in reassurance-seeking behaviors, such as checking bodily sensations, hand hygiene, visits to the doctor, or repeated media research, paradoxically increases anxiety. The CRBS consists of five items of reassurance-seeking behaviors which can be rated from 0 (not at all) to 4 (nearly every day), with reference to their expression over 2 weeks prior to response to the scale. Total scores range from 0 to 20, and a high score reflects high engagement in reassurance-seeking behavior. The CRBS has strong internal consistency reliability (α=0.90) [14]. In addition, an important problem in using the CRBS in Korean is the accuracy of the translation. In order to verify the accuracy of the translation, the original English version of the CRBS was translated into Korean by a bilingual expert, and the Korean version was translated back into English to ascertain accuracy.

Stress and Anxiety to Viral Epidemics-6 items

The Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale is a self-rating scale developed to measure viral anxiety [18]. It was derived from the SAVE-9 scale [19]. It consists of six items which are rated on a 5-point Likert scale ranging from 0 (never) to 4 (always). A high total score reflects a severe degree of viral anxiety. In this study, we applied the original Korean version of the SAVE-6, and Cronbach’s alpha among this sample was 0.805.

Generalized Anxiety Disorder-7 items

The Generalized Anxiety Disorder-7 items (GAD-7) is a self-report rating scale for measuring the severity of general anxiety [20]. It consists of seven items that can be rated on a 4-point Likert scale from 0 (not at all) to 3 (nearly every day). A high total score reflects a severe degree of generalized anxiety. In this study, we applied the Korean version of GAD-7 [21], and Cronbach’s alpha was 0.948 in this sample.

Patient Health Questionnaire-9 items

The Patient Health Questionnaire-9 items (PHQ-9) is a selfrating scale which was developed for measuring the severity of depression [22]. It consists of nine items that can be rated on a 4-point Likert scale from 0 (not at all) to 3 (nearly every day). A higher total score reflects a severe degree of depression. In this study, we applied the Korean version of PHQ-9 [21], and Cronbach’ alpha was 0.930 in this sample.

Statistical analysis

In this study, we utilized IBM SPSS v26 (IBM Corp., Armonk, NY, USA), RStudio (RStudio, Boston, MA, USA), JASP (JASP, Amsterdam, The Netherlands) and Microsoft Office Excel 365 for data management. Descriptive statistics (percentages, mean, and standard deviation [SD]) were carried out to examine the distribution of responses. The reliability and validity of the Korean version of the CRBS were assessed utilizing both classical and item response theory (IRT). Both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were carried out to explore and confirm the factor structure of the scale. Multigroup CFA was run to assess the measurement invariance of the scale across the variables of sex, depression, generalized anxiety, and viral anxiety. Item analysis was run to estimate the corrected item-total correlation, internal consistency reliability, and average inter-item correlation. The graded response model (GRM) was created in accordance with IRT. Slope and threshold parameters were estimated for the GRM. Moreover, IRT reliability and rho coefficient were calculated. Pearson’s product-moment correlation was run to estimate the association between PHQ-9, GAD-7, and SAVE-6.

RESULTS

All 400 participants were residents of Seoul (n=117, 29.3%), Pusan (n=30, 7.5%), Daegu (n=18, 4.5%), Daejeon (n=9, 2.3%), Gwangju (n=10, 2.5%), Incheon (n=31, 7.8%), Ulsan (n=8, 2.0%), Gyeonggi Province (n=108, 27.0%), Chungcheong Province (n=20, 5.0%), Jeolla Province (n=11, 2.8%), Gyeongsang Province (n=29, 7.3%), Gangwon Province (n=7, 1.8%), and Jeju Province (n=2, 0.5%). Demographic characteristics, responses to COVID-19 questions, and rating scale scores are presented in Table 1.

Table 1.

Clinical characteristics of the study subjects (N=400)

Variable	Value
Male	204 (51.0)
Age (yr)	44.7±13.0
18–29	70 (17.5)
30–39	73 (18.3)
40–49	87 (21.8)
50–59	93 (23.3)
≥60	77 (19.3)
Marital status
Single	132 (33.0)
Married, with kids	220 (55.0)
Married, without kids	25 (6.3)
Others	23 (5.8)
Questions on COVID-19
Have you experienced being quarantined due to infection with COVID-19? (yes)	46 (11.5)
Have you experienced being infected with COVID-19? (yes)	7 (1.8)
Have you been vaccinated? (yes)	366 (91.5)
Psychiatric history
Have you ever experienced depression, anxiety, insomnia, and received treatment for it? (yes)	76 (19.0)
Do you think you are depressed or anxious now and need help with your mood? (yes)	52 (13.0)
Symptoms rating
Coronavirus Reassurance-Seeking Behaviors Scale	3.4±4.0
Stress and Anxiety to Viral Epidemics-6 items	14.7±4.6
Patient Health Questionnaire-9 items	4.7±5.7
Generalized Anxiety Disorder-7 items	3.5±4.7

Values are presented as number (%) or mean±standard deviation.

Initial exploratory factor analysis

Table 2 shows the item-level properties of the Korean version of the CRBS. The Kaiser–Meyer–Olkin value (0.799) and Bartlett’s Test of sphericity value (p<0.001) (Table 2) suggested the sampling adequacy and data suitability for conducting EFA. The corrected item-total correlation values of all items (ranging between 0.577 and 0.777) were above the minimum cut-off value (0.30). The EFA analysis showed the single factor model of the Korean version of the CRBS (eigenvalue=3.397). All items loaded highly on the explored factor (ranging between 0.0704 and 0.878) (Table 3).

Table 2.

Scale-level psychometric properties of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale

Psychometric property	Score	Suggested cut-off
Mean inter-item correlation	0.595	Between 0.15 and 0.50
Cronbach’s alpha	0.867	≥0.7
McDonald’s omega	0.871	≥0.7
Split-half reliability (odd-even)	0.911	≥0.7
Composite reliability	0.879	≥0.7
Standard error of measurement	1.440	Smaller than SD, 3.98/2
Rho coefficient	0.879	≥0.7
IRT reliability	0.812	≥0.7
Results of exploratory factor analysis
Determinant	0.055	>0.0001
KMO measure of sample adequacy	0.799	0.50
Bartlett’s test of sphericity (p value)	1,153.541 (<0.001)	Significant
Eigenvalue	3.397	1 or above
Variance	67.943
Model fits of confirmatory factor analysis
χ2 (df, p value)	5.475 (5, 0.361)	Nonsignificant, <5
CFI	0.999	>0.95
TLI	0.998	>0.95
RMSEA [90% CI value] (p value)	0.015 [0.000–0.073] (0.786)	<0.08
SRMR	0.051	<0.08

IRT, item response theory; KMO, Kaiser–Meyer–Olkin; CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual; SD, standard deviation

Table 3.

Item properties of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale

Item	Response scale (%)					Descriptive (mean±SD)	CITC	CID	Factor loading
	0	1	2	3	4				EFA	CFA
1. I felt dizzy, lightheaded, or faint, when I read or listened to news about the coronavirus.	41.0	33.5	11.0	5.5	9.0	1.08±1.24	0.577	0.883	0.704	0.619
2. I had trouble falling or staying asleep because I was thinking about the coronavirus.	45.3	31.8	15.0	5.0	3.0	0.89±1.03	0.753	0.822	0.850	0.820
3. I felt paralyzed or frozen when I thought about or was exposed to information about the coronavirus.	65.3	19.8	9.3	4.3	1.5	0.57±0.93	0.774	0.819	0.878	0.849
4. I lost interest in eating when I thought about or was exposed to information about the coronavirus.	64.8	19.0	11.0	4.8	0.5	0.57±0.90	0.777	0.820	0.878	0.842
5. I felt nauseous or had stomach problems when I thought about or was exposed to information about the coronavirus.	82.8	8.3	5.8	2.5	0.8	0.30±0.75	0.661	0.851	0.798	0.705

0=not at all, 1=rare, less than a day or two, 2=several days, 4=more than 7 days, 5=nearly every day over the last 2 weeks. SD, standard deviation; CITC, corrected item-total correlation; CID, Cronbach’s alpha if item deleted; EFA, exploratory factor analysis; CFA, confirmatory factor analysis

Confirmatory factor analysis

CFA results showed a good fit for the CRBS model (χ2=5.475, df=5, p value=0.361, χ2/df=1.095, comparative fit index [CFI]=0.999, Tucker–Lewis index [TLI]=0.998, root mean square error of approximation [RMSEA] [90% confidence interval, CI] [p value]=0.015 [0.000–0.073] [0.786], standardized root mean square residual [SRMR]=0.051) (Table 2). Factor loadings in CFA were good. They ranged between 0.619 and 0.849 (Table 3 and Figure 1). Multigroup CFA results showed that the Korean version of the CRBS measured reassuranceseeking behavior consistently across sex (CFI=1.000, TLI=1.011, RMSEA=0.000, SRMR=0.054), depression (PHQ-9 ≥10) (CFI=1.000, TLI=1.097, RMSEA=0.000, root mean square residual [RSMR]=0.063), general anxiety (GAD-7 ≥10) (CFI=1.000, TLI=1.001, RMSEA= 0.000, RSMR=0.061), and viral anxiety (SAVE-6 ≥15) (CFI= 1.000, TLI=1.008, RMSEA=0.000, RSMR=0.064).

Figure 1.

Factor structure of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale (CRBS).

Graded response model analysis

Table 4 presents the GRM parameters (slope and threshold parameters) of the Korean version of the CRBS. Table 4 demonstrates that all the items have very high slope parameters (ranged between 1.925 and 4.493; mean=3.531). Among the items, item 3 has the highest discrimination ability and item 1 has the lowest discrimination ability. Overall, these items are highly efficient and able to provide good information about the latent trait assessed by the Korean version of the CRBS (Figure 2). Regarding threshold parameters, item 5 is the most difficult item among all the items, then item 3 and 4. In items 3, 4, and 5, all threshold coefficients are positive and suggest that above average levels of latent traits are required to endorse all the Likert-type response options. The scale information curve (Figure 3) shows that the Korean version of the CRBS provides more information about people between the 0.0 and 3.0 θ level.

Table 4.

Item fit, slope and threshold parameters of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale

Item	Slope parameter (a)	Threshold parameter (b)
		b1	b2	b3	b4
Item 1	1.925	-0.287	0.969	1.537	1.892
Item 2	3.568	-0.119	0.859	1.600	2.050
Item 3	4.493	0.436	1.128	1.698	2.235
Item 4	3.861	0.436	1.086	1.751	2.731
Item 5	3.807	1.031	1.440	1.967	2.564

Figure 2.

Item response category characteristic curve for each item of Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale.

Figure 3.

Scale information curve of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale. SE, standard errror.

Reliability of the Korean version of Coronavirus Reassurance-Seeking Behaviors Scale and evidence based on relations to other variables

Scale-level psychometric properties were shown in Table 2. The mean item-total correlation (0.595) of the CRBS was above the recommended range (between 0.15 and 0.50). We checked that inter-item correlations and none of the correlations between the items were too high, revealing an absence of multicollinearity. The internal consistency reliability was good (Cronbach’s alpha=0.867, McDonald’s omega=0.871, and split-half reliability=0.911). The standard error of measurement (1.44) was below the cut-off (smaller than SD, 3.98/2). Furthermore, this scale had good IRT reliability (0.812) and a Rho coefficient (0.879). The Korean version of the CRBS showed a good convergent validity with SAVE-6 (r=0.431; 95% CI, 0.347–0.507; p<0.001), GAD-7 (r=0.574; 95% CI, 0.505–0.636; p<0.001), and PHQ-9 (r=0.575; 95% CI, 0.506–0.637; p<0.001). Moreover, the CRBS total score was significantly higher among those with depression (PHQ-9 ≥10, t[398]=10.36, p<0.001), those with general anxiety (GAD-7 ≥10, t[398]=11.14, p<0.001), and those with viral anxiety (SAVE-6 ≥15, t[398]=8.33, p<0.001).

DISCUSSION

In this study, we explored the reliability and validity of the Korean version of the CRBS among the general population during this COVID-19 pandemic. CFA results showed a good fit with the CRBS model, and a multigroup CFA revealed that it measures reassurance-seeking behavior consistently across variables of sex, depression, general anxiety, and viral anxiety. All the items have very high slope parameters, and the scale information curve shows that the Korean version of the CRBS provides more information about people between the 0.0 and 3.0 θ level. The internal consistency reliability was good, and it showed a good convergent reliability with other existing rating scales.

We observed that the Korean version of the CRBS is a valid and reliable rating scale which can be applied to measure reassurance-seeking behavior in this COVID-19 pandemic. Item analysis indicated that in terms of both classical test theory (corrected item-total correlation) and modern test theory (slope parameter of the GRM), this scale is able to discriminate reasonably between low scorers and high scorers. Threshold parameters of the GRM also show that items are comparatively difficult. Above average levels of a latent trait are needed to endorse at least the lowest response option (nearly every day over the last 2 weeks). The scale’s information curve provides a clearer picture of its efficiency in assessing the Korean people’s coronavirus reassurance-seeking behaviors. This scale is accurate at assessing reassurance-seeking behaviors of people between 0.0 and 3.0 θ level. Psychometric information related to IRT indicated the superiority of this scale in comparison to previously published studies reporting on the psychometric properties of the CRBS [14,23].

In this study, we observed good internal consistency in the Korean version of the CRBS, in accordance with previous studies [14,23]. Additionally, there was good convergent validity with another viral anxiety scale (SAVE-6) and the generalized anxiety scale (GAD-7). Since the CRBS is a rating scale measuring reassurance-seeking behavior specific to the viral epidemic, we explored the convergent validity of the CRBS with a nonspecific anxiety scale and an anxiety scale which was specific to the viral epidemic. The CRBS showed good convergent validity with the GAD-7 and SAVE-6 scales. The SAVE-6 was developed to measure an anxiety response specific to viral epidemics. It has already been validated among the general population [18,24] and special populations such as medical students, public workers, and cancer patients [25-27]. Results of this study support the usefulness of the CRBS in examining the mechanism of the association between viral anxiety and reassurance-seeking behaviors in relation to viral epidemics.

This study has some limitations. First, we could not find a reassurance-seeking scale that was not specific to the viral epidemic, as a measure of convergent validity of the CRBS. Although we applied the anxiety scale specific to viral epidemics, it would have been more informative if we could have used another reassurance-seeking behavior scale which was not specific to a viral epidemic. Second, we conducted this study using a professional online survey company. This may have led to bias, since the use of responses generated from panelists already registered in the survey system may have impacted results. This could have been avoided if the participants had been absolutely anonymous. Furthermore, self-report responses may evoke biases such as those of social desirability, and may result in the reluctance of participants to report their reassurance-seeking behaviors. Finally, 91.5% of the participants were already vaccinated, a factor that may have also influenced the results.

In conclusion, we confirm the Korean version of the CRBS as a psychometrically sound tool. It is a valid and reliable rating scale that measures reassurance-seeking behaviors associated with viral epidemics. As a model of anxiety disorder in relation to illness, it will assist with the examinations of people’s viral anxiety and reassurance-seeking behaviors in this COVID-19 pandemic.