Validation of the COVID-19 Disbelief Scale: Conditional indirect effects of religiosity and COVID-19 fear on intent to vaccinate.

The COVID-19 pandemic uprooted economies, infected millions, and altered behaviors. Yet, the invisible nature of the disease, paralleled symptoms to the common flu, and misinformation generated COVID-19 disbelief. Many believed COVID-19 was a hoax. Many believed case numbers were fabricated. Others claimed it was a ruse for sociopolitical reasons. The construction of the 8-item COVID-19 Disbelief Scale (CDS) measures the false belief COVID-19 was not real and life-threatening. The CDS demonstrated discriminant validity and robust reliability across two studies. Predictive analysis evinced COVID-19 disbelievers feared COVID-19 less and had lower intent to get vaccinated. In the U.S., certain religious organizations spread misinformation. Religiosity associated with greater COVID-19 disbelief. Among disbelievers, conditional indirect effects of religiosity associated with greater COVID-19 fear and higher intent to get vaccinated. The moderated mediation model validated utility of the CDS as a concise instrument to study variable relationships. Published by Elsevier B.V.

Introduction

Disbelief persists as a problem with new disease mitigation with the expansion of speculatory misinformation spread online (e.g. H1N1, Ebola, COVID-19) (Aiyewumi & Okeke, 2020; de Bruin et al., 2020; Sell, Hosangadi, & Trotochaud, 2020; Tandoc & Lee, 2020). The microscopic and obscure nature of diseases requires rudimentary epidemiology to understand, but unfiltered statements can muddle public understanding. For example, many classified COVID-19 as the common flu because of similar symptoms (e.g. fever, dry cough) (Rothan & Byrareddy, 2020; Struyf et al., 2020). While widespread, not everyone personally experienced severe COVID-19 symptoms (e.g. ventilation or hospitalization) (Menni et al., 2020). The gravity and distinguishing features of COVID-19 infections was felt by some, while misconceived by others (Bendau et al., 2020). Media and communication from others influenced COVID-19 beliefs (Cinelli et al., 2020). The current study constructs and validates a new measure to identify COVID-19 disbelievers because of this problem.

In the digital age, the surfeit of misinformation competes with scientific health developments which warranted the creation of a COVID-19 disbelief measure (Bode & Vraga, 2018; Tandoc & Lee, 2020). Conflicting early official medical pandemic statements stated facial coverings would not prevent transmission (Tandoc & Lee, 2020). Soon after, it became evident that respiratory air droplets transmit COVID-19 (Sell et al., 2020). However, disinformation campaigns established COVID-19 disbelief (Bruder & Kunert, 2020; Uscinski et al., 2020). By the time public health officials attempted to correct misinformation, many viewed the COVID-19 pandemic as another season of the common flu. Bruder and Kunert (2020) found generic conspiracy beliefs to strongly correlate with the belief COVID-19 were a hoax. This implored COVID-19 disbelief as a construct measure to develop.

A COVID-19 disbelief instrument was absent at the time of this study. Researchers created scales on fear of COVID-19 (Ahorsu et al., 2020), COVID-19 stress (Taylor et al., 2020), COVID-19 anxiety (Lee, 2020), and COVID-19 burnout (Yildirim & Solmaz, 2020). However, widespread disbelief and disregard for safety policies contributed to spread of the disease (Kowalski, Marchlewska, Molenda, Górska, & Gaweda, 2020). For example, persons that disobeyed national U.S. shelter-in-place/lockdown guidelines, visited a crowded event, and returned home to multigenerational households related to communal spread of COVID-19 in 2020 (Atrubin, Wiese, & Bohinc, 2020; Mahale et al., 2020). The COVID-19 Disbelief Scale (CDS) provides an instrument to identify disbelievers and study related behaviors.

Furthermore, misinformation was inevitable when trial and error was part of the emergency medical response. This was apparent in experimenting with hydroxychloroquine and lopinavir–ritonavir as a treatment strategy (Danion et al., 2020). U.S. politicians used these experiments for political gain to abate fears (Alvarez & Barney, 2020). These experimental results eventually did not yield significant benefits that outweighed side-effects (Al-Bari, 2020). However, commercial hydroxychloroquine as a treatment for COVID-19 remained falsely a part of societal beliefs (Carbone, Green, Bucci, & Lednicky, 2020). This falsehood muddled COVID-19 perceptions as a real threat because of perceived readily available treatments (actually harmful). Meanwhile, many took the pandemic seriously by sheltering-in-place (i.e. remaining at home, except for essential reasons). However, boredom and desensitization overtime resulted in ignoring shelter-in-place/lockdown mandates (Danckert, Boylan, Seli, & Scholer, 2020). Some poorly reasoned lack of personal infection demonstrated COVID-19 was not real.

Many in the U.S., participated in the disbelief campaign responding to the Twitter post #FilmYourHospital (Ahmed, Seguí, Vidal-Alaball, & Katz, 2020; Gruzd & Mai, 2020). Disbelievers recorded hospital waiting rooms and parking lots as supposed video evidence of fake COVID-19 case numbers (Ahmed et al., 2020; Gruzd & Mai, 2020). This faulty logic implied media outlets and hospitals conspired to create a bogus scare to infringe on personal liberties (Uscinski et al., 2020). Many hospitals did not overflow with patients because providers admitted only severe cases and advised mild cases to self-quarantine because of high contagion risks (Klompas, 2020). Unfortunately, these conspiratorial beliefs interfered with patient care and added to society's pandemic hysteria. The CDS provides a tool for researchers to study antecedents and outcomes associated with disbelief that hinders society from reopening.

In the U.S., religiosity has been an influential factor in the misinformation generated about COVID-19 (Bentzen, 2020; Hill, Gonzalez, & Upenieks, 2021). Religious affiliation related to greater disregard to shelter-in-place guidelines to remain at home (Hill et al., 2021). Biblical literalist conservative Protestants studied from 1984 to 2010 increasingly opposed scientists on moral and social issues (Evans, 2013). Strong evangelical beliefs related to anti-science attitudes (Gauchat, 2008). March 2020, Louisiana pastor Reverend Tony Spell packed the Life Tabernacle Church in East Baton Rouge with over 1000 parishioners (Slisco, 2020). Regarding COVID-19 Rev. Spell stated, “It's not a concern… we are going to assemble no matter what someone says” and “Our church is a hospital where the sick can come and get healing”. Meanwhile, Virginia Bishop Gerald O. Glenn congregated in-person at Richmond's New Deliverance Evangelistic Church despite safety guidelines discouraging gatherings larger than ten (Brown, 2020). March 22, 2020, Bishop Gleen stated, “I firmly believe that God is larger than this dreaded virus” and “people are healed” in his church. Soon after, he died from COVID-19 complications. To their last breath, many COVID-19 disbelievers laid in their hospital bed claiming it was not real (Elliott, 2020). COVID-19 disbelief cost lives. Given the communication of disbelief from religious organizations with large followings in the U.S., studying the CDS with religiosity was inherent for this study.

With COVID-19 variants and most of the global population yet to vaccinate, nations will continue to combat the disease for many years (Johns Hopkins University & Medicine, 2021; Mahase, 2021). Immunization through vaccination eradicated smallpox worldwide in the 1970's (Anderson & May, 1985). Childhood vaccination of measles, mumps, pertussis and rubella curtailed outbreaks in the U.S. (Jones & Helmreich, 2020). Mass vaccination can safely reach herd immunity (Britton, Ball, & Trapman, 2020). Herd immunity occurs when a large portion of the population no longer transmits the disease because of inoculation (Betsch, Böhm, Korn, & Holtmann, 2017). Unfortunately, COVID-19 disbelief poses a problem to reach herd immunity. If not real, why vaccinate? This study models the CDS on intent to vaccinate to establish predictive validity and utility of this instrument (Hinkin, 1995).

Literature review

Terror management theory and coping with the COVID-19 pandemic

The initial outbreak of the novel COVID-19 struck uncertainty and fear around the world (Bendau et al., 2020; Bitan et al., 2020; Satici, Gocet-Tekin, Deniz, & Satici, 2020). Pandemic news reports, social conversations about COVID-19, and societal shutdowns instilled life-threating concerns (Basch, Kecojevic, & Wagner, 2020). Terror management theory postulates humans are aware of eventual death, but major events heighten fears that humans cope with behaviors and belief systems (Greenberg, Pyszczynski, & Solomon, 1986; Solomon, Greenberg, & Pyszczynski, 1991). For example, humans seek prayer and developed belief of an afterlife to allay human existence ends (Jonas & Fischer, 2006; Vail et al., 2010). The COVID-19 pandemic was a major worldwide event resulting in losses. Humans reacted by consuming media to understand how to survive and cope (Bendau et al., 2020; Cinelli et al., 2020).

Information at the onset of the COVID-19 pandemic was muddled by speculations (Brennen, Simon, Howard, & Nielsen, 2020; Singh et al., 2020). In the U.S., misinformation spread on social media reverberating politically motivated false statements such as “It is just like the flu” (Samore, Fessler, Sparks, & Holbrook, 2020; Uscinski et al., 2020). Researchers found conservatism linked to lower perceptions of COVID-19 as a threat because of source of misinformation (Calvillo, Ross, Garcia, Smelter, & Rutchick, 2020). Misinformation cannot be easily unseen or removed from someone's cognition (Davis & Loftus, 2007; Mazzoni & Vannucci, 2007; Patihis et al., 2013). While public health officials tried to correct the spread of misinformation, misunderstanding of COVID-19 as a real threat had permeated society (Bode & Vraga, 2018; Miller, 2020). Hence, the researchers defined COVID-19 disbelief as the false perception COVID-19 was not a real novel disease with potential life-threatening consequences.

Misinformation from media sources hampered preventative behaviors that would have reduced the spread of COVID-19 (e.g. wearing a face mask) (Romer & Jamieson, 2020). For example, COVID-19 vaccine misinformation was found to impede intent to vaccinate (Loomba, de Figueiredo, Piatek, de Graaf, & Larson, 2020). Widespread vaccination intends to reach herd immunity. Herd immunity occurs when the disease does not incubate and continue spreading from one person to another over a succession of time (Fine, 1993; Randolph & Barreiro, 2020). Theoretically, effective herd immunity requires “55% to 82%” of the population to vaccinate (DeRoo, Pudalov, & Fu, 2020; Sanche et al., 2020). Vaccination represents the future for society to reopen because it will curb transmission of COVID-19. Disbelief of COVID-19 assumes the false pretense that vaccinating serves no or little purpose.

With required shelter-in-place/lockdown orders, social media usage rose during the pandemic (Bendau et al., 2020; Ferrara, 2020). Unfortunately, coping by consuming social media misinformation did not change the real-life threat of COVID-19 (Imhoff & Lamberty, 2020). In fact, Imhoff and Lamberty (2020) found COVID-19 disbelief associated with lower participation in preventative behaviors (e.g. mask wearing). An important preventive COVID-19 behavior includes vaccination. Hence, vaccination hypothetically serves negligible purpose to COVID-19 disbelievers. Fig. 1 illustrates the proposed model of relationships.

Fig. 1

Hypothesized model of effects on intent to get COVID-19 vaccine.

Higher COVID-19 disbelief will associate with lower intentions to get a COVID-19 vaccine.

Terror management theoretically copes to impending death, made salient by mortality media coverage (Burke, Martens, & Faucher, 2010). As self-protectionism, this psychological mindset can be held by individuals carrying right or wrong beliefs (Strachan, Pyszczynski, Greenberg, & Solomon, 2001, p. 116). These beliefs can alter perceptions of a threat or finality of someone's lifespan (e.g. spirits live after death) (Greenberg et al., 1986). COVID-19 coverage and perceived threat was inescapable during the initial outbreak (Haroon & Rizvi, 2020). Wall-to-wall news coverage, constant social media posts, and COVID-19 talk was a part of life (Kousha & Thelwall, 2020). Some people stockpiled goods to cope (Fischer, Twardawski, Steindorf, & Thielmann, 2021). Another way to cope with this threat was to believe it was not true (Imhoff & Lamberty, 2020). Many people bought into the false narrative it was not real (Salvi et al., 2021). Disbelief in COVID-19 acted as one method to manage fears (I. Ali, 2020). If COVID-19 was believed not real, it could not hurt someone as portrayed by news outlets.

Higher COVID-19 disbelief will associate with lower fear of COVID-19.

If COVID-19 was believed as not real, why fear something that doesn't exist? With the constant barrage of COVID-19 media conveying danger, ignorance can be bliss. Whereas, those higher on neuroticism worried more about the COVID-19 pandemic and experienced greater negative emotions (Kroencke, Geukes, Utesch, Kuper, & Back, 2020). Believing the pandemic was not real provided false comfort. The alternative meant experiencing fear. This negative emotion can conjure thoughts of pain and death. Terror management theory expresses humans seek explanation and behaviors to cope with fear (Pyszczynski, Greenberg, & Solomon, 1999). Humans can cope with fear through denial and suppression of thoughts and emotions (Plusnin & Pepping, 2015; Yum & Schenck-Hamlin, 2005). Humans can also behave in accordance to their emotions. Vaccination provided an action to allay COVID-19 fears. The greater the fear, the more likely an action will be taken.

Higher fear of COVID-19 will associate with higher intentions to get a COVID-19 vaccine.

Religiosity and COVID-19 response

In times of crisis, religion has provided a positive source of solace and community for many (Jonas & Fischer, 2006). Religious practices have helped individuals cope with uncertainty and perceived threats like COVID-19 by placing their fate with divinity (Jaja, Anyanwu, & Iwu Jaja, 2020). With in-person mass gathering restrictions during the COVID-19 pandemic, online religious services connected churches and their members (Bentzen, 2020). However, many U.S. churches reverberated the false rhetoric that COVID-19 was a hoax (Bentzen, 2020; Calvillo et al., 2020). These churches adamantly refused to comply with mandated restrictions and encouraged in-person congregations (Hill, Gonzalez, & Burdette, 2020). Many of these churches claimed prayer and faith protected against infection (Carbone et al., 2020; Dutta, Acharya, Shukla, & Acharya, 2020). Unfortunately, no reputable scientific studies and health organizations supported these claims (Khalid et al., 2020). Many pastors that held in-person sermons caught COVID-19 and some died (Hill et al., 2020).

Further, religion in the U.S. (e.g. Christianity and Catholicism) teaches actions have consequences and humans innately sin (Curry-Roper, 1990; McMinn, Ruiz, Marx, Wright, & Gilbert, 2006). Sermons often emphasize the dangers of temptations, wrongdoers, and the unknown. With an approaching judgment day, church members are compelled to repent and seek forgiveness (Allais, 2008). Such messaging predisposes members to respond to fear (Ragsdale & Durham, 1986). Assuaging fears tied to death are part of religious practice such as prayer and confessionals (Gao et al., 2017). While COVID-19 disbelief among churchgoers was a part of their perceptions of the pandemic, public health emphasis on COVID-19 dangers was hypothesized to resonate with greater religiosity. Said another way, higher COVID-19 disbelief will associate with higher religiosity; and fear of COVID-19 will increase preventive actions compared to less religious COVID-19 disbelievers. Actions, like vaccination, help allay fears authoritative health officials emphasized which covers them on top of their spiritual practices.

Religiosity will moderate the relationship between COVID-19 disbelief and intentions to get a COVID-19 vaccine. Higher levels of religiosity and COVID-19 disbelief will dampen the lowered intentions to get a COVID-19 vaccine among disbelievers.

Religiosity will moderate the relationship between COVID-19 disbelief and fear of COVID-19. Higher levels of religiosity and COVID-19 disbelief will heighten fears of COVID-19; thereby, increasing intentions to get a COVID-19 vaccine compared to those less religious.

Method

The researchers developed the CDS to measure lack of belief of COVID-19 as a novel life-threatening disease. The researchers reviewed statements about COVID-19 disbelief commonly stated online and in society (Center for Disease Control and Prevention, 2020; Sahoo, Padhy, Ipsita, Mehra, & Grover, 2020). These statements reflected perceptions of the disease as not a real novel threat to humans. Careful wording was designed to capture one idea per statement (Hinkin, 1995). Simple and matching language reflected COVID-19 disbelief statements permeated online to not bias results (e.g. “COVID-19 is not real”) (Clark & Watson, 1995; Malhotra, 2006). A list of 21 statements was refined to 18 statements. The researchers conducted two studies with online participants to establish validity and reliability of the measure. Analysis of the proposed CDS with a variety of measures evaluated discriminant and convergent validity. Large national U.S. samples completed the surveys with the initial 18 items. Participants rated COVID-19 disbelief items on a 7-point Likert like scale from (1 – Strongly disagree to 7 – Strongly agree). Participants continued in the survey to complete intent to vaccinate, fear of COVID-19, and religiosity sections. Thereafter, participants completed an assortment of variables to establish discriminant validity (e.g. mistrust and adventurous). Presentation of each item within each variable section was randomly displayed to counterbalance. Demographics was collected at the end of the survey.

Participants

Study 1 participants

The researchers sampled 728 participants nationally from the U.S. Twenty-one did not complete the survey and unsuccessfully completed attention checks (N = 707). Males composed 39% of respondents and ranged between 19 and 83 years old (M = 41.75, SD = 13.14). The median household income was between $50,000 and $59,000. An associate college degree or higher was held by 70% of participants.

Study 2 participants

The researchers sampled 1014 participants nationally from the U.S. Eighteen did not complete the survey and unsuccessfully completed attention checks (N = 996). Males composed 41% of respondents and ranged between 19 and 88 years old (M = 41.68, SD = 13.28). The median household income was between $50,000 and $59,000. An associate college degree or higher was held by 72% of participants.

Measure refinement procedures

Item reduction

Short measures compared to long measures demonstrated similar reliability for a variety of validated measures (Donnellan, Oswald, Baird, & Lucas, 2006; Schmidt, Le, & Ilies, 2003). Designing a short economical CDS was intended to increase utility while reducing potential complications with lengthy surveys. Lengthy surveys create mental fatigue, relating to lower rates of completion and less accurate results (Ward, Meade, Allred, Pappalardo, & Stoughton, 2017). Thereby, the initial list was reduced. Evaluation of the 18 initial items for inter-item correlations was adequate for 11 items across the two studies with 1703 total participants. Three items had low inter-item correlations (|r|s < 0.30) (Tabachnick, Fidell, & Ullman, 2007). Items with scores lower than this threshold were removed from the list following guidelines for item reduction in scale development (Worthington & Whittaker, 2006). For example, despite the popular misconception that “The COVID-19 pandemic is something the media created”, this item was low on the inter-item correlation threshold. Items displaying values below 0.30 were removed at this time using objective and statistical results to refine the list. Kaiser-Meyer-Olkin (KMO) results exhibited suitable sampling adequacy (see Table 1 ). Bartlett's test of sphericity indicated the remaining 8 items appropriate for factor analysis.

Table 1

Sampling adequacy statistics.

	Study 1	Study 2
Kaiser-Meyer-Olkin measure of sampling adequacy	0.931	0.935
Bartlett's test of sphericity	χ2(28) = 5047.517⁎⁎⁎	χ2(28) = 6939.947⁎⁎⁎

p < .001.

Factor loadings

Evaluation of the scree plot and parallel analyses exhibited one factor to be appropriate for the scale. The first item explained most of the common variance (70.99%). Direct oblimin rotated analysis with two principle component factor loadings generated scores underneath 0.20 for the second factor. Forced two factor analyses generated insufficient scores (Dunn, Seaker, & Waller, 1994; Gibbons, Clark, Cavanaugh, & Davis, 1985). Therefore, the new measure was assessed satisfactory as one factor.

Researchers evinced factor loading scores suffice a 0.40 to 0.70 threshold in exploratory analysis (Hair Jr Joseph, Black William, Babin Barry, & Anderson Rolph, 2010, p. 125; Hulland, 1999; Peterson, 2000). CDS items demonstrated factor loading scores above the 0.70 stricter threshold (Chyung, Winiecki, Hunt, & Sevier, 2017). Table 2 presents the factor loadings of the 8-item CDS. Additional analysis was conducted using the 8-item CDS.

Table 2

Item-factor loadings and item-level descriptive statistics for COVID-19 disbelief scale (CDS).

Item		Study 1 (N = 707)			Study 2 (N = 996)
		Alpha = 0.939			Alpha = 0.926
		M	(SD)	Factor loading	M	(SD)	Factor loading
1)	COVID-19 is a hoax	2.39	(1.85)	0.901	2.30	(1.85)	0.893
2)	COVID-19 is not real	3.78	(1.81)	0.764	3.66	(1.87)	0.744
3)	If you do not believe COVID-19 exists you will not contract it	2.22	(1.79)	0.863	2.24	(1.83)	0.870
4)	COVID-19 is a whole bunch of political nonsense	2.89	(2.05)	0.876	2.75	(2.00)	0.882
5)	COVID-19 affects virtually nobody	2.34	(1.79)	0.883	2.32	(1.81)	0.876
6)	People claiming to have COVID-19 do it for popularity	2.55	(1.82)	0.861	2.55	(1.85)	0.881
7)	Testing is the reason why there are more cases of COVID-19	3.51	(2.12)	0.762	3.41	(2.07)	0.756
8)	Catching COVID-19 is just like catching the flu	3.41	(1.97)	0.813	3.31	(2.00)	0.829

Results

Convergent and discriminant validity

Correlations with various measures was performed to discern convergent and discriminant validity of the CDS (Maloney, Grawitch, & Barber, 2012; Mathieu & Farr, 1991). Table 3 depicts strength of relatedness and differentiation of variables with the 8-item CDS (Lucas, Diener, & Suh, 1996). In study 1, fear of COVID-19 (i.e. degree of fear to the novel coronavirus) (alpha = 0.88) (Ahorsu et al., 2020), mistrust (i.e. general suspicion) (alpha = 0.83) (Wright & Simms, 2014), generic conspiracy beliefs (i.e. credence to unproven conspiracies) (alpha = 0.95) (Brotherton, French, & Pickering, 2013), and social networking (i.e. social media interest) (alpha = 0.69) (Goldberg et al., 2006) variables demonstrated significant Pearson correlations with the CDS. Higher CDS scores correlated with higher fear of COVID-19 (r = 0.317, p < .001), religiosity (r = 0.330, p < .001), mistrust (r = 0.371, p < .001), generic conspiracy beliefs (r = 0.293, p < .001), social networking (r = 0.155, p < .001). Meanwhile, higher CDS scores correlated with lower COVID-19 vaccination intent (r = −0.145, p < .001).

Table 3

Correlations of 8-item COVID-19 Disbelief Scale (CDS) with variables and demographics.

	Study 1				Study 2
Variables	M	(SD)	r		M	(SD)	r
Fear of COVID-19	3.47	(1.72)	.317	⁎⁎⁎	3.47	(1.63)	.346	⁎⁎⁎
Religiosity	4.46	(1.91)	.330	⁎⁎⁎	4.47	(1.94)	.357	⁎⁎⁎
Intent to get COVID-19 vaccine	4.81	(2.00)	−.145	⁎⁎⁎	5.02	(1.96)	−.134	⁎⁎⁎
Mistrust	3.66	(1.18)	.371	⁎⁎⁎	–	–	–
Generic conspiracy beliefs	2.91	(0.98)	.293	⁎⁎⁎	–	–	–
Social networking	4.87	(1.38)	.155	⁎⁎⁎	–	–	–
Perceived vulnerability to disease	–	–	–		4.30	(0.81)	−.183	⁎⁎⁎
Vaccine hesitancy	–	–	–		2.98	(1.17)	.363	⁎⁎⁎
Compassion	–	–	–		5.32	(1.15)	−.566	⁎⁎⁎
Adventurousness	–	–	–		4.37	(0.96)	−.262	⁎⁎⁎
Unusual beliefs	–	–	–		2.37	(1.80)	.770	⁎⁎⁎
Patience	–	–	–		4.64	(1.24)	−.233	⁎⁎⁎
Emotion-based decision-making	–	–	–		3.76	(0.81)	.240	⁎⁎⁎
Demographics
Age	41.75	(13.14)	−.076	⁎	41.70	(13.28)	−.150	⁎⁎⁎
Household income	6.09	(3.00)	−.091	⁎	6.03	(2.91)	−.131	⁎⁎⁎
College degree	0.70	(0.46)	.149	⁎⁎⁎	0.72	(0.45)	.143	⁎⁎⁎

College degree was dummy coded 1 with an associate degree or higher and 0 without.

p < .05.

p < .001.

In study 2, vaccine hesitancy (i.e. reluctance to inoculate) (alpha = 0.78) (Shapiro et al., 2018), perceived vulnerability to disease (i.e. estimated harm of illnesses) (alpha = 0.82) (Duncan, Schaller, & Park, 2009), compassion (i.e. concern for suffering) (alpha = 0.84) (Goldberg, 1992), adventurousness (i.e. inclination to new experiences) (alpha = 0.77) (Johnson, 2014), unusual beliefs (i.e. possess magical abilities) (alpha = 0.83) (Wright & Simms, 2014), patience (i.e. capacity to remain calm) (alpha = 0.88) (Ashton & Lee, 2009), and emotion-based decision-making (i.e. proclivity to respond from feelings) (alpha = 0.73) (Tett, Fox, & Wang, 2005) variables demonstrated significant Pearson correlations with the CDS. Higher CDS scores correlated with higher fear of COVID-19 (r = 0.346, p < .001), religiosity (r = 0.357, p < .001), vaccine hesitancy (r = 0.363, p < .001), unusual beliefs (r = 0.770, p < .001), and emotion-based decision-making (r = 0.240, p < .001). Meanwhile, higher CDS scores correlated with lower COVID-19 vaccination intent (r = −0.134, p < .001), perceived vulnerability to disease (r = −0.183, p < .001), compassion (r = −0.566, p < .001), adventurousness (r = −0.262, p < .001), and patience (r = −0.233, p < .001).

Demographic analyses indicated higher CDS scores positively correlated with college degree (study 1: r = 0.149, p < .001; study 2: r = 0.143, p < .001). Meanwhile, higher CDS scores negatively correlated with age (study 1: r = −0.076, p < .05; study 2: r = −0.150, p < .001) and household income (study 1: r = −0.091, p < .05; study 2: r = −0.131, p < .001).

Reliability

The 8-item CDS demonstrated high internal consistency across both studies (study 1: alpha = 0.939; study 2: alpha = 0.926).

Moderated mediation analysis

Measures

Independent variables

The 8-item COVID-19 Disbelief Scale acted as an independent variable in the model. Fear of COVID-19 measured the strength of COVID-19 phobia using 7-items on a 5-point scale (e.g. “I am afraid of losing my life because of coronavirus-19.” 1 – Strongly disagree to 5 – Strongly agree) (Ahorsu et al., 2020). Fear of COVID-19 acted as the mediator in the model.

Moderator

Religiosity was measured averaging two items (“Religion/spirituality was an important part of my upbringing.” and “I currently consider myself to be a member of a religious or spiritual organization.”) on a 7-point scale (Strongly disagree – 1 to Strongly agree – 7). Participants were on average slightly more religious than neutral (study 1: M = 4.46, SD = 1.91; study 2: M = 4.47, SD = 1.94). Brief 1 and 2 item measures demonstrated validity and reliability for constructs they intended to measure (Littman, White, Satia, Bowen, & Kristal, 2006; Wanous & Hudy, 2001). The religiosity items captured a general religious and spiritual affiliation to match the diversity of faiths in the U.S. Other religiosity scales measured degree of faith with one particular religion or centrality to life too narrow for this study (Huber & Huber, 2012; Joseph & DiDuca, 2007).

Dependent variable

Getting a COVID-19 vaccine represents the future to overcome the COVID-19 pandemic when societies safely reach herd immunity (Randolph & Barreiro, 2020). Intent to get a COVID-19 vaccine was measured on a 7-point scale (“I will get the COVID-19 vaccine”, Strongly disagree – 1 to Strongly agree – 7).

Predictive validity results

Table 4, Table 5 depict SPSS PROCESS V3.5 moderated mediation analysis results for studies one and two (Hayes, 2012, Hayes, 2018). Bootstrapped 10,000 resampling procedures were applied to estimations (Hayes, 2018). Statistically significant results were at the 95% confidence interval when the intervals did not include zero. Intent to get a COVID-19 vaccine leaned slightly positive on the 7-point scale (study 1: M = 4.81, SE = 2.00; study 2: M = 5.02, SE = 1.96).

Table 4

Study 1 moderated mediation results of COVID-19 disbelief on intent to get vaccine.

	Outcome
	Fear of COVID-19				Intent to get vaccine
Antecedent	Coeff.	SE	t	p	Coeff.	SE	t	p
COVID-19 disbelief	−0.737	0.125	−5.912	<.0001	−1.094	0.147	−7.429	<.0001
Fear of COVID-19	–	–	–	–	0.380	0.044	8.721	<.0001
Religiosity	−0.309	0.062	−4.974	<.0001	−0.324	0.073	−4.438	<.0001
COVID-19 disbelief × religiosity	0.191	0.023	8.290	<.0001	0.146	0.028	5.245	<.0001
Covariates
Age	−0.013	0.005	−2.828	<.05	0.009	0.005	1.676	.094
Household income	−0.069	0.020	−3.501	<.001	0.043	0.023	1.877	.061
College degree	0.399	0.132	3.042	<.05	0.782	0.153	5.129	<.0001
Model summary	R2 = 0.230				R2 = 0.243
	F(6, 700) = 34.934, p < .0001				F(7, 699) = 32.103, p < .0001

Notes: Fear of COVID-19 was the mediator in the model. College degree was dummy coded 1 with an associate degree or higher and 0 without.

Table 5

Study 2 moderated mediation results of COVID-19 disbelief on intent to get vaccine.

	Outcome
	Fear of COVID-19				Intent to get vaccine
Antecedent	Coeff.	SE	t	p	Coeff.	SE	t	p
COVID-19 disbelief	−0.427	0.105	−4.070	<.001	−0.907	0.124	−7.324	<.0001
Fear of COVID-19	–	–	–	–	0.444	0.037	11.932	<.0001
Religiosity	−0.232	0.050	−4.696	<.0001	−0.363	0.059	−6.197	<.0001
COVID-19 disbelief × religiosity	0.138	0.019	7.184	<.0001	0.122	0.023	5.308	<.0001
Covariates
Age	−0.005	0.004	−1.443	.150	0.014	0.004	3.325	<.001
Household income	−0.066	0.017	−3.949	<.001	0.076	0.020	3.847	<.001
College degree	0.194	0.108	1.793	<.1	0.633	0.127	5.006	<.0001
Model summary	R2 = 0.184				R2 = 0.228
	F(6, 989) = 37.036, p < .0001				F(7, 988) = 41.673, p < .0001

Notes: Fear of COVID-19 was the mediator in the model. College degree was dummy coded 1 with an associate degree or higher and 0 without.

Fig. 2 illustrates results of the mediation analysis. Results of the omnibus moderated mediation test for religiosity and fear of COVID-19 demonstrated a significant indirect effect with COVID-19 disbelief on intent to vaccinate [study 1: effect = 0.073, SE = 0.073 (LLCI 0.051 ULCI 0.097); study 2: effect = 0.061, SE = 0.010 (LLCI 0.042 ULCI 0.081)] (Hayes, 2015, Hayes, 2018). Higher COVID-19 disbelief associated with lower intent to get a COVID-19 vaccine [study 1: b = −1.094, t(699) = −7.429, SE = 0.147, p < .0001 (LLCI −1.384 ULCI −0.805); study 2: b = −0.907, t(988) = −0.732, SE = 0.124, p < .0001 (LLCI −1.150 ULCI −0.664)] (H1). Higher COVID-19 disbelief associated with lower fear of COVID-19 [study 1: b = −0.737, t(700) = −5.912, SE = 0.125, p < .0001 (LLCI −0.982 ULCI −0.492); study 2: b = −0.427, t(989) = −4.070, SE = 0.105, p < .0001 (LLCI −0.633 ULCI −0.221)] (H2). Higher fear of COVID-19 associated with higher intent to vaccinate [study 1: b = 0.380, t(699) = 8.721, SE = 0.044, p < .0001 (LLCI 0.295 ULCI 0.466); study 2: b = 0.444, t(988) = 11.932, SE = 0.037, p < .0001 (LLCI 0.371 ULCI 0.517)] (H3). The researchers controlled for age, household income, and college degree in the model. SPSS AMOS V25 fit analysis was satisfactory based on recommended thresholds (study 1: χ /df = 4.682, p < .001, RMSEA = 0.072, SRMR = 0.070, CFI = 0.710; study 2: χ 2 /df = 5.816, p < .001, RMSEA = 0.072, SRMR = 0.062, CFI = 0.716) (Hu & Bentler, 1999; MacCallum, Browne, & Sugawara, 1996). These results indicated fear of COVID-19 as an indirect motive to vaccinate.

Fig. 2

Study 1: effects on intent to get COVID-19 vaccine.

Note: Models utilized 8-item COVID-19 Disbelief Scale *** = p < .001, **** = p < .0001.

Moderated mediation results

Conditional indirect effects with religiosity and COVID-19 disbelief on fear of COVID-19 and intent to get a vaccine demonstrated statistical significance. Fig. 3, Fig. 4 illustrate these conditional indirect effects for the two studies. SPSS PROCESS bootstrapping with 10,000 resampling was performed at the 95% confidence interval (Hayes, 2015). The confidence intervals for the conditional effects are presented in Table 6 for studies one and two. Moderated mediation analysis with low religiosity and high COVID-19 disbelief dampened intent to get a COVID-19 vaccine (study 1: effect = −0.802, LLCI −0.990 to ULCI −0.613; study 2: effect = −0.662, LLCI −0.822 to ULCI −0.502) (H4). Meanwhile, high religiosity and high COVID-19 disbelief exacerbated fear of COVID-19 (study 1: effect = 0.507, LLCI 0.409 to ULCI 0.606; study 2: effect = −0.151, LLCI −0.288 to ULCI −0.015) (H5). The indirect moderated mediation path increased intent to get a COVID-19 vaccine (study 1: effect = 0.084, LLCI 0.051 to ULCI 0.122; study 2: effect = 0.117, LLCI 0.081 to ULCI 0.157).

Fig. 3

Study 1 Moderated religiosity with COVID-19 disbelief on fear of COVID-19.

Study 1 Moderated religiosity with COVID-19 disbelief on intent to get vaccine.

Fig. 4

Study 2 Moderated religiosity with COVID-19 disbelief on fear of COVID-19.

Study 2 Moderated religiosity with COVID-19 disbelief on intent to get vaccine.

Table 6

Conditional direct and indirect effects with religiosity as moderator.

	Left-leaning	Average	Right-leaning
	Low (−1 SD)	Mean	High (+1 SD)
Study 1
COVID-19 Disbelief → Intent to get COVID-19 Vaccine	−0.802 (LLCI −0.990 ULCI −0.613)	−0.362 (LLCI −0.453 ULCI −0.272)	−0.143 (LLCI −0.264 ULCI −0.022)
COVID-19 Disbelief → Fear of COVID-19	−0.354 (LLCI −0.515 ULCI −0.193)	0.220 (LLCI 0.144 ULCI 0.297)	0.507 (LLCI 0.409 ULCI 0.606)
COVID-19 Disbelief → Fear of COVID-19 → Intent to get COVID-19 Vaccine	−0.135 (LLCI −0.199 ULCI −0.072)	0.084 (LLCI 0.051 ULCI 0.121)	0.193 (LLCI 0.137 ULCI 0.253)
Study 2
COVID-19 Disbelief → Intent to get COVID-19 Vaccine	−0.662 (LLCI −0.822 ULCI −0.502)	−0.295 (LLCI −0.373 ULCI −0.218)	−0.112 (LLCI −0.212 ULCI −0.012)
COVID-19 Disbelief → Fear of COVID-19	−0.151 (LLCI −0.288 ULCI −0.015)	0.263 (LLCI 0.199 ULCI 0.327)	0.470 (LLCI 0.389 ULCI 0.550)
COVID-19 Disbelief → Fear of COVID-19 → Intent to get COVID-19 Vaccine	−0.067 (LLCI −0.126 ULCI −0.005)	0.117 (LLCI 0.081 ULCI 0.157)	0.208 (LLCI 0.156 ULCI 0.265)

Discussion

General discussion

Results demonstrated the 8-item CDS to have robust internal consistency, convergent validity, and discriminant validity. COVID-19 disbelief positively correlated with generic conspiracy beliefs, unusual beliefs, vaccine hesitancy, and social networking (i.e. a source of disbelief misinformation). Meanwhile, the CDS negatively correlated with perceived vulnerability to disease and patience. Statistical analyses demonstrated the CDS as a unique one factor measure distinct from a variety of measures. Within the context of terror management theory, people coped with COVID-19 fears by believing the pandemic was not real. Through blissful ignorance, people could discount COVID-19 as not a real threat.

Further, predictive validity results supported hypotheses and utility of the measure. Within the context of COVID-19 disbelief, vaccination for the supposedly fictious disease seems irrelevant. Direct effects showed COVID-19 disbelief to negatively associate with intent to vaccinate. Meanwhile, fear of COVID-19 associated with higher intent to get a COVID-19 vaccine. Vaccination safely increases odds of reaching herd immunity and represents a future with curbed COVID-19 infections. Mediation analysis depicted greater COVID-19 disbeliefs associated with lower fear of COVID-19 and thereby lowers intent to vaccinate. While a negative emotion, fear can act as a driver to participate in preventative behaviors important to public health. Results support the need to dispel disbelief to increase cooperation to vaccinate.

Both religious and nonreligious participants believed COVID-19 was a hoax. Direct effects showed greater religiosity to negatively associate with intent to vaccinate. However, religiosity dampened the strong negative intent to vaccinate among disbelievers. Conditional indirect effects evinced religiosity and COVID-19 disbelief to associate with greater fear of COVID-19. Religiosity associated with greater fear of COVID-19. Those with greater fear of COVID-19 demonstrated greater intent to get a COVID-19 vaccine. These results indicate fear messaging can increase preventative behavior outcomes and particularly those with greater religiosity even among disbelievers. This implies accurately messaging the fears of COVID-19 can still resonate with disbelievers to take part in preventative measures.

Meanwhile, CDS correlations demonstrated a fascinating counterintuitive positive relationship with college degree. Shelter-in-place and social distancing heighten isolation (Castaneda & Saygili, 2020). The pandemic escalated social media usage during the COVID-19 pandemic (Ferrara, 2020). Users sought connections with others and shared a whirlwind of COVID-19 content (Papakyriakopoulos, Serrano, & Hegelich, 2020). In the U.S., political speeches circulated during the 2020 election year with false claims that COVID-19 was a hoax and another common flu strain (Calvillo et al., 2020; Uscinski et al., 2020). Social media posts (like #FilmYourHospital) provided plausible inaccurate evidence COVID-19 cases were made up by the media (Ahmed et al., 2020). While this study found college degree associated with higher COVID-19 disbelief, it can be explained by social media posts casting doubt. Study 1 collected data on social networking (interest) (Goldberg et al., 2006) which positively correlated with greater COVID-19 disbelief. Social networking also positively correlated with college degree (study 1: n = 707, r = 0.077, p = .041). Further research on the relationship between social media usage, college behaviors, and COVID-19 disbelief can reveal insight on acceptance of misinformation. False information can alter perception of eye witnessed stressful events (Christianson, Goodman, & Loftus, 1992). The presence of COVID-19 misinformation can be more difficult to override than the absence of misinformation. The CDS gives researchers a tool to identify COVID-19 disbelievers for such studies.

Implications

Results from this study evinced COVID-19 disbelief acted as a degree of terror management to the pandemic. Disbelievers agreed with the CDS statement “If you do not believe COVID-19 exists you will not contract it”. However, regardless of ethnicity and background people can contract COVID-19 (Aiyewumi & Okeke, 2020; Ali, Asaria, & Stranges, 2020; Bowleg, 2020). Despite likely origination from Wuhan, China, the disease crossed borders spreading throughout Italy and the world without selectively choosing who to infect (Sanfilippo, Bignami, Lorini, & Astuto, 2020). COVID-19 was a real human disease, spread to others without finite discretion. Yet, somehow disbelief was extant worldwide. The feverish pandemic coverage elicited a strong fear response many coped by tuning out. The CDS provides an important tool for researchers to study these individual differences and related behaviors.

Furthermore, widespread COVID-19 disbelief illustrated a societal deficit in basic epidemiological knowledge. CDS results illustrated flawed agreement from disbelievers that “Catching COVID-19 is just like catching the flu”. While COVID-19 mutated from a prior strain of the coronavirus that caused flu-like symptoms, COVID-19 was not the same. COVID-19 was more contagious, life-threatening, and largely untreatable at the onset (Johansson et al., 2021; Klompas, 2020; Menni et al., 2020). Months after the World Health Organization declared COVID-19 a pandemic, scientists continued studying the unique properties of the disease and variants (Ledford, 2021; Mahase, 2021; Van Oosterhout, Hall, Ly, & Tyler, 2021). Disbelief of the COVID-19 pandemic highlights an ongoing problem with public acceptance of scientific knowledge. Despite major advances in science, most of the general population lacks detailed knowledge. Lifesaving medicines, transmission of diseases, and vaccines are guised in a shroud of mystery with the absence of knowledge. Lack of understanding sets up a population to distrust scientists and disbelief to take hold in society. The CDS allows researchers to identify disbelievers to test if education can improve understanding and therefore believe COVID-19's existence. It can also reveal if lack of understanding was a source of disbelief. Lack of basic epidemiology suggests higher susceptibility to disbelief, rather than sift through academically available information online.

The scientific community spent substantial effort reeducating the misinformed public and debunking myths wildly spread on social media (Brennen et al., 2020; Marin, 2020). Preventing sources of misinformation from spreading, reduces wasted resources on baseless claims. Many social media websites, like Twitter, were breeding grounds for COVID-19 conspiracies (Ahmed et al., 2020; Gruzd & Mai, 2020). Hidden behind the guise that more information reveals the truth, there was a profit motive for misinformation. These posts stirred emotions. They boosted engagement with comments and shares. Social media organizations profit from time users spend viewing their ads (Kopf, 2020). Whether reinforced myths about COVID-19 or adorable cat videos; time spent on social media websites results in ad revenue. This scale helps researchers identify disbelievers with the ability to study their social media behaviors. It can help social media organizations evaluate how to reduce the spread of misinformation; beyond flagged refuted claim statements.

Limitations

Memory of diseases depends on preservation of history and education. Overtime, people may forget COVID-19 existed, like the 1918 Spanish Flu, H1N1, and polio (Flecknoe, Charles Wakefield, & Simmons, 2018; Tsoucalas, Kousoulis, & Sgantzos, 2016). As memory fades, COVID-19 disbelief may change and likely increase. The CDS depends on time sensitive societal recollection. For example, many behaved like the COVID-19 pandemic was over early 2021, posting pictures of large social gatherings on social media. However, most of the world was unvaccinated and people continued to contract the disease (Center for Disease Control and Prevention, 2020; Johns Hopkins University & Medicine, 2021). Studies using the CDS in the near future can provide insight on how to safely reopen society. The CDS can also reveal how to address challenges to the next pandemic. We should expect a flurry of misinformation, conspiracies, and vaccine resistance given not much changed to address the root of the problem. In fact, given its profitability through online views, health officials should expect more of the same (Arora & Grey, 2020; Basch et al., 2020). While vaccines have eradicated many deleterious diseases from infecting most of the general population, it also created a false sense of security and false premise of nonexistence. With COVID-19 disbelief and variants scientists expect prolonged circulation of the disease (Grubaugh, Hanage, & Rasmussen, 2020).

Future research

With the CDS, researchers can study how far individuals will act on their disbelief and how to direct fears toward positive behaviors. For example, U.K. health officials effectively advertised, “Keep Calm and Remember Hands, Face, Space” to redirect fear toward preventative behaviors (Ashford Borough Council, 2021). Changes in an environment can trigger physiological responses (Champion & Dekker, 2011; McQuiggan, Lee, & Lester, 2006). The COVID-19 pandemic caused major societal changes resulting in many losing their jobs, loved ones, and normal way of life (Arora & Grey, 2020). Many tried to cope with the changes by behaving with a semblance of normalcy (i.e. as if the pandemic never existed). Disbelievers performed humiliating public acts to try to convince others it was not real. Such behaviors included shouting at healthcare workers (Ahmed et al., 2020; Gruzd & Mai, 2020), spitting on strangers (Jeung, 2020; Lashua, Johnson, & Parry, 2020), and expressing xenophobia at retail stores (Wang, 2020). These individuals not only believed falsehoods but acted out with negative behaviors. This poses public safety concerns. Researchers can investigate redirection of emotions and advertising differences.

Meanwhile, investigation of social media behaviors and the spread of COVID-19 transmission myths can help inform social media policies. Social media algorithms present content to users based on prior behaviors, searches, group affiliations, and other factors (Spohr, 2017). Often users become part of an information bubble that reinforces their beliefs (Marin, 2020). Counter evidence can dissuade users to spend time on social media because it presents information challenging their beliefs. For example, a basketball fan may dislike seeing footage of a rival team dunking and winning games. If users are in a COVID-19 misinformation bubble, they will likely see content that supports false narratives (Cinelli et al., 2020).

Moreover, elected leaders and businesses pushed to reopen safely to serve the public. A society reopened generates business and revenue. Balancing closures and enacting safety policies was a delicate necessity to reduce COVID-19 infections. Despite public opposition, evidence-based knowledge drove COVID-19 response policies (Brennan, 2020). Understanding when disbelievers will cooperate, can help health experts design and message policies. For example, disbelievers discounted the potential impact of COVID-19 on their health. Disbelievers protested by defying shelter-in-place/lockdown orders and ignoring mask mandates (Brennan, 2020). However, free mask dispensaries at grocery stores could address convenience relevant in the moment if denied access to purchase goods. The end goal for public health safety remains evidence-based practices that reduced contagion whether if they believe or not. Another person vaccinated, means someone less likely to die from COVID-19 and someone less likely to asymptomatically spread the disease (Lipsitch & Dean, 2020). Further research with the CDS can investigate what disbelievers prioritize (e.g. open stores and normalcy). “Vaccinate so COVID-19 does not exist”, may be a message believers and disbelievers can both support.

Conclusion

The 8-item CDS provides a succinct tool for researchers to study behaviors associated with disbelief of the COVID-19 pandemic. Display of predictive validity results on intent to vaccine demonstrated utility across two studies. Further use of the scale can reveal what to expect as society enters the new norm and likely behaviors in the next pandemic. The predictive model validated utility of the CDS. The CDS provides a tool for researchers to study related behaviors and safe reopening of society.

CRediT authorship contribution statement

Stephen Bok: conceptualization, software data curation, data analysis, writing- original draft, supervision

Daniel Martin: conceptualization, methodology, validation, writing- revisions and added sections

Maria Lee: conceptualization, visualization, writing- reviewing, added sections and editing

Data sharing statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.

Declaration of competing interest

The authors whose names are associated with the manuscript “Validation of the COVID-19 Disbelief Scale: Conditional indirect effects of religiosity and COVID-19 fear on intent to vaccinate”, certify having no affiliations with or involvement in any organization or entity with any financial interests. This includes no educational grants; participation in speakers' bureaus; honoraria; employment; memberships; consultancies; equity interests; stock ownerships; patent-licensing arrangements; or expert testimony. Further there are no non-financial interests. This includes no affiliations, knowledge, beliefs, personal relationships, or professional relationships in the subject matter or materials discussed in the manuscript.