Analytic thinking predicts accuracy ratings and willingness to share COVID-19 misinformation in Australia.

The classical account of reasoning posits that analytic thinking weakens belief in COVID-19 misinformation. We tested this account in a demographically representative sample of 742 Australians. Participants completed a performance-based measure of analytic thinking (the Cognitive Reflection Test) and were randomized to groups in which they either rated the perceived accuracy of claims about COVID-19 or indicated whether they would be willing to share these claims. Half of these claims were previously debunked misinformation, and half were statements endorsed by public health agencies. We found that participants with higher analytic thinking levels were less likely to rate COVID-19 misinformation as accurate and were less likely to be willing to share COVID-19 misinformation. These results support the classical account of reasoning for the topic of COVID-19 misinformation and extend it to the Australian context.

Introduction

The World Health Organization (WHO) has warned that the current COVID-19 pandemic has fueled an “infodemic”—the outbreak of misinformation about this disease (WHO, 2020). Misinformation about COVID-19 can inflict serious harm—for instance, through the spreading of false cures or baseless preventative measures (Mian & Khan, 2020; Swire-Thompson & Lazer, 2020; Tangcharoensathien et al., 2020). In response, cognitive and behavioral scientists have begun to study the psychological mechanisms that enable the spread of COVID-19 misinformation and propose interventions to mitigate it (Van Bavel, Baicker, et al., 2020).

Misinformation is any message that is not supported by the best available evidence (Swire-Thompson & Lazer, 2020). While some misinformation is innocuous, a recent review of research on health-related misinformation found that harmful misinformation is abundant, particularly on social media platforms (Wang et al., 2019). This is of considerable concern because health-related misinformation has been linked with poor health decisions in previous pandemics. For example, during the 2014–2016 Ebola outbreak, Americans who believed conspiratorial misinformation about the disease were significantly less willing to seek medical assistance should they develop symptoms (Earnshaw et al., 2019). Exposure to conspiratorial misinformation has also been associated with the deterioration of COVID-19 preventive health behaviors (Allington et al., 2020; Barua et al., 2020). Moreover, the sharing of COVID-19 misinformation on Twitter has been associated with a future increase in case numbers in that geographical area (Singh et al., 2020). It stands to reason that reducing the spread of misinformation could help mitigate harmful health behaviors during the COVID-19 pandemic.

There are multiple causes of belief in and sharing of misinformation, including many cognitive biases (Van Bavel, Harris, et al., 2020). Recent work on cognitive biases has focused on the dual-process framework that distinguishes between two reasoning processes (Evans & Stanovich, 2013; Kahneman, 2012; Pennycook et al., 2015b): “Type 1” or intuitive processes that do not require working memory and that tend to be fast and automatic, and “Type 2” or analytic processes that require working memory and that tend to be slow and deliberative. In this framework, an important function of analytic processes is to monitor intuitive processes for mistakes and intervene when necessary. In the context of misinformation, a natural prediction is that engaging Type 2 processes facilitates the discernment between true and false claims (Pennycook & Rand, 2019a). An individual’s propensity to engage in analytic (rather than intuitive) processes is often measured using the Cognitive Reflection Test (CRT; Frederick, 2005). Consider an item from this measure: “A bat and a ball cost $1.10 in total. The bat costs $1 more than the ball. How much does the ball cost?” An intuitive answer that rapidly comes to mind is 10 cents. However, reflecting upon this question reveals that the correct response is 5 cents. Better performance on the CRT predicts correct responses to many thinking and reasoning tasks (Toplak et al., 2011), lower levels of epistemically suspect beliefs (Pennycook et al., 2015a), and less belief in and sharing of misinformation (Bago et al., 2021; Bronstein et al., 2019; Pennycook & Rand, 2019a, 2019b; Ross et al., 2021; Sindermann et al., 2020).

The classical account of reasoning

According to the classical account of reasoning, people who fail to override their Type 1 responses are more likely to perceive misinformation as true (Pennycook & Rand, 2019b). Consistent with this hypothesis, research indicates that people who rely more on analytic thinking and less on emotions are less likely to believe in misinformation (Martel et al., 2019); people tend to rationalize information that aligns with their attitudes when put under time pressure but are more likely to form objectively correct interpretations when allowed more time for reflection (Bago et al., 2021); and people higher in analytic thinking are less likely to believe political misinformation, even when it aligns with their political ideology (Pennycook & Rand, 2019b; Ross et al., 2021).

Extending this line of research to misinformation about COVID-19, there is evidence that lower levels of analytic thinking (among other individual differences) are associated with believing misinformation about COVID-19. This association has been found in samples from Canada (Pennycook, McPhetres, Bago, et al., 2021), Italy (Salvi et al., 2021), Iran (Sadeghiyeh et al., 2020), Slovakia (Čavojová et al., 2020), Turkey (Alper et al., 2020), the U.S. (Pennycook, McPhetres, Zhang, et al., 2020; Salvi et al., 2021; Stanley et al., 2020), and the UK (Pennycook, McPhetres, Zhang, et al., 2020). While there is evidence that higher levels of analytic thinking predicts being better able to discern between accurate and inaccurate information about COVID-19 in Italy (Salvi et al., 2021) and the United States (Calvillo et al., 2020; Salvi et al., 2021), there is evidence that analytic thinking does not predict the acceptance of accurate COVID-19 information in Slovakia (Čavojová et al., 2020). Better understanding these predictors of beliefs about COVID-19 is important because people who believe misinformation about COVID-19 are less likely to follow public health advice designed to mitigate this disease’s spread (Roozenbeek et al., 2020).

To our knowledge, only one study has investigated the relationship between analytic thinking and the willingness to share COVID-19 misinformation, finding a significant negative relationship with a U.S. sample (Pennycook, McPhetres, Zhang, et al., 2020). More work on this topic is also needed to help with developing interventions to reduce its spread.

Rationale

The relationship between analytic thinking and COVID-19 misinformation beliefs and willingness to share this misinformation has not yet been examined in Australia. Australia provides a useful point of comparison with existing research from other countries because per capita rates of COVID-19 cases and fatalities are relatively very low (WHO, 2021), with less than one-tenth of the fatality rate per capita compared with other countries where the relationship between analytic thinking and COVID-19 misinformation belief had been studied (Roser et al., 2020), with the exception of Slovakia (see Table 1).

Table 1

COVID-19 cases and fatalities as of 2 May 2020 (the date our data collection commenced)

Country	Fatalities per million	Cases per million
Italy	474.8	3,462.2
United Kingdom	413.0	2,750.7
United States	210.3	3,454.0
Canada	120.0	1,534.8
Iran	73.29	1,148.3
Turkey	39.6	1,474.7
Slovakia	4.4	257.7
Australia	3.7	266.6

Nonetheless, due to the global nature of digital communication, Australians have been exposed to substantial amounts of COVID-19 misinformation. A nationally representative survey conducted in April 2020 found that 59% of Australians indicated they had encountered COVID-19 misinformation at least “some of the time” (Park et al., 2020). This is likely to be an underestimate of misinformation exposure because people who agree with COVID-19 misinformation would not regard it as misinformation. Moreover, there is evidence that many Australians believe COVID-19 misinformation (Pickles et al., 2021). A nationally representative survey conducted in May 2020 found that many Australians reported believing at least one example of debunked misinformation about COVID-19, with 39% believing that the virus was engineered by a Chinese laboratory in Wuhan and 12% believing that the 5G network has been used to spread the virus (Essential Research, 2020).

In summary, the Australian context provides an excellent opportunity to investigate misinformation beliefs and the willingness to share misinformation in a country where exposure to COVID-19 misinformation was high, but the relative risk of contracting COVID-19 was low.

Hypotheses

We preregistered six hypotheses relating to the role of analytic thinking in the spread of COVID-19 misinformation. As the classical account of reasoning predicts that people with higher levels of analytic thinking would be better at identifying accurate information, we hypothesized that people with higher levels of analytic thinking would be less likely to perceive COVID-19 misinformation as accurate, would be more likely to perceive information as accurate, and would be better able to discern between the two.

H: Higher analytic thinking predicts lower belief in COVID-19 misinformation.

H: Higher analytic thinking predicts higher belief in COVID-19 information.

H: Higher analytic thinking predicts better discernment between COVID-19 misinformation and information.

While the classical reasoning account does not make explicit predictions about the willingness to share misinformation, it stands to reason that people with higher levels of analytic thinking would be less likely to share misinformation since most people report that accuracy is very important when deciding what information to share (Pennycook et al., 2021).

H: Higher analytic thinking predicts lower willingness to share COVID-19 misinformation.

H: Higher analytic thinking predicts higher willingness to share COVID-19 information.

H: Higher analytic thinking predicts better discernment between the willingness to share COVID-19 misinformation and information.

Methods

In a confirmatory approach, we preregistered our hypotheses and analysis plan using the As Predicted repository #40309.1 Ethics approvals were granted by the University of Kent #202015872211976468, Macquarie University #52020640915322, and the Australian National University #2020/235. Analyses were performed by using R (Version 4.03; R Core Team, 2018) with the psych (Revelle, 2020) and MASS (Venables & Ripley, 2020) packages. The data, analysis code, wording for all questions, and other materials are available on the Open Science Framework (https://osf.io/gcfb7/).

Participants

Australian participants were recruited as part of the International Collaboration on the Social & Moral Psychology of COVID-19 (ICSMP) project (https://icsmp-covid19.netlify.app/index.html) (Van Bavel, Cichocka, et al., 2020). Australian residents who completed the ICSMP survey were randomly allocated either to additional questions regarding COVID-19 statements that are the focus of the present study or to a separate study. The answers to questions from the ICSMP and the additional Australia-only survey were combined to test the present study’s hypotheses. The sample size was based on available financial resources and is comparable to those used in related misinformation research (e.g., Pennycook, McPhetres, Zhang, et al., 2020).

Participants were recruited via Lucid, a professional surveying company that has been assessed as suitable for social science research (Coppock & McClellan, 2019). Data were collected between 2 May and 12 May 2020, 45–55 days following the formal declaration of a human biosecurity emergency in Australia due to the COVID-19 pandemic (Commonwealth of Australia, 2020). Our sample was quota matched for age, gender, and state- or territory-level residence within Australia. No analyses were conducted until the data collection was complete. As per our preregistered analysis plan, we used two attention check questions from the main ICSMP survey. In the first of these, we asked participants to move a slider to the left, providing the value zero on a numerical scale. In the second, we asked participants to type the number “213” as their response. In line with the preregistered plan, any participant whose response to either of these questions was incorrect was removed from the data set before conducting the analysis. In total, 342 participants failed at least one of these attention checks and were removed, resulting in a sample of 742 participants. The median survey completion time for participants was 21.3 minutes. Participants were not forced to answer survey questions. Those participants with partially incomplete observations for any survey item used in a given analysis were removed from that particular analysis, resulting in a small number of differences in the number of observations across the analyses.

Variables

As part of the (ICSMP) project, participants completed a modified version of a three-item Cognitive Reflection Test (CRT) designed to measure the tendency to engage in analytic thinking (Frederick, 2005). The psychometric properties of the original three-item version of the CRT are generally considered to be reliable and valid (Bialek & Pennycook, 2018). New items were developed for the ICSMP because the original CRT has become widely known, and correct responses can be easily found online (Bialek & Pennycook, 2018). The total number of correct answers on this test form the CRT score variable, which ranges from 0 to 3 (M = 0.83, SD = 0.99). See Supplementary Fig. S1 for a histogram of CRT scores. As a test of internal consistency, Cronbach’s alpha is known to be less effective for nonnormally distributed data with low numbers of items (Sijtsma, 2009). As an alternative, we adopted the Greatest Lower Bound test (Jackson & Agunwamba, 1977; Ten Berge & Sočan, 2004) and used the procedure specified by McNeish (2017, pp. 431–432). This test showed the CRT scale to be internally consistent (glb = 0.80).

Participants were presented with ten statements (see Tables 2 and 3) about COVID-19, including five “information” statements that public health agencies had made (e.g., the Australian Department of Health and the WHO) and five “misinformation” statements that had been debunked by signatories of the International Fact-Checking Network’s code of principles (Poynter, 2020). We slightly reworded some of these misinformation statements while maintaining their central factual claim. These statements were presented to participants in a random order. See Supplementary Table S1 for the original statements and their amendments.

Table 2

Perceived accuracy and willingness to share misinformation statements

Misinformation statements	Perceived accuracy (%)	Willingness to share (%)
1. “The coronavirus is not a virus. It’s 5G that’s actually killing people and not a virus. They are trying to get you scared of a fake virus when it’s the 5G towers being built around the world.”	5.6	8.5
2. “The coronavirus pandemic can be dramatically slowed or stopped completely with the immediate widespread use of high doses of vitamin C.”	9.8	12.9
3. “The truth is that the WuXi pharma lab located in Wuhan, China, is where COVID-19 was developed and conveniently broke out.”	39.7	34.3
4. “Boil some orange peels with cayenne pepper in it. Stand over the pot and breathe in the steam so all that mucus can release. Keep blowing your nose too. Mucus is the problem; it is where the virus lives.”	7.7	9.1
5. “They started mass vaccination for COVID-19 in Africa, and the first 7 children who received it died on the spot.”	5.3	8.8

Table 3

Perceived accuracy and willingness to share information statements

Information statements	Perceived accuracy (%)	Willingness to share (%)
1. “COVID-19 presents a more serious risk to people aged 70 or over, people aged 65 and over with chronic medical conditions, and people with a compromised immune system.”	94.4	90.1
2. “If you live in an apartment with a security entrance, don’t allow delivery people to enter the building or use lifts or internal stairways. This minimizes the risk to any older or vulnerable people who share the common areas of the property.”	80.2	66.2
3. “Being able to hold your breath for 10 seconds or more without coughing or feeling discomfort does not mean you are free from COVID-19.”	53.5	37.9
4. “You should clean and disinfect frequently used objects such as mobile phones, keys, wallets, and work passes to stop the coronavirus from spreading.”	89.9	86.0
5. “One way to slow the spread of viruses, such as coronavirus, is social distancing (also called physical distancing). The more space between you and others, the harder it is for the virus to spread.”	94.7	91.5

Participants were randomly allocated to either the accuracy group (n = 378) who were asked about the accuracy of the statements or the sharing group (n = 364) who were asked about their willingness to share the statements. We split participants into two groups because experimental research demonstrates that asking people about accuracy can nudge them toward greater truth discernment in what social media headlines they are willing to share (Fazio, 2020; Pennycook, McPhetres, Zhang, et al., 2020; Pennycook & Rand, 2021; Pennycook et al., 2021; Roozenbeek et al., 2021).

Those participants in the accuracy group were asked to indicate whether each misinformation and information statement was accurate: “You will be presented with claims that have been made about the Coronavirus (COVID-19). To the best of your knowledge, are these claims accurate?” The response options were “yes” or “no.” The number of “yes” responses on the misinformation statements was added to create the perceived misinformation accuracy scale, ranging from 0 to 5 (M = 0.68, SD = 1.00). The scale was internally consistent (glb = 0.92). The “yes” responses to the information statements were tallied into a perceived information accuracy scale, ranging from 0 to 5 (M = 4.13, SD = 1.03). This scale was also internally consistent (glb = 0.87).

Those participants in the sharing group were asked whether they would consider sharing each misinformation and information statement: “Would you consider sharing these claims with your family or friends (such as during a phone call, in a text message, or via social media)?” Unlike some earlier studies in this literature (e.g. Pennycook & Rand, 2019b), this question was worded to enquire about any form of sharing (i.e., not specifically about sharing via social media) because previous research had found that the most common way Australians share information about COVID-19 is through in-person conversations (Park et al., 2020). The response options were “yes” or “no.” The “yes” responses to the misinformation statements were added together to create a willingness to share misinformation scale, ranging from 0 to 5 (M = 0.74, SD = 1.08). This scale was internally consistent (glb = 0.89). The “yes” responses to information statements were added to create a willingness to share information scale, ranging from 0 to 5 (M = 3.72, SD = 1.22). This scale was internally consistent (glb = 0.89).

Histograms of these four variables are available in Supplementary Figs. 2 to 5. We deviated our preregistration in that we used Spearman’s correlations instead of Pearson’s correlations since the data are not normally distributed (Spearman, 1987).

Following the procedure developed by Pennycook and Rand (2019b), we compiled an accuracy discernment variable by subtracting the participants’ misinformation accuracy score from their perceived information accuracy score and standardizing this difference by converting them to a z-score. Likewise, we compiled a sharing discernment scale by subtracting the participants’ misinformation sharing score from their information sharing score and standardizing those values.

Results

Table 2 shows the perceived accuracy ratings and sharing ratings for COVID-19 misinformation statements. A total of 42.9% of participants reported that at least one of these statements was accurate, and 43.9% of participants indicated they would be willing to share at least one of them.

Table 3 shows the perceived accuracy ratings and willingness to share COVID-19 information. A total of 96.2% of participants reported that at least one of these statements was accurate, and 98.4% of participants indicated they would be willing to share at least one of them.

Analysis

Regarding hypothesis H1, as predicted, there was a significant negative association between CRT score and the perceived misinformation accuracy scale, r(378) = −.20, p < .001, d = −0.41. This relationship is shown in Fig. 1a and indicates that participants with higher analytic thinking levels were less likely to perceive misinformation about COVID-19 to be accurate.

Fig. 1

Associations between CRT and perceived accuracy and the willingness to share. Note. Each dot represents an individual participant. The value on the accuracy scale shows how many items (out of five) the participant rated as accurate. The value on the willingness to share scale shows how many items (out of five) the participant indicated they were willing to share.

However, we found no support for H2, as there was no significant association between CRT scores and the perceived information accuracy scale, r(378) = .09, p = .094, d = 0.17 (see Fig. 1c). Supporting H4, we found a significant negative relationship between CRT score and the misinformation willingness to share scale, r(364) = −.19, p < .001, d = −0.38 (see Fig. 1b). However, H5 was not supported as we found no association between CRT score and the information willingness to share scale, (r(364) = −.05, p = .305, d = −0.11 (see Fig. 1d).

Regarding hypothesis H3, we found a significant positive association between CRT scores and accuracy discernment scores, r(378) = .21, p < .001, d = 0.42, which is shown in Fig. 2a. In line with our predictions, this shows that people who had higher levels of analytic thinking could discern the COVID-19 misinformation from the COVID-19 information statements more accurately.

Fig. 2

Associations between CRT score and perceived accuracy discernment and sharing discernment. Note. Each dot represents an individual participant. A higher rating on the perceived accuracy discernment scale indicates the participant was better able to discern between COVID-19 information statements and misinformation statements. A higher rating on the sharing discernment scale indicates the participant was more discerning in their willingness to share COVID-19 misinformation statements compared with COVID-19 information statements; y-axis values are standard deviations from the mean.

However, we found no relationship between CRT scores and sharing discernment scores as predicted by H6, r(364) = .08, p = .127, d = 0.16 (see Fig. 2b). Contrary to our predictions, this means that higher analytic thinking levels did not predict better discernment between the COVID-19 misinformation and information items when it comes to the willingness to share them with other people.2 Correlations between all variables are reported in Supplementary Tables S3 and S4.

Discussion

In this study we sought to establish whether analytic thinking would predict whether Australians believed or were willing to share misinformation about COVID-19, in line with the classical reasoning account of misinformation perception (Pennycook & Rand, 2019b). Our results do support this account, finding a significant negative relationship between analytic thinking and COVID-19 misinformation beliefs and willingness to share this misinformation in Australia.

More broadly, and in line with other research (Essential Research, 2020; Pickles et al., 2021), we find that many Australians rate some debunked COVID-19 misinformation as accurate: 42.9% reported believing at least one of our misinformation items, and 43.9% reported that they were willing to share at least one of them. These findings confirm that Australians have been affected by the “infodemic” declared by the World Health Organization, even though Australia’s COVID-19 infection and fatality rates were very low at the time the study was conducted.

As predicted, participants higher in analytic thinking (measured using the CRT) were significantly less likely to perceive COVID-19 misinformation as accurate compared with participants lower in analytic thinking. These findings are consistent with similar research on beliefs about COVID-19 misinformation in Canada, Italy, Iran, Slovakia, Turkey, the United States, and the UK (Alper et al., 2020; Čavojová et al., 2020; Pennycook, McPhetres, Bago, et al., 2021; Sadeghiyeh et al., 2020; Salvi et al., 2021). We also found that people with higher analytic thinking levels were less willing to share debunked misinformation about COVID-19, which is consistent with research from the United States (Pennycook, McPhetres, Zhang, et al., 2020).

Given that Australia has much lower COVID-19 fatalities per capita than many other countries, our results make a contribution to the classical reasoning literature by showing that the relationship between analytic thinking and COVID-19 misinformation beliefs persists even when the risk level is relatively low. While the classical reasoning account is silent about the willingness to share misinformation, our results finding a negative relationship between analytic thinking and willingness to share COVID-19 misinformation stand to reason because people report that accuracy is an important consideration when deciding what information to share (Pennycook et al., 2021) and people higher in analytic thinking are better able to identify misinformation, and thus, avoid sharing it.

However, our hypothesis that people with higher analytic thinking levels would be more likely to believe objectively accurate information statements about COVID-19 was not supported. This is a somewhat curious finding, but it does align with similar research conducted in Slovakia, where cases per capita and fatalities per capita were also low and where no relationship was found between analytic thinking and perceiving coronavirus facts as true (Čavojová et al., 2020). Our prediction that people with higher analytic thinking levels would be more willing to share COVID-19 information statements was not supported either. A potential explanation for this lack of associations is that cognitive reflection might only be strongly impactful for rejecting false claims (particularly when they are implausible), not for endorsing true claims (particularly when they are plausible). This possibility is consistent with research that found only inconsistent evidence for a relationship between analytic thinking and belief in legitimate news headlines despite strong associations with fake news headlines (Pennycook & Rand, 2019b). Alternatively, the lack of associations with belief in and willingness to share information statements may reflect methodological limitations of our study (see below).

As predicted, we also found that people with higher analytic thinking levels could better discern true COVID-19 information statements from debunked misinformation, in line with research from in Italy (Salvi et al., 2021) and the United States (Calvillo et al., 2020; Salvi et al., 2021). However, contrary to our prediction, we found no evidence that people with higher analytic thinking levels were any better at discernment when it comes to the willingness to share misinformation. It is possible that people do not always consider whether a claim is accurate before deciding to share it (Pennycook, McPhetres, Zhang, et al., 2020).

Taken together, these findings provide support for the classical reasoning account of misinformation perception (Pennycook & Rand, 2019b), consistent with a growing body of evidence accumulated across the world (Bago et al., 2021; Bronstein et al., 2019; Martel et al., 2019; Pennycook, McPhetres, Zhang, et al., 2020; Pennycook & Rand, 2019b).

Limitations

Our study has some limitations. First, we used self-reported measures of beliefs and behavioral intentions, which might not match true beliefs and behaviors. While there is evidence that the self-reported willingness to share misinformation correlates with real-world sharing behavior (Mosleh et al., 2020), it is conceivable that our survey questions have led to either socially desirable responding (Paulhus, 2002) or insincere responding (Levy & Ross, 2021) in reporting their beliefs and intentions, and as such there may be an Intention-Behavior Gap (Sheeran & Webb, 2016). For example, consider misinformation statement three, claiming that the COVID-19 virus was created in a laboratory in China, which received a much higher level of support than the other debunked statements. Some respondents may have provided responses based on racial or geopolitical attitudes rather than a sincere belief in the statement.

Second, the null results regarding information items could reflect ceiling effects. This is because the information items used in our study were generally perceived to be highly accurate, and these high levels could have suppressed variation in perception scores (i.e., a ceiling effect). Further studies might benefit from using information items that are less obviously accurate.

Finally, there is debate about the extent to which the CRT captures the disposition to rely upon analytic thinking as opposed to measuring numeracy or general cognitive ability (Isler et al., 2020; Pennycook & Ross, 2016; Sinayev & Peters, 2015). Future work on the relationship between analytic thinking and misinformation beliefs and sharing intentions would benefit from using additional measures of analytic thinking or controlling for general cognitive ability. We also note that as this study was correlational in design, we cannot make causal inferences about the relationship between analytic thinking and beliefs, or analytic thinking and sharing intentions.

Conclusion

Our results provide insights into the psychological correlates of COVID-19 misinformation beliefs and the willingness to share that misinformation among Australians. Supporting the classical account of misinformation perception, we find that people who are lower in analytic thinking are more likely to perceive debunked COVID-19 misinformation as accurate and are also more willing to share such misinformation.

(DOCX 920 kb)