Does fact-checking habit promote COVID-19 knowledge during the pandemic? Evidence from China.

OBJECTIVES: Promoting health knowledge during a public health crisis is essential. This study aims to examine how fact-checking habit influences COVID-19 knowledge in the COVID-19 infodemic. STUDY
DESIGN: This study uses a cross-sectional survey.
METHODS: During the early outbreak of COVID-19 in China, we conducted an online survey and collected data from 3000 representative Chinese Internet users. The study measured COVID-19 knowledge as a dependent variable, fact-checking habit as an independent variable, and general science knowledge and negative emotion as moderators. Internet use and several demographic factors were used as control variables. Ordinary least squares (OLS) linear regression analysis was conducted to examine the relationship between fact-checking habit and COVID-19 knowledge as a function of science knowledge and negative emotion.
RESULTS: Fact-checking habit was negatively associated with COVID-19 knowledge, and the relationship was moderated by general science knowledge and negative emotion. For those with less science knowledge or higher levels of negative emotion, COVID-19 knowledge was lower with the increase of experience in fact-checking.
CONCLUSIONS: During a pandemic, individuals may not be able to obtain high-quality information, even if they regularly fact-check information, and especially when they lack knowledge about science or are influenced by negative emotion. To promote health knowledge during a public health crisis, basic science literacy must be promoted, and the psychological impact of the crisis on the population must also be considered.

Introduction

Amid the COVID-19 global pandemic, people were eager to know more about the virus, which was seen by the sharp increase in information seeking related to COVID-19. Health knowledge can advance health behaviors; , therefore, improving health knowledge is essential in health promotion. However, during the pandemic, the abundance of both accurate and inaccurate information makes it difficult for people to obtain knowledge about COVID-19. Often, people are overwhelmed by the infodemic and misled by inaccurate information. According to the World Health Organization (WHO), an infodemic is ‘too much information, including false or misleading information, in digital and physical environments during a disease outbreak. It causes confusion and risk-taking behaviours that can harm health’. During the early outbreak of COVID-19 in China, a large amount of inaccurate information about COVID-19 misled people into excessive prevention behaviors. Outside of China, misleading preventive advice (e.g. drinking bleach, which went viral on the Internet) resulted in approximately 5800 people to be admitted to hospital and at least 800 deaths by August 12, 2020.

Given the increasing need for COVID-19 knowledge and the negative impacts of inaccurate information, there is an urgent call for fact-checking to cope with the COVID-19 infodemic. It is expected that individuals can identify inaccurate information and obtain useful knowledge by fact-checking what they read. Against this background, the current study aims to examine how fact-checking practices can influence COVID-19 knowledge.

Research framework

Obtaining knowledge from a massive amount of information requires skill and literacy. Fact-checking is one of the applications of information literacy.11, 12, 13 Fact-checking (also known as ‘verification’ or ‘authentification’ , ) refers to an individual's behaviors of determining whether a piece of information exists or is true (e.g. by using a search engine to search for more related information). Therefore, fact-checking is a process of learning. Ideally, individuals who fact-check what they read are more likely to obtain knowledge instead of inaccurate information, and the knowledge can help people make correct decisions. An increasing number of journalists and scholars are encouraging people to fact-check what they read.16, 17, 18 Eysenbach considered fact-checking as one of the four pillars of infodemic management. However, to date, the relationship between fact-checking and knowledge obtainment has not been empirically examined. Do people who are more experienced in fact-checking obtain more knowledge about COVID-19 than those who are not? The current study aims to answer this question.

When individuals perform fact-checking, they expose themselves to more information. However, not everyone can refine knowledge from what they read efficiently. Whether people can gain new knowledge largely depends on their existing knowledge, which is known as the ‘Matthew effect’ in education. Those who have more reading ability obtain more literacy by reading. , Following this line of reasoning, those who have more existing science knowledge should be more capable of judging the authenticity of COVID-19 information and obtain more knowledge by fact-checking.

The outcome of learning can also be affected by emotion. Studies suggest that students' learning outcomes can be enhanced by positive (e.g. happy, joyful) but not negative (e.g. anxious, fear) emotion in learning. , However, it is important to note that fact-checking under the influence of an infodemic is different from learning in school. People are exposed to both accurate and inaccurate information when they fact-check. Usually, negative emotion alerts individuals and elicits systematic as well as accommodative information processing, which leads them to focus on the actual details of the world. , On the contrary, positive emotion signals a predictable environment and induces assimilative and constructive processing, so the individuals rely more on existing knowledge and heuristic, schematic thinking to perform a task. , As a result, people with positive emotion are more likely to fall for misleading clues when they are exposed to both accurate and inaccurate information. Also, people tend to hold false memories of what they have seen. Therefore, when individuals perform fact-check, negative emotion can help them process the given information more systematically, whereas positive emotion increases the likelihood of being misled by inaccurate information.

Based on the current literature, the present study aims to investigate the impacts of fact-checking habit on the obtainment of COVID-19 knowledge and to examine how the effects vary with different levels of science knowledge and negative emotion. The findings of this study will help understand the determinants of health knowledge during a public health crisis.

Methods

Data collection

The data for this study were collected between 2 March and 23 March 2020 in mainland China. Data collection was outsourced to a commercial survey research company who have 4 million Internet panel members in China. To achieve a representative sample, we used a stratified quota sampling technique to recruit respondents. The quotas for subcategories of gender, age, and education groups were based on the most recent China Internet Network Information Center (CNNIC) report. This sampling method was used to recruit 3000 respondents aged >18 years, with a response rate of 24.6%.

Sample size

We followed Daniel and Cross's formula to calculate the survey sample size. According to the 44th CNNIC report, there are 939,840,000 Internet users in China. To reach a criterion of 3% margin of error and 99% confidence level, the sample size should be 1844. However, given the geographical diversity of the Chinese population, we increased the target sample size to 3000.

Measures

COVID-19 knowledge

This study measured COVID-19 knowledge by examining respondents' trust in six popular false statements about COVID-19 in China. To avoid the examination effect, two of the sentences were reversely stated as true statements. Respondents were asked to indicate whether they believed the statements on a 4-point scale, ranging from 1 ‘definitely false’ to 4 ‘definitely true’. The answers for the false statements were reversely coded. Respondents scored 1 when they thought the statement was ‘definitely true’ or ‘true’ and scored 0 when they thought the statement was ‘definitely false’ or ‘false’. The total score of all six items formed the knowledge index (M = 1.41, SD = 0.35).

Fact-checking

By adapting Edgerly et al.’s measures of fact-checking, we asked the respondents how likely were they to perform the following checking strategies after reading information online: check other major news outlets, ask friends/family members, use search engines, check social media (e.g. Weibo, WeChat) and consult other sources. Respondents reported their answer on a 5-point Likert scale, from 1 ‘very unlikely’ to 5 ‘very likely’. The average of these items formed the fact-checking index (α = 0.88, M = 3.74, SD = 0.87).

Science knowledge

It is important to measure the objective science knowledge of individuals rather than their perceived science knowledge because extremists in science topics tend to overestimate their knowledge. To measure objective science knowledge, we selected 10 of the 15 items from Fernbach et al.’s scales of scientific literacy. Respondents were asked whether the 10 statements were correct or not. We added up the number of correct answers to form the science knowledge index (M = 2.74, SD = 0.38).

Negative emotion

By adapting Yeung and Fung's (2007) measures of emotional responses, participants were asked to rate the levels of ‘sadness’, ‘fear’, ‘anger’ and ‘shock’ in response to COVID-19 on a 5-point Likert scale (1 = not at all, 5 = very intensive). The items were averaged for each respondent as an indicator of negative emotion (α = 0.85, M = 3.47, SD = 1.08).

Control variables

Respondents were also asked to report their age, gender, education, income, region of residence and Internet use frequency. For Internet use, respondents were asked how often they use the desktop and mobile devices to access the Internet, from 1 ‘never’ to 5 ‘always’. The average scores of these two items formed the index of Internet use (α = 0.69, M = 4.24, SD = 0.78).

Analysis

To analyze the data, a series of ordinary least squares (OLS) linear regression analyses were performed with SPSS version 26.0. We first tested a model with demographic and control variables only, as a baseline model. Then, the key independent variable fact-check habit and the two moderators (i.e. science knowledge and negative emotion) were entered into the model sequentially. Finally, we explored the moderation effects by including interaction terms in the regression equations.

Results

Before formal analysis, we checked the sample representativeness of our data. The demographic information is summarized in Table 1 . The distributions of demographic variables (age, gender, education, and income) of the sample are very close to those reported in the 44th CNNIC report, which is also shown in Table 1.

Table 1

Demographic characteristics of respondents (N = 3000).

Variable	Present sample (%)	Sixth China Census data (%)
Age in years
18-29	32.50	25.69
30-39	29.40	20.42
40-49	21.50	21.86
50-59	8.30	15.19
≥60	8.30	16.85
Sex
Male	52.40	51.19
Female	47.60	48.81
Education
Primary school or below	18.00	33.75
Secondary school	38.10	41.70
High school	23.80	15.02
College	10.50	5.52
University or above	9.70	4.01
Income
<6000	7.50	–
6001–10,000	28.80	–
10,001–30,000	49.60	–
30,001–60,000	10.10	–
≥60,001	4.00	–
Region
Rural	39.20	49.73
Urban	60.80	50.27

The descriptive statistics of the main variables were also checked (see Table 2 ). The majority of respondents (70.5%) knew that the statement ‘Drinking alcohol won't reduce coronavirus risk’ was false. However, only 51.8% of respondents knew that the following statement was true: ‘The coronavirus lasts longest on the smooth, non-porous surface; thus, the virus survives shorter on a sweater than the metal surface’. The accuracy rates of science knowledge items ranged from 44.0% to 79.7%. In terms of emotional reactions to the pandemic, more respondents felt shocked (58.4%) than fear (44.1%), sadness (49.6%) or anger (40.4%). In general, the likelihood of fact-checking was reasonable (see Table 2). More than half of the respondents reported that they were likely to fact-check online information by checking major news outlets and using the search engine. About a half would perform fact-check by asking friends or family members and checking social media, such as Weibo and WeChat. We also examined the diversity of fact-checking strategies. The percentages of people who are ‘likely’ or ‘very likely’ to perform one, two, three, or four types of fact-checking strategies when reading online information are 13.2% (n = 242), 17.2% (n = 315), 15.3% (n = 281), and 14.4% (n = 265), respectively. One in five respondents (21.5%, n = 395) reported that they were ‘likely’ or ‘very likely’ to adopt all of the fact-checking strategies; however, 18.5% (n = 338) of respondents reported no intention to use any of the examined fact-checking strategies when reading online information.

Table 2

COVID-19 knowledge, fact-checking habit, science knowledge, and negative emotion.

Variables	Percentage
COVID-19 knowledge (scale 1–4)	Score 3–4, very likely–very likely
Drinking alcohol will not reduce coronavirus risk. (True)	70.5%
Viruses are more virulent in cold and wet weather, thus turning on air-conditioners or heater up to 30° could fight the coronavirus. (False, reverse code)	53.7%
The coronavirus lasts longest on the smooth, non-porous surfaces; thus, the virus survives longer on a sweater than the metal surface. (True)	51.8%
The coronavirus is a bio-weapon developed by the United States. (False, reverse code)	56.0%
Going out with ginger slices in the mouth can prevent the coronavirus. (False, reverse code)	60.7%
The coronavirus is only infecting and killing Asians, but not Caucasians (False, reverse code)	65.6%
Fact-checking habit (scale 1–5)	Score 4–5, likely-very likely
Check other major news outlets.	56.4%
Ask friends/family members.	49.3%
Use search engines.	58.4%
Check social media (e.g. Weibo, WeChat).	49.8%
Consult some other sources.	44.6%
Science knowledge (scale 0–1)	Score 1, correct
Antibiotics kill viruses as well as bacteria. (False, reverse code)	44.0%
The center of the earth is very hot. (True)	77.5%
All radioactivity is manmade. (False, reverse code)	63.8%
The oxygen we breathe comes from plants. (True)	76.7%
All insects have eight legs. (False, reverse code)	68.9%
Men and women normally have the same number of chromosomes. (True)	54.7%
The continents have been moving their location for millions of years and will continue to move. (True)	79.7%
Lasers work by focusing sound waves. (False, reverse code)	37.0%
Electrons are smaller than atoms. (True)	62.4%
All plants and animals have DNA. (True)	69.7%
Negative emotion (scale 1–5)	Score 4–5, agree- strongly agree
Sadness	49.6%
Fear	44.1%
Anger	40.4%
Shock	58.4%

The regression results are shown in Table 3 . Among the control variables, age (b = 0.14, 95% confidence interval [CI] = 0.01–0.02, P < 0.001), education level (b = 0.06, 95% CI = 0.02–0.12, P < 0.01), and income (b = 0.04, 95% CI = 0.00–0.11, P < 0.05) all have positive effects on the obtainment of COVID-19 knowledge. Individuals who use the Internet (b = −0.15, 95% CI = 0.04–0.17, P < 0.001) know more about COVID-19 than those who do not use the Internet. However, fact-checking was negatively related to the obtainment of COVID-19 knowledge (b = −0.36, 95% CI = −0.66–0.53, P < 0.001).

Table 3

Regression analysis: predicting COVID-19 knowledge.

COVID-19 knowledge
	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
Age	.14∗∗∗	.05∗	.02	.02	.02	.02
	[.01,.02]	[.00,.01]	[-.00,.01]	[-.00,.01]	[-.00,.01]	[-.00,.02]
Gender	-.02	-.02	-.03	-.02	-.02	-.02
	[-.17,.04]	[-.16,.03]	[-.16,.02]	[-.14,.04]	[-.14,.03]	[-.15,.02]
Education	.06∗∗	.03	.04	.04∗	.04∗	.03∗
	[.02,.12]	[-.01,.08]	[-.00,.08]	[-.00,.09]	[.00,.09]	[.00,.08]
Income	.04∗	.04∗	.04∗	.04∗	.03∗	.04∗
	[.00,.11]	[.01,.12]	[.00,.10]	[.00,.10]	[.00,.10]	[.01,.10]
Residence	.01	-.00	-.03	-.01	-.01	-.01
	[-.09,.13]	[-.11,.10]	[-.17,.03]	[-.13,.06]	[-.13,.07]	[-.13,.06]
Internet use	.06∗∗	.20∗∗∗	.10∗∗∗	.13∗∗∗	.014∗∗∗	.13∗∗∗
	[.04,.17]	[.30,.44]	[.12,.25]	[.17,.29]	[.18,.31]	[.18,.31]
FCH		-.36∗∗∗	-.29∗∗∗	-.20∗∗∗	-.21∗∗∗	-.22∗∗∗
	–	[-.66,-.53]	[-.53,-.41]	[-.39,-.27]	[-.41,-.28]	[-.43,-.31]
SK			.36∗∗∗	.32∗∗∗	.32∗∗∗	.31∗∗∗
	–	–	[1.24,1.48]	[1.09,1.34]	[1.10,1.35]	[1.05,1.30]
NE				-.23∗∗∗	-.22∗∗∗	-.21∗∗∗
	–	–	–	[-.34,-.25]	[-.34,-.25]	[-.33,-.24]
FCH∗SK					.05∗∗	–
					[.09,.37]
FCH∗NE	–	–	–	–		-.12∗∗∗
						[-.21,-.13]
R2(%)	2.2∗∗∗	12.4∗∗∗	24.0∗∗∗	28.1∗∗∗	28.4∗∗∗	29.5∗∗∗

Note: The table shows standardized coefficient beta, with a 95% confidence interval in brackets.

The interaction terms have been centered.

∗P < 0.05, ∗∗P < 0.01, ∗∗∗P < 0.001.

FCH, fact-checking habit; NE, negative emotion; SK, science knowledge.

The relationship between fact-checking habit and obtainment of COVID-19 knowledge depends on the levels of science knowledge (b = 0.05, 95% CI = −0.34–0.25, P < 0.01). Science knowledge had a direct effect on susceptibility to misinformation (b = 0.36, 95% CI = 1.24–1.48, P < 0.001). The interaction effect is shown in Fig. 1 . For people with high science knowledge, their experience of fact-checking barely changed their knowledge about COVID-19. However, for people with less science knowledge, people who fact-check frequently obtained less COVID-19 knowledge.

Fig. 1

Interaction effect of fact-checking habit and science knowledge on COVID-19 knowledge.

Negative emotion also moderated the impact of fact-checking habit on COVID-19 knowledge (b = −0.12, 95% CI = −0.21–0.13, P < 0.001). Results also showed that negative emotion toward the pandemic had a negative effect on COVID-19 knowledge (b = −0.23, 95% CI = −0.33–0.24, P < 0.001). The interaction effect is shown in Fig. 2 . However, contrary to our hypothesis, for people with more negative emotion, knowledge about COVID-19 dropped significantly if they fact-check more. Among those who had a high level of negative emotion, COVID-19 knowledge decreased slightly as their fact-checking experience increased.

Fig. 2

Interaction effect of fact-checking habit and negative emotion on COVID-19 knowledge.

Discussion

Previous studies suggest that if individuals perform fact-checking on the information they consume, they are more likely to obtain knowledge. Our findings suggest the opposite in the context of the COVID-19 pandemic: people may not be able to obtain high-quality information, even if they have a good fact-checking habit, and especially when they lack knowledge about science or were influenced by negative emotion.

First, fact-checking habit had a negative relationship with COVID-19 knowledge. There are several possible explanations for this result. First, the new coronavirus brings a new health crisis to the world. When the virus swept through China, lots of unverified information about the virus appeared on the Internet, while scientists and health departments were still striving to understand the virus. Therefore, when facts are missing, the habit of fact-checking does not necessarily help to identify the useful knowledge. Second, people tend to fact-check what they believe for confirmation. Meanwhile, it is more difficult to counter wrongful beliefs when individuals have accepted them, which is called the continued or persistent effect of inaccurate information. , According to the cognitive bias theory, people prefer messages that are in line with their prior beliefs and resist the opposite. Thus, if people have accepted a wrongful idea, pre-existing bias will make people less likely to obtain the facts about COVID-19, even if they fact-check what they read from the Internet. Third, even if people fact-check without the influence of pre-existing beliefs, the information environment they are facing may not allow them to encounter diverse opinions. The Internet, especially social media, makes it easier for individuals to block voices of opposing opinions and selectively expose themselves to information that supports their views. , Fact-checking in a homogenous information environment or social network keeps people away from information they need.

This study further examined whether the effects of fact-checking habit on knowledge obtainment depend on existing science knowledge and negative emotion. The result is largely consistent with the Matthew effect of literacy acquisition. The data showed that people with less science knowledge gain less knowledge about COVID-19 as their experience of fact-checking increases. Among people with high science knowledge, COVID-19 knowledge barely changed with fact-checking habit. The finding highlights the importance of early science education in increasing the likelihood of knowledge intake. However, many reports demonstrated the difficulties in promoting science education. Although both developed and developing countries see the necessity of science education, there are so many challenges and problems, such as declining interest in science studies, inequality in teaching recourses and insufficient family involvement. ,

Among people who hold more negative emotion toward the pandemic, their fact-checking habit decreases the likelihood of knowledge obtainment. The finding implies that mental health is important in knowledge obtainment during a health crisis. Recent studies show that the problem of negative emotion during the pandemic is commonly seen. A study in China showed that young people suffer more from anxiety disorders and depressive symptoms than older age groups, and healthcare workers have the highest rate of poor sleep quality. In addition, a study in the United States found depression skyrocketed during the COVID-19 pandemic among adults. Therefore, healthcare departments and organizations should pay more attention to mental health issues within the population during the pandemic given its potential to influence knowledge gain.

It is important to point out the limitations of this study. First, the fact-checking measurement scale adopted by this study did not capture all aspects of fact-checking behavior and excluded some unmeasured fact-checking strategies that people have been shown to use. Therefore, developing a comprehensive scale for fact-check behavior could benefit similar research endeavors in the future. Second, because this study was conducted during the COVID-19 pandemic, and people were not very likely to hold a positive emotion toward the pandemic, it only examined the effects of negative emotions. Future studies can examine the effects of positive emotions on fact-checking and knowledge gain. Third, future studies should further explore the mechanisms behind information verification and the obtainment of science knowledge. Information sources and trust in these sources might also play an important role. ,

In conclusion, this study has two main findings. First, we found a negative relationship between fact-checking habit and the levels of COVID-19 knowledge during the pandemic. This result indicates that encouraging fact-checking behaviors might not be an effective solution to fighting an infodemic. Governments, the media, and non-governmental organizations (NGOs) should directly engage in promoting scientific health knowledge instead of encouraging people to fact-check in a low-quality information environment. Second, this study sheds light on the practice of knowledge promotion by suggesting that a lack of general science knowledge and increased negative emotion can lead to less knowledge obtainment, even if they are active in information fact-checking. To promote health knowledge during a public health crisis, basic science literacy must be promoted and the psychological impact of the crisis on the population must also be considered. In the long run, science literacy education is important to alleviate the inequality in health knowledge obtainment.

Author statements

Ethical approval

This study was approved by the Institutional Review Board of Fudan University.

Funding

This study was supported by the Science Popularization and Risk Communication of Transgenic Biotechnologies project (Grant ID: 2016ZX08015002-005), the National Science and Technology Major Project of the Ministry of Science and Technology of China.

Competing interests

None declared.