Pre-Service Teachers' Analysis of Claims About COVID-19 in an Online Course.

Along with the COVID-19 outbreak, which has been a global threat for public health, the unconfirmed information about the pandemic in circulation has become another threat. Hence, it has become important to improve public understanding of science with a focus on explaining the nature of uncertainty in science and its impacts. The goal of the present study was to explore pre-service teachers' analysis of claims related to the COVID-19 pandemic throughout an 8-week online implementation of a pre-service teachers' analysis task, focus group interviews, and instructor's feedback to this analysis in a course focusing on critical and analytical thinking. In order to achieve this purpose, the researchers used the claims that contain fallacies, conspiracy theories, and scientific arguments related to the COVID-19 pandemic as an assessment tool. The researchers developed and used a rubric consisting of the high, moderate, and low levels of analysis in three different categories including evaluation of claims, demarcation of fallacies and conspiracy theories from scientific arguments, and judgment of the credibility of sources. The findings indicate that the participants analyzed the claims rarely at a high level before the focus group interviews. However, after the focus group interviews, almost every participant's analysis scores of evaluation, demarcation, and judgment increased. The results also revealed their commitment to various fallacies and conspiracy theories while arguing the claims. Concluding remarks are made for the further implications of teaching critical evaluation of claims based on evidence. © Springer Nature B.V. 2020.

Introduction

We are living in a post-truth era, in which misinformation and disinformation are circulated very rapidly among the public. The spread of misinformation and disinformation has harmful effects on the public. A harmful effect of post-truth is the public loss of confidence in fact-based science (Kienhues et al. 2020). In the post-truth epoch, the disconnection between the scientific community and the public has prepared a suitable environment for groups that lead the spread of information pollution and fake news about the pandemic. More than ever, misinformation and disinformation are at the highest threat level for human beings (Pennycook et al. 2020). As the importance of social media increases day by day, it is significant to know the features of the information and news that spread in this realm and to take various precautions accordingly. The information that people get strongly affects their behavior (Cinelli et al. 2020) as well as their decisions. However, people tend to get information from sources without evaluating its trustworthiness. Therefore, science educators should find ways to foster students’ ability to make informed decisions by considering the trustworthiness of the sources of information.

Sadler (2009) argued that the individuals’ decisions are dependent upon their personal experiences, situations appealing to their emotions and morality. There are a number of variables that affect and manipulate individuals’ emotions and morality. These variables play a decisive role in decision-making on uncertain events. People often make their decisions on uncertain events such as the result of an election or whether someone is guilty or not, and cognitive bias is one of the variables that affect these decisions that originated from reliance on judgmental heuristics. Reasoning tends to rely on heuristic by consistently trying to find reasons for our opinions and against those who oppose to support our side (Mercier and Sperber 2017). The reliance on heuristic may lead to cognitive biases, and these biases can be observed not only among laymen but also among experienced researchers when they think intuitively (Tversky and Kahneman 1974). The sources of information may be helpful to trigger or avoid these biases. From this point of view, digital media have become a source of misinformation and disinformation despite its benefits of providing opportunities to laypeople engaging in debates about science and health issues. The spread of misinformation and disinformation ignites ill-informed and dangerous public decisions. It necessitates equipping social media users with the ability and tools to identify and share high-quality information and overcome misinformation and disinformation about COVID-19 (Nguyen and Catalan 2020).

The lack of the ability to identify and share high-quality information among the public emerged again with the COVID-19 outbreak. Misinformation and disinformation about COVID-19 spread among social media in a very short period of time due to the novelty of this virus and its rapid infection throughout the globe. In some countries, education was suspended, while in other countries curfews were enforced because of the outbreak. In such turmoil, various confirmed and unconfirmed information about COVID-19 began to spread as the coronavirus itself on social media streams became the primary news source of people with the digital age. This abundance of information, which makes it hard for people to access reliable information and the trustworthy resources they need, was described as infodemic (WHO 2020a). Central to this issue, creating scientifically literate citizens is crucial to deal with this infodemic. Scientific literacy requires all citizens to make informed decisions about health by enabling them to understand the world around them, critically analyze the suggested claims about scientific issues, identify questions, draw conclusions based on evidence, and make informed decisions about environmental and health issues (Rennie et al. 2001).

There are two visions of scientific literacy, vision I and vision II (Roberts 2007). Vision I focuses on products and processes of science while vision II emphasizes on situations, which students are likely to encounter as a citizen, with a scientific component. On the one hand, the categories including the concepts in science, nature of science, and ethics that control scientists’ work are based on vision I. On the other hand, interrelationships of science and society, interrelationships of science and humanities, and the differences between science and technology are related to vision II. The most problematic issue in adopting vision I is narrowing students’ experience with the scope of science as a human endeavor (Roberts 2007). However, in the post-truth era, the individuals are exposed to various information about controversial issues, as in the case of the COVID-19 pandemic. Therefore, it is necessary to adopt vision II of scientific literacy in examining learners’ understanding of the COVID-19 pandemic in this post-truth era.

Scientific literacy, in its fundamental sense, requires comprehending, interpreting, analyzing, and criticizing texts rather than scientific knowledgeability by acquiring isolated pieces of scientific information. Recognizing science as constituted by text and the resources that text makes available facilitates understanding the substantive science content and the epistemology of science (Norris and Phillips 2003). Previously mentioned, the researchers of this study adopted vision II of scientific literacy, as the ultimate goal of examining individuals’ analysis of claims is to create citizens who are able to evaluate the quality of scientific information based on its source and the methods it generates (Roberts 2007). From this point of view, it is important to help individuals critically evaluate the texts they read and detect information disorders in these texts.

Scientific literacy is also the ability to interpret and construct science-based ideas in popular media and other everyday contexts accurately and effectively (Cavagnetto 2010). However, evaluating claims and sources of information has become more difficult in the post-truth era. Therefore, the post-truth era necessitates a critical evaluation of online information about scientific information (Sinatra and Lombardi 2020). From this perspective, scientific literacy requires not only the critical evaluation of claims, evidence, and sources of information but also the ability to resolve uncertainty in science.

Uncertainty is a cognitive feeling which refers to individuals’ subjective experience of doubt or lack of knowledge and being unsure of a particular phenomenon (Jordan and McDaniel Jr. 2014). Uncertainty is inherent in science. Consequently, uncertainties affect public decisions about health issues. For example, the uncertainties about the side effects of vaccines and vaccination may have an impact on parents’ decision to vaccinate their children. Therefore, improving public understanding of science requires a focus on explaining the nature of uncertainty in science and its impacts (Kampourakis and McCain 2019). In the COVID-19 case, uncertainty is inevitable and essential since all citizens encounter uncertainty that includes different information containing misinformation and disinformation, have different opinions based on different information, and make different decisions based on the same information. As deep uncertainty increases, limited scientific knowledge and understanding of the novel coronavirus fails to meet the public’s need for certain information and advice, and this situation lays the ground for unbounded news, unverified claims, and fabricated stories (Nguyen and Catalan 2020). Kirch (2010) observed a similar pattern of identifying and resolving uncertainty in elementary school students’ investigations as in the discussions among the scientists. She concluded that the process of identifying and resolving uncertainty is a conserved and highly structured activity. The learners, thus, need to understand what uncertainty means in science, and the educators should find ways to create scientifically literate citizens and protect them against post-truthism (Kienhues et al. 2020). Developing strategies to achieve this goal requires understanding and describing the challenges that the learners encounter to resolve uncertainty in science and, for the case of this study, in the COVID-19 pandemic.

Although science includes uncertainty, teachers and students prefer certainty that provides definitive answers. The main problem of numerous cases of public resistance to science such as antievolutionist views, climate change denials, and antivaccine movement is the lack of understanding of uncertainty in science rather than scientific knowledge itself (Kampourakis 2018). The lack of knowledge on the uncertainty leads to the emergence of information disorders in controversial issues. Because of people’s need for certainty and difficulty in making sense of the uncertainty in science, they become more vulnerable to misinformation and disinformation on controversial issues. The topics where there is no consensus upon as a result of rejecting scientific evidence, information pollution, inadequate understanding of uncertainty in science, political stance, or group affiliation can be classified as controversial. COVID-19 has been discussed as a controversial issue for the reasons listed above. As in the current COVID-19 case, the public’s inadequate understanding of uncertainty has led to misinformation and disinformation. Misinformation and disinformation about the novel coronavirus pandemic trigger mistrust in science, as, for example, in the case of climate change denial. Despite the growing mistrust in science, science has a guiding role for society in many vital issues such as vaccination and climate change. Understanding the uncertainty in science and grasping the role of science in society are crucial in this manner. Therefore, there is a growing need to educate action-oriented and socially responsible citizens who engage in evidence-based reasoning and critical thinking processes. From this point of view, educational attempts need to adopt innovative ways to create such citizens (Erduran 2020a).

Managing uncertainty is a social endeavor and accomplished through argumentative discourse. The argumentative practice depends on the learners’ existing knowledge to deal with the uncertainty and their understanding of what counts as claim, data, evidence, and reasoning as well as the degree of their ability to solve the problems to deal with uncertainty (Chen et al. 2019; Chen 2020). Achieving this goal requires redefinition of scientific literacy and skills, ability, and competence that learners need to deal with COVID-19 misinformation and disinformation.

The COVID-19 pandemic and its impact on science, society, and education may last for a long time. Therefore, studies explore the implications of social distancing and how schools and higher education institutions organize themselves in adapting to the pandemic (Erduran 2020a). Along with this pandemic, the spread of misinformation and disinformation about this virus also increased, so did the necessity of creating further socially oriented citizens who critically evaluate information and make informed decisions about health issues. Curfews and social distancing entailed the schools and universities to organize their courses and lessons in online environments. This situation led the researchers of this study to implement an online course to examine pre-service teachers’ analysis of claims that contain fallacies, conspiracy theories, and scientific arguments about COVID-19.

The researchers of the present study argued that analysis of various claims including the evaluation and demarcation of fallacies and conspiracy theories from scientific arguments related to COVID-19 as well as the judgment of the sources of evidence can be used as an effective context for exploring and promoting learners’ critical thinking skills and their scientific literacy. In addition, curfews and social isolation that the governments applied in their countries brought the necessity of implementing online teaching activities all around the world. It is hard to say the exact time of the end of the pandemic and what educational systems would be like in the future. It can be inferred at least that online teaching in schools and universities may be applied for an extended period. Thus, examining and promoting learners’ evidence-based reasoning and critical thinking abilities in online learning settings seems to be significant in this manner. Since pre-service teachers will create such citizens, it is necessary to explore and enhance their evidence-based reasoning to let them critically evaluate the claims containing fallacies and conspiracy theories and demarcate them from scientific arguments by criticizing the sources of evidence. In order to promote the public understanding of uncertainty in science, we decided to start with pre-service teachers. Depending on this background, the study presented here aims to investigate pre-service teachers’ analysis of claims about COVID-19 that includes the evaluation of fallacies and conspiracy theories besides scientific arguments, as well as their decisions about the demarcation between fallacies, conspiracy theories, and scientific arguments and credibility of sources of evidence throughout an 8-week implementation of the pre-service teachers’ analysis task, focus group interviews, and instructor’s feedback to this analysis in a critical and analytical thinking course. For this purpose, we addressed the following research questions in this study:

What is the level of pre-service teachers’ evaluation of the claims about COVID-19 critically throughout an online course?

What is the level of pre-service teachers’ demarcation of fallacies and conspiracy theories from scientific arguments about COVID-19 throughout an online course?

What is the level of pre-service teachers’ judgment of the credibility of the source of the claims about COVID-19 critically throughout an online course?

Theoretical Framework

A thorough examination of pre-service teachers’ analysis of claims requires understanding the information disorders including fallacies and how to deal with them in order to facilitate critical evaluation and decision-making skills. Following these discussions, the paper presented here will introduce an online course in which pre-service teachers analyzed and discussed written tasks on the claims of COVID-19 and the result of their analysis throughout the course.

Before discussing the information disorder, it is necessary to explain what information is. According to Wiener (1954), information is the content that we exchange in adapting to the outer world. The information we exchange in communication processes may sometimes be correct or intentionally or unintentionally incorrect. Information disorders are generally related to the reality of information and the intentionality about its unreality. Information disorders can be listed as misinformation, disinformation, missing information, mal-information, or self-deception (Froehlich 2017). Since we focused on the information spread in the current COVID-19 context in this study, we are interested in misinformation and disinformation.

Disinformation is incorrect information; however, it is deliberately incorrect, its content might be fabricated or manipulated, and its context is also incorrect (Wardle and Derakhshan 2018). With the rise of post-modernism, disinformation was circulating on a series of topics related to basic scientific facts. People who do not have sufficient information literacy and scientific literacy began to spread this information unintentionally. The rapid spread of misinformation and disinformation on such basic issues is a red flag for the researchers. In particular, information disorders about COVID-19 have many consumers among people and quickly shared. Most of the misinformed and disinformed arguments in social media posts contain fallacies and are mostly based on conspiracy theories.

Fallacies About COVID-19

An argument can be defined as a set of propositions where one is claimed to follow from the others (Walton 1987). People use arguments to persuade others about something by justifying a conclusion and accepting that given conclusion. Simply put, each argument contains one or more premises and a conclusion. Arguments generally have two types, deductive and inductive (Tait 2009). In a deductive argument, the premises provide full support for the conclusion, and if those premises are true, therefore the conclusion must be true. In an inductive argument, the premises provide some degree of support for the conclusion and that conclusion is based on probability, and the conclusion is merely based on premises that contain probability. Inductive arguments are also known as informal arguments. Fallacies occur in both inductive and deductive arguments. However, if the premises of the deductive arguments are true, the arguments must be valid. The fallacies of deductive arguments are accepted as formal ones. In daily debates, people generally commit informal fallacies. In sum, the arguments in which given premises are irrelevant, unacceptable, or insufficient in order to reach the conclusion and are caused by an error in reasoning are described as fallacies (Hurley and Watson 2018; Theodore and Vaughn 2014).

Fallacies are divided into various types due to their different characteristics. Aristotle was the first philosopher who categorized the fallacies systematically into thirteen types of error in the book of Sophistical Refutations. Today, there are countless types of fallacies in the reasoning process that have been identified. However, the authors of this research have discussed the fallacies relevant to those that are included in this study.

Appeal to Authority

Appeal to authority is a fallacy that assumes an argument is valid because the arguer is an authority. Celebrity endorsements often involve appeals to authority fallacy, because fame does not give you any special expertise (Theodore and Vaughn 2014). The conclusion reached in an argument is evaluated based on its premises. The conclusion is independent of the reputation or expertise of the person who says it. Aristotle’s theories were taken as sacred in the middle ages and accepted as the last word in the debates on many subjects until scientific development eroded his authority (Woods et al. 2004). Accepting his arguments as valid without discussing the premises of his arguments is an example of an appeal to authority fallacy.

Burden of Proof

The burden of proof fallacy basically involves drawing a conclusion on the basis of the lack of evidence against this conclusion (Tindale 2007). The one who commits this fallacy places the burden of proof on the wrong side of the argumentation. Briefly, in this fallacy, the claimant burdens the proof that his/her claim is false to the other party instead of supporting his/her claim with premises. For example, to claim that aliens exist since no one has ever proven that aliens do not exist is an example of such a burden of proof fallacy. Even though it seems quite persuasive in daily discussions, the claimant is obliged to prove his or her claim.

Appeal to Emotion

This fallacy occurs when the conclusion is supported by emotions of the listeners such as pity, fear, or envy (Copi et al. 2014). Claimant manipulates listeners’ emotions in order to get them to think a claim as true. In this fallacy, emotions are used as evidence for the claim, and the other party is expected to accept those emotions as evidence. An example of this fallacy is that charity organizations emphasize that we ourselves are not hungry while demanding help to hungry people. In this example, charity organizations try to collect aid by manipulating the emotions of their benefactors, rather than building robust justifications to support their arguments.

Post Hoc Fallacy

It is a kind of fallacy created by establishing an imaginary link between the premises and the conclusion (Hurley and Watson 2018). This fallacy is often used by politicians, charlatans, and alternative therapists. The claim that a solar eclipse causes earthquake because they happened successively may count as a post hoc fallacy. The successive occurrence of more than one event does not provide sufficient evidence to show that these events are interconnected. Linking more than one event together just because they happen sequentially is also known as magical thinking.

Irrelevant Conclusion

An irrelevant conclusion is a fallacy that detracts people from the relevant point subject to discussion and draws an irrelevant conclusion (Barbone 2019). In such fallacy, an unrelated or unverified topic is included in the discussion and the focus of the discussion is shifted towards another direction to convince the opponent. While discussing the protection of the mask against coronavirus, referring to the rights and freedoms can be given as an example of an irrelevant conclusion fallacy.

According to McIntyre (2020), one of the common features of science deniers is committing logical fallacies. Therefore, it is extremely important to understand the structure of an argument and to be able to detect errors in reasoning, regardless of the subject being discussed. Fallacies act as useful tools for critical thinking (Walton 2010). Encouraging students to construct good arguments and justify their claims with reasoning as well as to evaluate arguments requires promoting their knowledge of common fallacies and the sources of these errors in the argumentation process (Zeidler 1997).

Conspiracy Theories About COVID-19

After COVID-19 became a global threat, multiple theories began to come forward in social media. Many arguments of these theories include fallacies. We will give some examples of countless theories and the fallacies that these theories contain in this section, explain the errors, and compare them with scientific facts.

One of the conspiracy theories that first went into circulation is the claim that the novel coronavirus causing the outbreak emerged, strengthened, or moved by radio waves emitted by 5G (fifth-generation) mobile networks. Advocators of this conspiracy theory argue that after 5G mobile networks have been rolled out in Wuhan, China, the novel coronavirus appears in the same city. Although the argument seems valid, it does not present any concrete evidence for the claim and it has errors in reasoning. This argument contains post hoc fallacy that establishes an imaginary link between the premises and the conclusion (Hurley and Watson 2018), because correlation does not imply causation. Moreover, although a 5G mobile network rolled out in numerous cities, there is no report that the novel coronavirus has emerged in other cities.

In another theory, it is claimed that the novel coronavirus was artificially created in the laboratory for a secret aim of the government (Mian and Khan 2020). The aforementioned government varies according to the country where the conspiracy theory was put forward. In addition, claims of creating the virus include a wide variety of purposes ranging from using biological weapons to marketing vaccines to population control by targeting certain religious groups. Although there is no credible evidence for these claims, a significant number of people believe and disseminate these claims. Moreover, genome studies on the origin of the virus have shown that the novel coronavirus originated in nature from animal sources (Lu et al. 2020).

One conspiracy theory is the claim that some TV series (Carras 2020) and novels (Roberts 2020), which had been published in the past, predicted the novel coronavirus years ago. Reviews by fact-checking organizations have shown that the images in the TV series are altered digitally (Mason 2020), and also in the later editions of that sci-fi novel, it was found that there was no similarity in the subjects such as the incubation period and mortality rate other than the Wuhan-400 name given to the virus (Evon 2020). In addition, the first version of the novel was written in the cold war era and the virus was called Gorki-400 in the first edition of the book.

Other popular information in circulation claim that home remedies such as coconut oil (Pennycook et al. 2020), vitamin C and garlic (Mian and Khan 2020), and alcohol consumption, very hot or very cold weather conditions, hand dryers, UV lamps, and antibiotics can kill the novel coronavirus. In order to prevent the risks caused by such claims and inform the public, WHO launched a “Mythbuster” information site (WHO 2020b) where dangerous rumors are refuted by scientific facts.

In addition to conspiracy theories, a number of fallacious arguments are also circulated in mainstream and social media during infodemic. Several mediatic medicine doctors in Turkey claimed that the novel coronavirus can be eliminated by making and gargling salt and vinegar (Flu 2020), the Turkish people would not be affected by the outbreak because of their genetic structures because nobody had proved otherwise, and drinking a traditional soup would prevent the novel coronavirus. The media spread these statements by highlighting the titles of the doctors. These claims include the appeal to authority and burden of proof fallacies. The appeal to authority fallacy occurs when someone affirms that an argument is correct because a field expert uttered that it is correct (Copi et al. 2014). However, there is always a possibility that experts’ opinion might be incorrect, unless she/he does not support their ideas with data. Therefore, the title of the person does not provide sufficient justification to accept a claim as correct. The case presented here included statements that were not based on any evidence, although they were experts in the field of medicine. Hence, to accept this argument as correct and to claim that it must be accepted as correct contain fallacies. The burden of proof fallacy is committed when someone places the burden of proof on the wrong side of the argument (Bennett 2012). In the example above, claiming that the genes of Turkish people are not affected by COVID-19 and that there is no proof otherwise put forward by anyone is an example of the burden of proof fallacy. As a matter of fact, the person who suggests extraordinary claims is supposed to provide evidence for this claim.

Last but not the least, in social media and messaging applications, some rumors in circulation include the claims that tea prevents people from the novel coronavirus due to its antioxidant features, the virus is exaggerated, and the activist of “68 Generation” has been forcibly closed at home. All these statements contain irrelevant conclusions and appeal to emotions, rather than providing evidence for the claims. Appeal to emotion is a form of fallacy that refers to emotional strategies that are irrelevant to the situation (Walton 1987). There is no apparent link between isolating the older people who are at risk to protect them from the fatal effects of the novel coronavirus and them being activists of “68 Generation.” The claimant targets the feelings of the people with this argument. An irrelevant conclusion fallacy occurs when premises support a different conclusion from the one that is proposed (Copi et al. 2014). The premise that the antioxidant feature of the tea and the conclusion that this feature provides protection against the novel coronavirus can be given as an example of the irrelevant conclusion fallacy.

Although some of the information and conspiracy theories that spread seem harmless, health advice, in particular, can cause serious problems and even deaths. Health-related disinformation poses serious risks to human and public health and causes people to waste time and money by producing pseudoscientific solutions to real problems (Dyer and Hall 2018). People are more prone to believe in what is simple rather than complex explanations in cases of uncertainty. At the same time, individuals are surrounded by countless biases (Kahneman 2011) even when attempting to understand complex processes. On the other hand, conspiracy theories minimize complexity (Byford 2011). We argue that people’s simplicity preference is the reason why such theories are so common in social media streams. The posts from conspiracy pages on social media are more likely to be liked and shared than the posts from science pages (Bessi et al. 2015). Falsehood and fake news spread farther, faster, deeper, and wider than truth (Vosoughi et al. 2018). Individuals often continue to believe in conspiracy theories, even if there is considerable amount of contradictory evidence. Even if all the conspiracy theories proposed about AIDS have been refuted, AIDS deniers continue to insist (Mian and Khan 2020). After an instructional implication in a critical thinking course, Dyer and Hall (2018) found that conspiracy theories are the most resistant realm to change.

Scientific Arguments About COVID-19

Scientific arguments are based on evidence and linked to knowledge justification in which claims should include logical clauses and be based on data and evidence from different sources (Jiménez-Aleixandre and Erduran 2007). Unlike the fallacies and conspiracy theories regarding the COVID-19 pandemic, scientists put forward a number of scientific arguments as a result of their research studies.

Along with many other factors, researchers simultaneously investigated the factors that trigger the spread of the novel coronavirus, the tools that prevent people from COVID-19, and the symptoms of COVID-19 patients. According to Palatnik and McIntosh (2020), the novel coronavirus transmits through aerosol when people produce droplets during sneezing or coughing. The effects of COVID-19 on patients were also a matter of curiosity. Scientists then began reporting symptoms of COVID-19. Afterwards, researchers investigated common symptoms of COVID-19 by conducting meta-analysis and review studies on the reports. In their systematic review, Abebe et al. (2020) stated that the common symptoms in COVID-19 patients are fever, dry cough, fatigue, and dyspnea. Due to the highly contagious nature of the novel coronavirus, scientists investigated the optimum conditions in which transmission to other people can be prevented. Scientists have published papers on these conditions in a short time with significant results. Chu et al. (2020) reviewed the former studies about COVID-19 and found that social distancing, face masks, and eye protection prevent the novel coronavirus transmission from person to person.

Scientific arguments about COVID-19 are based on research studies because of their nature. These studies originate from different countries, sources, and patients. The common feature of the previously mentioned scientific arguments about COVID-19 is that they are based on data and evidence from different sources. However, in the pandemic environment, all kinds of information including fallacies and conspiracy theories as well as scientific arguments have been spreading rapidly. For this reason, it is important to note that individuals would be exposed to not only scientific arguments but also information disorders about COVID-19.

The abundance of information necessitates individuals not only detect information disorders but also distinguish data, evidence, and information because sometimes the problem is people’s understanding and interpretation of data and news, rather than information disorders. Therefore, it is also important to educate people to understand what counts as data, evidence, and information (Cavagnetto and Hand 2012). Analyzing the claims in texts, thus, seems to play a crucial role in promoting individuals’ critical thinking skills and scientific literacy in the post-truth era.

The post-truth era, in which fallacies and conspiracy theories are easily spread among the public, requires educating every individual to be able to evaluate information critically and make informed decisions. In this manner, the ultimate goal of science education is to facilitate the understanding of scientific practices rather than science content. For achieving this aim, science education needs to equip individuals to answer the questions addressing which claims are trustworthy and why, which experts to be trusted, especially when they have disagreements on an issue in question, and how scientists draw a conclusion. In order to answer these questions, it is necessary to educate individuals to critically evaluate the claims by analyzing their reliability and make informed decisions on personal and social issues (Allchin 2013).

Facilitating decision-making skills necessitates avoiding learners’ biased thinking. From this perspective, science educators should provide them opportunities to evaluate claims and evidence critically and engage in argumentation. As the learners understand and evaluate evidence and arguments, they begin to evaluate judgments by realizing and criticizing counter-arguments (Lee 2007). Miller (1998) considers scientific literacy as “the ability to read and write about science and technology” (pp. 203–204). Competence to comprehend and follow arguments in the text is necessary to achieve this ability (Miller 1998). In order to comprehend and follow arguments in the text, it is also necessary to critically evaluate these arguments. Critical evaluation in science education also includes connections between evidence and alternative descriptions (McNeill et al. 2006).

In light of this theoretical background, it is extremely important for individuals to be able to evaluate claims, detect fallacies and conspiracy theories that arguments contain, demarcate fallacies and conspiracy theories from scientific arguments, and make a critical judgment about the credibility of the sources of evidence. Features such as scientific literacy, scientific content knowledge, and higher educational levels do not guarantee not falling into these traps in reasoning. Studies have replicated the finding that there is no relationship between knowledge of scientific facts and skepticism about pseudoscientific claims (McLaughlin and McGill 2017). As people do not think deeply about whether the content is correct or not when sharing misinformation and disinformation about COVID-19 (Pennycook et al. 2020), it is necessary to include these topics in different instructional settings. In the post-truth era, people continue to spread information pollution while creating their own alternative realities. Therefore, it is important to develop critical thinking and decision-making skills on controversial issues.

Sinatra and Lombardi (2020) suggested the need to enable students to evaluate the link between information sources and claims, for instance, scientific explanations about complex phenomena. They also argued that explicitly reappraising plausibility judgments may be a critical point in evaluating the relationships between sources of information and knowledge claims. While searching for and reading a scientific article online, the individuals need to ask the plausibility of the explanations and how they know it. It is significant, therefore, to create further citizens who critically evaluate information by questioning its source and make informed decisions about vital issues such as health-based or nutrition-based issues, which spread as information pollution in public. Teacher education courses should include educating pre-service teachers as they create such citizens.

We argue in this paper that it is necessary to probe into pre-service teachers’ analysis of fallacies and conspiracy theories about the pandemic that we live today, namely, COVID-19, in which we face various claims about it in order to find out how to facilitate the critical evaluation of these claims and make informed decisions. It is especially important to examine the analysis of pre-service teachers whose major is other than science because they are presumably unfamiliar with health issues. Examining pre-service teachers’ analysis of claims that contain fallacies, conspiracy theories, and scientific arguments about COVID-19 in an online course may be helpful to develop and improve future critical thinking courses to be given online. This exploration will bring new light to the science education and teacher education literature in terms of constructing teacher education courses to enable pre-service teachers to critically evaluate and make informed decisions about health issues such as a pandemic.

Method

Participants

Ten pre-service teachers who enrolled in the “Critical and Analytical Thinking” course in the Arabic Language Teaching Department of Education Faculty of a private university in Turkey participated in this study. This course was a faculty elective course aiming to promote pre-service teachers’ critical and analytical thinking skills and their ability to design activities to improve their students’ critical and analytical thinking skills. All of the participants were female and socioeconomically middle class. They are sophomores of the Education Faculty, and all of them completed the courses of reading, writing, and verbal communication skills, as well as the pedagogical course including an introduction to education, educational psychology, and educational sociology. Paul and Elder (2005) suggested competence in reading and writing among the criteria of critical thinking skills. Besides, as mentioned previously, the skills for comprehending, interpreting, analyzing, and criticizing texts are important aspects of scientific literacy in a fundamental sense. For this reason, the students whose major is language teaching were selected as the participants of this study. This competence and clarity will broaden teacher educators’ perspectives in teaching critical thinking skills.

Procedure

When COVID-19 spread worldwide after January 2020, the Ministry of Health of the Turkish Republic (MoH) began to take a series of precautions in other countries as it is in. The Turkish government formed a scientific committee consisting of scientists under the responsibility of the Ministry of Health, and the aforementioned committee prepared a guide for healthcare professionals (MoH 2020a). At the end of January, the Minister of Health of the Turkish Republic made statements regarding the etiology of the virus, following the criticism received due to the increase in the number of COVID-19 cases worldwide and the continuation of international passenger transportation, and stated that there was no cause for concern (MoH 2020b). However, after a while, the Turkish Ministry of Health (2020c) declared the first COVID-19 case on March 11 and the first death on March 17 in Turkey. At the same time, the Presidency of the Turkish Republic (2020) suspended education in schools and announced that education will be conducted online. After a short time, while a curfew was imposed on weekends in certain cities, a full-time curfew was started to be applied for the citizens who is over the age of 65. In addition, various precautions such as the obligation to wear masks in public areas and restrictions on travel to the other cities were taken. In May 2020 when a recent study was being carried out online, according to the Turkish Ministry of Health (2020c), 163,942 people were infected and 4540 people died due to COVID-19 in Turkey. According to the World Health Organization, there were 5,934,936 infected people and 367,166 deaths worldwide on the same dates (WHO 2020b). This study was conducted under these circumstances.

The first author of this paper was the instructor of the study. Following the introduction of these concepts, the authors implemented the study. The course began with face-to-face instruction of the concepts of thinking and reasoning, analytical and critical thinking. However, following teaching these topics, the COVID-19 pandemic began to spread in Turkey and this course is turned into an online course. The instructor taught the topics of inductive and deductive reasoning, valid and invalid arguments, fallacies, and credibility of the sources of information online. The participants also learned to differentiate the claims and arguments and their relationship to evidence, and they constructed claims and arguments on various topics by providing evidence for each claim in the online course. Following the introduction of these concepts, the authors implemented the study.

After shifting to online education, the instructor designed all the tasks including claim analysis task and focus group interviews online. The participants attended online classes muted and cameras off. Even if the instructor asked them to unmute themselves and keep their cameras on, they were usually unwilling to do so during the discussions of the aforementioned topics. However, they unmuted themselves during focus group interviews and willingly participated in the discussions. Although they continue keeping their cameras off, their willingness to participate in the oral discussions during focus group interviews was promising.

The study lasted over 8 weeks, during which the pre-service teachers analyzed claims and evidence in two different tasks. The first task included 15 claims, 5 of which were fallacies, 5 of which were conspiracy theories, and the rest were scientific ones, and the second task included a claim and a counterclaim which contain dilemmas. All of the participants were assigned to the 1st analysis task in which all the claims were the same for all the participants, while only volunteer participants whose pre- and post-analysis of the 1st task was, respectively, low were assigned to the 2nd analysis task in which the claims were different for each participant. The implementation of this study began with the participants’ submissions of the 1st pre-analysis task in which they analyzed each claim and evidence in written tasks and the credibility of the sources that they utilized in order to evaluate each claim. Following 3 weeks, the first author conducted focus group interviews over 3 weeks with volunteers. In the 2nd, 3rd, and 4th weeks of intervention, the instructor conducted focus group interviews with 4, 6, and 5 interviewees, respectively. Finally, the participants resubmitted their revised version of the analysis task, namely, the 1st post-analysis task. After analyzing each participant’s pre- and post-analysis tasks, the instructor gave feedback to their tasks and assigned volunteer participants to the 2nd analysis task. Table 1 illustrates the flow of the implementation.

Table 1

The flow of the implementation

Weeks	Activity
1	Participants’ submission of the 1st pre-analysis task
2–4	Focus group interviews with volunteers
5	Participants’ submission of the 1st post-analysis task
6	Instructor’s feedback to pre- and post-analysis tasks
7	Volunteer participants’ submission of the 2nd analysis task

The instructor gave feedback on four criteria including the evaluation of the claims, demarcation of fallacies and conspiracy theories from scientific arguments, selection of the sources of the claims, and discussion of the credibility of the sources. The participants were informed about the quality of their evaluations, demarcations, and judgments as well as the errors they make while discussing each criterion. Before submitting their 2nd analysis task, the instructor answered the participants’ questions in an online course or visual communication via social media platforms. Finally, in the seventh week of implementation, volunteer participants submitted their 2nd analysis task.

The researchers of this study decided to include the most common fallacies and conspiracy theories during the beginning of the pandemic in Turkey. During these days, people were mostly exposed to appeal to authority fallacies and they became viral on social media. Therefore, we included two examples of this type of fallacy in the list. In order to include equal examples of fallacies, conspiracy theories, and scientific facts and provide a variety of claims, the researchers decided to create a list of 15 claims including each category. Table 2 lists each claim presented to the participants and the category that each claim was involved in.

Table 2

Claims and their categories

Categories	Types	Claim
Fallacies	Appeal to authority fallacy	According to Dr. X, the novel coronavirus stays in the throat for 4 days before going down. Therefore, gargling with salt, vinegar, and hot water will be helpful.
	Appeal to authority fallacy	According to Dr. Y, drinking traditional Turkish soup day and night will prevent the infection of coronavirus.
	Appeal to emotion fallacy	The old people who you look down on, humiliate, and force to stay at home are Generation 68 activists. They fought on the streets for human rights, for their children’s rights, and for your rights. And what do you do to thank them? You lock them at their home for a virus that does not have risks as much as it is exaggerated.
	Burden of proof	Turkish people are not affected and will not be affected by this virus because of their genetic properties (ACE2 protein). Nobody could prove the opposite so far.
	Irrelevant conclusion	Tea can heal COVID-19 because of its antioxidant property.
Conspiracy theories	Conspiracy theory	A 1981 book, namely, The Eyes of Darkness, predicted the novel coronavirus 2019 outbreak including all the symptoms of the disease.
	Conspiracy theory	The novel coronavirus is produced with biotechnology by genetic engineers to be used against some countries for warfare purposes.
	Conspiracy theory	The novel coronavirus is produced by the Chinese government in order to decrease the population of China.
	Conspiracy theory	An episode of The Simpsons in 1993 predicted the novel coronavirus outbreak.
	Conspiracy theory	Just after the use of 5G technology, the novel coronavirus began to spread in China. The deaths occurred because of the signals carried by radio waves.
Scientific arguments	Scientific arguments	Viruses mutate, turn into other species, and cause diseases on different hosts.
	Scientific arguments	Antibiotics do not affect coronavirus; so, it is unnecessary to use antibiotics.
	Scientific arguments	The novel coronavirus is a respiratory virus which spreads primarily through droplets generated when an infected person coughs or sneezes or through droplets of saliva or discharge from the nose. Therefore, it is necessary to cough or sneeze into a tissue or inside the elbow.
	Scientific arguments	The novel coronavirus affects the respiratory system. So, smokers are more vulnerable to this disease.
	Scientific arguments	The incubation period of the novel coronavirus can last for 14 days. Therefore, people can spread it without realizing it.

This study has three data sources: the 1st analysis task, focus group interviews, and 2nd analysis task. The researchers of this study identified each claim about COVID-19 and prepared the questions to ask the participants in the 1st analysis task. In order to identify each one, the researchers searched the literature, Internet, and social media. They selected the most common fallacies and conspiracy theories that circulate on the Internet and social media. The researchers also included 5 scientific facts that the scientists had most consensus on. In the written texts, the participants were presented 15 claims and asked to evaluate each claim by providing evidence, demarcate fallacies and conspiracy theories from scientific arguments by identifying scientific argument for each fallacy and conspiracy theory, and finally identify and list the sources they utilized while searching claims by discussing the reliability of each source of evidence. Since the analysis of fallacies requires evaluating the relationship between claims and evidence, demarcating fallacies from scientific arguments, and judging the reliability of the sources of each claim, the researchers identified these three criteria to assess the participants’ analysis.

Following the submission of the 1st pre-analysis task, the instructor conducted the focus group interviews with the participants. The researchers used focus group interviews as both an intervention and a data source. Depending on the participants’ responses to the questions for each claim, the instructor began the interview by probing into their initial beliefs of each claim. She asked critical questions such as “How do you know that?” or “Is it your belief or do you have any evidence for this claim?” without directly giving the exact answer. The main purpose of these questions was to facilitate the critical evaluation of each claim and demarcation of fallacies and conspiracy theories from scientific arguments. For example, when a participant explained that this virus can be produced to decrease the population of the world, the instructor asked for evidence for this claim. This question led the participants to discuss what counts as evidence and based on the provided evidence how scientific arguments differ from unscientific ones. She also answered the participants’ questions on how to search for each claim by mentioning the criteria of the sources of evidence. For example, one of the participants asked whether it is enough to say that the websites of the WHO and Ministry of Health in Turkey are trustworthy sources; she reminded them depending on data comes first to decide the credibility of the sources.

After focus group interviews, the participants submitted their 1st post-analysis tasks. The researchers calculated the participants’ average scores of the 1st pre- and post-analysis tasks and decided to assign those whose average score is below 2.0 to an additional task after the instructor’s feedback. A 2.0 score was the reference point to make this decision for the researchers as it is a moderate level of analysis. In this way, they examined the effectiveness of the feedback on their level of analysis. They identified 5 out of 10 participants to be assigned to the 2nd analysis task. Table 3 indicates the claims and counterclaims to which each participant was assigned in the 2nd analysis task.

Table 3

The claims and counterclaims in the 2nd analysis task

Number of the task	Claim and counterclaim
1	Claim: Nature is saved thanks to the pandemic. Counterclaim: It cannot be inferred that nature benefited from the pandemic.
2	Claim: Using a mask is very effective for protection from the novel coronavirus. Counterclaim: Using a mask is not effective for protection from the novel coronavirus.
3	Claim: It is necessary to make high-stakes tests by taking the necessary precautions during the pandemic. Counterclaim: It is not necessary to make high-stakes tests during the pandemic even if required precautions are taken.
4	Claim: In the 2020–2021 instructional year, the lessons should be carried out online because of the coronavirus outbreak. Counterclaim: In the 2020–2021 instructional year, the lessons may be carried out face to face by taking required precautions.
5	Claim: The people with a particular blood type are infected with the novel coronavirus than others. Counterclaim: There is no relationship between the infection of the novel coronavirus and the blood type.

Data Analysis

The researchers created a rubric to score the participants’ analysis of the claims. First, they determined the criteria of analysis of claims based on research questions. After determining these criteria, they decided to include three levels to easily classify the quality of each criterion. Each of these levels was determined based on the answers given by the students after the content analysis. Following the discussions on the levels, they coded the participants’ answers at each level for each criterion independently. They discussed their conflicts until they reach a full agreement on coding. Table 4 shows the rubric to assess their analysis and examples for each criterion and level. The researchers assigned scores of 3, 2, and 1 to high, moderate, and low levels of analysis, respectively, for each of the category of each claim.

Table 4

Rubric for the pre-service teachers’ analysis of claims

Dimension	Level	Description	Example
Evaluation	High	The participant correctly explained a causal relationship between claim and evidence by reflecting a deeper analysis and elaborating on the evaluation of claim and evidence.	There is no relationship between being activists of Generation 68 and their risks of getting the disease. Lockdown is a precaution to protect old people from the disease, not an intention to humiliate them. This is an example of an appeal to emotion fallacy. The people who propose this argument try to manipulate our emotions to convince us.
	Moderate	The participant correctly explained a causal relationship between claim and evidence without elaborating on the evaluation of claim and evidence.	The scientists could not find a relationship between the infection rate and the race.
	Low	The participant was unable to employ a relationship between claim and evidence.	This claim cannot be true because the World Health Organization (WHO) did not make any explanation about the protection of tea against COVID-19.
Demarcation	High	The participant demarcated fallacies and conspiracy theories from scientific arguments and vice versa clearly, and she explained the scientific background of the claim successfully.	The claim that the novel coronavirus stays in the throat for 4 days before going down is an appeal to authority fallacy. Dr. X’s authority does not ensure his arguments’ validity. He does not provide any kind of evidence for his claim.
	Moderate	The participant successfully detected whether the given claim is a scientific argument or not but could not justify why it is scientific or unscientific. She also committed a fallacy or produced a conspiracy theory in some of her own explanation.	This argument is not a fallacy because using antibiotics makes bacteria more resistant.
	Low	The participant was apparently challenged to demarcate fallacies and conspiracy theories from scientific arguments.	The claim that the book entitled The Eyes of Darkness predicted the novel coronavirus is an appeal to ignorance fallacy. It is impossible to predict the future.
Judgment	High	The participant critically judged the credibility of each source that she utilized while researching the claim by comparing different sources of evidence.	There are some criteria to judge the credibility of the sources. For example, providing evidence by presenting data is among these criteria. The sources I used to search for this claim meet this criterion. Besides, the website of the Ministry of Health is a trustworthy source since it is one of the primary sources in Turkey for scientific explanations about health.
	Moderate	The participant critically judged the credibility of only one source that she utilized while researching the claim, or some of the sources she utilized were not appropriate.	This source cites the WHO. So, I think it is reliable.
	Low	The participant was apparently challenged to judge the credibility of the sources of the claim.	This source is credible because the claims they suggest are true.

The researchers collaboratively created the claims and questions in the analysis tasks as well as the categories of the rubric and independently analyzed and coded the participants’ responses. The initial percentage agreements of their coding were 74% and 80% for the 1st and 2nd analysis tasks, respectively. They discussed their conflicts until they reach a complete consensus on the codes.

The researchers transcribed the focus group interviews verbatim and analyzed participants’ explanations independently. Then, they discussed each participant’s explanations until they fully agree with the groups’ general understandings and beliefs. Finally, they translated the quotations they made from participant responses from Turkish to English.

Results and Analysis

The study presented here investigated pre-service teachers’ analysis of fallacies about COVID-19 throughout an online implementation in an analytical and critical thinking course. The implementation included participants’ analysis of various claims about COVID-19 and focus group interviews. For this purpose, the researchers examined the participants’ analysis of fallacies, conspiracy theories, and scientific arguments about COVID-19. Table 5 illustrates the participants’ analysis of each claim before and after focus group interviews.

Table 5

The frequencies and percentages of each level in the 1st pre- and post-analysis results

		Pre						Post
		High level		Moderate level		Low level		High level		Moderate level		Low level
Participant*	Skills	f	%	f	%	f	%	f	%	f	%	f	%
Abigail	Evaluation	–	–	7	46.66	8	53.33	3	20	10	66.66	2	13.33
	Demarcation	4	26.66	11	73.33	–	–	6	40	9	60	–	–
	Judgment	–	–	–	–	15	100	3	20	6	40	6	40
Emma	Evaluation	2	13.33	8	53.33	5	33.33	6	40	9	60	–	–
	Demarcation	1	6.66	13	86.66	1	6.66	4	26.66	11	73.33	–	–
	Judgment	–	–	7	46.6	8	53.3	–	–	7	46.6	8	53.33
Hazel	Evaluation	11	73.33	4	26.66	–	–	12	80	3	20	–	–
	Demarcation	12	80	2	13.33	1	6.6	13	86.66	2	13.33	–	–
	Judgment	5	33.33	1	6.66	9	60	8	53.33	–	–	7	46.66
Hillary	Evaluation	1	6.66	8	53.33	6	40	1	6.66	10	66.66	4	26.66
	Demarcation	–	–	15	100	–	–	1	6.66	14	93.33	–	–
	Judgment	–	–	5	33.33	10	66.66	2	13.33	9	60	4	26.66
Karen	Evaluation	1	6.66	10	66.66	4	26.66	3	20	8	53.33	4	26.66
	Demarcation	–	–	15	100	–	–	3	20	12	80	–	–
	Judgment	1	6.66	7	46.66	7	46.66	1	6.66	7	46.66	7	46.66
Keira	Evaluation	10	66.66	5	33.33	–	–	10	66.66	5	33.33	–	–
	Demarcation	11	73.33	4	26.66	–	–	11	73.33	4	26.66	–	–
	Judgment	14	93.33	1	6.66	–	–	14	93.33	1	6.66	–	–
Melissa	Evaluation	8	53.33	5	33.33	2	13.33	9	60	5	33.33	1	6.66
	Demarcation	9	60	4	26.66	2	13.33	9	60	4	26.66	2	13.33
	Judgment	6	40	5	33.33	4	26.66	7	46.66	6	40	2	13.33
Megan	Evaluation	–	–	1	6.66	14	93.33	–	–	3	20	12	80
	Demarcation	–	–	9	60	6	40	–	–	13	86.66	2	13.33
	Judgment	–	–	–	–	15	100	–	–	–	–	15	100
Mila	Evaluation	7	46.66	8	53.33	–	–	11	73.33	4	26.66	–	–
	Demarcation	9	60	6	40	–	–	11	73.33	4	26.66	–	–
	Judgment	–	–	2	13.33	13	86.66	2	13.33	3	20	10	66.66
Yoselin	Evaluation	12	80	3	20	–	–	11	73.33	4	26.66	–	–
	Demarcation	11	73.33	2	13.33	2	13.33	11	73.33	4	26.66	–	–
	Judgment	7	46.66	5	33.33	3	20	2	13.33	3	20	10	66.66

*Pseudonyms are used for participants’ names

The 1st Pre- and Post-analysis Results

Evaluation of Claims About COVID-19

As shown in Table 5, more than half of the participants (Abigail, Emma, Hillary, Karen, Megan, and Mila) were able to evaluate the claims mostly at moderate and low levels before the implementation. However, less than half of the participants (Abigail, Emma, Hillary, and Karen) evaluated the claims at a moderate level and only one participant (Megan) evaluated the claims at a low level in general after the implementation. The number of the evaluation level of the claims for each participant seems to have increased as well. After the implementation, the number of claims that the participants were able to evaluate at a high level increased. Four out of 10 participants (Hazel, Keira, Melissa, and Yoselin) were able to evaluate more than half of the claims at a high level before treatment, while 5 out of 10 participants (Hazel, Keira, Melissa, Mila, and Yoselin) were able to do that after the treatment. The number of claims that these participants were able to evaluate at a moderate level did not change. The number of participants who were able to evaluate more than half of the claims were 4 (Emma, Hillary, Karen, and Mila) in the pre-analysis task, while this number is again 4 (Abigail, Emma, Hillary, and Karen) in the post-analysis. After the implementation, the number of claims that these participants were able to evaluate at a low level decreased. Two out of 10 participants (Abigail and Megan) evaluated more than half of the claims at a low level, while only one participant (Megan) evaluated more than half of the claims at a low level. These results revealed an increase in the evaluation level of the participants overall.

Demarcation of Fallacies and Conspiracy Theories from Scientific Arguments

It is interesting to note that the participants’ ability to demarcate fallacies and conspiracy theories from scientific arguments is greater than their evaluation of claims. As indicated in Table 5, before the implementation, the majority of participants were able to demarcate the fallacies or conspiracy theories in the claims from scientific ones at high (Hazel, Keira, Melissa, Mila, and Yoselin) and moderate (Abigail, Emma, Hillary, Karen, and Megan) levels. After the implementation, 4 out of 10 participants (Hazel, Keira, Melissa, Mila, and Yoselin) seem to have been able to demarcate claims at a high level and 5 out of 10 (Abigail, Emma, Hillary, Karen, and Megan) in general. However, the participants rarely demarcated fallacies and conspiracy theories from scientific arguments at a low level (only Megan did so 40% of the claims) before treatment and even less (Melissa and Megan did so 13.33% of the claims) after treatment.

Judgment of Credibility of the Sources

On the one hand, the participants seem to have benefited from the implementation in terms of evaluation of claims and demarcation of fallacies and conspiracy theories from scientific arguments. On the other hand, the participants seemed to have been challenged in judging the credibility of sources both before and after the implementation. As evident in Table 5, the vast majority of participants could not critically judge the credibility of sources before the implementation. The participants’ judgment on the credibility of the sources was mostly at a low level before the implementation, and they were rarely able to judge the credibility of the sources at a higher level even after the implementation. Six out of 10 participants (Abigail, Emma, Hazel, Hillary, Megan, and Mila) judged the credibility of more than half of the sources at a low level before the treatment, while 4 out of 10 participants (Emma, Megan, Mila, and Yoselin) made more than half of these judgments at a low level after treatment. Only Keira judged the credibility of the sources at a high level even before and after the treatment, while Hillary made more than half of the judgments at a moderate level after treatment.

The Thematic Analysis of the Analysis Task

In addition to the analysis of research questions, the researchers of this study detected fallacies, biases, and conspiracy theories in students’ explanations while analyzing the claims. It is interesting to note that some of the participants made fallacious arguments while analyzing the claims even in their post-analysis task. The following themes emerged from the participants’ responses.

Bias in Judgment of Sources

While evaluating each claim, some of the participants argued that the claim would not be correct because the public health institutions did not make any explanation about this claim and they accepted it as evidence. They seem to have recalled the discussions in the class about the explanations of public health institutions without considering the claim and the evidence that needs to be provided for the claim.

Confirmation Bias

Some of the participants applied this bias while judging the credibility of the sources, while some of them judged the credibility of the sources by relating it to the claim itself. The quotes from their post-analysis task demonstrating this situation are given below:

In some other examples, the participants committed various fallacies while evaluating and demarcating the claims in their post-analysis task.

The source I applied is trustworthy. Because I don’t think radio waves will create viruses. The source supports my idea. (Emma)

The claims that tea cures COVID-19 are unfounded. For this reason, I do not trust the sources. (Hillary)

When I consider the instructor’s explanations about this website, I think the source is trustworthy. (Hazel)

Circular Reasoning

It is evident that the participants think that the sources are trustworthy since they support their views.

Nirvana Fallacy

The following explanation can be an example of this fallacy:

It is true that it is necessary to cough or sneeze into a tissue or elbow to protect against COVID-19, but not sufficient. We need to take some other precautions to be protected. Therefore, this claim is a fallacy. (Emma)

Burden of Proof

As shown in the following quotation, this participant shifted the burden of proof to the other party while evaluating the claim:

The information that the viruses could mutate is correct because no one could prove the opposite. (Melissa)

Argumentum ad Populum

The same participant applied this fallacy instead of relating the evidence with the claim:

Furthermore, this information is already accepted by everyone. (Melissa)

Appeal to Authority

Hillary based her argument on the experts without providing any source of evidence while discussing the incubation period of the novel coronavirus:

The information of the incubation period of the novel coronavirus can last for 14 days is correct because the experts made similar statements on this issue. (Hillary)

Conspiracy Theories

Some participants applied conspiracy theories while demarcating the conspiracy theories from scientific arguments in their post-analysis task. The quotations that exemplify these conspiracy theories are as follows:

The participants correctly identified the conspiracy theories in the claims that they were presented. However, they produced speculative claims which are not based on any data while demarcating them from scientific arguments.

I think the claims about the Simpsons are conspiracy theory. People who raised that claim may have aimed to make negative psychological effects on Chinese people. (Emma)

The claim that tea cures COVID-19 is a conspiracy theory. This claim may have been raised to increase tea consumption and marketing. (Karen)

The claim that Turkish people will not be affected by the novel coronavirus because of their genetic features is a conspiracy theory. This claim may have been raised to glorify the Turkish people or to prevent their struggle against the virus by making Turkish people trust their genes. (Karen)

The Results and Analysis of Focus Group Interviews

After the participants submitted their 1st pre-analysis task, the instructor conducted focus group interviews throughout 3 weeks. The first week included the conspiracy theories about COVID-19. The following excerpt depicts the confusion of the four volunteer participants about this issue:Hillary and Melissa also mentioned this conspiracy theory, but Hillary mentioned that the virus may be produced by China itself to decrease its population and Melissa argued that a Chinese scientist and an American scientist collaboratively produced the novel coronavirus to destroy the USA. Neither of them provided any evidence for these claims.

Hazel: I have heard that the virus is produced in a laboratory. I think it may have been produced in a lab, but I don’t believe that China did it because Chinese has economic and human loss just like other countries.

Instructor: Who do you think may have produced this virus?

Hazel: Some rich families may have done it. They might have tried to control the growth of the population all over the world.

Instructor: Do these sources provide any evidence for this claim?

Hazel: They argue that the sources of the world is limited.

Instructor: This is a reason, not evidence.

Hazel: They mention a series taking part in Korea about this issue. A virus called coronavirus that affected lungs was produced in a lab in this series. Its symptoms are the same.

Instructor: Do you think the idea of an outbreak in a film can be a sort of evidence?

Hazel: I am not sure that it is an evidence, but I have suspicions. These incidents occur secretly. They don’t usually come to light. Does the lack of evidence imply that we should ignore these claims?

Instructor: One can argue that he/she does not have to provide evidence for the counterclaim, instead you need to provide evidence for this claim if you put it forth.

Emma put a very interesting claim forward and detected the claim that this virus is transmitted through respiration is a conspiracy theory. When the instructor asked why she replied that people are infected by contact with some objects rather than respiration, she also claimed that masks are exaggerated.

The second week of focus group interviews was carried with the discussions of fallacies and credibility of the sources of evidence. The discussions about this issue are presented in the following dialog:

Following these discussions, Megan asked the doctors’ explanations about antibiotics, Melissa questioned the relationship between the 5G technology and the pandemic, Mila talked about the usage of viruses as a biological weapon, and Emma discussed the claim that the novel coronavirus was produced to decrease the human population. The instructor suggested them to question the number of participants, the methods used in a study, and how researchers conclude these results. She told them to ask these questions besides the credibility of the sources while evaluating the claims.

Hazel: You told that an argument should be evaluated independent from the profession of a person. Fallacies may also be committed by experts. What are we supposed to do then? May I trust the explanations of Ministry of Health (MoH) in Turkey or World Health Organization (WHO)? Do I need to prove them too?

Instructor: You need to be careful about this issue. Of course, authority and expertise should be taken into account during evaluating a claim, but you need to draw a line here. If the explanations of an expert conflicts the data, then the expertise does not mean anything. We can trust the explanations of MoH and WHO because their explanations based on the data.

Hazel: A few websites mentions about a claim which is among the list that you gave us. These websites cited a book entitled “The Eyes of the Darkness” mentioning about an outbreak of a virus similar to novel coronavirus. I cannot find the book. Under these circumstances how can I judge the source and evaluate this claim?

Instructor: Do you remember the websites?

Hazel: Newspaper and fact checking websites.

Instructor: If a news provide evidence for the claims and have a reference list it can be trustworthy. Otherwise we always question the validity of the claims. Fact checking sites usually meet the requirement of testing a claim by checking it from different sources. Therefore, their trustworthiness is higher than newspapers.

The third week of focus group interviews again included the discussions of the evidence. The instructor reminded them of the importance of data and providing evidence while reasoning the data. She also emphasized that we need to question the methods of gathering data and the conclusions that were drawn based on data even if we cannot gather this data. Keira gave an example of photoshopped images of the series called The Simpsons to spread a conspiracy theory, Yoselin mentioned a fallacy about drinking traditional Turkish soup day and night to prevent the infection of coronavirus, Hillary suggested the scientific argument that novel coronavirus mutated from SARS-related coronavirus, Melissa demarcated fallacies from scientific arguments correctly, and Keira cited MoH and WHO during her explanations. The discourse among them and the instructor provided evidence that their evaluation of fallacies and sources of evidence increased throughout the implementation. However, a question that Mila asked revealed confusion about authority as quoted below:

The discussions carried out during focus group interviews seem to have been beneficial in participants’ analysis of claims; however, the analysis tasks revealed that they still have problems while evaluating claims and especially judging the reliability of the sources. Based on these findings, the researchers decided to apply the second stage of the implementation by assigning a second task to the participants whose average analysis scores of the 1st analysis task is below the moderate level (score of 2.0) in order to increase their analysis skills.

Hazel: I read a lot of explanations of different doctors about this issue? Can we trust them?

Instructor: Of course, we can trust an expert opinion. However, as I mentioned before, if only one doctor says the opposite of the other doctors this doctor needs to provide evidence for his/her claims.

Five of the participants including Hazel (2.49), Keira (2.78), Melissa (2.39), Mila (2.19), and Yoselin (2.58) had an average score above the moderate level (> 2.0) while the average scores of the rest including Abigail (1.83), Emma (1.9), Hillary (1.79), Karen (1.86), and Megan (1.29) were below the moderate level (< 2.0). The participants whose average scores of the 1st analysis task were below 2.0 were submitted a 2nd analysis task.

The 2nd Analysis Results

Five of the participants including Hazel (2.49), Keira (2.78), Melissa (2.39), Mila (2.19), and Yoselin (2.58) had an average score above the moderate level (> 2.0) while the average scores of the rest including Abigail (1.83), Emma (1.9), Hillary (1.79), Karen (1.86), and Megan (1.29) were below the moderate level (< 2.0). The participants whose average scores of the 1st analysis task were below 2.0 were submitted a 2nd analysis task. Table 6 illustrates the task that the five selected participants were assigned and their analysis level for each category for each claim.

Table 6

The participants’ analysis level for each category for each claim in the 2nd task

		Claim 1		Claim 2
Participant and task	Skills	Level	Explanation	Level	Explanation
Abigail (3)	Evaluation	2	She critically argued the claim that it is necessary to make high-stakes tests in an era of the pandemic by providing related evidence; however, some of the evidence were not found in the references that she listed.	2	She critically argued the claim that it is not necessary to make high-stakes tests in an era of the pandemic by providing related evidence; however, some of the evidence were not found in the references that she listed.
	Demarcation	2	She critically argued the risks of making high-stakes tests in an era of the pandemic; however, she did not discuss whether this decision is not based on scientific evidence.	2	She critically argued that it is necessary to postpone making high-stakes tests for a few months; however, she did not discuss why this argument is based on scientific evidence.
	Judgment	2	She utilized a variety of sources and judged the credibility of each source based on the plausibility of the claim rather than particular criteria.	2	She judged the credibility of the sources based on the quantitative data they involve; however, she also stated her confusion whether the given information is proved or not.
Emma (2)	Evaluation	3	She correctly explained a causal relationship between claim and evidence by reflecting a deeper analysis of claim and evidence while discussing the necessity of using a mask.	3	She correctly explained a causal relationship between claim and evidence by reflecting a deeper analysis elaborating the evaluation of claim and evidence while discussing why it is necessary not only to use a mask but also to take other precautions.
	Demarcation	2	She critically argued the necessity of using a mask; however, she did not detect the fallacy that using a mask is not effective for protection from the novel coronavirus.	2	She critically argued the necessary precautions for protection from the novel coronavirus; however, she did not detect the fallacy that using a mask is not effective for protection from the novel coronavirus.
	Judgment	3	She critically judged the credibility of each source that she utilized while researching the claim by comparing different sources of evidence.	3	She critically judged the credibility of each source that she utilized while researching the claim by comparing different sources of evidence.
Hillary (1)	Evaluation	3	She critically argued the benefits of curfews and lockdowns to the environment by discussing the relationship between claim and evidence and elaborating the evaluation of claim and evidence.	3	She critically criticized the excessive usage of plastics and the negative impact of their wastes on the environment by reflecting a deeper analysis of the balance of ecosystems.
	Demarcation	2	Despite her ability to critically evaluate each claim, she was likely to commit a fallacy about the optimistic approach by articulating that she agreed on the first claim more than the second.	2	Although she elaborated on the negative impact of the pandemic on wildlife and ecosystems, she was inclined to believe in the fallacious argument that the pandemic healed the Earth as she stated that she agreed on the first claim.
	Judgment	3	She critically judged the credibility of each source that she utilized while researching the claim by comparing different sources of evidence.	3	She critically judged the credibility of each source that she utilized while researching the claim by comparing different sources of evidence.
Karen (5)	Evaluation	3	She critically argued the relationship between the blood type and the infection rate of the virus by citing related research studies.	3	She critically argued that there is not enough evidence to claim that there is a relationship between the blood type and the infection rate of the virus, and further research is needed to draw such a conclusion.
	Demarcation	3	She clearly criticized the validity of the first argument by demarcating the conclusions that are based on scientific evidence and those that are based on insufficient data.	3	She demarcated the validity of the second claim compared to the first one by arguing that there is not enough evidence for the first claim.
	Judgment	3	She critically judged the credibility of each source that she utilized while researching the claim by citing the articles published in both the national and international peer-reviewed journals. She also emphasized that this claim is published in an Internet site distributing unpublished e-prints about health sciences.	3	She critically judged the credibility of each source that she utilized while researching the claim by citing the articles published in both the national and international peer-reviewed journals. She also judged that the articles published in peer-reviewed journals are more credible than unpublished ones.
Megan (4)	Evaluation	2	She evaluated the claim and related evidence correctly; however, her evaluation lacked any elaboration in terms of the consequences of online teaching.	2	She evaluated the claim and related evidence correctly; however, her evaluation lacked any elaboration in terms of the health conditions that can occur during face-to-face interaction in teaching.
	Demarcation	2	Although she evaluated each claim and evidence correctly, one cannot infer from her explanations of what scientific argument is.	2	Although she positioned herself in favor of face-to-face teaching, she could not construct a sound argument including rebuttals considering different aspects of the situation.
	Judgment	2	Despite her correct judgment about the credibility of sources and choice of peer-reviewed journals, she utilized only one source to evaluate each claim and evidence.	2	Despite her correct judgment about the credibility of sources and choice of peer-reviewed journals, she utilized only one source to evaluate each claim and evidence.

Evaluation of Claims and Counterclaims About COVID-19

As indicated in Table 6, three out of five participants (Emma, Hillary, and Karen) were able to evaluate both claim and counterclaim critically. These participants provided related evidence to their claim and explained a causal relationship between claim and evidence. Moreover, the participants elaborated on the evaluation of claims by reflecting a deeper analysis of the claim and discussing the other situations apart from their claims. They cited related research studies and decided whether the evidence was enough to evaluate this claim. However, the other two participants (Abigail and Megan) evaluated the claim and counterclaim at a moderate level. These participants evaluated claim and counterclaim by relating evidence correctly; however, they did not analyze their claims deeply and did not elaborate on their evaluation. Furthermore, although one of these participants (Abigail) evaluated her claims by citing various evidence, she did not include these citations in the reference list. It is noteworthy to observe that none of the participants made an evaluation at a low level. This finding reveals an increase in the evaluation level of the participants compared to the results in Table 5. Unlike their first analysis, they did not commit fallacious arguments while evaluating claims and evidence in their second analysis.

As shown in Table 6, four out of five participants (Abigail, Emma, Hillary, and Megan) were able to demarcate fallacies and conspiracy theories from scientific arguments at a moderate level. These participants were also able to demarcate the argument correctly but could not adequately discuss whether the argument was scientific or not, or they committed fallacies although they were able to demarcate the argument correctly. Karen was able to demarcate fallacies and conspiracy theories from scientific arguments and clearly criticized the validity of fallacious arguments based on scientific evidence. She also compared the claim and counterclaim and argued that there was not enough evidence to support the fallacious argument and the other one was based on scientific evidence. In short, despite the improvement in the participants’ demarcation of fallacies and conspiracy theories from scientific arguments, they were still challenged to justify whether the argument is scientific and even likely to commit fallacies while demarcating the arguments.

In the first phase of the intervention, it was inferred that participants seemed to have been challenged in judging the credibility of the sources. As seen in Table 6, the majority of participants were able to judge the credibility of the sources critically in the last phase of the intervention. Three out of five participants (Emma, Karen, and Hillary) critically judged the credibility of each source that they utilized during research. These participants generally utilized more than one source, and they compared each of them while they were judging the credibility of sources. In addition, the other two participants (Abigail and Megan) were able to evaluate the sources which they used throughout their research at a moderate level. Although Megan correctly judged the source she used throughout the research, she used only one source to evaluate the claims and counterclaims. Although Abigail used more than one source when evaluating the claims, she judged those sources based on plausibility rather than objective criteria. In other words, she thought that the source is credible since the claims they suggest are plausible. In addition, although Abigail started her argument by mentioning that quantitative data is a criterion for the credibility of the sources, she seemed to have been confused during the rest of her argument. She argued that the information cannot be trustworthy unless it is proven. However, she also got the position that the information given in this source is logical; thus, she can rely on it. In sum, different from the participants’ first analysis, they began to search not only educational, organizational, and fact-checking websites but also national and international articles published in peer-reviewed journals despite their confusion about the judgment of the credibility of the sources.

Discussion

Understanding Uncertainty in Science

The pandemic context has revealed the necessity of increasing students’ understanding of uncertainty in science and developing their critical thinking skills and ability to engage in argumentation (Erduran 2020b). Lee et al. (2020) stressed that students have less tolerance to deal with uncertainty and suggested educators illustrate the uncertainty in science, discussing how to deal with these issues with students and creating more collaborative classroom cultures to deal with this intolerance. Uncertainty of controversial issues, such as COVID-19, requires developing innovative methods that facilitate the critical evaluation of claims and evidence. This study is an attempt to deal with learners’ intolerance for uncertainty in science and encourage them to critically evaluate the claims and evidence.

Since uncertainty is inherent in science, individuals are constantly faced with logical fallacies when discussing, thinking, and confronting a claim in daily life. The failure to recognize these fallacies will lead to misdirection to make informed decisions (Christoforides et al. 2016). It is important to include logical fallacies encountered in daily life in the course in a planned and programmed manner because determining logical fallacies is one of the critical thinking skills (Ricco 2007) and necessary to be a scientifically literate citizen (Roberts 2007). The results of this study also show that pre-service teachers benefited from discussing logical fallacies in terms of evaluating the claims, demarcating the fallacies and conspiracy theories from scientific arguments, and judging the credibility of sources.

The findings of the 1st analysis task revealed that the participants argued some of the claims without basing on evidence and sometimes applied fallacies, biases, and conspiracy theories even if they correctly demarcated fallacies and conspiracy theories from scientific arguments. Although these fallacies and conspiracy theories were also mentioned and discussed during focus group interviews, the participants seemed to have continued to create fallacious arguments even after these interviews. Their 2nd analysis task indicated that their evaluation of claims and demarcation of fallacies and conspiracy theories from scientific arguments as well as the variety and trustworthy sources they utilized improved throughout the intervention. In sum, encouraging pre-service teachers to analyze claims about an uncertain phenomenon seems to have had a positive impact on their evaluation of arguments and utilization of various sources.

Justification of Arguments and Judgment of Credibility of Sources

Despite the development of pre-service teachers’ evaluation, demarcation, and judgment throughout an online course in which collaborative discourse and instructor’s feedback were implemented, their analysis still lacked clear justification of their arguments and rebuttal for counterargument. Liu and Roehrig (2019) stated that although science teachers grounded their arguments about controversial issues on evidence, the evidence was insufficient to justify the claims. On the other hand, there are deficiencies in evaluating the credibility of the evidence that pre-service teachers used. Saribas and Akdemir (2020) found that pre-service elementary teachers listed a limited number of criteria to make judgments about the trustworthiness of evidence. The results of the current study are also consistent with the findings of these studies by indicating the need to overcome the abovementioned drawbacks about pre-service teachers’ argumentative skills especially considering justification of their arguments and judgment of the credibility of sources on controversial issues in this post-truth era.

In this post-truth era, access to information has evolved with technological advances and become a part of daily life. In particular, individuals who were born after the technological revolution differ from previous generations in terms of accessing information. The young generation obtains information and news from online platforms and shares them there (Nygren et al. 2019). Considering the lack of a control mechanism in online platforms and the fact that everyone can share any information at any time, unlike traditional media, the credibility of these information becomes crucial. Even though Internet access has removed the borders between people, it also leads to the dissemination of information disorders containing misinformation and disinformation, fake news, and speculation (Vosoughi et al. 2018). Young students are highly skilled in accessing information online, but they have trouble evaluating these information (Wineburg et al. 2016). There are a number of studies that used various methods to enable students to judge the credibility of the claims. These studies indicated that traditional methods are insufficient to provide skills for evaluating the claims (Parsazadeh et al. 2018). The researchers of this study suggest designing educational settings containing discussions on fallacious claims on online platforms as well as scientific arguments on controversial issues such as health issues or climate change to help students in K-12 grade levels as well as pre-service teachers understand uncertainty in science. In addition, there is no digital tool or online service that can reveal whether the sources are credible in this period when information is spreading rapidly online (Faraon et al. 2020). Therefore, it is also necessary to provide students with methods to facilitate the judgment of the credibility of the sources.

Dealing with Misinformation and Disinformation

The findings of this study revealed an improvement in pre-service teachers’ analysis of claims including evaluation of each claim, demarcation of fallacious arguments from scientific arguments, and judgment of the credibility of sources throughout this online course. Argumentation is a form of discourse that needs to be taught through appropriate instruction and used by students (Jiménez-Aleixandre and Erduran 2007). The discourse was implemented in this study through the instructor’s and participants’ discussions during focus group interviews as well as the instructor’s written feedback to the analysis task and oral feedback via an online course or social media by answering their questions. According to Duschl (2007), the feedback on thinking can come from both the teacher and the students themselves; however, it is the teacher who creates the environment to support students’ scientific thinking and reasoning. Formative feedback from teachers, thus, plays an important role in encouraging students’ scientific reasoning. The results of this study supported this view by enhancing pre-service teachers’ evaluation of claims, demarcation of scientific arguments from fallacies and conspiracy theories, and judgment of sources by implementing the focus group interviews and instructor’s continuous feedback to the pre-service teachers’ analysis. Zhu et al. (2017) found that automated scoring and feedback that was integrated into an online module supported students’ construction of written arguments about the factors that affect climate change. The results of the study presented in this paper also support the view that providing online feedback is a beneficial method to improve the learners’ argumentative skills on controversial issues such as COVID-19. Since misinformation and disinformation spread along with online environments, educators’ online feedback seems to be promising to deal with this misinformation and disinformation.

Conclusion

The study presented here examined pre-service teachers’ analysis of claims about COVID-19. For this purpose, the researchers of this study first examined the pre-service teachers’ level of evaluation of claims about COVID-19 throughout an online course by the 1st and 2nd analysis tasks and focus group interviews. The results revealed that the majority of the participants were able to evaluate the claims at a moderate or low level before the implementation. After the 1st analysis task, the researchers found that the number of the evaluation level of the claims for each participant increased. However, they also found that participants still evaluate the claims in mostly moderate and low levels. Following the 1st analysis task, they conducted focus group interviews to discuss the claims with the participants who had difficulties to evaluate the claims. Focus group interviews showed that some participants still had some problems while evaluating claims. The researchers asked the participants who had problems while evaluating claims to complete the 2nd analysis task. The findings of the 2nd analysis task indicated that only two participants were able to evaluate the claims at a moderate level and the other participants were able to evaluate the claims at a high level. It can be concluded from this result that the intervention implemented in this study has contributed to the pre-service teachers’ evaluation of the claims about COVID-19.

Second, the researchers explored the level of pre-service teachers’ demarcation of fallacies and conspiracy theories from scientific arguments about COVID-19 throughout the course. The results of the 1st analysis task showed that the demarcation levels of the participants were mostly at moderate and high levels before the implementation. In the 2nd analysis task, the participants were able to demarcate scientific arguments from fallacies and conspiracy theories correctly, but they also commit fallacies and apply conspiracy theories when they were demarcating them from scientific arguments. Furthermore, the participants were still challenged to justify whether the claims are scientific or not.

Finally, the researchers investigated the level of pre-service teachers’ judgment of the credibility of sources of the claims about COVID-19 throughout the intervention. They found that pre-service teachers’ judgment on the credibility of the sources was mostly at a low level in the 1st analysis task. Although there was an increase in their judgment level in the 2nd analysis task, they were rarely able to judge the credibility of the sources at a high level. It is also evident from the results of the focus group interviews that the participants still had difficulties in judging the sources. The 2nd analysis task revealed that the variety and the trustworthiness of the sources that the participants utilized increased. However, some of them still had difficulties while judging the credibility of the sources. These participants mentioned plausibility and proof while judging the sources.

Limitations and Suggestions

The study presented here contains the following limitations. First of all, we carried out this study after a pandemic which spread all over the world. For this reason, we conducted this study at a time when students continued online education obligatory. However, the participants have not taken online courses before and therefore did not have experience of online courses. In addition, the researchers were not able to gather data through face-to-face interactions with the participants. Thus, the data sources came from focus group interviews and written analysis tasks.

The research presented in this paper is based on analyzing the participants’ evaluation, demarcation, and judgment of claims and sources. Since the results of this study indicate that the participants still lacked clear justification of their arguments and rebuttal for counterargument and judgment of the credibility of sources on controversial issues, it can be inferred that justification and rebuttal of arguments as well as judgment of the credibility of sources should primarily be focused in further studies. Further research focusing on the improvement of the participants’ judgment of the credibility of sources and its impact on their ability to justify their arguments and refute counterargument may bring new light to this issue.

In order to educate scientifically literate citizens, researchers of this study suggest including the credibility of the sources in the science curriculum and making deliberate attempts to help students acquire the ability to judge the credibility of the sources while evaluating claims about controversial issues. This study is one of the initial attempts to increase pre-service teachers’ scientific literacy in the context of COVID-19. Considering the uncertainty in controversial issues, such as the case of this study, COVID-19, further research studies investigating the different methods in teaching controversial issues, i.e., implementing the collaborative discourse and instructor’s feedback on the credibility of sources while evaluating claims, will bring new insight in learners’ understanding of uncertainty in science and ability to engage in argumentation.