Effects of Reading Interventions on Student Understanding of and Misconceptions about Antibiotic Resistance.

Students possess informal, intuitive ways of reasoning about the world, including biological phenomena. Although useful in some cases, intuitive reasoning can also lead to the development of scientifically inaccurate ideas that conflict with central concepts taught in formal biology education settings, including evolution. Using antibiotic resistance as an example of evolution, we developed a set of reading interventions and an assessment tool to examine the extent to which differences in instructional language affect undergraduate student misconceptions and intuitive reasoning. We find that readings that confront intuitive misconceptions can be more effective in reducing those misconceptions than factual explanations of antibiotic resistance that fail to confront misconceptions. Overall, our findings build upon investigations of intuitive reasoning in biology, examine possible instructional interventions, and raise questions about effective implementation of reading interventions in addressing persistent misconceptions about biology.

INTRODUCTION

In recent years, scholars have directed increasing attention in biology education research to intuitive thinking—informal, instinctive ways of reasoning about the world (1–4). Our own past research has demonstrated that, in some cases, intuitive reasoning can lead to misconceptions about biological concepts (4–6). Misconceptions are persistent misunderstandings that arise as students make sense of new information in the context of existing knowledge frameworks (3, 7). These misconceptions are often associated with specific types of intuitive reasoning. The current study investigates how one type of intuitive reasoning—teleological thinking—might influence student misconceptions about antibiotic resistance.

Relationships between teleological thinking and biological misconceptions

Teleological thinking occurs when an individual conceives of an outcome as goal-directed (e.g., plants produce oxygen so that animals can breathe) (8). It emerges early in development (9, 10), persists into adulthood (2, 5), and is even evident in PhD-level scientists when responding under time pressure (11). As such, teleological thinking is a pervasive and compelling form of intuitive reasoning.

Students who spontaneously generate teleological explanations for biological concepts also tend to agree with teleological misconceptions (4, 8, 12). In particular, teleological thinking has been linked to misunderstandings about evolution (10, 12, 13) and natural selection (14), a concept commonly misunderstood among students across levels of expertise (13, 15). Moreover, one example of natural selection, antibiotic resistance—whereby random genetic variations allow some bacteria to survive antibiotics and reproduce to form a resilient population—is highly relevant to individual and public health (16–18) and susceptible to teleological thinking (12); making it an important domain to interrogate both misconceptions and possible interventions.

Strategies to address intuitive misconceptions about biology

Although numerous studies have established the ubiquity of intuitive biological misconceptions, fewer have examined methods to avoid or correct these misconceptions in the classroom. One study showed that changing the framing of biological misconceptions can impact student agreement, elaborations, and/or corrections of misconceptions, demonstrating the viability of intervention on intuitive misconceptions (19). In an intervention with elementary school children, teaching evolution with a storybook that was specifically focused on avoiding teleological reasoning reduced teleological thinking and improved student understanding (20).

Metacognition refers to a student’s general ability to attend to and reflect on their own thought processes (21). Inducing metacognition is one intervention that has been shown to be beneficial to student learning across disciplines (22), including biology (3) and other domains of science (for reviews, see references 23, 24). Students who demonstrate higher metacognition also show higher science reading comprehension (25), greater knowledge after a science learning activity (26), and general problem-solving skills (27). Metacognition has also been linked to intuitive thinking (28, 29), and it has been argued elsewhere that metacognition can specifically reduce teleological thinking in evolution (30, 31), although empirical work is lacking.

One type of intervention that has been used to address biological misconceptions is known as refutation text, which highlights a common misconception and then directly refutes it, providing correct information to supplant the misconception (see references 32 and 33). We see this method as metacognitive because it focuses the learners’ attention on their own thought processes and understanding as well as the target biological concepts, thereby compelling the learner to reflect on the friction between what they understand and scientific information (e.g., references 7, 32); but see (34–37). Empirically, refutation texts have been shown to improve both short- and long-term science learning compared to other types of interventions (32, 33). Recent studies have found that these strong effects for refutation texts hold in important topics such as climate change (38) and evolution (39). Thus, refutation texts present a promising intervention to address biological misconceptions associated with teleological thinking, yet there is a gap in the literature regarding their effectiveness in this context. In addition, there is an open question as to the most effective way to use refutation text to address misconceptions, whether to use a text that focuses on scientific language or one that draws attention to and refutes intuitive reasoning.

In the present study, we examined how different reading interventions affected student misconceptions and production of intuitive reasoning by presenting students in an advanced undergraduate biology class with readings about antibiotic resistance at two time points during the semester. At Time 1, students read a short article on antibiotic resistance with either (i) teleological language, (ii) no teleological language, or (iii) language that explained and then countered teleological misconceptions. At Time 2, the same students read one of two new articles on antibiotic resistance containing language intended to induce metacognition by either (i) refuting the misconceptions by explaining why they are scientifically inaccurate or (ii) refuting the misconceptions by explaining intuitive reasoning. At both time points, student-generated explanations of antibiotic resistance and agreement with a teleological statement were collected both before and after reading. We addressed two research questions: (i) How do different reading interventions—emphasizing scientific, teleological, or metacognitive language—impact students’ use of intuitive reasoning and acceptance of misconceptions about antibiotic resistance? (ii) How do different types of metacognitive refutation texts, refuting either scientific or teleological misconceptions, impact students’ use of intuitive reasoning and acceptance of misconceptions about antibiotic resistance?

Methods

Participants and recruitment

Participants were advanced biology majors taking a required course at a large, urban university. This course is the last of three core biology courses at this institution, which is required of all biology majors across multiple specialized concentrations. It was decided to obtain a large, advanced student sample across all specialized concentrations, as this course is populated primarily by students with junior and senior status (for a similar sampling strategy, please see reference 12; 40, 41). The content of the course was focused on genetics, and the topic of antibiotic resistance was not included in the course. The pedagogical approaches of the course included regular active learning and were taught by faculty alumni of professional development efforts in scientific teaching on the campus (42).

The first coauthor (S.B.P.) collected data during lecture sections at two time points, where an assessment was given as part of an in-class activity, with permission of the course instructor. All participants were given the option to opt out of having their responses including in the study. Data were collected across the two sections of the course, one section with 66 students enrolled and another with 47, for a total of 113 students in the course. Since not all students were present on each data collection day, 101 and 102 students were invited to participate at time points 1 and 2, respectively. Students who were not present for both time points were excluded from analysis. Student assessments were also excluded for noncompliance (10% Time 1; 7% Time 2) or if students did not identify as biology majors (8% in Times 1 and 2). After applying these inclusion criteria, the participation rate for this study was 62% and 63% for time points 1 and 2, respectively. Sixty-four participants were included in the analyses and comparisons described below. The final study population consisted of 78% women (50/64) and 82% students of color (53/64).

Reading interventions

To examine the extent to which instructional language affects student misconceptions and production of intuitive reasoning, we developed a set of reading interventions, presented at two points during the semester. These consisted of a short article framing antibiotic resistance in contrasting ways. At Time 1, we presented three framing conditions: (1) Reinforcing Teleology (T): used phrasing thought to underlie teleological misconceptions about evolution and antibiotic resistance; (2) Asserting Scientific Content (S): explained antibiotic resistance in a manner that would be acceptable to biology experts and avoided intuitive language; and (3) Promoting Metacognition (M): directly addressed teleological misconceptions and countered them with an explanation of antibiotic resistance without intuitive language (see Appendix 2 in the supplemental material). An illustration was included at the end of the reading to illustrate the concepts with legends that mirrored the framing of each reading condition.

At Time 2, we presented two conditions, intended to further explore the metacognitive intervention from Time 1: (i) the Alerting to Misconceptions (MIS) condition and (ii) Alerting to Intuitive Reasoning (IR) condition. The new readings were based on text from reading M (Time 1), but differed in that they described and refuted common student misconceptions about antibiotic resistance. For the MIS condition, the misconceptions were refuted with an explanation of why they are scientifically inaccurate. In the IR condition, the statements were refuted with an explanation of intuitive reasoning. These readings were also reformatted from those at Time 1, including bolded subheaders and without the cartoon (see Appendix 2). During data collection, students were told that their previous responses had informed the development of the new readings. Students were randomly assigned to conditions at both time points, and consequently their assignment at Time 1 was independent of their assignment at Time 2.

Assessment tool

To examine student ideas, we adapted a written assessment tool previously developed by Richard et al. (12) (see Table 1; Appendix 1), and checked the face validity of the assessment prompts with a subset of students. Students were presented with two target prompts both before and after each reading intervention. First, students were presented with an open-ended prompt (“How would you explain antibiotic resistance to a fellow student in this class?”) to assess their ideas and explanations of antibiotic resistance without any additional cues. Next, they were presented with a teleological misconception prompt (“Individual bacteria develop mutations in order to become resistant to an antibiotic and survive”). For this prompt, participants indicated their agreement along a 4-point Likert scale and explained their choice in writing. As described in Richard et al. (12), we predicted that this would prompt participants to consider the accuracy of the idea that bacterial mutation occurs with the purpose or goal of survival.

TABLE 1

Pre- and post-reading assessments completed by student participants

Pre-Reading Assessments1. “How would you explain antibiotic resistance to a fellow student in this class?”†2. “Individual bacteria develop mutations in order to become resistant to an antibiotic and survive.”‡† —Hypothesized teleological statementPost-Reading Assessments3. “What key ideas did you take away from the reading?”†4. “How would you explain antibiotic resistance to a fellow student in this class?”†5. “Individual bacteria develop mutations in order to become resistant to an antibiotic and survive.”‡† —Hypothesized teleological statement6. “The reading changed my mind about how antibiotic resistance works.” ‡†

In the space below, please explain with as much detail as possible.

“Please read the statement below and circle a response on the scale: Strongly Disagree, Disagree, Agree, Strongly Agree”

Data collection

The procedure was identical at both testing times. Students were informed that the assessment addressed the topic of antibiotic resistance and was part of their class activity for the day, but was not a test or assignment with any impact on their grade. Students were ensured that their responses would be anonymous and that only researcher S.B.P. (not the course instructor) would be reading their responses. All student participants gave informed consent. Researcher S.B.P guided participants through the assessment one question at a time and students were given 3–4 min to complete each written item.

The reading intervention was distributed at random. Participants were informed that there were multiple versions of the article and that each version explained antibiotic resistance in a different way. Students first completed the open-ended and teleological prompts, then proceeded to read the passage. To encourage students to read carefully and actively, they were also informed that they could write notes on the article and were alerted to the next step, in which they were asked to list key ideas from the reading. After listing key points, students again completed the open-ended and teleological prompts. Finally, they were asked to reflect on the impact of the reading on their thinking about antibiotic resistance. Overall, the completion of the assessment took approximately 45 min.

Demographic survey

Upon completion of the in-class assessment, students were asked to provide information about their personal and educational backgrounds, including self-identified gender and ethnicity. This allowed us to understand the makeup of the participating student population.

Data coding and analysis

(i) Coding of intuitive reasoning from student writing. Student’s written explanations for the open-ended and misconception prompts were analyzed for the presence of three types of intuitive thinking. Based on previous work (e.g., reference 4), we defined intuitive thinking as language consistent with teleological (assuming a purpose, goal, or function as cause for a change or event), essentialist (assuming that members of a categorical group all share an underlying property or “essence” that unifies them), and anthropic (anthropomorphizing, or attributing human characteristics to organisms or objects, reasoning by analogy to humans by default, considering humans to be special or unique relative to other biological organisms) thinking. Examples given in Appendix 3. Each response was coded by S.B.P. based on an established rubric used to identify intuitive reasoning (4, 12). A second researcher (K.D.T.) independently coded a random sample of 10% of responses for each prompt. All coding was done with coders unaware of the condition for each response. All qualitative coding reached 80% or greater consensus.

(ii) Quantification of student agreement on the teleological misconception prompt. Student agreement with the teleological misconception statement was scored based on responses to the Likert scale, coded categorically as “agree” (rating of 3 or 4) or “disagree” (rating of 1 or 2).

Statistical analysis

To look at differences in student responses in the pre- and post-assessments for each intervention condition, we used a 2 × 2 McNemar’s test. All statistical comparisons were made using JMP software, version 14W (SAS Institute Inc., Cary, NC).

Human subjects approval

This study was approved by the Institutional Review Board of San Franscisco State University on May 17, 2018 under protocol number E18-128a.

RESULTS

The results of our analyses are presented below in the order in which participants responded to the assessment prompts.

Time 1: Investigating scientific, teleological, and metacognitive reading interventions

At Time 1, students received one of 3 intervention articles: “Asserting Scientific Content” (S; n = 17); “Reinforcing Teleology” (T; n = 20); and “Promoting Metacognition” (M; n = 27). We examined how variations in language and framing in these articles (see Methods) influenced student reasoning assessed before and after presentation of the articles.

Shifts in student responses to the open-ended prompt: “How would you describe antibiotic resistance to a fellow student in this class?”

The first assessment item (Table 1; Appendix 1) asked participants to complete a written explanation of antibiotic resistance, allowing us to examine students’ production of intuitive reasoning without any particular prompting (Fig. 1, Table 2). Overall, before reading any intervention, 77% of all students produced some form of intuitive reasoning in their explanations. Prior to the intervention, fewer students in condition S produced intuitive language, compared to students in condition T and condition M; however, the difference was not significant. Following the reading intervention, we observed a decrease in the number of students producing intuitive language (45% overall). Although decreases were observed for each intervention group, the difference was significant only among students in the M condition (χ2 = 9.31, n = 27, P = 0.0023).

FIG 1

Time 1. Percentage of students producing intuitive language in response to the open-ended prompt: “How would you explain antibiotic resistance to a fellow student in this class?” A significant difference was observed between student pre- and post-assessment responses for students in the M reading group. *, P < 0.05. No significant difference was observed between students’ pre- and post-assessment responses for students in groups S and T.

TABLE 2

Student production of intuitive language and agreement with the teleological misconception statement (total N = 64)

Time 1	Produced intuitive language on open-ended prompt		Endorsed misconception on teleological statement		Produced intuitive language on teleological statement
Reading Intervention	Pre-reading	Post-reading	Pre-reading	Post-reading	Pre-reading	Post-reading
Asserting Scientific Content (S)	65% (n = 11)	35% (n = 6)	59% (n = 10)	59% (n = 10)	29% (n = 5)	12% (n = 2)
Reinforcing Teleology (T)	80% (n = 16)	60% (n = 12)	70% (n = 14)	70% (n = 14)	50% (n = 10)	45% (n = 9)
Promoting Metacognition (M)	81% (n = 22)	41% (n = 11)	56% (n = 15)	37% (n = 10)	52% (n = 14)	22% (n = 6)

Shifts in student responses to the teleological misconception prompt: “Individual bacteria develop mutations in order to become resistant to an antibiotic and survive”

To investigate the extent to which students would initially embrace misconceptions hypothesized to be rooted in teleological reasoning, we presented students with a teleological challenge statement (Fig. 2). Using a Likert scale, students indicated their agreement or disagreement with the statement and explained their rationale. In the pre-assessment, most students across intervention conditions indicated their agreement with the misconception statement (61%) (Fig. 2, Table 2). Following the reading intervention, we observed a slight decrease in agreement among all students (53%). This difference was driven primarily by a decrease in agreement among students in condition M; however, the decrease was not significant. Students in conditions S and T did not exhibit a shift in agreement. Overall, student endorsement of the misconception statement was not affected by the reading intervention.

FIG 2

Time 1. Student agreement and production of intuitive language in response to the teleological misconception statement “Bacteria develop mutations in order to become resistant to an antibiotic and survive.” No significant difference was observed in student agreement with the challenge statement for any reading group.

Time 2: Investigating misconception and intuitive reasoning reading interventions

At Time 2, students received one of two new intervention readings: “Alerting to Misconceptions” (MIS; n = 34) or “Alerting to Intuitive Reasoning” (IR; n = 30), which differed in how the misconceptions were specifically rebutted (see Methods). We again examined how the new reading interventions influenced student reasoning and endorsement of misconceptions.

Student production of intuitive reasoning was assessed by examining written explanations of antibiotic resistance on the first assessment prompt (Fig. 3, Table 3). In the pre-assessment, 45% of all students produced some form of intuitive reasoning in their explanations. The percentage increased slightly in the post-assessment following the reading intervention (52%). Upon examining the two intervention groups individually—MIS and IR—we observed no significant difference in student use of intuitive language. We did note that student use of intuitive language at Time 2 was similar to that of the post-assessment findings at Time 1. This suggests that the second iteration of the readings did not further influence student reasoning related to this prompt.

FIG 3

Time 2. Percentage of students producing intuitive language in response to the open-ended prompt “How would you explain antibiotic resistance to a fellow student in this class?” No statistically significant differences were observed in either intervention group.

TABLE 3

Student production of intuitive language and agreement with the teleological misconception statement (total N = 64)

Time 2	Produced intuitive language on open-ended prompt		Endorsed misconception on teleological statement		Produced intuitive language on teleological statement		Rejected the teleological inaccuracy
Reading intervention	Pre-reading	Post-reading	Pre-reading	Post-reading	Pre-reading	Post-reading	Pre-reading	Post-reading
Alerting to Misconceptions (MIS)	38% (n = 13)	50% (n = 17)	59% (n = 20)	21% (n = 7)	26% (n = 9)	32% (n = 11)	32% (n = 11)	62% (n = 21)
Alerting to Intuitive Reasoning (IR)	53% (n = 16)	53% (n = 16)	60% (n = 18)	7% (n = 2)	33% (n = 10)	17% (n = 5)	20% (n = 6)	50% (n = 15)

We next investigated the extent to which students would embrace misconceptions hypothesized to be rooted in teleological reasoning before and after the reading at Time 2, by presenting them with the teleological challenge statement (Fig. 4, Table 2). Initially, 59% of students indicated their agreement with the statement (comparable to the Time 1 pre- and post-assessment levels, 61% and 53%, respectively). By contrast, we observed a large decrease in agreement on the post-assessment, where only 14% of all students endorsed the statement. When examined by reading intervention groups, there were significant decreases in agreement from pre- to post-reading assessment for both the MIS and IR groups (MIS: χ2 = 11.27, n = 34, P < 0.0001; IR: χ2 = 16.00, n = 30, P = 0.0008).

FIG 4

Time 2. Percentage of students endorsing the teleological misconception statement “Bacteria develop mutations in order to become resistant to an antibiotic and survive.” Student agreement with the statement was significantly lower following both reading interventions. **, P < 0.001.

Rejection of the teleological misconception

Following the reading intervention, we observed a significant decrease in student endorsement of the teleological misconception. What changes in student thinking might accompany this shift? In a previous study, we examined relations between understanding of antibiotic resistance and teleological reasoning in faculty as well as students (12). Unlike the undergraduate students, few faculty participants endorsed the teleological misconception statement, and none of them produced teleological language in their written responses. Moreover, we observed that faculty tended to explicitly refute the teleological inaccuracy of the challenge statement, pointing to the incorrect notion that bacterial mutations occur in a purposeful, intentional, or goal-oriented way. They also acknowledged that mutations are random and exist prior to antibiotic treatment and that antibiotic resistance and bacterial survival occurs because of genetic variation in bacterial populations.

In the Time 2 student responses, we observed a dramatic decrease in endorsement of the misconception statement following the reading interventions, but no shift in intuitive reasoning, suggesting a lack of association between the two. To further explore the nature of students’ responses, we examined the extent to which participants in this study explicitly rejected the teleological inaccuracy of the misconception statement (i.e., responses referencing the idea that mutations (i) occur randomly or by chance and/or (ii) are not purposeful, a choice, or goal oriented). To do so, we qualitatively analyzed their written responses to the teleological misconception prompt (Fig. 5, Table 3). Prior to the reading intervention, 27% of students rejected the teleological inaccuracy. Our analysis of the post-assessments revealed that significantly more students rejected the teleological inaccuracy following the reading intervention (56%). The increase was statistically significant for both reading intervention groups (MIS: χ2 = 7.12, n = 34, P = 0.0076; IR: χ2 = 7.36, n = 30, P = 0.0067). Representative quotes from students in each reading intervention condition are shown in Table 4.

FIG 5

Time 2. Percentage of students rejecting the teleological inaccuracy of the misconception statement in their written responses. Rejection of the teleological inaccuracy was significantly higher among postintervention responses for students in both reading intervention groups. **, P < 0.001.

TABLE 4

Examples of student rejection of the teleological inaccuracy in response to the teleological misconception prompt following the reading intervention

Student quotesAlerting to misconceptions (MIS) reading	Student quotesAlerting to intuitive reasoning (IR) reading
“These bacteria that are resistant to a drug have existing mutations that randomly occur in bacteria's DNA. Bacteria cannot actively create a mutation to become resistant to an antibiotic for survival.”	“The mutations were already there. The bacteria that started out with these changes are the ones that survived and were able to pass down resistance to all offspring. Mutations that grant resistance to new antibiotics are few and far between, and it never occurs to meet that specific goal.”
“Mutations do not develop as a survival response. Mutations are random and pre-existing mutations may cause antibiotic resistance.”	“The statement makes it seem that bacteria choose to change their DNA to become resistant and survive, that is not true! Bacteria cannot deliberately choose to have mutations or the specific kind of mutations. Genetic mutations are random and not all mutations lead to antibiotic resistance.”
“They cannot develop mutations in order to become resistant to an antibiotic and survive. Some bacteria are already ‘born’ with the ability to fight off and live in an antibiotic environment.”	“The bacteria don't necessarily mutate in order to survive. A small population of them are born genetically different and happened to be immune to the antibiotic being used against them.”
“The bacteria develop these mutations, however it is not intentionally in response to an antibiotic. The mutations were developed earlier and just enable the bacterium to survive exposure to [an] antibiotic.”	“Mutations cannot be developed willingly or on command, they happen randomly and by chance. Bacteria do not develop mutations to become resistant to antibiotics, they acquire mutations by chance that allow them to survive after being exposed to the antibiotic making them resistant.”

Discussion

Intuitive ways of knowing can inadvertently lead students to scientifically inaccurate thinking, as when teleological thinking leads students to misconceive antibiotic resistance as a goal-oriented process of evolution. Although the connection between intuitive reasoning and misconceptions about biological concepts has been well-documented by education researchers, much less attention has been focused on investigating interventions aimed at addressing such intuitive misconceptions. In this study we begin to address this gap by investigating classroom reading interventions aimed at teleological misconceptions about antibiotic resistance, with particular emphasis on metacognition-promoting refutation texts. We sought to answer the following research questions. (i) To what extent do reading interventions affect students’ use of intuitive reasoning? (ii) To what extent do reading interventions affect student acceptance of misconceptions? We found variable efficacy in using a reading intervention to shift student thinking. In the following sections, we explore our findings, along with the implications of our results and potential future research directions.

Not all intervention readings are effective in shifting student conceptions

At Time 1, we observed varying degrees of change among student written responses and agreement with the teleological statements before and after the intervention readings. While student agreement with the teleological misconception statement remained largely unchanged by any reading intervention, student-generated explanations to both prompts used significantly less intuitive language following reading intervention M, but not after reading interventions S and T. These findings are largely consistent with our predictions. Because reading M explicitly acknowledged and refuted an intuitive reasoning-based misconception, students may have been more likely to recognize the inaccuracy in their prior explanations and make corrections (43, 44), engendering less intuitive language in their postintervention responses. In contrast, we found that student explanations were relatively unchanged following reading S, consistent with literature showing that expository texts are less effective in shifting conceptual thinking compared to metacognitive refutation texts (see references 32 and 43).

Although we observed shifts in student use of intuitive language following reading M, we were surprised by the lack of robust decrease in student agreement with the teleological misconception statement. Reading M may not have clearly directed students’ attention to produce the necessary cognitive conflict for conceptual change (see references 33 and 45). For example, although students were told that they would have plenty of time, perhaps they skimmed and did not read carefully, or were not engaged by the refutation of the misconception. As a result, students’ reading strategies or experiences may not have been sufficient for facilitating metacognition and conceptual change based on the text content (46).

Reading interventions designed to more explicitly promote metacognition appear to influence student thinking

Motivated by the inconsistent impact of reading M, new reading interventions were implemented at Time 2, where we observed more significant shifts in student thinking. This included both a large decrease in student agreement with the teleological misconception statement as well as an increase in responses that pointedly rejected the teleological inaccuracy of the statement after the readings. The differences between the results at Times 1 and 2 could be due to changes to the content and/or structure of the interventions. In the MIS reading, the teleological misconception was stated and refuted with an explanation of the scientific inaccuracy of that thinking. This updated content may have helped students to identify the inaccuracy of the misconception statement as well as in their own thinking, leading to a correction in their post-reading responses. In the IR reading, the statement was also refuted with an explanation of intuitive reasoning. This may have allowed students to recognize both the inaccuracy and a common pattern in their reasoning underlying that inaccuracy.

The structure of the updated readings was also designed to more explicitly highlight the teleological misconception and refutation. This included visual cues like headings to alert students to the focus of each paragraph, and bullets to draw attention to key ideas. These changes might have helped to direct students to the refutation content, more effectively facilitating a recognition of inaccuracies in their own thinking, and as a result, a corresponding shift in their agreement and written explanations. Previous studies have suggested that structure is an important element of the refutation text as a guide for reader attention (41, 47–49). The importance of drawing reader attention, particularly to the misconception, is likely significant for promoting metacognition as part of cognitive conflict and conceptual change (33–35). The increased efficacy of the updated readings, based on our findings, contributes to existing literature suggesting that metacognition reduces intuitive thinking, including teleological thinking about evolution (28–31).

Conclusions, limitations, and future directions

Our findings indicate that refutation texts have the potential to shift student misconceptions rooted in persistent intuitive reasoning. Refutation text readings provide students with metacognitive opportunities, allowing them to recognize their misconceptions and integrate more scientifically accurate ideas. However, our findings are also complicated by the nature of the study design. Because the same population of students participated at both time points, we cannot rule out possible interactions of the two experiences on student responses.

Beyond exploring variations on the implementation of a refutation text reading intervention, few studies have addressed the efficacy of a refutation reading relative to forms of live instruction, such as demonstrations and active learning, that can be used to promote metacognition (50) and significantly increase learning gains across STEM disciplines (51, 52). Some evidence suggests that refutation texts may be more effective in combination with pre-reading activities or other forms of instruction designed to promote conceptual change (33, 46, 53–56). Future studies could continue to investigate the efficacy of different classroom instructional approaches, particularly active learning approaches, in combination with and comparison to refutation texts, to reduce misconceptions and improve student learning.