Enhancing Conceptual Knowledge of Energy in Biology with Incorrect Representations.

Energy is an important concept in all natural sciences, and a challenging one for school science education. Students' conceptual knowledge of energy is often low, and they entertain misconceptions. Educational research in science and mathematics suggests that learning through depictive representations and learning from errors, based on the theory of negative knowledge, can potentially foster students' knowledge of abstract concepts such as energy. Thus, we propose here an instructional approach that combines these two strategies to foster conceptual knowledge of energy. It involves inserting an error in a biological energy flow diagram, an error that we derived from two prevalent misconceptions about energy: 1) plants get some of their energy from the soil or 2) energy cycles in an ecosystem. The approach's effect on students' conceptual knowledge of energy was tested in an intervention study with pre-post design and 304 ninth grade students (M = 14.79 years). Students who successfully identified and explained the error achieved larger gains in conceptual knowledge than students learning with a correct diagram. Thus, the proposed instructional approach holds promise for improving energy teaching.

INTRODUCTION

Energy is a fundamental conceptual element of all natural phenomena and thus is extremely important in all natural sciences. It also has enormous societal relevance. For example, switching from fossil to renewable energy supplies is crucial for future sustainability, and public acceptance is essential for this transition (World Energy Council, 2015). Therefore, energy is an important topic in both science and general education at school. However, due to its abstractness, the energy concept is highly challenging both to teach and learn. In recent decades, various studies have shown that students have limited knowledge of energy (e.g., Duit, 1984; Trumper, 1993; Opitz ) and even advanced students maintain multiple misconceptions (Liu and Tang, 2004; Chabalengula ). Thus, there is a clear need for effective instructional tools for fostering development of a sound energy concept.

Such tools could incorporate depictive representations, which can help students to grasp abstract concepts by visualizing phenomena that cannot be seen by the human eye (Ryoo and Linn, 2012). Incorporation of errors could also be potentially beneficial, according to the theory of negative knowledge, which holds that reflecting on errors can lead to a more comprehensive concept of learning content (Oser , 2012). Moreover, educational researchers have demonstrated that reflecting on errors in descriptive representations (i.e., texts and equations) can improve knowledge acquisition (e.g., Booth ). Thus, as reported here, we have developed and tested a tool that is intended to foster conceptual knowledge of the abstract scientific concept “energy,” based on use of a diagram with a deliberate error rooted in common misconceptions. To elucidate the rationale of our empirical study, we describe in the following section its three theoretical pillars: the energy concept, learning through representations, and learning from errors.

THEORETICAL BACKGROUND

The Energy Concept

As mentioned earlier, despite being primarily a physical concept, energy is highly important in all natural sciences and many other disciplines (e.g., economics), but here we primarily consider its importance in biological contexts, particularly ecological processes. Diverse biological topics can only be explained by applying the energy concept; for instance, energy is frequently considered when teaching nutrition, photosynthesis, muscle movement, and ecosystems (Needham, 2014; Harms, 2016). The discipline of biology deals with open systems, which follow the physical laws of thermodynamics (Stoy, 2010). In an ecosystem, energy can be transferred to and from the surrounding environment, and although energy cannot be directly measured, the energetic changes in a system can. Disregarding chemosynthesis, the sun is the ultimate energy source for any ecosystem. Its radiation is used by plants to synthesize biological molecules by photosynthesis. That is why plants are the basis of food chains. Food chains then show sequences of trophic levels (i.e., groupings of organisms based on their main source of nutrition). Through feeding relationships, chemical energy is transferred from one trophic level to the next. This transfer of chemical energy, however, accounts for only ∼10% of the energy (Kozlovsky, 1968), which is regarded as the reason why food chains rarely encompass more than five links (Trussell ). The reason for the inefficiency of energy transfer through food chains is that every energy transformation in an organism generates thermal energy, which is released to the environment. Moreover, some chemical energy is passed to decomposers, for example, in feces and food waste. Energy cannot be recycled, so a continuous energy input from the sun is needed to sustain the system.

Various studies have shown that students have limited knowledge of the energy concept (e.g., Duit, 1984; Trumper, 1993; Opitz ), particularly of principles of energy transfer (e.g., Lin and Hu, 2003; Sadler ). Moreover, students have been found to maintain misconceptions about the energy sources of an ecosystem. A popular misconception is that plants get their food or energy from the soil. This belief, which was also disseminated among philosophers and scientists before photosynthesis was discovered (Métioui ), is frequently entertained by lower and upper secondary school students (e.g., Boyes and Stanisstreet, 1991; Burger, 2001), and even teachers (Beals ). Boyes and Stanisstreet (1991) found that most first-year undergraduate science students surveyed in the United Kingdom knew that plants get energy from the sun, but they often named water, soil, or air as additional sources of energy. Similarly, they often stated that animals get their energy from food, but also from air and water. It seems likely that these students could not distinguish between energy supply and other requirements of plants and animals that are essential for living but do not supply energy (Boyes and Stanisstreet, 1991). Özay and Öztaş (2003) drew the same conclusion from a survey of ninth graders’ ideas in Turkey. This is consistent with the finding of Leach that 16-year-old students in England frequently used the terms “energy,” “matter,” “food,” and “nutrients” interchangeably in the context of ecosystems. DeBoer found a similar lack of distinction between matter and energy when students in the United States were asked to construct diagrams: some students included the sun in a food web diagram that was supposed to show the transfer of matter, while others did not include the sun in an energy flow diagram. Both students and teachers have been found to believe that energy can be recycled (e.g., by decomposers) and reused (e.g., by plants) in an ecosystem (Burger, 2001; Beals ; Lancor, 2014), which also shows confounding of energy flow and matter cycles. Altogether, there is strong evidence that energy is a critical and challenging concept for comprehending fundamental ecological processes. We applied these insights to formulate an error based on common misconceptions to incorporate in our instructional tool (with a depictive representation showing characteristic energy flow through a forest ecosystem) intended to foster conceptual knowledge of energy.

Learning through Representations

Instruction without using representations (signs that stand for something else) is impossible. Language (spoken or written) and pictures are both external representations but are based on different sign systems. Descriptive representations (texts and mathematical equations) consist of symbols, whereas depictive representations consist of iconic signs (Schnotz and Bannert, 2003). The specific function of depictive representations in science is to visualize phenomena that cannot be seen by the human eye, especially processes on a molecular or cellular level and abstract concepts (Winn, 1989; Kozma and Russell, 2005; Tsui and Treagust, 2013). For example, depictive representations are regarded as helpful for illustrating energy transfer in a food chain by making the abstract process more concrete (Winn, 1989). More recently, intervention studies have provided evidence that depictive representations can help students to grasp aspects of the energy concept. For example, Van Heuvelen and Zou (2001) found that bar chart representations of work–energy processes help university students to understand energy conservation in physics. Similarly, Ryoo and Linn (2012) showed that dynamic and static depictive representations promote middle school students’ understanding of energy transformations in photosynthesis. However, as yet we have little evidence about the particular features of representations used in the science classroom that foster understanding of energy.

An analysis of the representation of energy in biology textbooks revealed that flow diagrams with arrows indicating sequences are frequently used to illustrate energetic aspects of photosynthesis, matter cycles, and ecological energy flows (Wernecke ). Their abstractness is often lessened through combination with more concrete elements such as schematic drawings, realistic drawings, or photographs. However, representations with such multiple layers are especially demanding: students must both relate the concrete elements to the corresponding things in the real world and decode the meaning of symbols such as arrows (Roth ). Moreover, diagrams should include a caption to give an interpretative context (Ametller and Pintó, 2002).

Of course, good representations (of anything) are not sufficient. A requirement for comprehension of a representation is cognitive processing. According to the theory of constructivism and cognitive psychology, the individual builds up an internal, mental representation of the content and connects it with prior knowledge (e.g., Smith ; Mayer, 2011). Cromley investigated students’ cognitive activities while learning with text and complex biological diagrams. They found that students used a higher proportion of high-level strategies and a lower proportion of low-level strategies when processing diagrams than when processing text. High-level activity is associated with increased comprehension. However, students often just skim over depictive representations and thus fail to exploit their full potential (Cromley ). To counteract superficial processing, learners should be given picture-oriented tasks (Weidenmann, 1989).

Kragten found evidence that prior knowledge is crucial for solving tasks related to flow diagrams in biology. Therefore, prior knowledge should be activated when students are supposed to learn with such diagrams. Another requirement for learning from diagrams is sufficient knowledge of diagrammatic conventions, as shown by Cheng and Gilbert (2015) specifically for biological flow diagrams. The middle school students surveyed by DeBoer reportedly had little difficulty interpreting basic graphical representations of ecosystems like flow diagrams. However, two groups of researchers (Barman ; Gotwals and Songer, 2010) found that, when middle and high school students in the United States, Australia, and Canada considered a food chain or web, they could describe feeding relationships between the organisms but did not realize that the arrows represented energy transfer in the system. This corroborates the apparent challenges the concept of energy poses for students attempting to understand representations of an ecosystem.

Learning from Errors

Making errors is a natural part of the learning process. A lot of errors do not occur randomly, but originate from misconceptions (Nesher, 1987). Nevertheless, although errors should be eradicated, different theoretical approaches on learning see them as beneficial for learners when used in a constructive way (Smith ). According to the theory of negative knowledge, reflecting on errors can lead to the acquisition of so-called negative knowledge, that is, knowledge of how something is not in contrast to how it really is (Oser ; Oser and Spychiger, 2005). Concerning the internal representation of negative knowledge, it seems that incorrect parts of an initial mental model may be labeled as incorrect or as not belonging to this concept and that correct knowledge may be added, resulting in construction of a more comprehensive mental model. Oser and Spychiger (2005) describe negative knowledge as inherent in every piece of knowledge and claim that it serves various functions, inter alia realizing that what seemed correct is not actually correct helps recognition of what is right. In other words, the scope of a concept is sharpened by clarifying what does not belong to it. Thus, certainty is gained, because correct and incorrect conceptions can be more clearly distinguished. Various studies have corroborated the idea that misconceptions may not be extinguished but coexist with scientific conceptions, notably in the context of energy, studies by Liu and Tang (2004) and Solomon (1983). Negative knowledge is assumed to prevent the learner from using the corresponding misconception (Oser and Spychiger, 2005).

A critical issue in this context is that learners’ own errors are likely to arouse negative emotions like shame, which may hinder the learning process, so an error-tolerant classroom culture may be crucial for learning from errors in classroom settings (Oser ). However, substantially more research is required to clarify such requirements (Steuer and Dresel, 2015). Therefore, it seems reasonable to use constructed errors as a vicarious rather than personal experience in educational contexts. Moreover, learners’ own errors occur individually and spontaneously, while the application of constructed errors can be planned. Several studies that have addressed such issues, in various domains, indicate that reflecting on constructed errors promotes acquisition of skills and knowledge. For example, this approach has been used to improve the professional competence of firefighters (Joung ) and medical students (Stark ), highlighting its cross-domain applicability. In school education, the utility of the approach has been investigated particularly intensively in the domain of mathematics, for example, by inserting an error into the solution of an equation and asking students to state which step was incorrect, why it was wrong, and what must be done to solve the equation correctly. Reflecting on incorrect examples has been shown to improve learners’ understanding of mathematical topics such as algebra (Booth ), probability (Große and Renkl, 2007), decimal magnitude (Durkin and Rittle-Johnson, 2012), and fractions (Heemsoth and Heinze, 2014).

The effectiveness of learning from errors may depend on the learner’s prior knowledge. Students with low prior knowledge benefit more from learning with correct examples, while advanced students learn more from incorrect examples (Heemsoth and Heinze, 2014). Similarly, Große and Renkl (2007) reported that learning with correct and incorrect examples is beneficial for students with high prior knowledge, whereas students with low prior knowledge benefit more from learning with correct examples. Prior knowledge is needed to identify and explain errors. Therefore, highlighting an error lowers the demand on the learner, according to Große and Renkl (2007), who found that students with low prior knowledge profit from support such as highlighted errors. However, in contrast to Große and Renkl (2007), Durkin and Rittle-Johnson (2012) found no indications of an interaction between amount of prior knowledge and either learning with correct examples or learning with correct and incorrect examples.

Students enter the classroom with conceptions resulting from their experience in everyday life. The key feature of learning from errors is that misconceptions can be addressed by errors to prevent students from using them (Oser and Spychiger, 2005). Aside from incorrect equations in mathematics, it has been shown that learning from science texts that explicitly address misconceptions (so-called refutation texts) effectively fosters positive learning outcomes (e.g., Özkan ; Tippett, 2010). Refutation texts require simultaneous activation of correct and incorrect conceptions, which helps learners to become aware of the inconsistency between the misconception and the scientific conception. This is a crucial step toward conceptual change (Posner ; Van den Broek and Kendeou, 2008). Refutation texts have been successfully applied in the context of energy (Diakidoy ).

To date, descriptive representations (i.e., texts and equations) have been mainly used in material designed to implement this approach. However, given the importance of depictive representations in communicating scientific concepts and facilitating the development of correct mental models of these concepts, a significant challenge for researchers is to “design visualizations that specifically target … faulty beliefs” (Rapp, 2005, p. 56).

THE ERROR-BASED PICTORIAL TOOL TO FOSTER CONCEPTUAL KNOWLEDGE OF ENERGY

Depictive representations of abstract science concepts have proven advantageous for learning about energy. Moreover, educational research in mathematics has shown that learning from errors in descriptive representations can potentially foster students’ knowledge of abstract concepts. We hypothesized that combining the learning from errors approach with learning through depictive representations—an error-based pictorial tool—would promote knowledge acquisition about the abstract concept energy more so than learning with a correct representation. Our underlying assumption was that alerting students to the fact that an error had been deliberately placed in a diagram would encourage them to examine the representation more thoroughly, rather than skimming over it. If so, learning with diagrams incorporating deliberate errors should foster acquisition of conceptual knowledge of energy more effectively than learning with a correct representation. This is because, as previously stated, addressing common errors that are rooted in misconceptions may promote learning by making negative knowledge explicit.

Hence, in the intervention study reported here, we inserted an error in an energy-related diagram that showed the energy flow through a simple food chain in a typical ecosystem. This context was chosen for two reasons. First, energy flow is an important topic in biology and comprehending it requires sound understanding of the energy concept. Second, there is well-documented evidence that students possess prevalent misconceptions regarding this topic. Our empirical study addressed one hypothesis and four explorative research questions:

Hypothesis: On the basis of the findings of several studies on learning in mathematics (as described in the previous section), we hypothesize that learning with an incorrect representation would foster acquisition of conceptual knowledge of energy more effectively than learning with a correct representation.

Research question 1: Do students learn more when the error has already been encircled in the diagram when they receive it, or when they have to identify it by themselves? Concerning the underlying mode of action, trying to identify the error may encourage students to examine each part of the diagram thoroughly. However, failing to identify the error may be a barrier to learning, as students might continue to believe that the incorrect feature is correct or they might assume that a correct feature is incorrect.

Research question 2: Does students’ amount of prior knowledge influence how much conceptual knowledge of energy they gain?

Research question 3: What features of the diagram do students consider to be incorrect besides the deliberate error (an energy backflow arrow)?

Research question 4: What explanations do students give for the encircled error?

METHODS

Procedure and Sample

An intervention study with pre/posttest design and three experimental groups was conducted. Students’ conceptual knowledge was assessed before and after an intervention; the design thus allows gauging differential change of conceptual knowledge between conditions (i.e., effects of the intervention). The three groups differed in received learning material, which consisted of three variants of a diagram (one correct; one with an encircled error; and one with that same error, but not encircled) showing energy flow through a forest ecosystem and associated tasks (see Table 1 and Figure 1).

TABLE 1.

Learning materials provided to the three groups

Group 1	Group 2	Group 3
Incorrect diagram, without the error encircled	Incorrect diagram, with the error encircled	Correct diagram
Identify the error Explain the error Explain energy flow	Explain the error Explain energy flow	Explain energy flow

FIGURE 1.

Incorrect diagram with encircled error (provided to group 2).

The study was conducted by U.W. and a trained university student. The pretest was administered 2 weeks before the intervention. Although the topic should already have been treated in biology classes, the intervention started for all groups with a standardized presentation on energy flow in ecosystems to ensure that all students were provided with the essential information to identify and explain the error (see the Supplemental Material). Afterward, students received their respective learning material (see Table 1), which they worked on independently. In an effort to minimize class effects, all three variants of the material were distributed almost equally in each class. During the seatwork, students could use printouts of slides of the presentation. After students were finished, slides and learning material were collected by the administrator. Students then completed the posttest. At the close of the session, the administrator fully debriefed the students and also provided standardized information about correct solutions of the tasks. Teachers were not directly involved in conducting the study.

Power analyses with G*Power 3.1 (Faul ) indicated that, for one-way analysis of variance (ANOVA) and a middle effect size of f = 0.25, a sample size of N = 252 students would be required. To meet this requirement, we recruited325 ninth graders from 12 secondary schools on the highest track of the ability-grouped German school system (“Gymnasium”) to participate in the study. However, 15 students missed the second measurement occasion and were excluded from the sample. Another six students were not compliant, as they did not work on the diagram tasks or the posttest. Thus, the remaining sample comprised 304 students (52.2% female; M = 14.79 years, SD = 0.48 years): 102, 100, and 102 in groups 1, 2, and 3, respectively.

Materials and Instruments

On the basis of the energy flow diagrams in textbooks surveyed by Wernecke and flow diagrams of textbooks not surveyed there (e.g., Beyer ; Markl, 2010; Braun ), we designed the energy flow diagram shown in Figure 1. This diagram is a depiction of the energy transfer processes in an ecosystem, which cannot be observed directly due to temporal and spatial constraints. The diagram includes realistic drawings of organisms as representatives of trophic levels and contains labels, an identifying headline, and a caption concerning the scale. The indicated percentages are approximations. To reduce complexity, we applied two simplifications compared with typical energy flow diagrams in textbooks. First, we chose to portray the radiation from the sun as being only split into radiation that is not absorbed by plants (e.g., radiation reflected by the earth’s surface, as explained in an additional caption; some textbooks differentiate between reflected and thermally dissipated energy) and radiation absorbed by plants (set to 100% of the energy entering the system). Second, the chosen food chain “tree → mouse → eagle owl” consists of only three trophic levels, whereas many other flow diagrams show four or five trophic levels. However, all energy pathways can be traced in the reduced version, and the principles of energy flow are fully delineated.

The error inserted into the incorrect versions of the energy flow diagram targets widespread misconceptions. This is an additional arrow from the decomposers in the soil to the producers, suggesting that plants get some of their energy from the soil (Boyes and Stanisstreet, 1991; Leach ). This arrow implied that energy, just like matter, flows in a cycle and can be recycled (Burger, 2001; Beals ; Lancor, 2014). Three versions of the diagram were designed and complemented with tasks, as described and shown in Table 1 and the Supplemental Material.

The students’ conceptual knowledge of energy was assessed by a test with 18 items. These included eight multiple-choice items (regarded as representative in terms of context, difficulty, and discrimination) from the test developed to assess students’ general understanding of energy and validated by Opitz . The other 10 items (eight multiple-choice and two open-ended items) specifically concern energy flow. Three of these items were newly created, while the others were compiled, with some modification, from sources shown in the Supplemental Material. Despite pretesting before the study, internal consistency was low for the pretest (α = 0.57), but acceptable for the posttest (α = 0.71).

The following control variables were also assessed: cognitive abilities, using the N2 scale (graphic analogies) of the Kognitiver Fähigkeitstest (KFT) cognitive ability test (Heller and Perleth, 2000); effort appraised on a 10-point scale (Organisation for Economic Co-operation and Development, 2010); and time spent on tasks, the pretest, and the posttest. Moreover, each student completed a form reporting his or her sex, age, and biology mark on the last report card.

Data Preparation and Analysis

All test items were coded dichotomously. The objectivity of the ratings concerning the two items with an open-ended answer format was checked by rerating 25% of the posttest answers (72 and 69, respectively). Cohen’s kappa was chosen as a chance-­corrected coefficient of rater agreement (Cohen, 1960); it indicated at least substantial agreement (κ = 0.96 and 0.76, respectively; Landis and Koch, 1977). To address our hypothesis and research questions 1 and 2, we conducted mixed-model ANOVAs using SPSS 23, and computed a latent difference score model using Mplus 7.

To address research questions 3 and 4, we investigated students’ performance on the diagram tasks using descriptive statistics, focusing on the identification and explanation of the error by participants in groups 1 and 2. For a check of the objectivity of the ratings, 25% (26 in each case) of the explanations of students in groups 1 and 2 were randomly selected and coded by a second rater. The Cohen’s kappa values of the ratings of groups 1 (κ = 0.87) and 2 (κ = 0.63) indicated almost perfect and substantial rater agreement, respectively (Landis and Koch, 1977). Differences between means were regarded as significant if p < 0.05.

RESULTS

Initially, to ascertain that randomly distributing students to groups did not result in meaningful differences between the groups we conducted a set of one-way ANOVAs. Students’ pretest scores, indicating their prior knowledge, did not significantly differ between the three groups, F(2, 286) = 0.69, p = 0.500 (cf. Table 2). Moreover, there were no significant differences between the three groups of students in time spent on the pretest, posttest, and working with the diagram, F values < 2.70, p values > 0.05. Similarly, no significant differences were observed between the three experimental groups in cognitive abilities, last report card mark in biology, or effort, F values < 1.73, p values > 0.05.

TABLE 2.

Means (and standard deviations) of control variables

	Group 1	Group 2	Group 3
Pretest score (prior knowledge)	6.96 (2.69)	6.56 (2.99)	6.99 (2.67)
Working time: pretest (min)	22.67 (5.14)	24.07 (5.02)	24.06 (3.91)
Working time: posttest (min)	16.47 (4.49)	16.65 (4.47)	16.86 (3.98)
Working time: diagram (min)	15.86 (5.27)	15.46 (4.66)	16.01 (5.34)
Cognitive abilities	18.95 (3.60)	19.13 (3.93)	18.75 (3.96)
Biology gradea	2.35 (0.79)	2.46 (0.71)	2.38 (0.75)
Effort	6.07 (2.00)	6.08 (2.18)	6.55 (2.13)

aThe German grading system has a numerical 1–6 format, in which 1 is the highest and 6 the lowest grade.

Descriptive statistics showed that only n = 11 (10.8%) of the students in group 1 identified the incorrect arrow in the diagram and satisfactorily explained why it was incorrect, while n = 28 (28%) of the students in group 2 satisfactorily explained why the encircled arrow was incorrect. Because students were not informed about the correct solution before the posttest, students who failed to find and explain the error had no opportunity to build up a contrast between correct and incorrect knowledge. We further split groups 1 and 2 into two subgroups of students (total number of groups = 4), those who did and those who did not solve their respective tasks, thereby forming five sets with the students of group 3. With these five groups, we conducted another one-way ANOVA to investigate whether prior knowledge played a role in whether or not students solved their respective tasks; differences in the pretest scores were indeed statistically significant, F(4, 284) = 2.73, p = 0.029. Assessment of pairwise differences between the sets by post hoc tests with Bonferroni correction revealed one significant difference: the successful students in group 2 had a higher pretest score (M = 7.89) than students who failed in this group (M = 6.00), p = 0.024. One-way ANOVA also detected a significant difference between the five sets in effort, F(4, 294) = 4.294, p = 0.020, but post hoc tests with Bonferroni correction detected no significant difference between any pairs of sets in this respect.

To evaluate our hypothesis that learning with an incorrect representation fosters acquisition of conceptual knowledge of energy more effectively than learning with a correct representation, we conducted a one-way repeated-measures ANOVA, again initially with the three experimental groups. The main effect of measurement occasion shows that test results increased substantially from pre- to posttest, F(1, 285) = 233.48, p < 0.001, ηp² = 0.450. However, the interaction effect did not indicate differential improvement of students’ knowledge of the energy concept between the three groups, F(2, 285) = 2.43, p = 0.090, ηp² = 0.017. The main effect of group was not significant, F(2, 285) = 0.17, p = 0.842, ηp² = 0.001. This also implies that encircling the error in the incorrect representation did not result in higher or lower knowledge gain than use of the same learning material without the presumed support offered by encircling it (research question 1).

However, when we conducted the one-way repeated-measures ANOVA comparing the five (sub)groups of students, in addition to the significant main effect of measurement occasion, F(1, 283) = 199.92, p < 0.001, ηp² = 0.414, there was an interaction effect indicating differential improvement of knowledge of the energy concept, F(4, 283) = 5.94, p < 0.001, ηp² = 0.077. Post hoc tests with Bonferroni correction revealed that students in group 2 who provided a sufficient explanation of the error learned significantly more than students in that group who did not (p < 0.001). They also learned more than the students who failed to identify and explain the error in group 1 and more than students in group 3 (p = 0.003 and 0.029, respectively). Differences between the other pairs of sets were not significant (Table 3).

TABLE 3.

Means and standard deviations of energy test score gains

Group	n	M	SD
Group 1	96	2.07	0.27
Successful	11	4.64	0.74
Failed	85	1.74	0.27
Group 2	93	2.82	0.29
Successful	28	4.11	0.43
Failed	65	2.46	0.36
Group 3	99	2.44	0.28

To assess the relationship between pretest scores and knowledge gain (research question 2) and investigate the robustness of our results across analytical approaches, we also analyzed the data using latent difference score models in Mplus. Corroborating results from the ANOVAs, this showed that posttest scores were significantly higher than pretest scores (p < 0.001). Wald χ² (W) tests of parameter equality indicated that there were no significant differences in the improvement of students’ knowledge between groups 1 and 3, W(1) = 0.92, p = 0.339, or between groups 2 and 3, W(1) = 1.87, p = 0.339, but students in group 2 learned significantly more than students in group 1, W(1) = 5.61, p = 0.018.

We then used a latent difference score model to assess differences between students of group 1 who did and did not identify and explain the error. A Wald test indicated that the successful students improved their conceptual knowledge of energy significantly more than students who failed to solve their task, W(1) = 14.83, p < 0.001. More importantly, students in group 1 who solved their task also gained significantly more knowledge of energy than students in group 2, who learned with the marked error, W(1) = 4.92, p = 0.027, and group 3, who learned with the correct diagram, W(1) = 8.57, p = 0.003. Similarly, successful students of group 2 who provided a satisfactory explanation of the error learned significantly more than students in group 2 who failed, W(1) = 8.83, p = 0.003. They also learned more than the students who failed to identify and explain the error in group 1, W(1) = 22.85, p < 0.001, and students in group 3, W(1) = 11.21, p = 0.001. The knowledge gains of successful students in groups 1 and 2 were not significantly different, W(1) = 0.41, p = 0.521. Moreover, knowledge gains of students who failed in groups 1 and 2, and students in group 3 were not significantly different (p values > 0.05). Hence, our hypothesis that learning with an incorrect representation fosters acquisition of conceptual knowledge of energy more effectively than learning with a correct representation was supported for students who identified and explained the error.

Wald tests also revealed that the successful students of group 1 self-reportedly put more effort into work with the diagram than students of their group who did not identify and satisfactorily explain the error, W(1) = 6.98, p = 0.008. No such difference in effort between successful and unsuccessful students was found in group 2, W(1) = 3.10, p = 0.078.

Regarding research question 2, the correlations between pretest score and change score in all groups and subgroups were significantly negative (r values = −0.215 to −0.559, p values < 0.05), except for students who successfully explained the error in group 2 (r = −0.253, p = 0.153). Thus, the increase in students’ conceptual knowledge of energy was higher in all (sub)groups the less prior knowledge they brought to their respective tasks.

Task Performance

To address research questions 3 and 4, we investigated students’ performance on the diagram tasks.

Group 1 (n = 102)—Incorrect Diagram without the Error Encircled.

Only 11 students (10.8%) in group 1 correctly identified the incorrect arrow and satisfactorily explained why it was incorrect. Another eight students (7.8%) marked it correctly but gave an inadequate or no explanation. Most of the students in the group (n = 69, 67.6%) marked one of the following features of the diagram.

The arrow indicating energy flow through food from the mouse to the eagle owl (39.3% of the incorrect markings). Most of the students who selected this feature stated that the accompanying percentage should be 10% rather than 1%, failing to grasp that the percentages referred to percentages of the original amount of energy entering the system that are transferred between the trophic levels: “Then there are the secondary consumers. Here, again 10% of the energy is transferred. At this spot, there is an error in the diagram, because the mouse does not pass on just 1% of its energy.”

The arrow indicating dissipation of thermal energy from the decomposers to the atmosphere (27.9% of the incorrect markings). Students marking this arrow as incorrect usually noticed the inconsistency between the amounts of energy flowing to and from the decomposers, but they considered the energy backflow arrow to be correct and changed the percentage associated with the thermal energy arrow: “If you sum it up, you get that the decomposers have 44.5% of the energy. However, decomposers pass 10% to the producers and thus can emit just 34.5% of the thermal energy.”

The arrow indicating energy flow through dead biomass from the eagle owl to the decomposers (16.4% of the incorrect markings). Some students who marked this as incorrect argued that the accompanying percentage should be 0.4% to continue the sequence of percentages of energy flow through dead biomass. “On the sheet, the value of 0.5% is false, because the number gets consistently smaller by a zero point → 40% → 4% → 0.4% so there should be nothing with a 5.”

The arrow indicating dissipation of thermal energy from the producers to the atmosphere (4.9% of the incorrect markings). Students who selected this feature expressed the belief that plants do not give thermal energy to the environment: “The tree does not release thermal energy because it does not move.”

The remaining students marked another detail in addition to the incorrect arrow (three students, 2.9%), or either made an ambiguous marking or no marking at all (n = 11, 10.8%).

Group 2 (n = 100)—Incorrect Diagram with the Error Encircled.

Twenty-eight (28%) students in group 2 sufficiently explained why the encircled arrow was incorrect, by referring either to plant nutrition (plants do not get energy from the soil or the decomposers/plants only get energy from the sun) or general characteristics of energy flow (energy cannot be recycled/energy does not flow in a cycle). Most students (n = 63, 63.0%) gave an unsatisfactory explanation. The most frequent statement (39.7%) was that plants get energy from the soil, but the percentage accompanying the arrow was too high or too low, for example, “The marked feature is false because the consumer [sic] must absorb 100%.”

Additionally, a considerable number of students (27.0%) said that no energy would be left for backflow. These students noticed the inconsistency between the energy flowing to the decomposers and being released from them: “The decomposers cannot give 10% of the energy back to the producers, because they get a total of 44.5% of the energy from the food chain, but these 44.5% are lost by thermal energy. Thus, they do not have energy left they could hand over to the producers.” However, a solely mathematical argument was not considered a sufficient explanation.

A few other students stated that the arrow is correct (n = 2, 2.0%): “The red marked area is not a mistake as decomposers as well as consumers cannot produce their own energy. However, they give off part of the food they received previously. This part transferred to the tree, amounts to generally 10% of the received energy.” The remaining seven students (7%) did not give any explanation at all.

Group 3 (n = 103)—Correct Diagram.

Students in group 3 described the correct diagram. Their texts were not analyzed further in the context of this study.

DISCUSSION

The instructional tool presented and considered here integrates the approaches of learning from errors and learning through depictive representations, by incorporating a pictorial error based on well-known misconceptions of the flow of energy derived from the literature. The tool was used to explore whether learning with an incorrect flow diagram enhances students’ conceptual knowledge of energy more than learning with a correct diagram. The results indicate that learning from such errors fosters acquisition of conceptual knowledge of energy more effectively than learning with a correct diagram, provided that the students succeed in (identifying and) explaining the error. Failure to perform associated tasks adequately does not seem to be particularly harmful, as students who failed to identify and explain the error did not gain less knowledge than students learning with the correct diagram. Strikingly, only 11% of students who learned with the incorrect diagram (group 1) and 28% of students who learned with the incorrect diagram with encircled error (group 2) satisfactorily completed their respective tasks. Successful and unsuccessful students did not significantly differ in cognitive abilities or biology grade. However, successful students in group 1 self-reportedly put more effort into work with the diagram than those who failed. This indicates that identifying the error may be especially demanding, as each part of the diagram must be examined thoroughly, and students must take the task seriously. However, generally, students learning with an incorrect diagram did not reported making more effort than those learning with a correct diagram. The effort might have been influenced by study-related factors that were equal in all groups (and not present in regular lessons), for instance, the extraordinary situation or the fact that the results would not affect the students’ biology grade.

Concerning the relevance of prior knowledge, there was no statistically significant difference in pretest scores (and thus assumed prior knowledge) between successful and unsuccessful students in group 1, but this could be due to the small proportion of successful students (11%), and consequent low statistical power. Moreover, successful students in group 2 had a higher mean pretest score than those who failed. This finding supports the assumptions that prior knowledge is needed to identify and explain the error, and highlighting errors reduces demands on learners (Große and Renkl, 2007). Research on learning with correct worked examples also shows that learners’ prior knowledge should be considered when selecting prompts intended to evoke self-explanations (Neubrand ; Neubrand and Harms, 2017). Thus, matches between instructional support and learners’ prior knowledge are crucial for learning with both correct and incorrect learning material. We conclude that both learning with correct worked examples in combination with prompts and learning from errors enhances cognitive activation of the learner, because both address students’ prior knowledge (Baumert ).

In contrast to patterns typically observed in mathematics education, correlations between pretest scores and knowledge gains were negative in all groups. Learning from the incorrect diagram showing energy flow through a forest ecosystem benefited students with low prior knowledge, implying that our approach is not solely applicable to high-achieving students. The higher students scored on the pretest, the lower the conceptual knowledge of energy they gained during the intervention. However, this finding is not the result of a ceiling effect in the energy test. It is possible that students in group 1 stopped examining the diagram once they thought they had identified the incorrect feature, thus forfeiting the chance to further advance their knowledge. Hence, not specifying the number of errors inserted in the diagram might be advantageous. This might be a crucial difference from learning from errors in mathematics education, where students usually have to solve problems correctly, and thus have to examine complete representations.

Große and Renkl (2007) and Heemsoth and Heinze (2014) found that students with high prior knowledge benefit more from learning with incorrect learning material than students with low prior knowledge, whereas Durkin and Rittle-Johnson (2012) found no such difference. We found the opposite pattern, but our results clearly showed a need for high prior knowledge to identify and explain the error. This is not surprising, because, as exemplified in the theoretical background, prior knowledge of the content and about diagrammatic conventions is needed to learn successfully with biological diagrams (Cheng and Gilbert, 2015; Kragten ). Considering the various features in the diagram that were erroneously identified as incorrect by the students in group 1, we found little evidence of difficulties with diagrammatic conventions such as arrows, spatial relationships, and colors. However, we detected content-related difficulties. The feature identified most frequently as incorrect—the arrow indicating energy flow from the mouse to the eagle owl—was erroneously regarded as incorrect due to misunderstanding of the percentages of energy flowing into the system. We decided to include percentages in the diagram for two reasons. First, energy can be conceptualized as an abstract accounting quantity defined in a system (Duit, 2014). For example, the given percentages illustrate that energy is finally transferred to the atmosphere, thereby indicating that an ecosystem depends on a constant energy source. Second, percentages are typical features of energy flow diagrams in biology textbooks. However, they also gave students a clue about the error and offered them the possibility to argue mathematically (e.g., “There is no energy left for backflow”) instead of biologically (e.g., “Plants do not get energy from the soil”). Thus, the students may have expected to find a small error in the numbers, rather than considering a whole arrow to be incorrect. To counter this potential learning obstacle, a diagram without percentages, representing the error solely in depictive information, could be used in future studies.

Interpreting the results, it should be noted that the set tasks and correct responses were discussed only after the posttest. In an authentic class setting, discussion of such an error would be a crucial step of the instructional process to ensure that misconceptions are not reinforced. Students who do not identify and explain the error, and thus do not establish the contrast between correct and incorrect knowledge by themselves, must be confronted with the correct knowledge through discussion to facilitate learning.

Limitations

The study presented here has several methodological limitations. First, despite pilot tests, the internal consistency of the energy test was rather low, especially for the pretest. Second, a follow-up test of students’ knowledge of the energy concept, assessing whether the differential knowledge gain is maintained, would be instructive (i.e., investigation of the sustainability). Third, analysis of the learning of a group without a diagram would have allowed estimation of the knowledge gain induced by the presentation alone. However, we decided to settle for assessing the learning from errors approach in relation to comparable instruction in terms of the arrangement (individual work) and time spent on tasks. Fourth, systematically implementing the error-based pictorial tool in ecology instruction across participating classes was not practical (especially because of schools’ internal curricula). Given these limitations, the study should be regarded as a first step toward combining learning from errors and depictive representations, particularly in biological energy contexts.

Implications for Instruction and Future Research

Identifying and reflecting errors, thereby establishing a contrast between positive and negative knowledge, requires prior knowledge. Therefore, a depictive instructional tool incorporating an incorrect representation should not be used to introduce a new concept or topic. Moreover, teachers should ensure that all students are eventually provided with the positive knowledge, for example, by thoroughly discussing the tasks and correct responses in the classroom.

The learning from errors approach has not yet found its way into school teaching; correct examples are mainly used in classrooms (Booth ). Therefore, more practical teaching material implementing the approach is needed. In this study, a biological flow diagram was used, partly because this type of representation is frequently (and advantageously) used to depict energy flows in biological and ecological systems. Further studies should investigate whether learning from errors is transferable to other types of depictive representations and other scientific domains. Ideally, inserted errors should address prevalent misconceptions, and the students must have sufficient prior knowledge of the conventions of the type of representation used. We also need to learn more about the role of prior knowledge in learning from errors in different domains and from different representations, given the heterogeneous research results. Moreover, a sociosemiotics perspective (e.g., Kress and van Leeuwen, 1990) might provide valuable insights for the use of incorrect depictive representations in science teaching.

A potentially harmful effect of presenting students with incorrect representations is that they may memorize their content. Several studies have shown that, when conflicting information is presented in picture and text, students are more inclined to remember the information embedded in the picture (Peeck, 1989; Crisp and Sweiry, 2006). Hence, future studies should include a follow-up test to assess this potential risk and the sustainability of students’ knowledge acquisition. Moreover, closer examination of strategies that students use while looking for errors in a picture is warranted and could lead to the design of supporting provisions besides marking the error. For this purpose, process-oriented studies using eye-tracking methodology and think-aloud protocols might prove valuable.

According to the theory of negative knowledge and preliminary empirical studies, an error-tolerant classroom culture is needed for students to learn from their own errors (Oser ; Steuer and Dresel, 2015). Instructions concerning deliberately introduced errors might contribute to a constructive error climate by illustrating that it is possible and desirable to use errors as opportunities for learning. Beyond the direct effect on students’ knowledge about abstract concepts such as energy, this instructional approach might thus enhance science teaching more generally.

Click here for additional data file.