Learning about Chemiosmosis and ATP Synthesis with Animations Outside of the Classroom.

Many undergraduate biology courses have begun to implement instructional strategies aimed at increasing student interaction with course material outside of the classroom. Two examples of such practices are introducing students to concepts as preparation prior to instruction, and as conceptual reinforcement after the instructional period. Using a three-group design, we investigate the impact of an animation developed as part of the Virtual Cell Animation Collection on the topic of concentration gradients and their role in the actions of ATP synthase as a means of pre-class preparation or post-class reinforcement compared with a no-intervention control group. Results from seven sections of introductory biology (n = 732) randomized to treatments over two semesters show that students who viewed animation as preparation (d = 0.44, p < 0.001) or as reinforcement (d = 0.53, p < 0.001) both outperformed students in the control group on a follow-up assessment. Direct comparison of the preparation and reinforcement treatments shows no significant difference in student outcomes between the two treatment groups (p = 0.87). Results suggest that while student interaction with animations on the topic of concentration gradients outside of the classroom may lead to greater learning outcomes than the control group, in the traditional lecture-based course the timing of such interactions may not be as important.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

Recent calls for reform in STEM education cite the need for increased student interaction with course content both inside and outside of the classroom (1, 2). Traditionally, lecture-centered instruction has accounted for content delivery in many large-enrollment undergraduate classrooms. These student/content interactions inside the classroom have more recently become focused on strategies such as active learning and inclusion of authentic research in undergraduate laboratory environments (3–5). Research around such strategies has noted their benefits on a number of different occasions (6–8); however, their levels of adoption can fluctuate across educational settings (9, 10).

To date, there has been little research to determine the most effective way to engage students with the instructional material outside of the formal classroom setting. Instructional strategies designed to promote interaction outside of the classroom can vary widely depending on instructors’ pedagogical practices, course subject, and course level. Despite this, these interactions outside of the classroom have been widely shown to promote greater learning outcomes (1, 11). Examples of such successful engagement strategies include textbook reading assignments (12), worksheets (13), viewing of animations and videos (14), online modules (15), and instructor-mediated blogs (16). These methods can be characterized broadly into two main categories: pre-class preparation and post-class concept reinforcement.

The purpose of this study is to compare these two distinctly different categories of student interaction outside of the classroom, as well as compare both of these strategies with a no-intervention control. Recent innovations in the development of online instructional resources have provided students a platform where they can interact with course material on their own time and in their own environment. Hence, we investigate the use of such multimedia resources in support of learning in introductory biology.

Review of the literature

Student preparation has long been a key aspect of undergraduate instruction. Traditionally, preparation strategies have required students to read material in the textbook prior to attending class (12, 17). While these reading assignments have been shown to promote student preparation, motivation to complete such activities can fluctuate (11, 17, 18). Research conducted by Gross et al. (19) supports the role of preparation by noting that students who interacted with content prior to class performed 12% higher on follow-up exams than students who did not. To capitalize on outcomes such as these, various methods to promote student motivation and completion of these preparatory activities have been developed. Examples include the use of reading quizzes (18), online learning modules (15), and monitored discussion groups (20). While classroom instructional styles following the preparation assignments can vary, the learning outcomes appear to be positive. This places a possible emphasis on pre-class preparation in the learning process in many introductory biology students.

Not unlike preparation, post-class concept reinforcement has also been used to increase conceptual understanding (21). Reinforcement assignments can vary in their specific format and can be associated with a grade or simply left to the discretion of the student (22–25). While reinforcement assignments have been shown to increase exam scores in numerous studies (24, 26–28), motivation to complete such assignments has again been shown to vary (27). One method that instructors have implemented as a means of tracking the progress of such reinforcement assignments and hopefully increasing student participation is the use of web-based multimedia learning resources (23, 24, 26, 29). Recent studies have shown that, in many situations, these online, computer-based assignments lead to higher achievement on concept assessments than a more traditional paper-based format (23, 30, 31). These online resources are typically available to students at their convenience and may provide them a sense of technological familiarity that could motivate their completion. The rising popularity of multimedia resources in the undergraduate classroom makes investigation into their development and implementation an important emerging aspect of education research. Therefore, we focus on the use of one of these multimedia resources, animation, in our investigation into learning outside of the classroom.

The field of biology is particularly well adapted to the use of multimedia resources, as it has been suggested that many biological processes are more effectively depicted using animations than their static counterparts (32–34). As a means of conceptual introduction, dynamic animations have been shown to provide students accurate depictions of biological concepts in a way that allows the students to make connections that could ultimately lead to greater understanding (34–36). With proper concept introduction prior to class being such an important aspect of some learning environments (19), animation could contribute to the preparation process. Likewise, with reports of the efficacy of online multimedia as a means of reinforcement assignments (23, 26, 29), the integration of animation as reinforcement given after class could promote learning in introductory biology students.

The research presented here investigates the learning outcomes of students introduced to the topic of concentration gradients and their role in ATP synthase activity. These topics constitute key components of the mechanisms involved in cellular respiration and are typically presented as part of introductory biology instruction. Misconceptions concerning cellular respiration and its many components have been shown to be widely held by many introductory biology students (37, 38). Furthermore, these misconceptions have been shown to persist even after repeated instruction and advancement through the biology curriculum (39–41). With the ever evolving field of cellular and molecular biology, such misconceptions could prove detrimental to the learning process of students attempting to form foundational mental models in introductory biology (42, 43). Results of this study aim to provide empirical evidence of how different methods of student engagement with the material outside of the classroom can affect learning gains.

Research question

While the benefits of both preparation and reinforcement have been individually researched, a deeper understanding as to which instructional strategy is more effective in a traditional classroom is needed. Here we conduct an investigation into the comparison of these two strategies as a means of increasing student engagement with material in undergraduate introductory biology. As part of a three-group design, we also look at the contribution of both preparation and reinforcement compared with a control group that received neither treatment.

The research question guiding our study was, “How does learning about concentration gradients and ATP synthase differ when students view animations before or after instruction compared with a no-intervention group?” Previous research has supported the introduction of course material prior to classroom instruction (19, 20). However, it has also been noted that these benefits may be a result of instructional practices in the classroom and not the preparation assignments themselves (44). With the reported fluctuation in effectiveness of in-class instructional strategies (45), this could suggest that the role of preparatory activities could vary drastically between courses. By contrast, reinforcement assignments following classroom instruction have consistently led to higher achievement when students complete them compared with when they do not (26, 27, 46). Constructivist theory (37) might suggest that, in a traditional lecture-centered classroom, reinforcement assignments could facilitate the “concept application phase” of learning, where students apply previously learned material to new content-related problems. Regarding this study and the use of animation as a means of reinforcement, this could apply to the accurate formation of mental representations of scientific mechanisms. In addition, animations could also act as a metacognitive organization strategy that could lead students to greater understanding (47). Based on these theories, we hypothesize that students who view animations as reinforcement of instruction on topics related to concentration gradients and ATP synthase will outperform those who view animations as preparation for class instruction or for an assessment focused on the presented concepts. The findings of this research will provide insight into instructional “best practices” regarding the use of animation as preparation for and reinforcement of introductory cellular respiration concepts. Understanding the best timing to implement animated instructional resources could provide instructors with guidance on strategies that encourage the highest learning gains in introductory biology students.

MATERIALS AND METHODS

Participants and treatment groups

Participants (n = 732) were enrolled in the introductory biology course at a large public university in the southeast United States during either the fall or spring semester, and all research was conducted in accordance with IRB protocol # 0004606. In this quasi-experimental study, sections were randomly assigned to one of three treatments. The “preparation” group (n = 133) consisted of two class sections (one fall and one spring) that viewed an animation developed as part of the Virtual Cell Animation Collection on concepts related to concentration gradients and ATP synthase prior to attending a lecture-centered class session on the topic. The “reinforcement” group (n = 316) consisted of three class sections (two fall and one spring) that viewed the same animation as a means of reinforcement after they attended a classroom lecture on the topic. The “control” group (n = 283) consisted of two class sections (one fall and one spring) that only attended a classroom lecture on concentration gradients and ATP synthase. This group did not view the animation on the topic either prior to or following instruction. All course instructors (n = 5) were determined to have similar instructional styles and content delivery strategies. Multiple observations of each instructor revealed that all instructors dedicated ~75% of class time to lecture, augmented with ~25% of class time devoted to other interactive techniques (e.g., clicker questions, think-pair-share, etc.). Two of the instructors taught more than one section in this study; however, to control for possible instructor bias, their treatment group varied between sections. Variation in treatment group size was due to uncontrollable variability in student enrollment between course sections. Such variation in course section size is common at this university, and instructors typically do not vary teaching strategies between sections as a result of their enrollment numbers.

Assessment and measures

The assessment used to obtain information on student conceptual understanding was a 10-question instrument (α = 0.66) constructed using questions selected from two commonly used Biology textbooks that were slightly modified to fit the level of the course in this study (Appendix 1). The instrument was designed to remain short so as to prevent interfering with the course syllabus while maximizing student participation. Modifications to make questions more appropriate for the introductory level consisted of removing confusing phrasing and images that were more representative of upper-level biology course concepts. Assessment questions were categorized by the authors according to Bloom’s taxonomy as requiring either lower-order cognitive skills (LOCS), comprised of Bloom-level questions pertaining to knowledge, comprehension, or logic, or higher-order cognitive skills (HOCS), comprised of Bloom-level questions pertaining to analysis, synthesis, or evaluation (48). Six of the questions were determined to require LOCS, while the remaining four were determined to require HOCS, suggesting an overall low- to middle-order of cognitive skill level.

In order to obtain background information concerning student preference for multimedia learning, we included the following question with a five-point Likert scale (1 = strongly agree; 5 = strongly disagree), used to gather information on students’ feelings toward learning with multimedia resources: “I learn best when information is presented in a visually stimulating (i.e., animations/video) fashion.”

Student demographic information was obtained from the University registrar and matched to student performance on the aforementioned assessment. Student identifier data were removed from the dataset.

Instructional animation

The instructional animation used in this study was entitled “ATP Synthase (Gradients)” and is a part of the Virtual Cell Animation Collection (NSF awards: 0086142, 0618766, and 0918955). This set of multimedia resources was developed using the research-based principles of multimedia design (49, 50), and they are free to use for both instructors and students. The Virtual Cell Animation Collection currently consists of 24 animations available for either streaming or downloading in multiple formats from the project’s website (http://vcell.ndsu.edu/animations/).

Experimental procedures

Considering their introductory status, students were all assumed to have had a similar basic introduction to cellular respiration and its components as part of their high school instruction. A sampling of secondary science standards notes that this includes a basic knowledge of concentration gradients, with little application to cellular respiration. At the appropriate point on the instructional calendar, students were introduced to the topic of biological gradients and their role in the functions of the ATP synthase molecule using the experimental treatments outlined below (Fig. 1). All sections were conducted similarly in a traditional, lecture-centered style. Due to the quasi-experimental design of this study and the fact that students participated outside of class, we minimized potential confounding variables when possible. For example, student participation in the viewing of animations was monitored, and those who did not fully complete all assignments were excluded from the research results. All animations were uploaded to the Blackboard learning management system (LMS) page for the course, and student participation with the content was tracked using the statistical features of the Blackboard software package. Course structure did not allow for pretesting of students in this study; however, following instruction, they completed a 10-question assessment instrument (Appendix 1) designed to examine student knowledge on the given topic.

FIGURE 1

Experimental treatment groups as defined by the presence and timing of their interaction with Virtual Cell animations.

Statistical analysis

For each condition, descriptive statistics were compiled and inferential analysis run comparing treatment groups using the R statistical programing package. Student achievement was measured by their score on the assessment instrument following treatment. Analysis of covariance (ANCOVA) was initially used to investigate the effect of possible explanatory variables on assessment score. Variables selected were based on previous suggestions of their contribution to learning with multimedia resources. Following ANCOVA, Tukey’s analysis (51) was used to compare assessment scores across treatment groups and to calculate p values and 95% confidence intervals for differences in means between groups.

RESULTS

Previous studies have noted the possible confounding effects of various demographic factors on learning with multimedia resources (52–56). Therefore, we used statistical methods to examine possible contributors to assessment scores. Demographic variables were based on factors suggesting prior knowledge (previous enrollment in the course), student standardized test scores (total SAT and ACT composite scores), feelings towards multimedia learning (learning preference as defined in methods), and general demographic information (year in school, student gender, and student ethnicity). In an attempt to account for the inability to conduct a pretest, we included both student standardized test scores and previous course enrollment as a proxy for prior knowledge. Student year in school was classified as either underclassman (freshman/sophomore) or upperclassman (junior/senior). Likewise, student ethnicity was classified as either white or underrepresented minority. ANCOVA shows no significant contribution to assessment scores by any of the extraneous variables tested (Table 1). However, the results show a significant influence of treatment condition on assessment scores (F (2, 360) = 14.92, p < 0.001).

TABLE 1

Analysis of covariance table for possible extraneous variables.

Variable	df	Sum Sq	Mean Sq	F Value	p Value
Treatment condition	2	161.95	81.48	14.92	< 0.001
Multimedia learning preference	4	5.56	1.39	0.26	0.91
Gender	1	0.06	0.06	0.01	0.91
Ethnicity	1	4.08	4.08	0.75	0.39
Year in school	1	6.97	6.97	1.28	0.26
SAT composite score	1	0.30	0.30	0.05	0.82
ACT composite score	1	1.78	1.78	0.33	0.57
Previous enrollment	1	0.94	0.94	0.17	0.68
Residuals	360	1,966.06	5.46

Three-group comparison of conditions

In a three-group comparison, students who viewed animations on concentration gradients and ATP synthase activity as either pre-class preparation (mean [M] = 6.43, standard deviation [SD] = 2.46) or post-class reinforcement (M = 6.55, SD = 2.12) both had higher mean scores on the concept assessment compared with students in the control group (M = 5.37, SD = 2.35) (Fig. 2, Appendix 2). Post-hoc comparison of means using Tukey’s analysis shows that, when compared with the control group, both the preparation group (d = 0.44, p < 0.001) and the reinforcement group (d = 0.53, p < 0.001) scored significantly higher on the assessment instrument (Appendix 3). Comparison of means between the preparation group and the reinforcement group shows no significant difference between these two treatment groups (p = 0.87) (Appendix 3).

FIGURE 2

Descriptive statistics for mean score on the follow-up assignment by treatment condition. Bars in the boxes represent the median; the box represents the range between the first and third quartile, and the whiskers represent the standard deviation.

DISCUSSION

Strategies to increase student interaction with material outside of the classroom typically requires participation in activities that either prepare students for classroom instruction or reinforce concepts that have been presented in the classroom (27, 46, 57, 58). As a possible resource for these methods we investigated the use of an animation on the topic of concentration gradients and their role in ATP synthase produced by the Virtual Cell animation project. None of the possible extraneous variables examined in this study were shown to contribute to assessment scores on the topic of concentration gradients and their role in the actions of ATP synthase. This is of particular interest considering most introductory biology courses are populated by a diverse group of students. Multimedia resources that can be effective despite this variability could be beneficial to introductory biology instructors seeking alternative methods of instruction. We do however note that the sample in this study is representative of one institution and may not be representative of all universities. Future extensions of the study presented here would benefit from the investigation of a more diverse sample of student backgrounds. Such a representation may provide a more accurate representation of institutions nationwide.

The experimental focus on the use of animations provides evidence that perhaps multimedia can be a reliable means of content interaction outside of the traditional, lecture-centered classroom, regardless of timing. Reports show that many STEM educators either still rely on this traditional method of content delivery or have experienced negative results when using active learning in the classroom (45, 59). This is also the case at the university where this study was conducted, as the introductory biology instructors typically still use these traditional instructional methods. In this study, we wanted to focus on the specific timing of student interactions outside of the class and not the instruction itself. As part of this focus, the use of a lecture-centered classroom environment allowed us to control for as many possible confounding factors as possible in terms of instructional style, while still maintaining a robust, representative sample population. Our results suggests that, in such a setting, student/content interaction is beneficial but there is no significant difference in learning outcomes between students interacting with content as either preparation or reinforcement. However, it would be of interest to see whether these results could be replicated in an environment where in-class instruction differs in style, such as a more active learning–centered class design. Jensen et al. (44) suggest that, in such an environment, preparation may not be as significant as the classroom instruction itself. Comparison of the results between these two instructional methods could further the understanding of when the implementation of animations outside of the classroom is most effective.

We hesitate to make broad scoping generalizations of these findings due to the relatively short length and lack of full validation of our assessment instrument. However, our results showed that regardless of timing, students who were exposed to animations outside of the classroom performed higher on an assessment on the topic of concentration gradients than the control group. These results support the call for increased student interaction with biology concepts outside of the classroom and point to dynamic animation as an effective means of this interaction. Further expansion on this research could provide a deeper understanding of both student preparation and reinforcement in the learning process.

Limitations and future studies

We acknowledge that the quasi-experimental design of this study introduces a number of possible confounding variables. Our attempts to account for this using random selection of classroom section and the random assignment of classroom sections to treatments helped to minimize the impact of many of these potential confounders. However, future investigations could benefit from a completely randomized experimental design. This design would allow for smaller sample sizes that could be assessed more comprehensively to gain insight into the learning process. Together with the current study, the results of such a randomized study could aid in making more powerful conclusions concerning the use of animations outside of the classroom.

In addition, we feel that it is important to compare student performance using a variety of different topics within the Virtual Cell Animation Collection. The topic of concentration gradients and their role in the actions of ATP synthase is considered relatively novel to students in introductory biology. It would be of interest to see how our results compare with a situation where students are introduced to a more familiar topic (mitosis for example). Further investigation using a variety of different topics and multiple replications could therefore provide insight into which topics provide the most benefit when used as either preparation or reinforcement of concepts.

CONCLUSION

Recent calls to action in the field of undergraduate STEM education have placed a focus on the interaction of students with course materials outside of the classroom setting. Two instructional practices that have been implemented in a number of introductory biology classes to meet these needs are pre-class assignments focused on student preparation prior to class and post-class assignments that place an emphasis on concept reinforcement. In this study we focus on the benefits of these two strategies by using animations on the topic of concentration gradients and their role in the actions of ATP synthase developed by the Virtual Cell Animation Collection. Ultimately, the results of our study show that Virtual Cell animations on the topic of concentration gradients led to equally high achievement when used as either preparation prior to instruction or reinforcement following instruction compared with a non-treatment control group. These findings, together with the results of the presented future extensions, aim to provide introductory biology instructor empirical evidence on the “best practice” for implementation of Virtual Cell animations in instruction. These practices could provide insight into the use of animations as part of introductory biology instruction and how the timing of their implementation could affect the level of student understanding and achievement.

Accessing materials

Materials presented in this paper can be accessed using the Virtual Cell Animation Collection website (http://vcell.ndsu.edu/animations/). There are no requirements needed for access; however, there is an optional registration prompt for individuals who choose to download the materials for personal use. In addition to the project website, the Virtual Cell Animation Collection also has a YouTube site (http://www.youtube.com/user/ndsuvirtualcell) and a free Apple iOS application (http://itunes.apple.com/us/app/virtual-cell-animations/id427893931?mt=8) that will provide access to Virtual Cell content.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.