Investigation of a Stand-Alone Online Learning Module for Cellular Respiration Instruction.

With the recent rise of alternative instructional methodologies such as flipped classrooms and active learning, many core concepts are being introduced outside of the classroom prior to scheduled class meeting times. One popular means for external concept introduction in many undergraduate biology courses is the use of stand-alone online learning modules. Using a group of four large introductory biology course sections, we investigate the use of a stand-alone online learning module developed using animations from Virtual Cell Animation Collection as a resource for the introduction of cellular respiration concepts outside of the classroom. Results from four sections of introductory biology (n = 629) randomized to treatments show that students who interacted with the stand-alone online learning module had significantly higher normalized gain scores on a cellular respiration assessment than students who only attended a traditional lecture as a means of concept introduction (p < 0.001, d = 0.59). These findings suggest a superior ability to convey certain introductory cellular respiration topics in a stand-alone manner outside of the classroom than in a more traditional lecture-based classroom setting.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

A byproduct of the recent push for implementing active learning environments during face-to-face class time in undergraduate science courses is the emphasis that is now placed on student interactions with course material outside of the physical classroom (1, 2). Such a “flipped” classroom environment often requires students to be introduced to core course content prior to class meetings as preparation for the in-class activities (3). This being the case, it is imperative that the resources be created with both accuracy and efficacy in mind. While many instructors design and implement their own materials for out-of-class learning (4, 5), the success of such resources can often vary.

One method of concept introduction outside of the classroom that has previously been shown to be effective is the use of online learning modules with which students can interact on their own time (4, 6–8). In order to develop truly effective multimedia resources, one typically requires knowledge of the many instructional design principles that have emerged from the multimedia learning literature and their effects, as following these design elements has been shown in the literature to have a positive effect on learning (9–11). Those who teach science courses, however, may not have the requisite knowledge of effective multimedia design, or perhaps do not have the technical skills to develop their own stand-alone online learning modules. Hence, the formation of a research-driven database of effective resources for concept delivery outside of the classroom, including resources such as instructional animations and online learning modules, could benefit instructors in the implementation of a more active learning–centered classroom environment.

The Virtual Cell (VCell) team has been working to create freely accessible, online learning resources to support K–12 and undergraduate biology education. One example of this is a comprehensive series of dynamic molecular animations developed using proven principles of multimedia learning (12). Previous research on VCell multimedia resources has shown the ability of VCell animations to propagate learning when they are used both as part of an introductory biology lecture (6) and as a means of either concept introduction or reinforcement (13). Additionally, previous research investigating an online learning module developed using VCell animations on the topic of meiosis has shown its superiority in conveying concepts when compared with a traditional classroom lecture (p < 0.05) (14). Recently, the team has expanded its efforts by developing an online learning module on the topic of cellular respiration that leverages many existing VCell animations. The purpose of this study is to explore the initial efficacy of this online learning module and investigate its usefulness outside of a traditional classroom setting.

The virtual cell animation collection

The VCell Animation Collection (NSF awards: 0086142, 0618766, and 0918955) is a free-to-use series of 26 animations developed to represent the common introductory concepts of cellular and molecular biology (15, 16). Currently, the complete VCell Collection is openly available for either streaming or downloading in a number of different formats from the project’s central website (http://www.vcell.science), the collection’s YouTube channel (http://www.youtube.com/user/ndsuvirtualcell), and a free Apple iOS application that is currently in development. The VCell development team has recently focused on the expansion of their multimedia database by integrating their pre-existing animations into stand-alone learning modules. Previous research on the efficacy of a meiosis learning module developed by the VCell team has shown its ability to promote concept introduction and student achievement at a level greater than a traditional lecture-based classroom session (p < 0.01) (14). As an expansion of these findings, details on the investigation into the use of the newly developed online learning module on the topic of cellular respiration are outlined below.

Common misconceptions about cellular respiration

Cellular respiration is a concept that is central to understanding cellular and molecular biology. However, previous research has shown that introductory biology students harbor many misconceptions concerning cellular respiration that tend to persist even after traditional instruction (16–18). Among these are the role of oxygen in the formation of key molecules, the importance of biological gradients, and the reciprocal relationship between cellular respiration and photosynthesis (16, 17, 19). Research has suggested that possibly central to all of these is that students often associate respiration with breathing, making it difficult to conceptualize the cellular mechanisms involved in energy transformation (20). This deeper understanding of the molecular mechanisms that comprise respiration requires students to coordinate knowledge of biological organization on multiple levels (21). Such connections are often difficult for students and could contribute to the persistence of misconceptions in STEM learning (22).

One instructional resource that has previously been shown to aid in the connection of scientific concepts is the use of dynamic molecular animations (6, 15, 23, 24). In an attempt to address the difficulties in cellular respiration instruction, the VCell Animation team has developed an online learning module on the topic that can be used as a means of concept introduction in a stand-alone manner that students can interact with on their own time and in their own environment. This online learning module was developed using evidence-based multimedia guidelines, such as pre-training and segmentation, that have been shown to increase learning outcomes on many topics (25).

The online learning module is comprised of a general introduction to the topic of cellular respiration that outlines key factors to look for throughout the learning process and is followed by multiple individual sections on the specific steps involved in the molecular mechanism of respiration. Each individual section consists of a short introduction that includes a number of key factors for that section, followed by an animation outlining the mechanism involved in that specific step, and concluding with a brief assessment used to gauge students’ understanding of that specific portion of the module. The module ultimately ends with a number of summative questions emphasizing the key points of the topic overall. These questions were not the assessment questions used to gather normalized gain scores for this study. The investigation into this stand-alone learning module provides empirical evidence for its use in the introductory undergraduate biology classroom.

We attempt to further illustrate the potential benefits of online learning modules developed using animations produced by the VCell Animation Collection by answering the question, “To what extent do Virtual Cell online learning modules aid in the instruction of introductory biology concepts on cellular respiration compared with a traditional classroom lecture?” Due to the strict adherence to published guidelines on multimedia development, we hypothesize that students who interact only with online learning modules on cellular respiration topics will outperform students who only attend a traditional lecture on the same topics on a concept assessment. Results of this investigation will add to the body of empirical evidence on the efficacy of online learning modules developed using animations from the Virtual Cell Animation Collection.

METHODS

Participants and treatment groups

Participants were randomly enrolled in the introductory biology course at a large public university in the southeastern United States. Study participants (n = 629) self-enrolled in one of four sections of an introductory biology course that were randomly assigned to one of two treatments. The learning module treatment group (n = 341) consisted of two class sections that were assigned to only interact with the online cellular respiration learning module as a means of conceptual introduction. The traditional lecture group (n = 288) consisted of two class sections that received instruction on the same cellular respiration content as the module group only as part of a traditional lecture-centered classroom environment. It was determined that all course instructors in the traditional lecture treatment had similar instructional styles and content delivery strategies. Observation of these instructors by the research team revealed that all instructors dedicated ~75% of class time to lecture intermixed with ~25% of class time devoted to other interactive techniques. All research presented was conducted in accordance with Institutional Review Board protocol #0004606 and was granted exempt status prior to data collection. Subject consent was therefore not required prior to participation. However, subjects were not required to complete the study as part of their course if they chose not to.

Assessment and measures

The assessment instrument consisted of 14 questions: ten questions focused on basic understanding of cellular respiration and were used to identify treatment outcomes (Appendix 1); four questions were designed to quantify basic demographic information (Appendix 2). Questions were selected from previous instructor exams assessing introductory-level understanding congruent with introductory biology concepts. Each question was selected to examine typical introductory level cellular respiration concepts, and distractors were included to conform to common misconceptions on specific cellular respiration content. Common points of misconception assessed were the role of oxygen in the biological processes, the function of biological gradients, and the transformation of energy. Included in the demographic information was a question focused on students’ preference for multimedia learning. This information was gathered using the following question on a five-point Likert scale: “I learn best when information is presented in a visually stimulating (e.g., animations/video) fashion.”

Weighted Bloom’s Index was assessed by two independent sources and was calculated for the assessment and found to be 46.67, suggesting a middle-order of cognitive skill level (26, 27). In order to evaluate student outcomes as a result of treatments, a posttest identical to the pretest was given and normalized gain scores were calculated for each student. Cronbach’s alpha was used as a measure of internal consistency of the assessment and was found to be on the low end of the acceptable range (α = 0.67). Strong Cronbach’s alpha scores are typically considered to be 0.80 and above. We note this as a weakness of our study but feel that it directly relates to both the wide variety of prior knowledge amongst students and the relatively small number of questions on our concept assessment.

Experimental procedures

All participants were given an identical pretest during the first week of spring semester classes designed to assess the students’ baseline understanding of the concepts to be introduced later in the semester (Fig. 1). Later in the semester, approximately the midpoint, students from both treatment groups were presented instruction on the topic of cellular respiration. The module group (n = 341 students) was assigned the cellular respiration online learning module as a standalone, out-of-class activity that was to be completed by the student entirely through the Blackboard learning management system. After completing the learning module, students were directed to immediately complete a posttest that measured student knowledge of the presented concepts. Blackboard recorded both the completion of the learning module and the amount of time it took students to finish. Students in the traditional treatment group (n = 288) attended class as normal, and all instruction was given as a part of a traditional classroom lecture. After classroom instruction, students in the traditional treatment immediately completed the posttest via the Blackboard learning management system.

FIGURE 1

Experimental design assessing the effectiveness of the cellular respiration learning module developed from VCell animations as a stand-alone tool in introductory biology.

Student demographic information was obtained from both the University registrar and the student assessment and matched to student achievement outcomes. All student identifier data were removed from the dataset, and statistical analysis was conducted.

Statistical analysis

Only students who completed all aspects of the study were included in the analysis of the results. Normalized gain scores were calculated as a measure of student achievement (28), descriptive statistics were reported, and inferential analysis comparing treatment groups was conducted. Analysis of covariance was used to assess for treatment effect as well as possible explanatory variables on student outcomes. If significant results were found, Cohen’s d was also reported.

RESULTS

A descriptive comparison of the treatment groups shows that students in the online learning module group (n = 341, M = 0.56, SD = 0.39) have higher normalized gain scores than those who received instruction as part of the traditional lecture treatment (n = 288, M = 0.23, SD = 0.69) (Fig. 2, Table 1). Mean normalized gain scores for the online learning module group were categorized as “medium G” (0.7 ≥ G ≥ 0.3) while the traditional lecture group were categorized as “low G” (0.3 ≥ G), as defined by Hake (28). Inferential analysis shows that the differences in learning gains between the two treatment groups were statistically significantly (t(437.77) = 7.15, p < 0.001, d = 0.59).

FIGURE 2

Mean normalized gain scores on the topic of cellular respiration by treatment type. Bold lines represents group means, distribution is represented by width of the plot. *** indicates p < 0.001.

TABLE 1

Descriptive statistics for cellular respiration instruction.

Normalized Gain Score
	Module	Traditional
Min	−1.00	−4.00
1st Q	0.33	0.00
Median	0.66	0.40
Mean	0.56	0.23
3rd Q	0.83	0.67
Max	1.00	1.00
SD	0.39	0.69
95 % CI	0.42 > μ > 0.23

Q = quartile; SD = standard deviation; CI = confidence interval.

Analysis of selected extraneous variables

A number of demographic factors have previously been shown to have possible confounding effects on learning with multimedia resources (24, 29–32).To account for these factors, we used linear regression modeling to examine possible contributors to posttest assessment scores. Demographic variables that were examined were based on factors suggesting prior knowledge (pretest scores and previous enrollment in the course), standardized test scores (total SAT and ACT composite scores), feelings towards multimedia learning (multimedia learning preference as defined in experimental procedures), and other general demographic factors (year in school, student gender, and student ethnicity). Student year in school was classified as either underclassman (freshman/sophomore) or upperclassman (junior/senior). In addition, due to the uneven distribution, student ethnicity was classified as either white or nonwhite. Using the equation outlined below (Eq. 1), linear regression modeling showed a significant association of only pretest score (p = 0.03) on posttest scores from the extraneous variables tested (Table 2). In addition to pretest scores, results show a significant influence of treatment group on assessment scores (p < 0.001).

TABLE 2

Linear regression table for possible extraneous variables, streamlined (cellular respiration).

Variable	Estimate	Standard Error	t Value	p Value
Intercept	4.67	1.06	4.41	< 0.001***
Treatment condition	1.28	0.21	6.03	< 0.001***
Pretest score	0.12	0.06	2.19	0.03*
Previous enrollment	0.12	0.29	0.42	0.67
Total SAT score	0.00	0.00	0.37	0.71
ACT composite score	0.04	0.05	0.75	0.45
Multimedia learning preference	0.03	0.13	0.24	0.81
Year in school	−0.07	1.26	−0.06	0.95
Student gender	0.07	0.23	0.31	0.76
Student ethnicity	−0.45	0.30	−1.50	0.14

p < 0.05.

p < 0.001.

In an attempt to further streamline the analysis of possible extraneous variables, the original equation was truncated to include only those variables initially recognized as significant (Eq. 2). Results again show a significant influence of both treatment condition (p < 0.001) and pretest score (p < 0.001) (Table 3).

TABLE 3

Linear regression table for possible extraneous variables (cellular respiration).

Variable	Estimate	Standard Error	t Value	p Value
Intercept	5.92	0.21	28.68	< 0.001 ***
Treatment Condition	1.37	0.14	9.79	< 0.001 ***
Pretest Score	0.15	0.04	3.94	< 0.001 ***

p < 0.001

Analysis of data for possible section effect

As a means of accounting for a possible class section effect on learning outcomes, subsections defined by treatment condition were conducted using one-way ANOVA to analyze the effects of student section on the normalized gain score. No significant differences in normalized gain scores across sections were noted for either the online module treatment group (F(1, 339) = 0.36, p = 0.55) or the traditional lecture treatment group (F(1, 286) = 0.77, p = 0.38).

DISCUSSION

Recent reports on the state of STEM education have called for reform in a number of different facets of undergraduate studies. Among these is an emphasis on increasing student interaction with course materials outside of the classroom (1, 33). As one possible resource to meet these needs, we examined the use of an online learning module produced using animations from the Virtual Cell Animation Collection. Results indicate that students who interacted with a stand-alone learning module on the topic of cellular respiration showed higher learning gains than students who attended a traditional lecture on cellular respiration (p < 0.001, d = 0.59). With previous studies suggesting a prevalence of misconceptions on this topic (17–19), these results could have implications for alternative instruction strategies in introductory biology. One setting where these findings may be beneficial is a flipped classroom environment, where preparation prior to face-to-face instruction is key (34). A stand-alone learning module that outperforms a traditional lecture setting could provide instructors with trustworthy resources that can be used to present concepts outside of the physical classroom. This could free up classroom time for other instructional strategies that have been shown to increase conceptual understanding, such as active learning (2). Well-developed online learning modules that can deliver content in a fashion equivalent to a traditional lecture could also provide students with an effective means of concept review. Such trusted review materials could serve as a meaningful refresher in upper-level classes that typically build upon introductory concepts. Reinforcement of these basic concepts could provide students a foundation on which to promote deeper learning of upper-level concepts. Both of these aspects of online learning modules could have important implications for the learning process in undergraduate biology.

Results of this study also show that the educational benefits of the stand-alone online learning modules tested here were not affected by many of the common extraneous variables that have been previously shown to influence learning with multimedia resources. The investigation into online learning modules on the topic of cellular respiration show only a possible extraneous contribution by student pretest score. Students enter their undergraduate years with varying levels of prior understanding of cellular respiration (18), causing variation in the starting point of many undergraduate curricula. Due to previous reports on the contribution of prior knowledge to successful interactions with multimedia (35, 36), we suspect that, at least in part, students with higher levels of prior knowledge may be extracting more conceptually from the online learning module than those who exhibit lower levels of prior knowledge. The contributions of pretest score in this study most probably represent a student’s previous exposure to the topic as part of their K–12 education and exhibit this effect of prior knowledge on concept extraction. We acknowledge that students’ familiarity with educational technology could also play a role in the outcomes of this study. However, due to the regression analysis of their self-reported multimedia ability (Eq. 1), we believe that the majority of subjects were equally comfortable in using the technology associated with this report.

Due to the method of data collection used in this study, individualized item responses could not be obtained from student assessments. This data would have allowed us to focus more specifically on individual misconceptions and provided insight on how the module contributed to correcting these matters. We do, however, feel that the assessment used provides an effective broad-reaching net to gather information on student learning about cellular respiration and functions adequately as an introductory tool for analyzing module efficacy. Follow-up studies on this and other VCell modules will examine specific misconceptions in greater depth and will focus on student responses to individual questions pertaining to specific concepts. This will provide a more in-depth view of the benefits of the tested materials and will contribute to the overall goals of the VCell project.

Limitations

While the results of the investigation presented here show the benefits of learning with a stand-alone online learning module compared with a traditional lecture setting, it would be interesting to see whether these benefits are also exhibited when compared with an active learning–centered, flipped classroom environment. Research has previously shown that an active learning–centered environment can outperform a traditional classroom setting (2). We would therefore hypothesize that such a comparison would lead to more comparable learning gains. However, we note that such an active learning–centered classroom environment is not typical in many large undergraduate institutions (37).

We also acknowledge that the quasi-experimental design in this study does have limitations and suggest that smaller-scale studies may be beneficial. The size and scale of the experiments here present a more realistic view of the undergraduate population at the institution of our study and more appropriately answer the research question proposed. However, qualitative data from smaller-scale studies in the future could aid in the development of additional learning modules. We also note that the internal consistency reliability of our instrument did not reach the commonly accepted value of α = 0.80. While the instrument was reviewed by a number of experienced biology faculty at the sample university prior to its use, it could benefit from revisions before being used again in the future.

CONCLUSIONS

In response to the recent emphasis on effective content interaction outside of the classroom, we aimed to answer the question, “To what extent does a Virtual Cell online learning module aid in instruction on cellular respiration concepts compared with a traditional classroom lecture?” Using a stand-alone online learning module produced in accordance with research-supported design guidelines (12, 15), we investigated learning from two different types of content introduction (lecture or online learning module) across four sections of a large introductory biology course. Results show that students who interacted only with an online learning module had significantly higher normalized gain scores than those who were introduced to the topic as part of a traditional classroom lecture (d = 0.59, p < 0.001). Analysis of possible extraneous variables shows only pretest scores as a possible extraneous contributor to the achievement measures in this study. Results of this study provide empirical evidence for the use of online learning modules as a stand-alone form of concept introduction for introductory biology students on the topic of cellular respiration. We plan to expand upon these findings by designing further research targeting specific cellular respiration misconceptions so as to provide further evidence of the efficacy of this learning module. Such evidence can provide instructors confidence in these resources and support the use of such materials to complement alternative instruction strategies in the introductory biology classroom.

Click here for additional data file.