Interactive Computer Simulations as Pedagogical Tools in Biology Labs.

Student learning in biology may be impaired by instructional environments that emphasize technical methodology over analysis. We hypothesized that time gained by experimenting with accurate computer simulations could be used to engage students in analytical, creative learning. The effects of treatments that combined a week of simulated lab instruction with a week of standard lab instruction in different order (E-to-S and S-to-E) were examined using a controlled experimental design with random assignment of lab sections and hierarchical linear modeling analysis to account for possible clustering within sections. Data from a large sample of students ( N = 515) revealed a significant increase (1.59 SD) in posttest scores for both treatment groups over the control. We posit as a plausible explanation the reinforcement of psychomotor learning due to strong engagement of cognitive processes facilitated by the computer simulation. This study supports a wider use of computer simulations as learning tools in laboratory courses.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

Efforts to increase quantitative literacy across the biology curriculum are often hampered by the difficulty of adding more instruction into an already packed schedule and by a paucity of financial and human resources needed to re-educate biology instructors in math and computer science or to staff multidisciplinary teaching teams with biologists, mathematicians, science, technology, engineering, and mathematics (STEM) education specialists, and computer scientists. As a result, students graduating from many biology programs lack sufficient quantitative skills (Gross, 2004; Hoy, 2004). This systemic problem particularly handicaps biology graduates compared with other STEM majors and hinders or even thwarts their success in life science careers that require quantitative skills. Contemporary biology curricula aim to educate students in three complementary areas: context, concept, and literacy (National Research Council, 2003). Traditional biology education focuses primarily on content and concept, while quantitative literacy in mathematics and computer science is addressed through courses offered by specialized departments (Brent, 2004; Gross, 2004; Hoy, 2004). Such separation of biological context and concept from quantitative literacy instruction transfers the burden of integrating knowledge from those areas squarely upon the students. Unfortunately, many otherwise excellent students who graduate from biology programs tend to have an overly contextualized understanding of basic concepts (Masatacusa ). Halpern and Hakel (2003) suggested that decontextualization needs to be actively promoted in biology teaching and recommended abstract representations of scientific concepts as effective decontextualization techniques. The use of simulations and models specifically has been recommended as a technique to enhance the transfer of conceptual knowledge to new contexts (Salomon and Perkins, 1996). Computer hardware and software advances have allowed instructors to model complex biological phenomena. While computer simulations based on mathematical modeling have been developed and used in physics and engineering, and to some degree in chemistry education, their use to enhance biology laboratory courses has been slower to develop (Lemerle ). Methods for integrating computer simulations have been tried and found to either improve student achievement, student attitudes, or both (e.g., Hu ,b), and a review of the relevant literature concludes that the positive learning effects of computer simulations are especially strong in the laboratory setting (Rutten ). Simulations driven by deterministic equations allow students to examine biological phenomena without the encumbrance of noise and experimental errors. Stochastic models, on the other hand, take advantage of mathematical precision while allowing for statistical variations in simulated outcomes, thereby providing students with more realistic simulated data (e.g., Linh and Ton, 2011; Lv and Wang, 2011; Chevalier and El-Samad, 2012).

The conceptual framework guiding this study was also founded in Woods’s (2007) theory of interdisciplinary communicative competence, which defines interdisciplinary competence as an outgrowth of communication across academic knowledge domains and effective communication among academic cultures. In Woods’s theory, the integration of multiple academic areas through a model of inquiry will provide better opportunities to develop more useful competencies and communication skills as well as more positive attitudes.

This study began with the creation of three computer simulations of an acid phosphatase–catalyzed chemical reaction using the scientific programming language MATLAB. The simulations accurately reproduced the results of experimental protocols that have been employed for many years in a standard general biology laboratory course, BIOL 300L. The deterministic simulation works by solving a system of ordinary differential equations, while the stochastic simulation implements the Gillespie algorithm (Gillespie, 1977). Students participating in the study were not instructed in the theory and algorithms behind either simulation, but they could change key simulation parameters by means of an interactive graphical user interface (GUI). Aside from the simulation, all other aspects of the laboratory exercise, including data analysis, report preparation, and assessments, were exactly the same for control and treatment groups. Results from pretest and posttest assessments and quizzes were collected from a large sample of students (N = 515). Hierarchical linear modeling (HLM) analysis of the data confirmed the lack of between-section variability and initial group equivalence. Furthermore, HLM revealed that treatment group posttest scores were higher (1.59 SD) than those of the control group, supporting the general conclusion that computer simulations increased students’ learning of enzyme function.

MATERIALS AND METHODS

Enzyme Kinetics Experiments

The enzyme kinetics laboratory exercise is a part of the BIOL 300L General Biology Laboratory course that all students majoring in biology and biochemistry at University of Maryland Baltimore County (UMBC) are required to take. The exercise comprises 2 weeks of experiments on wheat germ acid phosphatase (Sigma-Aldrich) and a third week that is devoted to data analysis, discussion, and report preparation. During the first week, the exercise explores the effect of temperature and pH on a commercial acid phosphatase preparation that exhibits optimum activity at 37°C and pH 4.8. Supplemental Figure S1, A and B, shows representative results obtained by students during week 1. During the second week, students learn to use Michaelis-Menten and Lineweaver-Burk plots to calculate kinetic parameters, Vmax and KM, for acid phosphatase. They also investigate the effect of two reversible chemical inhibitors, phosphate ion and fluoride ion, on acid phosphatase Vmax and KM (Supplemental Figure S1C). During both weeks, the students measure the activity (rate of reaction) of acid phosphatase on the artificial substrate p-nitrophenyl phosphate, which the enzyme hydrolyzes to the reaction product p-nitrophenol. The standard incubation period for this reaction at 37°C and pH 4.8 is 15 minutes. To calculate the rate of reaction v0 in micromoles of product produced per minute, the students construct a standard curve of concentration versus absorbance for p-nitrophenol. This standard curve reflects the Beer-Lambert law of light absorption by a substance in solution. Further details of the experimental protocols used in weeks 1 and 2 are provided in the Michaelis-Menten.

Enzyme Kinetics Simulations and Graphical User Interface

Computer simulations of the acid phosphatase reaction system were first programmed in MATLAB using a modification of previously published work (Higham, 2008). Briefly, a system of seven mass-action kinetics equations representing a Michaelis-Menten enzyme (Michaelis-Menten) is solved numerically, either in a deterministic (DRRM) or a stochastic (SSAM) manner. The deterministic method is based on a differential equation–solver algorithm (MATLAB), while the stochastic method represents an application of the Gillespie algorithm (Gillespie, 1977). Temperature dependence is modeled with two logistic functions: a negative logistic function represents temperature-dependent enzyme degradation, while a positive logistic function represents temperature-dependent activation. The dependence on pH is represented as an equilibrium between the active enzyme and two inactive forms, EH+ and EOH−, that result from the reversible binding of H+ and OH− ions, respectively (Supplemental Material). Both methods are presented to the students as GUIs that are available from simlabs.umbc.edu, either as downloadable, stand-alone versions or online. Supplemental Figure S2 depicts an annotated screenshot of the stand-alone GUI for a generic enzyme without inhibitor used in week 2, which is also analogous to the online GUI. A user can select the simulation type and input values of elementary rate constants for enzyme and inhibitor (edit control boxes 1–7); reaction parameters (edit control boxes 9–14) such as starting concentrations of substrate, enzyme, and inhibitor (S0, E0, and I0); reaction volume; and simulation and assay times. Values can be entered by hand or selected from a pull-down menu (control box 8) that offers a choice of four different sets of prefit enzyme systems for a generic enzyme, acid phosphatase (without inhibitor), acid phosphatase plus competitive inhibitor (phosphate ion), and acid phosphatase plus non-­competitive inhibitor (fluoride ion). These parameter values are specific for commercial preparations and may change as the enzymes show decreased activity over time. Consequently, the default values of reaction constants listed in the Supplemental Material in Supplemental Table S1 should be recalibrated before use.

The simulations produce concentration profiles of substrate, enzyme, enzyme–substrate complex, and inhibitor complexes that are plotted at the bottom of the GUI. A rate of reaction v0 is calculated automatically for a specified assay time using a spline interpolation method. Individual concentration versus time plots of each chemical species can be selected for display using plot control check boxes labeled “S” (substrate), “E” (enzyme), “ES,” “P” (phosphate), “I” (inhibitor), “EI,” and “ESI.”

Education Research Protocol

A controlled design with random assignment of laboratory sections to experimental groups was used to test the effectiveness of computer simulations in an existing laboratory course, BIOL 300L General Biology Laboratory. The experimental protocol involving human subjects was reviewed and approved in advance by the UMBC Institutional Review Board. Hereafter, the capital letters “E” and “S” that appear in the group designations E-to-E, S-to-E, and E-to-S indicate the type of laboratory protocol used during weeks 1 and 2 (Supplemental Figure 2). The E (experimental) protocol was a standard enzyme kinetics protocol designed to teach students how to set up and incubate enzyme reaction assays and measure the concentration of the yellow-colored reaction product p-nitrophenol using a spectrophotometer. The S (simulation) protocol replaced the physical experiment with an accurate computer simulation of the enzyme-catalyzed reaction. Implementing the S protocol required additional instruction on how to use GUI controls to change the starting concentrations of substrate, enzyme, and inhibitor; how to read and record rate of reaction results; and how to interpret concentration versus time plots. During week 1, the S protocol used specialized GUIs and instructions on how to operate pH and temperature control sliders (Supplemental Material). Table 1 lists the technical learning goals specific to the E and S protocols and the conceptual learning goals that were common to both protocols.

TABLE 1.

Technical and conceptual learning goals of the E and S protocols

Technical learning goals specific to the E protocol

Learn to use volumetric equipment, such as reagent dispenser bottles, graduated cylinders, and pipettes to mix reagent stocks of buffer, substrate, enzyme, and inhibitors Learn to use strong alkali (NaOH) to stop the enzymatic reaction and increase the specific absorbance of the reaction product, p-nitrophenol. Learn to control the pH of solution, and accurately time the duration of the assay (e.g., 15 minutes). Learn to use a temperature-controlled water bath. Learn the Beer-Lambert law of light absorbance and how to use a spectrophotometer to measure absorbance and the amount of a chemical (p-nitrophenol) dissolved in water. Learn various laboratory psychomotor skills important for a profession in the life sciences.

Learn to use a graphic user interface (GUI) to control reaction parameters: pH, temperature, reaction time, and chemical concentrations (enzyme, substrate, inhibitors)

Learn to use a GUI to select preset simulation parameters: elementary rate constants for forward and reverse binding reactions for substrate and inhibitors; enzyme turnover rate.

Learn to read out the amount of product made predicted by the simulation and interpret concentration versus time plots for all chemical species included in the Michaelis-Menten model.

Learn basic concepts of experimental protocol design and execution.

Learn to calculate the activity of an enzyme from the amount of product made during a fixed period (assay time).

Learn to find the optimum pH and temperature of an enzyme-catalyzed reaction.

Learn to calculate the enzyme’s kinetic parameters Vmax and KM for a given substrate and how reversible chemical inhibitors can be used to manipulate those parameters.

Demographic data on gender, English language learning, academic seniority, and major were obtained from student records and from individual responses to the psychometric survey. The sample of 515 students was 56% female and 44% male; 83% of respondents indicated English was their primary language, while 17% declared English was a second language. The sample was composed of juniors and seniors, mostly biology BA, biology BS, and biochemistry and molecular biology majors (90%), who are required to take the BIOL 300L course. The remaining 10% of students had declared majors in chemical engineering, computer science, psychology, and health administration and policy.

A random-number generator was used to assign laboratory sections, each consisting of 24 individuals, to the control and treatment groups. In Spring and Fall, the S-to-E and E-to-S groups comprised four sections each, while the control group E-to-E comprised two sections. In the Summer offering of the course, there were three sections, each assigned randomly to one experimental group. To maintain the same learning objectives for both E and S protocols, when students carried out the S protocol, we did not require them to investigate what would happen if they changed the rate constants of a simulation, but instead told them what would likely happen and allowed them to spend time trying different “what-if” scenarios.

Assessments

Three types of assessments—pretest, quizzes (1 and 2), and posttest—were used to evaluate learning outcomes. The assessment schedule for weeks 1–3 is summarized in Supplemental Figure S3. Immediate feedback on the exercises carried out during weeks 1 and 2 was obtained from two short quizzes that were given in weeks 2 and 3. Quizzes used in the Spring semester were modified for the Fall semester to follow recommendations from item response theory. Further details of Spring and Fall semester quizzes are provided in the Supplemental Material.

During the Spring and Summer semesters, posttest course outcome evaluations were based on eight final exam questions relevant to the enzyme kinetics exercise (Supplemental Material). Item response theory indicated weak discrimination power for two questions, prompting their subsequent modification for the Fall semester section. A total of 515 pretest, quiz, and posttest responses were collected during a three-semester period.

An 11-question, five-point Likert scale psychometric survey was administered to students of all groups at the end of the Spring and Fall semesters. The objective of the survey was to gain demographic information on gender and English language, as well as insight into students’ perceptions and biases and preferences regarding the use of experiments, computers, and computer simulations in a laboratory environment (Supplemental Material).

Missing Data

Of 515 students who participated in this study, 154 (29.9%) were missing at least one value, and all variables had missing data, except the treatment condition indicator variable. Little’s (1987) test of missing completely at random (MCAR) indicated that the missing data patterns were not MCAR, χ2 (30) = 68.99, p < 0.001. Multiple imputation (Rubin, 1987), an expansion of multiple-regression imputation based on Bayesian inference, was used to impute values for the missing data. Multiple imputation is the preferred imputation method for data with missing patterns that are not MCAR (Gelman ). The imputation model regresses each missing data point on every other variable. The true value for the missing data point is considered the mean of a distribution, and sampling error will result in a potentially different value each time a multiple-regression imputation is run. Logistic regression was used to impute missing values for indicator variables. The multiple imputation procedures were run using IBM SPSS Statistics v. 25. Five imputed data sets were produced, as recommended by Brick and Garson (2009).

Data Analysis

Because treatment condition was assigned by section, two-level HLM (Raudenbush and Bryk, 2002) was used to examine possible clustering within sections. The first level (students) consisted of scores on the pretest, quiz 1, quiz 2, and posttest along with gender and English language indicator variables. The second level (section) consisted of the treatment condition and semester indicator variables. The sample consisted of 515 students in 23 sections, an average of 22 students per section. Sections were randomly assigned to treatment condition. Using Optimal Design software (Spybrook ), statistical power was estimated for an alpha level of 0.05. For an intraclass correlation (ICC) of 0.10 (the minimum needed to justify the use of HLM; Byrne, 2012), a statistical power of 0.80 was estimated for a minimum effect size of 0.47. All estimates were computed with full maximum likelihood. All HLM analyses were computed with an HLM of 7.03 (Raudenbush ).

RESULTS

Five imputed data sets were computed from the original. The imputed data sets were comparable to the original data set with no statistically significant t tests between the imputed data sets and the original (Table 2). The overall research goal was to examine the impact of the computer simulation treatment on the posttest. Descriptive statistics by treatment group are therefore provided (Table 3). All three groups appeared to show growth in their means from pretest to posttest. To determine the degree to which that growth was statistically significant and differed across groups, we used HLM to account for the clustering of students within sections. The HLM analyses consisted of five models:

TABLE 2.

Sample size, means, and SDs for each imputed data set

		Imputed data set
	Original	1	2	3	4	5
Pretest
Sample size (N)	513	515	515	515	515	515
Mean	0.69	0.69	0.69	0.70	0.69	0.69
SD	0.30	0.30	0.30	0.30	0.29	0.29
t Ratio from original	—	0.03	0.02	0.05	0.02	0.03
Quiz 1
Sample size (N)	497	515	515	515	515	515
Mean	0.68	0.68	0.68	0.68	0.68	0.67
SD	0.21	0.21	0.21	0.21	0.21	0.21
t Ratio from original	—	0.04	0.01	0.01	0.01	0.11
Quiz 2
Sample size (N)	492	515	515	515	515	515
Mean	0.72	0.72	0.72	0.72	0.72	0.72
SD	0.26	0.26	0.26	0.27	0.26	0.27
t Ratio from original	—	0.03	0.00	0.02	0.08	0.15
Posttest
Sample size (N)	513	515	515	515	515	515
Mean	0.98	0.98	0.98	0.98	0.98	0.98
SD	0.26	0.27	0.27	0.26	0.26	0.26
t Ratio from original	—	0.03	0.01	0.02	0.02	0.03

TABLE 3.

Descriptive statistics by treatment groupsa

	E-to-E (control)			S-to-E (treatment week 1)			E-to-S (treatment week 2)
Variable	N	Mean	SD	N	Mean	SD	N	Mean	SD
Pretest	284	0.686	0.256	111	0.705	0.337	118	0.702	0.340
Quiz 1	270	0.700	0.206	109	0.651	0.208	118	0.644	0.222
Quiz 2	270	0.711	0.276	107	0.729	0.247	115	0.724	0.252
Posttest	284	0.886	0.227	111	1.066	0.268	118	1.113	0.261

aDescriptive statistics are based on the original data set (un-imputed). The E-to-E group consisted of 13 sections. S-to-E and E-to-S groups consisted of five sections each.

Unconditional model (no predictors at the section or student levels)

Analysis of covariance (ANCOVA) model to account for student-level effects on the posttest

Random effects model to assess the degree of between-section variability in the student-level covariates

Treatment effects model to determine the effect of being in one of the treatment groups on the posttest.

Treatment comparison model to determine differences in the effect of being in treatment week 1 (S-to-E group) or treatment week 2 (E-to-S group) on the posttest.

Unconditional Model

The unconditional model (Eqs. 1 and 2) was computed to determine the overall amount of variance at the section level (Level 2) and to determine the amount of variance between sections using the ICC.

where

POST = the posttest score for student i in section j

γ00 = the intercept term representing the grand mean on the posttest

u0 = the unique effect for each section j on the posttest grand mean (i.e., variance term)

β0 = the section means on the posttest

r = the unique effect for each student i in section j on the posttest grand mean (i.e., variance term)

The values for the unconditional model were γ00 = 0.972, p < 0.001; u0 = 0.024, χ2 (df = 22) = 292.76, p < 0.001; r = 0.046. The ICC was 0.342, meaning that ∼34% of the variance was at the section level. Byrne (2012) recommended a minimum ICC of 0.10, so the continued use of HLM was deemed to be justified. The reliability estimate for the group means (β0) was 0.920, providing further evidence that the group means varied substantially across sections. Robust SEs were nearly identical to the ordinary SE estimates, indicating that the assumption of a normal distribution was unlikely to have been violated. Using the unconditional model as a baseline, the student model was developed to explain the impact of as many student characteristics as possible that may have been confounded by section effects (Ma ).

ANCOVA Model

The student model analysis began with an ANCOVA model, in which student-level predictors of the posttest were added to the model as fixed effects, that is, with no variance term at the section level. Using backward regression to develop the student model, all student-level variables were added to the unconditional model (Eqs. 3–9).

where

POST = the posttest score for student i in section j

γ00 = the intercept term representing the grand mean on the posttest

γ10 = the overall female slope

γ20 = the overall English language learner (ELL) slope

γ30 = the overall pretest slope

γ40 = the overall quiz 1 slope

γ50 = the overall quiz 2 slope

u0 = the unique effect for each section j on the posttest grand mean (i.e., variance term)

β0 = the section means on the posttest

β1 = the section female slopes

β2 = the section ELL slopes

β3 = the section pretest slopes

β4 = the section quiz 1 slopes

β5 = the section quiz 2 slopes

r = the unique effect for each student i in section j on the posttest grand mean (i.e., variance term)

Because the slopes were fixed effects, the section slopes (β1–β5) were held constant across sections to their overall respective slopes (γ10–γ50). This model does not test differences between sections; it measures the overall impact of the predictors on posttest. The female and ELL variables were dichotomous, so the slopes represent the impact on the posttest of being female or an English language learner. The variables pretest, quiz 1, and quiz 2 were group mean centered in this and all subsequent models, so their slopes (coefficients in Table 4) represent the impact on the posttest of being higher or lower than the section average on the predictor variables.

TABLE 4.

Fixed and random effects of the ANCOVA model

Fixed effects	Coefficient	SE	df	t Ratio
Posttest mean, γ00	0.978	0.033	22	30.02***
Female slope, γ10	−0.007	0.019	320	−0.35
ELL slope, γ20	−0.011	0.024	23	−0.45
Pretest slope, γ30	0.158	0.036	487	4.41***
Quiz 1 slope, γ40	0.141	0.042	487	3.37***
Quiz 2 slope, γ50	0.109	0.040	487	2.74***

***p < 0.001.

The reliability estimate for the section means (β0) was 0.929, indicating that even after adding student-level predictors to the model, the group means varied substantially across sections. Pretest, quiz 1, and quiz 2 scores were significant predictors of the posttest. The slopes for the female and ELL indicator variables were statistically nonsignificant (Table 4). These two variables were therefore removed from the model for parsimony, resulting in Eqs. 10–14. Coefficients and tests of statistical significance (Table 4) were similar to those of the full ANCOVA model.

Robust SEs were nearly identical to the ordinary SE estimates, indicating that the assumption of a normal distribution was unlikely to have been violated.

Random Effects Model

The next model (Eqs. 15–19) added random effects to pretest, quiz 1, and quiz 2 slopes to determine the degree to which the impact of the predictors on posttest varied between sections.

where

POST = the posttest score for student i in section j

γ00 = the intercept term representing the grand mean on the posttest

γ10 = the overall pretest slope

γ20 = the overall quiz 1 slope

γ30 = the overall quiz 2 slope

u0 = the unique effect for each section j on the posttest grand mean (i.e., variance term)

u1 = the unique effect for each section j on the pretest slope (i.e., variance term)

u2 = the unique effect for each section j on the quiz 1 slope (i.e., variance term)

u3 = the unique effect for each section j on the quiz 2 slope (i.e., variance term)

β0 = the section means on the posttest

β1 = the section pretest slopes

β2 = the section quiz 1 slopes

β3 = the section quiz 2 slopes

r = the unique effect for each student i in section j on the posttest grand mean (i.e., variance term)

All fixed effects (γ’s) were statistically significant (Table 5). Pretest, quiz 1, and quiz 2 were therefore retained as predictors of posttest. Robust SEs were nearly identical to the ordinary SE estimates, indicating that the assumption of a normal distribution was unlikely to have been violated.

TABLE 5.

Fixed and random effects for random effects model

Fixed effects	Coefficient	SE	df	t Ratio
Posttest mean, γ00	0.972	0.034	22	28.88***
Pretest slope, γ10	0.170	0.038	22	4.44***
Quiz 1 slope, γ20	0.144	0.052	22	2.79*
Quiz 2 slope, γ30	0.104	0.040	22	2.61*

*p < 0.05.

***p < 0.01.

The reliability estimate for the group means (β0) was 0.932, indicating that section variability was large for the posttest. The reliability estimates for the section slopes for pretest (β1), quiz 1 (β2), and quiz 2 (β3) were 0.249, 0.144, and 0.146, respectively. These reliability estimates indicated that section variability was small for all three predictor variables. Their random effects were also not statistically significant (u1, u2, and u3 in Table 5). The lack of between-section variability in the pretest provides evidence of initial group equivalence. Their variance terms (u coefficients) were therefore removed in subsequent models for parsimony. Their removal made the revised ANCOVA model (Eqs. 10–14; Table 6) the comparison model for subsequent models.

TABLE 6.

Fixed and random effects for revised ANCOVA model

Fixed effects	Coefficient	SE	df	t Ratio
Posttest mean, γ00	0.972	0.033	22	28.89***
Pretest slope, γ10	0.160	0.032	489	4.96***
Quiz 1 slope, γ20	0.142	0.048	489	2.99**
Quiz 2 slope, γ30	0.109	0.037	489	2.94**

**p < 0.01.

***p < 0.001.

Impact of Treatment Condition

The next model added the treatment condition to determine its impact on the posttest mean (Eqs. 20–24). The treatment variable combined sections in week 1 (S-to-E) and week 2 (E-to-S) treatment groups into a single group to determine the overall effect of being in a treatment group.

where

POST = the posttest score for student i in section j

γ00 = the intercept term representing the mean of the control group on the posttest

γ01 = the slope of treatment on the control group posttest mean

γ10 = the overall pretest slope

γ20 = the overall quiz 1 slope

γ30 = the overall quiz 2 slope

u0 = the unique effect for each section j on the posttest (i.e., variance term)

β0 = the section means on the posttest

β1 = the section pretest slopes

β2 = the section quiz 1 slopes

β3 = the section quiz 2 slopes

r = the unique effect for each student i in section j on the posttest grand mean (i.e., variance term)

Because the treatment variable was dichotomous, its slope (γ01) represents the value added to the posttest control group mean for being in the treatment group, which was statistically significant (Table 7). The reliability estimate for the section means (β0) was 0.884, indicating that section variability remained large for the posttest after accounting for the effect of the treatment.

TABLE 7.

Fixed and random effects for treatment model

Fixed effects	Coefficient	SE	df	t Ratio
Posttest mean, γ00	0.884	0.035	21	25.22***
Treatment slope, γ01	0.203	0.053	21	3.82***
Pretest slope, γ10	0.160	0.032	489	4.96***
Quiz 1 slope, γ20	0.142	0.048	489	2.99**
Quiz 2 slope, γ30	0.109	0.037	489	2.94**

**p < 0.01.

***p < 0.001.

The treatment model resulted in a 41.8% reduction in variance in the posttest from the revised ANCOVA model (Eqs. 10–14; Table 6). The t value for γ01 (fixed effect for treatment) corresponded to an effect size of Cohen’s d = 1.59 (Lipsey and Wilson, 2001), meaning that the treatment group scored on average 1.59 SD higher than the control group.

Comparison of Treatment Conditions

Some treatment sections occurred during week 1 (S-to-E group) and others in week 2 (E-to-S group). The treatment comparison model (Eqs. 25–29) examined the effect of the order of the treatment.

where

POST = the posttest score for student i in section j

γ00 = the intercept term representing the mean of the control group on the posttest

γ01 = the slope of treatment week 1 (S-to-E group) on the control group posttest mean

γ02 = the slope of treatment week 2 (E-to-S group) on the control group posttest mean

γ10 = the overall pretest slope

γ20 = the overall quiz 1 slope

γ30 = the overall quiz 2 slope

u0 = the unique effect for each section j on the posttest (i.e., variance term)

β0 = the section means on the posttest

β1 = the section pretest slopes

β2 = the section quiz 1 slopes

β3 = the section quiz 2 slopes

r = the unique effect for each student i in section j on the posttest (i.e., variance term)

The slopes for both treatment groups were statistically significant (Table 8). The coefficients for each group were used to compute an effect size difference. The t value for γ01 (fixed effect for treatment in week 1) corresponded to an effect size of Cohen’s d = 1.11 (Lipsey and Wilson, 2001). The t value for γ02 (fixed effect for treatment in week 2) corresponded to an effect size of Cohen’s d = 1.46 (Lipsey and Wilson, 2001). The effect size difference was small, 0.35, which is less than half an SD.

TABLE 8.

Fixed and random effects for treatment comparison model

Fixed effects	Coefficient	SE	df	t Ratio
Posttest mean, γ00	0.884	0.035	20	25.47***
Treatment week 1 slope, γ01	0.175	0.066	20	2.66*
Treatment week 2 slope, γ02	0.230	0.066	20	3.50**
Pretest slope, γ10	0.160	0.032	489	4.96***
Quiz 1 slope, γ20	0.142	0.048	489	2.99**
Quiz 2 slope, γ30	0.109	0.037	489	2.94**

*p < 0.05.

**p < 0.01.

***p < 0.001.

DISCUSSION

The present study showed that combining a hands-on laboratory curriculum with a computer simulation of the same experiment (herein described as “treatment”) increased posttest scores on average 1.59 SD relative to a control group, allowing for the conclusion that a computer simulation improved conceptual understanding of the reaction type under study. A treatment comparison model designed to test a possible effect of the order of the treatment (S-to-E vs. E-to-S) uncovered an effect size difference of 0.35 in favor of the E-to-S treatment (treatment week 2). However, that difference was less than half an SD, and its meaning remains uncertain until a further, larger study can be carried out. In the future, it would be interesting to explore the possibility that undertaking a physical experiment to develop psychomotor skills in advance of a simulation may afford students the knowledge structures, or schemata, to which they could “attach” the concepts contained in the simulation (Ambrose ). The takeaway of our analysis is clear: combining a physical instructional environment with a simulation that recapitulates an important part of that environment can enhance learning. This concept will inform others’ use of simulations in laboratory courses.

Training students to become skilled in technical tasks has practical value, but it also detracts from teaching them how to think about the science behind the experiments. Generating accurate enzyme kinetics data with computer simulations reduced the amount of time and effort that students had to devote to technical chores during the class period. A perception survey (Supplemental Material) confirmed the students’ general acceptance of the simulation exercises, which will reassure instructors of the appropriateness of using computer simulations in comparable courses.

Although our data demonstrated that using a quantitatively accurate simulation of an actual laboratory experiment can enhance conceptual understanding, we believe that we will see a stronger effect on learning when we employ the simulation more robustly—that is, as a tool for students to actively explore the relationship between the variables (enzyme concentration, substrate concentration, inhibitor concentration, reaction rate constants, etc.) and the outcome of the experiment, rather than just doing a single run of the simulation according to one set of conditions, as was done here. Evidence shows that two of the strongest factors influencing learning are repeatedly testing oneself or being tested on the concept (Glover, 1989; Karpicke and Roediger, 2007) and receiving immediate feedback on the result (McKendree, 1990; Hattie and Timperley, 2007). A simulation is in fact a perfect tool to achieve both types of cognitive stimulation, as students can change the input variables to test an idea or hypothesis they have—or even in the absence of an idea or hypothesis—and immediately see the result. Effectively, students can perform hundreds of experiments using a simulation in the same time it will take them to do one or a few physical experiments. Having students experiment with the simulation and extract the conceptual principles from their results is probably the most robust use of a simulation, and it is the method we plan to employ in the future with this simulation and others.

Fundamentally, we want students to understand not just acid phosphatase function, but enzyme function in general, and to understand not just the kinetics of enzyme-catalyzed reactions, but rate of change in general as a component of biophysical systems. That is, we want students to be able to decontextualize the concepts they learn from one example to the extent that they can transfer them across situations. Learning that is too bound to context has been described as having an “irrelevant concreteness” (Sloutsky ) that limits development and scope of conceptual understanding. By employing an accurate simulation, we can accomplish more decontextualization and transferability to other systems and other courses (Salomon and Perkins, 1996).

Click here for additional data file.