What's in a Prerequisite? A Mixed-Methods Approach to Identifying the Impact of a Prerequisite Course.

Despite the ubiquity of prerequisites in undergraduate science, technology, engineering, and mathematics curricula, there has been minimal effort to assess their value in a data-driven manner. Using both quantitative and qualitative data, we examined the impact of prerequisites in the context of a microbiology lecture and lab course pairing. Through interviews and an online survey, students highlighted a number of positive attributes of prerequisites, including their role in knowledge acquisition, along with negative impacts, such as perhaps needlessly increasing time to degree and adding to the cost of education. We also identified a number of reasons why individuals do or do not enroll in prerequisite courses, many of which were not related to student learning. In our particular curriculum, students did not believe the microbiology lecture course impacted success in the lab, which agrees with our analysis of lab course performance using a previously established "familiarity" scale. These conclusions highlight the importance of soliciting and analyzing student feedback, and triangulating these data with quantitative performance metrics to assess the state of science, technology, engineering, and mathematics curricula.

INTRODUCTION

Prerequisite courses are a highly ingrained facet of higher education, yet only a handful of studies examine their impact in a scholarly manner (Rovick ; Forester ; Soria and Mumpower, 2012; Reilly and Tomai, 2013). These works typically generate conclusions based on various surveys of student achievement (e.g., comparing overall course grade for cohorts who have or have not completed a recommended prerequisite course) or by measuring the correlation between graduate school grades and grades in undergraduate prerequisite courses (Choudhury ; Green ; McMillan-Capehart and Adeyemi-Bello, 2008; Wright ; McRae, 2010). Such broad evaluations may not accurately measure the possibly nuanced influence of prerequisite courses on student learning, as they assume a large degree of overlap between a prerequisite and a later course or program. For example, a prerequisite molecular biology course that discusses translation may set the stage for a cell biology course, but its direct impact on exams focusing only on cell–cell communication or the role of the cytoskeleton may be more minimal.

To achieve a more granular assessment of prerequisites, we previously developed a familiarity scale, in which we defined familiarity as the ability of a student to answer an exam question based on the content taught in a prerequisite course (Shaffer ). Very familiar (VF) questions are those a student should be capable of answering after completing the prerequisite, familiar (F) questions cover concepts that were touched upon in the prerequisite, and not familiar (NF) questions focus on material that was not discussed in the prerequisite. In this way, it is possible to examine the impact of prerequisites on exam performance on a topic-by-topic basis. From these results, we identified that, at best, students performed better only on VF exam questions, and there was no difference in performance on F versus NF questions (Shaffer ).

Exam data used to measure course or program effectiveness can also be combined with other types of information, including peer evaluation of faculty instruction, quantitative or qualitative student feedback, or student performance on validated instruments, among others (Berk, 2005; Borden and Kernel, 2010; Kuh ). This array of available resources also highlights another issue with prerequisite assessment: it generally occurs solely through measures of student performance, as is the case with the examples cited earlier. This ignores the impact of a prerequisite course on student attitudes regarding science, confidence, belongingness, or other noncognitive measures (Abouserie, 1995; Luzzo ; Hoffman ; Perkins ). Additionally, a lack of student input in such an assessment paints a one-sided picture of a course or program, as research has shown that student and instructor perspectives of a given experience can vary (James ; Kazerounian and Foley, 2007; Tanner ).

With the many calls in the past decade to improve science, technology, engineering, and mathematics (STEM) education (National Research Council, 2009; American Association for the Advancement of Science, 2011; President’s Council of Advisors on Science and Technology, 2012), it is important that changes to programs or curricula be driven by data. Our study focuses on one such modification to the biological sciences curriculum at an R1, research-intensive, university, where it was decided that a long-standing microbiology lecture (MLec, a prerequisite) and microbiology lab (MLab) course pairing would be uncoupled and that students would be free to enroll in the lab course without first completing the lecture. This was done based on the MLab instructor’s perception, through anecdotal evidence, that success in that course was independent of MLec. To see whether this is the case, we implemented our familiarity scale to uncover the impact of MLec on MLab exam performance and coupled this analysis with student interviews and survey data to identify student perceptions regarding prerequisites in this particular scenario and in general. Both sources of data provided complementary evidence that MLec does not impact success in MLab, affirming the curriculum change, while uncovering a number of interesting points regarding students’ thoughts on prerequisite courses. This work reinforces the importance of data collection to support program changes and the value of the student perspective in our drive to improve curricula for the next generation of STEM practitioners.

METHODS

Data Collection

This study was conducted at a large, public, R1 research university in the western United States and focused on microbiology lecture (MLec) and microbiology laboratory (MLab) courses. MLec and MLab are upper-division courses taken by third- through fifth-year students (the vast majority being in their fourth year). These individuals are primarily biological sciences majors, but a small fraction consists of other majors such as public health. These students have completed a series of lower-division “core” courses before enrolling in MLab and/or MLec, which include genetics, molecular biology, biochemistry, and ecology. Before 2013, MLec was a required prerequisite course for MLab. After this point, MLec was no longer required for enrollment in MLab, although it was treated as a recommended prerequisite. The previously mentioned lower-division molecular biology course now acts as the MLab prerequisite. The curricula and course structures of the MLec and MLab courses from 2013 until the 2014–2015 academic year (when data collection occurred) remained essentially unchanged. For this analysis, three types of data were collected: 1) surveys and 2) interviews with students who had completed MLab in Winter 2015, and 3) MLab exam data from the Fall 2014 and Winter 2015 academic quarters.

An online survey (Supplemental Material) was distributed to students who had just completed MLab in Winter 2015. Participation in the survey was completely optional (there was no participation grade associated with completion of the survey, as the quarter was over), although the response rate was still 63% of the total course enrollment. The survey contained both multiple-choice and free-response questions regarding student perceptions of prerequisites in the context of MLab and in general, as well as why they chose to take or not take MLec. One of the questions in the survey asked whether students were willing to participate in an interview regarding their perceptions on prerequisites. Forty-seven subjects responded that it was acceptable to contact them.

Semistructured interviews were conducted with 29 of the 47 students. Those individuals not interviewed either could not be interviewed due to scheduling conflicts or did not respond to two emails asking to schedule an interview. The interviews focused on a variety of prerequisite-related topics (questions provided in the Supplemental Material), including positive and negative attributes of prerequisites and the perceived impact of MLec on MLab performance. Students were probed to clarify unclear responses or to elaborate on responses when the interviewer felt it necessary. Interviews were audio-recorded, deidentified transcripts were generated, and transcripts were coded by a team of two researchers (L.A.C., V.W.C.).

Additionally, exam data from students enrolled in MLab during Fall 2014 (35 exam questions) and Winter 2015 (36 exam questions) quarters were collected from three exams in each quarter. Exams included both open-ended and multiple-choice questions. Before the analysis, all exam questions were normalized (by converting points earned to a percentage) so that each question carried equal weight in our analysis regardless of its point value. To aid in analysis, we obtained student demographic data, including gender, ethnicity, and grade point average (GPA), from the university registrar (Table 1).

TABLE 1.

Descriptive statistics of MLab studentsa

	MLab Fall 2014	MLab Winter 2015
Number of students (n)	94	76
Gender
Male	42 (44.7%)	29 (38.2%)
Female	49 (52.1%)	46 (60.5%)
Unknown	3 (3.2%)	1 (1.3%)
Ethnicity
White	11 (11.7%)	13 (17.1%)
Asian	67 (71.3%)	54 (71.1%)
URM	13 (13.8%)	8 (10.5%)
Unknown	3 (3.2%)	1 (1.3%)
MLec enrollment
Spring 2014	33 (35.1%)	22 (28.9%)
Summer Session 1 2014	9 (9.6%)	8 (10.5%)
Summer Session 2 2014	8 (8.5%)	10 (13.2%)
Did not take MLec	44 (46.8%)	36 (47.6%)

aDescription of students who enrolled in MLab during the two study quarters. URM includes African-American and Hispanic students. MLec enrollment refers to the quarter in which the indicated students took MLec.

This study was performed with approval from the University of California, Irvine, Institutional Review Board (HS# 2012-9191).

Familiarity Designation

MLab exam questions were characterized by familiarity (using either MLec lecture slides or instructors) and Bloom’s level, as previously described (Krathwohl, 2002; Crowe ; Shaffer ). Lecture slide familiarity was designated by a team of four researchers (A.K.L., U.A., J.V.D., S.J.D.) using MLec lecture slides from either the Spring 2014, Summer Session 1 (SS1) 2014, or Summer Session 2 (SS2) 2014 academic quarters. A separate familiarity characterization for MLab exam questions occurred using each set of lecture slides. All four members of the team independently characterized the familiarity level (VF, F, NF) identically for 78% of the exam questions, and three of the four individuals agreed on 95% of the questions. The team discussed questions for which the familiarity designation was not unanimous until a consensus was reached. Each of three different instructors (P.M., G.P., J.E.B.) taught one quarter of MLec and characterized the familiarity level of the MLab exam questions in the context of their particular courses. Thus, the MLab exam questions were categorized by familiarity in six different manners (Spring 2014 MLec lecture slides, Spring 2014 MLec instructor, SS1 MLec lecture slides, SS1 MLec instructor, SS2 MLec lecture slides, SS2 MLec instructor).

Data Analysis

Student responses to the online survey were segregated based on whether or not the student had completed MLec before enrollment in MLab. Quantitative survey data (yes/no or Likert-scale responses) were tabulated and, when appropriate, were analyzed using the Kruskal-Wallis rank-sum test to identify whether there were differences in the distribution of responses between students who did and did not take MLec (Figure 1). Qualitative responses to a question asking why students did or did not enroll in MLec were coded by a team of two researchers (L.A.C., V.W.C.). This was an iterative process. The researchers initially read through 25% of the responses independently and then met to discuss the codes each had created, as codes were not established before reading the responses. The team met with B.K.S. and J.H.S. to discuss whether the decided-upon codes overlapped to establish the list of codes used to analyze the remaining responses. L.A.C. and V.W.C. were open to the addition of further codes when reading all responses, although no further codes were added during this process. When complete, they had agreed on the characterization of 92% of the comments with a Cohen’s kappa (κ) of 0.88. The team discussed any comments not originally agreed upon until a consensus was reached.

FIGURE 1.

Students’ perceptions of the value of MLec for later success in MLab. (A) Interviews were conducted with students who had completed the MLab course. They were asked the question “Did you feel that there would be an advantage to taking MLec before MLab?” in the context of before the MLab course began and after it was completed. The sample population interviewed consisted of 16 students who had taken MLec and 13 who had not. The graph indicates the fraction of interviewees who thought “yes,” there would be an advantage. (B) An online survey asked students who had completed MLab to rate their agreement with the statement “I believe that someone completing MLec before enrolling in MLab would earn a higher grade in the lab” on a 5 point Likert scale. Responses were categorized based on whether the student had (n = 34) or had not (n = 28) completed MLec before enrolling in the lab. Responses from students who did or did not take MLec were not significantly different (p = 0.28 by Kruskal-Wallis rank sum test).

Interview transcripts were similarly analyzed with both quantitative and qualitative methodologies (Tables 2 and 3). Three different types of comments (why students did/did not enroll in the prerequisite, perceived positive attributes of prerequisites, and perceived negative attributes of prerequisites) were coded by the same two researchers, who agreed on the characterization of 91, 80, and 84% of the comments with κ = 0.80, 0.76, and 0.79, respectively. This process was similar to the one described earlier regarding survey response analysis, although in the case of the “why students did/did not enroll” question, the coding scheme had already been generated using the survey responses.

TABLE 2.

Student perceptions of prerequisites

Category	Fraction of interviews with representative comment	Example quotea
A. Students’ perceived positive attributes of prerequisites
Background knowledge	89.3%	“To give you a background before you take the class so you’re not completely lost or new to the subject.”
Acts as a safety net for students	35.7%	“Prerequisites are handy in that they allow students to be prepared for the material and not go in floundering.”
Responsible for future success	25.0%	“I think the benefit that they’re aiming for is to make people more successful in the class in terms of grades.”
Contributes to interest in subject material	21.4%	“It exposes you to different aspects of the field of biology…and then you can find things that you might like to do.”
Positively impacts how instructors teach	14.2%	“Since ten weeks is kinda limited, whoever’s teaching that class can…review quickly [the first week] what was in that prerequisite so that he has nine weeks to teach what he wanted to.”
Improves student behaviors	7.1%	“I don’t think it’s the material that they want you to remember. I think it’s how to approach the material and how to study it and what to take out of it.”
Improves overall quality of students	7.1%	“It’s a good thing because it makes the class size smaller and my peers now are more dedicated.”
Scheduling	7.1%	“Juniors or seniors who may be looking to graduate…might want to take a certain lab class or upper division class…and that would sort of keep freshmen and sophomores from taking spots in that class.”
B. Students’ perceived negative attributes of prerequisites
Scheduling	51.7%	“If someone really wants to take a lab or class and they have so many prerequisites then they wouldn’t be able to fit it in their schedule. It wouldn’t be cool if someone was really motivated to take a class that they couldn’t take.”
Waste of student’s time or money	37.9%	“The downside is that you have to really invest a lot of time into a path that you might not even like in the end.”
Not used as intended by faculty	31.0%	“The classes seem like the faculty aren’t communicating. The professors should talk about what they teach…so they don’t have to teach it again.”
Students are prepared without the prerequisite	17.2%	“I could’ve taken the class without the prerequisite and done probably fine as well.”
Not used as intended by students	13.8%	“Sometimes if some people are more concerned with the end result then they can gloss over the first class to take the next class and not pay attention to what is happening.”

aInterview responses to the question “What are positive (A) and negative (B) aspects of prerequisites?” n = 29 interviews.

TABLE 3.

Factors that influence prerequisite course enrollment

Category	Fraction of interviews with representative comment	Example quotea
A. Students’ reasons for taking the recommended MLec prerequisite
Background knowledge	46.7%	“I wanted to take micro and I thought, oh well, it might be a good idea to take the lecture before lab…so that you could have some sort of base to build off of.”
Scheduling	40.0%	“Just for scheduling, I thought taking the lecture during that quarter would work out with my other classes.”
Graduate school requirement/future plans	26.7%	“I want to go into nursing, so a lot of them usually require both [lecture and lab].”
Interest in course	13.3%	“I thought it would be interesting.”
Thought it was a prerequisite	13.3%	“I honestly thought it was a prerequisite for the lab.”
Upper-division elective	6.7%	“I was trying to find a class that would fit my upper-division elective.”
B. Students’ reasons for not taking the recommended MLec prerequisite
Not a prerequisite	50.0%	“My friend told me it was do-able without the lecture.”
Scheduling	35.7%	“When I tried to build my schedule, I couldn’t fit micro lecture in so I didn’t really put any thought into it.”
No interest in the subject	14.3%	“I’m not particularly interested in microbiology as a subject.”
Prior background	7.1%	“I felt that I had a good enough background in microbiology and bacterial techniques that I could pick up things as I went along and not be left behind.”

aInterview responses to the question “Why did you (A, n = 15 students) or did you not (B, n = 14) take MLec before enrolling in MLab?”

Exam performance was analyzed in two distinct ways.

In the first analysis (Figure 2, Table 4, and Supplemental Table S2 and Figure S1), we segregated MLab students based on whether they did or did not previously enroll in MLec. Those who enrolled in MLec were further segregated based on the particular quarter they had enrolled in that course (Spring, SS1, or SS2). Average performance on VF, F, and NF MLab exam questions was then compared between these MLec groups and the students who did not take MLec (Spring MLec student performance vs. no MLec student performance, SS1 vs. no MLec, SS2 vs. no MLec). These comparisons were made using two different methods to assign question familiarity (MLec lecture slides or instructors). The significance of the differences in average performance on VF, F, and NF questions between students who did and did not take MLec in these various scenarios was determined using multiple pairwise Student’s t tests.

FIGURE 2.

Comparison of MLab performance on familiar questions between students who had or had not taken MLec before enrolling in MLab. Mean scores and SEM on VF (A) or F (B) MLab exam questions. Students in MLab were segregated and familiarity was assigned based on the specific MLec section they enrolled in. Familiarity was designated by either MLec lecture slides or an MLec instructor as indicated. Differences were not significant by t test. MLec sections include Spring 2014 (Sp), Summer Session 1 2014 (SS1), and Summer Session 2 (SS2). Differences in the heights of each bar across MLec sections or familiarity designation methods is due to the fact that questions are segregated distinctly in each of these scenarios.

TABLE 4.

Multiple regression analyses examining factors influencing MLab exam performance; familiarity designated by Spring MLec instructora

	Estimate (±SEM)	p Value
Familiarity category: very familiar (r2 = 0.22)
Intercept	0.19 (0.09)	0.04*
Ethnicity (Caucasian)	0.03 (0.03)	0.41
Ethnicity (URM)	−0.03 (0.03)	0.38
Gender (M)	0.02 (0.02)	0.44
GPA	0.13 (0.15)	5.3e–06***
MLec (yes)	−0.00 (0.02)	0.76
Familiarity category: familiar (r2 = 0.16)
Intercept	0.16 (0.11)	0.16
Ethnicity (Caucasian)	0.01 (0.04)	0.42
Ethnicity (URM)	0.03 (0.04)	0.42
Gender (M)	−0.01 (0.02)	0.68
GPA	0.15 (0.03)	1.7e–05***
MLec (yes)	0.02 (0.03)	0.53
Familiarity category: not familiar (r2 = 0.10)
Intercept	0.09 (0.15)	0.54
Ethnicity (Caucasian)	0.07 (0.05)	0.18
Ethnicity (URM)	−0.03 (0.05)	0.55
Gender (M)	0.04 (0.03)	0.27
GPA	0.11 (0.05)	0.01*
MLec (yes)	−0.02 (0.03)	0.50

aSummary data from three independent multiple regression models of MLab exam question performance on very familiar (VF), familiar (F), and not familiar (NF) questions analyzed in the context of student demographics, including GPA (on a 4.0 scale), ethnicity (Caucasian, Asian, or URM [African American or Hispanic]), gender (male or female), and MLec completion (yes or no). The baseline variables for the models are Asian, female, and no MLec. The estimate (presented as the unstandardized coefficient) highlights the increase or decrease in scores (out of 100% presented in decimal form) for students based on the indicated factors. Data were combined for students in Fall 2014 MLab and Winter 2015 MLab sections, although conclusions were similar for each individual course. The estimate, SEM, and p values are indicated for each comparison. For this set of models, familiarity was designated by the Spring MLec instructor. The remaining 15 models (with familiarity designated by Spring, SS1, or SS2 lecture slides or instructors) are presented in the Supplemental Material (Supplemental Table S2, A–E).

*p < 0.05.

***p < 0.001.

To then control for differences in student populations, we ran multiple linear regression models. With the response variable being average exam performance on questions in each familiarity category, we controlled for student GPA (continuous variable on a 4.0 scale), ethnicity (categorical variable: Caucasian, Asian, underrepresented minority [URM, consisting of African-American and Hispanic students]), gender (categorical variable: male, female), and whether or not the student enrolled in MLec (categorical variable: yes, no). All factors were included in the model and retained, even if they did not significantly affect the response variable, to highlight that they were not significant factors in student performance and because of historical data highlighting the significance of such variables. This includes both gender and ethnicity, which were not significant in our models but have been shown to be significant indicators of performance both at the study institution and other institutions of higher education (Aguilar-Roca ; Creech and Sweeder, 2012; Sato ). Separate models were run for each group of MLec “yes” students (Spring 2014 MLec vs. those who did not take MLec, SS1 vs. no MLec, SS2 vs. no MLec) and for each method of familiarity designation (lecture slide or instructor). Descriptive information regarding the student populations analyzed is presented in Table 1.

A second method used to analyze exam performance (Figure 3 and Table 5) was a comparison of average performance on VF versus F versus NF exam questions. This included data only from students who had completed MLec, as previously described (Shaffer ). Questions were categorized by either MLec lecture slide or MLec instructor, as described earlier. The average performance on questions of each familiarity level was then calculated. The significance of the differences in average exam scores for questions in each familiarity category was determined using multiple pairwise Student’s t tests. Multiple linear regression models were then run to control for the familiarity and Bloom’s level of each exam question. With the response variable being average score for exam questions, we controlled for question familiarity (categorical variable: VF, F, NF) and Bloom’s level (categorical: Bloom’s 1–6). Separate models were run for each method of familiarity designation (lecture slide or instructor).

FIGURE 3.

Comparison of performance on MLab questions of differing familiarity. Mean scores and SEM of VF, F, and NF MLab questions are presented, and familiarity was assigned as indicated. Only exam data from students who completed MLec were included in this analysis. The average Bloom’s level of questions in each category is noted. Pair-wise comparisons of exam questions of different familiarity levels (VF vs. F, F vs. NF, VF vs. NF) were not significant by t test except for one comparison indicated on the graph. Differences in the heights of each bar across MLec sections or familiarity designation methods is due to the fact that questions are segregated distinctly in each of these scenarios. *p < 0.05.

TABLE 5.

Multiple regression analysis examining factors influencing MLab exam performancea

	Estimate (±SEM)	p Value
A. Familiarity designation: instructor (r2 = 0.15)
Intercept	0.83 (0.13)	1.1e–09***
Baseline: not familiar
Very Familiar	0.07 (0.04)	0.07
Familiar	0.04 (0.04)	0.30
Baseline: Bloom’s 1
Bloom’s 2	−0.21 (0.13)	0.12
Bloom’s 3	−0.24 (0.13)	0.07
Bloom’s 4	−0.32 (0.14)	0.02*
Bloom’s 5	−0.47 (0.14)	6.2e–4***
B. Familiarity designation: lecture slides (r2 = 0.15)
Intercept	0.87 (0.13)	6.9e–11***
Baseline: not familiar
Very Familiar	0.07 (0.04)	0.13
Familiar	0.07 (0.04)	0.07
Baseline: Bloom’s 1
Bloom’s 2	−0.23 (0.13)	0.09
Bloom’s 3	−0.27 (0.13)	0.04*
Bloom’s 4	−0.35 (0.14)	0.01*
Bloom’s 5	−0.52 (0.14)	1.8e–4***

a Multiple regression model data looking at performance on exam questions when controlling for question familiarity and Bloom’s level. The means by which familiarity was designated is indicated on each table. The estimate (presented as the unstandardized coefficient) highlights the increase or decrease in scores (out of 100% presented in decimal form) for VF or F (as indicated) questions vs. NF. The estimate, SEM, and p values are indicated for each comparison of VF or F vs. NF questions. For each of the models, the baseline values are Bloom’s level 1 and NF familiarity. Data from Fall MLab and Winter MLab courses were combined for analysis purposes, although conclusions were similar for each individual course.

*p < 0.05.

***p < 0.001.

For both of these exam analyses, Fall 2014 and Winter 2015 data were combined. All conclusions were similar regardless of whether data were presented per quarter or combined.

RESULTS

Student Perceptions Regarding the Value of Prerequisite Courses

To identify student perceptions of prerequisites in general, as well as specifically within the context of MLec and MLab, we solicited opinions from students who had completed MLab through both an online survey and follow-up semistructured interviews. The interviews highlighted a number of common themes regarding perceived positive and negative attributes of prerequisites (Table 2, A and B). Not surprisingly, students cited prerequisites as a source to obtain background knowledge and as being beneficial to future success, but also as a place to identify areas of interest for future study and logistically helpful to ensure that senior students are able to enroll in more heavily impacted courses (Table 2A). Scheduling issues were the primary negative attribute of prerequisites, and many cited the heavy impact on time to graduation and the corresponding costs to students, especially in light of the frequently observed disconnect between instructors teaching a prerequisite and a linked course (Table 2B).

We were particularly interested in students’ perceived value of MLec for success in MLab, and began by asking students who either did or did not take MLec before MLab their reasons for doing so. For those who took MLec, students cited a need for background knowledge, the course being a requirement for graduate school, or interest in the subject as reasons for enrollment. Other surprising results were that it was merely a class that fit their schedule or they were under the impression that MLec was a required prerequisite (Table 3A). The most common responses for why students did not enroll in MLec before MLab include the fact that it was not required, highlighting that students are guided strongly by the program rules laid out for them, or scheduling issues prevented them from doing so (Table 3B). The survey responses contained similar data (Supplemental Table S1).

Students were asked in the interviews whether they believed an individual who completed MLec would have an advantage in MLab, and whether that belief changed throughout their time enrolled in MLab. This produced a striking dichotomy, wherein those who did take MLec almost unanimously believed initially that it would produce an advantage, whereas those who did not were much less convinced (Figure 1A). Nearly all, though, felt by the end of the quarter that MLec was not beneficial for MLab performance. As explained by one student in response to this question of whether MLec completion would provide an advantage in MLab,

Maybe a little bit on the first day because [the instructor] started going over stuff and I was like “oh my goodness”. But then I started realizing I can go over the notes and the text and it covered everything really well.

Another stated,

There wasn’t any point where I felt like there was a lack of [background]material… So it would be like, oh I didn’t study what was there enough, but it wasn’t like it wasn’t there. So I never felt like taking a lecture in addition to that would’ve helped me.

Students who had recently completed MLab were asked in the survey for their level of agreement with the statement, “I believe that someone completing MLec before enrolling in MLab would earn a higher grade in the lab.” This produced a majority of neutral or negative (disagree, strongly disagree) responses (Figure 1B). Students who did and did not enroll in MLec did not respond in a significantly different manner (Kruskal-Wallis rank-sum test, p = 0.28). In agreement with these findings, 90% of students interviewed believed that MLec should continue to be optional for MLab enrollment.

Impact of MLec on MLab Performance

We next wanted to determine whether student perceptions regarding the lack of impact that MLec has on MLab performance agrees with an MLab exam analysis. Using our previously established familiarity scale (Shaffer ), we characterized MLab exam questions as VF, F, or NF. If MLec is beneficial for MLab performance, we would expect to see that students who enrolled in MLec outperformed their peers who did not on familiar questions (either VF or F). In the context of three separate MLec prerequisite sections, all of the conditions tested showed no statistical difference in performance on VF or F questions for students who had completed MLec versus those who had not (Figure 2, A and B). Similarly, there was no difference in performance on NF questions (Supplemental Figure S1). To ensure that differences in student demographics did not impact our results, we performed multiple linear regression models controlling for undergraduate GPA, gender, ethnicity, and whether a student did or did not take MLec. While GPA was a significant predictor of student performance, the other variables (including MLec completion) did not impact performance on MLab VF, F, or NF exam questions (Table 4 and Supplemental Table S2). Additionally, students who did or did not enroll in MLec were not significantly different in terms of GPA (Student’s t test p = 0.14), ethnicity (Pearson’s chi-squared p = 0.43), or gender (Pearson’s chi-squared p = 0.67).

Another means to assess the impact of MLec is to compare performance on specific MLab questions based on familiarity. If a student enrolled in MLec, we speculate that he or she should perform best on VF MLab questions. An increase in performance on VF questions was seen in certain cases for genetics/molecular biology and human physiology/human anatomy course pairings we previously examined, although there was no impact on F versus NF performance (Shaffer ). In this case, we saw no difference in performance on VF, F, or NF MLab exam questions for students who had previously taken MLec, with the exception of a significant difference on VF and NF question performance with familiarity designated by MLec instructor (Figure 3). To partially control for question differences, we also ran multiple regression models controlling not only for familiarity but also the question’s Bloom’s level (Table 5, A and B). While Bloom’s level was a significant predictor of performance on exam questions, familiarity had no effect, highlighting that the one significant result seen with the raw data was due to a slight imbalance in average Bloom’s levels between the questions in each familiarity category.

DISCUSSION

With the increased emphasis in recent years on improving undergraduate education, curriculum reform is a common topic of interest for STEM programs. One integral component of a curriculum is prerequisites, which, while in theory act to prepare students for future courses, are not commonly assessed to determine whether they fulfill this goal. A means to measure their impact is by examining student performance in a linked course, and we recently developed a novel familiarity scale through which to view potential effects. This familiarity scale was developed not as an endpoint for curricular reform but to create a baseline upon which future modifications can be measured. By applying the scientific method, these data can inform future policy decisions, which can then be reanalyzed in the same manner to determine their efficacy.

Assessment of prerequisite effectiveness can take a variety of forms, and in addition to performance data in a later course, can include comprehensive qualitative feedback from students and instructors. We were curious to hear the student perspective concerning prerequisites, and through semistructured interviews and online surveys, we found that, while students highlighted a number of perceived benefits, they also emphasized a number of logistical problems that arise, including scheduling issues created by demand for prerequisites and increased time to graduation based on the requirement to enroll in additional courses. In light of these potentially harmful consequences of prerequisites, coupled with the rising costs of higher education, it is essential that we can demonstrate to students that the positive attributes exist.

This study focused on a particular curriculum change made in the recent past without the assistance of scientific data. While lecture and laboratory courses are often coupled, it appears that, in this case, the microbiology lecture does not impact lab success. This conclusion was drawn by an examination of performance on questions of varying levels of familiarity (Figure 3 and Table 5) and a comparison of exam performance on familiar questions between students who had and had not completed MLec (Figure 2, Table 4, and Supplemental Table S2). While other studies have also examined prerequisite impact by comparing students who have or have not taken a prerequisite (Forester ; Choudhury ; Wright ), their focus on overall course grade may have missed a more conservative impact that the prerequisite had on future success. Our familiarity analysis applies a focused, question-by-question approach, which would be more likely to identify a positive effect of prerequisite completion. Despite this, our results identified no performance benefit of prerequisite completion for the microbiology lecture/lab series.

These exam-related results were not surprising based on the student interview and survey data we collected, and we believe our work emphasizes the need to solicit and use student input, ranging from course evaluations or graduation surveys to in-depth interviews, when making curriculum-related decisions. Student feedback at the individual course and instructor level is a standard part of the higher education experience, and while considerable work has examined the validity and reliability of such data (Cashin ; Falchikov and Goldfinch, 2000; Marsh, 2007), it is still up for debate as to whether this is the most accurate manner in which to evaluate instructor performance (Cashin ; Trout, 1997; Wright, 2000; Murray, 2007) and there is great uncertainty as to how frequently such data are actually used to improve the learning experience for future students (Cohen, 1980; Marlin, 1987; Kember ). Student feedback collected at the curriculum level is much rarer, and even less evidence is present as to whether this information is used in a meaningful way (Richardson, 2005; Richardson ). In our study, student feedback complemented the exam familiarity analysis and introduced data concerning prerequisites and course selection that may not be obvious from the faculty or administrative point of view. As students are obviously invested in the success of higher education programs, it is important that they are partners in the decision-making processes that lead to curricular modifications. We hope that this work and the methodologies used can act as part of the plan institutions implement when performing curricular assessment.