Team-Based Learning in a Pipeline Course in Medical Microbiology for Under-Represented Student Populations in Medicine Improves Learning of Microbiology Concepts.

As part of an undergraduate pipeline program at our institution for students from underrepresented minorities in medicine backgrounds, we created an intensive four-week medical microbiology course. Team-based learning (TBL) was implemented in this course to enhance student learning of course content. Three different student cohorts participated in the study, and there were no significant differences in their prior academic achievement based on their undergraduate grade point average (GPA) and pre-course examination scores. Teaching techniques included engaged lectures using an audience response system, TBL, and guided self-directed learning. We hypothesized that more active learning exercises, irrespective of the amount of lecture time, would help students master course content. In year 2 as compared with year 1, TBL exercises were decreased from six to three with a concomitant increase in lecture time, while in year 3, TBL exercises were increased from three to six while maintaining the same amount of lecture time as in year 2. As we hypothesized, there was significant (p < 0.01) improvement in performance on the post-course examination in years 1 and 3 compared with year 2, when only three TBL exercises were used. In contrast to the students' perceptions that more lecture time enhances learning of course content, our findings suggest that active learning strategies, such as TBL, are more effective than engaged lectures in improving student understanding of course content, as measured by post-course examination performance. Introduction of TBL in pipeline program courses may help achieve better student learning outcomes.

INTRODUCTION

Members of minority communities in the United States traditionally have poorer outcomes from preventable and treatable diseases than those who are not part of a minority community (5). Physicians of diverse racial and ethnic background are more likely to practice in medically underserved communities, which helps improve healthcare delivery in these areas (7, 21). However, fewer than 10% of physicians in the US are Non-Hispanic Black, Non-Latino other, or Latino (2). This is in sharp contrast to the ethnic diversity seen in most urban communities in the US. To address this disparity, pipeline programs are essential to provide students from racial and ethnic backgrounds traditionally underrepresented in medicine with early exposure to biomedical professions, which may ultimately increase racial and ethnic diversity amongst healthcare providers in underserved urban communities (6, 27, 30). Thus, these pipeline programs provide the opportunity to play an important role in the improvement of healthcare delivery to urban populations. Additionally, these programs could help these students to become interested in other biomedical science careers. Seeing the value of these programs, many institutions of higher education have already implemented pipeline programs in order to increase the numbers of underrepresented in medicine (URM) students pursuing careers in the health sciences (8, 11, 28, 29).

Pipeline programs provide important opportunities for URM students with regard to exposure to health science curricular content as well as teaching and learning strategies essential for success in medical school. The achievement gap in science, technology, engineering, and mathematics (STEM) with regard to URM students is well documented (1, 28). Pipeline programs aimed at closing this gap typically provide students with exposure to educational content necessary for future success in higher education (28). More importantly, we feel that these programs need to provide URM students experience with learning and teaching strategies that will help them cope with the challenges of higher education programs in the health sciences.

The Premedical Urban Leaders Summer Enrichment (PULSE) program was created at our institution with the intent of strengthening academic performance and providing health career experiences to enhance the portfolio of students from disadvantaged and/or URM backgrounds or those who matched the mission of our institution, which emphasizes service to underserved communities. As part of the PULSE program, we developed an intensive four-week course in medical microbiology. Although this course includes graded examinations, no undergraduate credit was assigned upon completion. Therefore, student participation and achievement were primarily driven by student interest in the course material as well as their intrinsic motivation to learn. Course content was presented through instructor-led, engaged lectures complemented by self-directed learning time and team-based learning (TBL) exercises. These teaching and learning methods are ones that are also utilized at our medical school and therefore provide exposure to strategies that participating students are likely to see in their future studies.

In a meta-analysis of 225 studies reporting exam scores and failure rates, Freeman et al. showed that active learning results in better performance on examinations compared with traditional lecturing (12). TBL is a form of active learning that has been shown to improve student engagement and learning of course content (23). During TBL exercises, students are divided into teams that engage in peer-to-peer teaching prompted by multiple-choice, mastery-level questions and complex, higher-level application questions. These exercises are based on materials assigned for review prior to the session. This teaching method allows students to reinforce and apply reviewed concepts through engaging discussions and by solving higher-order experimental or clinical problems (17). TBL also promotes the development of essential problem-solving and critical reasoning skills necessary for better understanding of course content by requiring students to apply learned concepts to difficult problems (31). TBL has been shown to disproportionately help those students who are struggling most with course material (20). Additionally, our group has shown that the introduction of TBL exercises to a first-year medical school course improved student final-examination performance in addition to reducing the number of course failures, also suggesting that TBL mainly helps struggling students (4). Thus, we hypothesized that increased use of active learning exercises, such as TBL, would improve understanding of course content as assessed by performance on an end-of-course examination in the Medical Microbiology course of the PULSE program.

MATERIALS AND METHODS

Student population

PULSE is a competitive program open to undergraduate students from US colleges and universities. URM and educationally/financially disadvantaged students are the target population. Committee selection is based on a holistic process that takes into account applicant grades, school activities, community service, personal statement, and letters of recommendation. Phase I applicants must have completed one year of college (by the start of program) and the following coursework: one semester each of biology, chemistry, math, and expository writing or college composition. A minimum grade point average (GPA) of 3.0 is required. The numbers of students in each cohort were as follows: year 1, 24 students; year 2, 21 students; year 3, 22 students.

Course description

The Medical Microbiology course was divided into seven blocks. Educational methods used in this course included the following: 1) Engaged lectures, 2) Self-directed learning under the supervision of a first-year medical student, and 3) TBL exercises. A detailed description of learning topics and learning objectives as well as the course syllabus are shown in Appendix 1. Achievement of the listed learning objectives was demonstrated through readiness assurance tests and an end-of-course examination.

1) Content delivery

The same professor delivered the course content and TBLs in all three years of the study. Course content was provided via engaged lectures, which employed an audience response system (ARS) using the peer-teaching method developed by E. Mazur (24).

In year 1, students felt that more lecture time was necessary. Therefore, for some difficult topics, we doubled the lecture time (Table 1). This extra time was not used to deliver additional content but rather to include more interactive questions through use of the ARS. To compensate for this additional lecture time, we decreased the number of TBL exercises from six in year 1, to three in year 2 (Table 1), which allowed us to investigate the effect of changes in the number of TBL exercises on student learning. Due to a decrease in grades on the TBLs as well as on the post-course examination, we adjusted the course structure in year 3 to incorporate both extra lecture time and more TBL exercises by adding one more class day each week. This change in educational time allowed us to increase the number of TBL exercises from three in year 2 to six in year 3.

TABLE 1

Medical Microbiology course structure in years 1, 2, and 3.

Lecture Topic	Year 1		Year 2		Year 3
	Lecture Time (min)	TBL Number	Lecture Time (min)	TBL Number	Lecture Time (min)	TBL Number
1) Structure Prokaryotes	80	1	80	1	80	1
2) Bacterial Metabolism	80	1	160	1	160	1
3) Microbial Growth and Control of Microbial Growth	80	2	80	1	80	2
4) Microbial Genetics	80	3	160	2	160	3
5) Animal Viruses	80	4	160	2	160	4
6) Host-Microbe Interactions	80	5	80	3	80	5
7) Innate and Adaptive Immunity	80	6	160	3	160	6
Total Lecture Time	630 minutes		990 minutes		990 minutes
Total TBL Exercises	6		3		6

TBL = team-based learning.

2) Self-directed learning

Students were provided with dedicated, self-directed learning time two afternoons per week for three hours each session. These sessions were designed to introduce students to the concept of independent learning and peer-teaching and allowed for individual and/or group study under the guidance of a medical student teaching assistant, who was experienced in both self-directed and active-learning strategies. Students typically reviewed information independently and reinforced any difficult topics through discussions with the medical student or within small student groups. In the second part of the activity, students were provided with multiple-choice questions related to the lecture topics. These questions were completed individually and later discussed in small groups under the supervision of the medical student teaching assistant. These discussions typically focused on the more difficult questions. The teaching assistant had real-time direct access to the course instructor for unresolvable questions/discussions. These sessions were designed to help students prepare for the TBL exercises. Each content topic was discussed after the corresponding lecture either on the same day or one day later. All material was discussed prior to the corresponding TBL exercise.

3) Team-based learning activity

All content topics in the course were covered as part of the TBL exercises (Table 1). The following description of the TBL exercises adheres to the guidelines for reporting TBL in the medical and health sciences education literature (14). The TBL method was composed of three phases: independent learning, readiness assurance, and application. In the first phase, students independently reviewed lecture materials with the optional guidance of a medical student teaching assistant in their self-directed learning time as described above.

In the second phase, students were tested for mastery of course learning objectives. Students were divided into predetermined teams of four to six students based on undergraduate GPA, class standing (freshman, sophomore, junior, senior), sex, race, and undergraduate institution, in order to increase diversity in the group resources, which has been shown to improve team function (25). Student teams were created by the course faculty without input from the students or teaching assistants. This portion of the exercise began with a closed-book, graded, individual readiness assessment test (iRAT). After completion of the iRAT, students reviewed the iRAT questions with their respective teams. During this portion of the exercise, called the group readiness assessment test (gRAT), the teams each discussed the quiz questions to reach a consensus answer. Misconceptions and confusion about the material were addressed through peer-to-peer teaching. Once the team arrived at a consensus answer, the correct response was revealed using an immediate feedback-assessment technique (IF-AT) scratch card. This card allowed the students to immediately see whether they had answered the question correctly. If the response was incorrect, the team continued to discuss the question and chose an answer from the remaining choices. At the completion of the gRAT, the instructor led an optional discussion of difficult questions that were not adequately addressed by the peer-to-peer teaching process. Grading for the TBL exercise was based on an assignment of 33.3% to the iRAT and 66.7% to the gRAT.

In the third phase, the application exercise, students were encouraged to apply concepts mastered during the iRAT/gRAT to experimental problems or clinical cases. The students worked within their teams to discuss and investigate these problems and were able to use course notes to assist in their investigations. The teams then presented their findings to the class, and this was followed by a class discussion with guidance and feedback from the instructor. This exercise was not graded. However, the team that answered the most questions correctly was publicly awarded a coffee shop gift card. If two or more teams got the same number of questions correct, the gift card was given to one of the teams at random. An example of a TBL exercise corresponding to Innate and Adaptive Immunity (TBL 6 in years 1 and 3 and TBL 3 in year 2) can be found in Appendix 2.

Data collection and analysis

Use of course data for analysis as part of this study was approved by the Institutional Review Board (IRB) at Rowan University under number Pro2015000520.

The distributions of GPAs of the students accepted to the Medical Microbiology course were analyzed using analysis of variance by SIGMA PLOT 12.5, and p < 0.01 was considered statistically significant.

In each year, on the first day of class, students took a pre-course examination consisting of 37 multiple-choice questions with five to six questions per lecture topic. On the last day of class, students took a post-course examination consisting of the same 37 questions. Students were unaware that the questions on the pre-course and post-course examinations were the same. The percentage of students answering each question correctly for both the pre-course and post-course examinations was calculated. For each year of the study, the change in percentage of students correctly answering each question between the pre-course and post-course examinations was calculated. The distributions of these calculated values amongst the three years were compared using a Kruskal-Wallis one way analysis of variance of ranks followed by a pairwise multiple comparison (Student-Newman-Keuls) using SIGMA PLOT 12.5, with p < 0.01 considered statistically significant.

To further analyze the effect of changes in the Medical Microbiology course on higher learning and critical thinking, pre-course/post-course examination questions were subdivided according to their Bloom’s taxonomy of learning domains. Two of the authors (O.L. and K.B.) reviewed the questions on the pre-course/post-course examinations and independently determined the Bloom’s taxonomy for each question. These two authors met and resolved differences in their Bloom’s taxonomy designation, coming to a consensus taxonomy for each question. Questions were then divided into two groups, Bloom’s taxonomy level 1 (remember) and Bloom’s taxonomy levels 2, 3, and 4 (understand, apply, and analyze). The questions were divided in this fashion (remember: n = 19; understand, apply, and analyze: n = 18), because this allowed for separation of Bloom’s taxonomy level 1 questions, which only require recall, from Bloom’s taxonomy level 2, 3, and 4 questions, which require manipulation of recalled information, a skill fostered by the TBL exercises. The percentage of students answering each question correctly for both the pre-course and post-course examinations was calculated, and for each year of the study, the change in percentage of students correctly answering each question between the pre-course and post-course examinations was calculated. The distributions of these calculated values amongst the three years were compared using a Kruskal-Wallis one way analysis of variance of ranks, and, in the case of data related to Bloom’s taxonomy level 2, 3, and 4 questions, this was followed by a pairwise multiple comparison (Student-Newman-Keuls). SIGMA PLOT 12.5 was used to perform the statistical analysis, and p < 0.01 was considered statistically significant.

Student course evaluations

Cooper Medical School of Rowan University (CMSRU) personnel distributed and collected the student evaluations immediately after the students took the final examination. They also administered the course examinations, collected the course evaluations, and stored the course documents, and they were not directly involved in any other aspect of the course. Students were asked to rate survey statements using a Likert scale where 1 = strongly disagree, 2 = somewhat disagree, 3 = neutral, 4 = somewhat agree, 5 = strongly agree. Student course evaluation data were analyzed using Microsoft EXCEL for Mac 14.3.9 to calculate means and standard deviations.

RESULTS

Student demographic data

Over the three years of our study, 67 students completed the Medical Microbiology course. Demographic data for students included in this study are shown in Table 2. Student ethnicity, gender, and undergraduate major were also examined. There was no significant difference (p = 0.317) among the three student populations in undergraduate grade point average (GPA) (Fig. 1). Extensive student diversity was noted throughout the study with non-Caucasian, URM, and female students representing the majority of students attending the course.

TABLE 2

Demographic data for students participating in the medical microbiology course.

	Year 1 (% of students)	Year 2 (% of students)	Year 3 (% of students)
Academic Level
Freshman	16%	29%	43%
Sophomore	24%	28%	38%
Junior	36%	38%	14%
Senior	20%	0%	0%
Post-graduate	4%	5%	5%
Ethnicity
Caucasian	16%	22%	8%
Non-Caucasian	84%	78%	92%
Under-represented minorities in medicine	72%	68%	52%
Sex
Male	24%	38%	19%
Female	76%	62%	81%
Undergraduate Major
Majors	Biology/Biomedical Sciences: 84%	Biology/Biomedical Sciences: 96%	Biology/Biomedical Sciences: 95%Spanish: 5%
Undeclared	16%	4%	0%

FIGURE 1

Distribution of undergraduate GPA amongst students participating in the Medical Microbiology course during years 1, 2, and 3 of the study. A comparison of the distribution of undergraduate GPA amongst the three years of the study was performed using analysis of variance (ANOVA), showing no statistical difference amongst the three groups (p = 0.317). The upper and lower limits of each box represent the 75th and 25th percentiles of the data, respectively, while the upper and lower whiskers represent the 90th and 10th percentiles, respectively. Data points outside of the whiskers, the outliers, are represented by solid circles. The horizontal line within each box represents the median. GPA = grade point average.

Examination of student demographic data also revealed that in year 1, there was a relatively even distribution of students at the different academic ranks, ranging from freshmen to seniors. However, in years 2 and 3, course enrollment was dominated by underclassmen (freshmen and sophomores).

Student performance on TBL exercises

Figure 2 shows the average scores for the iRAT and gRAT exercises in the three years of the study. The average iRAT scores decreased in year 2 compared with years 1 and 3. Interestingly, the iRAT scores for the first TBL were low in all three years of the study. However, this is not surprising as it likely reflects unfamiliarity with the teaching strategy. Nevertheless, in years 1 and 3, average iRAT scores improved with subsequent TBL exercises while they remained low in year 2.

FIGURE 2

Average and standard deviation of iRAT (panel A) and gRAT (panel B) scores for each TBL in Years 1, 2, and 3. There were three TBL exercises in year 2 (checkered, TBL 1, 2, and 3) and six TBL exercises in years 1 and 3 (black and grey bars, respectively). The iRAT and gRAT exercises have a maximal possible score of 10. Error bars reflect the standard deviation of scores for the iRAT (panel A) and gRAT (panel B) exercises in each TBL session. iRAT = individual readiness assessment test; gRAT = group readiness assessment test; TBL = team-based learning.

Student performance on the pre-course and post-course examinations

As part of this study, student scores from the pre-course examinations were analyzed using ANOVA. No significant differences were found in the pre-course examination scores of students in years 1, 2, and 3, indicating that students, on entering the course, had similar funds of knowledge. This result is consistent with the lack of significant differences found in the GPAs of students in the three groups (Fig. 1).

We calculated the change in the percentage of students answering each question correctly between the pre-course and post-course examinations for each year of the study, as these values are a measure of the true learning that occurs during the course. This analysis revealed a significant increase in the distributions of changes in the percentage of students answering each question correctly between years 1 and 2 and years 2 and 3. However, there was no significant difference when comparing years 1 and 3 (Fig. 3). These data suggest that students perform better when six TBL exercises are used than when three are used.

FIGURE 3

Change in percentage of students answering questions correctly between the pre-course and post-course examinations. The distribution of changes in the percentage of students answering each question correctly between the pre-course and post-course examinations was examined using a one way analysis of variance followed by a pairwise multiple comparison (Student-Newman-Keuls). This analysis showed that there was a significant difference in the distributions between years 1 and 2 (p < 0.0007) and years 2 and 3 (p < 0.0039) but not between years 1 and 3. There was no significant difference in the distribution of p values when comparing years 1 and 3 (p < 0.1820). The upper and lower limits of each box represent the 75th and 25th percentiles of the data, respectively, while the upper and lower whiskers represent the 90th and 10th percentiles, respectively. Data points outside of the whiskers, the outliers, are represented by solid circles. The horizontal line within each box represents the median.

On further analysis of the change in percentage of students answering the questions correctly between the pre-course and post-course examinations with respect to the Bloom’s taxonomy level of each question, we found that there was no change in performance on the Bloom’s level 1 question in response to changes in course structure (Fig. 4A). However, there was considerable improvement in performance (p < 0.001) on the Bloom’s taxonomy level 2, 3, and 4 questions in years 1 and 3 of the study, when there were six TBL exercises, compared with year 2, when there were only three (Fig. 4B). These results suggest that increased use of TBL exercises correlates with improved higher learning and critical thinking skills.

FIGURE 4

Change in percentage of students answering questions correctly between the pre-course and post-course examinations according to Bloom’s taxonomy classification. The distribution of changes in the percentage of students answering each Bloom’s taxonomy level 1 (remember) (A) or Bloom’s taxonomy levels 2, 3, and 4 (understand, apply, and analyze) (B) questions correctly between the pre-course and post-course examinations was analyzed using a one way analysis of variance. In the case of the Bloom’s taxonomy level 2, 3, and 4 questions, this was followed by a pairwise multiple comparison (Student-Newman-Keuls). For the Bloom’s taxonomy level 1 questions, there was no significant difference in the distributions between years 1, 2, and 3 (p = 0.213). For the Bloom’s taxonomy level 2, 3, and 4 questions, there was a significant difference in the distributions between years 1 and 2 (p = 0.001) and years 2 and 3 (p < 0.001) but not between years 1 and 3 (p = 0.587). The upper and lower limits of each box represent the 75th and 25th percentiles of the data, respectively, while the upper and lower whiskers represent the 90th and 10th percentiles, respectively. Data points outside of the whiskers, the outliers, are represented by solid circles. The horizontal line within each box represents the median. statistical difference amongst the three groups (p = 0.317). The upper and lower limits of each box represent the 75th and 25th percentiles of the data, respectively, while the upper and lower whiskers represent the 90th and 10th percentiles, respectively. Data points outside of the whiskers, the outliers, are represented by solid circles. The horizontal line within each box represents the median. GPA = grade point average.

Student opinion regarding usefulness of TBL exercises to learning microbiology

Table 3 shows data detailing student opinion of the Medical Microbiology course over the three years of the study. An anonymous survey was administered at the end of the course, after the post-course examination, and survey questions were answered using a Likert scale. Survey responses from the students were averaged for each question and are displayed in Table 3. Over the three years of the study, the students were generally highly satisfied with the course, and their opinions, for the most part, remained invariant. However, there was some variation in their responses to the question, “The study sessions provided opportunities to discuss the content with my fellow students in a helpful way to practice and test my knowledge and problem solving skills.” This may have been related to differences in interactions with the first year medical students, who were different in each of the three years of the study.

TABLE 3

Student opinion of the Medical Microbiology course.

	Year 1	Year 2	Year 3
The course was well organized	4.72	4.91	4.43
The course was efficiently presented	4.40	4.86	4.67
The pace of the course was appropriate	4.16	4.05	3.91
The information taught was appropriate	4.84	4.42	4.55
I was provided opportunities for practicing and testing my knowledge and problem-solving skills throughout the course	4.56	4.81	4.57
The ungraded applications in the TBL exercises were helpful in practicing and testing my knowledge and problem solving skills	4.92	4.71	4.57
The use of clickers for questions in lecture is a helpful way to practice and test my knowledge and problem solving skills	4.24	4.24	4.62
The study sessions provided opportunities to discuss the content with my fellow students in a helpful way to practice and test my knowledge and problem solving skills	3.60	4.38	4.00
The iRAT/gRAT evaluated my knowledge and helped me identify areas where I needed to focus my learning	4.40	4.43	4.48
The final exam evaluated my knowledge and problem-solving skills	4.24	4.37	4.48
The instructor is an effective lecturer	4.12	4.57	4.29
PowerPoint slides were helpful	4.24	4.48	4.42
Instructor’s notes were helpful	4.60	4.81	4.40
This course will help me in my career in biomedical sciences	4.62	4.71	4.48

TBL = team-based learning; iRAT = individual readiness assessment test; gRAT = group readiness assessment test.

Additionally, as part of the anonymous course evaluations, students were asked to respond to the following question: “What is the single best aspect of this course that needs to be continued?” There was a variety of student responses. However, 91% of students in year 1, 67% of students in year 2, and 75% of students in year 3 included the term TBL in their responses. Most of the other responses referred to the use of clickers in lecture and the course instructor. These data suggest that students were satisfied with the course in general, and that they enjoyed the TBL activities.

DISCUSSION

Our results strongly suggest that the number of TBL exercises in a course is positively correlated with student learning outcomes. We were able to show that student performance on the post-course examination improved with more TBL exercises, as demonstrated by a statistically significant positive change in the percentage of students answering each question correctly on the post-course examination compared with the pre-course examination (Fig. 3). Further analysis showed that student improvement on the post-course examination was more prominent in higher Bloom’s taxonomy level questions in years 1 and 3 when more TBL exercises were used, suggesting that the TBL exercises resulted in improved higher-level learning and critical thinking skills (Fig. 4). Improvement in student performance in years 1 and 3 compared with year 2 was not likely due to changes in student class standing (freshman, sophomore, junior, senior), as the year with the greatest percentage of underclassmen, year 3 (Table 2), showed one of the greatest improvements in student performance (Fig. 3).

Notably, over the three years of the study, there were substantial structural changes in the course in response to student feedback and student performance. These changes included a reduction in TBL exercises and concomitant increase in lecture time in year 2, followed by an increase in TBL sessions in year 3 while maintaining the same amount of lecture time as year 2. Over this time period, the course content and instructor, who is the corresponding author for this work, did not change. Even though there was an increase in lecture time in year 2 compared with year 1, it seems that the concomitant decrease in TBL exercises in year 2 outweighed any benefit the students may have derived from increased lecture time, resulting in an overall decrease in post-course examination improvement.

Students from year 1 wrote in their evaluations that they felt that increased lecture time would improve delivery of educational content in the course (data not shown). However, when lecture time was increased in year 2 at the cost of removing TBL exercises, this change actually resulted in poorer student outcomes (Figs. 2 and 3). Perhaps the students felt that increased lecture time would improve learning because of their natural comfort with this teacher-centered approach. Conversely, they may not have asked for more TBL exercises because of their unfamiliarity with the value of this active learning strategy, although they spoke highly of TBL in their course evaluation (Table 3). Indeed, Fatmi et al., in an extensive review of research on the use of TBL in health professions, showed that even though TBL almost always improves student performance, students often show mixed satisfaction with the method (10).

In year 3, when the number of TBL exercises was increased to six while keeping the same amount of lecture time as in year 2, the students once again had levels of improvement in performance on the post-course examination similar to that seen in year 1. It is interesting to note that improvements in performance on the post-course examination in years 1 and 3 were similar, suggesting that the addition of teacher-centered activities, such as engaged lectures, does not necessarily improve student learning, and that active learning strategies such as TBL may be more successful in this respect. Notably, Deslauriers et al. also found that students performed better when active learning strategies were used as opposed to teacher-centered approaches, regardless of the experience of the instructor (9).

Since the course’s inception in year 1 of the study, lectures have been delivered in an engaged format with intralecture questions using a pairwise instruction method developed by Eric Mazur at Harvard University (24). The goals of these intralecture questions were to help the instructor assess student understanding of the presented material, to acquaint students with pairwise share techniques, and to help make the lectures more enjoyable. Although this method proved to be an enjoyable and effective way to deliver lecture content, it seems that it was not sufficient to overcome decreased use of TBL exercises, as evidenced by the decrease in improvement on the post-course examination in year 2 compared with year 3.

Importantly, the Medical Microbiology course also included a self-directed learning component, which was designed to help the students prepare for the TBL exercises. The World Federation for Medical Education (WFTE) has endorsed self-directed learning as an important component of medical undergraduate education (19). The ability of future physicians to successfully engage in self-directed learning activities is essential to the development of life-long learning skills, which are necessary to keep pace with the ever changing landscape of medical practice (18). Providing our pipeline students with self-directed learning experiences, under the guidance of an experienced undergraduate medical student, gives them a head start on learning practices necessary for success in medical school and beyond.

Over the course of the study, as noted above, we saw a change in student class standing in the course, with the majority of students in years 1 and 2 being juniors (36% and 38%, respectively) and a shift toward underclassmen in year 3 (43% freshmen and 38% sophomores) (Table 1). This change likely resulted from increasing numbers of program applicants due to increased regional and national recognition of the PULSE program, which allowed the program coordinators to select for more underclassmen. One may speculate that having students with higher academic standing would improve learning outcomes due to increased experience with undergraduate coursework and a greater fund of knowledge. However, improvement in performance on the post-course examination in years 1 and 3 was similar despite the increase in underclassmen in year 3 (Figs. 2 and 3). These findings suggest that active learning strategies, such as TBL, may be able to overcome decreased experience with undergraduate academic coursework and a decreased fund of knowledge in more junior undergraduate students.

The vast majority of students enrolled in the Medical Microbiology course are URM students (Table 2). Prior research has shown that URM students struggle with performance in STEM courses (16, 22). The ability of TBL to improve performance in the Medical Microbiology course demonstrates the utility of this teaching strategy for URM students participating in STEM courses. Indeed, several researchers have shown that introduction of active learning strategies improves performance of URM students in introductory STEM courses (13, 15) and increases their scores on higher-level thinking questions (26).

Students who take the Medical Microbiology course do not receive undergraduate course credit and are taking the course primarily due to interest in pursuing careers in healthcare-related fields. The only external incentive for good performance in the TBL exercises was a gift card that was publicly awarded to the highest performing team on the Application exercise. Some students expressed frustration when their team did not achieve the award. Thus, it appears that this modest incentive may have been sufficient for some to improve their performance in the TBL. Interestingly, our first-year medical students resented the grading of the Application exercise and felt that this practice increased student anxiety (4). Perhaps the use of external incentives other than grading may be a future direction for investigation into alternative ways to provide external motivation without increasing student anxiety.

Anecdotal reports from prior students revealed that participation in the Medical Microbiology course has sparked an interest in pursuing further Microbiology and Immunology coursework at the undergraduate level. Indeed, the perception is that this course will help with future careers in biomedical sciences (Table 3). Of the 25 students in our first student cohort, 16 were accepted to medical school, one is in a Physician Assistant program, and five are still in college and/or post-baccalaureate programs with plans to apply to professional school in the next year or two. Three of these students have decided not to pursue a health career profession. In our second cohort of students, six have been accepted to medical school, and eleven others will apply to medical school or other health professions programs this year. Students from the third cohort remain in college. Many will apply for another phase of our pipeline program this summer.

Additionally, some PULSE students, who are now medical students, have reported the importance of the Medical Microbiology course in providing them with a strong background as they participate in microbiology, immunology, and infectious diseases courses at the medical school level. Therefore, courses such as the Medical Microbiology course offered as part of our institution’s PULSE program provide URM students with the opportunity to explore advanced scientific topics, which may help them choose a future in a healthcare-related field, as well as provide a solid foundation for future success in medical and health-related professions.

Ultimately, our study conclusions are limited by numerous factors inherent to examination of the effect of novel active learning strategies on educational outcomes. Notably, our student population changed from year to year. However, as stated above, we found that even despite the larger number of underclassmen in year 3 compared with year 2, we still saw improvement in post-course examination performance in association with increased use of TBL.

Overall, we feel that increasing TBL exercises fosters greater retention of course material, because it reinforces what students learn through didactic lectures and independent study. Also, TBL exercises provide students with opportunities to apply learned concepts to higher-level problems, promoting critical reasoning skills (17). Additionally, the opportunity to experience self-directed learning under the guidance of experienced undergraduate medical students provides students with a head start in using this very important skill. Finally, pipeline programs that foster interest in the biomedical sciences amongst undergraduate URM students could help reverse recent declines in enrollment of certain groups of URM students in medical schools (3). Use of active learning strategies, such as TBL and supervised self-directed learning, in these pipeline programs could further enhance participant success, which could lead to increased URM student enrollment in medical school as well as increased interest in careers in healthcare-related fields and biology.

Appendix 1: List of topics, content, and learning objectives for the Medical Microbiology course

Appendix 2: Sample TBL on innate and adaptive immunity