The effects of team-based learning on learning outcomes in a course of rheumatology.

INTRODUCTION: We evaluated the effects of implementing Team-Based Learning (TBL) on student engagement, accountability, satisfaction, and preference for lecture or team-based learning. Moreover, we assessed the effect of TBL on knowledge retention and application over time through short answer questions based on clinical scenarios addressing history taking and diagnosis skills in medical students.
METHODS: The study was conducted in a quasi-experimental design. The study population were all of the third-year medical students (n = 84) participating in a course of rheumatology in Shariati Hospital, which is a teaching hospital affiliated to Tehran University of Medical Sciences. We compared TBL with the conventional lecture-based method. The assessments were performed after implementation of TBL by the Classroom Engagement Survey (CES) and Team-Based Learning Student Assessment Instrument (TBL-SAI). The assessment for application of knowledge was conducted in 3 time-points through short answer questions on rheumatic diseases. The comparison of results was made by Student's t-test and repeated-measures analysis of variance (RM-ANOVA) using SPSS software, version 16.
RESULTS: The CES scores indicated a high level of engagement in TBL (Mean±SD=26.7±3.70, p=0.0001) but not in the lecture-based sessions (Mean±SD=23.80±4.35, p=0.09). The total mean score (SD) for TBL-SAI was 159.68 (14.14) for TBL sessions indicating a favorable outcome (p=0.0001). The student scores obtained from the short answer questions showed that over time the students' scores had declined significantly less for the TBL sessions in comparison to the lecture-based sessions, F (2, 166) = 4.624, p=0.011.
CONCLUSION: The results indicated higher student engagement, satisfaction and long term learning by TBL.

Introduction

In recent decades, many educational institutions have emphasized the necessity of shifting from conventional teacher-centered and discipline-based curricula towards a more integrated and learner-centered curricula in medical education (1). TBL was first introduced by Michaelsen, et al. for business courses (2). Due to its specific features that made it suitable for medical education, TBL was later adopted by various medical schools (3). TBL is based on the pedagogical principles and the constructivist learning theory (4). The principles of student-centered learning and supportive scaffolding are essential in the constructivist learning theory (4). Based on these principles, the learner is defined as an individual that is responsible for his/her learning and the learner is required to determine his/her own educational needs, guide the process of learning for his/her own learning, and actively participate in solving the problems in the group. In TBL, the learning process occurs with combining the information from the new learning experience (i.e. second and third phases of learning in TBL) with the previous information (i.e. first phase of TBL and the activities prior to the class). Learners must process the new subjects and integrate them with their existing ‘information structure’ to form or restructure the cognitive schema, which is unique to them based on their learning process. Also, supportive scaffolding is emphasized as a tool for supporting peers in small groups. In these groups, the learners have the opportunity to discuss the challenging subjects in team readiness assurance tests (TRAT) and group application exercises.

TBL also possesses the advantages of the social family model and the cooperative learning approach (5). The social family model emphasizes the tasks that enhance social interaction and academic learning (6). Cooperative learning promotes group interactions for analyzing the problems, finding solutions, taking decisions, and reporting the decisions as a team (1,5,7-10). Moreover, TBL focuses on knowledge application in a collaborative environment, increases the motivation for learning, and creates a concept map that would lead to deep learning (11-13). In group application exercises the individuals can develop and improve their critical thinking abilities in a process of active learning. Reflection in action, which is a crucial factor in effective learning, is improved in TRAT and in solving the case problems in the group application exercises (4).

The seven core elements in TBL include (a) team formation, (b) readiness assurance, (c) immediate feedback, (d) sequencing of in-class problem solving, (e) structure for developing team application tasks (i.e. the “four Ss"), (f) incentive structure, and (g) peer evaluation (14). These elements lead to improvement of self-directed learning, critical thinking, team-work, interpersonal communication, peer learning, and problem-solving skills in the TBL participants (5,8,11,14-22).

In recent years, various medical schools have applied TBL as an interdisciplinary and integrated approach to education for pre-clinical courses (11,23,24), internship (25,26), and residency (27) programs as well as continuing medical education programs (28). According to Inuwa, et al. implementation of TBL leads to deep understanding, and improvement of creative thinking and analysis skills of the learners (8). In a study by Koles, et al. (29) it was shown that TBL can effectively enhance the performance of weaker students in the final exams. Moreover, a study by Pogge has shown that application of TBL could improve the students’ confidence in patient counseling and improve their communication and team-work skills (30). In a study in an anatomy course, Vasan, et al. predicted that using clinical cases for TBL leads to the learners’ improved ability to apply anatomy knowledge for a longer period in clerkship years (1). They also piloted this prediction in a small section of the biochemistry course, while retaining most of the lectures, and their results were encouraging.

A systematic review study in 2013 which evaluated the effectiveness of TBL on learning outcomes in health professions education underlined the positive effects of TBL in improving knowledge scores (31). However, the study found mixed results in the literature with respect to the learning reactions of the learners to the TBL method. Moreover, the review found no reports in the literature evaluating the effect of TBL on knowledge retention over time. The study also recommended further research on learner satisfaction and accountability in TBL. In addition, this study highlighted the need for further research to evaluate the effect of TBL on higher level learning outcomes such as knowledge application.

In the present study, we have assessed the effect of TBL on accountability, recall, attention levels, and satisfaction of the students. Moreover, the learners’ engagement with TBL was compared with the conventional lecture-based method in a course of rheumatology. In addition, in this study we evaluated the effect of TBL on knowledge retention over time. We also assessed the knowledge application skills of the learners through short answer questions based on the clinical scenarios on history taking and diagnosis skills.

Methods

The study was performed in a quasi-experimental design. The participants were all of the third year medical students (n=84) who were participating for the first time in a course of rheumatology at Shariati Hospital, Tehran University of Medical Sciences.

Educational interventions

Educational interventions were comprised of TBL and lecture-based sessions, concerned with the history taking, physical examination, and diagnosis skills in the rheumatic diseases. TBL was implemented in sessions for rheumatic fever and periartheritis (3 sessions) and was compared to the conventional lecture-based sessions for systemic lupus erythematosus and ankylosing spondylitis (3 sessions). Each session lasted for about 3 hours and the sessions were held at one week intervals.

We conducted both interventions on the same group of participants. Lecture-based sessions were held at the beginning of the course before TBL sessions. The students had no prior exposure to the course content (i.e. the skills and the knowledge that was presented in the course). Coordination sessions about the course content and the teaching methods were held before the courses had started. The team of instructors was comprised of three rheumatologists and a specialist in medical education. Planning and design of the course as well as the preparation of group activities, educational cases, and readiness assurance tests and the short answer questions (SAQ) were developed by these instructors.

In order to implement the TBL, an orientation session was held for the learners to introduce them to the different steps in a TBL session. Then, the educational intervention was applied through the following steps: The first step was “preparation”. Prior to the commencement of the session, the students were provided with reading assignments that were based on the educational content for each session. The students were asked to study the material that was provided to them and to get prepared for the TBL session. At the beginning of each session, groups were formed. As the students were previously familiar to each other they were allowed to choose their team members to form a group comprising of six members. In the second step, Individual Readiness Assurance Test (IRAT) was conducted. IRAT in the current study was comprised of 11 multiple-choice questions (MCQs) that were based on the intended educational material for the course. In the third step, we conducted the Team Readiness Assurance Test (TRAT). Immediately after the individual tests, the students were asked to retake the same MCQ test but this time in a group. At this stage, the students were expected to discuss the questions with group members to reach an answer. At the end of the TRAT the answers provided by each group were corrected by the facilitator and immediate feedback was provided to the groups. Individual and team readiness tests were performed to promote learning and teamwork skills. “Appeals” was the next step. At this point, the teams were allowed and encouraged to refer to their assigned reading materials to appeal for any missed questions in order to reason and defend their answers. After the test the students could fill in appeal forms for their missed questions at TRAT. In the present study, no appeals were submitted by the students. In the next step, the faculty members (i.e. facilitators) provided feedback to the students. At this stage the questions that raised more discussion among the students were further discussed. The last step was Team Application (TAP). It is the most important step in TBL that provides the grounds for group interactions in order to analyze the problems, find solutions, take decisions, and report back the decisions in a team. The assignments in this step involved the application of reading contents through scenarios that engaged higher-level cognitive processes (1,5,7-10,21). At this step, all groups worked on the same scenarios (i.e. problems) and were asked to simultaneously report back their answers (5). Students were asked to find the problem in the case, decide on the key history questions and physical examinations, and provide a list of differential diagnoses.

Assessments

a) Team-Based Learning Student Assessment Instrument (TBL-SAI)

Team-Based Learning Student Assessment Instrument (TBL-SAI) was filled in by each participant after the TBL intervention. TBL-SAI is a 39-item instrument using a 5-point Likert scale from strongly disagree to strongly agree as possible response options (32). Three domains of accountability (13 questions), preference for lecture or team-based learning (16 questions including student recall, and attention levels, and student satisfaction (10 questions) are measured by TBL-SAI. Accountability “occurs when students prepare in advance for a class and/or contribute to other members of the team” and is measured by the accountability subscale. Student recall refers to “the ability of students to retrieve previously learned knowledge for use at a later time” and is operationally defined by the questions in preference for lecture or team-based learning subscale. Attention levels i.e. “the student’s ability to stay focused and concentrate on the course content during the conventional lecture or TBL activities” are measured by questions in preference for lecture or team-based learning subscale. Student satisfaction includes “positive feelings toward either TBL activities or the conventional lecture” and is measured by student satisfaction subscale (20,32,33).

b) Classroom Engagement Survey (CES)

The Classroom Engagement Survey (CES) is an 8-item questionnaire that evaluates student engagement in the class (34). Two types of engagement were considered in this regard: a) engagement with content that “occurs as the student thinks about the course content” and, b) engagement with peers that “occurs when the students interact with each other and participate in discussions and course activities”. CES was filled in by the participants after the TBL and lecture-based sessions.

Validity and reliability of the TBL-SAI and CES instruments were confirmed in a study by the authors. The instruments had shown acceptable internal consistency and reproducibility as measured by Cronbach’s alpha and Intra class correlation coefficient (ICC). Cronbach’s alpha and ICC for the TBL-SAI instrument were 0.79 and 0.82, respectively. In addition, Cronbach’s alpha and ICC for the CES instrument were 0.71 and 0.75, respectively (35).

c) Short Answer Questions Based on the Clinical Scenarios

We developed the short answer questions (SAQs) to assess the “know-how” of history taking, examination, and diagnosis skills among the participants. We selected SAQ as it allows appropriate content coverage and provides a higher chance for the assessment of clinical reasoning and decision making capability (36). The assessment of higher levels of learning and the students’ skills for knowledge application were assessed by five SAQs based on clinical scenarios. According to the educational content for each session, SAQs were developed based on a similar stem and structure by an expert panel. The questions related to rheumatic fever and periartheritis were intended for TBL sessions and the questions on systemic lupus erythematosus and ankylosing spondylitis were developed for lecture-based sessions by the panel (see the Methods section for further description about the expert panel). In the present study, pre-testing was not conducted in order to keep away from pre-test bias effect on the study results. The students were tested in the intervention and control groups at three time points i.e. immediately after the sessions, at two weeks, and at four weeks after each session.

Statistical methods

The frequency and percentages were used for the description of sample characteristics. Mean and standard deviations (SD) were calculated for all instrument scores. One-sample t-test was used for the comparison of TBL-SAI subscale and total scores as well as CES scores with the reference values (20,33). We used repeated-measures analysis of variance (RM-ANOVA) to evaluate the effect of educational interventions (i.e. TBL vs. conventional lecture-based sessions) on knowledge retention over time (i.e. at 0, 2, and 4 weeks after the intervention) in a within-subjects design. In this regard, the “Type of Educational Intervention” and the “Time of Assessment” were regarded as repeated measures variables and the students’ scores obtained from the SAQs as the outcome measure. We used repeated contrasts for post hoc analysis. P-values less than 0.05 were considered statistically significant. We used partial eta squared (ηp2) for effect size calculations. We used ηp2 as a quantitative measure of the strength of the observed effects in RM-ANOVA. According to a general “rule of thumb” ηp2 effect sizes equal to 0.01, 0.06 and 0.14 are considered small, medium, and large effect sizes, respectively (37). We used SPSS software, version 16 for statistical calculations.

Results

Eighty-four medical students from Tehran University of Medical Sciences participated in the study. The participants had a mean age±SD of 22±2.0 years. There were thirty males (34.88%) and fifty-six females (65.11%).

a) Readiness Assurance Tests

The mean score for IRAT was 8.53 out of 11. The mean score at the group stage (i.e. TRAT) was 10.25 out of 11 showing a statistically significant improvement (p=0.001).

b) Team-Based Learning Student Assessment Instrument (TBL-SAI)

Table 1 shows the mean scores of the participants at different domains of TBL-SAI after the TBL intervention.

Table 1

The mean scores of participants obtained from the Team-based Learning Student Assessment Instrument (TBL-SAI)

TBL-SAI Domains	NO.	Mean±SD	Reference Value *	Max	p
Accountability	13	46.43±6.54	39	65	0.0001
Preference for lecture or team-based learning	16	51.90±5.50	48	80	0.0001
Student satisfaction	10	38.08±4.02	30	50	0.0001
TBL-SAI (Total)	39	159.68±14.14	117	195	0.0001

One-sample t test was used to compare the mean scores of the participants with the reference values which indicate the neutral value for each domain of the instrument.

Reference values are obtained from (33).

c) The Classroom Engagement Survey (CES)

The mean scores for the student engagement were 26.7±3.70 and 23.80±4.35 for the TBL and lecture-based sessions, respectively. The comparison of results with the reference score of 24 showed a significantly high level of engagement in the TBL group (p=0.0001). However, the difference from the reference value was not statistically significant in the lecture-based sessions (p=0.09). There were also no association between the CES scores and the SAQ scores in the TBL group (p=0.08).

d) Short Answer Questions Based on the Clinical Scenarios

The mean scores for the SAQs obtained from the students after the TBL sessions were 26.97±5.01, 26.73±6.28, and 26.45±8.66 at the first, second and third assessments, respectively. After the lecture-based sessions, the mean scores obtained from the students at the first, second and third assessments were 28.11±5.96, 26.87±5.64, and 25.04±6.67 respectively (Figure 1). The results of RM-ANOVA showed that there was not a significant ‘main effect’ of the Type of Education (p= 0.845) and the Time of Assessment (p= 0.191) on the students’ SAQ scores (Table 2). We found a significant interaction effect between the time of assessment and the type of educational intervention,(p = 0.011, ηp2 = 0.05, Table 2). The significant interaction effect between the Type of Education and the Time of Assessment in this study indicated that the decline in SAQ scores over time had been significantly different across the educational methods (i.e. TBL vs. lecture-based sessions). In other words, although the students’ SAQ scores after the teaching sessions had naturally declined over time, the students’ SAQ scores had a significantly slower rate of decline after the TBL sessions in comparison to the lecture-based sessions. The slower decline in SAQ scores after the TBL sessions indicated a significantly better retention of knowledge over time after the TBL sessions. Planned contrasts revealed that the decline in the student scores after the TBL and lecture-based sessions (i.e. the difference in the rate of decline across the educational interventions) had not been significantly different at two weeks (p=0.099), but had become statistically significant at 4 weeks (p=0.004, ηp2=0.10) after the educational intervention (Table 2). These results indicated that the effect of TBL on knowledge retention had become more pronounced over time.

Figure1

Comparison of student scores obtained from the short answer questions (SAQs) after the TBL and lecture-based sessions

Table 2

Effects of educational methods on the participant’s short answer question (SAQ) scores over time

Main effects and interaction effect
Within-subjects variable	Sum of squares	dfM	dfR	F	p-value	ηp2
Type of Education	2.431	1	83	0.03	0.84	NS
Time of Assessment	154.51	2	166	1.70	0.19	NS
Type of Education * Time of Assessment	256.12	2	166	4.62	0.01	0.05
Planned contrasts 1
1st assessment vs. 2nd assessment	300.96	1	83	2.77	0.09	NS
1st assessment vs. 3nd assessment	1022.01	1	83	8.79	0.004	0.10

dfM= degrees of freedom for the effect of the model, dfR= degrees of freedom for the residuals of the model, NS = not significant, ηp2 = partial eta-squared

Panned contrasts are only reported for the significant interaction effect of Time of Assessment * Type of education.

Note: First, second, and third assessments were performed at 0, 2, and 4 weeks after each educational intervention, respectively.

Discussion

The results of the current study showed a favorable outcome with regard to the accountability, satisfaction and recall of the participants in the TBL sessions. Our results also demonstrated a downward trend for the student test scores obtained from SAQs that had a significantly slower rate after the TBL sessions in comparison to the lecture-based sessions.

In the current study, the comparison of student scores with the reference values for the instrument (32,33) showed that the student ratings obtained from the different domains of TBL-SAI had been significantly high for the TBL sessions. The accountability subscale in TBL-SAI concerns the students’ preparation prior to the class and their contribution to the other team members. TBL requires prior reading and having enough mastery of the subject so that the learners can actively participate in the group discussions and analyze and solve the group assignments and the clinical scenarios. The students’ engagement in the teaching and learning process resulted in reaching a higher level of understanding (38). The results of the current study, demonstrated a favorable accountability for the TBL group that was comparable to the results by Corbridge et al. (16) and Mennenga (20). The current study also demonstrated the learners’ preference for TBL to the lecture-based sessions that was in line with the other reports (11,16,20,22,30,39). The teamwork and the students’ engagement in the process of learning in small groups have led to students’ preference for TBL.

Learners’ satisfaction and collaborative environment are among other factors that influence effective learning. The learner reaction and satisfaction from TBL have been a debated subject in the literature (31). Fatmi, et al. hypothesized that the “increased student workload” and “shift in culture towards peer assessment” and “accountability” could negatively affect student reaction to TBL (31). In the present study, the learners showed positive reactions to the TBL that could be partly explained by the positive effects of having an interactive learning experience in a small group. The TBL sessions provided the opportunity for peer learning in the small groups and familiarized the learners with their peers’ approaches to history taking and physical examination. In this study, peer learning in the small groups could possibly have had synergistic effects on the students’ learning experience (40,41). and the improvement of their teamwork and clinical skills led to satisfaction with TBL. Moreover, as the educational system in Iran is based on the didactic teaching methods, experiencing interactive teaching in the course of rheumatology and facing with practical cases in the process of teaching could have possibly affected the students’ learning experience and satisfaction with the TBL. However, performing a qualitative study is recommended to explore the factors that affect learner’s satisfaction. The results of the current study showed significant improvement in the readiness assurance test scores at the group stage. Further, the results of the study showed a significantly higher student engagement for the participants in the TBL sessions. These results are in line with other reports that have confirmed the role of TBL in creating a favorable environment for collaborative learning (11,15,16,19,20,22). The different components of TBL, such as developing an interactive team and engaging the learner in the process of learning, positively affect the process of learning, learner’s satisfaction, and accountability. Different studies have shown various effects of TBL on the cognitive domains (11,15,22,30). The aim of implementing TBL is not necessarily to improve students’ knowledge but it is to also improve communication and problem-solving skills, interprofessional collaboration, and life-long learning that are achieved by the teamwork (42,43). Therefore, in TBL, it is expected that by putting emphasis on pre-reading techniques, self-directed learning, and working in small groups improvement in higher level knowledge is also achieved. In this regard, the application of individual tests before group sessions facilitates problem solving during a group session that leads to improvement in the group scores and reduction in the time spent on group activities (17). Improvement in the TRAT scores in comparison to the IRAT scores in the present study is in line with many reports that have emphasized the synergistic effects of group work and peer-learning in this regard (16,17,20,22,23,30,39). Therefore, it is possible to expect that in the small groups of TBL, with adequate understanding of the clinical scenarios, the learners can develop better problem solving, clinical reasoning, and teamwork skills.

The results of our study showed a significant difference in the trends of decline over time in the SAQ scores obtained after TBL sessions versus lecture-based sessions (Figure 1). In this study, we observed a statistically significant interaction effect between the time of assessment and the type of educational intervention, which indicated a different rate of decline in the SAQ scores over time across the educational interventions. The contrasts revealed that although the decline in the SAQ scores had not been significantly different between the TBL and lecture-based sessions, at two weeks, it had become significant at four weeks. In the TBL sessions, the learners attempt-both individually and in the group-to solve the problems related to the clinical cases, and providing feedback by the facilitator, positively affects their learning and the knowledge retention in learners. Therefore, it is possible to predict that the application of TBL would result in a more long-term learning and retrieval of information. This is in line with a study by Imanieh et al., where the problem-based learning (PBL) led to a significant improvement in learning and recalling output as compared to the traditional method over time (44). It has been shown that learning in PBL could be improved by active participation of learners in the homogeneous small groups (45). A study conducted by Clark, et al. among 70 undergraduate nursing students, with the application of TBL in a 7-week educational course, led to improvement in problem solving and communication skills of the learners (11). In a study by Koles, et al., related to the education of “Pathology-Based Content” in two separate semesters with different content materials, a comparison was made between TBL and other educational methods (16). The results showed higher knowledge test scores for TBL in comparison to the other methods. According to Koles, et al. the students in the lowest academic quartile benefited more from TBL sessions than the students in the highest quartile. These results also emphasize the positive long-term effects (for two semesters) of TBL that would lead to better academic performance over time (19). Improvement in knowledge scores has been more prominent in the studies where TBL was applied for a longer period or when MCQs were used for knowledge testing (15,22,30,46,47).

Limitations

One of our limitations was the quasi-experimental design of the study. However, since the purpose of this study was evaluating the effect of TBL on knowledge retention over time (which is presumably very much affected by the cognitive abilities of the learner) using a quasi-experimental time series design allowed us to control for the individual difference among the learners in retention of knowledge. In the present study, we applied TBL for a short period of time. Also, the number of TBL sessions and assessments in our study were limited. Therefore, although the results of the present study concerning better “knowledge retention” after the TBL sessions are very promising, further research is still required to evaluate the long-term learning effects of TBL for an extended period of time (i.e. after four weeks).

Conclusion

TBL is a learner-centered method that can be implemented in integrated educational programs and interprofessional training in different disciplines in medical sciences. The results of this study showed that implementing TBL increases learners’ participation and satisfaction with the course. This could lead to deeper learning, long-term knowledge retention and better performance regarding the practical knowledge. Therefore, it is recommended to apply TBL in interprofessional educational programs and in pre-clinical courses by using real clinical cases.