Comparing the impacts of reciprocal peer teaching with faculty teaching: A single-centre experience from KSA.

OBJECTIVES: This study aimed to compare the effectiveness of peer-led learning with faculty-led teaching activities in fifth-year students during their Internal Medicine II placement.
METHODS: We conducted a retrospective analysis of the results of multiple-choice questions exams at the end-of-placement for 2016 to 2017.
RESULTS: During the study period, 120 students in three cohorts completed the placement. There was a statistically significant difference (p < .001) between the mean student grade (% of correct answers) for questions related to lectures (n = 85, M = 68.7, 95% CI: 66.4-70.9), case scenarios (n = 24, M = 68.6, CI: 65.1-72.1), tutorials (n = 15, M = 64.9, CI: 59.9-69.8), and seminars (n = 24, M = 60.0, CI: 56.5-63.5). The first cohort had statistically significant better grades (M = 79.7, CI: 77.9-81.6) than the other two cohorts (group 2 M = 57.2, CI: 51.8-62.5; group 3 M = 60.5, CI: 58.1-63.0; p < .001).
CONCLUSIONS: In our study, students performed worse on questions drawn from reciprocal peer-taught seminars. This result could be attributed to students' poor teaching quality, as they were randomly selected to lead a seminar. However, this study was limited by heterogeneity in the number and topics of different teaching activities and certain other factors.

Introduction

A decade has passed since ten Cate and Durrning summarised the twelve most important reasons for implementing peer teaching in curricula. It offers a comfortable and safe educational environment, ensures that education is at an appropriate cognitive level, enhances motivation, and provides alternative methods of studying. Further, it helps students socialise, gain experience in leadership, and prepares them to be educators and supervisors—roles and responsibilities recognised as necessary for most modern doctors.,

Beyond the positive effects on the tutors themselves, students who are taught by peer-teachers have been found to achieve comparable performance to those taught by experienced clinical teachers in a variety of settings, from problem-based learning, to clinical, and surgical skills. Although in other disciplines, peer tutoring has a positive impact on students' academic achievement, in the context of medical education, there appears to be no significant difference in knowledge or skills outcomes of students taught by either student peers or faculty members.

When the College of Medicine at Al Imam University, KSA, was founded in 2007, it established a new curriculum that follows an innovative approach to medical education. It was developed by a fundamental committee that contained national and international experts with funding by the Saudi Government, and the program's clinical activities are supported by the King Saud Medical City tertiary care centre.

Due to the recognition of the benefits of peer teaching, reciprocal peer-led seminars became an integral part of the curriculum alongside tutor-led lectures, case scenarios, tutorials, problem-based learning, and clinical skills sessions. All students are required to lead seminars on a pre-determined topic, starting from their second year to their fifth and final year. To prepare them for this role, students receive training in presentation skills as part of the Learning Skills II course during their first year of university and an Evidence Based Medicine course in the second year, in which they learn how to identify relevant evidence and present important findings effectively.

Reciprocal peer teaching, as implemented at our institution, in which students of similar levels alternatively assume the roles of tutor and learner to meet identified educational outcomes, is inadequately addressed in the literature., In most other studies, students volunteer to become tutors, undergo specific training, or are selected based on their academic performance or some other assessment., Volunteering introduces selection bias, as tutors comprise those who have the skills, enthusiasm, and appropriate attitude towards teaching, thus increasing the chances of a successful program. Furthermore, in their systematic review, Rees et al. identified only ten studies that reported on peer teaching in undergraduate medical education, while Gazula et al. found only two studies supported by objective measurements specifically in reciprocal peer teaching. We could only identify one study in KSA that utilised reciprocal peer-teaching, which was in the form of small, informal peer-assisted learning groups among dentistry students and found that students with better pre-course grades benefited the most. The authors examined only one group without making a comparison with faculty teaching, and the study period was particularly short (two weeks).

Further, it is important to assess the success of an educational program from a pragmatic standpoint, beyond strict research protocols. Typically, subjective and objective measurements can be utilised—from student and faculty opinions to exam performance., As informal feedback from students had highlighted that complicated clinical cases and exam-related topics were not stressed sufficiently, we wanted to evaluate the program in a twofold manner. In a separate study, we sought students' perceptions and demonstrated that students generally have positive opinions of the peer teaching program in our institution, both as students and tutors. However, more students (38.4%) felt that the peer-led seminars did not prepare them for the exams than those who felt it did (27.9%). In this study, we aimed to examine whether objective measurements can be used to appraise the program. The findings may be used to inform changes in the way the peer-teaching program is set up and whether it is feasible to utilise objective measurements in a real-world setting.

Therefore, we conducted a retrospective study to compare peer teaching to teaching by faculty or experienced clinical staff in students who completed the Internal Medicine II placement. At the end of the clinical placement, students are required to appear for an exam of 50 multiple-choice questions (MCQ) and an objective structured clinical examination (OSCE). The former primarily tests students' knowledge, and the latter mainly tests their clinical history and examination skills. The objective was to identify students'’ exam performance on topics that were taught by peer teachers and compare it with performance on topics taught by faculty. We hypothesised that there would be no difference between exam performance for both types of teaching.

Materials and Methods

Setting and students

We studied the exam performance of fifth-year medical students during the Internal Medicine II placement for 2016–2017. The students had a curriculum with well-defined specifications, goals, and objectives and rotated through one week each of Critical Care and Oncology and two weeks of Neurology. The placement was chosen because the authors had direct access to the exam questions and results. Three groups of medical students completed the course each year, each of which was divided into two subgroups.

Intervention

Reciprocal peer teaching was conducted in the form of one-hour-long student-led seminars. All students were randomly selected at the beginning of the year to present and lead a discussion on a set topic (Table 1) during a seminar in their subgroup. The subject and objectives of each activity were pre-determined within the curriculum, and there was uniform coverage of all clinical subjects with the focus on knowledge. A faculty member was present as a strict observer to provide feedback to the student leading the seminar.

Table 1

Teaching activities.

Activity	Format	Conducted by	Number
Lecture	A formal presentation.	faculty or experienced clinical teacher	18
Case scenario	A formal discussion covering multiple and varied case scenarios, supported by presentations.	faculty or experienced clinical teacher	7
Tutorial	An interactive small group presentation.	faculty or experienced clinical teacher	3
Seminar	An interactive small group discussion supported by a presentation.	same-level peer (medical student)	5

The activities have set objectives and goals that are pre-determined in the curriculum, and there is no cross-covering of topics between activities.

Comparators

During the placement, students also attended faculty-led lectures, case scenarios, and tutorials (Table 1), which lasted 1 h each. There was no cross-coverage of topics between peer-led seminars and faculty-led activities. Students also attended clinical skills sessions, and as such, they could not serve as a comparator to the peer-led seminars.

Outcome measure

As an outcome measure, we chose MCQ exam performance in percentage of correct answers.

Students were required to appear for a 50-item MCQ exam at the end of the placement that tested their knowledge on the various topics encountered through the above activities. A committee comprising four experienced faculty members, who did not participate in any form of teaching, prepared the exam questions.

Each question tested a specific objective unique to a teaching activity, which could comprise a peer-led seminar or faculty-led lecture, case scenario, or tutorial. One to two questions were drawn from the topics of every teaching activity and, as such, each exam had a proportional number of MCQs per teaching activity, strictly adhering to each activity's objectives and goals.

If more than 80% of students answered a question incorrectly or correctly, the question was considered too difficult or too easy, respectively. It was therefore removed and did not count towards the overall grade.

Overall exam performance was measured as a percentage of correct answers in the final MCQ exam, and performance relating to a teaching activity was measured as a percentage of correct answers in MCQs based on topics and objectives taught only during that specific type of activity. The university uses the following grading system: <50% is fail, 50%–59.99% is pass, 60%–74.99% is good, 75%–89.99% is very good, and ≥90% is excellent.

Peer-led seminars did not focus on clinical history and examination skills, which are taught through clinical skills sessions and assessed through OSCEs. Therefore, performance on OSCEs was not considered in this study.

Statistical analysis

Statistical analysis was performed using JASP Version 0.9.0.1, and non-parametric tests were used because all assumptions for parametric tests were not met. Descriptive statistics were provided in numbers (n), percentages (%), and means (M). The Kruskal–Wallis Test was used to identify differences between groups in terms of exam performance. Further, Friedman's test was used to identify differences between performance relating to each teaching activity in terms of percentage of correct answers for each student. Moreover, the Mann–Whitney U Test was used to identify differences between gender or individual groups in terms of exam performance, overall, or peer-teaching activity. Additionally, the Chi square test was used to examine differences in the number of questions relating to the different teaching activities in each exam. A p-value of < 0.05 was set as significant.

Results

During the 2016–2017 academic year, three groups of a total of 120 students completed the Internal Medicine II placement. There were 45 male students in the first group, 24 female students in the second group, and 51 male students in the third group. Therefore, three sets of exams were analysed.

Most multiple-choice questions (57%) related to objectives met with lectures (Table 2), and only 10% of the questions related to tutorials, whereas 16% of questions were associated with peer-led teaching. A Chi-square test indicated that the variation in the numbers of each type of question between the three exams was not significant, χ = 4.66, p = .588. The ratio of questions per type of teaching activity ranged from 1.14 for case scenarios to 1.67 for tutorials (Table 2).

Table 2

Number of questions per teaching activity for each group.

Teaching Activity	Group 1	Group 2	Group 3	Total	Ratio Q/N
Lecture	30	31	24	85	1.57
Case scenario	7	7	10	24	1.14
Tutorial	6	3	6	15	1.67
Seminar	5	9	10	24	1.60
Total	48	50	50	148

A Chi-square test indicates that the variation in the numbers of each type of question between the three exams was not significant, χ2 = 4.66, p = .588. The Ratio of Questions (Q) per number of activities (N) is also displayed.

Comparing the grades relating to the various teaching activities across the whole cohort, students performed worse in questions relating to seminars (Figure 1). There was a statistically significant difference (p < .001) between the mean student grade (% of correct answers) on questions relating to lectures (n = 85, M = 68.7, 95% CI: 66.4–70.9), case scenarios (n = 24, M = 68.6, CI: 65.1–72.1), tutorials (n = 15, M = 64.9, CI: 59.9–69.8), and seminars (n = 24, M = 60.0, CI: 56.5–63.5). A non-parametric Friedman test of differences among repeated measures was conducted and rendered a Chi-square value of 28.81, which was significant (p < .001). Post-hoc comparisons indicate that there was a statistically significant difference between the grades relating to seminars and those relating to lectures (p < .001), case-scenarios (p < .001), and tutorials (p < .001).

Figure 1

Mean student grade (% of correct answers) per teaching activity for the whole cohort, with 95% Confidence Intervals. There was a statistically significant difference (p < .001) between the mean student grade (% of correct answers) on questions relating to lectures (n = 85, M = 68.7, 95% CI: 66.4–70.9), case scenarios (n = 24, M = 68.6, CI: 65.1–72.1), tutorials (n = 15, M = 64.9, CI: 59.9–69.8), and seminars (n = 24, M = 60.0, CI: 56.5–63.5).

Overall, the first group had the highest percentage of correct answers (M = 79.7, SD = 6.09), followed by the third (M = 60.5, SD = 8.6), and finally by the second group of students (M = 57.2, SD = 12.6) (Figure 2). Although the Kruskal–Wallis test indicates that the first group's grades were statistically better than that of the other groups (p < .001), the second and third group results were quite similar (p = .333).

Figure 2

Boxplot of overall grade (% of correct answers) of each group. Group 1 scored higher (M = 79.7, CI: 77.9–81.6) than either group 2 (M = 57.2, CI: 51.8–62.5) or group 3 (M = 60.5, CI: 58.1–63.0), and this was statistically significant (p < .001). Groups 2 and 3 had similar grades.

Students from the first group scored better in MCQs drawn from tutorials (M = 86.7, SD = 12.6), while those of the second and third groups scored better in MCQs relating to lectures (M = 61.8, SD = 11.9 and M = 63.2, SD = 0.2 respectively) (Table 3). The first group scored better across all teaching activities compared to either the second or third groups, and this difference was found to be statistically significant (p < .001). The differences between the second and third groups were only statistically significant for the MCQs drawn from tutorials.

Table 3

Grades (% of correct answers) relating to each teaching activity per group.

Teaching Activity	Group	Mean	SD	Lower 95% CI	Upper 95% CI
Lecture	1	78.6	8.09	76.1	81.0
	2	61.8	11.98	56.8	66.9
	3	63.2	10.22	60.3	66.0
	All	68.7	12.47	66.4	70.9
Case scenario	1	82.5	11.35	79.1	85.9
	2	56.6	22.87	46.9	66.2
	3	61.9	16.13	57.4	62.1
	All	68.6	19.49	65.1	72.1
Tutorial	1	86.7	12.61	82.9	90.5
	2a	29.2	28.34	17.2	41.1
	3	62.4	14.85	58.2	66.6
	All	64.9	27.31	59.9	69.8
Seminar	1	74.2	16.31	69.3	79.1
	2	50.9	19.08	42.9	59.0
	3	51.8	14.10	47.8	55.7
	All	60.0	19.35	56.5	63.5

Mean grade in % of correct answers, SD is Standard Deviation.

Students of group 2 only had three questions relating to tutorials, which is likely the reason for the large SD.

There was a statistically significant difference between the overall grade of male (M = 69.5, SD = 12.2) and female (M = 57.2, SD = 12.6) students, U = 560.0, p < .001. The difference was also significant for grades relating to all teaching activities (p < .001).

Discussion

This study indicates that students tend to perform worse in subjects taught through peer-led seminars. The reasons for the difference in performance are likely multifactorial. One cannot dismiss the possibility that in this set-up, peer-led teaching is insufficient to meet the standards required. Students receive training in presentations and research skills early in the curriculum, and by their fifth year, they may already have previous experience in teaching from earlier years. However, they are likely unprepared to lead such an activity in the context of the ever-increasing complexity of medical knowledge and required skills. Informal feedback received from the student-learners seems to corroborate this. They felt that complicated clinical cases were inadequately discussed, and exam-related topics were not sufficiently stressed.

A lack of role clarity can lead to confusion among students and peer teachers, and this is likely to have occurred in our setting, as peer-tutors were from the same level and were not specifically chosen for this role. In addition, as most exam questions related to topics discussed during lectures, students may have perceived the objectives discussed in peer-led seminars to be of less importance than topics taught by faculty. They may have thus not prepared adequately for these subjects, further explaining the difference in performance.

Notably, the nine-percent difference in grades per teaching activity, although statistically significant, is not significant from an educational perspective. On average, there were 29 questions relating to lectures and 8 to seminars (Table 2). Two-and-a-half wrong questions from lectures (9% of 29) or an extra correct question from seminars (9% of 8) are enough to close the gap.

The differences in performance between the three groups may be related to admission policies, as the two questions excluded from the exam would only change the overall grade by 4%. Admission to the university is consecutively dependent on academic performance during secondary school, and this likely explains why there are clusters of top-tier students in some groups across the course. Performance relating to different teaching activities is uniform across all groups, except for group 2, which performed worse in MCQs relating to tutorials (Figure 3). This group had only three questions based on tutorials (Table 2), and as such, this discrepancy is unlikely to represent a true difference. Notably, although the MCQs are different for each exam, there are policies in place to ensure consistent difficulty across all groups.

Figure 3

Mean student grade (% of correct answers) per teaching activity of each group with 95% Confidence Intervals. Groups 1 and 3 were all males, and group 2 only contained female students. Students of group 2 scored quite low on subjects taught in tutorials (M = 29.2, CI: 17.2–41.1); however, they only had three such questions. Refer to Table 3 for detailed means and confidence intervals.

Based on the previous discussion, it is also not possible to infer gender differences in the performance of our cohort. Generally, on average, there should not be much of a difference between genders, and in some cases, female students may even outperform male undergraduate students. Al-Mously demonstrated that Saudi female medical students demonstrated a superior academic performance to male students in pre-clinical courses. Although this is the first time female students were admitted to study medicine at the College of Medicine at Al Imam University, the cohort comprises fifth-year medical students. There should have been enough time for both the students, the faculty, and the administration to acclimatise. Considering that there is no statistical difference between the second group (all females) and the third group (all males), the first group might have been unusually better and therefore increase the average performance of male students.

Multiple Choice Questions are quite appropriate for testing content knowledge, and if designed appropriately, can also test higher levels of understanding. As they produce quantitative results, they are easy to grade, results can be obtained quickly, and there is no subjective factor in marking. However, the assessment of teaching effectiveness should be multifactorial and rely on multiple measures., Therefore, the results of this study should not be interpreted as direct evidence of the effectiveness of reciprocal peer teaching compared to faculty teaching. Instead, they indicate that there are small differences in performance in exam questions relating to seminars, and further work is necessary to identify the underlying reasons for these differences.

Limitations

The main limitation of this study is that the topics and objectives are different between the activities, and there is no cross-over in the curriculum. This is a confounding factor that may also explain the variations in scores.

There is also heterogeneity in the numbers of students per group, the number of questions relating to each teaching activity in each exam, and, quite possibly, the quality of teaching (at least relating to peer teachers). In addition, the exams were different for each group, and the equality of the difficulty level cannot be guaranteed.

Finally, we did not account for attendance. Although leading a peer-taught seminar is compulsory and all students participate in a pre-determined schedule, differences in attendance rates could affect the results due to the low attendance of some activities.

Conclusion

Students performed worse on questions relating to subjects taught during reciprocal peer-taught seminars. Although group and gender differences were observed in overall exam performance, the interpretation is inconclusive due to possible confounding factors that inflated the results of one of the male groups. This study is limited by heterogeneity in the number and topics of the different teaching activities, as well as the biases and challenges inherited by retrospective observational studies.

Recommendations

The evaluation of educational programs requires multimodal review, and the results of this study in isolation are inadequate to inform policy. However, we have highlighted a need to assess the effectiveness of reciprocal peer teaching further because there is a statistically significant difference in the percentage of correct answers between different groups of MCQs despite the methodological limitations.

Beyond experimental designs such as a randomised controlled study, future studies require a larger number of students and an exploration of confounding factors. These may include the Cumulative Grade Point Average of the students, attendance rates, the teaching experience of the lecturers (faculty and peers), studying the preferences of the students (for example, to determine whether students receive additional informal tutoring), and the distinct objectives of each session. A review of the policies to ensure equivalent difficulty is necessary for settings that include groups of students across multiple cohorts.

Source of funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

The author has no conflict of interest to declare.

Ethical approval

This project was reviewed by the internal review board (IRB) of Imam Mohammad Ibn Saud Islamic University (IMSIU).