Use of Student-Generated Multiple Choice Questions to Enhance Team-Based Learning of Anatomy at the University of Rwanda.

INTRODUCTION: Teaching of human anatomy has undergone significant changes in the past three decades. At the University of Rwanda, anatomy is being taught using team-based learning (TBL). While student-generated multiple choice questions (MCQs) stimulate deeper thinking on a given topic, their impact on anatomy learning is not known. This study aimed to find out the impact of student-generated MCQs on the current anatomy teaching method at the University of Rwanda.
METHODS: In this comparative interventional study, two similar chapters on anatomy were selected; one was taught using TBL while for the other one, in addition to TBL, students were encouraged to set MCQs while studying. Pre- and post-test scores were analyzed using SPSS 23 and the Student's t-test was used to compare the mean scores obtained.
RESULTS: Thirty-one medical students were recruited. Pre-test mean scores were 25.10 and 25.19 out of 50 for chapters 1 and 2, respectively. Although the students' post-test scores improved after teaching for each chapter, the improvement was much greater for chapter 2 than for chapter 1, with mean scores of 39.97 and 32.45 out of 50, respectively (P<0.05). Despite such improvement, almost half of the students found that setting MCQs was not easy.
CONCLUSION: This study found that student-generated MCQs can be used as a simple and cost-effective tool to enhance TBL of anatomy.

Introduction

Human anatomy is the science of human structures and their relationships with one another. It is subdivided into gross anatomy, which deals with the structures and position of organs; histology, which is occupied with the microscopic appearance of cells and tissues; and embryology, which studies fetal development.1 Human anatomy is one of the cornerstones of medical curricula and is fundamental for safe clinical practice, especially in surgery.2–4

With time constraints and increasing numbers of medical students per class, in addition to the limited number of cadavers for dissection, anatomy teaching has undergone significant changes in the past three decades.1,5,6 Traditional teaching methodologies of anatomy basically comprised cadaver dissections, inspection of prosections or plastinations, and didactic teachings.1 With the increasing use of technology in academia, methods have shifted to the use of models in teaching, computer-based learning and teaching based on radiological images.1,2

While instructors of human anatomy have been overloaded by the large volume of anatomy study materials and time constraints, other teaching methodologies, such as team-based learning (TBL), have been tried, where students are encouraged to be active and take responsibility for learning.7 These teaching styles give an opportunity for students to interact with others and to apply conceptual knowledge through working in small groups.7,8 TBL also overcomes the problem of a shortage of anatomy lecturers in some universities.9

All of these teaching changes are intended to increase the knowledge of anatomy. However, no single method has been found to be best, and a multimodal approach is favored.2,5

Several other methodologies have been tried in medicine to cope with curricula and student satisfaction, one of which is student-generated multiple choice questions (MCQs). It is thought that if students are encouraged to make up their own MCQs while studying, this stimulates deeper understanding of the topic; and it has been found that the knowledge required to set a good MCQ is usually greater than that required to answer one.10

At the University of Rwanda, didactic methods were initially used to teach anatomy, but with an increasing number of medical students, this has shifted toward PowerPoint presentations, and since 2013, TBL has been introduced at the university as the teaching methodology, only for anatomy. No prior studies have been conducted at the university to address the impact of such changes, although they are satisfactory for both lecturers and students. As there is still much room for improvement using other interventions aimed at improving anatomy science delivery at the university, student-generated MCQs could be one of the simplest interventions, if applied correctly.

Methods

This study aimed to improve anatomy learning and knowledge retention at the University of Rwanda by determining the impact of student-generated MCQs on team-based anatomy learning.

A comparative interventional study was conducted by comparing TBL and TBL enhanced by student-generated MCQs (TBL+) as teaching methodologies for the anatomy course at the University of Rwanda.

The study recruited 31 final year medical students who were rotating in the surgical department at the University Teaching Hospital of Kigali (CHUK) from January 2019 to February 2019. During their clinical rotation, two chapters on surgically oriented anatomy were chosen among other teaching aids and were included in this study. These chapters were selected based on their having similar length and level of complexity in the standard anatomy textbooks. Both chapters were taught by the same lecturer but not at the same time; chapter 1 was taught first, and after its completion, chapter 2 followed.

During the first chapter (innervation of the upper limb), students used the current teaching methodology for anatomy learning at the University of Rwanda, which is team-based learning.

Description of Chapter 1 Teaching

Teaching on the chapter lasted for 1 week.

On the first day, the lecturer gave an introduction to the chapter and study materials to the students (textbooks, CDs).

A pre-test was given to the students to measure their baseline knowledge of the chapter.

Each group was formed by five or six students, chosen randomly.

Students were instructed to spend a minimum of 2 hours/day discussing the topic in their respective groups.

After 5 days, the lecturer conducted individual and team readiness assurance tests using MCQs, and guided the students in clinical problem solving.

The post-test assessment test was carried out on day 7 using a different set of MCQs.

The second chapter (innervations of the lower limb) was taught using the new teaching methodology (TBL+), where students were instructed to set their own MCQs while studying.

Description of Chapter 2 Teaching

Teaching on chapter 2 also lasted for 1 week. The same environment as for Chapter 1 was provided for learning.

The lecturer started by providing an introduction to the chapter and giving study materials to the students (textbooks, CDs).

Students were also taught about how to set high-quality MCQs and introduced to Bloom’s taxonomy of categorizing questions into different levels of complexity and specificity.

A pre-test was performed to measure the baseline knowledge of students on the chapter with the use of MCQs.

Students remained in their respective groups of five or six students.

A minimum of 2 hours/day was the requirement for each group to discuss the topic.

During this chapter, they were encouraged to set at least five MCQs/group and to discuss them while studying.

After 4days, the lecturer collected student-generated MCQs to assess their quality and to find out whether they met the study objectives. All well-set MCQs were given back to students on day 5 to discuss them in depth before the post-test assessment, which was carried out on day 7. On the same day (day 5), students were also assessed on individual and team readiness using MCQs set by the lecturer and guided in clinical problem solving.

Post-Study Survey

To understand students’ perceptions on the new teaching methodology, a survey questionnaire was developed and given to the students after the study. This questionnaire, based on a five-point Likert scale, was initially tested in four pilot students (about 12% of respondents) to address whether there was confusion regarding the items and whether the respondents understood the questions in the same way as the questionnaire developer.

Data Collection and Analysis

The study included the following variables: student demographics, pre- and post-test scores for the innervations of the upper limb, and pre- and post-test scores for the innervations of the lower limb. SPSS software version 23 was used for data analysis. The mean and standard deviation were basic statistics calculated for individual students’ scores and were also used to report demographic data. The Student’s paired t-test was employed to determine and compare the mean scores obtained after the two chapter assessments. Post-study evaluation consisted of a SurveyMonkey online questionnaire of prepared and tested questions based on a Likert scale ranging from 1 to 5 points.

Results

The study recruited 31 medical students. Their age ranged from 23 to 30 years, with a mean±SD age of 25±1.599 years. There were 22 males (71%) and nine females (29%). Although they were all final year medical students, some were on level 5 (15 students) while 16 were on level 6 of training.

The pre-test minimum score for the upper limb was 17/50 and the maximum was 30/50, with a mean±SD score of 25.10±3.18. The minimal pre-test score for the lower limb was 16/50 and the maximum was 32/50, with a mean score of 25.19±3.73, and there was no statistically significant difference between the mean scores obtained (P=0.885) (Figure 1).

Figure 1

Students’ pre-test scores for upper and lower limb nerve supply.

The independent sample t-test was used to determine differences in mean score between level 5 and level 6 students’ pre-test scores for both chapters. Although the students were at different levels of training, their baseline scores were not statistically significantly different (Table 1).

Table 1

Pre-Test Scores Variation Depending on the Level of Training of Students

Pre-Test Scores	Level of Training	Number (%)	Mean Score/50	SD	P value	Mean Difference	95% CI of the Difference
							Lower	Upper
Chapter 1	Level 5	15(48.38)	24.4	2.613	0.245	−1.350	−3.677	0.977
	Level 6	16(51.62)	25.75	3.606
Chapter 2	Level 5	15(48.38)	25.13	3.441	0.933	−0.117	−2.910	2.676
	Level 6	16(51.62)	25.25	4.107

Post-test scores for both chapters were compared with pre-test scores using the Student’s t-test. Overall, there was an increase in students’ scores after the tests for both chapters (Table 2). However, students scored better on the post-test for chapter 2 than for chapter 1, with a mean difference of 7.515 (P<0.05) (Figure 2).

Table 2

Paired t-Test for Chapter 1 and Chapter 2 Comparing the Mean Pre-Test and Post-Test Scores

Chapter Scores	Mean	N	SD	Paired Differences					t	P value
				Mean Diff.	SD	SEM	95% CI of the Difference
							Lower	Upper
Pre-test (Chapter 1)	25.1	31	3.187	−7.355	4.378	0.786	−8.961	−5.749	−9.353	0
Post-test (Chapter 1)	32.45	31	4.675
Pre-test (Chapter 2)	25.19	31	3.736	−14.77	5.714	1.026	−16.87	−12.67	−14.39	0
Post-test (Chapter 2)	39.97	31	5.05

Figure 2

Post-test scores for chapter 1 and chapter 2.

During the second chapter, the students set 35 MCQs. The questions were corrected and analyzed by the lecturer to appraise their quality, by assessing mainly:

Whether the set MCQs covered the study objectives

Whether they were written in clear and understandable language

The way in which the proposed answer and distractors were constructed.

Quality of MCQs Generated by Students During Chapter 2

Of 35 MCQs, six (18%) were found to be poor-quality questions based on the above criteria and the remaining 29 were further assessed for their complexity based on different domains of Bloom’s taxonomy. Among good MCQs, 62.5% demonstrated adequate medical knowledge by recalling the anatomical concepts, while only the remaining 37.5% of questions demonstrated a higher level of thinking (Table 3).

Table 3

Categorization of Student-Generated MCQs in Different Domains of Bloom’s Taxonomy

Bloom’s Categorization of MCQs Generated by Students	% of Questions
Knowledge (questions that are only based on recognition or recall of memorized information)	62.5
Comprehension/Application/Analysis/Synthesis/Evaluation (questions that demonstrate a higher level thinking and which make important goals for education)	37.5

Post-Study Survey Results

All 31 students answered the post-study survey, which was prepared based on a five-point Likert scale. The developed questionnaire comprised eight questions which were tested in a pilot study with four students randomly selected from the study participants. These pilot students could clearly grasp the meaning of five out of eight questions on the survey. The other two questions were either difficult or meaningless for them to answer and therefore were removed from the survey questions. The remaining question on the survey asked about the level of difficulty in setting MCQs; of 31 students, 14 (45%) found that it was not easy to set MCQs, while it was neither easy nor difficult for 5 students (16%). However, only 12 students (39%) found that it was very easy to set MCQs

The rest of the survey questions and results are presented in Figure 3; almost 70% of students found self-generated MCQs very helpful in their learning during the second chapter.

Figure 3

Students’ perceptions on the new teaching methodology.

Discussion

This interventional study was the first one conducted at the University of Rwanda to test the impact of student-generated MCQs on the current anatomy teaching methodology (team-based learning).

This study recruited medical students in their final year of training. They had prior exposure to the anatomy course in their first year of medical school. Ideally, the best potential candidates for the study would have been first year medical students without prior knowledge of anatomy. However, we selected two anatomy topics which are more surgically oriented than gross anatomy and which the students had not learnt in depth during the first year of medical school. Unlike other studies on the impact of MCQs on learning, this is the only study to have evaluated the impact of student-generated MCQs on TBL of an anatomy course.

Previous studies on the impact of MCQs on student learning did not mention the prior knowledge of students on the study materials to which they were exposed.11,12 The fact that the medical students recruited into this study were not at the same level of training (level 6 and level 5) did not result in selection bias as they all had similar baseline anatomy knowledge, with mean score differences of 1.35 (P=0.245) and 0.117 (P=0.933) for the first and the second chapters, respectively.

Students’ scores on two different topics are usually expected to differ based on various factors such as differences in complexity between the topics, length, environment and available time for studying. These confounders were controlled by selecting two similar topics in the standard anatomy textbooks, and pre-tests for both chapters were undertaken before the start of teaching, which showed almost the same level of baseline knowledge (Figure 1), with student pre-test mean scores of 25.10/50 and 25.19/50 for chapters 1 and 2, respectively. However, the fact that the learning should have improved over time after the first chapter was difficult to control, and the decision to start the experiment with the chapter on upper limb innervation rather than starting on the lower limb was not based on any other criteria, but was also found to be a confounder.

In addition to TBL, students were requested to set MCQs while studying chapter 2. Their mean scores improved from 25.19 to 39.93 for pre-test and post-test, respectively (P<0.05). Compared with chapter 1, there was a great improvement in student mean score at the end of the chapter assessments, from 32.45 to 39.93 for control and intervention arms, respectively. Analysis with the independent t-test showed that the difference in scores was statistically significant (P<0.05). These results are comparable to those found by Shakurnia et al in their study which tested the effect of MCQ generation on midwifery students’ learning on an immunology course, where there was a significant increase in post-test score in the group that used MCQs as a learning tool compared with the control group.12 Unlike that study, where the students did not have prior knowledge of immunology, our students had prior knowledge of anatomy from their first year of medical school, which can make interpretation of the results difficult.

The improvement in student scores after chapter 2 should be viewed from different angles. The fact that chapter 2 was taught after completion of chapter 1 should have led to improved learning skills of the students over time and therefore improved post-test scores. A crossover design would have been suitable for this study or, if possible, students should have been randomized into two categories where one group could use either method. The improved student score should have also resulted in chapters which were different; although the students demonstrated the same baseline medical knowledge based on the pre-test score, the chapters were not identical and students would probably have had fewer difficulties in studying chapter 2 than chapter 1.

During the second chapter, students were asked to set MCQs while studying. The quality of MCQs set was analyzed using Bloom’s taxonomy and classified according to its cognitive domains for questions which test knowledge, comprehension, application, analysis, synthesis and evaluation.13

Of the good questions set by the students, 62.5% were testing knowledge, as students were only recalling the memorized anatomical concepts. However, only 37.5% were categorized into the higher cognitive domain. These findings are similar to those found by Palmer et al, who evaluated the impact and quality of student-constructed MCQs on learning clinical surgery at the University of Adelaide, where students were able to produce only 25% of MCQs testing higher cognitive skills, while many questions tested knowledge and comprehension.10 The students were unfamiliar with the task of setting MCQs, and did not have sufficient time to develop the skills necessary for setting MCQs.

Several other studies have highlighted the benefits of MCQs for learning in general, which include active learning and development of higher thinking skills in problem solving.10,11 As shown in our study, the adjunctive effect of student-generated MCQs on current TBL of the anatomy course was positive based on student performance scores and appreciation after the study, but as this was the first study conducted on anatomy teaching, randomized studies are needed.

Setting MCQs is an exercise that requires adequate knowledge of the subject and higher thinking skills. However, none of the studies that proved the benefits of student-constructed MCQs as a stimulus to learning evaluated the time required to accomplish this task. As anatomy constitutes a large module in the preclinical years of medical school, students may be overloaded by the extra work involved in generating MCQs on top of the preexisting huge amount of anatomy study material, and this needs to be addressed in further studies.

In contrast to our findings, the study by Grainger et al recognized the benefits of student-generated MCQs on improvement of learning, but the students perceived negatively the task of setting questions, mainly because of inadequate question-writing skills and the extra work requiring a high cognitive load for them to generate high-quality MCQs.14

Study Limitations

These chapters in the standard anatomical textbooks were very short; usually, TBL at the University of Rwanda is carried out by assessing students after they have studied a large component of the course, such as anatomy of the head, neck, trunk or upper limb. Therefore, students could easily memorize the chapters used in this study, which could potentially impact the post-test scores.

The fact that the scores obtained would not have an impact on students’ continuous assessment could have led to some students not to making an effort while studying.

The students’ perceptions on the new learning methodology were based on a five-point Likert scale, which did not allow them to express all of their thoughts.

The small sample size was also a limitation of this study; we recommend that further studies recruit an entire class instead of a small group and use a two-arm randomization process for better understanding of the impact of student-generated MCQs on TBL of anatomy.

Conclusion

The results of this study show that there was much improvement in student performance using student-generated questions as a stimulus to learning, and therefore this is a simple and cost-effective tool to enhance TBL of anatomy at the University of Rwanda. However, students need to be familiarized with the principles of setting good and formative MCQs.