Multiple-choice questions in assessment: Perceptions of medical students from low-resource setting.

BACKGROUND: In curriculum reforms under Competency-Based Medical Education (CBME) introduced by the National Medical Commission, medical students are recognized as one of the stake holders. The well-designed multiple-choice questions (MCQs) help students in assessment as well as promote deep learning. The objective of this study is to assess the perceptions of the Indian medical students towards the examination patterns in their undergraduate curriculum with particular reference to MCQs.
MATERIALS AND METHODS: An observational cross-sectional study was conducted among the medical colleges in India. A total of 7875 students took part in the study. The study instrument was a validated pretested self-administered questionnaire. The study was conducted as an online survey using the Google Forms platform. Inclusion criteria were students from all professional years as well as interns and those who were willing to participate in the study.
RESULTS: For the question of initiation of MCQs as a tool for the assessment, many of the students (46%) expressed that it should start from the 1st MBBS itself. The majority of the students (85%) felt that MCQ oriented preparation will be helpful for MBBS professional (theory and practical) examinations. Most of the students (90.9%) preferred that the MCQs should be concept and case scenario based. To the question of whether the current system of the university theory examinations of essays and short notes be replaced in total with MCQs so that they are more prepared for NEXT and USMLE examination, 65.5% responded with an answer of yes.
CONCLUSION: The findings of this study provide input to the regulatory authorities and other stakeholders responsible for designing and implementing the medical curriculum. Copyright:

Introduction

The National Medical Commission (NMC) has been constituted by an act of Parliament known as the National Medical Commission Act, 2019, which has come into force in place of the Medical Council of India with the aim to improve access to quality and affordable medical education.[1] Currently, the doctor to the population ratio in India is about 0.77: 1000 as per the current population estimate of 1.33 billion against the WHO norms of 1 doctor for 1000 population.[2] There are more than 500 medical colleges in the country, and on average 82,950 students graduate from the medical schools annually.[3]

However, the skill set of these students has been questioned and to set this right, the erstwhile Board of governors in supersession of the MCI has introduced the Competency-Based Medical Education (CBME) from the year 2019. The CBME focuses on the development of competencies designed to train the Indian medical graduate to be a physician of the first contact within the health system.

The Miller's Pyramid depicts the various levels of acquiring clinical competencies starting with the knowledge to actually acquiring the skill. Hence, the foundation of a good knowledge base cannot be overemphasized. The assessment of knowledge is by means of various methods such as essays, short answers, and very short answers, and multiple choice questions (MCQs). The Bloom's taxonomy provides the taxonomic levels from knowledge to synthesis in increasing levels of complexity. However, there has been a criticism that the assessment of the students knowledge is usually at the level of fact gathering. The testing of recall or comprehension is through essays and short essays. The higher levels of knowledge acquisition, i.e., interpretation and application can be assessed by methods such as case presentations, essays, and extended matching type MCQs. The MCQs especially well-designed help students to be assessed for application which in turn promoted deep learning. Hence, MCQs have become an important assessment method used in the screening tests for medical students to get admission to postgraduate studies.

In curriculum reforms under CBME, the medical students are recognized as one of the important stakeholders. However, there is limited literature about the perceptions of medical students about the examination pattern and the use of MCQs as a method of assessment. The aim of the study is to promote the use of MCQs in the assessment of students at the undergraduate level. The objective of this study is to assess the perceptions of Indian medical students towards the examination patterns in their undergraduate curriculum with special reference to MCQs.

Materials and Methods

Study design and setting

An observational cross-sectional study was conducted among the medical colleges in India. The study participants were undergraduate medical students. The study was conducted with data collection in November 2020. Inclusion criteria were students from 1st, 2nd and 3rd MBBS Part I and Part II as well as interns and those who were willing to participate in the study.

Study participants and sampling

Exclusion criteria were students who have completed an internship or those who were transferred in from medical school from outside India. The minimal sample size obtained is 5644 using formula 4pq/l2 (taking proportion [p] of students who preferred MCQs [83%] from the pilot study conducted on 1234 students, absolute error [l] of 1%, at 5% alpha error, 80% power). The sampling technique used was convenience sampling. A total of 7875 students took part in the study. The study instrument was a validated pretested self-administered questionnaire. The study variables included background information such as name and year of study, type of institutes such as government or private, gender, state, and perception about the current examination pattern. Institutional ethics committee clearance was obtained for conducting the study. The confidentiality of the participants was maintained.

Data collection tool and technique

The study was conducted as an online survey using the Google platform. The questionnaire was designed as a Google Form and sent as an online survey form across all the colleges across the country. Two student research organizations, namely, Andhra Medical College Undergraduate Research Association (AURA) and Global Association of Indian Medical Students (GAIMS) supported the data collection. One of the authors contacted the student executives of GAIMS of various colleges and they in turn, disseminated it among their students. A video was made discussing the objectives of the study to give a clear idea to the volunteers.

Statistical analysis

Responses were received to Google account and collected as MS Excel sheet. Data cleaning was conducted to remove double entries, incomplete responses, and to exclude responses from ineligible participants. Statistical analysis was performed using the Statistical Package for the Social Sciences (SPSS) version 24.0 (SPSS, Inc., Chicago, IL, USA). Tests of significance used were Chi-square test to determine the role of sociodemographic characteristics on relevant questions. The statistical significance level was set at P < 0.05 (two-sided) with a confidence interval of 95%.

Results

A total of 7875 responses were obtained from 200 colleges. After data cleaning, 7416 final responses were analyzed. Among the 7416 students, 51.8% were from government medical colleges and 48.2% were from private medical colleges.

The mean age of the students in the study was 20.7 + 1.49 years. About 51% were in the age group of 21–24 years, 48.2% were in the age group of < 20 years, and 1.8% were above 25 years of age. The majority of the study participants were females (63.3%) and 36.7% were males. About 31.5% of students belonged to 1st MBBS, 25.5% from 2nd MBBS, and 40.4% from 3rd MBBS Part I and II. Interns accounted for 2.6% of the study participants. There were more girls than boys in all the groups.

Regarding the perception on whether they should start practicing MCQs, the majority of students responded in the affirmative. A higher proportion of students from senior classes responded that they should start preparing MCQs to appear for competitive examinations such as National Exit Test (NEXT), National Eligibility cum Entrance Test (NEET) and United States Medical Licentiate Examination (USMLE) as compared to students from MBBS first and second years. This difference was found to be statistically significant [Table 1a].

Table 1a

Association of student’s responses on various questions with respect to their current academic year

Questions	Current academic year					Total
	1st MBBS (%)	2nd MBBS (%)	3rd MBBS part 1 (%)	3rd MBBS part 2 (%)	Internship (%)
Q1
No	240 (10.3)	172 (9.1)	122 (7)	103 (8.2)	8 (4.1)	645 (8.7)
Yes	2095 (89.7)	1717 (90.9)	162 (93)	1148 (91.8)	185 (95.9)	6771 (91.3)
P			0.001
Q2
1st MBBS	1407 (63)	725 (38.4)	676 (38.7)	479 (38.3)	63 (32.6)	3413 (46)
2nd MBBS	675 (28.9)	809 (42.8)	495 (28.3)	328 (26.2)	38 (19.7)	2345 (31.6)
3rd MBBS part 1	160 (6.9)	302 (16)	460 (26.3)	248 (19.8)	40 (20.7)	1210 (16.3)
3rd MBBS part 2	14 (0.6)	18 (1)	58 (3.3)	120 (9.6)	12 (6.2)	222 (3)
Internship	16 (0.7)	35 (1.9)	59 (3.4)	76 (6.1)	40 (20.7)	226 (3)
P			0.001
Q3
No	263 (11.3)	282 (14.9)	312 (17.8)	223 (17.8)	34 (17.6)	1114 (15)
Yes	2072 (88.7)	1607 (85.1)	1436 (82.2)	1028 (82.2)	159 (82.4)	6302 (85)
P			0.001			7416
Q4
250-500	673 (28.8)	503 (26.6)	424 (24.3)	324 (25.9)	54 (28)	1978 (26.7)
500-1000	507 (21.7)	453 (24)	493 (28.2)	303 (24.2)	50 (25.9)	1806 (24.4)
>1000	399 (17.1)	377 (20)	353 (20.2)	241 (19.3)	35 (18.1)	1405 (18.9)
Don’t know	756 (32.4)	556 (29.4)	478 (7.3)	383 (30.6)	54 (28)	2227 (30)
P			0.001

To the question of the timing of initiation of MCQs, the majority of students (46%) expressed that it should start from 1st MBBS itself. Only 6% stated that the preparation may start in 3rd MBBS part II or internship. The difference in perception among the different MBBS years was found to be statistically significant [Table 1a] [Figure 1]

Figure 1

From which year they should start preparing for multiple-choice questions?

The majority of the students (85%) felt that MCQ-oriented preparation will be helpful for MBBS professional (theory and practical) examinations. It was observed to be around 88% among 1st MBBS and declined slightly in higher class students with 85% among 2nd MBBS students and 82% among 3rd MBBS students and interns. This difference was found to be statistically significant [Table 1a] [Figure 2].

Figure 2

Distribution of study participants who felt multiple-choice questions-oriented preparation useful for MBBS Professional examinations

Regarding the number of MCQs which students should practice, after learning from the textbook, 30% responded that they were not sure, 24% opined that 500–1000 would be sufficient and 27% felt that 250–500 questions would be sufficient. Another 18% opined that 1000 MCQs need to be practiced. There were variations among the perceptions of students from the various MBBS years, and this difference among the different MBBS years was statistically significant [Table 1a] [Figure 3]

Figure 3

Distribution of study participants based on number of multiple-choice questions to be prepared after reading the subject

To the question of whether MCQs are more concept-based or memory-based in USMLE as compared to NEET PG, 73.9% of students responded that they are concept based and another 24% responded that they do not know the difference. It was observed that 1st MBBS students (30.9%) responded that they do not know the difference as compared to the rest of the students. This difference was found to be statistically significant [Table 1b].

Table 1b

Association of student’s responses on various questions with respect to their current academic year

Questions	Current academic year					Total
	1st MBBS (%)	2nd MBBS (%)	3rd MBBS part 1 (%)	3rd MBBS part 2 (%)	Internship (%)
Q5
Concept and case based	1550 (66.4)	1425 (75.4)	1385 (79.2)	966 (77.2)	152 (78.8)	5478 (73.9)
Don’t know	722 (30.9)	420 (22.2)	330 (18.9)	265 (21.2)	40 (20.7)	1777 (24)
Memory based	63 (2.7)	44 (2.3)	33 (1.9)	20 (1.6)	1 (0.5)	161 (2.2)
P			0.001
Q6
Concept and case based	2135 (91.4)	1699 (89.9)	1594 (91.2)	1135 (90.7)	178 (92.2)	6741 (90.9)
Memory based	200 (8.6)	190 (10.1)	154 (8.8)	116 (9.3)	15 (7.8)	675 (9.1)
P		0.472
Q7
50 percent	824 (35.3)	535 (28.3)	341 (19.5)	280 (22.40	31 (16.1)	2011 (27.1)
60 percent	573 (24.5)	505 (26.7)	446 (25.5)	315 (25.2)	53 (27.5)	1892 (25.5)
70 percent	566 (24.2)	499 (26.4)	532 (30.4)	404 (32.3)	68 (35.2)	2069 (27.9)
80 percent	372 (15.9)	350 (18.5)	429 (24.5)	252 (20.1)	41 (21.2)	1444 (19.5)
P			0.001
Q8
No	738 (31.6)	652 (34.5)	546 (31.2)	515 (41.2)	104 (53.9)	2555 (34.5)
Yes	1597 (68.4)	1237 (65.5)	1202 (68.8)	736 (58.8)	89 (46.1)	4861 (65.5)
P			0.001

Q1. Should you also practice MCQ’s for competitive exams such as NEXT, NEET PG, USMLE? Q2. If yes for above question, from which year should a student should start solving MCQ? Q3. MCQ oriented preparation will be helpful for MBBS professional (theory and practical) examinations? Q4. How many numbers of MCQ’s should a student prepare after reading text book for each subject? (based on the current academic year) Q5. MCQ for NEET PG and USMLE are slightly different, what do you think USMLE focuses on? Q6. Which type of MCQs do you like? Q7. How many questions should be from clinical subjects in NEXT? Q8. MBBS professional theory examination are for 14 subjects in total. Do you prefer a change to 100% MCQ computer-based examination instead of the current essay and short answer, so that you are more prepared for NEXT and USMLE examination? MCQ’s=Multiple choice questions, NEXT=National exit test, USMLE=United States medical licentiate examination, NEET-PG=National eligibility cum entrance test postgraduate

Most of the students (90.9%) preferred that the MCQs should be concept and case scenario based. The preference for concept-based MCQs was 91.4%, 89.9%, 91.2%, 90.7%, and 92.2% among 1st MBBS, 2nd MBBS, 3rd MBBS part 1, 3rd MBBS part II, and interns, respectively. The differences among the students of various years were not statistically significant [Table 1b] [Figure 4].

Figure 4

Multiple-choice questions preference based on case-based, concept-based, and memory-based

In the present study, 27.90%, 25.5% and 27.9% of the students opined that in the NEXT examination, MCQs from clinical participants should be 50%, 60% and 70% of total marks respectively. About 19.5% of students stated that clinical participants should be allotted 80% of marks. More 1st MBBS students (35.3%) preferred lesser allotment (50%) of MCQs from clinical participants as compared to the students from higher years. The differences between the opinions of the students of various MBBS years were found to be statistically significant [Table 1b] as 1st-year students will not have adequate exposure to the clinical participants [Figure 5].

Figure 5

Perception of students regarding allotment of multiple-choice questions on clinical participants in NEXT exam

To the question about whether the current system of university theory examinations of essays and short notes be replaced in total with MCQs so that they are more prepared for NEXT and USMLE examinations, 65.5% responded with a “yes” and 34.5% said “no.” Lesser number of students in internship (46.1%) and 3rd MBBS part II (58.8%) opined that MCQs should replace the current method of theory examination as compared to 1st MBBS students (68.4%) and this difference was statistically significant [Table 1b].

Discussion

In the present study, the majority of students (85%) opined that MCQ-oriented preparation is useful for MBBS examinations. This is in contrast to a study among preclinical medical students by Bird et al.,[4] which revealed that 54% of students reported preferring a Context Rich Short Answer Questions (CR-SAQ) examination format over MCQs format.

The majority of the students in the present study expressed that MCQs should be case scenarios or concept based which tests higher cognitive learning. Moeen-uz-Zafar Khan and Badr Aljarallah[5] concluded in their study that a well-constructed MCQ is superior to Modified Essay Questions in testing higher cognitive skills.

With regard to the present examination pattern in the University Examinations, about 65.5% of students preferred a change to 100% MCQ computer-based examination instead of the current essay and short answer so that they are more prepared for NEXT and USMLE examination. Hift[6] debated that MCQ format is better than open-ended question format and suggested phasing out of open-ended question format in summative assessment.

The majority of the students are of the opinion that MCQ preparation should be from the first MBBS. A few studies have also been conducted where writing MCQs was given as a task to the students themselves and study its effect on the performance in exams. The studies reported that such activities had a positive influence on learning behavior in terms of developing deep learning (Grainger study[7] ).

Herroro et al.[8] concluded that MCQs may be a valuable tool for medical education as a learning tool. Bottemly et al.[9] observed in their study that involving medical students in preparing MCQs that the students are eager participants and produced a large repository of relevant, good quality MCQs. There is also a widely accepted fact that formulating good quality MCQs is a demanding assignment. Chandre et al.[10] in their study emphasized the need for faculty development programs on the creation of MCQs. Mitayani Purwoko et al.[11] in a study observed that 61.2% of MCQs were of recall method and 50% of questions are to be revised. They recommended a faculty development program. Scully et al.[12] in their paper reiterated that MCQs have the capacity to assess certain higher-order skills and also provided practical recommendations for test developers seeking to purposefully construct such items.

Limitation and recommendation

The study was a cross-sectional study done among medical students to assess perceptions about examination pattern only. Students were not assessed by means of MCQs nor was there any comparison between the students’ performance on different assessment tools.

The findings of this study may provide inputs to the regulatory authorities and other stakeholders instrumental in designing and implementing the medical curriculum. The use of MCQs in assessment may be used as a learning tool as well as an assessment method. This may be more widely used in undergraduate medical training.

Conclusion

The study participants included students from across the country. The majority of the students had a positive opinion about MCQs and that MCQs should be used from 1st year. Students preferred MCQs which assessed higher-order cognitive skills. They opined those clinical participants should get more weight-age in assessment.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.