Hamid Mahmood1. 1. Department of Ophthalmology, Services Institute of Medical Sciences (SIMS), Services Hospital (SHL), Lahore.
Abstract
OBJECTIVE: To determine the psychometric worth of Multiple-Choice Questions (MCQs) and Short-Essay Questions (SEQs) used in internal summative assessment of undergraduate ophthalmology annual send-up examination and to ascertain the quality of items using subjective measures. STUDY DESIGN: Correlational analytical study. PLACE AND DURATION OF STUDY: Fatima Jinnah Medical College, Lahore, from January to March 2012. METHODOLOGY: Fourth year MBBS students appearing in their written send-up annual examination in ophthalmology. The MCQ and SEQ scores were correlated for validity and reliability. Construct validity was estimated (Pearson correlation coefficient). MCQ's split-half reliability (Spearman correlation coefficient) and SEQ's inter-rater reliability (Pearson correlation coefficient) were also estimated. RESULTS: Moderate correlation was seen between MCQ and SEQ scores (r=0.5, p < 0.01). Split-half reliability of MCQs had moderate correlation (r=0.4, p < 0.01) while inter-rater reliability of SEQs scores showed high correlation (r=0.9, p < 0.01). Most of the MCQs (55%) and SEQ's (90%) were at recall level of Bloom's taxonomy. MCQs (30%) attained a higher level (interpretation) as compared to SEQ's (10%). None of the MCQ and SEQ attained problem solving level. CONCLUSION: Using variety of tools for assessment improves both the validity and reliability enabling examiners to draw fair conclusions about a student's ability. Validity and reliability can be further improved by arranging training opportunities for item writers to achieve accurate transfer of hypothetical construct in the form of an item. This is crucial since essential and important learning outcomes can only be assessed with valid and reliable tools.
OBJECTIVE: To determine the psychometric worth of Multiple-Choice Questions (MCQs) and Short-Essay Questions (SEQs) used in internal summative assessment of undergraduate ophthalmology annual send-up examination and to ascertain the quality of items using subjective measures. STUDY DESIGN: Correlational analytical study. PLACE AND DURATION OF STUDY: Fatima Jinnah Medical College, Lahore, from January to March 2012. METHODOLOGY: Fourth year MBBS students appearing in their written send-up annual examination in ophthalmology. The MCQ and SEQ scores were correlated for validity and reliability. Construct validity was estimated (Pearson correlation coefficient). MCQ's split-half reliability (Spearman correlation coefficient) and SEQ's inter-rater reliability (Pearson correlation coefficient) were also estimated. RESULTS: Moderate correlation was seen between MCQ and SEQ scores (r=0.5, p < 0.01). Split-half reliability of MCQs had moderate correlation (r=0.4, p < 0.01) while inter-rater reliability of SEQs scores showed high correlation (r=0.9, p < 0.01). Most of the MCQs (55%) and SEQ's (90%) were at recall level of Bloom's taxonomy. MCQs (30%) attained a higher level (interpretation) as compared to SEQ's (10%). None of the MCQ and SEQ attained problem solving level. CONCLUSION: Using variety of tools for assessment improves both the validity and reliability enabling examiners to draw fair conclusions about a student's ability. Validity and reliability can be further improved by arranging training opportunities for item writers to achieve accurate transfer of hypothetical construct in the form of an item. This is crucial since essential and important learning outcomes can only be assessed with valid and reliable tools.