Ahmed Hesham Sobh1, Zubin Austin2, Mohamed Izham M I3, Mohammad I Diab4, Kyle John Wilby5. 1. College of Pharmacy, Qatar University, PO Box 2713, Doha, Qatar. Electronic address: as1304797@qu.edu.qa. 2. Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario, Canada. Electronic address: zubin.austin@utoronto.ca. 3. College of Pharmacy, Qatar University, PO Box 2713, Doha, Qatar. Electronic address: mohamedizham@qu.edu.qa. 4. College of Pharmacy, Qatar University, PO Box 2713, Doha, Qatar. Electronic address: Mohamed.diab@qu.edu.qa. 5. College of Pharmacy, Qatar University, PO Box 2713, Doha, Qatar. Electronic address: kjw@qu.edu.qa.
Abstract
BACKGROUND AND PURPOSE: Objective structured clinical examinations (OSCEs) are considered gold standard performance-based assessments yet comprehensive evaluation data is currently lacking. The objective of this study was to critically evaluate the psychometric properties of a cumulative OSCE for graduating pharmacy students in Qatar for which policies and procedures were adapted from a Canadian context. EDUCATIONAL ACTIVITY AND SETTING: A 10-station OSCE was conducted for graduating students in Qatar. Evaluation included assessment of pass rates, predictive validity, concurrent validity, internal validity, content validity, interrater reliability, and internal consistency. FINDINGS: Twenty-six students completed the OSCE. Three stations achieved pass rates < 80%. Scores from professional skills and case-based learning courses, formative OSCEs, and cumulative grade point averages correlated with OSCE scores (p < 0.05). Average correlation between assessors' analytical and global scoring was moderate (r = 0.52). Average interrater reliability was excellent for analytical scoring (ICC = 0.88) and moderate for global scoring (ICC = 0.61). Excellent internal consistency was demonstrated for overall performance (α = 0.927). Students generally agreed stations represented real practice scenarios (range per station, 30-100%). DISCUSSION AND SUMMARY: The evaluation model identified strengths and weaknesses in assessment and curricular considerations. The OSCE demonstrated acceptable validity and reliability as an adapted assessment.
BACKGROUND AND PURPOSE: Objective structured clinical examinations (OSCEs) are considered gold standard performance-based assessments yet comprehensive evaluation data is currently lacking. The objective of this study was to critically evaluate the psychometric properties of a cumulative OSCE for graduating pharmacy students in Qatar for which policies and procedures were adapted from a Canadian context. EDUCATIONAL ACTIVITY AND SETTING: A 10-station OSCE was conducted for graduating students in Qatar. Evaluation included assessment of pass rates, predictive validity, concurrent validity, internal validity, content validity, interrater reliability, and internal consistency. FINDINGS: Twenty-six students completed the OSCE. Three stations achieved pass rates < 80%. Scores from professional skills and case-based learning courses, formative OSCEs, and cumulative grade point averages correlated with OSCE scores (p < 0.05). Average correlation between assessors' analytical and global scoring was moderate (r = 0.52). Average interrater reliability was excellent for analytical scoring (ICC = 0.88) and moderate for global scoring (ICC = 0.61). Excellent internal consistency was demonstrated for overall performance (α = 0.927). Students generally agreed stations represented real practice scenarios (range per station, 30-100%). DISCUSSION AND SUMMARY: The evaluation model identified strengths and weaknesses in assessment and curricular considerations. The OSCE demonstrated acceptable validity and reliability as an adapted assessment.
Authors: Alberto Bellido-Esteban; Ana Isabel Beltrán-Velasco; Pablo Ruisoto-Palomera; Pantelis T Nikolaidis; Beat Knechtle; Vicente Javier Clemente-Suárez Journal: Front Psychol Date: 2021-03-24