An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives.

Background: The objective structured clinical examination (OSCE) is the gold standard and universal format to assess the clinical competence of medical students in a comprehensive, reliable and valid manner. The clinical competence is assessed by a team of many examiners on various stations of the examination. Therefore, it is found to be a more complex, resource- and time-intensive assessment exercise compared to the traditional examinations. Purpose: The objective of this study was to determine the final year MBBS students' and OSCE examiners' perception on the attributes, quality, validity, reliability and organization of the Medicine and Therapeutics exit OSCE held at the University of the West Indies (Cave Hill) in June 2017.
Methods: At the end of the OSCE, students and examiners were provided with a questionnaire to obtain their views and comments about the OSCE. Due to the ordinal level of data produced by the Likert scale survey, statistical analysis was performed using the median, IQR and chi-square.
Results: A total of 52 students and 22 examiners completed the questionnaire. The majority of the students provided positive views regarding the attributes (eg, fairness, administration, structure, sequence, and coverage of knowledge/clinical skills), quality (eg, awareness, instructions, tasks, and sequence of stations), validity and reliability (eg, true measure of essential clinical skills, standardized, practical and useful experiences), and organization (eg, orientation, timetable, announcements and quality of examination rooms) of the OSCE. Similarly, majority of the examiners expressed their satisfaction with organization, administration and process of OSCE. However, students expressed certain concerns such as stressful environment and difficulty level of OSCE.
Conclusion: Overall, the OSCE was perceived very positively and welcomed by both the students and examiners. The concerns and challenges regarding OSCE can be overcome through better orientation of the faculty and preparation of the students for the OSCE.

Introduction

Several methods of assessing the clinical competence of medical students exist. Traditional methods include short case and long cases and the viva voce examination, all of which have been criticized for lacking structure and standardization, having poor inter-rater reliability, and not minimizing examiner bias.1 Harden et al2 proposed the objective structured clinical examination (OSCE) in medical school as a means of overcoming these issues and improving the quality of clinical performance of the students. He described an OSCE as “a timed examination in which medical students interact with a series of simulated patients in stations that may involve historytaking, physical examination, counselling or patient management.”2 The OSCE with multiple standardized cases and scoring rubrics is now widely used as the gold standard and universal format to assess the clinical competency of medical students.3 Over the past few decades it has been proven to be a valid and reliable tool which can assess all three learning domains (cognitive, affective, and psychomotor)4–6 and the “shows how” level described in the Miller pyramid.7 It has been widely adopted as the examination for clinical competence all over the world in various medical disciplines in undergraduate, postgraduate and licensing examinations.4 However, it has been found to be a complex, resource- and time-intensive assessment exercise.7,8

The University of the West Indies (UWI), a regional indigenous university with campuses and medical faculties each with its own Dean in Barbados, Jamaica, Trinidad, and additionally a clinical site in the Bahamas, has a total enrollment of approximately 3,000 medical students in a 5-year MBBS degree. The degree is divided into the mainly preclinical phase 1 (years 1–3) and clinical phase 2 (years 4–5). There is an integrated modular curriculum with continuous in-course and final summative assessments. On the completion of the final (fifth) year of the MBBS, students have a final exit examination in the three major disciplines of clinical medicine—Medicine and Therapeutics which includes internal medicine, pediatrics, psychiatry and family medicine, obstetrics and gynecology, and surgery. Students passing this examination are eligible to be provisionally licensed as interns in most English-speaking Caribbean countries.

The final year MBBS exit exam for Medicine and Therapeutics has 2 written papers (both of which are a mixture of single best answer and extended matching questions) and an OSCE. It is held twice a year, in May/June (when the majority of the students sit) and November/December. For 52 years following the inception of UWI in 1948, the final MBBS clinical examination in Medicine and Therapeutics was carried out using traditional assessment methods.9 In 2000, an OSCE was introduced to examine clinical competence.9 Although, OSCEs are also used in the individual clerkship examinations of the specialties that are included in the final Medicine and Therapeutics exit examination (internal medicine, pediatrics, psychiatry and family medicine), these clerkship examinations in Barbados have fewer than eight different testing stations, under 15 candidates and local examiners. In contrast the Final MBBS Medicine and Therapeutics OSCE is about 4 hours duration, has 17 testing stations not including four to six rest stations, and involves 17 examiners many of whom are from other UWI campuses. In Barbados students are assigned to one of two concurrent circuits each with its own pool of examiners. Each circuit examines over 20 students. This same process is repeated sequentially at all four campuses of the UWI with total number of students sitting this exam exceeding 600 in any given examination year. There are two external examiners (usually an internist and a pediatrician) who move from campus to campus and observe these examination processes to ensure adequate standard and uniformity.

Above all, the process of maintaining harmonization and uniformity in terms of the testing material and methods employed at all of the four sites, each located in independent island nations, requires careful attention and organization. As a result, the process was found to be very stressful and tiring for both the students and the examiners (most of the examiners move from one campus to another). It is a common realization that even in the formative and low-stakes settings, OSCEs were found to be an anxiety-inducing experience for students and a challenging and labor-intensive experience for the examiners and organizers.10–19 It was therefore conceptualized that any positive or negative perception on the implementation of the Medicine and Therapeutics exit OSCE may affect students’ anxiety and stress level as well as their performance in such a high stakes examination. Very few research studies have investigated the examiners perception of the OSCE process and on their role as examiners for this process.11,18,19 Examiners' opinions are required to be sought as examiners play a critical role in executing the OSCE, contribute to the design of the OSCE stations, identify the competences to be tested, and provide individual or group feedback to the students after examination.20 There is therefore a need for investigating both student and examiner perception of this complex and potentially overwhelming process to identify areas that need further attention to improve the standard and quality of this examination. Medical schools from other countries would benefit from our findings on the organization of such a complex, time-consuming, resource-intensive assessment exercise. In addition,there are significant financial costs associated with implementing the exit OSCE and all four campuses are incurring huge expenses from examiner travel between islands, recruiting standardized patients, external examiners, support staff, and the other administrative expenses. Though a number of studies have been published on OSCEs in other campuses of UWI,9,10,21–24 this study was the first to evaluate the UWI exit OSCE.

Against this background, we aimed to conduct a study to determine examiner and student perception on the attributes, quality, validity, reliability, and organization of the Medicine and Therapeutics exit OSCE held at the Cave Hill Campus in 2017 and to identify areas that need further attention to improve the standard and quality of this examination.

Methods

OSCE setting

The exit MBBS in Medicine and Therapeutics OSCE consists of 17 examiner observed active stations and seven rest stations, thus in total 24 stations. Stations are of 7 minutes duration except for the history-taking stations which are of 23 minutes duration for the adult history and 15 minutes for the pediatric history. There is one minute transit time between stations. Of the 17 active stations, five are devoted to pediatrics, two to psychiatry, one to community health and the remaining nine stations are devoted to adult medicine (Table 1). The examination at each of the stations, except the history taking stations, lasted for 5 minutes with additional 2 minutes for presentation of findings and discussion. At the history taking station in pediatrics candidates were required to take a history for 10 minutes and then had 5 minutes to collect their thoughts, make a presentation and answer questions, while in the adult history station, they had 15 minutes to take history and 8 minutes to collect their thoughts, make a presentation and answer questions. Performance of the required tasks was calibrated according to specifically designed checklists, containing 10 to 20 items, each with an assigned score corresponding to the key skills.

Table 1

OSCE stations in medicine and therapeutics

● Paediatrics: 5 stations	● Adult Medicine: 9 stations
∘  History taking [P]	∘ History taking [S3]
∘ Cardiovascular examination [P]	∘ Respiratory examination [HV, E]
∘ Abdominal examination [P]	∘ Neurologic examination [HV, E]
∘ Neurologic examination [P]	∘ Cardiovascular examination [HV, E]
∘ Technical skill [S]	∘ Abdominal examination [HV, E]
	∘ Dermatological examination [HV, E]
● Community Health: 1 station	∘ Rheumatology station [HV, E]
∘ Counselling skills [S2] or	∘ Funduscopic examination station
∘ Community health problems [S2]	∘ Basic life support (BLS) [M]
	● Psychiatry: 2 stations
	∘ Psychiatric case evaluation

Abbreviations: P, real patients; S1, simulated patient and/or parents; S2, simulated patients with predefined case scripts; S3, simulated patients; M, mannequin; HV, healthy volunteer; E, examiner.

Research design

To realize the study aims, a cross-sectional survey approach was adopted and the participants were asked to complete a self-administered questionnaire at the end of the examination.

Participants

All students and examiners who participated for the May/June OSCE held at the Cave Hill campus, UWI on May 20, 2017 formed the part of the study population recruited through voluntary response sampling technique.

Study instrument

Study instruments for student10,21,25 and examiner11,19 were designed based on questionnaires used in previous studies. The student questionnaire evaluated perception of the recently completed OSCE under the following domains using Likert scales as follows: attributes from strongly disagree (score of 1) to strongly agree (score of 5); quality from not at all (score of 1) to a greater extent (score of 4); validity and reliability from not at all (score of 1) to a greater extent (score of 4); organization from very poor (score of 1) to excellent (score of 5); and comparison of the OSCE, multiple choice questions (MCQ), essay/short answer (SAQ) and clerkship assessment formats on 3-point scales in terms of their level of difficulty, fairness, learning opportunities and most preferable usage during clinical years. Additionally, there were three open-ended questions.

The examiner questionnaire evaluated perception of the overall fairness of the OSCE, range of clinical skills and knowledge tested, validity of the measure of clinical competence, exam administration, provided the level of information required and clarity of instructions at each station, adequacy of time allocation for each station, level of stress experienced by the students and minimized their chances of failing. In addition,examiners were asked to rate the impact of the OSCE on student learning, whether it is preferable to other formats of clinical examination and if it should be used more often in the clinical years of undergraduate programme. All questions were answered on a 5-point Likert scale (1 “strongly disagree” to 5 “strongly agree”). There was one open-ended question allowing examiners to express their opinion about the OSCE.

Data collection

In collecting the data for the present study, participants (students and examiners) were asked to fill-out the questionnaire at the end of the OSCE, which was guided by voluntary participation, anonymity, and assured confidentiality of the collected data. A printed sheet providing a brief description of the study and a request of consent was attached to the questionnaire. Those who consented to participate in this study signed the consent form and filled out the questionnaire. This process was completed on site at conclusion of the OSCE for all of the candidates.

Data analysis

The data was entered into the Statistical Package for the Social Sciences (SPSS) version 23.0 (IBM Corporation, Armonk, NY, USA). Mean and SD were calculated for data on age of students and year of experience as an OSCE examiner. Cronbach’s alpha was calculated to establish the reliability of the instruments used. Specifically, percentages, median, IQR, and chi-squared analysis with Yates’ correction were used to determine if the distribution of frequencies of responses in each item were sufficiently different to reject the null hypothesis that the distribution was due to chance. The critical value used to reject the null hypothesis was P≤0.01. The information collected from open-ended questions was collated and presented thematically.

Ethical approval

Ethical approval was obtained from the Institutional Review Board, The University of the West Indies, Faculty of Medical Sciences, Cave Hill Campus, Barbados (IRB No: 180209-B).

Results

Fifty-four students completed the questionnaire. The response rate was 100% and of the respondents 66.7% were female, mean age was 24.4 (SD ±1.6) years, 48.1% were from Barbados and 44.4% from Trinidad. There were 22 examiner respondents (response rate: 55%); more than half were male (54.5%). The mean years of experience of the respondents being an OSCE examiner is 7.60 (SD ±6.80).

Median, IQR and chi-squared values are provided for each survey question in Table 1–4. The frequency distribution and percentage of student and examiner responses are also shown in each table. The internal consistency of the survey was good with Cronbach’s alpha values of 0.89 and 0.90 for student and examiner questionnaire respectively.

Table 4

Students’ perception of assessment format

Which of the following formats is easiest?	N	Chi-squared	Median (IQR)	Difficult	Undecided	Easy
MCQ	52	10.89*	2 (0)	9 (17.3%)	28 (53.8%)	15 (28.8%)
Essay/SAQ	48	13.88*	2 (1)	15 (31.3%)	27 (56.3%)	6 (12.5%)
OSCE	49	19.14*	1 (1)	25 (51%)	22 (44.9%)	2 (4.1%)
Clerkships ratings	49	14.74*	2 (0)	10 (20.4%)	29 (59.2%)	10 (20.4%)
Which of the following formats is fairest?				Unfair	Undecided	Fair
MCQ	51	11.41*	2 (1)	6 (11.8%)	20 (39.2%)	25 (49%)
Essay/SAQ	44	14.23*	2 (1)	3 (6.8%)	22 (50%)	19 (43.2%)
OSCE	48	9.13**	2 (1)	7 (14.6%)	24 (50%)	17 (35.4%)
Clerkships ratings	45	20.93*	2 (1)	2(4.4%)	27 (60%)	16 (35.6%)
From which of the following formats do you learn most?				Learn very little	Undecided	Learn a lot
MCQ	50	7.24**	2 (1)	8 (16%)	19 (38%)	23 (46%)
Essay/SAQ	45	18.53*	3 (1)	2 (4.4%)	18 (40%)	25 (55.6%)
OSCE	48	10.50*	2.5 (1)	6 (12.5%)	18 (37.5%)	24 (50%)
Clerkships ratings	48	9.13**	2.5 (1)	7 (14.6%)	17 (35.4%)	24 (50%)
Which of the following formats should be used more often in the clinical years?				Used much less	Undecided	Used much more
MCQ	49	9.84*	2 (1)	6 (12.2%)	21 (42.9%)	22 (44.9%)
Essay/SAQ	44	3.05	2 (1)	11 (25%)	20 (45.5%)	13 (29.5%)
OSCE	48	4.88	2 (1)	9 (18.8%)	18 (37.5%)	21 (43.8%)
Clerkships ratings	45	12.13*	3 (1)	5 (11.1%)	16 (35.6%)	24 (53.3%)

Notes: Values are statistically significant *P<0.01. **P<0.05.

Abbreviations: MCQ, multiple choice questions; SAQ, short answer questions; OSCE, objective structured clinical examination.

Students’ perception of OSCE

Table 2 reveals that a majority of students (63–91%) perceived positively about the attributes of OSCE, i.e. fair examination, covered required knowledge and competencies, well-administered and well-sequenced OSCE stations. However, most of the students (67–79%) found the OSCE stressful and intimidating, and constrained with time. It was further revealed that the majority of the students remained neutral with the following statements: examination process minimized their chance of failing and allowed student to compensate in some areas. Approximately 80% students did not agree that the OSCE was less stressful than other examinations.

Table 2

Student perception of attributes and organization of OSCE (N=54)

Attributes of OSCE	Chi-squared	Median (IQR)	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Examination was fair	77.91*	4 (0)	0	1 (1.9%)	10 (18.5%)	37 (68.5%)	6 (11.1%)
Wide knowledge area covered	70.76*	4 (0)	0	3 (5.6%)	5 (9.3%)	36 (66.7%)	10 (18.5%)
Needed more time at stations	15.60*	4 (2)	2 (3.7%)	6 (11.1%)	10 (18.5%)	21 (38.9%)	15 (27.8%)
Examination well administered	73.66*	4 (0)	0	0	10 (18.5%)	36 (66.7%)	8 (14.8%)
Examination very stressful	40.22*	5 (1)	0	2 (3.7%)	10 (18.5%)	12 (22.2%)	30 (55.6%)
Examination well structured and sequenced	59.31*	4 (1)	1 (1.9%)	2 (3.7%)	2 (3.7%)	32 (59.3%)	17 (31.5%)
Examination minimized chance of failing	53.54*	3 (0)	1 (1.9%)	11 (20.4%)	32 (59.3%)	8 (14.8%)	2 (3.7%)
OSCE less stressful than other examinations	40.94*	2 (1)	23 (42.6%)	20 (37.0%)	8 (14.8%)	1 (1.9%)	2 (3.7%)
Allowed student to compensate in some areas	37.43*	3 (1)	1 (1.9%)	6 (11.1%)	26 (48.1%)	18 (33.3%)	3 (5.6%)
Highlighted areas of weaknesses	36.93*	4 (1)	1 (1.9%)	3 (5.6%)	13 (24.1%)	28 (51.9%)	9 (16.7%)
Examination intimidating	24.62*	4 (2)	0	4 (7.4%)	12 (22.2%)	23 (42.6%)	15 (27.8%)
Student aware of level of information needed	39.43*	4 (1)	0	5 (9.3%)	15 (27.8%)	28 (51.9%)	6 (11.1%)
Wide range of clinical skills covered	58.82*	4 (0)	0	1 (1.9%)	11 (20.4)%	33 (61.1%)	9 (16.7%)
Organization of OSCE	Chi-squared	Median (IQR)	Excellent	Above average	Average	Below average	Very poor
Orientation of OSCE before examination	27.18*	4 (2)	15 (27.8%)	18 (33.0%)	20 (37.0%)	1 (1.9%)	0
Announcement of venue and known to student	31.16*	4(2)	18 (33.0%)	22 (40.7%)	14 (25.9%)	0	0
Timetables were available and known to student	18.82*	4.5 (2)	27 (50.0%)	12 (22.2%)	11 (20.4%)	2 (3.7%)	2 (3.7%)
Revision of clinical procedures before OSCE	15.08*	3 (2)	8 (14.8%)	13 (24.1%)	21 (38.9%)	10 (18.5%)	2 (3.7%)
Staff answered queries related to OSCE	20.86*	4 (1)	16 (29.6%)	17 (31.5%)	18 (33.3%)	3 (5.6%)	0
Quality of OSCE rooms ie setup, lightening etc.	28.91*	4 (2)	22 (40.7%)	19 (35.2%)	12 (22.2%)	0 (0%)	1 (1.9%)
Availability good equipment including simulators	29.91*	4 (1)	15 (27.8%)	23 (42.6%)	15 (27.8%)	0 (0%)	1 (1.9%)

Note: *Values were statistically significant P<0.01.

Abbreviation: OSCE, objective structured clinical examination.

Table 2 also shows a majority of the respondents rated the organization of the OSCE as “excellent” and “above average”. However, more than half of the students felt dissatisfied with the clinical procedures’ revision done before the examination.

In relation to quality of OSCE performance, the majority of students were satisfied; however, approximately one quarter or less of the respondents had expressed their dissatisfaction with the allotted time, authenticity of setting and context at each OSCE station, and opportunities to learn provided by the OSCE (Table 3). More than half of the students thought that the tasks included in the OSCE “somewhat” reflected those taught in the clerkships.

Table 3

Student perception of quality and validity/reliability of OSCE performance (N=54)

Quality of OSCE performance	Chi-squared	Median (IQR)	To a greater extent	Somewhat	Very little	Not at all
Aware of the nature of the examination	44.51*	4 (1)	30 (55.6%)	23 (42.6%)	1 (1.9%)	0
Tasks reflected those taught	40.53*	3 (1)	23 (42.6%)	29 (53.7%)	1 (1.9%)	1 (1.9%)
Time at each station was adequate	19.00*	3 (1)	10 (18.5%)	27 (50%)	14 (25.9%)	3 (5.6%)
Setting and context at each station felt authentic	14.69*	3 (2)	18 (33.3%)	23 (42.6%)	11 (20.4%)	2 (3.7%)
Instructions were clear and unambiguous	48.80*	4 (1)	33 (61.1%)	20 (37.0%)	0	1 (1.9%)
Tasks asked to perform were fair	44.54*	4 (1)	32 (59.3%)	20 (37.0%)	2 (3.7%)	0
Sequence of stations logical and appropriate	38.83*	4 (1)	30 (55.6%)	21 (38.9%)	2 (3.7%)	1 (1.9%)
Examination provided opportunities to learn	13.39*	3 (2)	18 (33.3%)	22 (40.7%)	12 (22.2%)	2 (3.7%)
Validity and reliability of OSCE	Chi-squared	Median (IQR)	To a greater extent	Somewhat	Very little	Not at all
OSCE scores provide true measure of essential clinical skills	20.96*	3 (1)	9 (16.7%)	29 (53.7%)	11 (20.4%)	5 (9.3%)
OSCE scores are standardized	60.10*	3 (0)	10 (18.5%)	39 (72.2%)	5 (9.3%)	0
OSCE is a practical and useful experience	23.84*	3 (1)	15 (27.8%)	29 (53.7%)	4 (11.1%)	6 (11.1%)
Personality, ethnicity and gender will not affect OSCE scores	17.21*	3 (1)	9 (16.7%)	25 (46.3%)	17 (31.5%)	3 (5.6%)

Note: *Values were statistically significant P<0.01.

Abbreviation: OSCE, objective structured clinical examination.

Though the majority of the respondents were satisfied with the OSCE performance criteria, one quarter to one third of the students showed concerns as to whether the OSCE scores provided a true measure of essential clinical skills (29.7%) and whether the OSCE is a practical and useful experience (22.2%). Three quarters of the students believed that factors like gender, ethnicity, and personality did affect the results of OSCE.

When students were asked to compare the assessment instruments, the majority of the students identified the OSCE as the most difficult and MCQ as the fairest assessment formats. Students also mentioned that they learnt most from essay/SAQ format and suggested to use clerkship rating more in the clinical years (Table 4).

Answers to open-ended questions showed students were satisfied with all aspects of the organization of the OSCE and suggested a number of ways to improve future OSCE experiences. The positive aspects of the OSCE highlighted by the students included: well-structured OSCE; cooperation of examiners, staff, and patients; inclusion of wide variety of stations including multiple rest stations; availability of adequate staff with good mix of local/foreign examiners, and coverage of wide area of knowledge and skills. Respondents felt that the time allocated to perform expected tasks was insufficient, and that the procedure was stressful and intimidating. Suggestions for improvement included more mock examination sessions, more time per station, split OSCE into pediatrics and internal medicine, feedback after OSCE clerkships, and better layout of OSCE to avoid confusion.

Examiners' perception of OSCE

With regard to perception of examiners about their respective stations, majority of the examiners agreed to strongly agree that the examination was fair, covered a wide range of clinical skills and knowledge, was well-organized and well-administered, students were aware of the level of information needed, tasks asked to perform at each station were fair, and the OSCE was a standardized examination for all students (Table 5). However, some of the examiners felt that the OSCE failed to create a positive impact on student learning and OSCE scores did not truly reflect competence in clinical skills.

Table 5

Examiners' perception of OSCE (N=22)

	Chi-squared	Median (IQR)	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
Overall, the examination was fair	16.49*	4 (1)	2 (9.1%)	0 (0%)	1 (4.5%)	13 (59.1%)	6 (27.3%)
Wide range of clinical skills and knowledge covered	10.69**	4 (1)	2 (9.1%)	1 (4.5%)	3 (13.6%)	12 (54.5%)	4 (18.2%)
Examination was well-organized and well-administered	11.69**	4 (1)	2 (9.1%)	0 (0%)	1 (4.5%)	9 (40.9%)	10 (45.5%)
Students were aware of level of information needed	10.19**	4 (1)	2 (9.1%)	1 (4.5%)	1 (4.5%)	11 (50%)	7 (31.8%)
Instructions at each station were clear and unambiguous	7.19	4 (1)	2 (9.1%)	2 (9.1%)	1 (4.5%)	7 (31.8%)	10 (45.5%)
Setting and context at each station felt authentic	8.29	4 (1)	2 (9.1%)	1 (4.5%)	3 (13.6%)	11 (50%)	5 (22.7%)
Sequence of stations was logical and appropriate	9.29	4 (1)	3 (13.6%)	0 (0%)	4 (18.2%)	11 (50%)	4 (18.2%)
Tasks asked to perform at each station were fair	14.49*	4 (0)	1 (4.5%)	1 (4.5%)	3 (13.6%)	13 (59.1%)	4 (18.2%)
Time at each station was adequate	3.19	4 (2)	4 (18.2%)	2 (9.1%)	3 (13.6%)	9 (40.9%)	4 (18.2%)
OSCE scores provided true measure of essential clinical skills	3.09	4 (2)	3 (13.6%)	4 (18.2%)	3 (13.6%)	9 (40.9%)	3 (13.6%)
Examination were stressful for the students	7.49	3 (2)	1 (4.5%)	0 (0%)	7 (31.8%)	6 (27.3%)	8 (36.4%)
OSCE minimizes chances of failing	7.79	4 (1)	2 (9.1%)	2 (9.1%)	2 (9.1%)	11 (50%)	5 (22.7%)
OSCE has a positive impact on student learning	0.79	3 (2)	3 (13.6%)	5 (22.7%)	6 (27.3%)	4 (18.2%)	4 (18.2%)
OSCE is a standardized examination for all students	11.19**	4 (1)	1 (4.5%)	2 (9.1%)	2 (9.1%)	12 (54.5%)	5 (22.7%)
OSCE scores truly reflect competence in clinical skills	3.89	3 (2)	5 (22.7%)	5 (22.7%)	3 (13.6%)	8 (36.4%)	1 (4.5%)
OSCE is preferable to other formats of clinical examination	4.29	3 (1)	2 (9.1%)	2 (9.1%)	8 (36.4%)	6 (27.3%)	4 (18.2%)
OSCE should be used more often in the clinical years	1.99	4 (1)	3 (13.6%)	2 (9.1%)	5 (22.7%)	8 (36.4%)	4 (18.2%)

Note: Values are statistically significant *P<0.01. **P<0.05.

Abbreviation: OSCE, objective structured clinical examination.

Discussion

In the present study, the response rates for the students and examiners were adequate26 and the number (N=17) and timing of the stations (>5 minutes) exceeded the requirements to achieve a reliable OSCE.27 Both students and examiners expressed their satisfaction with the attributes, quality indicators, validity/reliability criteria, organization/settings of the Medicine and Therapeutics exit OSCE. In FMS, UWI, Mona Campus (Jamaica), Pierre et al10 used OSCE as an assessment instrument during the pediatric clerkship in year five to examine student acceptance of the OSCE and evaluation of the clerkship. Similar to our study, the authors recorded “overwhelming acceptance of the OSCE”: comprehensiveness (90%), transparency (87%), fairness (70%), and authenticity of the required tasks (58–78%). In both campuses, students expressed some concerns regarding OSCE, e.g., stressful and intimidating event and insufficient time to complete the stations. Similar acceptance and concerns of the OSCE have been previously described by the students in other medical schools worldwide.11,12,28,29

In the present study, more than three-quarters of the students felt that the OSCE induced higher levels of stress compared to other examination formats. Similar concerns were reported in other studies with medical and other health professional students.10,12–14,30,31 Examinations and assessment procedures in medicine were found to be anxiety provoking and stressful. Brand and Schoonheim-Klein13 reported that students had to prepare better for OSCEs and their expectation to pass OSCE was also significantly higher which might be the cause of stress. Moreover in OSCEs, timed test, interaction with patients and examiners, close monitoring and observation by examiners, and rude and apathetic approach of some of the examiners were responsible for students' increased stress levels.14,15,31–33 It was demonstrated that stress level did not decrease with increasing exposure to OSCEs.33,34 These increased stress levels, as Marshall and Jones14 mentioned, might be due to the different material tested rather than the assessment methods themselves. As high levels of stress may interfere with performance,35 careful preparation of students prior to OSCE is required to minimize the students’ anxiety level.16 In the present study, 70% of the students identified the OSCE as intimidating and this has been reported in other studies.10,36 Approximately 80% of the students disagreed that an OSCE was less stressful than other examinations and half of the students (51%) identified the OSCE as being more difficult than other forms of assessment instruments such as MCQ, essay/SAQ, and clerkship rating. Brand and Schoonheim-Klein13 reported greater stress levels among dental students during an OSCE in comparison to written and practical assessment methods. In a study conducted in Pakistan, more than 90% of the students found OSCEs “more stressful and mentally tougher” than other traditional examination formats. These views appear to contrast with findings from a study conducted in Ethiopia that students found long case and short case examinations in the clinical years more stressful than OSCE.17 This may be due to “an unsympathetic interaction between examiner and examinee” along with other factors;17 which include “poor briefing for the student or training for the examiners, poorly designed stations or a mismatch between what is tested in the OSCE and the curriculum and teaching and learning programme”.20 At the Cave Hill Campus, we do not use long and short cases for examinations in the Medicine and Therapeutics course and students perceived OSCE more stressful than written and other examinations. The inclusion of long and short cases needs to be considered as Khan37 questioned the validity of OSCE in clinical examinations and recommended to use workplace-based assessments by reducing the role of OSCEs “to put the ‘art’ back into medicine”. The author argued that students prepare strategically to pass the OSCE and adopt a robotic “tickbox” approach, but “struggle to translate this into skills which are practically useful in the dynamic, ever changing environment of real frontline medicine”. These concerns are supported by the view of the examiners in the present study—36% of the examiners felt that OSCE failed to create positive impact on student learning and 45% of examiners disagreed that OSCE scores truly reflect competence in clinical skills. These areas need further investigation.

Though students and examiners were in agreement with various aspects of the OSCE, there was a divergence in opinion among students and examiners regarding timing at OSCE stations. The majority of the students (66.7%) reported that they needed more time at stations whereas examiners felt that the time at stations was adequate (59.1%). A similar difference of opinion was also observed between students (41.7%) and examiners (11.1%) in a study conducted by Omu et al11 in Kuwait. Some studies have raised concerns that time was a problem and suggested that they focus on restricting time but rather should concentrate on how well students can perform.38,39 However, Schoonheim-Klein et al40 demonstrated that increased time per station had no impact on student’s performance. Stowe and Gardner41 suggested that the instructions in the OSCE form should be short and clear to give time to complete the task by the students.

The present study has a number of limitations. This cross-sectional study involved only one campus (out of four campuses) and had a small sample size; therefore, caution needs to be taken to generalize the data to other settings. One of the strengths of the study is seeking feedback both from students and from examiners; however, examiners’ comments may be biased as they examined only one station.

Conclusion

In this study, students and examiners reported favourable opinions on the process and organization of the OSCE conducted during the Medicine and Therapeutics exit exam. However, students felt that the OSCE was stressful and intimidating, and time allocation was inadequate for the assigned tasks. More practice sessions/mock exams with adequate feedback may better prepare students and create a better environment to assess skills expected of a doctor in clinical practice. Further, multicentred studies are required to be carried out to assess whether there is any difference in actual clinical performance between students assessed by traditional formats compared to those assessed by an OSCE and to ascertain long-term impacts of OSCE on clinical management of patients later in their professional life.