Feasibility and acceptability of objective structured clinical examination (osce) for a large number of candidates: experience at a university hospital.

OBJECTIVE: To assess the feasibility and acceptability of using objective structured clinical examination (OSCE) for a large number of medical students.
METHODS: All medical students (291) who had completed the basic surgical course were examined by objective structured clinical examination (OSCE) at the College of Medicine, Riyadh, for the first time. A 5-scale questionnaire was filled by the examiners at the end of the examination each day. Another questionnaire was filled by the students as a feedback.
RESULTS: All students agreed that the organizational aspect of the examinations was smooth and the time for each station was adequate. 86% of the students agreed that the stations were within the content of the course, 82% agreed that the examination was fair and objective and 93% wanted this method to be followed in the assessment of third year medical students, instead of the traditional examination (written and single long case). Similar responses were received from the examiners who were involved in the exams.
CONCLUSION: OSCE is a practical and acceptable method for assessing medical students' basic surgical skills, even for a large number of candidates, if facilities are available in the examination center. Replacing written exams with OSCE depends on the design of stations to test knowledge adequately in scope and depth probably at problem solving level.

INTRODUCTION

In recent years, there has been growing dissatisfaction in medical schools with the traditional methods of student assessment based on written examinations and faculty ratings of performance in clinical training. This is because of the limited skills assessed through written tests and psychometric problems associated with the rating of performance.1 Clinical competence refers to a complex set of skills that include the abilities to interview, perform a physical examination, make diagnostic and treatment decisions, and while demonstrating good interpersonal skills communicate with a patient and his or her family.2 The importance of the assessment of these skills, which are usually not systematically done in medical schools has been identified by many associations,3 addressed in several conferences and reports,4–5 and already has been applied in some medical schools in many parts of the world.7–8

The purpose of this prospective cross-sectional study is to examine the feasibility and acceptability of an objective structured clinical examination (OSCE) which was used to examine a large number of medical students for the first time at our institution.

MATERIAL AND METHODS

A total population (two hundred and ninety one) of medical students who had completed the basic surgical course (mainly history-taking and physical examination) at the College of Medicine, Riyadh, were examined by OSCE.

Forty consultant surgeons (examiners) were involved in a two-day examination, in four surgical wards. In addition, 8 surgical registrars organized the examinations. During the examination the students rotated round ten stations in each surgical ward at the same time, spending 4 - 5 minutes at each station. On a bell signal, the student moved to the next station. The time assigned was the same for all stations. A further 30 seconds was allowed for the student to move to the next station and for the examiner to finalize comments on the previous student's performance. Where the distance between two stations was long, a rest station was placed between them.

The assessment at each station was limited to the techniques of history-taking, physical- examination and differential diagnosis. Stations were either manned, with real or simulated cases or unmanned with pictures of clinical cases. The examiners used checklists to record the performance of the students at the manned stations. A 5-point scale answer questionnaire comprising 12 questions was completed (Appendix I). This was completed each day by all examiners immediately after the examination. Another feedback questionnaire was constructed for the students examined. Both questionnaires were in English.

RESULTS

The total number of students who were examined was 291 (208 male and 83 females). Almost all of the students agreed that the organization of the examinations was smooth and the time allotted for each station was adequate. The exam was described as fair and objective by 86% of the students, and 93% wanted this to be the method of assessment for third year surgical course.

All of the examiners agreed that the organization was smooth and the stations were within the scope of the course. The vast majority of the examiners (90%) agreed that the examinations were fair and objective and 78% preferred this method of assessment to the traditional method (written and single long case examination).

Forty clinical cases (real and simulated) were used on both days of examination. Patients were cooperative despite being examined by a large number of students on the same day. Another group of clinical cases were used on Day 2.

DISCUSSION

In our institution, the method used in the assessment for the third year medical students who have completed the basic surgical course, are the traditional written and long case clinical examinations. These methods have many shortcomings. Firstly, there is no guarantee that students would be able to use their knowledge in the care of patients or apply their clinical skills in the appropriate situations. Several studies910 have shown that students’ clinical performances are rarely observed by faculty in written examinations. Secondly, with regard to the long case clinical examination, there is great variability both in the patients assigned to students and in the criteria used by individual faculty members in rating students’ performance. Therefore, the current methods tend to be mostly subjective and not standardized. These negative effects of traditional methods of assessment have often been reported.11–13

In 1979, Harden14 described the first objective structured clinical examination (OSCE). This method has dramatically changed the assessment of clinical competence and had a significant impact on future doctors’ training and practice.15 OSCE fulfills most of the criteria needed to assess clinical competence especially because of its greater objectivity and the fact that the areas tested are uniformly applied by the examiners. It is envisaged that students’ skills in history-taking and physical examination, the essential content of the basic surgical course for the third year medical students would be tested. This would leave investigations, diagnosis and treatment to be included in the final year surgical course. In OSCE, the various components of clinical competence such as history-taking, examining the abdomen, commenting on a picture of a patient…etc are tested in phases; each component assessed in turn and dealt with at one of the stations in the examination.

All the examiners who participated in the study agreed that the stations were within the purview of the course. For our students, this method of assessment was highly acceptable. The faculty in turn could decide in advance the items to be examined (history-taking, physical examination and differential diagnosis) and design the stations accordingly. Furthermore, the content, structure and complexity of the examination (e.g. more straight-forward cases for junior students) are easily controlled. Besides, the use of the checklists by the examiners resulted in a more objective assessment, and with ten stations, a larger sample of the student’ skills were tested. For this reason, 78% of the teaching staff who responded to the questionnaire recommend this method for examining the third year medical students, even though OSCEs are time-consuming and labor-intensive.16

The OSCE is considered a significant contribution to the improvement of the methods of testing students’ clinical skills in medicine,17 and is known to be more valid in the assessment of clinical skills, both in undergraduate and postgraduate training.18 OSCEs combine the reality of live clinical interactions, the standardization of problems and the use of multiple observations of each student. Consequently, it is rapidly replacing other forms of assessment at all levels of medical and health professional education, licensure and certification.19

The organization of OSCE is complex and time consuming especially when many stations are to be used.20 These and other logistic limitations have restricted its application to smaller groups.18 Although this method of assessment was used for the first time at our institution, it was successful for a large number of candidates (291 medical students).

Simulated cases which were well-controlled by checklists were used in a few stations (e.g. asking the student to examine a normal abdomen or to take a history from the examiner himself). Standardized patients (SPs) have been used before in the OSCE format. SPs are individuals with or without actual disease, who have been trained to portray a medical condition in a consistent manner.21 SPs can also evaluate skills in interviewing interpersonal relationships, and communication.22 With the proper training, SPs were known to provide consistent and accurate simulations and recordings of performance by medical students and professionals.23 They are the gold standards for measuring the competence of students and the quality of the practice of physicians.24 Since the use of OSCE was the first in our surgical course, no SPs were used, but there are plans to use them in the coming exams.

In conclusion, OSCEs are practical and acceptable methods for assessing medical students’ basic surgical skills. If facilities are available (manpower, surgical wards, clinical cases and an enthusiastic organizing committee), a large number of candidates can be accommodated. In our institution, the OSCE is now the method of assessment for the third year medical students who have completed the basic surgical course.