Five years after commencing the objective structured clinical examination: are we getting it right? Medical students' assessment as the measuring index.

INTRODUCTION: The aim of this study was to assess our use of OSCE from the perception of final year medical students.
MATERIALS AND METHODS: This is a cross sectional survey of final year medical students undergoing the final examination in Surgery. All 102 medical students in the class were given the self-administered questionnaire to fill. The data were collated into excel spreadsheets and analysed using the SPSS version 21.
RESULTS: A total of 79 completed questionnaires were retrieved (return rate of 78%).All the students knew about and had participated previously in OSCE, 94.9% accorded the OSCE fair. 76(93.2%) wanted the examination as the main method of clinical assessment, 38(46.6%) had adequate preparation.In terms of the OSCE stations not mirroring real clinical scenarios, 38(48.1%) disagreed, and 26(20.5%) strongly disagreed. 34(43%) disagreeing that the logistics was poor. The students rating of the OSCE, on a scale of 1 - 10, gave a mean score of >8 for spread of the OSCE stations, detail of the questions and objectivity of the examination. The nearness to clinical reality was rated as 7.52 with artificiality of the stations rated as 4.12.
CONCLUSION: The OSCE has gained acceptance amongst final year medical students.

Introduction

The objective structured clinical examination is the most common way of assessing the clinical skill and proficiency of medical students[1]. From its humble beginnings in 1974 by Harden et al[2], it has gained traction worldwide in medical students' assessment and clinical competency training. It has undergone various modifications and alterations and has been domesticated in medical schools worldwide[1,3,4,5,6]. From when it was first introduced it has slowly gained ground to become the main method of clinical assessment of medical students[3,5,6].

Its main attraction has been its objectivity and versatility in testing clinical competency whilst retaining excellent reliability[1,3,5,7]. However, it is tedious to plan, design and effect[1,3,4,5,8]. Logistics involved are extensive and test the intellectual, pedagogical, commitment, versatility and physical energies of the examiners[3,7,8]. It also enjoys the use of various objects, devices and software in creating clinical scenarios to enhance the clinical experience whilst enabling the extinguishing of harm to patients or subjects in the controlled environment that it provides[3,4,5].

In the University of Benin Medical School, the department of surgery was the first to use it as a means of final examination. This followed its introduction in 2011 to the end of clinical surgical clerkship of the 400 Level students in that year. It has since come to supplant the traditional clinical examination model of the long case and short case. This year marks the fifth year of its introduction and thus this study was designed to evaluate it using a student based assessment. The aim of this study was to assess our use of this mode of examining medical students with a view to ascertaining the perception of final year medical students to the use of OSCE.

Materials and methods

This is a cross sectional survey of final year medical students undergoing the part V examinations in Surgery. The study population is the 600 level medical students who were undergoing the final MBBS Surgery examination using the OSCE. There was no attempt made at stratification or randomization. The research instrument used was a questionnaire which was designed by the authors to gauge the medical students' assessment of the OSCE Examination and its processes. The questionnaire had face and content validation by examiners in the department of surgery. There were 102 medical students in the class, and all of them were approached by the authors and given the questionnaire to fill. All students who presented themselves during the period of the examination were asked to fill the questionnaire which was self-administered. The questionnaires were immediately collected by the authors and collated into excel spreadsheets. The data was then analysed using the SPSS version 21 and the results presented in tables and charts.

Results

A total of 79 completed questionnaires were retrieved. This had a return rate of 78%.

All the students agreed that they knew about OSCE and had participated in one previously, however only 94.9% accorded the OSCE a fair examination.

In terms of desirability of the OSCE, 76(93.2%) wanted the examination as the main method of clinical assessment though only 38(46.6%) accepted to have had adequate preparation for the examination. Figure 1

Figure 1

shows a composite chart of adequate preparation and wanting the OSCE

Concerning the comprehensiveness of the scope of the OSCE, 41(51.9%) students agreed and 23(29.1%) strongly agreed. In terms of the OSCE stations not mirroring real clinical scenarios, 38(48.1%) disagreed, and 26(20.5%) strongly disagreed. Most of the students {32(40.5%)} agreed that the timing of each OSCE station was too short whereas 26(20.5%) students disagreed. The logistics for the examination was highly rated with 34(43%) disagreeing that the logistics was poor but 20(25.3%) were unsure. Figure 2

Figure 2

The students rating of the OSCE, on a scale of 1 – 10, gave a mean score of >8 for spread of the OSCE stations, detail of the questions and objectivity of the examination. However, clarity of the questions had a mean score of 6.99 and difficulty of the question was 6.86. The ease of answering the question was rated with a mean score of 5.54 Table 1

Table 1

Students Scoring of the OSCE Stations

	ITEM	Mean score	Standard deviation
1	Spread of the OSCE station	8.00	1.51
2	Detail of the OSCE questions	8.01	2.07
3	Objectivity of the OSCE stations	8.12	1.53
4	Clarity of the OSCE questions	6.99	2.01
5	Difficulty of the OSCE questions	6.86	1.90
6	Ease to answer the questions	5.54	1.88
7	Time duration of the OSCE stations	6.23	2.45
8	Nearness of the stations to real life clinical scenerios	7.52	1.61
9	Ability to guess the right answers to the questions	3.93	1.81

The nearness to clinical reality was rated as 7.52 with artificiality of the stations rated as 4.12. The duration of the station had a mean score of 6.23 whilst the ease of guessing the right answers had a mean score of 3.93. The main suggestions towards improving the OSCE examination were: more exposure of the students to OSCE preparation which was suggested by 10(12.7%) students; improvement in the objectivity as suggested by 9(11.4%) students); improvement in the logistics which was suggested by 8 (10.1%) students; longer duration of the stations and clarity of the pictures had 7(8.9%) students each; 4(5.1%) students suggested more training of the simulators and 2 (2.5%) students suggested improvement in the station design. Table 3

Table 3

Students suggestions for improving the OSCE Stations

Suggestions	Frequency (%)
More exposure of the students to OSCE preparation	10(12.7%)
Improvement in the objectivity	9(11.4%)
Improvement in the logistics	8(10.1%)
Longer duration of the stations	7(8.9%)
Clarity of the pictures	7(8.9%)
More training of the simulators	4(5.1%)
Improvement in the station design	2(2.5%)

Discussion

The OSCE has been demonstrated in various studies to exhibit excellent validity and reliability in testing knowledge and competence of medical students both in terms of clinical relevance as par what they would encounter in medical practice and also to test a wide range of skills in a controlled environment[1,6,8]. It has supplanted the traditional long case and short case examinations in many medical schools as the main mode for final examinations in surgery[3,5,7,8]. Its importance however goes beyond summative assessment to usefulness in formative assessment, thus increasing its usefulness in the medical school[1,3]. The need to engage the medical students more constructively amidst introduction of OSCE in postgraduate medical training instigated the introduction in the medical school.

Proper planning and preparation goes into the logistics for the OSCE and that includes training of both examiners and students about the various aspects of the OSCE[1,5]. Students' preparation is key to ensuring that medical students are able to properly participate in the examinations as the various skills and competencies being tested are necessarily compartmentalized and standardized to the OSCE format for testing[6,9,10]. In our study, though students' preparation was adjudged poor by the students, yet it did not affect their desire to have it as the main format for their final examination with most of them finding it a fair examination. This is keeping with other studies from recent introduction of OSCE in medical colleges were students felt it was fairer,ore relevant and more objective than traditional methods, therefore finding wide acceptance amongst students[11,12,13].

There are definitely logistic challenges in our deployment of OSCE as these were highlighted by poor scores. These included the timing of the stations, organization of the process and student preparation. Other studies have identified these as major challenges in the deployment of the OSCE with most medical schools devising means of localizing the examinations by the introduction of local content and modifications[14,15].

The use of standardized patients (simulators) has been established as a means of instituting objectivity, standardization and reliability to the OSCE. However, these have the tendency of introducing some artificiality to the exams that may not mirror real life clinical scenerios. Use of actors and specifically trained simulators must be standardized to help to create the proper setting to enable the candidate being tested to demonstrate appropriate competence and skill levels either in communication or otherwise. A mean score of 7.8 for stations mirroring real life situations demonstrates the fact that more needs to be done in creating that proper experience for students. A study[12] done at Islamabad Medical and Dental College showed that students felt uncomfortable with the use of simulated patients as opposed to real patients.

In the face of the commencement of OSCE, our study demonstrates that with a high acceptance rate amongst final year medical students, the examination does show good promise though logistics is still a problem as judged by the suggestions for improvement which were mainly in that area.

Conclusion

Five years after commencement of the OSCE as the main means of the final MBBS Surgery examination, the medical students have given a very favourable assessment of its use with the main challenge being students' preparation for the OSCE mode of examination and changes being suggested in improvement in logistics.

Table 2

shows the reason why the students like OSCE

What do you like about the OSCE?

SN		Frequency	Percentage
1	Provides a level ground for all students	22	36.7
2	It has a wide scope	5	5.0
3	Students face the same exams	14	1.7
4	It is objective	45	56.7
	Total	79	100