The prospective study of change of perception of postgraduate students on objective structured clinical examination in burns and plastic surgery.

BACKGROUND: In this study, we attempted to assess the change of perception of postgraduate students on objective structured clinical examination (OSCE) in burns and plastic surgery after first five OSCE.
METHODS: A prevalidated feedback questionnaire was used to assess and score the perception of postgraduate students on OSCE in burns and plastic surgery. The opinion of postgraduate students on the feedback questionnaire after first and fifth assessment tests based on OSCE was analyzed. The results were compiled on a data sheet and analyzed using Microsoft Excel and plotted as graphical interpretation. The statistical analysis was done using MedCalc software.
RESULTS: The results of the study showed that there is a positive change in perception of students in favor of monthly assessment based on OSCE in burns and plastic surgery after fifth assessment. The mean students' favorable perception score after the first assessment with was 30.2 ± 2.828 (mean ± standard deviation [SD]) and after fifth assessment, 43 ± 2.828 (mean SD) with Student's test t = 10.119 and P < 0.0001 which is highly significant statistically in favor of OSCE after fifth assessment.
CONCLUSIONS: We observed in our study that the monthly assessment based on OSCE is well accepted by the students of our subspecialty after few assessments; however, further studies are required to augment the evidence. Copyright:

Introduction

The objective structured clinical examination (OSCE) is being conducted monthly in our institution for student skill assessment in all departments. Periodic assessment of postgraduate students of Magister Chirurgiae (M Ch) burns and plastic surgery postgraduate training program of most medical schools based on different methods is a standard norm. These assessments are important in helping a student to become aware about his shortcomings and hence plan a course correction before appearing for the final assessment. It also helps the teachers in medical education to evaluate the level of interest and appreciation of a subject. The OSCE is interesting tool which we are applying at our institution to test the cumulative knowledge and skills acquired by a student. These assessment tools are to be evaluated for the objectives these are intended for. The prevalidated feedback questionnaire is an essential component of the evaluation system tool to complete the loop to assess the efficacy of that tool of assessment.

Materials and Methods

Study design and setting

The prospective comparative study on the perception of postgraduate students on OSCE was conducted in the postdoctoral department of burns and plastic surgery of our institution from February 2019 to April 2021.

Study participants and sampling

The theoretical knowledge and its application in clinical and practical skills of all ten students of our department were assessed by OSCE stations.

Data collection tool and technique

The OSCE stations included a total of four stations with predefined scenario-one each for soft skills, clinical case and planning, clinical radiology, instruments, and pathological specimen. Each station had a structured question which had to be answered by performing specific skills. The observer had to score the students according to the predefined checklist. Immediately after the examination, a prevalidated feedback questionnaire [Table 1] was used to assess the students’ favorable perception score (SFPS) for OSCE in burns and plastic surgery. The written consent from each participant was taken, and the identity of participants has not been revealed at any stage of the study conducted in accordance with the declaration of Helsinki version 2013. The questionnaire used was validated by three independent researchers. The prevalidated questionnaire [Table 1] comprises 15 questions with three response options which are agree, neither agree nor disagree, and disagree. The response agrees, neither agree nor disagree, and disagree carries score of 3, 2, and 1, respectively. The total score from each participant called SFPS by us is calculated accordingly by adding the score against each question. The SFPS for OSCE after first [Table 2] and fifth [Table 3] assessments were compiled on a data sheet and analyzed using Microsoft Excel and plotted as graphical interpretation.

Table 1

Questionnaire for survey on the perception of postgraduate students after first and fifth objective structured clinical examination in burns and plastic surgery

Name (optional): Age: Years: Sex: Male/female
Please tick the response: After first OSCE: ( ), After fifth OSCE: ( )
Question	Agree score 3	Neither agree Nor disagree Score 2	Disagree Score 1
1. Examination conducted was fair?	( )	( )	( )
2. Examination was not stressful?	( )	( )	( )
3. Examination was not difficult?	( )	( )	( )
4. Examination was well structured?	( )	( )	( )
5. Wide knowledge area covered?	( )	( )	( )
6. Questions and procedures asked were fair?	( )	( )	( )
7. Students aware of level of information required?	( )	( )	( )
8. Examination provided opportunity to learn?	( )	( )	( )
9. Examination helped to highlight your weaknesses?	( )	( )	( )
10. Time for examination was sufficient?	( )	( )	( )
11. Score was standardized?	( )	( )	( )
12. Examination pattern is true measure of clinical and soft skills?	( )	( )	( )
13. Attitude of examiner is better?	( )	( )	( )
14. Examination pattern may influence method of teaching?	( )	( )	( )
15. Examination pattern should be used as method of assessment in burns and plastic surgery?	( )	( )	( )

OSCE=Objective structured clinical examination

Table 2

Students’ favorable perception score after first objective structured clinical examination in burns and plastic surgery

Student number	Agree (score)	Neither agree nor disagree (score)	Disagree (score)	SFPS
1	12	16	3	31
2	9	18	3	30
3	15	14	3	32
4	12	14	4	30
5	6	18	4	28
6	12	16	3	31
7	9	18	3	30
8	15	14	3	32
9	12	14	4	30
10	6	18	4	28

SFPS=Students’ favorable perception score

Table 3

Students’ favorable perception score after fifth objective structured clinical examination in burns and plastic surgery

Student number	Agree (score)	Neither agree nor disagree (score)	Disagree (score)	SFPS
1	42	2	0	44
2	39	2	1	42
3	42	2	0	44
4	42	0	1	43
5	39	2	1	42
6	42	2	0	44
7	39	2	1	42
8	45	0	0	45
9	42	0	1	43
10	39	0	2	41

SFPS=Students’ favorable perception score

Statistical analysis

The statistical analysis was done using MedCalc statistical software. MedCalc Software Ltd Acacialaan 22 8400 Ostend Belgium. The paired t-test was used to compare the quantitative data after the first and fifth assessment tests. Statistical significance is set at 5% (P < 0.05).

Ethical consideration

The study (protocol/02/BPS) is done in accordance with declaration of Helsinki version 2013, and the identification of participants is not disclosed.

Results

The opinion of postgraduate students recorded as per the feedback questionnaire [Table 1] after first and fifth assessment tests were recorded, and statistical results were tabulated in Tables 2 and 3, respectively, with subsequent statistical comparison of the mean SFPS after first and fifth assessment [Table 4] using OSCE in burns and plastic surgery. The mean SFPS after the first assessment with was 30.2 ± 2.828 (mean ± standard deviation [SD]) and after fifth assessment 43 ± 2.828 (mean SD) with Student's test t = 10.119 and P < 0.0001 which is highly significant statistically in favor of OSCE after fifth assessment. This shows that there is a positive change in perception of students in favor of assessment based on OSCE in burns and plastic surgery [Figure 1].

Table 4

Comparison of the perception of postgraduate students after first and fifth objective structured clinical examination in burns and plastic surgery

OSCE	Mean SFPS±SD	Student’s t-test	Level of significance (P)
After first OSCE	30.2±2.828	10.119	<0.0001
After fifth OSCE	43±2.828

OSCE=Objective structured clinical examination, SFPS=Students’ favorable perception score, SD=Standard deviation

Figure 1

Line chart of students’ favorable perception score after first and fifth objective structured clinical examination

Discussion

The key components of the learning climate in the medical education system are teacher, learner (student), curriculum, assessment, and evaluation. The evaluation of the learning climate and assessment tool is performed through a feedback system from the learner and teacher. The Accreditation Council for Graduate Medical Education classifies medical competence into six domains: medical knowledge, patient care, professionalism, interpersonal and communication skills, systems-based practice, practice-based learning, and improvement.[1] The clinical vignette-based multiple-choice question paper is considered to be a valid tool for assessing the medical knowledge and its application in decision-making and planning while OSCE for other five domains.[12] Thus, OSCE is considered the gold standard of assessment methods.[3] However, OSCE has two major drawbacks that it is expensive and time-consuming.[4] There are number of studies published which have showed a positive change in perception of undergraduate students in favor of their assessment based on OSCE.[56789] The recent study also showed that the Student's perception was positive especially regarding to organization and the time attributed to each station, and furthermore, the students considered that the topics and questions applied in each station were relevant.[10] Our study has showed a positive change [Figure 1] in perception of students in favor of assessment of students in burns, plastic, and reconstructive surgical super specialty based on OSCE. Since the significance of student feedback for assessment tool in undergraduate and postgraduate medical education is being increasingly recognized,[11] we conducted such study on perception of students of plastic surgery super specialty on objective structured clinical/practical examination.

Our students were generally satisfied with assessment system as indicated by their positive feedback and high mean students’ favorable score after few assessments using OSCE. However, our study was limited by sample size which can be increased in future studies and may need to augment the evidence by conducting multi-institutional or multiple such studies.

Conclusions

We observed in our study that the monthly assessment based on OSCE is well accepted by the students of our subspecialty after few assessments, however, further studies are required to augment the evidence.

Ethical statement

The written consent from each participant was taken and the identity of participants has not been revealed at any stage of the study conducted in accordance with the declaration of Helsinki version 2013.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.