Summative objective structured clinical examination as a reference of learners' need before entering their internship.

BACKGROUND/
PURPOSE: Objective structured clinical examination (OSCE) has found increasingly wide use in the education of clinical dentistry in recent years. The objectives of this study were to find the predictability of OSCE in representing a student's actual ability in a clinical environment and to provide instructors with better ideas to guide an individual student in a certain field.
MATERIALS AND METHODS: Thirty-one dentistry students' qualification OSCE scores were obtained after the conclusion of their dental clerkship in December 2014. The OSCE scores were compared with the various scores given by instructors from different departments during their internship a year later. The correlation was investigated between these two scores.
RESULTS: The scores of the first station of OSCE-explaining a treatment plan for the restoration of a missing tooth-were correlated with the scores of prosthodontics in the internship. The scores of the fourth station of OSCE-explaining to a mother regarding the obtainment of a radiograph of her child's tooth-were correlated with the scores of orthodontics. In addition, the total score of the six OSCE stations was correlated with the scores of orthodontics, oral surgery, and pediatric dentistry during dental internship.
CONCLUSION: The results of clerkship qualification OSCE could provide instructors with insights into the needs of the students before their entering internship.

Introduction

Many issues have been widely discussed in the field of clinical dental education, including core competencies, curriculum development, and the evaluation systems of clinical training.1, 2, 3, 4 Patient-centered medical care has been shown to be the most important core value in clinical education by results from various studies. Memorization of medical knowledge is no longer the mean assessment of clinical competence, and clinical skills and doctor–patient interactions should also be included in the evaluation of a student's clinical ability. Objective structured clinical examination (OSCE) has been commonly used in education and license exams around the world. It has also been used in dental education in recent years.5, 6, 7, 8 The results of OSCE can provide educationists with valuable information regarding the planning of clinical dental education programs even though no correlation has been found between training outcome evaluation and future clinical performance.9, 10, 11, 12 Numerous related studies have been conducted in Taiwan, and the aim of this study is to investigate the correlation between OSCE and the results of clinical training during internship among dentistry students.13, 14, 15 Thus, the following aspects are evaluated: (1) the predictability of OSCE in representing a student's actual ability in a clinical environment and (2) whether OSCE could be used to provide instructors with better ideas to guide an individual student in a certain field.

Materials and methods

This is a retrospective cohort study, which included 31 dental students from National Taiwan University in 2014. The scores obtained from OSCE in December 2014 after the conclusion of these students’ dental clerkship (D5) were compared to the various scores given by instructors from different departments during their dental internship (Ri) a year later (2015). If a correlation existed between these two scores, it was investigated.

Introduction of qualification OSCE of dental clerkship

The qualification OSCE was designed and organized by well-experienced clinical instructors from the Department of Dentistry, National Taiwan University Hospital. Six 10-minute OSCE stations were designed: Station 1(S1), explaining a treatment plan for the restoration of a missing tooth; Station 2(S2), taking a study cast impression with alginate; Station 3(S3), explaining a treatment plan for restoration of an inlay; Station 4(S4), explaining to a mother regarding the obtainment of a radiograph of her child's tooth; Station 5(S5), placing a rubber dam on a designated tooth; and Station 6(S6), explaining pulpitis diagnosis and treatment.

A checklist was used at each station as an assessment tool. The students were evaluated according to a pre-existing assessment criterion by one appraiser at each station. The scores on the checklist were then converted into a percentage system to better understand the clinical abilities of every student.

Internship training program of school of dentistry, National Taiwan University

The internship training program at National Taiwan University (NTU) included 7 sub-departments: endodontics (ED), orthodontics (OT), oral and maxillofacial surgery (OS), oral diagnosis (DX), prosthodontics (PS), periodontics (PR) and pediatric dentistry (PD). Instructors in each department designed a different training course to provide the students with specialized skills. At the end of training, the instructors evaluated each student based on their professional knowledge, clinical application skills, professionalism and technique. The percentage system was used for grading; with 100 percent being the highest score and 60 percent being the pass mark.

Statistical analysis

SPSS version 11.0 for Windows (SPSS Inc., Chicago, IL, USA) was used for statistical analysis. First, the average and standard deviation of the scores of every student at each station of OSCE and the total score of OSCE were calculated. The average and standard deviation of the different scores of provided by each department during internship training were also calculated. Second, Spearman rank correlation coefficient was used to determine the correlation between the scores from OSCE and those from internship training. If a correlation was found, simple linear regression analysis was used to detect if a linear relationship exist between the two groups. Lastly, multiple regression analysis was utilized to design a forecasting model for the scores of internship training using the scores of OSCE as a self-variable.

Results

A total of 31 dental students was included in this study, 19 (59.4%) male students and 12 (40.6%) female students. They were aged from 22 to 32 years, with a median age of 24 years (Table 1). The performance of these 31 students showed that the third station of OSCE — explaining a treatment plan for restoration of an inlay — had the highest scores (mean ± standard deviation [SD], 81.46 ± 11.19). The sixth station of OSCE — explaining pulpitis diagnosis and treatment — yielded the lowest scores (mean ± SD, 61.67 ± 12.69) (Table 2). In the dental internship group, among differential departments, oral and maxillofacial surgery (OS) had the highest scores (mean ± SD, 91.06 ± 3.08) and prosthodontics (PS) had the lowest score (mean ± SD, 83.35 ± 10.07) (Table 3).

Table 1

Demographic data of participants.

		No. of participants	Percentage
Gender	Male	19	59.4%
	Female	12	40.6%
Age	≤23	9	29.0%
	23–27	18	58.1%
	≥28	4	12.9%

Table 2

Means and standard deviations of the checklist scores of dental objective structured clinical examination in dental clerkship (D5) group.

Station	Mean	Standard Deviation
S1	70.56	12.71
S2	81.33	12.52
S3	81.46	11.19
S4	77.38	15.25
S5	78.75	12.78
S6	61.67	12.69
Total score	451.14	38.66

Station 1— Explaining a treatment plan for the restoration of a missing tooth.

Station 2— Taking a study cast impression with alginate.

Station 3— Explaining a treatment plan for restoration of an inlay.

Station 4— Explaining to a mother regarding the obtainment of a radiograph of her child's tooth.

Station 5— Placing a rubber dam on a designated tooth.

Station 6— Explaining pulpitis diagnosis and treatment.

Table 3

Means and standard deviations of the training course scores in dental internship (Ri) group.

Training course	Mean	Standard Deviation
ED	85.55	4.51
OT	89.62	3.56
OS	91.06	3.18
DX	87.66	3.75
PS	83.35	10.07
PR	90.15	6.76
PD	89.35	4.40
Total score	3801.63	290.08
Average score	82.64	6.31

Training course: ED: Endodontics, OT: Orthodontics, OS: Oral and maxillofacial surgery, DX: Oral diagnosis, PS: Prosthodontics, PR: Periodontics, PD: Pediatric dentistry.

The scores of the first station of OSCE — explaining a treatment plan for the restoration of a missing tooth—were correlated with those of prosthodontics (PS). The scores of the fourth station of OSCE—explaining to a mother regarding the obtainment of a radiograph of her child's tooth—were correlated with those of oral diagnosis (OD) and orthodontics (OT). Last, correlations could be found between the total score of the six stations of OSCE and the scores of OT, OS, and PD during dental internship training (Table 4). A linear relationship between these scores could not be found after further analysis.

Table 4

Spearman Rank Correlation Coefficient between the scores from clerkship (D5) OSCE and internship (Ri) training.

	Station 1		Station 2		Station 3		Station 4		Station 5		Station 6		Total Score
	r	p-value	r	p-value	r	p-value	r	p-value	r	p-value	r	p-value	r	p-value
ED	0.03	0.8810	0.09	0.6548	0.31	0.1009	0.24	0.1938	−0.02	0.9183	0.11	0.5790	0.26	0.1724
OT	−0.08	0.6615	0.25	0.1848	0.13	0.4999	0.41	0.0243*	0.19	0.3049	0.20	0.2894	0.38	0.0387*
OS	0.06	0.7559	0.26	0.1708	−0.11	0.5718	0.33	0.0793	0.27	0.1520	0.22	0.2461	0.36	0.0483*
OD	0.37	0.0451*	0.005	0.9765	0.04	0.8485	−0.19	0.3162	0.32	0.0879	0.01	0.9445	0.15	0.4193
PS	0.37	0.0418*	0.09	0.6444	0.26	0.1603	−0.03	0.8723	0.21	0.2646	0.11	0.5529	0.33	0.0726
PR	0.15	0.4428	0.23	0.2149	0.20	0.3007	0.12	0.5282	0.09	0.6423	0.16	0.3904	0.28	0.1324
PD	−0.14	0.4554	0.22	0.2387	0.35	0.0576	0.31	0.0976	0.26	0.1610	0.33	0.0717	0.44	0.0144*
Total Score	0.27	0.1432	0.11	0.5480	0.23	0.2266	0.10	0.5669	0.16	0.4096	0.13	0.5095	0.32	0.0801

ED: endodontics, OT: orthodontics, OS: oral surgery, OD: oral diagnosis, PS: prosthodontics, PR: periodontics, PD: pediatric dentistry.

*: p-value < 0.05.

The results of multiple regression analysis showed that the scores of Station 5 was the only variable that was strong enough to predict the grades of future internship (R Square = 0.1767, F = 6.01, p = 0.0207).

Discussion

This study had a limited number of samples, with only 31 students included. More students are needed for a more comprehensive result. This study is a cohort study, which lacked control groups. To remedy this situation, further studies are required, and more extensive data, such as feedback from students and instructors, may also be useful in further evaluating the applications of OSCE in clinical dental education.

There are two different measurement tools applied in this study and it is uncommon in pretest-posttest research design. However, our hypothesis is that previous summative OSCE could be learners' need assessment of following learning stage. We would like to see if learners’ performance better or worse in summative OSCE is correlated with future learning outcome in some specific aspects. In other words, to evaluate the predictability of the summative OSCE, we certainly have to use different measurement tool even with more detailed information, such as learning outcome of the following stage, to establish the predictability.

Because this study focused on the transition from dental clerkship to internship. During dental clerkship training, teaching strategy consists of the simulation training, model practice and sometimes real patient encounter. Under the concern of the comprehensive assessment of knowledge, attitude and skill, OSCE is a reliable measurement tool with good representability for clerkship summative assessment. However, dental internship training consists almost all real patient practice and department scores therefore provide more detailed information and better reliability than an OSCE. This is also an important reason that this study utilize a more detailed measurement to evaluate the predictability of a brief, general measurement tool in terms of research design.

The results of this study showed that the OSCE grades on restorative dentistry at the end of clerkship correlated with the grades on the same topic during their internship in the following year. This result was expected because the assessment qualities of OSCE were similar to the important skills needed during internship training on the same topic. Furthermore, the correlation of the scores acquired at Station 4 of OSCE concerning imaging in pediatric dentistry with those in OD may have been caused by the two topics sharing similar techniques in interpreting dental images. Correlation analysis also showed that the total score of OSCE had a positive correlation with the grades of OD, OS, and PD during internship. These departments focused more heavily on doctor–patient communication and professionalism, which were the same traits that were assessed during OSCE, which may be the reason why these scores were highly correlated. This finding can alert instructors in OD, OS, and PD to more closely monitor and provide guidance to students with low total OSCE scores. The results of multiple regression analysis showed that only the scores at the fifth station of OSCE can predict 17.67% of the grades during internship training, which implies the existence of more unknown factors influencing the outcome of dental clinical training.

Conflicts of interest

All authors deny any conflicts of interest related to this study.