Deliberate Practice as an Effective Remediation Strategy for Underperforming Medical Students Focused on Clinical Skills: a Prospective Longitudinal Study.

BACKGROUND: It is critical to develop remedial education for underperforming medical students, but little is known about how to create an effective remediation program. Deliberate practice (DP) is a structured and reflective activity that is designed to optimize performance. Here we applied the concept of DP to create remedial education to improve the clinical practices of medical students. We also analyzed the effectiveness of the remediation program.
METHODS: Based on the expert performance approach of DP, we designed a 4-week remedial program for clinical performance that included feedback and reflection. There were 74 student participants in this program from 2014 to 2017. Their clinical performance was re-evaluated after completion, and changes in their clinical performance scores were analyzed.
RESULTS: Students who completed the remediation program showed significant improvements in clinical performance scores (P < 0.001). Most students found the program to be instructive and helpful for improving their clinical performance. They reported that role play with peers was the most helpful for improving their skills.
CONCLUSION: The DP-based remediation program improved the clinical performance of failing medical students. This remediation program should continue to be offered to underperforming students to ensure that medical school graduates are competent.

INTRODUCTION

Medical schools have a responsibility to produce educated, competent physicians who can play important roles in community healthcare. Accordingly, the schools must work to improve the clinical competency of students who perform poorly in school. Most medical students have outstanding academic performance and excellent grades in high school; nonetheless, some struggle in medical school. They have difficulty in integrating large amounts of knowledge, poor time and stress management and interactions with patients and colleagues. Those who fail to meet clinical competency standards need remediation and tailored education, but little is known about effective methodological strategies or how to verify that remediation was successful.123 In addition, teaching staff may be unfamiliar with remediation or with how to give feedback to students.4567

Deliberate practice (DP) is a concept initially described by Ericsson.8 DP provides immediate feedback, protected time for problem solving and opportunities for repeated performances to refine behaviors, and this strategy is strongly associated with the achievement of expertise in a wide variety of fields.910

Few studies have focused on the remediation of medical students, and remediation strategies have not been standardized. Most studies have focused on the remediation of specific skills.1112 DP with mastery learning and rigorous outcome measurement is considered an effective and evidence-based educational method, and the benefits of DP have been documented in medicine for improving surgical skills.11

Based on the principles of DP, we designed a remediation program for improving the clinical competency of underperforming students. Here we report the results of this remediation program as well as feedback from students.

METHODS

Subjects

All 4th-year medical students at Seoul National University College of Medicine take a two-session objective structured clinical examination (OSCE) every March and August to evaluate their clinical performance. The OSCE examination includes six stations, each representing an acute or chronic clinical case. The students perform a 10-minute standardized patient (SP) encounter at each station. Their performances are scored using a checklist that includes six categories: overall assessment, history taking, physical examination, physician's manner, patient education and the physician-patient interaction. Based on the first OSCE scores, we identified students who were performing poorly and who needed remediation to improve their clinical performance.

The remedial program

The remedial course for clinical performance was designed to help students develop the skills used in patient encounters: history taking, physical examination, and clinical reasoning and communication skills. We assessed the competencies and deficiencies of the participants based on each student's OSCE score and on feedback from the SPs as well as by reviewing videos of the SP encounters.

In 2014, we designed a 4-week DP-based clinical performance remediation course using the principles of the expert performance approach of DP as well as feedback and reflection. The learning objectives of remediation program are 54 clinical presentation (CP) such as chest pain, hematochezia or insomina and the level of difficulty was appropriate to achieve for 4th-year medical students. The students had trained in focused and repetitive practices on making schema, history taking and physical examination with rigorous, reliable measurement. During the program, individual and repetitive feedback with monitoring and error correction was given in a small group or in a one-on-one setting. The tutors of remediation program were from multiple medical specialties and they were required to complete workshop on DP and feedback methodologies.

The main learning activities were DP, feedback and reflection. Before each class, students were assigned homework in which they had to make schema and checklists for each CP. During each class, the students checked their schema and checklists using information in the main presentation and then refined their schema and checklists through discussion. Various clinical situations were presented to the students, who then practiced role play with a tutor or with each other. During and after the discussion and role play, the tutors repeatedly provided feedback on each student's schema, history taking, physical examination and physician-patient interaction skills. Tutors also demonstrated history taking, physical examination and information sharing with patients as needed. There were also interactive lectures, SP encounters and video review of various cases. After completing the remediation program, each student’s clinical performance was reassessed using different set of OSCE examination. The changes in clinical performance scores (before versus after remediation) were analyzed.

Students' response and outcomes

The students were asked to complete a post-remediation questionnaire that included the following items: 1) overall satisfaction with the remediation program; 2) whether they would recommend this program to others; 3) which methods were most effective in improving their clinical performance; 4) any other comments or suggestions regarding the remediation program. Students were asked to respond using a five-point scale that ranged from 1 (strongly disagree) to 5 (strongly agree).

To compare changes in OSCE scores between students with versus without remediation, we introduced a standardized T-score. We used this due to variability in the difficulty of the OSCE examination in different years. The T-score was determined as follows:

Data analysis

Statistical analysis was performed using SPSS version 23 (SPSS, Chicago, IL, USA). The changes in Clinical Performance Examination scores with versus without remediation were analyzed by analysis of covariance. A P value < 0.05 was considered to indicate a statistically significant difference. The students' evaluations of the remediation program were analyzed both quantitatively and qualitatively.

Ethics statement

This study was approved by the Institutional Review Board (IRB) of Seoul National University College of Medicine (IRB No. 1901-049-1002) and the board waived the requirement to obtain informed consent.

RESULTS

Participants

Following the first OSCE examination, students who had failed to show minimal competency of clinical performance were obligated to attend the remediation program. From 2014 to 2017, 74 (11.9%) of a total of 620 undergraduate medical students were referred to and participated in the remediation program. The number of participating students each year ranged from 15 to 22 (Table 1); all of the referred participants completed the remediation program. Of the 74 referred participants, 11 were women. The proportion of women students was significantly lower in the remediation group than in the control group (14.8% vs. 39.7%; P < 0.001). Most of the 74 students had multiple deficits e.g. in medical knowledge and in clinical and communication skills. Their mean grade point average (GPA), 2.84, was lower than the mean GPA of the control group, 3.20 (P = 0.001). The clinical performance score and GPA showed a significant positive correlation (r = 0.324, P < 0.001).

Table 1

Characteristics of medical students with or without remediation

Characteristics		Students without remediation (n = 546)	Students with remediation (n = 74)	P value
Year
	2014	135	18
	2015	146	19
	2016	137	15
	2017	138	22
Gender, men:women ratio		329:217	63:11	0.000
GPAa		3.20 (0.54)	2.84 (0.56)	0.001

GPA = grade point average.

aThe GPA value is reported as the mean (standard deviation).

The remediation program took place during the 4th-year students' elective period. During this period, students can perform research at an institution or hospital outside the campus; accordingly, some of participants were very reluctant to participate in the remediation program. However, as the remediation program progressed, most of the students were willing to continue to participate as they felt that the program was improving their clinical performance. All 74 completed the program.

Improvement in clinical performance after the remediation program

The mean OSCE score improved significantly more in the remediation group than in the control group (Table 2). As the OSCE scenarios and their difficulty varied, we used a standardized score in order to compare OSCE scores. Comparison of the OSCE scores between the two groups showed that the remediation program improved the clinical performances of students who took part in remediation in all categories such as history taking, physical examination and physician-patient interaction. The average total OSCE T-score of the remediation group was 36.54, which improved to 45.85 following the 4-week program. In the control group, the total T-score changed from 51.79 to 50.54. The scores for history-taking, physical examination and physician-patient interaction all improved following the remediation program.

Table 2

Clinical Performance Examination scores (T-score)

Variables		Students without remediation	Students with remediation	P value
No.		546	74
Total score				0.000
	1st exam	51.79 (8.61)	36.54 (7.75)
	2nd exam	50.54 (9.60)	45.85 (11.29)
History taking				0.002
	1st exam	51.15 (8.74)	41.72 (9.57)
	2nd exam	50.56 (9.60)	45.85 (11.15)
Physical examination				0.001
	1st exam	50.87 (9.62)	43.10 (9.51)
	2nd exam	50.25 (9.82)	47.81 (10.38)
Physician-patient Interaction				0.001
	1st exam	50.97 (9.47)	42.86 (10.03)
	2nd exam	50.42 (9.86)	46.66 (9.73)

Data are presented as average (standard deviation).

Program evaluation

A total of 61 of the 74 (82.4%) students who participated in the remedial course completed the post-program questionnaire (Table 3). General satisfaction with the program was high (4.14/5.0), and most of the students would recommend this program to their friends or to junior students (4.23/5.0). With regard to the specific remediation activities, many students (37.3%) felt that role play with peers (with tutor feedback) was most helpful in terms of improving clinical competencies. The students also indicated that the SP encounters and tutor feedback were both helpful following role-playing.

Table 3

Students' responses to remediation program

Variables		Mean	SD
The remedial course was helpful in improving clinical performance skills		4.14	0.81
I would recommend this program to others		4.23	0.88
Most effective methods to improve clinical performance, %
	Role play with tutor's feedback	37.3
	SP encounter	20.3
	Tutor's demonstration of patient encounter	15.3
	Making a schema or checklist of each clinical presentation	15.3
	Study books about clinical practice	10.2
	Lecture	1.7

Mean and SD derived from 5-point Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neither disagree nor agree, 4 = agree, 5 = strongly agree.

SD = standard deviation, SP = standardized patient.

DISCUSSION

These findings demonstrate that a DP-based remediation program involving feedback was effective in improving the clinical skills of undergraduate medical students. This 4-year longitudinal study found that the program improved the students' clinical performance and that the improvement was not confined to specific skills. The intervention aimed to help medical students attain clinical skills and improve their approach to common clinical problems by engaging in DP. DP involves 1) the repetitive performance of psychomotor skills in a focused domain, coupled with 2) rigorous skills assessment that provides learners with 3) specific, informative feedback, resulting in 4) improved skills performance in a controlled setting.1314 The results of prior studies also show the importance of DP versus innate ability for the acquisition of mastery-level skills in specific domains; indeed, the use of DP is a much more powerful predictor of professional accomplishment than experience or academic aptitude.13 Practice with reflection and feedback is the key rule of this educational intervention.

In our study, the learners were highly motivated, since they had to pass the clinical performance examination in order to graduate and the learning objectives were well-defined. The class size was relatively small, so the interactions between tutors and students were sufficient for evaluating performance and providing feedback.

Before the introduction of the DP-based remediation program, we tried other programs that were based on, for example, didactics or SP encounters, in order to increase the students' patient encounter experience. Notably, without an appropriate intervention, poorly performing students are more likely to fail again if they do not change their unsuccessful learning methods.1415

Our study has several limitations. First, it was performed at a single institution. There are conditions and circumstances that are specific to each medical school, so our results may not be generalizable to other institutions. Second, we focused on the DP method; we did not consider the students' individual issues, such as possible mental health issues. Therefore, a poorly performing student with economic problems or a psychiatric issue such as depression should be cared for on a case-by-case basis in an appropriate way. Third, we selected poorly performing students based on OSCE scores; thus, we may not have included all learners who perform poorly in real world situations in which their clinical performance might differ from their performance on the OSCE. In addition, as standardized T-score is dependent to the average score and standard deviation, there might be a limitation in accurate re-assessment of students' clinical competency using T-score, although we set 6 OSCE station evenly selecting among diverse CPs, not focusing on specific CP, and average difficulty index in each year was similar. To show improvement of students' performance in remediation group, we also had shown the improvements of OSCE scores in remediation group in all categories including PPI which has common checklist items constant throughout 4 years in all OSCE examinations. Finally, we did not analyze the influence of other factors, such as each student's specific deficits or their willingness to participate. Some deficits could be corrected easily, while others could not.

In conclusion, this DP-based remediation program improved the clinical performance of medical students. The program should be analyzed further to better understand what made it effective, and it should continue to be developed for students who are not performing well in medical school.