Orthopaedic Resident Remediation: Frequency, Interventions, and Outcomes.

BACKGROUND: Despite multiple studies in other areas of surgical resident education, there is no current literature regarding orthopaedic resident remediation. The goal of the present study was to use a survey format to determine the frequency of remediation, the underlying etiology, the intervention strategies utilized, and the outcomes.
METHODS: The program directors of all current 159 Accreditation Council for Graduate Medical Education-recognized orthopaedic residency programs in the United States were e-mailed a non-identifying digital survey. Follow-up e-mails were sent monthly for a 3-month period. The data remained anonymous, with no identifying information. The data are reported with use of descriptive statistics.
RESULTS: Seventy (44%) of 159 program directors responded; most were from academic institutions with 4 to 5 residents per class. One hundred and fifty-eight residents were remediated, with the greatest number being remediated during the third postgraduate year (PGY-3). Professionalism, patient care, and communication were the most common deficiencies requiring remediation. Mentorship, feedback, and probation were the most common interventions. Of the 117 residents for which the outcome was reported, 58 graduated on time, 14 graduated from another program, 25 graduated from another specialty, 14 were terminated, 3 pursued litigation, and 3 chose a non-medical career. Rehabilitation, didactics, feedback, and mentorship were associated with the highest rates of on-time graduation.
CONCLUSIONS: Most remediated residents were PGY-3, suggesting increased scrutiny as residents moved from junior to senior responsibilities. The greatest number of deficiencies requiring remediation pertained to the affective domain, which highlights the importance of screening measures such as away rotations and interviews. The relationships formed during increased feedback sessions and mentorships can help problem residents to graduate on time.

The American Board of Internal Medicine defines a problem resident as “a learner who demonstrates problem behaviors significant enough to require intervention by program leadership….”[1] The problem could present in various aspects of performance, including emotion management, knowledge base, social interaction, and surgical skills. This intervention could be in the form of remediation, defined as “intending to correct or improve one’s skill in a specific field.”[2] In an effort to prevent the need for future remediation, researchers have attempted to identify selection criteria to predict a successful orthopaedic resident. Although no consensus exists, United States Medical Licensing Examination Step 1 score, honor clerkship grades, motor ability, and affective domain characteristics have been correlated with resident success[3,4].

The Accreditation Council for Graduate Medical Education (ACGME) adopted 6 core competencies in 1999 to standardize the evaluations and expected performance of residents and faculty[5]. This framework for objective evaluation has led to an increase in publications regarding potential strategies to identify and help problem residents. In the Clinical Competency Committees: A Guidebook for Programs, the ACGME provides options for remediation such as intensive mentoring, additional readings, skill laboratories, added or repeated rotations, extended education, or counseling to consider another specialty or profession[6].

Surveys sent to program directors have been utilized to gauge trends and strategies currently being implemented in other areas of medicine, including internal medicine, otolaryngology, and general surgery[7-9]. However, we are not aware of any current literature from organized committees or national surveys regarding orthopaedic surgery resident remediation. The goal of the present study was to utilize a national survey to determine the frequency of remediation, the underlying etiology for remediation, the intervention strategies utilized, and the outcome for orthopaedic residents.

Materials and Methods

The program directors of all current 159 ACGME-recognized orthopaedic residency programs in the United States were sent a non-identifying digital survey via e-mail. No programs were excluded. After the initial e-mail, follow-up e-mails were sent every month for a 3-month period to the program directors who had not responded. The survey had a check-box format that included the option to select all answers that applied or to include free text when indicated (see Appendix). The first question pertained to the total number of residents remediated at said program during the last 10 years. The remaining questions (questions 2 through 8) were asked for each resident reported in the first question. For example, if 3 residents had been remediated during the last 10 years, then the answer to Question 1 would be 3. Questions 2 to 8 would then be asked sequentially in 3 different sets to account for each of the 3 residents. No identifying data on individual residents were collected. The data were analyzed with use of descriptive statistics with the assistance of a trained biomedical statistician.

Results

Seventy of 159 program directors responded, for a response rate of 44%. The mean program size was 4.5 residents per year (range, 1 to 9 residents per year). Fifty-eight program directors reported that they had implemented remediation interventions, and 12 reported that no residents had been remediated. Sixty-three respondents were from academic institutions and 7 were community hospitals. Of the 58 directors who reported remediation, 53 were from academic institutions and 5 were from community hospitals.

The results of the survey indicated that, during the last 10 years, 158 residents had required remediation. The highest number of residents remediated from 1 program was 8, and the mean was 3. Of the 158 residents who had been remediated during the last 10 years, 91 had been accepted into the program via a traditional route and 13 had been accepted via a non-traditional route; the route was not specified for the remaining 54 residents. A traditional route is defined as an applicant matching into a first postgraduate year (PGY-1) position of an ACGME-accredited orthopaedic program. Most residents requiring remediation were identified during PGY-3 (Fig. 1).

Fig. 1

Bar graph showing the total number of residents remediated during each postgraduate year over a 10-year period.

Communication prompted the most instances of remediation followed by communication and professionalism (Fig. 2). Mentorship, increased feedback, and probation were the most commonly used interventions (Fig. 3). The outcome was reported for 117 residents, with 72 graduating from an orthopaedic program (Fig. 4). Most (58) graduated on time. Of the remaining residents, 25 graduated in another medical field, 14 graduated from another orthopaedic program, 14 were terminated, 3 pursued litigation, and 3 chose a non-medical career. The “select all that apply” aspect of the survey allowed for differences in the number of reported remediated residents, deficiencies, interventions, and outcomes.

Fig. 2

Bar graph showing the total number of residents requiring remediation for each deficiency over a 10-year period. Patient care includes Clinic and OR (operating room). Communication includes Residents, Attendings, Staff, and Patients. Professionalism includes Untrustworthy, Substance abuse, Unreliable, and Relationships.

Fig. 3

Bar graph showing the numbers of reported instances of remediation interventions.

Fig. 4

Bar graph showing the reported outcomes following resident remediation over a 10-year period.

The outcomes associated with each remediation strategy are shown in Table I. The rate of on-time graduation was 66.7% for didactic intervention, 59.5% for feedback, and 58.5% for mentorship. Rehabilitation was associated with the highest rate of on-time graduation (71.1%) but also was associated with the highest rate of termination (28.6%). Leave of absence predictably was associated with the lowest rate of on-time graduation (40.0%), followed by counseling (41.2%). Leave of absence also was associated with the second-highest rate of termination (20%).

Discussion

The response rate of 44% was higher than anticipated but lower than desired. The programs that responded were mostly at academic institutions and included 4 to 5 residents, but a large range of diverse programs was included. These programs averaged 3 residents remediated in the past 10 years. The majority of residents required remediation at PGY-3, indicating concern about graduating residents from junior to senior-level responsibilities. More PGY-5 residents than interns required remediation, suggesting that orthopaedic evaluation is difficult until responsibilities increase. The large jump in remediation from PGY-1 to PGY-2 supports this theory.

As responsibilities increase, the resident’s motor skill, affective domain, and cognitive ability are increasingly scrutinized. However, these characteristics can also be evaluated in medical students. In 2 separate studies regarding resident orthopaedic success, clinical clerkship grades were associated with better performance and better interpersonal skills[3,10]. These findings support the importance of clinical clerkships, away rotations and, to a lesser degree, interviews. Spending a month with an applicant allows evaluators to observe how the applicant handles stress, interacts with staff and peers, and presents himself or herself to others. Unfortunately, there is no panacea, but longer evaluations in varied environments offer the best chance for a program to evaluate the affective domain in the current system. Once an applicant has been accepted into a program, it becomes increasingly difficult to change his or her personality, values, and core beliefs.

However, improving medical knowledge or operative skill may be more feasible. The Orthopaedic In-Training Examination (OITE) provides a standardized, objective way to identify residents who may be candidates for remediation. In the present study, medical knowledge was cited as the reason for remediation in 60 cases, and more residents were remediated for this reason than were remediated for both operating room performance and clinical patient care. This difference may be due to the objective nature of identifying knowledge deficiencies and is consistent with the general surgery literature[11]. Although motor skills are often a focus of aspiring surgeons, the importance of a comprehensive understanding of pathophysiology and anatomy cannot be overlooked.

In their survey-based study of general surgery residency remediation, Torbeck and Canal reported that only 52% to 75% of programs had a specific policy for remediation[8]. Hauer et al.[11] and Smith et al.[12] proposed formal guidelines for remediation with 4 and 3 steps, respectively, applicable to residents in all specialties. Standardization of the process could make these strategies readily available and could decrease the national variation in current practices[11-13].

Although there is a need for formal remediation strategies, preventing and identifying the deficiencies should not be overlooked. In an effort to prevent the need for remediation, the fourth year of medical school could be restructured with a more rigorous curriculum, which could include basic surgical skill laboratories, increased opportunity for patient care, and continued growth with a focus on the affective domain. Several strategies also could be implemented to improve the interview and away rotation processes. Ethical situation rooms could present an opportunity to further evaluate the affective domain. Increasing the number of away rotation positions could allow for better understanding of the quality of the applicant and could lead to fewer applicants on formal interview days as these rotations are essentially month-long interviews. These strategies have not been validated and may not be feasible. However, they could provide a change to the current system and could be investigated further in the future.

A large number of residents who required remediation graduated from an orthopaedic program, with most of them graduating on time. Others were able to transition to another orthopaedic program or another medical field and graduate as well. The low number of terminations and even lower number of reported litigation cases suggest that residents may not have left the field of orthopaedics or medicine on poor terms. Furthermore, increased use of remediation strategies in general surgery residencies has been associated with decreased attrition[14]. These findings may be due to communication and relationships established during the remediation process, specifically mentorship and feedback sessions.

Although the response rate in the present study was only 44%, we believe that this finding was not a limitation, for 2 reasons: (1) a large number of diverse programs were included in the present study and (2) this rate is higher than those in similar published studies in the general surgery and orthopaedic literature[8,10]. However, the present study had many other limitations. The self-reporting nature of surveys and the personal nature of these conflicts could lead to recall bias as well as selection bias. Recall bias may be stronger in cases of affective domain issues due to the personal and potentially egregious nature of these issues; learners who simply needed an additional reading plan may be less memorable to a program director 10 years later.

Another major limitation is the user error of the survey. For example, 58 program directors reported remediated residents, but only 52 program directors answered the follow-up questions specific to their remediated residents. Furthermore, some of the question sets were incomplete for remediated residents. For example, 7 of 10 questions may have been answered for 1 resident while 6 of 10 were answered for another. This limitation could have been mitigated if the survey had been piloted to program directors. Also, a survey format with a “select all that apply” option has inherent limitations such as the potential for multiple, overlapping reported outcomes. These limitations led to the discrepancy in the reported data as seen in the figures.

The survey described in the present report identified the number of residents who had been remediated, the reason for remediation, subsequent interventions, and outcomes. This information may allow program directors to understand the most commonly utilized and effective strategies for resident remediation to better serve residents in need.

Appendix

The full survey sent to program directors is available with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A80).

TABLE I

Outcome Associated with Remediation Intervention

	Graduated on Time	Graduated from Another Orthopaedic Program	Other Medical Field	Termination	Litigation	Non-Medical Career
Increased feedback	59.5%	2.6%	20.3%	13.5%	2.7%	1.4%
Mentorship	58.5%	5.3%	20.2%	10.6%	3.2%	2.1%
Probation	47.7%	3.1%	27.7%	16.9%	3.1%	1.5%
Psychological counseling	41.2%	7.8%	25.5%	17.6%	3.9%	3.9%
Strict guidelines	47.2%	8.3%	19.4%	13.9%	5.6%	5.6%
Didactics	66.7%	3.7%	13.0%	9.3%	3.7%	3.7%
Leave of absence	40.0%	6.7%	13.3%	20.0%	6.7%	13.3%
Learning assessment	55.6%	0	44.4%	0	0	0
Substance abuse rehabilitation	71.4%	0	0	28.60%	0	0