A Systematic Approach to Working With Medical Learners in Difficulty: A Faculty Development Workshop.

Introduction: For medical educators, applying a systematic approach to working with struggling learners (learners in difficulty) can improve faculty success and satisfaction with the remediation process. Use of the familiar SOAP diagnostic framework can ensure that faculty develop a thorough differential diagnosis and target their interventions to address underlying issues affecting learner success.
Methods: We developed a workshop to teach medical education faculty essential skills for supporting learners in difficulty. Teaching methods included didactic presentation, large-group discussion, and small-group work with role-plays. Over three 2-hour sessions, participants learned to assess a learner in difficulty, develop an initial remediation plan, and evaluate their learning system with the goal of improving support to learners in difficulty. Evaluation included pre- and postsession assessment of learner self-perceived confidence and skill with working with struggling learners, as well as brief postsession evaluation.
Results: Ninety-nine faculty participated in the Learners in Difficulty workshop over 7 years. Participants' overall rating of the workshop was 4.9 (1 = poor, 5 = outstanding). Pre- and postworkshop evaluation showed a statistically significant increase in perceived self-confidence to "Meet the needs of a struggling learner," from an average of 4.4 to 7.6 on a 10-point scale (mean Δ = 3.2; 95% confidence interval, 2.6-3.8; p < .001). Discussion: This workshop provides a stepwise approach to working with learners in difficulty and assessing participants' educational systems to identify strengths and weaknesses. Evaluations indicated participants felt more confident in their ability to engage in this topic following completion of the workshop.

Educational Objectives

By the end of this workshop, participants will be able to:

Use a common clinical model for assessing a learner's problem behaviors.

Describe the steps involved in identifying, assessing, and addressing problem behaviors.

Explain the rationale for partnering with the learner to work on the issue together.

Develop a learning plan.

Demonstrate the application of a learning plan through role-play.

Analyze their learning systems regarding support of learners in difficulty.

Introduction

All medical schools and residency programs have subsets of learners who struggle with some aspect of their learning or clinical performance. Reviews of residency training programs have found a prevalence of 7%–9% of learners with some kind of difficulty.[1,2] These learners take up a disproportionate amount of faculty time and energy and can be frustrating to work with.[1] Learning a systematic approach to working with learners in difficulty can allow faculty to be both more successful in their remediation and more satisfied with the process.

Literature on working with medical learners in difficulty recommends that clinical teachers use a similar diagnostic approach to that employed with patients presenting with undifferentiated concerns.[3,4] The SOAP (subjective, objective, assessment, plan) model is one such approach. Developing a broad, thorough differential diagnosis is an important step in the process of remediating learners in difficulty. We have found in our own experience that faculty colleagues tend to prematurely anchor their differential diagnoses around areas of medical knowledge and clinical synthesis, whereas a significant number of underlying issues are nonacademic and related to confidence, well-being, response to stereotype threat, or external stressors. Other remediation programs have found similar themes.[5] The existing MedEdPORTAL literature about learners in difficulty (using the search terms problem, difficult, and struggle, with and without the term learner) focuses on remediating academic difficulties (clinical synthesis[6] and test-taking skills[7]) and interpersonal difficulties.[8,9] Our workshop is unique in that it uses a holistic approach, starting with describing the problem behavior and generating a differential diagnosis before creating a remediation plan. A holistic approach of zone-based remediation has also been described recently.[10]

We describe a workshop conducted at the University of California, San Francisco (UCSF), Family Medicine Faculty Development Fellowship to teach skills for working with learners in difficulty. The workshop has been updated and refined over several years and is appropriate for faculty at all levels who work regularly with clinical learners.

Through the workshop, participants learn to assess a learner in difficulty using a systematic approach; develop an initial remediation plan including discussing the plan with the learner; and examine elements of their learning system, which, if improved, could also help improve the support of learners in difficulty.

All sessions use a combination of didactic presentations, large- and small-group discussion, and role-play to teach these skills.

Methods

Our Learners in Difficulty workshop (LID workshop) was part of the Northern California Faculty Development Fellowship, which consists of 24 full-day sessions over 8 months. We conducted the workshop annually from 2011 to 2018 for 99 participants from 12 academic and community-based primary care residency programs and one osteopathic medical school in Northern California. Each LID workshop was delivered to a group of 15–20 faculty members, using two to three facilitators, and was conducted over 1 day.

It is beneficial but not essential for participants to be familiar with both feedback models (including a focus on learner-centered and appreciative approaches)[11-15] and systems improvement approaches[16] prior to the LID workshop. Facilitators should have personal experience in working with learners in difficulty and developing remediation plans. Because much of the workshop is conducted in small groups, there should be a minimum of one facilitator per 10 participants.

The LID workshop centered around four key skills that help faculty members work more systematically with learners in difficulty. The first three skills (constructing a differential diagnosis for the observed behaviors, generating target behaviors for the learner, and drafting strategies that the learner can use to reach the target behaviors) form the groundwork/preparation that the faculty do before meeting with the learner. The fourth skill is a framework for the initial discussion (intervention conversation) with the learner.

Consistent with contemporary adult learning theory, we included a variety of educational approaches in the LID workshop, including large-group work to more easily cover didactic information and small-group work to facilitate active participation. The use of participants’ own cases during the workshop increased the relevance and applicability of the material for each learner. The LID workshop in its full format was presented over 6 hours. Considering that not all facilitators will have this much time, we have included instructions in the facilitator guide (Appendix A) for condensing the material into shorter versions, including a 1.5-hour presentation with only large-group participation. Furthermore, we have broken the workshop into three separate components, parts A-C (see Appendix A), which can be done individually or longitudinally.

Facilitators used the facilitator guide (Appendix A) to prepare for the workshop and created the PowerPoint (Appendix B) that was used throughout the presentation. In advance of the workshop, we asked participants to reflect on a learner in difficulty they had worked with and to answer a series of brief questions in the guidelines for developing a case (Appendix C). We also provided additional worksheets and tools, both print and digital versions, for participants to use during the workshop (Appendices D–K).

We began the workshop with a large-group discussion that introduced the objectives of the workshop, using the annotated PowerPoint presentation (Appendix B). We then solicited examples of learners in difficulty from participants (Appendix C) and wrote brief summaries of the case examples using a flip chart and markers. We chose one case to use as an example, and in a large-group discussion, we used the PowerPoint and flip charts to both describe the groundwork/preparation components and create those components for the example case. Having seen the process for the example case, participants broke into small groups and took turns generating differential diagnoses, target behaviors, and strategies for their individual cases, using their small groups as advisory committees. Participants used a handout presenting four key skills (Appendix D), a list of suggested interventions by differential diagnosis area (Appendix E),[17] and the small-group instructions (Appendix F) as they worked through their cases. The small groups used laptops/tablets to access the appendices and take notes about their individual cases. Alternatively, facilitators could distribute the appendices as paper handouts, and the small groups could use notepaper and pens to work on their cases.

The large group reconvened, and we described the framework for the intervention conversation. We asked one participant to role-play the conversation from the participant's personal case in front of the large group, with another participant playing the role of the learner (instructions contained in the PowerPoint and in Appendix F). The small groups then did their own role-plays with their individual cases.

As a large group, we asked participants to reflect on the skills they had learned and their experiences in working through their own cases. We then introduced a full learning plan for learners who struggle in multiple domains (Appendix G) and showed a learning plan template (Appendix H) and sample learning plan (Appendix I). We followed with a large-group discussion of systems-level components important for the success of remediation efforts (Appendix J).

At the conclusion of the workshop, participants set personal goals for implementing the skills learned in the workshop (Appendix K) and completed an evaluation of the workshop (Appendix L) using an online, anonymous survey platform (Qualtrics).

Evaluation instruments were self-designed and were validated internally by an expert in faculty development who provided feedback and suggestions. The evaluation included pre- and postsession surveys to measure self-perceived confidence and skill with working with learners in difficulty, using a 10-point Likert scale. Additionally, we created postsession evaluations to seek learner perception of the importance of the topic and the quality of the presentation. This was done on a 5-point Likert-type scale (Appendix L). Pre- and postsession assessments were analyzed using a t test with a cutoff of p < .05. Learners’ overall perceptions of the workshop, including strengths and areas for improvement, were collected anonymously and evaluated for themes by the workshop leaders.

Results

The first iteration of this curriculum was conducted in 2011. A total of 99 faculty from more than 10 academic and community-based family medicine residencies and one osteopathic medical school participated in the curriculum between 2011 and 2018. Based on initial feedback, expansion of the curriculum was made in 2014, and the curriculum in its current format has been offered to 55 participants. Participants had little to no exposure to formal training on working with learners in difficulty prior to the workshop. Measures of the current format included participant satisfaction with the curriculum, as well as pre- and postparticipation self-efficacy. Participant ratings of satisfaction on a 5-point scale (1 = poor, 5 = outstanding) were high. The average participant rating for the prompt “The differential diagnosis framework and 4 Key Skills are a clear, useful tool for approaching learners in difficulty” was 4.8. For the prompt “I am more confident in my ability to work with learners in difficulty,” the average rating was 4.6. Finally, the average response to “Please indicate your overall rating of the Learner in Difficulty workshop” was 4.9.

Participant ratings of pre- and postworkshop experience, summarized in the Table, revealed a statistically significant increase in perceived self-confidence from an average of 4.4 to 7.6 on a 10-point scale (1 = little confidence, 10 = high confidence; mean Δ = 3.2; 95% confidence interval [CI], 2.6–3.8; p < .001). Similarly, participants significantly increased from an average of 2.4 to 3.9 on a 5-point scale (1 = limited skill, 5 = extensive skill; mean Δ = 1.5; 95% CI, 1.2–1.7; p < .001) on their self-assessed ability of “Dealing with learners in difficulty.”

Table.

Mean Participant Ratings With 95% CIs on Pre-/Postsurveys

	Preworkshop(N = 54)		Postworkshop(N = 48)
Item	M	95% CI	M	95% CI	p
1. How confident are you to meet the needs of a struggling learner?a	4.4	4.0–4.9	7.6	7.3–8.0	<.001
2. Assess your current skill level in dealing with learners in difficulty.b	2.4	2.2–2.7	3.9	3.7–4.0	<.001

Abbreviation: CI, confidence interval.

Rated on a 10-point scale (1 = cannot do at all, 10 = certain can do).

Rated on a 5-point scale (1 = limited skill, 5 = extensive skill ).

Strengths of the workshop were noted to be the systematic approach, use of specific tools, and use of real-world cases to enhance applicability. Areas for improvement were noted to be the desire for more time to work on the case studies and the desire to solicit more input and advice from session facilitators. A thematic analysis of participant comments is summarized next.

Systematic approach:

“Having a systematic process to diagnose a learner in challenge was a huge help.”

Specific tools:

“The SOAP approach and 4 step approach were intuitive and clear.”

“The differential diagnosis handout will anchor my approach.”

Utilizing real-world cases:

“Working with a group with an actual case and working through the process of having an intervention conversation was hard and helpful.”

“Using our own cases made a big difference.”

More time for engaging in case studies:

“I wanted more time to work on my own case and hear about other cases.”

“It was great to struggle through my own case, but I wanted more time.”

Expanded opportunity to seek advice from experts:

“I would have liked time to ask more questions from [the instructor].”

“Wish we had more time to problem-solve with [the facilitators].”

“The session was great, but I feel like I need ongoing support as I work with learners who are struggling.”

Discussion

To address the need for more robust systems to support learners in difficulty, we designed, conducted, and evaluated this three-part workshop. The first two parts aim to strengthen clinical faculty members’ ability to identify and develop plans to address learner struggles. The third and final portion of the workshop is targeted to medical educators responsible for examining and improving systems to support our learners in difficulty.

In our workshop, we teach educators to apply the same clinical reasoning skills they use in diagnosing and managing their patients to assist their learners in difficulty. Anchoring the assessment of learners in a familiar framework (the SOAP approach) allows educators to quickly master the material and feel confident in its application to learners. Additionally, by incorporating cases of real learners with whom the educators are currently working, we help educators grasp the relevance of our material to their work and immediately apply practical ideas to common teaching situations.

When we first started teaching this workshop, adapting it from materials used by our predecessors, the overall framework was similar; however, we had not yet divided it into discrete sections. We quickly learned that participants found the framework unwieldy and difficult to learn and apply. We also saw that the practical approaches we used with our own learners in difficulty deviated from the framework we were teaching. Therefore, in 2014, we made substantial changes to the curriculum, generating the Four Key Skills model we currently use in the workshop and dividing the initial work with the learner (part A) from a more complex learning plan (part B) and a consideration of the learning system (part C). We also learned that to facilitate the sense that this work was manageable, we needed to include stories of success and best practices, and we therefore added a large-group reflection session at the end of the workshop for participants to share successes. Reciprocally, we have also used feedback from participants in the workshop to improve our own remediation work at our programs. This Four Key Skills model has not been described previously.

Results from surveys of participants about the workshop at the UCSF Family Medicine Faculty Development Fellowship revealed that participants rated the workshop highly and that it significantly boosted confidence and self-perceived ability of working with learners in difficulty.

One strength of the curriculum is its three-part structure. The three progressive parts allow session leaders to choose portions of the curriculum to effectively meet participant needs based on their own roles. Part A is relevant to all medical teachers, whereas parts B and C are directed at more advanced educational faculty. Furthermore, parts A-C can be conducted in a single, daylong workshop or longitudinally over three sessions.

One limitation of this workshop is that the evaluation is focused on participant self-assessment of skills and confidence and does not include learner-centered outcomes. It would be difficult to measure outcomes of learners in difficulty at our fellows’ institutions before and after participation in the workshop. An observed-structured teaching evaluation is a potential alternative assessment tool that would allow workshop instructors to observe and provide feedback on skills and behaviors taught during the workshop. Additionally, we did not directly evaluate our objectives as articulated.

This workshop does not address learning disabilities or difficulties with language fluency, both of which are factors that may impact learner performance but are beyond the scope of this workshop. Also of note, the workshop leaders and most of our participants are faculty in graduate medical education programs. Faculty working primarily with medical students may have other resources to address performance issues.

Working with learners in difficulty can be a complex topic, and assessing faculty and system readiness for this topic is important. For clinical faculty with foundational skills in giving feedback, development of a differential diagnosis and an initial remediation plan would be appropriate. The portion of the workshop on improving systems is more appropriate for medical educators in leadership roles who have some experience with system improvement initiatives. Workshop instructors are encouraged to begin with the resources presented and adapt them to their personal environments and the needs of their specific participants.

Future possible directions include expanding the case-based component of the workshop to allow the participants to bring their cases, with proposed target behaviors and strategies, back to the large group for further group and expert input. Another possibility is to hold a follow-up session 3–6 months after the initial workshop to discuss case updates and outcomes. Finally, participants could be asked to do an assessment of their current system and offer recommendations or be asked to implement changes to improve their systems.

A. LID Facilitator Guide.docx

B. Learners in Difficulty Presentation.pptx

C. Guidelines for Case Development.docx

D. Four Key Skills.docx

E. Interventions by Differential Diagnosis Area.docx

F. Small-Group Instructions.docx

G. Full Learning Plan How-to.docx

H. Learning Plan Template.docx

I. Learning Plan Example.docx

J. Building a Successful System.docx

K. Postworkshop Goal-Setting Form.docx

L. Pre- and Postworkshop Assessment & Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.