Is there a way for clinical teachers to assist struggling learners? A synthetic review of the literature.

Struggling medical trainees pose a challenge to clinical teachers, since these learners warrant closer supervision that is time-consuming and competes with time spent on patient care. Clinical teachers' perception that they are ill equipped to address learners' difficulties efficiently may lead to delays or even lack of remediation for these learners. Because of the paucity of evidence to guide best practices in remediation, the best approach to guide clinical teachers in the field remains to be established. We aimed to present a synthetic review of the empirical evidence and theory that may guide clinical teachers in their daily task of supervising struggling learners, reviewing current knowledge on the challenges and solutions that have been identified and explored. A computerized literature search was performed using Medline, Embase, Education Resources Information Center, and Education Source, after which final articles were selected based on relevance. The literature reviewed provided best evidence for clinical teachers to address learners' difficulties, which is presented in the order of the four steps inherent to the clinical approach: 1) detecting a problem based on a subjective impression, 2) gathering and documenting objective data, 3) assessing data to make a diagnosis, and 4) planning remediation. A synthesized classification of pedagogical diagnoses is also presented. This review provides an outline of practical recommendations regarding the supervision and management of struggling learners up to the remediation phase. Our findings suggest that future research and faculty development endeavors should aim to operationalize remediation strategies further in response to specific diagnoses, and to make these processes more accessible to clinical teachers in the field.

Introduction

Clinical teachers are faced with the daily challenge of managing patients while simultaneously supervising medical trainees from all levels of training and with varying degrees of autonomy.1 Steinert has defined struggling learners as medical trainees who do “not meet the expectations of a training program”.2 Such trainees, who experience more difficulties, often pose an even greater challenge to clinical teachers, since these learners warrant closer supervision,3 which may be time-consuming and compete with time spent on patient care, but may also require specific teaching skills.4 As a result, the supervision of struggling learners has been associated with reactions from clinical teachers ranging from helplessness to frustration and avoidance,2,5 as clinical teachers often perceive that they are ill equipped to supervise these learners efficiently.6

The discomfort that clinical teachers may feel when supervising struggling learners may have concrete consequences for learners and patients alike. Clinical teachers in the field are often best positioned to detect difficulties early on, due to their privileged access to learners’ actual performances with real patients, and to the fact that struggling learners very rarely step forward themselves to ask for help.7 When difficulties are not addressed as soon as they are detected, undue delays might occur before remediation and closer follow-up can take place.8,9 In certain cases, learners’ difficulties may in fact never be addressed, which may eventually allow less than fully competent trainees to be promoted to independent practice.10–12

Although clearly delineated guidelines for supervising struggling learners could offer a useful solution to these issues, no such guidelines exist, due to the paucity of evidence to guide best practices in remediation.9,13 Intervention studies regarding remediation have often been limited to small single-institution endeavors, and have focused on the remediation of cognitive difficulties.9,14 Although recommendations for remediation at an institutional level have recently been published,15 the best approach for clinical teachers in the field remains to be established.

Therefore, we aimed to present here a synthetic review of the empirical evidence and theory that may guide clinical teachers in their daily task of supervising struggling learners. This article reviews our current knowledge on the challenges and solutions that have been identified and explored.

Materials and methods

Our methodology was based on Grant and Booth’s16 typology of reviews, which defines a literature review as the “examination of recent or current literature”, whose “analysis may be conceptual or thematic” and is “typically narrative”. A search of the relevant literature was performed in both medical (Medline, Embase) and educational (ERIC, Education Source) databases, using the following search terms: underperformance, fail, at risk, difficult or problem, combined with clinical or medical student or resident. The search covered the 1995–2015 period, and was limited to the English and French languages. This search strategy generated 440 titles. After an initial title-based screening, 70 abstracts were read, of which 37 were rejected based on relevance. A total of 33 articles were thus included in this review (Figure 1).

Figure 1

Flow diagram for article inclusion.

Articles were considered for inclusion if they addressed the issue of struggling learners from the perspective of medical teachers in the field, or if they presented empirical evidence or theory that could inform their perspective. Because this review focused on clinical teachers’ daily supervision tasks, articles that addressed the issues of learner difficulties solely from an institutional or systematic perspective were excluded. A meta-analysis was not deemed feasible, due to the wide variety of research contexts and methodologies encompassed, nor was an exhaustive review the purpose of this article. Rather, it aimed to synthesize the most relevant concepts to guide clinical teachers when working with struggling learners.

Results

Theoretical framework

A theoretical framework for addressing struggling medical trainees was provided by Langlois and Thach,17 who first suggested that the SOAP (subjective, objective, assessment, plan) clinical model could be used to structure the approach to learners with difficulties. This model has since been used repeatedly in medical education.18–20 It takes advantage of the parallel often drawn, since Irby21 pointed it out, between the clinical and pedagogical diagnostic approaches.1,22–24 When applied to struggling learners, the SOAP model divides the task into four steps that are well known to clinicians: 1) detecting problems, based on a subjective impression; 2) gathering and documenting objective data, according to diagnostic hypotheses; 3) making a pedagogical diagnosis based on this assessment; and 4) planning a targeted remediation. These steps can guide clinical teachers to approach learners’ difficulties in the same order that they assess patient symptoms. In this review, the relevant literature and recommendations (Table 1) are also organized as per these four main steps.

Table 1

Suggested approach to struggling learners

Subjective: Detecting a problem based on a subjective impression

• Clinical teachers should trust their impressions.

• Doubts should prompt further observation and documentation.

• The goal should be early identification of difficulties, within the first quarter of a rotation.

Objective: Gathering and documenting objective data

• Data should be based on more than one context and on as many observations as possible, depending on available sources: direct or indirect observations, notes in patient files, formal and informal interactions with faculty and staff.

• Predefined milestones and EPAs are useful tools to document objective discrepancies with the expected performance level.

• At least a minimal amount of direct observation is recommended.

• An informal discussion, or a diagnostic conversation, should be held with the learner before any further step is undertaken.

Assessment: Making a pedagogical diagnosis based on assessment of the collected data

• A pedagogical differential diagnosis must consider cognitive, attitudinal, and mental health issues.

• These three types of difficulties are often interrelated. When this is the case, issues must be addressed one at a time, starting with that which has the most impact on clinical performance.

• Difficulties of cognitive origin are the most frequent; among them, both clinical reasoning difficulties and insufficient knowledge should be considered.

Plan: Planning a targeted remediation

• The first step in planning a targeted and efficient remediation is to pinpoint the underlying issue as precisely as possible.

• A clear process for remediation should be in place locally, since its absence could deter clinical teachers from following up on the identification of difficulties.

• Ideally, this remediation process should be integrated into the learner’s regular clinical activities.

Abbreviations: EPAs, “entrustable” professional activities.

Subjective: Detecting a problem based on a subjective impression

Between 10% and 15% of learners will experience significant difficulties during their medical training.25–27 The various steps leading to their formal identification are most often initiated based on the subjective impressions of clinical teachers.7 These subjective impressions, or intuitions, can be either shaped by direct observation of the learner in action or by interacting formally or informally with the student.28 Clinical teachers’ perceptions are indeed considered to be a reliable predictor of learners’ difficulties. Weller et al,3 for instance, found that when clinical teachers were asked to identify subjectively which learners required closer supervision, a reliability coefficient of 0.7 was reached with nine observers, whereas 50 observers were required to reach this same coefficient when more traditional measures were used.

In theory, clinical teachers’ recognition of difficulties in a learner should prompt the next steps of documentation, assessment, and remediation rapidly, since early identification is considered the gold standard to attain.23 In practice, however, the process is very often halted at this stage.2,9,18,29,30 One of the most frequently cited reasons for this, according to many exploratory studies done with clinical teachers, is the discomfort of not knowing which steps should be followed and how to achieve them.6,31–33

The consequence of not acting on clinical teachers’ early impressions is the frequent delay in identifying and addressing learners’ difficulties. Late identification of difficulties was defined by Lacasse34 as identification that takes place after the first quarter of a rotation. When this occurs, practical implications are that less time is left for efficient remediation, but also that critical incidents may take place, with concrete consequences to patients, before a red flag is raised.8,28 A US survey of program directors in internal medicine in 2000 drove the point home by revealing that up to 59% of their residents in difficulty were indeed identified after a critical incident.35 In order to avoid consequences associated with late identification of difficulties, and because their subjective impressions can be considered reliable, clinical teachers should be encouraged to act on their perceptions of difficulties in learners as soon as a doubt arises,34 beginning with the gathering and documenting of objective data.

Objective: Gathering and documenting objective data

According to Bearman et al,30 establishing a diagnosis in medical education consists in “identifying discrepancy between the expected performance standard and the demonstrated performance, and then trying to establish the reason for underperformance”. In many clinical programs, expected abilities at various stages of training are defined through milestones,36 while key tasks in a discipline that learners should eventually be trusted to perform are defined as “entrustable” professional activities (EPAs).37 Milestones and EPAs can both be very useful in identifying and documenting discrepancy. Once it is identified, however, underperformance should be seen as a symptom, not as a diagnosis in itself, which once made could justify ending the observation phase.17,34,38 The first step in addressing difficulties should consist, on the contrary, in further observations to define the issue better28 and to clarify its cause better, in order to determine the most appropriate response.18,19 Further verification of the initial impression formed also helps determine whether the learner is in fact experiencing difficulties that require follow-up or whether the issue was contextual or even an isolated incident.39

Through this collection of data, tangible examples must be gathered. This should be done as much for clinical teachers, who will thus verify their impressions, as for the learner, who is more likely to see credibility in the feedback received,35 and for the institution, who will be able to justify its decisions better.18,34 Objective data are indeed an integral part of a fair approach.28 According to many authors, this initial documentation is the responsibility of clinical teachers in the field, because only they have access to learners’ performances in a real clinical context.30,40,41 The documentation of clinical performance by clinical teachers is essential, since only 2%–6% of struggling learners will self-identify.7,35 Various hypotheses have been put forward to explain why such low rates of learners will themselves ask for help. These hypotheses range from fear of academic and social consequences35 to self-appraisals, which are significantly different from actual performance.42 Kruger and Dunning43 have indeed shown that those who are “unskilled” in a certain discipline are often unaware of it, because the same knowledge is often necessary to be both competent and have an accurate perception of one’s competence in a given field. For instance, participants in the Kruger and Dunning study whose test scores were in the 12th percentile estimated themselves to be in the 62nd percentile. These participants only recognized their errors once their skills had improved.

How to collect pedagogical data

Data collection can be achieved either through indirect or direct observation of learners’ performances. The latter occurs relatively rarely unless procedures or techniques are involved,9,44 but will often be more effective. In a national US survey of the Association of Program Directors in Internal Medicine, a clear majority (82%) reported that problem-learner difficulties had been identified in their institution through direct observation.35 Difficulties, however, can also be inferred from the manner in which cases are presented or from indirect observation,45 but where indirect observation is the only type of observation, certain difficulties could be masked.29 Data can also be collected by reviewing notes written by the learner in patient files or through their formal and informal interactions with staff and faculty. No matter which approach is preferred or available, data should always be based on more than one context and on as many observations as possible.9,46

Many observations are indeed necessary, due to the subjective nature of observations made in clinical settings, but also due to case specificity. As Nendaz et al47 put it, the competence to diagnose varies from one case to another for the same clinician. It is thus more reliable to question a learner more briefly on many cases. For an observed behavior to lead to a diagnosis, it ought to have been observed repeatedly9 and in various types of situations. Indeed, each performance depends as much on the context as it depends on the content,48 and hence Jouquan’s statement49 that the nonmanifestation of a certain knowledge does not systematically equate with its absence [translation].

Hinson et al50 stressed the importance of an additional source of data, which they called “diagnostic conversations”: these conversations consist of “discussing the problem and getting a fuller picture of everything that is going on for that student”. This conversation allows a more accurate pedagogical diagnosis, but may also bring to light potential gaps in perceptions, giving an opportunity for both parties to share the same understanding of a given situation.34 The diagnostic conversation can henceforth be used as a common starting point, which will facilitate a shared appreciation of whether or not progress has been made later on.30 When relevant, it will also allow the taking into account of the learner’s context and reality with regard to the remediation plan, allowing it to be more realistic and increasing the chance that it will be followed.50 In fact, the reasons for the observed difficulties should be determined jointly with the learner, since only the learners hold the whole picture of their personal situation.28,30,38 Faculty can only gain from adding this insider’s perspective to the outside perception of the learner’s professional performance. Diagnostic conversations can take the form of a brief exchange at the end of a shift or of a more substantial discussion in light of all the assembled information to date, but Lacasse34 recommends that at the very least, an informal discussion be held with the learners themselves before any further step is undertaken.

Assessment: Making a pedagogical diagnosis based on assessment of the collected data

Underperformance may be due to one of three main categories of difficulties – related to cognition, mental health, or attitude – among which cognitive difficulties are the most frequent.27,35,51 When considering which category is most likely involved, it must be borne in mind that causes of under-performance are often interrelated.34 A study by Guerrasio and Aagaard,14 for instance, showed that among 53 learners followed for clinical reasoning difficulties, three-quarters had “comorbidities”, ranging from insufficient knowledge (17% of students) to mental health issues (12% of residents). In such cases, the best approach is to address difficulties one by one, beginning with the predominant difficulty for that learner or the one that interferes most with their work.

One prominent finding when reviewing the literature on the causes of underperformance to consider when a learner is in difficulty is the absence of consensus among authors on what the differential diagnosis should include and on how to classify it. Indeed, up to 17 different classification systems of underperformance have been identified in this review. They have been synthesized in one table for the purpose of this article, and are presented in Table 2. This striking disparity of classifications seems to be related to the disparity of sources used by each author to classify learner difficulties. Cited sources vary from regulating institutions, whose classification systems are based on expert consensus7,35,52 to systematic reviews34,53 to compilations of clinical teachers’ narrative descriptions.51,54,55 In a few cases, no source was cited.18,22,56,57 This multitude of classification systems also stems from the fact that none of these systems was reused in its entirety by authors other than the original author. Rather, new classification systems were created each time by adapting or summarizing one or more preexisting classifications.17,19,20,58

Table 2

A synthesis of classification systems of underperformance in medical education

Cognitive causes	Insufficient knowledge7,17,18,20,22,34,35,53–56,58	Insufficient investment in studies17–19,34Learning disorders17,18,22,34,35,54,56Organization related difficulties19,20,34,51,52,54
	Clinical reasoning7,18,34,71	Insufficient knowledge20,34Difficulty organizing information34,51,55,56Clinical reasoning difficulties71
Noncognitive causes	Attitude problems7,18,20,22,35,52,53,57	Ignorance of professional responsibilities20Different values and beliefs17,18,34,55Poor insight and self-regulation18,22Poor social skills7,19,22,52,56Insufficient motivation22,52–56Conflicts in the workplace34,35,54,58
	Affective problems7,17,19,34,53	Anxiodepressive mood disorders17,19,20,34,35,51–55Other mental health issues19,20,22,35,56,58Substance abuse17,19,20,22,34,35,56,58Stress (family, personal, relational)34,35,52–54,58

A quick look at the synthesis of these classification systems in Table 2 reveals that among the main causes of underperformance that are cognitive in origin, a lack of knowledge is cited significantly more often than clinical reasoning difficulties. However, while insufficient knowledge is indeed much more prevalent among medical students in difficulty, clinical reasoning difficulties have been shown to be at least as frequent during residency.7 The fact that its consideration as a cause of underperformance is so markedly less frequent (four authors of 17 cited it explicitly) could also suggest a lack of knowledge transfer from one research field to the other: although clinical reasoning has been the subject of 40 years of research, it may at times be confined to a debate among experts,29 which may be partly due to the hermetic terminology used in cognitive psychology, as suggested by Kempainen et al.59 Moreover, there seems to exist a misconception that clinical reasoning errors are driven by a lack of medical knowledge, rather than an inability to apply that knowledge in clinical practice.60

Plan: Planning a targeted remediation

Remediation was defined by Guerrasio et al7 as “additional teaching above and beyond the standard curriculum, individualized to the learner who without the additional teaching would not achieve the necessary skills for the profession”. As pointed out by Hauer et al’s9 thematic review of the literature on remediation, no standardized and universally accepted remediation processes have yet been developed. Remediation initiatives will thus often resort to ad hoc and uncoordinated processes27,61 or to the sterile repetition of inconclusive strategies.30,62–64 One possible cause for the absence of a standardized remediation process may very well be the aforementioned absence of consensus on the specific causes of underperformance. It is indeed much more difficult to solve a problem that has not been well defined, and the demonstrated disparity among classification systems of underperformance means that there exists at present no consensual basis on which to build common guidelines for managing difficulties.

In practice, obstacles to efficient remediation also arise when the pedagogical diagnosis has not been well established at the outset. Remediation may in fact be fruitless if the previous step, clarifying the diagnosis, has been overlooked.28 Therefore, the first step in planning a targeted and efficient remediation is to have pinpointed the underlying issue as precisely as possible. This point cannot be overstressed, knowing that Bearman et al30 have shown, consistent with our anecdotal experience, that the very idea of matching a remediation intervention with the underlying difficulty is rarely a consideration for clinical teachers in the field. This approach may lead, for instance, to simply adding a supplemental clinical rotation as a remediation strategy for issues related to professionalism. The lack of well-defined remediation strategies for clinical teachers in the field can only (understandably) deter them from naming difficulties that they would not know how to address. Therefore, there is a strong need for further research with regard to remediation that is evidence-based, individualized, and targeted according to pedagogical diagnosis.2,18,35,65,66 In the absence of formal guidelines, however, some guiding principles may be drawn from previous studies on remediation.

In situ remediation

Whereas learners with noncognitive difficulties should be referred, it is generally recommended that cognitive difficulties first be addressed within the learners’ current rotations,9,18,67 an approach that Bearman et al30 termed “in situ learning”. Only if this approach fails should the issue be brought to the next institutional level. Remediation that is integrated with the learners’ usual clinical activities is generally regarded as the most efficient,18,30 because in situ remediation is based on the cognitive psychology principle of experiential learning. First, it allows the learner to anchor new knowledge to skills that are already mastered and to a context that is already familiar.65 Second, it also allows this new knowledge to be contextualized and transferred immediately to an authentic practice setting.68,69 Finally, in situ remediation takes advantage of experiential learning by focusing the learning process on real-life issues that the learner truly has to solve.34 Unfortunately, in practice, a survey conducted by Saxena et al65 among 71 US medical schools showed this integrated form of remediation to be the least utilized, possibly (according to the same study) because of the lack of confidence clinical teachers often express regarding their own capacity for remediating learners’ difficulties.

Time-efficient remediation

The management of struggling learners is notorious – and sometimes dreaded – for being time-consuming.7,14,20,34,61 In one study based on 151 medical learners with difficulties, Guerrasio et al7 observed that the remediation of clinical reasoning difficulties was among the most time-consuming remediation processes, with an average of 20 hours spent on individual interventions per student. There is an obvious need for time-efficient remediation strategies, because for remediation activities to be realistically integrated with a regular rotation, it is crucial that these activities be time-efficient, as they will otherwise not be acceptable to clinical teachers.70

Discussion

Our review has shown that many principles are well established regarding the approach to struggling medical learners. As presented in Table 1, an effective approach to the clinical supervision of struggling learners should aim for early and precise identification of difficulties, based on repeated observations and on an initial informal discussion with the learner. A pedagogical diagnosis should be made, supported by various sources of data, acknowledging that difficulties can be interrelated. Only once this pedagogical diagnosis is clear should a targeted remediation strategy be developed and ideally integrated with the learner’s regular clinical activities. This review has also shown, however, that there remain both theoretical and practical challenges to address, in order to operationalize these principles further. Three-pronged recommendations are discussed in response to these challenges.

Faculty development

Certain barriers to applying best practices in the field could be addressed by faculty-development endeavors. Firstly, a lack of proper documentation by clinical teachers has been identified as one of the main causes of failure. It has been suggested that this step may be omitted, because clinical teachers do not know what to observe and document specifically. Knowledge of key elements to document proof of competence in a learner, or lack thereof, should be made explicit to clinical teachers, in accordance with the expected development path at respective institutions. Secondly, the identification of precise pedagogical diagnoses to explain underperformance has repeatedly been identified by clinical teachers themselves as a pressing faculty-development need. Since the next practical steps will derive from these diagnoses, faculty support in identifying the underlying issues would be instrumental in helping clinical teachers carry through an effective approach to struggling learners.

Institutional procedures

This review has also pointed out some steps in the management of struggling learners, which if implemented locally as preset procedures could then serve as common references among faculty. We hypothesize that having a simple and preestablished procedure to which learners could be referred should foster earlier identification of struggling learners. In this perspective, the first process to formalize should be a comprehensive description of steps to follow when referring a learner for remediation. Such processes should be determined locally, based on best evidence in remediation and organized according to available resources. For instance, clinical teachers, as well as rotation supervisors, should know ahead who to notify, formally and informally, if an issue arises. It should be clear who is responsible for overseeing the remediation process, and who serves as faculty advisor for this purpose. A second procedure, which could be preestablished locally, is a list of relevant descriptors for clinical teachers to observe and document, in order to situate the learner effectively with regard to the expected level of competence. Such descriptors will generally be drawn from the milestones relevant to each level of training.

Further research

The absence of a common framework to organize learner difficulties and from which to build standardized remediation strategies has been highlighted in this review. The establishment of such a framework should be given priority, in order for future research to build around a common structure. This objective also appears as a preliminary step to address a second pressing need: that of developing and evaluating efficient remediation strategies on a wider scale. Such endeavors should focus on targeting remediation strategies according to diagnosis, hence the need for a common framework to build from, and should aim for them to be time-efficient, so as not to impede their implementation in practice.

Conclusion

This article aimed to review the theory and empirical evidence available regarding the global approach to struggling learners during clinical supervision. Our review has shown that the perception that a learner is experiencing difficulty is assessed intuitively and reliably by clinical teachers, but that their unfamiliarity with the following steps, from pedagogical diagnoses to remediation strategies, leads to delays before the formal identification of these learners. Because early identification of struggling learners should be considered a gold standard, the subsequent steps for clinical teachers have been presented here, organized around the clinical model SOAP. Many obstacles have been identified to their practical application, among which the lack of standardized remediation procedures is not a lesser deterrent. Is there a recipe for supervising struggling learners? The answer remains to be completed, and future research and faculty-development endeavors should pursue the common goal of further operationalizing remediation strategies in response to specific diagnoses and making these processes more accessible to clinical teachers in the field.