Deliberate apprenticeship in the Pediatric Emergency Department improves experience for third-year students.

INTRODUCTION: The Pediatric Emergency Department (PED) provides medical students with learning in a high-volume, fast-paced environment; characteristics that can be stressful for new students. Shadowing can improve transitioning, yet this alone does not facilitate students' development of independent medical care competencies. This study evaluates if third-year medical students' deliberate apprenticeship with senior residents increases students' comfort and patient exposure in the PED.
METHODS: This study took place over the 2011-2012 academic year, and study participants were all third-year medical students during their pediatric clerkship rotation. This was a prospective educational intervention assigning students to randomized control blocks of deliberate apprenticeship (DA) intervention or control. DA students were paired with a senior resident who oriented and worked with the student, while control students were unpaired. All students completed a 20-question structured survey at shift end, which included questions about their perception of the learning environment, comfort with, and number of patient care responsibilities performed. We used independent Mann-Whitney and t-tests to compare experiences between the groups. Statistical significance was defined as p<0.05. We used the constant comparative method to qualitatively analyze students' comments.
RESULTS: Response rate was 85% (145/169). Students also rated on 5-point Likert-scale their level of comfort with defined aspects of working in the PED. DA students (n=76) were significantly more comfortable obtaining histories (4.2 versus 3.8) and formulating differential diagnoses (3.9 versus 3.4). DA students also performed more physical exams (2.9 versus 2.4). We categorized themes from the qualitative analysis of the students' comments about their PED experience. The titles for these themes are as follows: PED provides a good learning experience; uncertainty about the medical student's role in the PED; third-year medical students compete with other learners for teaching attention; opportunities provided to medical students for inclusion in patient care; personal knowledge deficits limit the ability to participate in the PED; PED pace affects learning opportunities.
CONCLUSION: DA constitutes a feasible approach to the clinical learning environment that increases students' patient care experiences and may ease transitioning for undergraduate medical students to new clinical environments.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

The transition to new clinical environments can be a stressful experience for medical students. Lack of knowledge of their roles and lack of familiarity with their environments contributes to the anxiety-provoking nature of transitioning work places.1 The pediatric emergency department (PED), in particular, is a high-volume, fast-paced environment that provides rich learning and training opportunities not found in other healthcare settings. Yet these very characteristics can be stressful for students not accustomed to this environment. Orientations, however, allow students to understand the “mechanics and processes of the healthcare team.2 Expectations, such as how many patients to see, what information should be conveyed to the preceptor, and how much time to spend with each patient, are valued by the student.3,4

Deliberate apprenticeship (DA) is the process of how learners gradually become part of a profession through progressively shared activities with established professionals. This concept of DA, in conjunction with orientation to learning objectives, is a key theoretical framework of how learning occurs.5,6 For instance, existing research shows that shadowing of current house officers can improve transitioning from the undergraduate to graduate medical training years, as well as comfort in a new practicing physician’s role.7,8 Emergency department (ED) observational experiences have been shown to increase career interest in emergency medicine for first-year medical students by linking these students with well-entrenched learners in the field.9 DA extends the principle of simply shadowing, by incorporating active involvement and participation as necessary steps to developing independent medical skills. Although there have been qualitative studies investigating the socialization and transition to new environments for house staff, there have been few educational interventions on how DA can improve the overall experience of the transition from the pre-clinical to the clinical years for undergraduate medical students.

From our post-clerkship surveys conducted prior to the initiation of this study, it appeared that third-year medical students’ ratings of their experience in the PED varied based on how actively the student was oriented and engaged during their shift. The absence of assignment responsibility for orientation for specific trainees leaves considerable variability in students’ contacts with residents at different levels of training, as well as with attendings. We hypothesized that DA of third-year medical students with senior residents in the PED would improve medical students’ overall experience in this setting compared to the usual practice, at this institution, of students just showing up for their shifts.

METHODS

Study Population

The study population was the third-year class of 169 medical students completing their pediatric clerkship for the academic year of 2011–2012.

Study Setting

This was a prospective educational intervention study conducted at the PED at a tertiary care children’s hospital over a 12-month (rotation) period. This study was deemed exempt by the local institutional review committee. The pediatric clerkship at this institution consists of a total of eight weeks: four weeks in the inpatient hospital setting and four weeks of ambulatory experience. The PED experience consists of two eight-hour shifts that occur during the ambulatory period. Prior to the start of the overall pediatric clerkship, students receive a four-hour orientation. An online website contains additional information regarding various aspects of their pediatric clerkship. Included is a PowerPoint presentation about the PED that students are encouraged to review before their ED shifts. Specifics about the PED experience are not verbally presented during the four hour orientation. Instead, the overall orientation instructs students to simply “show-up” and “jump-in” during the PED shifts.

Study Protocol

Students were assigned in randomized blocks to DA (intervention) or control groups. The DA group consisted of students specifically paired with the senior residents whose shift schedule had the most significant overlap with the students’ schedule. This resident was in charge of not only orienting, but also closely working with the student. These senior residents received an email with the date, shift time, student name, and specific instructions on how to orient their medical student. The control group consisted of students who received the traditional pediatric orientation: i.e. showed up for their shifts without a pre-assigned resident. The third-year medical students not paired with senior residents were not aware that other groups of students had specific pairings to residents.

Outcome Measurements

We evaluated this educational intervention by collecting medical students’ perceptions about their experience during their PED shifts. Students were asked to complete a web-based (Qualtrics™) 20-question anonymous online instrument via email approximately one hour after completion of their first PED shift. Outcome data elicited from students included the number of patients seen, histories/physicals performed, and procedures observed. Students also rated on 5-point Likert-scale their level of comfort with defined aspects of working in the PED. We also provided a section for written comments regarding their PED experience. Students were explicitly informed that their participation was voluntary and anonymous.

Data Analysis

Quantitative

We analyzed the data using SPSS, Version 19 (Armonk, NY). Descriptive statistics for each variable were calculated. We used independent Mann-Whitney analyses, for ordinal-level ratings of comfort and overall PED teaching and experience, to compare the experiences of students in the DA and the control groups. We compared the number of patients that DA and control students saw and procedures they observed, as well as the number of histories/physical examinations they independently performed, using independent t-tests. We studied this over an academic year to determine empirically whether differences emerged over time, i.e. early versus late rotation periods with analysis of variance over the 12 rotations. Statistical significance was defined as p<0.05. We used a Cohen’s d to calculate effect size; based on a population of 169 students, with 95% confidence level, effect size 0.5, power level of 0.8 and 80% response rate, this was a total of 102 students.

Qualitative

Using the constant comparative method, consistent with grounded theory, the comments provided by the medical students were independently read and coded for emergent themes by three investigators, who were blinded to the comment status as generated by the DA or the control group.10 The analysis proceeded iteratively until all comments were coded into themes consistently across reviewers.

RESULTS

Quantitative Results

A total of 145 out of 169 (85%) students responded to the online survey instrument. Figure illustrates the breakdown of individuals completing the survey. Most (74.1%) of the students reported that they had read the online orientation PowerPoint prior to their PED shift. The majority of students (85%) reported receiving an individual orientation during their PED shift. Table summarizes the findings comparing the DA and control groups. At a statistically significant level, DA students were more comfortable with taking histories and with creating a differential diagnosis. In addition, DA students performed significantly more physical examinations. There was no significant difference in students’ rating of their overall PED experience in comparing mean ratings of the two groups. No statistically significant difference occurred in comparing mean number of professional care responsibilities across the 12 rotations during the academic year.

Figure

Study population.

Table

Differences between deliberate apprenticeship (intervention) of third-year medical students in a pediatric ED and control groups.

	Control mean (SD)	Intervention mean (SD)	p-value
Comfort obtaining histories†	3.8 (1.0)	4.2 (1.0)	0.02*
Comfort with physical examinations†	3.7 (1.1)	4.1 (1.0)	0.07
Comfort with formulating a differential diagnosis†	3.4 (1.0)	3.9 (0.9)	0.001*
Overall, comfort level working in the pediatric emergency department†	3.7 (1.1)	4.0 (1.0)	0.11
Overall teaching†	3.4 (1.0)	3.4 (1.1)	0.88
Overall experience†	3.4 (1.0)	3.5 (1.1)	0.40
Number of patients seen	5.8 (1.7)	5.5 (1.6)	0.31
Number of histories performed	2.6 (1.4)	3.0 (1.4)	0.08
Number of physical exams performed	2.4 (1.3)	2.9 (1.5)	0.03*
Number of procedures observed	2.0 (1.0)	2.0 (1.2)	0.76

p<0.05,

Likert-type scale very uncomfortable (1) to very comfortable (5).

Qualitative Results—Themes from Student

There were 58 comments from the students. Six themes emerged from the analysis of these comments. A summary of each theme, noting the percent of students contributing comments categorized within the theme category, is provided. The themes were as follows: PED provides a good learning experience (39.7%); uncertainty of the medical student’s role in the PED (27.6%); opportunities provided to medical students for involvement/inclusion in patient care (12.1%); third-year medical students compete with other learners for teaching attention (3.4%); personal knowledge deficits limit the ability to participate in the PED (3.4%); and PED pace affects learning ability (13.8%). A description of each theme, with illustrative quotations, is provided below.

PED Provides a Good Learning Experience

Many medical students characterized the PED as a valuable part of their training. One student commented, “I think that it is an important experience because you get to see/hear very pertinent findings that are often not present in clinic (ex: real nasal flaring and retractions or even in my case a patient in status (control).” Faculty and residents were also individually acknowledged as contributing to the medical students’ having an overall good experience. For instance, one student wrote, “Dr. [X] was GREAT, so was [Y] the resident…they were great teachers and definitely included teaching as well as feedback (DA).”

Uncertainty of the Medical Student’s Role in the PED

Many medical students reported that faculty and house staff did not know how to incorporate them into the flow of patient care. One medical student commented, “It was a little hectic when I arrived and I wasn’t informed of exactly what I would be expected to do—nobody seemed to know what my role was exactly and whether I should see patients on my own, etc (control).” Students reported that if expectations were provided, they would be better suited to participate in this environment. One student wrote, “At the beginning nobody looked like they were expecting to see an M3 show up for the shift; nor were we given instructions at peds orientation to ask for Dr. X (attending, resident, etc.) on arriving at the ED. Once I introduced myself and got setup with a resident everything was great, but knowing who on staff to talk to when you get there would make things run smoother” (control).

Opportunities Provided to the Medical Student for Involvement/Inclusion in Patient Care

Students reported that when they were afforded the opportunity to participate directly in patient care, their overall learning improved. A student commented that, “[I] loved my time in the ED, as I was able to jump from patient to patient,…and really able to see whatever patients I wanted to…[I] loved working with residents who let me take the history myself, and gave me more autonomy in the physical exam, ordering tests, discharging patients” (DA).

Third Year Medical Students Compete With other Learners for Teaching Attention

Medical students are not the only trainees in the PED. Medical students characterized this presence of other levels of learners as working against medical students seeing patients. One of the students wrote that, “There were so many providers (1 intern, 1 resident, 1 M4 and a faculty member plus myself)…I didn’t get to do an interview, I barely talked to a patient all day” (control).

Personal Knowledge Limit the Ability to Participate in the PED

Medical students felt that their own knowledge gaps, as well as being new to the clinical environment, affected their ability to participate in the PED. One student exclaimed that, “as an early M3 who hasn’t yet completed internal medicine I felt woefully under-prepared to work with most ER patients and without having done anything except two weeks of well child exams on outpatients I feel even less prepared to take histories on pediatric patients” (control).

PED Pace Affects Learning Ability

The PED is often a hectic place, yet there are also times when there is low volume. One student expressed this experience of environmental variability by writing that this is a “busy place even when it’s not ‘busy’” (control). Another student reported that “both shifts I was there, the PED was pretty quiet, so I didn’t get to see as many interesting things as I otherwise would have” (DA).

DISCUSSION

Medical educators have recognized that traditional medical education may insufficiently prepare students for the transition from preclinical to clinical education.11,12 One response to this challenge has been the development of transitional courses.1,7,8 We know that the PED is a rich learning environment and that some EM educators have noted should be incorporated into early training.13–15 In this study, we tested if DA for third year medical students would improve their experience in the PED. Yet, the qualitative themes show that students found the PED to be problematic in terms of uncertainty of role, level of inclusion, and self-perceived knowledge deficits. We found that our DA intervention helped in these domains by performance of more physical examinations and increased comfort with taking histories and formulating differential diagnoses.

The theory of situational learning has been prominent in identifying the active social methods that learners and professionals use in DA to facilitate the progressive awarding of participation and autonomy of learners. In DA, the apprentice student observes the master demonstrating how to do different parts of the task.5,6 The master deliberately makes the target processes visible, often by explicitly showing the apprentice what to do, for example describing how they evaluate a patient and thinking out loud through the clinical reasoning. This extends what has previously been described as simply “orientation,” “shadowing,” to “legitimate peripheral participation in a community of practice.”16 By pairing students with senior residents who not only oriented, but also closely engaged the students, DA enhanced students’ perception of their learning. Further, students who were actively engaged during their PED shifts commented on having better individual experiences. In particular, comments on students’ level of inclusion with residents and/or faculty allude to the idea that DA increases participation in the unique clinical environment of the PED.

As medical school curricula evolve, there is a move towards earlier emergency medicine experiences.17,18 In light of this, it is important to know and address some of the pitfalls of third-year students in the potentially chaotic PED. One key issue is uncertainty with their role, with some students requesting a clear orientation. This study also showed that the majority of students reported receiving an orientation to the PED, whether or not they were assigned to a senior resident. Even so, they remained uncertain of their role. This concept of providing information to a student, yet the student not perceiving that it has been provided, is not uncommon. For example, one study on feedback for medical students showed statistically significant differences in the regularity of feedback provided by faculty as compared to that perceived by the student.19 Another potential reason for this discrepancy is that, although they were oriented to the PED, they did not receive further instruction for the duration of their shift or simply could not independently perform without direct instructions. This is best put in the words of one student who wrote, “I did appreciate one of the residents had me look up a treatment regimine [sic] for one of the cases we saw and present that to her.” Such directed learning may be necessary for some individuals to have an overall better experience in the PED.

The themes culled from student comments help us to understand the experience of students in the PED. We recognize that we need to be more intentional in orienting and engaging students in clear roles, inclusion in patient care and learning. DA may help with many of these issues but further study is warranted.

LIMITATIONS

Although the senior residents did receive an email with specific instructions on what/how to orient their students, we studied only the medical students’ experiences. It is therefore possible that some of the seniors were not able to orient their pre-assigned medical students during their shift, or oriented students to whom they were not paired, raising the issue of the fidelity of the study with its intended implementation. The study did not explore the extent to which participation in DA might have enhanced faculty engagement. In our institution, as is true for other clerkships, the length of the PED rotation is limited. Our medical students completed two shifts in the PED, thus limiting assessment of improvement over time. Furthermore, DA students were not necessarily paired with the same senior resident each shift. In addition, not all students entered comments. Although the number of students participating in the study enabled detecting a statistically significant difference between the DA and control groups at alpha=0.05, the effect size was moderate, with some overlapping confidence intervals. Furthermore, we made 10 comparisons between the DA and control group in regards to comfort levels in various domains, as well as number of medical skills completed. These multiple comparisons may have led to a Type 1 error. Moreover, in seven of these comparisons, there were no differences between our groups, thus the full extent of DA needs to be further elucidated. If the alpha levels are further adjusted for multiple comparisons with the conservative Bonferroni Correction, which divides alpha by the number of comparisons made, only comfort in making a differential diagnosis remains as a comparison representing a statistically significant difference between the groups. Finally, the study was performed at a single institution, thus limiting generalizability.

CONCLUSION

This educational intervention study extends existing research on DA by providing insights into the kinds of outcomes such intentional allocation of resources may achieve. This study adds to our current professional discourse on the move from time-based to competency-based education. Much of the discussion about this move focuses on the need to be more intentional in designing education promoting cognitive and procedural competence. This randomized controlled block study, using mixed quantitative and qualitative approaches, builds on and extends this literature by focusing on intentional approaches to student socialization to the work-place learning site. The intervention evaluated represents a theoretically-grounded and feasible course of action amenable to being adopted in other clinical learning sites.

The emergent themes from analysis of the students’ comments illuminate how active engagement can improve experience. Although students’ ratings of their overall experience in the PED did not differ across the two groups, students who underwent DA performed more physical exams, and had greater comfort in obtaining histories and formulating a differential diagnosis. Above all, DA is a novel, simple, and feasible technique to ease the transition to new clinical environments for undergraduate medical students.