Asking what do residents value most: a recent overview of internal medicine residents' learning preferences.

BACKGROUND: Little is known about the preferred learning experiences of today's internal medicine residents. We conducted a survey of the educational experiences in an internal medicine residency to determine the learning opportunities internal medicine residents value most and why.
METHODS: An online, anonymous survey of 182 internal medicine residents was performed, with each resident receiving a survey each day over nine days. Participants were asked to state their most valuable learning experience over the past day, describe why it was valuable, and rank it on a 5-point Likert-type scale. Resident free-text responses were coded and grouped into themes. The location of and participants in the experience were also examined.
RESULTS: The 182 residents completed a total of 303 surveys. Of the 303 surveys, 92% (N=277) of the responses noted their chosen learning experience was useful. An attending was involved in 50% (N=152) of experiences; the patient was noted as a participant in 8% (N=25) of experiences. Free-text responses were coded into five thematic groups descriptive of why residents found their learning experiences to be valuable: Repetition in Learning, Effective Pedagogy, Clinical Problem Solving as an Individual or Collaboratively, Opportunity for Active Engagement, and Bedside Learning.
CONCLUSION: Our data provide a broader framework for designing and implementing future faculty development and resident curricula that emphasize interprofessional education and the patient as a key educational figure.

Introduction

Residency is a critical, formative, multidimensional educational experience that bridges the gap from medical student to independent practitioner. A resident is the consumer of numerous learning sessions designed to provide the knowledge, communication skills, and hands-on “know-how” to succeed as a high-caliber, informed, and caring physician. The Accreditation Council for Graduate Medical Education (ACGME) mandates that internal medicine residency programs provide a myriad of educational experiences to support resident learning.1,2 However, little is known about which educational experiences internal medicine residents value most.

The prior literature has evaluated and described medical student and resident learning style preferences;3–10 however, there are few papers analyzing residents’ perceptions of the most valuable learning experiences in their daily work.11 Several published studies do examine individual aspects of the resident educational experience, including learning preferences on rounds; reading habits; preferences in noon-conference presentations; educational experiences as they relate to ACGME competencies; and the relationship between resident reading, conference attendance, and in-training exam scores.12–16 These studies offer important contributions to our understanding of resident learning. However, understanding which educational methods and strategies today’s residents value most remains a significant challenge facing the graduate medical education community. There are two studies in the surgical literature which take on this question, asking residents to reflect on their most significant learning experiences.17,18 However, to our knowledge, there are no prior studies examining the learning preferences of internal medicine residents. The purpose of this study is to provide a current overview of what educational experiences internal medicine residents value most and why.

Methods

Survey distribution

This study was approved by the Partners Human Research Committee, in July 2016. We conducted an anonymous survey study of the Brigham and Women’s Hospital internal medicine residency during the 2016–2017 academic year. The survey instrument was created by a third-year resident (JC) in the Brigham and Women’s Hospital internal medicine residency and was reviewed by a senior faculty member (HS), our statistician (SP), and the residency program director (Dr. Joel Katz). The survey instrument was created using Qualtrics, a computer software that makes respondents’ email addresses anonymous, available at Harvard University and Harvard Medical School. Completion of the survey was deemed to be informed consent. Because the email addresses were made completely anonymous by the Qualtrics software, it was not possible to determine on how many days each resident completed the survey.

All members of the internal medicine residency (N=182) received a daily email with a link to an online survey on nine separate weekdays. The survey was distributed Monday through Thursday during week 1 and Monday through Friday during week 2. The survey administration period included a rotation switch day so that most residents participated in more than one rotation during the study. Residents were informed that their responses would be confidential. The total number of possible participants on each day of the survey administration varied from 116 to 120 residents. This reflected that only residents who were participating in internal medicine inpatient or ambulatory rotations at Brigham and Women’s Hospital, or an elective rotation in research at Brigham and Women’s Hospital, at the time of survey administration, were eligible for the study. This means that residents who were on vacation, sick call, a non-internal medicine rotation, or on a rotation at an unaffiliated site were excluded from this study. All residents received a link to the online survey, but if a resident noted that he or she was in one of these ineligible groups, then the survey was terminated automatically.

Residents were asked, “Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you”. They were then asked to rate their chosen learning experience on a 5-point Likert-type scale and were asked two follow-up questions about the experience: “Where did this experience occur?”, and asked to choose only one location, and “Who else was involved?”, and invited to mark all that applied. They were also asked to provide their postgraduate year and current rotation with each survey (Table S1).

Data analysis

After data collection, qualitative analysis of resident responses to the question “Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you” was performed to look for repeated concepts and ideas. These initial concepts were organized into a series of categories, which were then grouped together into themes.19,20 Open coding was initially performed by one of the study investigators (JC), a third-year internal medicine resident, to identify why respondents found their chosen learning experience to be valuable. Based on the identification of keywords, phrases, and concepts in the text, 22 codes were generated (Table 1). Each response could be coded with one, several, or zero codes depending on the content of the response. A senior faculty member from the department of medicine (HS) then reviewed each response and assigned codes with the first coder (JC). Each response was discussed until consensus was reached between JC and HS. Axial coding was then performed, wherein the responses were all reexamined in the context of their assigned codes and based on commonalities the specific codes were divided into five thematic groups (Table 1). Next, the responses were reviewed by a second coder (SS), a second-year internal medicine resident, who independently assigned codes to each response from the list of codes generated by JC and HS. The two sets of coded data (produced by JC and SS, respectively) were then assessed for agreement. A third coder (ED), also a second-year internal medicine resident, then reviewed and independently coded all discordant responses. The second and third coders (SS and ED) also reviewed each of their assigned codes with the same senior faculty member from the department of medicine (HS) to ensure that each code was thoughtfully assigned. For a response to be included in analysis, there needed to be agreement between two out of the three coders. We considered agreement to mean either assignment of the same code or assignment of two codes from the same thematic group. Any responses that were discarded because of discordance among coders were reviewed to ensure that they were not systematically different from those that were included.

Table 1

List of codes and thematic groupings

Codes		Thematic groups
• Synthesizing information• Solidifying learning• Skill practice		Repetition in Learning
• Expert teaching• Multiple learning modalities• High yield/useful• Varied teaching settings• Clarity of teaching• Relevance		Effective Pedagogy
• Creative thinking• Honing clinical judgment• Diversity of perspectives• Clinical tool• Learning from error		Clinical Problem Solving as an Individual or Collaboratively
• Interactive• Small group• Ability to ask questions• Collaborative learning		Opportunity for Active Engagement
• Patient as teacher• Expert modeling patient care• High-stakes learning• Longitudinal care		Bedside Learning

In summary, our coding was done methodically by having each coder read the item out loud, process it, discuss it, and then jointly assign it a code. We did not use a systematized rubric or other validated tool that could easily be translated into numeric values. Thus, our data do not lend themselves to calculation of Cohen’s kappa or Fleiss score when analyzing interrater reliability.

Results

Resident respondents

In total, the 182 internal medicine residents completed 303 surveys during the nine-day study period. The overall response rate varied by day and ranged from 20% to 49% during the study period (Table S2). Respondents were from all years of internal medicine training and participating in a wide variety of rotations (Table 2). Because the survey software made the respondent’s email addresses completely anonymous, it was not possible to know how many residents were repeat responders and exactly how many times each resident responded.

Table 2

Participating residents’ year of training and current rotation

	N (%)
Postgraduate year
1	111 (37)
2	76 (25)
3	87 (29)
4+	25 (8)
No response	4 (1)
Rotation
Inpatient rotation at the main teaching hospital	115 (38)
Ambulatory rotation	67 (22)
Night float rotation	45 (15)
Other	21 (7)
Intensive care unit rotation	14 (5)
Inpatient rotation at another site	13 (4)
Clinical elective	13 (4)
Research elective	12 (4)
Inpatient rotation at the VA	3 (1)

Abbreviation: VA, Veterans Health Administration Medical Center.

Characteristics of residents’ most valuable learning experiences

The 182 residents completed a total of 303 surveys. Of the 303 surveys, 92% (N=277) of the responses noted their chosen learning experience was useful, with 63% (N=191) reporting the experience to be “very useful” and 29% (N=86) reporting it to be “somewhat useful” (Figure 1). The residents’ most valuable learning experiences included a wide variety of other individuals. An attending was cited most frequently, and involved in 50% of total experiences (N=152). The patient was involved in 8% (N=25) of all experiences. Nurses (N=14, 5%) and other allied health care professionals (N=12, 4%) were the least frequently cited individuals (Table 3). The experiences occurred in a variety of settings. The most commonly cited setting was the team workroom (N=94, 31%; Table 3).

Figure 1

Internal medicine residents were surveyed during the 2016–2017 academic year about their most valuable learning experiences and were asked to rate these experiences on a 5-point Likert-type scale.

Note: This figure depicts the frequency of each response on that scale (N=303).

Table 3

Participants’ most valuable learning experience and experience setting

“Who else was involved?”a	N (%)
Attending	152 (50)
Coresident	109 (36)
More senior resident	77 (25)
More junior resident	66 (22)
Medical student	42 (14)
Just me	40 (13)
Patient	25 (8)
Consulting service	21 (7)
Other	15 (5)
Nurse	14 (5)
Other allied health care professional	12 (4)
“Where did this experience occur?”b	N (%)
Team workroom	94 (31)
Other	75 (25)
Patient bedside	37 (12)
Medical ward	27 (9)
Missing/no answer	24 (8)
Clinic room	19 (6)
At home	15 (5)
Library	8 (3)
Radiology reading room	3 (1)
Pathology microscope	1 (<1)

Notes:

Multiple answers were allowed.

Only one answer was allowed.

Coding

Of the 303 responses, 266 surveys (88%) included a response to the free-text question, “Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you”. Of these 266 responses, 152 respondents (54%) provided a description of why the experience was valuable. The remaining 114 responses named a valuable experience without providing an insight into why it was valuable. Of these 152 complete responses, 138 were assigned open codes with agreement between at least two out of the three coders (91%), leaving only 14 responses uncoded. These 14 responses were reviewed to see if they were somehow different, more interesting, or important compared to the other responses. We found that the discarded responses tended to be shorter and more vague than the included responses. In 13 of the 14 discarded responses, at least one of the coders did not feel that a code could be assigned based on the information provided, suggesting that these responses were more vague than the responses included in the dataset. Because each response could be assigned more than one code, there were a total of 162 codes with agreement between at least two out of the three coders. As described above, the open codes were divided into five thematic groups representing the key feature of the resident’s most valuable learning experience: Repetition in Learning, Bedside Learning, Clinical Problem Solving as an Individual or Collaboratively, Effective Pedagogy, and Opportunity for Active Engagement (Table 1). Figure S1 depicts the distribution of each thematic group within the sample. Of interest, the five thematic groups represented different aspects of resident learning and pedagogy. The most commonly identified themes were Repetition in Learning (N=48, 35%) and Effective Pedagogy (N=47, 34%), followed by Clinical Problem Solving as an Individual or Collaboratively (N=34, 25%), Opportunity for Active Engagement (N=22, 16%), and Bedside Learning (N=16, 12%). Of note, each survey response could be coded into zero, one, or multiple thematic groups.

Thematic groups

Verbatim examples of resident responses categorized into each of the five thematic groupings are presented in Table 4.

Table 4

Internal medicine residents were surveyed during the 2016–2017 academic year about their most valuable learning experiences

Theme	Sample responses
Repetition in Learning	“Admitting a patient last night [because] I had to think through how to best summarize the critical aspects of heart failure.”“Me teaching the interns via a case I saw Talking through it solidified my own knowledge.”“As part of the geriatrics rotation, I engaged in an observed clinical experience [OSCE] focused on end-of-life communication. I found it tremendously useful to practice using language around hospice and end-of-life care.” (This was also coded as Opportunity for Active Engagement and Effective Pedagogy)
Effective Pedagogy	“[My most valuable learning experience was] an attending pointing out an EKG finding I had never seen. This was valuable because hearing things out loud from experts makes things stick and makes things make sense in a way that [reading in] a book does not.” (This was also coded as Repetition in Learning)“[My most valuable learning experience] was noon conference with [a senior infectious disease attending] – an incredibly high yield review of HIV management from a topical expert.”“My resident taught me about hyponatremia in an extremely clear way during about [5 minutes] on rounds.”“This AM our team discussed CRT-D. The experience was valuable because the senior resident and attending highlighted the most salient points about said topic.”
Clinical Problem Solving as an Individual or Collaboratively	“I was working in the ED and working collaboratively with one of the ED [senior residents]. I presented a case to her and she provided feedback from a different specialty perspective about my differential that helped remind me not to anchor and highlighted another clinical entity.”“[I] learned about vertigo and the quick way to screen for central [versus] peripheral vertigo from my attending. [This was] useful [because it] imparted a few key points to remember, and gave a useful clinical tool to use [in either the inpatient or outpatient] setting.”“[My most valuable learning experience was in] morning report – it focused on [the case of a patient] that I had taken care of while on [service], so it was helpful to hear the discussion from a lot of different perspectives (on a case that I had already thought about a lot).”“[I admitted] a sick/complicated patient overnight, [which forced] me to try to think creatively about what might explain their presentation.”
Opportunity for Active Engagement	“[My most valuable learning experience was] attending teaching rounds because it was an interactive small group scenario.”“The most valuable learning experience I had…was a lecture I had on readmission risks and quality and safety. The lecture was valuable because it fostered participation and enthusiastic discussion.”
Bedside Learning	“[My most] significant teaching moment [was] with my resident based on a change in patient status [leading to] decompensation. [It was] so salient because I was nervous and every ounce of me was trying to pay attention to details.”“[My most valuable learning experience occurred] in my primary care clinic [with] a patient [experiencing] persistent symptoms after a recent ED diagnosis. [I] learned about the workup required and had a chance to see the [patient’s] course after [he was] discharged from [the] ED/hospital”. (This was also coded as Clinical Problem Solving as an Individual or Collaboratively)“[My most valuable learning experience was] a rapid response … [for] a patient with nearing-unstable atrial fibrillation with RVR. [I] learned [the] steps needed to cardiovert a patient on the floor in an urgent (but not emergent) setting.”

Notes: Internal medicine residents were surveyed during the 2016–2017 academic year about their most valuable learning experiences. Responses were coded and categorized into five thematic groups. This Table lists representative resident responses for each thematic grouping.

Quotations could be coded into more than one thematic group. This is noted throughout the table.

Abbreviations: AM, morning; CRT-D, cardiac resynchronization therapy defibrillator; ED, emergency department; RVR, rapid ventricular response.

Repetition in Learning

This theme captures the idea that residents found value in repeated exposure to or practice with core concepts and skills. Residents valued learning through a large volume of clinical experience with exposure to various clinical topics repeatedly and through a variety of modalities: at the bedside, at the microscope, in clinical team discussion, through self-guided learning, or formal didactics.

Effective Pedagogy

Several residents described what they felt was effective learning facilitated by either great teachers or successful teaching methods. The exemplary teachers described were most often attendings with specific expertise, but also included coresidents who used engaging teaching strategies.

Clinical Problem Solving as an Individual or Collaboratively

Many responses focused on learning how to tackle a specific clinical problem. Residents valued group brainstorming, opportunities for individual creative thinking, and the acquisition of new skills or tools to use in clinical decision making.

Opportunity for Active Engagement

Some residents felt that they were learning best because they had the opportunity to participate actively in the learning process. They described interactive lectures, small group didactics, or team rounds where they were given space to participate, discuss, and ask questions.

Bedside Learning

Several responses touched on the salience of learning that occurs while taking care of a patient in real time. Residents found that active engagement in individual patient care helped make learning memorable.

Discussion

To our knowledge, this is the first study to provide an overview of internal medicine residents’ perceptions of what learning experiences are most valuable and what makes them valuable. The strengths of our study are the breadth of the experiences we captured, novel delineation of the setting and cast of characters in resident learning, as opposed to just the content, and exploration of why residents find more value in certain learning experiences. Our findings depict a rich tapestry of learning opportunities within residency education. The image of learning in residency that emerges from this single-academic-center study highlights the importance of diversity of experiences and educational modalities.

Specifically, and surprisingly, we noted a conspicuous absence of the patient as a central figure in the residents’ reported most valued learning experiences. Residents cited the patient as one of the people involved in their chosen learning experiences in only 8% (N=25) of responses (Table 3). Forty percent (N=121) of the most valuable learning experiences occurred in either the team workroom (31%, N=94) or on the medical floor (9%, N=27), locations where the residents engage in work and learning related to patient care; yet, the patient is not given credit as a major educator in these experiences. Examining the thematic groupings used to code our resident responses, “Bedside Learning” was the least well-represented category, accounting for only 12% of codes (Figure S1). While it seems that many of residents’ high-yield learning experiences relate specifically to care of individual patients, residents are only associating the patient as a key figure in their learning experiences a minority of the time.

Bedside learning, defined as learning in the presence of the patient, is recognized as an integral and essential component of resident learning21–25 and has been the focus of several recent, published interventions.26–28 Furthermore, spending time at the bedside engaged in patient care has been shown to combat physician burnout.21 The divide between enthusiasm about learning from clinical care and identifying the patient as a central part of that learning is demonstrated in our study. We believe this is a relevant and unexpected finding that should encourage residency programs to actively incorporate the patient as a teaching partner.28

In addition, it is notable that the residents surveyed mention nurses and other allied health care professionals infrequently as participants in their useful learning experiences. Nurses are noted as a participant in the most valuable learning experience in 5% of responses (N=14), and other allied health care professionals are noted in 4% of experiences (N=12; Table 3). Interprofessional learning is emerging as a key educational goal for today’s residents and has been shown to improve patient care outcomes.29–31 Thus, our finding of only few resident surveys noting interprofessional learning experiences is of significant importance and provides a rationale for the development of increased and innovative opportunities for interprofessional education.

It is important to note that on a survey administered by a residency program, there is a possible source of implicit bias on the part of respondents toward recollecting educational experiences that involve their formal educators. This potential bias may partially explain the prevalence of formal educators as participants in residents’ most valued experiences, and may have overshadowed the patient and other allied health care professionals as additional possible valuable contributors. However, our survey asked that residents select as many individuals as were involved in their chosen learning experience. Thus, the rare mention of patients, nurses, and allied health care professionals is an important finding.

Our data describe the educational methods and strategies that today’s residents value most, and in doing so provide an opportunity to develop new strategies that residency programs can use to increase the visibility of the patient and allied health care professionals in valuable resident learning experiences. Based on our data, we challenge educators to create space for all team members, including the patient, to engage in clinical interactions and discussions and ask questions both during rounds and throughout the day. The next steps in our research will include evaluation of whether tailoring residents’ educational experiences to those they find most valuable will have a positive outcome on resident learning, job satisfaction, and quality of resident life.

Our study has several limitations. First, the daily response rate averaged 37% over the nine days of survey administration. However, this response rate is within the range of what is reported in the literature for trainee and physician response rates to surveys.32,33 This may be attributed to survey fatigue, especially noting that the response rate trended down over successive days in the study period.34 Perhaps, this would be mitigated in future studies if the same number of surveys were spaced out over a longer period. Second, we surveyed residents from a single internal medicine residency program at a single academic hospital. Third, our study was a snapshot capturing nine days. It would be more informative to collect data over a longer period of time. Fourth, because our survey instrument anonymized the email addresses of respondents, we do not know how many times each resident responded to our survey over the nine-day study period. This is a limitation that would be correctable when future computerized survey instruments are programmed for anonymous distribution. Fifth, our study examined resident impressions of learning, but did not measure resident learning itself as an outcome. Future studies are needed to assess the relationship between resident learning preferences and specific identified learning outcomes.

Conclusion

Our data and detailed analyses provide a new and broader recognition of the learning preferences of today’s internal medicine residents. Our results indicate that internal medicine residents value exposure to a wide variety of patient care-centered and core didactic activities, and value learning that is relevant to patient care, actively engages them in the learning process, and is delivered by skilled and expert physician educators. We clearly illustrate that the residents surveyed value learning experiences centered on patient care, but they do not recognize the patient as a central educational figure. In addition, residents are rarely engaging in, recognizing, or finding value in interprofessional educational experiences. In the future, educators will need to partner with the patient and other members of the health care team to deliberately design learning experiences that are focused, high yield, and inclusive. We look forward to bridging this divide by creating, designing, and implementing educational experiences that inspire today’s residents and faculty, based on these results.

Distribution of thematic groupings.

Table S1

Survey instrument

Survey questions

1. What year are you

○ PGY1

○ PGY2

○ PGY3

○ PGY4+

2. Think back over your last 24 hours, were you on:

• Inpatient rotation at the main teaching hospital

• Inpatient rotation at another hospital site

• Inpatient rotation at the VA

• Intensive care unit rotation (including other hospital sites)

• Night float rotation (including other hospital sites)

• Ambulatory rotation

• Research elective

• Clinical elective

• Other elective (specify)______

• Away rotation (including global health)

• Vacation

• Other _____

3. Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you.

4. Please rate this learning experience.

1 = I didn’t learn anything new

2 = I learned something, but it was not particularly memorable or useful

3 = I learned something that may be useful in future clinical encounters

4 = I learned something that will be useful in future clinical encounters

5 = I learned something new and/or important that I believe I will retain and consistently use in future clinical encounters

5. Where did this experience occur? (choose only one)

• Patient bedside

• Clinic room

• Team workroom

• Medical ward

• Library

• Radiology reading room

• Pathology double-headed microscope

• At home

• Other (Please specify: _____)

6. Who else was involved? (Choose as many as applicable)

• Attending

• Co-resident

• More senior resident

• More junior resident

• Medical student

• Nurse

• Other allied health care professional (pharmacist, social work, care coordinator, other) Specify:________

• Consulting service fellow or attending

• Patient

• Just me

• Other (Please specify: ____)

Abbreviation: VA, Veterans Health Administration Medical Center.

Table S2

Daily survey response rate

Study day	Number of responses	Total possible respondents	Response rate (%)
1	59	120	49
2	53	120	44
3	56	120	46
4	42	120	35
5	53	120	44
6	43	116	37
7	35	116	30
8	23	116	20
9	34	116	29