Residents Take the Lead: A Modern Collaborative Approach to Research During Residency.

BACKGROUND: Participation in scholarship is a requirement for Internal Medicine (IM) residencies, but programs struggle to successfully integrate research into busy clinical schedules. In 2013, the IM residency at Brigham and Women's Hospital implemented the Housestaff Research Project (HRP)- a novel residency-wide research initiative designed to facilitate participation in scholarship. The HRP had two components-a formal research curriculum and an infrastructure that provided funding and mentorship for resident-led, housestaff wide projects.
METHODS: This is a mixed-methods study of 190 IM residents and two HRP-supported research projects. Seventy-seven residents responded to an electronic survey about their interests in research exposure in residency. Fifty-six residents responded to an electronic survey about their participation in the HRP. The success of HRP-supported projects was evaluated through resident comments, interviews with three residents leading the first two HRPs and a description of the success of the projects based on resident involvement and dissemination of the results.
RESULTS: Eighty-seven percent (n= 67/77) of residents were interested in additional research exposure during residency. Ninety-five percent (n = 53/56) of residents had heard of the HRP, and 77% had participate in at least one aspect of it. Approximately 20 residents were directly involved in the two resident-led projects. HRP-supported projects resulted in presentations at three local and three national conferences, one manuscript in press, and one manuscript in preparation. The resident project leaders felt that a strength and unique aspect of the HRP was the collaboration with co-residents.
CONCLUSION: The HRP successfully created a culture of research and scholarship within the residency. The HRP leaders and residents that participated in HRP-supported projects expressed the most direct benefits from the program. All residents were exposed to research concepts and methods. Future directions for the HRP include selecting projects that maximize the number of resident participants and integrating a more robust research curriculum.

Introduction

The Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Internal Medicine (IM) identify participation in scholarship as a core residency requirement.1 Exposure to research teaches residents to appraise the literature and make evidence-based clinical decisions, promotes critical thinking, and helps determine the structure of their future careers. IM residency programs recognize that these exposures ultimately foster better clinicians but identify lack of faculty time and funding as the main barriers to resident research at university programs.2

Residencies have pursued various pathways to facilitate scholarship and foster resident participation in research activities and publications. Some of the pathways that have been explored include appointing a dedicated director to facilitate resident research,3 creating a comprehensive three-year research curriculum,4 and developing a research-focused residency track.5 Other specialties have created Resident Research Days,6 structured research curricula,7–9 and promoted a point-based reward system.10 Such programs have resulted in increased scholarship by residents.4,5,7–9 For example, one research curriculum implemented in a general surgery program incorporated structured mentoring with annual milestones. Resident publications increased by 720%.9 Appointing designated faculty to oversee resident scholarship is another strategy for increasing publications by residents.3,4

Traditionally, Brigham and Women’s Hospital (BWH) IM residents have worked on individual research projects in basic science, clinical, or translational research. The residency did not have a formal research curriculum. In the late 1990s, residents received additional exposure to research through participation in quality improvement and safety projects led by attendings,11 similar to initiatives at other institutions.12–14 These programs encouraged housestaff participation and input but were led by fellows or attendings.

In this paper, we describe the development and implementation of the Housestaff Research Project (HRP) in the IM residency at BWH. Beginning in 2013, the HRP is a novel residency-wide research initiative designed to facilitate participation in scholarship. The HRP has two components—a formal research curriculum and an infrastructure for providing funding and mentorship for resident-led, housestaff wide projects. We present a description of each component of the HRP and a mixed methods evaluation of the program. The evaluation explored residents’ interest in research exposure during training, their experience with the HRP, and the success of the first two HRP-supported projects.

Methods

Subjects and Settings

All BWH IM residents had the opportunity to participate in the HRP.

Description of the Housestaff Research Project

Selecting Resident-Led Research Projects

Resident-led research projects supported by the HRP were selected every one to three years. This ensured there was at least one HRP-supported project active at all times. Residents were invited to submit proposals to answer research questions based on their clinical experiences. They were required to propose a project in which the entire residency could participate in all elements of the study—from project design and institutional review board (IRB) submission to data collection, data analysis, manuscript composition, and publication. After an HRP-supported project was selected, a faculty advisor with expertise in the area was identified who mentored the resident leaders and participants.

The mechanism to choose the resident research projects evolved over the course of the HRP to maximize resident participation. For the first HRP-supported project (HRP1) selected in 2013, five groups of interested residents met with an associate program director (APD) to propose their research project. Subsequently they submitted a research proposal that was reviewed by a committee of residency leadership. One research proposal for a prospective study was selected. The second HRP-supported project (HRP2) was selected in 2015. HRP2 was designed to support a retrospective chart review using a centralized clinical data registry available to BWH faculty and residents. A short lecture was held on research methods and included a description of the registry. Following the lecture, residents in small groups brainstormed research topics and submitted one or more Requests for Applications (RFA) by the end of the conference. On the same day of the conference, an electronic RFA was sent by email to all residents. All RFAs were reviewed and a committee composed of two APDs and a chief medical resident selected the top six proposals. Through a “shark tank”-style selection process, the finalists were asked to present their proposals at a noon conference and answer questions from their fellow residents. All residents were invited to vote anonymously to select the winning proposal.

Research Curriculum

During the second year of the HRP, we incorporated research-focused lectures into our noon conference series. The content is described in Box 1. The lecture topics were chosen by residency leadership based on feedback from prior and current residents engaged in research during residency. Faculty were chosen based on known local expertise in the lectured areas. This lecture series was repeated every two years. Additionally, brief updates about HRP-supported projects were incorporated into the noon conference series. At the beginning of the conference, the HRP leaders provided a project update, received feedback from residents, and invited additional residents to join the project. Noon conference updates continued every two to three months while the projects were active.

Box 1

Housestaff Research Project Lecture Series, Brigham and Women’s Hospital AY 2014–2015

Introduction to Research MethodsAs the first lecture of the HRP conference series, residents were introduced to the process by which a clinical question can be refined into a research question, including writing a hypothesis and specific research aims. The lecture was developed by the Director of Clinical Research for the Division of Endocrinology.Introduction to the Research Patient Data Registry (RPDR)The RPDR is a centralized clinical data registry, which can be used for retrospective chart reviews. It includes data on adult patients seen at Brigham and Women’s Hospital and Massachusetts General Hospital. Residents were introduced to the power of a large, granular dataset and asked to think of a new study idea, generate a research hypothesis, and submit a proposal for the second HRP. The lecture was developed by a member of the research faculty who uses this database and has mentored prior resident projects using this data registry.Evaluating Scientific Research: An Editor’s PerspectiveGiven by a deputy editor of the New England Journal of Medicine, housestaff were introduced to the process by which editors evaluate manuscripts for publication, including the initial review process by associate and deputy editors, the peer review process, the most common reasons for rejection, and scientific misconduct.How to Write and Publish a Paper: Lessons Learned as an Author, Mentor and EditorThe Chair of Medicine at BWH provided residents with the keys to a successful academic career—encouraging residents to do rigorous research, to publish to establish their scientific legacy and provided prospective on how to accomplish these goals.

Study Design

We used a mixed methods study design, including electronic surveys and semi-structured interviews, and evaluated the success of the first two HRP-supported projects. Two electronic surveys were conducted—in Academic Year (AY) 14–15 and AY15-16. Both surveys explored the research needs and goals of our residents. The AY15-16 survey asked residents about their involvement with the HRP, which had started approximately two years prior. Interviews were conducted with the residents leading the HRP-supported projects. The interviews sought to assess the impact of the HRP on the resident leaders and solicit their insights and advice on the program. Additionally, we evaluated the success of the first two HRPs, including the study design, level of housestaff involvement, and the dissemination of the work through local or national presentations. The evaluation of this program was reviewed by the Partners Human Research Committee and determined to meet criteria for exemption.

Quantitative Data via Survey

Two electronic surveys were sent to all IM residents via email. In AY14-15 and AY15-16, the survey asked about their prior research experience and research interests during residency. In AY15-16, residents were asked about their involvement with the HRP. To maintain confidentiality, the only demographic information collected was postgraduate year (PGY).

Qualitative Data via Survey and Interviews

As part of the AY15-16 survey, all residents were asked for free text responses about the pros and cons of the structure of the HRP. In AY16-17, the three residents leading HRP1 and HRP2 were invited to participate in semi-structured interviews. They were asked about their decision to apply to lead an HRP-supported project, how other housestaff were involved, and how this project compared to any prior research experiences. The interviews were recorded and transcribed for review with the residents’ permission.

Data Analysis

Quantitative

Results are reported by the number of responses to the individual question. Missing data was accounted for as “no response.”

Qualitative

Two researchers (AJA, MAY) performed a qualitative content analysis to identify the themes that emerged from the AY15-16 survey comments and transcripts of interviews with resident leaders. Both coders independently reviewed the transcripts and identified themes. The identified themes were compared and discussed until consensus was reached. Representative quotations from each theme were identified.

Results

Demographic Characteristics

The BWH IM residency program has 190 residents per year. The program includes categorical medicine residents, primary care residents, and interns completing a preliminary medicine year. The response rates for the AY14-15 and AY15-16 survey were 41% (n = 77/190) and 29% (n = 56/190), respectively. In AY14-15, almost 50% of respondents were PGY-1s. In AY15-16, the three classes were more equally represented. (Table 1). Three residents who led the HRPs were interviewed—two male and one female. At the time of the interviews, two were PGY-3 residents and one was a first-year fellow.

Table 1

All Housestaff Survey on Research Interest During Residency and Involvement with the Housestaff Research Project (for AY15-16 Only)

	AY14-15 N (%)	AY15-16 N (%)
Total responses	77 (41)	56 (29)
Year in training
PGY-1	38 (49)	17 (30)
PGY-2	21 (27)	20 (36)
PGY-3 or higher	18 (23)	18 (32)
No response	0	1 (1.8)
Do you consider yourself to have significant research experience?
Yes	49 (64)	30 (56)
No	28 (36)	26 (46)
No response	0	0
Do you feel comfortable designing a research project without faculty guidance?
Yes	23 (30)	18 (32)
No	54 (70)	38 (68)
No response	0	0
Have you submitted an IRB?
Yes	38 (49)	32 (57)
No	37 (48)	24 (43)
No response	2 (2.6)	0
Have you submitted a manuscript for publication?
Yes	67 (87)	50 (89)
No	9 (12)	6 (11)
No response	1 (1.3)	0
Are you interested in more research exposure during residency?
Yes	67 (87)	38 (68)
No	9 (12)	18 (32)
No response	1 (1.3)	0
What percentage of time to you expect to dedicate to research in the future?
0	2 (2.6)	6 (11)
20	34 (44)	30 (56)
50	18 (23)	9 (16)
80	9 (12)	9 (16)
100	0	1 (1.8)
No response	14 (18)	1 (1.8)
Have you heard of the housestaff research project?
Yes	Not asked	53 (95)
No	Not asked	3 (5.4)
No response	Not asked	0
How have you been involved with the housestaff research project?
Have not heard of it	Not asked	5 (8.9)
Helped with HRP1	Not asked	1 (1.8)
Helped with HRP2	Not asked	4 (7.1)
Attended at least 1 noon conference as part of research curriculum	Not asked	20 (36)
Attended at least 1 noon conference with resident update on HRP	Not asked	37 (66)
No response		8 (14.2)

Quantitative Survey Results

The vast majority of residents were interested in additional research exposure during residency. Though many had prior research experience, most felt they still needed significant faculty mentorship for research projects. Almost all residents expected to include research in their future careers, including over 30% who plan to spend at least half of their time doing research. The results are summarized in Table 1.

In AY15-16, 95% of residents had heard of the HRP and 77% had participated in the HRP, mainly through noon conference updates and feedback sessions on current projects. Nine percent (5/56) of survey respondents were directly involved with the day-to-day work of the HRPs.

Qualitative Results

In AY15-16, residents were asked to provide feedback on the pros and cons of the current structure of the HRP via electronic survey. Seventeen residents provided comments. When asked to describe the benefits for the HRP, three themes emerged: (1) collaboration and community building, (2) flexibility, and (3) formal instruction in research. Residents appreciated the research curriculum and having the opportunity to work with other residents on joint research projects, which allowed for varying degrees of commitment. When asked to describe the challenges of the HRP, three themes emerged: (1) difficulty being involved if not the project leader, (2) timing of meetings, and (3) preference for support for individual projects. Residents described the challenges of getting involved in a project if not the project leader and requested additional structure to the HRP projects. Finally, some residents requested more support identifying mentors for individual projects. The themes and representative quotations are included in Table 2.

Table 2

Benefits and Challenges of the Housestaff Research Project as Described by Residents: Analysis of Comments from Survey of All Housestaff

Theme	Representative Quotation(s)
Benefits of Housestaff Research Project
Collaboration and community building	It’s great to have [a] HRP because it’s a wonderful opportunity for us to collaborate on something together and meet other residents with similar research interests.
Flexibility	[The HRP] allows as much involvement as individuals would like.
Formal instruction in research	I like the idea of exposing residents to research methods/skills in a more formal way.
Challenges of Housestaff Research Project
Difficult to be involved if not the project leader	It tends to have a lot of “cooks in the kitchen,” which makes it difficult to determine where I can fit in.It is tough to get involved as an intern. Despite expressing interest and helping with small, well-defined tasks, I was never able to become a fully-fledged member of the team that moved the project forward.
Timing of meetings	I was unable to attend the initial meeting/noon conferences and found it hard to get involved. I emailed one of the organizers and never heard back. It should be easier to get involved.I think building more structure around the HRP, including more formal, regular meeting times and other clearly defined roles for interested housestaff members might take some of the burden off the project leaders and help engage more residents.
Preference for support for individual projects	The current structure of HRP is really directed and a valuable experience to the people who are spearheading it. …. I think instead more should be done to support residents searching for their own research project. Navigating how to find a mentor. Creating an IRB, etc.

All three residents who led HRP-supported projects agreed to participate in interviews. Themes in four categories emerged from these conversations and all themes and representative quotations are included in Table 3. The resident leaders felt the benefits of the HRP were numerous, including: (1) learning from peers and utilizing everyone’s unique strengths, (2) collaborating with peers, (3) focusing on problems identified by residents, (4) creating a resource for the housestaff, (5) distributing work, and (6) creating a culture of scholarship and process improvement. The three resident leaders felt that the HRP was unique from their prior research, largely because the project was designed around collaboration with their co-residents. Both groups were proud to have created new datasets that could be used and expanded for future resident projects.

Table 3

Benefits and Challenges of the Housestaff Research Project as Described by Residents—Analysis of Interviews with Residents Leading First Two Housestaff Research Project-supported Projects

Theme	Representative Quotation(s)
Benefits of Housestaff Research Project
Learning from peers and utilizing everyone’s unique strengths	People who come from many different research backgrounds … so it’s just been cool to see how different people approach a question[The] diversity of perspectives and past experience working with [co-residents] was awesome.
Collaboration with peers	… there were a lot of people involved and at the point of taking care of the patient. ….All the data I have is from the housestaff … anytime a study was ordered by a house officer those are the data I am using,
Focusing on problems identified by residents	[The study question] originated from being a house officer and I talked to people about it like yes this is something we would like to learn about and we think we can impact.
Creating a resource for the housestaff	Having a cleaned data set for the use of housestaff would be the perfect resource for people who are short on time but have high ambitions of conducting research during residency that we could actually create a resource that could be used for years on like a variety of different topics.
Distributing work	And so the idea with this is to actually distribute some of the work again to make it more manageable for conducting a study while you are in residencyMy experience with research in the past has been that the lead author does 95% of the work … so the idea with [the HRP] is to actually distribute some of the work.”
Creating a culture of scholarship and process improvement	I think it’s just a really special thing that we do and I am lucky we are all lucky that we do it and this culture of changing and studying
Challenges of Housestaff Research Project
Difficulties accommodating all residents who express interest	We have gotten more interest than we have been able to accommodate. More people have been interested. . . than we have been able to either delegate work for or create learning opportunities for.
Scheduling challenges	Everyone has very little free time and we do get these windows of time where we have the bandwidth to do concentrated amounts of work but those hardly ever align with anyone else who is doing projects.The resident schedule is unpredictable.
Limited resident time	Everyone’s busy and sometimes it’s difficult to explain what the project is when you are in the heat of intern year.
Difficulty collaborating on certain tasks	Some of the tasks just are not easy to do collaboratively.Writing a manuscript is not easy with 10 people so we are dividing that part up and saying lit review versus methods part versus results versus impression but at the end of the day there will be one or two of us making sure the language was same and together.
Keys to Engaging Residents in a Housestaff Research Project
Resident champions	But probably the most important thing is having one, or two … champions that sort of just actually still are doing the majority of the work, driving things forward, setting up the meetings, things like that.Telling people hey this is still going on even though you were in the MICU for a month and then did 3 months of inpatient, you can jump in at anytime … as long as there is someone doing that you can remind people to come back.
Noon conference updates	Noon conference updates are probably the most useful thing we do because it’s like the largest audience . . . and you can catch them at this great opportunity which is critical.
Suggestions for Improving the Structure of the HRP
Creation of a research track	I think one thing that could make particularly this type of project more successful is to sort of create a scheduling track like similar [tracks] in the residency, to have that coincide with classes that you could take at the School of Public Health for example. I think that that would help get around the scheduling issue and help to also satisfy the desire to have some formal teaching curriculum around this.

Challenges of the HRP as identified by resident leaders were: (1) difficulties accommodating all residents who expressed interest, (2) scheduling challenges, (3) limited resident time, and (4) difficulty collaborating on certain tasks. They sought to balance maximizing resident engagement with busy resident schedules and the need for a standard approach for chart reviews and manuscript preparation. They identified resident champions and noon conference updates as critical to facilitating and maintaining resident engagement. Finally, some advocated for the creation of a structured research track for all residents interested in the HRP to minimize scheduling challenges and allow for the creation of a more robust research curriculum.

Description and Success of Prior Housestaff Research Projects

The descriptions of HRP1 and HRP2 are based on information from the interviews with HRP leaders and the senior author’s knowledge of each project. HRP1 was a prospective study on the impact of an electronic clinical decision tool for cardiovascular imaging tests on patient care, resource utilization, and cost. The faculty mentor was an Associate Professor in Medicine and Radiology at Harvard Medical School (HMS) and Associate Director of the Cardiovascular Imaging Program with a research expertise in cardiovascular imaging. Housestaff involvement included: a literature review, developing the clinical decision-making tool, and case review. Six residents were actively involved, and about 100 residents provided feedback on the clinical decision-making tool. This project has been presented at two local (BWH Cardiovascular Imaging Poster Session and BWH Housestaff Research Day) and one national (Annual Scientific Session of the American Society of Nuclear Cardiology) conferences.

HRP2 was a retrospective chart review on the relationship between psychiatric illness and hospital readmissions and cost. The faculty mentor was an Associate Professor of Medicine at HMS and an experienced researcher in clinical outcomes research. Housestaff involvement included: hypothesis generation, dataset variable generation, literature review, and a chart review to validate their definition of psychiatric illness. Ten residents were actively involved with the project. Another ten residents expressed interest but did not participate. This project has been presented at one local (BWH Housestaff Research Day) and two national (Society for General Internal Medicine and Academy Health) conferences.

The manuscript for the first project is in press, and a manuscript for the second project is in preparation. The resident leaders were the first author or co-first author on the manuscripts. The remaining residents are listed under “Brigham and Women’s Hospital Internal Medicine Residency.” In the acknowledgement, individual names are listed alphabetically. Both projects have resulted in large databases which can be used for future resident projects.

Discussion

Participation in scholarly activity in IM training is required by the ACGME and critical to the professional development of residents. Despite the clear benefits, research opportunities in residency are a challenge in IM and other specialties. We presented the HRP as one strategy for creating meaningful research opportunities in IM training. It was designed to engage all IM residents in research methods and projects to inspire them to become better clinicians and collaborative researchers. The only prior study describing a resident research program that engaged all housestaff in a resident-driven project required significant oversight from faculty leadership.14 To our knowledge, the HRP is the first research program that was centered on resident-led projects in which residents took the lead in engaging and collaborating with their co-residents.

The creation of the HRP broadened the exposure of research concepts and methods to all BWH IM residents. Similar to the experience of other programs,4,6,8 the HRP created a culture of research and scholarship to which all residents were exposed. Our survey data suggested that most residents were familiar with the HRP, even if they were not directly involved in the HRP-supported projects. Through residency-wide noon conference updates, all residents learned from the HRP. The noon conference curriculum and updates exposed residents to research in three ways. First, by asking residents to propose research questions and select the research project, they become engaged in hypothesis generation. Second, project updates illustrated the challenges of robust data collection and ensuring the integrity of the data. Third, the presentation of the final data introduced core concepts about data analysis. HRP project leaders identified noon conference updates as critical to housestaff engagement. Other studies demonstrated that a longitudinal research curriculum was important for maintaining momentum and keeping residents engaged in research projects.4

Resident feedback on the HRP was generally positive. Residents appreciated the ease with which they could identify problems from their clinical rotations and work to better understand and address them. They enjoyed working with their co-residents and learning from them in new ways. Some residents valued the flexibility afforded by the HRP as they could be engaged in some aspects of a research project without having full responsibility. HRP resident leaders identified the distribution of work across many residents as a benefit of the HRP, especially given busy and variable resident schedules. Though not reflected in resident comments, our survey data and prior studies suggest that faculty mentorship is a critical aspect of resident research and residents likely benefited from ease of mentorship through the HRP.

The HRP was not without challenges. Both HRP-supported projects struggled at times with “getting things done.” Having strong resident leaders proved necessary for successful projects, as they were responsible for ensuring the work was completed while simultaneously trying to generate additional interest to maximize housestaff involvement. Resident leaders balanced residents’ expectations, including the time necessary to complete tasks. This led to the success of the projects, including creating large databases that can be used for future research projects, presentations at national and local meetings, and publication in peer-reviewed journals manuscripts. However, the resident leaders described challenges incorporating all of their peers who expressed interest in the day-to-day work of the project, especially for tasks that were difficult to perform collaboratively.

Some residents preferred leading their own individual projects while others appreciated joining a group project, like the HRP. The resident leaders and residents that participated in HRP1 and HRP2 expressed the most direct benefits from the program. Though they sought to engage all interested residents, a continued challenge was how to maximize the number of participants while maintaining the integrity and the value of the project for each individual resident. When the HRP projects were not a good fit for a resident, residents interested in research were still able to pursue individual research projects in the Department of Medicine.

Still in its early years, the HRP is now an integral component of research education at BWH. Our evaluation of the HRP has allowed us to continue to refine its structure. Two different approaches were used for the selection of HRP-supported projects. We favor the “shark-tank” selection approach implemented for HRP2. This approach resulted in a larger group of residents becoming more actively involved from the beginning of the project. Expanding the research curriculum remains a priority for the HRP. The research curriculum described sought to educate residents about the research process. Residents continue to express a desire to learn about research techniques and methods, but their schedules remain the main challenge to disseminating any curriculum outside of the typical conference times. Other programs have successfully incorporated research methods curricula, but this required significant curricular time.4,7,9 One strategy suggested by the HRP leaders was the creation of a research pathway which would include more curricular time on research methods, similar to the PRIME pathway at University of California, San Francisco.5

This study had several limitations. We presented results from a single institution that may not be generalizable to other programs. Our ability to determine the impact of the HRP on the entire housestaff was limited by a low response rate to the AY15-16 survey and we were unable to measure the HRP’s ability to inspire residents to pursue their own independent research projects.

The HRP significantly impacted our resident leaders and many resident participants. We plan to broaden the reach of the HRP by selecting future projects that maximize the number of resident participants and integrating a more robust research curriculum. We plan to fund one project every one to three years using the “shark-tank” selection approach. It is beneficial for residents to learn about both prospective and retrospective research studies and therefore, will vary the RFA to allow proposals of each study type. We plan to assess the impact on residents’ research careers five years after graduation.

We present this description of the HRP as a possible new approach for other programs that seek to facilitate and encourage research and scholarship within their residencies.