Next generation pathways into biomedical informatics: lessons from 10 years of the Vanderbilt Biomedical Informatics Summer Internship Program.

OBJECTIVES: To examine roles for summer internship programs in expanding pathways into biomedical informatics, based on 10 years of the Vanderbilt Department of Biomedical Informatics (DBMI) Summer Research Internship Program.
MATERIALS AND METHODS: Vanderbilt DBMI's internship program is a research-intensive paid 8-10 week program for high school, undergraduate, and graduate students. The program is grounded in a "Windows, Mirrors, and Open Doors" educational framework, and is guided by an evolving set of design principles, including providing meaningful research experiences, applying a multi-factor approach to diversity, and helping interns build peer connections.
RESULTS: Over 10 years, 90 individuals have participated in the internship program, with nine students participating for more than one summer. Of 90 participants, 38 were women and 52 were men. Participants represented a range of racial/ethnic groups. A total of 39 faculty members have served as primary mentor for one or more interns. Five key lessons emerged from our program experience: Festina Lente ("Make haste slowly"), The Power of Community, Learning by Doing, Thoughtful Partnerships Lead to Innovation, and The Whole is More Than the Sum of Its Parts. DISCUSSION: Based on our experience, we suggest that internship programs should become a core element of the biomedical informatics educational ecosystem. Continued development and growth of this important educational outreach approach requires stable funding sources and building connections between programs to share best practices.
CONCLUSION: Internship programs can play a substantial role in the biomedical informatics educational ecosystem, helping introduce individuals to the field earlier in their educational trajectories.

INTRODUCTION

Biomedical informatics is a multi-disciplinary field, drawing from medicine, computer science, engineering, social sciences, and beyond., This broad foundation is a strength, but also leads to challenges in establishing clear educational pathways into the field., The biomedical informatics educational ecosystem continues to evolve, reaching into new environments such as undergraduate programs,, community colleges, and even working with high school students., Programs such as the American Medical Informatics Association (AMIA) High School Scholars Program and the AMIA First Look Program have expanded educational outreach in biomedical informatics. Despite these programs, significant questions remain about how to attract diverse students into informatics educational pathways, even earlier in their educational and career trajectories.

Fortunately, biomedical informatics can draw on the lengthy educational outreach experience of other Science, Technology, Engineering, Mathematics, and Computing (STEM-C) fields. Multiple disciplines have recognized the importance of earlier engagement with students as a major contributor to attracting diverse young talent. Public engagement activities such as Computer Science Education Week, National Engineers Week, and Scientist in the Classroom Programs increase awareness about educational opportunities and career paths. These short-duration outreach activities are important, but disciplines have also developed longer-term engagement strategies. Research Experience for Undergraduates (REU) programs, funded through the National Science Foundation (NSF), have promoted substantial engagement in a wide range of scientific fields for over 30 years., Fields such as astronomy and space science have long-standing internship programs for high school and undergraduate students, funded through NASA, with the aim of developing the next generation of scientists and engineers in space exploration., Individual educational institutions have also explored targeted STEM-C engagement activities at the high school level, such as the Harvey Mudd Women’s Introduction to Science, Technology, Engineering, and Mathematics Program.

Vanderbilt University’s Department of Biomedical Informatics (DBMI) recognized the immense potential of earlier interaction with students and established a summer research internship program in 2008. The objective of this manuscript is to examine roles for summer internship programs in expanding pathways into biomedical informatics, based on ten years of the Vanderbilt DBMI Summer Research Internship Program. The manuscript presents the model underlying the program, describes the program’s evolving design principles, discusses the unique programmatic elements that have led to successful engagement with almost 100 talented young individuals, and explores next steps for educational outreach in biomedical informatics.

METHODS

Vanderbilt DBMI’s internship program is a research-intensive 8–10 week summer program for high school, undergraduate, and graduate students. The internship program has a multilayered design that builds on a “Windows, Mirrors, and Open Doors” framework, with the goal of addressing needs of students, faculty, and the department.

Conceptual framework

The conceptual framework underlying the program can be described as “Windows, Mirrors, and Open Doors” (Figure 1), building on concepts from diversity studies in education and children’s literature. Biomedical Informatics is inherently difficult to grasp as a field, in part because of the wide range of informatics specializations and the dense jargon common in technical fields. Windows into the field, such as minimizing use of jargon and using communication strategies to reach non-experts,, can help novices to informatics understand what the field entails. Research on women pursuing STEM degrees has demonstrated that the presence of similar individuals in a field can help build confidence and retain individuals. Groups that are historically and currently under-represented in biomedical informatics, including women and members of under-represented racial/ethnic groups, face challenges in identifying similar role models. Efforts such as the Women in AMIA Initiative seek to establish Mirrors that can assist members of under-represented groups with picturing themselves in biomedical informatics. Learning about a field and seeing potential for involvement are both important, but without access to get involved, interested individuals cannot actually reach the opportunity. This leads to the third component of this conceptual model: Open Doors. Providing access to opportunities to explore a field is a necessary third component of the conceptual model. Our program seeks to address all three model components, providing learning opportunities to expose interns to the field, mentoring and interaction with graduate students, faculty, and peers, and the opportunity to join a research team.

Figure 1.

Windows, mirrors, and open doors conceptual framework.

Program goals

The internship program has grown and evolved over the last 10 years, including the goals of the program. The program started with a broad, open-ended goal: to explore how a department focused on research and an educational program focused on graduate and post-doctoral training could engage with undergraduate students. A substantial motivation behind this goal was developing diverse and inclusive pathways into informatics, including building new pathways for women and members of other under-represented groups. This goal has continued as a driving force behind the program over the past 10 years. We have developed additional specific program goals over time. One substantial goal added later to the program was building partnerships at the high school level, to support involvement of high school students and educators in informatics. Another additional goal was attracting diverse young people to careers in the biosciences, in research, and in biomedical informatics through program participation.

Program design principles

Several core program design principles motivated the initial program structure, with additional design concepts emerging as the program grew and as participants provided feedback (Table 1). Two foundational principles have guided the program since inception: providing students with meaningful research experiences and linking students with mentors to provide guidance on research projects and educational/career trajectories. The program has taken a multi-factor approach to diversity that continues to evolve, with a current goal of ensuring diversity across gender, race/ethnicity, socioeconomic status, age, educational level, and specialization. Recruiting interns from different backgrounds and perspectives requires special attention and effort. Since biomedical informatics is a broad field with many different potential specializations, students need illustrations of different pathways, beyond their specific research project, and mentor. Ensuring students remain engaged throughout their experience and on track with individual internship goals are crucial components, especially considering the 8–10 week duration. A major program design principle that emerged based on student feedback was the importance of building connections with peers, a concept that interns have repeatedly discussed both as a short-term and long-term program benefit. Finally, when high school students were included in the intern cohort, the program incorporated additional elements to address the unique requirements of working with minors.

Table 1.

Program design principles and program components

Program design principle	Program component	Component type
Providing a meaningful research experience	Project ideas submitted by faculty mentors; matching process between projects, mentors, and students	Individual; program administration
Mentoring on research, education, and career	Mentor-mentee agreement; meetings between faculty and intern; inclusion of graduate students in mentoring; seminar series	Individual
Multi-factor approach to diversity	Interns are paid for their work; recruiting across multiple channels; including high school, undergraduate, and graduate students as interns; seminar from medical school diversity-focused partner	Program administration
Recruiting	Multiple approaches to recruiting; partnership with local high school	Program administration
Illustrating educational and career pathways	Twice-weekly seminar series on biomedical informatics topics and career pathways; encouragement to meet with additional faculty members depending on research interests	Group
Student engagement	Meetings between interns and faculty mentors; seminar series; role of program administrative co-ordinator	Individual; group
Building connections with peers	Seminar series; presenting research results to peers; co-location of intern workspaces; opportunities to socialize	Individual; group
Addressing special requirements when working with minors	Protection of minors training; partnership with local high school; role of program administrative co-ordinator	Program administration

Program implementation

The Vanderbilt DBMI internship program implemented the design principles through multiple program components, grouped into program administration, individual program elements, and group program elements (Table 1).

Program administration

An underlying program administration structure supports internship program activities. Program administration tasks include: recruiting intern candidates, collecting faculty research projects, matching students with mentors and projects, overseeing routine administrative tasks (eg, completion of required paperwork, access to equipment/supplies), and promoting intern engagement in the program and DBMI organizational culture. Key roles involved in program administration include an administrative co-ordinator and a faculty co-ordinator, working collaboratively to ensure the program runs smoothly and to rapidly resolve any issues. The program administration also implements the program’s multi-factor approach to diversity through activities such as: securing program funding, connecting with partner organizations, and recruiting across multiple channels. When the internship program began working with high school students in program year 5, the program administration also took on tasks specific to the special requirements of working with minors including ensuring that all individuals working closely with minors complete Protection of Minors training. The program administration has also worked to build a partnership with a local high school, to better understand high school student needs and to engage teachers in biomedical informatics educational outreach.

Individual program elements

Individual program elements consist of interns working on their specific research projects. Faculty mentors have direct responsibility for overall research projects, while graduate students and research staff often become involved in day-to-day project activities. At the internship start, interns and their faculty mentors complete a mentor-mentee agreement, to set expectations and provide groundwork for discussions of important internship details (Supplementary Appendix SA). Vanderbilt DBMI internships are paid positions, although compensation varies according to educational level and funding source.

Group program elements

The group program elements are activities in which all interns participate, to build a peer network of students interested in biomedical informatics. Intern workspaces are typically located together to support interactions and networking. Interns also have opportunities to socialize during work breaks, through activities outside of work, and with other programs through the Vanderbilt Summer Science Academy. All interns participate in twice-weekly seminars presented by DBMI faculty, with topics including: research areas, educational/career paths, research methods, and diversity/inclusion. At the end of the summer, interns present their work to peers, graduate students, and faculty. Collectively, the program administration, individual components, and group components form a stable support structure for interns.

Program evaluation and assessment

Identifying appropriate approaches to evaluate and assess the emergent internship program design has proven challenging, particularly as the educational trajectories of many former interns are still in process. Using information from faculty mentors and staff associated with the program, the career website LinkedIn (https://www.linkedin.com/), and university websites, we retrospectively assessed educational/career trajectories of former interns. Educational/career trajectories were divided into leading indicators (eg, enrollment in a technology, bioscience, or health-related educational or training program) and lagging indicators (eg, completion of a biomedical informatics degree, enrollment in a graduate degree program in a STEM field, current employment in a role related to biomedical informatics) to account for the many past participants still enrolled in educational or training programs. We also used PubMed, the AMIA Knowledge Center, and information from faculty mentors to identify publications with mutual authorship between a program participant and their faculty mentor.

In addition, on a prospective basis with each cohort of interns, the program has conducted an end-of-internship participatory design workshop with interns, beginning in program year 7. The workshop purpose was two-fold: to encourage interns to reflect on their internship experience for future plans and to gather data about what was working well in the program and potential improvement areas. Qualitative data from the design workshops, combined with quantitative analyses of educational/career trajectories and discussions among the authors regarding program performance, helped to inform the development of key program lessons explored in detail in the results.

RESULTS

Participant characteristics

Over 10 years, 90 individuals have participated in the internship program, with nine students participating for more than one summer, for a total of 99 internship experiences (Table 2). During the 99 internship experiences, most participants were undergraduate students, whereas high school students were added to the program in Year 5. Graduate students also participated during some program years.

Table 2.

Cohort size and educational level

Program year	Year	Interns (total per year)	Educational level at time of internship (number of interns per level)
			High schoola	Undergraduate	Graduate
1	2008	2	0	2	0
2	2009	5	0	5	0
3	2010	12	0	8	4
4	2011	12	0	10	2
5	2012	10	1	9	0
6	2013	10	1	8	1
7	2014	10	2	8	0
8	2015	10	3	5	2
9	2016	13	3	9	1
10	2017	15	5	8	2
	Total	99	15	72	12

Includes students between high school graduation and first year of college.

Of 90 participants, 38 were women and 52 were men. Participants represented a range of self-reported racial/ethnic groups, with 41% (n = 37) White/Caucasian, 26% (n = 23) Asian, 19% (n = 21) Black/African American, 8% (n = 7) Hispanic/Latino, and 4% (n = 4) from groups not represented by other categories.

Mentor and research project characteristics

Thirty-nine faculty members served as primary mentor for one or more students over the past 10 years. Over 10 years, 18 faculty mentored one intern, 10 mentored two interns, three mentored three interns, three mentored four interns, and five mentored five or more interns. Faculty mentors represent the range of research within DBMI, including bioinformatics, clinical informatics, data science, human factors engineering, and sociotechnical systems. Likewise, research projects have spanned the range of research within DBMI. While some projects were computationally-intensive and required strong computer science skills, other projects applied qualitative methods, opening the program to students with social science backgrounds. The primary intern project requirement was a well-bounded research project feasible to complete within 8–10 weeks, with other details and requirements left to faculty mentors. Mentors have also included graduate students and research staff in DBMI, providing another mentoring layer. Faculty mentors are required to meet with interns at least once per week to provide guidance and feedback, although in practice faculty-intern interactions occur more frequently. The actual time involved in mentoring interns varies substantially depending on the research project and on the intern. Many mentoring activities especially for staff and graduate students occur on an impromptu basis, when interns need basic help with resolving challenges or discuss research-related questions during social interactions like coffee breaks.

Program financial support

Several different funding sources have supported the program since 2008, including training grants from the National Library of Medicine (NLM), funds from individual investigators, and other internal funding sources (Table 3). Initial program funding came through a small number of short-term training positions on the department’s NLM-funded training grant. The major program growth period was supported through a supplement to DBMI’s NLM training grant, as part of the American Reinvestment and Recovery Act, and later through an NLM-funded R25 grant. When the NLM transitioned away from R25 participation, internal Vanderbilt funding supported the 10th program year, providing a bridge while we worked to identify new potential funding sources. The majority of program funds throughout the 10 years of the program have gone to participant support costs, including paying a monthly stipend or an hourly wage to participating students, depending on funding source. Due to limitations of funding sources, for the first 10 years of the program, students were expected to cover costs such as housing and meals on their own. Internal funds have supported the majority of program administration costs throughout the length of the program, with the R25 mechanism providing 10% support for the faculty co-ordinator and 20% support for the administrative co-ordinator during the 10 weeks of the internship. The faculty co-ordinator role was initially integrated into the Vice Chair of Education for DBMI (C.S.G.) who estimated running the program took 20% effort for 15 weeks each year. Beginning in 2016, the faculty co-ordinator role was transitioned into a separate role (K.M.U.), with work supported by internal educational funds. Support for faculty mentoring activities was covered as part of existing internal compensation for faculty departmental activities.

Table 3.

Financial support for internship

Year	Financial support for internship (number of interns per funding source)
	Facultya	NLM T15 STTPb	ARRA T15c	R25d	Other internal fundse
2008	0	2	0	0	0
2009	0	3	2	0	0
2010	0	4	8	0	0
2011	0	5	0	7	0
2012	0	2	0	8	0
2013	0	3	0	7	0
2014	1	0	0	9	0
2015	0	3	0	7	0
2016	0	4	0	9	0
2017	1	4	0	0	10
Total	2	30	10	47	10

NLM: National Library of Medicine.

Financial support directly from a faculty sponsor.

Short-term training positions for individuals from diverse backgrounds and women (NLM T15LM007450, PI: CS Gadd).

Supplement to T15 training grant from the National Library of Medicine as part of the American Reinvestment and Recovery Act (NLM T15LM007450, PI: CS Gadd).

Vanderbilt Biomedical Informatics Summer Research Experience Program (NLM R25LM011174, PI: CS Gadd).

Financial support from Vanderbilt funds not directly tied to a specific student-mentor relationship.

Retrospective analysis identified post-internship educational/career trajectories for 74 of our 90 former interns (Table 4). Over one-third (n = 36) of the past interns, who we were able to follow are currently involved in an educational program. A substantial number of past interns (n = 33) remained involved in research in some way, either as a researcher or in roles such as research co-ordinators or research program managers. Over half of students (n = 40) were retained in computer science or CS-related technical fields, with half of the students (n = 45) remaining involved in the Biosciences or Healthcare fields. Participation in a professional or graduate degree program is a lagging indicator of program impact, as interns must first complete their undergraduate degree. As expected from a lagging indicator, a smaller number of students completed or are currently pursuing a healthcare professional degree (n = 16) or graduate degree in the Biosciences or Healthcare fields (n = 18). A total of 15 interns are currently directly involved in biomedical informatics in some way, either through pursuit of a graduate degree (n = 8) or through employment in biomedical or clinical informatics roles, such as working for an electronic health record vendor. Notably, three former interns are currently graduate students in Vanderbilt DBMI. Approximately a third of interns (n = 28) were included as co-authors on peer-reviewed conference submissions or journal publications, including 9 peer-reviewed journal articles.

Table 4.

Post-internship educational and career trajectory

Category	Yes	No	Too early to identify
General educational/career path
Involved in any research-based capacity (eg, researcher, research assistant, research manager/co-ordinator)	33	29	12
Involved in any bioscience or healthcare field (eg, healthcare provider, research-related role, pursuing a healthcare professional degree)	45	18	11
Employed in or currently pursuing a degree in CS or a CS-related technical field	40	23	11
Pursuit of graduate education
Completed or currently pursuing a healthcare professional degree (eg, MD, DDS, PharmD)	16	47	11
Completed or currently pursuing a masters or doctoral degree in a Biosciences or Healthcare field (eg, MS, MPH, PhD)	18	44	12
Involvement in biomedical informatics
Completed or currently pursuing a biomedical informatics graduate degree (eg, biomedical informatics, bioinformatics, health informatics)	8	54	12
Directly involved in biomedical informatics (eg, completed or currently pursuing a biomedical informatics graduate degree, employed in biomedical or clinical informatics, conducting biomedical informatics research)	15	47	12
Involved in a field adjacent to biomedical informatics (eg, computational biology, human genetics, biostatistics)	8	54	12
Publishing with faculty mentor
Interns included in a peer-reviewed publication with their faculty mentor in any author position, including conference submissionsa	28	62	NA
Interns included in a publication with their faculty mentor in any author position, peer-reviewed journals onlya	9	81	NA

Notes: Subcategories within each high level category have some overlap. For example, a student may be both involved in research and employed in a CS field. Table only contains data on 74 interns whose trajectories we were able to track, except rows marked with a which contain data on all 90 interns.

CS: computer science.

As an annual assessment of program performance, the output of the end-of-internship participatory design workshop has led to several program refinements. For example, interns discussed challenges at the internship start, related to confusion about where to go for administrative needs and lack of clarity around their research projects. As a result, the program revised the orientation process to provide additional administrative details. We also added a mandatory mentor-mentee agreement to establish common ground and set research project expectations at the internship start. Interns also provided feedback about what is working well in the program and about what program elements are important to them, including the twice weekly seminar series and proximity of individual workspaces to other interns.

Key program lessons

In assessing ten years of experience at Vanderbilt, we have identified five key lessons for development and growth of internship programs in biomedical informatics:

Festina Lente (“Make haste slowly”).

The Power of Community.

Learning by Doing.

Thoughtful Partnerships Lead to Innovation.

The Whole is More Than the Sum of Its Parts.

Each key program lesson is discussed in greater detail below, and explored through a case study of Vanderbilt DBMI’s high school outreach efforts (Box 1).

Box 1

High school outreach: a case study of the windows, mirrors, open doors framework and program lessons in action

Framework application

Five program lessons

Windows (learning about informatics) Worked closely with a high school computer science teacher so she could learn about the scope and content of biomedical informatics DBMI faculty presented to high school students early in the partnership to discuss informatics and the upcoming internship opportunity Remaining goal: faculty doing monthly “lunch and learn” sessions at the school Mirrors (seeing yourself reflected in informatics) Focused on students between their junior and senior years for summer internships; students report back to classmates about their experience and have continuing support for further exploration in following school year Connected high school interns with undergraduate and graduate students to understand next steps in informatics career pathways Ensured high school students had a welcoming experience during their DBMI internship Open doors (Increasing access to informatics pathways) Funded at least two high school interns each year Matched students with projects/mentors that were within a reasonable scope for their knowledge and skills Encouraged eligible students to apply to the AMIA High School Scholars Program

1. Festina Lente (“Make haste slowly”) Started small with high school outreach: one student intern, one high school Expanded as we learned about challenges and benefits of working with high school students 2. The power of community Incorporated layers of support for high school students into the program structure: high school peers, undergraduate students, graduate students, research staff, and faculty mentors Matched high school interns with faculty/projects that had other students/staff involved to ensure they could seek help and feedback from multiple sources, when possible 3. Learning by doing High school interns were given ownership over an appropriately-scoped research project High school interns presented on outcomes of the research project at the end of the summer Provided feedback to students when who applied to the AMIA High School Scholars Program 4. Thoughtful partnerships lead to innovation Focused on a high school partnership model, ensuring both our program and the high school benefited from the partnership Funded computer science teacher at our partner school to attend an AMIA education-focused conference, where she participated in a K-12 outreach panel and attended conference sessions Explored potential STEM-focused funding opportunities with other local colleges/universities in collaboration with our partner high school 5. The whole is more than the sum of its parts Too early in the educational trajectory of our former high school interns to identify clear biomedical informatics outcomes Strong feedback from high school teacher, administrator, and students about the impact of the program: students gained confidence in technical and academic skills, pursued new academic directions, applied to colleges/universities they had not considered before the internship, and thought about education and educational outcomes differently

Festina lente (“Make haste slowly”)

Although the Latin phrase Festina Lente or the similar adage “Go slow to go fast” might seem counter-intuitive, this concept was a guiding principle in developing the summer internship program. The program started on a small scale, allowing us to test program ideas, learn from experiences, and build on lessons from each program iteration. Over 10 years, the program has steadily gained traction and momentum, establishing a stable foundation for growth and exploration. For example, faculty engagement is a critical aspect of the program, as faculty provide research direction and mentoring. Understanding faculty needs, lowering barriers to involvement, and communicating about the value of the internship program have helped to ensure engagement of faculty mentors. The program infrastructure, built around the design principles in Table 1, provides crucial support by managing the administrative aspects and organizational requirements of an internship. Having the right components in place when needed has supported the program’s thoughtful and purposeful evolution.

The power of community

A second key program lesson is that opportunities to build peer connections need to be designed into the internship experience, because a peer community provides a powerful source of support. The group components of our internship program encourage interactions among the intern cohort, with peers at similar education levels and with similar interests, providing a foundation for development of a sense of community among interns. In addition, when possible, we locate intern work spaces near each other to further facilitate peer interactions. Over the years, intern feedback indicates that this informal peer community is an important element of the internship experience. For example, although interns worked on different research projects, they have used similar methods or tools or dealt with similar concerns about faculty interactions. The intern peer community has provided support to help interns resolve problems, answer questions about their projects, and learn from others.

Learning by doing

The program allows interns to learn about biomedical informatics by doing research. More importantly, the scope of what interns learn during their internship is wider. Even at an early stage in their educational trajectory, interns learn that presenting their scientific research improves the scientific process. By taking ownership over a portion of a research project, interns also learn about how research teams collaborate and about the importance of feedback. In applying their skills and knowledge to a research question and reporting about their results, interns gain confidence in their capabilities. In several cases, by completing our internship program, interns have learned that biomedical informatics is the field they want to pursue.

Thoughtful partnerships lead to innovation

Getting messages about biomedical informatics to students early in their educational trajectory has proven challenging. We have learned that identifying and partnering with appropriate schools can serve as a major facilitator for student recruitment. For example, as our internship program expanded to the high school level, we developed a partnership with a local high school that offered an AP Computer Science course. The high school partnership model has assisted with recruiting interns with appropriate skillsets. Furthermore, thoughtful partnerships can lead to innovations in educational outreach efforts. One area that we are currently exploring is outreach to K-12 educators. Our high school partnership has provided input and guidance in that effort, pushing boundaries and providing support for innovative thinking in educational outreach.

The whole is more than the sum of its parts

Based on our program experience, short-term research experiences benefit students, faculty, and Vanderbilt DBMI as a whole to a much greater extent than the sum of each individual program component. The internship experience has provided interns with meaningful biomedical informatics research experiences and has served as a recruitment tool for potential graduate students. Through our program, interns learned about a specific research topic and about the broader field of biomedical informatics. For faculty, working with summer interns has provided an entry into mentoring students at different skill and knowledge levels, crucial experience for ongoing academic mentoring work. Helping to mentor summer interns has also provided access to mentoring and teaching opportunities for DBMI graduate students. Summer interns have also helped with developing new research areas, assisting faculty with gathering preliminary data for grant applications and extending research into new areas not covered by current funding. Each program component has contributed an important element, but 10 years of experience demonstrates that the actual picture of what the program has contributed to our biomedical informatics educational ecosystem is much bigger.

DISCUSSION

The development and evolution of the Vanderbilt DBMI Summer Internship Program illustrates important lessons regarding the benefits and limitations of short-term training experiences in medical informatics. The program also highlights the role that internship programs are beginning to play in the biomedical informatics educational ecosystem. The program’s integrated model provides a supportive and inclusive community for interns. While interns conduct meaningful individual research, being part of a larger internship program allows interns to engage with peers, expands their knowledge of the field, and provides support for day-to-day needs. While the program has benefitted interns, it has also benefitted faculty mentors and Vanderbilt DBMI as a whole.

Should internship programs be a core element of the biomedical informatics educational ecosystem?

Ten years of experience here at Vanderbilt strongly supports the idea that internship programs should become a core element of the biomedical informatics educational ecosystem. Answering this question requires examining three additional questions. First, should biomedical informatics educational outreach continue expanding? Based on developments in biomedical informatics education such as undergraduate degree programs and high school outreach, the answer to the first question is an unequivocal yes. Biomedical informatics is now a well-established field and the demand for individuals with biomedical informatics training and skills continues to grow. Continuing to expand educational outreach in biomedical informatics is a logical way to build new pathways into the field and attract new talent earlier in their careers.

Second, are individual internship opportunities an adequate outreach approach or are there benefits to a structured internship program? This question requires a more nuanced answer. Individual approaches to funding short-term training opportunities, such as diversity supplements to several types of NLM-funded grants, are one important pathway to engaging more students in the field, although there are some challenges in connecting students to opportunities. As our article clearly shows, substantial effort is needed on an upfront and ongoing basis to develop, maintain, and grow an internship program. Has that effort really been worth the cost in resources and time? The experience of 10 years of the Vanderbilt DBMI Summer Internship Program demonstrates the added value that a program structure contributes to student and faculty experiences. In discussing internships with programs around the country and in our own experience, one major barrier to faculty involvement in mentoring interns is a concern regarding productivity and bandwidth. A robust program structure helps to overcome this barrier. The structure of Vanderbilt’s internship program allows faculty to focus on mentoring by engaging a faculty co-ordinator in managing initial internship components and an administrative co-ordinator in managing ongoing logistical components. For students, the internship program structure helps provide support and promotes a safe and inclusive environment, where interns are able to voice concerns and where potential issues are address proactively. Through our internship program, students are able to build connections with peers interested in the same area, forming a community and a network they can continue to build on after their internship.

Third, and perhaps most significantly, can the costs of internships be justified relative to the benefits they provide to students, departments, and the field? On a per-participant basis, internships are relatively inexpensive in large part because of the time-limited duration of the interaction. Adding program administration increases costs, but delivers substantial benefits. However, as a whole, how can we measure whether the cost of the program is worth it? Over the years, we have heard many powerful stories about how program participation influenced educational and career decisions. Taking a closer look at the quantitative metrics about program outcomes (Table 4) backs up the anecdotes that we have heard: well over half of our interns have gone on to education or careers in the biosciences, in healthcare, in computer science, and in research. Several interns have made a substantial enough research contribution to be included on one or more conference submissions or other publications with their faculty mentor. The number of interns going on to informatics graduate education is small but notable, given the challenges of recruiting students to graduate degree programs in the biosciences. As recent intern cohorts complete undergraduate degrees, we anticipate that this number will continue growing. Balancing the relatively low costs of an internship program with these tangible outcomes provides justification to pursue these types of programs.

What is needed to continue the growth of internship programs in biomedical informatics?

More work is needed to move beyond the current level of internship programs in biomedical informatics. Stable and reliable funding sources are one of the greatest needs in starting and growing internship programs, both to provide compensation for interns and to support development of helpful program components. Providing a reasonable wage or stipend for interns is critical for the important goal of expanding opportunities in the field. While some students may be able to work without wages in the summer, providing a wage/stipend supports access to opportunities for individuals across socioeconomic classes. Paying interns for their work also supports a standard of compensation for academic effort and represents an aspect of social justice, with the idea that people should be fairly compensated for work. In addition, as informatics research competes for the same skillsets as summer internship opportunities in industry, compensation is a requirement. Although paying interns is one important component of funding a summer internship program, administrative program components, and extra mentoring activities by faculty also need funding support. Multiple administrative components have been needed to support the success of our internship program model, and finding funding for these administrative components is challenging. Working with students of various educational levels in a short-term research experience requires additional effort from faculty mentors compared to a longer-term graduate or post-graduate mentoring relationship. Faculty are rarely compensated for this additional effort, creating a barrier to engaging faculty in short-term research programs.

The other substantial need for continued expansion of internship programs is the need to connect across programs, to share best practices and other information. The lessons that Vanderbilt DBMI has shared about our internship program through this article are one piece of the picture. What is urgently needed is a consortium of internship programs in biomedical informatics, so that programs can share lessons they have learned and strategies for development, maintenance, and growth of internship programs. With a rising interest in diversifying pathways into informatics and existing structures such as the Academic Forum, AMIA could take a leadership role in developing and supporting this type of consortium. In addition to sharing best practices across programs, a consortium of internship programs could also aid in disseminating information to individuals interested in starting internship programs, helping to further expand this model of educational outreach.

Future directions for the Vanderbilt DBMI Summer Internship Program

Vanderbilt DBMI was recently awarded a grant from the NSF to establish an REU site. Our REU site, called the Program for Access to Training in Health Informatics (REU-PATHI), will allow us to continue to evolve our model for engaging diverse undergraduate students in biomedical informatics research and mentoring. REU-PATHI groups biomedical informatics research projects under three focus areas: Computing, Precision Health, and Human-Technology Interaction. This new program will expand our outreach efforts in six ways to benefit undergraduate students: increased program visibility on the national scale, greater emphasis on diversity in recruiting, more equitable and inclusive financial benefits for students (including support for housing and meals), connections across research areas, additional mentoring in career pathways and educational planning, and rigorous program evaluation. The REU funding enables us to move towards a more thorough study of the costs and benefits of the program. As our program continues to evolve, we will continue working on improving tracking of student outcomes after program completion and increasing dissemination of student internship project results.

CONCLUSION

Internship programs have strong potential to become sustained components of the biomedical informatics educational ecosystem, providing important new paths to engage talented students early in their educational/career trajectories. Increased levels of funding support along with sharing information about program design across institutions would enable summer internship programs in biomedical informatics to more fully achieve their potential. The lessons presented here based on 10 years of experience can provide a foundation for further growth of internship programs in biomedical informatics.

FUNDING

Preparation of this manuscript was supported through funding from Vanderbilt University Medical Center. The program described in this manuscript was supported by multiple funding sources including: NLM T15LM007450, NLM R25LM011174, Vanderbilt University, and Vanderbilt University Medical Center.

CONTRIBUTORS

K.M.U. drafted the manuscript, incorporated edits, and prepared initial responses to reviewer feedback. B.Y.Y., R.J., and C.M. contributed to data collection and data collation. C.R. reviewed drafts, provided feedback and assisted with identifying references. K.J. and C.G. provided feedback on drafts and suggestions redirection of manuscript and assisted with responses to reviewer feedback.

Conflict of interest statement. None declared.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Click here for additional data file.