Applying Experiential Learning to Career Development Training for Biomedical Graduate Students and Postdocs: Perspectives on Program Development and Design.

Experiential learning is an effective educational tool across many academic disciplines, including career development. Nine different institutions bridged by the National Institutes of Health Broadening Experiences in Scientific Training Consortium compared their experiments in rethinking and expanding training of predoctoral graduate students and postdoctoral scholars in the biomedical sciences to include experiential learning opportunities. In this article, we provide an overview of the four types of experiential learning approaches our institutions offer and compare the learning objectives and evaluation strategies employed for each type. We also discuss key factors for shaping experiential learning activities on an institutional level. The framework we provide can help organizations determine which form of experiential learning for career training might best suit their institutions and goals and aid in the successful design and delivery of such training.

INTRODUCTION

Experiential learning is a hands-on educational process that produces knowledge and skills through a combination of experience, reflection, conceptualization of the experience, and use of learned ideas to make decisions or solve problems (Kolb, 1984). This mode of learning is valuable when applied to graduate career exploration, when doctoral students and postdocs seek to gain an understanding of various professions and job cultures, assess whether their skill sets are a good fit, and further develop their transferable skills in order to make informed career decisions (Schnoes ; Chatterjee ). Just as these PhD students and postdoctoral fellows (collectively referred to as “trainees”) may rely on experiential learning in a laboratory setting to hone their research and technical skills, hands-on learning can be similarly beneficial for career development.

Although there are many valuable approaches and components to the career exploration process that do not include experiential learning, such as traditional lectures, seminars, career panels, advising, and mentoring, those formats are more passive, require less engagement from the trainees, and do not provide the experience of being “in the shoes” of the professional. Experiential learning opportunities, however, allow for the trainee to be immersed in the actual (or simulated) work environment, acquire and apply skills needed in particular careers, and complete hands-on projects directly related to that profession (Binder ).

The motivation for incorporating experiential learning into career development opportunities is grounded in the well-established concept of self-efficacy. Albert Bandura (1994) described how practicing and experiencing a specific task will lead to increased confidence in a person's ability to perform this task, and this positive belief in oneself feeds back into better performance of the task at hand. Self-efficacy is experience-based and serves as a foundation for successful performance (Lent ). Indeed, at the undergraduate level, hands-on research experiences have proven effective in building self-efficacy and research skills, two variables that can predict student aspirations for research careers (Adedokun ). Following this logic, opportunities to experiment with careers with which they were previously unfamiliar, even on a small scale, can lead to trainees' increased self-efficacy and enhanced confidence in career decision making. This concept further maintains that even rejection of a career path, after the small-scale career experience, is a valuable outcome, as it will prevent a trainee from making an uninformed job commitment, which might ultimately lead to disappointment, frustration, disengagement, and underperformance.

As the biomedical research workforce continues to expand and diversify, integrating experiential learning into the graduate curriculum has tremendous value in helping trainees identify career paths that will be right for them, especially considering that currently only a minority of all U.S.-trained PhD graduates will enter the professoriate (National Institutes of Health [NIH], 2012; Schillebeeckx ; Offord, 2016; Langin, 2019). Further, exposure to opportunities for recognizing the transferability of their existing skill sets, and for developing new skills, can prepare trainees for job satisfaction across a wide range of careers in academia and beyond (Sinche ). Until relatively recently, traditional graduate programs have provided students with only limited exposure to careers outside academia. To address this training discrepancy in the biomedical sciences, the NIH established the Broadening Experiences in Scientific Training (BEST) Program in 2013, awarding grants to 17 institutions to pilot career and professional development programs that enhance the preparedness of PhD graduates and postdocs entering the biomedical workforce, both within and outside academia (Meyers ; Lara ; Lenzi ). All of the NIH BEST programs incorporated experiential learning opportunities (NIH, 2017), which have been refined over the 5-year grant period through local assessment and sharing of successes, challenges, and best practices.

In this article, we describe the range of experiential learning approaches our institutions offer and classify them into four types: job simulations, employer site visits, job shadowing, and internships. We compare the structure, duration, resource requirements, learning objectives, and assessment strategies employed for each type with the goal of helping institutions determine which approach will best serve their local environments. In addition, we discuss key strategies for implementation based on our collective experience. We hope the framework we provide here will inspire and encourage readers to create or expand experiential activities that help graduate students and postdocs achieve successful and fulfilling careers in research and beyond.

DEFINING EXPERIENTIAL LEARNING ACTIVITIES: FOUR TYPES

Experiential learning approaches can incorporate a vast spectrum of activities that provide hands-on experience and learning, both inside and outside a classroom setting. For the purpose of this article, we focus on experiential learning activities that are immersive or include an element of practicing or exercising a skill or set of skills. After performing a cross-site comparison of the BEST Program–associated experiential learning activities delivered by our nine institutions during the award period (over 5 years and involving hundreds of trainees), we classified them into four major types: job simulation, site visits, job shadowing, and internships, as described in Table 1 and detailed in the following sections. Beyond the experience itself, most of the described activities also include either concurrent or subsequent reflections, which are intended to help participants articulate the impact that the experience had on their career decision making. As a component of the eight principles of good practice for all experiential learning activities, the National Society for Experiential Learning defines “reflection” as a critical element to elevate simple experience to a learning experience (National Society for Experiential Education, 1998). This is an element we highly recommend incorporating into all activities through an exercise, discussion, postassignment, or survey designed for looking back on and considering the experience.

TABLE 1.

Types of experiential learning opportunities offered by BEST programsa

aThis table describes the four types of experiential learning opportunities offered by the BEST programs. Although there is substantial variation within each type, key differences exist in the amount of time trainees commit to the experience, where the activities take place, whether there is staff coordination. and the extent to which mentorship is provided by external professionals.

Our comparison of experiential learning activities developed across the BEST Consortium revealed that the activities tended to include the simulation of duties involved in a specific job, experience in a physical workplace, or both. For example, job simulations focus on the practice of tasks that are involved in a specific job and need not actually involve immersion in the work environment, whereas site visits and job shadowing at work locations constitute a temporary immersion in the environment. Experiential learning in the form of internships blends immersion and simulation. These experiential learning examples differ along other key dimensions, including the time commitment for the participants, location, institutional and staff involvement, and the role of mentors. Specific examples from individual BEST institutions are provided below (see also Table 1). Our selection is not intended to be comprehensive, but rather to give program directors, administrators, faculty, and trainees ideas for how experiential learning can be incorporated into institutional programming at different doses and with varied budgets.

Job Simulation

Job simulations are organized, hands-on activities that are often case or scenario based and provide trainees with the experience of performing a set of tasks that would be required in a specific professional environment (see Table 1). For example, a job simulation for a patent agent in the arena of intellectual property may involve reading a patent application, interviewing the scientist who proposed the innovation, and using online databases to assess prior art. Simulations often take place in 1 day or less and can be facilitator led (usually in a classroom or learning studio setting, but sometimes offered at the job site) or self-directed via an online platform. This flexibility makes job simulations an attractive choice in circumstances in which travel or in-person opportunities are limited.

Job simulations typically include a period of reflection, which may be achieved through guided exercises, a reflection-discussion, or an informational interview with a professional from that career field. In contrast to an internship, in which the trainee often receives in-depth training and is required to complete a project for the company or organization, a job simulation is not generally expected to yield a usable deliverable or provide a service of value. Rather, a job simulation exercise should inspire next steps for additional training or reveal whether or not the particular career path aligns well with a trainee's skills, interests, or values.

One example of job simulations uses an in-person series of full- or half-day workshops branded as “mini-internships” (Virginia Tech, n.d.). These job simulation workshops are offered primarily at the home institution and are delivered by external PhD-level professionals from a variety of career paths such as science policy, industry, or publishing. Before each mini-internship, a Virginia Tech faculty member host works with the guest professional(s) to set learning objectives and guide the design of the team-based job simulation exercises. For example, one science policy job simulation workshop required trainees to negotiate and present a consensus policy position for their mock scientific society, negotiating between two groups with opposing policy viewpoints. Actual policy experts from nonprofit and government agencies guided the trainees through the exercise. In another example, a PhD-level science reporter from broadcast media led trainees through development of a news story, which they subsequently recorded and evaluated. Following the activity session, participants engage in a reflective discussion with the professional(s), during which they are also directed to additional training to grow related skill sets and networking opportunities.

Job simulations also can be delivered via online platforms, for instance, the “Inter-SECT: Interactive Simulation Exercises for Career Transitions” developed for PhD-level trainees in both the sciences and humanities at Washington University (Nguyen, 2019), which is used by UCSF BEST Motivating Informed Decisions (MIND) trainees (University of California San Francisco (UCSF) BEST MIND trainees participate). The job simulation is a self-guided activity that contains the background and instructions to complete a task related to the trainee's career of interest. Simulations include, but are not limited to, job tasks associated with careers in advocacy, market analysis, clinical trials, journal editing, medical writing, regulatory affairs, education outreach, patent law, and university administration. Participants monitor and track their progress in the program and complete a self-reflection guide to determine whether further investigation into a particular career track is warranted. At the conclusion of the exercise, a framework is provided for setting up an informational interview with a professional in order to learn more about the career path. This program is completely free and available online to any individual so that all universities can promote the use of this resource as an experiential career exploration platform (NIH, 2017; Langin, 2018).

In addition to offering job simulations as stand-alone activities, some institutions have incorporated them as one component of more time-intensive courses or training programs. For example, at one institution, trainees with an interest in a teaching career can take a series of six training sessions on teaching skills and pedagogy that incorporate a 15-minute teaching simulation and preparation of a teaching philosophy statement for a mock job application (CU Denver/Anschutz Medical Campus). The simulated teaching deliverables are evaluated by expert faculty teachers and critiqued by peers. Pre- and postassessments measure the self-efficacy of the trainees regarding their specific skill competence and reveal whether the training had an impact (Hartley ). Similar examples for teaching-focused careers can be found throughout the Center for Integration of Research Teaching and Learning Network (CIRTL, 2019) and the successful Institutional Research and Career Development Awardee (Rybarczyk ; Eisen and Eaton, 2017) programs, in which training opportunities include experiential learning principles to improve mentoring, teaching, and communication as well as diversity and inclusion practices.

Employer Site Visits

The employer site visit format was adopted by a number of BEST institutions as a way for trainees to engage in workplace exploration. As described in Table 1, site visits usually entail a small group of trainees spending anywhere from 2 hours to a full day at a company location where the visit is immersive, observational, and informational. Employer site visits are generally organized by university staff or by trainees in collaboration with professionals and often include coordination with the on-site human resources department of the company. Site visits typically include a tour of the facility, a briefing about the company, and a panel discussion including employees of the company at various stages in their professional careers, ranging from entry-level PhD positions to the most senior management and high-level company officers (e.g., chief operating officer, vice president, chief scientist). Panels may focus on a single type of career at the company (e.g., bench research) or may include a wide variety of roles (e.g., R&D scientists, regulatory affairs professionals, business development managers, and project managers). Representatives from human resources might present on the recruiting, application, and interview processes and address issues such as company benefits and convey the overall company culture.

Because employer site visits can accommodate groups of trainees, the format also works well when organized locally by one or more programs at a single institution (e.g., collaboration between the FUTURE Program and the Internship and Career Center at University of California Davis [UC Davis]) or by a consortium of regional institutions, exemplified by Enhancing Local Industry Transitions Through Exploration, a collaboration between the University of North Carolina’s Training Initiatives in Biomedical & Biological Sciences (TIBBS)/Immersion Program to Advance Career Training (ImPACT), National Institute of Environmental and Health Sciences Office of Fellows' Career Development, and Duke University's Office of Postdoctoral Services. These collaborative models leverage the employers' time as well as that of the institutional staff organizing the visits and have the added benefit of trainees from different programs or institutions getting to interact around career exploration.

Some institutions (University of Chicago, Cornell University, Virginia Tech, and Vanderbilt University) have also employed the “trek” format, which consists of 2–4 days of employer site visits in a geographic location chosen for its proximity to certain industries or career types that are well represented in that area. A primary goal of the trek format is to highlight the wide variety of company sizes, structures, and cultures that exist within a high-density region of the country. Past treks include career pathways in biopharma (Boston), entrepreneurship (Silicon Valley), data science/computational biology (San Francisco Bay Area), and science policy (Washington, DC). Typically, between four and 10 senior graduate students or postdoctoral fellows are selected through a competitive application process to participate in a trek, and each day consists of 1–4 site visits, usually organized and coordinated by university staff.

Whether as a stand-alone activity or a more intensive engagement such as a trek, employer site visits allow trainees to get a sense of what it would be like to work with a particular employer. They experience firsthand the physical work spaces and also get a glimpse into the professional environment and workplace culture. As a result of participating, trainees have reported feeling better informed and able to evaluate whether they can see themselves working for the company they have just visited (Supplemental Figure S1).

Job Shadowing

Several institutions have chosen to use job shadowing to enable their trainees to gain practical career experience. As described in Table 1, job shadowing consists of accompanying professionals in their workplaces in a one-on-one relationship. The goal of the shadowing experience is to understand the professional environment and skills required for that career and create contacts on-site (Herr and Watts, 1988). These engagements are often initiated by university staff and last anywhere from 1 to 10 days, which may be spread out over as much as a semester. Shadowing is arranged on an individual basis, offers a flexible schedule, and is tailored exactly to the trainee's career interests. Because the trainees are simply observing a professional, there is no specific work product completed at the end of this experience. Job shadowing builds on the immersive benefits that employer site visits offer by giving trainees the opportunity to observe the workplace in action. Through the experience, they come to understand the work culture, including how employees interact with one another, how work products are developed and presented, and how work is evaluated.

At one institution (Rutgers University) trainees are eligible to apply for a shadowing experience once they have demonstrated prior engagement with career development activities on campus. Program staff match about 20 trainees per year with a professional in a local company or organization whose job aligns with a trainee's interest. These professionals have included bench scientists, regulatory affairs experts, public policy specialists, consultants, medical communications professionals, patent lawyers, and a range of other experts from the nonprofit and industry sectors. The trainees spend about 72 hours shadowing one or more individuals at the host site and experiencing the workplace over the course of a semester. Although hands-on training is not required, sometimes trainees are given small tasks and generate deliverables. Trainees complete self-reflection assignments and hosts complete evaluations of the trainees.

Another less resource-intensive approach is to implement mechanisms that support and encourage trainees to identify their own job shadowing experiences. For example, one university (Virginia Tech) implemented a competitive travel award program to help offset expenses for traveling to various job settings for the purpose of job shadowing. In the application, trainees must provide details on the opportunity, including their motivation for pursuing the experience, how their time would be spent, expected gains, and how they will share what they learned with other trainees. The trainees are responsible for initiating and planning the shadowing experience through use of their own personal or professional networks, reaching out to alumni, or through contacts identified through other Virginia Tech–BEST activities. However, guidelines (National Society for Experiential Education, 1998) for trainees and the job shadow host, as well as for writing letters of inquiry, are available through the university career office. This model also affords trainees the opportunity to experience job settings that may not be available near the institution and promotes the dissemination of these experiences through trainee blogs, social media, and in-person formats.

Internships

Internships are a broadly recognized form of experiential learning used in many academic domains, such as undergraduate career development and master's student professional training (Professional Science Master's, n.d.), that provide the academic trainee firsthand experience in a formal work environment. Indeed, a systematic review of 57 peer-reviewed articles found that experiential learning in the form of business internships improved chances of employment, enhanced job and social skills, and assisted the trainees in determining a career path (Velez and Giner, 2015). Business internships have also been associated with higher academic performance and greater employment outcomes (Knouse ) and have been suggested to create more realistic expectations that positively influence later job satisfaction.

Internships developed for biomedical trainees by our BEST institutions have focused on a variety of career areas, including science/medical communication, nonprofit management, data science, regulatory affairs, advocacy/policy, and business development and venture capital, among others. Most internships take place off campus in a company, nonprofit organization, or government office and provide the invaluable experience of receiving hands-on training in a professional setting. Trainees are typically assigned a project and are taught the skills necessary to complete the project, which often results in a deliverable such as a report, study, presentation, or publication.

The landscape for internships for biomedical trainees has changed significantly over the past decade. Although discipline-specific opportunities existed as early as 1988, such as the NIH Biotechnology Predoctoral Training Program (National Institute of General Medical Sciences [NIGMS], 1988), which requires an industrial internship, and career-specific programs followed, such as the Christine Mirzayan Science and Technology Policy Fellowship (National Academies of Sciences, Engineering, and Medicine, 2009), it was not until much later that more formalized, institutionally driven internship programs for biomedical graduate students began to emerge (e.g., Graduate Student Internships for Career Exploration Program at UCSF; Schnoes ; Internship and Career Center at UC Davis). The BEST Program catalyzed the development of additional programs, starting in 2013. A year later, guidance from the Office of Management and Budget clarified for faculty and administrators that trainees appointed to research grants have a dual role as trainees and employees (NIH, 2014), thus removing one potential barrier for institutions to allow participation in internships. Indeed, recognizing the important role that internships play in enhancing trainees' professional and career development, the National Science Foundation now provides additional incentive for NSF trainees to engage in mentored collaborative research internships (NSF, 2015).

The role of the academic institution in facilitating internships can vary, ranging from taking full responsibility for identifying and developing the internship to empowering trainees to find and pursue their own internships. In all cases, program staff provide support through facilitation of paperwork, such as processing leave of absence requests or assisting with continuing benefits/payroll enrollment if trainees remain on institutional funds, referrals to resources, or having NSF-funded faculty support intern funding applications. Several of our institutions have created a credit-bearing internship course that all students can register for, including international students on F1 visas, in order to be granted Curricular Practical Training and maintain their immigration status (Homeland Security, n.d.; International Student, n.d.). The NSF also offers supplemental funding for up to 6 months for graduate students seeking nonacademic research internships (NSF, 2018).

While the traditional “full-time for 3–6 months” internship format was employed successfully by several BEST institutions (e.g., UCSF; Schnoes ), other programs experimented with shorter or part-time internship models. For example, the UNC Chapel Hill ImPACT Program empowers trainees to identify the internship host, and then the trainee is supported by program staff to formally apply for and develop a 160-hour internship format that best suits both parties, either across multiple months (part-time) or for a single month (full-time). Intern, internship host, and faculty advisors are all asked to complete postinternship evaluations, which are collected by the ImPACT team.

Internships at both Vanderbilt Augmenting Scholar Preparation and Integration with Research-Related Endeavors (ASPIRE) and U Chicago myCHOICE (Chicago Options in Career Empowerment) are strictly limited to a part-time format (8-10 hours/week maximum for 10–12 weeks) and in both cases, it is the program staff who recruit potential hosts (both internal and external to the university) and formalize internships. ASPIRE and myCHOICE both select trainees through a formal application process, and at the culmination of the internship, both the intern and host complete evaluation surveys that offer an opportunity for self-reflection. An advantage of these part-time internships is that they can be pursued contemporaneously with graduate and postdoctoral research.

Some BEST programs (e.g., UC Davis FUTURE Program) focus on empowering trainees to find and pursue their own internships and provide support by linking trainees to opportunities for internships. The process of identifying and applying for an on-campus or off-campus internship is thus driven by the trainee, which is a viable model when staff resources are limited.

While off-campus internships are well recognized for their exposure to professional nonacademic settings, the on-campus internship model is particularly flexible and works well in research-adjacent fields, including technology transfer, corporate relations, biosafety, or internal review board administration. For example, at Cornell's Technology Licensing Office, interns learn how to perform prior art searches or relevant literature evaluations for a technology, work with intellectual property lawyers, help create one-page documents to market a new disclosure or patent application, and pitch ideas to potential clients. These on-campus internships are especially beneficial for schools that are located in a region with less biomedical industry. Similar to job shadowing, these experiences require minimal paperwork to establish and are well suited for graduate students and postdocs who need flexible internship schedules or international scholars who cannot perform work at off-campus sites.

LEARNING OBJECTIVES FOR BEST EXPERIENTIAL LEARNING ACTIVITIES

To tackle the broad career and professional development goals outlined earlier, each institution designed its experiential learning activities to meet particular learning objectives, which are an important component of any educational program. Learning objectives describe the intended outcomes that learners should achieve as a result of participating in an educational activity and outline the demonstrable skills or knowledge that should be acquired by the participants. Developing training activities with a clear and thoughtful set of learning objectives in mind is extremely helpful in effectively designing, delivering, and evaluating the success of these educational activities. In some cases, programming may require formal approval by the various governing bodies at our institutions, in which case learning objectives and assessment implementations may be a necessary component of program design. While not all BEST awardee institutions considered formal learning objectives for each experiential activity, doing so has certainly aided in program design and the creation of assessment tools to ensure we are developing evidence-based approaches toward career and professional development training (Millet ).

Interestingly, a review of our collective BEST experiential learning activities identified a set of common and highly prioritized learning objectives that we aimed for our trainees to meet, listed in Table 2. Although a quantitative comparison of assessment results among all activity types was not part of the BEST experimental design, the table provides an empirical assessment of the extent to which the common learning objectives were typically met through each experiential learning approach. We further classified these learning objectives into three important categories based on how they would be met by the trainee: 1) directly through the completion of a task or process (experiential), 2) requiring self-reflection and higher-level thinking (reflective), or 3) involving planning and/or application of new knowledge toward future actions (prospective).

TABLE 2.

Common learning objectives for experiential learning activities across institutions a

aThis table contains examples of key learning objectives that were shared across institutions for their experiential learning activities in career and professional development. Learning objectives are categorized as experiential (met directly through the hands-on experience of the activity), reflective (requiring self-reflection and higher-level thinking), and prospective (relating to decisions on future plans and application of new knowledge). The capacity of each experiential learning activity for meeting these learning objectives will vary and has been scored qualitatively as low (•), medium (••), or high (•••) for each objective based on the authors' experiences.

Notably, the majority of our activities aimed to meet objectives within each of these categories that spanned the many levels of learning from simple to complex, as defined by Bloom's taxonomy of educational objectives (Bloom, 1956). For example, some objectives focused on the more concrete, knowledge-based goals of understanding and remembering (e.g.“Explain job expectations”), while others focused on applying and analyzing (e.g., “Compare pathways for pursuing careers in a given area”). Other objectives were indicators of more complex and abstract levels of learning such as the analyze, evaluate, and create domains (e.g., “Identify one's skills gap for achieving success in a particular work sector” or “Expand network with individuals in an industry sector of interest”).

Three categories of learning objectives are reviewed in the following sections, reflecting the four types of experiential learning activities we have delivered.

Experiential

These learning objectives assess the ability to acquire and use knowledge obtained through direct experience, such as observation of activities within one or more work environments, engaging intellectually with professionals in that environment, or completing tasks to learn or practice new skills. Each of the four activity types were viewed to be successful in meeting experiential learning objectives to some extent, although the participating institutions observed that activities allowing for more extensive observational or hands-on opportunities, such as job shadowing and internships, were best positioned for addressing these types of objectives. By contrast, less-intensive activities like employer site visits, which typically do not involve the practice of skills or other experiences to provide firsthand visibility into challenges or decision making, often did less to promote new knowledge or skills for meeting higher-level learning objectives in this category.

Reflective

These objectives are often met through a period of reflection directly associated with the delivered activities (either throughout the experience as it unfolds, or at a postactivity discussion session) or through subsequent reflection on an individual basis (such as through revising one's individual developmental plan [IDP] postactivity or written assignments; Vincent ). Reflective objectives often require having done some level of self-assessment and subsequent evaluation of one's skills, values, and interests as they relate to the experience of a particular career path. They may require weighing prior experiences and assumptions to form new conclusions that influence future actions. Cultivating reflective practices encourages participants to internalize what they have learned such that they may not only recall learned information, but also construct meaning, understand how learned elements relate to one another, and develop actionable plans based on learned concepts.

Just as with experiential objectives, reflective objectives were included for all four types of educational activities delivered at our institutions, with the expectation that longer, more in-depth career exposure activities would go further in meeting these objectives for the learners. For example, institutions did not expect learners to pursue employer site visits if their primary learning objective was to determine the alignment of the career with their skills, interests, and values. Instead, in that case, the trainee would be encouraged to pursue an internship or job shadowing experience, where this objective could be achieved more readily.

Prospective

Learning objectives in this category reflect the value of the activity for producing future tangible items or next steps. For example, an activity may provide the knowledge, contacts, or tools for trainees to expand their professional networks, identify future activities for expanding relevant skill sets, determine appropriate professional organizations for future participation, or even produce a deliverable for inclusion in their portfolios and or to enhance their curricula vitae. For trainees interested in producing a tangible deliverable as evidence of their ability to apply the learned skill set (such as a publication, patent application, or even a start-up company), internships are an ideal format, and a deliverable of some sort (i.e. report, presentation, poster) is typically a required component of the experience. However, internships are often time and resource intensive, and participation is limited. Thus, job simulations may also be a good format for producing a small-scale deliverable or at least conveying knowledge required for learners to produce future deliverables.

Thus, a variety of common learning objectives can be met to various extents through all four experiential learning approaches that are important for career exploration and professional skill building. While there are, of course, many other learning outcomes institutions may consider in their program design, those outlined here can serve as a good starting point for individuals looking to design, structure, and evaluate new experiential learning activities. As indicated in Table 2, each approach has a different capacity for meeting particular objectives, which needs to be balanced against other factors, such as time commitment, resources required to deliver the activity, trainee goals, and institutional culture. For example, despite the clear advantages of internships in experiencing a profession, institutions noted that job simulations required less commitment on the part of trainees and fewer institutional, staff, and mentorship resources, making them an attractive option for exploring a wide spectrum of job-related skills. Shorter experiential learning formats are also ideal for synergizing with other activities, such as informational interviews; when combined, these activities may tackle a broader range of learning objectives. Ultimately, experiential learning activities should be considered valuable components of a well-rounded professional development program that collectively targets and measures a range of learning outcomes.

EVALUATING THE EFFECTIVENESS OF EXPERIENTIAL LEARNING

A variety of evaluation methods and tools can be employed as part of an institution's assessment plan to ensure that both individual learning objectives and broader programmatic goals are met. In Table 3, we provide an overview (including definitions and examples) of some key evaluation methods that have been used at our institutions; these can vary with respect to the ease of administration, time commitment, and level of expertise required.

TABLE 3.

Evaluation methods used for assessment of experiential learning activities and programsa

Method	Description	Examples	Common uses and measures
Survey	A series of questions designed to gather opinions and insight about program activities and experiences (usually using an online tool)	Skill survey pre- and postexperiential learning event Retrospective survey on perceived value of specific activity (workshop, career trek, etc.) or entire program	Measure self-reports on select learning objectives and satisfaction with activity Indirectly measure longitudinal impact of an activity (pre/post)
Assignment	An assigned task designed to assess (and potentially develop) skills, such as written products, visual demonstrations, projects, or presentations	Science policy brief (or science communication blog) Completion of a consulting project Construction of a museum display Reflection essay describing an informational interview	Directly measure whether a skill has been attained Ensure an action item has been performed or a deliverable produced
Performance assessment	Review submitted by an expert in the field who has thorough knowledge of trainee's performance in an experiential learning activity (e.g., internship supervisor)	Letter of reference from supervisor of internship or other experiential learning activity Completion of an evaluation rubric	Directly assess the extent to which an objective has been met Define areas for individual improvement Assess efficacy of activity in meeting standards of an expert
Interview	Structured discussion or questioning of individual trainees or a focus group to gain qualitative input at the program level	Focus group with standardized questions about the experiential learning activities facilitated by the program Exit interviews	Identify unexpected benefits of activity and areas for improvement Identify new learning objectives
Outcome tracking	First job placement in career- field of choice; subsequent career outcomes	Matching of LinkedIn job titles of former trainees with their career interests (defined by a standard career taxonomy)	Measure long-term impact of activity on career decisions Relate results of other evaluation methods (e.g., surveys) to career outcomes

aThis table contains examples of the methods used to evaluate experiential learning activities and programs at the authors’ institutions. The methods vary in terms of their applicability to specific assessment goals (e.g., presenting to stakeholders, scholarly publication, internal program adjustments, individual feedback) and the tailored resources required to deploy them for a given activity or program.

Surveys are the most commonly used evaluation method across our programs, given their ease of administration and their efficiency for gathering input on a variety of program aspects for formative and summative assessment as well as the ability to collect anonymous data. However, surveys are indirect measures and thus are not the tools best suited for evaluating skill attainment. Furthermore, survey language can bias the results.

For more direct measures, some institutions employed assignments or performance assessments conducted by experts in the field (Hartley ; Wortman-Wunder and Wefes, 2020). While more time-consuming to implement and review, these mechanisms are effective ways to assess individual trainee competence regarding specific skills and may encourage stronger performance if embedded within a course.

Interviews or focus groups led by experienced evaluators can also require substantial time and resources (question preparation, interviews, recording/transcription services, and data analysis), but are effective at gauging the perceived value of experiential learning opportunities for career decision making and for the overall training experience.

Finally, outcome tracking can provide useful information on program effectiveness toward career preparation (e.g., whether trainees identified and achieved their career goals) and highlight trends that suggest how programs should evolve over time (e.g., include emerging career fields). However, effectively tracking trainee career trajectories after exiting the institution can be challenging and time-consuming and relies on using a clear and consistent career taxonomy (Mathur ).

Once collected, the results of these program evaluations can be valuable when shared with different audiences at the institution. For trainees, self-reports and assignments (such as reflections, IDPs, and projects) can be included in e-portfolios and incorporated into their annual performance discussions with their advisors to inform new career and professional development goals (Vincent ). For program managers, the results can be used to create formative assessments, guide pruning of less-effective experiences, improve existing programming, and identify new opportunities to address gaps. For graduate programs, publishing results of evaluations and career outcomes can augment recruitment and retention success and potentially inform curricular gaps. When shared with faculty, evaluation results may increase buy-in/support for trainee participation in future experiential learning activities. Applications for training grants and other funding mechanisms can also be enhanced by the addition of professional development activities for which efficacy is supported by assessment and outcomes data (e.g., see the NIGMS expectations for including experiential learning, career exploration, and outcomes transparency in T32 training grant applications; NIH, 2020). Finally, positive results may increase the willingness of employers to support experiential learning activities and may also be crucial in seeking funding from institutional leaders, alumni, and other sources.

Ultimately, all institutions must identify metrics that align with their culture, priorities, and strategic goals and create an evaluation plan accordingly. The decision of timing, frequency, and method of assessment will be defined by the nature of the activities planned, the practicability of conducting the assessment, possible survey fatigue, and pressing deadlines. A pre/post design may be optimal in some cases, and a retrospective onetime survey may be more efficient in other cases. The most important consideration should be how the data can be analyzed and used to inform and guide the improvement of future program offerings and their evaluations.

KEY FACTORS FOR SHAPING INSTITUTION-SPECIFIC APPROACHES

All academic institutions must consider a number of unique factors when determining the type, breadth, and quantity of experiential learning opportunities they can offer to graduate students and postdocs. Our BEST institutions have shared best practices over the course of the past 5 years and have identified several key considerations that can support successful and sustainable program development (e.g., NIH, 2017). Several institutions have adopted a broader approach to experiential learning to encompass all doctoral science, technology, engineering, and mathematics disciplines and even social sciences, humanities, and arts (e.g., Cornell University; Wayne State University, n/d/). They have found that the interactions across the interdisciplinary foci of the participants enriches the experiences and might even enable broader institutional support. For example, when asked what aspect of the program was most valuable, a trainee at UC Davis reported, “One of the best parts of the FUTURE Program was the cohort itself. I loved having life and career discussions with other grad students and post-docs. I loved seeing my experiences mirrored in others as well as hearing new perspectives.” Although the experiences described in this paper are focused on biomedical trainees, it is clear that the same principles apply across disciplines.

Local Expertise.

For universities just getting started with graduate- and postdoc-level experiential learning, we recommend starting small and trying one or two activities that seem manageable within the institution's capabilities. Matching programming to the available campus expertise and leveraging the institution's strengths are well advised. For example, if a university is strong in specific areas of expertise (e.g., policy, law, entrepreneurship, tech transfer), campus partners in these fields might be eager to help plan or facilitate experiential learning opportunities. Such a partnership could be designed to be mutually beneficial, especially when it can bring together multiple units across the institution that share similar professional development goals.

Geographic Location.

While it is desirable to expose trainees to the full breadth of careers represented within the biomedical workforce, this breadth of expertise is not equally represented across all geographic locations. For example, some universities are not located near a hub of biopharma companies, which can be a disadvantage for PhD scientists potentially interested in industry careers. These geographic limitations may guide implementation of specific types of experiential learning, such as job simulation or employer site visit treks, and make others, such as industry internships, less feasible.

Staff Allocation.

Most institutions do not have sufficient staff to implement all of their ideas for career and professional development, including experiential learning programming. Finding ways to leverage existing staff and offer less staff-intensive experiential learning opportunities are effective strategies when building or expanding a program. Some universities take a less staff-intensive approach to facilitating (vs. developing) internships by advertising existing opportunities to participants and equipping them with the knowledge and tools to organize and pursue internship opportunities on their own. Alternatively, facilitating on-campus job shadowing opportunities with PhDs employed in university staff roles may also be more manageable.

Support of trainee-led career-affiliation groups who are interested in planning and implementing their own programming is a productive way to multiply the effectiveness of limited staffing resources while increasing project and program management skills of the trainees involved. This successful model has been embraced by many student organizations (Advancing Science and Policy and Cornell Graduate Consulting Club at Cornell, ComSciCon franchises across North America). A similar outcome has been observed when a single workshop has led to the development of exclusively student- and postdoc-driven interest groups, for example, the Academia-Industry-Alliance, at the University of Colorado Denver/Anschutz Medical Campus; this BEST Program spin-off organizes many career development opportunities, including an annual Biotechnology Mini-Symposium to which all local biotech companies are invited.

Alumni Participation.

We have found that alumni networks can provide a rich source of qualified individuals who are eager to give back to the university and happy to participate in experiential learning activities (on-site or remotely), often without any expectation of compensation. For example, alumni may conduct individualized, Web-based tours or in-depth informational interviews or may travel to participate in-person in experiential learning activities, such as case-based projects or hands-on workshops with a job simulation component. Indeed, combining remote activities such as online workshops with in-person career consultations is an especially useful strategy when geographic or financial constraints exist. For institutions that are fortunate enough to be located in regions with an extensive local alumni network, alumni can be a valuable resource for hosting shadowing or local internship opportunities at their employment sites. Alumni participation may be solicited in a number of ways, such as through the institution's Office of Alumni Relations (or equivalent), directories, data collection or social media sites managed by graduate and postdoctoral training programs, or personal connections of the staff or faculty (such as former trainees or colleagues from the institution). Ultimately, we have received overwhelmingly positive feedback from our participating alumni (see Supplemental Table S1 for representative feedback) and conclude that efforts to meaningfully engage alumni and create a culture of giving back benefit everyone involved (Lara ).

Faculty Advocacy.

One recurrent concern regarding career program development relates to faculty perception of time that trainees spend on professional development activities, including experiential learning. Increasingly transparent data show that the majority of trainees are entering careers outside academic tenure-track positions, for which additional training is highly beneficial (NIH, 2012; Coalition for Next Generation Life Science, 2019). Concern over extended time to degree for those who participate in professional development events, including internships, seems to be unfounded (Schnoes ; Cornell University, Rutgers University, University of Chicago, University of North Carolina, Vanderbilt University, Virginia Tech, unpublished results). In fact, spending time exploring and preparing for careers can increase the motivation to complete graduate or postdoctoral training, resulting in a greater focus on research progress (Mathur ; Watts ; UNC, unpublished data). Efforts to build faculty support can be achieved through broadly sharing doctoral career outcomes with faculty to enhance their awareness of careers that PhD graduates are actually pursuing. These data can be accompanied by information about experiential learning opportunities for trainees at their universities, which can help augment their own mentorship. Identifying prominent faculty champions to help message the potential benefits of experiential learning activities within their departments is also a valuable strategy to build support. Success of experiential learning programs is contingent on open discussion about career goals between trainee and mentor, which can be facilitated through the use of tools such as the IDP (Vincent ). Additional incentives to embrace professional development activities are increasingly provided by national funding sources that predicate funding on the existence of career development support structures at the institution (NIGMS, n.d.). This is an excellent example of how national entities are leveraging their institutional capital to bring about culture change.

Ineligible Populations.

Not all trainees are able to participate in specific types of training activities due to visa restrictions. Time constraints and the conditions set by particular funding mechanisms may also prevent postdoctoral scholars from participating in certain activities such as internships. Being aware of these limitations and providing a variety of experiential learning activities that are compatible with such limitations will ensure that the needs of international and other scholars are being met.

CONCLUSIONS

The collective experiences of our nine institutions demonstrate that career-focused experiential learning for PhD-level trainees can occur in many forms. Lessons learned include: 1) experiential learning activities vary broadly and can be effective even on a small scale, which may be especially important for institutions new to experiential learning programming or with resource constraints; 2) there should be components of both practical engagement and reflection to maximize the benefits of experiential learning; 3) regardless of the form of experiential learning, evaluations of the activity should be intentional, planned in advance, and aligned with predetermined learning objectives; and finally, 4) sharing of information and programming experiences across the BEST Consortium has made the insights described here possible. Thus, we strongly encourage others to continue sharing across the broader training community. In sum, introducing experiential learning in a variety of doses and formats can be extremely beneficial to the professional development of trainees and does not have to be expensive, unilateral, or time/resource-intensive. Our experiences described here provide a framework that can aid in the design and development of experiential learning programs that meet the needs of biomedical graduate and postdoctoral trainees across a range of institutions.

Note. During revision of this article, the landscape for biomedical training has shifted due to the COVID-19 pandemic. As faculty, trainees, and employers now lean more heavily on virtual and online platforms, while limiting in-person interactions, this presents both challenges and opportunities for incorporating experiential learning into biomedical training. While the job simulation format described here provides an avenue for gaining career insight and experience without being physically present at an employer's site, there is also new opportunity and need to create innovative solutions for delivering internships remotely, and for bringing trainees “on-site” for virtual visits and shadowing.

Click here for additional data file.