Building Diverse Careers in Clinical and Translational Research: Evaluation of a Certificate Program in Translational Research.

Introduction: The Certificate Program in Translational Research (CPTR) at the Georgia Clinical and Translational Science Alliance provides PhD students, post-doctoral fellows and faculty with didactic, mentored, and experiential training in clinical and translational research.
Methods: Quantitative evaluation includes tracking trainee competency, publications, grants and careers in clinical and translational research. Qualitative evaluation includes interviews with trainees about program experiences.
Results: The CPTR provided knowledge and skills in clinical and translational research through coursework, clinical rotation, and collaboration with interdisciplinary scientists. Trainees reported increased confidence in 22 program competencies. Trainees have published more than 290 peer-reviewed articles and received over four million dollars in grants from the NIH, over $15 from the U.S. Department of Defense, and more than $300,000 from foundations. Trainees who completed the program remained in clinical and translational research. Conclusions: Programs like the CPTR are needed to train investigators to advance biomedical discoveries into population health.

Introduction

Progress in biomedical research is increasing at an exponential rate [1]. Unfortunately, there is a well-documented disconnect between advances in basic science research and translation of clinical and translational research data to real-world settings [2, 3]. Major obstacles have been identified in the performance of “bench-to-bedside” and “bedside-to-community” research [4-8]. Scientific research silos tend to occur during two major stages of translational research; namely in the transition to the T1 phase (the transfer of new knowledge from the laboratory to testing in humans) and in the transition to T2 phase (the transfer of knowledge from clinical studies into clinical practice, the community, and health decision making), as defined by the conceptual model of the varying stages of translational research developed by NIH [1, 5]. A potential solution to begin to address these scientific roadblocks is to equip biomedical research scientists with the knowledge and skills to facilitate the translation of biological knowledge into tools to improve human health [1]. Ph.D.-degree granting programs in the biomedical sciences have traditionally directed scientists towards narrow and specialized career paths without providing exposure and training on how to translate their research towards meaningful, clinical findings [4, 8]. This has resulted in many Ph.D. scientists not being prepared to appreciate medically relevant principles that are critical for carrying out T1 research and translating laboratory findings and discoveries to humans to benefit health. In response to such a need, several successful Ph.D. degree programs specifically aimed at promoting translational research have been developed [8, 9]. These full-degree programs, however, are intensive and may not be feasible for trainees who choose to remain in basic science laboratory settings, and would like to gain some exposure to clinically relevant research and how best to work with others in translating their findings to improve health. Furthermore, postdoctoral fellows and junior faculty, with or without clinical backgrounds, who have already completed their doctoral degrees may also benefit greatly from additional training in clinical and translational research [1, 10].

Here, we describe the evaluation of the Certificate Program in Translational Research (CPTR) developed at Emory University in collaboration with our Clinical and Translational Science Award (CTSA) partners at the Morehouse School of Medicine (MSM) and the Georgia Institute of Technology (Georgia Tech) in Atlanta. The aim of the CPTR is to provide a didactic and experiential training program at the interface of medicine and science to facilitate a trajectory for predoctoral and postdoctoral trainees to pursue diverse careers in translational research. The CPTR was initially funded by the Howard Hughes Medical Institute “Med into Grad Initiative” which had goals of integrating medical knowledge into Ph.D.-type graduate education and to bring trainees to the interface of science and medicine and facilitate translation from basic science discoveries to the clinical environment [11]. Our CPTR was subsequently institutionalized by incorporating the program into our Clinical and Translational Science Award research education program [12, 13] which allowed us to also broaden the trainee pool beyond the initial target of Ph.D. graduate students to include postdoctoral fellows and faculty members from all disciplines.

Setting, Participants, and Enrollment

The CPTR program has supported training for Ph.D. students, postdoctoral fellows, and junior faculty members (both those with M.D. and Ph.D. degrees) since 2011. Individuals from the Emory University, Morehouse School of Medicine (MSM), and the Georgia Institute of Technology (GA Tech) which were the partner institutions in our NIH-funded Clinical and Translational Science Award (the Atlanta Clinical and Translational Science Institute [ACTSI]) between 2007 and 2017 were eligible to apply; with refunding, the name of our NIH-funded Clinical and Translational Science Award was changed to the Georgia CTSA with the addition of the University of Georgia as the fourth partner. Ph.D. graduate students begin the CPTR at the onset of their second or third year of graduate school after completing comprehensive examinations. Exposure early in their training ensures an impact of translational research in their dissertation and career direction. The application to the CPTR includes: an NIH-style biosketch from the applicant and lead mentor/advisor, a personal statement, a transcript from the current graduate degree program, a letter of recommendation from the Ph.D. Program Director (graduate student), and a letter of support from the Lead Mentor. The letters from the mentors and program directors ensure full support and acknowledgment of the protected time required away from the mentors’ laboratory to complete the CPTR program. Our CTSA Research Education Executive Committee reviews applications. Applicants are accepted based on their interest, quality of mentor(s) or mentoring team, and potential for a career in translational research.

Program Description

The CPTR involves a foundation of didactic classes complemented by clinical-related rotations (see Fig. 1 for full descriptions). The program is designed to provide a better understanding of the clinical research infrastructure and enterprise and how to translate findings from the bench to the beside and from the bedside to the community as well as how to access resources that will support translation. The CPTR is also designed to provide exposure and interaction with a diverse group of investigators outside of the trainee’s scientific milieu (including KL2 scholars and other junior and senior clinical and translational research investigators and clinicians), study participants, and patients who may be future study participants [10]. The didactic program requires 14-credit hours of core clinical and translational research classes provided through the Laney Graduate School of Emory University and taught by faculty across the three CTSA partner institutions plus an additional elective course (two or more credits). CPTR trainees matriculate in 7 courses developed for the Emory Master of Science in Clinical Research (MSCR) program (e.g., Introduction to Clinical and Translational Research, Research Colloquium, Ethical, Legal, and Social Issues of Responsible Clinical and Translational Research, and Scientific and Grant Writing), also supported by our CTSA (Fig. 1). Specific courses were developed to focus on the particular needs of CPTR trainees (e.g., Fundamentals of Epidemiology, Biostatistics for Translational Research, and Translation to Clinical Medicine).

Fig. 1

Curriculum for the Certificate Program in Translational Research at the Georgia Clinical and Translational Science Alliance.

Among the experiential requirements is a rotation in the Georgia CTSA Clinical Research Network (CRN). The CRN provides physical and intellectual resources to facilitate the conduct of clinical research by investigators (e.g., research nursing, laboratory and bionutrition support, equipment and space available), and incorporates several outpatient and inpatient clinical research units at our Georgia CTSA institutions. During the CRN rotation, trainees learn about the resources available through our CTSA for clinical and translational research, have the opportunity to meet with the principal investigators (PIs) of NIH-funded and Institutional Review Board (IRB)-approved CRN projects, and attend Scientific Advisory Committee meetings for CRN protocol review. Additional CPTR requirements include attendance at an IRB meeting and a monthly journal club in clinical and translational research with MSCR trainees.

The Translation to Clinical Medicine course is a unique and key component of the CPTR, designed specifically for trainees who have had little or no practical experience interacting with patients or study subjects. This 2-credit hour course, scheduled on an individual basis during CPTR course of studies, provides initial training and practical experience in common pathophysiologic diseases and in working with patients and study subjects (e.g., the informed consent process). In addition, as part of the course, CPTR trainees participate in an individualized clinical medicine rotation in which the Course Director links each trainee with a clinical investigator working in their area of research interest. The clinical investigator becomes a navigator for the trainees in the clinical research and clinical medicine world. The clinical rotation component of the course occurs over at least 20 contact hours in an inpatient and/or an outpatient setting. Opportunities for trainees may include rounding with a clinical service at Emory- or MSM-affiliated teaching facilities, observing diagnostic or therapeutic procedures (e.g., imaging, surgery, physical examinations) in tertiary and community-based research sites; understanding and observing state-of-the art hospital-based analytical technologies; and/or shadowing additional multidisciplinary inpatient and/or outpatient teams caring for patients with disorders or diseases relevant to the CPTR trainee’s research.

CPTR trainees may complete the program in 1 or 2 years; however, the 2-year option is ideal for trainees who wish to minimize time away from the laboratory or other research setting. For Ph.D. graduate trainees, the CPTR was constructed in such a way that it does not increase time to degree. Students are evaluated based on the successful completion of their didactic coursework, and additional requirements such as participation in the monthly journal club. Students receive letter grades for the didactic classes dependent on results of assignments and exams. A major project for all students is the completion of a full grant written in the style of an NIH F, K, or R-grant as a requirement for the Scientific and Grant Writing course. Following completion of all requirements, the Laney Graduate School of Emory University confers a formal Certificate in Translational Research to be applied to the trainee’s official transcript.

Materials and Methods

The CPTR evaluation measured program goals and outcomes using a mixed methods approach that includes quantitative measures (CPTR trainee self-assessments of skills and competencies, demographics, publications, grants received, careers in clinical and translational research) and qualitative interviews.

Quantitative Data Collection

The trainees complete a self-assessment of competencies at the beginning and end of the program. The competencies were developed from the national CTSA competencies for master degree programs in clinical and translational research and were revised to reflect the content and duration of the CPTR. The evaluator tracks grants and publications from various sources (see Data sources and instruments section) into a program database of all former and current CPTR trainees.

Qualitative Data Collection

The qualitative evaluation includes interviews with program participants to assess the impact of the CPTR on their career path. Qualitative research captures rich description and context about participant experiences and allows for more in-depth exploration of program impact [14, 15]. For this study, the CPTR program evaluator (D.L.C.) conducted semistructured, one-on-one interviews with trainees to collect data about the program over time (n=9). Qualitative interviews were conducted either in person or by phone using a standardized instrument. Interviews lasted between 20 and 50 minutes, were audio-recorded, and transcribed verbatim by a professional transcriptionist. The Emory IRB determined that the study did not require formal approval because it was conducted as part of a standard educational program evaluation.

Data Sources and Instruments

Quantitative Data Sources

Publications by former and current CPTR trainees are tracked by semiannual searches in PubMed (http://www.ncbi.nlm.nih.gov/pubmed). NIH funding as a principal investigator and/or program director is tracked through the Research Portfolio Online Reporting Tools (RePORT) Expenditures and Results Tool (RePORTER; http://projectreporter.nih.gov) on an annual basis. Non-NIH federal funding and nonfederal funding is tracked by obtaining updated curriculum vitae and NIH biosketches from former trainees on an annual basis. In addition, a quantitative self-assessment survey is given to all trainees at their entrance and upon completion of the CPTR. This instrument was developed to collect the trainee self-reported confidence, on a scale of 1 (low confidence) to 3 (high confidence), in 22 competency areas of clinical and translational research that cover 5 core thematic areas: (1) spectrum of translational research; (2) designing clinical and translational research studies; (3) research ethics and the responsible conduct of research; (4) implementing team science and translational research studies; and (5) communication in clinical and translational science. A total of 48 trainees completed the self-assessment at the beginning, and 23 of these individuals did so at the end of the program. This might introduce nonresponse bias however early versions of the survey did not assign unique identifiers to respondents making it difficult to assess whether respondents and nonrespondents to the exit survey were different at baseline. The confidential survey was administered online via SurveyMonkey® (San Mateo, CA, USA).

Qualitative Data Sources

A qualitative interview guide was developed to collect information about the attainment of program goals and objectives. After the evaluator developed the guide, the CPTR program directors (H.M.B., T.R.Z.) reviewed the guide and provided feedback. Revisions were made to capture data about important evaluation domains. The guide included both open-ended and close-ended questions to assess knowledge and skills gained, mentoring relationships, coursework, the impact of the program on career development, and areas for program improvement. Interviews were conducted from September 2013 to December 2015. Interviews were digitally audio-recorded and transcribed verbatim.

Analysis

Quantitative Analysis

For quantitative metrics (e.g., engaged in a career that encompasses clinical and translational research, publications, grant funding) descriptive statistics were calculated. For the self-assessment survey, descriptive and inferential statistics were calculated using SAS 9.4 (SAS Institute, Cary, NC, USA). The 22 competency area items were recoded into numeric variables with a value of “1” representing a rating of “low” confidence, “2” representing “medium,” and “3” representing “high.” The mean and standard deviation were calculated for each item at baseline and exit. Effect size and 95% confidence intervals were calculated using PROC GLM.

Qualitative Analysis

For qualitative interviews, digital recordings were transcribed verbatim. Two researchers independently reviewed the verbatim transcripts and identified codes from key questions. Deductive codes were developed from the interview guide and inductive codes emerged from the data. The coding focused on major themes pertaining to the evaluation domains.

Results

To date, 61 trainees have enrolled in the CPTR and 37 have graduated from the program. In total, 40 (66%) trainees are women and 21 (34%) are men; 28 (46%) are white, 19 (31%) are Asian, 8 (13%) are African American or Black, and 6 (10%) are Hispanic (Table 1). The largest proportion of trainees were Ph.D. graduate students (n=30, 49%), 5 trainees were students in the combined M.D./Ph.D. program, 12 trainees were Ph.D. postdoctoral trainees (20%), 2 trainees were resident physicians or clinical fellows, 10 (16%) were faculty members, and 2 trainees were research staff members (1 was an M.D. and the other was a Ph.D.). In total, 55 (90%) of the trainees were from Emory, 4 (7%) trainees from MSM, and 2 (3%) were from GA Tech.

Table 1

Characteristics of Certificate Program in Translational Research (CPTR) Trainees, 2011–2017 (n=61, including 24 currently enrolled trainees)

Demographics	n (%)
Gender
Female	40 (66)
Male	21 (34)
Race/ethnicity
White	28 (46)
Asian	19 (31)
African American/Black*	8 (13)
Hispanic*	6 (10)
Position during training
Faculty	10 (16)
Ph.D. graduate student	30 (49)
M.D./Ph.D. student	5 (9)
Postdoctoral fellow (Ph.D.)	12 (20)
Resident/fellow (M.D.)	2 (3)
Research staff (Ph.D. and M.D.)	2 (3)
Institution affiliation
Emory University	55 (90)
Morehouse School of Medicine (MSM)	4 (7)
Georgia Institute of Technology (Georgia Tech)	2 (3)

Underrepresented minorities as defined by the NIH (n=14, 23%).

By the end of the program, trainees reported increased confidence in the 22 competencies in clinical and translational research (Table 2). The effect sizes for 19 of the competencies were greater than 0.14, indicating that the magnitude of improvement was large. The competencies with the largest magnitude of improvement were (1) summarizing the principles and practices of the spectrum of community-engaged research (M=1.46 vs. M=2.74, η2=0.47); (2) propose a study protocol for addressing clinical or translational research question (M=1.75 vs. M.2.78, η2=0.44); and (3) collaborating with bioinformatics specialists in the design, development, and implementation of research projects (M=1.67 vs. M=2.74, η2=0.42); these gains in competency spanned critical domains of clinical and translational research training. The competencies with lowest magnitudes of improvement were: (1) conduct a comprehensive and systematic search of the literature using informatics techniques (M=2.33 vs. M=2.65, η2=0.06); (2) devise and rigorously tests an experimental hypothesis (M=2.40 vs. M=2.83, η2=0.10); (3) extract desired information from the medical record (M=1.96 vs. 2.61, η2=0.14). The high mean levels of confidence at baseline may explain why the magnitude of improvement was lowest for these competencies.

Table 2

Mean confidence levels before and after completing the program and effect size in 22 competencies in the Core Thematic Areas of the Certificate Program in Clinical and Translational Research

Core thematic area	Competencies	Pretest (n=48) [mean (SD)]	Post-test (n=23) [mean (SD)]	Effect size [partial η2 (95% CI)]
Spectrum of translational research	Relate translational research to the spectrum of moving research to practice	1.77 (0.66)	2.52 (0.67)	0.22 (0.07, 0.38)
	Describe the spectrum of current cutting-edge translational research	1.54 (0.62)	2.48 (0.59)	0.35 (0.17, 0.49)
	Describe the role of biostatistics and bioinformatics in biomedical and public health research	1.81 (0.64)	2.78 (0.42)	0.39 (0.21, 0.52)
	Collaborate with bioinformatics specialists in the design, development, and implementation of research projects	1.67 (0.66)	2.74 (0.45)	0.42 (0.24, 0.55)
	Critique clinical and translational research	1.88 (0.61)	2.74 (0.45)	0.35 (0.17, 0.49)
	Describe quality improvement research	1.71 (0.71)	2.43 (0.59)	0.21 (0.06, 0.36)
	Describe the culture of medicine and how it is practiced	1.71 (0.80)	2.48 (0.59)	0.20 (0.05, 0.35)
Designing clinical and translational studies	Conduct a comprehensive and systematic search of the literature using informatics techniques	2.33 (0.63)	2.65 (0.57)	0.06 (<0.01, 0.19)
	Devise and rigorously test an experimental hypothesis	2.40 (0.68)	2.83 (0.39)	0.10 (0.01, 0.25)
	Propose a study protocol for addressing clinical or translational research question	1.75 (0.60)	2.78 (0.42)	0.44 (0.27, 0.57)
	Summarize principles and practices of community-engaged research	1.46 (0.71)	2.74 (0.45)	0.47 (0.30, 0.46)
	Extract desired information from the medical record	1.96 (0.80)	2.61 (0.66)	0.14 (0.03, 0.31)
Research ethics and the responsible conduct of research	Use high legal and ethical standards in the conduct of researchDescribe the relevance of cultural and population diversity in clinical research design	2.46 (0.65)1.75 (0.79)	2.96 (0.21)2.91 (2.29)	0.16 (.05, 0.28)0.41 (0.23, 0.54)
Implementing team science and translational research studies	Build an interdisciplinary team that matches the objectives of the research problem	1.79 (0.77)	2.61 (0.58)	0.23 (0.07, 0.38)
	Manage a team of interdisciplinary translational scientists	1.58 (0.65)	2.35 (0.57)	0.25 (0.09, 0.40)
	Conduct clinical or translational research project	1.71 (0.68)	2.61 (0.58)	0.30 (0.13, 0.45)
	Integrate elements of translational research into given study designs that could provide the basis for future research	1.81 (0.70)	2.78 (0.42)	0.35 (0.17, 0.49)
	Generate simple descriptive and inferential statistics that fit the study design chosen and answer research question	1.94 (0.76)	2.61 (0.58)	0.17 (0.04, 0.32)
Communication in clinical and translational science	Prepare grant proposals that are judged successfully competitiveEffectively communicate detailed information in manuscript form	1.75 (0.64)2.13 (0.67)	2.65 (0.49)2.74 (0.45)	0.34 (0.17, 0.49)0.19 (0.05, 0.34)
	Communicate information to a research subject in a clinical trial	1.88 (0.79)	2.91 (0.29)	0.35 (0.20, 0.49)

Scale: 1, low confidence; 2, medium confidence; 3, high confidence

To date, all 37 (100%) of the trainees who have completed the CPTR have continued with a career that encompasses translational research (Table 3). In total, 24 are in academic positions (65%), 11 (30%) are working in industry, 1 (3%) works for the federal government, and 1 (3%) works for state or local government. The 37 trainees who have completed the program have published 290 peer-reviewed articles, of which 108 (37%) were first author and 18 (6%) were senior author publications. They have also authored 13 book chapters and 1 book. The CPTR graduates have received a total of 22 grants as a principal investigator. This includes 8 NIH grants totaling over $4 million (F31=3; K01=2; K23=1; R01=1; F30=1); 3 U.S. Department of Defense grants which exceed $15 million; 4 foundation grants which exceed $300,000; 6 internal institutional grants exceeding $200,000; and 1 pharmaceutical company grant ($20,000).

Table 3

Accomplishments of Certificate Program in Translational Research graduates: careers in clinical and translational science, publications, and grants

Accomplishment	n (%)
Careers in clinical and translational research	37 (100)
Academic	24 (65)
Industry	11 (30)
Federal government	1 (0.03)
Local/state government	1 (0.03)
Total number of published articles	290
First author	108 (37)
Senior author	18 (6)
Total number of grants awarded	22
NIH Federal funding ($4 million)	8 (36)
Non-NIH federal funding (DoD) ($15 million)	3 (14)
Foundation grants ($300,000)	4 (18)
Internal institutional grants ($200,000)	6 (27)
Pharmaceutical ($20,000)	1 (4)

Qualitative Results

In the qualitative interviews, trainees discussed the influence of the CPTR on their career pathways in clinical and translational science. Five of the interviewees were women and 4 were men. The interviewees had completed the program from 1 to 3 years before the interview. Six of the interviewees completed the CPTR as Ph.D. students, 1 trainee was a research staff member who already completed her Ph.D., and 2 were physicians. Seven of the participants were trained or working at Emory, 1 participant was from Georgia Tech and 1 was from Morehouse School of Medicine.

Flexibility of Program

Trainees reported that the flexibility of the CPTR allowed them to pursue training in translational research while remaining on track with their own graduate or postdoctoral programs. In particular, the CPTR did not delay their research project completion or for Ph.D. students, expected graduation date. This made the program more desirable than a Master’s degree which could require 1 or 2 additional years of training or time to Ph.D. degree. For example, one trainee explained, “The certificate program…didn’t [require me to] push back [my dissertation] defense date…and the class schedule was very amenable to my research schedule.” Another trainee explained, “I wanted to do something…alongside my Ph.D. program without it causing huge delays in my Ph.D. progress or time to graduation so that’s why I chose the certificate over the master’s [degree].” Trainees found that the CPTR training enhanced their ability to include clinical and translational research in their current or future career goals. One trainee explained, “I have a really great core knowledge from what I learned from the Certificate program. So for me, it was actually a really, really nice thing to be able to do…[ and it] helped me launch my research career once I got to a fellowship.” In addition, the program provides an opportunity to enroll in electives that are relevant to translational research but outside a trainee’s primary program. For example, one trainee used the CPTR to gain exposure to clinical and translation research methods and also learn the process of patenting new technology. This was invaluable to her own career trajectory which included drug discovery for HIV. She stated, “I was allowed, under the umbrella of the program, to take any elective I wanted…I liked that there was flexibility, so you could…look anywhere at the university and say I want to take that because I think it’s interesting.” Another trainee explained, “I wanted…to be able to…apply my findings to human populations… and with a PhD, it’s not easy to do everything at the same time… I wanted to be able to apply my knowledge, my findings in the lab to more human populations and see if we can cure disease….”

Coursework in Clinical and Translational Research

The CPTR courses exposed scientists to important concepts, methods and approaches to research with human populations. Trainees reported that the courses in biostatistics, clinical trial research design, and scientific and grant writing were the most beneficial to their training. In particular, these courses enabled the trainees to think critically about how to analyze studies with data about humans compared with their own research based on animal models, and then describe a translational research project in a grant proposal. One trainee explained, “the clinical trials introduction course was really [helpful]. It was an understanding of clinical trial design that I think was really, really helpful.” Another trainee said, “I learned about statistics that were not usually used in animal models [but] were used in… human studies [and] the way to analyze human studies. So it made it easier to read papers in clinical trials.” Trainees also realized that the exposure to clinical and translation research methods and concepts prepared them to work collaboratively with diverse scientists. One trainee explained, “the biostatistics course was really helpful. I don’t use [biostatistics] on a day-to-day basis but when I meet with our biostatistician…I can have an intelligent conversation as a direct result of my experience.”

Many trainees discussed the value of the course on grant writing because the curriculum focused on the specific elements of translational science in research proposals. Moreover, trainees crafted an individual grant throughout the semester and received feedback on their writing and research design. One trainee explained:

What was new to me was when you do patient oriented research, the additional things you need to add in the grant and consider before you start writing grants. So you know, the budgeting related to translational research. I have not learned that before [when I was a post-doc]. It was all from basic science perspective, so when I did the translational research, even the budgeting, writing grants, the structure and submitting to grants—all of those things were different.

Another trainee described the process of conceptualizing a translational research project throughout his work in the grant writing course. Although at first his lack of direction seemed like a hindrance, he developed an innovative project while completing the tasks assigned by the instructors. He explained,

The project I was working on the first half of the year fell through because of not enough pre-clinical data and I shifted gears into a new project right around the time I started [the grant writing] class and I was kind of frustrated. I thought “well, I don’t have much to show for what I’ve done so far” but I started – I just started from scratch right at the beginning of that course and I wrote a practice grant. And through that process, I really got into the trial design and into that project deeper and deeper and that actually really helped me design [my graduate trainee] study.

The grant proposal required in the grant writing course influenced his research interests. At the time of the interview, several years after completing the course, he was submitting that same study for NIH governmental funding. He explained:

I’ve applied for several grants using…the information I learned in that class…I’m thankful for the experience in that class to have helped me…be prepared…I use…the information I learned in that class all the time. So I think that was actually the most practical, most helpful of all the classes in the Certificate program.

For some trainees, the grant writing class product focused on an NIH K or F award. Trainees reported that the experience and expertise of the instructors in the Scientific and Grant Writing course improved their proposals and research projects. One trainee explained:

For the grant writing course, I used what I was planning to submit for my K award. I mean, it was just to have that hands-on [guidance]. [The instructors] really read over [the grant] and gave me really good critical feedback…going through the whole process, a whole class on that was just – it was just perfect. It was vital for me to keep me going and to have a finished project at the end. It was wonderful. And that’s the one that ultimately ended up being funded.

Other trainees echoed similar sentiments, stating, “The grant writing course…[was] the number one most helpful part of the program.”

Value of a Clinical Rotation

As part of the CPTR, nonclinical trainees are required to complete a 20-hour clinical rotation with a clinical investigator that is conducting research in an area related to the trainee’s research as part of the “Translation to Medicine” course. Ph.D. graduate students and postdoctoral fellows reported that the clinical rotation of the course had a major impact on their understanding of translational science. Many had only worked in laboratory settings and lacked exposure to the clinical side of research. Interacting with actual patients with disorders relevant to their preclinical work contributed to a more sophisticated understanding of translational science, and solidified the importance of linking their own scientific research to their ultimate desire to improve human health. For example, one trainee explained that the clinical rotation reinforced the critical transition of bench research to patient populations.

In our [PhD] training…you are working on some real-world applications but even then, it felt like there was still a lack of attention to the clinic. And what I’d really been interested in was those clinical applications right away… So I think that [the CPTR] helped me understand that transition from pre-clinical to clinical areas, you know, just to get a little more of that real-world application.

Another trainee stated, “Seeing the actual human side of what I was researching was—it made that connection that I didn’t previously have…And so it just drove home that we really need to develop a medication that actually works and alongside that is a therapy that actually works.” Another trainee explained:

Before [the clinical rotation] I really hadn’t – I’d never shadowed a physician, I didn’t – I didn’t really know what was going on behind the scenes when you just go into a regular doctor’s office or a clinic. And it was interesting to just see that side of health care, which I’d never really been exposed to.

Learning and Collaborating With Diverse Colleagues of Different Academic Levels

Several CPTR trainees discussed that the opportunity for team science collaboration with peers and colleagues who were medical students, physicians in training (residents and fellows), and Ph.D.s broadened their exposure to diverse research processes and enriched their understanding of their own translational research interests. They learned how to work with scientists from diverse disciplines and with varying methodological expertise, both in their CPTR classes and their joint classes and Translational Research Journal Club with the MSCR trainees. One trainee explained:

When it comes to clinical research, I think…PhDs and doctors can complement each other’s knowledge… I kind of got the sense of how it’s going to look like when you work in a hospital. Scientists and medical doctors can try to work together to…come up with a solution to solving a particular…disease problem or…a condition that…you need to find some sort of treatment for it. At the end of the day, if you really want …to do translational research…it takes both medical doctors and PhDs to work together. In this kind of [program], we get the exposure of how PhDs think, how medical doctors think. So I think that the whole transition from academia to medical settings and getting to work with each other, it will be much smoother.

Another trainee expressed a similar view regarding the value of having courses with a combination of medical and Ph.D. trainees. They explained:

I think another strength of the program was the fact that we had…a mix of medical school students and PhD students so you know, when it comes to like different topics that people research, it…was very valuable for me to know how medical school students are…thinking about the problem versus how PhDs are thinking about the problem and how different groups have different knowledge so we…can complement each other.

Another trainee expressed the value in learning from diverse peers. They explained:

We [PhD students] tend to think about it from a very, you know, molecular level…So we tend to think from the small and slowly build it up and go to the – get the larger picture potentially whereas like the medical school students start from the large, like the big picture and then slowly narrow it down. So I was like, oh my…the way that they think is the exact opposite of what we think…

Exposure to Interdisciplinary Science

The CPTR expanded the trainees’ scope of expertise and their ability to think broadly about the implications of their research and the connections between their disciplines and a broader network of clinical and translation research. Linking diverse research areas made them stand out from other candidates in their field during the subsequent academic journey. For example, one trainee stated:

I think you come out of grad school and you have an advanced degree, people know you’re smart but what are you smart in? And if all you can talk about is your pre-clinical research and what you did for your dissertation, it kind of leaves you kind of a subject matter expert in a very precise field. And what the CPTR enabled you to do is to talk broadly about a whole field that you wouldn’t be able to talk about previously.

Another trainee explained that the CPTR training demonstrated how interdisciplinary research is carried out in an academic and clinical setting. The coursework combined with her clinical rotation augmented her Ph.D. training and facilitated opportunities for interdisciplinary work. She explained,

The program had a huge impact. When I entered the program, I knew that…I wanted to do something that is interdisciplinary, like basic science and also clinical research… the program taught me how interdisciplinary research operates. And then when I finished my PhD, and when I applied for [my] fellowship, the only thing that they asked me about was my experience with the Certificate program and what I learned from my clinical rotation…So I would say the Certificate played a very major role in me getting the fellowship.

Another trainee who completed the CPTR applied for the NIH loan repayment program and received the award. She explained that the program officers told her that she “actually got the loan repayment grant because of the translational research program.”

Advantages on the Job Market

Several trainees were encouraged by their faculty advisors and professors to enroll in the CPTR to expand their career opportunities after graduation and increase their competitive edge on the job market. One trainee reported that the credentials of the CPTR demonstrated her commitment to clinical and translational research compared with her colleagues who simply expressed interest without pursing formalized training. She explained, “not that many people do [the CPTR program] and they kind of just stay in their own little microcosm of science or research. And they might be aware of the translational aspects but actually pursuing it and trying to understand it further, I don’t think very many people do that.” As a result, she has discussed her Certificate in job applications and interviews to highlight her skills and commitment to clinical and translational research. She said, “the [CPTR] basically allowed me to go into a job interview and talk about clinical research…it helped me…transition into the industry and tweak my career… pivot into a more clinical application of what I’d been doing, which I wouldn’t have otherwise.” One trainee stated, “I needed some sort of way to transition…to get myself a leg up amongst everybody else who was also getting a Ph.D.” Another trainee explained, “when I’m talking to people, networking, sending out resumes [the CPTR] always comes up as a conversation piece… it opens doors.” Another trainee explained, “I really believe having the certificate on my biosketch [and] my CV was a strength.”

Overall Impact

Overall, the trainees reported that the CPTR fulfilled its goals to provide trainees with a means to pursue translational research throughout their careers. Enrolling in the CPTR allowed trainees to engage in research with a more direct human connection, and many stated that this stimulated their passion as a researcher—it enhanced their ability to conduct experiments and write papers for academic audiences toward developing a sense of bettering the health of communities in a more direct manner. One trainee explained, “The strengths of the program are, I guess as it’s said in the title, it’s the ‘Certificate Program in Translational Research’ and it bridges people from the basic sciences to the application of that science to patients and makes that bridge a more tangible thing.” While reflecting, another trainee stated:

I would say [the CPTR] developed my…appreciation of where I fit in the health care scheme and health care as a whole. I think the program really bridged a lot of different areas from…medical doctors at an actual clinical site and what they’re conducting and what’s being performed there, to how that [information is being] analyzed, how that is interpreted and how the results are…disseminated to the public. So I think from that standpoint, it just helped me develop an appreciation of…clinical research.

Another trainee reported, “I got a lot of short-term and long-term benefit from [the program]…the biggest thing was that it just expanded my knowledge of clinical trials and clinical research. Ph.D. trainees, at least in my graduate division, are not typically exposed to clinical applications or even just bedside application of what they were working on.” One trainee explained, “[the program] made the translational aspect of it real, that what I’m doing in the lab, even though I’m working with animal models, still has an impact on…what happens in real life in humans and I think that’s the main takeaway that I’ve gotten from this [program].”

Recommendations for Program Improvement

Although the trainees expressed positive experiences with the program, they also had recommendations for improvement. Several trainees recommended a longer rotation in the clinic so that their experiences with patients and physicians were more in-depth or a more comprehensive list of elective courses that could benefit diverse career and research interests. Expanding the coursework options could further tailor the program to their individual interests. Another trainee suggested more professional development such as negotiating salaries and contracts while another recommended adding program events related to careers outside of an academic setting. For example, aligning CPTR trainees with a mentor in industry would strengthen the networking opportunities with diverse scientists in settings outside of academia. Another trainee recommended a building a strong network for the alumnae of the program so that they could continue to build career and research collaborations with each other. Along the same lines, one trainee thought that it would be beneficial to have a specific mentor assigned who could guide them in clinical and translational research careers in addition to the program directors who worked in this capacity.

Discussion

There is a pressing need to increase the pipeline and diversity of a multidisciplinary workforce engaged in clinical and translation science and to translate new discoveries to improve the health of communities [3, 5, 16, 17]. Clinical and translational research training to enhance multidisciplinary research teams has been emphasized as a major National Center for Advancing Translational Sciences (NCATS) priority [18-20]. The incorporation of more flexible and innovative clinical and translational research training and a team science orientation was emphasized by the 2013 Institute of Medicine report on the CTSA programs [1], and echoed by other thought leaders [21-24]. These programs need to accommodate the needs of a wide range of trainees and faculty that enter the field from diverse disciplines, a range of degree-seeking programs and various stages of career. The CPTR program developed with support of a Howard Hughes Medical Institute grant and subsequently institutionalized through support of our CTSA is one such example. As reported by program participants, the flexibility in the CPTR facilitated trainees attaining skills in clinical and translational research possible while continuing their original academic training (e.g., Ph.D. graduate degree program or postdoctoral fellowship training) and was felt by trainees to enhance their career trajectory. The CPTR confirmed and enhanced their desire to translate basic science discoveries to improve human health. All trainees who have completed the program remain in careers that encompass translational research. The CPTR offers an important pathway into clinical and translational research as an alternative to other training efforts such as a master’s degree in clinical and translation science, which may be overly time-intensive for some professionals who are pursuing a Ph.D. degree or already have a doctoral degree but don’t have the skills or knowledge on how to translate their findings from the lab to the bedside or from the bedside into the community. The CPTR model we have developed in our CTSA program increases the translational research workforce and enhances the career development of future leaders of the biomedical research workforce, a major mission of NIH and NCATS [1, 21].

Qualitative evaluation of coursework and training, and quantitative data about trainee accomplishments and confidence in clinical and translational research skills, illustrate that the length and content of the CPTR provide substantial skill-building and career support in the essential areas of clinical and translational research [25]. The CPTR will continue to recruit a strong and diverse pool of potential candidates at the Georgia CTSA partner institutions, Emory, MSM, Georgia Tech, and our new partner, the University of Georgia. In addition, the CPTR is adopting new program components to continue to ensure success, based on trainee feedback, our CTSA External Advisory Committee, and NIH/NCAT clinical and translational guidance for increasing the future translational research workforce [1, 26–28]. Each CPTR trainee is required to complete an individual development plan (IDP) [29] to outline their personalized training pathway goals in areas such as research methodology, team science, mentorship and leadership training, as well as plans for publication and grant submissions. Mentors and program leadership provide feedback on the IDP to ensure that each trainee’s goals are attainable and within the scope of the program. The IDP is the foundation for the evaluation of each scholar’s progress throughout the program.

The program also works with trainees to ensure they have the flexibility to enroll in courses and trainings within their specialty area while adhering to the core CPTR competencies. For example, mentors and program directors review the scholar’s IDP and recommend additional electives that will enrich their training and career goals in clinical and translational science—this includes, for example, relevant electives in the business school and/or school of public health. Given the emergence of “big data” from rapidly expanding omics technologies [30-32], the CPTR has also added to the curriculum a new course that began in fall semester 2017 entitled “Fundamentals of Big Data for Clinical and Translational Research.” The CPTR is also enhancing professional development training. Recently, we implemented a series of lectures and activities on the science of team science [33], which prepares trainees to propose and collaborate on projects appropriate for NIH funding as well as entrepreneurship training. Trainees are guided through a process of customer discovery and business model generation as another means to further translate discoveries to benefitting human health. CPTR trainees are also encouraged to participate in leadership training that takes place in conjunction with the NIH-funded Broadening Experiences in Scientific Training (BEST) program at Emory—a program that prepares trainees for potential careers outside of academia such as positions in biomedical and pharmaceutical industries [27]. The CPTR is also providing opportunities for externships with local and national biomedical industry partners (e.g., through Georgia Bio, http://www.gabio.org), state and federal agencies (e.g., Centers for Disease Control and Prevention), foundations (e.g., American Cancer Society), and local and national CTSA partner institutions. Externships provide trainees with new scientific, theoretical, and practical knowledge, including an appreciation of technology, regulatory, and commercialization challenges in developing and bringing a biotechnology or pharmaceutical product to clinical use. Finally, to bolster the translational pipeline, the CPTR has implemented a new mentor training program that focuses on transitioning participants from trainees to future mentors [34-36]. This is a critical step in building a cadre of influential mentors for the next generation of clinical and translational scientists.

The CPTR will continue to use the evaluation results to revise and develop new aspects of training in translational research. For example, our evaluation showed that past program participants would like to have continued contact with other trainees through a more formalized alumni network and annual gatherings. This would strengthen networking and mentoring between scientists and could possibly broaden career opportunities in clinical and translational research. We are in the process of implementing an online social networking site to transmit program updates and important messages (such as upcoming events, job openings, and other career opportunities) to and between alumni. An alumni network such as this can facilitate collaborations across academic, industry, and governmental institutions. This will link current trainees to past program participants and optimize the potential to learn about diverse career pathways—an expressed interest that emerged in the evaluation interviews. Solidifying a network of scientists in this manner works towards a major goal of the Georgia CTSA to grow the number of clinical and translational research networks and research projects locally and nationally. In addition, we will continue to conduct a mixed methods evaluation of the CPTR to determine the long-term impact of the program. In particular, we will continue to assess the number of trainees who continue with careers in clinical and translational science and the diversity of their careers.

Limitations

This study is subject to certain limitations. The results of the study are limited to the experiences of the participants in the program. However, the evaluation contributes important lessons on how certificate programs can successfully train scientists to become clinical and translation researchers. The interviewees were selected from a convenience sample and, thus, do not represent all of the program participants. Nonetheless, efforts were made to collect data about the strengths as well as the weaknesses of the program. More participants completed the pretest survey than the post-test survey and we did not collect unique identifiers to measure change at the individual level. Since these analyses, we have added questions to the survey that allow us to match pretest and post-test surveys and we are more actively recruiting trainees to complete the exit survey so that we have better response rates. This study only includes trainees who completed the CPTR in the past 6 years. The program is still relatively new and we are unable to capture longer-term data. Additional research is underway to capture data about the program impact of our CPTR trainees during the course of their professional careers.

Conclusion

Our CPTR was developed in an effort to increase the number of scientists with the skills to translate and disseminate new scientific discoveries across the translational pipeline. This study explored the experiences of trainees in our CPTR program which is supported by our NIH-funded Clinical and Translational Science Award, now known as the Georgia CTSA. Quantitative and qualitative data indicate that the program participants felt that they received valuable comprehensive training in clinical and translational research. Graduates of the program have remained dedicated to translational research through multiple career avenues, including positions at academic medical centers, industry, and local and federal government agencies. The CPTR serves as a model for other institutions with interests in implementing translational research training programs to a diverse pool of new scientists interested in clinical and translational research to benefit human health.