Sowing the "CEED"s of a more diverse biomedical workforce: The Career Education and Enhancement for Health Care Research Diversity (CEED) program at the University of Pittsburgh.

PURPOSE: The need to diversify the biomedical research workforce is well documented. The importance of fostering the careers of fledgling underrepresented background (URB) biomedical researchers is evident in light of the national and local scarcity of URB scientists in biomedical research. The Career Education and Enhancement for Health Care Research Diversity (CEED) program at the University of Pittsburgh Institute for Clinical Research Education (ICRE) was designed to promote career success and help seal the "leaky pipeline" for URB researchers. In this study, we aimed to quantify CEED's effect on several key outcomes by comparing CEED Scholars to a matched set of URB ICRE trainees not enrolled in CEED using data collected over 10 years.
METHOD: We collected survey data on CEED Scholars from 2007 to 2017 and created a matched set of URB trainees not enrolled in CEED using propensity score matching in a 1:1 ratio. Poisson regression was used to compare the rate of publications between CEED and non-CEED URB trainees after adjusting for baseline number of publications.
RESULTS: CEED has 45 graduates. Seventy-six percent are women, 78% are non-White, and 33% are Hispanic/Latino. Twenty-four CEED Scholars were matched to non-CEED URB trainees. Compared to matched URB trainees, CEED graduates had more peer-reviewed publications (p=0.0261) and were more likely to be an assistant professor (p=0.0145).
CONCLUSIONS: Programs that support URB researchers can help expand and diversify the biomedical research workforce. CEED has been successful despite the challenges of a small demographic pool.

Introduction

In academia, a successful biomedical research career is one in which the investigator is able to design and implement a research project, present and publish the findings, and compete for grant funds in order to carry out basic, clinical, or translational research. Additionally, understanding the academic environment is vital. Programs designed to prepare early-career researchers from underrepresented backgrounds (URB) for academia are increasingly common, with a 2012 environmental scan documenting their presence at 29% of 124 MD-degree-granting medical schools. The importance of fostering the careers of fledgling URB biomedical researchers is especially important in light of the national and local scarcity of URB scientists in biomedical research.–

The Career Education and Enhancement for Health Care Research Diversity (CEED) program at the University of Pittsburgh Institute for Clinical Research Education (ICRE) is a purposefully designed program of multi-level mentoring, targeted coursework, monthly seminar series, and networking opportunities designed to promote success in academic research for URB biomedical researchers. CEED’s goal is to provide a solid foundation for a successful research career for URB scientists while ensuring a supply of well-qualified clinical and translational investigators in the biomedical research pipeline. CEED aims to achieve this goal by providing mentoring and training in scientific and grant writing, research presentation, and other skills required to apply for competitive career development awards or other types of grant funding appropriate to each Scholar’s level.

While CEED has grown into a competitive, year-long professional and research development program for URB biomedical researchers, its overall effectiveness is unknown. In this paper, we describe the components of the program, quantify program effectiveness on several key outcomes, and discuss future directions.

Description of the CEED Program

CEED was explicitly designed to achieve an increase in the URB biomedical research workforce. The program is housed within the ICRE, which supports the education and training components of the University of Pittsburgh’s Clinical and Translational Science Institute (CTSI). CEED leverages the resources of the ICRE to support CEED Scholars. The program is led by two co-directors (KZA, NEM), with administrative support from a program coordinator. An 11-member Executive Committee reviews and rates applicants for acceptance into the program and provides guidance for future directions. Executive Committee members also serve as advisors to CEED Scholars.

CEED employs a unique, four-component structure that includes a monthly seminar series, multi-level mentoring, targeted coursework, and networking opportunities.

The Monthly Seminar Series is the core of the CEED program and runs from July through April of each program year. CEED Scholars have an opportunity to present their research projects and receive critiques and encouragement in a collegial and friendly environment. The CEED program creates a “safe” environment where issues of race, ethnicity, and gender can be openly discussed. Presentations are complemented by a variety of talks designed to frontload important information and skills needed to succeed in an academic research environment. Topics include Giving Effective Presentations, Curriculum Vitae, Negotiation, Difficult Conversations, Writing Productivity, NIH Training (K) and R Grants, and Letters of Recommendation.

CEED incorporates three levels of mentoring. These levels run concurrently throughout the course of the program to enable the Scholar to integrate the support of each mentor into a cohesive mentoring experience (Fig. 1).

Fig. 1.

The Career Education and Enhancement for Health Care Research Diversity (CEED) levels of mentoring.

In addition to the program-specific focus on mentoring, each CEED Scholar is expected to participate in the annual Mentoring Matters Workshop at the ICRE. This interactive, half-day workshop guides participants in developing a mission statement for their careers, finding the best mentor to help achieve their goals, developing expectations for their mentoring relationship, and finding strategies for ensuring success. The workshop provides another unique opportunity for Scholars to work with a diverse and multidisciplinary group of senior faculty members and to network with both clinical and translational researchers from across the University. All ICRE programs require the applicant to have a mentor when enrolled in the ICRE but there is not a formal mentoring program across the entire ICRE.

Courses offered through the ICRE follow a competency-based model and provide CEED Scholars with an opportunity to further their education in areas of research in which they may have had little experience or exposure. The ICRE offers several courses that are particularly beneficial to CEED Scholars (Table 1).

Table 1.

Courses beneficial to The Career Education and Enhancement for Health Care Research Diversity (CEED) scholars

Course Name	Description	ICRE Competencies Addressed
Research Design and Development (Grant Writing)	Scholars learn the phases of the research process from conception to design and, ultimately, to implementation of the research. Through a combination of group sessions and independent work, trainees use a research topic of their choice to develop their own research proposal in the form of an NIH grant application.	Problem FormulationMethodologySamplingMeasurementOral CommunicationWritten CommunicationMultidisciplinary Teamwork
Introduction to Grant Writing	For those Scholars who are not yet prepared to take Research Design and Development.	Oral CommunicationWritten CommunicationMultidisciplinary TeamworkEthics and Professional Norms
Medical Writing and Presentation Skills	Medical educators and researchers must be able to present their work clearly and effectively. The main objective of this course is to help students develop excellent medical writing and presentation skills.	Oral CommunicationWritten CommunicationMultidisciplinary TeamworkEthics and Professional Norms

Throughout the program year, CEED Scholars are given multiple opportunities to network and connect with senior faculty members. At the beginning of the year, there is a Meet-and-Greet that brings together participants in CEED and CEED II (CEED II is a program for URB medical students and is beyond the scope of this manuscript), CEED program directors, the CEED Executive Committee, the Scholars’ mentors, and CEED alumni. This allows the Scholars to meet with program leaders and converse in an informal setting. At the end of the program year, CEED Scholars present their scholarly work through a poster presentation to all Scholars’ mentors and the CEED Executive Committee. This popular session highlights each Scholar’s research accomplishments for the past year and provides another networking opportunity.

Methods

Study Design

This was a retrospective study design that included all CEED and non-CEED URB trainees who enrolled in the ICRE between the years 2007 and 2017, inclusive. All trainees complete an ICRE-developed survey upon matriculation into any ICRE program. This instrument assesses a wide variety of career success metrics.– Trainees also are asked to complete it annually while enrolled in any ICRE program, upon graduation, and then every other year as alumni. The number and type of publications and grants were abstracted from submitted CVs while demographic and professional characteristics were self-reported.

Statistical Analysis

Our primary question focused on the effect of the CEED program among URB trainees within the ICRE over 10 program years. Because the CEED and non-CEED trainees were not determined randomly, there were differences between both groups in demographic and other characteristics. Additionally, as the retrospective study design (vs prospective) can add additional bias, we utilized propensity score matching to minimize bias and create a comparable cohort of CEED and non-CEED URB trainees. Logistic regression was used to model the propensity of being in the CEED program as a function of race (White vs non-White), ethnicity (Hispanic vs non-Hispanic), degree (MD vs PhD), and faculty status at the beginning of a trainee’s program year. We created a matched set of non-CEED URB trainees also enrolled in ICRE programs in a 1:1 ratio, after assessing between-group imbalance using standardized mean differences. Descriptive statistics included means, standard deviations, and sample proportions; between-group differences were assessed with two-sample t-test or chi-square tests. Poisson regression was used to compare the rate of publications between CEED and non-CEED URB trainees after adjusting for baseline number of publications. Due to missing publication data, we performed sensitivity analysis by imputing zeros for all missing cases and re-running the Poisson regression on these data.

Statistical analysis was conducted using SAS version 9.3. All statistical testing was done with 5% type I error without adjustment for multiplicity.

Results

Our cohort comprised 45 CEED and 61 non-CEED URB trainees from 2007 to 2017. While similar in terms of gender distribution between groups, there were noticeable differences on race (78% vs 59% non-White; p=0.0072) and ethnicity (33% vs 54% Hispanic/Latino; p=0.0274). Note that URMs may be designated as White and Hispanic/Latino. CEED trainees were more likely to hold PhDs (56% vs 25%; p=0.0018) as well as hold a faculty position at baseline (22% vs 5%; p=0.0177). Finally, the mean number of publications per trainee was higher among the CEED cohort (5.44 vs 2.74; p=0.0007) (left-hand side of Table 2). After using propensity score matching to create a comparable cohort of 24 non-CEED URB trainees, demographics and professional characteristics were similar between groups (right-hand side of Table 2). This was further validated by calculated absolute standardized mean differences less than 0.2 (see supplemental Fig. 1).

Table 2.

Characteristics of The Career Education and Enhancement for Health Care Research Diversity (CEED) participants vs Non-CEED participants (2007–2017)

	Before Propensity Matching			After Propensity Matching4
Descriptive Characteristics	CEED	Non-CEED URB1	p-Value‡	CEED	Non-CEED URB1	p-Value‡
Graduates	45	61		24	24
Female, n (%)	34 (76)	38 (62)	0.1483c	19 (79)	17 (71)	0.5050c
Race						0.7453c
Non-White, n (%)	35 (78)	44 (59)	0.0072c	17 (71)	18 (75)
White, n (%)	8 (18)	23 (31)		7 (29)	6 (25)
Ethnicity						0.7628c
Hispanic/Latino, n (%)	15 (33)	33 (54)	0.0274c	9 (38)	8 (33)
Degree2						0.7555c
MD, n (%)	20 (44)	44 (72)	0.0018c	17 (71)	16 (67)
PhD, n (%)	25 (56)	15 (25)		7 (29)	8 (33)
Faculty Status3(baseline)
Faculty, n (%)	10 (22)	3 (5)		4 (17)	3 (13)
Non-faculty, n (%)	34 (76)	50 (82)		20 (83)	21 (87)
Other, n (%)	0 (0)	2 (3)	0.0177f			>0.9999f
Number of Publications (baseline), N, Mean (STD)	44, 5.44 (6.04)	43, 2.74 (4.30)	0.0007w	23, 4.38 (5.21)	17, 3.83 (5.68)	0.2099w

c = Pearson’s chi-square test; f = Fisher’s exact test; w = Wilcoxon rank-rum test.

We used the NIH definition of URB: a person who indicates that they belong to at least one of the following groups: Hispanic/Latino, American Indian/Alaskan Native, Black/African American, or Native Hawaiian/Other Pacific Islander. This analysis only includes trainees that started the first non-CEED program in 2007 and beyond.

Degree was dichotomized into MD vs PHD, where MD includes MD, MD/PhD, MD/Masters and DO. PHD=all other.

Baseline faculty status (when they started the programs) was categorized into three groups: faculty (if positions are “Assistant Professor,” “Instructor (Clinical Educator),” or “Faculty”), non-faculty (if positions are “Clinical Doctorate Student,” “DrPH Student,” “Fellow/Postdoc,” “Medical Student,” “PhD Student,” “Resident/Housestaff,” or “Undergraduate”), and other.

Out of 43 CEED alumni and 61 non-CEED underrepresented minority alumni with covariate information, 24 pairs were matched based on propensity model that controls for gender, race, ethnicity, degree, and the baseline faculty status.

Only 44 CEED and 43 non-CEED URB trainees had publication data for analysis. Prior to propensity score matching, CEED trainees had a higher rate of peer-reviewed publications than their non-CEED counterparts (11.02 vs 6.20 publications per person-year; p=0.0005) after adjusting for mean number of publications at baseline. After propensity score matching, this difference was attenuated, yet still significant (10.0 vs 7.69 publications per person-year; p=0.0261). Because only 16 matched pairs had complete publication data, we performed a sensitivity analysis by imputing zero for all missing cases. In this scenario, the results were robust in that CEED trainees continued to have higher publication rates compared to their non-CEED counterparts (8.73 vs 4.69 publications per person-year; p<0.0001) (Table 3).

Table 3.

Comparison of number of peer-reviewed publications of The Career Education and Enhancement for Health Care Research Diversity (CEED) graduates vs University of Pittsburgh Institute for Clinical Research Education (ICRE) non-CEED graduates, 2007–2017

		After Propensity Matching
X		Complete Publication Data	Sensitivity Analysis
CEED	11.02 (N = 44)	10.00 (N = 16)	8.73 (N = 24)
Non-CEED	6.20 (N = 43)	7.69 (N = 16)	4.69 (N = 24)
p-Value	0.0005	0.0261	<0.0001

After adjusting for matched pairs, CEED graduates were more likely to hold an assistant professor position (54% vs 17%; p=0.0145) and trended toward being in the tenure stream (25% vs 4%; p=0.0971). We found no differences in Career Development Awards (p=0.6085) or Research Project Grants (p=0.4894) (Table 4).

Table 4.

Comparison of number and type of academic positions of The Career Education and Enhancement for Health Care Research Diversity (CEED) graduates vs ICRE non-CEED graduates, 2007–2017

	Before Propensity Matching			After Propensity Matching
Descriptive Characteristics	CEED	Non-CEED underrepresented background (URB)	p-Value‡	CEED	Non-CEED URB	p-Value‡
Academic Position, University of Pittsburgh, n (%)	18 (40)	38 (62)	0.0177c	9 (38)	18 (75)	0.0088c
Assistant Professor, n (%)	11 (24)	5 (8)	0.0230c	4 (17)	3 (13)	>0.9999f
Academic Position, Other
Institution, n (%)	21 (47)	13 (21)	0.0067c	12 (50)	4 (17)	0.0305f
Assistant Professor, n (%)	16 (36)	5 (8)	0.0006c	9 (38)	1 (4)	0.0102f
Assistant Professor (Overall), n (%)	27 (60)	10 (16)	<0.0001c	13 (54)	4 (17)	0.0145f
Tenure Track (Overall), n (%)	13 (29)	3 (5)	0.0017f	6 (25)	1 (4)	0.0971f
With Grants Awarded as PI or Multi-PI1, n (%)	18 (40)	6 (9)	0.0002c	9 (38)	4 (17)	0.1930f
Career Development Award, n (%)	7 (16)	2 (3)	0.0348f	3 (13)	1 (4)	0.6085f
Research Project Grant, n (%)	4 (9)	0 (0)	0.0300f	2 (8)	0 (0)	0.4894f
With Grants Awarded as co-I, n (%)	13 (29)	5 (8)	0.0050c	6 (25)	4 (17)	0.7238f
Research Project Grant, n (%)	6 (13)	2 (3)	0.0691f	2 (8)	2 (8)	>0.9999f

c = Pearson’s chi-square test; f = Fisher’s exact test; w = Wilcoxon rank-rum test.

PI = Principal Investigator; F/I = Foundation/Institute.

Discussion

CEED was designed to promote career success and help seal the “leaky pipeline” for URB researchers with a purposefully designed program of multi-level mentoring, targeted coursework, monthly seminar series, and networking opportunities. In this study, we aimed to quantify CEED’s effect on several key outcomes by comparing ICRE CEED Scholars to a matched set of ICRE URB trainees not enrolled in CEED using data collected from 2007 to 2017. Our findings indicate participating in CEED can promote career success, thus contributing to retaining URB researchers in the pipeline, and expanding and diversifying the biomedical research workforce. CEED Scholars were significantly more likely to have peer-reviewed publications and hold academic positions at the assistant professor level. There were also more CEED Scholars holding tenure track positions, although this finding did not reach statistical significance. There were also more CEED Scholars with career development awards or research project grants, but these results also did not reach statistical significance. Notably, this success is beginning to be recognized outside the ICRE. CEED leadership is working with other divisions and departments to introduce CEED into the interviewing process for fellowship and junior faculty positions as a potential component of their experience, which may reinforce to URB interviewees the University’s commitment to breaking down the barriers to biomedical research careers. CEED has also been recognized with the University of Pittsburgh Chancellor’s Affirmative Action Award.

The literature describes other programs designed to promote URB faculty development and recruitment in biomedical research,, , – and includes at least one program in which developers noted participation was tied to enhanced inclusion, although there was no specific intent to achieve this outcome. In those designed specifically for URB participants, most reported on the availability and characteristics of their programs, and evaluated their programs on the basis of the number of grant applications and manuscripts produced or satisfaction with program content. Of those we examined, none compared their program participants with a comparison group.

This study’s limitations include the fact that CEED participants self-select into the program (and acceptance is based on a competitive process as described above). However, participants in the ICRE programs also self-select and would self-identify as interested in research. Nevertheless, it is possible that those who elect to participate in a program such as CEED are different in important, unknown ways from those who do not. An additional limitation is the potential confounding effect of mentoring. While mentoring is a key programmatic component of CEED, it is also a part of the larger ICRE (to which CEED belongs). As a result, it is possible that scholars who benefited from CEED also benefited from effective ICRE mentorship. However, since mentoring was implemented across the numerous programs of the ICRE with a higher degree of variability (i.e. degree programs vs career development programs), it is possible that any effect due to the CEED program was underestimated since non-CEED ICRE trainees may have also received effective mentoring. Lastly, the retrospective design of the study allows for biases to impact the overall study results, which ultimately limits our ability to draw any causal conclusions. While propensity score matching helps to mitigate these biases by controlling for known confounding, it cannot prevent unknown confounding.

Our results allow CEED to quantify 10 years of successful advancement of the careers of URB investigators and faculty while expanding the retention of diverse faculty. Because CEED has been successful despite the challenges of a small demographic pool and constrained funding, we believe it is likely other institutions will be able to replicate our results by following our program’s model.