Institutional Interventions That Remove Barriers to Recruit and Retain Diverse Biomedical PhD Students.

The faculty and student populations in academia are not representative of the diversity in the U.S. POPULATION: Thus, research institutions and funding agencies invest significant funds and effort into recruitment and retention programs that focus on increasing the flow of historically underrepresented minorities (URMs) into the science, technology, engineering, and mathematics (STEM) pipeline. Here, we outline challenges, interventions, and assessments by the University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences (GSBS) that increased the diversity of the student body independently of grade point averages and Graduate Record Examination scores. Additionally, we show these efforts progressively decreased the attrition rates of URM students over time while eliminating attrition in the latest cohort. Further, the majority of URM students who graduate from the GSBS are likely to remain in the STEM pipeline beyond the postdoctoral training period. We also provide specific recommendations based on the data presented to identify and remove barriers that prevent entry, participation, and inclusion of the underrepresented and underserved in the STEM pipeline.

INTRODUCTION

The United States has experienced significant demographic shifts in recent years, wherein whites currently make up less than half the population of minors, and the general population is projected to be “majority minority” by 2044. Major contributors to this shift include changing immigration trends and higher birth rates within some ethnic populations. More than 50% of the total population growth between 2000 and 2010 has been attributed to an increase in the population identifying as Hispanic. Additionally, more than one-third of the U.S. population reported their race or ethnicity as belonging to a non-Hispanic minority group; a growth of close to 30% compared with the previous decade (U.S. Census Bureau, 2010). States in the southern and western regions have experienced more dramatic changes, while Texas, California, New Mexico, the District of Columbia, and Hawaii report majority minority populations (U.S. Census Bureau, 2010). These changes in the general population have had progressively diminishing impacts beyond K–12 levels of the education pipeline. This lag in representation is most pronounced in graduate/professional training such as in the sciences, where Blacks, Hispanics, and Natives (American Indians, Alaskan Natives, Native Hawaiians, and Pacific Islanders) combined comprise 30% of the population but only 13% of the science, technology, engineering, and mathematics (STEM) workforce (Antonio, 2002; Gibbs ; Heggeness ; National Science Foundation [NSF], 2017).

The lack of diversity of the STEM workforce is concerning for several reasons. First, STEM, through its impact on health and defense, represents a significant component of the U.S. economy for which there are major commitments of national resources in training at the graduate and professional levels. Questions regarding equity and access to heavily subsidized training resources and opportunities arise as certain populations remain underrepresented in these training pipelines. Second, this inequity in training limits the opportunities for those from underrepresented groups to participate in this highly skilled, impactful, and economically robust workforce. Third, the limits in participation in the scientific workforce, as demonstrated through the lack of diversity, especially at the levels of leadership, have consequences. These consequences include a lack of input in the decisions that are made as to the foci of our scientific efforts, especially in areas that disproportionately impact the populations that are underrepresented and often underserved (Poodry, 2003). Finally, there is a growing body of literature that argues that diverse teams and environments outperform advanced teams that are homogeneous (Watson ; Erhardt ; Bell ; Marcolino ). Thus, attaining diversity in STEM remains a major challenge for the United States (NSF, 2017).

The national effort to address the issue of diversity in the scientific workforce has focused on leveraging the higher interest in science seen in underrepresented students at the K–12 levels to enhance participation and persistence levels of students as undergraduates. Research-intensive institutions, with support from funding agencies, have targeted recruitment and retention as the major targets for increasing the numbers of graduate and professional degrees awarded to underrepresented populations in the sciences (women, ethnically diverse groups, individuals with disabilities, individuals from culturally diverse backgrounds) (NSF, 2014; National Institutes of Health [NIH], 2015; Valantine and Collins, 2015; Mervis, 2016). However, the state of representation at these institutions would indicate that these initiatives have failed in the broader sense to accomplish their objectives across underrepresented populations. While females are equally represented in the training population and junior professional levels for several disciplines in the biosciences (50% of the population vs. 51–58% of the workforce), racial and ethnic underrepresented minorities (URMs) remain disproportionately low within the scientific workforce (30% of the population vs. 13% of the workforce), especially as faculty at research institutions (30% of the population vs. 4% of the faculty at research institutions) (Gibbs , 2016; Heggeness ; NSF, 2017). Furthermore, under the present system to sustain research at highly competitive research institutions, funding rates for URM scientists are significantly lower than funding for the research of well-­represented scientists, and worse, there is evidence of discriminatory practices that disadvantage some URM populations (Ginther ). In this regard, Black scientists are less likely than their well-represented peers to secure an individual research grant (R01) from the NIH (Ginther ). The lower numbers of URMs represented in STEM and their disproportionally lower funding rates, along with their lower level of interest in pursuing faculty positions at academic research institutions (Gibbs ), presents a challenge to diversity in STEM and an opportunity for change.

While models provide explanations for when and where URMs divert from the STEM academic pathway, there is a disconnect between these models and analyses of interventions that are thought to reduce attrition in STEM careers (Koenig, 2009; Whittaker and Montgomery, 2012; Allen-Ramdial and Campbell, 2014). Additionally, data measuring outcomes of URMs in STEM education are phenomenological; they generally lack a clear description of what, if any, interventions or barriers have played a role in the outcomes reported. These observations highlight the need for deliberate and detailed assessments of the effects of such interventions at the graduate level to allow scaling of successful interventions aimed at addressing three major challenges in the biomedical sciences.

Admissions: Institutional awareness and commitment to mitigate the impact that the Graduate Record Examination (GRE) and other quantitative metrics have on the diversity of the scientific landscape. An admissions committee is charged with predicting and identifying the best prospective doctoral students based on an application that consists of GRE scores, transcripts, a personal statement, a research statement, and letters of recommendation. While the Educational Testing Service cautions against use of the GRE to set cutoff scores for the triage of applicants,, the GRE is a key factor in graduate admissions at many institutions. Further, in light of data that the GRE is not an indicator of success (Miller and Stassun, 2014; Wolf, 2014; Cahn, 2015; Hall ; Moneta-­Koehler ), predicts race and gender, is biased toward physical science majors, and selects against socioeconomically disadvantaged populations (Miller and Stassun, 2014), it is possible that many schools that rely heavily on the GRE in the decision-making process are indirectly selecting against racial, ethnic, and socioeconomic minorities. Thus, we hypothesize that changes to the admissions process and a holistic review of applicants that considers all attributes of each candidate would drastically change the demographics of students admitted to the graduate school.

Recruitment: Building partnerships and trust between the graduate school and target institutions through bilateral site visits, faculty and student workshops at minority-serving institutions (MSIs), and customized site visits of the graduate school based on the needs and research interests of prospective students. Institutions seeking to increase the diversity of their student bodies typically rely heavily on recruitment activities at major diversity conferences (i.e., Society for the Advancement of Chicanos and Native Americans in Science [SACNAS], Annual Biomedical Research Conference for Minority Students [ABRCMS]) and recruitment fairs at target institutions. However, it is possible that this one-dimensional approach to recruiting a diverse student body is limited, as these events are dependent on 1) the time allotted for each recruitment fair relative to the number of attendees and 2) the chance of a recruiter meeting qualified candidates in a sea of exhibitors, faculty, staff, students, and other recruiters. Thus, it is possible that limiting recruitment activities to recruitment fairs may limit an institution’s ability to reach its full diversity potential. We hypothesize that increased recruitment fair activity coupled with focused efforts on personalizing the experiences of prospective applicants, faculty, and administrative mentors through on-site visits, admissions workshops, and recruitment seminars would create a sustainable presence beyond the recruitment event. We hypothesize that these relationships will function to increase the pool of highly qualified diverse doctoral applicants through continuous and secondary recruitment efforts by students and mentors with whom the graduate school has established trust.

Retention: Implementation of social and academic programming that focuses on creating an inclusive environment such that all students thrive and persist in the biomedical sciences. Unfortunately, only around half of the students who enter STEM doctoral programs will actually graduate with a doctoral degree (Nettles and Millett, 2006). These high rates of attrition have resulted in implementation of many prevention and intervention programs focused on academic preparation before enrollment, while others have focused on enhancing the experiences of students on campus via social and peer-­support groups (Gandara and Maxwell-Jolly, 1999; Maton and Hrabowski, 2004; Hoffmann and McGuire, 2009). In this regard, we sought to merge successful elements from these programs and provide a hybrid of academic, peer, and social programming for students to create an environment of support that we hypothesized would improve the retention rates of URMs at the graduate school and overall persistence in the biomedical sciences beyond graduation.

To address these challenges, we have measured the effectiveness of specific interventions implemented in the recruitment, admissions, and retention processes at the University of Texas MD Anderson Cancer Center UTHealth Graduate School of Biomedical Sciences (GSBS) on long-term URM student outcomes over a 10-year period (Figure 1). The data presented show that these interventions are successful at reducing the attrition rates of URMs at the graduate school and, moreover, that these interventions have resulted in increased numbers of URM graduates who have remained in research and science-related careers. Thus, our results provide an approach for diversity recruitment and degree completion in the biomedical sciences. This strategy can also serve as a scalable and sustainable model to maximize efforts focused on increasing participation of URMs in the biomedical sciences by reducing attrition at key transition phases: predoctoral to doctoral, precandidacy to postcandidacy, and postdoctoral training to employment.

FIGURE 1.

Timeline of GSBS interventions.

METHODS

Data Sources

All work was conducted at the University of Texas MD Anderson Cancer Center UTHealth GSBS Dean’s Office. The data presented were extracted from the graduate school’s admissions, student, and alumni databases as described in the following sections.

Detailed information about the graduate school can be found in Supplemental Figure 1.

Admissions, Student, and Alumni Databases.

All database systems are managed at the graduate school, are customized, and use the Microsoft Office Access platform to capture, update, and report admissions, student, and alumni information. Each person in the database is assigned a unique identifier that facilitates the linkage of information from the admissions, student, and alumni databases to a single individual. Subidentifiers are assigned to each individual that distinguish between applicant, student, and alumni information, as basic information in each database is stored so that the data are consistent regardless of which database is being queried. However, some data are unique to specific databases. Thus, the following information describes information that is database specific.

Admissions Database.

The GSBS manages all applications via EMBARK Admissions Software, which allows for the collection of application forms, documents, scores, and recommendation letters. This software also allows the GSBS to create custom groups, manage admissions processes, and manage scores and ranks of candidates assigned to applicants by members of the admissions committee. Importantly, this software allows the Office of Admissions to download and transfer applicant information, test scores, grade point averages (GPAs), areas of research interests, previous education and degrees, interview information, curricula vitae, and visitation outcomes into the student database for long-term storage of information, material and postadmissions reviews and analyses.

Student Database.

Information in the admissions database for applicants who are admitted and matriculate into the graduate school is downloaded into the student database. Additional data collected in this database are: courses, grades, area of research, date of entry, program affiliation, tutorial mentor, probation status, research mentor, committee members, candidacy examination results, enrollment status, funding, awards, exit process, dissertation title, and degree received.

Alumni Database.

The information in the admissions and student databases are linked to the alumni database once a student graduates from the graduate school. Additional data collected are alumni contact information, employment history, education and degrees, awards, and activities. Maintenance of the alumni database through extensive data collection and editing of alumni employment information is conducted annually through a contracted third-party vendor.

Participants

Prospective Students.

Data collected from diversity recruitment events (n = 51) between 2012 and 2016 were used to query the admissions database to determine whether any prospective minority students from these events (n = 7691) applied to the doctoral program within a 4-year period and were subsequently admitted. Data collected from prospective students were the first name, last name, email address, and the name of the undergraduate or graduate institution.

Applicants.

Data were collected from the admissions database to determine the number of permanent residents or U.S. citizens who applied to the doctoral program, were offered admission, and accepted the offer of admission in 2009–2012 and 2013–2016; these data were exported to Excel for data analyses.

2009–2012: applicants (n = 958), offers of admission (n = 305), acceptances of admissions offer (n = 179)

2013–2016: applicants (n = 1041), offers of admission (n = 298), acceptances of admissions offer (n = 155).

Demographic information and metrics for applicants who accepted the offer of admission from the graduate school were collected, clustered, and analyzed for the 2009–2012 (n = 179) and 2013–2016 (n = 155) academic school years.

Students.

Data on the student body demographics, GPAs, GRE scores, and candidacy examination results were collected by querying the student database for the indicated information. Student records that were incomplete (missing data: GRE scores, GPAs) were eliminated from the study before data analyses. Only one record was omitted from the analyses of student metrics in the 2004–2005 time point because of nonstandard GPA scores. The student (white, female) is a graduate of a U.S. undergraduate institution at which letter grades are awarded on a 12.0 scale.

Total student body information includes data collected from the student database for all domestic students, permanent residents, and international students enrolled at the graduate school during academic years 2004–2005 (n = 452), 2008–2009 (n = 466), 2012–2013 (n = 441), and 2016–2017 (n = 336).

Domestic student body information includes data collected from the student database for students who were either U.S. citizens or permanent residents during academic years 2004–2005 (n = 248), 2008–2009 (n = 224), 2012–2013 (n = 199), and 2016–2017 (n = 182).

Candidacy examination results were collected and clustered for all URM students (Black, disabled, Hispanic, American Indian, Alaskan Native, Native Hawaiian, and Pacific Islander) before (2011–2013; n = 33) and after (2014–2016; n = 16) social and academic support was offered at the graduate school.

Attrition data were collected for the 2005–2013 academic years from the student database by year of matriculation into the PhD program for URM students (Black, disabled, Hispanic, American Indian, Alaskan Native, Native Hawaiian, and Pacific Islander) who left the program with a master’s degree or without a degree (n = 104) before (2005–2007) or after on-site interviews and social and academic support were offered to students (2008–2013).

Alumni.

Alumni employment data were extracted from the alumni database for analyses of career outcomes of 81 URM doctoral graduates. These data were exported to Excel and labeled as “Total” to provide a comprehensive view of career choices for up to 10 years postgraduation. To further understand outcomes or URM graduates beyond the postdoctoral fellowship training period, we split these data into two categories: 0–5 years postgraduation (postdoctoral training period, n = 38) and 5–10 years postgraduation (career entry period, n = 43).

Statistical Analyses.

Data extracted from the applicant, student, or alumni databases were exported and prepared for analysis using Microsoft Excel. Statistical analyses were conducted using PRISM software (analysis of variance [ANOVA]) and Excel (averages and standard deviations). Data on student body metrics were analyzed using one-way ANOVA with a post hoc Tukey test where a significant overall effect was observed. Significance was assigned to p values ≤ 0.05. The complete results of the analyses can be found in Supplemental Figure 2.

PROCEDURES

Intervention 1: On-Site Interviews of Applicants.

To improve the quality of students who are offered admission, the GSBS began conducting on-site interviews of top-tier doctoral applicants in 2008. Interviews of these students allowed the graduate school to 1) ensure that the candidate for admission was just as strong as the application for admission, 2) coordinate meetings of candidates with students and faculty of the graduate school, 3) allow candidates to discuss their research in a formal setting, 4) tour institutional facilities, ask questions, and have discussions about the graduate school and scientific community, and 5) customize each candidate’s visit to his or her research interests through interviews with faculty and staff and informal meetings with graduate program directors and administrators.

Intervention 2: Holistic Applicant Review.

In 2013, the admissions process was overhauled at the graduate school. Rolling admissions practices were halted, and admissions committee discussions were organized to shift the initial focus away from the quantitative metrics of each applicant to research experience and accomplishments, potential for a career in research, and quality of references. The addition of an optional statement of adversity was also added to the application to allow applicants to disclose obstacles or disadvantages they may have overcome for consideration by the admissions committee.

See Supplemental Figure 1 for admission committee description, detailed information on applicant review, and applicant scoring categories and criteria.

See Supplemental Table 1 for an example scoring sheet.

Intervention 3: Recruitment of Diverse Students

Recruitment fairs—GSBS representatives attended and participated in graduate school recruitment fairs at SACNAS, ABRCMS, the California Forum for Diversity (northern and southern California fairs), McNair Recruitment Fairs, the NIH Graduate and Professional Fair, the Society for Neuroscience, the Emerging Researchers National conference, and the American Association for Cancer Research conference. It should be noted that the GSBS representatives at these events (and recruitment seminars; see point 2) were biomedical scientists with doctoral degrees that have a thorough knowledge of the graduate school, faculty, and programs of study to reduce ambiguity and improve responses to questions from students about interdisciplinary research and opportunities.

Recruitment seminars—To extend support to and build pipelines with STEM students from MSIs, GSBS representatives host regular admissions and graduate application seminars in a classroom setting of senior STEM students interested in pursuing a graduate education in the biomedical sciences each year. The goal of these seminars/workshops is to improve graduate application quality and success rates for prospective diverse students. Target institutions were University of Puerto Rico System Schools (Mayaguez, Bayamon, Rio Piedras, and Cayey campuses), Dillard University, Texas Southern University, Morehouse College, Spelman College, Howard University, Trinity Washington University, Xavier University, Prairie View A&M University, University of Texas–El Paso, Florida International University, and Hampton University.

Application fee waivers—Many students who apply to the GSBS from underrepresented groups also declare that they are socioeconomically disadvantaged. Thus, another barrier to increasing the pool of diverse applicants is financial; it dissuades minority application submission. Beginning in 2013, the GSBS waived application fees to any students, regardless of race, ethnicity, or socioeconomic status, who sent an email request to the Office of Admissions.

GSBS Site Visits—Following recruitment seminars at target institutions, the GSBS hosts an annual 2-day on-site “reverse recruitment” event for STEM leaders and faculty at target MSIs. The foci of these visits are to offer workshops on writing informative and supportive letters of recommendations for competitive prospective graduate applicants; to provide tours of the facilities and graduate school; to supply information and answer questions about the graduate admissions process; and to build trust and relationships with said leaders, so they return to their respective institutions and promote the GSBS as a premier research institution for students who are interested in pursuing their graduate education in the biomedical sciences (i.e., reverse recruitment).

Intervention 4: Student Retention through Social and Academic Support.

Before 2014, there was very little social or academic support centered on creating a culture of equity and inclusion at the graduate school. Thus, the following programs were inaugurated to increase the retention rates of URM students.

Pairing social support groups to academic success—A unique feature of many social support groups at the GSBS is their primary focus on scholarship and professional development. The objectives of groups such as the Association of Biomedical Researchers (AMBR) are to foster a supportive environment and to promote and encourage the scholarship of minority students during their training period to ensure their retention in the GSBS. AMBR students organize and take part in a wide array of scholarly and social activities that enable participants to achieve the best possible outcomes during their graduate careers. AMBR is dedicated to providing the tools for professional development and success in graduate school and transition trainees into future careers. Some of the activities and workshops that the group has hosted include soft-skills workshops, candidacy examination workshops, research seminar and lab meeting support, and providing an audience for training grant representatives for training grants with a focus on increasing diversity.

Candidacy examination support—The GSBS candidacy examination gives students an opportunity to demonstrate: 1) their knowledge in a particular area/field; 2) their ability to think critically and analytically; 3) their capacity to synthesize and integrate knowledge; and 4) their potential to conduct independent research, based on the defense of a research proposal that they have written and presented to an examination committee. The prerequisites, deadlines, and examination procedure are well defined for each program at the GSBS, and many URM students tend to struggle during this process, which can contribute to attrition rates. Thus, the AMBR group provides mock candidacy examinations for trainees in the program to better prepare them for the process while improving outcomes.

a. Structure: 1) Mock candidacy examinations take place ∼4 weeks before the program-based candidacy examination of each trainee is scheduled to take place. 2) Students must submit their candidacy proposal to the mock examination committee, which consists of a panel of postdoctoral mentors to the group, 2 weeks before the mock candidacy examination is scheduled to take place. 3) The committee examines the student based on program-specific requirements of the oral and written candidacy examination. This meeting will be recorded, and the recording will be immediately available to the student for review and preparation. 4) Once the exam is complete, the student leaves the room while the mock examination committee openly discusses the outcome of the exam. 5) Upon completion of the discussion, the student returns to the room and is given an evaluation of the exam, including a score (e.g., unconditional pass, conditional pass, recommendation for a re-examination, and fail) and written comments/suggestions from each committee member on the presentation and written proposal.

b. Possible outcomes: 1) unconditional pass: the student passes without conditions/deficiencies; 2) conditional pass: the student passes, but with conditions as defined by the committee that should be resolved in a specified amount of time; 3) re-examination: the committee elects to re-examine the student before rendering a decision; and 4) fail: the committee has decided, in clear terms, that the student has failed the candidacy examination and does not have the potential to successfully complete the PhD program.

RESULTS

Changes in Doctoral Student Body Demographics over Time

To understand whether interventions may have an effect on diversity in the population of students, we initially examined snapshots of the demographics of the GSBS student body. We examined the entire student body demographics during the 2004–2005 (preintervention), 2008–2009, 2012–2013, and 2016–2017 academic years at the GSBS (Table 1A). The demographics of the total GSBS student population (domestic and international students combined; Table 1A), were compared with domestic students alone (Table 1B). While the number of Asian-American males and females have decreased over time, there were significant increases in the number of Native, disabled, and Hispanic males and Black, Native, disabled, and Hispanic females. Interestingly, these trends were consistent for both the total student body (Table 1A) and domestic student body (Table 1B), despite nearly half of the student body during these periods being composed of international students (Table 1A). Further, racial and ethnic information is not collected by the graduate school for enrolled international students, so the demographic makeup of the student body is better represented by limiting our analyses to the domestic student body for interventions linked to increases in the population of URM students at the GSBS. Thus, the following analyses in this study will be limited to the domestic student body.

TABLE 1.

Student body demographics: Percentage distribution of the (A) total and (B) domestic student body population enrolled at the MD Anderson UTHealth GSBS in 2004, 2008, 2012, and 2016 by race, gender, ethnicity, and disability statusa

aAI, American Indians; AN, Alaskan Natives; NH, Native Hawaiians; PI, Pacific Islanders. Populations of students who have increases or decreases in numbers over time are denoted in green and red, respectively.

The shift in the student body demographics has been the result of the commitment of the graduate school to systematically identify and remove barriers that that may affect the entry and success of URMs in the biomedical sciences. Over the past 10 years, the GSBS has made a significant investment in diversity recruitment, diversity retention, and student support programs. We have undertaken this analysis to measure the impact of these initiatives. The following questions were addressed: 1) What interventions resulted in the changes observed? 2) Are these changes associated with changes in GRE scores and GPAs? 3) What are the outcomes of students who have benefited from the outlined interventions? Given the culture of the institution to collect data in real-time on the outcomes measured, to catalogue and access data with ease (see Materials and Methods: Admissions, Student, and Alumni Databases), and to easily query and analyze data that have been collected, we are able to provide a model of successful interventions.

GRE Scores, GPAs, and GSBS Student Demographics Are Not Associated

The GSBS significantly overhauled its admissions processes in 2013 to “holistically” review all applicants. We measured whether GRE scores and GPAs were associated by first analyzing snapshots of the demographics of the entire GSBS student body in 4-year intervals to quickly assess changes in the student body before and after the changes to the admissions process were implemented. We analyzed the GPA and GRE (Quantitative [Q], Verbal [V] percentiles) averages of the student body in 2004–2005, 2008–2009, 2012–2013, and 2016–2017 by race and ethnicity (and gender; unpublished data) to determine 1) whether there were any statistical differences in GRE scores between groups, and 2) whether any changes impacted the demographics of the student body (Table 1B vs. Figure 2, A–C). The GPA averages of white males significantly changed over time, while other groups had no significant changes during the time periods examined. While there were increases in GRE scores over time in all groups (Figure 2, B and C), significant changes were intragroup changes in Hispanic females and white males and females (Figure 2C). Thus, there were no overall significant differences in student body GPA or GRE scores during the time periods examined (with the exceptions noted for Hispanic females and white males and females) pre- and postintervention within demographic groups (Table 1B). These findings provide support for a holistic applicant review in GSBS admissions and allow examination of the interventions that resulted in alterations in the student body demographics, independent of GRE and GPA scores.

FIGURE 2.

Domestic student body metrics. The average GPA (A) and GRE (B and C) scores of the domestic student body at the MD Anderson UTHealth GSBS in 2004, 2008, 2012, and 2016 by race, gender, and disability status. GPAs (A) and Quantitative (B) and Verbal (C) GRE scores for all domestic students are included next to their respective academic year. Groups with significant ANOVA values are highlighted by asterisks. ANOVA analysis, *p < 0.05. AI, American Indians; AN, Alaskan Natives; NH, Native Hawaiians; and PI, Pacific Islanders.

Changes in Recruitment Activities and Admissions Processes Increase Diversity Independent of Student Metrics

In popular social science theory, organizational, group, and social changes do not occur unless there is a shift from the focus on individual behaviors or items to a focus on changing the system (Sarayreh ). Changing the system occurs through creating awareness, making systematic changes, assessment, and management (Sarayreh ; Elrod and Kezar, 2015; Estrada ). Thus, to further understand whether the changes in the student body demographics (Table 1) are linked to increased recruitment activities (Figure 3A, left) and changes in admissions processes, we analyzed the number of applicants, offers of admission, and acceptances of the offers of admission before (2009–2012) and after (2013–2016) these changes. These efforts have been successful, as the percentage of competitive URM applicants increased from 2013 to 2016 (Table 2), with a corresponding steady increase in the percentage of entering URM PhD students who were direct recruits from these events (Figure 3A).

FIGURE 3.

Recruitment and admissions interventions. Interventions focused on improving the recruitment and admission of a diverse student body are measured by (A) the number of matriculating students who representatives of the graduate school met at a recruitment event and (B–D) entering student metrics. AI, American Indians; AN, Alaskan Natives; NH, Native Hawaiians; and PI, Pacific Islanders.

TABLE 2.

Domestic applicant demographics and metrics: Applicant and entering student body demographics before (2009–2012) and after (2013–2016) an overhaul of the graduate school’s admissions processesa

aAI, American Indians; AN, Alaskan Natives; NH, Native Hawaiians ; PI, Pacific Islanders. Populations of students who have increases or decreases in numbers over time are denoted in green and red, respectively.

Further, to understand the significance of the changes that we observed in admissions, we next examined whether there were differences in the numbers of students who had applied versus offered an interview versus accepted an offer of admission in preintervention versus postintervention groups. Because of the low numbers of students in some groups, we combined all URM students (Black, disabled, Hispanic, American Indian, Alaskan Native, Native Hawaiian, and Pacific Islander) and the majority students (white and Asian American) (Supplemental Figure 3). First, we examined whether significant differences existed between preintervention groups (majority students or URM students). We observed no significant difference in majority or URM populations regardless of gender (males, chi-square = 5.06, p = 0.080; females, chi-square = 1.48, p = 0.48), suggesting that there are no differences within majority or URM populations before interventions. Next, we examined whether differences existed between postintervention groups (majority students or URM students) and observed significant differences in both males (chi-square = 34.73, p = 0.00001) and females (chi-square = 6.25, p = 0.044), suggesting that after our interventions there were significant differences between majority and URM populations in both genders. To further understand the postintervention differences between groups, we used Fisher’s exact probability test to examine whether there were differences in URM versus majority groups with respect to the following groups: 1) applied versus offered an interview, 2) offered an interview versus accepted, and 3) applied versus accepted an offer (Supplemental Figure 3). We observed significant differences in males between applied versus offered an interview (p < 0.00001) and offered an interview versus accepted (p < 0.00001). No significant difference was observed in applied versus accepted (p = 0.21). These data suggest that interventions may play a role in the increases in male URM applicants who are offered an interview and ultimately accept the offer of admission. In females, significant differences were observed in the applied versus offered postintervention groups (p = 0.013), while the other postintervention comparisons were not significant (offered an interview versus accepted, p = 0.137) and applied versus accepted (p = 0.82), suggesting that our interventions may play a role in the increases in female URM applicants who are offered an interview. Next, we compared the numbers of majority and URM applicants before and after interventions. We observed significant differences in URM males (chi-square = 22.13, p < 0.000016) and no significant differences between URM females or majority applicants regardless of gender. We used Fisher’s exact probability test to examine whether there were differences in numbers of URM males who had applied versus were offered an interview versus accepted an offer of admission before and after interventions. We observed a significant increase in applicants who applied versus those offered an interview. These data suggest that interventions may play a role in the increases in male URM applicants offered an interview.

Importantly, the changes to the admissions process are associated in time with increased diversity of students who are offered admission, accept the offer of admission, and matriculate into the graduate school (Table 2) without significantly affecting GPA (Figure 3B), GRE-Q (Figure 3C), or GRE-V (Figure 3D) scores. This suggests that the holistic application review has had little impact on quantitative student metrics while enhancing student diversity. This also implies that moving the focus away from the metrics of each applicant to a holistic review and customized on-site visit increases the pool of highly competitive URM applicants who accept the offer of admission.

Personalized On-Site Interviews of Prospective Students Coupled with Academic and Social Support Programs Drastically Improves the Retention Rates of Underrepresented Minority Doctoral Students

Minority trainees have a lower level of interest in pursuing faculty positions at academic research institutions (Gibbs ), which creates a significant challenge to persistence and retention of minority students in research and research-related careers. The attrition of URM doctoral students who entered the GSBS before 2008 was around this average at ∼45% (Figure 4B, 2005–2007). Interestingly, students admitted to the graduate school in 2007 were not interviewed before the offer of admissions. Thus, based on our hypothesis that personalized on-site interviews before admission of prospective students would improve the quality of entering students and subsequently reduce their attrition rates, we analyzed the outcomes of students following the implementation of this initiative.

FIGURE 4.

URM outcomes. Interventions aimed at improving the progress and retention of URM students are measured by PhD candidacy examination results (A) and attrition rates of students over time who left the PhD program with an MS or no degree by year of matriculation (B). As a reference, the years in which implementation of specific interventions were implemented are labeled in red.

An analysis of the attrition rates by year of entry of URM students who entered the graduate school subsequent to this intervention revealed a significant impact on the attrition rates of URM students. Attrition rates of URMs following on-site interviews in 2008 showed a significant decrease (−50 to −75%) in the number of students who left the doctoral program with a master’s degree or before degree completion (Figure 4B, 2005–2007 vs. 2008–2013). Further, the attrition rates following changes in recruitment efforts, admissions processes, and support systems show an elimination of the attrition rate for students who entered the graduate school in 2013, providing further evidence of the need for such interventions (Figure 4B, 2013).

Further, to ensure that all students at the GSBS thrived and excelled in their graduate school experiences, we implemented social and academic programming to enhance the quality of education for all GSBS student in the form of candidacy examination support and peer mentoring and support organizations. We inaugurated a new student group, the Association of Minority Biomedical Researchers, or AMBR, in 2013. The goals for creating this group were to foster a supportive environment and promote student scholarship during the training period, reduce attrition rates, and improve doctoral candidacy examination outcomes. Because the candidacy examination is one area in which there is a leak in the STEM pipeline, we hypothesized that improving candidacy examination outcomes would significantly reduce attrition rates of URM students at the GSBS. Thus, the assistant dean of diversity worked with the AMBR group to establish a panel of postdoctoral fellows who function as a mock candidacy examination committee for members of the group as a method of candidacy examination preparation and to improve outcomes. Comparison of first-time candidacy examination outcomes before (2011–2013) and after (2014–2016) this intervention revealed a shift toward an unconditional pass outcome by first time URM students (Figure 4A), suggesting that the mock candidacy examination practice provided by the AMBR group helped to improve outcomes and repair a critical leak in the STEM pipeline: precandidacy to postcandidacy.

Repairs in the Pipeline at Critical Transition Phases Result in Increased Persistence of URM Doctoral Students in the Biomedical Sciences That Last beyond Graduation

According to national trends, URM doctoral students do not persist in academic careers beyond obtaining the doctoral degree, which can significantly impact the scientific landscape (Gibbs ; Feig ; Silva ; NSF, 2017). While many of the students who benefited from the interventions described in this study are current students at the graduate school, we wanted to understand the outcomes of URM graduates over the past 10 years to inform new programming. Thus, employment data were collected and analyzed for URMs who graduated from the GSBS within a 10-year period. These data were also analyzed by career stage (i.e., postdoctoral training period: 0–5 years postgraduation; nontraining period: 5–10 years postgraduation) to more accurately determine career outcomes (5–10 years postgraduation) and persistence in the biomedical sciences without consideration of postdoctoral training and fellowship periods. Surprisingly, we found that URM students who graduate from the GSBS are more likely to remain in the biomedical sciences (Figure 5) than what is reported in the literature (Gibbs ). Together, these data provide strong evidence supporting the implementation of such efforts across graduate institutions in the biomedical sciences.

FIGURE 5.

The impact of interventions on URM students in STEM careers. A measurement of “retention” in biomedical research and science-related fields based on career choice, postgraduation, 2007–2016 (n = 81).

CONCLUSIONS AND RECOMMENDATIONS

Nearly 20% of the GSBS student body is composed of URM students, resulting in one of the most diverse student bodies in graduate STEM education in the United States (“The Top Producers of Minority STEM Graduates,” 2013). While the GSBS is well on its way to reaching its goal of training diverse scientists who will provide the intellectual depth and breadth necessary to tackle STEM and healthcare needs of the nation, it is still far from the target—a student body that reflects the community that it will serve. According to the 2010 Census report, Houston is classified as having “majority minority” population (U.S. Census Bureau, 2010). Also, the report highlighted Hispanics as the largest group whose population made the most significant increase in numbers, yet these gains are not yet reflected in the student body demographics at the GSBS (Table 2). Thus, it is possible that additional interventions could improve recruitment strategies for specific communities.

However, our results do show that systematic changes and institutional efforts to enhance the scientific landscape are possible, and we provide a clear outline of interventions that can be used to broaden participation and enhance the experiences of URM students in the biomedical sciences. Thus, we propose the following data-driven recommendations to improve and maintain diversity in the biomedical sciences.

Recruitment. Participation in STEM recruitment fairs focused on historically underrepresented groups in STEM is the most obvious and helpful method to recruit high-quality applicants who have previous research experience and to increase the applicant pool of URMs. In this regard, >70% of the underrepresented students who matriculated in the 2017–2018 class met with recruiters before application submission (Figure 3A, right). However, in addition to participation in these events, we have found that removal of socioeconomic barriers can also improve the number of students who complete the application process. Many students from underrepresented groups who apply to the graduate school also declare that they are socioeconomically disadvantaged, so providing application fee waivers to all students who submit an email request to the Office of Admissions, regardless of race, ethnicity, or socioeconomic status or membership in STEM organizations, has increased the pool of diverse applicants. Finally, establishing and cultivating relationships with STEM faculty and administrators at MSIs and providing graduate application seminars and workshops to senior undergraduate STEM students and MSI faculty has also been successful as a recruitment tool. These approaches not only provide valuable information about the admissions process but, importantly, result in MSI faculty “buy-in” and encouragement for promising students to apply to the graduate school.

Admissions. The holistic review of prospective applicants is of paramount importance to diversity in STEM. Additionally, decreased numbers of applications assigned to admission committee members and a requirement for individual criteria scores prevent the use of GRE scores as a means to quickly manage the volume of applicants assigned to an individual faculty member. Thus, careful management of applicant review prevents the use of quantitative measures as a decisive factor in the admissions process and enables selection based on holistic review.

Inclusion and retention. Another factor linked to persistence in STEM is gender, racial, and ethnic stereotyping associated with the academic competence of women and URMs. Independent research studies consistently report that URM trainees have feelings of being perceived as incompetent and less capable than white peers, which negatively influences the perceptions of self-efficacy of trainees, impacts mentor–mentee relationships, and reduces the desire of trainees to pursue academic careers (Carlone and Johnson, 2007; Hurtado ; Griffin ). On the opposite end of the spectrum, “model minority” stereotypes of Asian trainees as high-achievers, although seemingly harmless, can also negatively affect the desire to pursue an academic career (Cheryan and Bodenhausen, 2000). Thus, support groups focused on academic and social support, such as AMBR, First Generation Student Group, and LGBT Student Alliance Groups at the GSBS, are essential to creating an environment of inclusion and support for students who are considered visible (based on racial and ethnic statuses) and/or invisible (based on gender identity, first-generation status, and sexual orientation) minorities. The long-term impact of academic and social support fostered in these groups for student persistence in the biomedical sciences is well documented and essential to increasing diversity in STEM (Figure 4B; 2013) (Gloria and Kurpius, 2001; Nicpon ; Rayle and Chung, 2016).

Faculty demographics. Faculty mentors are an important component in this pathway, as they provide support, guidance, and information to trainees about the field. Specifically, faculty mentors are essential for the persistence of women and URM trainees in the biomedical sciences, because these students generally have fewer role models and mentors in the field (Nelson and Rogers, 2010; Towns, 2010; Gibbs ). Thus, if the goal of any institution is to work toward diversity management, faculty hiring practices will also have to be reviewed, assessed, and overhauled so that faculty demographics mirror the student body these institutions have worked so hard to diversify.

We recognize that implementing such programs not only requires an appreciation of diversity and inclusion by an institution, but a significant financial investment to implement the outlined interventions. The list of recommendations outlined above are far from exhaustive, but have proven useful as interventions that can help graduate institutions address issues related to diversity in STEM education in the presence and absence of financial support. Specifically, institutions that are interested in increasing diversity in a fiscally conservative manner can begin by creating programs aimed at promoting an inclusive environment coupled with academic support (e.g., academic student support groups). While students usually lead these programs, incorporating peer mentorship and student–postdoctoral mentorship programs provides foundational support that can be used as a strategy to improve retention and outcomes with some administrative support from the graduate school. Second, changes to the admissions process can greatly improve the diversity of an institution. Careful selection of admissions committee members who are diverse in their own graduate education experiences, able to invest their time to carefully review applicants, and knowledgeable about unconscious bias has the potential to impact the review of applicants and demographics of students who are offered admission.

Other factors that should be considered as we continue to measure the impact of our programs on the diversity of the graduate school are the impact of local, state, and federal policies on the demographics of the student body. It is possible that changes in how student tuition and fees and immigration status are considered will affect the outcomes of our assessment in a manner that is independent of our interventions. Thus, we challenge the government, academia, and educational administrative leadership to work toward shifting the balance from one of tradition to that of inclusivity, as movement toward diversity management is the only way to achieve true equality and inclusion.

LIMITATIONS OF THIS STUDY

There are limitations to the data and analysis presented in this study. Factors we did not measure may have contributed to the effects observed, and study designs using repeated sampling are generally limited, because of the inability to control for what would happen in the absence of the outlined interventions. For example, enhanced federal funding meant to increase diversity in STEM (NSF, 2014; NIH, 2015; Valantine and Collins, 2015; Mervis, 2016; Hall ) before graduate school application may have enhanced URM applicant competitiveness over time. Additionally, while we focused on “URM” students in our study, they were not a homogeneous group of students: students in these groups may be members of other categories that were not analyzed (e.g., gender identity, sexual orientation, first-generation status, and changes in citizenship status), and we were unable to examine for whom the interventions work best and individual psychological variables that may have affected outcomes. There were also limitations of statistical power for testing some of the differences observed because of limited numbers of students in certain groups. Finally, this study was limited to one highly contextualized institution.

Click here for additional data file.