Moving diversity, equity, and inclusion from opinion to evidence.

Over the past decade, institutions have earmarked more resources to diversity, equity, and inclusion (DEI). However, many struggle to measure the effectiveness of their efforts and communicate them to the scientific community. We offer suggestions for applying rigorous scientific methodology to DEI and ways to promote diversity in scientific journals.

Main text

Embracing a wider range of diversity, equity, and inclusion metrics

Many clinical and academic institutions, especially academic medical centers, have (at least verbally) recognized the value of diversity, equity, and inclusion (DEI) to both innovation and excellence. DEI at most institutions is moving from being secondary to clinical, research, and educational missions to becoming acknowledged as core components of institutional success. This change has been a result of national standards set by governing bodies for accreditation (AAMC), research funding agencies (NIH, NSF, HHMI), and peri-academic societal pressures (the protests and civil unrest of 2020). Yet, these standards often just scratch the surface of DEI and mostly quantify actual diversity (difference) within an organization. Effectively measuring equity and inclusion has been just as important for institutional advancement, but less emphasized in programmatic efforts and evaluations.

For example, in medical and doctoral programs, there is a great deal of attention paid to the diversity of an entering class of students. Recruitment efforts are often made through pipeline programs or visits to colleges. Application, interview, and matriculation numbers of students from underrepresented groups are tracked, and medical schools tout the percentage of students recruited who are underrepresented in medicine along with the overall GPA and MCAT scores of the entering class. However, when assessing inclusion, the unfavorable results of diversity climate surveys, which can measure how well faculty and students feel like they belong to and are valued in an environment, are sometimes not shared with the survey participants and can leave institutions unsure of how to improve in those areas. Moreover, institutions shy away from the more difficult actions that could affect immediate change, such as removing a long-time departmental chair or addressing pay equity.

Defining DEI goals and objectives

The scientific community must begin to embrace a wider range of tools for measuring DEI in their organizations. This begins with defining DEI goals and objectives. Some goals and objectives can be made at the national level (Figure 1). These are often most effective at ushering change, but hardest to agree upon and implement. National affinity groups, scientific organizations, and funding agencies hold great power in this domain. Even academic publishers and national conferences can be gate keepers to advancing DEI at the national level. Other DEI goals and objectives can be defined at the institutional level. Institutions such as hospitals, companies, and schools should identify on an annual basis their objectives and specific aims for increasing diversity, ensuring equity, and promoting inclusion. These should be widely shared with the community and led by institutional leadership at the highest level. For example, many academic campuses welcome DEI initiatives, but don’t often make sufficient space for it: the easiest way being to make it of equal importance to other work. If DEI is truly valued, this should be reflected in the weight that it is given on faculty promotion committees. Consider the impact of losing all of the DEI service work and activities that are conducted by faculty and others to change institutions and service students. Could your institution thrive without it? If the answer is no that should be valued and rewarded accordingly.

Figure 1

Social ecological framework for defining DEI goals

Inspired by the Social-Ecological Model, this framework posits that DEI is not solely determined by individual factors, but instead is influenced by a collection of subsystems that occur at various levels. Therefore, institutions should aim to develop goals, objectives, and subsequent programmatic activities and metrics that address each of the subsystems for maximal effectiveness.

Some programs and policies may of course differ from department to department and are best tailored for a particular environment. There are even benefits of micro-environments, such as laboratories and offices, defining their own goals and objectives around DEI. We have seen excellent examples of this that help promote cultural change throughout an institution. Finally, individuals must also commit to personal goals and commitments such as serving on a DEI committee (roles not just reserved for historically marginalized and underrepresented groups), recognizing their privilege in certain spaces, and becoming better allies to other marginalized groups. Commitment to change starts with the individual.

Establishing metrics to track DEI efforts

Once goals and objectives are defined, institutions should seek to move from expert opinion and commentary on effective measures for advancing DEI to objective, validated, and evidence-based research and evaluation. After the spark of civil unrest and highlight of social injustice in 2020, many academic faculty would often propose a stronger investment into early middle or high school pipeline efforts that increase the number of individuals from underrepresented groups who enter medicine and science. While this is necessary, it is not sufficient and does not address the issues of racial inequity highlighted by the two pandemics (COVID-19 and racism). In fact, it suggests that some may be missing the point and not recognizing the racism, discrimination, and bias occurring within their own spaces. The events of 2020 created a momentum of awareness and desire to further advance DEI, and we must not let this wane. Our efforts moving forward must be rooted in both frameworks and evaluative metrics that have been shown (or will be tested) to address the actual problem. PRESS, for example, is an evidence-based framework for achieving racial equity. Without precise and rigorous standards to track the effectiveness of DEI efforts, institutions can regress to traditional, well-meaning approaches, limiting an organization’s return on investment.

When designing DEI objectives and their corresponding metrics, one must consider focusing on equity and inclusion in addition to diversity. For example, a residency program director may seek to build a diverse class of clinical residents by aiming to increase the percentage of residents from underrepresented groups who select that program and thereby ensuring a diverse representation of the entering class. Furthermore, one might consider periodically evaluating the experience of those residents through surveys and focus groups and establishing interventions and programs that help to achieve a more inclusive and equitable environment. Well-designed metrics will help those who are committed to DEI manage discrimination blind spots, which are mindsets and practices that encourage uniformity but are essentially concealed. Metrics that show an institutional return on investment can be used to engage stakeholders, bolster leadership commitment, guarantee additional resources, and advocate for further change. Additionally, metrics that are inclusive of staff (not just students, faculty, or leadership) will foster employee trust, satisfaction, and loyalty and reinforce an organization’s culture, brand, and reputation.

Scientific approaches to measuring inclusion

Institutions often use classic metrics, such as those previously discussed. However, these metrics might only consider diversity and neglect inclusion, which is typically harder to evaluate within institutions. Inclusion is often missed because it is not as easily measured as diversity. Inclusion typically needs to be measured through a mixed method approach that might require greater innovation. For example, microaggressions and macroaggressions occur on and off campus and are rarely measured. Microaggression experiences such as microinsults, microassaults, and microinvalidations can diminish the ideas, emotions, and identity of persons excluded because of their ethnicity or race (PEERs),, resulting in increased marginalization. Microaggressions at an institution create a barrier to inclusion. Although people may not always report them, tracking the rate of microaggressions among faculty, staff, and students is nevertheless important. The scientific community must embrace methodology and further develop new mechanisms and collaborations for tracking and understanding mistreatment such as microaggressions. Psychosocial instruments such as the Racial Microaggressions Scale, which is an anonymous survey that quantifies microaggressions by considering both their frequency and the distress they cause, can be helpful but should be tailored for the relevant populations and fields.

Microaggressions are also tightly associated with a sense of belonging, which can similarly be measured. Although not frequently evaluated, sense of belonging is crucial for PEERs as its absence can hinder student success., Prior studies found that women and PEER students report a lower sense of belonging than majority students, principally due to weak interpersonal relationships, perceived incompetence, and a lack of STEM identity., Therefore, quantifying sense of belonging, which includes (1) the sense of individual involvement in everyday practices and (2) the sense of being included in the general environment, is the first step in fixing underlying issues that cause a feeling of not fitting in. For the general environment, microaggressions and profiling surveys can help to understand general inclusivity and sense of belonging at a macro scale. For individuals, a scale that seeks to determine and quantify specific causes of a lack of sense of belonging is available. This can also be measured as a “pulse” by the Challenged Sense of Belonging Scale, a very short questionnaire that measures participation, self-identity, and connectedness. Importantly, predominantly white institutions, historically black colleges and universities, and liberal arts colleges can, as a group, differ in students’ rates of sense of belonging. This should be considered when making cross-institutional comparisons. A lack of sense of belonging ultimately may signify a lack of equity and inclusion for PEER students and faculty at a university, making it an important metric to consider.

It is also important to understand the level of stress at an institution and how it may be harmful to PEERs. One of the best ways to understand this is by measuring stress levels in the work environment. Stress can be especially high among PEER students and faculty, making metrics and surveys that anonymously ask about stress in various job facets important. For example, the Stress in General scale measures stress by considering job happiness, intentions to continue with the institution, blood-pressure, and job aptitude. Stress and subsequent burnout can occur when PEERs end up taking on multiple responsibilities and bear the burden that comes with being a minority in majority environments and systems. For PEERs, the long-term stress resulting both from work and discrimination can lead to racial battle fatigue or John Henryism, a psychological construct for coping with stress, with negative physiological effects. Therefore, beyond just measuring stress in the work environment, it is also important for institutions to measure general mental health incidents and their association with gender and race to understand overall stress levels of students and employees. These results can then be used to reduce stress and promote equity among faculty and students, which helps to avoid conditions such as John Henryism.

Finally, we believe institutions must look inward at their own attitudes to better understand the inclusion of their faculty and students. While many institutions measure diversity using hiring or admission rates, sometimes these metrics lack nuance and do not consider culture. Simply put, culture is the essence of an organization that determines how individuals behave and the organization runs. Measuring culture is difficult, but key aspects of culture and bias within an institution can be measured. For example, institutions can assess their interview processes for cultural humility and adaptability, i.e., their ability to rapidly learn and conform to organizational cultural norms. Cultural humility and adaptability lead to more promotions, more favorable performance evaluations, higher bonuses, and higher retention. Retention remains a pertinent issue, and it is important to consider why some institutions and fields do not retain their faculty or postdoctoral fellows. For example, this question could be better understood by regularly surveying faculty and trainees on their general satisfaction and intention to continue in various career paths., Lack of general satisfaction can explain why some STEM trainees do not stay in the pipeline.

Assessing the gender and race pay gap among employees can also measure institutional culture and help quantify institutional equity. These cultural aspects can be ascertained by assessing pay by demographics and ensuring that pay is equitable and fair. Beyond culture, we believe leaders should focus particular attention on promoting their institutional vision for inclusion and equity, which will create a wider feeling of workplace inclusivity. The Inclusive Leadership Scale, which measures aspects such as promotion of PEERs, empowerment, and fairness of a leader, is a useful measure of this effort. Together, these methods may be included in overall metrics used to measure institutions to ensure that equity, inclusivity, and diversity are met at every level.

Publishing DEI in scientific and medical journals

Many DEI papers published in science, health, and medical journals have focused on opinions, explaining a DEI issue, or telling a personal story. Although these do have a place, we hope to see a balance between them and DEI publications that utilize qualitative and quantitative methods to focus on solutions. We want to encourage more publishing of data from scientifically rigorous approaches to DEI efforts in the biomedical and STEM community. Hannah Valentine encouraged a rigorous scientific approach to DEI that is consistent with the ways in which we address science and human health. A degree in data science or social sciences is not required to investigate solutions to the DEI challenges that we all face. Collaboration is key to advancing in these areas. For example, collaborative studies from basic and social scientists to quantify and better understand the effects of mentorship are desperately needed in the biomedical fields. Even commentaries can be improved by using data to focus on the solution rather than the problem, which will empower individuals and serve as a call to action. In past articles, we have discussed the leaking pipeline of STEM trainees leaving the field and provided solutions such as early training and specific support programs that should be funded.

Finally, journals have a role to play as well as they are often the gatekeepers to scientific research sharing. Social media has become a powerhouse for better airing of scientific, mentoring, and DEI issues. For example, social media has amplified the discrepancy between rejection rates for authors from underrepresented groups versus well-represented groups. Furthermore, social media has also highlighted the lack of gender equality in citation rates, which remains an issue among some journals. Such journals could address this issue through allyship and increasing equity. We are happy to see some journals take steps to instill and measure DEI efforts in publishing. If journals do not commit to creating spaces for DEI publications, the change that we hope to see will stall.

Improving access to editors at journals

One last suggestion for diversifying the scientific space is to give individuals from underrepresented groups greater access to becoming reviewers and journal editors. It is common knowledge that many journal editorial boards consist of a majority of individuals who are from high-income countries, white, and/or male. Improving access for PEERs to journal editorial boards can bolster a larger focus on DEI. This can be implemented through pipeline programs. For example, we have observed recent programs such as the American Society for Cell Biology’s (ASCB) Early Career Editor, which allows post-docs and new faculty to begin learning the process of being a journal editor. While the full impact of these programs remains to be seen, based on the success of other STEM pipeline programs,, we believe pipeline programs that facilitate becoming academic editors will be a crucial way for young faculty and PEERs to get involved and have an impact at the journal level.

Conclusion

While institutions have been increasingly supportive of DEI efforts, they must continue to be informed by experience, values, and scientific evidence. Here, we define a framework for conceptualizing DEI goals and objectives and bring to light how pre-existing instruments can be used to measure the impact of DEI efforts on institutional climate. We also discussed steps that must be made for shifting DEI to utilizing more rigorous methodology, encouraging institutions to be more forthcoming in publicizing and reporting these results, and improving equity in both academic institutions and scientific journals. We hope that this will serve as a guide to increase representation in science, become more effective at institutional commitments to DEI, utilize quantification in DEI commentaries and research, and drive home the importance of inclusivity in science.