Diversity and inclusion for the All of Us research program: A scoping review.

The All of Us Research Program (All of Us) is a national effort to accelerate health research by exploring the relationship between lifestyle, environment, and genetics. It is set to become one of the largest research efforts in U.S. history, aiming to build a national resource of data from at least one million participants. All of Us aims to address the need for more diversity in research and set the stage for that diversity to be leveraged in precision medicine research to come. This paper describes how the program assessed demographic characteristics of participants who have enrolled in other U.S. biomedical research cohorts to better understand which groups are traditionally represented or underrepresented in biomedical research. We 1) reviewed the enrollment characteristics of national cohort studies like All of Us, and 2) surveyed the literature, focusing on key diversity categories essential to the program's enrollment aims. Based on these efforts, All of Us emphasizes enrollment of racial and ethnic minorities, and has formally designated the following additional groups as historically underrepresented: individuals-with inadequate access to medical care; under the age of 18 or over 65; with an annual household income at or below 200% of the federal poverty level; who have a cognitive or physical disability; have less than a high school education or equivalent; are intersex; identify as a sexual or gender minority; or live in rural or non-metropolitan areas. Research accounting for wider demographic variability is critical. Only by ensuring diversity and by addressing the very barriers that limit it, can we position All of Us to better understand and tackle health disparities.

Introduction

The All of Us Research Program (All of Us), funded by the National Institutes of Health (NIH), is a national effort to accelerate health research by exploring the relationship between lifestyle, environment, and genetics [1]. The program is set to become one of the largest health care research efforts in U.S. history, aiming to build a longitudinal resource of multiple data types and biosamples from at least one million participants.

A main goal of the NIH Precision Medicine Initiative (PMI) Working Group [2] was that All of Us “should broadly reflect the diversity of the U.S.” As stated in the 21st Century Cures Act, All of Us will “ensure inclusion of a broad range of participants, including consideration of biological, social, and other determinants of health that contribute to health disparities.” [3] The recognition that not all groups have benefited equally from research is a major step towards health equity. A key contribution that All of Us can make is setting the stage for more diverse participation in precision medicine research to come. We describe the efforts made to date to guarantee that All of Us reflects the broad diversity of the U.S. and enables precision medicine research for all.

To ensure that All of Us is rigorous in its efforts to enroll a diverse group of one million participants, we assessed the demographic characteristics of other U.S. biomedical research cohorts. To better understand which groups are represented or underrepresented in biomedical research, we 1) conducted a broad survey of the literature focused on ten demographic categories essential to the program’s diversity aims, and 2) reviewed the enrollment characteristics of national cohort studies similar to All of Us. By identifying these populations, we can understand the barriers and gaps in enrollment of key underrepresented groups. Based on previous evaluations of diversity and inclusion by race and ethnicity, non-White groups have already been established as underrepresented. [4-6] It is important to note here a distinction between minority health and underrepresentation of other groups, especially with regards to health disparities research and biomedical research representation. Minority health refers to “health characteristics and attributes of racial and/or ethnic minority groups (defined by Office of Management and Budget), who are socially disadvantaged due in part by being subject to potential discriminatory acts.” [7] Historically, women have been underrepresented as well, though their proportion now exceeds 50% in NIH-funded research. [8] In addition to racial and ethnic diversity in biomedical research, this work proposes that researchers examine other demographic and social determinants when considering inclusion of diverse participant populations. This review established a landscape of demographic characteristics and social determinants meant to guide All of Us towards measuring diversity among its participants.

A Guide for diversity inclusion

Defining underrepresentation for All of Us and stakeholder input

All of Us has relied on designated definitions of diversity guided by the leading authorities on health disparities. In addition, we have considered the inclusion of groups that may be underrepresented in biomedical research, specifically, through consultation with a variety of resources and stakeholders, as follows. The National Institute on Minority Health and Health Disparities (NIMHD), congressionally mandated to designate disparity groups, [9] has designated the following groups as most affected by health disparities in the U.S.: Blacks/African Americans, Hispanics/Latinos, American Indians/Alaska Natives, Asian Americans, Native Hawaiians and other Pacific Islanders, socioeconomically disadvantaged populations, underserved rural populations, and sexual and gender minorities. We also used the guidelines for including women and minorities in clinical research set forth by the NIH [10], as well as a review of the U.S. Census data (Fig 1) [11]. We sought input from NIH leadership and associated stakeholders, including the NIMHD; the Office of Research on Women’s Health; the Sexual and Gender Minority Research Office; the National Human Genome Research Institute; the National Institute of Child Health and Human Development; the National Institute on Aging; the Office of Extramural Research; and the All of Us Research Program. Lastly, input was obtained from the All of Us Participant Provided Information Committee (PPIC), an All of Us governance body who developed the program’s baseline demographic survey that uses existing measures from other national surveys. [1] A diverse team of PPIC experts, including All of Us participant representatives, and All of Us participant engagement teams vetted these measures to ensure appropriate terminology and phrasing and ensured consistency in terminology for guiding the diversity categories developed here, some of which are being implemented as of this writing. We sought to examine if these major groups have historically been underrepresented in the biomedical research literature, including clinical trial studies, and whether these categories would help measure the impact of the All of Us Research Program for these groups.

Fig 1

(A-B). Projected Age, Race, & Ethnicity, and Sex Composition of the United States through 2040.

Based on these major inputs, All of Us prioritized nine essential diversity categories to complement racial and ethnic minority inclusion. An important point to note is that, aside from racial/ethnic minorities and sex assigned at birth, the other categories have not been broken out demographically in most other studies. Therefore, it is not explicitly known whether they are truly underrepresented in biomedical research. Explicitly defining and evaluating all these demographic characteristics will be important for understanding their impact on social determinants of health. These include (listed alphabetically): access to care, age, annual household income, disability, educational attainment, gender identity, geography (i.e. rurality), sex assigned at birth, and sexual orientation. We anticipate that eventually 75% of the All of Us cohort will be from historically underrepresented groups in biomedical research and expect nearly half of this diversity to be from racial or ethnic minorities. [1] Of the nine categories, six are included in the NIMHD definitions of “disparity populations,” three are related to socioeconomic status, two to sexual and gender minorities, and one to rurality. For each category, we verified a designation of groups as either historically represented or underrepresented through a review of the literature.

Source selection

We established a team with pertinent expertise to plan and execute the review. The team included representatives from the NIH and participating All of Us institutions with expertise in library and information science, enrollment monitoring and reporting, underserved populations, health disparities research, and strategic planning for large-scale national consortia.

Searches of English language articles were conducted using PubMed and Google Scholar along with a mixture of Medical Subject Headings (MeSH) terms and keywords relevant to the nine diversity categories (S1 Table for search strategies and updated retrieval counts). Excluded from the results were duplicate articles, those not relevant to human subjects research, and those that did not contain data to either support or refute justification for considering designation of representation. Meta-analyses and systematic review articles published since 2012, specific to biomedical cohort studies conducted in the United States, and emphasizing results or conclusions specific to the categories of interest were prioritized for review by a subgroup of the authors. We also used handsearching and consulted the subject matter experts on our team to identify additional potentially relevant white papers and unpublished reports. Examples of ways evidence about representation in research was assessed included: evaluation of data from existing U.S. biomedical research studies and trials, evidence depicting how study eligibility criteria contributed to underrepresentation of these groups, and evidence outlining barriers to participation unique to these groups. Results were refined to include a small set of distinctive articles that emphasized findings specific to underrepresentation of the key groups (i.e., why they were considered underrepresented, whether the groups were included in study recruitment, data analysis, or conclusions). The literature was reviewed over the course of several months in 2017 through 2018 (S2 Table for final counts of screened and selected articles, S2 File for selected bibliography).

Complementary assessment of core diversity characteristics for similar cohort studies

Additionally, a selection of other national cohort studies was evaluated for demographic characteristics benchmarking a diversity baseline for All of Us. We included cohort studies with an enrollment of >75,000 and a study design or scope like All of Us. Those selected were U.S. based except for one international study, the U.K. Biobank. [12] Information was collected using publicly available sources including websites, datasets, and publications. When demographic information was not readily searchable, we made attempts to gather the information from study authors.

Of the national cohort studies evaluated, we identified fifteen as most relevant to informing All of Us based on the aims of the research and mixture of data types collected from participants. Five (33%) were still enrolling participants at the time of our review, and seven (47%) had begun enrollment in 2006 or later. We confirmed that thirteen (87%) collected questionnaire data, collected biospecimens, and linked to external data sources. Eight (53%) collected electronic health records. We also prioritized review of diversity categories including age, educational attainment, household income, U.S. geographic region, disability, race and ethnicity, sex assigned at birth, and sexual and gender minority status. See Fig 2 for a brief demographic snapshot of the cohorts we evaluated.

Fig 2

Core demographic characteristics of similar cohorts evaluated to support development of the All of Us guide for diversity and inclusion.

Results

Roughly 645 articles were screened throughout the literature review and 170 were selected as significant for informing the All of Us Guide for Diversity and Inclusion (S2 Table for exact counts by search category). Based on the body of literature and consultation with stakeholders, All of Us emphasizes inclusion of racial and ethnic minorities and has prioritized the following additional groups as being possibly underrepresented in biomedical research: individuals with inadequate access to medical care; <18 or >65 years; with an annual household income at or below 200% of the federal poverty level (FPL); who have a cognitive or physical disability; or have less than a high school education or equivalent. Also included are those individuals who: are intersex; identify as a sexual or gender minority; or live in rural or non-metropolitan areas. Including the racial and ethnic groups, Table 1 shows historically represented and underrepresented characteristics of each category and how they are measured by the survey responses. [13] Researchers interested in studying specific subsets of these groups (for example, individuals < 3 or >80), are able to do so based on the way in which these demographic data have been collected.

Table 1

The All of Us Guide for diversity and inclusion.

Diversity Category	Represented in Biomedical Research	Underrepresented in Biomedical Research
Race and Ethnicitya	Individuals who identify as White and non-Hispanic	Individuals who identify as other than White and non-Hispanic (i.e. Asian; Black, African or African American; Hispanic, Spanish, or Latino; Native Hawaiian or Pacific Islander; Middle Eastern or North African)
Access to Care	Individuals who have had a needed medical visit in the past 12 months or can obtain and pay for medical care as needed	Individuals who have not had a needed medical visit in the past 12 months or cannot easily obtain or pay for medical care as needed
Ageb	Adults ages 18–64	Children 17 or younger and adults 65 or older
Annual Household Income	Individuals with household incomes above 200% of the Federal Poverty Level	Individuals with household incomes equal to or below 200% of the Federal Poverty Level
Disability	Individuals without a physical or cognitive disability	Individuals with either a physical or cognitive disability
Educational Attainment	Individuals with a high school degree or equivalent	Individuals with less than a high school degree or equivalent
Gender Identitya	Individuals who identify as either a man or a woman	Individuals who identify as gender variant, non-binary, transgender, or something else
Geography	Individuals who reside in urban metropolitan areas	Individuals who reside in rural and non-metropolitan areas
Sex Assigned at Birtha	Male or female individuals	Individuals who are neither male nor female (i.e. intersex)
Sexual Orientation	Individuals who identify as straight	Individuals who identify as asexual, bisexual, gay or lesbian or something else

a Racial and ethnic minorities and women: The NIH Revitalization Act of 1993 set forth policy that mandates the inclusion of women and minorities in NIH-funded clinical research unless their exclusion is justifiable and approved by the relevant NIH institute. The resulting policy guided the enrollment criteria developed by All of Us. For the purpose of this work, however, women are not considered UBR. Representation of women in NIH-funded research has steadily risen to over 50%. [4]

b Adults aged 65 or older: NIH Policy and Guidelines on the Inclusion of Individuals Across the Lifespan as Participants in Research Involving Human Subjects defines older adults as individuals 65 years of age or older and underrepresentation of this entire group has been confirmed by the literature. Adults 75 or older, however, may be especially underrepresented; there is additional federal guidance on ensuring their inclusion. [14]

These designations are supported by assessing the other national cohort studies. Historically underrepresented groups, where we were able to obtain data, were underrepresented across all fifteen studies evaluated. Annual household income was unknown, not collected, or not reported for thirteen (87%). Of those reporting income, one (7%) is U.S. based and only a small portion of the participants had an annual household income under $25,000. [15] Educational attainment data were obtained for seven (47%) cohort studies. Percentage of participants with less than a high school degree or GED ranged from 1% to 15% for those U.S. based studies. [15-18] We confirmed inclusion of older adults in only five (33%) studies and their representation ranged from <17% to 37%. [16,19-22] Participants under 18 were enrolled in two (13%) studies. [22,23] Only one (7%) study reported on representation of participants with a disability; these participants made up 18% of that total sample. [17]

Fourteen (93%) studies reported demographic representation of participants by sex. Males were not eligible for four of the studies. Ten (67%) were eligible to both males and females and reported on sex. Of those ten, females were overrepresented in five (50%). [12,18,21,22,24] None of the fifteen studies reported intersex. Five (33%) reported participants’ geographic region. Those mostly recruited from only one U.S. geographic region though two (13%) reported geographic dispersity stretching across at least four regions. [15,25] Data to specifically assess rurality were not readily available. Only two (13%) studies reported overrepresentation of racial and ethnic minorities compared to White participants. [23,24]

Lastly, most did not draw an intentional distinction between sex assigned at birth and gender. Data to permit evaluation of gender minority representation, however, were not collected from participants or not publicly reported for any of the fifteen studies. This was the case for sexual orientation data as well. See Fig 2 for aggregate findings from the cohort assessment and Table 1 for All of Us Guide for Diversity and Inclusion.

Discussion

Most of the articles sought to illustrate major health care and research barriers impacting racial and ethnic populations, along with other underrepresented groups. The literature demonstrated that racial/ethnic minorities and other groups are typically impacted by at least one of several barriers including financial (e.g., insurance coverage, income), cognitive (e.g., health literacy, technical proficiency), language (e.g., lack of translation resources), communication and cultural factors (e.g., cultural humility, lack of trust), structural (e.g., transportation, geographic accessibility), racial and ethnic discrimination, or study design (e.g., eligibility criteria limiting participation or inclusion, failure to reach diverse groups due to recruitment limitations). [7,26-34] These barriers have significant effects on multiple groups. Access to research opportunities is critically impacted by structural barriers like transportation, but also by language, communication, and unnecessary restrictions in study design. For example, willingness to participate in research does not significantly differ between individuals who reside in rural areas and those who live in urban locations. Rather, the limiting factor is accessibility impacted by either structural barriers or recruitment limitations. [31,34,35] Older adults are also impacted by these barriers. The literature demonstrated that older adults have been marginalized in research due to a number of reasons including not meeting eligibility criteria (e.g., clinical trial restrictions on comorbid conditions due to risk of side effects), perceived burden on study staff (e.g., longer screening process or more “high touch” retention tactics needed), and cognitive or physical impairments that affect the ability to consent or fully participate. [36-38] Similar barriers are faced by racial or ethnic minorities. [39-41] Cultural barriers, specifically lack of trust, and not being trustworthy, are defining barriers across racial or ethnic minorities, particularly Black and African American individuals, [34,42] and also across many other groups including rural populations and sexual or gender minorities. Specifically in cancer-related clinical trials, studies have shown no racial or ethnic differences in participation rates [43], but this does not seem to be generally applicable to other types of clinical research settings. [44] Importantly, overcoming a lack of trust is through engagement and carefully designed and strategies [45] by which participants are invited to participate, in a respectful way, in research programs.

Representation of underrepresented groups is significantly affected by the lack of consistent demographic data collection and gaps in research domains specific to the health disparities that impact them. For example, unlike sex assigned at birth, which is a biological construct, gender is a construct that includes many dimensions. Domain experts recommend researchers include measures to capture both an individual’s assigned sex at birth and gender identity to fully represent one’s physical and internal sense of self. [46] The literature confirmed that gender identity as a social determinant of health is not only understudied but also unrecognized. [47,48] This body of literature is comprised of articles on evaluation of instruments that gather this information (e.g., questionnaires, electronic medical records, etc.), their limited use within healthcare and research [49,50] and the lack of knowledge and sensitivity of medical personnel about different gender identities. [51] Similarly, sexual orientation measures are not often collected from patients and research participants, especially noted for older adults. [52,53] These omissions were observed in our own review of national cohorts. The literature also noted that biomedical research has generally neglected exploration of sexual orientation, apart from research focused on understanding sexually transmitted disease. This implies a substantial gap in research that could lead to improvements in broader health outcomes for sexual and gender minorities. [54,55] The lack of data collection measures and demographic reporting does not allow us to confirm whether individuals that identify as sexual or gender minorities are truly underrepresented. Data collection is a mandatory first step, however, in addressing the obvious gaps in research results that directly impact health disparities faced by this population. For this reason, sexual and gender minorities are included as an underrepresented group.

We identified 12 articles that examined sex as a biological factor in biomedical research. Several of the articles call for more comparison of research findings by sex, noting that, like race and ethnicity, sex differences in many domains are still widely understudied. [56-60] Regardless of actual representation of females in biomedical research, many health outcomes specific to them will remain unknown unless researchers prioritize exploring differences across sex assigned at birth, race, and ethnicity. In addition, sex is predominantly perceived as a binary biological construct and individuals born as intersex, or those with disorders of sexual development, have difficulties “fitting” into a male/female binary category. Including intersex as a biological construct is important for health care since individuals’ needs may not apply to either the male or female classification. [59] The literature indicates that this is also true for transgender individuals who have undergone gender affirming surgeries or who receive hormonal treatments as well. [61]

We identified literature that explained underrepresentation of select minority populations due to issues in how race and ethnicity data are traditionally collected and reported. [62,63] For example, there has been confusion among survey respondents over the categorization of Hispanic or Latino as an ethnicity rather than a race. [63] A lack of sufficient categories for race and ethnicity data, often in combination with underrepresentation within the research samples, means results are often reported in aggregated categories that can mask underlying disparities. For example, Middle Eastern and North African populations have often been classified as White in research questionnaires that do not specifically call out a category for Middle Eastern and North African, similarly Asian American, Native Hawaiian, and other Pacific Islanders have historically been grouped as Asian or Other. [62,63]

Limitations of this approach

The broad scope and depth of this review was limited since complex strategies were needed to canvass the literature, returning exceptionally large retrieval sizes. This work is meant as a survey of trends, relying mainly on reviews obtained from the literature to best inform the state of the science on groups historically underrepresented in biomedical research. Our evaluation of other large national cohort studies was also limited because we conducted most of this work using publicly available sources. In some instances, we were unable to confirm the accuracy or currency of the information. In addition, All of Us is a hypothesis and disease-agnostic research program with minimal inclusion/exclusion criteria. Most of the studies we identified in our evaluation differ from All of Us in both areas. For example, clinical trials-enrolled participants with specific medical conditions or those that may benefit from a therapeutic treatment. They may, by their very nature, exclude certain demographic groups. Finally, many studies do not collect comprehensive socio-demographic data, therefore, we do not know whether their populations include groups we assert to be underrepresented.

Commentary

There is a rich diversity within the U.S. that must be leveraged for understanding factors contributing to health and disease for the entire population. Historically, biomedical research has neglected this diversity to our detriment. A recent report [64] showed that, of the drugs removed from the U.S. market between 1997 and 2000, 80% were removed due to side effects or fatalities that occurred in women. Another report [65] found that racial factors, although important, do not solely account for the disparities in health care. Societal and environmental factors are equally important to account for context. [66] A widely-used antiplatelet drug, clopidogrel, needs to be converted to an active form to be effective. [67] However, due to a genetic variation that affects the body’s ability to metabolize it, only about 50% of individuals of Asian ancestry are able to benefit, along with other individuals who carry the variant. This is one example of a modern scientific advancement, in this case a life-saving medication, offering a high benefit for some individuals within populations, but significant risks for others. [68]

An early research opportunity for All of Us will be examining health disparities among these racial and ethnic populations intersected across other demographic groups and their impact on social determinants of health. For a summary of significant conclusions, see Table 2. The All of Us Research Program Guide for Diversity and Inclusion (Table 1) is an essential product for All of Us, serving as the program foundation for 1. future demographic enrollment monitoring, 2. gauging the success of new recruitment strategies as they are implemented and providing opportunity to revise when key enrollment gaps are identified, and 3. gauging the variability and strength of the resulting research data repository, and ensuring that it will support health disparity research for all historically underrepresented groups as it grows over time. This guide is a novel attempt to emphasize and define diversity and inclusion for All of Us based on a series of biologically and socially significant constructs. There are considerable nuances that impact diversity and inclusion monitoring and we respect that they should not be treated independently of one another or considered universally equal. For example, there is an enormous body of literature confirming the importance of racial and ethnic diversity in biomedical research. The All of Us guide emphasizes minority representation in biomedical research, and the additional demographic characteristics and social determinants highlighted are meant to complement, not dilute, that inclusion.

Table 2

Key points learned in developing the All of Us guide for diversity and inclusion.

• The rich, cultural, geographic, and demographic diversity of the U.S. is underrepresented in U.S. biomedical research.

• Significant barriers impact representation of key populations. The biomedical research community must acknowledge and address them to ensure inclusivity.

• A lack of data collection for many key demographic characteristics and social determinants is a critical gap across the entirety of biomedical research. For example, educational attainment and many other socio-economic status factors have a robust relationship with overall mortality. Yet, these measures are almost completely absent from the research landscape. All of Us, and the biomedical research community at large, must address this.

• Sexual and gender minority measures, except for research exploring sexually transmitted disease, have mostly been ignored. Although a lack of measures does not necessarily suggested underrepresentation, disparities impacting these groups are understudied. To understand representation of these groups, All of Us and the biomedical research community must start asking about sexual orientation and gender identity.

• A guide for inclusion is necessary to ensure equity in research outcomes. The guide, however, must be viewed as a starting point, not a final product in understanding and addressing barriers that impact participation, can we position All of Us to better understand and tackle health disparities for all.

Conclusion

Research that accounts for wider demographic variability is critical. It is time that we move beyond relying on research generated from largely homogeneous populations of usually White, male, urban, and higher socio-economic status. [69] Only by incorporating more diverse factors as key variables, ensuring inclusion, and identifying and addressing barriers that limit research participation, can we position All of Us to better understand and adequately tackle health disparities.

Research is used to provide the basis by which we discover new pathways and identify new treatment approaches. As such, only by narrowing the gaps of health disparities and increasing our understanding of how social determinants of health impact health outcomes can we realize a future of research that is more precise. We note that discrimination is a standing issue in workforce diversity, [70] which impacts minority participation in research studies like ours. Based on the work presented, and in conjunction with community partners, All of Us has developed resources for engaging diverse communities (https://www.joinallofus.org/en/community/community-resources). These provide a springboard for researchers to more readily incorporate diversity and inclusion strategies to local settings [45,71] or disease specific contexts, or to use the https://www.researchallofus.org/ platform to build diverse cohorts. Within All of Us, the attention paid to these important factors can help surface the intersectionality of these barriers to research participation and help the research community to be better equipped to engage diverse populations.

The approach taken by All of Us was broad, methodical, and intentional in conceptualizing an inclusive guide. Despite the limitations, it encompasses a depth of inclusion that is unprecedented in other national cohorts. The emphasis on historically underrepresented minorities and other populations provides a reference for any future study that aims to ensure diversity and equity. The All of Us Research Program is well on its way to creating a national resource that reflects and supports the broad diversity of the U.S. necessary for advancing precision medicine for all.

Search strategies by diversity category and January 2020 retrieval counts.

(DOCX)

Click here for additional data file.

Number of articles screened and selected in original literature search.

(DOCX)

Click here for additional data file.

Bibliography of selected references by search category.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

3 Apr 2020

PONE-D-19-34035

Diversity and Inclusion for the All of Us Research Program: A Scoping Review

PLOS ONE

Dear Mrs. Mapes,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

As you can see the reviewer comments are minor, given that your paper was well-written and informative.

We would appreciate receiving your revised manuscript by May 15 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Dr Emma Louise Giles

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.plosone.org/attachments/PLOSOne_formatting_sample_main_body.pdf and http://www.plosone.org/attachments/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. One of the noted authors is a group or consortium"the All of Us Research Program". In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Manuscript number: PONE-D-19-34035

Title: Diversity and Inclusion for the All of Us Research Program: A Scoping Review

Overall evaluation: In this well-written, well-organized, and concise paper, the authors present the results of a scoping review investigating the demographic characteristics of participants enrolled in US biomedical research cohorts in order to identify what groups are underrepresented in biomedical research. The results are unsurprising but present a snapshot of the lack of diversity across multiple sociodemographic characteristics within biomedical research. The authors present a set of guidelines for specific groups that researchers should make exceptional efforts to represent in biomedical research.

This is an important contribution to the literature in that it provides a moment in time in which researchers can see how we have failed to serve many underrepresented groups in research, and a call for rectifying this as we build future cohorts. While I understand that the focus of these guidelines are on population-based research, I appreciate that the authors also include some information about the lack of representation of women and racial and ethnic minority groups in clinical trials research, which has a tremendous impact on the safety of drugs and procedures for diverse groups in the US.

I have only one strong recommendation for the authors. There is a brief mention of the role of lack of trust in the low participation rate of Black and African American participants in biomedical research. This is concerning, as there is a growing body of evidence (e.g., https://doi.org/10.1002/cncr.28483) that when there is equal access to participate in and individuals are explicitly ASKED to participate in clinical trials, African Americans are no less likely to agree to participate nor to be willing to participate in clinical trials research. I fear that not including this in the current manuscript may irresponsibly perpetuate the lack-of-trust excuse that researchers often use to justify a lack of diversity in their research. We often characterize groups as “hard-to-reach” for any number of reasons, when in reality what we are saying is that they are just “easy to ignore.” I believe that the authors of this manuscript are working to change this norm in biomedical research, so I think that it is important to recognize how citing trust plays a role. Similarly, I think it is important to include any evidence we have for whether when other underrepresented groups – those from rural areas, those with disabilities, etc. -- are offered equal access to clinical trials they will accept, but this may be an important point to include in this paper to support the authors’ efforts to shift norms towards including UBR groups in research.

Other major comments:

1. Can you provide an explanation for how “Individuals who have had a needed medical visit in the past 12 months” is representative of access to care? Does this mean that they went to a medical facility, therefore they have access to care? It was unclear to me how this represents a lack of access to care.

2. In Figure 1, it is very difficult to distinguish the lines in the charts (and therefore the reported proportions of race and ethnicity) because of the colors. Also, why does the projected population of Native Hawaiian and Pacific Islanders reach 0%? Are they eliminating that category from the census?

3. Are there any actual study participants included in the Participant Provided Information Committee, particularly those representing any of the groups in the proposed guidelines? If so, this could certainly be a strength to include in the manuscript.

Minor comments:

1. All of the figures and tables are difficult to read, even as the TIFF files.

2. Line 188 need space between “and” and “170”

3. Line 209, 220, 224, possibly other instances: Data are plural, so data “were” not “was”

Reviewer #2: This article is very well written.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Katherine S. Eddens

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

15 May 2020

Title: Diversity and Inclusion for the All of Us Research Program: A Scoping Review

Manuscript number: PONE-D-19-34035

Reviewer #1: I have only one strong recommendation for the authors. There is a brief mention of the role of lack of trust in the low participation rate of Black and African American participants in biomedical research. This is concerning, as there is a growing body of evidence (e.g., https://doi.org/10.1002/cncr.28483) that when there is equal access to participate in and individuals are explicitly ASKED to participate in clinical trials, African Americans are no less likely to agree to participate nor to be willing to participate in clinical trials research. I fear that not including this in the current manuscript may irresponsibly perpetuate the lack-of-trust excuse that researchers often use to justify a lack of diversity in their research. We often characterize groups as “hard-to-reach” for any number of reasons, when in reality what we are saying is that they are just “easy to ignore.” I believe that the authors of this manuscript are working to change this norm in biomedical research, so I think that it is important to recognize how citing trust plays a role. Similarly, I think it is important to include any evidence we have for whether when other underrepresented groups – those from rural areas, those with disabilities, etc. -- are offered equal access to clinical trials they will accept, but this may be an important point to include in this paper to support the authors’ efforts to shift norms towards including UBR groups in research.

Response: Thank you for raising this point as it is important to highlight. We acknowledge as well that there is evidence supporting the argument that black and African American individuals are not less likely to participate in clinical trial research, particularly cancer trials, when explicitly asked. Clinical trials, however, are associated with better care and more potential for therapeutic benefit than other types of research activities. We believe there are two issues with trust to note – there is a lack of trust (participant) and not being trustworthy (researchers/institutions). Inviting individuals to participate, in a respectful way, may aid in overcoming the lack of trust barrier. This may not mean, however, that the barrier does not exist. We have addressed this point and clarified in the manuscript that the reason to cite low trust is so that researchers become more trustworthy (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6143205/) and that racial differences in trust are not just historically important to note (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6802409/).

Reviewer #1: Can you provide an explanation for how “Individuals who have had a needed medical visit in the past 12 months” is representative of access to care? Does this mean that they went to a medical facility, therefore they have access to care? It was unclear to me how this represents a lack of access to care.

Response: Access to care can be measured according to several major elements including timeliness of receiving care when a need arises. This element is often measured according to one’s ability to receive care when needed over the course of a 12 month timespan.* This means the individual had the resources required to receive care when needed (ex. Transportation, money, locality to a medical care clinic, etc.) The All of Us Research Program recommends that both individuals who have not received a needed medical visit in the past 12 months or cannot easily obtain or pay for care as needed should be considered underrepresented in biomedical research.

*1. Elements of Access to Health Care | Agency for Health Research and Quality [Internet]. [cited 2020 Apr 19]. Available from: https://www.ahrq.gov/research/findings/nhqrdr/chartbooks/access/elements.html

Reviewer #1: In Figure 1, it is very difficult to distinguish the lines in the charts (and therefore the reported proportions of race and ethnicity) because of the colors. Also, why does the projected population of Native Hawaiian and Pacific Islanders reach 0%? Are they eliminating that category from the census?

Response: Figure 1 has been revised so that the lines are wider and easier to distinguish. Please note that the project population of Native Hawaiian and Pacific Islanders are as follows: 0.2% (2016), 0.2% (2020), 0.3% (2030), 0.3% (2040). The figure originally submitted listed the 2020 population as 4% in error.

Reviewer #1: Are there any actual study participants included in the Participant Provided Information Committee, particularly those representing any of the groups in the proposed guidelines? If so, this could certainly be a strength to include in the manuscript.

Response: Yes, more than 30 participant representatives are active members of the All of Us governance committees. The Participant Provided Information Committee roster includes two participant partners. This is now noted in the manuscript.

Reviewer #1: Minor comments:

1. All the figures and tables are difficult to read, even as the TIFF files.

2. Line 188 need space between “and” and “170”

3. Line 209, 220, 224, possibly other instances: Data are plural, so data “were” not “was”

Response: Thank you for this feedback.

1. The figures and tables have been revised to improve readability. Figure 3 has been repurposed as a table (Table 3).

2. This grammatical error has been corrected.

3. These corrections have also been made.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

8 Jun 2020

Diversity and inclusion for the All of Us Research Program: a scoping review

PONE-D-19-34035R1

Dear Dr. Mapes,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Emma Louise Giles

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Thank you for revising your manuscript and considering all of the reviewer comments. The final minor comment I wish to make is that there are formatting errors on Page 15, which may need to be corrected. Thank you for such an interesting manuscript.

Reviewers' comments:

18 Jun 2020

PONE-D-19-34035R1

Diversity and inclusion for the All of Us Research Program: a scoping review

Dear Dr. Mapes:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Emma Louise Giles

Academic Editor

PLOS ONE