A systematic review to identify research priority setting in Black and minority ethnic health and evaluate their processes.

BACKGROUND: Black, Asian and minority ethnic communities suffer from disproportionately poorer health than the general population. This issue has been recently exemplified by the large numbers of infection rates and deaths caused by covid-19 in BAME populations. Future research has the potential to improve health outcomes for these groups. High quality research priority setting is crucial to effectively consider the needs of the most vulnerable groups of the population.
OBJECTIVE: The purpose of this systematic review is to identify existing research priority studies conducted for BAME health and to determine the extent to which they followed good practice principles for research priority setting.
METHOD: Included studies were identified by searching Medline, Cinnahl, PsychINFO, Psychology and Behavioral Sciences Collection, as well as searches in grey literature. Search terms included "research priority setting", "research prioritisation", "research agenda", "Black and minority ethnic", "ethnic group". Studies were included if they identified or elicited research priorities for BAME health and if they outlined a process of conducting a research prioritisation exercise. A checklist of Nine Common Themes of Good Practice in research priority setting was used as a methodological framework to evaluate the research priority processes of each study.
RESULTS: Out of 1514 citations initially obtained, 17 studies were included in the final synthesis. Topic areas for their research prioritisation exercise included suicide prevention, knee surgery, mental health, preterm birth, and child obesity. Public and patient involvement was included in eleven studies. Methods of research prioritisation included workshops, Delphi techniques, surveys, focus groups and interviews. The quality of empirical evidence was diverse. None of the exercises followed all good practice principles as outlined in the checklist. Areas that were lacking in particular were: the lack of a comprehensive approach to guide the process; limited use of criteria to guide discussion around priorities; unequal or no representation from ethnic minorities, and poor evaluation of their own processes.
CONCLUSIONS: Research priority setting practices were found to mostly not follow good practice guidelines which aim to ensure rigour in priority setting activities and support the inclusion of BAME communities in establishing the research agenda. Research is unlikely to deliver useful findings that can support relevant research and positive change for BAME communities unless they fulfil areas of good practice such as inclusivity of key stakeholders' input, planning for implementation of identified priorities, criteria for deciding on priorities, and evaluation of their processes in research priority setting.

Introduction

Current evidence demonstrates disproportionately poorer health outcomes for Black and Minority Ethnic (BAME) groups. In particular, the prevalence of Type 2 diabetes is reported to be as much as six times higher in UK South Asians compared to Europeans [1] and disparities in mental health care for BAME groups represent a serious public health concern [2] with a significantly disproportionate number of people from BAME backgrounds detained under UK mental health legislation in hospitals in England and Wales [3].

The extent and seriousness of disparities in health has been further demonstrated in the recent global pandemic of severe acute respiratory syndrome caused by covid-19 and which has disproportionately affected vulnerable and marginalised populations such as BAME groups [4] being up to twice as likely to die from covid-19 in the UK than people of White British ethnicity [5]. However, it is worth noting that this is a new pandemic and inconsistent and emerging findings continue to be reported. This pandemic has exposed the severe extent of existing socioeconomic health and structural inequalities, ranging from poverty to barriers to accessing care, and crowded living conditions [6] among these groups that have been exacerbated by covid-19.

Research priority setting has the potential to reduce disparities in health by making research more efficient in solving health problems. Involving the local population addresses the issue of equity and attends to the needs of the most vulnerable groups within the population, while reinforcing the links between research, action and policy [7]. There is no consensus on the definition of research priority setting but there is agreement on a range of activities that centre on identifying, prioritising and reaching agreement on the research areas or questions deemed important to stakeholders [8]. Historically, researchers and funders have generally set healthcare agendas themselves [9]. More recently, it has been recognised that research needs to address questions that are relevant to the people it intends to make a difference for, give them a voice [10] and mitigate waste [8]. Key stakeholders include healthcare professionals, policy makers, patients, and their families, as well as the public more generally. These questions should be answered using the most appropriate methods, and research results need to be reported in a manner that is comprehensive, transparent, and accessible [11].

The past decade has seen an increase in research priority setting exercises in a range of areas [12] and there are increasing efforts to identify shared research priorities using explicit processes [13]. In their narrative review of health research priority setting methods, models and frameworks, Bryant et al. (2014) found that among eleven different priority setting exercises identified from the United Kingdom, Australia, the United States and Canada, none had been evaluated to check their prioritisation processes or assess the extent to which the exercise had achieved its objectives [14]. It is also unclear whether research prioritisation exercises have been undertaken for BAME health. This systematic review is interested in identifying whether there has been any progress in research priority process evaluation since then, with a specific focus on BAME health, given that BAME communities suffer worse health outcomes than the wider population [15], the stark increase in research priority setting in the past decade [12], along with an increase of discourse around evaluation of research priority setting initiatives [13].

Therefore, this study aimed to identify and evaluate existing research priority setting studies conducted for BAME health. Applying a critical lens to their processes may inform ways to improve future research prioritisation for BAME populations and increase the value and contribution of research aimed to improve the health of BAME communities.

Are there health research priority setting studies conducted for BAME health?

Have they adhered to good practice principles in health research priority setting?

Methods

The systematic review followed the standards of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [16] (S1 File). The search was undertaken between July 6th-7th, 2020, in four electronic health databases: CINAHL, MEDLINE, PsychINFO and PBSC. The following Boolean search term combinations were used:

“research priority setting” [all fields] OR “research prioritization” [all fields] OR “research prioritisation” [all fields] OR “research priorities” [all fields] OR “research agenda” [all fields]

AND

“Black and minority ethnic” [all fields] OR “BAME” [all fields] OR “ethnic group” [all fields] OR “ethnic groups” [all fields] OR “minority groups” [all fields] OR “multicultural” [all fields] OR “Asians” [all fields] OR “immigrants” [all fields] OR “indigenous” [all fields] OR “Aborigines” [all fields]

To ensure the full scope of published literature within each database was targeted, we searched databases from their inception to July 2020. Titles and abstracts published in English only were included due to time limitations. Abstracts were screened for relevance. Given its limited timeline, the principal researcher (HI) independently conducted the article search. Studies were included in the full text screening that used a qualitative, quantitative and mixed method design. Searches in the grey literature included: reference lists of included articles, Google Scholar, Cochrane methods priority setting, and the James Lind Alliance (a well-established priority setting partnership method). The search string ‘research priority setting and Black and minority ethnic health’ was applied to Google Scholar. The first ten pages of Google were examined for eligible articles. All authors contributed and refined the review’s search strategy. Two authors (HI and MC) applied the critical appraisal criteria.

Inclusion and exclusion criteria

The review was developed to include any study that outlined a process of conducting a research prioritisation exercise. Studies must outline the characteristics of the participants, discuss the methods used to obtain research and identified well-established outcomes. Studies that made no mention of health research or did not describe the research prioritisation process were excluded. Studies were included if they focused on obtaining research priorities specifically for BAME populations. This includes studies that sought to identify research priorities for ethnically mixed populations and involved them in the process, provided that differences in priorities from BAME groups and the wider population were described. See Table 1 for the inclusion and exclusion criteria.

Table 1

Inclusion and exclusion criteria.

Included	Excluded
Studies that directly elicited and identified research priorities (e.g., topics or questions) for BAME health	Studies assessing priorities for practice and policy (quality indicators)
Studies must outline a process of research priority setting, including participants characteristics, study type and an outcome	Non-research articles (policy documents, clinical guidelines, commentaries, editorials)
Studies that sought to identify research priorities in White and BAME populations and involved the public in the exercise must display the differences in identified priorities between both groups	Studies that involved White and BAME groups in identifying their health priorities yet did not discuss disparities between priorities
UK and international studies	Study protocols
Studies written in the English language only	Conference reports, workshop or meeting that failed to include information about the participants and methods
	Interventions to improve BAME health
	Priority setting exercises that were non-health research priority focused

To ensure credibility of the process, all authors discussed and agreed the selected papers. References were managed with EndNote X9 for ease. After removing duplicates, HI independently screened the title and abstract of 1,080 records including three found in grey literature. In total, 32 studies were selected for full text examination. The PRISMA flowchart, including reasoning for study exclusions, are displayed in Fig 1.

Fig 1

PRISMA flow diagram.

Quality appraisal tool

Each of the identified studies were assessed using a quality appraisal tool specifically designed for health research priority setting. In the absence of a gold standard approach to research prioritisation, a checklist of nine common themes for good practice in health research priority setting was [17] used to determine whether the research priority setting exercises from the studies adhered to good practice principles in their processes as reported by the checklist. This checklist has been used previously to evaluate research priority setting exercises [13, 18–20], and has effectively identified weaknesses prevalent in research prioritisation exercises. As this tool was specifically designed with health research prioritisation in mind, it could identify issues that may otherwise have been overlooked by traditional quality appraisal tools.

The checklist is organised into three domains which were used to critically appraise the studies: preparatory work, deciding on priorities, and after priorities have been set. Each domain contains corresponding practices that further identify the goals in each step. Within preparatory work, there are five related practices: context, use of a comprehensive approach, inclusiveness, information gathering, and planning for implementation; within deciding on priorities: criteria, and methods for deciding on priorities; and within after priorities have been set: evaluation and transparency. See Table 2 for a detailed description of each theme.

Table 2

Checklist for health research priority setting [17].

Theme	Description
Preparatory work
1—Context	1 The resources available for the exercise were reported2 The focus of the exercise was clearly stated (what it was about and who it was for)3 The underlying values or principles were clear4 The health environment in which the process took place was described5 The research environment in which the process took place was described6 The political environment in which the process took place was described7 The economic/financial environment in which the process took place was described
2—Use of a comprehensive approach	8 The process of priority setting was described in detail
3—Inclusiveness	9 The participants involved in setting research priorities were described10 An appropriate representation of expertise was included11 An appropriate representation of sex was included12 An appropriate representation of regional participation was included13 Relevant health sectors and other constituencies were included
4—Information gathering	14 The information and sources used to inform the priority setting exercise were referenced
5—Planning for implementation	15 Plans for translation of research priorities were discussed16 Who will implement the research priorities and how?
Deciding on priorities
6—Criteria	17 Relevant criteria to focus discussion on setting priorities were stated
7—Methods for deciding on priorities	18 Approach for deciding on priorities was described (e.g., consensus or metrics based)
After priorities have been set
8—Evaluation	19 When and how evaluation of the established priorities and the priority setting process will take place were defined (e.g., multiple sessions)
9—Transparency	20 Clarity about the approach used, i.e., how priorities were set

Data collection process

We conducted a descriptive synthesis to summarize the study characteristics and outcomes, as well as how well each study matched up against the good practice principles as reported by the checklist. A quality score was assigned to each study and was based on the number of 20 good practice criteria it had met in Table 2. One researcher (HI) independently extracted study characteristics, methods, and outcomes. Comprehensive data extraction checklist forms for the quality synthesis were developed to extract the relevant data. Two researchers (HI and MC) extracted the data to quality appraise the studies.

Results

We identified 17 studies meeting our inclusion criteria. These studies were conducted in research priority setting for BAME health in a range of different topic areas. Three studies were conducted for mixed ethnic groups [21-23]. All 17 studies can be seen in Table 3. The topic areas were child obesity [21, 24], mental health [25], suicide [26, 27], cancer [28], E-health [29], knee replacement surgery [30], pre-term birth [31], healthy school development [32], and in health more generally [22–24, 33–36]. The prioritisation exercises were conducted to determine health research priorities in these topic areas for a range of different ethnicities in high income countries. They concerned Latino health in the US [24, 37], refugee and immigrant health in the US [25, 26, 32], the health of indigenous Australians [28, 29, 36], the health of native Americans [27], the health of Asian youth in New Zealand [34, 35], South Asian health in the UK [30, 33], the health of minority and underrepresented communities in the US [22, 23], and Black and Hispanic health in the US [21, 31]. Six studies did not include patient and public involvement (PPI) as participants in establishing research priorities [21, 24, 25, 27, 32, 35], Five studies had PPI involvement alongside other stakeholders such as healthcare professionals, academics, researchers, and decision makers [26, 28, 30, 33, 37]. Four studies involved only BAME PPI as participants [29, 31, 34, 36] and 2 studies involved multi-ethnic PPI in identifying priorities and highlighted the differences in identified priorities by ethnicity [22, 23]. The most common methods used to identify priorities were surveys, focus groups, interviews, Delphi techniques, and workshops.

Table 3

Study characteristics for the included empirical studies with quality score.

Study ID	Country	Title	Topic and scope	Population included in the Identification of priorities	Method	Main outcome (research Priorities)	Quality score (Based on met criteria in the checklist–see Table 2)
Flores et al., 2002 [24]	USA	The health of Latino children: urgent priorities, unanswered questions and a research agenda	General health	Paediatricians, health researchers, academic dentist, anthropologists, academic nurse, environmental health expert, dean of a school of public healthNo public involvement(total n = 13)	Workshop	Research agenda included:(1) greater inclusion of Latino children in medical research (2) analysis of study data by pertinent Latino subgroups (3) more research on Latino child health issues that can elucidate social and economic determinants of health and use of health services for all children, such as cross-border health and the healthy immigrant effect (4) enhancing early educational opportunities for Latino children (5) training healthcare professionals more extensively in cultural competency	6/20 (30%)
Colucci et al., 2010 [25]	Australia	Setting research priorities in refugee mental health	Refugee mental health	Academics, key practitioners, and policy makersNo public involvement(total n = 71)	2 online surveys	Key research priorities included (1) the design and delivery and location of mental health services for refugee clients (2) how existing services can be adapted and extended for refugee clients (3) the prevalence of mental health problems in refugee clients (4) factors promoting resilience and successful transition to life in the new country of settlement	7/20 (35%)
Colucci et al., 2017 [26]	Australia	A suicide research agenda for people from immigrant and refugee backgrounds	Suicide in immigrant and refugee populations	Policy makers, service providers, academics, service users, carer advocates(total n = 138)	Online Delphi with two rounds of questionnaire.	Greatest priority was given to:(1) access and engagement with suicide prevention services (2) suicide protective and risk factors compared to populations not from immigrant and refugee backgrounds (3) culturally appropriate assessment of suicide risk	9/20 (45%)
Goold et al., 2017 [22]	USA	Priorities for patient-centered outcomes research: the views of minority and underserved communities	General health in minority and underserved communities	Academic and community partners (n = unknown)Members from minority and medically underserved communitiesWhite (63)Black or African American (98)Other (22)(total n = 183)	Interviews	Greatest priority was given to: (1) quality of life, (2) patient-doctor, (3) access, (4) special needs (5) compare approaches.	12/20 (60%)
					Focus groups	Black participants were less likely to prioritize research on causes of disease, new approaches, and compare approaches than White participants.
Manikam et al., 2017 [33]	England	Using a co-production prioritization exercise involving South Asian children, young people and their families to identify health priorities requiring further research and public awareness	co-production of child health research and public awareness agendas	Heath care practitioners from a range of backgrounds (n = 27)South Asian children and families (n = 35)(total n = 62)	Systematic literature reviewScoping surveyFocus groups	Health care practitioners prioritized public awareness on obesity, mental health, healthcare access, vitamin D and routine health checks and research on nutrition, diabetes, health education and parenting methods.South Asians prioritized research into the effectiveness of alternativeMedicines. Both healthcare practitioners and South Asians prioritized increased research or public awareness on mental health illness, blood and organ donation, obesity, and diet.	12/20 (60%)
McNeely at al., 2017 [32]	USA	How schools can promote healthy development for newly arrived immigrant and refugee adolescents: research priorities	identification of research priorities for promoting the school success of immigrant and refugee youth	Researchers, service providers, educators, policymakersNo public involvement(n = 132)	Modified CHRNI framework	Highest priority research options were:(1) evaluating newcomer programs identifying how family and community stressors affect newly arrived immigrant and refugee adolescents’ functioning in school (2) identifying teachers’ major stressors in working with this population (3) identifying how to engage immigrant and refugee families in their children’s education	13/20 (65%)
Morris., 2017 [28]	Australia	Identifying research priorities to improve cancer control for indigenous Australians	identify emerging research priorities in Indigenous cancer control.	Researchers, public health practitioners, advocacy groups, allied health and other related professionals, Indigenous cancer survivors and their families, and Indigenous community groups.(total n = 225)	Online survey	Identified research priorities included: (1) cancer prevention and early detection (2) health literacy (3) culturally appropriate care for Indigenous patients, survivors, and families.	10/20 (50%)
Franck et al., 2018 [31]	USA	A novel method for involving women of color at high risk for preterm birth in research priority setting	A research agenda for pre-term birth in women of colour	BAME women at high risk of preterm birth(total n = 12)	Novel RPAC frameworkFocus groups	A list of Top 10 research priorities including:(1) How does a mother’s stress affect the baby?(2)-What are the most effective ways to improve patient-provider communication, particularly when patients perceive insensitive and rude comments from health care workers?(3) What is the most effective care for pregnancy and high-risk pregnancy? For example, if African American women are at higher risk, why isn’t there specialized care to improve outcomes?(4) What causes Sudden Infant Death Syndrome?(5) Does the type of insurance you have determine the type of care that you get, or the quality of your care and is care different based on insurance status or race?	12/20 (60%)
Ramirez., 2011 [37]	USA	Salud America! Developing a national Latino childhood obesity research agenda	To identify research priorities to address Latino childhood obesity	Academics, researchers, health educators, administrators, managers, clinicians, public health workers, students, community(total n = 313)	Modified three-round Web-based Delphi	25 research priorities identified within the domains of society; community; school; family; individual. These include:Society: Policies that subsidize accessibility of healthy foods to improve diet among Latino familiesCommunity: built environment policies involving collaborations with multiple stakeholders to promote physical activitySchool: health, nutrition, and active physical education classes as part of the school curriculumFamily: engaging Latino families as advocates of childhood obesity prevention initiatives at the community and school levelsIndividual: programs making physical activity more attractive than watching TV or playing video games	13/20 (65%)
Peiris-John et al., 2016 [35]	New Zealand	Stakeholders views on factors influencing the wellbeing and health sector engagement of young Asian New Zealanders	priorities on the health and wellbeing of Asian youth	Opinion leaders, key decision makers on Asian youth health from the academic field, in health service planning and community organisationsNo public involvement(total n = 6)	interviews	Key priority themes identified were:(1) cultural identity (2) integration and acculturation (3) barriers to help-seeking	5/20 (25%)
Maar et al., 2010 [29]	Canada	Reaching agreement for an Aboriginal e-health research agenda: the Aboriginal telehealth knowledge circle consensus method	Develop an Aboriginal E-health research agenda	Aboriginal council members; Aboriginal community; regional and provincial, and federal leaders, and policy makers(total n = 40)	Novel Aboriginal telehealth knowledge consensus method containing 7 cycles	Priorities fell into 6 distinct topics(1) ethical principles for Aboriginal e-health Research (2) internet-based national information for Aboriginal e-health initiatives; (3) research related to e-health education and professional development; (4) sustainability; (5) best practices; and (6) broader applications and impact of e-health on Aboriginal culture and communities.	12/20 (60%)
Goold et al. 2018 [23]	USA	Members of minority and underserved communities set priorities for health research	General health in minority and underserved communities	Academic and community partners (n = unknownMembers from minority and medically underserved communities45% White ethnic30% Black African American8% Hispanic6% Native American4%Arab American, Arab, or Chaldean(total n = 519)	Focus groupsSurveys	Highest priority was given to:child health research and mental health research. Other prioritised topics were Aging, access, promote health, healthy environment, and what causes diseaseBlack/African American participants were less likely to prioritise mental health researchNative American and Arab American participants prioritised research on culture and beliefs	11/20 (55%)
Spurling et al., 2017 [36]	Australia	‘I’m not sure it paints an honest picture of where my health’s at’-identifying community health and research priorities based on health assessments within an Aboriginal and Torres Strait Islander community: a qualitative study	Identify health priorities of the Aboriginal and Torres Strait Islander communities that could be translated into research themes	Key informants from an urban Aboriginal and Torres Strait Islander community(total n = 21)	Interviews	Three themes emerged, to be translated into research priorities(1) complex, interrelated intergenerational nature of health involving social, cultural, and environmental determinants of health(2) ambivalence about health assessments(3) community strength	10/20 (50%)
Wexler et al., 2015 [27]	USA	Framing health matters: advancing suicide prevention research with rural American Indian and Alaska native populations	Suicide research priorities in indigenous populations	Suicide researchersNo public involvement(n = unspecified)	3-day consensus workshop	Two main themes:(1) Expansive commitments of indigenous approaches to inquiry: holistic perspectives; focus on the past as well as the present and future(2) Community-level factors: conceptualising suicide as a social problem: localizing indigenous suicide rates in specific community contexts; development of community capacity and collaboration on design of local programs; emphasis on protective factors, resilience, and well = being	7/20 (35%)
Bryan et al., 2020 [30]	Canada	A research agenda to improve patients’ experience of knee replacement surgery: a patient-oriented modified Delphi study of South Asian origin in British Coumbia	Identify a research agenda for South Asian patients who undergo knee replacement surgery	South Asian patients and caregivers, healthcare professionals(total n = 53)	Focus groupsModified Delphi survey	A list of 25 prioritiesTop priorities both for patients and caregivers and for clinicians were (1) promoting exercise following surgery and (2) self-management after hospital discharge.One of the highest ranked topics for patients and caregivers was improving knee implants. Patients and caregivers prioritized research on promotion of exercise and self-management following surgery and improvement in knee implants.	10/20 (50%)
Gallagher et al., 2010 [21]	USA	Identifying interdisciplinary research priorities to prevent and treat pediatric obesity in New York city	Child obesity in underrepresented Black and Hispanic communities	Obesity experts among different faculties at Columbia university including clinical scientists, clinicians, educators, service providers, and public health researchersNo public involvement(total n = 55)	4 focus groupsSurvey	A list of top 10 priorities including:(1) Integration of behavioural and cultural components into research(2) Contribution of health disparities on rates of childhood obesity in our communities(3) Social determinants of health and identification of previously unmeasured factors(4) Effectiveness of behavioural approaches targeted toward families(5) Translating current evidence into practice in the clinic and community activity	12/20 (60%)
Wong et al., 2015 [34]	New Zealand	Priorities for Asian youth health: perspectives of young Asian New Zealanders	identify priority areas for research on Asian youth health	Asian youth(total n = 15)	Focus groups	Themes identified were:(1) cultural differences and identity(2) racism and discrimination(3) access mental health issues	4/20 (20%)

E-health is the health services and information delivered through the internet and related technologies

The main outcomes of studies were the identification of a range of research priorities related to the topic area. The research priorities were expressed as prioritised research topics, priority themes, top 10 prioritised lists, and more extensive lists of research questions. Top identified priorities included the need for greater inclusion of Latino children in medical research [24], the design and delivery and location of mental health services for refugees [25], access and engagement with suicide prevention services for people from immigrant and refugee backgrounds [26], Expansive commitments of indigenous approaches to inquiry for suicide in indigenous populations [27], cancer prevention and early detection in indigenous Australians [28], ethical principles for aboriginal e-health research [29], research on quality of life for minority and underserved populations [22], child health research and mental health for minority and underserved populations [23], complex, intergenerational nature of health involving social, cultural, and environmental determinants of health for Aboriginal and Torres strait islander communities [36], research into the effectiveness of alternative medicines for South Asian children [33], promoting exercise following surgery for South Asian patients undergoing knee replacement surgery [30], evaluating newcomer programs identifying how family and community stressors affect newly arrived immigrants and refugee adolescents functioning in school [32], how does a mothers stress affect the baby? for women of colour [31], policies that subsidise accessibility of healthy foods to improve diet among Latino families [37], integration of behavioural and cultural components into research to prevent and treat paediatric obesity in predominantly Black and Hispanic communities [21], cultural differences and identity for young Asian New Zealanders [34] and cultural identity, integration and acculturation in young Asian new Zealanders [35].

Assessing study quality against the checklist of good practice in research priority setting

None of the studies fulfilled all good practice principles as proposed by the checklist (see Table 4).

Table 4

Appraisal of comprehensiveness of reporting.

Item	Study	Total studies n (%)
Context
1-The resources available for the exercise were reported	[22, 23, 31]	3 (17.6)
2-The focus of the exercise was clearly stated (what it was about and who it was for)	[21–37]	17 (100)
3-The underlying values or principles were clear	[21, 22, 24–37]	16 (94.1)
4-The health environment in which the process took place was described	[21, 24–33, 35–37]	14 (82.3)
5-The research environment in which the process took place was described	[21, 24–29, 31–37]	14 (82.3)
6-The political environment in which the process took place was described	[29]	1 (5.8)
7-The economic/financial environment in which the process took place was described	[32]	1 (5.8)
Use of a comprehensive approach
8-The process of priority setting was described in detail	[22, 23, 29, 31–33]	6 (35.2)
Inclusiveness
9-The participants involved in setting research priorities were described	[21–23, 26, 28–37]	14 (82.3)
10-An appropriate representation of expertise was included	[22, 23, 28, 30–33, 36]	8 (47)
11-An appropriate representation of sex was included	[22, 23, 33, 34, 36]	5 (29.4)
12-An appropriate representation of regional participation was included	[21–23, 28–31, 33, 34, 36, 37]	11 (64.7)
13-Relevant health sectors and other constituencies were included	[22, 23, 26, 28–30, 32, 36, 37]	9 (52.9)
Information gathering
14-The information and sources used to inform the priority setting exercise were referenced	[21–33, 37]	14 (82.3)
Planning for implementation
15-Plans for translation of research priorities were discussed	[21, 31, 37]	3 (17.6)
16-Who will implement the research priorities and how	[21, 31, 37]	3 (17.6)
Criteria
17-Relevant criteria to focus discussion around setting priorities were stated	[22, 23, 26, 32, 33, 37]	6 (35.2)
Methods for deciding on priorities
18-Approach for deciding on priorities was described (e.g., consensus or metrics based	[21–27, 29–33, 37]	13 (76.4)
Evaluation
19-When and how evaluation of the established priorities and the priority setting process will take place were defined (e.g., multiple sessions)	0	0
Transparency
20-Clarity about the approach used was stated, i.e., who set the priorities how priorities were set	[21–23, 25–32, 36, 37]	13 (76.4)

Theme 1: Context

Every study reported some contextual factors. All studies made the focus of their exercise clear and overall, studies reported the values and principles behind their exercise. However, only three studies explicitly included information regarding the resources they used [22, 23, 31]. These included a detailed discussion around an interactive device used for the process [22, 23] and the materials used for each rounds of the process (e.g., flip chart paper, markers; audio recorders). The health environment was described by nearly all studies aside from [22, 23, 34] and the research environment was described by all studies excluding [22, 23, 30] yet only one study described the political environment in which the prioritisation exercise occurred [29]. Similarly, only one study described the economic environment context [32].

Theme 2: Use of a comprehensive approach

Six studies identified using comprehensive frameworks to conduct their research priority setting exercises [22, 23, 29, 31–33]. Four of them modified existing frameworks. One from the Child Health and Nutrition Research Initiative [32]; one from the James Lind Alliance [33]; two adapted an existing deliberation exercise: CHAT framework [22, 23]. Two studies developed comprehensive protocols and used them to guide their research priority setting exercises [29, 31]. The remaining eleven studies did not report using an established framework to guide the process.

Theme 3: Inclusiveness

Overall, most studies described the participants involved in the process. However, some studies provided more demographic information than others, including age, ethnicity, sex, and occupation of participants. Two studies did not provide any demographic details [24, 27]. The inclusion of a diverse range of key technical experts such as policy makers, service providers, academics, researchers and health care practitioners was described in most cases and these groups formed a core working group/council in the first stage of the prioritisation exercise in some studies [22, 26, 29, 30, 32]. These experts either had experience of working with the groups they were setting priorities for, or extensive knowledge of the groups (according to the authors). There was PPI in the research team albeit in fewer instances [29, 30].

Underpinning most of the prioritisation exercises were strong notions of equality, fairness and justice for people underrepresented in research, with a focus on community research ownership, community action, collaboration and partnership. An appropriate level of BAME PPI involvement was included in most studies [22, 23, 28–31, 33, 34, 36, 37]. Six studies did not report any BAME PPI [21, 24, 25, 27, 32, 35] and one study used a very small sample of BAME PPI [26].

Of the ten studies that mentioned the sex of participants, half contained a disproportionately larger number of female participants. However, it was noted that females are underrepresented in research more generally, so over-representation was not deemed an issue under this circumstance. The majority of studies reported an appropriate representation of regional participation and had included relevant sectors in their priority setting process.

Theme 4: Information gathering

The majority of studies used a working group or planning committee consisting of experts in the field, in order to define domains and categories in which to direct the research priorities from the beginning of the exercise. Most studies reported the range of technical data required to inform the discussion on research priorities ranging from presentations, literature reviews, informal and formal discussions, conferences, health assessments and surveys. Initial documentation containing stakeholder priorities and research priorities set by external bodies in the field were used in some instances to compare previous research priorities and those identified in the exercise. Consultation with BAME community stakeholders prior to the studies was reported in a minority of studies [25, 31, 33].

Theme 5: Planning for implementation

A limited number of studies reported plans to convert research priorities into projects. One included the prioritised list in a request for proposals to address issues around pre-term birth [31]. Another disclosed that workshops and targeted seminars were developed, along with a grant application submission, and sponsored projects to reduce childhood obesity [21] and several projects in the community were established to aimed at obesity reduction in Latino children as a result of their research priority setting study [37]. Plans for pilot studies were also established from research agendas [21, 31, 37].

Theme 6: Criteria

Six studies stated that criteria had been chosen to focus the discussions [22, 23, 26, 32, 33, 37] yet criteria was only made explicit by [33] who cited; burden of illness, inequalities, cost to the NHS, and impact on family and child as criteria for topic submission, and [32] who listed; answerability, significance, and practical application, as criteria to set priorities.

Theme 7: Methods for deciding on priorities

Studies adopted either a consensus-based approach, [24, 27, 29, 33] a metrics-based approach [21, 26, 28, 30, 37] or a combination of both [22, 23, 25, 31, 32]. A small minority of studies used thematic analysis to identify themes borne from focus group discussions and described these as the research priorities [34-36]. Ranking and/or consensus was not used to determine these priorities. Likert scales were the most popular ranking method used for prioritization. Dot voting was used to rank in one study [31]. Surveys, interviews (phone and face to face), focus groups, the Delphi method (adapted and original), the nominal group technique (adapted and original), were identified as common methods for deciding on priorities. One study did not disclose its methods for deciding on priorities [24].

Theme 8: Evaluation

No studies reported plans to update research priorities. Despite this, in a small number of studies, researchers did go on to conduct further research priority exercises for the same ethnic population they studied in an earlier research priority study [22, 25, 35], albeit with the aim of more diverse stakeholder inclusion [23, 34] and to establish a research agenda in a different topic area [26]. The most recent study included in this review expressed a desire to conduct a similar study with other ethnic groups [30].

Theme 9: Transparency

Most studies were explicit about who set the priorities by providing information on participant characteristics. Studies were also transparent in describing the stages of the process that involved different stakeholders. For instance, initial topic areas may have been identified by a core group; community/patients/service users, or by professional stakeholders, to be ranked in the final stage of the process by another group of stakeholders. Most studies detailed how they set priorities; however, the extent to which an explanation on how priorities were set varied across studies.

Some studies made transcripts available [31], as well as reports [25] along with an interim report which included the research plan, research tools and a list of key informants [29] and finally, a consensus statement [24]. Few studies provided an evaluation of their priority setting process [22, 25, 26, 30, 31, 33]. Examples included: an acknowledgment of little to low involvement from BAME communities in setting priorities either at any given stage of the exercise [25, 26, 30]; evaluation of the information gathering stage as well as the plausibility of incentivizing responses to increase response rates by expert stakeholders [33]; discussion around data collection about ethnicity encountered technical difficulties which resulted in the loss of data for 25% of the participants, which could have markedly influenced the study findings [22].

Discussion

This review provides an overview of research priority setting within studies for BAME health. Publications in a range of different topic areas were identified such as knee surgery, mental health, preterm birth, and child obesity. Identified priorities include the design and delivery and location of mental health services for refugees; research into the effectiveness of alternative medicines for South Asian children; cancer prevention and early detection in indigenous Australians, and integration of behavioural and cultural components into research to prevent and treat paediatric obesity in predominantly Black and Hispanic communities. By applying a checklist of good practice in research priority setting by Viergever et al. [17] to the prioritisation exercises, a number of strengths and weaknesses were identified which influenced the quality of the research prioritisation exercises within the studies. None of the studies fulfilled all the good practice criteria. This suggests there remains significant work to do to achieve effective research priority setting in BAME health.

Our findings suggest that the greatest failure of studies when assessed against the good practice checklist, concerned the criteria evaluation. The majority of studies presented procedures and outcomes which were said to inform and assist funders and policy makers on making better decisions and assist researchers in doing more work in the area, especially in involving BAME populations in setting research priorities. Yet very few exercises made explicit any plans to translate the priorities into projects. It should be noted that this issue is not limited to research prioritisation for BAME health only. Prioritisation literature has demonstrated that lack of evaluation of outcomes is a common problem across research prioritisation studies more widely [14]. However, this has been further identified as a barrier to BAME participation in research. It is perceived that there would be no personal benefit of participating [38] and it is a common complaint from BAME communities that they are not informed of the outcomes of a study after taking part [39]. Thus, it is very important to disseminate results back to the community and participants involved so they can be assured that their views were in fact incorporated and their involvement was not tokenistic. Strategies to promote outcome evaluations in BAME population research could include reporting the acceptability of the exercise to those involved in the process [14] and performing an impact analysis, for example, as a review of research performed can be valuable in that it can enforce discussion on issues surrounding implementation [17].

None of the studies used the original versions of established research priority setting approaches, recognizing that they were unsuitable for involving BAME participants. For instance, some studies advised against using well known comprehensive frameworks where BAME patient and public stakeholders co-produced research priorities alongside professional stakeholders, noting that it may lead to the possibility of further muting seldom heard voices, instead, they either adapted comprehensive approaches where opinions could be expressed freely, or developed their own, deeming them more inclusive [29, 31, 33].

It is widely recognised that community engagement in priority setting is a key means for setting research questions and topics that are relevant and beneficial to them. Yet, without addressing power dynamics, their engagement can be tokenistic [40]. The studies in this review that developed their own framework, focussed on strong notions of patient/community-oriented research, co-production, community engagement, and BAME research ownership, and cited the importance of participatory approaches to research. They were especially mindful of involving BAME participants in setting priorities and described a more thorough recruitment strategy to fulfil this objective. Examples included examining potential barriers to recruitment and steps taken to overcome them such as arranging transportation and accommodation of issues around language and low literacy.

One particular area where some studies were lacking was the appropriate involvement of BAME communities in establishing research priorities. Sample sizes of BAME groups were either too small or there was no involvement of these groups at all, in any stage of the process. Discourse has established that marginalized communities are often excluded from priority setting exercises due to issues surrounding language barriers and difficulties accessing communities [41]. Given that research findings show that BAME groups react favourably and show a willingness to be involved in research, perceiving research participation as an opportunity or even a right [42], this is a missed opportunity. As well as this, ignoring BAME concerns has generated scepticism and community anger towards health research, especially among BAME groups [43].

Interestingly, many of the studies that scored highly in quality, had involved a high number of BAME research participants in various stages of their prioritisation exercise [22, 29, 31, 33, 37]. Some of them had developed novel frameworks to conduct research priority setting specifically for BAME groups and recommended that their framework be used by other researchers aiming to set research priority agendas in BAME health. Involving BAME communities in each stage of the process was deemed fundamental in their studies. For instance, they ran pilot studies prior to the prioritisation process or enlisted members of the community to provide feedback on the topics guide used to generate discussion around priorities. They made efforts to build trust and facilitate smooth participation in their exercises. This was done in various ways such as enlisting community facilitators to guide discussions, utilising bilingual interpreters, recruiting community leaders in a position of trust in their communities, organizing the exercises in locations familiar to participants, holding focus groups specifically for different cultural groups, and running women only groups. In the literature, these have all been identified as effective ways of facilitating BAME community participation in research [38, 44]. This is in line with the UK’s National institute for health research INVOLVE guidance [45] on co-producing a research project that emphasizes embracing diversity and the development of structures and practices to allow for the involvement of all stakeholders required for a project. There is also an argument that experiential knowledge from those directly affected by the issue being researched improved the quality and relevance of the research such as identifying appropriate research questions and improving the clarity of communications [46].

With regards to applying criteria to generate research priorities, only two studies explicitly stated the criteria to guide the process [32, 33] it was evident that an ethical framework supported the process of priority setting in most of the studies as justice and fairness were explicitly mentioned by researchers when describing the context of the proposed research. However, this may not be sufficient given that explicitly defined criteria is particularly important in that it could provide justification to satisfy funders and policy makers so they may fund, support and utilise the priorities [47, 48].

Strengths and limitations

To our knowledge, this is the first systematic review to characterise and evaluate published research priority setting studies relevant to the health of BAME populations. The strengths of this review include the description of reported health research priorities in BAME populations, and the application of a comprehensive methodological framework to evaluate processes. However, the review is not without its limitations. For example, the search was limited to include studies in the English language only, which could have excluded research priority studies in other languages.

Implications and recommendations for future research

Since the publication of the narrative review from Bryant et al. (2014) and according to the results from this review, barely any improvements have been made in terms of evaluation from prioritization processes in exercises. Quality research that adheres to good practice guidelines are required to make a difference to BAME communities and improve outcomes, thus helping close the health inequality gap amongst BAME communities. Identifying potential barriers to recruiting BAME communities in research priority setting and putting measures in place to overcome these, could be very useful in increasing their involvement. This ensures relevance of the research to meet their needs and address health inequalities.

Study PRISMA checklist.

(DOC)

Click here for additional data file.

12 Mar 2021

PONE-D-20-29158

Research priority setting in Black, Asian and minority ethnic health: A systematic review

PLOS ONE

Dear Dr. Iqbal,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Apr 26 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Baltica Cabieses, PhD

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please specify the time frame for your search - i.e., did you search for articles from the inception of the databases? In addition, please ensure you include all relevant articles published to date.

3. Please state how many authors participated in the article search.

4. Please ensure that you include a title page within your main document. You should list all authors and all affiliations as per our author instructions and clearly indicate the corresponding author.

5. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 and 2 in your text; if accepted, production will need this reference to link the reader to the Table.

6. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

7. We note that Figure 1 in S5 includes an image of a participant in the study.

As per the PLOS ONE policy (http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research) on papers that include identifying, or potentially identifying, information, the individual(s) or parent(s)/guardian(s) must be informed of the terms of the PLOS open-access (CC-BY) license and provide specific permission for publication of these details under the terms of this license. Please download the Consent Form for Publication in a PLOS Journal (http://journals.plos.org/plosone/s/file?id=8ce6/plos-consent-form-english.pdf). The signed consent form should not be submitted with the manuscript, but should be securely filed in the individual's case notes. Please amend the methods section and ethics statement of the manuscript to explicitly state that the patient/participant has provided consent for publication: “The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details”.

If you are unable to obtain consent from the subject of the photograph, you will need to remove the figure and any other textual identifying information or case descriptions for this individual.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The systematic review “Research priority setting in Black, Asian and minority ethnic health” well executed and scientifically founded study. Research question, hypothesis, methods and results are clearly described.

Reviewer #2: The manuscript gives an interesting critical view of the evidence on research priority setting in BAME population. The strengths of this study include the application of a comprehensive methodological framework to evaluate processes. As well as reporting the non-compliance to good practice guidelines. However, there are missing details in the methodology to ensure the standards of the PRISMA statement. Besides description of the characteristics and main results of included studies, since the reader might be interested in having more information of priority research topics in BAME populations, beyond the quality of these studies. Furthermore, there are punctuation errors that must be revised throughout the document.

Title

It might be necessary to specify the scope of the review, considering that the authors proposed: “to determine the extent to which the evidence followed good practice principles”.

Abstract

Page 1. I suggest including a brief description of search terms and inclusion criteria.

Page 2. The results section does not report the priority research topics (mental health, preterm birth, child obesity etc.) found in this study. Moreover, I suggest highlighting the concern related to evaluation criteria, as stated in the main text.

Introduction

Page 4. Although the authors have mentioned evidence on priority setting methods and the increase of priority setting exercise, there is no cited literature related to the specific context of BAME population research. Therefore, I recommend to clearly state the knowledge gap in BAME population and provide reference for “given the stark increase in research priority setting in the past decade, along with an increase of discourse around evaluation of research priority setting initiatives”.

Methods

Page 6. Regarding the study selection, I suggest specifying characteristics of the admitted population, study types and a well stablished main outcome. The latter is necessary to understand the applied criteria by the authors and to justify their selections. Therefore, it will support the statement “This included studies where an ethnically diverse population was described in order to determine how priorities may differ between ethnic groups”. Table 1. I recommend revising the description of inclusion and exclusion criteria in order to make a clear parallel between them.

Page 7. Although the quality appraisal tool is presented in detail, there is some missing information related to the quality score interpretation (table 3 only shows percentages based on meet criteria).

Page 9. There is a description of selection process that should be included before the quality appraisal. Furthermore, there is no information of data collection and synthesis (data extraction procedures and qualitative analysis). I suggest revising and follow a consistent structure with the PRISMA statement.

Results

Page 11. The first paragraph of the result section does not include a brief description of the characteristics (participants when possible, study type etc.) and main outcomes of included studies. This section must inform the readers of what is known about the research priority topics to highlight the current needs in BAME population. The results are focused on the quality criteria, instead of providing a comprehensive analysis of all the data that has been collected.

Table 3. I suggest including columns for participant’s description and main outcomes related to the topic and scope of each study. Furthermore, the quality score column might include a brief explanation, so the reader will identify the flaws of each study. Please Define E-health at the table legend.

Discussion

Page 19. The first paragraph of the discussion must be accompanied with a brief critical synthesis of priority topics, beyond just mentioning them.

Page 20. I suggest citing o propose strategies to promote evaluation of outcomes in BAME population research.

Page 21. The authors state “sample sizes of BAME groups were either too small or there was no involvement of these groups at all, in any stage of the process”, which raise questions about the selection criteria; since the reader might be expecting studies that include BAME population through the process.

Page 23. The Last paragraph of the discussion does not highlight the concern related to evaluation of the process. Please report a summary of study strengths and limitations

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Minor_comments_SystematicReview_PONE-D-20-29158.docx

Click here for additional data file.

15 Apr 2021

Academic editor’s comments

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming

Author - The style changes have been made throughout the manuscript, including naming

2. Please specify the time frame for your search - i.e., did you search for articles from the inception of the databases? In addition, please ensure you include all relevant articles published to date

Author - The following sentence has been added “we searched databases from their inception to July 2020” (p.7)

3. Please state how many authors participated in the article search

Author - The following sentence has been added “the principal researcher (HI) independently conducted the article search” (p.6)

4. Please ensure that you include a title page within your main document. You should list all authors and all affiliations as per our author instructions and clearly indicate the corresponding author

Author - A title page has been added and has adhered to PLOS1 author guidelines

5. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 and 2 in your text; if accepted, production will need this reference to link the reader to the Table

Author - Table 1 is now referred to in the text on p.8. Table 2 is referred to on p.10

6. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines

Author - The supporting Information file now includes captions which can be found at the end of the manuscript. In text-citations have been updated.

7. We note that Figure 1 in S5 includes an image of a participant in the study.

As per the PLOS ONE policy (http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research) on papers that include identifying, or potentially identifying, information, the individual(s) or parent(s)/guardian(s) must be informed of the terms of the PLOS open-access (CC-BY) license and provide specific permission for publication of these details under the terms of this license. Please download the Consent Form for Publication in a PLOS Journal (http://journals.plos.org/plosone/s/file?id=8ce6/plos-consent-form-english.pdf). The signed consent form should not be submitted with the manuscript, but should be securely filed in the individual's case notes. Please amend the methods section and ethics statement of the manuscript to explicitly state that the patient/participant has provided consent for publication: “The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details”.

If you are unable to obtain consent from the subject of the photograph, you will need to remove the figure and any other textual identifying information or case descriptions for this individual

Author - Studies included in the review have been removed from the supporting information files. Included studies can be located in the reference list. Therefore, S5 no longer exists with the image requested to be removed

Reviewer 1 comments

Grammar errors on page 6, 15, 16, 20, 21, 22, 23

Author - The errors have been corrected

Since in the introduction the example of health conditions from UK Black African women was mentioned, the question rises why the search term “Africans” was not included in the search, to increase the reach of articles including this specific sub-group - Page 4. The example given of health conditions among UK Black African women has been removed

Introduction: “This systematic review is interested to see if there has been any progress in research priority process evaluation since then, with a specific focus on BAME health, given the stark increase in research priority setting in the past decade, along with an increase of discourse around evaluation of research priority setting initiatives” � Recommendation to include the conclusion that since the publication of the narrative review from Bryant et al. (2014) and according to the results from this review (none of the studies fulfilled all the good practice criteria) barely any improvements have been made in terms of evaluation from prioritizations processes in exercises

Author - Page 42. The recommendation has been added to the section Implications and recommendations for future research

Reviewer 2 comments

Title

It might be necessary to specify the scope of the review, considering that the authors proposed: “to determine the extent to which the evidence followed good practice principles”. Author - Title has been changed to “A systematic review to identify research priority setting in Black and minority ethnic health and evaluate their processes”

Abstract

Page 1. I suggest including a brief description of search terms and inclusion criteria -

Author - page 2. A description of search terms and inclusion criteria have been added to the abstract

Page 2. The results section does not report the priority research topics (mental health, preterm birth, child obesity etc.) found in this study. Moreover, I suggest highlighting the concern related to evaluation criteria, as stated in the main text

Author - Page 2. The results section has been amended to include the different topic areas found in the study. I have added in a sentence highlighting the concern related to the evaluation, which now states “None of the exercises followed all the good practice principles as outlined in the checklist"

Introduction

Page 4. Although the authors have mentioned evidence on priority setting methods and the increase of priority setting exercise, there is no cited literature related to the specific context of BAME population research. Therefore, I recommend to clearly state the knowledge gap in BAME population and provide reference for “given the stark increase in research priority setting in the past decade, along with an increase of discourse around evaluation of research priority setting initiatives”

Author - Page 5. The following sentence has been added to address the reviewer’s concern: “It is also unclear whether research prioritisation exercises have been undertaken for BAME health” Page 5-6. References have been provided for the statement quoted by the reviewer

Methods

Page 6. Regarding the study selection, I suggest specifying characteristics of the admitted population, study types and a well stablished main outcome. The latter is necessary to understand the applied criteria by the authors and to justify their selections. Therefore, it will support the statement “This included studies where an ethnically diverse population was described in order to determine how priorities may differ between ethnic groups”. Table 1. I recommend revising the description of inclusion and exclusion criteria in order to make a clear parallel between them

Author - Table 1 has been amended to make this distinction clear

Page 7. Although the quality appraisal tool is presented in detail, there is some missing information related to the quality score interpretation (table 3 only shows percentages based on meet criteria)

Author - page 12. the information has been added under the section data collection

Page 9. There is a description of selection process that should be included before the quality appraisal. Furthermore, there is no information of data collection and synthesis (data extraction procedures and qualitative analysis). I suggest revising and follow a consistent structure with the PRISMA statement

Author - Page 7 -8. The section has been amended to include this information

Page 12. Information on data collection and synthesis has been added

Results

Page 11. The first paragraph of the result section does not include a brief description of the characteristics (participants when possible, study type etc.) and main outcomes of included studies. This section must inform the readers of what is known about the research priority topics to highlight the current needs in BAME population. The results are focused on the quality criteria, instead of providing a comprehensive analysis of all the data that has been collected.

Author - Page 14. The first paragraph of the results section now includes a brief description of the participants in each study and the methods used in studies. Although Table 3 now includes a column displaying the main outcomes (research priorities) for each study, as suggested by the reviewer, multiple research priorities were identified throughout the studies, in different topic areas, as is typical in priority setting exercises as they encompass a wide spectrum of views. As such, we are unable to provide a comprehensive analysis of the priorities themselves in the first paragraph of the results section. Instead, we listed the top identified priority in each study. If the reader would like to view the full list of priorities for each study, the studies can be found in the reference list.

Table 3. I suggest including columns for participant’s description and main outcomes related to the topic and scope of each study. Furthermore, the quality score column might include a brief explanation, so the reader will identify the flaws of each study

Author - Columns have been added displaying the population and the main outcome of each study. We have added information in the quality score column, in the first box and signposted to Table 2 which provides detail of the quality criteria

Please Define E-health at the table legend

Author - Page 27. health has been defined at the table legend

Discussion

Page 19. The first paragraph of the discussion must be accompanied with a brief critical synthesis of priority topics, beyond just mentioning them.

Author - Page 37. Examples of priority topics have now been given

Page 20. I suggest citing o propose strategies to promote evaluation of outcomes in BAME population research

Author - Page 38-39. Strategies have been proposed

Page 21. The authors state “sample sizes of BAME groups were either too small or there was no involvement of these groups at all, in any stage of the process”, which raise questions about the selection criteria; since the reader might be expecting studies that include BAME population through the process

Author - The selection criteria have been amended and it is now clear that BAME participation in studies was not a pre-requisite for study inclusion

Page 23. The Last paragraph of the discussion does not highlight the concern related to evaluation of the process. Please report a summary of study strengths and limitations

Author - Page 42. The concern regarding evaluation of the process is now highlighted

Page 42. Strengths and limitations of the study have been provided

Submitted filename: Response to reviewers.docx

Click here for additional data file.

3 May 2021

A systematic review to identify research priority setting in Black and minority ethnic health and evaluate their processes

PONE-D-20-29158R1

Dear Dr. Iqbal,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Baltica Cabieses, PhD

Academic Editor

PLOS ONE

19 May 2021

PONE-D-20-29158R1

A systematic review to identify research priority setting in Black and minority ethnic health and evaluate their processes

Dear Dr. Iqbal:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Baltica Cabieses

Academic Editor

PLOS ONE