A realist review of community engagement with health research.

Introduction: Community engagement is increasingly recognized as a critical aspect of global health. Recent years have seen an expansion of community engagement activities linked to health research, but debates and inconsistencies remain about the aims of different types of engagement, mechanisms underpinning their implementation and impact, and influential contextual factors. Greater commitment to and consistency around community engagement by health research programs, implementers and funders requires a more coherent evidence base. This realist review is designed to improve our understanding of how and why community engagement contributes to intended and unintended outcomes (including research and ethical outcomes) in different contexts. Given the breadth and diversity of the literature on community engagement in health research, the review will initially focus on malaria research in low- and middle-income countries (LMICs) and draw on wider global health literature where needed. Methods and analysis: Community engagement in practice is often a complex set of interventions. We will conduct a realist review - a theory driven approach to evidence synthesis - to provide explanations for how and why community engagement with health research produces the pattern of outcomes observed across different contexts of application. We will consolidate evidence from a range of documents, including qualitative, quantitative and mixed method studies. The review will follow several stages: devising an initial programme theory, searching evidence, selecting appropriate documents, extracting data, synthesizing and refining the programme theory, and reiteration of these steps as needed. Ethics and dissemination: A formal ethics review is not required for this literature review.  Findings will be disseminated in a peer reviewed journal, through national and international conferences, and through a set of short briefings tailored for audiences with an interest in community engagement. Outputs and presentations will be informed by and feed into our network of community engagement experts. PROSPERO registration number: CRD42019125687.

Introduction

Background

Community engagement is increasingly recognized as a critical element of global health research, recommended by ethicists, funders and international ethics guidelines, such as the 2016 Council for International Organization of Medical Sciences (CIOMS) guidelines [1– 3]. However, ‘community engagement’ remains a relatively ill-defined term with varied meanings and practices across the domains of health promotion, health related research, health programmes and international development [4]. The diverse conceptual underpinnings of community engagement, the range of goals ascribed to it and wide variety of activities undertaken all complicate the evaluation of community engagement [4].

Community engagement has been defined as a process of collaborative work with groups of people affiliated by geographic proximity, interest or health issue, to address social and health challenges affecting those people [5]. In practice in health-related research, community engagement encompasses a wide variety of activities and strategies, such as conducting meetings with community members and representatives, working with community advisory boards [6, 7] and involving members of the community in designing and implementing research activities [4, 8]. Community Engagement also involves a wide range of different stakeholders in a dynamic set of social interactions with considerable relational complexity [3, 9, 10].

Our review aims to focus on CE in large research programmes in LMICs in order to understand how community engagement works in practice in such settings and assess claims made for contributions to research related and ethical outcomes. In addition to understanding ‘common current practice’ in community engagement in large health research programmes, we also want to understand potentially different dynamics developed through community engagement in other contexts in order to highlight alternative approaches and the distinctive mechanisms which may be brought into play in such settings.

Goals and outcomes of community engagement

Scholars have identified a series of overarching goals of community engagement in health research [2, 11]. A distinction is often made between the instrumental goals of improving the quality and relevance of research [12], including achieving recruitment and retention targets [8, 11, 12], and a range of ethical goals of community engagement, including: respecting individuals, communities and stakeholders [3, 11, 13, 14]; building trust and social relationships [9, 11, 15– 18]; determining appropriate benefits; minimizing risks, burdens and exploitation [3, 4, 11, 19– 21]; supporting the consent process [10– 12]; understanding vulnerabilities and researcher obligations [3, 11, 23, 24]; and gaining permissions, approvals and building legitimacy [3, 11, 14, 25, 26].

Community engagement initiatives in health research often have more than one goal, however, and the distinction between instrumental and ethical goals in practice can be unclear [10]. Further, studies that have attended to the relational dynamics of engagement, rather than formal ethical procedures [15], have made visible the ‘human infrastructure’ of research [3] and highlighted how a concern with ethical negotiation of relationships may be integral to achieving more instrumental research goals. Aspirational outcomes for community engagement with research do sometimes include community empowerment and community capacity building [27– 29]. This is more common in participatory social development contexts, where there may also be a focus on some degree of co-production of the research itself [30, 31]. However, in large health research programmes in LMICs, co-design of the research with communities from the outset and throughout all stages of the research process appears rare. The bulk of current literature on community engagement in global health focuses on the contribution of community engagement to research-related outcomes rather than the range of potential ethical outcomes, including the impact on stakeholder relationships.

Community engagement goals and activities are also potentially affected by the type of research and interventions in which initiatives are embedded and the opportunities and constraints for engagement that they entail [4, 8, 32]. For instance, in mass drug administration studies and vaccine studies, where study success depends upon a high population coverage, community engagement may be emphasized to achieve study related aims rather than other goals [8, 33, 34]. In hospital based clinical trials, community engagement may focus on engaging with patients at the hospital rather than broader communities. There are guidelines for good participatory practice in clinical trials aimed at maximising the meaningful involvement of individuals and communities [28], but some have nevertheless argued that the nature of clinical trials can circumscribe the scope for community involvement compared to other forms of social and health research [35].

Community engagement mechanisms

From a realist perspective, diverse community engagement activities may rely on a more limited number of relational mechanisms. Potential mechanisms hinted at in the literature include: respectful interactions facilitate trust [9, 16]; listening and acting to show understanding and express recognition, leads to a sense of people being heard [11, 16]; and responsiveness of research processes increases motivation to participate [14, 16, 36– 38]. A range of different engagement activities may contribute to a mechanism coming into play, and differences in context will affect how these mechanisms work in particular settings, producing a pattern of outcomes [39, 40]. Figure 1 provides a simplified example.

Figure 1.

Example mechanism.

At the same time, several different mechanisms may work in combination, some of which may contribute to intermediate outcomes that, in turn, become an important new context within which other mechanisms may operate. Drawing on some of the potential mechanisms highlighted in the above paragraph, we can see how they may combine and interact with each other (recognising that whether or not they do in practice is an empirical question for the review to consider). For example, building of relationships between researchers and community members may help to establish trust and the credibility of research. At the same time, where community members can provide feedback and voice their concerns about research through meetings and discussions, and have these acknowledged and responded to, they may feel valued and respected and this may also contribute to the establishment of trust. Established trust may be an important precursor to a collaborative partnership between researchers, community members and other stakeholders, which may in turn enhance community members’ sense of ‘being a part of’ the research and may create a sense of responsibility and ownership in the research. In this way, engagement is likely to involve a number of different mechanisms in combination and their sequencing may prove to be important.

Influence of context

While some of the above mechanisms may be common across a wide range of types of health research, whether or not they come into play may be affected by a particular local social and cultural context [8, 40]. For instance, the impact of community engagement on targeted malaria elimination studies that aims at wide community mobilisation may be enhanced in a cohesive community [14] compared to a politically fragmented community [38, 40]. In recent years, the details of social and cultural context have been increasingly reported in accounts of community engagement around health research [14, 36, 37, 41, 42]. For example, the geo-political context of Thai-Myanmar border, livelihoods based on subsistence farming, the influence of traditional healing practices and remote and limited health service provision have been highlighted as affecting engagement outcomes [38]. A recent systematic review of the last century’s literature on community engagement and population coverage in mass antimalarial administration identified a range of contextual factors [8], including: how community engagement and population coverage could be affected by the social cohesion of the communities [33]; the influence of social hierarchies [14] and political factions [34, 36, 38]; cultural beliefs around blood tests (including rumours such as blood stealing) [14, 16]; the influence of traditional healing practices [43]; and, more broadly, perceptions around the concept of research [37], its rationale and the impacts [42, 44].

To date, however, there is a lack of analysis and explanation of how the factors of local social and cultural context have affected particular outcomes. Similarly, as noted above, a number of potential mechanisms have been hinted at in the literature, but these are often left implicit without any systematic analysis, including how they work in combination and are influenced by context.

Addressing diversity

Realist review and synthesis provides a framework to look at configurations of context, mechanism and outcome and to draw out regularities and patterns that endure across context [45, 46]. Some of the team involved in the current review previously attempted to conduct a conventional systematic literature review, but faced great challenges due to the extraordinary complexity and diversity of the community engagement literature [47]. They recommended that a realist review may be better suited to address such complexity [47– 50]. Using a realist approach, the current review will focus initially on malaria research to explore: the intended (and unintended) outcomes of community engagement; mechanisms by which community engagement strategies are believed to generate the outcomes; and important contexts that affect various mechanisms and outcomes. The ultimate aim of this review is to develop transferrable learning about community engagement in health research and programmatic interventions.

Methods

Glossary of terms

Realist review utilizes specific realist concepts with particular meanings, which are briefly described here [51].

Context: It refers to the backdrop of programs and research. For example, context can include cultural norms and history of the community in which a program is implemented, scope and the extent of existing social networks or the infrastructure in which the programme is built. They can be trust-building processes, geographic location effects, funding sources, opportunities or constraints. Context can thus be broadly understood as any condition that trigger and/or modifies the behaviour of a mechanism.

Mechanism: There are many definitions of mechanism. What they have in common is that mechanisms generate outcomes. Mechanisms are the agents of change. They describe how the resources embedded in a programme influence the reasoning and ultimately action of programme ‘subjects’. Mechanisms are underlying entities, processes, or structures which operate in specific contexts to generate outcomes of interest. Mechanisms are usually hidden and are sensitive to variations in context and generate outcomes.

Outcome: The intended and unintended results of interventions or programmes. Complex programmes may involve intermediate outcomes that provide important pre-conditions for other outcomes to be realised.

Programme theory: A programme theory is “an abstracted description and/or diagram that lays out what a program (or family of programs or intervention) comprises and how it is expected to work”. It is usually made up of a set of interlinked propositions explaining the causal relationships involved in how a programme works in practice

Context-mechanism-outcome configurations (CMOc): A CMO configuration explains the casual relationship between a particular aspect of context, whether or not a mechanism of interest is triggered by it, and the outcomes produced. A programme theory usually combines a range of different CMOcs, which may overlap or be nested to explain the observed patterns in phenomena. Configuring CMOs is a fundamental process in realist review to generate or refine the theory that becomes the final product of the review.

Review aim

This review aims to improve our understanding of the ways in which (i.e. how, why and in what contexts) community engagement interventions contribute to (or do not contribute to) reported outcomes (both explicitly aimed for and unintended), including the range of research related and ethical outcomes highlighted above under the section ‘Goals and outcomes’, and the roles of context and mechanisms in each case. Given that outcomes are not always clearly articulated in reports of community engagement, the review will also consider revealed outcomes, again attempting to understand the influence of particular mechanisms and context.

Our review will focus initially on large malaria research programmes to understand ‘common current practice’ in community engagement in settings of global health research. In addition, we may draw on wider literature to better understand some of the mechanisms underpinning community engagement outcomes such as empowerment and community capacity building, where adequate data cannot be found in the literature on large research programmes.

The review will focus initially on malaria research because it is a major strand of global health research, it is an area where members of the review team have experience and knowledge as well as access to practitioners and expertise, and it provides a pragmatic way of focusing the review to make it manageable.

Review objective

To conduct a realist review to understand the ways in which community engagement contributes to intended and unintended outcomes, and through which key mechanisms. This will be done with (A) engagement with a diverse range of literature, (B) the development of a programme theory and (C) feedback and advice from stakeholders experienced in the field.

Review research question

Within the existing and available literature pertinent to malaria research, what are the causal explanations for the ways in which community engagement contributes to intended and unintended outcomes?

Sub-questions:

What are the intended and unintended outcomes of community engagement strategies?

What are the key mechanisms by which community engagement strategies result in their intended and unintended outcomes?

What are the important contextual influences on the ways in which different mechanisms produce intended and unintended outcomes?

Study design

Realist review is increasingly recognized as an effective process for consolidating evidence and learning from complex social programming, particularly in public health and community development [48– 50]. The proposed review design is based on the realist review approach [46], which aims to explore how community engagement produce outcomes in specific contexts through the operation of a number of key mechanisms. This review will initially focus on the community engagement embedded in malaria research in low and middle income countries. The initial search commenced on 21 st February and screening from 12 th March, 2019. We anticipate the review will be completed by the 30 th June, 2020.

Over the years, realist reviews have explored the configurations of context, mechanisms and outcomes for complex issues, such as antimicrobial prescription by doctors [52], access to primary care for socio-economically disadvantaged elderly populations in rural areas [53], and the process of appraising the performance of doctors [54]. Because community engagement activities are complex interventions that work through a variety of different mechanisms to produce different outcomes across different contexts, realist review holds the promise of bringing greater clarity and understanding to the variation of engagement in practice.

A realist review eschews the traditional hierarchy of evidence and incorporates a wide variety of data, including those derived from qualitative, quantitative, mixed methods studies, as well as grey literature. The review will include all the pertinent documents published in English language only. In contrast to a more conventional review, in which the data are extracted and aggregated across a selection of studies deemed to be of sufficient quality, in realist review, diverse data are drawn upon depending on their potential value to contribute to refinement of programme theory. Within realist reviews assessments are not made for risk of bias of individual studies or meta-bias across studies [55]. Findings derived from this realist review are expected to be transferable because they will focus on commonly occurring mechanisms through which community engagement produce both intended and unintended outcomes. This will enable us to produce recommendations likely to be useful across domains beyond malaria within global health research. For the sake of clarity we have set out the realist review as six distinct steps. However, as is common in many realist reviews, our expectation is that we may need to make changes to what we have initially planned and set out in our protocol in order to answer our review questions [56]. Finally, we acknowledge that realist reviews incorporate iterative cycles within the steps outline below and of engagement with the literature and with stakeholders with relevant knowledge.

Stakeholder engagement

We aim to involve a wide range of stakeholders with expertise and experience of community engagement. At key stages in the review these stakeholders will provide input and feedback on relevant papers, our literature search strategies, and evolving learning. Figure 2 shows the structure of inputs into the review. We have established a ‘content expert advisor’ group comprised of experts from low- and middle-income countries (LMICs) who are community engagement practitioners and scholars. This group has been involved in the generation of the initial candidate programme theories for the review, and will provide input at key moments throughout the process. Our advisory group draws on expertise in bioethics, malaria, realist review and includes health research funders. In addition, we aim to solicit input from community members involved in community engagement processes, including in Kenya and on the Thai/Myanmar border. In both of these contexts, core team members and content experts are able to support this process. The website for the review is a programme hub within the MESH community engagement website [ https://mesh.tghn.org], which brings together a large LMIC based network including over 1000 practitioners of community engagement. We aim to use this platform to draw in and consult CE practitioners. An evolving group of wider stakeholders, including academics, policy-makers, funders, research programme managers, implementers and engagement practitioners, will also provide input. The wider stakeholder group will be identified using a snowball approach through the authors’ professional networks, engagement at conferences, and through more systematic outreach based on identifying stakeholders from different disciplines, identified in literature and relevant programming initiatives.

Figure 2.

Review team and stakeholder input.

Step 1: locating existing theories

As a first step in realist review, an initial scoping search is conducted to identify theories that begin to explain and develop our understanding of community engagement. This stage is crucial to visualize the underpinning assumptions about why certain components and processes of community engagement are required to achieve one or more desired outcomes [45].

At the initial stage, these theories are explored in two main ways: 1) drawing on exploratory searches of relevant literature in repeated cycles; and 2) consulting with key content experts who have practical experience of implementing community engagement.

The first stage of our initial scoping of key literature to identify elements of programme theory involved reviewing key literature in the field of community engagement recommended by our content experts. A summary of commonly recognised outcomes of community engagement, potential mechanisms and important elements of context was then discussed among the core team and our context experts to produce elements of an initial programme theory and related visualisation. This was subsequently refined through further discussion among the team. Figure 3 is the consolidated visualisation of elements of an overall programme theory that was then used to help focus our search strategy. It should be noted that at this stage the diagram is more a summary collection of still (heterogenous) elements, not strictly configured or consistent (where the labelling of context, proto-mechanisms and outcomes is only provisional), and that some of the terms are indicative at this stage rather than strictly defined (since their meaning in practice will depend on how they are evidenced in the literature). In addition, the visualisation includes an initial sense of the distinction between more background ‘conditions’ and the more active elements of ‘context’ which are expected to have a more salient influence on the mechanisms of interest. Making putative contexts, mechanisms and outcomes as explicit as possible is an important first step to help focus our ongoing systematic literature searches. This stage was not exhaustive but helped to capture key aspects of engagement and to help focus more systematic literature searches in stage 2.

Figure 3.

Initial programme theory visualisation.

This initial set of elements gathered in the programme theory visualisation ( Figure 3) will be further simplified to produce a version that can be used in community consultations in Kenya and the Thai Myanmar border. Figure 4 shows an early iteration of a simplified version of the diagram, notable for the way it focuses on our initial sense of potential mechanisms and omits aspects of context. A further refined version of this visualization may be used in addition to some open-ended questions as part of consultation with community representatives, to both solicit community perspectives, but also ‘test’ this initial sense of what may be important in community engagement. Depending on how plans for the community consultations develop, it may or may not be that the use of such a visualisation is considered to be helpful. We include it here to provide a simplified illustration of some of the expected key relational mechanisms at the outset of the review.

Further development of programme theory will be iterative and incorporate insights from discussions within the project team and advisors, and findings emerging from review of the literature. Regular meetings will be conducted with the aim of challenging, sense-making and synthesizing a range of different elements into particular configurations of context, mechanism and outcome, and integrating these into an overall programme theory.

Step 2: searching for evidence

We anticipate that searching for evidence will involve literature searches in up to three broad categories. The first search will identify literature that describes or discusses the common current practice of community engagement in large research programmes, focusing on clinical trials (for example, mass antimalarial administration and malaria vaccine trials) in LMICs as defined by World Bank criteria. This first search will include a supplementary search based on citation chaining of key international ethical guidance and policy documents on community engagement with clinical trials. If more relevant data are needed then our second search will involve identification of literature that describes community engagement practices in other research paradigms and across disciplines. In addition, the searching will be flexible and if needed will incorporate relevant studies published up to the completion of the review. This search will initially still focus on community engagement in malaria, but we will loosen this requirement if additional relevant data are still needed for programme theory development and testing. Finally, if needed, the third search will involve identification of literature that discusses failure, challenges and problems in community engagement. In addition to searches of electronic bibliographic databases, techniques including forward and backward citation chaining, and methods to identify ‘kinship’ or ‘sibling’ studies of relevant documents will also be employed after the screening to help identify the relevant materials [51].

The goal of such a search strategy is to identify adequate literature that can further inform the development of a more detailed programme theory. The process of designing, piloting and conducting the formal searches will be conducted with the support of an information specialist (CD). Modifications and adaption of search strategies following the pilot will be documented and implemented across source types.

The following electronic bibliographic databases will be searched: MEDLINE, Embase, Global Health, CINAHL, The Cochrane Library, the Web of Science Citation Indexes (Core Collection), Scopus, the Global Index Medicus, IBSS and ASSIA. Additional databases identified by the information specialist may be added later. Informed by the initial programme theory developed in step 1, each search strategy will be built around two main concepts: malaria and community engagement. Additional terms will be added to identify literature in the three categories described above. A comprehensive set of free text and subject heading terms will be used to identify the relevant documents. Search terms will be chosen based on key documents identified by the project team and wider content expert groups, discussion in these groups, and the initial programme theory. The searches will not be limited by the date and will be limited to English language. Searching is underway, and the search strategies employed for the first of the planned searches are presented as Extended data [58].

All screening will be undertaken by BA. RV will screen a random sample of 10% of records to support discussion and refinement of inclusion criteria. Disagreements and consistency between these two reviewers will be discussed amongst themselves first and with the project team members if and when necessary.

Initial screening will be conducted based on the title, abstract and keywords. We will use following inclusion criteria to determine if a document is likely to contain the relevant data:

Community engagement in malaria research. By community engagement, we are predominantly referring to the range of strategies undertaken alongside research, for example meetings and discussions with the stakeholders, and training and devolvement of responsibilities to community volunteers.

Document type: all study designs and documents that may contain relevant data.

Types of (participants) studies: documents that include research focused on community engagement or community engagement embedded in any malaria related research.

Types of intervention: community engagement conducted alongside research to promote research and ethical outcomes or relevant case studies of community engagement in long established research institutions.

Outcome measures: both intended and unintended outcomes of community engagement will be explored, for example: (1) research outcomes - recruitment, support for research programme and higher quality research; and (2) ethical outcomes - identification of non-obvious interests, acting on stakeholders’ feedback, and enhanced legitimacy of research.

During the screening process, documents will be excluded based on their content using following criteria:

Research documents which have only briefly mentioned community engagement but with no further details on how the community engagement was conducted, what it entailed or related outcomes.

Additional searching

As the aim of the realist review is to include a broad range of documents to further inform the development of the programme theory, where needed we will look across disciplines, outside of malaria and in different research paradigms, particularly in relation to the exploration of mechanisms for community engagement to produce both intended and unintended outcomes of interest. For example, we may undertake additional searches as the programme theory develops for a number of reasons: to fill in evidence gaps; develop understanding of potential mechanisms; and borrow analogies or theories from other relevant disciplines.

Step 3: document selection

Documents included from screening of titles and abstracts will be considered for selection into the review. The full text of documents initially screened into the review will be further sorted and selected for inclusion primarily based on two criteria: 1) relevance in terms of how and to what extent they can contribute to the programme theory development and refinement; and 2) rigour, which refers to the credibility and trustworthiness of methods used to generate the data [45]. Documents pertinent to community engagement in clinical trials of malaria or programmes will be initially prioritized for inclusion and analysis. Other health research with relevant information on community engagement will also be subsequently incorporated based on its’ potential to strengthen our understanding of community engagement processes. These papers will be categorized as having potentially major or minor contributions to answering the research questions.

Major contributions include:

Documents which contribute to answering the research questions and conducted in the field of clinical trials related to malaria in LMICs

Documents which contribute to answering the research questions but are not focused on clinical trials; for example, descriptive account of community engagement strategy in a research institution.

Documents which do not focus on malaria, but health research, for example HIV, tuberculosis or Ebola research, which can significantly inform our review in terms of understanding the processes and mechanisms.

Minor contributions include:

Documents which report community engagement in high income countries, non-clinical trial contexts, for example development science, but where the mechanism could plausibly operate in the circumstances of LMICs.

This process, together with the developing discussions around the literature, will enable reviewers to focus on data extraction and analysis of papers that provide a conceptually rich contribution, while still including documents that are less conceptually rich.

Step 4: data extraction

Following the methods outlined in a previous realist review [59], data extraction will take place in two stages. At first, the selected documents including their characteristic details will be extracted into a table. This will provide a descriptive account of the documents included. In the second stage, all the selected documents will be analysed using NVivo. Extraction of data will be undertaken first by BA and will be independently reviewed by RV for consistency and refinement of the codes. Discussions will be held with the core project team members for consistency of codes, interpretation and the (interim) findings. Discussions will be held with the wider project team when there are disagreements within the project team which cannot be resolved. Any discussions and their outcomes among core team members and the wider project team members will be recorded.

Step 5: data synthesis

The main aim of the data synthesis in realist review is to develop and then confirm, refute or refine parts of the programme theory. The initial programme theory will be further developed by drawing on the data found within included documents. Analysis of the data will entail using a combination of various methods of reasoning that includes a deductive approach, in which the codes used to code data are based on the initial programme theory, an inductive approach in which codes will emerge from the documents reviewed and a retroductive approach, where inferences are made based on interpretations of the data contained within included documents of underpining mechanisms.

Analysis will following the process set out by Papoutsi et al. [60]. Primary reviewers BA and RV will develop initial CMOCs and these initial CMOCs together with the emerging CMOCs will be discussed amongst the core team for validation and refinement.

The review will follow the standard Realist and Meta-Review Evidence Synthesis: Evolving Standards (RAMESES) guidelines on quality and reporting [61].

Step 6: refine programme theory

The final step in realist review is the refinement and validation of the programme theory [46]. To ensure that the final programme theory makes pragmatic sense, experts and practitioners of community engagement will be consulted to inform final refinement of the programme theory [62, 63]. Incorporating the inputs of community members and community engagement practitioners (field staff) will help us to make it practical and realistic. We also aim to hold an international validation workshop approximately 15 months into the review, in order to validate the findings but also to gather input to help develop tailored versions of the findings for a number of different audiences, including funders and policy-makers and global health research managers and engagement practitioners [46].

Towards the end of this process of refinement, the review team will revisit parts of the review that require re-scrutinizing. This process will be continued until no new information is provided by the evidence or stakeholder involvement, essentially reaching theoretical saturation [46].

To our knowledge, this is the first realist review aiming to synthesize evidence and produce conceptualizations on community engagement with health research in LMICs.

A realist review should enable us to understand the complexity of community engagement and its outcomes in diverse contexts. The programme theories developed through the process should be relevant across contexts.

Stakeholder engagement during programme theory development will ensure a range of perspectives inform the review and support the relevance and uptake of the findings.

The breadth and diversity of the community engagement literature remain a challenge. Our review will use community engagement with malaria research as a pathfinder topic and will draw on wider literature on community engagement where necessary.

Ethics and dissemination

Ensuring that the outputs from this review are useful to the community engagement practice in health research is a key priority for us. Aligning with this value, we will produce relevant and appropriate outputs that target a range of audiences, in conjunction with stakeholder consultation:

Academic forums: we aim to publish in a high-impact peer-reviewed journal including sharing our work in relevant academic conferences where inter-disciplinary scientists attend.

Plain English summaries and briefing documents: apart from the academic outputs, we will produce documents which will be simplified for a range of particular audiences, including for a non-expert audience, with an aim of maximum uptake and dissemination of our evidence.

Study status

The study has piloted the preliminary search strategy and the number of studies resulting from this initial search are undergoing screening and has not been completed yet.

Data availability

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Harvard Dataverse: Replication data for a realist review of community engagement with health research. https://doi.org/10.7910/DVN/DZ4B2Z [58]

This project contains the following underlying data:

Appendix 1.docx (database search strategies for preliminary searches)

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

I have read the authors' responses to each of my concerns, and I believe they have given them thorough consideration and addressed them sufficiently in the revised version of the submission.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

I am happy with the revisions that have been made.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

This is a very interesting study protocol and the research that ensues will make a positive contribution to the field.

I have several comments however some of which relate more to the study design, hence may be difficult to change but aspects could be used to improve the paper and possibly refined the methodology.

I think that the rationale for using a realist approach needs to be summarized earlier in the paper. Some detail in provided in the methods section however terms relevant to realist evaluation are used earlier in the paper, hence need to be defined at an earlier stage. It would be good to have a text box providing information about this in addition to the Figure 1 example of a mechanism. With reference to this, it can be difficult in practice to distinguish between an intervention/activity and a mechanism. This highlights the importance of providing a full explanation of realist evaluation theory at an early stage of this paper.

In your aims and objectives you state that you will develop a programme theory. This needs to be explained. Normally each intervention/CE strategy would have there own programme theory that accounts for the context, the problem, what change is be activated and how this will be measured in terms of outcomes. If I understand correctly you are trying to develop a theory for CE strategies and maybe seeking to identify what are the main mechanism that support 'effective' CE in different contexts. Again mechanisms may vary significantly across contexts, cultures and also be affected by the institutional cultural of the organisations involved.

Could you justify more clearly why you are focusing on malaria research. This may have just been a pragmatic decision but please clarify.

The stakeholders involved are mainly based at international health research centers - I wonder whether this may result in this work focusing more on CE in these settings rather than more broadly. It is great that you have representation across different institutions, continents and countries, but some areas of the world are missing. This could be due to you focus on malaria research. Thinking about Haiti in the Caribbean and parts of South America.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Dear Tracey,

We are grateful to your suggestions and comments. Your suggestions were very helpful and we believe that these suggestions have improved our manuscript significantly.

We have added our responses below with relevant corresponding changes in the manuscript.

We look forward to your kind consideration.

Sincerely yours,

On behalf of co-authors,

Bipin Adhikari

This is a very interesting study protocol and the research that ensues will make a positive contribution to the field.

I have several comments however some of which relate more to the study design, hence may be difficult to change but aspects could be used to improve the paper and possibly refined the methodology.

I think that the rationale for using a realist approach needs to be summarized earlier in the paper. Some detail in provided in the methods section however terms relevant to realist evaluation are used earlier in the paper, hence need to be defined at an earlier stage. It would be good to have a text box providing information about this in addition to the Figure 1 example of a mechanism. With reference to this, it can be difficult in practice to distinguish between an intervention/activity and a mechanism. This highlights the importance of providing a full explanation of realist evaluation theory at an early stage of this paper.

AUTHORS: Thanks for the comments. We have added an explanation of what a realist programme theory is and tried to explain more clearly the initial and provisional nature of our programme theory visualisation. The added glossary of terms also aims to bring additional clarity to key realist terms.

AUTHORS: Thanks for the comment. Initial scoping of the literature suggested that there may be a smaller set of relational dynamics that underpin the much wider variety of engagement interventions and activities. We do expect the mechanisms to have some relevance across context but also, as is key to realist reviews, aim to ascertain where particular contexts make a difference to how they do or do not come into play and in what combination

In your aims and objectives you state that you will develop a programme theory. This needs to be explained. Normally each intervention/CE strategy would have there own programme theory that accounts for the context, the problem, what change is be activated and how this will be measured in terms of outcomes. If I understand correctly you are trying to develop a theory for CE strategies and maybe seeking to identify what are the main mechanism that support 'effective' CE in different contexts. Again mechanisms may vary significantly across contexts, cultures and also be affected by the institutional cultural of the organisations involved.

Could you justify more clearly why you are focusing on malaria research. This may have just been a pragmatic decision but please clarify.

AUTHORS: Malaria is both an important are of health research in its own right, and a key area of practical experience for members of the team. But as the reviewer indicates, it is also a way, pragmatically, to focus the review initially in order to make it manageable and achievable.

The stakeholders involved are mainly based at international health research centers - I wonder whether this may result in this work focusing more on CE in these settings rather than more broadly. It is great that you have representation across different institutions, continents and countries, but some areas of the world are missing. This could be due to you focus on malaria research. Thinking about Haiti in the Caribbean and parts of South America.

AUTHORS: Thanks for the comment. Indeed, the initial focus of the work is on large programmes of international health research given the bulk of engagement work undertaken in this type of setting. We hope this will allow an understanding and assessment of some of the claims being made for engagement in what is a predominant context for engagement in global health research. As noted in responses to other reviewers, there will be both benefits and limits to this particular focus which we will aim to clearly document. We also aim in our Search B to look beyond this particular setting to consider how engagement, and the mechanisms and contexts involved may differ. The point about stakeholder relationships is an important one, and we will seek to systematically seek input from different geographical regions as we develop the stakeholder network.

I am enthused about the prospects of conducting the realist synthesis on community engagement. I think the field can benefit from such a study. The team has the right mix of competencies to successfully complete a realist synthesis of a complex area such as community engagement.

Despite my enthusiasm for such a realist synthesis, I am not fully convinced that the approach as proposed will generate useful knowledge to help plan more meaningful community engagement for global health research. Key concerns that I have include the following points: Comments on Rationale

Defining ‘outcomes’ from a community lens: Perhaps my primary concern is that the paper provides very little focus on defining what “outcomes” actually mean. There is little to no discussion on what outcomes would be measured to understand benefits to the community. This is a significant oversight. The promise of a realist perspective in this study is its potential to generate knowledge of the context and mechanisms that might be associated with community engagement. Because the proposed protocol makes little effort to define outcomes from the perspective of the community, I am a little concerned that this study might not generate useful, actionable information.

Understanding of the community engagement process: I found very limited discussion on what is community engagement. The process of community engagement is not discussed. I would have ideally preferred a little greater clarity on the dynamic nature of community engagement. As it presently reads, the construct of community engagement is under-theorized.

Towards deeper understanding of the multiple shades of community engagement: Given the formidable complexities that might underlie community engagement, I am sensitive to the challenges of understanding the CMO configurations underlying community engagement. I would recommend a more thoughtful discussion on why community engagement around research in malaria might be a little different than community engagement for other goals (for example, community engagement for gender empowerment). I found the discussion around the concept of community engagement quite mechanical.

A more explicit focus on LMIC contexts: Given that the research is being doing in LMIC settings, I did not find a discussion about the challenges of conducting research in LMIC setting. I found myself wondering which specific contexts in such settings will be explored. In what ways are the mechanisms associated with community engagement different in LMIC contexts?

The need to more explicitly focus on values: Underlying much community engagement is a set of values. Community engagement can be quite exploitative or can be very respectful recognizing that the community is the seat of the decisions. The present version of the protocol takes a very instrumental approach to community engagement. I was somewhat surprised by an absence of any discussion around values in a realist synthesis protocol on community engagement.

Taking unintended outcomes seriously: There is a claim that the review will explore unintended outcomes, but there is very little discussion on how the authors plan to explore this. Most evaluations that might form the basis of such a synthesis do not often report on unintended outcomes; in this light it’s not clear how the authors intend to report on a synthesis of unintended outcomes.

The need for prioritization: Given the complexity of community engagement, wide varieties of CMO configurations might be possible. In the absence of clear prioritization, the complexity of having many CMO configurations with potentially limited evidence (across the multiple configurations of contexts and mechanisms) needs to be more explicitly discussed.

Interrogating the theory of change: Much of the theorizing is from a researcher’s perspective. The review presently does not have a strong plan to incorporate a community perspective to challenge, interrogate and refine the program theory. For example, I would have liked the authors to more clearly discuss how the community group members within the Wider Stakeholders Group can be utilized to refine the theory of change.

While the rationale for the study is well described, it’s unclear in the absence of information on the outcomes how the proposed realist synthesis plans to achieve the objectives of the study. As pointed out earlier, the study also suffers from taking very much of a researcher’s perspective on potential benefits of research without sufficiently incorporating community members in either clarifying what community engagement means or in refining the theory of change.

Appropriateness of Study

While there is much to be enthusiastic about the application of realist synthesis in understanding community engagement, many aspects of the proposed approach need to be further refined. The lack of a clear prioritization of CMO configurations, under-theorization of the dynamic nature of community engagement, and an absence of the discussion of the LMIC context that might be appropriate reduce my confidence in the proposed approach’s ability to refine and develop a useful program theory for community engagement. I would recommend focusing on some of these points.

Details of the methods

As it presently reads, the details of the realist synthesis follow a general cookie-cutter approach and the complexity of conducting a realist synthesis for a complex area like community engagement especially in the domain of malaria research is not sufficiently considered. I also would argue that the Figure 3 as presented is not sufficiently clear as the starting point for developing an initial program theory.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Dear Sanjeev,

We are grateful to your suggestions and comments. Your suggestions were very helpful and we believe that these suggestions have improved our manuscript significantly.

We have added our responses below with relevant corresponding changes in the manuscript.

We look forward to your kind consideration.

Sincerely yours,

On behalf of co-authors,

Bipin Adhikari

I am enthused about the prospects of conducting the realist synthesis on community engagement. I think the field can benefit from such a study. The team has the right mix of competencies to successfully complete a realist synthesis of a complex area such as community engagement.

Despite my enthusiasm for such a realist synthesis, I am not fully convinced that the approach as proposed will generate useful knowledge to help plan more meaningful community engagement for global health research. Key concerns that I have include the following points:

Perhaps my primary concern is that the paper provides very little focus on defining what “outcomes” actually mean. There is little to no discussion on what outcomes would be measured to understand benefits to the community. This is a significant oversight. The promise of a realist perspective in this study is its potential to generate knowledge of the context and mechanisms that might be associated with community engagement. Because the proposed protocol makes little effort to define outcomes from the perspective of the community, I am a little concerned that this study might not generate useful, actionable information.

AUTHORS: Thank you for highlighting the importance of outcomes from the perspective of communities being engaged. As noted in our response to reviewer 3 we may have not made clear enough how the search strategy in the review aims to access and understand such potential outcomes. While our initial searches (search A) focus on large health research trials, and these may pick up community input into research protocols, informed consent processes and benefits of research participations, community perceptions of researchers, there may still be little emphasis in this setting, on more emancipatory outcomes for the community. Our search B aims to help elucidate mechanisms around community empowerment by looking at examples beyond common current practice in large clinical trials. However, this needs to be more clearly brought out in the paper as we have done in the introduction to the paper and in the section on review aims on page 4. It remains the case however that much community engagement with research proceeds as if the research is going to be done anyway and in a more instrumental fashion, and we feel it is important to understand the dynamics and impacts of this currently predominant practice.

I found very limited discussion on what is community engagement. The process of community engagement is not discussed. I would have ideally preferred a little greater clarity on the

AUTHORS: On page 3 we attempt to summarise some of the range of activities that appear under the umbrella of community engagement, and we also highlight that there may be a smaller sub-set of relational mechanisms underpinning this diversity. In this sense we have indicated some fruitful lines of enquiry. However, these are often under-theorised in the present literature, which is one of our rationales for conducting the review. There are undoubtedly complex relational dynamics involved in engagement, some of which we have signalled in our visualisation in Figure 3. In addition, there is a prevailing ambiguity in the way engagement is presented, as to whether and how responsive the overall research process is to community influence and input. We expect to make contributions to better conceptualisation of engagement and understandings of the relational mechanisms underpinning it by conducting the review and to clarify how engagement works in practice.

Given the formidable complexities that might underlie community engagement, I am sensitive to the challenges of understanding the CMO configurations underlying community engagement. I would recommend a more thoughtful discussion on why community engagement around research in malaria might be a little different than community engagement for other goals (for example, community engagement for gender empowerment). I found the discussion around the concept of community engagement quite mechanical.

AUTHORS: The review aims to focus on malaria as an initial pathfinder topic, both to draw on the considerable experience of the team and our content experts in this area, but also to make the review accomplishable, given the complexities noted by the reviewer. Indeed we expect there to be differences between engagement around malaria, and other diseases, such as HIV for example, as well as other initiatives with more of a social development focus. The role of stigma and power relations for example, may be less prominent within malaria research than within some other settings. The focus on engagement with health research, and the fact that engagement may have been typically deployed in a more instrumental way in this area, will need to be transparently flagged in the findings of the review. However, as noted above, we also aim in our search B to elucidate some broader engagement dynamics by drawing on literature from different disciplines and settings as far as is possible within the time and resources constraints of the review.

Given that the research is being doing in LMIC settings, I did not find a discussion about the challenges of conducting research in LMIC setting. I found myself wondering which specific contexts in such settings will be explored. In what ways are the mechanisms associated with community engagement different in LMIC contexts?

AUTHORS: It is because we expect there to be particular characteristics of large health research programmes in LMICs that we chose to focus on such settings, in order to help clarify some of the contextual features that may have an important impact on engagement mechanisms, which may be quite different from those brought into play by research conducted in high income countries. In particular, the influence of colonial or neo-colonialism relations may be an important and distinctive part of LMIC settings, potentially demanding more comprehensive engagement processes. The LMIC focus is also partly about where the focus of the team’s ongoing work resides, In addition, it provides one way to focus the review to provide a manageable analysis process. Some relevant contextual features associated with LMIC settings that have been documented in the literature are highlighted in the background section on pages 3 and 4, but we await the review analysis to fully understand the influence of such factors and if and how different mechanisms may be implicated in LMIC settings.

Underlying much community engagement is a set of values. Community engagement can be quite exploitative or can be very respectful recognizing that the community is the seat of the decisions. The present version of the protocol takes a very instrumental approach to community engagement. I was somewhat surprised by an absence of any discussion around values in a realist synthesis protocol on community engagement.

AUTHORS: Much of the CE literature is aspirational, and it is not always clear at what level the values and principles being invoked actually affect practice. There may be an important role for making values explicit in community engagement, something which we hope the review will draw out if it is significant.

There is a claim that the review will explore unintended outcomes, but there is very little discussion on how the authors plan to explore this. Most evaluations that might form the basis of such a synthesis do not often report on unintended outcomes; in this light it’s not clear how the authors intend to report on a synthesis of unintended outcomes.

AUTHORS: In the community engagement literature there is often a lack of clear statement of intended outcomes. As is typical in realist reviews, we will review the literature for examples of outcomes whether these are explicitly outlined at the outset or not, and seek to draw on relevant data explaining the contexts in which they have developed, and the mechanisms which may have been implicated. An example is provided by work documenting how the division between between participants and non-participants in vaccine trials in some settings has led to exacerbation of community differences and sometimes fuelled rumours about research (for eg. Gikonyo et al 2008/doi:  10.1016/j.socscimed.2008.02.003). In this way there may be interests that have been affected by research (usually negatively) that were not identified, acknowledged and safeguarded effectively through the community engagement. While unintended outcomes of community engagement, these processes are important to understand for the way they influence planned community engagement process and outcomes. The reviewer rightly points to the challenge that evaluation of community engagement is itself underdeveloped, and that finding unintended outcomes documented may be difficult. This issue of the availability of data on specific types of outcomes that we are interested in is an inherent limitation of literature reviews of any type – we can only analyse what is ‘out there’. As such, whilst we may make try our best to search for data on unintended outcomes, if we are limited by what is available, we will report this to be the case in future publications. However, at this stage of the review, scoping of the literature does indicate that there are data to draw on, and it remains important to consider outcomes beyond those explicitly aimed for in such a diverse and under-conceptualised field.

Given the complexity of community engagement, wide varieties of CMO configurations might be possible. In the absence of clear prioritization, the complexity of having many CMO configurations with potentially limited evidence (across the multiple configurations of contexts and mechanisms) needs to be more explicitly discussed.

AUTHORS: We agree with the reviewer that there is a need for prioritisation. It is this that drove our initial focus on Malaria, in large health research programmes, in LMICs. At the same time, our concern that such a narrow focus might obscure the types of community outcomes such as empowerment, led us to our additional search components outlined above. We have attempted to strike a balance between making the review manageable and not neglecting engagement mechanisms that are outside of the mainstream. We also expect to have to progressively focus our review – as is expected of realist reviews [ http://ramesesproject.org/media/Realist_reviews_training_materials.pdf] . We will do so using a combination of the judgement of the project team’s content expertise, feedback and advice from our stakeholders and/or as directed by the data in the included documents.

Much of the theorizing is from a researcher’s perspective. The review presently does not have a strong plan to incorporate a community perspective to challenge, interrogate and refine the program theory. For example, I would have liked the authors to more clearly discuss how the community group members within the Wider Stakeholders Group can be utilized to refine the theory of change.

AUTHORS: Much of the published literature is from the researcher’s perspective. It is for this reason that we have a community consultation strand, which we expect to inform the programme theory and search strategy (including delving into different research paradigms) as the review progresses. We also aim to draw on the network of practitioners gathered through the MESH website and international community of practice. In addition, some of the published literature does include qualitative work that draws on interviews with a wider variety of research stakeholders, At the same time, there is value in greater clarity about what engagement from a researcher’s perspective actually aims at and actually achieves in practice in the dominant paradigm, in order to contextualise some of the volume of more aspirational claims made in the field.

While the rationale for the study is well described, it’s unclear in the absence of information on the outcomes how the proposed realist synthesis plans to achieve the objectives of the study. As pointed out earlier, the study also suffers from taking very much of a researcher’s perspective on potential benefits of research without sufficiently incorporating community members in either clarifying what community engagement means or in refining the theory of change.

AUTHORS: As noted above, given the diversity and complexity of the field, it is an important step to critically understand how engagement is being defined and developed in relation to some of the outcomes defined on page 3, in a major area such as engagement with large health research programmes in LMICs. Recognising the emphasis of existing literature on large research programmes and researcher’s perspectives led us to include a community consultation component which we expect to inform iterations of the programme theory, validation of emerging findings and review outputs. Recognising that there are always limits to what one review can cover, we believe the way we have designed the review will enable us to gain clarity on ‘common current practice’ around engagement in large health research initiatives, but that our search and review strategy should also enable us to draw out additional engagement and community dynamics that have been developed in other settings and can highlight alternatives approaches and related mechanisms.

While there is much to be enthusiastic about the application of realist synthesis in understanding community engagement, many aspects of the proposed approach need to be further refined. The lack of a clear prioritization of CMO configurations, under-theorization of the dynamic nature of community engagement, and an absence of the discussion of the LMIC context that might be appropriate reduce my confidence in the proposed approach’s ability to refine and develop a useful program theory for community engagement. I would recommend focusing on some of these points.

AUTHORS: We appreciate the reviewers concerns. However, the very issues he has raised are in part related to the need to carry out a realist review – the need being that there is at present a poor understanding of the causal processes for many of the outcomes (of all types) of community engagement. It is potentially problematic to prioritise in advance CMO configurations as doing so may ‘shut down’ explorations of possibly more important causal explanations. We do intent to better theorise community engagement – albeit in the form of a realist programme theory – as the output of our review. We do intend to explore whether or not and what aspects of LMICs function as context within the CMO configurations we plan to develop. In addition, we feel that we have already addressed many other aspects of the issues raised in this comment in our responses above. These include questions raised around prioritisation and focus, around theorisation of community engagement and LMIC context. In this way we feel confident that the review can make an important contribution to developing the field.

As it presently reads, the details of the realist synthesis follow a general cookie-cutter approach and the complexity of conducting a realist synthesis for a complex area like community engagement especially in the domain of malaria research is not sufficiently considered. I also would argue that the Figure 3 as presented is not sufficiently clear as the starting point for developing an initial program theory.

AUTHORS: In some ways this is the nature of a protocol paper - we have to lay out a more generic process which we fully expect to need to be responsive and iterative as the review progresses depending on what emerges from the strands of literature and in response to stakeholder feedback. Figure 3, as noted in response to reviewer 1’s comments, is a collection of elements that are not yet configured and in this sense they do not represent a fully fledge programme theory. The elements gathered have sensitised us to particular issues to be attentive to during the analysis of the literature, including surfacing some potential mechanisms. We also wanted to include this diagram since the process of developing initial programme theory is often left unclear in realist review protocols, but we recognise we do need to be clearer about the exact status of the visualisation.

In Goals of community engagement, the goal of using this method for co-design of health research has been left out. Increasingly there are researchers who are undertaking co-design with communities, the whole approach of participatory research.

Outcomes could also include: community ownership, community empowerment, community leadership, sustainability. These are not considered. It may be that an intended or even unintended consequence of community engagement was a resultant community level outcome not just a malaria research or ethical obligation outcome.

The dissemination activities should also be informed and co-designed with community groups. One example could be the recently formed Civil Society for Malaria Elimination. Dissemination, at least of briefs, in languages other than English would be advantageous for reach.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Dear Maxine,

We are grateful to your suggestions and comments. Your suggestions were very helpful and we believe that these suggestions have improved our manuscript significantly.

We have added our responses below with relevant corresponding changes in the manuscript.

We look forward to your kind consideration.

Sincerely yours,

On behalf of co-authors,

Bipin Adhikari

In Goals of community engagement, the goal of using this method for co-design of health research has been left out. Increasingly there are researchers who are undertaking co-design with communities, the whole approach of participatory research.

AUTHORS: The reviewer usefully points out something that was discussed in our team but which we had not made explicit enough in the narrative. There are as far as we were aware very few examples of the co-design of large clinical trials and health research programmes. Many authors highlight this as an aspiration, and there are elements of research implementation that involve greater or lesser degrees of responsiveness to community concerns and ideas. Scoping for our programme theory suggested that there potentially were examples of outcomes of community empowerment linked to community engagement processes, but not usually associated with major programmes of health research, more often in the settings on international development or community development. It is for this reason that our search A looks at ‘common current practice’ of large clinical trials, while our search B aims to look beyond this to look at other paradigms and disciplines. We are aware that the causal logic and process and key related ‘mechanisms’ associated with community empowerment may not easily be found in the literature relating to engagement with clinical trials, and we may need to look elsewhere to find adequate specification of the relevant mechanisms and related data.

Outcomes could also include: community ownership, community empowerment, community leadership, sustainability. These are not considered. It may be that an intended or even unintended consequence of community engagement was a resultant community level outcome not just a malaria research or ethical obligation outcome.

AUTHORS: Thanks for the comment that there may also be community level outcomes from engagement, some of which we have subsumed under our description of ethical outcomes (though these may not always be intended outcomes according to engagement plans and strategies). We have flagged these potential outcomes, in particular those of community empowerment and community capacity in our amended paragraph on p 3 referring to community engagement outside the settings of large health research programmes.

The dissemination activities should also be informed and co-designed with community groups. One example could be the recently formed Civil Society for Malaria Elimination. Dissemination, at least of briefs, in languages other than English would be advantageous for reach.

AUTHORS: Thanks for the suggestions. We do have a community consultation component of the review, and this includes gathering input on dissemination and outputs. The addition of civil society Malaria forums as relevant stakeholders to develop outputs would be useful. The review process already aims to produce briefings and to support some networking to facilitate dissemination. We do not have the resources to do this in languages in addition to English in the project budget but there may be potential to develop some outputs in other languages depending on potential collaboration with other partners around the planned design of accessible review outputs.

This study protocol describes the undertaking of a realist review of community engagement for malaria research in LMICs. The protocol begins by identifying the need for greater understanding of community engagement and how different contexts shape engagement outcomes and, subsequently, have implications for health research outcomes. It then describes the objectives of the review, and details the search strategies that will be used.

Is the rationale for, and objectives of, the study clearly described? Is the study design appropriate for the research question? Are sufficient details of the methods provided to allow replication by others? Are the datasets clearly presented in a useable and accessible format? Additional Comments

Partly. The rationale for the review is well articulated. Under “Review objective” on page 4, one of the stated goals is “To conduct a realist review to understand the ways in which community engagement contributes to intended and unintended outcomes” – but it is not clear whether the authors mean by "outcomes". Does this refer to outcomes for the health research for which community engagement is being conducted, or outcomes of engagement in general? It is not until page 8 that it is specified that both research outcomes and ‘ethical’ outcomes will be examined. Additionally, it is not clear what is meant by ‘ethical outcomes’ – does this refer to outcomes that enhance the ethical conduct of the research? More clarification would be helpful here regarding exactly how 'outcomes' are being defined in the context of this review.

Partly. The authors are conducting a realist review, which is a method to develop/refine programme theory. However, it is unclear what ‘programme theory’ means in this context: is this a theory of how community engagement impacts health research? In other words, it is not clear what the programme theory will explain, so it is difficult to assess whether a realist review is the most useful study design. A scoping review would have similar elements (e.g. searching through both academic and grey literature; secondary searching of reference lists; consultation with stakeholders for additional inclusions, etc.) but without the goal of building theory, which may be more appropriate here if it is difficult to articulate exactly what the resultant theory is intended to explain.

Partly. There are some inconsistencies and missing information. For example, on page 5, the authors note that “The review will include all the pertinent documents published in English language only.” However, on page 7, they note that “The searches will not be limited by the date and (English) language, although they will be conducted in English.” Please clarify the search strategy and its language limitations. Additionally, location of community engagement should be part of the inclusion criteria for initial screening of literature in order to identify studies based in LMICs, which is a stated goal of the review. Please also specify the definition you are using to determine whether the literature is sourced from an LMIC context (e.g. World Bank definitions of LMICs?)

Not applicable. This protocol describes a literature review which is not yet completed, thus all data have yet to be collected. However, the final dataset (i.e. all literature included in the review) will be fully accessible as a part of the review article once it is completed.

The conceptual distinction between “Outcomes” and “Contexts” is unclear in Figure 1. In the example given, it would seem that suspicion/openness to research is more an outcome than a context. Perhaps ‘Prior research experiences (e.g. historical mistrust due to past abuses/lack of engagement, vs. trust in research due to positive experiences)” would more accurately capture the concept of context here. In the smaller (definitional) wheel, you might specify that context refers to underlying social, political, historical, and economic factors within which engagement interventions are embedded.

It is not always clear what the authors mean by “outcomes” – will the review focus on outcomes of engagement itself, or the outcomes (impact on) the health research for which engagement is conducted? For example, on page 4, 2 nd paragraph: The authors note “To date, however, there is a lack of analysis and explanation of how the factors of local social and cultural context have affected particular outcomes.” Which particular outcomes? Does this refer to research outcomes, or something else?

Figure 3 is very difficult to understand due to a lack of definitions and clear labeling. For example, what do the blue bars signify? What is the difference between “conditions” and “contexts”? How are “interaction qualities” different from “contexts”?

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Dear Suzanne,

We are grateful to your suggestions and comments. Your suggestions were very helpful and we believe that these suggestions have improved our manuscript significantly.

We have added our responses below with relevant corresponding changes in the manuscript.

We look forward to your kind consideration.

Sincerely yours,

On behalf of co-authors,

Bipin Adhikari

This study protocol describes the undertaking of a realist review of community engagement for malaria research in LMICs. The protocol begins by identifying the need for greater understanding of community engagement and how different contexts shape engagement outcomes and, subsequently, have implications for health research outcomes. It then describes the objectives of the review, and details the search strategies that will be used.

Is the rationale for, and objectives of, the study clearly described?

Partly. The rationale for the review is well articulated. Under “Review objective” on page 4, one of the stated goals is “To conduct a realist review to understand the ways in which community engagement contributes to intended and unintended outcomes” – but it is not clear whether the authors mean by "outcomes". Does this refer to outcomes for the health research for which community engagement is being conducted, or outcomes of engagement in general? It is not until page 8 that it is specified that both research outcomes and ‘ethical’ outcomes will be examined. Additionally, it is not clear what is meant by ‘ethical outcomes’ – does this refer to outcomes that enhance the ethical conduct of the research? More clarification would be helpful here regarding exactly how 'outcomes' are being defined in the context of this review.

AUTHORS: Thanks for the comments. We discuss some of the main goals for engagement and their contributions to research related outcomes and ethical outcomes, and give some examples of the range of things articulated in the literature on page 3, (and it is the aim of the review to further sharpen the distinctions being made and review the evidence supporting the claims for engagements contribution to such outcomes). We have better signposted this account of the range of outcomes in the Review Objective sub-section on page 4 and with the amended heading on page 3.

Is the study design appropriate for the research question?

Partly. The authors are conducting a realist review, which is a method to develop/refine programme theory. However, it is unclear what ‘programme theory’ means in this context: is this a theory of how community engagement impacts health research? In other words, it is not clear what the programme theory will explain, so it is difficult to assess whether a realist review is the most useful study design. A scoping review would have similar elements (e.g. searching through both academic and grey literature; secondary searching of reference lists; consultation with stakeholders for additional inclusions, etc.) but without the goal of building theory, which may be more appropriate here if it is difficult to articulate exactly what the resultant theory is intended to explain.

AUTHORS: Thanks for the comments. We have added a definition of what a programme theory is as understood within realist reviews on page 11 of the revised manuscript. Despite the variability of the literature there are also some fairly well-developed claims linked to instrumental outcomes for improvement of research, and for ethical outcomes, including some of those we highlight in our introduction (and as we also we note in our introduction, these outcomes are sometimes conflated). There is also some considerable empirical work in these areas, even while it remains relatively disparate. It is precisely the causal explanations of how engagement process work that we are interested in clarifying within our programme theory, as a way to begin to move the field forward and to underpin more effective evaluation, and this is what makes a realist review the synthesis framework of choice

Are sufficient details of the methods provided to allow replication by others?

Partly. There are some inconsistencies and missing information. For example, on page 5, the authors note that “The review will include all the pertinent documents published in English language only.” However, on page 7, they note that “The searches will not be limited by the date and (English) language, although they will be conducted in English.” Please clarify the search strategy and its language limitations. Additionally, location of community engagement should be part of the inclusion criteria for initial screening of literature in order to identify studies based in LMICs, which is a stated goal of the review. Please also specify the definition you are using to determine whether the literature is sourced from an LMIC context (e.g. World Bank definitions of LMICs?)

AUTHORS: Thanks for pointing out some of these inconsistencies that we need to address. The searches will be limited to the English language. We will restrict the initial searches to LMIC countries and use definition of LMICs as outlined by the World Bank and have added this detail into the paper on page 7. However, where we lack sufficient data to contribute to testing and refinement of particular aspects of the programme theory we may seek to draw on work outside LMIC contexts where it can provide a contribution to development of programme theory, as is common practice in realist reviews

Are the datasets clearly presented in a useable and accessible format? Additional Comments

Not applicable. This protocol describes a literature review which is not yet completed, thus all data have yet to be collected. However, the final dataset (i.e. all literature included in the review) will be fully accessible as a part of the review article once it is completed.

The conceptual distinction between “Outcomes” and “Contexts” is unclear in Figure 1. In the example given, it would seem that suspicion/openness to research is more an outcome than a context. Perhaps ‘Prior research experiences (e.g. historical mistrust due to past abuses/lack of engagement, vs. trust in research due to positive experiences)” would more accurately capture the concept of context here. In the smaller (definitional) wheel, you might specify that context refers to underlying social, political, historical, and economic factors within which engagement interventions are embedded.

AUTHORS: We have provided a glossary to explain what is meant by context and outcomes in a realist review (under methods section). It is also worth noting that this manuscript is a protocol and we have not completed the review yet. At this early stage in the review, we have provided Figure 1 only as an illustrative and speculative example of mechanisms ( http://ramesesproject.org/media/Realist_reviews_training_materials.pdf). In our simplified diagram we wanted to show that the prevailing context of either openness to research or suspicion to research would affect whether the mechanism of willingness to place trust comes into play (these may indeed be due to prior research experiences or historical mistrust, but it is the prevailing context we are signalling here). And in the smaller wheel, we wanted to highlight that what are considered contexts in realist analysis is the sub-set of the setting (often called wider context) that actively influences the mechanism of interest. Social, political economic and historic factors are present in the setting, but the notion of context aims to flag those aspects that are particular salient for the mechanism of interest (see related comment below about Figure 3)

It is not always clear what the authors mean by “outcomes” – will the review focus on outcomes of engagement itself, or the outcomes (impact on) the health research for which engagement is conducted? For example, on page 4, 2

AUTHORS: As noted in response to previous comments, we gave an overall treatment of the diversity of outcomes on page three. In addition, different studies give accounts of contributions to different outcomes and part of the aim of the review is to organise some of these distinctions and gather evidence for the mechanisms that underpin some of them. This is part of the inconsistency that the review is attempting to address. We have flagged this issue of diversity of outcomes at the outset of the paper on page 3, while also identifying some of the documented range. In addition as noted above, we flag this again when mentioning outcomes in the ‘Review Objective’ sub-section on p 4 to remind the reader of this diversity.

Figure 3 is very difficult to understand due to a lack of definitions and clear labeling. For example, what do the blue bars signify? What is the difference between “conditions” and “contexts”? How are “interaction qualities” different from “contexts”?

AUTHORS: Realist reviews are routinely expected to produce an initial programme theory. Such a theory is not expected to be definitive but to provide an illustrative overview of the possible issues and ideas that may need to be considered with the review. As such, any initial programme theory does often lack the precision and clarity of the finalised programme theory expected at the end of a realist review (Reference: http://ramesesproject.org/media/Realist_reviews_training_materials.pdf). We wanted to be as transparent as possible and include the visualisation since this was part of the initial process of identifying programme theory and this is often not made visible in protocol papers. However, the visualisation is more a collection of elements to help us focus searches and analysis, rather than a more strictly defined diagram as is highlighted in the paper. Given the tendency for it to be interpreted as the latter, this does need more emphasis and so we have attempted to do this on p6. In addition (as noted above in response to the previous comments about Figure 1) in Figure 3 our programme theory highlights some things as background ‘conditions’ while others are flagged as likely contextual influences. We have added some clarification on this distinction on page 6

This is a pertinent review in the field of Community Engagement (CE). Employing a realist perspective is a novel and appropriate method for CE. Their review of the literature is well structured and the need for a realist perspective is well justified. The proposed review shows considerable engagement with the current literature drawing robust conclusions. I agree that current literature on CE does not present a coherent evidence base that will enhance understanding of how different CE mechanisms contribute to research outcomes.

It is a very welcome review at a time when it seems now CE has become rather a technical imperative rather than an ethical imperative as it rightly be. I believe the lack of coherent literature has led to researchers relying on random literature which might not lead to desired or intended outcomes.

The methodology proposed is logical and appropriate. The authors have a clear picture of what the review will entail. I do have reservations on the selection of the broader stakeholders from snowballing. Snowballing for stakeholders will ultimately lead to people who have the same mindset and view of an issue and can be extremely biased at most. A more structured/stratified approach is recommended drawing from all schools of thought from within which snowballing may be employed.

Limitations of the review are well articulated by the authors. The difficulty of searching for CE literature as most of the literature is rarely written with the intention of illustrating CE activities/mechanisms but are usually implied in other types of manuscripts, should be underscored.

The review promises to bring clarity and understanding to CE literature.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Dear Rosemary,

We are grateful to your suggestions and comments. Your suggestions were very helpful and we believe that these suggestions have improved our manuscript significantly.

We have added our responses below with relevant corresponding changes in the manuscript.

We look forward to your kind consideration.

Sincerely yours,

On behalf of co-authors,

Bipin Adhikari

This is a pertinent review in the field of Community Engagement (CE). Employing a realist perspective is a novel and appropriate method for CE. Their review of the literature is well structured and the need for a realist perspective is well justified. The proposed review shows considerable engagement with the current literature drawing robust conclusions. I agree that current literature on CE does not present a coherent evidence base that will enhance understanding of how different CE mechanisms contribute to research outcomes.

It is a very welcome review at a time when it seems now CE has become rather a technical imperative rather than an ethical imperative as it rightly be. I believe the lack of coherent literature has led to researchers relying on random literature which might not lead to desired or intended outcomes.

AUTHORS: Thank you for the comments.

The methodology proposed is logical and appropriate. The authors have a clear picture of what the review will entail. I do have reservations on the selection of the broader stakeholders from snowballing. Snowballing for stakeholders will ultimately lead to people who have the same mindset and view of an issue and can be extremely biased at most. A more structured/stratified approach is recommended drawing from all schools of thought from within which snowballing may be employed.

AUTHOR: Thank you for the useful suggestion. In addition to an element of snowballing we are aiming to find stakeholders willing to give input and review interim outputs and iterations of the programme theory from a number of different disciplines (public health, epidemiology, biomedicine, psychology, social science, anthropology, critical community practice….) and a range of different positionings in relation to engagement as a practice (funders, implementers, research managers, engagement practitioners and community representatives). We have amended the paper on page 6 to highlight that we will pro-actively seek to find inputs from this profile of stakeholders beyond our existing networks of contacts, through current literature in different fields.

Limitations of the review are well articulated by the authors. The difficulty of searching for CE literature as most of the literature is rarely written with the intention of illustrating CE activities/mechanisms but are usually implied in other types of manuscripts, should be underscored.

The review promises to bring clarity and understanding to CE literature.

AUTHORS: Thanks