Priority setting for new systematic reviews: processes and lessons learned in three regions in Africa.

Priority setting to identify topical and context relevant questions for systematic reviews involves an explicit, iterative and inclusive process. In resource-constrained settings of low-income and middle-income countries, priority setting for health related research activities ensures efficient use of resources. In this paper, we critically reflect on the approaches and specific processes adopted across three regions of Africa, present some of the outcomes and share the lessons learnt while carrying out these activities. Priority setting for new systematic reviews was conducted between 2016 and 2018 across three regions in Africa. Different approaches were used: Multimodal approach (Central Africa), Modified Delphi approach (West Africa) and Multilevel stakeholder discussion (Southern-Eastern Africa). Several questions that can feed into systematic reviews have emerged from these activities. We have learnt that collaborative subregional efforts using an integrative approach can effectively lead to the identification of region specific priorities. Systematic review workshops including discussion about the role and value of reviews to inform policy and research agendas were a useful part of the engagements. This may also enable relevant stakeholders to contribute towards the priority setting process in meaningful ways. However, certain shared challenges were identified, including that emerging priorities may be overlooked due to differences in burden of disease data and differences in language can hinder effective participation by stakeholders. We found that face-to-face contact is crucial for success and follow-up engagement with stakeholders is critical in driving acceptance of the findings and planning future progress.

Priority setting for health research aims to ensure appropriate and efficient resource allocation and use, especially in resource-constrained areas.

Subregional priority setting activities using a mix of approaches has led to the identification of contemporary, contextual and region-specific health priorities for systematic reviews.

Embedding systematic review workshops helps stakeholders such as researchers, policy-makers, healthcare professionals, programme managers to discuss questions and prioritise topics in an informative way.

Emerging priorities may be missed if only one burden of disease database is used, and self-selection of stakeholders and language differences can be an important barrier in some regions.

A wide range of approaches that acknowledge language and cultural differences should be explored.

Introduction

Stakeholder-informed priority setting for health research helps to identify topical and relevant issues and their accompanying unresolved questions regarding prevention, diagnosis, treatment, prognosis and rehabilitation using an explicit, iterative and inclusive process.1 In addition, it should ensure the efficient use of scarce resources in both the design and implementation of programmes to address relevant healthcare issues for measurable regional and country-wide impact.2 Countries in sub-Saharan Africa are presently facing the twin problems of resource constraints and high disease burden and therefore priority setting is an essential component of resource allocation. Although the region still has high mortality and morbidity from infectious diseases such as HIV infection, tuberculosis (TB), malaria and lately viral haemorrhagic fevers, non-communicable diseases (NCDs) now account for a substantial proportion of those deaths and morbidities.3 Indeed, it is projected that by 2030, NCDs will account for thrice the disability-adjusted life years worldwide and about five times the combined mortality from infectious, nutritional as well as maternal and perinatal diseases in African countries.4 This disproportionate occurrence will place enormous strain on already overburdened and fragile health systems in these countries. As such, evidence on efficient strategies to prevent and manage NCDs are needed to inform policy and practice.

In line with Cochrane’s ‘Strategy to 2020’, several Cochrane groups have undertaken priority setting with a view to a more efficient use of resources for research synthesis, including a mix of formal and informal approaches.5 6 Although these approaches have been successful in identifying high priority topics,7 the views of African stakeholders have not been adequately represented. Consequently, questions relevant to the African context may not have been considered in the list of priority titles for systematic reviews that have emerged from these efforts. Given the dearth of human resources for health, poor health services financing, suboptimal deployment of life saving technology and essential medicines, as well as a complex mix of socioeconomic, cultural, gender and equity disparities across many of Africa’s health systems, it is reasonable to undertake priority setting at this time to answer the accompanying unresolved questions.

Although there is no gold standard framework, there are common principles about what constitutes good practice in setting priorities. These include the use of an explicit process, stakeholder engagement, information management, consideration of values and context as well as having mechanisms in place for reviewing decisions.8 9 To ensure credible outcomes, any specific approach should allow a transparent, inclusive and iterative process.10 The reliance on interpretive assessments using stakeholder (eg, consumers, healthcare practitioners, policy-makers) driven consensus dialogues should lead to relevant questions and subsequent implementation of the findings.11

Cochrane Africa is a network of institutions and researchers committed to supporting the production and use of context relevant priority systematic reviews to inform healthcare decision-making in the sub-Saharan region for optimum impact.12 Cochrane Africa comprises three hubs: West Africa, Francophone and Southern-Eastern (SE) hubs. These hubs have been involved in priority setting efforts in the past.13 14 Recently, hub-specific priority setting activities for systematic reviews took place in three African regions within this network. In this paper, we critically examine the various approaches used, share the results of the scoping of the literature to identify gaps in the evidence base, compare the various processes adopted and discuss the lessons learnt from implementing these independent priority setting activities across the region.

Identifying priorities for new systematic reviews

We conducted tailored priority setting in 2016 in Cameroon (Central Africa); in 2017 in Nigeria, Liberia, Ghana and Sierra-Leone (West Africa) and in 2017 in Kenya and South Africa (Southern-Eastern Africa). Specific approaches undertaken by each hub are described below. They are described separately but there are core elements that are common across the three approaches.

Modified Delphi approach

In the West African hub, a combination of methods, including a modified Delphi process and some of those identified by the Cochrane priority setting methods group,15 were used in an iterative way. The Delphi process involved two rounds of prioritisation where key stakeholders in the first round indicated top priorities via email and in the second round, were asked to add to the list just before stakeholder discussions. The Cochrane methods group identifies authors of systematic reviews as stakeholders in the process hence our decision to involve them. Electronic and face-to-face contacts with key informants were done at various stages of the process across four countries namely Nigeria, Liberia, Ghana and Sierra Leone. The stakeholders were selected based on their involvement with Cochrane activities in their region, links or involvement with disease control programmes, expertise in communicable and NCDs and work in specific disease areas. First, using the global burden of disease (GBD) data for 2016, national priority conditions across the region were identified.16 These were ranked by key informants from the Cochrane network including Cochrane authors and by health policy-makers and managers of disease control programmes across the region based on the importance of the condition and their knowledge of the prevalence and incidence. The key informants were requested to add any other health condition not on the country-specific GBD list but which was considered important. Twenty-nine stakeholders across Nigeria (12), Liberia (11) and Ghana and Sierra Leone (1 each) responded to the initial ranking of priority disease conditions. Stakeholder panels were held in Nigeria and Liberia to formulate questions using the PICO (Participants, Intervention, Comparison, Outcome) tool guided by a framework that focused on prevention, diagnostics, treatment and rehabilitation. This resulted in a list of 164 questions that cut across 4 health categories. An online survey tool was used to circulate the 164 review questions to stakeholders in Anglophone West Africa (Gambia, Sierra Leone, Liberia, Ghana and Nigeria) for scoring. Eighty-eight (63%) out of 140 stakeholders responded to the request to rank priority review questions. Each stakeholder gave a score (0–5) to each question using largely a priori factors namely the prevalence of the condition, the likelihood the condition will cause death or disability, cost effectiveness, the likelihood of the intervention improving health outcomes and feasibility of the intervention. Based on these scores, the review title with the highest score was ranked first resulting in 25 questions being nominated.

Following this, online databases (CENTRAL, MEDLINE (PubMed) and clinical trial registries) were searched in October 2017 for existing systematic reviews and clinical trials addressing identified questions. The search retrieved 3608 records, of which 24 were systematic reviews and 43 were randomised controlled trials. Two researchers independently screened, selected studies for inclusion and conducted data extraction. And discrepancies were resolved by a senior researcher. An analysis of the outputs was done to identify gaps for new reviews or those requiring updates. The process is shown in figure 1.

Figure 1

Priority setting process in West Africa Hub. PICO, Participants, Intervention, Comparison, Outcome. RCTs, Randomised Controlled Trials.

Multimodal approach

This was the approach adopted by the Francophone hub, coordinated from Cameroon. Here, the priority setting process started in 2016, identifying and merging national health priorities with those of the Effective Healthcare Research Consortium (EHCRC, http://www.evidence4health.org). The EHCRC focuses on ensuring relevant and priority healthcare problems such as malaria, TB, child and maternal health conditions in low-income and middle-income countries (LMICs) are addressed in systematic reviews.17 The key steps are as shown in figure 2 and listed below:

Figure 2

Priority setting process in Francophone hub.

Identifying relevant stakeholders (researchers, academics, clinicians, journalists, civil society organisations, ministry of health (MOH), non-governmental organisations (NGOs) and so on).

Creating a list of email and office addresses for selected stakeholders.

Developing an initial (non-exhaustive) list of priorities based on documented causes of mortality/morbidity from the GBD data for Cameroon and funder (EHCRC) priorities (essentially diseases of the poor especially affecting women).

Conducting systematic review workshops for clinicians and other stakeholders to build capacity on the evidence generation process.

Sharing the initial list of priorities with stakeholders with a request to rank them from 1 to 10, with one being the least priority and 10 being the highest priority. A total of 72 stakeholders in the city of Yaoundé were approached. They included 30 clinician/researchers, 26 clinicians, 1 IT expert, 7 researchers, 1 journal editor, 3 MOH representatives and 4 representatives of NGOs. Responses were collected in two ways:

via email; requested within 3 weeks from all stakeholders and

via door-to-door visits; for stakeholders within the city of Yaoundé who did not respond by email (mostly academics, ministry staff, clinicians and members of civil society organisations).

Identifying existing systematic reviews addressing top priority topics, defined by the highest mean rank.

Producing evidence assessments/translations of systematic reviews addressing top priority topics.

Multilevel stakeholder discussion

In the SE hub, the priority setting process involved countries in East Africa, with support from South Africa. The process started with the identification of a specific and often neglected priority field—chronic kidney disease—and approaching relevant decision-makers in that field. These included the Cochrane Kidney and Transplant Review Group (http://kidneyandtransplant.cochrane.org) and the Kenya Renal Association. The SE hub co-conducted two workshops in Nairobi in June and July 2017, with healthcare professionals and other decision makers working in the renal field. During the first workshop, which were attended by 32 participants including nephrology registrars, medical officers and pharmacists, the importance and steps of evidence-based healthcare and systematic reviews were introduced. The second workshop started with introduction to phrasing questions using the PICO tool and the importance of priority setting, whereafter participants were asked to generate questions they feel are important for their local research agenda. Twenty-five participants attended the second workshop, of which 19 had also attended the first workshop. Among the participants were eight physicians undertaking a fellowship in nephrology and a senior policy-maker from the Kenya Ministry of Health. They generated 30 questions which we grouped into the different types of questions (eg, questions on interventions, burden of disease, risk factors and so on). Subsequently, through engagement with two key kidney care specialists in the region, we compiled a short-list of six questions based on their expert opinions. Literature searches in the Cochrane Database of Systematic Reviews and MEDLINE (PubMed) were then conducted in October 2017 for existing reviews answering these questions. A search was conducted for each of the six short-listed questions. After screening titles and abstracts of retrieved records results by one researcher, 11 relevant systematic reviews, including 3 Cochrane reviews, were found. Evidence mapping (using the PICO format) of existing systematic reviews was conducted to identify the gaps in the current literature. After further discussion with experts and the Cochrane Kidney Transplant Review Group virtually, two questions, for which there was an evidence-gap, were identified (see table 1). It was agreed that the question of haemodialysis frequency and duration in patients with end-stage kidney disease was of high priority. An author team was put together and the title registered with the review group. The step by step process is presented in figure 3.

Table 1

Priority topics identified through priority setting processes across the three hubs

CAN hub	Priority topic/Question	Current status
West African	Community-based vs facility-based directly observed intermittent preventive therapy in pregnancy for preventing malaria	Registered with PROSPERO
	Booster dose of Bacille Calmette-Guerin vaccine for preventing tuberculosis in LMICs	Registered with PROSPERO
	Mass deworming or targeted deworming for preventing or treating anaemia or improving nutritional status and cognition in children	–
	Cognitive behavioural therapy plus pharmacological therapy versus pharmacological therapy for vaso-occlusive pain in children with sickle cell	–
	Early screening for preventing breast cancer	–
	Public health education for preventing breast cancer	–
	Hand hygiene for preventing infections in neonates	Registered with Cochrane Neonatal Review Group
	Self-breast examination for preventing cancer	–
Francophone	Effects of structured antiretroviral therapy treatment interruptions in chronic suppressed HIV-infected African adults	–
	Pattern of cardiovascular disease in African patients with chronic renal failure	–
	Cost-effectiveness of Procalcitonin in the follow-up of bacterial infections	–
	Prevalent pulmonary embolism, mortality rate, disease determinants and predictors of mortality following non-cardiac surgery in sub-Saharan Africans	–
Southern-Eastern	In patients with ESRD, what are the effects of different frequency (eg, thrice vs twice per week dialysis) or duration of dialysis (eg, 3 hours vs 4.5 hours) on quality of life and cost-effectiveness outcomes?	Title registered with Cochrane Renal and Transplant Review as ‘Less intensive vs conventional haemodialysis for people with end-stage kidney disease’
	Among adult patients on haemodialysis with intradialytic hypotension, what are the effects of changing dialysate sodium on reducing intradialytic hypotension?	–

CAN, Cochrane Africa Network; ESRD, end stage renal disease; LMICs, low-income and middle-income countries.

Figure 3

Priority setting process in the Southern-Eastern hub.

Priority topics identified

Although the approaches have been different, the outcomes reflect region-specific priorities. Some of the priority questions, titles and topics are listed in table 1.

Processes, findings and lessons learned

Our core objective as Cochrane Africa is to support the production of context relevant priority reviews that can inform policy and practice in the region. In many settings, arriving at what constitutes priority questions has been influenced by locoregional factors including sociocultural factors, available resources, burden of disease, nature of the health system as well as its financing.18 Often, the research agenda and the research priorities are set by global funders and multilateral agencies19 and may appear to limit the inclusion of value and equity issues relevant to those countries. Cochrane Africa’s work has demonstrated that a variety of approaches can lead to successful identification of priority review topics.7 We have summarised the comparisons of the processes, findings and lessons learnt in table 2.

Table 2

Comparison of processes, findings and lessons learnt

Domain	West Africa hub	Francophone hub	Southern-Eastern hub
Focus	Communicable and non-communicable diseases	Infectious diseases and diseases of poverty	Specific disease field—chronic kidney disease
Process	Modified Delphi	Multimodal approach	Multilevel stakeholder discussions
Stakeholders involved	Cochrane (internal) and non-Cochrane (external) including patients and consumer groups	Mainly non-Cochrane, key policy makers, clinicians, researchers	Professional group (Renal Association), specialists in the region and the Cochrane Kidney and Transplant Group
Stakeholder Engagement methods	Modified Delphi involving virtual contacts and face-face group discussions	Electronic, one-on-one contact and group discussions	In-person and virtual group discussions
Feasibility	Fewer or non-existent Cochrane stakeholders in many countries of the region	Language barrier—engagements in two languages,	Single country involvement in some steps of the process and wider consultations subsequently
Value-added	Involvement of patients and consumers. Workshop in systematic reviews	In-person contact of policy makers. Workshop in systematic reviews	Involvement of Cochrane Review Group and workshop in evidence-based healthcare and systematic reviews
Cost	More expensive holding stakeholder discussions across countries—so limited to two only (Nigeria and Liberia)	Cost savings as only stakeholders in Cameroon involved	Cost savings likely as stakeholder discussions for prioritisation of topics held virtually
Outcome	Priority review topics and registration	Translation and dissemination of Evidence Assessments	Title registration of a priority question with the Cochrane Kidney and Transplant Group
Weakness	Fewer or non-existent Cochrane stakeholders in many countries of the region	Language barrier—engagements in two languages, limited	Single country involvement in some steps of the process and wider consultations subsequently

Comparison of processes

Priority setting in resource poor settings of LMICs rely extensively on frameworks that make use of burden of disease data and cost effectiveness analysis, strategies that have been found to reasonably influence policies downstream.20 In the West African and Francophone hubs, these two frameworks have influenced decisions to prioritise particular conditions both in terms of identification of priority diseases and the ranking of questions that emerged from the process. This may have led to the exclusion of other priorities. For example, the use of the GBD data in the initial identification of prevalent conditions in the West African hub instead of data from the national bureau of statistics may have led to the exclusion of conditions such as trauma and mental health disorders. This limitation was resolved by asking the key informants to add any health condition they considered important but not on the original list of disease conditions extracted from country-specific GBD circulated. Furthermore, as the prioritisation process was across some countries of West Africa, GBD data were chosen to ensure consistency across the region. For the Francophone hub, the GBD data were the most up-to-date and reliable database with emphasis on diseases of poverty which was the focus of the funder. For Southern Eastern hub, the field of prioritisation was chosen based on feasibility and prior professional relationship with the renal community.

The processes had to be undertaken in two languages (French and English) in Cameroon. Language barriers pose a substantial challenge to effective dissemination and potential uptake of research evidence. Until recently, most research was only available in English language, thereby limiting access and use. Cochrane now has a multilanguage strategy as part of its translation efforts. It is expected that this initiative will improve access to research publications, increase capacity for use of research evidence and scale up impact in countries where English is not a native language.

Stakeholder involvement is essential and may serve to narrow questions to very critical areas and to save on resources needed for the entire priority setting process. It also enhances uptake of evidence.21 The strategy of engagement of stakeholders may be determined by available resources and in our case varied across the hubs. In the West African hub, the modified Delphi process was used across two countries and involved key consumer groups and patients apart from Cochrane authors, editors and so on. In the SE hub, a key professional group was engaged using face-to-face group meetings and virtually for ranking of priority topics while in the Francophone hub, key policy-makers were involved and engaged virtually (via email) and through one-on-one in-person meetings. Across all three hubs, systematic review workshops were embedded into the process. These enabled introduction of participants to the concepts and relevance of evidence-based healthcare and systematic reviews and allowed for informed discussions and prioritisation of topics. Capacity building efforts such as these are ongoing in the region.22 Stakeholder involvement makes for a transparent process and in an appropriate setting prevents domination of one group over the other.23 Furthermore, engaging a wide variety of stakeholders in the generation of evidence has also been shown to enhance end-user participation as well as identification of appropriate policy opportunities and likely direction of political support. Even in the adhoc policy environment in LMICs, stakeholder participation has a potential to be useful.24 The door-to-door model of stakeholder engagement proved to be useful in the Francophone hub although it required enormous resources to organise and limited the scope of stakeholders that could be reached.

A key concern when embarking on any priority setting exercise is to determine what resources are available for the scope of work ahead. For Cochrane, the involvement of both internal (authors, editors, consumers and so on) and external (policy-makers, clinicians, NGOs and so on) stakeholders in the priority setting process leads to useful partnerships and empowerment and ensures successful outcomes.6 This has been demonstrated across all three Cochrane Africa Network (CAN) hubs. We did not appraise the cost of the entire activity but presume that our use of electronic and face-to-face meetings in the West African and Southern Eastern hubs may have led to some cost savings.

Comparison of findings

The outcomes of the different priority setting exercises were varied. Several priority topics across preventative, diagnostic and therapeutic interventions were identified, and systematic reviews addressing some of these are underway with the relevant Cochrane review groups. The topics also varied because the different hubs had a different focus at the start of the process. Whereas the West African hub focused on a wide range of communicable and NCDs, the Francophone hub focused on diseases of poverty and the South-Eastern hub focused on a single NCD—chronic kidney disease. Owing to language concerns, the prioritisation efforts in the Francophone hub focused on identifying relevant existing reviews to help with the production of bilingual evidence summaries for wider dissemination, access and use by various stakeholders.

Lessons learnt

Lessons have been learnt following an assessment of the implementation of priority setting activities across the Cochrane Africa hubs. This involved examining the various approaches, comparing them, reflecting on the strengths and weaknesses and drawing lessons for future activities. Although the approaches have been different, the outcomes reflect region-specific priorities. The iterative nature of the approaches underscores the need to allow for an open and transparent process. Leveraging existing burden of disease data or other regional or national diseases burden data is a useful step but needs to allow for input from stakeholders on the ground. Focused priority setting on a single disease field and working with key stakeholders identified during the preparatory phase ensures that the outcomes are also focused and acceptable.

Stakeholder involvement across the entire process is a sine qua non for successful outcomes that reflect the health priorities of the regions in addition to enhancing evidence uptake. Indeed, stakeholder engagement and improved understanding of the prioritisation process has been identified as a key indicator of successful priority setting activities.8 9 For optimal impact, across all three hubs, stakeholders were selected based on their affiliation with Cochrane, work in an area of focus and capacity to influence policies. The engagement strategies have varied with the type of stakeholders and have been influenced by availability of resources. In the West African hub, we observed that patients tended to prioritise interventions for disease prevention rather than treatment. Patient involvement has been noted to increase the cost of prioritisation process and the time to reach consensus.25 However, in the West Africa hub, where patients and consumer groups have been involved, we did not study this directly. We think that the cost was not affected but time to reach consensus may have increased presumably because more time was spent to explain some concepts and the process to patient participants.

Although time-consuming and resource intensive, door-to-door meetings to obviate poor access to internet have proven to be useful. Finally, for Cochrane, the involvement of the appropriate Cochrane editorial group remains imperative to ensure a pipeline for conducting new Cochrane Reviews.

Conclusion

Priority setting is an iterative process, with issues emerging over time, and different approaches may be used to elicit region-relevant priorities. Through Cochrane Africa’s priority setting processes, we learnt that emerging priorities may be missed because of the drawn out process, that stakeholder involvement is critical to the success of the priority setting process, that face-to-face contact and follow-up after engaging with stakeholders is important, and that language can be a barrier across the entire spectrum of activities.