Strengthening prehospital clinical practice guideline development in South Africa: Reflections from guideline experts.

INTRODUCTION: De novo (new) guideline development methods are well described and supported by numerous examples, including comprehensive checklists. However, alternative guideline development methods, which draw from existing up to date, high quality clinical practice guidelines instead of re-inventing the wheel, have not been adopted so readily, despite the potential efficiencies of such methods compared to de novo development. In Africa, guideline quality and rigour of development, especially for prehospital care, remains poor. This paper firstly describes the opinions of international guideline experts on the African Federation for Emergency Medicine guideline project, and secondly updates a framework for South African prehospital guideline development.
METHODS: We conducted a qualitative study of expert reviews of an evidence-based guideline development project led by the African Federation for Emergency Medicine in 2016 for prehospital care in South Africa. We purposefully sampled key international and regional guideline experts from a range of organisations. Comments and voice memos, following a terms of reference guide, were thematically analysed through manual coding.
RESULTS: A total of seven experts gave feedback. Key themes revolved around existing international clinical practice guidelines not being enough to cover context specific evidence, blurring of guideline responsibilities and output, and transparency of guideline decisions and conflicts of interest. We showcase three fit-for-purpose guideline development approaches and provide an updated alternative guideline development roadmap for low-resource settings.
CONCLUSION: In order to create clinical practice guidelines that clinicians trust and use on a daily basis to change lives, guideline developers need rigorous yet pragmatic approaches that are responsive to end-user needs. Reflecting on the African Federation for Emergency Medicine prehospital guideline development project in 2016, this paper presents key guiding themes to strengthen guideline development in low- and middle-income countries and other low-resource settings and provides an updated hybrid guideline development approach.

Introduction

De novo (new) guideline development methods are well described and supported by numerous examples, including comprehensive checklists [1]. However, alternative guideline development methods, which draw from existing up to date high quality clinical practice guidelines (CPGs) instead of re-inventing the wheel, have not been adopted so readily, despite the potential efficiencies of such methods compared to de novo development [2]. Alternative guideline development methods include a variety of robust approaches, such as the ACA (adopt, contextualise or adapt), adolopment and use of the ADAPT framework [1,3,4]. These have been applied across various topics and disciplines including emergency care, stroke rehabilitation, psychiatry and chronic musculoskeletal pain [5].

However, within emergency care, de novo guideline development methods continue to be predominantly used when developing CPGs. A 2018 landscape analysis of global prehospital CPGs found that nearly 60% of prehospital CPGs were developed de novo, with less than 2% using alternative methods [6]. Guideline quality also varied, with a lack of methodological clarity in 32% of global emergency care CPGs. Furthermore, in sub-Saharan Africa (SSA), a similar scoping review found that 71% of emergency care guidance documents, including clinical care pathways and protocols, failed to report appropriate development methods or reference parent CPGs [7]. In SSA, the majority of emergency care guidance are produced by professional societies (58%), followed by national departments of health (21%) and academic/clinical institutions (19%), reflecting similar trends reported from high-income regions [6]. These trends are seen in other spheres, such as the primary care setting, where a cross-sectional analysis of selected CPGs highlighted guideline quality issues, especially in rigour of development, editorial independence and applicability [8].

Considering the substancial burden of trauma in Africa, it is essential that robust, high-quality guidance is produced and available for prehospital providers [9]. Although most SSA prehospital guidance documents seem to be end-user focused, many lack transparent reporting to support their clinical recommendations [[10], 11., 12., 13.]. This speaks to the urgency of promoting and strengthening the transition from eminence-based to evidence-based guidance for prehospital care in SSA. However, in strengthening both the development and implementation of CPGs in low- and middle-income countries (LMICs), especially alternative guideline methods, significant progress has been made both locally and internationally.

Indeed, many of the methods for advancing alternative guideline development have originated from Africa in the past 5 years [14], with numerous examples and case studies emerging to guide developers [4,5,[15], [16], [17], [18], [19], [20], [21], [22]], including online toolkits [23]. Examples include stroke [22], mental health [5], and emergency care [21], all of which have used alternative guideline development methods that are context-specific and fit for purpose for a LMIC setting. For example, in allied health, Ernstzen et al. [20] describe a four-phased contextualisation framework to produce a multidisciplinary CPG for primary health care of adults with musculoskeletal pain. Sampling patients' and practitioners' perspectives and preferences, they were able to contextualise/adapt recommendations to fit the local setting and needs [20]. Additionally, user-friendly and pragmatic clinical decision tools exist that can be used as templates for adaptation considering the best available evidence, such as those produced by the Emergency Medicine Kenya Foundation [11]. Other guideline development methods include streamlined de novo approaches such as used by the Belgium Red Cross in developing first aid CPGs for first responders in Africa [24], to end-user-centric approaches for developing clinical decision tools for primary care nurses [16]. These methods and examples, among others mentioned, will play an important role in shaping emergency care guideline development where resources are scarce.

In South African emergency care, progress has been made to transition to evidence-based clinical practice guidelines [25] with the African Federation for Emergency Medicine (AFEM) producing the first prehospital CPG for paramedics in South Africa [26]. Recent developments include scoping and appraisal of SSA prehospital guidance documents [7], critical reflections on guideline methods and roadmap of the AFEM guideline development approach [21], case studies [5,27], and exploration of paramedic perceptions to strengthen CPG uptake [28]. These have resulted in key priority actions to strengthen local prehospital guideline development and uptake, enhanced with knowledge translation activities [29]. However, various challenges still exist for guideline developers who use alternative methods, especially in emergency care. These include a lack of high-quality ‘seed’ guidelines to adapt or adopt, challenges in pooling recommendations from multiple guidelines, a complex and shifting implementation context, lack of experience in guideline development groups, and the undue influence of conflicts of interest and beliefs when considering recommendations for implementation [21,[30], [31], [32]]. Furthermore, when developing CPGs using alternative methods, even though these methods focus on implementation readiness, it does not automatically lead to successful implementation despite the availability of useful tools to aid in implementing CPGs [33].

In looking for solutions, a consolidated updated roadmap to successful guideline development and implementation would help strengthen future guideline projects for emergency care in South Africa and beyond, building on previously published challenges, roadmaps and lessons learnt by the AFEM prehopsital CPG project [21,27,28]. This paper firstly describes the opinions of international guideline experts on the AFEM guideline project, and secondly aims to update a framework for South African alternative prehospital guideline development by consolidating expert input.

Methods

Study design

We conducted a qualitative study of expert reviews of the AFEM guideline development project to explore their opinions on methods to strengthen guideline development and implementation, and provide a roadmap and update for future development and implementation of South African prehospital CPGs. We purposefully sampled key international and regional guideline experts, from a range of universities and organisations. We asked them to provide their comments on three AFEM-linked guideline publications [21,27,28] in writing or as a voice memo. The COREQ (Consolidated criteria for reporting qualitative research) statement guided our research reporting [34]. Ethics approval was obtained from the Stellenbosch Faculty of Medicine and Health Sciences ethics committee (S17/03/069). Written informed consent was obtained from participants.

Participants

Key guideline experts were purposefully sampled in order to maximise the diversity of data relevant to the study's aims. We used email to invite participants from a variety of sources, including guideline organisations (i.e. World Health Organisation, Guidelines International Network and National Institute for Health and Care Excellence), academic institutions and evidence-based health care units (e.g. International Centre for Allied Health Evidence, University of Cape Town Knowledge Translation Unit, Centre for Evidence-based Practice), and national (South African) and international emergency care organisation representatives. We sought experts with experience in conducting, developing or implementing CPGs (within or outside of emergency care) or who have published extensively in the field of CPGs, who would be able to provide adequate feedback in the allocated timeframe. We aimed to have a 1:1 ratio of local versus international guideline experts. A relationship was established via email before data collection, where most participants knew the researcher through professional networks.

Data collection and analysis

Participants received a terms of reference pack, which included study objectives and three documents: i) an AFEM CPGs methods paper, reflecting on challenges and lessons learnt [21]; ii) an AFEM CPG qualitative case study [27]; and iii) a study of paramedic guideline implementation perception challenges [28]. We also included a series of semi-structured prompting questions to guide their expert review (Supplementary File).

We collected two types of data from participants: i) written reports; and ii) self-recorded voice memos. Reports and voice memos were sent via email to the principle investigator (MM) and kept in a password-secure location. Voice memos were transcribed verbatim for analysis. Transcribed data were analysed thematically by MM with a deductive approach, based on the AFEM guideline process as an overarching guide [21], through manual coding as described by Erlingsson and Brysiewicz [35]. Themes were discussed among the author team. All transcripts were read as a whole to familiarise the analysts, followed by collapsing verbatim text into condensed meaning units. Next steps involved labelling condensed meaning units by formulating codes and then grouping these codes into categories. Where appropriate, with sufficient data depth, categories were merged into themes and across themes, to create overarching themes. Expert guideline development approaches were presented graphically to provide examples of guideline development.

In reviewing the AFEM methods, experts described and highlighted key approaches to strengthen development and downstream implementation. Through triangulation of previously published challenges, roadmaps and lessons learnt, we incorporated these suggestions into an updated graphical roadmap and tabulated narrative for future guideline development in resource-limited settings, considering the original AFEM methods and challenges described previously [21,28], and drawing from the themes presented in this paper.

Trustworthiness and reflexivity

We sought to ensure that the research process was trustworthy, so that our findings could be considered a credible reflection of participants' reality [36]. We took several measures to establish credibility (i.e. used quotes verbatim, peer scrutiny of the project), dependability (i.e. member checking and review of notes), confirmability (i.e. reflection of research beliefs and assumptions, debriefing sessions) and transferability (i.e. description of study context and participants, used participants terms/concepts in writing), where possible. The principle investigator (MM) has a background in prehospital emergency care and was involved as a methodologist in the AFEM CPGs as a core guideline panel member. During analysis, MM drew from his lived experiences as an AFEM guideline panel member [21,26,37] and past guideline research [23,27,28].

Findings and discussion

A total of 10 participants were invited, with three declining participation due to workload and time commitments. Participants were from both high-income and low-to-middle income countries, and three were from South Africa. Participants ranged from a variety of guideline organisations and backgrounds, from emergency medicine and primary care to allied health; from international guideline organisations to country specific guideline development or research units involved with guideline production; to heads of departments, senior researchers and professors. All experts provided written reports, while one expert provided both written and voice memo reports.

Overview and themes

Six major themes emerged from the data, summarising the various opinions and key considerations of the guideline experts regarding the AFEM guideline project. These are discussed below along with three examples of guideline development, followed by a revised roadmap for alternative guideline development and implementation for South African prehospital care, drawing from major themes and previous work [21].

Using existing international CPGs is not enough to cover context-specific evidence

Experts considered using high-quality international CPGs as an appropriate method of ‘short-cutting the laborious process of searching for evidence’ [Expert 1] compared to producing CPGs de novo. However, experts noted that when working with international CPGs, there is a risk to miss contextual evidence or context-specific interventions typically uncovered during searching for primary studies, resulting in recommendations that may not be deliverable in South Africa or any other setting.

“The understanding that ‘research evidence’ [guidelines] is important but not sufficient and needs to be integrated with other important forms of knowledge, is a key part of guideline development in order to make recommendations work ‘on the ground’.”

Three experts recommended two similar overarching solutions presented below as subthemes, pertaining to using other forms of knowledge and evidence in alternative guideline development processes:

Use local evidence: incorporating and aligning policy, local guidance, and end-user documents

Experts advised using evidence sources such as local policies, local CPGs or guidance documents, essential medicine lists and clinical decision tools such as algorithms, to ensure recommendations ‘[are] developed and grounded in a clinical setting rather than an abstract, generic manner’ [Expert 3]. Indeed, even with de novo methods which use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) framework [38], local evidence around feasibility, applicability, values and preferences and resource use is sourced in order to develop context-specific recommendations. Clinical Decision Support Tools or end-user documents such as Practice Approach to Care Kit 101 [39] for primary health care nurses use local CPGs and policy as the starting point, while supplementing knowledge gaps using an evidence synthesis database (such as British Medical Journal Best Practice) [40], ‘ensuring alignment to local priorities and resources’ [Expert 3]. The overarching premise was explained by Eisenberg (2002) that even when the evidence is abundantly clear, ‘local circumstances dictate how that evidence is translated into practice’, emphasising the consideration of local circumstances when using global evidence such as from CPGs [41]. When and how local evidence is incorporated during guideline steps varied between experts in our study; some used local evidence as the starting point, while others only included local evidence when recommendations were to be adapted from CPGs. However, a common thread among experts was that this step be actioned within a guideline panel process.

Carefully choose the guideline panel: enabling wider consultation

Across experts, the essential nature of a balanced guideline panel was emphasised. Guideline panels allow ‘evidence to be considered and mulled over, debated and developed into context-specific recommendations’ [Expert 2], and is a universal process in guideline development.

‘When it comes to resource-strapped settings, primary research is most often limited, and the input of a good expert panel is especially important.’ [Expert 4]

Experts suggested various stages and methods for involving experts in the guideline development process. However, all agreed that meeting face to face is the ideal format, especially to discuss and find consensus on guideline scope, questions and priority topics. Experts noted that during the AFEM guideline process, it was unclear how the expert panel engaged with the recommendations, whether this was face to face or online. Towards solutions, one expert suggested that for the AFEM CPGs, ‘two types of face-to-face meetings be organised: one with content experts, and one with stakeholders or users’ [Expert 4], to create user buy-in, and further facilitating better implementation by involving stakeholders from the ‘outset, throughout development and during implementation’, as part of a wider and open consultation process [Expert 3].

Blurring of responsibilities, separation of output

One expert was concerned around the impact of the AFEM CPGs beyond providing clinical guidance ‘being used to define limits of professional practice in order to regulate groups of practitioners’ [Expert 1]. Experts noted the CPGs were ‘doing what conventional guidelines do, which is guide clinical practice’ [Expert 5] but also noted that the CPGs were serving other purposes, including setting scopes of practice (who can do what and when), blurring responsibilities where regulators and health authorities should have stepped in regarding guideline implementation, as ‘implementation primary responsibility lies with those who are delivering the service or those who are regulating the delivery of the service’ [Expert 2].

Towards future solutions, one expert advised there should be a clear separation of guideline outputs. Firstly, the guideline team should only produce the clinical guidance and, where feasible, an end-user document; and secondly, regulators or health authorities should produce a scope of practice or other regulatory framework. This would separate clinical guidance from regulatory issues which touch on sensitive areas outside of a CPG team's ambit such as ‘professional identity issues, professional security and lack of clarity on future career pathing’ [Expert 6]. Creating separate outputs would have helped reduce the tangling of perceptions of the evidence-based CPGs with implementation policy and scopes of practice issues, as highlighted by one South African expert:

‘In retrospect, these two areas – clinical care and scope of practice may have been better in two documents. This would have allowed people to engage with them separately.’ [Expert 6]

Heterogeneous methods of heterogeneous evidence classifications

Experts provided conflicting options for dealing with heterogeneous levels of evidence classifications, a common issue when dealing with multiple CPGs, each of which might use a different level of evidence classification. Some suggested a conversion table to align the different classifications systems together with a writing guide ‘to ensure consistent decisions about the combined levels of evidence’ [Expert 7], while others used GRADE EtD or plain language descriptions to differentiate between higher and lower levels of evidence. It was also recognised that determination of the strength of recommendations (e.g. conditional or strong) from different CPGs still requires additional research, as guideline teams often do not report their decision making or context factors that affect the strength of recommendations.

In 2016, the AFEM guideline was faced with more than 50 different evidence classifications found across 264 included CPGs, and took the approach of reporting the original plain language meaning for each classification, described previously [21]. This reduced the workload on the guideline panel, who had more than 1000 recommendations to consider, where merging levels of evidence classifications was not feasible. Alternative classification merging options in the literature include the EtD framework; but this method does not scale well with large numbers of recommendations [42], or the National Health and Medical Research Council evidence matrix [43]. However, a potentially scalable approach was proposed by Grimmer et al, standardising evidence strength grading for recommendations from multiple CPGs, resulting in an overall strength of the body of evidence classification [15]. Further testing is needed to determine scalability and guideline teams with less experience.

Fit-for-purpose clinical practice guidelines: snapshot of three approaches

Three key guideline development approaches used in low-resource settings, are shown in Fig. 1 (where colours indicate development stage namely planning and scope, evidence synthesis or recommendations development). Experts noted that although these approaches have predominantly been used in LMICs, due to typical restrictions in human and fiscal resources these methods are by no means inferior to the typical de novo guideline development methods, or less applicable in high-income settings.

Fig. 1

Three examples of guideline development methods used in LMICs.

The first is a streamlined de novo approach, which streamlines the systematic review process to save time by producing a more focused evidence review with ‘lower sensitivity, which might result in missing some studies, but a balanced guideline expert panel, meeting face-to-face, serving as a backup’ [Expert 4]. To save time, this approach uses one reviewer ‘together with one or two content experts to prepare draft recommendations, which can be discussed during the panel meeting’ [Expert 4]. As an example, this approach was successfully used by the Belgian Red Cross in developing the first evidence-based first aid guideline for first responders in Africa [44].

The second is an approach focused on producing a clinical decision support tool or an end user document (referred as a 3rd generation knowledge product) by drawing from primary studies and systematic reviews (1st generation) and local CPGs and policies (2nd generation). This approach is useful for those who ‘don't have time or resources to develop first generation content de novo’ and for developing an end-user template that can easily be ‘updated, or adapted for in-country localisation to policy, skills and resources’ [Expert 3]. Examples include the Practice Approach to Care 101 for primary care nurses [39].

The third guideline approach is structurally similar to the original AFEM approach, whereby existing CPGs or other forms of guidance are used as the evidence base, and together with a writing guide to ‘amalgamate recommendations from multiple guidelines’ [Expert 1], recommendations are either adopted as is, contextualised (implementation caveats added) or adapted (changed completely) to the local context needs. This approach has been tried and tested in various settings, including the Philippines [45], and in South Africa for stroke rehabilitation [46]. Another expert proposed an inverted guideline development approach, whereby i) all CPGs on a broad topic are identified; ii) recommendations are listed with strength of evidence; and iii) for each target health care facility or service provider, each recommendation is either adopted or adapted with reasons, considering the facility/user setting.

Expanding alternative guideline development methods: balancing rigour with pragmatism

Experts commented that the AFEM prehospital CPGs produced in 2016 provided a ‘balance of rigour with practicality, an impressive task, given the scope of prehospital guidance and a final CPG that includes over a 1000 recommendations’ [Expert 1]. The AFEM adopt, contextualise or adapt approach provided a flexible, pragmatic and cost-effective manner to develop CPGs, of which the ‘clinical evidence-based part of the CPG seemed to be well received’ [Expert 6]. In reviewing the AFEM methods, experts described and highlighted key approaches to strengthen development and downstream implementation. We incorporated these suggestions into an updated roadmap for future guideline development in resource-limited settings (see Fig. 2), considering the original AFEM methods and challenges described previously [21,28], and drawing from the themes presented in this paper, including Fig. 1. The previous roadmap focused on the process of adapting, contextualising and adapting recommendations and lacked further alternative development options [21].

Fig. 2

Hybrid alternative guideline development approach and key considerations.

In conjunction with the guideline development roadmap (Fig. 2), Table 1 describes priority considerations highlighted by guideline experts when reflecting on the AFEM guideline methods. The roadmap (Fig. 2) and considerations presented in Table 1 should be read together, and aim to improve and update existing AFEM CPG development methods and support guideline development initiatives in low-resource settings, especially professional societies in prehospital care.

Table 1

Summary of key guideline steps and considerations in Figure 2.

Guideline development roadmap	Priority considerations highlighted by guideline experts when reflecting on the AFEM guideline methods
Guideline group	Consists of three entities: i) end-user and stakeholders (such as guideline decision makers); ii) expert panel; and iii) working group responsible for evidence synthesis. Working group together with the guideline methodologist support the expert panel with guideline processes.
Needs and contextual analysis	Map and describe the clinical context, considering:• Local resources and gaps• End-user needs and expectations
Priority setting, scope and question generation	Seek multi-stakeholder input for priority setting of questions and linked outcomes (including end-users and policy makers)Keep scope balanced and manageableGenerate a patient pathway to support question generation (e.g. logic framework to place questions in the patient journey)
Summarising and matching best evidence with questions	Systematically search, match and appraise best available evidence. Depending on time/resources, best evidence includes:• High quality, up-to-date CPGs• Systematic reviews• Evidence databases• Context-specific (local) policy and CPGs (2nd generation evidence)
Drafting recommendations for guideline panel consideration and input	For questions with matched CPGs with EtD:• Reassess EtD judgements (adolopment)For questions with matched CPGs with no EtD but with systematic reviews with Summary of Findings tables:• Develop EtD (adolopment)For questions with matched CPGs with no EtD and systematic reviews with no Summary of Findings tables:• Adopt, contextualise or adapt recommendationsFor priority questions with no matched CPGs or systematic reviews, consider a streamlined de novo approach:• Focused questions and evidence search• One reviewer screening and extraction• GRADE and EtD• Draft recommendations for expert panel to considerInput of local evidence to maximise implementation efforts is considered at this stageEnsure transparent, documented decisions for each recommendation
Publishing CPG	Seek formal endorsement by local health and service delivery authorities and organisations. Part of end-user and stakeholder panel input
End-user product	Ideally developed in conjunction with guideline implementers, stakeholders (decision makers and end-users) and the guideline group, ensuring the end-user product is:• End-user driven and tested• Based on parent CPG recommendations• Draws from existing end-user documents and local CPGs, guidance document and SOPs
Standard operating procedures and regulations	Separate and independent output produced by policy makers, health services and regulators, and informs the end-user product
CPG and end-user dissemination and implementation	Responsibility of service delivery stakeholdersUse local champions to support effortsEnsure regular and consistent communicationConsider local end-user needs and expectations

The hybrid alternative guideline development roadmap proposed in Fig. 2 draws from the strengths of both alternative and de novo guideline development methods and is further expanded in Table 1. In recent years, alternative guideline development frameworks have evolved from a focus on identifying source CPGs for adaptation to adapting specific recommendations to examining the evidence underpinning the adapted recommendations [30]. Our approach allows for flexibility regarding where and when the evidence synthesis steps occur, depending on the depth of reporting and quality of recommendations from the seed CPGs. For example, if recommendations have linked EtD summaries or systematic review Summary of Findings tables, then adolopment should be considered [30], which re-examines the evidence underpinning recommendations. If no evidence summaries are available, or when there are multiple recommendations for the same question, the Adopt, Contextualise and Adapt approach, which grades recommendations across CPGs, is a viable option [15] since the underpinning evidence is not readily available. Guideline groups can decide in advance which approach would best work for them, considering the available seed CPGs and their methodological expertise, scope, timeline and fiscal resources.

This updated framework for alternative guideline development for low-resource settings still needs to be evaluated independently, specifically in LMICs, where the needs for adaptive guideline methods is greatest. Our research has a key limitation: we did not conduct in-depth interviews with the experts, which may have provided richer data to expand on complex problems and methods mentioned by experts.

Conclusion

In order to create CPGs that healthcare professionals or healthcare workers trust and use on a daily basis to change lives, guideline developers need rigorous yet pragmatic approaches that are responsive to end-user needs. Reflecting on the AFEM prehospital guideline development project in 2016, we present, in this paper, key guiding themes to strengthen guideline development in LMICs and other low-resource settings. Furthermore, we present three distinct guideline development approaches used in SSA and summarise their approaches. For future guideline projects in LMICs and other low-resource settings, especially for professional societies in prehospital care, we propose an updated alternative guideline development roadmap.

Dissemination of results

The results of this research will be shared via targeted knowledge translation activities including on social media platforms, conference presentations and directly with guideline decision makers.

CRediT authorship contribution statement

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: XX contributed 65%; XY 25%; YY 10%. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Funding

This project was funded by the Harry Crossley Foundation, Faculty of Medicine and Health Sciences, Stellenbosch University and the Early Career Academic Development program.

Declaration of competing interest

MM was involved as a guideline methodologist and evidence reviewer in the original AFEM CPG commissioned via the Health Professions Council of South Africa Professional Board of Emergency Care. TY was involved as a consultant. Neither AFEM nor the Health Professions Council of South Africa Professional Board of Emergency Care were involved with the interpretation or final write up of this manuscript. MM is an editor of the African Journal of Emergency Medicine. MM was not involved in the editorial workflow for this manuscript. The African Journal of Emergency Medicine applies a double blinded process for all manuscript peer reviews.