Evaluating the effectiveness of the Emergency Neurological Life Support educational framework in low-income countries.

Background: The Emergency Neurological Life Support (ENLS) is an educational initiative designed to improve the acute management of neurological injuries. However, the applicability of the course in low-income countries in unknown. We evaluated the impact of the course on knowledge, decision-making skills and preparedness to manage neurological emergencies in a resource-limited country.
Methods: A prospective cohort study design was implemented for the first ENLS course held in Asia. Knowledge and decision-making skills for neurological emergencies were assessed at baseline, post-course and at 6 months following course completion. To determine perceived knowledge and preparedness, data were collected using surveys administered immediately post-course and 6 months later.
Results: A total of 34 acute care physicians from across Nepal attended the course. Knowledge and decision-making skills significantly improved following the course (p=0.0008). Knowledge and decision-making skills remained significantly improved after 6 months, compared with before the course (p=0.02), with no significant loss of skills immediately following the course to the 6-month follow-up (p=0.16). At 6 months, the willingness to participate in continuing medical education activities remained evident, with 77% (10/13) of participants reporting a change in their clinical practice and decision-making, with the repeated use of ENLS protocols as the main driver of change. Conclusions: Using the ENLS framework, neurocritical care education can be delivered in low-income countries to improve knowledge uptake, with evidence of knowledge retention up to 6 months.

Introduction

The Emergency Neurological Life Support (ENLS) framework was launched in 2012 as a new initiative by the Neurocritical Care Society (NCS).[1] It was designed to help healthcare professionals improve patient care during the first critical hours of a patient’s neurological emergency, using evidenced-based protocols with the aim of reducing overall morbidity and mortality from neurological injuries. In 2015, an updated ENLS course was launched incorporating 14 presentation-led modules designed around neurological illnesses.[2] Each module aims to improve care using standardized management protocols, facilitated with the inclusion of management checklists, and structured communication guides to assist the transfer of information between healthcare professionals.

Prior studies have shown that standard educational courses do not always result in a change in practice, and the Accreditation Council for Continuing Medical Education has highlighted the need for linking educational activities to changes in competence, performance or patient outcomes.[3,4] Consequently, educational providers and healthcare professionals increasingly need to be aware of which strategies are effective to achieve the educational objectives in their respective environments. The ENLS course was designed to be a valuable educational tool to update healthcare professionals regularly managing patients with neurological emergencies, although there has been no formal assessment of knowledge uptake and retention to validate the course framework. Furthermore, the course was initially designed for healthcare providers working in high-income countries (HICs) and upper-middle-income (UMICs) countries, and the applicability for low-income countries (LICs) and lower-middle-income countries (LMICs) is unknown.

Resource-challenged environments of LICs and LMICs face a significant burden of neurocritical illness.[5] Stroke, traumatic brain injury, epilepsy and neuro-infections such as meningitis, encephalitis and cerebral malaria are among the major causes of morbidity and mortality in these countries. The burden of these neurological illnesses is also much higher in lower income countries due to the contribution of poverty, malnutrition, environmental pollution, increasing tobacco use and a sedentary lifestyle.[6,7] Moreover, the impact of neurotrauma is expected to grow as increasing urbanization and the increasing use of motor vehicles are leading to a corresponding rise in the incidence of traumatic brain injury in LICs and LMICs.[8] The education of healthcare professionals and subsequent acute management of these patients is often limited due to constrained resources, limited specialty training, poorly developed trauma systems, lack of access to invasive neuromonitoring and restricted access to neuroradiological diagnostic facilities.

Consequently, we sought to evaluate the applicability of the ENLS framework in resource-challenged environments in a LIC, and the potential impact of ENLS on education during the recent major earthquake in Nepal, as the course and assessments occurred before and after these unfortunate events.[9] Additionally, as the first ever validation study of ENLS, we set out to formally evaluate acquisition of knowledge and decision-making skills after the course, and determine whether participants continued to retain knowledge up to 6 months.

Material and methods

A 1-day ENLS course was conducted at the Tribhuvan University Teaching Hospital in Nepal, 21 February, 2015 (Online Supplementary Appendix 1). Preparatory material was provided to all participants before the course. The study was submitted to the Research Ethics Board of Tribhuvan University Teaching Hospital and determined exempt from review in accordance with Article 2.4 of the Tri-Council Policy Statement 2nd Edition.[10] All targeted physicians received a personalized E-mail to enable them to access the survey website and complete the online questionnaires. Consent for participation was implied by the completion and return of the self-administered questionnaires.

We assessed participants’ knowledge of neurocritical care emergencies and decision-making skills at three time points: prior to the 1-day course; at the end of the course; and at 6 months following course completion to assess the retention/sustainability of new knowledge. A knowledge and decision-making skills assessment question bank, based on the ENLS course material and protocols, has previously been developed by expert course instructors for the first Canadian ENLS course.[11] For the current ENLS course, we created a knowledge and decision-making skills assessment tool using this question bank. The researchers conducted a pilot study to establish reliability for the assessment tool. Due to a small sample size of target users in the pilot study, reliability measures were unable to be determined. The tool evaluated participant knowledge and decision-making skills for 13 neurological emergencies including acute ischaemic stroke, resuscitation following cardiac arrest, intracerebral haemorrhage, subarachnoid haemorrhage, status epilepticus, meningitis and encephalitis, traumatic brain injury, intracranial hypertension, acute non-traumatic weakness, traumatic spinal cord injury, spinal cord compression, approach to the patient with coma and airway, ventilation and sedation. Each test consisted of 26 multiple-choice questions with only one correct answer. The assessment tool was E-mailed to all the participants before they received the preparatory course material, immediately after the course, and 6 months following course completion (Online Supplementary Appendix 2).

The authors surveyed the participants regarding the ENLS course and online educational material at two points in time—at the end of the course and at 6 months following course completion. The questionnaire captured participant demographics, collected data on perceived knowledge and perceived preparedness for neurological emergencies, and preferences for continuing medical education (CME) following the intensive educational course (Online Supplementary Appendix 3). The questionnaires were created using two online survey websites (surveymonkey.com, Palo Alto, CA, and surveygizmo.com, Boulder, CO). Each physician was contacted by an E-mail that explained the purpose of the study and provided a link to the web-based survey. Confidence was assessed using a 5-point Likert scale (1=not confident, 5=extremely confident).

Statistical analysis

Descriptive statistics were used to describe all survey domains, and were reported as medians and interquartile range (IQR) for continuous variables, and counts and frequencies for categorical variables. Ordinal paired variables were compared using the Wilcoxon signed ranks test. All data were analysed with SAS software (version 9.4; SAS Institute, Cary, NC), and a two-tailed p-value <0.05 was considered statistically significant.

Results

Demographics

The ENLS course held on 21 February, 2015 in Kathmandu, Nepal was the first course conducted in Asia (J.C. Hemphill III, personal communication). The course was attended by 34 physicians from several specialties actively involved in dealing with neurological emergencies in daily practice. Twenty-four participants (71%; 24/34) completed the satisfaction survey (Table 1). There was equal representation from residents (38%; 9/24) and staff (33%; 8/24); two critical care fellows from the first Nepalese Critical Care Fellowship programme attended. Most participants undertook their specialty training in Nepal (67%; 16/24), but only 13% (3/24) completed subspecialty training in critical care medicine. Most participants (87%; 20/24) said they had previously cared for critically ill neurological conditions, with many consults occurring in the general medical/surgical intensive care unit (ICU) (42%; 10/24) or the emergency department (42%; 10/24). Most attendees (67%; 12/18) reported receiving 1–4 weeks of formal training on the care of neurocritically ill patients during their residency. Furthermore, most participants received prior training in all the neurological illnesses covered by the ENLS course, apart from traumatic spinal cord injury, where only 32% (7/22) of participants had received prior training (Table 1). Knowledge was mainly acquired through exposure during residency, routine patient care and self-regulated learning or journal clubs (Table 1).

Table 1.

Respondents, demographics

Demographics	Total number (n=24)
Base specialty, n (%)
Anaesthesia	8 (33)
Emergency medicine	2 (8)
Family medicine	3 (13)
Internal medicine	1 (4)
Neurology	2 (8)
Neurosurgery	3 (13)
Other	5 (21)
Base specialty training in Nepal, yes (%)	16 (67)
Subspecialty training in Critical Care Medicine, yes (%)	3 (13)
Current level of training, n (%)
Resident	9 (38)
Fellow	2 (8)
Staff	8 (33)
Other	5 (20)
Number of years at current training level, n (%)
<1 year	10 (42)
1–2 years	6 (25)
3–5 years	5 (21)
5–10 years	1 (4)
>10 years	2 (8)
Care for critically ill neurological conditions, yes (%)	20 (87)
Place where critically ill neurological injured patients are cared for, n (%)*
Dedicated NeuroICU	3 (13)
General medical/surgical ICU	10 (42)
Emergency department	10 (42)
Neurological/neurosurgical wards	7 (29)
Other	1 (4)
Number of specific training weeks within a neurocritical care setting, n (%)†
1–4	12 (67)
5–8	1 (6)
9–12	1 (6)
13–16	0 (0)
4–6 months	3 (17)
>6 months	1 (6)
Prior specific neurological illness training, n (%)‡
Acute ischaemic stroke	14 (64)
Resuscitation following cardiac arrest	14 (64)
Intracerebral haemorrhage	14 (64)
Subarachnoid haemorrhage	13 (59)
Status epilepticus	12 (55)
Meningitis and encephalitis	11 (50)
Traumatic brain injury	12 (55)
Acute non-traumatic weakness	10 (45)
Traumatic spinal cord injury	7 (32)
Spinal cord compression	12 (55)
Approach to the patient with coma	12 (55)
Airway, ventilation and sedation	13 (59)
Methods for acquiring neurocritical care knowledge, n (%)¶
Resident exposure to neurocritical care	15 (65)
Lecture series	7 (30)
Routine patient care	14 (61)
Self-regulated education and journal clubs	13 (57)
Dedicated rotation	6 (26)
Elective rotation	7 (30)
Other	2 (9)

*Each participant could list more than one area in the hospital.

†Total number of respondents for this question was 18.

‡Total number of respondents for this question was 22.

¶Total number of respondents for this question was 23.

ICU, intensive care unit; n, number of responses.

Acquisition of knowledge and decision-making skills

Thirty-three (97%; 33/34) participants completed the pre-course assessment tool, and 18 (53%; 18/34) participants completed the post-course assessment tool. Significant acquisition of knowledge and decision-making skills was shown, with an improvement in paired-test scores from pre-course (median score 38%; IQR 19–46) to post-course (median score 61%; IQR 53–69; p=0.0008). For the 6-month post-course test, responses were received from 94% (17/18 of those surveyed, 50% of the original total group). The median score on the test after 6 months was 46 (IQR 38–65), which was not significantly different from the post-test scores (p=0.16) (Figure 1), and still significantly higher than pre-course performance (p=0.02).

Figure 1.

Knowledge and decision-making skills. Upper and lower edges of the box indicate the interquartile range (IQR), the line inside the box indicates median test score, the whiskers that extend from each box indicate the minimum and maximum test scores. Median pre-course test score 38% (n=33), post-course test score 61% (n=18), 6-month post-course test score 46% (n=17). *There was a significant improvement in test scores comparing paired data from pre- and post-course tests (p=0.0008). **There was a significant improvement in test scores 6 months after the course compared with pre-course data (p=0.02).

Changes in participants’ perceived knowledge and preparedness

Overall, the majority (95%; 20/21) of respondents strongly agreed or agreed that ENLS training prepared them in the management of neurocritically ill patients within the first hours of an emergency. This satisfaction level remained at 6 months, with 92% (12/13) continuing to strongly agree or agree (Online Supplementary Appendix 4). When assessing satisfaction with knowledge and skill acquisition for each of the modules, most participants found the ENLS course provided the knowledge and skills required to care for neurological emergencies in the critical first hours; this was similarly reported at 6 months (Figure 2). Four modules were identified where there was respondent disagreement or strong disagreement that ENLS provided adequate knowledge and skills (status epilepticus, resuscitation following cardiac arrest, acute non-traumatic weakness and airway, ventilation and sedation; Figure 2). The assessment of satisfaction with the breadth of information covered by the online material and on-site 1-day course showed similar levels of dissatisfaction in three of these four modules (status epilepticus, resuscitation following cardiac arrest and acute non-traumatic weakness; Figure 3), but overall participants were either very satisfied or satisfied with the breadth of information covered in the course. Before participating in the ENLS course, the level of confidence in managing neurological emergencies was either neutral, uncomfortable or very uncomfortable in half of those surveyed (Figure 4). After completion of the ENLS course, most participants were either very comfortable or comfortable managing all 13 neurological emergencies (Figure 4).

Figure 2.

(a) 1 week post-course: the ENLS course provided the knowledge and decision-making skills required to care for neurological emergencies in the critical first hours. (b) 6 months post-course: the ENLS course provided the knowledge and decision-making skills required to care for neurological emergencies in the critical first hours. AIS, acute ischaemic stroke; HIBI, hypoxic–ischaemic brain injury and resuscitation following cardiac arrest; ICH, intracerebral haemorrhage; SAH, subarachnoid haemorrhage; SE, status epilepticus; M&E, meningitis and encephalitis; TBI, traumatic brain injury; IC HTN, intracranial hypertension; Weakness, acute non-traumatic weakness; TSCI, traumatic spinal cord injury; SCC, spinal cord compression; Coma, included approach to the patient with coma; Airway included airway, ventilation and sedation.

Figure 3.

Level of satisfaction with the breadth of information covered in the ENLS course. **Includes assessment of the online material and on-site course. AIS, acute ischaemic stroke; HIBI, hypoxic–ischaemic brain injury and resuscitation following cardiac arrest; ICH, intracerebral haemorrhage; SAH, subarachnoid haemorrhage; SE, status epilepticus; M&E, meningitis and encephalitis; TBI, traumatic brain injury; IC HTN, intracranial hypertension; Weakness, acute non-traumatic weakness; TSCI, traumatic spinal cord injury; SCC, spinal cord compression; Coma, included approach to the patient with coma; Airway included airway, ventilation and sedation.

Figure 4.

(a) Level of confidence in managing neurological emergencies before the ENLS course. (b) Level of confidence in managing neurological emergencies after the ENLS course. AIS, acute ischaemic stroke; HIBI, hypoxic–ischaemic brain injury and resuscitation following cardiac arrest; ICH, intracerebral haemorrhage; SAH, subarachnoid haemorrhage; SE, status epilepticus; M&E, meningitis and encephalitis; TBI, traumatic brain injury; IC HTN, intracranial hypertension; Weakness, acute non-traumatic weakness; TSCI, traumatic spinal cord injury; SCC, spinal cord compression; Coma, included approach to the patient with coma; Airway included airway, ventilation and sedation.

At 6 months, most participants (92%; 12/13) found the 1-day course and protocols very important or important (Online Supplementary Appendix 5), and 77% (10/13) reported a change in their practice and clinical decision-making at 6 months after the ENLS course, citing the repeated use of the ENLS protocols as the main driver of change (Online Supplementary Appendix 6). Protocol use was mainly reported in the management of acute ischaemic stroke, intracranial hypertension and traumatic brain injury (Online Supplementary Appendix 7). All (100%) of the respondents said they would recommend the ENLS course to others, indicated that they would be interested in participating in continuing medical education following the course, and thought it would be useful to develop a tailored ENLS programme for LICs. Feedback from the surveys asked for the inclusion of movement disorders or pharmacological agents as additional modules. Participants thought adapting ENLS for lower income countries, expanding of the course to a 2-day workshop and increasing the interactive nature of the course with hands-on training would improve future courses (Online Supplementary Appendix 8).

Discussion

After successfully implementing the first ENLS course in Asia, our study demonstrates that a 1-day ENLS course improves knowledge and decision-making skills in the management of neurological emergencies, and participants retain this new knowledge at 6 months following course completion. The ENLS course was well received in Nepal, with participants reporting increased comfort in managing this patient population, and appears to be applicable to countries with limited resources. Physicians rated their satisfaction with the course highly, indicating that the learning objectives were met and the impact on both their practices and their patients was positive. Furthermore, participants who completed the follow-up survey at 6 months indicated that the changes they had made in their clinical practice were sustained, citing the use of the traumatic brain injury protocol as a main driver of change during the 6 months post-ENLS at a time during which two disastrous earthquakes hit Nepal.[9]

As efforts increase worldwide to ensure that CME programmes are effective, outcome-orientated and environment-appropriate it is becoming increasingly relevant to measure learners’ outcomes. Recommendations have previously called for the development and implementation of new assessment tools to measure learning and validate educational effectiveness for all CME programmes.[12] Outcome measures for CME programmes are becoming increasingly important for healthcare professionals, educational providers, statutory bodies, the public, and governments.[13] Our study is the first to evaluate the ENLS educational framework by assessing participants’ satisfaction, learners’ outcomes, and transfer of newly acquired knowledge to clinical settings.

As the goal of ENLS must ultimately be to improve patient outcomes, it is important to assess whether the material learned is retained to enable application of the knowledge to clinical practice. Our study revealed that ENLS participants retained new knowledge over a 6-month interval following the course. Participants also reported continual use of the ENLS protocols at 6 months, which probably facilitated participants’ knowledge retention. The use of checklists and protocols has previously been shown to improve adherence to processes of care, decrease complications in a variety of ICU settings and improve patient outcomes.[14-18] As a future, final evaluative assessment of the ENLS course, institutions may want to consider collecting ICU management and outcome data for this specific patient population to evaluate outcomes after the successful incorporation of the ENLS course into residency training programmes that deal with neurological emergencies including critical care, anaesthesiology, emergency medicine, neurology and neurosurgery.

Feedback from the course indicated that participants would welcome the addition of interactive hands-on sessions. Recent data from Hocker and colleagues successfully demonstrated the effectiveness of integrating simulations for neurological illnesses into a critical care fellowship programme in a HIC setting.[19] We suggest ENLS course directors may want to consider incorporating simulation-based learning for neurological emergencies into future courses. In resource-limited settings, access to high-tech/high-fidelity simulation could represent a significant barrier for such implementation, but less expensive and more easily obtainable simulation methods, such as low-tech/high-fidelity or low-fidelity tools, may represent a learning- and cost-effective alternative.

Our study does have several limitations. Like all surveys, our survey was limited by voluntary, rather than mandatory participation, introducing the possibility of selection bias as non-respondents may have provided alternative views on perceived satisfaction level. Similarly, those who completed the knowledge and decision-making assessment tool may be more secure in their knowledge with continued use of the ENLS educational resources, and therefore may be a non-representative sample of those who completed the course. However, our relatively high response rate should minimize this potential bias. Secondly, we did not conduct clinical sensibility testing of the tool, such as reliability and validity.[20] These are essential elements because these describe the quality of the measurement. Questionnaires lacking good psychometric values may not measure the construct they intend to assess, and may hamper the interpretation of results. Future research should be work to establish ENLS-specific knowledge and decision-making assessment tools, with established psychometric properties including construct and content validity, inter-rater reliability, test–retest reliability, and responsiveness.[21,22] These tools could be implemented in all ENLS courses as a quality indicator for the NCS, and enable institutions, using ENLS as an educational tool, to bench-mark their results with a validated assessment tool.

The current ENLS course format may not be optimal for lower income countries due to lack of resources, including human, material and financial, which prevents sustained provision of services. Many of the available technologies and interventions used in HICs and incorporated into the ENLS protocols are not available in LICs and LMICs due to the lack of funding or supporting infrastructure. However, this landmark ENLS course in Asia provides evidence that ENLS is applicable to resource-limited settings and there appears to be substantial demand for a structured educational course on the acute management of neurological illnesses in LICs. Future course development for resource-limited settings should specifically focus on the development of needs assessments for these countries and ethical checklists to endeavour to avoid ethical dilemmas, such as feasibility, sustainability, situational sensitivity and patient safety.[23] Better quality of care can improve health much more rapidly than can other drivers of health, such as economic growth, educational advancement or new technology, and so there is an urgent need to adapt evidenced-based protocols for resource-limited settings to improve clinical practice and thus the quality of care.

Conclusions

This is the first study to show that an ENLS course can be utilized as an effective knowledge translation activity to disseminate evidence-based practice in LICs. Understanding that knowledge retention is one of the critical factors for maintaining sustainable performance, our findings show that knowledge acquired during the course was retained at 6 months. Furthermore, early protocol-driven resource-appropriate education may help to improve patient care and outcomes in resource-limited environments. Future research should focus on validating assessment tools for general critical care courses in low-income settings.

Supplementary data

Supplementary data are available at .

Click here for additional data file.