Novel educational adjuncts for the World Health Organization Basic Emergency Care Course: A prospective cohort study.

INTRODUCTION: The World Health Organization's (WHO) Basic Emergency Care Course (BEC) is a five day, in-person course covering basic assessment and life-saving interventions. We developed two novel adjuncts for the WHO BEC: a suite of clinical cases (BEC-Cases) to simulate patient care and a mobile phone application (BEC-App) for reference. The purpose was to determine whether the use of these educational adjuncts in a flipped classroom approach improves knowledge acquisition and retention among healthcare workers in a low-resource setting.
METHODS: We conducted a prospective, cohort study from October 2017 through February 2018 at two district hospitals in the Pwani Region of Tanzania. Descriptive statistics, Fisher's exact t-tests, and Wilcoxon ranked-sum tests were used to examine whether the use of these adjuncts resulted in improved learner knowledge. Participants were enrolled based on location into two arms; Arm 1 received the BEC course and Arm 2 received the BEC-Cases and BEC-App in addition to the BEC course. Both Arms were tested before and after the BEC course, as well as a 7-month follow-up exam. All participants were invited to focus groups on the course and adjuncts.
RESULTS: A total of 24 participants were included, 12 (50%) of whom were followed to completion. Mean pre-test scores in Arm 1 (50%) were similar to Arm 2 (53%) (p=0.52). Both arms had improved test scores after the BEC Course Arm 1 (74%) and Arm 2 (87%), (p=0.03). At 7-month follow-up, though with significant participant loss to follow up, Arm 1 had a mean follow-up exam score of 66%, and Arm 2, 74%. DISCUSSION: Implementation of flipped classroom educational adjuncts for the WHO BEC course is feasible and may improve healthcare worker learning in low resource settings. Our focus- group feedback suggest that the course and adjuncts are user friendly and culturally appropriate.

African relevance

This pilot study suggests that the development of open access educational resources for the course is feasible.

Open access educational resources may result in greater knowledge acquisition and retention.

This may prove a useful adjunct for the World Health Organization Basic Emergency Care course.

Introduction

Emergency care, consisting of early resuscitation and stabilization, substantially reduces the morbidity and mortality associated with injury and a wide range of acute conditions [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]]. One estimate suggests that over half of all deaths and nearly one-third of disability adjusted life-years in low- and middle- income countries (LMICs) are potentially addressable by emergency care [11]. However, significant gaps remain in access to emergency care in LMICs [[12], [13], [14], [15]].

In 2006, a WHO report concluded that the lack of trained medical professionals is a significant barrier to adequate emergency care in LMICs [15]. Given this need, the WHO Basic Emergency Care course (BEC) was conceived and developed with input from stakeholders from 40 countries. The BEC is a five day, in-person course consisting of lectures and hands-on skills stations.

Furthermore, in 2007 the WHO published a Bulletin calling for more online open access educational resources (OERs) as part of a greater strategy to improve medical training [16]. The potential for OERs to link large audiences with crucial educational resources has been demonstrated by its implementation in the United States and other highly-resourced countries [[17], [18], [19], [20]]. Similar efforts underway in LMICs have shown promise, specifically in training healthcare providers [[21], [22], [23], [24], [25], [26], [27]]. OERs may help adults learn more effectively by empowering them to review materials at their own pace outside of classroom-based instruction; a concept proposed by the “flipped classroom” model [27].

Recognizing the importance of emergency care training and the powerful potential of OERs, we developed two novel OERs as adjuncts to the WHO BEC course. In developing the BEC-Cases and BEC-App, we sought to improve knowledge acquisition and retention from the BEC course, using the “flipped classroom” model. The BEC-Cases were designed to be engaging clinical scenarios that connect BEC concepts to realistic patient presentations. The BEC-App was designed to be a point-of-care reference tool that participants could use beyond the BEC course while caring for patients with emergency needs.

While the flipped classroom model appears to be effective among adult learners in high income countries, its utility in is not well studied among LMICs [[35], [36], [37], [38], [39], [40]]. The widespread use of mobile devices in many LMICs, now puts OERs and the “flipped classroom” approach within reach of educators in these settings. The goal of this pilot study was to determine whether the addition of the BEC-Cases and BEC-App to the standard BEC course would improve short and long-term knowledge acquisition and retention compared to the BEC Course alone. Additionally, we sought to determine if the BEC adjuncts were culturally appropriate and accessible among their intended users.

Methods

Development of the BEC-Cases began with a systematic evaluation of the WHO BEC course content. Study investigators identified and categorized all the key learning points in the course, which were subsequently classified by topic and arranged into specific learning goals.

Learning goals were grouped and thirty-two representative clinical case scenarios were developed in which all 219 unique learning goals were addressed in the BEC-Cases. In order to bring in both the learners' and educators' perspectives to case development, six residents and five attending physicians were invited to develop case scenarios. All cases were reviewed and edited by the study investigators. The final BEC-Cases were coded into an interactive html format (twinery.org). This format was selected for its small file sizes and thus accessibility in low-bandwidth internet settings, and for its compatibility with most operating systems and devices.

The BEC-App was designed as a point-of-care reference to be freely downloaded onto smartphones. Using the BEC course as a guide, study investigators collaborated with a software programmer to create a flowchart of the fundamental emergency care approach taught in the BEC: the ABCDE Assessment. Using this approach, the app algorithmically guides providers through decision branches in the care of life-threatening conditions. The BEC-App includes descriptions and images of emergency interventions and was cross checked against the BEC-Cases to ensure internal consistency.

The BEC-App and a selection of BEC-Cases underwent pre-release testing among a volunteer cohort of emergency providers at the African Federation for Emergency Medicine conference in 2016 in Cairo, Egypt. Additionally, user testing was performed among a convenience sample of medical attendants, nurses, and doctors at Muhimbili National Hospital in January 2017 in Dar es Salaam, Tanzania and the University of Cape Town/Stellenbosch University, Cape Town, South Africa. User feedback on case content and delivery was collected and edits were made accordingly.

We conducted a prospective, cohort study of adult healthcare providers in two district hospitals in the Pwani Region of Tanzania from October 2017 through February 2018. Participants were selected from a convenience sample of providers working at Bagamoyo and Kisarawe Hospitals, two district hospitals receiving emergency patients with similar proximity to the national referral hospital, Muhimbili National Hospital. The study design and content underwent ethics review at the University of California, San Francisco and the Muhimbili National Hospital, Dar es Salaam, Tanzania.

Eligible participants were adult (at least 18 years of age) doctors, nurses, and medical attendants who voluntarily enrolled in the educational course between October 2017 and February 2018. Participants were active healthcare providers with variable work experience and responsibilities but who have not participated in additional, formal emergency care training, such as an emergency medicine residency, a specialized emergency nursing curriculum, or a pre-hospital emergency care training course. Informed consent to participate in the study was obtained prior to enrollment in the course for all participants and participants received no financial incentives [Appendix A]. There were no further exclusion criteria.

Participants were Swahili speaking with English as a second language. Local provider educators fluent in English and Swahili were selected to teach the in-person BEC course. Written BEC and adjunct materials were provided in English and all interventions and assessments were conducted at the participants' location of work.

The study was designed with two arms. Kisarawe Hospital was arbitrarily selected to be Arm 1 and Bagamoyo Hospital to be Arm 2. We attempted to enroll 24 participants equally divided between arms with a goal of 12 participants in each arm as recommended by the BEC with each level of provider represented. Due to site-specific logistics, 14 participants were enrolled in Arm 1 and 10 in Arm 2 in an unequal distribution of healthcare cadres. Participants in Arm 1 underwent the traditional BEC course alone while participants in Arm 2 received online access to the BEC-Cases and App two weeks prior to the BEC course. In-person assistance was available to participants in Arm 2 to help overcome technical barriers to accessing online adjuncts, but no course content was discussed. Following the conclusion of the study, access to the BEC-Cases and BEC-App was made freely available to participants in Arm 1. Immediately after consent and enrollment, all participants were given a multiple-choice pre-test to assess learners' baseline knowledge prior to receiving the BEC course materials [Appendix B]. Following the pre-course assessments, the BEC course was conducted at each site following the standard WHO BEC Facilitator Guide. At the end of the course, all participants scoring 72% and above were awarded a Certificate of Completion through the Emergency Medicine Association of Tanzania (the national professional society conducting the course). Focus groups inclusive of all participants were held by course instructors after the post-test to obtain participants' opinions of the course and the adjuncts' accessibility, applicability, and potential improvements. [Focus group guide available in Appendix D.]

Approximately 7 months later in September 2018, participants in both arms were reassessed using a multiple-choice follow-up exam [Appendix C] to measure longer-term knowledge retention. A final focus group was performed for participants in Arms 1 and 2 according to the focus group guide with the goal of identifying users' opinions on the utility of the course content and the adjuncts in their professional lives since the course [Appendix E].

Outcomes of interest in this feasibility study were participant scores on the pre-test, post-test, follow-up exam, and qualitative data on the usability and appropriateness of the adjuncts among the target population. Basic demographic and professional information were gathered during assessments and feedback [Appendices B, C, D, E]. We analyzed intra- and inter-cohort data using SAS software and performed Fisher's exact t-tests and Wilcoxon ranked-sum tests along with descriptive statistics were used to assess for differences in knowledge, based on the assessment tests, and passing scores between Arms 1 and 2 before and after the BEC course and OERs. Subgroup analyses were performed among both cohorts comparing pre- and post-test performance among healthcare profession but is limited by significant asymmetry between the two groups.

Results

A total of 24 participants were enrolled and completed the first stage of this prospective cohort study. A total of 14 participants were enrolled in Arm 1 and 10 in Arm 2, due to the staffing needs in the hospitals at the time of the course. Arm 1 participants were more likely to be female (93%) and much more likely to be medical attendants (50%) than nurses (29%) and doctors (21%). Arm 2 participants were also more likely to be female (70%) and nurses (70%) (Table 1).

Table 1

Participant characteristics per arm enrolled including gender and job type.

Participant characteristics by study arm
Study arm	Arm 1 - Kisarawe (BEC-Course)	Arm 2 - Bagamoyo (BEC-Course + Adjuncts)
Total Participants	14	10
Female (%)	13 (93%)	7 (70%)
Job description
Doctor	3 (21%)	3 (30%)
Nurse	4 (29%)	7 (70%)
Medical Attendant	7 (50%)	0 (0%)

All participants completed the pre-tests and post-tests in the initial study period. Follow-up exam analyses were limited by significant (50%) participant loss to follow-up (Arm 1 had 7 lost to follow-up and Arm 2 had 5 lost to follow-up). Every effort was made to contact participants for follow-up, however due to clinical duties and staff movement many original participants were unable to complete the follow-up exam (Fig. 1). In our analysis, we included all data that was obtained even if incomplete.

Fig. 1

Flow diagram of participants.

There was a total of 24 participants, 14 in Arm 1 and 10 in Arm 2 at the initial assessment. The delayed assessment in Arm 1 had 7 participants with 7 lost to follow up. Arm 2 had 5 participants in the delayed assessment with 5 lost to follow up. Mean pre-test scores were comparable between Arm 1 (50%) and Arm 2 (53%). After completing the pre- test, participants in Arm 2 were given access to the OERs for two weeks prior to the BEC course.

Following the BEC course, mean post-test scores were improved in both groups with Arm 1 (74%) and Arm 2 (87%). The absolute mean percent score improvement was greater in Arm 2 (34%) than Arm 1 (24%), though this difference did not reach statistical significance (p=0.14). Participants required a 72% minimum score on the immediate post-test to receive a “Certificate of Completion” for the BEC Course as designated by the WHO. A total of 7 (50%) of participants passed the immediate post-course assessment test in Arm 1 compared with 9 (90%) in Arm 2 (p=0.08). At 7 months, the mean follow-up exam score remained lower in Arm 1 (66%) than in Arm 2 (74%) (Fig. 2), though interpretation of these data are significantly limited by many participants being lost to follow up.

Fig. 2

Average pre-, immediate post-test, and follow-up exam scores (%) by study arm.

The mean score improvement from pre-test to post-test was greater in Arm 2 (p=0.14). The mean scores at the follow-up exam were higher in Arm 2 than Arm 1.

In a subgroup analysis of participants' performance in the immediate pre-tests and post-tests based on job description, the largest overall improvement was seen among nurses in both arms while the smallest was seen among doctors. Overall, nurses' scores improved from 49% to 86% (absolute mean improvement of 37% among the 10 participating nurses), while doctors' scores improved from 61% to 81% (absolute mean improvement of 20% among the 7 participating doctors)

A secondary objective of this study was to gather qualitative feedback on the BEC course and adjuncts, their usability and appropriateness. Focus groups were held immediately following the BEC course in both arms and again following the follow-up exams. Comments from the first focus groups include, “They use simple and understandable language, as well as vivid examples like scenarios that happens in their life,” and, “Everything was clearly explained and was well understood.” Written feedback about course content included “I like because I change my ideas with different people on how to handle or care the emergency cases,” and, “The module was relevant to me.” While the small study size and technical challenges during the focus groups precluded thematic grouping, participant responses to open-ended questions included: “I loved the case scenarios, mostly about the Needle Decompression, I feel confident doing it so long as I have this BEC course,” “I feel like I have gained enough knowledge about BEC and I'm confident that I will attend emergency patients, I loved most the skills session,” “Case scenarios has helped a lot according to ABCDE approach we learned,” “Scenarios are good for really cause they give us a real picture,” and the “point of care app will help so much.”

Discussion

Emergency care training is critical in order to narrow the gap between patient needs and provider skills, facilitate more efficient use of existing resources, and reduce morbidity and mortality in LMICs [[28], [29], [30], [31], [32], [33], [34]].

While limited, the data from this pilot study show the WHO BEC course improved participants' knowledge of emergency care. Furthermore, participants that first received access to the OERs showed a trend towards improved post-test scores when compared to participants that underwent the BEC course alone. Thus, our data suggest that in this resource- limited setting, the flipped classroom model using mobile phone OERs may help adults learn more effectively. These differences appear to be persistent at 7 months following the BEC course although definitive conclusions are significantly limited due to participant loss to follow-up.

Additionally, trends among our data suggest that the BEC course and the OERs may lead to greater knowledge gains among nurses than doctors. This could indicate that certain provider groups stand to benefit more from the BEC course and OERs. Definitive conclusions are however not possible due to small sample size and convenience sampling.

Finally, focus group data suggest the BEC course and OERs were well received and appear to be delivered in an acceptable and accessible manner.

This is a pilot feasibility study and as such is subject to much bias including selection bias, interviewer bias, and response bias. As a pilot study, this investigation is limited in its power to detect statistically significant differences among study arms. Our results are thus meant to guide further studies of the BEC course and efforts to utilize and evaluate the flipped classroom approach in LMICs. Limitations of this study include selection bias and convenience sampling of participants, leading to a higher baseline level of medical training in Arm 2. Though Arm 2 had a higher proportion of doctor participants, pre-test scores were similar among both arms suggesting that knowledge of emergency care was comparable between arms prior to the BEC course and OERs.

Due to technical constraints, we were unable to evaluate how frequently OERs were used by participants. Known challenges to accessing OERs included limited and self-funded mobile data in rural areas where study sites were located and participants' older models of mobile phones with slow interfaces. Future implementation studies are planned to help mitigate these challenges.

Finally, further bias was introduced at the follow-up exam due to significant participant loss to follow up. Every effort was made to contact participants for follow-up, including notifying hospital leadership and calling participants on their mobile phones. However, due to clinical duties and staff movement many original participants were unable to complete the follow-up exam. In Arm 1, more doctors completed the follow-up exam, compared to only one doctor completing the follow-up exam in Arm 2. Given the scores that Doctors obtained on the pre- and post-tests, we expect this to skew the mean follow-up exam score to be higher in Arm 1, which could indicate that the improvement in scores in those exposed to the BEC adjuncts may have been greater than we know. Finally, our results are isolated to short and moderate- term knowledge assessments and do not evaluate long-term knowledge retention, hands-on application of skills, or patient-oriented outcomes.

While the above findings are from a pilot study, the study sites selected were district hospitals near Dar es Salaam, Tanzania. As such, we believe that the use of the BEC Course and OERs, as well as the flipped classroom model are feasible and show promise to be beneficial in other LMICs, warranting further thoughtful investigation.

Conclusions

Despite multiple limitations, this pilot study suggests that the development of BEC OERs is feasible and may lead to greater knowledge acquisition and retention via the flipped classroom model, compared to the BEC course alone.

Future studies are needed to assess long-term knowledge retention from the BEC course and accessibility of OERs, and to determine whether trends toward higher knowledge assessment scores among OERs users are specifically attributable to OERs.

Dissemination of results

Preliminary site-specific results were shared with staff members at each study site through an informal presentation. Results were also shared with collaborators at the referral hospital for each site.

Authors' contribution

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: SS contributed 20%; JCB and AT 15% each; PN, JM and HS 10% each; CR, TN, DH, and SM 5% each.

All authors approved the version to be published and agreed to be accountable for all aspects of the work.

The following are the supplementary data related to this article.

Appendix A

Study information for participants and informed consent.

Appendix B

BEC adjunct pre-course knowledge assessment.

Appendix C

BEC adjunt delayed knowledge assessment.

Appendix D

BEC pre course demographics and focus group facilitator guide.

Appendix E

BEC course delayed demographics and focus group facilitator guide.

Declaration of competing interest

The authors declared no conflicts of interest.