Evaluation of change in emergency care knowledge and skills among front-line healthcare providers in Ukraine with the Basic Emergency Care course: a pretest/post-test study.

OBJECTIVE: Evaluate the change in participant emergency care knowledge and skill confidence after implementation of the WHO-International Committee of the Red Cross (ICRC) Basic Emergency Care (BEC) course.
DESIGN: Pretest/post-test quasi-experimental study.
SETTING: Mechnikov Hospital in Dnipro, Ukraine. PARTICIPANTS: Seventy-nine participants engaged in the course, of whom 50 (63.3%) completed all assessment tools. The course was open to healthcare providers of any level who assess and treat emergency conditions as part of their practice. The most common participant profession was resident physician (24%), followed by health educator (18%) and prehospital provider (14%).
INTERVENTIONS: The 5-day WHO-ICRC BEC course. PRIMARY AND SECONDARY OUTCOME MEASURES: Change in pre-course and post-course knowledge and skill confidence assessments. Open-ended written feedback was collected upon course completion and analysed for common themes.
RESULTS: Participant knowledge assessment scores improved from 19 (IQR 15-20) to 22 (IQR 19-23) on a 25-point scale (p<0.001). Participant skill confidence self-assessment scores improved from 2.5 (IQR 2.1-2.8) to 2.9 (IQR 2.5-3.3) on a 4-point scale (p<0.001). The most common positive feedback themes were high-quality teaching and useful skill sessions. The most common constructive feedback themes were translation challenges and request for additional skill session time.
CONCLUSIONS: This first implementation of the WHO-ICRC BEC course for front-line healthcare providers in Ukraine was successful and well received by participants. This is also the first report of a BEC implementation outside of Africa and suggests that the course is also effective in the European context, particularly in humanitarian crisis and conflict settings. Future research should evaluate long-term knowledge retention and the impact on patient outcomes. Further iterations should emphasise local language translation and consider expanding clinical skills sessions. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Participants were drawn from diverse professions and levels of training.

The participant subgroup size was small, which may affect the ability to draw individual conclusions.

Change in knowledge and skills was evaluated with pre-course and post-course written and practical skill assessments.

Only 50 (63.3%) completed all four course assessment components and were included in the analysis.

Test performance may have been influenced by physician specialty and years in practice.

Introduction

Ukraine is the only country in Europe facing an ongoing armed conflict, resulting from the Russian annexation of Crimea in 2014 and subsequent occupation of the Eastern territories of Donetsk and Luhansk (figure 1). In February 2022, Russian forces invaded the rest of Ukraine sparking an ongoing war of devastating morbidity and mortality.1 During the first weeks of fighting, over 10 million Ukrainian civilians have been displaced from their homes, and indiscriminate bombing has caused significant destruction of infrastructure in many cities in Ukraine.2 Millions of Ukrainians have been forced to find shelter or flee to western Ukraine and neighbouring countries like Poland, Moldova and Hungary.3

Figure 1

Ukraine war in maps: tracking the Russian invasion. Source: Institute for the Study of War. Accessed 31 March 2022 (https://www.bbc.com/news/world-europe-60506682).

Despite the increased need for basic emergency medical care, it is estimated that conflict areas have lost 30%–70% of medical professionals, forcing many patients to travel long distances to access care.4 Damaged infrastructure and security concerns have also negatively impacted prehospital emergency care systems.5 The COVID-19 pandemic has further strained the situation, with over 1.1 million cases and over 20 000 deaths in the country to date.6 In addition, there have been numerous high-activity disinformation campaigns further hindering the healthcare response.7

The healthcare system in Ukraine is among the least developed of the post-Soviet countries in terms of important health outcome measures and organisational efficiency.8 Emergency medical care in particular is fragmented and underdeveloped, facing challenges related to lack of funding, equipment shortages and lack of standardisation in training.9 Recent reforms have required physicians to adopt international clinical guidelines and introduced the role of paramedics and emergency medical technicians to transition prehospital care delivery away from physicians and feldshers.9 10 Despite these steps forward, prehospital and hospital emergency medical care in Ukraine face several challenges. While emergency departments exist in Ukraine, they function mostly as triage and admission centres and lack specialised emergency personnel or equipment needed for patient evaluation and resuscitation. Opportunities for continuous medical education are limited and inconsistent throughout the country.11 While there are currently no studies examining the quality of provision and training of emergency care specifically in eastern Ukraine, these issues are likely exacerbated in the setting of fewer healthcare resources.12

In response to the ongoing conflict and need for increased training opportunities in emergency care in Ukraine, the ‘Emerging Technologies and Countermeasures to CBRN Agents: Advanced Training Response to Conflict and Security Challenges in East Ukraine’, was created by the North Atlantic Treaty Organization (NATO) Science for Peace and Security Programme and held at Mechnikov Hospital in Dnipro, Ukraine in November of 2019.13 In the wake of the armed Russian–Ukrainian conflict that erupted in 2014, this hospital scaled up and transformed itself into a trauma response centre treating thousands of soldiers and civilians wounded in the fighting. The WHO and International Committee of the Red Cross (ICRC) Basic Emergency Care (BEC) course was delivered to front-line healthcare providers during the event.

The BEC course is an open-access 5-day course designed to train providers how to assess and treat conditions requiring emergency care, mixing didactic sessions and skills station training.14 The topics and skills covered are listed in table 1. The course has been successfully implemented in several sub-Saharan African countries.15–20 It has not yet been implemented in a humanitarian context but given its focus on providing care in a resource-limited setting, it may be capable of improving access to care and patient outcomes in a setting such as eastern Ukraine. BEC also represented an opportunity to train a wide variety of first responders, an important goal in eastern Ukraine due to the significant loss of healthcare professionals from the region. Here we report the effectiveness and acceptability of implementing BEC course in Ukraine to improve emergency care knowledge and to enhance practical hands-on clinical skills.

Table 1

Topics and skills covered in the BEC course

Didactic topics	Practical skills
ABCDE Personal Protective Equipment Scene Safety Airway Obstruction Breathing Tension Pneumothorax Opiate Overdose Asthma/Chronic Obstructive Pulmonary Disease (COPD) Pleural Effusion/Haemothorax Circulation Pulselessness Shock Severe Bleeding Pericardial Tamponade Disability Hypoglycaemia Increased Pressure on the Brain Seizure/Convulsion Exposure Snake Bite Hypothermia SAMPLE History Signs and Symptoms Allergies Medications Past Medical History Last Oral Intake Events Surrounding Illness or Injury	Airway Skills Stations Basic Airway Manoeuvres Opening the Airway—Adult Head-Tilt/Chin-Lift Opening the Airway—Paediatric Head-Tilt/Chin-Lift Opening the Airway—Adult Jaw Thrust Opening the Airway—Paediatric Jaw Thrust Management of Choking Management of the Choking Adult and Larger Child Management of the Choking Infant and Small Child Airway Suctioning Basic Airway Device Insertion Oropharyngeal Airway Insertion Nasopharyngeal Airway Insertion Recovery Position
Trauma Trauma Primary Survey Scene Safety Cervical Spine Immobilisation Trauma Secondary Survey Head Injury Facial Fractures Penetrating Eye Injury Penetrating Neck Wound Chest Injury Abdominal Injury Spinal Cord Injury Internal Bleeding Pelvic Fracture Extremity Fracture Open Fracture Open Wound Crush Injury Blast Injury Burn Injury Trauma in Pregnancy	Breathing Skills Stations Giving Supplemental Oxygen Bag-Valve-Mask Ventilation Emergency Needle Decompression Management of Open Pneumothorax (Sucking Chest Wound) How to Make a Spacer from a Plastic Bottle Breathing assessment
Difficulty in Breathing Airway Causes Airway Foreign Body Severe Allergic Reaction Airway Swelling from Inflammation or Infection Airway Burns Lung Causes Pneumonia Asthma/COPD Pneumothorax Haemothorax Pleural Effusion Acute Chest Syndrome in Sickle Cell Disease Heart Causes Heart Attack Heart Failure Pericardial Tamponade Systemic Causes Anaemia Opioid Overdose Diabetic Ketoacidosis	Circulation Skills Stations Circulation examination External bleeding control Direct Pressure for External Bleeding Deep Wound Packing for External Bleeding Tourniquet Technique for Uncontrolled External Bleeding Uterine Massage for Postpartum Haemorrhage IV Cannulation Insertion of IV Cannula (Adult) Insertion of IV Cannula (Paediatric) IV Fluid Adjustment for Special Considerations IV Fluid Administration for Shock
Shock Due to Dilated Blood Vessels Severe Infection Spinal Cord Injury Severe Allergic Reaction Due to Fluid Loss Diabetic Ketoacidosis Severe Dehydration Burn Injury Due to Blood Loss External Bleeding Large Bone Fracture Abdominal Bleeding Bleeding in the Stomach or Intestines Haemothorax Ectopic Pregnancy Postpartum Haemorrhage Due to Problems with the Heart Heart Failure Heart Attack Abnormal Heart Rhythm Heart Valve Problem Pericardial Tamponade Tension Pneumothorax	Extended Physical Examination Skills Station Neurological Assessment Glasgow Coma Scale Alert/Verbal/Pain/Unresponsive (AVPU) Scale Secondary Survey (Head-to-Toe) Trauma Assessment
Altered Mental Status Rapidly Reversible Causes Hypoglycaemia Severe Dehydration Heat Stroke Hypoxia Infection Cerebral Malaria Inflammation or Infection around the Brain Severe Infection Rabies Metabolic Diabetic Ketoacidosis Toxic Alcohol or Drug Intoxication or Withdrawal Pesticide Poisoning Snake Bite Medication Reaction or Dosing Issue Gaseous Poisoning Other Causes Seizures/Convulsions Increased Pressure on the Brain Liver Disease Kidney Disease Head Trauma Ingestions of Chemicals or Toxins	Immobilisation Skills Station Cervical Spine Immobilisation Log Roll Spinal Immobilisation Positioning of the Pregnant Patient Fracture Immobilisation Closed Fracture Immobilisation Open Fracture Immobilisation Applying a Pelvic Binder
Transfer and Handover Destination Planning Transfer Patient Positioning Ongoing Care during Transport Handover Situation, Background, Assessment, Recommendations	Wound Management Skills Stations General Wound Management Burn Management Determine Total Body Surface Area Estimate Depth of Burn Fluid Resuscitation in Burn Injury Snake Bite Bandaging and Immobilisation
Medications Epinephrine Acetylsalicylic Acid (Aspirin) Diazepam Glucose (Dextrose) Magnesium Sulfate Naloxone Oxytocin Paracetamol (Acetaminophen) Salbutamol (Albuterol) Tetanus Vaccine

BEC, Basic Emergency Care; IV, intravenous.

Methods

The 5-day course was held in Dnipro, Ukraine as an optional track of the ‘Advanced Training Response to Conflict and Security Challenges in East Ukraine’ and was open to healthcare providers of any level who assess and treat emergency conditions as part of their practice. The class was taught by a group of three facilitators in English with real-time translation to Ukrainian and Russian. The two facilitators leading instruction were US-based attending emergency medicine physicians and medical school faculty, one of whom has taught several BEC courses, and the third facilitator was a US medical student completing a fellowship in Ukraine who supported course logistics. Course materials were translated into Ukrainian. Language of instruction was driven by the course participants. Both Russian and Ukrainian are commonly spoken and widely understood, so it is not uncommon for both languages to be used. Small group skills sessions were led by local providers who were identified by course leadership based on prior experience in delivering and teaching emergency care and participated in training specific to their assigned stations prior to the course. There was no course enrolment fee.

Participants completed 25 questions of written pretest and post-test to assess their knowledge of BEC content at the beginning of the first day and the end of the last day of the course. Participants are scored on a scale of 1 to 25, receiving 1 point per correct question. Participants also completed an 11-question pre-confidence and post-confidence self-assessment on BEC knowledge and skills in which a score of 1 (least confident) to 4 (most confident) is self-assigned in response to each question. Participants were also given the opportunity to provide open-ended written anonymous feedback on the course. These course assessment tools are part of the published BEC course package and were translated into Ukrainian. Participants were only awarded a completion certificate if attendance requirements were met, and all pre-assessment and post-assessment materials were completed. Pre-course and post-course assessment scores were compared using the Wilcoxon signed-rank test and p values were reported. Results were analysed in whole and by participant role. Course feedback was collected confidentially on paper, from each participant in Russian and Ukrainian and translated to English for analysis. Comments were closely reviewed by the authors and similar comments were grouped together to assess for positive and constructive themes. Participants who did not complete each of the assessment components were excluded from the quantitative analysis, although their comments, if provided, were included in the feedback results. Data verification was further conducted with the translated course feedback.

Patient and public involvement

The BEC course was included in the conference in response to a request for emergency care training of front-line providers by participants and coordinators. The research question and outcome measures were informed by a desire to demonstrate if the training is effective in caring for patients in this regional context. The study design was adapted from previous implementations which were informed by focus groups; however, patients were not directly involved in the design of this study. Patients were not involved in the recruitment or conduct of the study, nor did any patients participate in the study. The results will be disseminated to study participants through publication.

Results

Seventy-nine participants registered for the course of which 50 (63.3%) completed all four assessment components, which include the pre-knowledge and post-knowledge assessment examinations and the pre-confidence and post-confidence assessments. Participant home cities are shown in table 2. Participant professions are listed in table 3. Resident physicians are those who are undergoing postgraduate specialty training. Of the residents and physicians attending the course, the largest specialty was surgery (28%), followed by anaesthesia (17%) and medicine (17%).

Table 2

Participant home cities

City	N
Dnipro	21 (42%)
Kyiv	18 (36%)
Unknown	8 (16%)
Poltova	1 (2%)
Mykolaiv	1 (2%)
Rivne	1 (2%)

Table 3

Occupational demographics

Occupation		Time since training (years)	Time in current position (years)
Resident physician	12 (24%)	Not Applicable	1.5
Health educator	9 (18%)	3.4	1.8
Prehospital provider	7 (14%)	3.6	5.1
Physician	6 (12%)	10.2	5.5
Police officer	5 (10%)	14.5	8.3
Medical student	4 (8%)	N/A	1.9
Other*	7 (14%)	15	17

*Red Cross volunteer, dentist, nurse, engineer, nuclear plant worker.

Test and confidence scores

Table 4 summarises pretest and post-test and confidence assessment scores by profession. Knowledge assessment scores improved significantly after completing the BEC course from 19 (IQR 15–20) to 22 (IQR 19–23) on a 25-point scale. Of the 50 participants who completed both the pretest and post-test, 40 improved their score, 5 did not change their score and 5 had lower scores. Some positions were analysed as a group due to their smaller individual cohort sizes and denoted as ‘other’ (n=7). Only the overall analysis of all participants was adequately powered and is demonstrated in figure 2.

Table 4

Summary of pretest and post-test and confidence assessment (CA) scores by profession

Position	N (%)	Pretest median (IQR)	Post-test median (IQR)	P value	Power	Pre-CA median (IQR)	Post-CA median (IQR)	P value	Power
Resident physician	12 (24)	18 (16–19)	23 (18–24)	0.068	0.057	2.6 (1.9–3.0)	2.8 (1.8–3.1)	0.374	0.15
Health educator	9 (18)	19 (16–20)	22 (21–22)	0.065	0.41	2.6 (2.4–2.6)	3.2 (2.8–3.6)	0.008	0.27
Prehospital provider	7 (14)	19 (9–20)	20 (19–22)	0.042	0.031	2.6 (2.1–2.9)	2.8 (2.3–3.0)	0.345	0.21
Physician	6 (12)	20 (19–21)	23 (22–23)	0.027	0.21	2.6 (2.0–2.9)	3.0 (2.9–3.6)	0.028	0.2
Police officer	5 (10)	13 (9–18)	22 (21–22)	0.043	0.28	2.3 (2.1–2.7)	3.0 (2.8–3.0)	0.042	0.35
Medical student	4 (8)	21 (21–22)	23 (21–23)	0.713	0.29	2.4 (2.2–2.6)	2.9 (2.8–3.4)	0.68	0.07
Other	7 (14)	17 (12–19)	19 (18–21)	0.027	0.23	2.25 (2.1–2.2)	2.5 (2.3–2.8)	0.017	0.23
All participants	50 (100)	19 (15–20)	22 (19–23)	<0.001	0.59	2.5 (2.1–2.8)	2.9 (2.5–3.3)	<0.001	0.78

Other includes Red Cross volunteer, dentist, engineer, nurse, nuclear plant worker.

Figure 2

Overall median pretest and post-test scores.

Self-assessed confidence ratings in emergency care knowledge and skills also improved significantly after course completion from 2.46 (IQR 2.08–2.75) to 2.92 (IQR 2.50–3.25) on a 4-point scale. Of the 50 participants who completed the confidence assessment both before and after the course, 43 improved their score, 2 did not change their score and 5 had lower scores. Again, only the overall analysis of all participants was adequately powered and is demonstrated in figure 3.

Figure 3

Overall median pre-confidence and post-confidence assessment (CA) scores.

Course feedback

Feedback was collected during the final day of the course. Participants were asked to describe what they liked best and what they would change in the course. Of the 50 participants who completed all assessment components, 37 of 50 (74%) gave course feedback. Additional nine participants, who had registered for the course but did not complete one or more of the assessment components, gave course feedback that was included.

The most common positive feedback (16 of 46, 34.8%) surrounded quality of teaching. The practical skills station portion was also well received by participants, with 13 of 46 (28.3%) stating it was a highlight of the course. Positive comments also centred around the easy understandability of the material, course content, and team-building and positive class environment that the class offered. The most frequent suggestions for improvement focused on translation issues (11 of 46, 23.9%) regarding the course lectures and course materials. A significant number of participants (9 of 46, 19.6%) said they would like more hands-on skills station practice. Other comments centred on more standardisation in instruction of practical sessions and desire for more in-depth content. Positive and constructive comments are included in table 5. Comments were chosen as representative of the most common themes.

Table 5

Positive comments and constructive feedback from course participants

Positive comments

Constructive feedback

‘[I liked the] interaction of the participants in the course (under observation of trainers), structure, simplicity of the delivery of the material, a great variety of helping materials, precise timing with adequate breaks, lots of practice, regular review of the material, and practice scenarios!’ (Health Educator, city not recorded)‘I like the way it is structured. A lot of information about emergency was given in a quite simple way.’ (Health Educator, Kyiv)‘[I liked the] lecturers and instructors, understandable presentations, practice after theory, structured lectures with modern recommendations.’ (Resident Physician, city not recorded)

‘The long duration of the course, the language of instruction, more practice, better translations of the test are needed.’ (Resident physician, Kyiv)‘It would be great to have more practice and high-quality equipment so that interns can learn and repeat all the manipulations perfectly.’ (Prehospital Provider, location not recorded)‘More instructors that took care to use only the course program and not their own experience for example, 'we were taught', 'there was a case', 'I heard'. Thanks. More personal experiences.’ (Health Educator, Kyiv)

Discussion

The findings in this study were acquired from the first WHO-ICRC BEC course conducted in Europe. Knowledge assessment scores and self-assessed confidence scores improved significantly after completion of the BEC course. However, when stratified by subgroups, some professions trended to more improvement than others. While practising physicians, police officers, and the combined ‘other’ category saw significant improvements in both test and confidence scores, resident physicians and medical students did not. A possible explanation for this difference is that residents and students are receiving active education and some content from BEC may have been recently taught to them. However, this reasoning would suggest higher pretest scores for these groups in comparison with others, which was observed for medical students but not for resident physicians. Alternatively, physicians and police officers, and positions within the other category are more removed from recent training and this course may have provided a new learning or refresher opportunity. Prehospital workers and health educators were split in the significance of their improvements, which may reflect the diversity of training and experience in these categories.

This was also the first BEC course to be taught in Ukrainian or Russian, using real-time translation to cross language barriers similar to BEC implementations in other countries. While the real-time translation from English to Ukrainian slowed the course slightly, it was effective as reflected in the assessment tool and the comments. Participants in this course voluntarily formed small groups to translate the BEC Quick Cards, which are meant to summarise high-yield content, into Ukrainian and distributed copies real time to all students participating in the course.

This implementation demonstrated the feasibility of the BEC course to train providers planning to work in a humanitarian context: many of the participants plan to provide emergency care in the context of the eastern Ukraine humanitarian crisis and reported that the knowledge and skills acquired here will be helpful. The feasibility of training diverse providers was also shown: participants included prehospital providers and police officers, both of which function as first responders to the types of emergencies addressed by the BEC course that require timely decisions with limited resources and information. The benefit of the course may vary by specialty training and experience and further research should explore this concept as this study was underpowered to perform subgroup analyses. The scope of the course and depth of content was the subject of feedback in this and prior implementations.16 20 While this feedback may reflect a desire for both additional skills practice and content, this must be balanced with the opportunity cost of taking practising healthcare providers away from patient care duties for a 5-day course.

The findings in this manuscript are subject to limitations. Although 79 participants registered for the course and attended various portions throughout the 5 days, only 50 (63.3%) completed all four course assessment components and were included in the analysis. Incomplete assessment components were due to lack of attendance among these participants, reportedly due to clinical or professional duties and conflicting conference events. While this course was free to enroll in, participants were not paid a per-diem as they were in some other BEC implementations that reported higher levels of assessment tool completion. The participant subgroup size is small which may affect the ability to draw individual conclusions from them, and test performance may have been influenced by physician specialty and years in practice. Prior participant trauma, military or humanitarian experience was not collected, and this may have influenced results. Language barriers were highlighted in the participant feedback and suggest the need for formal translation of course materials. Another limitation includes the wide variation in types of learners who participated in this course with different levels of medical training and field experience. As noted earlier, however, this conflict caused significant loss of healthcare professionals from the region, making the recruitment of same-level learners challenging.

This course was the first step in implementing BEC throughout the region. Several participants in this course will complete a training-of-trainers course with the objective of establishing a dedicated regional training centre in Ukraine. This will allow for local trainers to lead additional courses in the region with the goal of improving access to emergency care which may bolster ongoing efforts to strengthen the specialty of emergency medicine in Ukraine. Future studies assessing knowledge retention, impact and sustainability of this train-the-trainer model will be implemented.

Conclusion

This is the first implementation of the WHO-ICRC BEC course in Europe and the first time this course was conducted in a region with an ongoing conflict. The results of this study suggest that the course was well received and led to significant improvement in emergency care knowledge for a variety of front-line first responders. Future directions will include participants completing a training-of-trainers course, allowing for continued expansion and eventual implementation of these acquired skills in a low-resource healthcare system during an ongoing conflict and war.