The impact of the Fundamental Critical Course on knowledge acquisition in Rwanda.

Background: Emerging critical care systems have gained little attention in low- and middle-income countries. In sub-Saharan Africa, only 4% of the healthcare workforce is trained in critical care, and mortality rates are unacceptably high in this patient population.
Objectives: We sought to retrospectively describe the knowledge acquisition and confidence improvement of practitioners who attend the Fundamental Critical Care Support (FCCS) course in Rwanda.
Methods: We conducted a retrospective study in which we assessed survey data and multiple-choice question data that were collected before and after course delivery. The purpose of these assessments at the time of delivery was to evaluate participants' perception and acquisition of critical care knowledge.
Results: Thirty-six interprofessional clinicians completed the training. Performance on the multiple-choice questions improved overall after the course (mean score pre-course of 56.5% to mean score post-course of 65.8%, p-value <0.001) and improved in all content areas with the exception of diagnosis and management of acute coronary syndrome and acute respiratory failure/mechanical ventilation. Both physicians and nurses improved their scores significantly (68.9% to 75.6%, p-value = 0.031 and 52.0% to 63.5%, p-value <0.001, respectively). Self-reported confidence in level of knowledge also increased in all areas. Survey respondents indicated on open-answer questions that they would like the course offerings at least annually, and that further dissemination of the course in Rwanda was warranted.
Conclusion: Deploying the established FCCS course improved Rwandan healthcare provider knowledge and confidence across most critical care content areas. Therefore, this course represents a good first step in bridging the gaps noted in emerging critical care systems. Contributions of the study: Critical care education in sub-Saharan Africa is limited and few staff have formal training. The aim of the study was to determine whether a focused course delivered in Rwanda on critical care management improved knowledge in key areas. Our retrospective study on results from a multiple choice question test and survey indicate that short courses may improve knowledge of critical care management.

Background

The universal application of critical care best practices is a very important but largely overlooked concept in global health. Historically, global health development efforts have focused on population-wide public health initiatives such as infectious diseases, maternal health, and paediatric health in low- and middle-income countries (LMICs). [[1]] Critical illness survivorship, however, is alarmingly low in LMICs, in contrast to the more frequently studied areas of infectious and chronic disease.[[2]] Data from the limited pool of available studies indicate that approximately 80% of the global mortality from critical illness derives from patients in LMICs.[[2]] This high burden of mortality is thought to be due to poor infrastructure, lack of essential medications, limited workforce, lack of awareness in the identification of critical illness, poorly defined concepts of critical care, and limited processes established to validate the efficacy of various care or process improvement initiatives.[[3,4]] Sub-Saharan Africa bears 24% of the global burden of disease, with only 4% of the workforce operating in critical care.[[5]] Anaesthesiologists are in charge of managing critically ill patients, yet the number of fully-trained providers is extremely low in comparison with high-income countries (HICs).[[6-8]] To address the lack of practitioners who have completed formal critical care training programmes, short courses on fundamentals of critical illness have been employed in sub-Saharan Africa.[[9]] A larger emphasis has been placed on courses focused on trauma, including the Advanced Trauma Life Support and Acute Trauma Care courses, yet the Society of Critical Care Medicine’s Fundamental Critical Care Support (FCCS) course has been administered in a format adapted to the local context and shown to be effective in Kenya and Zambia.[[10,11]] The FCCS course is a structured course developed in the United States that is designed to provide nonintensivists with sufficient critical care training to care for critically ill patients in the first 24 hours or until trained intensivists are available.[[12]]

The present study aims to retrospectively describe the knowledge acquisition and confidence improvement of practitioners who attended the FCCS course in Rwanda.

Material and methods

Study setting

Rwanda is a small, low-income country in the African Great Lakes Region, with a population of approximately 12.3 million. Patients requiring critical care services are cared for in three public and one private (with an academic institution affiliation) adult intensive care units (ICUs), which consist of a total of 26 critical care and 11 high-dependency unit (HDU) beds. These are managed primarily by 18 consultant-level anaesthesiologists assisted by nurses, few of whom generally has specialised critical care training. In addition to these deficits in staffing and fundamental skills training, there is an absence of infrastructure to assure continuing education for health professionals serving the critically ill.

Study procedures

In December 2018, the Society of Critical Care Medicine’s (SCCM’s) standardised Fundamental Critical Care Support (FCCS) course (Version 6.2, updated 2016) was conducted in Kigali, Rwanda, over a 2-day period. We performed a retrospective analysis of the pre- and post-course assessment materials collected during this course. The content of the material delivered included lectures and skills sessions that were developed by the SCCM. For the skills sessions, the class was split to form small groups with trainers leading each group, and the stations consisted of a variety of tools including airway mannequins, functioning ventilators and case-based discussions. Students were enrolled in the course through the University of Rwanda College of Medicine and Health Sciences; there were no prerequisites for registration and the participation fee was nominal. A 20-question pre-course multiple-choice question (MCQ) test, created and validated by the SCCM, was administered to all participants. The test assessed the medical provider’s level of knowledge in the topics to be addressed in a case-based format. A pre-course survey included demographic information, site and area of practice, and perceived level of knowledge in key areas of critical care. To assess the perceived level of knowledge, the survey employed responses categorised in a Likert-type scale. The responses ranged from 1 to 5, with 1 indicating Novice, 2 indicating Advanced Beginner, 3 indicating Competent, 4 indicating Proficient, and 5 indicating Expert. A 50-question post-course MCQ test assessing the same content but with different questions and an identical survey assessing perceived level of knowledge, was administered to all participants who completed the course. The post-course survey included questions assessing participants’ experience in the interprofessional learning environment with three Likert-type scales: 1 = Very Poor, 2 = Below Average, 3 = Average, 4 = Above Average, and 5 = Excellent; 1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Very Often, and 5 = Always; 1 = Strongly Disagree, 2 = Disagree, 3 = Undecided, 4 = Agree, and 5 = Strongly Agree. Additionally, this survey included free text questions related to opportunities for course improvement and future directions. Completed surveys and MCQ tests were de-identified prior to data analysis. Given that this was a retrospective analysis, written informed consent was not sought from the course participants.

Ethical aspects and study oversight

Surveys and testing data were collected in December 2018 as part of the course evaluation and feedback. No identifiable information was collected at this time. When an opportunity arose to further develop course materials, we decided to retrospectively study the anonymous data we had collected in 2018 to help educate us and develop a more tailored curriculum. Authors of this study are the Rwandan faculty who were involved in setting up the course initially, and we had their permission to proceed with the analysis as they felt from their perspective that no identifiable information would be disclosed, given the anonymity of the data, and no individual would be harmed. Ethical approval was then sought, and the study was approved by Virginia Commonwealth University’s Institutional Review Board (ref. no. HM20019868).

Statistical methods

Characteristics of the group are summarised using frequencies and proportions. The overall MCQ test results and results by practice role are reported as means (standard deviations (SDs)) and 95% confidence intervals (CI). Test results of group comparisons were made using equal variance, two-sample independent t-tests with equality of variances assessed using the Brown-Forsythe test. Pre- and post-course test results were compared using paired t-tests and reported for all participants and by practice role. Changes in test scores by content area after course participation were assessed using the χ2 test. p-values less than α (α=0.05) were considered significant.

Likert-type survey results are reported using medians, interquartile ranges, frequencies and proportions. Comments collected from the survey were assessed for themes, and frequencies and proportions of key themes are reported. The Wilcoxon signed-rank test was used to compare differences in self-reported confidence in critical care knowledge areas before and after the course. To assess for differences among the categories of participants’ self-assessment of critical care knowledge, the Kruskal-Wallis test with Bonferroni correction was utilised. A composite Likert score was created by summing the responses to each knowledge area of the participants’ scores pre- and post-course completion, with missing values imputed by multivariate normal imputing without rounding. To assess predictors of improved confidence in knowledge before and after the course, a beta regression with a re-scaling transformation of the change score as described by Zou et al. [[13]] was performed. All levels within the covariates in this model with the exception of gender were grouped to allow for proper model performance, given the small sample size. Data were analysed in JMP, version 15 (SAS Institute, USA).

Results

Thirty-six participants, including nurses, physicians and medical and nursing students, completed the FCCS course. The group had nearly an equal number of male and female participants (Table 1). The majority (71.4%) were in the age range of 30 - 39 and ~two-thirds were nurses, with the remainder being physicians (25.0%) and students (5.6%). The participants’ practice locations were diverse and included private and public referral hospitals in major cities (53.1% and 21.9%, respectively), rural district hospitals (6.3%) and others (Table 1).

Table 1

Participant characteristics

Characteristics	n (%)
Sex, N=36
Male	19 (52.8)
Female	17 (47.2)
Age, N=35
20 - 29	2 (5.7)
30 - 39	25 (71.4)
40 - 49	4 (11.4)
50 - 59	3 (8.6)
60 - 69	1 (2.9)
Primary role, N=36
Nurse	25 (69.4)
Physician	9 (25.0)
Student	2 (5.6)
ICU number of beds, N=30
0 - 4	3 (10.0)
5 - 9	21 (70.0)
≥10	2 (6.7)
Not applicable	4 (13.3)
Years of practice, N=34
<5	11 (32.4)
5 - 10	12 (35.3)
11 - 15	4 (11.8)
16 - 20	4 (11.8)
≥21	3 (8.8)
Practice site, N=32
Private tertiary hospital	17 (53.1)
Public tertiary hospital	7 (21.9)
University of Rwanda student or lecturer	5 (15.6)
District hospital	2 (6.3)
Ambulance service	1 (3.1)

ICU = intensive care unit

The mean (SD) baseline score (% of questions answered correctly) on the MCQ test was 56.5 (14.1)% which improved to 65.8 (9.3)% after the course (mean (SD) difference 9.3 (10.4)%, p-value <0.001) (Table 2). Most participants (85.6%) had a higher test score after completion of the course. Physicians had both higher pre- and post-course MCQ test scores when compared with that of non-physician participants. While nurses and physicians both improved their overall scores significantly after completion of the course, there was no difference in the magnitude of improvement between the two groups (difference (SD)=3.9 (4.2)%, p-value=0.626) (Table 2).

Table 2

Overall multiple-choice question test results

	Test	Mean (SD), %	95% CI	p-value
All participants (n = 36)	Pre-course test	56.5 (14.1)	51.7 - 61.3
	Post-course test	65.8 (9.3)	62.7 - 69.0
	Difference	9.3 (10.4)	6.5 - 13.6	<0.001
Nurses (n = 25)	Pre-course test	52.0 (11.6)	47.2 - 56.8
	Post-course test	63.5 (7.6)	60.4 - 66.6
	Difference	11.5 (10.4)	7.3 - 15.8	<0.001
Physicians (n = 9)	Pre-course test	68.9 (13.9)	58.2 - 79.5
	Post-course test	75.6 (8.9)	68.3 - 83.2
	Difference	6.7 (9.7)	0.0 - 15.7	0.031
Students (n = 2)*

SD = standard deviation

CI = confidence interval

* Student pre-course test scores were 70% and 45%; post-course test scores were 62% and 56%

For the content areas tested both before and after the course, all areas showed significant improvement, with the exception of ‘Diagnosis and management of acute respiratory failure/mechanical ventilation’ (where there was no significant change in score) and ‘Diagnosis and management of acute coronary syndrome’ (where the score declined) (Table 3).

Table 3

Multiple-choice question test results by content area

Content area	Pre-course Percent correct, %	Post-course Percent correct, %	p-value
					Percent correct (%)	Percent correct (%)
Diagnosis and management of acute coronary syndrome	77.8	53.3	<0.001
Diagnosis and management of acute respiratory failure/mechanical ventilation	60.4	64.7	0.231
Management of life-threatening electrolyte and metabolic disturbances	52.8	74.1	0.017
Life-threatening infections: diagnosis and antimicrobial therapy selection	25	66.1	<0.001
Monitoring blood flow, oxygenation and acid-base status	58.3	76.7	<0.001
Critical care in pregnancy	30.6	63.9	0.005
Recognition and assessment of the seriously ill patient	37.5	93.1	<0.001
Special considerations†	19.4	51.9	<0.001
Airway management		79.2
Cardiopulmonary/cerebral resuscitation		59.6
Diagnosis and management of shock		95.8
Neurologic support		80.1
Critical care in infants and children: the basics		86.1
Ethics in critical care medicine		86.1

* Special considerations include management of pulmonary embolism, hypertensive crises, gastrointestinal haemorrhage, abdominal compartment syndrom

Prior to participation in the course, the median confidence level was 3 (Competent) on a response range of 1 - 5, with 1 being Novice and 5 being Expert in all 16 knowledge areas of the FCCS, except for ‘Management of the paediatric airway’ (median=2.5) and ‘Diagnosis and management of acute coronary syndrome’ (median=2) (Supplemental Table 1: https://www.samedical.org/file/1836). After completion of the course, the majority of median confidence level scores were 4 (Proficient) (Supplemental Table 2: https://www.samedical.org/file/1836). Improvement in confidence was seen in all knowledge areas. However, we were unable to detect a significant difference among improvements in these content areas – i.e., we could not determine in which area the greatest improvement was seen owing to the high number of multiple comparisons (Table 4 and Supplemental Table 3: https://www.samedical.org/file/1836).

Table 4

Pre-course v. post-course confidence by knowledge area

Knowledge area	Pre-course median (IQR)	Post-course median (IQR)	p-value
				Recognition and assessment of the seriously ill patient	3 (3 - 4)	4 (3 - 4)	<0.001
Diagnosis and management of acute respiratory failure	3 (2 - 4)	4 (3 - 4)	<0.001
Mechanical ventilator low fidelity/high fidelity	3 (2 - 4)	4 (3 - 4)	<0.001
Management of the paediatric airway	2.5 (2 -3)	4 (3 - 4)	<0.001
Monitoring oxygen balance and acid-base balance	3 (2 - 4)	4 (3 - 4)	<0.001
Diagnosis and management of shock	3 (2 - 4)	4 (3 - 4)	0.006
Assessment of the critically ill child	3 (2 - 4)	4 (3 - 4)	<0.001
Diagnosis and management of neurological emergencies	3 (2 - 4)	4 (3 - 4)	<0.001
Diagnosis and management of obstetric haemorrhage	3 (2 - 3)	4 (3 - 4)	<0.001
Diagnosis and management of preeclampsia/eclampsia	3 (2 - 4)	4 (3 - 4)	<0.001
Management of life-threatening electrolyte and metabolic disturbances	3 (2 - 4)	4 (3 - 4)	<0.001
Life-threatening infections: diagnosis and antimicrobial therapy selection	3 (2 - 4)	3.5 (3 - 4)	<0.001
Performing the basic trauma and burn support	3 (2 - 4)	4 (3 - 4)	<0.001
Diagnosis and management of acute coronary syndrome	2 (2 - 3)	3 (3 - 4)	<0.001
Special consideration patients	3 (2 - 4)	4 (3 - 4)	<0.001
Diagnosis and management of sepsis	3 (2 - 4)	4 (3 - 4)	<0.001

In summing the Likert-type individual responses for all 16 knowledge areas into a Likert score for each individual, the mean pre-course confidence score was 46.8 (14.3) (95% CI 42.0 - 51.7), the mean (SD) post-course confidence score was 58.4 (12.5) (95% CI 54.1 – 62.6), and the difference (improvement in confidence) was 11.6 (9.8) (95% CI 8.2 - 14.9; p-value <0.001). With gender, age, site of practice, primary role, years of experience, and number of years spent working in the ICU as covariates (all grouped except for gender owing to small sample size), the beta regression model revealed no significant predictors of improved confidence Supplemental Table 4: https://www.samedical.org/file/1836).

Survey participants also indicated that they previously had average experiences learning in an interprofessional collaboration format (median=3 ‘Average’) and that they only occasionally participated in interprofessional learning experiences (median=3 ‘Sometimes’). Most participants (94.3%) agreed that the FCCS course as performed in an interprofessional environment positively impacted learning and the vast majority (97.2%) also agreed that the course would positively impact their practice Supplemental Table 2. Participants described the content areas of acute coronary syndrome and shock management in general as the most difficult areas. When asked how the FCCS course could improve participants’ knowledge in Rwanda, respondents indicated they would like to be offered the course more frequently and would like to have the course held over more than 2 days (Supplemental Table 5: https://www.samedical.org/file/1836).

Discussion

One of the fundamentals of critical care is early identification, evaluation of the critically ill patient and the ability to provide these patients with the needed therapies quickly to reduce mortality. For example, early recognition of sepsis and early antibiotic administration are now enshrined in the surviving sepsis guidelines as methods to reduce mortality.[[14]] The pre-test scores clearly demonstrated knowledge deficits in the ‘Recognition and assessment of the seriously ill patient’ content , with a significant increase in scores in the post-course test. As an extension, the knowledge gains from the ‘Life-threatening infections: diagnosis and antimicrobial therapy’ section highlighted the need for increased understanding of sepsis recognition and sepsis management.

Acute respiratory failure and mechanical ventilation are managed by anaesthesiologists and ICU nurses in Rwanda – there are no dedicated respiratory therapists. The anaesthesiologists continue to support the operating rooms contemporaneously with ICU oversight; hence, the burden of care falls primarily upon the bedside nurse to manage acute issues with the ventilator. While there was reasonable baseline knowledge of recognition of acute respiratory failure and management, knowledge gains were not significant, suggesting that the course content delivery should be tailored more methodically to meet local needs. This approach may entail more hands-on sessions or modifications in the didactic sessions to reflect the types of mechanical ventilators available as well as modes of ventilation employed.

The course content relating to acute coronary syndrome had the most intriguing results. Participants reported on the survey that they had a limited pre-existing knowledge base. While the acute coronary syndrome pre-course test score was high, the post-course test score was actually lower than the pre-course score and was the only score to decrease. It is possible that exposure to acute coronary syndrome in Rwandan medical personnel is low – due perhaps to a low incidence, under-diagnosis and little exposure to such courses as Advanced Cardiac Life Support (ACLS) – resulting in generally poor understanding of the material covered. Similar results were seen in the FCCS course conducted in Kenya by MacLeod et al. [[11]], in which the only area in which confidence did not improve was also cardiology content. This finding may suggest that there is some fundamental aspect of the way cardiology content is delivered in the FCCS course that needs modification.

Blunt trauma, including head trauma from road accidents, is a significant cause of critical injury. For example, 44% of admissions in one of Rwanda’s teaching hospital’s ICUs are from head trauma.[[15]] Accordingly, the course content for neurological support was adapted to focus on non-invasive neurological assessment to guide treatment owing to the absence of intracranial monitoring devices or advanced treatment options including thrombolytic therapies and embolectomies. Participants demonstrated a significant increase in their confidence in the management of neurological emergencies.

Maternal mortality remains high, despite evolving midwife programmmes.[[16]] In the content area ‘Critical care in pregnancy’, which includes pre-eclampsia and haemorrhage, significant knowledge gains were noted after the course. This underscores the utility of a more comprehensive programme disseminated widely in Rwanda, targeted at hospital-based and community-based providers. Additionally, the feasibility of obstetric haemorrhage teams should be explored in resource-constrained settings.

Finally, we found it encouraging that nearly half of all participants indicated that the FCCS course should be offered at least annually. Participants also felt that the course should be conducted in other areas of Rwanda and that local instructors should be involved. We conducted the FCCS course again in 2019 and included two ‘Train the Trainers’ days in which we educated eight Rwandan physicians and nurses on methods of teaching the FCCS course. To make this programme sustainable, it would require a critical mass of trainers and materials tailored to local needs. The authors continue to work collaboratively with local clinicians identified to be trainers and are working to establish a comprehensive multidisciplinary critical care training programme. Our study group has a comprehensive plan of having an ongoing FCCS course the year round and educating more local trainers, including clinical preceptorships at the bedside to improve patient outcomes. Our hope is that this will promote sustainability and dissemination of the course in Rwanda; success of this intervention will be the subject of future investigation.

Like the FCCS courses conducted in Zambia (in which FCCS was combined with the Acute Trauma Care course) and Kenya, we saw gains in knowledge and confidence. We also agree with the authors of the Kenyan study that modification of course content to meet local capabilities is required. To this end, we propose future research in Rwanda to determine the rate of performance of critical care management best practices. This will serve to assess the epidemiology and outcomes of patients admitted to the ICU to highlight gaps in performance of areas in which best practice standards are not maintained. Further, this endeavour will facilitate the identification of practice areas in which focused educational efforts, and a systematic evaluation of available material and personnel resources, could have the greatest impact on patient care. Through these efforts, a platform will be developed through which additional educational initiatives, including an even-further tailored version of the FCCS course, may be developed. Future research would also benefit from an ICU registry through which we can determine the efficacy of educational interventions. To date, no such registry exists in Rwanda and few have been developed in low-income countries.[[17,18]]

Study limitations

There are several limitations in the present study. The sample size was small and, while we were able to detect changes in knowledge and confidence in most areas, a larger sample size would have allowed for comparing degrees of changes and the identification of predictors of confidence improvements. Based on the methodology employed, we are unable to determine if the knowledge gained actually translates into changes in practice in the critical care setting. Finally, while the purpose of this study was to determine whether there was improvement in knowledge and confidence as a result of the material delivered in the course – and we feel confident that this was accomplished – we did not formally assess the content of the education with respect to its consistency with the technology and care processes routinely available at the practitioners’ disposal.

Conclusions

Deploying the established FCCS course improved Rwandan healthcare provider knowledge and confidence across most critical care content areas. Therefore, this course represents a good first step in bridging the gaps noted in emerging critical care systems. Recognising the high morbidity and mortality faced in LMIC emerging critical care systems requires attention as critical illness places significant burdens on overall global health outcomes. As such, it is imperative to evaluate programmes and resources to strengthen global health critical care. Further work is warranted in addressing sustainability, avoidance of redundancy, alignment of the education with local priorities, patient care resources and practices, retention of newly trained faculty, evaluating post-graduate critical care training pathways, and access to continuing medical education.

Supplemental Tables