Medical student satisfaction and confidence in simulation-based learning in Rwanda - Pre and post-simulation survey research.

INTRODUCTION: Simulation-based learning (SBL) has been shown to effectively improve medical knowledge, procedural proficiency, comfort with undertaking taught tasks, inter-professional communication, teamwork and teaching skills. This study aimed to evaluate Rwandan medical students' attitudes, satisfaction and confidence level with SBL.
METHODS: Fifth year medical students at the University of Rwanda were given a short course on paediatric acute care using simulation. The simulation sessions were locally developed cases based on the pRRAPID materials, developed at the University of Leeds (UK). Equipment included low fidelity infant mannequins, basic airway devices, IV access, and monitoring. A four-part, Likert-scale questionnaire was distributed to medical students before and after their four-week simulation program.
RESULTS: 57 pre-simulation and 49 post-simulation questionnaires were completed. Confidence in skills increased in all fifteen domains of the questionnaire with the total skill confidence score rising from 44.0 (±12.3) to 56.2 (±8.8) after the simulation-based intervention (p < 0.001). Satisfaction and attitudes towards simulation-based learning in this setting were very positive.
CONCLUSION: The simulation-based intervention was well received by students in this setting. Satisfaction was high and the simulation exercise increased the students' confidence. Previous research has demonstrated that SBL is effective and the results of this study now demonstrate that it is well received in our setting. As we move from knowledge-based education to a competency-based education culture, faculties in this setting should invest in providing SBL opportunities throughout the medical school curriculum.

African relevance

Simulation Based Learning (SBL) is an effective teaching modality.

There is a lack of evidence regarding SBL on the African continent.

This study demonstrates an increase in confidence level in Rwandan students using SBL.

Introduction

Simulation-based learning (SBL) introduces learners to scenarios and environments designed to closely approximate real-world situations and can be used for training and evaluation [1,2]. SBL has been used in high-risk industries such as aviation, nuclear power and the military [3]. Subsequently, SBL has been adopted in medical education giving learners the opportunity to acquire competencies in aspects of medical care such as resuscitation skills, technical procedures, behaviour and inter-professional communication skills [1]. Through SBL, students are exposed to clinical events without putting the health of real patients at risk [4].

SBL has been shown to effectively improve medical knowledge, procedural proficiency, comfort with undertaking taught tasks, inter-professional communication, teamwork and teaching skills [5., 6., 7.]. SBL has been shown, in several settings, to be more favourable than non-simulation interventions, such as didactic or observational learning, in terms of satisfaction and for developing competencies [5., 6., 7.]. Furthermore, simulation is becoming more accepted as a form of medical training because of the safety of environment, reproducibility, standardization of content and the ability to simulate critical events [6].

Regarding acceptance and satisfaction, the majority of literature demonstrates that medical students enjoy SBL and that it increases their confidence [2,7., 8., 9.]. Medical students report that SBL is a good method which allowed them to apply their theoretical knowledge in a safe environment. SBL improves students' peer learning and systematic approaches to cases as well as their competency, team cooperation and learning experiences [10].

However, SBL is known to have some challenges; some students report challenges in communication with patients, as mannequins cannot give verbal and non-verbal feedback, and that initially, they feel nervous and stressed by the scenarios [11]. Others have described not being satisfied with the laboratory facilities, the time required for learning sessions, and challenging interactions between students during scenarios. Barriers such as costly and cumbersome equipment, staff skills and small class sizes which require higher numbers of faculty to delivery the sessions have also been a challenge [9].

In low-income countries (LICs), there is frequently a low ratio of healthcare professionals (HCPs) to the population [12]. This poses two problems: firstly, doctors are expected to be competent, independent healthcare providers immediately upon graduating, and secondly, there is limited faculty available for training. Studies show that SBL enables medical students to acquire the competencies necessary to become effective medical doctors following graduation, however running simulation still requires staff [13]. The World Health Organization (WHO) recommends SBL in both developed and developing settings, but it may be that in a low-resource setting, low-fidelity simulation is more appropriate and available [14]. Meaney carried out a prospective quasi-randomized interventional trial in Botswana comparing instructor led simulation with manikin feedback and self-directed learning; they found little difference between the groups [15]. This might make the argument for higher fidelity simulation in LICs where lack of faculty is a major challenge.

A qualitative research study in 12 low- and middle-income countries (LMICs) from Africa, Latin America, and Asia evaluated barriers to patient care and how SBL can improve inter-professional communication and decision-making processes. Results found that simulation has been effective in overcoming challenges faced in LMICs and it was suggested to be a key component in further education systems for medical students in this setting [16]. Locally, at the University of Rwanda, simulation facilities have been established in partnership with collaborating partners [17,18]. SBL has been found to be effective, but there is little in our own setting to report on the satisfaction of students and whether it increases their confidence as future practicing HCPs.

This research project aimed to evaluate Rwandan medical students' attitudes, satisfaction and confidence level with simulation-based learning (SBL). It aimed to answer the specific research question: In a resource-limited setting, does the use of SBL increase the confidence of medical students in the management of acutely unwell children?

Methods

Before and after questionnaire research of a medical education intervention. Reporting of this study has been checked and verified in accordance with the GREET and the extensions to the CONSORT and STROBE statements checklist for simulation research [19,20].

University Teaching Hospital of Kigali (CHUK) is a tertiary hospital in Kigali, the capital of Rwanda. CHUK is one of the principal clinical teaching sites for students from the University of Rwanda (UR). Medical students at UR follow a traditional approach, primarily reliant on theoretical classroom training and hospital based clinical attachments with supervision limited by the availability of faculty in this setting [17].

Upon graduation, medical students can often be placed in isolated rural settings, immediately responsible for many patients, including paediatric care. The objective of the intervention was for students to develop competencies in ABCDE clinical assessment, inter-professional communication skills, decision making, time management, patient safety and prescribing, all in a protected environment. Inter-professional communication skills were taught using the SBAR (Situations, Background, Assessment and Recommendation) method.

Fifth-year medical students who were on their paediatric clinical rotations at CHUK were invited to participate in simulation scenarios based on the “Recognizing and Responding to Acute Patient Illness and Deterioration” (RRAPID) course [21]. Prior to simulation sessions, students were given free access to the pRRAPID online materials via the website and mobile phone application. [22]. A half-day, peer-taught symposium was delivered in conjunction with the simulation sessions to cover the important theoretical principles covered in the simulation sessions.

During the simulation sessions a low-fidelity mannequin was used in conjunction with a box of standard medical equipment. A low-fidelity vital-signs monitor was produced by laminating a picture of a monitor allowing the use of a white-board pen. Facilitators used pre-designed, locally specific, piloted simulation scenarios. The facilitators were paediatricians or paediatric residents who had been trained in how to teach using SBL. Groups were 5–8 students in size. Typically, the facilitator would ask three students to role-play a clinical scenario such as a child with difficulty breathing. While the team managed the ‘patient’, the remainder of the group observed the scenario and then provided the role-players with constructive feedback. The simulations were immersive and where possible, were held on the ward to maximize realism. The sessions ran for 1.5 h weekly over four consecutive weeks. No unplanned modifications in the intervention were made during the course of study. No measurement of attendance was taken; however, our experience is that these sessions were highly valued and attendance was very good. Several facilitators taught the scenarios simultaneously, with a faculty debrief after the session to ensure that the session had gone as planned. The sessions were co-coordinated by a student class-representative working with the faculty to ensure that frequency, timing and duration of sessions were delivered as scheduled.

Fifth-year medical students at UR in clinical rotation on the Paediatric ward at CHUK took part in the simulation sessions. Non-participation in the research did not exclude students from the SBL sessions. Students who voluntarily gave informed consent to participate were included. Participants were enrolled before their first simulation session. They were sent a link to the online questionnaire by the class-representative on the class WhatsApp Group. The questionnaire contained an explanation of the study and a consent statement in English. Unique Individual Patient Identifier codes were used when completing the questionnaires. Patient-identifiable information was kept in a separate database to protect confidentiality. The project proposal was reviewed by the research ethics committee (REC) at CHUK (Ref: EC/CHUK/586/2018).

We developed a Likert-scale questionnaire, structured in four sections that was based on previous simulation studies [23., 24., 25., 26.]. We used previously described questions assessing SBL, however we did not validate our own questionnaire. The sections of the questionnaire were: i. Self-reporting of skills; ii. Confidence; iii. Satisfaction; iv. Attitudes to SBL. The questions were reviewed by three local paediatricians with experience in medical education to ensure content validity. The first section (self-reporting of skills) was sent before the simulation sessions started. A complete questionnaire comprised of all four-sections was sent to students after completing the simulation sessions. The intervention did not include practical exposure to intraosseous (IO) insertion, use of a defibrillator or recognizing cardiac rhythms. Questions regarding these skills were included in the questionnaire to assess for acquiescence bias before and after the SBL experience. The questionnaires were implemented either electronically (Google Forms®) or in printed, paper-form for those without internet/phone access or those who did not feel comfortable with using electronic media. Data were downloaded from Google Forms® into Microsoft Excel® and analysed using SPSS® v24. English is the official, academic language of instruction at the UR. The simulation sessions were taught in English and therefore the questionnaire was written and completed in English.

Each Likert item was analysed and described using means and then combined to form total section scores which were weighted out of 100 to give meaning [27]. Comparative statistics used student's t-test. The questionnaires were completely anonymous and therefore paired analysis was not possible.

Results

The study period was from the 9th April 2018 to the 2nd of July 2018 and a total of 101 students attended the simulation sessions. 57 pre- and 49 post-simulation questionnaires were completed. Response rate was therefore 56.4% and 48.5% for the pre- and post-SBL questionnaires. During data cleaning, duplicate responses were found and removed; four from the pre- and one from the post-simulation questionnaires. Three subjects in the post-simulation questionnaire only gave data on the confidence level (Table 1) and did not complete the second part of the questionnaire and therefore, were not included in the post-simulation analyses (Table 2). The mean age of the participants was 25.8 years (SD ± 2.74), no other demographic details were collected.

Table 1

Skill confidence levels pre and post simulation.

“Rate from 1 to 5 (circle the number) with 1 being not confident to 5 being highly confident for the following skills”	Pre-simulation mean (n = 53)	Post-simulation mean (n = 48)	Difference	p-Value
Assessing perfusion	3.02 (±1.15)	4.35 (±0.76)	1.34	p < 0.001
Knowing when to administer IV fluid bolus	2.96 (±1.16)	3.94 (±0.93)	0.98	p < 0.001
Positioning the head correctly	3.42 (±1.13)	4.38 (±0.82)	0.96	p < 0.001
Recognizing apnoea	3.09 (±1.08)	4.04 (±0.90)	0.95	p < 0.001
Assessing mental status	3.15 (±1.05)	4.06 (±0.86)	0.91	p < 0.001
Providing ventilation	3.21 (±1.10)	4.08 (±0.92)	0.88	p < 0.001
Leading a code/resuscitation	2.66 (±1.11)	3.52 (0.90±)	0.86	p < 0.001
Recognizing shock	3.15 (±1.18)	3.96 (±0.80)	0.81	p < 0.001
Recognizing respiratory distress	3.87 (±0.94)	4.65 (±0.57)	0.78	p < 0.001
Doing chest compressions	3.32 (±1.14)	4.06 (±0.93)	0.74	p = 0.001
Placing an IV cannula/line	3.02 (±1.2)	3.69 (±0.95)	0.67	p = 0.002
Placing an intraosseous (IO) needle	1.62 (±1.02)	2.29 (±1.11)	0.67	p = 0.002
Using a defibrillator	1.60 (±0.95)	2.15 (±0.97)	0.54	p = 0.005
Recognizing cardiac rhythms	2.23 (±1.07)	2.77 (±1.06)	0.54	p = 0.012
Communicating with the team	3.68 (±1.2)	4.21 (±0.71)	0.52	p = 0.008
Total skill confidence score	44.0 (±12.27)	56.2 (±8.83)	12.15	p < 0.001

Q Questions ordered by magnitude of pre/post confidence change, not in the order presented in the questionnaire; L Likert scale used: strongly disagree = 1; disagree = 2; undecided = 3; agree = 4; strongly agree = 5; ±SD.

Table 2

Post-simulation confidence, satisfaction and attitudes.

	Post-simulation means (n = 45) (±SD)
Confidence
I am confident working in a team to care for a sick infant	4.22 (±0.79)
I am confident recognizing a sick infant	3.85 (±0.84)
I am confident managing a sick infant	3.67 (±0.63)
I am confident using the equipment and paperwork required to manage a sick infant	3.54 (±0.72)
Total post-simulation confidence score	15.28/20 (±2.46)
Satisfaction
I was able to ask questions to improve my knowledge and understanding	4.63 (±0.65)
Feedback provided was constructive	4.46 (±0.66)
I had the chance to work with my peers during the simulation	4.43 (±0.69)
The simulation allowed me to analyse my own behaviour and actions	4.35 (±0.67)
The simulation was designed for my specific level of knowledge and skills	4.33 (±0.85)
The scenario resembled a real-life situation	4.20 (±0.89)
The simulation provided me with learning materials and activities to promote future learning	4.07 (±0.80)
Total post-simulation satisfaction score	30.46/35 (±3.97)
Attitude
I would like more simulation teaching in the medical school curriculum	4.87 (±0.40)
I think simulation is a good way of learning clinical skills	4.76 (±0.48)
Simulation covers important topics such as paediatric resuscitation	4.15 (±0.94)
I would like to teach simulation	3.57 (±0.91)
Total post-simulation attitude score	17.24/20 (±2.00)

Q Questions ordered by magnitude of pre/post confidence change, not in the order presented in the questionnaire; L Likert scale used: Strongly Disagree = 1; Disagree = 2; Undecided = 3; Agree = 4; Strongly agree = 5; ±SD.

Students reported improved skills confidence in every domain, but there was a range of improvements (Table 1 and Fig. 1). The most significant improvement was in assessing perfusion, and the smallest improvement was in communicating with the team. For all the results our p-value was <0.001 demonstrating statistical significance. The overall confidence score for the 75 points of the 15 items was 44.0 (Mean 2.93), with an increase of 12.15 points through the domains after the SBL (total 56.2, mean 3.74). No items reduced in confidence.

Fig. 1

Confidence levels pre and post simulation.

Questions ordered by magnitude of pre-simulation self-reported score, not in the order presented in the questionnaire.

Highest satisfaction was the ability to ask questions to improve their knowledge and understanding; lowest satisfaction was regarding learning materials to promote future knowledge. Students enjoyed the simulation sessions and showed high interest in promoting more simulation-based teaching in their medical curriculum; yet, interestingly, they showed less interest in teaching simulation in the future (Table 2).

Discussion

We aimed to evaluate Rwandan medical students' attitudes, satisfaction and confidence level following the introduction of simulation-based learning (SBL). Students reported immediate increased comfort in assessing and treating acutely ill and deteriorating children and that they had high satisfaction levels with positive attitudes of SBL.

Significant progress has been made globally in reducing child mortality over the past decades. Despite this progress, an estimated 5.4 million children under the age of five died in 2017. Approximately half of these deaths occurred in sub-Saharan Africa [28]. Therefore, it is paramount that optimal teaching methods are identified to give HCPs the competencies they need to care for these sick children upon graduation. This study supports the use of simulation methods.

There were varying levels of skills confidence before simulation, the lower scores may represent lack of exposure with teaching through traditional methods [29]; this highlighted a learning need which SBL could achieve. The most significant improvement in skills after simulation training was in “Assessing perfusion”; this was emphasized in the Circulation component of each scenario and we postulate that repetition helped improve confidence in this critical skill. The highest post-SBL confidence was in recognizing respiratory distress. This is important as respiratory disease is globally the major contributor to non‑neonatal mortality of children under five-years of age [30]. The increase in confidence in this domain was relatively modest, reflecting the high pre-SBL confidence. Respiratory distress is a common complaint in Rwanda, so it is likely that students had significant pre-SBL exposure to clinical cases.

The intervention did not include practical exposure to intraosseous (IO) insertion, use of a defibrillator or recognizing cardiac rhythms. These three items saw modest improvements but represented the smallest increases in confidence, except for “communicating with the team” which had a high pre-SBL confidence and therefore, only a small range for potential improvement. This reveals that there was some acquiescence bias or that the SBL package had given the students a more global sense of confidence in themselves. Intraosseous insertion is an important competency in our setting, and it would be ideal to include it in the teaching curriculum. However, we only had low fidelity resources available and using animal tissue was not feasible. A cost-effective/pragmatic solution for low-fidelity insertion technique would be a welcome innovation for educators in this setting.

High-fidelity simulation equipment is costly and therefore not accessible to all educators [31]. Fidelity plays an important role in the choice of an appropriate simulation for a specific task but high-fidelity simulation is not always superior as it is dependent on the type of task and the learner's level [32]. Low-cost simulation resources and activities have been identified and described in the literature. Rather than focusing on the fidelity of the mannequins or equipment used, Ellianas describes the importance of identifying high-stakes performance tasks with educational gaps as a strategy [33]. An alternative is to use standardised patients, but this also has practical and cost implications [34]. It is important to note that our largest cost, was the labour cost of writing and piloting the simulation cases which are now available for future sessions in Rwanda, and to other educators in the region, with only minor amendments required.

Post simulation confidence was highest for “working in a team to care for a sick infant” (Table 2). This supports the use of SBL for practicing teamwork which is crucial in a resuscitation scenario. Students were moderately confident at “recognizing a sick infant” and “managing a sick infant” which is consistent with available literature [8]. This confidence in the simulation environment needs to transfer to the clinical environment to improve the quality of patient care [35]. The least confidence was in “using equipment and paperwork”; this may be because hands on experience was a new form of participation for many students, as opposed to passively observing. This increase in confidence in students is important, however, relating confidence with competence is complex, and it has been demonstrated that high levels of confidence doesn't necessarily translate into an increase in competency [36].

Satisfaction with the SBL package was high. Medical teaching in Rwanda continues to rely on didactic approaches where the large numbers of students may restrict their ability to ask questions to fill gaps in their understanding. Students attending this teaching package reported positively that they could “ask questions to improve my knowledge and understanding” [9]. Students were satisfied with SBL in that it “resembled real-life situations”. Students felt the “feedback provided was constructive” which is consistent with previous evidence indicating that debriefing is a core part of SBL, and also reflected the importance of providing training to the faculty who delivered the sessions [37., 38., 39., 40.]. Students enjoyed that simulation was designed for their specific level of knowledge and skills, reflecting the piloting work that was done prior to launching. Students were all given free access to the pRRAPID app and internet resources, 35 participants agreed (n = 20) or strongly agreed (n = 15) that they had the “learning materials and activities to promote future learning”. With a mean of 4.07, this was the lowest scoring item in satisfaction. This may suggest that the faculty didn't adequately signpost to the free eBook/app as 14 participants didn't agree with this statement [21,22].

After exposure to simulation sessions, medical students at the University of Rwanda have shown high acceptance levels. UR students rated simulation highly; they agreed it is a “good way of learning clinical skills”, “covers important topics” and they would “like more simulation teaching”. This is consistent with findings in pre-existing literature [16,41,42]. The highest scoring item on the questionnaire was the desire for more SBL in the medical school curriculum (Table 2). This desire for more SBL has been shown in South Africa where clinical simulation has been recognized to enhance medical curriculum and is now considered a required component in the curriculum [41]. Interestingly, students reported less desire to “teach simulation”; this may represent a lack of confidence and familiarity with the simulation process, and a lack of comprehension of the value of teaching for their own learning. Once established, an interesting future piece of work might investigate the use of near-peer SBL activities to enable students to maximize their exposure to SBL and also develop their own teaching skills.

English is the official, academic language of medical students. However, for most it is their second language and therefore assessing concepts of confidence and satisfaction in this language may have affected the responses. Alternative explanations for the improvement in scores post-simulation includes acquiescence bias, exposure other than simulation (e.g. alternative educational activities) or facilitator differences. As the data was entirely anonymous, without coding identifiers, there was no mechanism to statistically pair the changes in confidence scores. Furthermore, no control group was used. A sample size calculation had not been undertaken as we had a limited time period and student cohort to investigate. The majority of our data is descriptive and for the comparison all the pre and post-simulation confidence the results were significant, therefore a type II error is unlikely. More pertinent is that the second questionnaire was undertaken immediately after the SBL sessions, and no delayed measurement was made. We are therefore unable to report if the effects on confidence were sustained and it is feasible that they may diminish over time.

SBL is in its infancy in this region and this study is a good starting point for justifying the use of this teaching method. We created a well-designed questionnaire based on previous SBL and the questionnaire was piloted to ensure understanding and length to avoid “questionnaire fatigue.” Furthermore, this study was interventional, and results of the intervention were assessed. However, this study focused on 5th Year medical students only, therefore, information about satisfaction in different levels of medical students was not provided. Moreover, our study has no comparable or control group which is considered as a limitation. Finally, our measurement tool does not allow for us to assess if their new skills will transfer into clinical practice [42].

Our participants were in their penultimate year of studies and many were experiencing SBL for the first time. We speculate that simulation would have a greater impact if it is started early in the curriculum and was consistently applied as students gain new competencies [42]. SBL offers excellent exposure to students without risking patient safety and health [4]. We found that with simulation in a limited resource country, students acquired the confidence to recognize sick and deteriorating children and act accordingly.

Conclusion

Previous research has demonstrated that SBL is an effective strategy to enhance medical student's competencies. This study demonstrates that Rwandan students increase their confidence level in recognizing and responding to acutely unwell and deteriorating children and that SBL methods were well received as appropriate teaching methods. Considering our findings, we recommend more SBL in undergraduate education in this setting.

The scenarios for this intervention were designed specifically for the resource-limited setting and were piloted prior to use. Further improvements have been made based on the results of the study and on feedback from facilitators and students. The resources are freely available to any other faculty on contacting the corresponding author.

Dissemination of results

The results were presented as a poster presentation at the 2018 AfCEM conference held in Kigali, Rwanda. The findings have been shared with staff members within the paediatric department at UR.

Authors' contributions

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: ST contributed 50%, PTC 30%, AW 15% and HS 5%. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Conflicts of interest

The authors declare no financial conflicts of interest. The Rwanda Paediatric Association (RPA) gained a small fund to implement the pRRAPID package. This was awarded by the UK charity: OptIn (Overseas Partnering and Training Initiative), based in the Leeds Children's Hospital, an affiliate of the University of Leeds. PTC wrote the pRRAPID materials with a faculty member at the University of Leeds; however, he does not receive financial payment from University of Leeds.