Acquiring sepsis competencies through simulation-based learning bundle during intermediate care unit internship.

ABSTRACT: Intermediate care units (ImCUs) have been shown as appropriate units for the management of selected septic patients. Developing specific protocols for residents in training may be useful for their medical performance. The objective of this study was to analyze whether a simulation-based learning bundle is useful for residents while acquiring competencies in the management of sepsis during their internship in an ImCU.A prospective study, set in a tertiary-care academic medical center was performed enrolling residents who performed their internship in an ImCU from 2014 to 2017. The pillars of the simulation-based learning bundle were sepsis scenario in the simulation center, instructional material, and sepsis lecture, and management of septic patients admitted in the ImCU. Each resident was evaluated in the beginning and at the end of their internship displaying a sepsis-case scenario in the simulation center. The authors developed a sepsis-checklist that residents must fulfill during their performance which included 5 areas: hemodynamics (0-10), oxygenation (0-5), antibiotic therapy (0-9), organic injury (0-5), and miscellaneous (0-4).Thirty-four residents from different years of residency and specialties were evaluated. The total median score (interquartile range) increased significantly after training: 12 (25) vs 23 (16), P = .001. First-year residents scored significantly lower than older residents at baseline: 10 (14) vs 14.5 (19), P = .024. However, the performance at the end of the training period was similar in both groups: 21.5 (11) vs 23 (16), P = 1.000. Internal Medicine residents scored significantly higher than residents from other specialties: 18 (17) vs 10.5 (21), P = .007. Nonetheless, the performance at the end of the training period was similar in both groups: 24.5 (9) vs 22 (13), P = 1.000.Combining medical simulation with didactic lectures and a rotation in an ImCU staffed by hospitalists seems to be useful in acquiring competencies to manage critically ill patients with sepsis. We designed a checklist to assure an objective evaluation of the performance of the residents and to identify those aspects that could be potentially improved.

Introduction

Intermediate care units (ImCUs) have been developed to act as “step-up” or “step-down” units, providing specialty care for cardiac, neurologic, respiratory, or surgical conditions[ for some complex patients. ImCUs warrant the diagnosis, treatment, and management of patients whose clinical condition exceed monitoring and nursing care in conventional wards but do not require Intensive Care Unit (ICU) assistance or whose admittance to ICU is precluded due to their multiple comorbidities.

Sepsis is a life-threatening disease with increasing incidence and it has become a relevant public health problem, affecting millions of people and becoming one of the most important causes of in-hospital mortality.[ Early recognition of the problem, an early hemodynamic resuscitation, and an adequate antimicrobial therapy implementation are key for decreasing in-hospital mortality and length of stay.[ For these reasons, developing specific learning protocols in the management of sepsis becomes essential for education of medical residents.

Different medical simulation strategies have demonstrated their effectiveness to improve medical knowledge and procedural skills[ with a decrease in procedure-related complications.[ However, few of them focus on medical performance in a sepsis scenario.[ In addition, these studies are heterogeneous in terms of analyzed measurements, objectives, and the modality of simulation.

The objective of this study was to analyze the utility of a simulation-based learning bundle (SBLB) to acquire competencies in the management of sepsis for residents from different specialties during their internship in an ImCU staffed by hospitalists.

Methods

Data source and study population

We conducted a prospective study at Clinica Universidad de Navarra (CUN), a tertiary-care academic medical center in Pamplona, Spain. We enrolled those residents who performed their internship in our ImCU from January 2014 to September 2017. ImCU at CUN is staffed by hospitalists. It is independent and located parallel to ICU and it has a 9-bed capacity.

Due to the educational nature of the project, informed verbal consent was considered enough and it was obtained from all participants. The study was approved by the IRB of the CUN (project n. 2019.088).

We included residents from different years and different medical and surgical specialties according to the Spanish Residency Program. In order to enter in this program, medical school graduates must take a national exam. They are then ranked by their performance on this exam and they sequentially choose from the available residency training positions offered by the Spanish Ministry of Health. These positions covered 47 specialties distributed over more than 200 training centers.

Simulation-based learning bundle (SBLB). Simulation technology and resident evaluation

The 4 pillars of the SBLB are:

sepsis scenario in the simulation center,

instructional material,

sepsis lecture, and

management of septic patients admitted in an ImCU during their internship supervised by hospitalist staff.

A sepsis-case scenario was displayed in the simulation center that is located close by the Hospital, at the Medical School Building. The simulation center is equipped, among others, with an adult patient simulator (SimMan 3G Mannequin , Laerdal Medical, Stavanger, Norway) and a wireless communication and control room.

During their rotation in the ImCU (usually 1–2 months per rotation), residents were evaluated twice by the staff – before and after the rotation – in the simulation center. Each resident was his/her own control. For evaluating purpose and according to sepsis international guidelines, we previously developed a checklist including 5 areas that residents must fulfill during their performance in the simulation center (Table 1): hemodynamics (0–10), antimicrobial therapy (0–9), oxygenation (0–5), organic injury (0–5), and miscellaneous (0–4). Total score could vary from 0 to 33 points according to level of completion – 0 not performed, 1 performed. Responses to all questions were equally weighted. The trainees were encouraged to verbalize their thoughts and requirements in terms of blood tests, radiology … so the staff located in the control room could fulfill the checklist accurately.

Table 1

Checklist.

Parameter and score

Hemodynamics (0–10)

Peripheral volume access (0–1)

Assess volume status deficit (0–1)

Intravenous crystalloid/coloid fluid administration (0–1)

MAP >65 mm Hg (0–1)

Central line placement (timing and patient volume status through CVP measure, 8–12 mm Hg) (0–2)

Use of vasoactive agents (agent choice, dose, and administration site) (0–3)

Lactate levels assessment (concept and dynamic evaluation) (0–1)

Antimicrobial therapy (0–9)

Timing < 1 hour (0–1)

Empiric combination therapy (0–1)

Infection source (0–1)

Community adquired/nosocomial/health-care related (0–1)

Multi-resistant risk (antibiotics < 1 month, previous germs, recent admission) (0–3)

Prosthetic device (0–1)

Immunosuppression (0–1)

Oxygenation (0–5)

SpO2 >90% (0–1)

PaO2/FIO2 ratio <150 (0–1)

Supplementary oxygen (nasal cannula, Venturi mask, noninvasive ventilation, orothraqueal intubation …) (0–1)

ScvO2 ≥70% (0–1)

Transfusion of red cells according to hemoglobin/hematocrit levels (0–1)

Organic injure (0–5)

Kidney function assessment and urine output ≥0.5 mL/kg/hr (0–1)

Liver function assessment (0–1)

Hematologic and coagulation assessment (0–1)

Neurologic status (Glasgow scale/encephalopathy) (0–1)

Respiratory status (ALI/ARDS) (0–1)

Miscellaneous (0–4)

Glucose control (0–1)

Complete blood test, cultures, chest X-ray pre and postcentral line (0–3)

The sepsis scenario was different in the first and second simulations. The scenario started at the Emergency Department where the patient had arrived, with a background and history of present illness in a chart given to the resident. Both cases lasted around 30 minutes depending on the management of the resident. The care team comprised 3 participants including the team leader, nurse, and a proceduralist. The simulation control team included a simulation controller and a task performance monitor. The speaker “voice” of the mannequin allowed standardization of responses to questioning and complaints of the patient. Results of blood tests, blood gases, chest x-ray, or other tests were given depending on the request of the residents during the simulation. Vital signs and clinical improvement or worsening of the patient also varied according to the instructions given by the trainees. For more details, please check supplemental digital content.

During their internship, residents learned about appropriate management of sepsis in those patients admitted to the ImCU. They also received instructional material and a 2-hour interactive sepsis lecture based on updated guidelines given by a hospitalist staff (FL, NFR). In an effort to standardize this lecture, both attending physicians provided the same information according to a previously developed script. After the second evaluation in the simulation center, a debriefing session was made with each trainee.

Statistical analysis

We calculated the median and interquartile range (IQR) for pre- and post-instruction test score at the simulation center. Because of the small size sample, we used the nonparametric alternative to paired T test: Wilcoxon signed Rank test. We used Mann-Whitney test to search for differences within different subgroups. Statistical significance was set at P < .05. We performed all statistical analyses using IBM SPSS statistical software, version 20.0 (IBM Corp, Armonk, NY).

Results

A total of 34 residents from different years of residency and specialties were evaluated before and after their internship in our ImCU. Table 2 shows general characteristics of those residents. Twelve different specialties participated in the study. About 25% of the residents were from Internal Medicine and almost half of the cohort were evaluated during their first year of residency.

Table 2

General characteristics.

Characteristics	Number (%)
Specialty
Internal medicine	9 (26.5%)
Other specialties	25 (73.5%)
Year of residency
1st yr	18 (52.9%)
2nd yr	11 (32.4%)
3rd yr	2 (5.9%)
4th yr	3 (8.8%)
Year of simulation
2014	10 (29.4%)
2015	6 (17.6%)
2016	8 (23.5%)
2017	10 (29.4%)

Only one of the 34 residents did not perform the first simulation and 5 of the 34 residents (14.5%) failed to complete the second simulation.

The minimum score was 1/33 in the first simulation and 14/33 in the second one whereas the maximum score was 26/33 and 30/33, respectively.

The total median score (IQR) increased significantly after training: 12 (25) vs 23 (16), P = .001. These differences between the first and the second simulation remained significant in each of the evaluated areas (hemodynamics, oxygenation, antimicrobial therapy, organic injury, and miscellaneous) (Table 3). “Hemodynamics” was the area where the physicians in training improved most. On the other hand, assessing “respiratory failure” by PaO2/FiO2 was generally absent even in the second simulation scenario (data not shown).

Table 3

Pre and post total scores.

Metrics	Basal	Final	P
Hemodinamics (0–10)	3 (9)	9 (6)	<.001
Antimicrobial therapy (0–9)	3 (7)	6 (7)	<.001
Oxygenation (0–5)	2 (5)	3 (3)	<.001
Organic injury (0–5)	2 (4)	3 (5)	<.001
Miscellaneous (0–4)	2 (4)	2 (3)	.003
Total	12 (25)	23 (16)	<.001

Year of residency

First-year residents scored significantly lower than older residents at baseline: 10 (14) vs 14.5 (19), P = .024. However, performance at the end of the training period was similar in both groups: 21.5 (11) vs 23 (16), P = 1.000 (Table 4).

Table 4

Pre and post total scores according to year of residency.

Metrics	1st year basal	Older basal	P	1st yr final	Older final	P
Hemodinamics (0–10)	3 (7)	5,5 (7)	0.005	8 (6)	9 (4)	0.249
Antimicrobial therapy (0–9)	2 (4)	3 (6)	0.021	5.5 (6)	6 (6)	0.559
Oxygenation (0–5)	2 (3)	2 (5)	0.06	3 (3)	4 (3)	0.170
Organic injury (0–5)	1 (3)	2 (4)	0.034	3 (5)	3 (4)	0.589
Miscellaneous (0–4)	2 (2)	2 (4)	0.168	2 (2)	3 (3)	0.374
Total	10 (14)	14,5 (19)	0.002	21.5 (11)	23 (16)	0.215

Internal medicine vs other specialties

Internal Medicine residents scored significantly higher than residents from other specialties: 18 (17) vs 10.5 (21), P = .007. Nonetheless, performance at the end of the training period was similar in both groups: 24.5 (9) vs 22 (13), P = 1.000 (Table 5).

Table 5

Pre and post total scores according to specialty (internal medicine vs others).

Metrics	Internal medicine basal	Others basal	P	Internal medicine final	Others final	P
Hemodinamics (0–10)	6 (6)	3 (9)	0,001	9 (3)	8 (6)	.279
Antimicrobial therapy (0–9)	3 (5)	2 (4)	0,020	6 (6)	6 (6)	.649
Oxygenation (0–5)	2 (4)	2 (5)	0,154	4 (2)	3 (3)	.184
Organic injury (0–5)	3 (4)	2 (3)	0,008	3 (2)	3 (5)	.237
Miscellaneous (0–4)	2 (3)	2 (3)	0,102	3 (2)	2 (3)	.103
Total	18 (17)	10.5 (21)	0,001	24,5 (9)	22 (13)	.075

Discussion

The present study shows the utility of simulation-based learning as a supplementary tool for the training of residents. Development of new technologies and reliable models in medical simulation have allowed us to incorporate instruments that favor learning and its evaluation in a controlled environment. Simulation-based learning has been tested before in different settings: Lenchus[ published the results of the University of Miami-Jackson Memorial Hospital Center for Patient Safety (Miami, Florida) where he developed a specific procedural instruction curriculum. Preinstruction and postinstruction scores of 60 trainees were analyzed, showing a significant improvement in medical knowledge and procedural skills. Authors like Gauger et al[ showed similar good results in laparoscopic surgeries. Simulation-based training has also been used in different clinical scenarios such as cardiopulmonary resuscitation.[ However, and despite its clinical relevance, only few have referred to adult septic patient simulation.[ Besides, these studies are heterogeneous in terms of analyzed measurements, objectives, and the modality of simulation.

Different technologies and modalities are used in simulation.[ In our study, we wanted to analyze the utility of an SBLB (that includes the combination of instructional material, a sepsis lecture, an internship in ImCU, and the use of simulated septic patient scenarios) as an instrument to train residents to acquire competences in the management of septic patients. According to our results and using a checklist to evaluate their performance, the residents improved significantly their capacity to manage septic patients in a simulated scenario. We would like to emphasize that it was not the purpose of the study to analyze individually the weight of each of the 4 pillars that compounds our SBLB. Implementing this same teaching model in different hospitals with residency programs and in different clinical scenarios might be useful, although we are aware that the needed infrastructure, timing, and human resources could limit this approach.

The measurement tool used to evaluate the performance of the different trainees was a checklist that we developed based on International Sepsis Guidelines. Similarly, Ottestad et al[ focus their study on a scoring system to rate the management of septic shock of different ICU teams in a patient simulator. The technical score system, with a maximum of 16, was based on the Surviving Sepsis Campaign guidelines. Hansel et al[ in a randomize trial comparing the influence of simulator training with a crew resource management course and control group, used a similar checklist with a maximum of 15 to evaluate medical students’ performance. In the same way, Li et al[ used a 21 items checklist to evaluate the residents in their study. In the current study, we developed a checklist based on international Sepsis guidelines. We believe that the score system used in our study is wider and more accurate than the one used by the authors mentioned above, since we covered 5 different areas with a maximum score of 33. This checklist allowed us to differentiate in an objective way the performances of the trainees. In addition, we could detect aspects that could be reinforced in the future (in our case, assessing respiratory failure of the simulated-patient).

Elder trainees and those from internal medicine specialty had higher basal scores in the simulator than 1st-year residents and those from specialties different from internal medicine. This is expected since the clinical experience in both cases was higher. Interestingly, at the end of the internship, there were no differences between these different subgroups. highlighting the potential benefit of this bundle regardless of their clinical experience and year of residency.

It is noteworthy that at the end of the SBLB, the overall median score achieved was moderately good (70% of total score) but not optimal. These results are similar or even better than those reported by Ottestad et al[ (44% correct answers individually and 58% in team-based approach) or Hansel et al[ (73% of correct answers). Far from being a bad result, this highlights the complexity of the correct management of a septic patient and the importance of simulation training before any physician in training deals alone with real septic patients.

We would like to highlight the importance of the debriefing session after the second simulation. We usually spent around 10 to 15 minutes with each resident giving feedback and analyzing positive points and errors in an interactive way. We have not analyzed if debriefing session improves the performance of the trainee, but we believe that it is an essential part in the simulation-based learning. Other authors like Fanning et al[ support this belief.

Our study has several limitations. First, the study was performed in only 1 center with relatively few participants. Second, although we recommended trainees not to study sepsis management until the first scenario was done, we cannot assure that all of them accomplished it. This might lead to a bias. However, the statistically significant improvement in their scores with our SBLB clearly outperformed this bias. Third, the checklist used to evaluate the performance of the residents is not externally validated. However, the checklist was developed based on current management of sepsis guidelines goals and we think that it is complete and useful to evaluate trainees in an objective way. Finally, long-term retention of knowledge was not evaluated.

Conclusions

In summary, we have shown the utility of combining medical simulation with didactic lectures and a rotation in an ImCU staffed by hospitalists to acquire competencies to manage critically ill patients with sepsis. Moreover, we designed a wide checklist to assure an objective evaluation of the performance of the residents and to identify those aspects in the complex management of a septic patient that could be potentially improved in the education of future residents in our ImCU internship. The implementation of this kind of training methods into residency programs should be taken in account, if material and technology are available.

We firmly believe that this simulation bundle could have a positive impact on the application of treatments and protocols in routine clinical practice transforming and improving medical knowledge and resident training.

Studies regarding the clinical impact of SBLB on the prognosis and outcomes of septic patients should be designed. Furthermore, studies using SBLB in clinical scenarios different from sepsis are warranted.

Acknowledgments

We thank the Faculty of Medicine and Simulation Center Staff for their availability to perform the simulation case.

Author contributions

Conceptualization: Nerea Fernandez Ros, Félix Alegre, Belén Gil-Alzugaray, Manuel F. Landecho, Jorge Quiroga, Juan Felipe Lucena.

Data curation: Belén Gil-Alzugaray, Manuel F. Landecho, Jorge Quiroga, Juan Felipe Lucena.

Formal analysis: Nerea Fernandez Ros, Félix Alegre, Ana Huerta, Nicolás García, Jorge Quiroga, Juan Felipe Lucena.

Investigation: Jorge Quiroga.

Methodology: Nerea Fernandez Ros, Félix Alegre, Ana Huerta, Belén Gil-Alzugaray, Manuel F. Landecho, Jorge Quiroga, Juan Felipe Lucena.

Supervision: Félix Alegre, Ana Huerta, Nicolás García, Jorge Quiroga, Juan Felipe Lucena.

Validation: Jorge Quiroga.

Writing – original draft: Nerea Fernandez Ros.

Writing – review & editing: Nerea Fernandez Ros, Félix Alegre, Belén Gil-Alzugaray, Nicolás García, Jorge Quiroga, Juan Felipe Lucena.