An innovative and safe way to train novice ear nose and throat residents through simulation: the SimORL experience.

Introduction

Post-graduate specialist education is a progressive process. Studies show a variable range of competences as junior doctors enter their specialist clinical training [1,2]. Many junior residents are initially not comfortable with the clinical setting as it is perceived as a stressful environment, something that is only experienced to a degree during medical school [3].

It would be advisable, however, that junior trainees entering the clinics are equipped with a homogenous skill-set and clinical competences that allow them to safely approach the first on-calls.

In Italy, most Ear Nose and Throat (ENT) training programs use apprenticeship as their main teaching model (time-based education), where junior trainees learn from direct observation and trial from senior residents and attending physicians. This is a very common model in the medical sciences, and is particularly rooted in surgical disciplines [4]. This model has some limitations, however, as case and procedure exposure for observation and practice will be dependent on a specific institution’s characteristics, mentors personal subspecialty or preference and case-mix available at any given time. An alternative or integration to time-based education is competency based medical education. Competency-based medical education allows trainees to progress through their training based on competency, rather than fulfillment of a pre-defined amount of time [5,6].

Medical simulation offers an excellent tool to supplement this model by offering a safe and repeatable learning environment to integrate specific cases and procedures that can be tailored to the resident’s competency needs. Simulation allows the creation of custom-made learning programs that are able to provide junior trainees with basic tools to approach safely, among others, the first year of training and associated on-call duties.

The field of otorhinolaryngology is not new to the use of simulation and much literature is available, especially regarding simulation of specific surgical approaches [7-11]. Recently, some centres have launched short educational events aimed at introducing junior residents to the duties of being a new resident, the so-called boot camp. Almost all published boot-camps in ENT are from North America, are one day long, use simulation as a main teaching tool and focus predominantly on training about common ENT emergencies [12-17]. These events are designed with the goal of developing the cognitive, communication and procedural skills necessary for otolaryngology residency [18].

The aim of this work is to present and describe the model of SimORL – a two-day simulation event in ENT dedicated to junior residents – and investigate whether participants found the event beneficial to their professional development. We hypothesised that SimORL would increase participants confidence and be perceived as useful and educationally effective in teaching basic ENT topics of interest to the junior trainee.

Methods

SimORL (Simulation in Otolaryngology) is a two-day intensive, focused, training course that covers multiple topics of interest to the junior trainee. SimORL uses simulation as the main educational strategy, including different modalities ranging from cadaver models to high fidelity mannequins.

The first SimORL took place in 2018 at the SIMNOVA Healthcare Simulation Centre, Università del Piemonte Orientale, Novara, Italy.

Participants

SimORL was open to any junior ENT resident across the country. Invitees included novice otolaryngology - head and neck surgery trainees (postgraduate year one and/or two). The event was also advertised through social media channels.

Participants were divided into 5-person teams leveled out by training year. Participants signed an on-line written informed consent as part of their registration process for the event.

Curriculum design and content

The core curriculum of the event was developed to meet the following educational goals: 1) recognise and treat common ENT presentations; 2) perform simple instrumental diagnostics; 3) perform basic surgical interventions (tracheostomy was chosen as the surgical intervention); and 4) communicate effectively with teams and relatives. Over the two days a total of 9 + 1 educational stations were delivered (five per day) in three teaching categories. More information about the stations are reported in Table I. Each group rotated through all of the stations. All stations included debriefing and no assessment was carried out to grade participants in their actions.

Table I.

Type, goals and simulators used for each station during the two days of SimORL.

Educational station		Skill	Educational goals	Teaching cathegories	Equipment
1	Diagnostic nasal endoscopy	Diagnostic nasal fibre optic examination on prepared (prepared with ethmoidectomy-medial meatotomy-sphenoidotomy)/ not prepared nostrils	Perform simple instrumental nasal diagnostics	Task trainers and manual abilities	Cadaveric model, Olympus 4K technology
2	Diagnostic otoscopy	Virtual otoscopy	Perform simple instrumental otologic diagnostics Recognise and treat common ENT presentations	Task trainers and manual abilities	OtoSim®
3	Sutures	Principles of skin sutures (with silk, vicryl and prolene stitches)	Perform basic surgical interventions	Task trainers and manual abilities	Limbs and Things® Skin Pad
4	Surgical 3D anatomy	Human anatomy of head and neck	Perform simple instrumental diagnostics Recognise and treat common ENT presentations	Task trainers and manual abilities	3D virtual simulator, zSPACE AIO®
5	Surgical tracheostomy	Surgical tracheostomy	Perform basic surgical interventions	Task trainers and manual abilities	High-fidelity wet part-task trainer
6-7	Common ENT conditions, high-fidelity scenarios	4 high-fidelity scenario (posterior epistaxis and neuronitis on day 1, laryngeal respiratory distress due to glottic cancer and neck swelling in a retro-parapharyngeal abscess on day 2)	Perform basic surgical interventions Recognise and treat common ENT presentations Communicate effectively with teams and relatives	High-fidelity simulation	Gaumard HAL human patient simulator or standardised patient
8	Non-technical skills	Team building exercise	Communicate effectively with teams and relatives	Interactive discussions, non-technical skills	N/A
9	Surgical skills	Become familiar with ENT surgical instruments	Perform basic surgical interventions	Interactive discussions, non-technical skills	ENT surgical kits
10	Robotic surgery	Competitive robotic surgery station	Perform basic surgical interventions	Gamification	Robotic surgery DaVinci® XI simulator trainer

Program evaluation

Self-assessment data was collected from participants using anonymous pre-course and post-course surveys. Prior to attending the event, participants were asked to complete a survey on demographics and pre-course simulation experience, as well as confidence levels in performing the skills involved in the event. Post-event surveys assessed participants’ post-training confidence level with each of the skills and abilities presented. Moreover, participants completed a satisfaction questionnaire and expressed an educational effectiveness rating for each of the event stations. The previously validated Satisfaction with Simulation Experience Scale (SSES, a 5-item Likert scale) was used to assess satisfaction [19]. For the other sections, responders were asked to reply using an unanchored semantic scale ranging from “1 - strongly disagree/not confident at all” to “10 - strongly agree/very confident”. Residents were also given the opportunity to include free-text comment about the event, the stations and suggestions for improvement of the event.

Data analysis

Data was collected on paper-based forms and later transformed into digital spreadsheets. Descriptive statistics were summarised using median and percentiles. Pre-post-event differences in self-perception regarding confidence were tested using the Wilcoxon signed-rank test.

Results

Twenty-three ENT residents from eight different training programs around the country participated in the first SimORL; all completed the pre- and post-event questionnaires. The post graduate year (PGY) levels of the 23 participants were: 11 (48%) PGY-1, 6 (26%) PGY-2, 5 (22%) PGY-3, and 1 (4%) PGY-4. 15 (65%) were female. Only 3 (13%) individuals reported limited previous experience with simulation and the remaining 20 (87%) reported no experience with simulation. No individuals were excluded from the analysis.

Confidence

Post-event confidence levels demonstrated significant increases in all six stations. Confidence in performing diagnostic otoscopy improved from 6 (5-7) to 7 (6-8) (p = 0.021); in performing diagnostic nasal endoscopy from 6 (5-7) to 8 (6-8) (p = 0.0003); confidence with surgical anatomy from 4 (3-6) to 7 (6-8) (p < 0.0001); confidence managing common ENT conditions from 6 (5-7) to 7 (6-8) (p = 0.036); confidence performing a surgical tracheostomy from 5 (2-6) to 7 (5-8) (p < 0.0001) and finally confidence in suturing from 6 (5-7) to 8 (7-8) (p = 0.0011). Variations of confidence pre-post including min-max scores per station are presented in Figure 1.

Figure 1.

Radar chart presenting median self-confidence perception level before (dotted white) and after (grey) the event. Apexes of polygon represent each simulation station (1: diagnostic otoscopy; 2: diagnostic nasal endoscopy; 3: surgical anatomy; 4: common ENT management scenarios; 5: surgical tracheostomy; and finally 6: suturing). Axes in each radar chart represent the 10-points semantic unanchored scale.

Participant’s satisfaction with the event and the simulation experience

Overall participants found SimORL a useful learning experience, rating it 10 (9-10) of 10 for global satisfaction and educational value. The majority of participants highly rated each of the learning stations for educational effectiveness (Tab. II). The most appreciated station was diagnostic nasal endoscopy, while the least appreciated was the otoscopy station.

Table II.

Educational effectiveness of each simulation scenario/topic as rated by participants using a 10-point unanchored semantic scale. Results are presented as median and 25-75 percentile.

Skill	Rating
Diagnosic otoscopy	6 (6-8)
Surgical 3D anatomy	8 (6.75-10)
Suture	8 (7-10)
Diagnostic nasal endoscopy	10 (8.75-10)
Surgical tracheostomy	9 (8.75-10)
Common ENT conditions high fidelity scenarios - day 1	8 (7.75-9.25)
Common ENT conditions high fidelity scenarios - day 2	9 (8-10)
Non-technical skills	8 (7-9)
Surgical skills	7.5 (6.75-8)
Robotic surgery	9.5 (8.75-10)

Overall, the global SSES rating was very high with a median of 4.5 of 5 (4-5). All satisfaction measures are reported in Table III.

Table III.

The Table reported all satisfaction measures: responders were asked to reply using a -point unanchored semantic scale ranging from “strongly disagree/not confident at all” to “strongly agree/very confident”.

	Strongly disagree (%)	Disagree (%)	Unsure (%)	Agree (%)	Strongly agree (%)
Debrief and reflection
The facilitator provided constructive criticism during the debriefing	0	0	4	17	78
The facilitator summarised important issues during the debriefing	4	0	4	22	70
I had the opportunity to reflect on and discuss my performance during the debriefing	0	0	13	13	74
The debriefing provided an opportunity to ask questions	0	0	4	30	65
The facilitator provided feedback that helped me to develop my clinical reasoning skills	0	0	4	22	74
Reflecting on and discussing the simulation enhanced my learning	0	0	9	26	65
The facilitator’s questions helped me to learn	0	0	9	30	61
I received feedback during the debriefing that helped me to learn	0	0	9	17	74
The facilitator made me feel comfortable and at ease during the debriefing	0	0	4	9	83
I had the opportunity to reflect on and discuss my performance during the debriefing	0	0	4	17	78
Clinical reasoning
The simulation developed my clinical reasoning skills	0	0	4	35	61
The simulation developed my clinical decision-making ability	0	0	17	26	57
The simulation enabled me to demonstrate my clinical reasoning skills	0	9	9	35	48
This was a valuable learning experience	0	0	4	26	70
Clinical learning
The simulation caused me to reflect on my clinical ability	0	0	4	26	70
The simulation tested my clinical ability	0	0	13	30	57
The simulation helped me to apply what I learned from the case study	0	4	9	35	52
The simulation helped me to recognise my clinical strengths and weaknesses	0	0	9	26	65

Discussion

The results of our study suggest that SimORL can be a valuable tool to provide didactic, hands-on and confidence building experiences for junior otorhinolaryngology trainees. After completing the SimORL curriculum, participants referred significant improvement in self-confidence in several areas, including improvements in the diagnostics process (otoscopy and nasal endoscopy), general knowledge (anatomy), surgical procedures (suturing and tracheostomy) and complex clinical reasoning (management of common ENT emergencies).

Previous studies suggest a correlation between actual proficiency and self-perceived competence [20-22].

The high rating in the SSES likely reflects a high-quality formative process including well-delivered expert debriefing with non-judgmental constructive approaches throughout the entire event and the application of theoretical reinforcements of key teaching points by tutors.

The entire program was designed with the concepts of deliberate practice in mind. Based on the Ericsson research on skill acquisition, significant improvements in performance can be realised when individuals are: a) given a task with a well defined goal; b) motivated to improve; c) provided with feedback; and d) provided with opportunities for repetition [23].

SimORL allowed residents to increase their confidence by practice, trial-and-error and help from supervisors and teachers. A key aspect of the practice of surgical disciplines is the ability to perform practical procedures efficiently and safely. Patient safety, malpractice and increased mortality rates caused by human errors have increased the importance of medical simulation sessions for undergraduate and postgraduate training programs [24]. Simulation-based training offers well-described advantages as an educational tool. It provides many opportunities to practice both technical and non-technical skills in a safe environment, learning from achievements and errors without the consequences that may result from mistakes [25].

According to Italian training standards for the ENT residency program, at the end of the training period, residents becoming specialists should have proficiently mastered all the different regions and systems pertaining to the head and neck, including surgery. Other residency programs (such as Anesthesia or Emergency Medicine) include the option to use simulation as a complementary simulation tool in their national training requirements. SimORL could offer the opportunity to safely improve fields of training.

There are previously published studies with similar methodologies. Our results are in line with that of other similar events both in the field of otorhinolaryngology as well as in other medical specialties [26,27].

Recently, Dean et al. conducted a cross-sectional survey of all otolaryngology residency program directors in the United States and Puerto Rico: based on the participants’ answers, the authors concluded that simulation-based boot camps can be a significant and valuable component of residency training [28].

International data have shown that current training paradigms may fail to adequately prepare surgeons for independent practice [29]. Restrictions on trainee working hours, new patient safety initiatives and pressures for increased operating room efficiency have significantly reduced the number and quality of learning opportunities for surgical trainees [5,30].

This is, to our best knowledge, the first nationwide simulation event in the field of otorhinolaryngology in Italy. SimORL aims to be a stimulus for the integration of training with simulation, an approach to competency-based post-graduate medical education and, hopefully, provides a small contribution to the improvement of patient care.

There are several limitations to this study. The sample size was small and may not be representative of the overall trainee population. There was no summative assessment of skills and performance, as this was excluded to preserve the psychological safety of trainees who are new to the concept of simulation and debriefing, which, however, limits the ability to document improvement objectively.

There was no follow-up over time: this study presents immediate results, when the signals are stronger and no long-term follow-up of confidence self-perception has been performed nor did we assess for knowledge retention over time. Although longer than most ENT simulation events reported in literature, the length is still significantly shorter than other specialty boot camps than can span from one week to one month. The ideal program length is yet to be clarified.

Conclusions

The SimORL experience demonstrated that simulation for ENT residents can be a valid educational tool to improve confidence in performing specific ENT procedures, showing a very high overall global satisfaction rate. Further studies are needed to confirm these encouraging results.