The role of simulation in teaching sinus surgery in otolaryngology residency: A survey of rhinologists.

BACKGROUND: Simulation is currently recognized as an effective surgical training tool. However, no standardized curriculum exists for endoscopic sinus surgery (ESS) simulation training. The goal of this study was to obtain an understanding of current ESS simulation use to aid the future development of an ESS training curriculum.
METHODS: A 14-question survey regarding sinus simulation in residency training was developed through the education committee of the American Rhinologic Society. The survey was administered to academic American Rhinologic Society members in the United States, Canada, and Puerto Rico. The participants provided information regarding the type, amount, and effectiveness of simulation use in their residency program.
RESULTS: Responses were received from 67 training programs; 45% of the programs endorsed using simulation training, although only 23.9% used ESS simulation, and all the programs used cadavers. Only 12.5% of respondent programs required ESS simulation training before operating on live patients, and trainees had an average of <6 hours of simulation training before live operations. A majority of respondents observed subjective improvement in residents' endoscope handling, dexterity, and understanding of anatomy after ESS simulation. The greatest obstacles identified were associated cost and lack of realistic simulators.
CONCLUSION: A majority of responders observed improved surgical technique and knowledge in residents after simulation training. However, <25% of the survey responders used ESS simulation and cited cost and limited availability as the most common barriers. A curriculum of validated simulators has potential to improve the quality of ESS training during residency.

The purpose of graduate medical education is to develop residents into competent physicians and surgeons. Limitations in time, capital, and energy are barriers that all programs face in accomplishing this task. Due to a scarcity of critical resources, integrating methods of instruction that can accelerate competency development is valuable for any graduate medical education program. The Accreditation Council for Graduate Medical Education works to standardize understanding of competency as well as establish specialty-specific resident milestones toward competency attainment.[1]

Developing competency in surgical residents involves many unique challenges. In the traditional intraoperative setting, sporadic opportunities to perform specific procedures combined with frequent mismatch between resident ability and procedure complexity hindered focused learning.[2,3] In addition, evaluating residents in this setting is often inconsistent.[4] Over the past decade, simulation training has gained recognition as an effective surgical training tool. The use of simulation models allows deliberate practice of new skills, which can translate into intraoperative confidence and performance.[5-7] Simulation use also carries the promise of objective assessment of technical ability and a reduction of surgical complications.[4] Despite the validation of new simulation models and increasing evidence in support of simulation use, no formalized, progressive curriculum currently exists for endoscopic sinus surgery (ESS) training. An important initial step would include determining the current state of and barriers to ESS simulation use in North American otolaryngology residency programs. A survey was distributed to academic members of the American Rhinologic Society (ARS). The goals of this survey were to provide an accurate understanding of current ESS simulation use and to aid in the future development and implementation of a curriculum for ESS training.

METHODS

Institutional review board approval of this exempt protocol was obtained. A 14-question survey was developed through the education committee of the ARS regarding the role of sinus simulation in residency training (Table 1). Content validity was established by using a modified subject matter expert rater method. This is a process by which a draft survey is initially developed by expert consensus. The survey is then iteratively refined by integrating the input of several experts until consensus is achieved. Information collected included demographics, residency size, role of simulation, role of sinus simulation, perceived benefits, and hindrances to implementation of ESS simulation. SurveyMonkey (surveymonkey.com) was used to administer surveys to academic ARS members in the United States, Canada, and Puerto Rico. Of note, if the respondent answered “no” to question no. 6 (regarding whether any type of simulation was used in the residency training program), then the survey respondent was routed to the final question regarding barriers to implementation of simulation. The survey was not sent specifically to program directors (unless they happen to be ARS members) or to residents.

Table 1

Survey of 14 questions sent to academic-affiliated rhinologists

PGY = Postgraduate year; ESS = endoscopic sinus surgery.

RESULTS

Responses were received from 67 training programs (Table 2). Residents participated in some form of simulation training in 45% of the programs (30/67). However, only 23.9% of the programs (16/67) used ESS simulation. Only 2 of 16 of the programs (12.5%) required simulation before operating on live patients. On average, trainees spent <6 hours in ESS simulation before the first live ESS. The majority of programs performed ESS simulation in-house rather than sending residents to courses at other programs (13/16). All participating programs used cadavers, but 43.8% (7/16) also used manufactured simulators. Primarily, the residents were evaluated on their ability to complete a surgical task (13/16). Fluency and the time to task completion were not as often evaluated during simulation settings (4/16 and 5/16, respectively). The majority of respondents observed improved endoscope handling (15/16) and dexterity (14/16) after ESS simulation. The majority (10/16) also believed that the residents had a better understanding of anatomy, with a faster learning curve after simulation. The greatest obstacles to implementation of ESS simulation were the associated cost and the lack of realistic simulators.

Table 2

Responses to select survey questions

FESS = Functional endoscopic sinus surgery.

DISCUSSION

Increased educational expectations in the context of reduced training hours encouraged many disciplines within medicine to use simulation to augment and accelerate skills attainment. Due to the changing educational environment, questions have been raised about the effectiveness of current teaching methods and how we might better accomplish educational missions.[8] The Society of American Gastrointestinal and Endoscopic Surgeons has been at the forefront in establishing the need for curriculum development, surgical simulators, and assessment of proficiency.[9-12] Within otolaryngology, individual efforts have also been made, though no unified effort of this kind has taken place.[13] Development of a formalized curriculum to standardize education and surgical training holds promise to improve ESS training. Surgical simulation has been used in many areas of otolaryngology and general surgery, and this method of learning is congruent with the attitudes of the current generation of residents.[3]

Simulation is only a part of resident education, and traditional teaching, such as didactics, bedside rounds, and textbooks, remain paramount.[14] However, there is evidence that the current generation of students are open to technology and finding creative ways to learn.[15] Further, they prefer hands-on experiences rather than lectures and enjoy learning by trial and error.[16] One can learn much from errors, and, although certain mistakes are not permissible in the operating room, simulators allow residents a safe place to try new things and procedures. In accord with the learning methods of Generation Y, students generally showed a favorable attitude toward ESS simulation.[17]

Simulation is also beneficial because the skill set used for endoscopic procedures differs from open procedures.[18,19] Endoscopic procedures are performed in narrower confines than open procedures and, thereby, there is limited freedom of motion and accessibility. Also, the image on the monitor is displayed in two dimensions, whereas procedures are performed in three dimensions. Depth perception must be learned from other visual and tactile cues. Also, bleeding during live surgery compromises visualization, which adds considerable complexity (and risk) and can detract from learning. For these reasons, simulators offer residents safe alternatives to learn, practice, and improve their endoscopic skills.[18] The ideal model would be inexpensive and readily available, and, in addition, possess face, content, and construct validity.[20] Face validity relates to how realistic the model looks. Content validity involves confirming that the skills required while practicing on the simulator translate to real-life situations. Construct validity aims to ensure that the test measures what it is intended to be measured and establishes differences among different skill levels.

Most respondents in this survey maintained that a lack of availability and the high cost of current models prevented their programs from implementing ESS simulation. However, several ESS simulators have been developed and validated for ESS training.[21] These range from low to very high fidelity, but the ideal model does not yet seemingly exist. Leung et al.[22] and Wais et al.[23] used common materials to create a low-fidelity simulator. They demonstrated improvements in performing surgical tasks but did not establish construct validity. Another low-fidelity task trainer was creatively (and inexpensively) assembled by using household goods.[23] Face, content, and construct validity have all been established by using this model, although it is moderately time intensive to build.[24,25]

To provide a more-realistic endoscopic image, Burge et al.[26] designed a low-cost intermediate-fidelity trainer. The investigators demonstrated that participants learned to perform endoscopic tasks quicker and with fewer errors after using the trainer. Regarding high-fidelity simulators, Weghorst et al.,[27] Fried et al.,[28-31] and Arora et al.[32] have done monumental work in ESS trainers, and published that their virtual reality simulator with haptic feedback demonstrated validity for teaching and measuring the skills needed in ESS. Another validated high-fidelity virtual reality model has also been validated but also has a high cost.[33] The biggest challenge to this simulator is that few, if any, otolaryngology programs have the funds to purchase such a simulator. The price associated with a system like this was considered by rhinologists in this survey to be one of the foremost obstacles to implementation of simulation. When the available simulators and studies were evaluated together, it seemed that there were multiple iterations of variable levels of fidelity and validity. Due to the disparate nature, a cohesive systematic review of simulators remains problematic.

Respondents in this study observed improved instrument handling and dexterity among residents who participated in ESS simulation. However, scope handling and dexterity may not be the educational objectives for a more-advanced resident. A wide range of fidelity is found among current ESS models, and it is important to consider which models are best suited for residents at different levels of training. For instance, a high-fidelity model that mirrors the complex anatomy of the sinus cavities may overwhelm the early learner whose initial learning objective is scope handling and three-dimensional conceptualization. Approximately 45% of respondents to this survey used some form of simulation in resident education. However, only 23.9% of the programs reported using ESS simulation. This result differed from a recent survey of program directors in the United States, which found that ∼78% of the programs used some type of otolaryngologic simulation.[34] The difference in results may reflect a greater awareness or availability of simulation outside of rhinology. Whatever the cause, simulation is widely used among otolaryngology residency programs and sinus simulation lags behind.

There were several limitations to this survey. First, the survey method prevented establishing a specific survey response rate. Because the ARS does not have a separate membership category for academic members, the exact number of relevant recipients was unknown. The ARS does not have a current list of otolaryngology program directors, and, therefore, the survey could not be sent directly to this group. Although the survey was kept short to minimize the time that respondents had to commit, academic faculty may have had “survey fatigue.” In addition, because response was voluntary, it is possible that only those with a specific interest in simulation responded. It should also be noted that this survey did not acknowledge clinical intervention with endoscope and suction as an integral step in skill acquisition. Also, although respondents reported improvement in surgical skills after simulation usage, these results were often based on subjective assessments, which highlighted an important point: for simulation to be maximally effective, the component tasks required to be mastered and integrated to complete more complex tasks must be delineated. This has not been done in a systematic way for ESS, and, as a result, a curriculum of progressive skill attainment should be developed to build toward ESS competence.

The coordination of a viewing hand and an intervention hand while watching an unaligned monitor requires significant dexterity and control. The correct order of ESS learning environment remains undetermined and may be (1) endoscopic viewing (simulation and/or clinic), (2) endoscopic suctioning (clinic), (3) endoscopic intervention (simulation and/or operating room). Thus, the steps involved in learning how to perform ESS as well as the optimum environment from which to gain this experience is still open for debate. We believe that simulated environments may improve patient comfort and safety during this learning process.

CONCLUSION

Fewer than one-fourth of the survey respondents identified using ESS simulation in their respective programs. High cost and limited availability of ESS simulators are the most commonly reported barriers to implementing ESS simulation in residency training. A systematic effort to develop a progressive curriculum by using validated simulators will improve the quality of ESS training during residency.