Evaluation of a 3-Dimensional-Printed Head Simulation Technique for Teaching Flexible Nasopharyngoscopy to Radiation Oncology Residents.

PURPOSE: Simulation-based medical education is an effective tool for medical teaching, but simulation-based medical education deployment in radiation oncology (RO) is limited. Flexible nasopharyngoscopy (FNP), an essential skill for RO residents, requires practice that typically occurs on volunteer patients, introducing the potential for stress and discomfort. We sought to develop a high-fidelity simulator and intervention that provides RO residents the opportunity to develop FNP skills in a low-pressure environment. METHODS AND MATERIALS: Computed tomography images were used to create an anatomically accurate 3-dimensional-printed model of the head and neck region. An intervention incorporating didactic instruction, multimedia content, and FNP practice on the model was designed and administered to RO residents attending the Anatomy and Radiology Contouring Bootcamp. Participants completed pre- and postintervention evaluations of the training session and model fidelity, and self-assessments of FNP skill and confidence performing FNP. Participants were video recorded performing FNP pre- and postintervention. Videos were scored by a blinded observer on a predefined rubric. Changes in scores were evaluated using the Wilcoxon signed-rank test.
RESULTS: Twenty-four participants from 17 institutions and 4 countries completed the intervention, 50% were women, and most were senior residents. Postintervention, FNP confidence and FNP performance improved significantly (mean ± standard deviation on a 10-point scale: 1.8 ± 1.8, P < .001; 2.2 ± 2.0, P < .001, respectively). Participants felt the model was helpful (mean ± standard deviation on a 5-point scale: 4.2 ± 0.6), anatomically correct (4.1 ± 0.9), and aided in spatial comprehension (4.3 ± 0.8). Overall satisfaction for the intervention was high (4.3 ± 0.8). Participants strongly agreed the intervention should be integrated into RO training programs (4.3 ± 0.8).
CONCLUSIONS: A 3-dimensional-printed model and associated intervention were effective at improving FNP performance and the teaching method was rated highly by participants. RO residents may benefit from broader dissemination of this technique to improve trainee performance.

Introduction

Simulation-based medical education (SBME) is a key contributor to quality health professional training. SBME deployment in contemporary medical residency curricula has increased dramatically, reflecting the demonstrated efficacy of its use in improving the clinical and procedural competence of learners compared with traditional instructional methods. , SBME is most commonly used to improve procedural skills with the most effective interventions incorporated directly into curricula and involving deliberate practice and feedback in a nonclinical environment.

Effective interventions require high-fidelity simulators custom-designed for the particular skill. Three-dimensional (3D) printing has emerged as a method for producing high-quality, inexpensive teaching aids that have demonstrated efficacy over traditional materials. Their use in improving procedural performance has been established secondary to the ability to precisely reconstruct intricate anatomic structures.

SBME use in radiation oncology (RO) is less widespread than in many other specialties. The majority of reported interventions focus on improving contouring skills via screen-based simulators despite the fact that Radiation Oncologists are expected to maintain several procedural skills in the modern clinical environment. Flexible nasopharyngoscopy (FNP) is essential to the diagnosis, treatment, and surveillance of patients with head and neck (HN) cancer. Mastery of this skill requires practice that typically occurs in outpatient clinics on volunteer patients, introducing the potential for stress and discomfort. A recently reported simulation workshop designed to teach FNP demonstrated an improvement in confidence and procedural expertise when administered to RO residents. We sought to develop a novel intervention incorporating a high-fidelity 3D-printed simulator that provides RO residents the opportunity to develop FNP skills in a low-pressure environment.

Methods

Designing the 3D-printed simulator

A Radiation Oncologist identified a suitable model patient based on ease of performing FNP. A computed tomography (CT) scan of the patient used for radiation treatment planning was anonymized and imported into 3DSlicer version 4.10.2 software (The Slicer Community, open source www.slicer.org) for model design. Once imported, the 2.5 mm slice scan was viewed using a window width of 350 and window level of 40. After rendering the bulk volume from the CT scan, volumes were cropped using the crop volume module for the creation of segmentations.

The nasal cavity required adequate diameter to allow scope passage (scope diameter = 3.6 mm) while maintaining the anatomic integrity of the model. Custom local thresholds based on the Hounsfield scale were used to generate segmentations, ensuring a viable path for the instrument. Slices were manually edited to correct irregular discontinuities on the surfaces of the nasal cavity and outer shell, and to correct abnormalities related to dental artifacts. Manual edits were also used to reshape the exterior surface of the nose and nostrils.

After initial testing, additional manual edits were made to allow for additional widening of regions identified as barriers for passing the instrument. To conserve the use of filament and reduce printing time, a large hollow region was created within the head in areas not visible to the endoscope (Fig. 1 ). A supportive stand extending from the chin of the model was also added.

Fig. 1

Base computed tomography image with model creation overlay showing airways, hollow region, and supportive stand.

3-dimensional-printing the simulator

The simulator was printed with 3 mm EcoTough polylactic acid filament (www.filaments.ca) on a Lulzbot Taz 6 3D printer. Because of print area size contraints, the model was designed and printed in 2 component pieces: the head and the neck. The G-code (reference file format for 3D printers) was generated using Cura LulzBot Edition 3.6.13 software after the importation of stereolithography files, exported from 3DSlicer. The prints were scaled up to 1.6 times their original size to allow for improved instrument maneuverability. Both components were printed with main settings available in Table 1 . Remaining parameters used Cura default settings for the Lulzbot Taz 6 Standard Extruder.

Table 1

Lulzbot Taz6 printer settings

	First layer	Remaining layers
Bed temperature	60°C	60°C
Extruder temperature	210°C	205°C
Layer height	0.43 mm	0.25 mm
Layer speed	15 mm/s (2 layers)	60 mm/s

To allow for viewing of internal anatomy and easier facilitation of support material, the head print was paused at a level after the base of the nostrils had been completed (approximately z = 114). A thin piece of paper was cut and glued to the top of the incomplete model and, once dried, a modified G-code resumed the print from this level. The head section then was separated at the level of the paper, cleaned and cleared of support material, then attached back together with hinges. The neck was printed upside-down to conserve more time and filament; once cleared of support material it was glued to the head, to create the complete final product. The final assembled model is shown in Figure 2 .

Fig. 2

Photographs of the 3-dimensional–printed model.

Downloadable files for printing this 3D head are available at bit.ly/2z8XTWoHead (head) and bit.ly/2z3axpINeck (neck).

Assessment of simulator efficacy

Participants for this study were recruited from RO residents attending the Anatomy, Radiology & Contouring Bootcamp. Local research ethics approval and study consent were obtained.

Participants first completed a pre-intervention evaluation and were asked to perform FNP on a 3D-printed simulator. FNP performance was recorded using MobileOptx (MobileOptx LLC, Pennsylvania) smartphone adaptors.

After the baseline evaluation, the participants completed a teaching intervention, incorporating didactic, multimedia, and practical components of FNP. Didactic components included a review of relevant anatomy and discussion of flexible endoscope design, function, and operation, a systematic approach to FNP-aided physical examination of the upper aerodigestive tract, and tips for troubleshooting issues encountered while performing FNP. Participants also viewed the New England Journal of Medicine instructional video “Examination of the Larynx and Pharynx.” Finally, participants practiced on the 3D simulator, with feedback provided.

After the intervention, participants completed a posttest evaluation and repeat FNP recordings were obtained.

Evaluation questions included self-assessments of FNP skill and confidence performing FNP clinically (10-point scale) and assessments of 3D-simulator fidelity and the effectiveness of the training intervention (5-point Likert scale). FNP recordings were reviewed by a single, blinded, expert FNP provider and scored on a 10-point scale using a predefined rubric. The expert reviewer is an experienced HN Radiation Oncologist who frequently performs FNP for diagnostic and surveillance purposes. The reviewer had experience performing FNP on the 3D-printed simulator before reviewing and scoring participant video recordings. Statistical analysis included comparison of pre- and postintervention FNP scores to identify and quantify improvement in ability secondary to the intervention, using the Wilcoxon signed-rank test, Wilcoxon rank-sum test, or Kruskal-Wallis test as appropriate. Additionally, pre- and posttest surveys were analyzed to assess simulator fidelity and attitudes toward performing FNP in a clinical setting. All statistical analysis was performed using SAS version 9.4 software (SAS Institute, Cary NC), using 2-sided statistical testing at the 0.05 significance level.

Results

A total of 31 subjects consented to the intervention with 24 completing all aspects of the study and therefore available for analysis. Baseline characteristics are available in Table 2 . Participants were mostly senior residents many of whom had previously completed either a HN surgical or RO rotation. Most participants who had completed a HN rotation did not receive formal FNP training despite most performing the technique clinically. There was a wide variation in reported number of prior FNPs.

Table 2

Baseline characteristics

Characteristic	No. (%)
Sex
Female	12 (50)
Male	12 (50)
Postgraduate year
2	1 (4)
3	6 (25)
4	12 (50)
5	5 (21)
Prior HN surgical rotation
Yes	8 (33)
No	16 (67)
Performed FNP on HN surgical rotation
Yes	6 (75)
No	2 (25)
Received formal training on HN surgical rotation
No	8 (100)
Prior HN RO rotation
Yes	18 (75)
No	6 (25)
Performed FNP on HN RO rotation
Yes	17 (94)
No	1 (6)
Received formal training on HN RO rotation
Yes	3 (17)
No	15 (83)
No. of prior FNP performed∗
0	4 (17)
≥11	16 (67)
≥21	9 (38)

Abbreviations: HN = head and neck; FNP = flexible nasopharyngoscopy; RO = radiation oncology.

Categories are not mutually exclusive (for the 9 participants with ≥21 FNPs being also included in the 16 participants with ≥11 FNPs given the total exceeds 24.

FNP skill was objectively assessed pre- and postintervention via blinded review of video recordings. Skill was evaluated using a standardized rubric (Appendix E1). Mean ± standard deviation (SD) pre-intervention score was 4.5 ± 2.0. Postintervention mean ± SD score improved significantly to 6.7 ± 1.5 (P < .001). This translated to a mean ± SD increase of 2.2 ± 2.0. The improvement in score did not depend on postgraduate year (PGY) level (P = .261), whether or not the participant had previously completed a HN surgical rotation (P = .480) or a HN RO rotation (P = .300), or the number of prior FNPs (P = .270).

Participants were asked to report perceived attitudes toward FNP in a clinical setting on a 10-point scale (0 = low, 10 = high). Participants reported low confidence at the time of their first-ever lifetime FNP (mean ± SD: 1.9 ± 1.6). At the time of the study, reported mean ± SD confidence was higher (5.5 ± 2.2), but participants endorsed some anxiety when performing FNP in a clinical setting (3.9 ± 2.1). After the intervention, self-reported confidence significantly improved to 7.2 ± 0.9 (P < .001). Participant confidence consistently improved regardless of PGY level (P = .385); however, those without a previous HN RO rotation had a significantly larger improvement in confidence (P < .001). Participants with <11 prior FNPs had the greatest improvement in confidence (mean ± SD increase: 3.3 ± 2.2), although a significant improvement in confidence was maintained even in those participants with ≥21 prior FNPs (P = .016).

Participants evaluated various aspects of the intervention on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Participants agreed that the model accurately simulated the size and shape of human anatomy (mean ± SD: 3.8 ± 0.9) and was anatomically correct (4.1 ± 0.9). Realistic passage of the scope (2.9 ± 1.1) and simulation of human tissue (2.6 ± 1.1) were identified as potential areas for improvement. There was agreement that the model was helpful for learning FNP (4.2 ± 0.6) and should be promoted as a standard component of RO training (4.3 ± 0.8). Participants strongly agreed the intervention was a positive learning experience (4.7 ± 0.5) and would recommend the session to others (4.7 ± 0.6). Mean ± SD overall rating for the intervention (including didactic and practical components) was 8.2 ± 1.5 on a 10-point scale (0 = not beneficial, 10 = extremely beneficial).

Discussion

SBME use is uncommon in RO, yet there are numerous clinical and professional competencies that could benefit from simulation-based training. No participants in our study reported receiving formal training during HN rotations despite most performing the technique clinically. Instead, residents typically learn FNP in suboptimal, high-stress environments often under close scrutiny by patients, their families, and multiple members of the care team. As a direct result, provider and patient discomfort and the potential for iatrogenic injury are heightened. We report outcomes from an intervention that uses a custom-designed 3D-printed simulator and associated training session to teach RO residents FNP. The intervention was associated with significantly increased self-reported confidence performing FNP. When stratifying by number of prior FNPs performed we noted a significant improvement in confidence regardless of past experience but participants with less scoping experience had a greater magnitude of improvement compared with those with more experience. Objective performance, scored by a blinded, expert-reviewer, improved significantly after the intervention by a mean of 2.2 points on a 10-point scale. The improvement in objective performance was not dependent on PGY level or past experience. These findings support the model accurately simulating clinical FNP experiences. It allows for all trainees to improve by providing a systematic approach to the upper airway examination alongside deliberate practice with expert feedback. It also enables a safe, interactive learning environment for less experienced trainees to gain valuable confidence performing a highly specialized and clinically important technique without any risk to patients. This is reflected in the consistently high ratings garnered from participants when asked to evaluate the intervention.

As SBME use becomes more widespread, an increasing body of work has been published examining the hallmarks of effective simulations. , 8, 9, 10 In general, interventions should incorporate feedback and deliberate practice, use simulators of appropriate fidelity that are matched to the desired skills, integrate multimodal learning aids, and rigorously measure their outcomes. The most effective simulations are incorporated as a standard component of a training curriculum. We integrated this intervention into the curriculum of the Anatomy, Radiology & Contouring Bootcamp: a 3-day intensive training course designed to assist RO residents in learning anatomy skills relevant to the modern practice of RO. The session builds on anatomic knowledge gained in the course providing a bridge to clinical practice. This integration is important as effective simulations should complement clinical training rather than trying to supplant practice on real patients in clinical settings. This particular intervention is designed as a preclinical exercise allowing trainees with limited experience to gain the basic skills necessary to safely and competently perform FNP examinations before attending HN clinics.

An intervention designed to train RO residents to perform fiberoptic laryngoscopy has recently been described in the literature. The authors created a 2-phase approach that includes a workshop to provide an overview of HN anatomy and logistics of the examination. Participants subsequently perform fiberoptic laryngoscopy on a computer program and a mannequin. Postintervention surveys demonstrated a significant improvement in mean HN anatomy knowledge and self-reported confidence performing laryngoscopy. In contrast, the present approach relies on a 3D-printed model which may confer several benefits, including more accurate representation of the pertinent anatomic structures and reduced cost compared with simulation mannequins. In comparison to the present approach, the authors incorporated a follow-up period wherein participants who completed the workshop were later supervised performing FNP in a clinical setting and provided with immediate feedback. The addition of longitudinal feedback likely supports the efficacy of the intervention at improving FNP skill and should be considered for similar interventions in the future. Conversely, the authors based their conclusions on self-reported scores for confidence and HN anatomic knowledge. Our approach incorporates an objective measure of improvement in the form of scored video-recordings which strengthens the veracity of our conclusion that this intervention directly contributes to improved performance.

3D-printing is a novel manufacturing technology that uses an additive process to recreate intricate structures based on computer-generated models in a slice-by-slice fashion. The technology has been widely adopted for medical applications given the ability to leverage high-resolution CT imaging for model creation.12, 13, 14 In radiation therapy, 3D-printing has been successfully used to create customized bolus15, 16, 17 and patient-specific phantoms. 3D-printed models are effective tools for medical education. In a double-blind randomized controlled trial, medical students learning cardiac anatomy demonstrated significantly improved test scores when using 3D-printed models instead of cadaveric materials. Another randomized controlled trial demonstrated improved test scores for medical students learning skull anatomy on 3D-printed models versus cadaveric materials or traditional anatomy atlases.

3D-printed models are cost-effective and can be manufactured quickly. Any institution capable of supporting the capital requirements for a 3D printer can create their own models based on source code which can be widely disseminated via the Internet. Once the source files are obtained, commercial printing services can be leveraged to create models should a program not have access to an in-house printer. Given the affordable nature of most filaments, programs can easily print as many copies as required to fit their needs. The use of 3D-printed simulators is also a safe and effective way to facilitate training on invasive procedures with limited pathogen-exposure risk. During the Coronavirus (COVID-19) pandemic, there is a clear and urgent need for programs to leverage technologies, such as these in new and innovative ways to continue to meet the educational needs of medical trainees.

This study is strengthened by its focus on high-quality SBME principles. We combined the ability to create a highly accurate simulator via 3D printing with a comprehensive training session incorporating direct feedback and rigorous outcome measurement. The use of smartphone recordings to measure objective performance strengthens our conclusion that the intervention was effective at improving FNP performance and suggests the intervention could translate to improved patient outcomes. There were some noted drawbacks to our design, including the use of a single reviewer and the fact that the FNP smartphone attachments affected the balance of the endoscope, which may have negatively affected performance. Participants noted some deficits in the ability to pass the scope through certain regions of the model and also reported that the model did not accurately recreate the feel of human tissue. In the future, we aim to create a revised 3D-model which looks to improve endoscope maneuverability and plan to explore the use of different filaments to better simulate human tissue. These updated models are planned to include various pathologic findings to enable more oncology-relevant training scenarios and more rigorous evaluation of participants. Future modules could incorporate training on communication of examination findings and scored assessments of examination skill based on the identification of landmark features or pathology. We have elected to make the present version of the 3D-model available via free download to promote the proliferation of this technique to more training programs. As experience with 3D-printed simulators grows within the RO community we aim to complete a multi-institutional study on the effectiveness of these and similar interventions to better characterize the optimal parameters for their incorporation into training programs. In addition, as the efficacy of these interventions is better characterized, there are opportunities to create 3D-printed simulators for other aspects of RO training. RO interventions that require invasive, uncomfortable patient contact or an understanding of complex 3D anatomy (ie, pelvic examinations, brachytherapy training) could benefit from purpose-built simulators that allow for deliberate practice in a low-stress learning environment and should be investigated as potential avenues for expanded 3D-printing use in RO.

Conclusions

In conclusion, a 3D-printed model and associated training intervention were effective at improving objective FNP performance and the teaching method was highly rated by participants. We believe there is excellent potential for the expanded use of SBME in RO and would advocate strongly for future interventions to be designed in accordance with the principles of high-quality simulation interventions.