Repetitive simulation training with novel 3D-printed sinus models for functional endoscopic sinus surgeries.

Background: The purpose of this study was to find a utility of a newly developed 3D-printed sinus model and to evaluate the educational benefit of simulation training with the models for functional endoscopic sinus surgery (FESS). Material and methods: Forty-seven otolaryngologists were categorized as experts (board-certified physicians with ≥200 experiences of FESS, n = 9), intermediates (board-certified physicians with <200 experiences of FESS, n = 19), and novices (registrars, n = 19). They performed FESS simulation training on 3D-printed models manufactured from DICOM images of computed tomography (CT) scan of real patients. Their surgical performance was assessed with the objective structured assessment of technical skills (OSATS) score and dissection quality evaluated radiologically with a postdissection CT scan. First we evaluated the face, content, and constructive values. Second we evaluated the educational benefit of the training. Ten novices underwent training (training group) and their outcomes were compared to the remaining novices without training (control group). The training group performed cadaveric FESS surgeries before and after the repetitive training.
Results: The feedback from experts revealed high face and content value of the 3D-printed models. Experts, intermediates, and novices demonstrated statistical differences in their OSATS scores (74.7 ± 3.6, 58.3 ± 10.1, and 43.1 ± 11.1, respectively, p < .001), and dissection quality (81.1 ± 13.1, 93.7 ± 15.1, and 126.4 ± 25.2, respectively, p < .001). The training group improved their OSATS score (41.1 ± 8.0 to 61.1 ± 6.9, p < .001) and dissection quality (122.1 ± 22.2 to 90.9 ± 10.3, p = .013), while the control group not. After training, 80% of novices with no prior FESS experiences completed surgeries on cadaver sinuses.
Conclusion: Repeated training using the models revealed an initial learning curve in novices, which was confirmed in cadaveric mock FESS surgeries. Level of evidence: N/A.

INTRODUCTION

Functional Endoscopic Sinus Surgery (FESS) is one of the most frequently performed surgeries in rhinology. Despite its effectiveness, there is a risk of significant complications due to the proximity of the orbit and skull base. Surgical training is necessary to properly equip a training surgeon with the appropriate skills to be able to safely remove obstructing cells and disease and in so doing achieve the best possible outcome for the patient. In the past these skills were acquired by trainees firstly watching surgery and then by performing procedures under supervision on patients. This so‐called “on‐the‐job training” does expose patients to additional risk. During training there is no guarantee that registrars would have sufficient FESS cases to achieve necessary competency to be able to practice as independent surgeons. In addition, there are no objective evaluation methods to assess their surgical performance as each patient has different anatomy and complexity preventing comparison between trainees. Thus, traditional surgical training for FESS depends on the availability of suitable patients and lacks objectivity for trainee assessment.

Recently, advanced 3D printer technology has allowed the development of 3D sinus model for surgical simulation training. The models are produced from patient computer tomography (CT) scans of paranasal sinuses of actual patients undergoing FESS. The advanced printer technology has allowed the paranasal sinuses of these patient to be anatomically the same in shape and size as well as tissue feel. The mass‐producibility of the 3D models also provide trainees with plenty of opportunities for surgical training without any risk to patients. In addition, the 3D models facilitate objective assessment of dissection quality by post dissection CT scans. We recently reported the utility of the current 3D models in remote surgical training.

Our aims of this study are to validate the model and to evaluate the educational benefit of simulation training with the current 3D‐printed sinus model for FESS.

MATERIAL AND METHODS

Participants

The institutional review board approved the present study (No. 018‐043). Forty‐seven otolaryngologists voluntarily participated in the present study. The written informed consent was obtained from all the participants.

For the subsequent validation study, participants were classified into three groups: the experts (the officially certified board members of the Japanese otolaryngology society performing over 200 FESS cases), intermediates (the officially certified members performing less than 200 cases), and the otolaryngology registrars who had not yet been certified (novices, Figure 1). This differentiation was based on the Japanese residency program of the Japanese otolaryngology certification and on the criteria of the certified ESS instructor of the Japanese rhinology society. The Japanese residency program required 2 years of being junior residents and the following 4 years of being otolaryngology residency before taking an examination for the board certification of the Japanese Society of Otorhinolaryngology‐Head and Neck surgery. After the board‐certificated, surgeons who have performed 200 FESS cases are certified as the ESS instructor by the Japanese Rhinology society.

FIGURE 1

The flow of the present study. Forty‐seven otolaryngologists took part in the present study. The experts (n = 9), intermediates (n = 19), and novices (n = 19) were classified based on the certification of official board member of the Japanese otolaryngology society and the number of experienced FESS cases. The simulation trainings were conducted once by the experts and intermediates. Among the novices, those who requested more training performed simulation trainings total seven times (the training group). The other novices performed the simulation training three times (the control group).

Simulation trainings

The 3D sinus models were purchased from Fusetec (Adelaide, South Australia). The models were manufactured based on 3D‐printer technology from the axial CT scans of patients with chronic rhinosinusitis who had undergone sinus surgery (Figure S1A–C). The models are primarily 3D printed at 0.0125 mm slices, utilizing multiple materials simultaneously, with a unique voxel‐based software integration. To realize haptic feedback of human tissue, bone and skin structures are constructed with a Shore Hardness (D) 83–86 and a Shore Hardness (A) 28–33, respectively. The medial wall of the maxillary sinus, the anterior wall of the sphenoid sinus, and the inside space of the frontal sinus are visible from the outside so that to evaluate completeness of surgery and severe complication by direct vision (Figure S1D–H). Nine different models (Model 1 to Model 9) with different degrees of anatomical complexity have been developed. In this study, three different models (Model 2, Model 6, and Model 8) were used (Table S1). A 4‐mm rigid nasal endoscope and a monitor (Telepac, Storz, Tuttlingen, Germany), standard ESS instruments (Storz), and a powered microdebrider (Medtronic, Jacksonville, FL) were also prepared. Infrared reflective markers were attached to several surgical instruments for future motion capture study, although it was not a focus of the present study.

Before the study was performed, the purpose, design of the study, and each step was explained to the participants. The key steps of the simulation training are summarized in Table 1. Briefly, the simulation training was started with uncinectomy and maxillary antrostomy, followed by anterior and posterior ethmoidectomy, sphenoidotomy, and frontal sinusotomy. The order that the surgical steps were to be performed was set with the criteria of when the dissector could proceed to the next step detailed in Table 1.

TABLE 1

Surgical steps in the simulation trainings

Surgical steps	Procedures	Tasks	Criteria to start procedures
1	Uncinectomy/middle meatal antrostomy	Enlargement of ostia of maxillary sinus	N/A
2	Anterior‐ethmoidectomy	Resection of anterior ethmoidal cells including bulla ethmoidalis	After removal of posterior fontanelle in maxilla
3	Posterior‐ethmoidectomy	Resection of posterior ethmoidal cells	After the removal of bulla ethmoidalis
4	Sphenoidotomy	Enlargement of natural ostium of sphenoid	After the resection of lower part of the basal lamella
5	Frontal sinusotomy	Resection of cells in frontal recess	After the entrance into sphenoid
6	(Full‐house FESS)	Status of all sinuses completely opened and exposed	After the frontal sinusotomy finished

Abbreviation: FESS, functional endoscopic sinus surgery.

The coronal, sagittal, and axial CT images of the corresponding models were provided in advance. Dissection allocation time was up to 45 min. The dissectors attempted to complete a full house FESS (complete sphenoethmoidectomy with frontal sinusotomy and maxillary antrostomy) during that time. Feedback was provided on the dissection technique at the completion of the dissection. Participants' technical skills were assessed by the first author (MS) according to the objective structured assessment of technical skills (OSATS) score for FESS. Each dissection was video recorded for later assessment.

After completion of the dissection, the experts evaluated the reproducibility of the models and the educational benefit. Face value was defined as “a type of validity that is assessed by having experts review the contents of a test to see if it seems appropriate,” and was performed by the experts answering the question “Do the models reproduce human nasal cavity and paranasal sinus?” using a VAS score (0: strongly disagree, 25: disagree, 50: neutral, 75: agree, 100: strongly agree). The content validation, defined as “an estimate of the validity of a testing instrument based on a detailed examination of the contents of the test items,” was also assessed by the experts answering the question “Do the simulation training have educational benefits for FESS?” using the VAS score (0: strongly disagree, 25: disagree, 50: neutral, 75: agree, 100: strongly agree). Background information, including the age, gender, clinical experience, dominant hand, and prior surgical experience of FESS, was also collected.

The simulation dissection was performed once by the experts and intermediates (Figure 1). Regarding the learning curves analyses, novices were invited to perform the dissections repeatedly. The surgical performance of the novices who participated in the repetitive training sessions (seven sessions, training group) was compared to those novices who underwent three dissection sessions (control group) without formal training. The right side of a same models was utilized at the 1st (Model 2 Right side), and the left side of the model at the second and the last dissection sessions (Model 2 Left side). Other types of models (Model 6 and 8, Right and Left side) with increasing difficulty were utilized for the third, fourth, and fifth dissections (Table S1). Novices were allowed to join the dissections a maximum of twice a day. Finally, novices in the training group without any clinical experience of FESS surgery performed FESS on cadavers to simulate FESS on a patient (mock FESS surgery) after the 1st and final model dissection (Figure 1), according to the rules summarized in Table 1. All the dissections were conducted within 1 month to minimize the influence that exposure to FESS during the novices' ongoing day‐to‐day training would have on their skill acquisition and dissection performance.

Outcome measures for model dissection

OSATS score was used to evaluate participants' surgical performances in the model dissection. The scoring system is designed to score the participants performance on specific procedures performed during FESS on the scale of one (unable to perform) to five (performs easily with good flow). A score of three or more in each checklist is considered competent for the task. In this study, the checklist of the intranasal preparation was excluded because vasoconstrictor and local anesthesia was not required for the 3D models. For inter‐rater reliability assessments, 22 recorded videos were randomly selected, and OSATS score correlation was evaluated between the two independent raters.

Progress status of surgeries was evaluated by one author (MS) onsite and confirmed by a blinded expert rhinologists (YN) watching the recorded videos. The progress status was expressed as the number of procedures that the participants had started or completed by the end of the allocated time. For example, if a participant was performing a sphenoidotomy when time was up, the progress was expressed as “4: sphenoidotomy” (Table 1). The time taken to complete a mini‐FESS (middle meatal antrostomy and anterior ethmoidectomy) was also recorded. If surgeon did not complete mini‐FESS within allocated time of 45 min, the amount of time counted as 2700 s (=45 min) in the analysis.

Severe complications were assessed by the first author (MS) onsite, and by the other blinded assessor offsite using the video recordings and by evaluation of the dissected models. Severe complications were defined as injuries to the skull base, internal carotid artery, optic nerve, anterior and posterior ethmoid arteries, lamina papyracea, and nasolacrimal duct.

Qualitative assessment by postdissection CT scan

At the completion of all the dissections, CT images of the 3D models were taken using a 16‐slice multidetector‐row CT scanner (Hitachi, Tokyo, Japan) with collimation of 0.63 mm at 120 kV and 200 mA or less and a rotation time of 1.0 s. Sagittal and coronal multiplanar reconstruction (MPR) images were obtained from the axial images. The details of this analysis are described in Figure S2. Briefly, the same sagittal and coronal plane were chosen from the CT images of the 3D models after each surgery. Unresected bone seen in the sagittal plane and coronal planes were measured by a blinded observer (TS) using ImageJ 1.50i (National Institutes of Health, Bethesda, MD, USA) and ONIS 2.4 (Digitalcore, Tokyo, Japan).

Statistical analysis

All data were expressed as mean (±SD). Shapiro–Wilk tests were applied to evaluate if the data fitted a normal distribution curve. Parametric data (experienced year, experienced cases, and OSATS score) were assessed with a two‐tailed t‐test between two groups, or with one‐way ANOVA followed by Tukey's test among three or more groups. Nonparametric data (Time to complete mini‐FESS, the intensity of the region of interests [ROI], and the length of the remnant cells) were assessed with Wilcoxon test between two groups, or with Kruskal‐Wallis test when there was more than three groups. Progress status of the surgeries was compared with Wilcoxon test between two groups, or with Kruskal‐Wallis test when there was more than three groups. A chi‐square test was used for comparison of severe complications. Pearson coefficients analysis were used to assess correlation between the onsite and blinded OSATS scores, and intraclass correlation coefficients were calculated to assess the interrater reliability. p values of less than .05 were considered statistically significant. All the analyses were performed by using JMP 11 (SAS Institute Inc.).

RESULTS

Face and content validity of the 3D models

The characteristics of the participants are shown in Table 2. Overall face and the content validities of the current simulation training model were 79.6 ± 13.2 and 88.8 ± 10.1, respectively (Figure 2). The detailed validation of the reproducibility of the anatomy and specific surgical procedures are also shown in Figure 2.

TABLE 2

Characteristics and surgical performances of participants

	All participants (n = 47)	Experts (n = 9)	Intermediates (n = 19)	Novices (n = 19)	p value
Surgeon characteristics
Experienced years (ave.)	10.9 ± 8.7	21.6 ± 8.5	13.9 ± 4.9	2.9 ± 1.9	Experts vs. intermediates; p < .001 Expert vs. novices; p < .001 Intermediates vs. novice; p < .001
Gender (F/M)	6/41	0/9	4/15	2/17
Dominant hand (right/left)	44/3	9/0	17/2	18/1
Experienced FESS cases (ave.)	213.9 ± 500.1	955.6 ± 815.6	66.8 ± 35.1	9.8 ± 14.8	Experts vs. intermediates; p < .001 Expert vs. novices; p < .001 Intermediates vs. novice; p = .871
Surgical performance
OSATS score	55.3 ± 15.1	74.7 ± 3.6	58.3 ± 10.1	43.1 ± 11.1	Experts vs. intermediates; p < .001 Expert vs. novices; p < .001 Intermediates vs. novice; p < .001
Progress of surgeries (%)					Experts vs. intermediates; p = .008 Expert vs. novices; p < .001 Intermediates vs. novice; p < .001
Full‐house FESS	15 (31.9%)	9 (100%)	5 (26.3%)	1 (5.3%)
Frontal sinusotomy	11 (23.4%)	0	9 (47.3%)	2 (10.5%)
Sphenoidotomy	2 (4.3%)	0	1 (5.3%)	1 (5.3%)
Posterior ethmoidectomy	16 (34.0%)	0	4 (21.1%)	12 (63.2%)
Anterior ethmoidectomy	2 (4.3%)	0	0	2 (10.5%)
Uncinectomy/Middle meatal antrostomy	1 (2.1%)	0	0	1 (5.3%)
Time taken to complete a mini‐FESS (sec)	1219.1 ± 677.9	596.1 ± 232.3	1093.2 ± 464.8	1612.7 ± 731.1	Experts vs. intermediates; p = .084 Expert vs. novices; p < .001 Intermediates vs. novice; p = .020
ROI intensity in the sagittal CT views	104.5 ± 26.8	81.1 ± 13.1	93.7 ± 15.1	126.4 ± 25.2	Experts vs. intermediates; p = .024 Expert vs. novices; p < .001 Intermediates vs. novice; p < 0.001
Remnant length of the posterior ethmoid cells in the coronal CT views (mm)	6.8 ± 4.3	3.8 ± 3.7	4.8 ± 3.7	10.1 ± 2.6	Experts vs. intermediates; p = .475 Expert vs. novices; p < .001 Intermediates vs. novice; p < .001

Abbreviations: FESS, functional endoscopic sinus surgery; OSATS, objective structured assessment of technical skills; ROI, region of interest.

FIGURE 2

Face and content validity of the 3D models and the simulation training. The face validity and content validity of the 3D models and the training were assessed. (A) The face validation of the models was undergone by the experts using VAS score (0: not at all to 100: completely representative of human paranasal sinuses). (B) The content validation of the simulation training was also assessed by the experts using the VAS score (0: not at all to 100: exactly valid).

Construct validity of the simulation dissections

Figure 3, Table 2, and Video S1 shows the surgical performances by the experts, the intermediates, and the novices. The inter‐rater reliability of the OSATS scores was high (r = 0.957, p < .001). The average of OSATS score in the participants was 55.3 ± 15.1. The OSATS score of the experts, the intermediates, and the novices were 74.7 ± 3.6, 58.3 ± 10.1, and 43.1 ± 11.1, respectively. There was a significant difference in the OSATS score between the groups (ANOVA, p < .001, the experts vs. the intermediates, p < .001, the experts vs. the novices, p < .001, and the intermediates vs. the novices, p < .001, Figure 3A).

FIGURE 3

Surgical performance of the simulation training by the experts, the intermediates, and the novices. The simulation training by the experts, the intermediates, and the novices were assessed with OSATS score (A), progress of surgeries (B), time for mini‐FESS completion (C), and sagittal and coronal CT scan assessment scores (D and E).

All the experts (100%), five intermediates (26.3%), and one novice (5.3%) had finished the frontal sinusotomy and completed Full‐House FESS within the time allowance (Figure 3B). The experts completed significantly more surgical steps than the intermediates and novices (p = .008 for the intermediates, and p < .001 for the novices, respectively). There was also significant difference between the intermediates and the novices (p < .001).

For mini‐FESS, the experts, the intermediates and the novices took 596.1 ± 232.3, 1093.2 ± 464.8, and 1612.7 ± 731.1 s, respectively to complete the dissection (Figure 3C). The experts and the intermediates completed mini‐FESS significantly more quickly than the novices (ANOVA, p < .001. the experts vs. the novices, p < .001, and the intermediates vs. the novices, p = .020).

The ROI intensity in the sagittal CT views for the experts, the intermediates, and the novices was 81.1 ± 13.1, 93.7 ± 15.1, and 126.4 ± 25.2, respectively (Figure 3D). The intensity was significantly lower in the experts and the intermediates than in the novices (the experts vs. the novices; p < .001 and the intermediates vs. the novices; p < .001). There was also significant difference between the experts and the intermediates (p = .024). The remnant length of the posterior ethmoid cells in the coronal CT views was 3.8 ± 3.7 mm in the experts, 4.8 ± 3.7 mm in the intermediates, and 10.1 ± 2.6 mm in the novices (Figure 3E). The length was significantly longer in the novices than in the experts and intermediates (the experts vs. the novices, p < .001, and the intermediates vs. the novices, p < .001).

There were five severe complications observed in the 1st dissection session, including injuries to the nasolacrimal duct (n = 3), lamia papyracea (n = 1), and the skull base (n = 1, Figure S3). Four occurred in surgeries by novices and one injury of the nasolacrimal duct in surgery by an intermediate. Because of the low frequency of severe complications here were no significant differences in the complication rate between the experts, intermediates, and novices (p = .148).

Educational efficacy of the 3D simulation training in novice cohort

The novices were divided into two groups (the training novice group and the control novice group). The characteristics of the training and control group are shown in Table 3. Compared to the control group, the training group had significantly less experience as an otolaryngology registrar (the training group 2.1 ± 2.0 years and the control group 3.8 ± 1.3 years, p = .020) and had performed significantly fewer FESS cases (the training group 3.3 ± 9.4 cases and the control group 17.0 ± 16.8, p = .002). The OSATS score in the second dissection session was significantly lower in the training group than in the control group (training group; 41.1 ± 8.0 and control group; 54.8 ± 9.6, p = .004, Figure 4A). In the training group, the OSATS score significantly increased after third dissection session (third session; 52.2 ± 9.7, p = .002, fourth session; 55.3 ± 8.2, p < .001, fifth session; 58.2 ± 7.9, p < .001, sixth session; 59.6 ± 3.9, p < .001, and the final session; 61.1 ± 6.9, p < .001, Figure 4A and Figure S4A). In the control group, the score at the final dissection session did not change from the baseline (56.4 ± 9.8, p = .720). There were no significant differences of the score in the final dissection session between the training group and the control group (p = .243).

TABLE 3

Characteristics and surgical performance of the training group and the control group

Characteristics	All novices (n = 19)	Novice training group (n = 10)	Novice Control group (n = 9)	p value
Experienced years (ave.)	2.9 ± 1.9	2.1 ± 2.0	3.8 ± 1.3	p = .020
Gender (F/M)	2/17	2/8	0/9
Dominant hand (right/left)	18/1	9/1	9/0
Experienced FESS cases (ave.)	9.8 ± 14.8	3.3 ± 9.4	17.0 ± 16.8	p = .002

Abbreviations: FESS, functional endoscopic sinus surgery; OSATS, objective structured assessment of technical skills; ROI, region of interest.

FIGURE 4

Comparison of surgical performances in the training and the control group in second and final dissection session. To assess the educational efficacy of the training, surgical performance was compared between the training group and the control group in points of OSATS score (A), progress of surgeries (B), time for mini‐FESS completion (C), and sagittal and coronal CT scan (D and E).

Training group (second) vs. control group (second); p = .004

Training group (final) vs. Control group (final); p = .243

Training group (second) vs. Training group (final); p < .001

Control group (second) vs. Control group (final); p = .720

Training group (second) vs. Control group (second); p = .008

Training group (final) vs. Control group (final); p = .200

Training group (second) vs. Training group (final); p < .001

Control group (second) vs. Control group (final); p = .1796

861.6 ± 441.0

Training group (second) vs. Control group (second); p = .004

Training group (final) vs. Control group (final); p = .775

Training group (second) vs. Training group (final); p = .004

Control group (second) vs. Control group (final); p = .537

Training group (second) vs. Control group (second); p = .775

Training group (final) vs. Control group (final); p = .013

Training group (second) vs. Training group (final); p = .001

Control group (second) vs. Control group (final); p = .480

Training group (second) vs. Control group (second); p = .445

Training group (final) vs. Control group (final); p = .178

Training group (second) vs. Training group (final); p = .003

Control group (second) vs. Control group (final); p = .396

In the second dissection session, none of the training group (0%) completed Full‐house FESS and two (22%) did in the control group. The progress of surgeries in the second dissection session was significant less advanced in the training group than in the control group (p = .008). Through the repetitive dissection session, the progress was significantly improved in the training group (Figure S4B and Video S2). At the final dissection session, 6 (60%) in the training group completed Full‐house FESS, and 3 (33.3%) in the control group did. The difference between the groups was not significant in the final dissection session (p = .200, Figure 4B).

The training group spend more time to complete mini‐FESS than the control group in the second dissection session (training group; 1783.2 ± 577.8 s and the control group 915.0 ± 347.1 s, p = .004), but no significant differences were found in the final dissection session (training group; 861.6 ± 441.0 s and the control group; 844.7 ± 465.7 s, p = .775).

The intensities of ROI in the sagittal CT views in the second dissection session were not significant different between the training group and the control group (the training group 122.1 ± 22.2 and the control group 119.3 ± 22.3, p = .775, Figure 4D and Figure S5A–C). In the final dissection session, the intensity in the training group significantly decreased (90.9 ± 10.3, p = .001) and significantly less than in the control group (111.1 ± 17.9, p = .013). Also, the remnant length of the posterior ethmoid cells in the coronal CT views in the second dissection session was not different between the training group and the control group (the training group 9.2 ± 3.5 and the control group 7.8 ± 3.9, p = .445, Figure 4E and Figure S5D–F). At the final dissection session, the length was significantly shortened in the training group (3.6 ± 3.0 mm, p = .003), but not in the control group (6.0 ± 4.0 mm, p = .396).

The skill acquisition through the model dissection in the training group was confirmed by comparing the outcome criteria in the cadaver surgery dissection session before and after the repetitive training session. The OSATS score and the progress of surgery significantly improved in the cadaveric mock surgeries after the final dissection session when the outcome criteria were compared to those after the first dissection session (OSATS score 43.4 ± 11.1 to and after the 62.0 ± 6.4, p = .015, Figure 5A, and the progress of surgery p = .009, Figure 5B). In the novice group both of the OSATS score and the progress of surgeries in cadaveric mock surgeries significantly correlated to those in the dissection session (OSATS score; r = 0.828, p = .003 and progress of surgery; r = 0.953, p < .001, Figure 5C and D).

FIGURE 5

The assessment of the cadaveric mock surgeries by five nonexperienced surgeons after the second and final dissection session. (A) The OSATS score in cadaveric mock surgeries was significantly improved after the dissection session. (B) After the dissection session, the surgeries were significantly more progressed than before. Noteworthy, most of the novice (80%) completed Full‐House FESS in cadaveric mock surgery after the training session, despite that they had never experienced any actual surgeries. (C) There was significant positive correlation between OSATS score in dissection session and cadaveric mock surgeries. (D) The progress of cadaveric mock surgeries was significantly correlated with the progress in the dissection session.

DISCUSSION

This study illustrates the benefit of repeated surgical dissection on 3D‐printed models with significant improvement in the surgeon's ability to perform the surgical dissections for the different sinuses (OSATS score), progress of surgery and completeness of the surgical dissection (bone remnants on post dissection CT scan). In addition, the variety of anatomy and complexity of anatomy can be selected thereby providing an appropriate level of difficulty for the surgeon which can be tailored to the level of experience of the surgeon. The fact that the same anatomical model can be given to multiple different surgeons allows comparison of surgical skills between surgeons and allows for standardized teaching and evaluation of surgical skills for certification of surgeons. When considering restrictions on work hours and ethical concerns for patient safety, there is a growing demand for efficient surgical training programs. In the past there has been various training methods devised for nonpatient training of surgeons for FESS, from simplified training models , , to virtual reality (VR) applications , , and cadaveric dissections. Simplified models and specified skills training contributes to the acquisition of basic surgical techniques and simple endoscopic maneuvers of surgical instruments. , , However, these skills do not cover the skills and knowledge necessary to complete complex sinus surgery. VR provides further training opportunities without putting patients at risk. , , VR still might have issues with haptic feedback, which is essential for appropriate handling of surgical instruments. The gold standard of surgical training is still cadaver dissection. There is increasing lack of availability and difficulty in acquiring cadavers thereby limiting trainees exposure. One of the biggest drawbacks of cadaver training is that the anatomy found in cadavers is often simple and therefore lacks the complexity to allow development of the surgical skills necessary to manage complex sinus surgery particularly complex frontal sinus dissection.

Advanced 3D manufacturing and printing technology has been previously described for surgical training with models of the temporal bone, , , kidney, renal pelvis, and ureter, mandibular, aorta and heart all having been. In the field of rhinology, 3D‐printed models of paranasal sinuses , , , and skull base , , , , have also been developed. However, most of this research has been focused on creating and validating 3D models, not evaluating their educational efficacy. A small number of studies have examined the training effects but have lacked a control group and objective evidence of the training benefit.

In this study, we have shown significant educational efficacy of these 3D models by comparing the training group to the control group at multiple time points, including OSATS scores (overall surgical performance), the postdissection analysis of CT scans (for efficacy), the progress of surgeries, and time taken to perform a mini‐FESS (for efficiency). The surgeons' skill acquisition was also assessed by the assessment of performance on cadavers. Notably, 80% of the novices who had never previously performed an ESS before completed a full‐house FESS on a cadaver. This shows the value of the repetitive training using the 3D sinus models. This final evaluation utilizing the traditional gold‐standard cadaver dissection is a further validation of the value of dissection on these models. ,

The advantages of the 3D models are the ability of the trainee to perform the same surgery repetitively utilizing this repeated surgery to speed up skill adoption (30). In addition, there are nine different models with varying degrees of difficulty and anatomical complexity. This allows for a structured approach to training with the novice trainees starting on the simple models and as their skill set improves to progress to increasingly more complex and difficult surgeries. This allows certification boards to assess trainees' surgical skills on the same anatomy and surgical complexity and allows a minimum certification standard to be set as all trainees will be performing the same surgery on the same anatomy. A post dissection CT also allows an objective outcome assessment of the surgery. Finally the models do not have the issues associated with cadaver material such as availability, ethical considerations or infectious potential.

In this study, the OSATS score and the progress of surgical steps in the simulation training significantly correlated to the final score in the cadaveric mock surgeries. This correlation implies that the predictive validity of the simulation training correlates well with how the surgeon will per in the surgical suite. This predictive validity is considered the gold standard for evaluating training prior the training being incorporated into a training program.

This study has several limitations with the number of surgeons in each cohort being limited. There were only five novices in the cohort that underwent cadaveric mock surgery validation due to the lack of availability of cadavers. Therefore, their surgical performance could not be compared with those in the control group. There is a possibility that some of trainees may have acquired additional surgical skills in the period between evaluations. However, this was limited to 1 month so this would seem unlikely as the no‐one in the novice group performed surgery during this period.

CONCLUSION

Repetitive surgical dissection using the large range of anatomical variants in the newly designed 3D sinus models is beneficial for the acquisition of the surgical skills necessary for FESS. Postdissection CT analysis of the 3D models contributes to the objective evaluation of the acquired surgical skills.

AUTHOR CONTRIBUTIONS

Masanobu Suzuki, Atsushi Konno, Takashige Abe, A.J. Psaltis, and P.J. Wormald designed the project. Masanobu Suzuki, Kou Miyaji, Ryosuke Watanabe, Akira Nakazono and Akihiro Homma organized the simulation training and collected data. Masanobu Suzuki and Yuji Nakamaru analyzed OSATS score. Kou Miyaji, Takayoshi Suzuki, and Kotaro Matoba analyzed CT images. Masanobu Suzuki, Takashige Abe, and P.J. Wormald wrote the draft. All authors provided feedback on the manuscript and gave final approval to the submitted paper.

FUNDING INFORMATION

This work was partially supported by JSPS Grant‐in‐Aid for Scientific Research JP18H04102, 18K16871, 18KK0444, and 22K16923.

CONFLICT OF INTEREST

P.J. Wormald: consultant for Fusetec and receiving royalties from Fusetec. A.J. Psaltis: consultant for Fusetec. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figure S1 The appearance and endoscopic view of the 3D sinus models used in this study.

(A) The advanced manufactured 3D sinus models were used in this study. (B and C) Endoscopic views before (B) and after the surgeries (C). (D and E) The side view of the models before (D) and after the surgeries (D). (F) Endoscopic view of the enlarged natural ostium of maxillary sinus. The arrow in E and F indicated enlarged natural ostium of maxillary sinus. (G and H) The top view of the models before (G) and after the surgeries (H). (I) Endoscopic view of the enlarged frontal sinus. The arrow in H and I indicated enlarged frontal sinuses.

Click here for additional data file.

Figure S2 Representative CT views of the 3D models after trainings.

(A–C) The sagittal CT views of the 3D models after simulation training by a novice (A), an intermediate (B), and an expert (C), respectively. (D–F) The coronal CT views of the 3D models (Model 2 Right side) after simulation training by a novice (D), an intermediate (E), and an expert (F), respectively. For image analysis, a same sagittal and coronal plane were chosen. The selected sagittal plane is the one containing of frontal sinus drainage pathway. In the sagittal plane (A‐C), the amount of the remnant cell walls of frontal, ethmoid, and sphenoid sinuses were assessed by the intensity of region of interests (ROI; sky‐blue pentagon). The ROI was designed to the area enclosed the following anatomical points; the inferior wall of agger nasi cell, the superior edge of the frontal beak, the posterior wall of the frontal sinus, and the superior and inferior edge of the sphenoid sinus. This was set so that the degree of the remnant walls was evaluated, regardless of middle and inferior turbinate, unstable anatomical structures. The intensity of the ROI was measured by a blinded examiner (TS) using ImageJ 1.50i (National Institutes of Health, Bethesda, MD, USA). In this evaluation, the higher the intensity was, the more the bonny cell wall was remnant after surgeries.

In the coronal scan (D–F), the plane contains posterior ethmoid cells were selected for analysis. On the selected coronal scan, the length of the remnant of the inferior wall of the biggest posterior ethmoid cell (sky‐blue dotted line) were measured by the blinded examiner (TS) using ONIS 2.4 (Digitalcore, Tokyo, Japan), and subjected to the statistical analysis.

Click here for additional data file.

Figure S3 Representative images of the injuries of the nasolacrimal duct and the skull base.

(A‐D) The external appearance (A), endoscopic image (B), coronal (C) and axial (D) CT view of 3D models whose nasolacrimal duct was injured (the arrows). (E‐H) The external appearance (E), endoscopic image (F), coronal (G) and sagittal (H) CT view of 3D models whose skull base were injured (the arrows). The right images in C, D, and G are the CT views of a model without injuries for comparison.

Click here for additional data file.

Figure S4 The OSATS score (A) and progress of surgeries (B) in the training group through the repetitive training.

Click here for additional data file.

Figure S5 Representative CT views of the 3D models after trainings.

(A‐C) The sagittal CT views of the models after second training (A) and the final training (B) and that by an expert (C). Sky‐blue pentagon represents the region of interest measured for the analysis. (D‐F) The coronal CT views of the 3D models (Model 2 Left side) after second training (D) and the final training (E) and that by an expert (F). Sky‐blue dotted line represents the remnant bone wall of interest measured for the analysis.

Click here for additional data file.

Table S1 Three different 3D sinus models used in this study.

Click here for additional data file.

Video S1 Comparison of the simulation training among different surgeons.

The Model 2 Right side was dissected by novices (left), intermediates (middle), and experts (right). During the allocation time of 45 min, the expert completed Full‐house FESS, and the intermediate finished sphenoidotomy, while the novice did not complete posterior ethmoidectomy.

Click here for additional data file.

Video S2 Comparison of the simulation training before and after the repetitive training.

The Model 2 Left side was dissected by the same surgeon (the training group), before (left) and after (right) the repetitive training. Only maxillary sinus was addressed in the second simulation training before the repetitive training. After the repetitive training, the surgeon completed Full‐house FESS during the same allocation time and for the same model with the identical anatomy.

Click here for additional data file.