Teaching Radial Endobronchial Ultrasound with a Three-Dimensional-printed Radial Ultrasound Model.

BACKGROUND: Peripheral pulmonary lesion (PPL) incidence is rising because of increased chest imaging sensitivity and frequency. For PPLs suspicious for lung cancer, current clinical guidelines recommend tissue diagnosis. Radial endobronchial ultrasound (R-EBUS) is a bronchoscopic technique used for this purpose. It has been observed that diagnostic yield is impacted by the ability to accurately manipulate the radial probe. However, such skills can be acquired, in part, from simulation training. Three-dimensional (3D) printing has been used to produce training simulators for standard bronchoscopy but has not been specifically used to develop similar tools for R-EBUS.
OBJECTIVE: We report the development of a novel ultrasound-compatible, anatomically accurate 3D-printed R-EBUS simulator and evaluation of its utility as a training tool.
METHODS: Computed tomography images were used to develop 3D-printed airway models with ultrasound-compatible PPLs of "low" and "high" technical difficulty. Twenty-one participants were allocated to two groups matched for prior R-EBUS experience. The intervention group received 15 minutes to pretrain R-EBUS using a 3D-printed model, whereas the nonintervention group did not. Both groups then performed R-EBUS on 3D-printed models and were evaluated using a specifically developed assessment tool.
RESULTS: For the "low-difficulty" model, the intervention group achieved a higher score (21.5 ± 2.02) than the nonintervention group (17.1 ± 5.7), reflecting 26% improvement in performance (P = 0.03). For the "high-difficulty" model, the intervention group scored 20.2 ± 4.21 versus 13.3 ± 7.36, corresponding to 52% improvement in performance (P = 0.02). Participants derived benefit from pretraining with the 3D-printed model, regardless of prior experience level.
CONCLUSION: 3D-printing can be used to develop simulators for R-EBUS education. Training using these models significantly improves procedural performance and is effective in both novice and experienced trainees.

The prevalence of peripheral pulmonary lesions (PPLs) has been reported as 33% (range, 17–53%) in computed tomography (CT) screening studies of smokers at high risk of malignancy, and 13% (range, 2–24%) in the general population (1). Given findings of reduced lung cancer mortality with low-dose CT screening (2, 3), there have been recommendations for targeted screening programs (4). It is predicted that detection of PPLs will increase with the introduction of such initiatives (5).

Although up to 96% of PPLs are nonmalignant (2), a critical minority represent potentially curable lung cancer. Thus, obtaining an accurate tissue diagnosis without undue complications is paramount. Current sampling techniques include percutaneous methods such as CT-guided lung aspiration or biopsy or endobronchial techniques including electromagnetic navigation and radial endobronchial ultrasound (R-EBUS). Although CT-guided percutaneous lung aspiration or biopsy has excellent diagnostic yield for malignancy (6), this is offset by a relatively high false-negative rate of 20–30% with a need for further confirmatory testing in cases of suspected lung cancer. Furthermore, percutaneous sampling is associated with a higher complication rate of 24–39% (7). Although electromagnetic navigation offers enhanced navigational ability, the main disadvantage is the high operational costs and lack of direct visualization of the target lesion, with diagnostic yield varying from 59% to 85% (8).

R-EBUS is a bronchoscopic procedure performed for localizing and sampling PPLs. It uses a flexible ultrasound probe that is inserted through the working channel of a bronchoscope and advanced into different bronchial subsegments of the target lobe until the characteristic ultrasound image of a solid lesion is demonstrated (9). Although R-EBUS has reported sensitivity of 73% and an excellent safety profile (10, 11), a key factor impacting diagnostic yield is the ability to navigate the radial probe to be within the PPL before sampling (11). It is therefore important that the bronchoscopist possess the requisite skills and knowledge to accurately manipulate the radial probe, confirm its location by interpreting ultrasound images, and perform sampling maneuvers. R-EBUS has been demonstrated to be noninferior to CT-guided lung biopsy with a lower rate of complications including pneumothorax and hemothorax (3% vs. 27%, P = 0.03) (10).

R-EBUS is a technically challenging procedure associated with a steep operator learning curve (12). Indeed, experience in R-EBUS usually occurs after the proceduralist has attained proficiency in standard bronchoscopy (13). For similar interventional bronchoscopic techniques, credentialing bodies have suggested that 50 cases be completed under supervision to achieve competency (14). Currently, training for R-EBUS follows an apprenticeship model involving supervised procedures with live patients. However, there is increasing recommendation that simulation-based education be incorporated into training programs (15–17). Recent systematic reviews have shown that simulation training in bronchoscopy has significant benefits in procedural skills and time to completion compared with no intervention and has also been demonstrated to be more efficient than hands-on training with live patients (18–20). Furthermore, simulation has the additional benefit of protecting patients from safety risks posed by trainee participation while enabling greater standardization of skills assessment (21). However, a major barrier to implementation of simulation training is the prohibitive cost (22).

Three-dimensional (3D) printing technology offers a cost-effective solution and has been implemented for educational and procedural planning purposes in surgical specialties (23). In interventional pulmonology, there is emerging evidence for development of models for simulated flexible bronchoscopy (24–33). However, to date, there have been no published reports of such models for R-EBUS simulation.

We hypothesize that 3D printing technology can be used to produce ultrasound-compatible airway models with a high degree of anatomical representation and that training using these models will improve R-EBUS performance as evaluated by a dedicated R-EBUS assessment tool. The study objectives are 1) development of a 3D-printed airway model with ultrasound compatibility and 2) evaluation of the utility of this model as a training tool in respiratory medicine trainees with different levels of prior R-EBUS experience.

Methods

This prospective observational study was approved by the Austin Health Human Research Ethics Committee (HREC/56119/Austin-2019).

3D-Printed Model Development

The 3D-printed models were generated using high-resolution CT (HRCT) scans from two patients who had undergone R-EBUS at our institution. One patient was considered “low difficulty” with a PPL in a third-generation bronchus, and the other was “high difficulty” with a PPL in a sixth-generation bronchus.

The 3D models were created using 3D Slicer and Autodesk Meshmixer software and subsequently printed using an Ultimaker S5 3D Printer (34). A separate 3D lesion was designed to interlock on to the 3D-printed bronchial models to produce a realistic appearance when imaged under R-EBUS (Figure 1).

Figure 1.

High-difficulty three-dimensional–printed model. (A–E) Computed tomography image (A), Standard Tessellation Language (STL) image (B), model image (C), simulation (D), and radial endobronchial ultrasound image (E).

Further details of the 3D printing methodology with accompanying figures are provided in Appendix E1 in the data supplement.

The estimated cost to produce each model was AUD $150, with cost components as follows:

3D printer PLA and PVA filament: AUD $60

Ultimaker S5 printer consumables: AUD $10

Processing time and labor: AUD $80

3D-Printed Models for Improving R-EBUS Performance

Participants

All respiratory medicine trainees in Victoria, Australia, were invited to participate in the study. Participants were ranked by self-reported number of prior R-EBUS procedures performed and were matched in pairs, and one of each pair was randomly assigned to a specific group whereas the other was assigned to the other group, thereby resulting in two experience-matched groups (A and B).

Study protocol

Before attending the study, participants were provided with reading material describing endobronchial anatomy, the Kurimoto method for anatomical identification of PPLs, and R-EBUS technique; instructional videos demonstrating R-EBUS; and deidentified HRCT scans from which the two 3D models were generated. The aim was to simulate the preprocedural planning process.

The study was conducted at a dedicated endoscopy training center. All participants received a 40-minute lecture outlining the Kurimoto method (35) before separating into preassigned groups A and B. Participants in group A were regarded as the intervention group and received a 15-minute opportunity to practice R-EBUS, using the low-difficulty 3D-printed models, with instruction from an experienced endoscopist. Participants in group B did not receive this opportunity and remained in a waiting room during the 15-minute period that the intervention trainees were practicing on the model, and were regarded as the nonintervention group. Participants in both groups then proceeded to perform R-EBUS on the low-difficulty 3D-printed model. Their simulated performance was evaluated using an assessment tool (described below) by an experienced endoscopist who was blinded to the participant’s level of experience. It was determined that identification of the PPL was unsuccessful if the lesion was not visualized within 15 minutes. The process was repeated for the high-difficulty model.

R-EBUS assessment tool

An assessment tool specific for the R-EBUS procedure was modified from the Bronchoscopy Skills Tasks and Assessment Tool (BSTAT), which was validated as previously described (30, 36). This R-EBUS skills and tasks assessment tool (RE-STAT) examined six major components:

Anatomical recognition and navigation to the target airway

Precision of movement of the bronchoscope

R-EBUS performance using a guide sheath and identification of the PPL ultrasound image

Sampling the identified PPL

Equipment safety

Identification of different ultrasound images obtained by a R-EBUS probe

Each component was scored with a total possible score of 30 (Appendix E2). For practical purposes, sampling using forceps and washings was not performed during the study, and therefore, the total achievable score was 26.

Statistical Methods

Unpaired t tests were used to determine the significance of the difference between the mean RE-STAT scores of the two groups, for both low- and high-difficulty models. Data were reported as mean values ± standard deviation. Levene’s test was used to compare score variability between the two groups.

Linear regression was used to evaluate the effect of prior R-EBUS experience upon the benefit derived from using the 3D-printed models. The number of prior procedures performed was log(to base 10)+1 transformed to account for the skewness of this variable. To combine the findings from low- and high-difficulty models, analysis was performed using a linear mixed model with the participant as a random effect.

A P value <0.05 was regarded as statistically significant. Analyses were performed using Minitab Statistical Software.

Results

Participants

Twenty-one trainees representing nine institutions participated in the study. The majority of trainees were in their first and second years of specialist training. Although all trainees had experience in standard flexible bronchoscopy, self-reported previous R-EBUS procedures varied widely between 0 and 35 (median, 2; interquartile range, 0–6). The mean number of prior procedures was 7 for both the intervention and nonintervention groups. The flow of participants through the simulation is presented in Figure 2.

Figure 2.

Flow of study participants. (A) First simulation: low-difficulty model. (B) Second simulation: high-difficulty model. 3D = three-dimensional; EBUS = endobronchial ultrasound; RE-STAT = radial EBUS skills and tasks assessment tool.

For both the low- and high-difficulty 3D-printed models, group A (intervention group) performed significantly better than group B (nonintervention group).

For the low-difficulty model, group A achieved a mean RE-STAT score of 21.5, compared with 17.1 for group B, resulting in a statistically significant difference of 4.45 (95% confidence interval [CI], 0.61–8.28; P = 0.03), which reflects a 26% improvement in simulated performance (Table 1). Levene’s test showed group B to have significantly greater variability in scores than group A, with more low scores observed (P = 0.017). More specifically, the scores for the RE-STAT domains of performance of R-EBUS and lesion sampling were significantly higher in the intervention group (P = 0.02 and P = 0.04, respectively).

Table 1.

RE-STAT scores for performance of radial EBUS using the low-difficulty 3D-printed model

RE-STAT Component	Group A	Group B	Mean Difference (95% CI, P Value)
Anatomical recognition	3.9	3.8	0.1 (−0.28 to 0.42, P = 0.68)
Precision of movement	2.1	1.54	0.56 (0.15 to 0.95, P = 0.01)
Performance of R-EBUS	7.1	5.27	1.83 (0.27 to 3.38, P = 0.02)
Performance of lesion sampling	2.5	1.18	1.32 (0.09 to 2.54, P = 0.04)
Equipment safety	3.3	3.0	0.3 (−0.43 to 1.03, P = 0.40)
Identification of ultrasound images	2.2	2.82	−0.62 (−1.86 to 0.63, P = 0.31)
Mean total score (standard deviation)	21.5 (2.02)	17.1 (5.7)	4.45 (0.61 to 8.28, P = 0.03)

Definition of abbreviations: 3D = three-dimensional; CI = confidence interval; EBUS = endobronchial ultrasound; R-EBUS = radial endobronchial ultrasound; RE-STAT = R-EBUS skills and tasks assessment tool.

For the high-difficulty model, group A had a mean RE-STAT score of 20.2 versus 13.3 for group B with a statistically significant mean difference of 6.95 (95% CI, 1.37–12.49; P = 0.02), corresponding to 52% improvement in simulated performance (Table 2). Similarly, group B demonstrated significant variability in scores compared with group A (P = 0.049). Once again, the intervention group displayed significant improvement in the domains of performance of R-EBUS and lesion sampling (P = 0.02 and P = 0.03, respectively).

Table 2.

RE-STAT scores for performance of R-EBUS using the high-difficulty 3D-printed model

RE-STAT Component	Group A	Group B	Mean Difference (95% CI, P Value)
Anatomical recognition	3.7	2.0	1.7 (0.71 to 2.69, P = 0.001)
Precision of movement	1.8	1.45	0.35 (−0.18 to 0.87, P = 0.19)
Performance of R-EBUS	7.3	4.09	3.21 (0.47 to 5.95, P = 0.02)
Performance of lesion sampling	2.4	0.72	1.68 (0.21 to 3.14, P = 0.03)
Equipment safety	2.8	2.18	0.62 (−0.79 to 2.03, P = 0.37)
Identification of ultrasound images	2.2	2.82	−0.62 (−1.86 to 0.63, P = 0.31)
Mean total score (standard deviation)	20.2 (4.21)	13.3 (7.36)	6.95 (1.37 to 12.49, P = 0.02)

For definition of abbreviations, see Table 1.

Effect of Prior R-EBUS Experience

For the low-difficulty model, linear modeling of the RE-STAT score against prior R-EBUS experience showed a significant difference in the slopes of groups A and B. Participants with less prior experience derived more benefit from the intervention (P = 0.001) (Figure 3A). Viewed a different way, participants in the nonintervention group benefitted more from having prior EBUS experience, compared with those in the intervention group. Fitted regression equations can be used to predict RE-STAT scores for an individual based on the number of previous R-EBUS procedures:

Figure 3.

Plot of radial endobronchial ultrasound skills and tasks assessment score versus prior level of radial endobronchial ultrasound experience. (A) Low-difficulty model. (B) High-difficulty model. EBUS = endobronchial ultrasound.

Group A predicted RE-STAT score =  21.343 + 1.41 log(EBUS number+1)

Group B predicted RE-STAT score =  12.840 + 8.89 log(EBUS number+1)

For example, if the number of EBUS cases is 5, a participant in group A is predicted to have a score of 21.343 + 1.41 log(6) = 22.4, and a participant in group B is predicted to have a score of 12.840 + 8.89 log(6) = 19.8. If the number of EBUS cases is 10, the corresponding predictions are 22.8 and 22.1, which are closer scores due to increased EBUS experience.

For the high-difficulty model, there was no significant difference in the slopes of the study groups (P = 0.723) (Figure 3B). Although the mean difference between groups was significant (P = 0.018), there was an estimated (constant) difference of 6.50. Fitted regression equations can be used to predict RE-STAT score:

Group A predicted RE-STAT score =  17.99 + 4.21 log(EBUS number+1)

Group B predicted RE-STAT score =  11.49 + 4.21 log(EBUS number+1)

Therefore, if the number of EBUS cases is 5, a participant in group A is predicted to have a score of 17.99 + 4.21 log(6) = 21.3, and a participant in group B is predicted to have a score 6.5 less than this (i.e., 14.8).

When considering the combined results from the low- and high-difficulty models, a single regression model can be developed that incorporates prior experience level, whether the participant received intervention or not, and whether they performed on the low- or high-difficulty case:

Predicted RE-STAT score = 11.5 + [4.96 ×  log(prior cases+1)] + [5.6 × (Intervention Y/N] + [2.9 × (Difficulty High/Low)]

Discussion

This study describes a novel application of 3D printing to produce ultrasound-compatible tracheobronchial tree models. We show it is possible to produce such models at affordable cost, using readily available open-source software, 3D printing technologies, and materials, without needing extensive software training, and that the models can replicate ultrasonographic features of PPLs. Critically, this study demonstrates the utility of using a 3D-printed model as a training tool for R-EBUS. Even limited practice with these models significantly improved R-EBUS simulated performance in a cohort of predominantly inexperienced R-EBUS bronchoscopists. Importantly, the largest differences were observed in domains examining manual dexterity skills such as performance of R-EBUS and lesion sampling when comparing the intervention versus the nonintervention group.

As more countries implement lung cancer screening programs (4, 37, 38), it is expected that the incidence of PPLs will increase. As a key tissue sampling tool for PPLs, the corollary is that the demand for R-EBUS will also grow. As such, adequate training and quality benchmarking in this technique is essential.

Bronchoscopic techniques have traditionally been taught using an apprenticeship method (22). This approach requires balancing patient safety against trainee need for experience (39). Furthermore, it makes standardization across centers and objective skills evaluation challenging (40). Over the last 2 decades, there has been a paradigm shift from the apprenticeship method to structured training programs in respiratory medicine. Current guidelines from credentialing bodies place increasing emphasis on educational outcomes or competencies rather than volume-based certification (15–17). These guidelines also recommend incorporation of bronchoscopy simulation, particularly in the early stages of procedural training (18, 21). Given that 81% of participants recruited for our study were novice R-EBUS bronchoscopists, our results are likely to reflect the potential effect of instituting guideline recommendations for simulated R-EBUS training as part of a structured education program.

Bronchoscopy simulation training currently exists in two forms: high-fidelity virtual simulators and low-fidelity physical models (17, 41). The advantages of high-fidelity simulators are that they allow repeated training in a zero-risk environment, they expose trainees to a range of clinical scenarios, and they provide the ability to track performance metrics to give instant feedback (17). However, this often comes at prohibitive cost, regularly exceeding $100,000, and so uptake is limited (22, 42). In contrast, low-fidelity simulators are generally an order of magnitude cheaper than high-fidelity simulators (28, 29); however, their functionality has been comparatively more limited.

3D printing technology brings significant advantages to standard low-fidelity simulators. The cost of each model in this study is estimated at AUD $150 and is consistent with previously published reports (25–29). With greater availability of 3D printers, it is likely that production costs will decline, further enhancing the financial viability of adopting this approach to generation of low-fidelity simulators. Only one study has discussed the economics of using 3D-printed bronchoscopy simulators for a training program (29). DeBoer and colleagues calculated an approximate 40% annualized training cost reduction per trainee when comparing their model with the nearest-priced commercially available model (USD $416 vs. USD $730) (29). These results suggest that 3D-printed bronchoscopy simulators are economically viable and have the potential to significantly reduce training costs for hospitals. In the absence of on-site 3D-printing expertise, it is now eminently feasible and inexpensive for CT scan images to be sent digitally to centers with requisite experience and facilities to produce the models.

Critically, 3D printing provides a means to enhance functionality and interactivity of models by democratizing their design (25, 31, 32). Bespoke models have been produced for training in transbronchial needle aspiration, in which a preference for low-fidelity models and transferability of simulator-acquired skills to clinical settings has been demonstrated (43, 44). Models could incorporate pathological anatomy to facilitate procedural planning, as seen in applications of 3D printing in surgery (45–48). A growing body of literature demonstrates the potential to produce 3D-printed bronchoscopy simulators with high anatomical representation that provide a realistic endobronchial navigation experience (24–28). Training with 3D-printed bronchoscopy simulators also improves trainees’ quality of bronchoscopic task performance (27, 29, 33), decreases task completion times (27, 29, 31), and increases skills confidence (29, 49). Findings from this study suggest that 3D-printed models are not only useful for R-EBUS novices but also of value for proceduralists with at least intermediate experience.

This study had a number of strengths. With regard to the development of the model, modifications for R-EBUS such as the addition of an ultrasound-compatible lesion and the ability to sample the lesion have enhanced the anatomical realism of the model. Furthermore, the study was designed to recreate the recommended procedural planning process, which occurs before R-EBUS performance, including instructional references and expert teaching. As such, the only variable between the two groups in this study was the opportunity to train with the 3D-printed model before assessment.

We identify some limitations of our study. We acknowledge that the RE-STAT used in this study was modified from the validated BSTAT and is yet to be validated in its own right. However, there is currently no standardized test that measures the technical skills needed to perform R-EBUS specifically. As such, we sought to use the most appropriate assessment tool available from bronchoscopy literature, which has previously been modified for other bronchoscopic procedures such as linear probe EBUS. The RE-STAT is similar to the B-STAT in the domains of anatomical recognition, scope operator technique, performance of sampling, and adjunctive questions regarding ultrasound image interpretation. The two main differences from the BSTAT are the inclusion of a section addressing management of complications of transbronchial lung biopsy and transbronchial needle aspiration, and points for anatomical recognition correlated with imaging findings. As such, the modifications made to develop the RE-STAT do not diminish the face validity of the assessment of the radial EBUS procedure itself. Our study demonstrates that the RE-STAT has utility as a proof-of-concept tool and paves the way for a more formal validation study to be performed in the future.

Similar to other studies assessing the utility of 3D-printed models for standard bronchoscopy, the study validity would have been increased by comparison of R-EBUS performed in the simulation setting and in the true clinical setting (30). Validity may also be limited by small sample size. However, with more than 85% of the statewide cohort of trainees taking part in this study, with representation of all levels of training and nine institutions, our findings remain generalizable to other bronchoscopy training programs.

Although the intervention and nonintervention groups were matched for experience levels, concealed randomization was not possible and may be a source of bias. However, the rigor of study design has been balanced with pragmatic considerations of allowing all trainees to experience practice opportunities with the models.

The matching of participants between group A and group B was based on experience, not competence, and similar prior exposure to the procedure may translate in different levels of performance. However, previous studies have similarly used experience levels as opposed to tested competence levels. For example, Steinfort and colleagues evaluated a low-fidelity bronchoscopy simulation model in novice, intermediate, and advanced skill levels based on prior bronchoscopy experience and number of cases (30).

Finally, we acknowledge that the same model(s) were used for both training and assessment and that skills are only repeated between practice and testing, rather than transferred to the different simulation situation. Furthermore, this study measures short-term learning outcomes only (assessment immediately after training) and does not assess skills retention. This forms the focus of future studies.

Future directions include development of a 3D model in which multiple sampling modalities, such as brushings and biopsies of a lesion, can be performed. The findings from this study may be integrated into a structured bronchoscopy training program with longer follow-up, allowing assessment of long-term skills retention and more detailed calculation of training program cost savings. Lastly, these models have potential in the planning of complex R-EBUS procedures, in which preprocedural navigation and lesion sampling can be simulated.

Conclusions

We demonstrate the feasibility of developing a 3D-printed model with novel specifications for R-EBUS. We show that such models are a viable method of training and assessing R-EBUS and that pretraining on a 3D-printed model improves subsequent assessment task performance.