Learning curve for endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) of pancreatic lesions in a novel ex-vivo simulation model.

Background: Endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) is essential in the management of digestive cancers. However, teaching and learning this technique remain challenging due to the lack of cost-effective models. Material and methods: This was a prospective experimental study using a complete porcine upper gastrointestinal ex-vivo organ package, placed in an Erlangen Active Simulator for Interventional Endoscopy (EASIE-R), and prepared with one cyst and two solid masses (2 cm). Five fellows inexperienced in EUS-FNA were enrolled, performing 10 procedures on each lesion, alternatively. The total time, number of attempts for success, of needle view losses, and of scope handling were recorded, associated with an independent skills rating by procedure. We compared the first 15 procedures with the last 15 for each fellow.
Results: The fellows successfully performed all procedures in 2 to 40 minutes, requiring 1 to 6 attempts. All (5/5) improved their total time taken (P < 0.001), number of times when the EUS view of the needle was lost (P < 0.05), scope handling (P < 0.005), and skills rating (P < 0.001), whereas 4/5 (80 %) improved their number of attempts. The overall evaluation showed a significant decrease (P < 0.001) in the total time taken (11.2 ± 7.8 vs 4.3 ± 2.2 minutes), number of attempts (2.6 ± 1.2 vs 1.2 ± 0.7), number of times when the EUS view of the needle was lost (2.3 ± 2 vs 0.5 ± 0.7), and need for scope handling (1.1 ± 1.7 vs 0.1 ± 0.2). We also observed an improvement in skills rating (5 ± 1.9 vs. 7.7 ± 1.1).
Conclusion: This newly designed ex-vivo model seems to be an effective way to improve the initial learning of EUS-FNA, by performing 30 procedures.

Introduction

The role of endoscopic ultrasound (EUS) has dramatically increased in the field of gastroenterology for both diagnostic and therapeutic interventions 1 2 3. EUS-guided fine-needle aspiration (EUS-FNA) is essential in the diagnosis of cystic structures as well as in cancer staging of various malignancies 4 5 6. One of the challenges for developing proficiency in EUS-FNA relates to limited opportunities for high-quality training and the notable clinical learning curve required for improved efficacy 7 8 9 10. Indeed, with regard to cytopathological diagnosis of pancreatic cancer, sensitivity clearly increases with the endoscopist’s experience 11 12.

To date, two methods for learning EUS-FNA have been reported: formal training, consisting of fellowship in a dedicated training center for 6 – 24 months, and informal training, consisting of various didactic sessions which usually include short hands-on experiences. Hands-on training in EUS-FNA has traditionally used various models: (1) phantoms devoid of animal material (Olympus, Tokyo, Japan) 13; (2) models using porcine organs (upper digestive EUS-FNA) 14; and (3) live pigs 8 15 16. Simulators do not currently offer training in FNA. None of the aforementioned have demonstrated evidence to provide a cost-effective learning program 17, and between 8 % and 50 % of US endoscopists report that they are self-taught. While self-education is feasible and effective for simple endoscopic procedures 18, there is little validation for more complex procedures such as EUS-FNA 19. Moreover, the majority of gastroenterologists adopting EUS-FNA are compiling their learning curve via live human patients, which is suboptimal for patient safety 20. Finally, formal training programs are scarce in Europe and, where present, only allow training for a small number of endoscopists per year, due to the costs and accessibility of live animal models at dedicated didactic centers 8 9.

In 2001, the American Society for Gastrointestinal Endoscopy (ASGE) recommended that the minimum number of EUS procedures before being competent should be 150 supervised cases for diagnosis, with 50 EUS-FNA including 25 cases of pancreatic lesions 21. The European Society of Gastroenterology discourages self-learning of EUS-FNA. They instead recommend combining various simulators and, if available, live pig models to perform a minimum of 20 and 30 supervised EUS-FNAs of non-pancreatic and pancreatic lesions, respectively (Grade C recommendation) 22. In 2011, the British Society of Gastroenterology recommended that the trainees have to perform 250 EUS procedures on humans, including 75 EUS-FNA, 45 being on pancreatic lesions 23. However, there are no recommendations about pre-clinical non-human training, and it will be less and less acceptable in the future to start such procedures directly on patients.

Consequently, because of the costs, availability, and ethical dilemmas associated with live pig models as well as safety issues of training directly on patients, we describe the use of a novel ex-vivo simulator for training in EUS, so as to define the learning curve for EUS-FNA in ex-vivo models. Therefore, our aims were: (1) to assess the efficacy of a newly designed ex-vivo EUS model for training endoscopists in EUS-FNA for both cystic and solid pancreatic and peri-pancreatic lesions, at the beginning of their experience; (2) to elucidate an eventual learning curve for EUS-FNA procedure in this experimental setting.

Material and methods

The study was designed as a prospective, ex-vivo animal study. Modified Erlangen Active Simulators for Interventional Endoscopy (EASIE-R, Endosim, LLC, Hudson, Massachusetts, United States) specifically designed (Fig. 1) for EUS-FNA were used to perform the experiments. The study was exempt from internal review board approval as no human research subjects or live animal tissue was involved in the investigation.

Fig. 1

 Solid mass and cystic lesion: macroscopic view of specimen and EUS appearance. a Solid mass made with chicken heart colored in blue and heated in microwave, and finally rolled into small bowel. b Solid and cystic lesion sutured on the stomach. c EUS view of the cystic lesion: homogeneous, well limited, and hypoechoic. d EUS view of the solid mass: heterogeneous, hypoechoic, and hyperechoic.

Model setting

The ex-vivo simulator consisted of a complete porcine upper gastrointestinal organ package (esophagus, stomach, proximal duodenum, pancreas, liver, and spleen) suspended in the EASIE-R simulator (Endosim), which was filled with water (Fig. 2). The EASIE-R model has been validated in previous endoscopic experimental settings, including the endoscopic retrograde cholangiopancreatography (ERCP) training model 24, an endoscopic hemostasis model 25 26 27, and a gastric ESD model 28.

Fig. 2

 Complete EUS ex-vivo specimen placed in the EASIE-R simulator.

Three types of lesions were created within the ex-vivo model: one cyst and two solid masses, simulating a pancreatic pseudocyst (lesion 1), a pancreatic tumor (lesion 2), and a peri-pancreatic lymph node (lesion 3), respectively (Fig. 1 a). The position and size of the lesions were standardized, measuring 2 cm for all of the procedures. To simulate the cystic lesion, a pig bladder was sutured against the anterior wall of the stomach and filled with water mixed with Methylene Blue. The solid lesions were generated using chicken heart dyed with Methylene Blue and covered with small bowel. These were sutured against the posterior wall in the pancreatic area, and in the anterior wall 2 cm under the cardia to simulate a pancreatic tumor and peri-pancreatic lymph node, respectively (Fig. 1 b).

Operators and procedures

Five gastroenterology fellows participated in the evaluation of the ex-vivo EUS-FNA simulator. They were gastrointestinal fellows with experience in diagnostic and therapeutic colonoscopies and gastroscopies, but they had no previous clinical experience in endoscopic retrograde cholangiopancreatography, or in EUS and EUS-FNA. Each performed a total of 30 procedures over a period of 3 weeks (around 10 per week), carrying out alternately 10 procedures on each lesion (pancreatic pseudocyst, pancreatic solid lesion, and peri-pancreatic lymph node) using EUS-FNA 22 gauge needles (Cook and Boston Scientific). The number of 30 procedures was decided based on the ESGE and ASGE recommendations about EUS-FNA on pancreatic lesions 22 23, and on the Mertz and Gautman study, which suggested that the sensitivity increased through 30 cases in pancreatic masses 29. Before initiating the study, each endoscopist read the ESGE and ASGE recommendations for the EUS-FNA technique and had already observed numerous procedures performed by experts in humans. Subsequently, three procedures, one for each lesion, were performed by each investigator endoscopist for basic EUS instruction on ultrasonographic orientation within the model. The sequence of 30 procedures was designed to avoid confounding of the learning curve given the heterogeneity of lesion subtypes. Thus, for each endoscopist, the sequence was as follows: lesion 1 “procedure 1”, lesion 2 “procedure 1”, lesion 3 “procedure 1”, lesion 1 “procedure 2”, etc.). This prevented 10 consecutive procedures from being performed on any given single lesion.

Before each procedure, a senior endoscopist would place the echoendoscope 1 cm proximal to the location of the actual lesion and then the official procedure time would begin once the novice endoscopist was handed the endoscope. The first step was to take position directly adjacent to the lesion of interest with an adequate ultrasound view for FNA ( Fig.1 c, Fig. 1 d). Subsequently, the needle was set up and one or more attempts were carried out until achieving success as described below. For the cystic lesion, one attempt was defined as one try of in-out pass into the cyst; for the solid lesions, the FNA was standardized based on the ESGE technical recommendations in the literature 22 and one attempt corresponded to five in-out passes throughout the lesion, then the needle was removed and the sample was assessed by flushing it, as is performed during live patient care.

The procedure was considered successful and complete when 1 cc of blue fluid was aspirated back into the syringe (cystic lesion), or when a blue solid tissue sample (not fluid) was achieved into the needle upon flushing it (solid lesions), considering the sample was large enough for a histological analysis. If there was no blue liquid or tissue retrieved, this attempt was considered to be a failure, and thus the timer would be started again for the next attempt. A maximum number of six attempts was allowed for each procedure; otherwise the procedure was aborted and considered a failure.

Aims and measurements

The total times for the procedures including time for positioning the scope and setting up the needle, and time to perform each FNA attempt, were recorded as well as the number of attempts to achieve procedural success as defined above. Each attempt time was measured from once the needle was set up to its removal after five passes had been carried out. To evaluate the technical difficulties and risk of complications, we recorded the number of times when the EUS view of the needle was lost and when the assistant had to hold the scope to maintain the position. Finally, an external independent expert endoscopist (> 100 EUS-FNA procedure/year) evaluated the technical skills for each procedure on a scale between 0 and 10, 10 being the best performance.

The primary objective was to evaluate the efficacy of a newly designed ex-vivo EUS model for initially training endoscopists in EUS-FNA, and improving their technical skills before procedures on patients. The evaluation criteria were the improvement in total time and number of attempts after 15 procedures. The secondary objectives were to elucidate the learning curve for the EUS-FNA procedure and to document the improvement in safety and skills in this experimental setting.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics version 20 (IBM SPSS Inc., Chicago, Illinois, United States). There was no sample size calculation since there is no clear data in the literature about the EUS-FNA learning curve in such models. The number of fellows was based on their availabilities. Regarding the number of procedures, Mertz and Gautam demonstrated a consistent improvement in the sensitivity of EUS-FNA of pancreatic lesions through 30 – 40 FNA cases for a single endoscopist who did not undergo dedicated training in EUS, so we retained the number of 30.

Data are expressed as mean ± standard deviation. Comparisons of means of continuous variables between two groups were performed using t test or Mann-Whitney U test. For count data, the Poisson log-linear regression model was performed. Mixed model analysis of repeated measures was used to take into account the correlation of the data and to assess fixed and random effects. The repeated measurements and all data for the same operator tended to be correlated. For all tests, statistical significance was defined as P < 0.05. Scatterplots were used to analyze patterns in bivariate data (time and number of attempts).

Results

The five endoscopists enrolled in this present study performed all of the procedures successfully with procedure times ranging from 2 to 40 minutes and required 1 to 6 attempts to achieve success. One specimen (including esophagus, stomach, liver, pancreas, and spleen) was used per participant and only the cystic lesion was changed once because of many punctures being made which compromised the cyst’s ability to retain fluid. While no endoscopist undertaking the procedures in the study had previously performed EUS-FNA, we observed significant heterogeneity in terms of mean total procedure time during the first 15 procedures: 16.3 ± 9.9 minutes for operator 1, 16.0 ± 8.7 minutes for operator 2, 7.9 ± 2.7 minutes for operator 3, 10.7 ± 5.6 minutes for operator 4, and 4.9 ± 1.9 minutes for operator 5. However, such a difference was not observed for the final 15 procedures: 4.5 ± 2.9, 5.9 ± 2.2, 3.6 ± 1.9, 4.7 ± 1.9, and 3.1 ± 0.9 minutes, respectively. Alternatively, there was minimal heterogeneity among the participants for the mean number of FNA attempts, which were 2.6 ± 1.2, 2.6 ± 1.5, 2.8 ± 1.1, 2.5 ± 1.4, and 2 ± 0.8, respectively.

Comparison for each fellow (Table 1)

Regarding the efficiency of the model in improving the operators’ skills, all of the fellows had a significant improvement in mean total procedure time (P < 0.001) when comparing the first 15 procedures with the last 15 procedures. Moreover, four out of the five also showed a significant improvement in the mean number of FNA attempts within an individual procedure during the course of their 30 cases. Regarding the evaluation of potential risk of complications, all had a significant decrease (P < 0.005) in the number of times the EUS view of the needle was lost during the puncture and the need to have the endoscope handled during the procedure to maintain a stable position. The independent skills evaluation of each operator made by the expert also showed a significant improvement (P < 0.001) in the skills for all of the endoscopists involved.

Overall comparison

When pooling the combined initial 15 procedures (75 procedures, 15 per operator) and the final 15 procedures (75 procedures, 15 per operator), there was a significant improvement in the operators’ performance. We observed a significant decrease between the two periods (first 75 and final 75) in the total procedure time from 11.2 ± 7.8 to 4.3 ± 2.2 minutes (P < 0.001) and of the number of FNA attempts from 2.6 ± 1.2 to 1.2 ± 0.7 (P < 0.001). We also observed a significant decrease in the loss of EUS view of the needle from 2.3 ± 2 to 0.5 ± 0.7 (P < 0.001) and of the number of times an assistant was needed to hold the endoscope during the puncture from 1.1 ± 1.7 to 0.1 ± 0.2 (P < 0.001). Finally, the independent skills evaluation demonstrated a significant improvement between mean score during the first 15 procedures per operator and mean score during the last 15 procedures: 5 ± 1.9 vs. 7.7 ± 1.1 (P < 0.001).

Learning curve (graphs)

When analyzing the data patterns in bivariate analysis (mean procedure time and number of attempts), we observed a reproducible learning curve for all of the operators, which seemed to reach a plateau after 15 procedures (Fig. 3 and Fig. 4). After this number of procedures, we observed a flattening of the progression curve.

Fig. 3

 Learning curves for EUS-FNA per endoscopist showing the improvement in both procedure total time (left side) and number of attempts (right side).

Fig. 4

 Overall (all endoscopists) learning curve for EUS-FNA showing the plateau phase reached after 15 procedures for both procedure total time (top) and number of attempts (bottom).

Discussion

The role of EUS has increased substantially in recent decades, becoming a critical tool in the armamentarium of gastroenterologists to guide decision making in digestive oncology. Furthermore, the field of interventional EUS has blossomed, with applications such as cystogastrostomy, hepatico-gastrostomy, and vascular therapy. The learning and mastery of EUS-FNA must be achieved before performing such advanced therapeutic techniques, as it provides foundational skills for the first step in every interventional EUS procedure: the access by needle puncture. However, current education in EUS-FNA and therapeutic EUS remains a critical challenge, since there is a lack of reproducible, effective and low-cost models available to meet this need. Unfortunately, there is currently no effective computer-based or virtual reality simulator, as they currently do not offer verisimilitude to the tactile feel of puncturing a human lesion, cyst or lymph node. Live pigs serve as very good models because their anatomy is close to humans, which is particularly suited for learning diagnostic EUS 8. Fritscher-Ravens and colleagues reported a study in which they enrolled two fellows who performed 96 EUS-FNA on living pigs 16. They demonstrated a significant improvement in the procedure times on patients, when comparing their performance before and after this training. However, live animals are very difficult to obtain on a large-scale basis due to high costs, ethical issues, and challenges in creating target lesions. Regarding ex-vivo models for EUS simulation, there is at present a single study described by Matsuda et al. 14. In this trial, the prepared pig organ was embedded in layers of Jell-O, a plastic tube was placed and connected to a pump system to simulate the aorta and a Y-shaped tube was put on the pig’s esophagus to simulate the trachea. Grapes were embedded in the gelatin to provide simulation for lymph nodes. This model was evaluated in a workshop and seemed suitable for learning EUS and EUS-FNA, but no learning curve study had been performed. Finally, considering the evolution of medicine and medical responsibilities, it will become increasingly questionable for fellows to start learning new procedures directly on patients without any previous hands-on experience.

For these reasons, we decided to conduct an experimental study based on the EASIE-R ex-vivo simulator, which was already validated in several learning curve settings in endoscopy. We specifically modified this model for teaching and learning EUS-FNA, creating solid and cystic lesions for targets to train in the EUS-FNA procedure. In this study that included five gastroenterology fellows, we show a significant improvement in the efficiency (duration) of successful FNA procedures, efficacy (number of attempts for successful procedures), in the endoscopists’ skills (independent evaluation), as well as a trend towards decreasing markers for risk of complications (loss of EUS view, need to hold the scope). Moreover, comparing the first 15 with the final 15 procedures for each endoscopist, we demonstrated a statistically significant learning curve for EUS-FNA indicating a “plateau” phase after 15 procedures within this model. This was consistent for each endoscopist separately, as well as for the overall cohort.

We acknowledge that there are several limitations to this study: the lesions were in only three different locations, we did not compare the fellows to an expert ultrasound endoscopist, and there was no evaluation to confirm that these skills translate to improvements in actual EUS-FNA in human patients. Furthermore, we did not evaluate the histological quality of the samples. Additionally, this model did not contain vascular structures to simulate arteries and veins, which are important anatomic structures to assess in real-world EUS. Finally, it is important to underline that this model is likely dedicated to initial learning and with the objective to teach fellows about the technique of EUS-FNA before their first procedures on patients, to improve their confidence. Indeed, in this form, it is not suitable for complicated lesions, such as uncinate ones, or for advanced EUS.

To date, this is the first learning curve investigation confirming the efficacy of a novel ex-vivo model simulator with target lesions in improving the interventional skills of novice endoscopists. Based on our results, we also suggested that a minimum of 15 procedures is required to reach a plateau in terms of EUS-FNA performance within this model. Moreover, it has the advantage of being reproducible, low-cost, and reusable for multiple procedures. Indeed, with regard to the cost-effectiveness, we used a total of three porcine upper gastrointestinal specimens costing $ 275 each, successfully placed in one single tray ($ 2000 for purchase; $ 600 for rent). So, the total cost of the whole study, including 150 EUS-FNA procedures, was $ 2825 (the needles were provided by the companies).

In conclusion, this study shows that the modified EASIE-R ex-vivo model with simulated real lesions seems effective in improving the skills of novice endoscopists for EUS-FNA. This training of 30 procedures could be proposed as first line of training, in order to give preliminary tips or advice and acquire technical skills, before performing the first supervised procedures on patients. Additional studies with some improvements in the model and an evaluation of the impact on clinical practice of EUS-FNA should be conducted to confirm these promising outcomes.

Results and comparison between the first 15 procedures (5 per lesion) and the last 15 for each operator and overall for all operators.

Endoscopist	Procedures	Positioning time, min	Total time, min	Number of attempts	Number times when EUS view of needle lost	Number of times assistant had to hold scope to maintain a stable position	Skills rating (n/10)
1	First 15	2.5 ± 2.2	11.2 ± 7.8	2.6 ± 1.2	2.3 ± 2	1.1 ± 1.7	5 ± 1.9
	Last 15	1.1 ± 0.8	4.3 ± 2.2	1.6 ± 0.7	0.5 ± 0.7	0.1 ± 0.2	7.7 ± 1
	P	0.034	< 0.001	< 0.001	< 0.001	0.001	< 0.001
2	First 15	3.7 ± 2.9	16.3 ± 8.7	2.6 ± 1.5	1.5 ± 1.8	2.9 ± 2	4.4 ± 1.8
	Last 15	1.5 ± 0.8	5.9 ± 2.2	1.9 ± 0.7	0.5 ± 0.9	0.1 ± 0.3	7.2 ± 1
	P	0.013	< 0.001	0.075	0.042	< 0.001	< 0.001
3	First 15	2.3 ± 0.9	7.9 ± 2.7	2.8 ± 1.1	1.9 ± 1.6	0.2 ± 0.4	5.5 ± 2.1
	Last 15	1.3 ± 0.8	3.6 ± 1.9	1.5 ± 0.6	0.6 ± 0.7	0 ± 0	8 ± 1.3
	P	0.03	< 0.001	< 0.001	< 0.001	NA	< 0.001
4	First 15	3.1 ± 1.3	10.7 ± 5.6	2.5 ± 1.4	3.3 ± 2.7	0.5 ± 0.5	4.5 ± 2
	Last 15	1.7 ± 0.9	4.7 ± 1.9	1.5 ± 0.6	0.6 ± 0.6	0 ± 0	7.7 ± 1
	P	0.02	0.001	0.002	< 0.001	NA	< 0.001
5	First 15	1.5 ± 0.5	4.9 ± 1.9	2 ± 0.8	2.3 ± 1.9	0 ± 0	6.1 ± 1.5
	Last 15	1 ± 0.3	3.1 ± 0.9	1.4 ± 0.5	0.5 ± 0.8	0 ± 0	7.8 ± 0.9
	P	0.002	0.003	0.006	0.001	NA	< 0.001
Overall	First 15	2.6 ± 1.9	11.2 ± 7.8	2.6 ± 1.2	2.3 ± 2	1.1 ± 1.7	5 ± 1.9
	Last 15	1.3 ± 0.8	4.3 ± 2.2	1.6 ± 0.7	0.5 ± 0.7	0.1 ± 0.2	7.7 ± 1
	P	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001	< 0.001