When trainees reach competency in performing endoscopic ultrasound: a systematic review.

Background/Study aim The American Society for Gastrointestinal Endoscopy (ASGE) recommends that trainees complete 150 endoscopic ultrasound (EUS) procedures before assessing competency. However, this recommendation is largely based on limited evidence and expert opinion. With new evidence suggesting that this historical threshold is underestimating training requirements, we evaluated the learning curve for achieving competency in EUS. Patients/Materials and methods Two investigators independently searched MEDLINE for full-text citations assessing the learning curve for achieving competency in EUS in the period 1946 to 25 March 2016. A learning curve was defined as either a tabulated or graphic representation of competency as a function of increasing EUS experience. Results Eight studies assessing 28 trainees and 7051 EUS procedures were included. When stratifying studies based on procedural indication: three studies assessed competency in evaluating mucosal lesions, three studies assessed competency in EUS fine-needle aspiration (EUS-FNA), and two studies assessed comprehensive competency. Among studies assessing mucosal lesion T-staging accuracy, competency was achieved by 65 to 231 procedures. Among studies assessing EUS-FNA, competency was achieved by 30 to 40 procedures. Among the two studies assessing comprehensive competency in EUS, competency was not achieved in either study across all trainees. Only four of 17 trainees reached competency by 225 to 295 EUS procedures. Conclusion As EUS competency assessment has evolved to more closely reflect independent clinical practice, the number of procedures required to achieve competency has risen well above ASGE recommendations. Advanced endoscopy training programs and specialty societies need to re-assess the structure of EUS training.

Introduction

Endoscopic ultrasound (EUS) has emerged as a core diagnostic modality for evaluation and tissue acquisition in gastrointestinal disease 1. With this comes a critical need for a better understanding of the learning curve to achieve competency in EUS, highlighted by the increasing emphasis on competency-based medical education 2 3. Unfortunately, due to the current lack of clarity for competency-based assessment in EUS, training programs are still dependent on procedural thresholds to help ensure their trainees achieve competency. This is further complicated by societies 4 5 having to rely on limited data and expert opinion to estimate procedural thresholds with the American Society for Gastrointestinal Endoscopy (ASGE) recommending 150 supervised EUS procedures before competency should be assessed, 75 of which must evaluate the pancreatobiliary system and 50 must incorporate EUS fine-needle aspiration (EUS-FNA) 4.

Recently, systematic reviews for both colonoscopy 6 and endoscopic retrograde cholangiopancreatography (ERCP) 7 have directly questioned their procedural thresholds. Moreover, with new evidence 8 9 suggesting that the current recommendations for EUS also underestimate the volume of procedures required to achieve competency, we sought to systematically evaluate the learning curve for EUS. The goal of this review is to delineate appropriate procedural thresholds thus enabling policymakers in the creation of uniform and objective EUS training guidelines.

Patients/Materials and methods

This systematic review was completed in accordance with the meta-analysis of observational studies in epidemiology (MOOSE) recommendations 10.

Search strategy

For the period 1946 to 25 March 2016, two authors (NS and GO) independently searched MEDLINE using the following search strategy: (“endosonography [MeSH]” OR “endosonography” OR “endoscopic ultrasound” OR “EUS”) AND (“trainee” OR “training” OR “resident” OR “fellow*” OR “competenc*” OR “learning curve”). Subsequently, to evaluate the gray literature, the same authors independently searched the bibliographies of included studies, relevant guidelines, and pertinent review articles for further potential citations.

Study selection

Full-text citations, published in the English language and which assessed the learning curve in performing EUS, were considered for inclusion. A learning curve was defined as either a tabulated or graphic representation of competency as a function of increasing EUS experience. A minimum of two data points along the learning curve were required. Given the diversity of indications for EUS, quantifying trainee success rate for a specific component of the procedure such as cecal intubation rate for colonoscopy 6 or cannulation rate for ERCP 7 was not mandated for study inclusion.

Citations were subsequently independently assessed for potential exclusion under the following criteria: 1) not published in English; 2) only published as an abstract; 3) review articles, letters, commentaries, editorials or book chapters; 4) lacked trainee-specific or EUS-specific outcomes; 5) assessed non-gastroenterology or non-surgery-based trainees; 6) did not quantify a learning curve as defined above or trainee position along the learning curve was not clear; 7) only the initial aspect of the learning curve was assessed, which was set at < 25 EUS procedures; 8) solely used simulation models to quantify the learning curve; 9) focused on the assessment of trans-rectal ultrasound (TRUS), endo-rectal ultrasound (ERUS) or colorectal EUS; 10) focused on the assessment of advanced diagnostic modalities (e. g. elastography, endomicroscopy), interventional EUS or cytopathology training. For the purpose of this review, citations in which participating endosonographers who had completed EUS training but were learning EUS-FNA were deemed acceptable for study inclusion. Authors were contacted if further information was believed to potentially affect study inclusion.

Data extraction

Citations selected for study inclusion subsequently underwent independent data extraction by two authors (NS and GO), with variables including: 1) year of publication; 2) country of origin; 3) trainee description including previous endoscopic experience; 4) structure of advanced endoscopy training; 5) EUS case description; 6) trainee’s involvement during EUS; 7) EUS competency outcomes. Data was finalized through consensus between the two authors with disagreement at any stage of the systematic review process being resolved by involving a third study author (JT).

Outcomes and analysis

Our outcome of interest was the number of procedures required to achieve competency in EUS. Estimates were extracted as they were expressed within their respective citation, using their respective marker of competency. When a citation assessed multiple definitions of competency or provided multiple estimates for when competency was achieved, for the purpose of this review, what we perceived to be the primary definition/estimate was used to define when competency was achieved. Citations were subsequently stratified based on site/lesion as per the ASGE credentialing guidelines 4.

Descriptive statistics were used to present study findings as meta-analysis was thought to be inappropriate at this time due to notable heterogeneity.

Results

Search results and study description

In total, 385 citations were identified by electronic and grey-literature searches, of which 42 underwent full-text review (Fig. 1). Ultimately, eight studies 8 9 11 12 13 14 15 16 assessing 28 trainees and 7051 EUS procedures were included (Table 1), and 34 full-text citations were excluded as described in Fig. 1. Three studies 8 9 12 were prospective in study design with seven studies assessing gastroenterology-based trainees 8 9 11 12 13 14 16 and one study 15 assessing a surgery-based trainee. Upon stratifying included studies, three 11 15 16 assessed competency in mucosal lesion evaluation, three 12 13 14 assessed competency in EUS-FNA, and two 8 9 assessed comprehensive competency (Table 2).

Fig. 1

 Study selection. EUS, endoscopic ultrasound; ERUS, endo-rectal ultrasound; GI, gastroenterology; TRUS, trans-rectal ultrasound.

Description of included studies.

Study	Year	Country	Study design	Trainee background	# Trainees	# EUS
Park et al. 11	2015	Republic of Korea	Retrospective	Gastroenterology	4	553
Wani et al. 9	2015	USA	Prospective	Gastroenterology	12	4257
Wani et al. 8	2013	USA	Prospective	Gastroenterology	5	1412
Cote et al. 12	2011	USA	Prospective	Gastroenterology	3	305
Nayar et al. 13	2011	UK	Retrospective	Gastroenterology	1	110
Mertz and Gautam 14	2004	USA	Retrospective	Gastroenterology	1	71
Schlick et al. 15	1999	Germany	Retrospective	Surgery	1	112
Fockens et al. 16	1996	The Netherlands	Retrospective	Gastroenterology	1	231

Competency outcomes of included studies.

Study	Marker of competency	Outcomes
Mucosal competency assessment
Park et al. 11	Gastric cancer T-staging accuracy	Competency achieved @ 65 EUS examinations
Schlick et al. 15	Esophageal or gastric cardia cancer T-staging accuracy	Competency achieved @ 75 to 105 EUS examinations
Fockens et al. 16	Esophageal cancer T-staging accuracy	Competency achieved @ 101 to 231 EUS examinations
EUS-FNA competency assessment
Cote et al. 12	Adequacy of EUS-FNA specimens	Competency achievement not specified. The proportion of adequate specimens did not change along the learning curve
Nayar et al. 13	Pancreatic EUS-FNA diagnostic accuracy	Competency achieved @ 30 EUS examinations
Mertz and Gautam 14	Pancreatic EUS-FNA sensitivity	Competency achieved @ 30 to 40 EUS examinations
Comprehensive competency assessment
Wani et al. 9	Median score of “no assistance” across assessed items on EUS evaluation tool	Competency not achieved across all trainees; 2 /12 trainees achieved competency @ 225 to 245 EUS examinations
Wani et al. 8	Median score of “no assistance” across assessed items on EUS evaluation tool	Competency not achieved across all trainees; 2 /5 trainees achieved competency @ 255 to 295 EUS examinations

EUS, endoscopic ultrasound; EUS-FNA, endoscopic ultrasound fine-needle aspiration.

Competency assessment: mucosal lesions

Three studies 11 15 16, including six trainees performing 896 EUS procedures, assessed competency in mucosal lesion evaluation. All three studies focused on T-staging accuracy of esophageal and/or gastric cancer. Only one study 11 clearly defined competency, based on staff T-staging accuracy, whereas the other two studies 15 16 defined a threshold number of required procedures based on a statistically significant difference in trainee performance along the learning curve. It was deemed that competency was achieved across all trainees over a range of 65 to 231 EUS procedures.

Competency assessment: EUS-FNA

Three studies 12 13 14, including five trainees and 486 EUS procedures, evaluated competency in performing EUS-FNA. Two studies 13 14 focused on pancreatic EUS-FNA. Competency was clearly defined in one study 13 as a diagnostic accuracy of > 70 %. Of the two remaining studies, one study 14 defined a threshold number of required procedures based on a statistically significant difference in trainee performance along the learning curve. The focus of the other study 12 was to assess the feasibility of concurrent EUS-FNA training alongside EUS training; competency was not clearly defined. Of the two studies 13 14 in which competency was deemed to have been achieved, this occurred over a range of 30 to 40 EUS-FNA procedures.

Competency assessment: comprehensive competency

Two studies 8 9, including 17 trainees and 5669 EUS procedures, evaluated comprehensive competency. For both studies, a successful procedure was defined by a median score of 1 (no assistance) on a standardized EUS assessment tool; this included when applicable: 1) identifying the lesion of interest; 2) TNM staging; 3) sub-epithelial wall-layer characterization; 4) EUS-FNA. Using cumulative sum (CUSUM) analysis, acceptable and unacceptable failure rates of 10 % and 20 %, respectively, were used to define competency. Competency was not achieved in either study across all trainees; only four of 17 trainees reached competency between 225 and 295 EUS procedures.

Discussion

As EUS has become an integral tool in the armamentarium of the advanced endoscopist, increasing emphasis has been placed on refining EUS training. Naturally, similar to the maturation of competency assessment for colonoscopy 6 and ERCP 7, questions have been raised concerning the adequacy of current recommendations 4 with regard to the number of EUS procedures required before assessing competency. Furthermore, recent data indicate that, even with the current recommendations in mind, training programs are providing insufficient EUS procedural volumes 17. After these findings, studies began to emerge re-evaluating procedural thresholds for achieving competency in EUS 8 9. Alongside this, a systematic review has been published evaluating training and competency assessment in endoscopy, including EUS 18. However, this review included only two studies 8 19 assessing learning curves and procedural thresholds in EUS and, given the emergence of a landmark study in EUS competency 9, we felt that a systematic review focusing on EUS competency assessment was warranted. With the above in mind, our systematic review highlights that, as competency assessment in EUS has evolved and has begun to assess comprehensive competency, the number of procedures required to achieve competency in EUS remains unclear but has clearly risen above current ASGE recommendations. Given these findings and in the wake of competency-based medical education, advanced endoscopy training programs and specialty societies need to re-evaluate the current structure of EUS training.

Historically, colonoscopy 20 21, ERCP 22 23, and EUS have shared a similar evolution with initial recommendations for the number of procedures trainees should perform before assessing competency being largely based on limited data and expert opinion. Although this appears to be the nature of procedural development, with studies now evaluating comprehensive competency, guidelines must adapt and reflect the new era of EUS competency assessment. Traditionally, while the scope of EUS practice may have been more focused on a specific indication for EUS, we believe current trainees must at least be able to perform mucosal tumor evaluation, submucosal abnormality evaluation, pancreatobiliary evaluation, and EUS-FNA. Therefore, the concept of independent comprehensive competency should be the new standard for defining competency in EUS. Until competency-based assessment in EUS can be delineated, we agree with Wani et al. 9 that at least 225 EUS procedures should be performed before assessing competency. It is important to note though that this was the earliest that a trainee achieved “comprehensive competency” and that only four of 17 trainees achieved competency by 225 to 295 EUS procedures with an average of approximately 330 EUS procedures per trainee. Therefore, with the threshold for comprehensive competency remaining unclear, advanced endoscopy training programs should expect that most trainees will not reach competency at either 225 EUS procedures or 330 EUS procedures and need to modify their programs accordingly.

Although the evolution of the definition of competency in EUS may mirror that of colonoscopy and ERCP, the assessment of EUS competency is much more difficult. Both colonoscopy and ERCP have well-established quality indicators such as cecal intubation rate 24 and native papilla deep cannulation 25, which have been used to assess trainee competency. Unfortunately, EUS has no such quality indicator 26. This is in part due to the variety of indications for EUS, thus making it difficult to identify a universal EUS procedural component across all EUS indications; and while EUS-FNA sample adequacy rates, EUS-FNA sensitivity, and adverse events incidence rates could potentially be used as markers of competency, limiting factors have prevented their adoption 27. Concerning EUS-FNA sample adequacy rates and EUS-FNA sensitivity, as described above, these only pertain to EUS-FNA. Moreover, none of these can be readily quantified during or immediately after the procedure which limits their utility. Given the above, we commend Wani and colleagues 8 9 for the creation of an objective standardized assessment tool in EUS. Through the incorporation of cognitive evaluation, therapeutic EUS, as well as delineating a universal approach to station/procedural component scoring, it is our belief that this tool carries with it the potential to emerge as a means for objectively quantifying comprehensive competency.

Our study has a number of limitations. Most notably, due to multiple sources of heterogeneity including study methodology, the definition of competency in EUS, trainees’ previous endoscopic experience including duodenoscope experience, EUS training environment, and the indication for EUS, we felt that meta-analysis was not appropriate at this time. Moreover, quality assessment of included studies was not performed, to remain consistent with our previous reviews on competency assessment in colonoscopy 6 and ERCP 7. Our exclusion criteria were relatively restrictive, which potentially led to publication bias due to the exclusion of non-English publications, colorectal EUS, and interventional EUS. Alongside this, given our focus on gastroenterology and surgery trainees, we excluded studies evaluating competency in EUS for lung cancer amongst pulmonary trainees 28 29. Of note, recent recommendations from the European Society of Gastrointestinal Endoscopy (ESGE), in conjunction with the European Respiratory Society (ERS) and the European Society of Thoracic Surgeons (ESTS) have been published addressing this specific indication 30. Lastly, we included studies assessing the initial methods for training in EUS 14 15 16 which no longer reflect the current structure of EUS training.

In conclusion, our systematic review highlights that as EUS competency assessment has evolved to more closely reflect independent clinical practice, the number of EUS procedures required to achieve competency remains unclear but has clearly risen above current ASGE recommendations. Further research is required to establish standardized definitions and means of assessing comprehensive competency and all EUS indications, thus facilitating individualized competency assessment dependent on the trainee’s future breadth of practice. Moreover, as EUS training continues to incorporate therapeutic EUS, the definition of comprehensive competency will need to evolve alongside. Moving forward, advanced endoscopy training programs must embrace more conservative procedural thresholds for achieving competency in EUS and provide adequate procedural volumes to their trainees. However, it is critical to understand that trainees achieve competency over a range of EUS procedures. Therefore, each trainee requires individualized assessment, to ensure competency is achieved before entering independent clinical practice.