Assessment of Benefit of Advanced Inflammatory Bowel Disease Training: Challenges and Solutions.

Background: Advanced inflammatory bowel disease (IBD) fellowships are available for gastroenterologists who wish to increase their expertise in complex IBD. However, little is known about the outcomes of such training. The aims of this study were to assess clinical and academic outcomes following advanced training in IBD.
Methods: We surveyed gastroenterologists who completed advanced IBD fellowships and compared competency and outcomes to gastroenterologists focusing in IBD who completed gastroenterology training alone. Participants completed a survey via REDCap. Continuous variables were compared using the Wilcoxon rank-sum test. Categorical variables were compared using chi-square or Fisher's exact tests.
Results: A total of 104 physicians participated in the study. IBD fellowships were completed by 31 physicians (30%), of whom 29 (94%) felt their training was excellent. Management of complicated IBD (84%), research mentoring (74%), and career mentoring (71%) were felt to contribute most highly to professional development. Compared to non-advanced trained physicians, advanced trained physicians expressed higher levels of comfort with management of IBD during pregnancy (P = 0.003), complicated IBD (P = 0.057), and peri-operative IBD (P = 0.057). No significant advantage was detected in academic productivity. Common barriers to participation in IBD fellowships included feeling it was unnecessary (45%) and desire to begin a faculty position (42%). Conclusions: This study suggests there may be clinical benefit to advanced IBD training. Importantly, this study identified that there are also unique challenges to the assessment of clinical competency in IBD training. Efforts by the IBD community to establish a registry of advanced trainees and improve competency assessments are needed.

INTRODUCTION

The management of patients with inflammatory bowel disease (IBD) has become increasingly complex. In the past 2 decades, there have been a number of new therapies, biologic and non-biologic, approved for the medical management of IBD as well as continued exploration of new therapeutic mechanisms of action. Therapeutic paradigms have also evolved, advocating early use of effective therapy, consideration of using 2 or more agents together, “treat-to-target” algorithms, therapeutic drug monitoring, and risk stratification. In addition to these advances in medical therapeutics, procedural management has also continued to evolve allowing for enhanced colorectal cancer surveillance with chromoendoscopy and endoscopic treatment of stricturing Crohn’s disease. These advancements have improved medical care for patients with IBD. However, the ability to integrate this expanding wealth of information into a traditional 3-year gastroenterology fellowship training is constrained by the requirements already set forth by the Accreditation Council for Graduate Medical Education (ACGME). Trainees have expressed low levels of satisfaction with their IBD exposure as well as lack of comfort with management of complex IBD. Various solutions have been proposed to address this, including fourth year advanced training fellowships.

Advanced fellowship training programs provide physicians the opportunity for additional clinical training and mentorship in a subspecialty field. Examples of such fellowship training options in gastroenterology include interventional/advanced endoscopy, transplant hepatology, neuro-gastroenterology and motility, and IBD. Advanced IBD fellowships have been developed at several centers across the United States and Canada. While curriculum varies by program, the advanced IBD fellowship year focuses on outpatient, inpatient, and endoscopic management of complex IBD. In addition to furthering knowledge alone, such advanced training options at specialized centers may offer exposure to leading experts in the area, offering opportunities for mentoring and advancement in research, as well as networking with peers and mentors. Despite such fellowships having been available for over 10 years, there have been no prior studies attempting to quantify the clinical and academic benefit of advanced training in IBD.

In this article, we aim to achieve 2 objectives. First, we performed a survey assessing competency and academic performance of IBD physicians who have or have not completed an IBD fellowship, to assess incremental value of specialized training. We report the results of this survey. Secondly and importantly, we then address the challenges in quantifying any benefit (or lack thereof) of advanced training and suggest solutions to ensure systematic and robust assessment of the value of such advanced training.

METHODS

Survey Design and Cohort

The study design included a national, cross-sectional assessment of IBD physician outcomes, which was captured via an electronic survey. The population of interest included gastroenterologists who had completed an advanced IBD fellowship. The control population included gastroenterologists practicing with a clinical interest in IBD who had completed a general gastroenterology fellowship but had not completed an advanced IBD fellowship. This study was approved by the Partners institutional review board.

Recruitment and Participation

Advanced IBD graduates were identified either by their advanced IBD programs or by self-report on the completed survey. Twenty advanced IBD programs were identified through the American College of Gastroenterology database, the Crohn’s and Colitis Foundation advanced fellowship registry, and/or individual program websites. All programs were contacted to identify graduates who had completed a year of advanced fellowship at their sites at any point in time. Emails for graduates were obtained from either advanced programs, current division or hospital websites, published chapters, published journal articles, or through personal communication. During the survey, participants were asked “Have you completed an advanced year of training in Inflammatory Bowel Disease (IBD)?.” Those participants who responded “yes” were included in the advanced IBD training group.

The control group was recruited through 2 mechanisms. First, we identified gastroenterologists with an “interest” or “self-reported expertise” in IBD who had not completed a year of advanced training. One hundred thirty-seven ACGME approved gastroenterology programs with at least 10 years of accreditation were identified through the ACGME website or published program lists. The associated program, division, and hospital websites were then reviewed to identify physicians with a specialized interest in IBD or leading the clinical IBD programs at those centers. Emails were obtained through current division or hospital websites, published chapters, published journal articles, or through personal communication. While being biased towards those with greater self-reported expertise or comfort in IBD, we adopted this process to minimize comparing advanced IBD trained graduates with general gastroenterologists who are not routinely involved in the care of patients with IBD. Both advanced trainee graduates and controls were contacted with up to 3 emails to encourage response. Second, professional members of the Crohn’s and Colitis Foundation were approached for participation by mailing a link for the study through the foundation listserv as well as posting on the foundation website. The recruitment email contained information on the goals of the study and a link to the survey for those interested in participating. Information on informed consent was included at the start of the survey and completion of the survey indicated willingness to participate. The study period was from October 2017 to March 2018.

Survey and Data Collection

A 36-item survey assessing demographic information, IBD exposure during general gastroenterology training, barriers to advanced training among those interested in IBD, current components of advanced IBD training, level of clinical comfort with various IBD patient scenarios, ideal curriculum components, academic outcomes, and academic productivity was constructed by the research team (Appendix 1). As part of the survey, advanced trained physicians were also asked to reflect on components of training in their advanced year that were most important to their professional development. The survey was piloted among current trainees or recent graduates prior to implementation. In completing the survey, participants could choose to skip any questions they did not feel comfortable answering. Level of clinical comfort was assessed on a Likert scale with the following responses: “very comfortable,” “comfortable,” “uncomfortable” and “very uncomfortable.” When participants were asked “If you did not pursue an advanced fellowship, what was the reason(s)?,” they could select more than one reason for non-participation in advanced IBD fellowships as well as free text additional reasons.

Data was collected and managed using REDCap (Research Electronic Data Capture) hosted by Partners HealthCare Research Computing, Enterprise Research Infrastructure & Services (ERIS) group.

Statistical Analysis

Continuous descriptive data was presented as medians with interquartile ranges. Categorical descriptive data was reported as absolute numbers and percentages. Comparisons were performed by the Wilcoxon rank-sum test for continuous variables and chi-square or Fisher’s exact test, as appropriate, for categorical variables. For the purposes of statistical analysis, “very comfortable” was compared to all other levels of comfort. A sensitivity analysis of clinical and academic outcomes was performed using only those physicians who had been practicing for less than or equal to 10 years to ensure comparability in practice level between the groups. A P value of less than 0.05 indicated statistical significance. Missing data were excluded from the analysis. Analyses were performed using Stata IC 15.1 (Stata Corp, College Station, Texas).

RESULTS

Survey Cohort

The study population consisted of 106 survey respondents. Two individuals were excluded as they self-identified as non-gastroenterologists (colorectal surgeon, nurse practitioner). The mean age of the participants was 44 years (range: 30–91) and males comprised 65% of the group. Most physicians worked in an academic setting (90%) with a minority working in private practice (10%). The level of exposure to IBD in general GI training was considered excellent (53%), very good (21%), or good (20%) by most participants. Advanced IBD fellowships were completed by 31 physicians (30%) (Table 1). Advanced trained physicians were similar in gender distribution (P = 0.38) but were younger (37 vs 42 years, P < 0.001) and had significantly fewer years of independent practice (P < 0.001) than non-advanced trained physicians. Most graduates of advanced training in IBD were within their first 5 years of practice (74%). Furthermore, actual time spent exposed to inpatient IBD during general fellowship was lower (P = 0.01) among the advanced trained group. Additional survey data can be found in Supplementary Table 1.

TABLE 1.

Baseline Characteristics and General Gastroenterology Training Experience by Completion of Advanced IBD Fellowship Training

	Advanced trained (N = 31) N [%]	Non-advanced trained (N = 73) N [%]	P-value
Age	37 ± 7	42 ± 14	<0.001
Gender			0.38
Male	22 (71%)	44 (62%)
Female	9 (29%)	27 (38%)
Years in independent practice			<0.001*
First year	11 (35%)	2 (3%)
2 to 5 y	12 (39%)	19 (26%)
6 to 10 y	7 (23%)	24 (33%)
10 to 15 y	0	5 (7%)
Over 15 y	1 (3%)	23 (32%)
Exposure to IBD during general training			0.09*
Excellent	11 (35%)	44 (61%)
Very Good	9 (29%)	13 (18%)
Good	9 (29%)	12 (17%)
Poor	2 (6%)	3 (4%)
Time exposed to outpatient IBD (general fellowship)			0.15
<10%	7 (23%)	14 (20%)
10% to 30%	16 (53%)	26 (37%)
>30%	7 (23%)	31 (44%)
Time exposed to inpatient IBD (general fellowship)			0.01
<10%	9 (29%)	17 (24%)
10% to 30%	21 (68%)	33 (46%)
>30%	1 (3%)	21 (30%)
Time exposed to IBD research (general fellowship)			0.55
<10%	11 (35%)	21 (29%)
10% to 30%	11 (35%)	22 (31%)
>30%	9 (29%)	29 (40%)
Number of IBD faculty on staff (general fellowship)			0.74*
0 to 3	19 (61%)	41 (58%)
4 or greater	12 (39%)	30 (42%)
Crohn’s and Colitis Foundation Visiting Fellow Program			0.16*
Yes	5 (16%)	5 (7%)
No	26 (84%)	68 (93%)

*Fisher’s exact test.

Clinical Comfort With IBD Management

Most advanced trained gastroenterologists felt very comfortable with management of mild to moderate IBD (94%), complicated/severe IBD (94%), IBD during pregnancy (94%), peri-operative management of IBD (94%) and therapeutic drug monitoring (94%). Many advanced trained physicians felt very comfortable with preventative care (83%). However, fewer felt comfortable with management of ileal-pouch related disease (71%), chromoendoscopy (61%), and dilation of IBD related strictures (58%) and few reported that they were very comfortable managing short gut and nutritional deficiencies (29%).

Comparative to non-advanced trained physicians, advanced trained physicians felt more comfortable managing IBD during pregnancy (94% vs 66%, P = 0.003) with a trend to significance in complicated/severe IBD (94% vs 78%, P = 0.057) and peri-operative IBD (94% vs 78%, P = 0.057). A sensitivity analysis was performed using only those physicians who had been practicing for less than 10 years to ensure comparable periods of post-graduate experience between the 2 groups. In this subset, there was again a higher level of perceived comfort with IBD during pregnancy (93% vs 67%, P = 0.007) (Fig. 1). In addition, advanced trained physicians were more comfortable managing pouch related disease (70% vs 44%, P = 0.029), complicated/severe IBD (93% vs 76%, P = 0.046), and peri-operative IBD (93% vs 76%, P = 0.046).

FIGURE 1.

Percentage of physicians reporting the highest level of comfort with IBD scenarios by completion of advanced fellowship training (sensitivity analysis ≤ 10 years of independent practice). CD indicates Crohn’s disease. UC indicates ulcerative colitis. An * indicates a P value of less than or equal to 0.05.

Academic Outcomes

Of those who completed advanced IBD fellowship training, most respondents were within their first 5 years of independent practice (74%) with just under half in the first year (35%). A majority went on to work in an academic setting (27/31, 87%) and approximately half had obtained grant funding (15/31, 48%). In the past 3 years, more than half (55%) had published 6 or more manuscripts and more than two-thirds (68%) had written IBD chapters. Compared to non-advanced trained physicians, advanced trained physicians did not demonstrate superiority in grant funding, publications, and/or leadership responsibilities. In a sensitivity analysis of those with less than 10 years of independent practice, we found there was again no superiority in academic outcomes among those who completed an advanced year of fellowship though non-advanced trained group had greater numbers of years in independent practice (Table 2).

TABLE 2.

Academic Outcomes by Completion of Advanced History of Fellowship Training (Sensitivity Analysis ≦ 10 Years of Independent Practice)

	Advanced trained (N = 30) N [%]	Non-advanced trained (N = 45) N [%]	P-value
Years in independent practice			0.001
First year	11 (37%)	2 (4%)
2 to 5 y	12 (40%)	19 (42%)
6 to 10 y	7 (23%)	24 (53%)
Work setting			0.15*
Academic	26 (87%)	44 (98%)
Private	4 (13%)	1 (2%)
Grant funding	14 (47%)	27 (61%)	0.21
Intramural grant funding	6 (20%)	19 (44%)	0.03
Extramural grant funding	11 (37%)	17 (40%)	0.80
Publications in the last 3 y			0.75*
0	0	2 (5%)
1 to 5	14 (47%)	19 (43%)
6 to 10	8 (27%)	14 (32%)
11 to 15	3 (10%)	5 (11%)
16 or more	5 (17%)	4 (9%)
IBD chapter(s)	20 (67%)	26 (59%)	0.51
Peer reviewer	23 (77%)	35 (80%)	0.77
Editorial board	5 (17%)	6 (14%)	0.75*
Editor	3 (10%)	3 (7%)	0.69*
IBD director	8 (27%)	16 (36%)	0.38
Program director	1 (3%)	14 (33%)	0.002
Committee membership	19 (63%)	32 (73%)	0.39
Leadership (chair/vice chair)	7 (23%)	11 (26%)	0.83

*Fisher’s exact test.

Reasons for not Participating in Advanced IBD Fellowship Training

Of 71 physicians who did not complete an advanced training year, 24 (33.8%) considered advanced training in IBD. There were a variety of reasons that physicians decided not to pursue additional training (Fig. 2). The most common reasons included the feeling that additional training was not necessary (32/71, 45%), the desire to begin a faculty position (30/71, 42%), and financial reasons (16/71, 23%). A small group of physicians attributed not pursuing advanced training due to a lack of time (9/71, 13%) and very few (2/71, 3%) were not planning on sub-specializing in IBD. Twenty cited other reasons, including significant exposure during general training, geographical limitations, family reasons, and lack of opportunities for advanced IBD training at the time they graduated. Of 47 participants who did not consider completing the advanced fellowship, 8 (17%) were unaware of the advanced training fellowship option.

FIGURE 2.

Reasons for not pursuing an advanced year of training in IBD as reported by survey participants (sensitivity analysis ≤ 10 years of independent practice).

When isolating the analysis to only those physicians who had been in independent practice ≤ 5 years, more had considered advanced training in IBD (12/21, 57.1%) but the reasons for not pursuing advanced training were similar: the desire to begin a faculty position (10/21, 47.6%), the feeling that additional training was not necessary (9/21, 42.9%), financial reasons (6/21, 28.6%), lack of time (5/21, 23.8%), and not planning on sub-specializing in IBD (1/21, 5%). Only 1 physician was not aware of the advanced training option.

Education and Professional Development

Of the 31 participants who completed advanced training, all felt their training was excellent (29/31, 94%) or very good (2/31, 6%). During advanced training, exposure to outpatient IBD was common with 77% of advanced trained physicians reporting that more than half of their time was spent in the outpatient setting. Exposure to inpatient IBD was less common with most (68%) spending less than a third of their time in this area (Supplemental Fig. 1). Areas which were listed as the most important to graduates’ professional development included management of complicated IBD and IBD in special situations (84%), multidisciplinary case conferences and discussions (61%), and critical interpretation of the literature (58%). In addition to clinical training, most graduates reported research mentoring (74%) and career mentoring (71%) as being very important to their professional development (Fig. 3).

FIGURE 3.

Components of advanced training which were felt to be very important to graduates’ professional development

All 106 IBD physicians surveyed were asked their opinion on ideal curriculum components for advanced fellowship. A majority felt both outpatient and inpatient IBD were essential components of an advanced IBD curriculum (Fig. 4). Specifically, 103 (99%) felt outpatient IBD was essential and 94 (90%) felt inpatient IBD was essential. Fewer felt radiology teaching (59%), pathology teaching (50%), observing IBD surgeries in the OR (38%), formal training in research methods (33%), and rotating in pediatric IBD clinics (32%) were essential components of an advanced IBD curriculum.

FIGURE 4.

Percentage of providers who felt each educational component was an essential part of an advanced IBD fellowship curriculum.

DISCUSSION

There is a paucity of scientific literature that has systematically assessed the impact of advanced training in IBD on clinical proficiency and academic productivity. These outcomes are very relevant for trainees who have an interest in becoming IBD specialists as well as program directors who wish to guide trainees in their career development. Through a survey of 106 physicians, we found that those who completed an advanced IBD training reported greater comfort with managing IBD in pregnancy, peri-operatively and for complicated disease. However, there was no discernible difference in academic performance or career trajectory. In addition to these findings, our study highlights the unique challenges encountered in the objective assessment of advanced subspecialty training. These challenges merit further discussion in the IBD community and are highlighted below (Table 3).

TABLE 3.

Challenges in Quantifying Benefit of Advanced Training and Possible Solutions

Challenge	Possible solutions
Identification of advanced IBD trainees	Development of a prospective registry of advanced IBD graduates, which would facilitate prospective monitoring and future assessments of benefit of advanced training.
Variability in training during IBD fellowship	Develop a specific advanced IBD training curriculum that lays out minimum suggested recommendations for clinical exposure (including to surgery, pathology, and radiologic evaluation) and endoscopic training in complex procedures.
Defining “comparator populations”	Structured pre- and post-advanced fellowship competency assessment, including case-based scenarios, and development of a registry of providers designated as IBD specialists in their respective centers.
Assessment of competency	Development of case-based competency assessment tools covering various spheres of inpatient and ambulatory IBD care.
Assessment of mentorship	Develop direct and indirect metrics for assessment of mentorship quality including through trainee-completed surveys and face-to-face time spent in meetings with mentor.

Challenges in Assessing the Impact of Advanced Training

Identification of Advanced IBD trainees

Delineation of advanced trainees in IBD is challenging. Completion of an advanced training year is the most discrete and easily accessible measurement but has important limitations. For example, many trainees can utilize their second and third years of general gastroenterology training to spend more focused time in IBD clinics, IBD dedicated endoscopy sessions, and under the clinical and academic mentorship of IBD specialized gastroenterologists. Such trainees would not be captured as “advanced trained” but likely have accumulated a similar clinical acumen to many advanced trainees who did not complete a similar level of IBD training during their general gastroenterology fellowship. Second, while there are comprehensive listings of formal IBD fellowship programs, positions at other institutions may vary in their availability year to year based on funding availability and may not be captured in existing databases. Third, such databases exist primarily for programs based in the United States and would not capture trainees who obtain advanced IBD training in other countries. At this time, there is no national registry for advanced IBD trained physicians making identification of fellowship participants arduous, time-consuming, not comprehensive, and likely subject to bias.

Recommendation.

We propose the development of a prospective registry of advanced IBD graduates, which would facilitate prospective monitoring and future assessments of benefit of advanced training.

Variability in training during IBD fellowship

As with many training programs, there is no formalized curriculum for an IBD fellowship. There is considerable variation between institutions in number and expertise of faculty, proportion of ambulatory compared to inpatient training, and training in specialized endoscopic procedures such as chromoendoscopy and balloon dilation. While it may not be feasible to mandate specific numbers, it would be helpful to develop consensus suggestions for minimum requirements, which can set expectations for both the trainee and the program. There is also a need to develop a standardized curriculum that would provide a sound knowledge base contemporary ambulatory and inpatient IBD care.

Develop a specific advanced IBD training curriculum that lays out minimum suggested recommendations for clinical exposure (including surgery, pathology, and radiologic evaluation) and endoscopic training in complex procedures.

Defining “comparator populations”

To quantify the benefit of advanced training, fellows either need to be surveyed prior to and following the completion of IBD fellowship, or compared to peers who did not complete IBD training. Both of these present challenges and lend themselves to some solutions. There is no systematic assessment of IBD competency prior to fellowship. While most trainees complete their gastroenterology boards during their fellowship, which includes testing on IBD, this is often several months into their advanced fellowship and is not an accurate reflection of pre-training knowledge. Development of structured pre- and post-fellowship tests may help assessing comprehensiveness and value of training. Identifying peer groups is challenging, as it would be inherently less meaningful to compare IBD trainees with general gastroenterology graduates who manage IBD infrequently. Any changes thus identified may reflect volume of practice rather than impact of training. One comparison group, as we defined, could be providers who self-designate specialization in IBD at academic institutions or practice settings but have not completed IBD training. However, there is no listing of such providers and attempts at identifying this group would be biased towards academic institutions and individuals with published contact information through scientific publications.

Structured pre- and post-advanced fellowship competency assessment, including case-based scenarios, and development of a registry of providers designated as IBD specialists in their respective centers.

Assessment of competency

Clinical competency is a difficult variable to measure in advanced IBD training given the lack of standardized assessment tools across all institutions, the lack of standardized reporting of clinical competency outcomes, and the inherent bias that can be present when assessing clinical competency. For this study, we used a self-assessment of competency given the lack of an objective marker available for advanced IBD training. Self-assessment of clinical competency is inherently flawed in that some trainees may over- or under-estimate their level of expertise. Furthermore, recall bias is an important limitation of self-assessment surveys and could have influenced the results in this study. One solution to this problem would be the integration of case-based assessments during advanced training. For example, serial assessments of trainee knowledge over the course of the advanced training year would allow for discovery of incremental knowledge accrual during the year. Furthermore, creation of IBD core competencies would likely improve monitoring of educational progress as well as delivery of critical feedback during the advanced training year. Cohen et al. have proposed several core domains of competency which provide a framework for beginning this process.

Development of case-based competency assessment tools covering various spheres of inpatient and ambulatory IBD care.

Challenges in quantitative assessment of mentorship

Advanced fellowships offer additional benefits, which cannot necessarily be quantified, including mentorship by thought leaders in IBD, networking opportunities, and refinement of research skills. Reflection by advanced IBD graduates highlighted the importance of such benefits with over 70% citing the importance of research and career mentoring during this year on their professional development. Unfortunately, like clinical competency, the effects of mentorship are difficult to measure. However, through the use of a prospective registry, indirect assessments of mentorship could be assessed and tracked. For example, completion of publications (book chapters, original manuscripts) with mentors, attainment of career development grants with mentors, presence at networking events with mentors, and mentee assessments of mentorship impact could be tracked during the advanced training year.

Develop direct and indirect metrics for assessment of mentorship quality, including through trainee-completed surveys and face-to-face time spent in meetings with mentor.

In conclusion, advanced IBD training is an increasingly popular avenue to refine clinical skills and continue academic development. However, assessment of the benefits of advanced training remains difficult owing largely to the lack of standardized measurements of training efficacy and inability to capture intangible benefits such as mentorship and networking. Efforts to standardize competency measurements for advanced IBD training and capture longitudinal data from advanced trainees will likely help advance educational research in IBD training. We also presented some challenges in quantifying benefit of advanced training and propose some solutions to address this educational gap.

Click here for additional data file.

	Very comfortable	Comfortable	Uncomfortable	Very uncomfortable
Mild to moderate CD or UC
Complicated/Severe CD or UC
IBD during pregnancy
Peri- and post-operative IBD
Pouch Related Disease
Therapeutic Drug Monitoring
Preventative Care
Nutritional Deficiencies and Short Gut
Chromoendoscopy
Dilation of IBD Related Strictures

	Essential	Optional but would be nice	Not Necessary
Inpatient IBD
Outpatient IBD
Radiology teaching
Pathology teaching
Observing IBD surgery in the OR
Rotation in pediatric IBD clinics
Formal training in research methods (masters level courses)

Professional Responsibilities	Percentage of Time
Academic Research
Outpatient IBD Clinic
Management of Inpatient IBD
Administrative Duties
Teaching

	Yes	No
Current Editorial Responsibilities
Peer Reviewer
Editorial Board
Associate Editor/Editor
Positions in Societies
Committee Membership
Leadership (chair/vice chair)
Institutional
Program Director
IBD Center Director/Co-director
Other