Mandatory Hepatology Education for Internal Medicine Residents: Long-Term Effects and Implications for Workforce Needs.

We previously created a mandatory, inpatient, hepatology resident curriculum that immediately improved comfort, knowledge, and career interest in chronic liver disease (CLD). The durability of these effects needs to be known to use this intervention to address the hepatologist shortage. Thus, we aimed to assess this curriculum's long-term outcomes on internal medicine (IM) residents' CLD comfort, knowledge, and career interest. From 2015 to 2019 at a single institution, one IM resident was always assigned to the rotation. Similar anonymous assessments were administered to incoming postgraduate year (PGY)-1 residents and graduating PGY-3 residents, including a historic control cohort that graduated in June 2015. At residency completion, the intervention cohort (n = 61) had significantly higher comfort (1, not at all comfortable/strongly disagree; 5, very comfortable/strongly agree) with both hepatology (e.g., hepatitis C, 2.5 vs. 3.3, P < 0.001) and common IM topics (e.g., heart failure, 3.6 vs. 4.8, P < 0.001) but not specialty topics lacking curricula (e.g., inflammatory bowel disease, 2.8 vs. 2.7, P = 0.54). Compared to the historic cohort (n = 27), the intervention cohort was more comfortable in several CLD topics (e.g., cirrhosis, 3.2 vs. 3.8; P = 0.005) and answered more questions correctly (65% vs. 55%; P = 0.04), but career interest was unchanged (1.9 vs. 1.8; P = 0.45). Many residents (33%) would consider a hepatology career if training were separated from gastroenterology.
Conclusion: With the completion of a mandatory hepatology curriculum, residents' CLD comfort and knowledge durably improved and exceeded that of historic counterparts. Initial career interest was not sustained, perhaps due to prerequisite gastroenterology training. These findings suggest IM educational initiatives may better address hepatology workforce needs by generating comanagers than by recruiting trainees.

chronic liver disease

internal medicine

In‐Training Examination

Medical Knowledge Self‐Assessment Program

postgraduate year

The burden of chronic liver disease (CLD) in the United States is rapidly increasing. The number of inpatient hospitalizations for CLD now equals those of chronic obstructive pulmonary disease, and cirrhosis death rates have been worsening, with greatest increases seen among young adults due entirely to alcohol‐related liver disease.( , , ) These trends are only expected to worsen due to the aging of patients with CLD and the obesity, alcohol use, and opioid epidemics that are fueling significant increases in nonalcoholic fatty liver disease, alcohol‐related liver disease, and new hepatitis C infections, respectively.( , , , )

Despite this increasing burden, there is a growing shortage of providers trained to provide care for CLD.( ) By 2033, it is projected that there will be a shortage of 35% clinical full‐time equivalents in adult hepatology.( ) In order to bridge this widening deficit, it is imperative for internal medicine (IM) physicians to be trained to comfortably comanage basic aspects of CLD alongside gastroenterologists and hepatologists, which is the premise of successful programs, such as Project Extension for Community Healthcare Outcomes (ECHO) and the American Association for the Study of Liver Diseases’ Fundamentals of Liver Disease curriculum.( ) In fact, it has been argued that the only way for hepatologists to address the number of patients with hepatitis C virus requiring treatment is to enlist the assistance of internists( ); however, we previously demonstrated that graduating postgraduate year (PGY)‐3 residents did not choose to rotate on an elective hepatology rotation, despite viewing it as integral to their training.( ) Therefore, we created a mandatory, inpatient, hepatology rotation with a complementary didactic curriculum that was shown to improve comfort with and knowledge of CLD and also generated an increase in hepatology career interest following completion of the rotation.( ) Understanding the durability and longevity of these effects on a residency program is needed to better guide the use of a mandatory hepatology curriculum as a recruitment tool as well as the creation of other subspecialty curricula. This study aimed to prospectively analyze the long‐term effects of a mandatory hepatology curriculum within a cohort of IM residents over a 4‐year period.

Materials and Methods

This study was conducted at the University of Chicago Medical Center, a large, urban, academic hospital with a liver transplant program; the Center performed an average of 28 transplants per year from January 2014 to December 2018. The inpatient hepatology service at the University of Chicago provides care for patients with liver disease, including those with decompensated cirrhosis and after liver transplant. The service is comanaged by an academic hospitalist team and a hepatology team (comprised of a hepatology attending, a gastroenterology fellow, and a transplant hepatology fellow).( ) A plan for each patient is established during daily multidisciplinary rounds, which include the transplant surgery team, the transplant nurses, a transplant social worker, a transplant pharmacist, a transplant nutritionist, and the liver unit’s floor nurses. Before the 2014‐2015 academic year, a 2‐week hepatology rotation was an elective inpatient service offered to all IM and medicine‐pediatric residents. In 2014‐2015, in response to a needs assessment from the prior year,( ) the inpatient hepatology service became mandatory, the curriculum was restructured, and the aforementioned pilot study assessed the immediate impact of this intervention during that time period.( ) The current follow‐up study occurred from 2014 to 2018 and does not include data from the previous pilot study. The average patient census on this service during this study period was 10. Each 2‐week rotation included 1 to 2 PGY‐2 and/or 3 residents in addition to occasionally including interns or medical students. The features and learning objectives of the curriculum and rotation have all been published (Supporting Materials S1).( ) The addition of a web‐based module on hepatocellular carcinoma and updates/additions to the library of articles are the only modifications that occurred to the curriculum in this current study. The University of Chicago Institutional Review Board deemed that this study was exempt from full review.

Preresidency and Postresidency Assessments

Anonymous paper‐based assessments were administered to all incoming PGY‐1 residents and to the same cohort at the completion of their PGY‐3 year. Similar assessments were completed by a cohort of 27 PGY‐3 residents that graduated in June 2014. This group served as a historic control group that did not experience the new rotation and curriculum. The initial assessment for the PGY‐1 residents consisted of 40 Likert‐type questions to assess comfort and self‐perceived knowledge in managing various general medical and CLD conditions (no validity evidence for this portion) and 12 multiple‐choice questions from the Medical Knowledge Self‐Assessment Program (MKSAP) 16 to objectively assess knowledge of CLD (Supporting Materials S2). The postresidency assessment given to this cohort included the same 40 Likert‐type questions with the addition of four new questions as well as 24 multiple‐choice questions from MKSAP 15 or 16 and Digestive Diseases Self Education Program version 6.0 (Supporting Materials S3). The first 12 multiple‐choice questions were identical to the questions on the preresidency assessment to gauge improvement. The final 12 questions were identical to the questions administered to the historic cohort.

Statistical Analysis

Statistical analysis was performed using RStudio software (RStudio, Inc., Boston, MA). Wilcoxon rank sum tests and Mann‐Whitney tests were used to analyze the responses to the Likert‐type questions on the preresidency and postresidency assessments. Paired and unpaired t tests were used to compare the mean number of multiple‐choice questions answered correctly. For all tests, statistical significance was defined as P ≤ 0.05. Descriptive statistics were used to analyze the trends of the In‐Training Examination (ITE).

Results

Demographics

A total of 59 out of 68 (87%) residents completed both the prerotation and postrotation assessments in their entirety and were included in the intervention cohort. Two of the remaining 9 residents who did not answer the demographic questions but did answer the remainder of the questions were included in the subsequent subjective cohort analyses (total n = 61). Seven only partially completed one or more of their assessments and were excluded from the study. The intervention cohort was comprised of 25 (42%) men and 34 (58%) women from 30 different medical schools. In total, 57 (97%) residents in the intervention cohort completed the hepatology rotation before they graduated, with the majority completing the rotation in their third year (n = 37 [65%]) and the remaining in their first (n = 16 [28%]) and second years (n = 15 [26%]).

Additionally, 10 (18%) residents completed the rotation twice. A majority of the graduates reported participating in an elective gastroenterology rotation (n = 35 [59%]) and having completed at least some of the MKSAP 16 gastroenterology questions (n = 50 [85%]) during their residency.

The historic cohort was comprised of 27 residents; 10 (39%) of the residents were men, and 14 different medical schools were represented in the historic cohort. Only a minority of these residents participated in the previously offered elective hepatology rotation (n = 10 [37%]) or a gastroenterology rotation (n = 8 [30%]) before graduation.

Subjective Assessments of Knowledge and Comfort

A Likert‐type scale (1, not at all comfortable; 5, very comfortable) was used to assess resident comfort with commonly encountered topics within IM, gastroenterology, and hepatology. There was significant improvement in resident comfort with each of the hepatology and general medicine topics at the completion of residency training within the intervention cohort (Table 1). This trend was not observed for specialty topics without a dedicated curriculum (e.g., celiac disease, inflammatory bowel disease). Of the hepatology topics, graduates of the intervention cohort only showed a significant improvement in comfort with hepatitis C compared with those in the historic cohort.

TABLE 1

Percentage of Residents (n = 61) Who Were Somewhat Comfortable or Very Comfortable With Each of the Topics and Mean Likert Scores for Responses to “What Is Your Comfort Level Managing the Following ______?” (1, Not at All Comfortable; 5, Very Comfortable) from 2014 to 2018 at the University of Chicago

Topic Assessed		Intervention Cohort					Historic Cohort
		Intern Year		Graduating Year		P Value	Percent (%) Somewhat/Very Comfortable	Mean Likert Score	P Value
		Percent (%) Somewhat/Very Comfortable	Mean Likert Score	Percent (%) Somewhat/Very Comfortable	Mean Likert Score
General medical conditions	Congestive heart failure	67	3.61	97	4.83	<0.001	100	4.96	0.18
	Chronic obstructive pulmonary disease/asthma	75	3.75	98	4.78	<0.001	100	4.93	0.14
	Non‐ST elevation Myocardial infarction	52	3.44	98	4.73	<0.001	100	4.81	0.52
	Vaso‐occlusive crises	28	2.87	98	4.80	<0.001	100	4.81	0.99
	Tumor lysis syndrome	15	2.25	84	4.21	<0.001	74	4.04	0.42
Liver conditions	Spontaneous bacterial peritonitis	31	2.98	85	4.13	<0.001	81	4.12	0.94
	Acute liver failure	17	2.23	51	3.40	<0.001	44	3.26	0.53
	Hepatitis B	11	2.46	25	2.83	0.01	15	2.52	0.11
	Hepatitis C	13	2.46	48	3.26	<0.001	22	2.63	0.004
	Hepatorenal syndrome	18	2.36	44	3.39	<0.001	41	3.04	0.20
	Nonalcoholic steatohepatitis	20	2.67	43	3.30	<0.001	37	3.04	0.26
Specialty topics without curriculum	Celiac disease	30	2.92	36	3.05	0.37	30	2.78	0.24
	Inflammatory bowel disease	26	2.82	18	2.73	0.54	15	2.37	0.06

A Likert‐type scale (1, no knowledge; 5, strong knowledge) was also used to evaluate self‐perceived knowledge of 15 topics within CLD. There was significant improvement in self‐perceived knowledge across all CLD topics before the intervention cohort’s completion of residency (Supporting Table S1). This portion of the assessments was not administered to the historic cohort.

The final set of Likert‐type questions prompted residents to rate their agreement (1, strongly disagree; 5, strongly agree) with several statements. There were significantly more graduating residents in the intervention cohort who agreed that they felt comfortable managing cirrhosis, abnormal liver tests, and posttransplant recipients when compared with both the historic cohort and PGY‐1 residents in the intervention cohort (Table 2). Furthermore, significantly more graduates in the intervention cohort agreed that their training in gastroenterology and hepatology was adequate when compared to the historic cohort.

TABLE 2

Percentage of Residents (n = 61) Who Agreed or Strongly Agreed and the Mean Likert Scores for Responses to “Rate Your Agreement With the Following Statements” (1, Strongly Disagree; 5, Strongly Agree) from 2014 to 2018 at the University of Chicago

Statement	Intervention Cohort					Historic Cohort
	Intern Year		Graduating Year		P Value	Percent (%) Agree/ Strongly Agree	Mean Likert Score	P Value
	Percent (%) Agree/ Strongly Agree	Mean Likert Score	Percent (%) Agree/Strongly Agree	Mean Likert Score
Gastroenterology is an integral component of training in IM	100	4.75	98	4.69	0.83	100	4.52	0.11
Hepatology is an integral component of training in IM	98	4.61	98	4.59	0.66	96	4.56	0.85
Your residency program has provided sufficient education in gastroenterology	‐	‐	56	3.61	‐	22	2.93	0.003
Your residency program has provided sufficient education in hepatology	‐	‐	62	4.59	‐	26	2.70	<0.001
I feel comfortable managing liver cirrhosis	30	2.97	70	3.82	<0.001	41	3.22	0.005
I feel comfortable managing patients with a liver transplant	7	2.13	26	2.89	<0.001	11	2.19	0.005
I feel comfortable interpreting abnormal liver tests	66	3.67	92	4.25	<0.001	70	3.85	0.01
I feel comfortable performing a paracentesis	20	2.43	67	3.69	<0.001	78	3.89	0.56
I am interested in a career in gastroenterology	39	3.25	15	2.05	<0.001	7	1.63	0.14
I am interested in a career in hepatology	23	2.87	7	1.87	<0.001	7	1.77	0.45

A total of 27 (47%) PGY‐3 residents agreed that the comanagement of the hepatology service prepared them for practice after residency, 38 (66%) agreed that comanagement allowed for more time to focus education, and 9 (15%) agreed that comanagement prevented autonomy in patient care.

Objective Assessments of Knowledge

The mean percentage of questions answered correctly by the PGY‐3 residents in the intervention cohort was 7.8 out of 12 (65%) compared to 6.8 out of 12 (55%) in the historic cohort (P = 0.04) (Fig. 1A). Of the 12 topics assessed, the graduating seniors in the intervention cohort improved in eight topics when compared to the historic cohort. The mean percentage of questions answered by the PGY‐1 residents versus the PGY‐3 residents in the intervention cohort improved from 34% to 56% (n = 60; P < 0.001) (Fig. 1B). PGY‐3 residents improved in 11 of 12 topics. The majority of the residents (n = 52 [87%]) showed an improvement of their scores following the completion of residency.

FIG. 1

Mean percentage of residents who answered each topic correctly on the multiple‐choice questions between the years 2014 and 2018 at the University of Chicago. (A) Mean percentage of residents on the postrotation assessment from the historic cohort (n = 27) and intervention cohort (n = 61). (B) Mean percentage of residents during their intern year ([1], n = 61) and following graduation ([3], n = 60). Abbreviations: AKI, acute kidney injury; Dx, diagnosis; EtOH, ethanol; HBV, hepatitis B virus; HC, historic cohort; HCC, hepatocellular carcinoma; HCV, hepatitis C virus; HPS, hepatopulmonary syndrome; IC, intervention cohort; NASH, nonalcoholic steatohepatitis; OLT, orthotopic liver transplant; SBP, spontaneous bacterial peritonitis; Tx, treatment.

Career Interest in Hepatology

There were 9 residents (15%) in the intervention cohort intending to pursue a gastroenterology fellowship; 8 (89%) of these residents reported the hepatology rotation positively impacted this decision, and 2 residents (20%) planned to pursue a career in hepatology.

As seen in Table 2, residents’ career interest in hepatology declined in the intervention cohort and was unchanged when compared to the historic cohort. When asked if one would consider hepatology as a career if its training pathway was separated from gastroenterology, 20 (33%) of the intervention cohort agreed that they would.

ITE Trends

Before the establishment of the curriculum, the historic cohort from 2011 to 2013 experienced a linear decline in the overall percentage of hepatology‐related questions answered correctly on the ITE (71% in PGY‐1 vs. 63% in the PGY‐3) (Fig. 2A). A similar trend was observed nationally. In contrast, after the implementation of the curriculum, the cohorts testing from the years 2014 to 2016 and 2015 to 2017 experienced a linear increase in the overall percentage of hepatology‐related questions answered correctly on the ITE (PGY‐1, 65% vs. 72%; PGY‐2, 68% vs. 81%). Although a similar increase occurred nationally as well, the residents in the intervention cohort equaled or outperformed the national average each year (Fig. 2B,C).

FIG. 2

Mean percentage of hepatology‐related questions answered correctly on the ITE. (A) By University of Chicago residents and a national sample of residents between 2011 and 2013. (B) By one class of University of Chicago residents and a national sample of residents between 2014 and 2016 after implementation of the curriculum in 2014. (C) By a second class of University of Chicago residents and a national sample of residents between 2015 and 2017 after implementation of the curriculum in 2014. Abbreviation: UofC, University of Chicago.

Discussion

In response to both trainee‐reported deficits and the rapidly changing epidemiology of CLD, we implemented a nonelective hepatology rotation and an accompanying curriculum that uniquely placed residents in a comanagement role and included focused education on the management of various types of CLD. Looking beyond the previously demonstrated short‐term benefits, we sought to assess the long‐term effects of this educational intervention on a trainee cohort. Several key observations described below emerged from these data that promote further understanding of how to harness hepatology curricula in residency training to address workforce deficits and better inform the creation of other subspecialty curricula.

1. Implementation of a hepatology curriculum generated durable improvements evident at the completion of residency.

One major critique of medical education research is its focus on short‐term outcomes assessed nearly immediately after the completion of an intervention.( ) These outcomes are often assumed to be predictors of long‐term performance, but this is rarely proven. A major lingering question following our previous study was whether or not the subjective and objective improvements in comfort and knowledge of liver disease would be sustained throughout residency training. The present study demonstrated significant improvements in self‐rated comfort and knowledge and objectively assessed knowledge across two residency classes over the course of 4 years. These trends are likely attributable to the intervention because of the following: (1) a similar trend was not seen in other gastroenterology topics (e.g., inflammatory bowel disease) without a dedicated educational intervention but was seen in other core general medicine topics within the IM residency; (2) the historic cohort (which was not exposed to the intervention) was significantly less comfortable and knowledgeable with several topics at the time of graduation; and (3) no other programmatic changes occurred during this time to account for increased exposure to liver disease.

Interestingly, the intervention cohort noted higher comfort levels in managing cirrhosis, abnormal liver tests, and posttransplant recipients when compared to the historic control; but when assessing specific conditions (e.g., acute liver failure), the intervention cohort only rated higher levels of comfort with hepatitis C. Thus, at the completion of residency, a generalized comfort with CLD persisted in the intervention cohort but a specialized comfort for more nuanced decision making in CLD was lacking. This finding is not unexpected following a 2‐week intervention as improvements in comfort in many of these topics often require dedicated fellowship training. More importantly, this generalized comfort with CLD is what is needed to facilitate more comanagement alongside hepatologists as the majority of primary care providers state that willingness to comanage cirrhosis is impeded by a lack of confidence.( , ) Using subspecialty curricula to generate generalized comfort may be a means to partly offset the predicted shortage of 17,800 medical subspecialists by 2033.( )

2. A mandatory hepatology curriculum can generate career interest, but the effect is not durable.

One major observation in our previous study was a significant increase in career interest in hepatology immediately following completion of the curriculum, suggesting that curricula can be effective workforce recruitment tools. However, this current study highlights that this career interest actually wanes back to baseline as time passes during residency training.

A 2‐week curriculum can spark interest, but the effect was not durable in the study. This may be due to the lack of efforts in this study to ensure ongoing repeated exposure to CLD (i.e., the lower transplant volume at our center may have caused minimal exposure to CLD outside of the rotation). Further complementary interventions may be needed to maintain and augment initial interest generated by a curriculum if it is to be used as a means of recruitment. However, 89% of those planning to pursue a gastroenterology fellowship reported that the hepatology rotation positively impacted this decision, suggesting the curriculum remains an effective tool for generating career interest. An alternative explanation often debated is that the traditional training pathway of needing to complete a gastroenterology fellowship before a hepatology fellowship ultimately dissuaded residents and led to the decline in hepatology career interest.( ) This is supported by the 33% of residents who agreed that they would consider a career in hepatology if it were separated from gastroenterology.

3. The assigned mandatory nature of a hepatology curriculum overcame systemic deficits.

Before the intervention, only 37% of residents chose to rotate on the hepatology rotation despite seven opportunities to do so and overwhelming agreement that it was integral to their training. Not unexpectedly, only 26% reported sufficient training in hepatology at the completion of residency; there was a linear decline in ITE performance with liver‐related questions. In response to these data, the rotation and curriculum in this study were designed to be a mandatory experience. Following implementation, almost the entire intervention cohort (97%) had completed the rotation, and subsequently, there was a marked increase in agreement on the sufficiency of hepatology education as well as a linear improvement in ITE performance that exceeded national averages.

These findings raise the question whether or not some aspects of subspecialty education within IM residency training should be mandatory. The traditional paradigm of residents choosing subspecialty electives facilitates letters of recommendation and clinical experiences in the field of their desired fellowship, which is undeniably important to maintain. However, in this system, many may neglect subspecialties that are critical to their training or areas of weakness. Thus, residency program leadership can use ITE data as an individualized needs assessment to identify required subspecialty rotations that will complement, individualize, and improve a resident’s ultimate career path. The Alliance for Academic Internal Medicine Education Redesign Task Force endorsed a similar approach.( )

4. A rotation implementing a comanagement model has educational benefits.

A comanagement model of care is defined as a model in which hospitalists and specialists jointly manage patients and hospitalists write orders to dictate day‐to‐day care, diagnose and treat complications promptly, and facilitate transition of care to the outpatient setting.( ) This model has been shown to decrease the burden of tasks in the primary manager role but requires the specialist physician to assume more active responsibility for a patient's care plan compared to a consultant. Comanagement has been shown to improve the quality of care for patients with CLD and is increasingly recognized as a necessary model in the outpatient management of CLD.( , , , ) Our previous study was the first to assess the educational value of placing IM residents in the role of a specialist comanager; this uncovered notable strengths (e.g., elimination of excessive time spent on task completion to facilitate more time for educational opportunities) and weaknesses (e.g., lack of complete responsibility for patients) cited by the rotators. This study affirmed that the comanagement model allowed trainees more time to focus on education about the subspecialty itself and ultimately was not prohibitive of autonomy in patient care.

One limitation of our study is that it was performed at a single academic institution. Although it is reassuring that the improvements in comfort and knowledge were both durable and duplicated in two additional residency classes, it is unclear if similar results would be generated in a different residency program. Also, this study did not prospectively randomize residents to the cohorts but rather exposed all residents to the curriculum and included a smaller historic cohort for comparison. This approach was chosen to mitigate the confounding of results in the intervention cohort from intervention/exposure bias. Logistically, it was also not feasible to divide residency classes into two groups and expose them to two different types of hepatology rotations when only one hepatology inpatient service exists. However, through this approach, other extraneous factors (e.g., the rapid emergence and subsequent increased familiarity with direct‐acting antiviral therapies for hepatitis C during the time period of this study) may have confounded the results. It is still possible that intervention/exposure bias could have affected differences in comfort levels in the intervention cohort, but because assessments were administered at the completion of residency (months to years after exposure to the rotation) and 97% of the residents were exposed to the curriculum, this seems less likely. Another limitation is that improved comfort and knowledge does not necessarily translate into improved patient outcomes. Future areas of study need to focus on whether clinical outcomes have improved in response to the curriculum. Finally, because this was an inpatient‐only rotation, residents were not exposed to outpatient management of CLD, which may explain the worsening performance on some topics (e.g., hepatitis B and C) and may have also contributed to the waning career interest.

In conclusion, we have demonstrated that the implementation of a mandatory 2‐week inpatient hepatology rotation and accompanying curriculum not only lead to immediate improvements in comfort and knowledge of CLD seen at completion of the rotation but also sustained improvements that are still evident at the time of completion of residency. The generalized comfort with CLD present at the end of residency can be harnessed to facilitate more comanagement of CLD alongside hepatologists; ultimately, this can help address the growing burden of CLD and offset workforce deficits. These findings suggest IM educational initiatives may better address hepatology workforce needs by generating comanagers than by recruiting trainees, but separation of gastroenterology and hepatology training pathways could increase their recruitment potential as well.

Supplementary Material

Click here for additional data file.

Supplementary Material

Click here for additional data file.

Supplementary Material

Click here for additional data file.

Table S1

Click here for additional data file.