Does the "July effect" of new trainees at teaching hospitals impact outcomes for patients hospitalized with heart failure? Real-world analyses of more than half a million US admissions.

Introduction: The "July effect" refers to the potential of adverse clinical outcomes related to the annual turnover of trainees. We investigated whether this impacts inpatient heart failure (HF) outcomes.
Methods: Data from all adults (≥18 years) admitted with a primary diagnosis of HF at US teaching hospitals from the 2012-2014 National Inpatient Sample were analyzed. Non-teaching hospital admissions were excluded. The primary outcome was in-hospital mortality. Secondary metrics included hospital length of stay (LOS) and total cost adjusted for inflation. Logistic and linear regression models were used to adjust for confounders. Admissions were classified into 4 quarters (Q1-Q4), based on the academic calendar. Q1 and Q4 were designated to assess the effect of novice (July effect) versus experienced trainees, respectively.
Results: There were 699,675 HF admissions during Q1 and Q4 in the study period. Mean age was 71 ± 15 years and 48% were females. There were 20,270 in-hospital deaths, with no difference between Q1 and Q4; crude odds ratio (OR) 1.00, 95% confidence interval (CI) 0.94-1.07, p = 0.95. After risk adjustment, there was no in-hospital mortality difference between Q1 and Q4 admissions; adjusted OR 0.96, 95% CI 0.89-1.03, p = 0.23. There was no difference in hospital LOS or total cost; 5.8 versus 5.8 days, p = 0.66 and $13,755 versus $13,586, p = 0.46, in Q1 and Q4, respectively. Conclusions: In this nationally representative sample, there was no evidence of a "July effect" on inpatient HF outcomes in the US. This suggests that HF patients should not delay seeking care during trainee transitions at teaching hospitals.

Introduction

The “July effect” is a term used in academic medicine to describe the potential for adverse patient outcomes that may be consequential to clinical inexperience related to the annual turnover of trainees that occurs in the United States (US) on July 1st. The validity of this prevalent concern has been previously investigated [1]. For instance, major general and vascular surgical procedures early in the academic year are associated with higher morbidity and mortality, which may in part be attributable to the limited experience of new trainees [2]. To the contrary, this concern has been largely debunked in the cardiac surgery literature based on published results from analyses of over 470,000 cardiac procedures, where risk-adjusted in-hospital mortality for these procedures did not differ across academic year quartiles [3]. Other studies have investigated whether the “July effect” notion applies to medical and overall hospital admissions in the US, with conflicting results [4-6]. Hence, we sought to assess the relationship between academic year quarter and patient outcomes at US teaching hospitals in patients admitted for heart failure (HF).

Methods

To address this question, we used the National Inpatient Sample (NIS) database, the largest publicly available, all-payer, administrative database in the US. The NIS database comprises de-identified data from over 7 million inpatient admissions annually. Data are derived from a 20% stratified sample of US hospitals, with sampling weights that translate into representative national estimates for over 95% of the US population [7].

For the present analyses, data from all adults (≥18 years) admitted with a primary diagnosis of HF from 2012 to 2014 were included, with primary diagnosis identified by ICD-9 codes. Since the consideration of the “July effect” is unique to academic medical centers, non-teaching hospitals were excluded. The primary outcome of interest was in-hospital mortality. Secondary metrics of interest included hospital length of stay (LOS) and total hospitalization cost, adjusted for inflation. Logistic and linear regression were used for analyses of associations and to adjust for potential confounders. Candidate variables were first tested for univariable associations, and those with a p-value <0.2 were included in the final multivariable model. Charlson Comorbidity Index was used to adjust for patient comorbidities [8]. Continuous data are presented as mean ± standard deviation (SD) and categorical data as frequencies. The timing of HF admissions was classified into 4 quarters based on the academic calendar. Quarter (Q) 1 represented the months of July–September. Q4 represented the months of April–June, with all analyses comparing Q1 versus Q4. All analyses were performed using Stata (StataCorp. 2015 Stata Statistical Software: Release 14. College Station; TX: StataCorp LP). The Stata survey commands were used to account for clustering, stratification, and weighting of data in the NIS.

Results

There were 699,675 HF hospitalizations during Q1 and Q4 over the study period. Baseline patient and hospital characteristics are provided in Table 1. Overall, the mean age was 71 ± 15 years, and 48% were women. There were 20,270 in-hospital deaths, 95% confidence interval (CI) 19,442–21,097 (Q1 9,695, 95% CI 9,216–10,174 versus Q4 10,575, 95% CI 10,024–11,126), with no difference between Q1 and Q4; crude odds ratio (OR) 1.00, 95% CI 0.94–1.07, p = 0.95. After adjusting for age, sex, race, estimated household income, Charlson Comorbidity Index, hospital region, hospital size, and insurance type, there remained no mortality difference between Q1 and Q4; adjusted OR 0.96, 95% CI 0.89–1.03, p = 0.23 (Fig. 1). Similarly, there was no difference in adjusted hospital LOS or cost; 5.8 ± 7.2 versus 5.8 ± 7.0 days, p = 0.66; and $13,755 ± 27,182 versus $13,586 ± 27,517, p = 0.46, for Q1 versus Q4, respectively.

Table 1

Baseline patient and hospital characteristics of admissions in the first and last quarters of the academic year (Q1 and Q4).

	Q1 (N = 334,965)	Q4 (N = 364,710)	p-value
Age – years, mean (95% CI)	70.4 (70.2–70.6)	70.7 (70.5–71.0)	p < 0.001
Females, n (%)	160,785 (48)	174,325 (48)	p = 0.45
Race			p = 0.003
White, n (%)	190,105 (60)	210,445 (61)
Black, n (%)	85,425 (27)	89,760 (26)
Hispanic, n (%)	25,650 (8)	28,170 (8)
Other/unknown, n (%)	16,850 (5)	18,435 (5)
CCI, mean (95% CI)	3.4 (3.4–3.4)	3.4 (3.4–3.4)	p = 0.003
Median household income*			p = 0.10
Quartile 1, n (%)	110,215 (34)	118,270 (33)
Quartile 2, n (%)	78,550 (24)	87,030 (24)
Quartile 3, n (%)	76,035 (23)	83,335 (23)
Quartile 4, n (%)	63,485 (19)	68,940 (19)
Insurance status*			p = 0.11
Medicare, n (%)	238,595 (73)	261,530 (73)
Medicaid, n (%)	35,400 (11)	37,250 (10)
Private, n (%)	41,540 (13)	45,160 (13)
Self-pay, n (%)	11,765 (4)	12,540 (4)
Hospital bed size			p = 0.38
Small, n (%)	60,980 (18)	66,315 (18)
Medium, n (%)	97,630 (29)	105,540 (29)
Large, n (%)	176,355 (53)	192,855 (53)
Hospital region			p = 0.001
Northeast, n (%)	84,570 (25)	93,465 (26)
Midwest, n (%)	81,665 (24)	88,565 (24)
South, n (%)	127,084 (38)	136,319 (37)
West, n (%)	41,645 (12)	46,360 (13)

CCI = Charlson Comorbidity Index; CI = Confidence Interval.

Median household income and insurance status have missing values

All percentages rounded to nearest whole number

Significant p-values rounded to 3 decimal places

Non-significant p-values rounded to 2 decimal places

All counts (n) are weighted estimates of number of admissions

Fig. 1.

In-hospital mortality comparison between the first and last quarters of the academic year (Q1 versus Q4).

Discussion

In these analyses of a large, nationally representative sample of HF admissions in the US, and after risk adjustment, there was no difference in in-hospital mortality, hospital LOS, or total costs, when comparing admissions in the first versus last quarter of the academic year. The “July effect” theory has been put to the test in analyses of data from several cohorts and across different disciplines, with varying results. Among the key messages across investigations of the “July effect” is that adequate control for baseline risk factors is crucial, and evidence of a “July effect” seems only to be observed when adequate controls are not performed [1-3].

It is plausible that the “July effect” may be more relevant to surgical specialties, where new/junior trainees rely heavily on direct “hands-on” training for operative procedures that may have steep learning curves for improvement of skills and outcomes, with potential for adverse results despite this training having direct supervision. Whether the same situation applies to medical specialties is controversial. In 2017, Mims, et al. sought to assess the outcomes of July medical admissions; namely, myocardial infarction (MI), HF, and pneumonia [5]. They observed longer hospital LOS and higher costs associated with July HF admissions – findings that were not observed in the present results. There are several reasons why these two analyses may have yielded different findings. One major reason is that the present analyses using data from 2012 to 2014 comprises over 3 times more patients than the sample of HF patients from the 2011 NIS dataset included in the prior analyses. The larger sample size in the present cohort likely translates into more reliable estimates [5]. Additionally, the prior study included comparisons between teaching and non-teaching hospitals as well as Q1 versus all other quarters (Q2 through Q4). In the present analyses, only participants admitted in Q1 and Q4 (excluding Q2 and Q3), and admitted at teaching hospitals (excluding non-teaching hospitals) were included, so that the focus of comparison is the “novice trainee effect” and to minimize the effects of and associations with other factors, such as seasonal variations in HF admissions as well as inherent differences between teaching and non-teaching hospitals, that may have confounded the results [9,10]. It is also worth highlighting that the reported differences in hospital LOS and costs observed in the prior analyses were generally modest with marginal real-world significance [5].

In analyses of another large dataset of over 18 million admissions from the University Health System Consortium, which includes admission data from 120 academic medical centers and 333 affiliated hospitals in the US, there was no evidence of increased risk-adjusted mortality based on admission quarter [4]. A more recent study investigated the adverse outcomes in July/August (compared with the rest of the academic year) for medical and surgical admissions from the Medicare Patient Safety Monitoring System database between 2010 and 2017 [6]. While those analyses were stratified by medical and surgical admissions, the medical admissions were all pooled together (MI, HF, and pneumonia); thereby not allowing for HF-specific inferences; compromising comparison with the present results. However, the prior overall findings are largely consistent with the present results, where the risk of adverse events was not higher for July/August admissions. Taken together, these observations provide reassurance of consistent outcomes and suggest that patients should not delay seeking medical care in July (or first quarter the academic year) for concerns about the quality of care delivered by new trainees. For HF specifically, these findings may be credited to the well-defined guidelines that facilitate safe patient care with appropriate supervision.

Despite several strengths of the present study, including the large size of the NIS database and its standardized survey methodology, there are specific limitations. Notably, given the administrative nature of the NIS database, information on granular clinical information is not available, and diagnostics are based on ICD-9 coding. Additionally, the increasing role of hospitalists and non-house staff teams cannot be accounted for using this dataset. Furthermore, observation is limited to hospitalization, without the ability to assess longer-term outcomes or hospital readmission rates.

In conclusion, there was no evidence of a “July effect” on inpatient HF outcomes or other important metrics of care. In-hospital mortality, hospital LOS, and total cost did not differ between patients admitted in the first versus last quarter of the academic year. These results provide reassurance of consistent outcomes for patients admitted with HF during trainee transitions at teaching hospitals.