Association of residency work hour reform with long term quality and costs of care of US physicians: observational study.

OBJECTIVE: To determine whether 30 day mortality, 30 day readmissions, and inpatient spending vary according to whether physicians were exposed to work hour reforms during their residency.
DESIGN: Retrospective observational study.
SETTING: US Medicare. PARTICIPANTS: 20% random sample (n=485 685) of Medicare beneficiaries aged 65 years or more admitted to hospital and treated by a general internist during 2000-12. MAIN OUTCOME MEASURES: 30 day mortality, 30 day readmissions, and inpatient Medicare Part B spending among patients treated by first year internists who were fully exposed to the 2003 Accreditation Council for Graduate Medical Education (ACGME) work hour reforms during their residency (completed residency after 2006) compared with first year internists with partial or no exposure to reforms (completed residency before 2006). Senior internists not exposed to reforms during their residency served as a control group (10th year internists) for general trends in hospital care: a difference-in-difference analysis.
RESULTS: Exposure of physicians to work hour reforms during their residency was not associated with statistically significant differences in 30 day mortality, 30 day readmissions, or inpatient spending. Among 485 685 hospital admissions, 30 day mortality rates during 2000-06 and 2007-12 for patients of first year internists were 10.6% (12 567 deaths/118 014 hospital admissions) and 9.6% (13 521/140 529), respectively, and for 10th year internists were 11.2% (11 018/98 811) and 10.6% (13 602/128 331), for an adjusted difference-in-difference effect of -0.1 percentage points (95% confidence interval -0.8% to 0.6%, P=0.68). 30 day readmission rates for first year internists during 2000-06 and 2007-12 were 20.4% (24 074/118 014) and 20.4% (28 689/140 529), respectively, and for 10th year internists were 20.1% (19 840/98 811) and 20.5% (26 277/128 331), for an adjusted difference-in-difference effect of 0.1 percentage points (-0.9% to 1.1%, P=0.87). Medicare Part B inpatient spending for first year internists during 2000-06 and 2007-12 was $1161 (£911; €1024) and $1267 per hospital admission, respectively, and for 10th year internists was $1331 and $1599, for an adjusted difference-in-difference effect of -$46 (95% confidence interval -$94 to $2, P=0.06).
CONCLUSIONS: Exposure of internists to work hour reforms during their residency was not associated with post-training differences in patient mortality, readmissions, or costs of care. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

A core purpose of graduate medical education in all countries is to prepare physicians for unsupervised practice. In the US, a frequently debated subject is whether this objective continues to be met since the implementation of several major residency work hour reforms by the Accreditation Council for Graduate Medical Education (ACGME). The first reform, in 2003, established a maximum 80 hour work week and prohibited shift lengths from exceeding 30 consecutive hours (see box).1 The second reform, in 2011, further capped shift lengths at 16 consecutive hours for interns and 28 hours for other trainees.2 The third reform, in 2017, allowed for longer shift lengths for interns, among other changes.3 Reductions in residency work hours and increased shift work in US academic medical centers have led to speculation that physicians completing residency today have less robust clinical experience before entering unsupervised practice compared with pre-reform residency cohorts.4 5 6 7 8 9 10 In contrast with this view, the marginal reduction in clinical skill resulting from fewer work hours might be small, particularly given already long work hours, and it might also be possible that residents who are less fatigued consolidate knowledge better and have equivalent or greater clinical competency both during and after residency.11 12 Despite these alternative hypotheses, it is unknown whether the work hour reforms have had long term effects on post-training outcomes of physicians who completed residency during the period of these reforms.

Medical training in the US comprises four years in medical school completed after an undergraduate degree (typically four years), followed by three years of residency training for those who seek to practice internal medicine. The first year of residency training is called the intern year

Before 2003, physicians in training in US internal medicine residency programs routinely worked more than 80 hours a week, with shifts often lasting 30 hours or more

Concerns about high work hours for resident physicians first arose in New York state in 1984 after a widely publicized death in a teaching hospital. The state imposed a reduction in work hours in July 1989

In response to mounting concerns over medical errors as a result of fatigued resident physicians, and legislative action by the US House of Representatives proposing federal work hour limits, the Accreditation Council for Graduate Medical Education (ACGME) announced reforms to work hours in 2002, which took effect in July 2003

The 2003 ACGME reforms restricted resident physicians to a maximum of 80 hours work a week, limited shifts to 24 hours (not including time for handoffs of patient care), limited in-hospital call to every third night, and mandated four days off every 28 days (on average one day a week). Subsequent reforms were enacted in 2011 and 2017

Although several studies have evaluated the association of work hour reforms on outcomes of patients treated in teaching hospitals, with the purpose of understanding how resident work hours and shift structures influence patient outcomes, no national studies have been done of the association of these reforms with performance of physicians after residency training

To tackle similar concerns in Europe, the Council of Europe in Brussels developed the European Working Time Directive in 1998—a mandate that specified requirements for working hours, nighttime work, and rest periods for all public employees.10 The directive, which included physicians in training, limits employees to 48 hours work a week. The restriction in work hours has raised considerable debate as to its impact on graduate medical education and the performance of physicians entering independent practice.10 13

Previous studies of residency work hour reforms in the US have focused on the immediate impacts of these reforms on the quality of care provided by resident physicians, as measured by patient outcomes in teaching intensive hospitals,14 15 16 17 18 19 20 rather than the later post-training quality of care provided by physicians who were exposed to reduced work hours during their residency. Similarly, in the United Kingdom, a meta-analysis of 13 studies evaluating the association between reduced work hours and the quality of resident education found no association in six studies, a negative association in six studies, and improvement in resident education in one study.21 One multicenter study of patient outcomes found no relation between reduced work hours and patient mortality, whereas another study found a small reduction in adverse events and errors. 21 No studies have evaluated the impact of reduced work hours from the European Working Time Directive on the performance of physicians after completion of training.

We conducted a national analysis of quality and costs of hospital care provided by internists who completed their residency in the US during 2000-12 and were variably exposed to the 2003 ACGME work hour restrictions during that period. Data on hospital admissions of Medicare patients were linked to detailed information on residency training of the physician for each hospital admission. We analyzed how quality and costs of inpatient care provided by physicians in their first year after completion of their internal medicine residency (ie, newly independent internists) varied across cohorts completing residency during 2000-12. Our goal was to examine whether physicians who were exposed to the 2003 ACGME work reforms during their residency had worse patient outcomes or higher costs of care in their first year of independent practice compared with physicians who completed residency before the reform.

Methods

Data sources and study population

We used Medicare 20% Carrier and Inpatient Files to identify fee-for-service beneficiaries aged 65 years or more who were admitted to acute care hospitals between 1 January 2000 and 30 December 2012. These data comprise a simple, 20% random sample of all fee-for-service Medicare beneficiaries in a given year.

To study hospital outcomes and costs of care among patients treated by general internists, we first used established methods to assign an attending physician to each hospital admission based on the physician National Provider Identifier (NPI) in the Carrier File that accounted for the most Part B spending (evaluation and management services, tests, procedures) during that hospital admission.22 23 24 25 We then restricted our analysis to those admissions in which the assigned physician was a general internist. General internists were identified using specialty information in a comprehensive physician database obtained from Doximity, a company that provides online professional networking services for US physicians. The database includes information on all US physicians (both registered members of the service and non-registered physicians) obtained from several sources and data partnerships, including the US Department of Health and Human Services NPI Registry, specialty boards (eg, the American Board of Medical Specialties),26 state medical boards, and hospitals and medical schools that share data with Doximity on their current staff, students, and alumni. The database includes information on residency training (type of residency and year of completion) and subsequent specialty training. We focused on hospital admissions in which the assigned physician completed internal medicine residency without further specialty training. Physician NPI was used to match between Medicare and Doximity databases. Details of the database and its validation are published elsewhere.23 26 27

Although patient outcomes and costs of care are influenced by all providers involved in a hospital admission (some of whom may bill for care provided (eg, consulting physicians), and others who may not (eg, overnight cross coverage provided by an internist distinct from the assigned attending physician)), on the basis of previous studies we attributed patient outcomes and costs of care to a single physician.22 23 24 25 28 29

Outcome measures

Outcome measures included 30 day mortality from date of hospital admission, 30 day readmission from date of hospital discharge, and overall Medicare Part B spending for a given hospital stay, adjusted to 2012 dollars. We used Part B spending as a measure of overall costs of care because it includes professional and other services at the discretion of physicians (eg, specialty consultations, interpretation of imaging and laboratory testing by radiologists and pathologists, and procedures), and for a given hospital stay, Part A spending, which comprises most inpatient spending, is largely invariant to physician decisions owing to the fixed diagnosis related group payment. Part B spending proxies for the intensity of resource use by physicians, and thus likely correlates with other inpatient resource use (eg, imaging, laboratory testing) encompassed in the Part A payment).23 24

Patient characteristics

Patient demographics were obtained from Beneficiary Summary Files and included age, sex, race or ethnic group (non-Hispanic white, non-Hispanic black, Hispanic, and other). Each hospital stay was categorized by diagnosis related group to account for reason for hospital admissions. Using the Chronic Condition Warehouse database, we captured the presence of any of 27 chronic conditions.

Analysis: overview

We sought to examine whether physicians who were exposed to the 2003 ACGME reform during their residency had worse patient outcomes or higher costs of care in their first year of independent practice compared with physicians who completed residency before the 2003 reform. For each calendar year during 2000-12, we identified all hospital admissions in that year in which the assigned physician was an internist who completed residency in the previous year. For example, in 2003 we identified all hospital admissions in which the assigned physician completed internal medicine residency in 2002. Similarly, in 2007, we identified hospital admissions in which the assigned physician completed internal medicine residency in 2006. Physicians who completed their internal medicine residency in 2004, 2005, or 2006 onwards therefore had one, two, or three years of exposure, respectively, to the 2003 ACGME work hour changes during their residency.

Importantly, mortality of Medicare beneficiaries admitted to hospital declined over this period,30 implying that mortality of patients cared for by first year physicians would also be expected to decline each year. This could spuriously suggest that physicians with greater exposure to the 2003 ACGME reforms during their residency (ie, those who completed residency in more recent years) have lower patient mortality as a result of residency reforms. To account for declining trends in hospital mortality, we employed a difference-in-difference study design.28 For each calendar year, we identified all hospital admissions in that year in which the assigned physician was an internist who had completed residency 10 years previously (eg, in 2002 and 2007 we identified hospital admissions in which the assigned physician completed internal medicine residency in 1992 or 1997, respectively). Patients treated by these 10th year physicians served as a control group to account for nationwide trends in costs, mortality, and readmissions. The assumption behind this difference-in-difference approach is that overall trends in hospital care during 2000-12 impacted physicians of all levels of experience equally and that differences in trends between first year and 10th year physicians after the 2003 ACGME work hour reform could plausibly be attributable to the reform itself rather than to nationwide trends in hospital care.

Statistical analysis

Because patients treated by 10th year internits served as a control group for patients treated by first year internists, we began by comparing characteristics of patients treated by both groups of physicians. We tested for observable differences in patient demographics and pre-existing chronic conditions using t tests and z tests for proportions, as appropriate. The probability distributions of diagnoses (diagnosis related groups) for which patients were admitted was also compared between both groups of physicians. This analysis, which has been used in previous studies,31 32 was conducted to demonstrate similarity between first and 10th year physicians in why patients were admitted to hospital.

We then estimated hospital admission level multivariable linear models in which dependent outcomes were 30 day mortality, 30 day readmissions, and Part B spending. Owing to a failure of logistic regressions to converge with indicator variables for more than 800 diagnosis related groups and 5000 hospitals, we estimated linear models for binary mortality and readmissions outcomes.33 34 Independent variables included an indicator variable for whether the physician was a first year one (reference, 10th year physician), indicator variables for each year from 2000 to 2012, interaction terms between first year attending physician indicator variable and year indicator variables, patient characteristics (age, sex, race, diagnosis related group, and chronic condition indicators), length of stay, and hospital indicator variables, otherwise known as hospital fixed effects, to account for time invariant hospital characteristics (eg, teaching hospital status or other unmeasured but time invariant differences in patient populations across hospital regions) that may be correlated with patient outcomes. The inclusion of hospital fixed effects thereby allowed for comparison of patient outcomes between physicians in the same hospital who had varying exposure to work hour reforms during their residency.

We also adjusted for whether the attending physician for a given hospital admission was a hospitalist physician (indicator variable), since graduates of recent residency cohorts might be more likely to work as hospitalists and the hospitalist model of care has been associated with lower mortality.35 36 37 Hospitalists were defined by a validated approach: general internists with at least five evaluation-and-management billings in a given year who filed at least 90% of their total evaluation-and-management billings in inpatient setting as defined by Current Procedural Terminology codes 99221-99223, 99231-99233, and 99251-99255).38

To describe the estimated effects of the 2003 ACGME reform on quality and costs of care provided by first year physicians, we reported adjusted 30 day mortality, 30 day readmissions, and Part B spending for first and 10th year physicians each year from 2000 to 2012. Differences in trends between first and 10th year physicians beginning after 2006 (and plausibly after 2003 for cohorts partially exposed to the work hour reforms) would be consistent with effects of the ACGME reform on quality and costs of care provided by first year physicians. In addition to this approach, we estimated a difference-in-difference model in which the first year attending indicator was interacted with a single post-2006 indicator rather than year specific indicators, adjusting for the covariates previously described and separate time trends for first and 10th year physicians.

Additional analyses

We conducted several additional analyses. Firstly, in sensitivity analyses we analyzed 90 day mortality and readmissions to allow for longer follow-up of patients after hospital admission. Secondly, we conducted an analysis that excluded New York state, which implemented residency work hour reforms before the ACGME’s 2003 reform. Thirdly, we conducted a separate difference-in-difference analysis that compared the post-2006 period with the pre-2003 period since the intervening period consisted of variable exposure of physicians to reduced work hour environments (ie, some physicians were exposed to reduced work hours for one, two, or three years). Fourthly, because spending could be skewed, we re-estimated our baseline models with logarithmic transformations of spending. Fifthly, to study the longer term effects of physician exposure to work hour reforms, we analyzed outcomes and costs of care of physicians who in any calendar year had completed residency two years previously (or in an additional analysis, three years previously) as opposed to first year physicians in our baseline analysis. Finally, we re-estimated our baseline models using block bootstrapped standard errors.

All analyses were conducted using STATA version 14 (STATA, TX). Standard errors were clustered at the hospital level. A two sided P≤0.05 was considered to indicate statistical significance.

Patient and public involvement

No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community.

Results

Our sample included 258 543 patients admitted to hospital and treated by first year internists (treatment group) and 227 142 patients admitted to hospital and treated by 10th year internists (control group). Patient demographics and pre-existing chronic conditions were similar between both groups over the study period (table 1), with statistically significant differences being clinically small in magnitude and not in any systematic direction. The cumulative distributions of diagnosis related group categories were not statistically different between both groups of physicians (supplementary figure 1), suggesting that reasons for admission were similar between the two groups.

Table 1

Characteristics of study population by whether the treating physicians were exposed to work hour reforms during their residency. Values are percentages (numbers) unless stated otherwise

Characteristics	1st year internists (treatment group)		10th year internists (control group)	P value*
No of hospital admissions	258 543		227 142
Mean (SD) age (years)	75.6 (14.3)		75.2 (14.4)	<0.001
Women	59.3 (153 316)		59.0 (134 014)	0.06
White patients	82.2 (212 522)		82.6 (187 619)	0.16
Mean (SD) No of chronic conditions†	6.06 (2.1)		5.91 (1.9)	<0.001
Chronic conditions†
Coronary artery disease	55.1 (142 457)		53.1 (120 612)	<0.001
Alzheimer’s dementia	26.0 (67 221)		25.5 (57 921)	0.02
Atrial fibrillation	18.9 (48 865)		18.5 (42 021)	0.03
Chronic kidney disease	30.3 (78 339)		31.4 (71 323)	<0.001
Chronic obstructive pulmonary disease	32.9 (85 061)		30.9 (70 187)	<0.001
Diabetes	39.8 (102 900)		38.6 (87 677)	<0.001
Congestive heart failure	44.7 (115 569)		43.1 (97 898)	<0.001
Hyperlipidemia	44.8 (115 827)		43.5 (98 807)	<0.001
Hypertension	76.9 (198 820)		75.5 (171 492)	<0.001
Past stroke/transient ischemic attack	13.9 (35 937)		13.6 (30 891)	0.01
Cancer	12.7 (32 835)		12.3 (27 938)	0.01

Estimated using two sample t tests or z tests for proportions, as appropriate.

Presence of chronic illness assessed using indicators from the Chronic Condition Warehouse File. “Cancer” includes presence of any of breast, endometrial, prostate, or colon cancer.

Physicians’ exposure to work hour reform during their residency was not associated with statistically significant differences in 30 day mortality, 30 day readmissions, or inpatient spending. Overall, 30 day mortality rates for first year internists during 2000-06 and 2007-12 were 10.6% (12 567 deaths/118 014 hospital admissions) and 9.6% (13 521/140 529), respectively, and for the control group of 10th year internists were 11.2% (11 018/98 811) and 10.6% (13 602/128 331), for an adjusted difference-in-difference effect of physician exposure to work hour reform on subsequent 30 day patient mortality of −0.1 percentage points (95% confidence interval −0.8% to 0.6%, P=0.68) (table 2).

Table 2

Post-training outcomes in a difference-in-difference analysis of physicians exposed or not exposed (control group) to 2003 Accreditation Council for Graduate Medical Education work hour reforms

Unadjusted outcomes by period	1st year internists (treatment group)	10th year internists (control group)		Difference-in-difference change		P value for adjusted change
				Unadjusted, percentage point difference	Adjusted, percentage point difference (95% CI)
30 day mortality (% (No))
2000-06	10.6 (12 567/118 014)	11.2 (11 018/98 811)		-	-	-
2007-12	9.6 (13 521/140 529)	10.6 (13 602/128 331)		−0.5	−0.1 (−0.8 to 0.6)	0.68
30 day readmissions (% (No))
2000-06	20.4 (24 074/118 014)	20.1 (19 840/98 811)		-	-	-
2007-12	20.4 (28 689/140 529)	20.5 (26 277/128 331)		−0.4	0.1 (−0.9 to 1.1)	0.87
Medicare Part B spending ($)
2000-06	1161	1332		-	-	-
2007-12	1267	1599		−161	−46 (−94 to 2)	0.06

Overall, 30 day readmission rates for first year internists during 2000-06 and 2007-12 were 20.4% (24 074/118 014) and 20.4% (28 689/140 529), respectively, and for 10th year internists were 20.1% (19 840/98 811) and 20.5% (26 277/128 331), for an adjusted difference-in-difference effect of physician exposure to work hour reform on subsequent 30 day patient readmissions of 0.1 percentage points (−0.9% to 1.1%, P=0.87). Medicare Part B inpatient spending for first year internists during 2000-06 and 2007-12 was $1161 (£911; €1024) and $1267 per hospital admission, respectively, and for 10th year internists was $1331 and $1599, for an adjusted difference-in-difference effect of physician residency exposure to work hour reform on subsequent inpatient spending of −$46 (95% confidence interval −$94 to $2, P=0.06).

Similar findings were observed for 90 day mortality and readmissions outcomes (supplementary table 1) and in year-by-year analyses of mortality, readmissions, and costs of care. Specifically, no changes in 30 day mortality or readmissions were observed for first year internists before versus after 2006 (the year in which the first cohort of physicians that was fully exposed to new work hour reforms completed residency) or for the control group of 10th year internists (fig 1; unadjusted rates in supplementary table 2). For each year, the difference in mortality (and readmissions) between first and 10th year internists was not statistically significantly different from the difference between first and 10th year internists in 2003, P>0.15 for all year comparisons in a formal test of interactions.

Fig 1

Adjusted trends in 30 day mortality and readmissions for patients whose physicians were in their first year after residency completion versus control group of 10th year physicians. Dotted line reflects year in which all subsequent internal medicine residency cohorts were exposed fully to the 2003 Accreditation Council for Graduate Medical Education work hour reforms during their residency (ie, for three full years of residency training). For each year and for both mortality and readmissions models, coefficients on yearly difference-in-difference interactions (ie, outcomes difference between first and 10th year internists) were not significant (P>0.15) compared with 2003 difference

Trends in inpatient spending were also similar during 2000-12 between patients treated by first year internists versus the control group of 10th year inernists (fig 2, unadjusted spending in supplementary table 2). In particular, both physician groups exhibited similar spending trends before versus after the implementation of work hour reforms in 2006.

Fig 2

Adjusted trends in Medicare Part B spending for patients whose attending physicians were in their first year after residency completion versus control group of 10th year physicians. Dotted line reflects year in which all subsequent internal medicine residency cohorts were exposed fully to the 2003 Accreditation Council for Graduate Medical Education work hour reforms during their residency (ie, for three full years of residency training). For each year, coefficients on yearly difference-in-difference interactions (ie, Medicare Part B inpatient spending difference between first and 10th year internists) were not significant (P>0.15) compared with 2003 difference

Similar findings were observed in additional analyses (supplementary tables 3-5 and supplementary figures 2 and 3), including an analysis that excluded New York state; a separate difference-in-difference analysis that compared the post-2006 period with the pre-2003 period; an analysis with logarithmic transformation of spending; an analysis of outcomes and costs of care of physicians who in any calendar year had completed residency either two years or three years previously (as opposed to first year physicians in our baseline analysis); and an analysis in which confidence intervals were calculated using blocked bootstrapped standard errors.

Discussion

We analyzed whether exposure of physicians to work hour reforms during residency training was associated with post-training differences in patient mortality, readmissions, or costs of care. Using national data from Medicare, we found that physicians who were exposed to the 2003 work hour reform during their residency had no subsequent differences in 30 day mortality, 30 day readmissions, or costs of care after completion of their residency compared with physicians who completed their residency before the reform.

The potential impact of resident work hour reform on the competency of physicians completing residency has been much discussed in medical education.4 39 40 41 42 Because reductions in work hours and increased shift work could theoretically lead trainees to be less prepared for independent practice, understanding the implications of work hour reforms on the quality and costs of care provided by physicians entering independent practice is important. Although studies have examined whether reductions in residency work hours are associated with changes in mortality in teaching versus non-teaching hospitals,14 15 16 17 18 19 20 only one study has explored whether exposure of physicians to the 2003 Accreditation Council for Graduate Medical Education (ACGME) reform during their residency was associated with worse post-training clinical outcomes of these physicians. Using data on all internal medicine hospital admissions in Florida during 2000-09 linked to year of residency completion of the attending physician, that study found no relation between physicians’ exposure to the 2003 ACGME reform during their residency and inpatient mortality and length of stay in subsequent independent practice.28 That study was limited, however, by analysis of only Florida hospitals and inability to study post-discharge mortality, readmissions, or costs of care.

Several factors should be considered in interpreting our findings. Firstly, although reduced work hours and greater shift work could lead to less cumulative clinical experience during residency, these might be offset by greater consolidation of clinical knowledge mediated by reductions in resident fatigue and increased teaching by residency programs. Alternatively, if the requirement to maintain work hours below a threshold leads residents to miss periods of formal teaching, this could have the opposite effect. Ultimately, the incremental effect of reducing work hours on acquisition of clinical knowledge and experience is unknown and is an empirical question. Secondly, a potential concern with our analysis is that hospitals increasingly employed hospitalist physicians over our study period.43 Although care by hospitalists has been associated with lower mortality and costs, and hospitalists are disproportionately comprised of new residency graduates, our analyses adjusted for whether care was provided by a hospitalist physician.35 36 37 Thirdly, a potential concern with our analysis is that the 2003 ACGME reform led many hospitals to modify inpatient care models to include advanced practitioners, which might mitigate the adverse effects of less experienced first year physicians.44 A confounder such as this should be addressed by our difference-in-difference strategy as long as advanced practitioners had similar beneficial effects on both first and 10th year internists (our control group) within a hospital. Fourthly, it is possible that first year internists were increasingly employed by large teaching hospitals over our study period, which could bias our findings toward the null if teaching hospitals experienced differentially greater improvements in quality than non-teaching hospitals. Our analysis included hospital fixed effects, however, which would capture differences such as these provided first and 10th year internists were similarly affected within a hospital. Nonetheless, it is still possible that factors such as increasing use of advanced practitioners, multidisciplinary teams, electronic health systems, and inpatient pharmacists might have improved outcomes and lowered costs among first year internists but not among older physicians, which would bias our findings toward the null.45 Indeed, because hospital care is often the product of team work, the impact of any single physician in influencing patient outcomes and costs of care is muted by the effect of the team. This might not only help explain our findings but might also serve as a justification for future training reforms, which should recognize that the role of individual physicians in delivering care is mitigated by the increasing role of teams and technological advances in how medical care is delivered.

Strengths and limitations of this study

Our study has several additional limitations. Firstly, our analysis focused on 30 day mortality and 30 day readmissions, and in additional analysis on 90 day outcomes. Though mortality and readmissions are important quality measures and include the influence of inpatient care on outcomes after discharge, more refined process and outcome quality measures are important to consider. Previous studies examining the impacts of residency work hour reforms have, however, focused on mortality outcomes.14 15 16 17 18 19 20 Secondly, we used inpatient Medicare Part B spending to analyze whether ACGME reforms were associated with increased costs of care provided by new attending physicians. Though Part B spending captures professional service costs that proxy for overall inpatient resource use (including specialist consultations, interpretation of studies by radiologists and pathologists, and procedures), Part B spending cannot account for differential use of medications, imaging modalities, and laboratory testing, which are included in the Part A fixed diagnosis related group payment. Thirdly, our analysis was limited to quality and costs of hospital care rather than outpatient care. Because internal medicine residency training is so heavily focused on the development of inpatient medical management skills, it is possible that work hour reductions would have had limited impact on physician performance in this domain but a larger impact on outpatient performance or other areas less emphasized in the internal medicine training curriculum. Fourthly, our analysis focused on the impact of the 2003 ACGME reform because data were unavailable on new attending physicians who completed a full three year internal medicine residency after the 2011 reform. Fifthly, our baseline difference-in-difference approach was premised on the assumption that reductions in work hours occurred sharply in the first year of the 2003 ACGME reform. Hospitals in at least one state—New York—enacted work hour reforms before the ACGME reform, and other hospitals reported delays in adherence to the new work hour regulations.46 We conducted a year-by-year analysis to more directly visualize how specific cohorts of physicians might have been differentially affected by exposure to reduced work hour environments during their own residency training. We also conducted a sensitivity analysis that excluded New York state. Importantly, previous studies of work hour reforms have primarily assumed that exposure of physicians to reduced work hours began in 2003. Finally, our study did not analyze outcomes in surgery, where reductions in operative volume during residency have raised concerns about inadequate preparation for unsupervised practice after surgical residency.47 48

Conclusions and policy implications

Exposure of physicians to work hour reforms during residency training was not associated with post-training differences in patient mortality, readmissions, or costs of care. Further assessment of the impact of both the 2003 and the 2011 residency work hour reforms on other aspects of physician quality and in other specialties is important.

Concerns exist that residency work hour reforms in the United States, which have led to reductions in work hours and greater shift work during residency training, have lowered preparation of physicians for independent practice after residency

Post-training patient outcomes and costs of care of physicians who were exposed to the 2003 US Accreditation Council for Graduate Medical Education (ACGME) work hour reforms during their residency are unknown

Exposure of physicians to the 2003 ACGME work hour reforms during their residency was not associated with statistically significant differences in 30 day mortality, 30 day readmissions, or inpatient spending

These findings suggest that the large reductions in residency work hours in the US were not associated with declines in the quality of physician training