The impact of duty hours on resident self reports of errors.

BACKGROUND: Resident duty hour limitations aim, in part, to reduce medical errors. Residents' perceptions of the impact of duty hours on errors are unknown.
OBJECTIVE: To determine residents' self-reported contributing factors, frequency, and impact of hours worked on suboptimal care practices and medical errors.
DESIGN: Cross-sectional survey.
SUBJECTS: 164 Internal Medicine Residents at the University of California, San Francisco. MEASUREMENTS AND
RESULTS: Residents were asked to report the frequency and contributing factors of suboptimal care practices and medical errors, and how duty hours impacted these practices and aspects of resident work-life. One hundred twenty-five residents (76%) responded. The most common suboptimal care practices were working while impaired by fatigue and forgetting to transmit information during sign-out. In multivariable models, residents who felt overwhelmed with work (p = 0.02) and who reported spending >50% of their time in nonphysician tasks (p = 0.002) were more likely to report suboptimal care practices. Residents reported work-stress (a composite of fatigue, excessive workload, distractions, stress, and inadequate time) as the most frequent contributing factor to medical errors. In multivariable models, only engaging in suboptimal practices was associated with self-report of higher risk for medical errors (p < 0.001); working more than 80 hours per week was not associated with suboptimal care or errors.
CONCLUSION: Our findings suggest that administrative load and work stressors are more closely associated with resident reports of medical errors than the number of hours work. Efforts to reduce resident duty hours may also need to address the nature of residents' work to reduce errors.

INTRODUCTION

In July 2003, the Accreditation Council for Graduate Medical Education (ACGME) mandated that all residency programs limit their duty hours to fewer than 80 per week. Duty hour limitations were catalyzed by heightened public and governmental concern around issues of patient safety.1,2 The limitations affect 8000 ACGME accredited training programs, their 100,000 trainees,3 and the 16 million patients admitted to teaching hospitals each year.4

Although it seems logical that limiting residency duty hours might improve patient safety, little evidence supports this hypothesis. A study conducted over a decade ago, after New York State implemented duty hour reductions, found delays in test ordering and an increase in hospital complications.5 More recent reviews, moreover, found no evidence of improved mortality6 or patient safety7 after duty hours were reduced. Taken together, these studies suggest that resident errors may be due to factors other than long duty hours, but few data exist regarding residents’ perceptions of how errors are produced or how duty hour reductions may have affected these contributing factors.

To understand the contributing factors and frequency of residents’ perception of suboptimal care practices and errors as well as the impact of duty hours on those practices, we surveyed residents in a large university-affiliated training program after duty hours were reduced. We hypothesize that factors other than long duty hours are associated with resident self-perceived errors.

METHODS

Sites and Subjects

Descriptions of the survey protocol have been published previously.8 We performed our study at the University of California, San Francisco (UCSF) internal medicine training program. Residents rotate through 3 clinical sites: the San Francisco Veterans Affairs Medical Center, San Francisco General Hospital (a municipal teaching hospital), and Moffitt-Long Hospital (an academic medical center). All 164 internal medicine residents at UCSF were eligible for the study, which was approved by the university’s institutional review board.

In February 2003, each of the 3 hospitals implemented changes designed to reduce duty hours and comply with the ACGME mandates. Data collected for internal purposes from each site before the change showed that residents were routinely working more than 80 hours per week. These changes, designed with resident and faculty input, varied slightly from site to site, but included the use of daytime cross-coverage residents (“day float”) to provide coverage for postcall teams (thereby facilitating early departure of teams), overnight cross-coverage (“night floats”) to relieve the on-call team from early morning admissions and provide coverage of patients not cared for by the team, and early sign-out times facilitated and enforced by the chief residents at each site.

Survey Development

Through direct observation of residents working at Moffitt-Long hospital for 1 month, investigators identified salient domains relevant to residents’ workload, patient care practices, and errors consistent with others in the literature.9–12 We then developed a survey, which included open-ended questions within these domains. This survey was completed by a pilot group of UCSF residents ineligible for the study. The most common results from open-ended questions were categorized in a response format. To further establish content validity, we posed this survey to experts in the fields of medical education, outcomes research, patient safety, and psychometrics, after which items were reworded, reformatted, or eliminated. As a final check of usability and clarity, we pretested the survey in a second sample of noninternal medicine house staff at UCSF and recent residency graduates outside of UCSF.

Survey Measures

Suboptimal Patient Care Practices

We defined suboptimal care practices as processes that are clearly substandard but that might not necessarily lead to an error or adverse event. Specific practices assessed with a 5-point Likert scale (“never” to “very often”) included: forgetting to transmit important information during sign-out, making up information to give to a superior, working while impaired by fatigue, writing information in a patient’s chart that one is unsure of, or reporting information that one is unsure of.

Medical Errors

Factors contributing to resident errors were assessed by asking residents how often they made errors due to each of the following factors: fatigue, excessive workload, inadequate knowledge, inadequate supervision, distractions, stress, and inadequate time. The frequency of errors was assessed by the question “How often do you make the following types of errors?” with the following responses: cognitive (e.g., wrong diagnosis), technical (e.g., pneumothorax), and administrative (e.g., note in the wrong chart). Responses to these assessments were made on a 5-point scale ranging from “never” to “very often.”

Work Experience

The survey included a question about the amount of time spent doing tasks that could be completed by a nonphysician and a question about the frequency of feeling overwhelmed at work on a 5-point scale ranging from “never” to “very often.” Questions also addressed general issues about the duty hour reduction system changes including: timing of the last inpatient ward rotation, awareness and perceived success of the new system changes to reduce duty hours, and the number of hours worked per week.

Perceptions of Duty Hour Reduction

The impact of duty hour reduction on the chance of committing medical errors was assessed with a question asking what impact has the work hour reduction system had on the chance of committing medical errors with the following responses: “negative”, “no impact”, and “positive impact.”

Survey Methods

To ensure that all residents would have experienced duty-hour reductions by the time they filled out the questionnaire, we surveyed internal medicine residents beginning 1 month after duty hours were reduced and continuing for 4 months afterward. To maximize response rates, we mailed each survey to the residents’ home addresses with a $3 financial incentive, and sent multiple follow-up reminders to nonresponders via e-mail, postal mail, and in conferences. Surveys were randomly coded to track response rates; these codes were removed before data entry to ensure confidentiality. Data were entered by a professional vendor, and double keyed to ensure accuracy. Participation was voluntary and consent was implied with the return of the survey.

Statistical Analysis

To characterize the distribution of residents’ responses to survey questions, we first used univariate statistics and then conducted factor analysis among questions assessing suboptimal patient care and errors. Highly correlated questions loaded on the same factor were averaged; the average of correlated scores was termed a “summary score.”

Bivariate associations among variables were assessed with correlation analyses and t tests. Multivariate linear regression models identified factors associated with self-reports of suboptimal patient care practices and medical errors. We selected variables for entry into models based on our a priori hypothesis regarding factors related to suboptimal care practices and medical errors, observed relationships among variables, or to retain face validity of the model, and all variables were retained. Covariates in multivariable models included: age (>30 vs ≤ 30), sex, postgraduate year (PGY) level (PGY 1, PGY2 vs PGY3), perceived percent time spent on administrative tasks (non-physician tasks: less than 50% vs greater than 50%), the frequency with which residents felt overwhelmed at work, and the number of hours residents reported working per week (less than 80 hours vs 80 hours or more per week). All analyses were performed using SAS version 8.12 (SAS Institute Inc, Cary, NC.).

RESULTS

Respondent Characteristics

One hundred sixty-four eligible residents received surveys, and 125 (76%) responded. Sex, level of training, and type of program were similar between respondents and the small number of nonrespondents (p > 0.2, p > 0.45, p > 0.6, respectively). Respondents were equally distributed among year of training (PGY-1: 36.6%, PGY-2: 35.8%, PGY-3: 27.6%).

Most respondents were female (60%), enrolled in the categorical residency track (62%), and younger than 30 years of age (70%). All respondents had experienced an inpatient rotation before and after duty hours were reduced and all were aware of the system changes intended to reduce duty hours to fewer than 80 per week. Sixty percent (n = 72) agreed that the new system succeeded in keeping duty hours under 80 per week. The vast majority [93% (n = 116)] felt that residents should work fewer than 80 hours per week. After duty hours were reduced, 35% (n = 44) of residents reported working more than 80 hours a week, 18% (n = 22) of residents reported spending greater than 50% of their time doing nonphysician tasks, and 20% (n = 24) reported feeling overwhelmed at work “fairly often” or “very often.” Residents’ reports on the impact of duty hour reduction on errors were split, with 45% stating that it had had no impact and 45% stating that it had had a positive impact.

Resident Reports Regarding Frequency and Predictors of Suboptimal Patient Care Practices

Residents reported working while impaired by fatigue and forgetting to transmit important information during sign-out as the most common of 5 suboptimal patient care practices (Table 1). Factor analysis of those 5 practices loaded onto the same factor and these items were combined to form a summary score (Cronbach αα  = 0.75) with a mean of 2.16 (SD 0.54, range 1–5), with higher scores representing a greater likelihood of reporting a suboptimal care event. In multivariable models, factors independently associated with a greater likelihood of suboptimal practices included spending greater than 50% of one’s time in non-MD tasks (p = 0.02, 0.29 points more likely) and feeling overwhelmed at work (p < 0.001, 0.18 points more likely) (Table 2).

Table 1

Self-Reported Reasons for and Frequency of Suboptimal Care Practices and Errors*

Variable (survey question and response)	Mean (SD)
Frequency of engaging in common suboptimal patient care practices
During your most recent inpatient rotation work-week, how often did you...?
Work while impaired by fatigue	3.73 (0.85)
Forget to transmit important information during sign-out	2.13 (0.69)
Report information that you were unsure of	1.92 (0.81)
Write information in a patient’s chart that you were unsure of	1.74 (0.781)
Make up information to report to your superior	1.13 (0.54)
Suboptimal patient care practices summary score* (summary of the previous 5 responses, Eigenvalue = 2.5 ,Cronbach αα = 0.75)	2.16 (0.54)
Reasons for medical errors
How often do you make errors because of...?
Fatigue	3.11 (0.77)
Excessive workload	3.11 (0.84)
Inadequate time	3.04 (0.89)
Distractions	2.77 (0.86)
Stress	2.57 (0.91)
Work stress summary score (summary of the previous 5 responses, eigenvalue = 3.1, Cronbach’s αα = 0.85)	2.92 (0.67)
Inadequate knowledge	2.67 (0.62)
Inadequate supervision	2.12 (0.65)
Intellectual stress summary score (summary of the previous 2 responses, Eigenvalue 1.6, Cronbach αα = 0.60)	2.39 (0.54)
Frequency of medical errors
How often do you make the following errors?
Cognitive	2.67 (0.61)
Administrative	2.34 (0.73)
Technical	1.98 (0.50)
Medical error summary score (summary of the previous 3 responses Cronbach’s αα = 0.60)	2.32 (0.05)

*All items were asked on a 5-point scale ranging from 1 (never) to 5 (very often).

Table 2

Predictors of Self-Reported Suboptimal Patient Care Practices and Errors

	Suboptimal patient care practice*		Medical Error*
Variable	Parameter estimate (β)	P value	Parameter estimate(β)	p value
PGY-1	0.1771	0.1770	0.0204	0.8310
PGY-2	0.0379	0.7535	0.0401	0.6471
Working >80 hrs per week
(coded as yes = 1, no = 0)	0.0475	0.6330	0.0567	0.4330
Non-MD tasks >50% of time			0.0585	0.5221
(coded as yes = 1, no = 0)	0.2885	0.0204
Frequency of being overwhelmed at work			0.0739	0.0945
(coded never = 1 to 5 = very often)	0.1838	0.0019
Frequency of engaging in suboptimal patient care practices*			0.4152	<0.0001

Parameter estimates (β) are controlled for age and sex, with scores less than 0 indicating less frequent engagement in the outcome measure.

*Frequency of engaging in suboptimal patient care practices included in regression model for predictors of medical errors only.

*Outcome variables based on summary scores as described in Methods.

Self -Reported Contributing Factors to Medical Errors

Fatigue and excessive workload were the most common of 7 factors contributing to medical errors (mean 3.11 SD 0.77, mean 3.11 SD 0.84, respectively). Factor analysis of these 7 items revealed 2 factors: “work stress” (5 items: fatigue, excessive workload, distractions, stress, and inadequate time, Eigenvalue 3.1, Cronbach α = 0.85), and “intellectual stress” (2 items: inadequate knowledge and inadequate supervision, Eigenvalue 1.6, Cronbach α = 0.60). Residents reported making medical errors due to factors related to “work stress” (mean = 2.92 SD = 0.67) more frequently than factors related to “intellectual stress” (mean 2.39 SD = 0.54, p < .0001; Table 1). Although all 7 individual factors were correlated with making medical errors overall (with correlation coefficients ranging from 0.30 to 0.44, all p < .001), the scale scores of “work stress” and “intellectual stress” were more highly correlated with the frequency of committing medical errors than the individual responses (r = 0.46, p < .0001; r = 0.50, p < .0001, respectively).

Resident Reports of the Frequency and Predictors of Committing Errors

Residents reported committing cognitive errors more frequently (mean 2.66, SD 0.61) than administrative (2.34, SD 0.73) and technical (mean 1.98, SD 0.50) errors (F = 46.4, p < .0001). The medical errors summary score of these responses yielded a mean of 2.32 (SD 0.0516, Cronbach α = 0.60). In multivariable models, controlling for age, sex, year of training, hours worked, and fraction of time on administrative tasks, only the frequency of suboptimal patient care practices was independently associated with self-reported frequency of medical errors (p < 0.0001) (Table 2).

DISCUSSION

In this cross-sectional survey of internal medicine residents, residents who worked more than 80 hours per week did not report that they were more likely to engage in suboptimal patient care practices or commit medical errors. Instead, we found that residents who reported spending greater than 50% of their time on nonphysician-oriented tasks and those who were overwhelmed at work were more likely to report engaging in such practices and, in turn, committing medical errors.

The ACGME mandated duty hour reduction in part due to concerns about patients’ safety.1 Although recent research reported increased attentional failures and medical errors with extended work shifts,13,14 our data suggest factors other than the number of hours worked may be contributing to errors.

Forgetting to transmit important information during sign-out was a frequent suboptimal patient care practice reported in our study. This highlights a concerning consequence of duty hour limitations—the increase in discontinuity, which may be associated with adverse events.15 Based on our residents’ self-reports, as well as prior suggestions by residents16 and experts,17,18 building effective systems and processes for sign-out will be needed to enhance safety in the context of limited duty hours.

In our cohort, increasing administrative burden and feeling overwhelmed were associated with perceived delivery of suboptimal patient care. Prior data have linked burnout to suboptimal patient care practices.11 We believe that our data corroborate these findings, since residents who are overwhelmed and spend much of their time in completing administrative tasks are likely to suffer burnout. This problem is not unique to our program—20% of our residents reported spending considerable time in administrative tasks, a level similar to that found in other residency programs.19,20 Our data also linked perceived suboptimal patient care practices to medical errors—a logical relationship, but one that has not been demonstrated in prior studies to our knowledge.

Residents in our study reported making medical errors due to what we term “work stress”—fatigue, excessive workload, distractions, stress, and inadequate time. Interestingly, inadequate knowledge (a factor associated with “intellectual stress” in our study) was not a common factor contributing to medical errors, as noted previously.12 We also found that the number of hours worked was not associated with the chance of committing medical errors, corroborating prior data from the surgical literature.21 We may be detecting the effects of a changing landscape of residency, with the main stressors no longer relating to concerns about knowledge deficits, but rather to increased workload due to high patient turnover,22 distractions from pagers,8 inadequate time to complete tasks due to duty hour mandates, and fatigue and stress that persist even after duty hours reform.23–25

Although residents in our cohort were split in their response to the question “What is the impact of decreased duty hours on your chance of committing an error?”, we found that the number of hours was not associated with self-perceived errors. This contradiction, we believe, represents the multifaceted determinants (above simply hours worked) of residents’ perception of the impact of duty hour reduction. It is plausible that those residents who felt that decreased duty hours had a positive effect on the chance of committing errors may have been more efficient and less overwhelmed, and thus less likely to perceive making errors. A similar argument could also explain the half of residents who felt duty hour reduction had a negative impact on their chance of committing errors. Our regression models, which found no relationship between duty hours and perceived errors, are also a more sensitive measure than the answer to a single question.

Our study has several limitations. Because we performed it at a university-based training program, our results may not be directly applicable to training programs in private or community-based hospitals. Because we surveyed only internal medicine residents, our findings may not extend to other trainees. As a single system study, our results may have limited applicability to other settings, and the variability between the hospitals may have affected resident responses. However, our survey assessed the impact of a variety of “day float” and “night float” interventions implemented at our 3 hospitals. The approaches chosen at UCSF sites are very similar to those being attempted elsewhere in a variety of disciplines.26–28 The variability between hospitals mirrors those found in many other systems, thus increasing our study’s generalizability. Although residents’ recollections may be subject to recall bias, our study sought to minimize this bias by administering the survey soon after the duty hour changes, and by asking questions that explicitly framed the comparison we hoped to assess. Although it is unclear how reliable resident self-report may be as the mechanism for detecting medical errors, many errors studies do use self-reports as one of the measures, particularly since more direct measures are difficult to come by. Moreover, we believe that resident perception of errors is important, as it is likely to influence their attitudes, clinical practices, and willingness to participate in error reduction activities. Although statistically significant, the magnitude of the associations between some contributing factors and resident’s reports of errors were relatively small. However, if our results were replicated over many residency programs and over many patient-resident encounters, we believe the magnitude of associations would be greater. Finally, although we did not validate residents’ perceptions with direct measures of workload, our findings are consistent with existing time-motion studies of residents’ daily tasks.9,29

Duty hour limitations were intended to reduce fatigue and thereby resident errors. Although duty hour reduction undoubtedly has many benefits, it is unclear whether current incarnations of residents’ schedules and program changes are leading to fewer errors. Our data suggest that reform efforts that include changes to the nature of resident work as well as the number of hours residents work may have a positive impact on resident errors.