Assessing the impact of the COVID-19 pandemic on clinician ambulatory electronic health record use.

OBJECTIVE: The COVID-19 pandemic changed clinician electronic health record (EHR) work in a multitude of ways. To evaluate how, we measure ambulatory clinician EHR use in the United States throughout the COVID-19 pandemic.
MATERIALS AND METHODS: We use EHR meta-data from ambulatory care clinicians in 366 health systems using the Epic EHR system in the United States from December 2019 to December 2020. We used descriptive statistics for clinician EHR use including active-use time across clinical activities, time after-hours, and messages received. Multivariable regression to evaluate total and after-hours EHR work adjusting for daily volume and organizational characteristics, and to evaluate the association between messages and EHR time.
RESULTS: Clinician time spent in the EHR per day dropped at the onset of the pandemic but had recovered to higher than prepandemic levels by July 2020. Time spent actively working in the EHR after-hours showed similar trends. These differences persisted in multivariable models. In-Basket messages received increased compared with prepandemic levels, with the largest increase coming from messages from patients, which increased to 157% of the prepandemic average. Each additional patient message was associated with a 2.32-min increase in EHR time per day (P < .001). DISCUSSION: Clinicians spent more total and after-hours time in the EHR in the latter half of 2020 compared with the prepandemic period. This was partially driven by increased time in Clinical Review and In-Basket messaging.
CONCLUSIONS: Reimbursement models and workflows for the post-COVID era should account for these demands on clinician time that occur outside the traditional visit.

INTRODUCTION

The COVID-19 pandemic triggered sudden and dramatic changes in many aspects of the health care delivery system, including clinician workflows and use of electronic health record (EHR) systems. In a matter of weeks, health care organizations shifted a substantial portion of their ambulatory care delivery to phone- and video-based telemedicine encounters and saw large reductions in emergency and hospital care. Clinicians still delivering face-to-face care were forced to restructure many aspects of their practice to ensure the safety of patients and staff.

The pandemic-driven shifts toward virtual treatment, and the corresponding change in patient expectations and awareness of communication tools such as secure messaging via the EHR, have substantially altered the nature of ambulatory care. Further, as telehealth and the pandemic incentivized patients to become familiar with the use of online portals to access their health information and connect with clinicians, the time required for those clinicians to manage the care of their patients through the EHR increased. The long-term shift to a mix of face-to-face and virtual care, as ambulatory patient volume returned in the second half of 2020, may have exacerbated these issues as clinicians delivered care across multiple modalities in a single day. Given that many of these changes, such as increased patient familiarity with asynchronous messaging, may persist beyond the COVID-19 pandemic, it is critical to evaluate how they have impacted clinician work.

Although near-universal adoption of EHRs was an important enabler of the shift to telemedicine, increased reliance on EHRs may exacerbate existing issues of low job satisfaction and poor well-being amongst clinicians. Many clinicians spend a significant amount of time working in the EHR, and clinicians in the United States already face a greater EHR burden than their international peers., Total time spent working in the EHR, “after-hours” time working outside of scheduled clinic hours, and responding to In-Basket messages without protected time or reimbursement for messages have been associated with a variety of negative impacts, including burnout, which can translate into higher costs and lower-quality care. Further, if the increased EHR burden is driven by activities such as messaging with patients, these new demands on clinician time may be concentrated on tasks that are outside of the bounds of the traditional “visit” and are currently nonreimbursable for most clinicians.

To address this topic, we used national, longitudinal EHR metadata to examine three research questions. First, how did ambulatory clinician EHR use change in response to the COVID-19 pandemic? Second, what changes were observed across different EHR functions? Finally, what specific EHR interactions are responsible for changes in EHR active-use time? Our findings have important implications for understanding ongoing changes in clinical work.

MATERIALS AND METHODS

Data

The sample for our longitudinal, observational study consisted of deidentified EHR metadata from 366 ambulatory care health systems in the United States that use Epic as their EHR software vendor. Epic has the largest ambulatory care market share in the United States with 33.4% of the market, and the sample represents nearly all of Epic’s ambulatory customer base. Health systems that adopted Epic during the study period, or who switched from Epic to another EHR vendor during the study period, were excluded from the analysis. The sample included physicians and advanced practice practitioners (eg, physician assistants and nurse practitioners). In the data, each organization represents an EHR installation, which may include more than one facility, such as a hospital with multiple ambulatory clinics, multiple hospital systems using a single EHR instance, or a chain of ambulatory clinics. Data were aggregated at the organization level, and all measures were calculated weekly. Not all organizations reported data in each week, and our final analytic sample is an unbalanced panel from December 29, 2019 through January 2, 2021 with 19 398 observations at the organization-week level. We conducted robustness tests for all analyses using a balanced panel of the 245 organizations that reported data in every week. This study using deidentified data aggregated at the health system level was deemed exempt from human subjects research by the Institutional Review Board at Stanford University.

EHR metadata collection

Clinician EHR activity was monitored by the software, which collects metadata extracted via Epic System’s Signal data aggregation tool. These metadata track activity including mouse movement, keystrokes, clicking, scrolling, and any other interaction with the EHR. EHR time is defined in Signal as the time the user was performing active tasks in the software, and the system stops counting if no interactions are conducted after 5 seconds. Although this is a conservative measurement of EHR time, it allows our study to focus on active EHR work while excluding time where the EHR may be open that the clinician is performing non-EHR tasks. However, our data is unable to capture clinician work that is generated by the EHR but not conducted through it, such as a phone call with a patient that is initiated by receiving an In-Basket message. Our measures should be interpreted with this context in mind.

EHR time measures

We measured EHR active-use time across four different functions., These functions were Clinical Review (eg, time spent reviewing patient history or test results), Notes (eg, time spent documenting), Orders (eg, time spent inputting orders, performing diagnostic association for orders), and In-Basket messaging (eg, time spent reading or writing messages or otherwise managing the In-Basket).

We defined after-hours time as any time between 5:30 pm and 7:00 am local time on weekdays and any time on weekends, unless the clinician was scheduled during those times. After-hours active-use time in the EHR per day was measured as time spent in the EHR on clinical work, excluding tasks related to research, data analysis, customization, or performance measurement. This measure was developed by Epic to measure after-hours time and is broadly consistent with the literature on after-hours EHR work.,

We measured mean In-Basket messages received per clinician per day a user logged in to the system, in total and by source. In-Basket messages in the Epic EHR can be generated from seven possible sources: team, results, prescription, patient, system, custom, and other. These categories were defined to be consistent with previous literature on clinician messaging. We measured both counts of messages received, as well as message volumes normalized to each organization’s prepandemic baseline—an 11-week period from December 29, 2019, to March 14, 2020.

Daily encounter volume and COVID-19 case count

We measured mean ambulatory encounter volume per clinician per day for each health system in each week of the study using EHR metadata from the Signal tool to capture our measure of daily volume. Finally, we included a measure of national confirmed COVID-19 case counts from the Centers for Disease Control COVID Data Tracker to provide context regarding the state of the pandemic in each week.

Statistical analysis

We calculated descriptive statistics for our two primary measures of EHR active-use time—total EHR time per day and after-hours time per day per clinician—for each week in the study period, comparing those figures to national COVID-19 case counts and daily encounter volume. To estimate the change in EHR active-use time and after-hours time after the onset of the pandemic, while controlling for daily encounter volume per clinician in each week as well as unobserved organization-level variation, we used ordinary least squares regression models in an event study design. We used the date of implementation of the nation’s first shelter-in-place order—in the state of California on March 19—as our proxy for the onset of the pandemic, with the week beginning (March 1, 2020) as our baseline prepandemic week. Our models included controls for encounter volume per clinician per day during the current week, organization-level fixed effects to control for any time-invariant organizational characteristics, and robust standard errors clustered at the organization level to account for serial autocorrelation over time.

We calculated descriptive statistics for EHR active-use time per day in each of our four functions: In-Basket, Clinical Review, Orders, and Notes. We calculated descriptive statistics for mean In-Basket messages received per day, in total as well as by source, for each week. To facilitate interpretation, we normalized message volume by each organization’s prepandemic message volume during the 11-week period from December 29, 2019, to March 14, 2020.

To estimate the impact of messages by type on clinician EHR time, we ran an ordinary least squares regression model, with clinician time spent in In-Basket per day as the dependent variable, and number of messages of each type (system, team, results, prescription, patient, custom, and other) as the independent variables of interest. The model also included organization fixed effects and robust standard errors clustered at the organization level. We conducted a robustness test including week fixed effects to control for any secular changes during the pandemic. Finally, for all analyses, we conducted robustness tests with a balanced panel including only health systems that reported data in every week.

Limitations

Our study has limitations. First, our dataset only includes scheduled ambulatory appointments with physicians and advanced practice practitioners. Due to data limitations, we were unable to evaluate inpatient or emergency settings, nor were we able to consider the impact of the pandemic on other clinicians, such as registered nurse (RN) or medical assistants, who are critical members of care teams and may face significant pandemic-driven EHR burdens. Second, our measure of EHR active-use time is conservative and narrowly defined, counting only the time clinicians spent actively using the EHR. Our study may therefore underestimate the impact of COVID on clinician work, as time spent helping patients connect to telemedicine calls and reading patient histories without clicking in the EHR for 5 s was not captured in our measures of active-use time. We also lack data on clinician scheduling, making it impossible to normalize to 8 h of scheduled clinician time as some previous studies have been able to do. However, our data are internally consistent over time, and the main insights of our study focus on relative changes over time rather than absolute measures of EHR time. To provide clinical context, we have also expressed changes over time as percent changes relative to the prepandemic baseline where possible. Third, our data cannot differentiate between EHR time for telemedicine and virtual care compared with EHR time during face-to-face, in-person visits. Our results therefore estimate the impact of the COVID-19 pandemic in its entirety, rather than simply through the shift to telemedicine. Fourth, we use data with proprietary metrics from a single EHR vendor, whereas Epic Systems is the largest ambulatory care vendor in the United States, clinicians and health systems using other EHR vendors may have different experiences. Finally, we do not have data on patient phone calls and are unable to assess changes in this modality of patient-clinician communication relative to secure messaging or scheduled encounters.

RESULTS

EHR time per day

Mean clinician EHR time per day was relatively stable from December 29, 2019, to March 14, 2020, with weekly means ranging between 80.4 and 85.9 min/day and a mean of 84.3 min across weeks (Figure  1). After-hours time per day was similarly stable during this time period, with weekly means between 19.2 and 20.6 min/day and a mean of 19.4 min across weeks. Both EHR time per day and after-hours time per day dropped significantly during the onset of the pandemic, from March 15, 2020 through July 4, 2020. In this period, EHR time per day dropped to a low of 64.9 min (23.01% decrease) in the week beginning March 29, 2020, before recovering to 85.7 min (1.67% increase compared with prepandemic mean) during the week of June 28, 2020. Similarly, after-hours time per day reached a minimum of 12.1 min during the week of April 5, 2020, and increased to 20.5 min during the week of June 28, 2020. From July 5, 2020, onward, total EHR time per day was slightly higher than in the pre-COVID period, with a mean of 86.4 min (2.49% increase relative to prepandemic mean) across those weeks, while after-hours time saw a smaller increase to a mean of 19.8 min (2.06% increase relative to prepandemic mean) in this period.

Figure 1.

Total electronic health record time per day and after-hours time per day. Notes: Blue-dotted line represents national COVID-19 case count and green line represents average daily clinical volume in our sample.

We found similar results in our event study models adjusting for daily encounter volume and time-invariant organizational characteristics (Figure  2). When compared with our baseline week (March 1, 2020), by the week of July 5, 2020, mean EHR time per day increased by 3.9 min (95% confidence interval [CI]: 2.98–4.85 min), and remained at a similar level (excepting holidays, annotated in Supplementary Appendix Exhibit SA1) through the remainder of the year. After-hours time per day followed a similar trajectory, with an increase of 1.55 min over the prepandemic baseline by the week of June 28, 2020 (95% CI: 1.19–1.92 min).

Figure 2.

Caption: total electronic health record time and after-hours time, controlling for volume and organization. Notes: Graphs are event study plots controlling for organization fixed effects and daily volume. All point estimates are relative to the week, 2-weeks prior to the first state-wide shelter in place (SIP) order in California, our proxy for the onset of the pandemic. Gray regions represents 95% confidence intervals with standard errors clustered at the organization level.

EHR time by function

Clinician EHR time per day across functions varied significantly during the onset of the COVID-19 pandemic (Figure  3). Time in In-Basket per day increased from 11.21 min during the week of March 1, 2020, to 12.68 min (13.11% increase) in the week of March 15, 2020, decreased back to the prepandemic baseline, and then increased again and stayed higher than prepandemic levels throughout 2020. Time in Clinical Review per day decreased significantly at the onset of the pandemic, then increased to higher levels, with average time from December 29, 2019, through March 14, 2020, of 17.17 min, compared with 18.37 min (7.00% increase) from July 5, 2020, onward. Time in Orders and Notes per day both saw similar decreases in the early days of the pandemic and recovered to their prepandemic levels by July 5, 2020.

Figure 3.

Electronic health record time per day by component function.

In-Basket messaging

Overall In-Basket messages received per day increased slightly compared with the prepandemic baseline, with clinicians receiving 4% more messages the week of July 5, 2020, compared with the 11-week prepandemic average (Figure  4). The greatest increase was in messages from patients, with clinicians receiving 157% of their prepandemic average per day; that increased level of messages remained consistent through the end of the year. Team and prescription messages also remained at higher levels (relative to baseline) during the postpandemic period after July 5, 2020, whereas results messages remained below the prepandemic average after July 5, 2020. Other message types saw a similar pattern to the results messages, and raw message counts were also similar (Supplementary Appendix Exhibit SA2).

Figure 4.

In-Basket message volume by source type. Notes: weekly message volumes are normalized by organization’s prepandemic baseline level (11-week period from December 29, 2019, through March 14, 2020)

Our multivariate fixed effects regressions suggest that each patient message increased In-Basket time per day by 2.32 min (P < .001, 95% CI: 2.16–2.48). Each Results message also significantly increased In-Basket time per day by 0.24 min (P < .001, 95% CI: 0.20–0.28). No other message type was associated with a statistically significant increase in In-Basket time per day (Table  1). We found similar results in our robustness tests including week fixed effects to control for secular time trends and seasonality (Supplementary Appendix Exhibit SA3).

Table 1.

Impact of messages on clinician In-Basket electronic health record time per day

Message type	Coefficient	95% confidence interval
System	−0.01	(−0.01 to 0.01)
Team	0.18	(0.09–0.27)
Results	0.24*	(0.20–0.28)
Prescription	−0.01	(−0.13 to 0.11)
Patient	2.32*	(2.16–2.48)
Custom	0.07	(−0.01 to 0.14)
Other	0.2	(0.08–0.32)

Notes: N = 18 347 organization-weeks. Model includes organization-level fixed effects. Dependent variable is clinician In-Basket active use time per day. Coefficient represents the marginal effect of receiving each additional message of that type.

P < .01.

DISCUSSION

The COVID-19 pandemic resulted in a large and immediate shock to clinical work, as reflected in our EHR metadata. Our national, longitudinal study of ambulatory clinician EHR work throughout 2020 illustrates that time spent in the EHR and after-hours EHR time per day dropped dramatically at the onset of the pandemic, but recovered by July 2020 to a level consistently higher than the prepandemic baseline. These results held after adjusting for daily encounter volume and organizational-level characteristics, suggesting that delivering care in the COVID era required more EHR work by clinicians compared with the pre-COVID baseline. This increased EHR time was driven by two functions: Clinical Review, where clinicians view test results and patient history, and In-Basket, where clinicians send, receive, and manage messages from a variety of sources. In-Basket messages received by clinicians increased dramatically during the pandemic, including receiving 157% of their baseline prepandemic messages from patients per day. Those patient messages were a significant driver of clinician work, with each message per day resulting in an additional 2.32 min of EHR active-use time. These results provide insight into the short- and medium-term impact of the COVID-19 pandemic on clinician work. Furthermore, they have important implications for designing policy and payment systems for a future that includes greater use of telemedicine and patients who may be more willing to communicate with their clinicians through secure messages.

The fact that clinicians spent more time working in the EHR during the COVID-19 pandemic is unsurprising but still concerning. An obvious potential driver of this finding is that many clinicians delivered care virtually via telemedicine, including both phone and video visits. The rapid proliferation of virtual visits may have had implications not only for how clinicians deliver care during the encounter but also how patients expect to interact with their clinician outside of the boundaries of the scheduled appointment. Telemedicine may demand more EHR time, especially in EHR work outside of the traditional “visit” as clinicians rely more on EHR-based Clinical Review and receive more In-Basket messages. Team-based care, a promising mechanism to improve quality in the prepandemic era, suddenly required more EHR work as distributed teams had to communicate through In-Basket messaging. Clinicians may have even served as impromptu technical support for their patients trying to navigate video conferencing applications, which required troubleshooting audio and video issues before appointments could begin. These problems may have been compounded as clinicians delivered both virtual and face-to-face care, which are visit modalities with very different structures and demands on clinician time. One surprising aspect of our results was that, after the immediate post-COVID acclimation period, COVID case counts appeared to have little impact on either encounter volume or any of our EHR use measures. This speaks to the resilience and adaptability of the health care system, which managed to continue delivering outpatient care both virtually and in-person after a period of adjustment.

We found Clinical Review and In-Basket to be the main drivers of increased EHR work. For Clinical Review, it may be that telemedicine encounters led to an increased reliance on previous documentation and viewing of test results in the EHR, or the increase in messages required clinicians to go back and review their own documentation before responding to patients. The increase in messages received across a variety of sources, resulting in a significant increase in In-Basket time, offers an important clue as to how the pandemic has changed clinical work. The aforementioned increase in messages from other care team members may reflect a new reliance on asynchronous communication as teams had to work in a distributed manner—a phenomenon observed across multiple industries in the COVID era. The increase in prescription messages—likely refill requests—was sharpest at the onset of the pandemic as patients may have rushed to ensure they had sufficient medication supply in the face of an uncertain future, and decreased closer to prepandemic levels as the year progressed. The most striking and durable increase, however, was in patient messages, with clinicians receiving up to 157% of their prepandemic volume of patient messages. Increased patient messaging may have been due to a variety of factors. Initially, patients may have substituted messaging for in-person visits due to fear of COVID exposure. The proliferation of telemedicine may have encouraged more patient portal sign-up and use—both of which had historically been low. Once patients became familiar with portals, they likely became more willing to message with clinicians. Given that the increase in patient messages was relatively stable through the end of 2020—even after telemedicine volume declined and in-person visits increased nationally,—we suspect that the latter explanation is a more-prominent driver of increased messaging and that the higher message volume may persist in the post-COVID era.

Our results have important implications for policymakers as the US health care system prepares for the postpandemic period. Though calls for continued use of telemedicine have grown, finding the right reimbursement structure may be challenging, with many open questions such as whether phone visits should be reimbursed at the same level as video visits and how virtual care should be integrated into clinician workflows. Beyond the first-order impact of virtual care, the expansion of telemedicine may have resulted in an increased awareness among patients of the ability to communicate with clinicians through secure messaging. The distribution of that EHR work across types of activity, however, suggests that the additional demands on clinician time may prove challenging to integrate with existing fee-for-service payment systems while changing patient expectations regarding how they communicate with their clinicians. Time reviewing patient records and messaging with patients often takes place outside of a traditional visit and is rarely a service reimbursed by payers or health systems. Furthermore, many ongoing efforts to address EHR burden are focused on documentation work, such as the recent CMS changes to ambulatory E/M documentation codes, and are unlikely to address this new form of EHR work driven by patient messaging even if they are successful in reducing EHR documentation time. To the extent that the increase in patient messages is durable, clinicians will need to allocate more of their time to answering those messages, even after the pandemic has subsided. This is especially important given recent evidence identifying In-Basket messages in particular as a predictor of physician burnout, as well as other studies finding a significant increase in patient messages received by clinicians during the pandemic.,, Policymakers and health system leaders should keep these new demands on clinician time in mind as they develop future reimbursement models and workflows, taking care not to exacerbate EHR-driven clinician burnout.,

CONCLUSION

In our national study, we found that EHR active use time fell in the immediate onset of the COVID-19 pandemic, but by July 2020 clinicians spent more time in the EHR and more EHR time after-hours per day. This increase was driven by time spent in Clinical Review and In-Basket messaging, with clinicians receiving 157% of their prepandemic baseline messages from patients. These patient messages represented a significant demand on clinician time, with each requiring an additional 2.3 min of daily EHR work on average. Policymakers and health system leaders looking to create sustainable workflows incorporating telemedicine in the post-pandemic period should be aware of these new demands on clinician time not only to avoid clinician burnout but also to accommodate rethinking the model of ambulatory medicine as patient expectations for care expand beyond the scope of the traditional face-to-face visit.

AUTHORS CONTRIBUTION

AJH and NLD conceived the study, acquired the data, designed the analysis plan, drafted the manuscript, and participated in the critical revisions process. MT performed statistical analysis and helped draft the manuscript. CS, CL, and RH designed the analysis plan, drafted the manuscript, and participated in the critical revisions process.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

CONFLICT OF INTEREST STATEMENT

RSH reports receiving personal fees from Kaiser Permanente, Partners Healthcare, MD Anderson Cancer Center, OhioHealth, and Ochsner Health; serving as an advisory board member for RubiconMD, Arena, and Carrum Health; and being an uncompensated trustee of Brigham Health and the Brigham and Women’s Physicians Organization. No other disclosures are reported.

DATA AVAILABILITY STATEMENT

Data are can be requested in de-identified format from Epic Systems.

Click here for additional data file.