Protocol for pragmatic randomised trial: integrating electronic health record-based behavioural economic 'nudges' into the electronic health record to reduce preoperative testing for patients undergoing cataract surgery.

INTRODUCTION: Robust randomised trial data have shown that routine preoperative (pre-op) testing for cataract surgery patients is inappropriate. While guidelines have discouraged testing since 2002, cataract pre-op testing rates have remained unchanged since the 1990s. Given the challenges of reducing low-value care despite strong consensus around the evidence, innovative approaches are needed to promote high-value care. This trial evaluates the impact of an interdisciplinary electronic health record (EHR) intervention that is informed by behavioural economic theory. METHODS AND ANALYSIS: This pragmatic randomised trial is being conducted at UCLA Health between June 2021 and June 2022 with a 12-month follow-up period. We are randomising all UCLA Health physicians who perform pre-op visits during the study period to one of the three nudge arms or usual care. These three nudge alerts address (1) patient harm, (2) increased out-of-pocket costs for patients and (3) psychological harm to the patients related to pre-op testing. The nudges are triggered when a physician starts to order a pre-op test. We hypothesise that receipt of a nudge will be associated with reduced pre-op testing. The primary outcome will be the change in the percentage of patients undergoing pre-op testing at 12 months. Secondary outcomes will include the percentage of patients undergoing specific categories of pre-op tests (labs, EKGs, chest X-rays (CXRs)), the efficacy of each nudge, same-day surgery cancellations and cost savings. ETHICS AND DISSEMINATION: The study protocol was approved by the institutional review board of the University of California, Los Angeles as well as a nominated Data Safety Monitoring Board. If successful, we will have created a tool that can be disseminated rapidly to EHR vendors across the nation to reduce inappropriate testing for the most common low-risk surgical procedures in the country. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov identifier: NCT04104256. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To our knowledge, this pragmatic randomised trial protocol is the first to evaluate the use of behavioural economics ‘nudges’ to discourage pre-operative (pre-op) testing before cataract surgery.

Our testing of three distinct ‘nudges’ will offer insights as to whether certain behavioural economic framing strategies are more effective than others for changing physician behaviour.

Methods from this study protocol can be used to design similar interventions that target other types of testing before low-risk surgeries and low-value services more broadly.

A potential limitation to the study is that the three ‘nudges’ might not be powerful enough to overcome the deeply ingrained institutional and cultural clinical practices.

There is a possibility of the nudges unnecessarily overburdening a physician ordering tests for a reason other than pre-op evaluation during a pre-op visit.

Introduction

Background and rationale

Cataract surgery is the most common operation in the USA with 1.7 million procedures performed annually.1 Due to the ageing population, this number is expected to increase to 4.4 million by 2030.1 Cataract surgery is a low-risk procedure: it lasts an average of 18 min, it is almost always performed using topical anaesthesia, and serious complications are exceedingly rare.2–4 There is strong consensus based on robust randomised controlled trials that routine preoperative (pre-op) tests for cataract surgery such as blood testing, EKG, chest X-rays (CXRs) and cardiac stress tests are unnecessary and do not improve health outcomes.5–8

Since 2002, specialty societies globally have repeatedly published guidelines discouraging such testing.2 9–12 Despite this evidence and concerted efforts to reduce pre-op screening tests before cataract surgery, several studies have found that the majority of patients who undergo this procedure are subjected to these tests.6 7 This is a quintessential example of physician behaviour being driven by factors other than evidence. Reasons why pre-op testing continues despite the evidence that it is low-value include institutional requirements and culture, legal concerns, financial incentives and the belief among providers that another specialist expected the pre-op tests.13 Reducing routine pre-op testing before cataract surgery could reduce patient exposure to unnecessary and potentially harmful tests6 while saving upwards of US$50 million dollars annually.14–17

Recognising that empiric proof of non-efficacy alone will not lead to a reduction in the delivery of low-value care, research teams and policymakers are examining the application of behavioural economics as a way to encourage physicians to change their behaviour.18 19 Behavioural economic theory posits that humans have predictable cognitive biases and that these can be leveraged to reduce undesired behaviours. One application of behavioural economics is through pop-up alerts embedded within a health system’s electronic health record (EHR). This study will test the impact of various appeals to physician professionalism on physician-ordering patterns. Specifically, we will test whether framing risk of psychological harm, financial harm or an absence of improving patient safety will improve physician ordering patterns compared with usual care.

The study described here will test the hypothesis that an EHR-based behavioural economics intervention implemented in a real-world setting including all physicians in the healthcare system (ie, not just those who agree to participate) will reduce pre-op testing for cataract surgery. To our knowledge, no study has tested the use of an alert that incorporates a behavioural economics-based ‘nudge’ as a mechanism by which to reduce pre-op testing before surgery in a large academic health system setting. In behavioural economics and psychology, nudges are known as strategies that change behaviour in a predictable manner without restricting options.20 If successful, this intervention could be generalised to other EHR systems and disseminated to health systems across the country.

Study objectives

Our first aim is to integrate three distinct ‘nudge’ alerts into the UCLA Health EHR. Each nudge will test a different behavioural economics framing strategy. Our second aim is to conduct a four-arm randomised pragmatic trial to compare the effectiveness of the nudges to each other and to usual care. We hypothesise that exposure to the nudges will result in a reduction of pre-op tests ordered by physicians. We will randomise providers who conduct pre-op visits for patients with cataract surgery to one of four study arms: nudge 1, nudge 2, nudge 3 or no nudge (usual care) and measure the effect of the intervention on pre-op testing (see below).

Trial design

This is a pragmatic, parallel group, open-label, randomised controlled superiority trial. Groups are allocated 1:1 to the intervention arms (nudges) and control arm (routine care). This trial will be conducted between June 2021 and June 2022 with a 12-month follow-up period. This trial has been registered with the United States Clinical Trials Registry. This protocol manuscript is compliant with the Standard Protocol Items: Recommendations for Interventional Trials statement.21 In this pragmatic trial of a quality improvement intervention, neither patients nor physicians provide consent to participate in the trial. This protocol was approved by the Institutional Review Board (IRB) of the University of California, Los Angeles (IRB#18–0 01 240).

Methods

Setting

This study will take place at UCLA Health, a large urban academic health system located in Los Angeles, California and surrounding communities. UCLA Health operates four hospitals on two campuses (Westwood and Santa Monica) and over 250 outpatient practices. Approximately, 2520 physicians and 4200 nurses provide care throughout the system.

Participants

The participants in this study will be UCLA Health physicians who complete a pre-op visit for at least one patient undergoing cataract surgery at UCLA in the 12 months prior and 12 months after the study start date and all patients who are seen during such visits.

Inclusion criteria

All UCLA Health physicians perform a pre-op visit for a patient undergoing cataract surgery in the 12 months prior to the study start date as well as all physicians are anticipated to complete such a visit during the 12-month study period. In 2019, 322 physicians at 74 unique UCLA Health clinics performed such a visit. These physicians conducted a total of 1122 pre-op visits for 995 unique patients. To identify patients who presented for a pre-op visit within 30 days of cataract surgery, we used Surgical Consult Encounter Code 2100 in combination with one of four codes used to identify cataract surgery (online supplemental appendix).

Exclusion criteria

As this is a real-world pragmatic trial, all physicians who meet the criteria described above will be included in the study. Similarly, there are no exclusions for patients who present at a visit defined by the criteria described above. Patients with cataract surgery who do not have a pre-op evaluation with a UCLA physician, but have cataract surgery at UCLA, will not be included in this study.

Study intervention

At UCLA Health, cataract surgery planning is initiated when a patient is evaluated by a UCLA ophthalmologist and deemed an appropriate candidate for the procedure. The patient is then asked to make an appointment for a pre-op visit with their primary care provider (PCP), cardiologist or another UCLA physician. The Centres for Medicare and Medicaid Services and the Joint Commission require that a pre-op visit be completed within 30 days of the surgery.22 23

At this pre-op appointment, the provider completes a required history and physical, sometimes working off a checklist provided by the surgeon that may include a list of suggested tests such as EKG, cardiac stress test, Complete Blood Count (CBC), electrolytes and CXR. The ordering of these tests is left to the discretion of the physician completing the pre-op evaluation. The decision to order tests and procedures is influenced by the provider’s previous experiences, preferences and interpretation of the guidelines.

The intervention will deploy one of the EHR nudge alerts (control, nudge 1, nudge 2 or nudge 3) such that each physician in the study will only get one of the nudges (or none of them) for the duration of the study. These alerts will be triggered when a physician begins to order a pre-op test for patients during their pre-op visit. The full list of pre-op tests, informed by Chen et al and Steinberg et al,6 24 was identified using Current Procedural Terminology codes. These tests fall into several categories including chemistry panel, complete blood count, coagulation panel, urinalysis, EKG, cardiac stress test, CXR and pulmonary function tests. The full list of pre-op tests can be found in the Appendix. Details of the nudge alert including opt-out features are described below.

To inform the design of the nudges for this study, we considered findings from Liao et al25, which found that self-reported behaviour among physicians is most influenced by the patient and the potential for patient harm as compared with societal or institutional factors. We, therefore, tailored the three distinct nudges to highlight the following concepts1: patient safety,2 financial costs to the patient and3 the risk of psychological harm to the patient. Each of the three nudge reference guidelines from the American Society of Anesthesiologists (ASA) and the American Academy of Ophthalmology (AAO)26 27 include links that physicians can access via the EHR. Given that one common reason for ordering cataract pre-op tests is the belief that they are required by the surgeon, we included specific language stating that UCLA ophthalmologists and anesthesiologists advise against pre-op testing. We also included text to reassure the provider that the pre-op team at UCLA will order any necessary labs on the day of surgery. In addition, each nudge includes an image relevant to the message of the nudge and the potential impact pre-op tests may have on patients.

Three intervention arms

Nudge 1 (figure 1) will display as ‘UCLA Ophthalmologists and Anesthesiologists ADVISE AGAINST routine pre-op tests, which do not increase the patient’s safety and are considered inappropriate per current evidence-based recommendations (see UCLA Ophthalmology, UCLA Anesthesiology, ASA and AAO recommendations). The UCLA Pre-op Eval and Planning Center (PEPC) will order any necessary day of surgery labs. ’9 10

Figure 1

Nudge 1, alert highlighting the safety/liability aspects of preoperative tests.ASA: American Society of Anesthesiologists, AAO: American Academy of Ophthalmology, UCLA: University of California, Los Angeles.

Nudge 2 (figure 2) will display as ‘UCLA Ophthalmologists and Anesthesiologists ADVISE AGAINST routine pre-op tests, which can increase the patient’s out-of-pocket costs and are considered inappropriate per current evidence-based recommendations (see UCLA Ophthalmology, UCLA Anesthesiology, ASA and AAO recommendations). The UCLA PEPC will order any necessary day of surgery labs’.9 10 14

Figure 2

Nudge 2, alert highlighting the financial benefits to the patient of not experiencing preoperative tests.ASA: American Society of Anesthesiologists, AAO: American Academy of Ophthalmology, UCLA: University of California, Los Angeles.Disclaimer: the image in the figure does not depict a patient and is license-free.

Nudge 3 (figure 3) will display as ‘UCLA Ophthalmologists and Anesthesiologists ADVISE AGAINST routine pre-operative tests, which can cause aggravation and psychological stress for the patient and are considered inappropriate per current evidence-based recommendations (see UCLA Ophthalmology, UCLA Anesthesiology, ASA, and AAO recommendations). The UCLA Pre-op Eval and Planning Center (PEPC) will order any necessary day of surgery labs’.9 10

Figure 3

Nudge 3, alert highlighting the psychological benefits to the patient of not experiencing preoperative tests.ASA: American Society of Anesthesiologists, AAO: American Academy of Ophthalmology, UCLA: University of California, Los Angeles.Disclaimer: the image in the figure does not depict a patient and is license-free.

We designed these alerts to minimise the burden on physicians while also including a hard stop, so that if a physician chooses to order a pre-op test, they will need to type an explanation. This behavioural intervention is known as ‘accountable justification’, which has been shown effective in reducing low-value care.28 Specifically, as part of the alerts, physicians can remove the order with a single click on ‘Accept’. If they choose not to remove the order, they can click ‘Keep’ and will be required to click on the ‘Explain’ button and provide free text stating why they continued with placing the order after exposure to the nudge (Appendix).

Control arm

Providers who are randomised to usual pre-op care will not receive an alert, or any special information about testing guidelines when they attempt to order a pre-op test. All other aspects of the clinical encounter and documentation will be the same for both the intervention and control arms.

Surveys to measure provider experience

We are unaware of any existing provider experience survey specific to EHR interventions targeting pre-op testing before cataract surgery. The study team, therefore, developed this survey de novo (see Appendix). The Physician Experience Survey will be distributed to physicians in the intervention arm. This 11-item survey will elicit physician views on concepts such as knowledge about guidelines related to pre-op testing for cataract surgery and views on electronic decision support tools and whether they find these helpful or burdensome in the clinical decision-making process. We will email providers a link to the survey and providers will receive a US$25 gift card following completion of the survey.

Data collection methods

All data will be collected from the UCLA Health EPIC EHR ‘CareConnect’ and from surveys administered to providers. This data will be stored securely and only accessible to research personnel trained in confidentiality and privacy procedures. An independent Data Safety Monitoring Board will provide ongoing oversight and data audits. To compare the cost of pre-op testing among study arms, we will use publicly available Medicare payment rates and UCLA Health commercial payment rates.

Analysis

Randomisation strategy

The study statistician will randomise eligible physicians to one of the two study arms (figure 4). The randomisation will be stratified by the number of pre-op encounters conducted in the pre-intervention period (0, 1–2, 3–5,>6) to ensure appropriate control of physician’s experience or preference to conducting pre-op testing.

Figure 4

Randomisation strategy: Randomizing the UCLA Health physicians who perform pre-op testing to one of the three intervention arms or the usual care arm.

Primary and secondary outcome analysis

We will measure and compare rates of testing in the 12 months before and 12 months after initiation of the randomisation (primary outcome) to determine whether the EHR-based intervention is successful in changing physician behaviour. We will also compare each of the three intervention arms and the usual care arm and will perform exploratory comparisons between nudge 1, nudge 2 and nudge 3 (secondary outcome) to determine whether certain behavioural economic framing techniques are more effective than others at reducing pre-op testing.

Other secondary outcomes will include the percentage of patients who received specific categories of pre-op tests including labs, EKGs and CXRs, same-day surgery cancellations and physician-level experience measured by the surveys. We will also estimate cost savings associated with the intervention (described below). Evaluation of the primary and secondary outcomes will be performed using generalised linear mixed-effects models at the patient level. We will investigate the impact of all nudges versus control, which will require a threefold correction. As a secondary analysis, we will conduct pairwise comparisons of the nudges. Models will include fixed nudge 1, nudge 2 and Nudge 3 effects, a fixed study period effect (postnudge intervention vs prenudge intervention), second-order interactions between these effects, a fixed stratum effect (hospitalist vs PCP) and random physician effects. Physician effects will account for clustering of patients by physician. Comparisons between changes in utilisation rates across study arms will be estimated and tested using model contrasts. All pairwise comparisons of changes between study arms will be performed to identify the most effective nudge, and Bonferroni corrections will be applied to control the familywise type I error rate at 5%. Intervention effects will be summarised using ORs, 95% CIs and p values (with multiple comparison corrections). Differences between strata will be similarly summarised.

These models will also be adjusted for patient-level covariates (age, race, ethnicity and Elixhauser comorbidities). Because serious complication rates are so low from cataract surgery (<0.1%), we will not have adequate power to determine whether the intervention is associated with a change in complication rates, but we will extract from the EHR all cardiovascular events occurring within 30 days of surgery and compare these in an exploratory fashion among the intervention and control groups. Analyses will be performed using SAS V.9.4 (SAS Institute, Cary, North Carolina).

Power

Preliminary analyses of UCLA Health data show that the baseline rate of testing is 65%. In a prior quality improvement study conducted at a different health system,29 we reduced pre-op lab testing by 59% (a reduction from 90% to 31%). We conservatively estimate an effect half as large (30%) in each intervention arm.

Power will be evaluated using a χ2 test adjusting for clustering. This is a conservative simplification of the planned mixed-effects logistic regression model analysis. Each nudge will be assumed to independently reduce the rate of testing by 30%. An intraclass correlation of 0.25 will be assumed (correlation between patients seen by the same physician, after adjusting for patient-level characteristics). Assuming each physician sees an average of seven patients during the intervention period (based on preliminary data), and with an anticipated 195 physicians per study arm, using two-sided hypothesis testing and an overall alpha of 5% (controlled using Bonferroni corrections, the individual alpha for each contrast will be 0.008), we will have >90% power to detect a difference of 30% between the usual care arms and each of the intervention arms.

Cost analyses

We will measure several cost outcomes associated with the intervention, specifically (1) savings from reduced use of medical services, (2) net total savings after accounting for the cost of the intervention and (3) return on investment. We will also extrapolate the savings estimates we observe to the Medicare population to quantify the amount of savings that the Medicare programme could achieve through the reduction of low-value pre-op services for cataract surgery. We will also extrapolate longer term (ie, 2–5 year) savings that would accrue if the effects of the intervention persist, based on our prior work.29

To calculate savings from reduced use of medical services, we will multiply the estimates of utilisation (ie, quantity) for each of the pre-op services measured by the relevant payment rates for the services based on the coverage type for the patient (ie, Medicare and average commercial payment rates per service). We will then compare differences in spending overall and by specific services (ie, lab testing, EKGs) between the control arm and each of the intervention arms and a pooled estimate of the three-nudge intervention arms. We will disaggregate the savings to measure the amount and proportion accrued by the health system/payer versus the patient. An extension of this analysis will estimate how reducing the utilisation of pre-op testing affect different parties; we will disaggregate the savings estimates by type of payer contract (ie, capitated vs fee-for-service(FFS)) to derive estimates of savings to the health systems versus the payer. Under capitated payments, the savings accrue to the health system, while under FFS payments, the savings accrue to the payer and represent revenue losses for the health system.

We will extrapolate observed saving estimates to compute savings the Medicare FFS programme might expect to accrue if similar reductions in the utilisation of pre-op services for cataract surgery were observed in the Medicare population. To generate these savings estimates, we will compute the number of Medicare patients undergoing cataract surgery in a single year and the fraction of patients who have these pre-op tests. We will then take our estimates of reduction in utilisation and apply these estimates to the current use of pre-op testing utilisation in the Medicare FFS programme to determine the number of services that would be reduced. We would then apply FFS payment rates to the number of services that would be reduced to estimate savings to Medicare. Finally, we will develop estimates of longer term savings in a year–year time range, applying a discount rate to costs, assuming that the observed reduced amount of services used remained similar after the 1-year study period ended.

Discussion

Investigators are only beginning to learn how to use the EHR as a tool to promote desired care processes.30 This proposed pragmatic trial should break new ground in our understanding of how behavioural economic approaches can be used to discourage care that does not promote better patient outcomes. If effective, this study will inform approaches for using nudges not only to reduce pre-op testing before low-risk surgeries but also to reduce low-value care more broadly.

Ethics and dissemination

The study protocol was approved by the Institutional Review Board (IRB) of the University of California, Los Angeles (IRB number 18–0 01 240) as well as a nominated Data Safety Monitoring Board that will provide ongoing oversight for this study. All previous and future amendments of the protocol have and will be approved by IRB. Results will be published in peer-reviewed journals and uploaded to the US National Library of Medicine Clinical Trials.