Multicomponent intervention to improve blood pressure management in chronic kidney disease: a protocol for a pragmatic clinical trial.

INTRODUCTION: The purpose of this study is to incorporate behavioural economic principles and user-centred design principles into a multicomponent intervention for the management of uncontrolled hypertension (HTN) in chronic kidney disease (CKD) in primary care. METHODS AND ANALYSIS: This is a multicentre, pragmatic, controlled trial cluster-randomised at the clinician level at The Brigham and Women's Practice -Based Research Network of 15 practices. Of 220 total clinicians, 184 were eligible to be enrolled, and the remainder were excluded (residents and clinicians who see urgent care or walk-in patients); no clinicians opted out. The intervention consists of a clinical decision support system based in behavioural economic and user-centred design principles that will: (1) synthesise existing laboratory tests, medication orders and vital sign data; (2) increase recognition of CKD, (3) increase recognition of uncontrolled HTN in CKD patients and (4) deliver evidence-based CKD and HTN management recommendations. The primary endpoint is the change in mean systolic blood pressure between baseline and 6 months compared across arms. We will use the Reach Effectiveness Adoption Implementation Maintenance framework. At the conclusion of this study, we will have: (1) validated an intervention that combines laboratory tests, medication records and clinical information collected by electronic health records to recognise uncontrolled HTN in CKD patients and recommend a course of care, (2) tested the effectiveness of said intervention and (3) collected information about the implementation of the intervention that will aid in dissemination of the intervention to other practice settings. ETHICS AND DISSEMINATION: The Human Subjects Institutional Review Board at Brigham and Women's Hospital provided an expedited review and approval for this study protocol, and a Data Safety Monitoring Board will ensure the ongoing safety of the trial. TRIAL REGISTRATION NUMBER: NCT03679247. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This pragmatic clinical trial of a multicomponent intervention will be the first to incorporate clinical decision support, computable phenotypes, user-centred design principles and behavioural economic theories for the purpose of improving hypertension in chronic kidney disease patients.

This study’s success will be measured both by its clinical potential (ie, improvement of hypertension in chronic kidney disease patients), and by the feasibility and potential utility of its application, offering a real-world evaluation of implementation science frameworks.

Limitations include the potential for enrolled primary care physician (PCPs) to leave their clinic or transition to urgent care practice, the use previous year mean systolic blood pressure as a baseline, contamination of PCPs within clinics if the intervention is discussed, and possible workarounds to the clinical decision support system.

The use of the Reach Effectiveness Adoption Implementation Maintenance implementation science framework in this pragmatic trial will provide valuable lessons for researchers hoping to leverage multiple components, from electronic health records to usability principles, into a single application.

Background

Chronic kidney disease (CKD) is prevalent, afflicting 26 million Americans, and is associated with high morbidity and mortality. Medicare costs for CKD and end-stage renal disease total $84 billion and $36 billion, respectively.1 CKD diagnosis, monitoring and treatment must be improved in primary care. Hypertension (HTN) is a leading risk factor for long-term outcomes such as kidney failure, cardiovascular events and death. There are effective approaches to monitoring and treatment that must be disseminated broadly to cut costs and save lives. Dissemination efforts must focus on primary care clinics because 95% of patients with CKD have early disease and are cared for by primary care physicians (PCPs). Only 15% of patients whose estimated glomerular filtration rate (eGFR) is less than 60 mL/min/1.73 m2 are aware that they have CKD, so it is especially important that PCPs become aware of the diagnosis early.2–6 Furthermore, there is evidence that CKD is underdiagnosed by PCPs. Data from our 15 primary care clinics showed that only 15% of patients with CKD had a documented diagnosis of CKD and only 40% had a urine albumin test.7 Many effective approaches for recognition of CKD and treatment of uncontrolled HTN in CKD are appropriate for the primary care setting.

Behavioural science studies have shown that decision-making is complex and involves conscious and unconscious drivers. One simple strategy is to display clinical decision support (CDS) with prechecked, no-action defaults. These defaults greatly impact user behaviour, as shown in studies on organ donation, end-of-life planning and generic drug prescribing.8–11 One study tested an accountable justification intervention within the CDS that asked PCPs, who were in the act of prescribing an antibiotic, to explicitly justify this decision in a free-text response.12 Accountability improves decision-making accuracy, and requiring a justification frames the antibiotic prescribing behaviour as non-normative.13 14

Additionally, work from the behavioural economics literature suggests that human behaviour is influenced by emotional cues (eg, fear) and social circumstances (what they think their peers think) rather than reason, known as being ‘predictably irrational’.15 Therefore, another simple external intervention is to ask PCPs to publicly commit to a new behaviour. In one study, PCPs were asked to sign and post a letter that stated a commitment to following appropriate antibiotic use guidelines.16 The intervention led to a decrease in inappropriate antibiotic use. The effect was attributed to three factors: (1) PCP’s aim for consistency and fear that inconsistency will lead to disapproval by peers, (2) publicly committing to a behaviour connects that behaviour with the PCP’s self-image and (3) the visible poster affects patients’ behaviour. Our objective in this pragmatic clinical trial is to incorporate behavioural economic principles and user-centred design principles into a multicomponent intervention for the management of uncontrolled HTN in CKD in primary care.

Methods

Implementation science and randomised clinical trial conceptual frameworks

This study will use the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework to conduct the evaluation. The RE-AIM framework is well suited for technology innovation projects because it focuses on external validity in study design and guides the planning, conduct, evaluation and maintenance of the intervention.17 In this project, the RE-AIM framework will be used to derive practical measures of how well the intervention works in real-world clinical settings and to produce a 360-degree assessment of its efficacy.

Overall study design

We will evaluate the effectiveness of the intervention in a pragmatic, cluster-randomised controlled trial, randomised at the clinician level. The rationale for this is that the behavioural economic and usability components of intervention apply at the clinician level; additionally, PCPs are expected to learn from the CDS, and thus there would be potential for contamination if randomisation occurred at the patient level. The primary endpoint of the study is the change in mean systolic blood pressure (SBP) between baseline and 6 months compared across arms. Secondary endpoints include HTN-specific process measures, process measures for CKD quality of care, adverse drug events and hypotension. We also hope to assess whether the intervention improves process measures for quality of CKD care such as annual urine albumin tests. The overarching structure supporting the evaluation is the RE-AIM framework, and this protocol was reported in accordance with Standard Protocol Items: Recommendations for Interventional Trials guidelines (figure 1).

Figure 1

Schedule of enrolment, interventions and assessments. *Y, year; Q, quarter.

Study setting

The Brigham and Women’s Primary Care Practice-Based Research Network (BWPC PBRN) is one of 155 PBRNs nationally certified by the Agency for Healthcare Research and Quality. The BWPC PBRN is a network of 15 practices which includes hospital-based practices, community-based practices and community health centres affiliated with Brigham and Women’s Hospital.

Eligibility criteria

Patients: All patients over the age of 18 who have a visit with a PCP at one of the intervention practices during the 2 years preceding the visit date will be eligible for enrolment. The first inclusion criteria will be CKD, defined as two prior eGFRs 16–59 mL/min/1.73 m2 separated by 90 days, as calculated by CKD-EPI, or two prior UACR >30 mg/g separated by 90 days. The second inclusion criteria will be uncontrolled HTN, defined as at least one SBP >140 mm Hg within the 2 years preceding the enrolment visit, as well as SBP >140 mm Hg at the enrolment visit. Patients with a most recent eGFR ≤20 or two previous eGFRs within 2 years separated by at least 90 days≤15 will be excluded.

Clinicians: Our objective is to include PCPs who have a consistent panel of primary care patients and represent primary care clinicians broadly outside of the academic medical centre setting, and thus our criteria excludes clinicians who only see urgent care and walk-in patients. We also excluded residents in training. We compiled a list of currently employed physicians, physician assistants and nurse practitioners in our primary care network. The network includes 220 PCPs who care for approximately 150 000 patients. Of the 220 PCPs, 184 were eligible to be enrolled, and the remainder were excluded; no PCPs opted out of the trial.

Study intervention

One component of the intervention is a series of Epic Best Practices Advisories (BPAs). These CDS are designed to utilise computable phenotypes (CPs), defined as disease definitions or algorithms that allow curation of disease populations using data from electronic health records (EHRs).18 We developed five CPs, each with its own CDS recommendation. The first CP includes patients with CKD and uncontrolled SBP for whom it is advised that an ACE inhibitor (ACEi) be prescribed. The second includes patients with CKD and uncontrolled SBP for whom an angiotensin receptor blocker (ARB) should be prescribed. The third CP includes patients who are currently on an ACEi but not at an optimal dose, while the fourth CP includes those who are on a suboptimal dose of an ARB. The fifth CP includes patients who are maximised on an ACEi or ARB, but are not on a diuretic. The CDS prompts PCPs to make clinical decisions specific to these phenotypes. For example, the PCP of a patient exhibiting the first CP would be prompted to prescribe a starting dose of an ACEi (eg, lisinopril), while the PCP of a fourth CP patient would be prompted to increase the dose of the patient’s currently prescribed ARB (eg, losartan).

Another aspect of the multicomponent intervention, developed utilising user-centred design principles, is the display of patient-specific data explaining why the CDS fired. Our team performed contextual inquiry sessions and two rounds of usability testing (group design and individual think aloud sessions) on the CDS prototype. The main focus of the contextual inquiry sessions was to assess the different activities, steps and thinking processes involved in managing uncontrolled blood pressure using EHRs and any other resources used by PCPs during a patient encounter. The goal of the group design sessions was to validate the requirements gathered during the contextual inquiry sessions and gain a greater understanding of user preferences and mental models. The goal of the individual think aloud sessions was to uncover usability issues and validate design decisions. Additionally, hyperlinks to clinical guidelines supporting the CDS recommendation are included in the final CDS.19 20

Finally, a precommitment email was sent to all participating PCPs asking them to pledge to follow recommendations about blood pressure management. We will evaluate whether the pledge makes people more likely to either follow the recommendation or to add an accountable justification (stated rationale for why one disagrees with the recommendation) when interacting with the BPA, incorporating the behavioural economic principle of accountability into the intervention.

Implementation of CDS in Epic

The five CPs were translated to a set of rules, which were incorporated into our Epic system in order to create the BPAs. The BPAs appear at the time of chart opening by the PCP if criteria are met (see example in figure 2). When the trial begins, we will identify control patients in real time according to the same inclusion criteria as intervention patients. This will allow us to validate that the rules are accurately identifying patients and producing the correct recommendations through a chart review. The CDS was moved to the production environment in ‘silent mode’ for approximately 6 weeks before the scheduled start date of the trial, where a report recorded when it would fire but it was not displayed to the user. The CDS was also activated in the production environment for a pilot study.

Figure 2

Example CDS with explanation, clinical information, recommended actions and accountable justification (‘acknowledge reason’). CDS, clinical decision support.

In practice, the CDS will fire only on the initial enrolment visit, except if the PCP chooses the ‘remind me next visit’ option in the ‘acknowledge reason’ section. However, if the patient’s SBP has dropped below 140/90 mm Hg, the patient is already on the recommended medication or a new allergy to an ACE/ARB has been entered, this follow-up firing will be suppressed.

Outcomes and measures

Due to the intervention’s multiple components, ranging from aspects of clinical effectiveness (management of HTN in CKD patients) to usability principles and strategies (PCP adherence to CDS guidelines), outcomes and measures will be guided by the RE-AIM framework.

Reach will refer to the overall use of the CKD CDS, including the number of PCPs and patients for whom it fires. To assess the reach of the CDS, statistics on the quantity and types of firings will be collected through enterprise data warehouse and Epic queries. Concurrent manual review of Epic reports and chart review on CDS firing statistics will be conducted by team members to verify the automated monthly summaries. Analytic variables will include the percentage and types of clinicians in primary care who use the software, descriptions of excluded clinicians, PCP review and/or response to pledge email, PCP interaction with the CDS, signing of orders or accountable justification documentation within the CDS and whether the BPA fired appropriately during encounter.

Effectiveness will refer to the clinical efficacy (process and outcome measures), usability of the software in the primary care environment, process measures and both positive and negative unanticipated consequences. In evaluating effectiveness, the primary endpoint is the change in mean SBP between baseline and 6 months compared across arms. This outcome was chosen as the primary outcome because of the growing need in primary care to monitor patients’ HTN in order to help mitigate negative long-term outcomes of CKD such as kidney failure, cardiovascular events and death. A meta-analysis of three large cohorts of CKD patients without diabetes concluded that maintaining blood pressure below 140/90 mm Hg decreases risk of these outcomes significantly.21 Several guidelines have been issued to emphasise the importance of HTN control in CKD.22–24 Additional secondary outcomes are listed in tables 1 and 2.

Adoption will refer to the percentage and types of settings and staff that embrace the innovation. We will analyse the various ways in which the intervention PCPs accept, reject and generally interact with the BPA. Using a BPA report extracted through Epic, we will collect which CDS fired, the date and time of the firing, the medical records number (MRN) and demographic information of the patient on whom it fired, and the user (PCP). We will also extract the user follow-up action (whether the PCP indicated that they would order a medication or basic metabolic panel through the BPA) or which acknowledge reason the PCP chose. If ‘other’ is indicated as an acknowledge reason, the PCP will be asked to elaborate with a comment. Finally, in conducting our manual chart review, we will record whether the BPA firing for the patient was appropriate and whether the course of action that the PCP indicated they would take through the BPA differed from the course that they actually took.

Implementation will refer to the consistency of CDS use, any support resources used, any barriers and/or enabling factors that are identified, any workarounds to barriers that develop, any changes from preintervention to intervention period and any unintended consequences to patient safety or workflows. Prior to the clinical trial, a pilot study was conducted in live clinical settings. The BPAs were turned on for approximately 2 weeks for all intervention clinicians, and the patients for whom the BPAs fire during this period were classified as ‘Pilot Patients’. Interaction with the BPAs was monitored. The first time a BPA fired for a PCP, the research team contacted the PCP by email to gather feedback through a survey or an interview. The experiences of these PCPs were noted, but no changes were made to the BPA. However, some early workarounds that were discovered included selecting the accountable justification ‘other’ without a valid reason (eg, ‘x’) in order to circumvent the BPA, as well as ordering a medication or panel without completing the order by signing off on it.

Maintenance will refer to how well the innovation components and their effects are sustained, as well as any strategies that are used to uphold the intervention over time. This will be recorded by qualitative descriptions of system performance longitudinally, emerging workflow changes and long-term unintended consequences, and how the BPA fit into the existing PCP workflow. The methods used will include contextual inquiry sessions, interviews and surveys.

Sample size and power calculation

According to our analysis of patients who saw a PCP at one of the 15 BWPC practices in 2009, there were 3118 patients with two prior eGFR 16–59 mL/min/1.73 m2 measured at least 90 days apart between 2007 and 2008. We determined that 42% (n=1309) of these patients had at least two BP>140/90 mm Hg. Those patients with elevated SBP had a mean SBP of 153.9 mm Hg and SD of 14.0 mm Hg. We will assume that 71% of the enrolled patients will have at least one follow-up visit during the 6 month follow-up period.25 According to an analysis of 2013 data, 56 461 patients visited a PCP at one of the 15 BWPC practices in 2013.26 We identified 5593 patients with CKD who visited a PCP during 2013, when CKD was defined as two prior eGFR <60 mL/min/1.73 m2 or UACR ≥30 mg/g measured at least 90 days apart between 2007 and 2012. Combining our findings from the two studies, we expect 42% of these 5593 patients to fit both of our inclusion criteria, so n=2349 patients. We based our power calculation on an expected decrease in the mean of the final SBPs for patients in the intervention arm of at least 3 mm Hg as compared with the mean of the final SBPs in the control arm, which is a clinically important decrease. The two arms will be compared using a robust generalised estimating equations z-test for continuous data; this approach does not assume normality of the outcome and accounts for a possible cluster effect of patients within PCP.27 Using the GEE z-test with a two-sided type I error rate of 2.5%, we calculated that 497 evaluable patients per arm and an average of six patients per PCP would provide over 80% power to detect an average 3 mm Hg SBP decrease in the intervention arm. We assumed an intra-cluster (clinician) correlation coefficient of 0.1, as is commonly assumed in this type of cluster randomisation study.28 Therefore, we have power to detect a 3 mm Hg decrease in mean of final SBPs in the intervention arm.

Recruitment

This is a pragmatic clinical trial. Once the study period begins, each patient who has an office visit with a PCP and fulfil criteria for CKD and uncontrolled HTN will be automatically enrolled in the study. Both inclusion criteria will be assessed, in real time, for every adult patient who visits a PCP. The patients will be designated as belonging to the intervention or control arm based on the PCP that the patient sees. The subgroup of PCPs included in the pilot study will remain enrolled in the clinical trial; the patients they saw during this period will be excluded from the final analysis.

Allocation

This study will utilise a matched-pair cluster randomised design with the intervention on the cluster level, and the main outcome (6 month minus baseline change in SBP) measured at the patient level. We will have 174 clusters (made up of 184 clinicians) in the study (figure 3). Clusters were made up of either one PCP with their own panel of patients, or two PCPs who share a panel, also known as a ‘comanagement dyad’. We will match pairs of clusters with a similar number of patients and prior year mean blood pressure of patients in the cluster. One cluster in each pair will be randomised to the intervention and the other to usual care.

Figure 3

Participant timeline.

Data collection and follow-up

Patients will be electronically identified and included in the study over the course of 12 months. Patients seen by PCPs during the pilot study will be excluded. Retrospective data indicate that 70% of patients have a follow-up around 6 months. Outcomes assessment will occur at 180 days (±60 days). Clinical outcomes will be recorded and reviewed every month over the course of the trial.

Statistical methods

Descriptive statistics (eg, means and SD, medians and interquartile ranges, frequencies) will be used to examine the distributions of demographic (eg, age, sex, race/ethnicity) and clinical characteristics (eg, comorbidities, baseline eGFR, smoking status). We will compare intervention and control groups on baseline characteristics with Rao-Scott χ2 tests for categorical variables (accounting for clustering) and GEE z-tests for continuous variables (accounting for clustering).29 The primary endpoint is the change in mean SBP between baseline and 6 months compared across arms and the primary analysis will be an intent-to-intervene analysis. SBP at intermediate visits will not be used in this analysis. We will examine the baseline SBP in both intervention and control arms. If we find a difference in baseline mean SBP, we will perform a secondary analysis in which the outcome is SBP at 6 months and in which baseline SBP is included as a covariate. Using the Haybittle–Peto approach, the conventional p-value of 0.05 can be used in the primary analysis. In other words, the p-value for the GEE z-test must be ≤0.05 for the intervention and control groups to be declared significantly different.

Secondary analyses

We will perform a secondary analysis for patients who did not have a subsequent visit after the enrolment visit. We will use multiple imputation to estimate the final SBP for patients who have missing data. We will perform another secondary analysis, as ‘as-treated’ analysis, on patients where the clinician chose to order the recommended medication. We will also perform a secondary analysis at the clinician level. This ‘as-treated’ analysis will include only clinicians who did not choose to opt out after randomisation.

Patient and public involvement

Patients were not involved in the design of this research, since the intervention will be conducted at the clinician level. However, during the trial, patients who have completed the 6-month primary outcome timeframe may be contacted to evaluate their experience as part of the trial. This will only occur after primary outcome data collection has ended, so as not to contaminate any patient data.

Ethical considerations

The Human Subjects Institutional Review Board at Brigham and Women’s Hospital approved this study protocol, and a Data Safety Monitoring Board will ensure the ongoing safety of the trial. Potential harm outcomes will be monitored and addressed as needed. Any relevant protocol modifications will be communicated to relevant parties.

Discussion

During this study, we expect to demonstrate a decrease in mean SBP in CKD patients following the implementation of CDS in the form of an Epic BPA. We also expect to demonstrate that PCPs receiving the live BPA will show higher levels of action in the management of their CKD patients’ HTN than those receiving the silent BPA. Through this pragmatic, cluster-randomised controlled trial, we hope to build a greater base of information in how PCPs interact with CDS in real-world settings, as well as determine some of the most successful strategies for CDS implementation. Taking PCP feedback into consideration will be a vital aspect of this process, and we hope that the utilisation of user input collected during the usability testing sessions will help to decrease negative PCP interactions with the CDS such as alert fatigue. By using the RE-AIM framework, we can inform future implementations of this system in similar settings. The framework will provide an insight on the ideal approach for the application of CDS to assist in the management of chronic diseases in primary care. Potential areas of focus include other multidisciplinary chronic diseases with high HTN comorbidity such as diabetes mellitus, hyperlipidaemia or cardiovascular disease.

There are potential limitations to this study. One is the potential for enrolled PCPs to leave their clinic or transition to urgent care practice. The effect of this attrition would be expected to balance between arms. Contamination within clinics is also possible, as the intervention is randomised at the PCP level.30 However, given the time pressures of primary care practice, it is unlikely that PCPs would spend time discussing CDS. Additionally, although ideally PCPs would be matched and randomised according to the baseline patient SBP at trial enrolment, the constraints of the CDS design made it more efficient and less likely prone to error to assign PCPs to intervention and control arms prior to the start of the trial. The list of PCPs in each arm was hard-coded into the CDS. Another limitation is that PCPs could develop workarounds to the CDS system. Workarounds noted in the pilot phase included PCPs accepting the recommended order within the BPA, leading to a pended order, but then cancelling the pended order. This sequence of events would cause the BPA to refire when the chart was reopened. In the pilot phase, there were also some instances of PCPs indicating ‘other’ as an acknowledge reason and providing invalid free-text explanations (eg, ‘x’ or ‘None’). The main reason for requiring a free-text explanation is to improve the CDS, so this action does reduce the quantity of useful information we will collect. The addition of a second CDS tool at the end of the visit could prevent these workarounds, but the CDS is meant to allow flexibility in situations where the recommendations are not clinically appropriate. Furthermore, adding a second CDS tool would likely contribute to alert fatigue.

At the conclusion of this study, we will have: (1) validated an intervention that combines laboratory tests, medication records and clinical information collected by EHR to recognise uncontrolled HTN in CKD patients and recommend a course of care, (2) tested the effectiveness of said intervention and (3) collected information about the implementation of the intervention that will aid in dissemination of the intervention to other practice settings. There are various factors that contribute to successfully addressing these issues, from the participation of the care team to the specific medications that are prescribed. We believe that results of this trial will provide valuable information regarding the implementation of CDS to help detect and treat CKD and HTN in primary care settings. The lessons learnt in conducting this trial will also serve as references for future endeavours in implementation of CDS for primary care.

Ethics and dissemination

The Human Subjects Institutional Review Board (IRB) at Brigham and Women’s Hospital provided an expedited review and approval for this study protocol, and a Data Safety Monitoring Board will ensure the ongoing safety of the trial. As this is a pragmatic clinical trial randomised at the clinician level, informed consent from individual patients was not required by the IRB, and clinicians were offered the opportunity to opt out of the trial. A manuscript with the results of the primary study will be published in a peer-reviewed journal. Separate manuscripts discussing secondary aims and analyses will be submitted for publication in peer-reviewed journals as well. On completion of the trial, and after the publication of the primary manuscript, data requests can be submitted to the researchers at the Division of General Internal Medicine and Primary Care at Brigham and Women’s Hospital, Boston, Massachusetts.

Table 1

Outcome variables and measures for both arms

Measurement variable	Form of variable	Analysis metric	Time point from first visit
Primary
Mean SBP	Continuous	Change from baseline	6 months
Secondary
Mean SBP	Continuous	Change from baseline	12 months, 18 months
Controlled SBP rate	Dichotomous	Proportion of patients with controlled SBP rate	6 months, 12 months, 18 months
Urine albumin to creatinine ratio	Continuous	Urine albumin to creatinine ratio	6 months, 12 months, 18 months
eGFR	Continuous	eGFR	6 months, 12 months, 18 months
Mean SBP analysis with imputation of missing 6-month BP measurement	Continuous	Change from baseline	6 months
Controlled SBP rate analysis with imputation of missing 6-month BP measurement	Dichotomous	Proportion of patients with missing 6-month BP measurement	6 months
Mean SBP as-treated analysis (patient level)	Continuous	Change from baseline in patients whose clinician ordered the recommended medication	6 months
Mean SBP as-treated analysis (clinician level)	Continuous	Change from baseline in patients whose clinician ordered the recommended medication	6 months
Other Outcomes of Interest
Medication ordered	Dichotomous	Proportion of patients with recommended medication ordered	6 months
Basic metabolic panel ordered	Dichotomous	Proportion of patients with basic metabolic panel ordered	6 months
Referral to nephrology e-consults	Dichotomous	Proportion of patients with referral to nephrology e-consults	6 months
Adverse Clinical Outcomes
Mean SBP >110	Dichotomous	Proportion of patients with mean SBP of less than 110	6 months
Newly documented allergy	Dichotomous	Proportion of patients with newly documented allergy due to adverse drug events	6 months
Serum creatinine >2.0	Dichotomous	Proportion of patients with creatinine >2.0	6 months
K+>5.2	Dichotomous	Proportion of patients with K+>5.2	6 months
K+<3.6	Dichotomous	Proportion of patients with K+<3.6	6 months

BPA, Best Practices Advisories; eGFR, estimated glomerular filtration rate; SBP, systolic blood pressure.

Table 2

Outcome variables and measures for intervention arm only

Measurement variable	Form of variable	Analysis metric	Time point from first visit
Acknowledge reason entered	Dichotomous	Proportion of patients with acknowledge reason entered	6 months
Feedback button clicked	Dichotomous	Proportion of patients with feedback button clicked	6 months
PCP participation on pledge email survey	Dichotomous	Proportion of patients whose PCPs participated in pledge email survey	6 months
Guideline accessed	Dichotomous	Proportion of patients with guideline accessed	6 months
BPA acceptance	Dichotomous	Proportion of patients for whom BPA interaction was not an acknowledge reason	6 months

PCP, Primary Care Practice.