Improving Care for Patients With or at Risk for Chronic Kidney Disease Using Electronic Medical Record Interventions: A Pragmatic Cluster-Randomized Trial Protocol.

BACKGROUND: Many patients with or at risk for chronic kidney disease (CKD) in the primary care setting are not receiving recommended care.
OBJECTIVE: The objective of this study is to determine whether a multifaceted, low-cost intervention compared with usual care improves the care of patients with or at risk for CKD in the primary care setting.
DESIGN: A pragmatic cluster-randomized trial, with an embedded qualitative process evaluation, will be conducted.
SETTING: The study population comes from the Electronic Medical Record Administrative data Linked Database®, which includes clinical data for more than 140 000 rostered adults cared for by 194 family physicians in 34 clinics across Ontario, Canada. The 34 primary care clinics will be randomized to the intervention or control group. INTERVENTION: The intervention group will receive resources from the "CKD toolkit" to help improve care including practice audit and feedback, printed educational materials for physicians and patients, electronic decision support and reminders, and implementation support. MEASUREMENTS: Patients with or at risk for CKD within participating clinics will be identified using laboratory data in the electronic medical records. Outcomes will be assessed after dissemination of the CKD tools and after 2 rounds of feedback on performance on quality indicators have been sent to the physicians using information from the electronic medical records. The primary outcome is the proportion of patients aged 50 to 80 years with nondialysis-dependent CKD who are on a statin. Secondary outcomes include process of care measures such as screening tests, CKD recognition, monitoring tests, angiotensin-converting enzyme inhibitor or angiotensin receptor blocker prescriptions, blood pressure targets met, and nephrologist referral. Hierarchical analytic modeling will be performed to account for clustering. Semistructured interviews will be conducted with a random purposeful sample of physicians in the intervention group to understand why the intervention achieved the observed effects.
CONCLUSIONS: If our intervention improves care, then the CKD toolkit can be adapted and scaled for use in other primary care clinics which use electronic medical records. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT02274298.

What was known before

Patients with chronic kidney disease do not always receive primary care that aligns with recommendations from clinical guidelines. This may be due to knowledge gaps or low awareness of guidelines for chronic kidney disease among family physicians.

What this adds

We will implement a low-cost intervention that includes chronic kidney disease care tools for family physicians to determine whether it helps to improve care for patients. This toolkit can easily be adapted for daily use in a variety of primary care settings.

Background

Approximately 30% of older adults, or 2 million Canadians, are living with chronic kidney disease (CKD).[1] Unfortunately, the prognosis of end-stage kidney disease is worse than most cancers.[2] To prevent the progression to end-stage kidney disease, patients require the best possible care during the earlier stages of CKD.

Patients with early stage CKD are typically managed in the primary care setting and are referred to nephrologists when their kidney function worsens. Guidelines recommend that family physicians monitor kidney function regularly and manage cardiovascular risk factors.[3] A recent clinical guideline strongly recommends that all patients aged 50 years or older who have nondialysis-dependent CKD receive a statin.[4] This recommendation was based on high-quality evidence that statins help to reduce the risk of fatal and nonfatal cardiac events among this population.[5,6]

Quality of Care for Patients With CKD

In practice, family physicians do not always follow CKD guideline recommendations. For example, only 60% of the patients with CKD in one Canadian study were prescribed a statin.[7] Studies in the United States have found that the proportion of patients with CKD on statins range from 30% to 47%.[8-10] A large observational study in the United Kingdom also found that under half of the patients with CKD were prescribed a statin.[11]

To improve quality of care, it is important to determine the perceived barriers to providing appropriate care for patients with CKD among family physicians. An American survey found that compared with nephrologists, family physicians did not recognize that guidelines for CKD existed, disagreed that they have the resources required to provide appropriate care, and did not feel capable of providing care that would slow the progression to end-stage kidney disease. Furthermore, family physicians who had been practicing for more than 10 years were less likely to recognize patients with CKD and to refer to a nephrologist.[12] A qualitative study found that family physicians in New York were unaware of the K/DOQI (The National Kidney Foundation Kidney Disease Outcomes Quality Initiative) guidelines on defining, evaluating, preventing, and treating CKD.[13,14] As such, these physicians reported using outdated methods to diagnose and monitor CKD and were unsure when to refer their patients to a nephrologist.[14] This suggests that knowledge gaps among family physicians regarding CKD management represent an important determinant of the suboptimal care received by patients.

Implementing Best Practices for CKD

Development of evidence-based clinical practice guidelines for CKD is insufficient to produce changes in practice. However, advancements in health care technologies, such as point-of-care reminders, are promising tools to help provide better management for patients with CKD. Actual integration of recommendations into family physicians’ electronic medical records (EMR) through the use of clinical decision support system (CDSS) tools has been shown to be moderately effective at improving care.[15,16] Family physicians are generally more accepting of CDSS when they include the use of simple and concise messages, have limited information that needs to be input by physicians, fit with the users’ workflow, include components that target patients directly, and are developed and tested by family physicians.[17-20]

The few previous studies conducted to assess the effect of CDSS tools to help improve care for patients with CKD have shown variable success.[21-25] Two of these studies focused on patients with more severe CKD, which is not representative of typical patients cared for by family physicians.[21,24] One study was not a randomized clinical trial but rather used a before/after design to compare care prior to and after the intervention for only 2 centers.[22] Another study was also a nonrandomized evaluation of a CKD checklist where 4 of 13 primary care providers were assigned to use this tool.[25] To our knowledge, there have not been any randomized clinical trials in Canada which have used CDSS to improve the care for patients across CKD stages. A previous randomized clinical trial study in the United Kingdom involved workshops and academic detailing.[23] Although this previous study led to practice improvements among family physicians in the management of CKD, scaling strategies that require costly workshops and academic detailing to a larger audience are impractical.[23]

Study Hypothesis

The low-cost implementation in primary care clinics of a multifaceted intervention with CKD “tools” featuring CDSS, a Web site for practice audit and feedback, and customized patient handouts will improve identification and management of patients with stage 3+ CKD compared with usual care.

Methods

Study Design

We will conduct a pragmatic, cluster-randomized trial with 2 arms using a multifaceted intervention. The purpose of a pragmatic trial is to assess the effectiveness of an intervention when used in practice, and is characterized by broad eligibility criteria and a less controlled environment than an explanatory trial.[26] A cluster design will reduce contamination among physicians and patients belonging to the same clinical practice. The unit of randomization will be the primary care clinic/cluster and the unit of analysis will be the patient. Our comparison group will be an active control group for a second clinical trial on atrial fibrillation.[27] We will report the study in accordance with the Consolidated Standards of Reporting Trials (CONSORT) extensions for cluster trials, pragmatic trials, and trials of nonpharmacological interventions.[28-30] We will also use the Template for Intervention Description and Replication (TIDieR) guidelines to describe the intervention.[31] This study was approved by the Sunnybrook Health Sciences Centre Research Ethics Board.

Setting

Ontario primary health care

In Ontario, access to primary health care and laboratory tests is covered under the Ontario Health Insurance Plan program, with no co-pay for these or other medically necessary services. Over the past few years, many family physicians have switched from fee-for-service to a capitation model, where physicians are paid for the number of registered or “rostered” patients in their practice.[32] Outpatient medications from a minimally restricted formulary are covered for patients aged 65 years or older but not for most people under this age, unless they qualify for special disability assistance or other drug benefit programs.

Ontario Renal Network

The Ontario Renal Network, a provincial government organization within Cancer Care Ontario, is responsible for funding and managing the delivery of all CKD services throughout Ontario. This study was funded by the Ontario Renal Network.

Electronic Medical Record Administrative data Linked Database

This study will utilize the Electronic Medical Record Administrative data Linked Database (EMRALD®) housed at the Institute for Clinical Evaluative Sciences (ICES).[33,34] ICES is a “prescribed entity” under Ontario privacy legislation and is able to collect individual-level patient health information without requiring patient consent because it has the policies and procedures in place to maintain patient privacy and confidentiality.[35] At the time of study commencement, EMRALD® included clinical data for 140147 adult patients rostered to one of 194 family physicians in 34 clinics across Ontario using PS Suite® EMR. PS Suite is the leading market share provider of EMR software in Ontario.[36] For our study, we will have 31 clusters for these 34 clinics, because some of the clinics share resources and will be randomized together. Physician participation in EMRALD® is voluntary and recruitment is ongoing. At the time of study initiation, 19% of the clinics were rural, and the physicians included in EMRALD® have been using their EMR for an average of 4½ years.

Participants

Eligible physicians are those participating in EMRALD®, using PS Suite EMR for at least 2 years, and have 100 or more rostered patients. These eligibility criteria are to ensure that physicians have had sufficient experience with their EMR and have adequate patient data to assess baseline and outcome measures. Eligible patients are individuals identified in the EMR as being rostered to one of the participating physicians and who have an EMR record for at least 1 year at the time of allocation. This is to ensure that the patient has enough available data to assess baseline characteristics. Eligible patients also fit into one of the definitions described below for patients at risk for CKD or with stage 3+ CKD.

Patients at risk for CKD

Eligible patients at risk for CKD include individuals who do not have confirmed stage 3+ CKD (as defined in the following section) and not on dialysis but have at least one of the following: a diagnosis of hypertension,[7] diabetes,[37,38] or age above 60 years and below 75 years with ischemic heart disease.[39]

Patients with stage 3+ CKD

Evidence of stage 3+ CKD will be defined as having 2 estimated glomerular filtration rate (eGFR) values <60 mL/min/1.73 m2 separated by at least 3 months based on the definition outlined in clinical guidelines.[3] The eGFR values available in the EMR were calculated at the laboratories using the Modification of Diet in Renal Disease (MDRD) equation.[40] The algorithm for identifying evidence of CKD is based on eGFR tests for eligible patients who are not on dialysis. Evidence of dialysis is found by searching in the cumulative patient profile and past medical history notes of the EMR. The algorithm looks for each patient’s most recent eGFR in the EMR. If the most recent eGFR is normal or only 1 eGFR less than 60 mL/min/1.73 m2 is found in the patient chart, then the patient is flagged as not having CKD. If the most recent eGFR is less than 60 mL/min/1.73 m2, then the algorithm searches for the previous, consecutive eGFR test. If this previous test is also less than 60 mL/min/1.73 m2 but less than 3 months prior to the most recent test, then the algorithm continues searching for the next consecutive abnormal eGFR test. If this test is less than 60 mL/min/1.73 m2 and at least 3 months prior to the most recent test, then the patient is flagged as having CKD. Patients will be excluded if they are below 18 years of age as of the date the patient data were collected (between June 2014 and October 2014).

Allocation

We will allocate clinics to either the CKD intervention group or the active control group based on minimization using the software “MINIM.”[41] Minimization considers the characteristics of participants in each group which provides better balance across important baseline covariates than simple randomization, given the limited number of clusters in this trial.[42] The covariates will include total number of family physicians in each clinic/cluster, clinic rural location, average years of experience, average years on the EMR, number of patients above 65 years, average age of all patients, number of patients with hypertension, number of patients with ischemic heart disease, number of patients with diabetes, number of patients with atrial fibrillation, number of patients with stage 3+ CKD, number of patients meeting blood pressure (BP) target (140/90 or 130/80 mm Hg if patient has diabetes), number of patients 65 years and older with atrial fibrillation or have a CHADS score ≥ 1 (a clinical prediction score based on Congestive heart failure, Hypertension, Age ≥ 75 years, Diabetes, and prior Stroke) and an anticoagulation prescription, and number of patients between 50 and 80 years of age with stage 3+ CKD on a statin unless contraindicated. Allocation will be performed centrally by an independent analyst after recruitment is completed.

Intervention

We will use a multifaceted approach and provide family physicians who are in practices allocated to the intervention group with a “toolbox” of CKD resources. These resources were developed based on a thorough review of the literature, expert advice from the Ontario Renal Network, family physicians and nephrologists, and a modified Delphi panel to identify quality of care indicators for CKD.[7] Physicians are not required to use these tools; in this pragmatic trial, we are interested in how simply providing family physicians with access to appropriate resources and tools can influence the care that they provide for patients with CKD.

Intervention components include the following:

System for Audit and Feedback to Improve caRE (SAFIRE) web-based feedback on quality indicator performance for CKD:

All EMRALD® physicians already have access to the SAFIRE. SAFIRE is available through a secure Web site to provide confidential, individualized performance reports to physicians regarding their quality of care for patients with certain chronic diseases (eg, hypertension, diabetes, and ischemic heart disease). Clinics that are randomized to the intervention group will receive access to SAFIRE for CKD, which will provide audit-and-feedback reports based on CKD quality of care indicators. Data from the EMRs will be extracted approximately every 6 months to generate the SAFIRE reports. Physicians in the intervention group will then be notified that their reports are available to be viewed. These reports include (1) their performance in comparison with the top 10% of other EMRALD® physicians, (2) their performance in comparison with the top 10% of colleagues within their own practice site for physicians that practice in a group of 3 or more, (3) their clinic performance compared with other participating EMRALD® clinics, and (4) changes in their performance over time. Physicians are able to view their data from 2 perspectives: green “at target” graphs show how many of their patients are meeting guideline-based targets and red “high risk” graphs show how many of their patients are well beyond targets, at highest risk for adverse events and should be prioritized for addressing. In addition, SAFIRE allows physicians to download a list of their patients and import it into their EMR system, which then allows them to select a patient and pull up the patient chart within the EMR. Finally, physicians are also able to receive Continuing Medical Education credits (Main-Pro C) through this system. Examples of “at target” and “high risk” practice-level reports for CKD are shown in Figures 1 and 2, respectively.

Figure 1.

System for Audit and Feedback to Improve caRE example screenshot of chronic kidney disease performance report at the practice level: At target.

Figure 2.

System for Audit and Feedback to Improve caRE example screenshot of chronic kidney disease performance report at the practice level: High risk.

The CKD quality of care indicators used in SAFIRE were identified through a modified Delphi panel funded by the Ontario Renal Network.[7] This technique has been used previously to successfully identify consensus-based quality indicators for cardiac care.[43,44] We excluded the top 2 performing indicators as there was little room for improvement. We also excluded the indicator that examined serum potassium test ordering after initial angiotensin-converting enzyme (ACE) inhibitor/angiotensin receptor blocker (ARB) prescription as we were uncertain in our ability to capture initial prescription. We modified the time periods for indicator assessment and positive or negative perspective for some of the indicators. For example, rather than looking to see whether patients have a BP measure in the past 12 months, we changed this to 15 months to provide some flexibility because BP monitoring or other tests may not be completed within a 365-day time frame. Finally, we included the indicator on whether or not patients had a flu shot in the SAFIRE patient list, but this was not included in the physician performance reports. These SAFIRE adapted indicators are outlined in Table 1, along with the proportion of patients in our study at baseline who were meeting these indicators and the proportion at high risk.

Table 1.

Quality of Care Indicators for Patients at Risk for Chronic Kidney Disease or With Stage 3 or Greater Chronic Kidney Disease as Identified by the Modified Delphi Panel and Proportion of Patients in EMRALD® at Baseline Achieving These Indicators (At-Target) or Identified as High-Risk.

Criteria for on target	%	Criteria for high risk	%
Indicators for patients at risk for CKD
Percentage of patients with risk factors for CKD and an eGFR in the past 18 months	73.6	Percentage of patients with risk factors for CKD and no eGFR in the past 24 months	19.8
Percentage of patients with risk factors for CKD and an ACR in the past 18 months	28.0	Percentage of patients with risk factors for CKD and no ACR in the past 24 months	68.4
Percentage of patients with an eGFR <60 mL/min/1.73 m2 and another eGFR in ±6 months	47.5	Percentage of patients with an eGFR <60mL/min/1.73 m2 and no eGFR in ±12 months	32.1
Percentage of patients with an eGFR <60 mL/min/1.73 m2 and an ACR in ±6 months	16.3	Percentage of patients with an eGFR <60 mL/min/1.73 m2 and no ACR in ±12 months	80.9
Percentage of patients with diabetes and albuminuria with a prescription for an ACE inhibitor/ARB	74.7	Percentage of patients with diabetes and albuminuria with no prescription for an ACE inhibitor/ARB	25.3
Indicators for patients with stage 3+ CKD
Percentage of patients with an ACR in the past 18 months	34.2	Percentage of patients with no ACR in the past 24 months	62.0
Percentage of patients 50 to 80 years with a prescription for a statin	60.4	Percentage of patients 50 to 80 years with no prescription for a statin	39.6
Percentage of patients with a BP recorded in the past 15 months	91.6	Percentage of patients with no BP recorded in the past 24 months	3.9
Percentage of patients with at least one BP measure in the past 15 months meeting BP targets (<140/90 mm Hg for patients <80 years, <150/90 mm Hg for ≥80 years and <130/80 mm Hg for patients with diabetes)	78.2	Percentage of patients with at least one BP measure in the past 15 months with BP >160/100 mm Hg	5.6
Percentage of patients <80 years with an eGFR <30 mL/min/1.73 m2 and who have been seen by a nephrologist	66.7	Percentage of patients <80 years with an eGFR <30 mL/min/1.73 m2 and who have not been seen by a nephrologist	33.3
Percentage of patients with CKD documented in the CPP	27.1	Percentage of patients with CKD not documented in the CPP	72.9
		Percentage of patients with ≥1 prescription for NSAIDs	1.4
		Percentage of patients with a prescription for an ACE inhibitor and ARB on the same day	0.7

Note. EMRALD® = Electronic Medical Record Administrative data Linked Database; CKD = chronic kidney disease; eGFR = estimated glomerular filtration rate; ACR = albumin to creatinine ratio; ACE = angiotensin-converting enzyme; ARB = angiotensin receptor blocker; BP = blood pressure; CPP = cumulative patient profile; NSAID = nonsteroidal anti-inflammatory drug.

EMR-based clinical decision support and reminders:

We have developed a custom form that can be embedded into PS Suite EMR based on the Ontario Renal Network’s KidneyWise clinical algorithm that can be accessed to automatically generate the appropriate pathway of care based on an abnormal eGFR or albumin to creatinine ratio (ACR; ie, recommended screening, management and referral practices; see Figure 3).[45]

Figure 3.

Custom form for recommended identification, detection, and management of chronic kidney disease in primary care based on the Ontario Renal Network’s KidneyWise Algorithm.

Based on the quality of care indicators for CKD, we developed reminders for physicians to install in their EMR (see Online Appendix 1). Because reminders in the PS Suite EMR cannot be built to detect the two most recent eGFR or ACR values, these reminders will pop up based on the most recent eGFR or ACR values. Some reminder algorithms may require modification by the physicians depending on what nomenclature they use to record the various diseases and how their laboratory test results are labeled in their EMR. For reminders that relate to patients with known CKD, we identify these patients by recording of CKD in the problem list, or the most recent eGFR <60 mL/min/1.73 m2.

Patient education:

Studies have shown that CDSS interventions aimed at the patients in addition to physicians are more successful than interventions aimed solely at physicians.[20] Patient handouts are aimed at patients who are newly diagnosed with CKD and can be autopopulated based on data in the EMR (see Online Appendix 2). Physicians can download these forms into their EMR and provide them to patients. These personalized reports describe the function of the kidneys, what it means to have CKD, recommendations to lifestyle modifications (ie, diet, exercise, and smoking), as well as recommended medical changes to discuss with their physician. Included in the handout is a Web site link to the Kidney Foundation of Canada, which includes electronic patient educational brochures on important aspects of CKD including appropriate nutrition,[46] as well as a Web site link to the Canadian physical activity guidelines.[47]

Implementation support and physician education:

At initiation of the trial, physicians will be notified by e-mail that their SAFIRE feedback is available along with the toolkit and installation instructions and a physician handout from the Ontario Renal Network KidneyWise program (see Online Appendix 3). Clinics will also be offered a group presentation through a webinar, as well as one-to-one telephone correspondence and e-discussions with research team members with both technical and clinical expertise on the content and installation of the toolkit. An infographic developed to encourage physicians to use the toolkit will be e-mailed to physicians with the second round of feedback to remind physicians to use the CKD tools (see Online Appendix 4). This second round of feedback, showing current performance on quality indicators and changes over time, will be done after the EMR data are reextracted, approximately 6 months after the initial feedback and tool dissemination.

Usual Care

In this pragmatic trial, the clinics allocated to usual care will not receive the CKD toolkit but their approach to screening and/or management of CKD will not be otherwise standardized. We are concomitantly conducting a similar trial with an atrial fibrillation toolkit, such that consenting physicians will be randomized to either the CKD intervention group or the atrial fibrillation group (ie, an active control group).[27] Thus, the intervention group for CKD will serve as the atrial fibrillation control group and vice versa.

Study Outcomes

All outcomes will be assessed approximately 6 months following the second round of feedback and reminder to use the CKD tools. This time frame was chosen because data are extracted approximately every 6 months and updated SAFIRE reports are disseminated with every data extraction. The primary outcome will be the proportion of patients aged 50 to 80 years with stage 3+ CKD who are on a statin. This will be our primary outcome because statin use at baseline was suboptimal (60%), and recent clinical guidelines recommend that all patients aged 50 or older who have nondialysis dependent CKD should receive a statin.[4,5]

Secondary outcomes will include the proportion of patients (1) at high risk who are screened for CKD with an eGFR and/or an ACR test, (2) with an initial eGFR less than 60 mL/min/1.73 m2 and a follow-up eGFR and/or ACR test, (3) with diabetes and albuminuria and taking an ACE inhibitor or ARB, (4) meeting eGFR criteria for CKD and documentation of CKD in the problem list in the EMR, (5) meeting eGFR criteria for CKD with an ACR test in the past 18 months, (6) meeting eGFR criteria for CKD and meeting BP targets, and (7) aged below 80 years and an eGFR less than 30 mL/min/1.73 m2 being seen by or with a referral to a nephrologist.

Implementation outcomes will be assessed approximately 6 months after 2 rounds of feedback to determine the frequency by which physicians in the intervention group used the tools, including number of times they accessed SAFIRE, how many physicians uploaded the tools to their EMRs, and how many used the tools with their patients.

Data Collection

All patient variables in this trial, including baseline characteristics, outcomes, and covariates, will be extracted from the physicians’ EMRs. Patient identifiers will be removed before the data are securely transferred to the analyst who will be performing data quality checks and cleaning as per standard EMRALD® protocol.[33,34] The analyst will not be able to identify the patients and will be blinded to the allocation of the clinics. By using an EMR to obtain patient data, we will avoid unnecessary follow-up and clinical assessments, which will substantially reduce trial costs, as well as reduce the burden on physicians and patients. This is also consistent with the pragmatic design of the study.

Recruitment and Timeline

All eligible family physicians in EMRALD® have already signed consent to participate in our trial. After all clinics have been randomized, participating physicians who are in the clinics allocated to the intervention group will receive the CKD tools. Uptake of the tools will be evaluated, and updated feedback will be sent at baseline and 2 additional times at approximately 6 month intervals.

Analysis

Baseline variables for clinic, physician, and patient characteristics will be summarized. Continuous variables will be reported using means and standard deviations for normally distributed variables and medians and interquartile ranges for nonnormally distributed variables. Binary and categorical variables will be described using counts and percentages.

Primary and secondary outcomes will be analyzed at the patient level, so hierarchical modeling approaches will be performed to account for the effects of clustering.[48] We will report the intracluster correlation coefficient for each outcome. We will restrict the analysis to eligible patients who were rostered to the study physicians throughout the entire duration of the trial. We will use generalized estimating equations to estimate the relative risks and associated 95% confidence intervals. Random effect variables will be included in these models to account for the clustering of the patients within physicians and clustering of physicians within clinics. As secondary analyses, adjusted models will also be built for each outcome controlling for each outcome measure at baseline and the variables that will be used to perform the covariate-constrained randomization. An “exposure” variable will also be included in these adjusted models that accounts for whether or not physicians in the intervention group used the tools based on our implementation outcomes. Finally, an “as-treated” sensitivity analysis will be performed where we restrict the analysis to only the clinics in the intervention group that used the tools. No interim or subgroup analyses will be completed.

At the time of study initiation, there were 34 eligible clinics in EMRALD® and approximately 140 000 rostered patients who were 18 years of age or older, with 7315 of these patients having stage 3+ CKD. The estimated intraclass correlation coefficient for statin prescribing in primary care has been described for similar patients in the literature to be 0.019.[49] Based on these estimates and 5% alpha, we would require 16 clinics and 3424 patients to have 80% power to detect a 10% increase in the proportion of patients prescribed statins from baseline, which is a reasonable expectation based on findings from previous studies using similar interventions.[50]

In cluster-randomized trials, the majority of the statistical power is based on the number of included clusters. In our study, we do not expect any clinics to drop out, because clinic site leads are committed to this study. We expect some physicians to drop out of the study due to moving, retirement, or death, and some patients who will be lost to follow-up due to the dropout of their physician, change of family physician, or death; however, this will only minimally affect the statistical power of the study. To account for missing patient data in our analyses, we will use cluster mean imputation, which has been shown to be valid for studies with larger cluster sizes and missing data only at the individual level.[51]

Process Evaluation

For pragmatic trials that include complex interventions, it is important to qualitatively evaluate how the intervention could be improved and why it may not have worked as well as anticipated (especially if null results are found).[52,53] To assess this in our trial, we will use random purposeful sampling[54] to select approximately 10 to 20 physicians (depending on when data saturation is reached) in the intervention group to participate in semistructured interviews after study completion. The interview strategy will be guided by the normalization process theory[55] and will include questions regarding how often the physicians used the CKD tools, which ones they found most useful, and whether the intervention improved their attitudes and behaviors regarding care for patients with CKD. Each interview will be approximately 30 minutes and will be audio recorded and transcribed. NVivo software will be used to assist with coding, which will be performed using thematic analysis where the data from the interviews will be mapped to the theoretical constructs of the normalization process theory.[56]

Discussion

Feasibility and Limitations of Study Methodology

The family physicians who will participate in this study are individuals who use PS Suite EMR and have volunteered to participate in EMRALD®. Although previous analysis has shown EMRALD® physicians and patients to be similar across most characteristics to Ontario family physicians and Ontario patients,[34] the results of this study may not be generalizable to physicians who do not use an EMR or who use other EMR systems, although the multifaceted intervention in this trial can be applied to other EMRs. Furthermore, individuals in EMRALD® may be participating in other quality improvement strategies, so they may already be performing at a higher level of care than other family physicians.

The quality of data in EMRs is likely not at the standard of data typically collected for the purpose of a clinical trial, as we are using real world data. Furthermore, results will only be available for data that are captured in the EMR. This limitation prevents measurement of possible adverse events that may result from our trial, such as increases in the incidence of statin-associated myopathy or increased incidence of falls due to more intensive management of BP. However, we expect measurement bias to be consistent across treatments groups, so this would not affect the assessment of any differences in our specified outcomes between the intervention groups. Furthermore, we have conducted previous validation studies on other major chronic conditions and found the specificity to be close to 100% and sensitivity high.[57-59] The secondary use of EMR data, originally used for clinical purposes, to assess outcomes is a key feature of the pragmatic design and will substantially reduce the cost of conducting this trial. Registry-based trials such as this one are becoming more popular, and issues with methodology are being investigated and improved.[60,61]

Although a larger trial with longer follow-up would be powered to detect more clinically relevant outcomes such as end-stage kidney disease, the cost and time required to conduct such a trial would not be feasible. This pragmatic trial is feasible to do in a short time frame for a low cost and assess the impact of a multifaceted intervention on an important surrogate outcome.

Potential Impact of Trial

If our trial shows the potential to improve patient outcomes and process of care measures, then the CKD toolkit can easily be implemented into daily practice and also adapted for a variety of primary care settings and other EMRs at a low cost. Widespread use of the CKD toolkit can provide improved care for patients with CKD all throughout Ontario and is economically feasible, in contrast to other expensive quality improvement interventions such as academic detailing and workshops. The process evaluation component of this study will allow us to better understand the barriers that family physicians may encounter, which may limit their ability to provide the best possible care to patients with CKD. This will allow us to further improve the CKD tools. Finally, the methodology for this study can easily be extended to other complex chronic conditions that are managed in the primary care setting.