Impact of electronic clinical decision support on adherence to guideline-recommended treatment for hyperlipidaemia, atrial fibrillation and heart failure: protocol for a cluster randomised trial.

INTRODUCTION: Clinical practice guidelines facilitate optimal clinical practice. Point of care access, interpretation and application of such guidelines, however, is inconsistent. Informatics-based tools may help clinicians apply guidelines more consistently. We have developed a novel clinical decision support tool that presents guideline-relevant information and actionable items to clinicians at the point of care. We aim to test whether this tool improves the management of hyperlipidaemia, atrial fibrillation and heart failure by primary care clinicians. METHODS/ANALYSIS: Clinician care teams were cluster randomised to receive access to the clinical decision support tool or passive access to institutional guidelines on 16 May 2016. The trial began on 1 June 2016 when access to the tool was granted to the intervention clinicians. The trial will be run for 6 months to ensure a sufficient number of patient encounters to achieve 80% power to detect a twofold increase in the primary outcome at the 0.05 level of significance. The primary outcome measure will be the percentage of guideline-based recommendations acted on by clinicians for hyperlipidaemia, atrial fibrillation and heart failure. We hypothesise care teams with access to the clinical decision support tool will act on recommendations at a higher rate than care teams in the standard of care arm. ETHICS AND DISSEMINATION: The Mayo Clinic Institutional Review Board approved all study procedures. Informed consent was obtained from clinicians. A waiver of informed consent and of Health Insurance Portability and Accountability Act (HIPAA) authorisation for patients managed by clinicians in the study was granted. In addition to publication, results will be disseminated via meetings and newsletters. TRIAL REGISTRATION NUMBER: NCT02742545. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The main strength of this study is the design of the clinical decision support tool. The tool was developed with iterative clinician input to fit the workflow. And it uses complex data in its algorithms (such as risk scores and information only available through natural language processing) to enable the provision of individualised treatment recommendations for patients with three distinct yet common cardiovascular conditions. It efficacy will be evaluated with a rigorous study design using clinically meaningful endpoints.

The main limitations are that the study may lack generalisability—as the clinician decision support tool is a proprietary system, not yet widely available, tested in an academic medical centre.

Background

Cardiovascular disease is the leading cause of morbidity and mortality in the USA.1 Prevention and treatment guidelines for cardiovascular diseases aim to improve outcomes and cost effectiveness.2–4 However, successfully incorporating these guidelines into clinical practice remains a challenge.

Some of the largest gaps between evidence-based guidelines and clinical practice exist for the management of hyperlipidaemia, atrial fibrillation and heart failure.5 It is estimated that only 41% of the 61.8 million patients in the USA who are eligible for statins under the 2013 American College of Cardiology/American Heart Association (ACC/AHA) guidelines are receiving these medications.6 Of the 2.8 million patients with heart failure and left ventricular ejection fraction (LVEF) <40%, rates of compliance with the American College of Cardiology Foundation (ACCF)-AHA guidelines are as low as 7.3% for some measures.7 Similarly for atrial fibrillation, guideline compliance is suboptimal.8 9

Many interventions have been designed to address these gaps between evidence and practice, yet most achieve only minimal gains in guideline adherence.10 Although informatics-based clinical decision support tools have shown promise, most interventions aimed at improving the medical management of cardiovascular disease have shown only minimal gains in compliance to recommended treatment, with a recent systematic review showing a median improvement of 2.0% across interventions.11 Among studies looking at interventions aimed at improving prescribing practices for hyperlipidaemia, atrial fibrillation and heart failure, results ranged from no effect to 13.3% improvement.12–15 The most successful intervention was unique in that the tool provided clinicians with all the information needed to make a decision within the application. None of the algorithms used by the tools were particularly sophisticated in their ability to take into account common clinical conditions that might alter treatment recommendations. Only two of the four interventions were integrated into the electronic medical record (EMR).

Using the lessons learned from these studies and also from other studies of clinical decision support for chronic disease management,16–18 we devised a tool that took into account individual patient factors that might alter treatment recommendations and presented these factors along with calculated risk scores, decision aids and educational materials when a treatment recommendation was made. Clinician input was sought throughout the design process and when deciding on how to integrate the tool into the clinician workflow.

In this paper, we first describe the development of this clinical decision support tool, MayoExpertAdvisor and then outline the protocol for an ongoing clinical trial to formally evaluate its effectiveness. We hypothesise that MayoExpertAdvisor will increase clinician adherence to best practice treatment recommendations for patients with hyperlipidaemia, atrial fibrillation and heart failure.

Development and validation of MayoExpertAdvisor

Like many institutions, Mayo Clinic develops and implements institution-wide guidelines to standardise patient care. Mayo Clinic’s care process guidelines incorporate expert opinion, best practices supported by professional organisations, and local standards of care while allowing for individualisation of treatment. These guidelines are available to Mayo Clinic clinicians through a multifaceted computer-based tool called AskMayoExpert. AskMayoExpert is available via local Intranet to all clinicians at Mayo Clinic and its affiliated healthcare organisations. While the knowledge contained in such care process guidelines is a useful reference for clinicians, a fundamental challenge is faced when clinicians must look up data within the EMR and/or calculate risk scores to apply care algorithms.19 20 As an adjunct tool to complement AskMayoExpert, MayoExpertAdvisor was developed to automate these tasks. MayoExpertAdvisor pulls data from the EMR and calculates risk to determine appropriate care. It then notifies the clinician when care differs from the guidelines by delivering action-oriented recommendations, along with a brief justification, in real time at the point of care. MayoExpertAdvisor is embedded in the EMR to facilitate integration into the clinical workflow.

Institutional guidelines for hyperlipidaemia, atrial fibrillation and heart failure

Using the process described above, institutional guidelines for management of hyperlipidaemia, atrial fibrillation and heart failure were developed. Publicly available resources used as a basis for the guidelines are listed in table 1. Generally, Mayo’s institutional guidelines mirror nationally recognised best practices. For example, the institutional guideline for hyperlipidaemia uses the 2013 ACC/AHA guidelines on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk as a starting point but also incorporates other factors, such as the patients Framingham 30-year risk of atherosclerotic cardiovascular disease (ASCVD) and the use of a decision aid to discuss the pros and cons of statin therapy with a patient.21 22 The institutional guidelines used to develop MayoExpertAdvisor for hyperlipidaemia, atrial fibrillation and heart failure are included in the online supplementary appendices 1–3.

Table 1

Resources used for the development of institutional guidelines

Condition	Sources
Hyperlipidaemia	2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk3
	A summary and critical assessment of the 2013 ACC/AHA Guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular disease risk in adults21
Atrial fibrillation	2014 AHA/ACC/Heart Rhythm Society Guidelines for the Management of Patients with Atrial Fibrillation2
Heart failure	2013 ACC Foundation/AHA guideline for the management of heart failure4

ACC, American College of Cardiology; AHA, American Heart Association.

MayoExpertAdvisor development

Operationally, MayoExpertAdvisor (MEA) captures the patient-specific data required to determine appropriate care recommendations from the EMR and then presents relevant data alongside treatment suggestions to clinicians. To develop MayoExpertAdvisor, a team of clinicians and information technology (IT) experts worked together to translate the institutional guidelines into computational logic. Then, the IT team developed processes to automatically gather from the EMR the data needed to run the algorithms. For data not optimally captured from discrete sources (such as the International Classification of Diseases Ninth Revision (ICD-9) or 10 codes), natural language processing (NLP) was used. For example, common comorbidities were captured both from ICD-9 or 10 codes and from free text within problem lists. Some inputs such as alcohol use to determine bleeding risk were almost exclusively captured through NLP as this is not routinely coded in a discrete format by clinicians.

Clinicians worked with user experience designers to develop the user interface. A user-centred design process in which the team researched the needs of clinicians and aligned the design of the application to meet those needs was used. This process started with collaborative codesign working sessions, in which information needs and decision-making flows were elicited from clinicians and user interface design concepts were created. Industry best practice design principles for usability23 were used throughout the design process. Once an initial design was agreed on, the team conducted task-based usability testing to uncover any issues users might have in completing key tasks with the design. The user interface design was then revised based on the findings from this usability study to better meet user needs.

Validation of MEA

Level 1: receiving the correct data

MEA requires complex data sources, such as risk scores and information only retrievable through NLP. The accuracy of each data element needed to run MEA algorithms was tested until the error rate was less than 5% in 30 consecutive patient records.

Level 2: testing of the logic

MEA logic was tested on patients with hyperlipidaemia, atrial fibrillation and heart failure and the automated recommendations were compared with clinician chart review. The IT team was notified when any discrepancies were found so that programming errors could be resolved. This process was repeated until no errors were observed in 200 patient records.

Level 3: testing of user interface

A formal usability study was conducted, in which 12 clinicians walked through clinical scenarios using a working MEA prototype. Clinicians were observed by a facilitator who recorded any difficulties noted and also asked clinicians a set of standardised questions at the end of each scenario and solicited general feedback about the tool. Based on study findings, the user interface was redesigned to address observed difficulties navigating the tool and clinician concerns. For example, there was a general lack of clarity among participants of where to look in the interface for which information. To address this issue, the information was reorganised around conditions rather than by type of information (eg, care recommendations, risk scoring tools and knowledge resources) and the visual hierarchy of information was adjusted by changing font treatments and adding graphical elements such as icons to direct the user’s vision. Another issue was users needing to see current therapies being used for the patient, but not having this information available in the tool. This was addressed by adding this information into the interface for each condition.

Level 4: pilot

MEA was tested in a clinical setting with 26 primary care physicians for 4 weeks. User feedback was informally solicited throughout the pilot and formally collected through a postpilot survey. Survey results indicated clinicians were generally satisfied with the tool, but thought care recommendation notification process, though email, did not fit their workflow. They also indicated they often did not follow a care recommendation due to a comorbidity not captured by the tool, with the most often cited comorbidity being statin intolerance. Additional capabilities for recognising comorbidities were built into the tool after the pilot (ie, identifying statin intolerance), and the tool was further integrated into the EMR through the additions of coloured MEA alerts visible within the EMR that notified the clinician of the presence of a MEA recommendation.

MEA for hyperlipidaemia, atrial fibrillation and heart failure

Clinicians are notified in several locations in the EMR when a recommendation is available for a patient (see figure 1). When a clinician clicks on an alert or the MEA tab in the EMR, a screen opens that reveals the treatment recommendation along with relevant patient data, risk scores, decision aids and educational materials (see figure 2). Clinicians can manipulate patient data to visualise how lifestyle modification (eg, smoking cessation) or treatment changes (eg, starting a drug or changing drug dose) will affect risk. These manipulations are only temporary and are not stored in MEA or in the EMR.

Figure 1

MEA alerts. The EMR notifies the clinicians of MEA recommendations in three ways. (1) In the patient appointment section, MEA is seen with the number of recommendations in parentheses next to the patient’s name. (2) MEA also appears as a clinical alert. (3) Within the EMR banner, there is also a MEA alert that opens the MEA screen. EMR, electronic medical record; MEA, MayoExpertAdvisor.

Figure 2

MEA user interface. (A) Care recommendation. Depending on the individual patient’s date in the EMR, MEA makes a recommendation. (B) Vitals. Most recent outpatient vital signs. (C) Relevant patient data. The most relevant demographics, conditions and lab results for managing the given condition. (D) Resources for next steps. Additional condition-specific tools (eg, list of moderate and high-intensity statins) to assist in recommendations. (E) Risk calculators. Condition-specific risk calculators with a patient’s data prefilled for real-time calculations. (F) Decision aids. Mayo-vetted shared decision-making tools. Field values are prefilled with patient data. ACC ASCVD, American College of Cardiology Atherosclerotic Cardiovascular Disease; EMR, electronic medical record; LDL, low-density lipoprotein; MEA, MayoExpertAdvisor.

There are several treatment recommendations that MEA can give for each cardiac condition. For hyperlipidaemia, MEA suggests a specific statin intensity. For atrial fibrillation, MEA decides in favour or against anticoagulation based on the CHA2DS2-VASc24 and the HAS-BLED scores.25 For heart failure, MEA recommends (1) initiation of beta blocker therapy, (2) uptitration of beta blocker therapy, (3) initiation of ACE inhibitor/angiotensin receptor blocker (ARB) therapy (4) and uptitration of ACE inhibitor/ARB therapy. MEA also notes when patients with the above conditions are well managed according to institutional guidelines, but this does not generate an alert. For a detailed list of possible MEA recommendations, please see table 2.

Table 2

MEA recommendations

Condition	Recommendations
	Type	Text
Hyperlipidaemia	Low to moderate-intensity statin	Consider low to moderate-intensity statin therapy due to diabetes and elevated cholesterol
		Consider moderate-intensity statin therapy due to diabetes and elevated risk
		Consider moderate-intensity statin therapy due to ASCVD (age>75)
		Consider moderate-intensity statin therapy due to elevated risk
	Moderate to high-intensity statin	Consider moderate to high-intensity statin therapy due to ASCVD
		Consider moderate to high-intensity statin therapy due to LDL ≥190
		Consider moderate to high-intensity statin therapy due to diabetes and elevated risk
	Lifestyle modification	Therapeutic lifestyle modifications recommended due to elevated triglycerides and risk of pancreatitis
	Fibrate/omega 3	Fibrate and/or omega 3 therapy recommended due to elevated triglycerides and risk of pancreatitis
	Well managed	Lipids are controlled on current medication
Atrial fibrillation	No anticoagulation	No anticoagulation therapy indicated due to CHA2DS2-VASc score=0
	Initiate anticoagulation	Consider anticoagulation therapy, if HAS-BLED score <3, due to CHA2DS2-VASc score=1
		Consider anticoagulation therapy, if HAS-BLED score <3, due to CHA2DS2-VASc score 2 or greater
	Well managed	Patient on anticoagulant well managed according to guidelines for management of atrial fibrillation
Heart failure	Initiate beta blocker	Consider beta blocker due to heart failure
	Initiate ACE inhibitor or ARB	Consider ACE inhibitor or ARB due to heart failure
	Titrate ACE inhibitor or ARB	Consider increasing dose of ACE inhibitor or ARB to target dose for heart failure, monitor creatinine and potassium levels
	Titrate beta blocker	Consider increasing dose of beta blocker to target dose for heart failure
	Titrate aldosterone blocker	Consider increasing dose of aldosterone blocker to target dose for heart failure, monitor creatinine and potassium levels
	Medication contraindicated	Diltiazem contraindicated because of heart failure
		Verapamil contraindicated because of heart failure
		Non-steroidal anti-inflammatory drugs contraindicated because of heart failure
		Rosiglitazone contraindicated because of heart failure
		Cilostazol contraindicated because of heart failure
		Itraconazole contraindicated because of heart failure
		Dronedarone contraindicated because of heart failure
		Flecainide contraindicated because of heart failure
		Propafenone contraindicated because of heart failure
	Well managed	Patient is on appropriate dose of beta blocker
		Patient is on appropriate dose of ACE inhibitor or ARB
		Patient is on appropriate dose of aldosterone blocker.

ARB, angiotensin receptor blocker; ASCVD, atherosclerotic cardiovascular disease; LDL, low-density lipoprotein; MEA, MayoExpertAdvisor.

Methods

Study design

MEA is undergoing evaluation in an ongoing cluster-randomised, non-blinded clinical trial with clinician care teams as the unit of randomisation. If MEA is effective at the end of the trial, it will be made available to both the intervention and control arm.

Setting

The study is being conducted at four geographically distinct primary care practice sites affiliated with the Mayo Clinic in Rochester, Minnesota, an academic medical centre.

Eligibility criteria

All primary care clinicians (physicians, nurse practitioners and physician assistants) practising in the divisions of Family Medicine or Primary Care Internal Medicine at Mayo Clinic in Rochester, Minnesota were eligible to participate. Physicians in training (ie, medical students and residents) were excluded. Patient data were used for determination of study outcomes. To be included for data extraction purposes, patients needed to have one of the specified clinical conditions and to be in the panel of one of the clinicians in the trial. All patients seen by eligible clinicians are Olmsted County residents and thus the demographics reflect the local population. Patients who did not authorise the use of their medical record for research were excluded.

Recruitment of clinicians

Clinicians were recruited through presentations at division meetings, emails, phone calls or by individual face-to-face contact. If they indicated they were interested in participating, a meeting with a study team member was scheduled to review the details of the study. Any individual who indicated they did not wish to participate in the study was placed on a do not contact list and no further contact was initiated. To avoid potential coercion, clinicians were not recruited by anyone in a supervisory or position of perceived power over them.

Randomisation

Clinicians were cluster randomised by care teams to minimise the potential contamination that could occur if multiple members of a care team saw the same patient during the study period. Care teams varied in size from five to eight clinicians. There were 20 care teams representing three practice types (family medicine, internal medicine and combined family medicine and internal medicine), located at four distinct clinical sites. Prior to randomisation, these 20 teams were stratified into six strata to create balance of practice type and physical location. There was one mixed strata with three family medicine and one internal medicine care team pooled from two physical locations; the remaining five strata were homogeneous in physical location and practice type. Each stratum comprised two, four or six care teams. A stratified, blocked randomisation schedule was generated by the study biostatistician (REC) to produce an equal number of care teams per condition using a randomly generated code. The randomised assignments remained with the study biostatistician until the study start, at which time access to MEA was granted, if applicable, and consented clinicians were notified by study staff of the group assignment.

Clinician education (both groups)

All clinicians received a 4 min online educational module that reviewed the current institutional guidelines for management of hyperlipidaemia, atrial fibrillation and heart failure.

Intervention group

Clinicians in care teams assigned to the intervention arm received training by the research team on the use of MEA were given access to MEA and will be alerted of MEA care recommendations throughout the study period.

Control group

Clinicians in care teams assigned to the standard of care arm continue to have access to institutional guidelines through the AskMayoExpert knowledge resource but do not have access to the MEA screen. MEA recommendations are run in the background for data collection purposes (see ‘Date sources’ section below), but control group clinicians do not have access to these recommendations.

Data sources

Baseline assessment

The MEA algorithm was run on all eligible patient records at the start of the study. The specific nature of the MEA recommendations for each patient (if any) were recorded; if patients were already ‘well managed’ this was also recorded. These data will be used to assess the baseline adherence to the institutional guidelines for hyperlipidaemia, atrial fibrillation and/or heart failure.

Daily previsit MEA assessment of patient records

The MEA algorithm is applied to all eligible patients the evening prior to a scheduled outpatient visit. As with the baseline assessment, the daily previsit assessment includes the specific nature of the MEA recommendations for each patient and also which patients are well managed. These data will be used to assess the previsit adherence to the institutional guidelines for hyperlipidaemia, atrial fibrillation and/or heart failure.

Daily postvisit MEA assessment of patient record

The MEA algorithm is applied the evening after the visit to the same patients to whom the MEA algorithm was applied the evening prior and again extracts information about MEA recommendations for each patient. These data will be used to assess the postvisit adherence to the institutional guidelines for hyperlipidaemia, atrial fibrillation and/or heart failure.

Study completion MEA assessment

The MEA algorithm will be run on all eligible patients at the completion of the study to provide a similar set of data as described for the Baseline MayoExpertAdvisor Assessment.

MEA application measures of use

Clicks to view MEA recommendations are being tracked by standard timing and click metrics and are stored as a component of the application’s operation.

Survey data

A nine-item survey was sent to all study clinicians at the start of the study ascertaining (1) efficiency in managing the three clinical conditions; (2) frequency of using risk scores in managing the three clinical conditions and (3) the overall use/burden of clinical decision support tools. The same survey will be administered at 2-month intervals throughout the study. At the completion of the study, we will ask clinicians in the intervention arm to complete a 25-item survey to evaluate their impressions of MEA in domains of effectiveness, adoption, implementation, maintenance, usability and overall satisfaction. We will also solicit specific suggestions for improvement.

Primary outcome

The primary endpoint is the percentage of guideline-based recommendations acted on by clinicians for hyperlipidaemia, atrial fibrillation and heart failure. We hypothesise that care teams with access to MEA will provide more guideline-consistent care than care teams without access to MEA.

Secondary outcomes

Secondary endpoints include the following: (1) the percentage of previsit care process guideline based recommendations acted on by clinicians for each condition individually (ie, hyperlipidaemia, atrial fibrillation and heart failure); (2) the change in percentage of patients on optimal therapy for each condition from the initiation to the completion of the study; (3) self-reported efficiency in managing the three clinical conditions; (4) self-reported use of risk calculators; (5) overall use/burden of clinical decision support tools; (6) overall impressions of MEA in the domains of effectiveness, adoption, implementation, maintenance, usability and overall satisfaction and (7) overall use of the application.

Statistical considerations

Primary analysis

The study design provides two levels of hierarchy in the data. First, the unit of randomisation was the care team, which includes multiple clinicians. The second level of hierarchy is the individual clinician. For the purpose of the analysis, we expect that the variance component due to the care team to be negligible and potentially not estimable. Therefore, the primary endpoint will be tested with a generalised estimating equation (GEE) model using an identity link and binomial distribution with the clustering variable being the individual provider identification number. The primary parameter of interest in this model will be the main effect of the MEA intervention. With the identity link, the parameter estimate is interpretable as the change in percentage points of the primary outcome between groups. The clustering will be incorporated into model by means of the robust variance estimator using care team as the clustering indicator. As a sensitivity analysis, Rao-Cramer adjusted χ2 tests will be computed to compare the primary outcome between interventions. In addition, the primary outcome over time will be modelled using GEE to determine if temporal trends in the data are present. Secondary endpoints will be tested using a similar analytical strategy. For the primary analysis, we will assume that if a patient has more than one encounter in course of the study, that each of his or her encounters is statistically independent. This is to address the potential lack of identifiability of multiple encounters within patient due to the deidentification process as well as possible convergence issues with the higher order hierarchical model.

Sample size estimation

The sample size calculations are based on the following factors: (1) target power 80%; alpha=0.05 (two sided); no interim analysis/alpha spending function; (2) randomisation of clusters: approximately 10 provider teams (clusters) to MEA and approximately 10 provider teams (clusters) to standard of care; (3) an estimate that 2% of the patient encounters that are not exposed to the MEA will have a change in the care plan that would have been suggested by information directly reported by the MEA system had it been available; (4) an estimate that MEA will increase the percentage of patient encounters with a change in their care plan to 4% (this change is consistent with the median change seen in a systematic review of similar studies11) and (5) an intraclass correlation (ICC) range between 0.005 and 0.0075 based on the pilot study data. Based on these assumptions, the total number of patient encounters per cluster required is estimated to be between 263 (ICC=0.005) to 772 (ICC=0.0075). For study planning, the higher number is selected such that the total sample size per arm is 7720 (15 440 total patient encounters). Assuming each care team will have at least 20 patient encounters per clinic day, a minimum of 39 clinic days are expected for each team. We conservatively estimate that it will take approximately 6 months to provide some protection against the uncertainty of the ICC, variable staffing loads and vacation schedules over the summer and for sufficient time to monitor usage patterns over time. The total number of patient encounters during this period of time cannot be determined with certainty and it is expected to vary by care team. The institutional review board approval allows the protocol to use the first 50 000 patient encounters during the 6-month study period.

Ethics and dissemination

The clinicians were deemed human subjects and written informed consent for their participation was obtained. Health Insurance Portability and Accountability Act (HIPAA) authorisation for clinicians did not apply. A waiver of informed consent and of HIPAA authorisation was granted for the clinicians’ patients. All patient records that are used for research in the state of Minnesota must include research authorisation. Records for which the patient has declined authorisation were excluded from the study.

A waiver of informed consent for patients was granted by the institutional review board as the use of MEA was considered to have minimal risk to the patients and to not adversely affect their rights or welfare. Additionally, it was deemed that the research could not be practically conducted without waiver of consent as the tools could not run only on select patients for computational efficacy reasons and due to the size of the study. A waiver of HIPAA authorisation for patients was granted as: (1) it was deemed there was minimal risk to the privacy of individuals as the data will be deidentified by the IT team before being shared with the research team; (2) the research could not practicably be conducted without the waiver for the reasons stated above and (3) the research could not be conducted without access to and use of the protected health information, as the MEA computing algorithm uses protected health information in its algorithms.

Results of the study will be disseminated though publication, as well as meetings and newsletters targeted at key stakeholders.

Discussion

Our study will rigorously evaluate the effectiveness of an automated clinical decision support tool for improving adherence to best practices for patients with hyperlipidaemia, atrial fibrillation and heart failure. This notification system addresses many of the limitations of previous tools by integrating recommendations seamlessly into the EMR and by providing clinicians with both relevant patient data and actionable recommendations.

This builds on our earlier work in an inpatient setting that used a predecessor of MEA26 and illustrates the continued evolution of our clinical decision support system.27 If effective, this decision support tool could be scaled to address the management of other common chronic diseases.

Strengths and limitations

The strengths of this study are that the clinical decision support tool is integrated within the clinical workflow and was designed with iterative clinician input. The study enrols primary care clinicians from two specialties (family medicine and internal medicine) and evaluates the impact of the system on three distinct yet common clinical conditions. Further, the study rigorously tests the effectiveness of the tool through a randomised trial using objective and clinically meaningful endpoints (ie, changes in care to align with ideal management).

The main weakness of the study is that the findings may lack generalisability as MEA is a proprietary system, not yet available to unaffiliated institutions, and it was tested in an academic medical centre. Other limitations include our lack of tracking of outcome measures such as ED visits, hospitalisations or deaths, which during the study time frame would mainly be to ensure that the implementing MEA did not inadvertently lead to unintended negative consequences. We also did not stratify clinicians based on baseline adherence to the guidelines, which could lead to imbalances in the groups at baseline.