Effect of computerised, knowledge-based, clinical decision support systems on patient-reported and clinical outcomes of patients with chronic disease managed in primary care settings: a systematic review.

OBJECTIVES: Chronic diseases are the leading cause of disability globally. Most chronic disease management occurs in primary care with outcomes varying across primary care providers. Computerised clinical decision support systems (CDSS) have been shown to positively affect clinician behaviour by improving adherence to clinical guidelines. This study provides a summary of the available evidence on the effect of CDSS embedded in electronic health records on patient-reported and clinical outcomes of adult patients with chronic disease managed in primary care. DESIGN AND ELIGIBILITY CRITERIA: Systematic review, including randomised controlled trials (RCTs), cluster RCTs, quasi-RCTs, interrupted time series and controlled before-and-after studies, assessing the effect of CDSS (vs usual care) on patient-reported or clinical outcomes of adult patients with selected common chronic diseases (asthma, chronic obstructive pulmonary disease, heart failure, myocardial ischaemia, hypertension, diabetes mellitus, hyperlipidaemia, arthritis and osteoporosis) managed in primary care. DATA SOURCES: Medline, Embase, CENTRAL, Scopus, Health Management Information Consortium and trial register clinicaltrials.gov were searched from inception to 24 June 2020. DATA EXTRACTION AND SYNTHESIS: Screening, data extraction and quality assessment were performed by two reviewers independently. The Cochrane risk of bias tool was used for quality appraisal.
RESULTS: From 5430 articles, 8 studies met the inclusion criteria. Studies were heterogeneous in population characteristics, intervention components and outcome measurements and focused on diabetes, asthma, hyperlipidaemia and hypertension. Most outcomes were clinical with one study reporting on patient-reported outcomes. Quality of the evidence was impacted by methodological biases of studies.
CONCLUSIONS: There is inconclusive evidence in support of CDSS. A firm inference on the intervention effect was not possible due to methodological biases and study heterogeneity. Further research is needed to provide evidence on the intervention effect and the interplay between healthcare setting features, CDSS characteristics and implementation processes. PROSPERO REGISTRATION NUMBER: CRD42020218184. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

In contrast to other reviews, this review focuses on interventions involving clinical decision support systems (CDSS) that are knowledge-based, are embedded within electronic health record systems, provide electronic alerts and are used by physicians in primary care.

The electronic search strategy was piloted, and hand searching of reference lists of previous reviews permitted a sufficient level of confidence that all relevant articles were included.

Some limitations arose related to studies being of 1-year duration on average, making it difficult to deduce long-term effects on patient outcomes.

The scope of the review was restricted to nine chronic diseases; however, the selection included the most common chronic diseases; yet, there were no eligible studies in five of them.

The included studies were of low-to-moderate quality of evidence, and heterogeneity among the included studies limited the ability to conduct a meta-analysis.

Introduction

Non-communicable chronic diseases are the leading cause of disability and mortality worldwide, affecting one in three adults globally.1 Rising healthcare expenditure because of chronic disease prevalence has led to an ever-growing demand for effective primary care services for the long-term management of chronic conditions.2–4 Considering that health systems have mainly adapted to providing periodic care for acute problems, an even greater need to achieve efficient, comprehensive and coordinated long-term care persists.5 Models such as the chronic care model have been proposed to improve the quality of care in chronic care systems.6 This model recommends ‘decision support’ as one of six applicable changes in primary care settings that promote better patient outcomes, if successfully implemented.6–8 Within this and other models of primary care and chronic condition management, clinicians are encouraged to provide patient care according to evidence-based clinical practice guidelines (CPGs) to support improved patient outcomes.9 10 Given the frequent updates to CPGs resulting from rapid advancements in medical knowledge, a major challenge is the maintenance of clinicians’ knowledge of evidence-based CPGs.10 Consequently, clinicians’ adherence to CPGs may be variable and frequently insufficient, which may result in suboptimal disease management outcomes.10 11

Clinical decision support is an umbrella term that includes various care structures, processes and tools.12 Clinical decision support systems (CDSS) range from non-computerised to computerised systems and basic to advanced systems.12 13 Currently, most attention is driven toward computerised advanced CDSS, which perform complex decision-making functions from electronic health record (EHR) patient data.12 Such systems are termed knowledge-based, as they use software algorithms to generate actionable patient-specific outputs in the form of recommendations to the practitioner using conditional (IF–THEN) rule statements derived from CPGs.14 15 They also differ in their technical and clinical interface, whether passive (providing user-initiated prompts to generate clinical advice, eg, by clicking buttons) or active (automatic system-initiated prompts).12 16 17 CDSS also differ in system–user communication, which could be of consulting nature (ie, provide appropriate ‘next-steps’ clinical recommendations) or of critiquing nature (verify decisions taken by practitioners).12 16 Given the differences in interfaces, the scope of this review includes computerised, knowledge-based CDSS embedded in EHRs that are of either passive or active, consulting or critiquing, nature.

Although CDSS are ubiquitous, few have been successfully adopted.18 Previous reviews appraised the efficacy of different types of CDSS in primary care, presenting a greater consensus on the useful role of CDSS on practitioner-related outcomes such as guideline adherence.19–23 However, the impact of CDSS on patient outcomes remains largely uncertain.19–23 Bryan et al demonstrated a positive association between the use of CDSS in primary care and patient outcomes.23 However, a need to further evaluate the effect of CDSS on patient outcomes was identified, particularly in primary care and specifically for chronic disease management. The novel focus of this review is in the combined focus on the effect of computerised, knowledge-based, CDSS and their effect on patient-reported and clinical outcomes of patients and with chronic disease and managed in primary care settings. This particular focus was chosen to inform more specific and efficacious implementation strategies in primary care management of chronic disease and attract the industry’s consideration of CDSS use in primary care settings, which a more generalised focus on multiple care settings, multiple diseases beyond just chronic disease or lack of focus on patient-reported outcomes may not achieve. There remains a need to assess the capabilities of CDSS in distinctive clinical professional areas and disease entities, particularly on patient outcomes.18 A recent review on EHR-embedded CDSS focused on process delivery (practitioner outcomes) and was not limited to primary care settings and chronic conditions.24 Despite obvious heterogeneity, clinically meaningful improvements in care processes were shown.24 However, it was unclear under what clinical circumstances these improvements could be replicated in every clinical context to affect clinical outcomes presumptively.24

In this systematic review, patient outcomes refer to quantifiable clinical biomarkers such as glycated haemoglobin (HbA1c) and low-density lipoprotein cholesterol (LDL-C) levels, clinical outcomes such as mortality and morbidity, and patient-reported outcomes, such as health-related quality of life. To the best of our knowledge, and in the view of rapid upgrades in the world of CDSS, no review exists solely on the effect of CDSS embedded within EHRs on patient outcomes for patients with chronic disease managed in primary care. This review aims to fill this knowledge gap by collating all the available evidence on this classification of CDSS.

Methods

Study design and eligibility criteria

A systematic review of the existing literature was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.25 26 The research question, inclusion and exclusion criteria were defined using the PICOS framework.27 Nine common chronic diseases were selected as part of the inclusion criteria, following a review by Reynolds et al 28 namely asthma, chronic obstructive pulmonary disease (COPD), heart failure, myocardial ischaemia, hypertension, type 2 diabetes mellitus (T2DM), hyperlipidaemia, arthritis and osteoporosis. Studies reporting on patient-related and clinical outcomes managed by primary care clinicians using CDSS-EHR compared with usual care were included. Studies involving participants<18 years of age, multidisciplinary (non-clinician) system users, and other varieties of CDSS (not linked with EHR) were excluded. A detailed view of the criteria is provided in online supplemental table 1.

Figure 1 depicts an analytic framework of the intervention adapted from a framework published by Lobach et al.22 The framework provides a theoretical context for possible features that could underpin the success, or lack of success, of a complex intervention such as implementing CDSS in a primary care setting. The framework guided the extraction of relevant data from studies to inform the narrative synthesis.

Figure 1

Analytic framework. CDSS, clinical decision support system; CPGs, clinical practice guidelines; EHR, electronic health record.

Search strategy

Electronic databases Medline, Embase, Scopus, Cochrane Central Register of Controlled Trials (CENTRAL) and Health Management Information Consortium, and one clinical trial registry (clinicaltrials.gov), were searched from inception to 24 June 2020. Cochrane reviews were screened for relevant search terms and subject headings (MeSH) terms under the following categories ‘registries supported by CDSS’ AND ‘General Practice/Primary care’ OR each of the nine chronic diseases combined with AND ‘Asthma’, ‘heart failure/myocardial ischaemia’, ‘hypertension’, ‘diabetes mellitus’, ‘hyperlipidaemia’, ‘COPD’, ‘arthritis’, ‘osteoporosis’ (online supplemental tables 2 and 3). The search was limited to articles in English.

Terms for patient-reported outcome measures (PROM) were excluded. Existing PROM filters are not comprehensive, and the use of those filters in the search strategy could increase the risk of missing relevant studies.29 However, we manually screened papers to identify any ‘patient-reported outcomes’ measured in studies. PROM may be described in various ways; thus, attempting to list specific patient outcomes for each entity could introduce bias and lead to missing relevant studies. Previous systematic reviews around CDSS in primary care were also searched for potentially eligible studies.19–21 23 24

Study screening and selection

Search results were exported to RefWorks, then screened using Covidence. After removal of duplicates, screening was conducted by two reviewers independently (MLEA and KID). Disagreements were resolved by consensus within the review group. After an initial screening of titles and abstracts, the full text of potentially relevant articles was assessed.

Data extraction

A data extraction form was adapted using the ‘Cochrane data collection form for intervention reviews: randomised controlled trials (RCTs) and non-RCTs’ and aided by Lobach et al’s data abstraction guidance.30 31 The form was piloted and two reviewers (MLEA and KID) completed data extraction independently.

Quality assessment of individual studies

Risk of bias (RoB) was assessed using the Cochrane RoB tool,32 33 by two reviewers independently (MLEA and KID). The methodological RoB was assessed per the following elements in RCTs: selection bias, performance bias, detection bias, attrition bias, reporting bias and other biases.33 34

Quasi-RCTs were reported as high risk for random sequence generation. Cluster RCTs were assessed for an additional element ‘selective recruitment of cluster participants’. All judgements were made with guidance from the Cochrane Handbook and the guidelines of the Cochrane Consumers and Communication Review Group.33–35

Studies were considered as high risk if they scored ‘high’ or ‘unclear’ for either random sequence generation or allocation concealment, based on the rising evidence that these two elements are particularly significant sources of bias.34 A summary ‘RoB’ figure was generated for all risk assessments of elements with a narrative commentary for each study. Additionally, a detailed adapted table was generated to visualise subelements that fit under each element in the Cochrane RoB tool.34

Quality assessment of outcomes

The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) tool was used to assess the quality of outcomes from individual studies.36 The tool involves five elements: RoB, inconsistency, indirectness, imprecision and publication bias. The tool is primarily used for meta-analyses; however, it can be adapted for use in narrative reviews. Study outcomes were appraised, ranked as high, moderate, low or very low and justifications were provided, in line with Cochrane guidance.37

Data synthesis

Treatment effect was reported as ORs with 95% CIs and p values for dichotomous outcomes, where available. Continuous data were presented as mean differences with 95% CIs and p values.

Heterogeneity in the results was anticipated due to the wide spectrum of possible patient-related outcomes reported by the studies. Methodological heterogeneity was assessed using Cochrane RoB tool. Heterogeneity in included studies varied in terms of intervention components, population characteristics and outcome measurements; thus, conducting a meta-analysis was deemed not appropriate by the four-person author group.

Patient and public involvement

No patients were involved.

Results

The search identified 5430 articles. Following removal of duplicates, 4498 articles remained for screening by title and abstract. Of these, 266 articles were selected for full-text review and 10 articles met the inclusion criteria. Among them, two articles were published protocols of included articles; thus, eight different studies were finally included. The detailed search results are presented in the PRISMA flowchart (figure 2).

Figure 2

PRISMA flowchart detailing the systematic search results. CDSS, clinical decision support systems; CENTRAL, Cochrane Central Register of Controlled Trials; HMIC, Health Management Information Consortium; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Study characteristics

Characteristics of included studies are presented in table 1. Included studies comprised of patients in four disease areas: T2DM, asthma, hyperlipidaemia and hypertension.38–45 No studies on COPD, heart failure, myocardial ischaemia, arthritis and osteoporosis were identified. Six studies were cluster RCTs, one study was parallel-group RCT and one was quasi-RCT of stepped-wedge design. Most studies were conducted in the USA, and three studies took place in the UK, Spain and Belgium. Studies, both among the disease conditions and within each disease entity, differed appreciably in participant covariates, setting attributes and CDSS characteristics. These factors, compounded with the heterogeneity of the studies (as described in the Methods section) suggested that the reported outcomes were not fit for statistical pooling; hence, these factors justified not conducting a meta-analysis. Follow-up ranged from 6 months to 1.5 years. No study incorporated financial reimbursement before the implementation of interventions. Further details are presented in table 1.

Table 1

Characteristics of included studies

Study	Disease	Study design	Duration	Sponsorship	Setting	Condition identification from EHR	Participants		Comparability			Target outcomes
							Intervention	Control	Age	Sex	Disease severity
Schnipper et al 38 USA	T2DM	RCT	6 months	The Agency for Healthcare Research and Quality	Partners HealthCare System, 10 practices at Brigham and Women’s Hospital and Massachusetts General Hospital (academic)	NS	3431 patients	3578 patients	Comparable	IG: more females (statistically significant)	Comparable	Patients meeting management goals
Gill et al 39 USA	T2DM	Cluster RCT	1 year	Sanofi, USA	Primary care practices in Delaware and Maryland (community)	ICD-9CM EHR code	5 offices29 PCPs4484 patients Analysed (by subgroups): HbA1c: 2041 patientsLDL-C: 2793 patients	5 offices23 PCPs1902 patients Analysed (by subgroups):HbA1c: 723 patientsLDL-C: 931 patients	Overall: 18–75 IG: older (statistically significant)	Comparable	IG: better glycaemic control, lower LDL-C	HbA1c and LDL-C
Heselmans et al 40 Belgium	T2DM	Cluster RCT	1 year	None	Belgian primary care practices using OneHealth EHR(community)	ICPC-2EHR code or confirming lab tests	25 offices60 PCPsAnalysed:14 offices21 PCPs1351 patients	26 offices60 PCPsAnalysed:15 offices31 PCPs2464 patients	Overall: ≥18 years IG: older (significant statistically)	CG: more females(statistically significant)	Comparable for: mean HbA1c, mean BP, % patients meeting target for HbA1c, BP and LDL-C	HbA1c, LDL-C and BP
Eccles et al 41 UK	Asthma and angina	Cluster RCT	1 year	UK NHS research and development	General practices in north-east England (community)	Asthma or anginaEHR code in prior 12 months	Asthma: 31 offices Angina: 29 offices	Angina: 29 offices Asthma: 31 offices	≥18 years; no details presented	NS	NS	Quality of life and condition-specific outcomes
Gill et al 42 US	Hyperlipidaemia	Cluster RCT	1 year	None	MQIC of primary care practices across the country(academic and community)	Identified as having LDL levels recorded in EHR	12 offices49 PCPs26 696patients	13 offices56 PCPs37 454 patients	Overall: 20–79 yearsCG younger	CG: more males (statistically significant)	CG lower disease risk	LDL-C
Cobos et al 43 Spain	Hyperlipidaemia	Cluster RCT	1 year	Novartis	General practices in Catalonia	Identified as having TC between200 mg/dL and 400 mg/dL	20 offices1046 patients	22 offices1145 patients	Overall: 18–94 years	CG: more females(statistically significant)	Comparable	Lipid values
Hicks et al 44 USA	Hypertension	Cluster RCT	1.5 years	The Agency of Healthcare Research and Quality	Primary care practices affiliated with the Brigham and Women’s Hospital	Identified as having two or more hypertension- related outpatient visits using EHR codes	7 offices786 patients	7 offices1048 patients	Comparable	Comparable: >20 years	IG: significantly higher % of patients with controlled BP at first visit	BP control
Lopez et al 45 USA	Hypertension	Non- RCT	1 year (2016)	Centre for Disease Control	Primary care practices in New York	Identified as having at least 1 ICD code for hypertension in EHR	7 offices	7 offices	18–85 years	NS	NS	BP control

BP, blood pressure; CG, control group; EHR, electronic health record; HbA1c, glycated haemoglobin; ICD, International Classification of Diseases; ICPC-2, International Classification of Primary Care, 2nd edition; IG, intervention group; LDL-C, low-density lipoprotein cholesterol; MQIC, medical quality improvement consortium; NS, not stated; PCPs, primary care physicians; RCT, randomised controlled trial; TC, total cholesterol; T2DM, type 2 diabetes mellitus.

CDSS characteristics

CDSS interventions were characterised based on their origin (commercial/local), delivery mode (user or system-initiated), user response (optional/mandatory), events (system functions), coupled EHR and auxiliary features (local clinician involvement in system development process, system accessibility to other team members, provision of performance feedback or coupled with user education). Additional information such as user training, user adherence and reported usability was recorded. Details are provided in table 2.

Table 2

CDSS characteristics of interventions in included studies

Study	Disease	Origin	Delivery mode	User response	Events	Guidelines	Integrated EHR (name)	User training	User adherence	Usability	Auxiliary features
Schnipper et al 38	Diabetes mellitus	Commercial	User- initiated	Optional	Requests missing data. Alerts outputs of assessments of clinical care. Recommends orders for medication changes, laboratory studies, appointments and referrals	NS	Longitudinal medical record	Brief instructions at startmonthly tips	Tracked use by clinician and sent customised emails	Used for 5.6% of eligible patients	Local user involvement in development process
Gill et al 39	Diabetes mellitus	Commercial	System- initiated	Optional	Generates reports for patient-specific recommendations	American diabetes Association;European Association for the study of diabetes and ICD 9 coding	Multiple EHRs (previously employed)	Not provided	Qualitatively assessed through informal interviews	4/5 practices reported use	Recommendations also received by non-clinicians in care team
Heselmans et al 40	Diabetes mellitus	Commercial	User- initiated	Unclear	Alerts for reminders, therapeutic suggestions and diagnosis-specific guideline links	Belgian and International Guidelines	HealthOne	Not provided	Not assessed	Not assessed	None
Eccles et al 41	Asthma and angina	Local	System- initiated (4 months) then altered to user- initiated	Optional	Alerts management and prescription options and patient referrals	NS	AAH Meditel or EMIS systems	1-day training	Measured by a usage log	Median number of system interactions was 0	Education on system guidelines
Gill et al 42	Hyperlipidaemia	Commercial	System- initiated	Optional	Alerts for overdue lipid tests. Alerts if patient has not met personalised lipid goals. Generates disease risk categories	ATP-III guidelines	Electronic medical record (centricity)	1-year familiarity with system	Not measured	Not measured	Generates a list of patients who have not met lipid goals, outside office visits. Listed patients were sent a letter at the start and after 6 months of the study
Cobos et al 43	Hyperlipidaemia	Local	System- initiated	Optional	Recommends therapeutic options. Recommends follow-ups. Recommends lab testing	ESCHM	Not specified	None	Monitored by CDSS	71%	None
Hicks et al (2008)44	Hypertension	Local	System- initiated	Optional	Alerts for therapeutic options	AHA/ACC 2001 guidelines	Not specified	None	Assessed prescription of a recommended drug class within 1 week of clinic visit	NS	None
Lopez et al 45	Hypertension	Local	System- initiated	Optional	Alerts for missing BP readings Alerts for therapeutic options	JNC- 7 guidelines	eClinicalWorks or MD Land	2-day training at start, 7 training sessions during second year and ad-hoc sessions as needed	Semi-structured interviews	Reported all practices have used all CDSS components	Generates reports of patients by status to manage visits

AHA/AAC, American Heart Association/American College of Cardiology; ATP-III, Adult Treatment Panel-III; CDSS, clinical decision support systems; EHR, electronic health record; ESCHM, The European Society of Cardiology and other societies for Hypercholesterolemia Management; ICD, International Classification of Diseases; JNC-7, The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure; NS, not stated.

RoB and quality of the evidence

Online supplemental tables 4 and 5 provide risk assessments details. Overall, the included studies were impacted by RoB, mainly due to incomplete outcome data and uncertainty on blinding of outcome assessment and personnel. In most studies, it was unclear if outcome assessors were blinded. Additionally, blinding of physicians (CDSS users) was not possible across all studies, as they were the intended system users. Most studies reported substantial withdrawal rates due to poor follow-up. These limitations resulted in analysing subgroups (rather than the initially intended full cohorts) in some trials; hence, most studies were considered of high risk for incomplete outcome data.

Overall, quality of the evidence was rated as either moderate or low, with the most impact resulting from methodological risks of bias within studies. Online supplemental table 6 presents the quality of evidence evaluated using GRADE, with reasons stated for quality downgrading.

Narrative synthesis

Diabetes mellitus

Three studies assessed the effect of CDSS on clinical and patient-reported outcomes in patients with T2DM.38–40

Gill et al retrieved slight, yet statistically significant, reduced HbA1c and LDL-C with the CDSS intervention compared with the control group, with a small between-group difference in HbA1c and LDL-C levels of 0.12% and 3.57 mg/dL, respectively (table 3).39 The reported reductions in HbA1c and LDL-C were clinically small; however, statistically, the intervention group had 52% higher odds of achieving personalised HbA1c levels, 56% higher odds of achieving HbA1c <7.0% and 34% higher odds of achieving an LDL-C <100 mg/dL compared with usual care patients over 1 year of follow-up. Nevertheless, compared with the control group, the intervention group had better-controlled baseline levels of HbA1c and LDL-C. The study by Gill et al involved system-initiated ‘push’ CDSS, which were integrated into EHRs already operating at participating practices. Reported limitations were of EHR-CDSS interoperability difficulties, which resulted in missing patient lab values.39 Other missing lab values also resulted from patients not returning for lab tests. In view of these limitations, a subset of randomised patients without missing HbA1c and LDL-C values were analysed (table 3).

Table 3

The effects of CDSS on patient health outcomes

Study	Disease	Outcome	Outcome definition	Absolute effects		Relative effects	Conclusions	Quality
				Intervention	Usual care
Schnipper et al 38	Diabetes	Odds of patients meeting management goals at the end compared with baseline	Targets undefined	Not reported		ITT: OR=1.02; 95%CI 0.99 to 1.04, p value not reported.PP: OR=1.16; 95% CI 1.08 to 1.25, p<0.001	Intention-to-treat: non- significant odds of intervention patients meeting management goals compared with control patients at 6 months.Per Protocol: significant odds of intervention patients meeting management goals compared with control patients after follow-up period	⊕⊕◯◯Low
Gill et al (2017)39	Diabetes	Difference in A1c from baseline.Difference in LDL-C from baseline.Odds of patients achieving A1c and LDL-C goals	HbA1c goals:<7% and<7.5% in patients with microvascular complications. LDL-C goals:<100 mg/dL	Difference in HbA1c from baseline: −0.08%±1.15%.Difference in LDL-C from baseline: −4.35±24.54mg/dL	Difference in HbA1c from baseline: −0.14%±1.51%.Difference in LDL-C from baseline: −1.70±26.16 mg/dL	Adjusted HbA1c between-group difference (%): 0.12%, 95% CI 0.02 to 0.22; p=0.02.Adjusted LDL-C between-group difference (mg/dL): 3.57 mg/dL, 95% CI 1.80 to 5.34, p<0.0001.Odds of achieving personalised HbA1c goals: aOR=1.52, 95% CI 1.24 to 1.86, p<0.0001.Odds of achieving HbA1c <7%: 1.56, 95% CI 1.27 to 1.91, p<0.0001.Odds of achieving HbA1c ≤9%: 1.07, 95% CI 0.75 to 1.52, p=0.71.Odds of achieving LDL-C <100 mg/dL: 1.34; 95% CI 1.11 to 1.61,p=0.002	Clinically small, yet statistically significant between-group reductions in HbA1c and LDL-C	⊕⊕⊕◯Moderate
Heselmans et al 40	Diabetes	Mean change of HbA1c.Mean change in BP.Mean change in LDL-C levels	NA	Not reported		Between-group difference in HbA1c means: −0.09%, 95% CI − 0.18 to 0.01, p=0.06.Between-group change difference in SBP: 0.13% (95% CI −0.91 to 1.16, p=0.81.Between-group change difference in DBP: 0.12% (95% CI −1.25 to 1.49, p=0.86.Between-group change difference in LDL-C: 1.76% (95% CI −0.46 to 3.98, p=0.12	No significant changes in between-group means for HbA1c, BP or LDL-C	⊕⊕⊕◯Moderate
Eccles et al 41	Asthma and angina	Patient-reported outcomes using generic measures: The Short Form 36 Health Survey Questionnaire (SF-36) and European Quality of Life Five Dimension (EQ-5D).Condition-specific measures: the Seattle Angina Questionnaire and the Newcastle Asthma Symptoms Questionnaire		Not reported			No effect on any patient outcome	⊕⊕⊕◯ Moderate
Gill et al 42	Hyperlipidaemia	The percentage of patients whose most recent LDL-C was at goal (%)	Defined by the ATP-III guidelinesLDL goals:<100 for high-risk, <130 for moderate risk and <160 for low risk patients	High risk:Baseline: 50%End: 53.3%P<0.001 Moderate risk: Baseline: 64.1%End: 64.7%P>0.1Low risk: Baseline: 88.5%End: 87.9%P<0.05	High risk:Baseline: 52.3%End: 56.1%P<0.001 Moderate risk:Baseline: 67.1%End: 68.5%p<0.001Low risk:Baseline: 90.2%End: 90.9%P<0.001	High risk: OR=1.17 (95% CI not reported), p<0.05.Moderate risk: OR=0.29 (95% CI not reported), p<0.05.Low risk: OR=1.74 (95% CI not reported), p<0.05	A significant increase in proportion of patients attaining goals for all categories except the moderate-risk IG.No significant difference in improvement in any category compared with the CG	⊕⊕◯◯Low
Cobos et al 43	Hyperlipidaemia	Between- group differences in means of lipid values (mg/dL)	NA	TC:233.8LDL-C: 149.2HDL-C: 58.0Triglycerides:136.6	TC:231.0LDL-C: 146.5HDL-C: 56.3Triglycerides: 135.2	Difference, 95% CI, P valueTC:−2.8 (–1.7 to –7.3), 0.218LDL-C:−2.7 (–1.7 to –7.1), 0.227HDL-C:−1.6 (–0.6 to –3.6), 0.142Triglycerides:−1.4 (–8.3 to –11.2), 0.766	No impact detected on final lipid values	⊕⊕⊕◯Moderate
Hicks et al 44	Hypertension	Absolute effects: Percentage of patients with controlled BP (%).Mean baseline and end SBP and DBPRelative effects: Odds of BP control at outcome visitBaseline and end difference in SBP and DBP	Controlled BP:<130/80 mm Hg for patients with diabetes or renal failure<140/90 mm Hg for other patients	48%baseline SBP: 136 mm HgEnd SBP: 138 mm HgBaseline DBP: 78 mm HgEnd DBP: 77 mm Hg	45%baseline SBP:138 mm HgEnd SBP: 137 mm HgBaseline DBP: 80 mm HgEnd DBP: 78 mm Hg	OR: 0.96Baseline between-group difference in SBP: 2 mm Hg, p =0.59Baseline between-group difference in DBP: 2 mm Hg, p=0.07End between-group difference in SBP: 0 mm Hg, p=0.67End between-group difference DBP: 1 mm Hg, p=0.05	No differences between IGs in mean SBP or DBP readings at (end) outcome visits	⊕⊕◯◯Low
Lopez et al 45	Hypertension	Relative effects: odds of BP control post-intervention	Controlled BP:<140/90 mm Hg for all patients	Not reported		OR=1.36, 95% CI 1.08 to 1.71,p=0.009	Significant odds of improvement of BP in the IG compared with usual care	⊕⊕◯◯Low

aOR, adjusted OR; ATP-III, Adult Treatment Panel-III; BP, blood pressure; CG, control group; DBP, diastolic blood pressure; HbA1c, glycated haemoglobin; HDL-C, high-density lipoprotein cholesterol; IG, intervention group; LDL-C, low-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; SBP, systolic blood pressure; TC, total cholesterol.

Schnipper et al (RCT) reported the odds of patients meeting management goals (table 3).38 Intention-to-treat analysis showed non-significant changes in patients’ odds of meeting management goals; a per-protocol analysis, however, showed slightly significant odds of attaining management goals in the intervention group compared with the control group (table 3). Among the three studies, this intervention is the only one that involved user training (table 2). Auxiliary features also included feedback on performance and local user (physician) involvement in the CDSS development process; yet, system usability was reported to be used for only 5.6% of patients (table 2).

Heselmans et al (cluster RCT) reported non-significant between-group differences in HbA1c, LDL-C, diastolic blood pressure (DBP) and systolic blood pressure (SBP) (table 3).40 Levels of HbA1c were relatively well controlled, with reasonable mean values for HbA1c in both groups at baseline, especially in the control group (table 1).40 Subsequent analysis of a subgroup of 601 patients with HbA1c >7.0% retrieved a statistically significant between-group difference in HbA1c favouring the intervention, though not clinically relevant (−0.40% (95% CI −0.70 to −0.09)). The same analysis retrieved non-significant differences in LDL-C and BP. User adherence and usability were not measured.

In all three studies, there was no mention of the provision of reimbursement for the implementation of the CDSS. All CDSS were commercial, and user response to the system alerts was optional.

Asthma and angina

One study assessed the impact of CDSS on self-reported quality of life and condition-specific outcomes in patients with asthma and angina.41 The angina intervention group acted as the control group for the asthma intervention group and vice versa (justified by authors to eliminate a possible Hawthorne effect in practitioner behaviour on the primary outcome). Questionnaires were administered in three rounds (a year before the intervention, beginning of the intervention and a year after the intervention), which showed non-significant differences in both intervention groups post-intervention (table 3). Completion of the three rounds of questionnaires occurred in 46% and 35% of patients with angina and asthma, respectively. The implemented CDSS were locally produced and embedded in two widely used health systems (table 2). The CDSS were modified 4 months into the study from system to user-initiated, as requested by participating physicians. User training was limited, and system usability was very low, measured using a log with a median of zero user–system interaction during most of the study (table 2).

Hyperlipidaemia

Two studies involved participants with hyperlipidaemia.42 43

The study by Gill et al involved implementing a commercial CDSS in participating practices of one institution.42 All patients enlisted under participating practices with recorded LDL-C values were enrolled. A high correlation in outcomes was found at baseline and end measurements, causing patients not meeting LDL-C levels after randomisation to be analysed only. The control group also had a lower risk of disease and was significantly younger. Patients were stratified into three guideline-based risk groups, and the percentage of patients reaching the predefined goals at baseline and end were recorded (table 1). All categories in the intervention and control groups showed a significant increase in the proportion of patients attaining lipid goals, except in the moderate-risk group. Calculated differences were not presented; however, p values of differences within the intervention and control groups suggested no significant improvement in any risk category.

In Cobos et al despite a usability of 71%, the intervention had no superiority over usual care in terms of between-group differences in lipid values (table 3).43 The study reported >25% patients lost to follow-up for clinic visits or LDL-C testing (table 1).

Hypertension

Two studies assessed the effect of CDSS on BP in hypertensive patients.44 45 Both studies employed CDSS that were system-initiated with optional user responses (table 2).

Hicks et al reported no differences between intervention groups in BP control (table 3).44 Differences in baseline and end SBP and DBP between groups were not significant. Nonetheless, the intervention group had a significantly higher percentage of patients with controlled BP (table 1). No user training was provided, and system usability was not measured (table 2). Less than 80% of recruited patients were followed-up due to withdrawals caused by the study’s inclusion criteria, which required patients to have a recorded ethnicity in the EHR. Subsequently, patients without BP recordings were considered uncontrolled. However, a subset analysis excluding these patients still retrieved a non-significant difference in BP values.

Lopez et al’s study was a quasi-RCT and reported significant odds of BP improvement in the intervention group compared with the control group (table 3).45 Notably, extensive training was provided for participating clinicians compared with all other studies, and semi-structured interviews revealed that all practices appeared to use the system components (table 1).

Discussion

Main findings

As primary care providers are increasingly required to monitor and manage chronic diseases, there is a need to assess the capabilities of CDSS on patient outcomes. This systematic review included the eight available studies on the effect of computerised, knowledge-based, EHR-embedded CDSS on clinical and patient-reported outcomes. Several study designs were included in the eligibility criteria; however, studies fit for inclusion were trials, namely, RCTs, cluster RCTs and non-RCTs (table 1, online supplemental table 1). Two studies reported a statistically significant impact of CDSS on patient outcomes,39 45 one reporting clinically small yet significant reductions in HbA1c and LDL-C and another reporting significant increases in the proportion of hypertensive patients who achieve goals, which may be clinically relevant.45 Another two studies reported statistically significant findings but in subgroup analyses.38 40 One study reported statistical significance in partial outcomes and three studies did not report any statistically significant impact of CDSS on outcomes at all.42–44 Overall, evidence was not conclusive to imply a differential effect, either positive or negative, of clinician use of CDSS on patient or clinical outcomes compared with usual care, mainly due to the quality of evidence impacted by the prominent RoB in the included studies. The systematic review also noted a limited availability of studies evaluating the CDSS of interest directed to clinicians in primary care for the management of the common chronic diseases. This is especially true for COPD, arthritis and osteoporosis, which were eligible conditions, yet no such studies were available.

Implications of the review

The systematic review infers the importance of considering how the interplay of factors such as the nature of a healthcare setting (academic and community) and CDSS interact with clinicians (or users per se). System-initiated CDSS seemed to have a better reflection on patient outcomes compared with user-initiated systems; however, this relationship was confounded by many factors such as poor process measures, namely, low system usability and EHR–CDSS interoperability difficulties. A common limitation reported across most studies was poor patient follow-up, which highlights the importance of patient behaviour as a confounding factor to the performance of CDSS interventions in clinical settings. Some interventions included helpful auxiliary features such as involvement of local physicians in CDSS development and feedback on performance; yet, these features did not seem to reflect positively on patient outcomes. Understanding user (clinician) experiences qualitatively in the context of using such CDSS to manage chronic diseases in primary care may be of benefit to inform on their practical use. This review assessed CDSS functionality when used by clinicians; however, CDSS accessibility to other members of a multidisciplinary team may have an added positive effect, which could translate into better patient outcomes. Multidisciplinary teams in primary care have been shown to aid successful implementation of CDSS by increasing adherence to CPGs and improving patient outcomes in the absence of CDSS.46 47

The healthcare setting and intervention implementation itself may dictate the success or failure of an intervention. Large academic care settings may be better equipped with IT and financial resources than community centres to ensure successful implementations of CDSS.48 Included studies revealed limitations due to poor implementation of the CDSS, namely, interoperability issues, lack of IT support, poor usability and lack of user training, which could have hindered the intervention from reflecting significant outcomes compared with usual care. Models have been offered to help explicate how structural elements of healthcare settings and various levels of healthcare organisations levelling up to the industry may influence clinician behaviour and uptake of health IT to influence patient outcomes.17 49 Finally, it is imperative to consider the effect of non-clinical factors such as patient behaviour outside clinic visits, their perceptions and adherence to clinical advice.50

Comparison with previous reviews

Previous reviews of CDSS of multiple types have indicated impact on patient outcomes in some instances. A recent review showed that CDSS systems achieve small to moderate effects on processes of care; yet, the extent of translation to patient outcomes and under what clinical contexts and conditions this was achieved was not determined.24 Previous reviews were not limited to primary care settings and included various other CDSS, namely, paper-based, web-based, stand-alone electronic systems, and patient-directed systems that do not necessarily alert a clinician at the point of care.19 21 51 However, reviews assessing the impact of various types of CDSS on chronic disease patient outcomes in primary care identified inconclusive results, with a need to define more clearly the exact role of CDSS compared with usual care in subsequent studies.20 23 Reviews evaluating patient outcomes noted conceptual between-study heterogeneity, as this review did, especially when several diseases and clinical outcomes were considered.52 In terms of patient outcomes, CDSS seem to affect clinical outcomes for cardiovascular risk factors such as hyperlipidaemia and diabetes.11 53 Nevertheless, a much more recent review on the effect of CDSS on cardiovascular risks revealed no definitive clinical benefits after their implementation.53 On the other hand, the evidence is less robust for other non-cardiovascular risk-related chronic conditions such as cancer and other primary care sensitive cases (PCSCs) such as vaccination, which were not part of the eligibility criteria in this review.54

A review of CDSS embedded within EHRs in all-purpose healthcare settings on overall mortality and morbidity suggested no effect of CDSS on mortality. However, it noted that potential benefits or risks would highly depend on setting and disease characteristics, which is an evident implication in this review.52 A review by Bryan et al validated the use of CDSS in primary care to improve outcomes; however, they reported substantial variability in intervention effectiveness related to the type and implementation of the different types of CDSS.23 Additionally, other reviews incorporated studies that involved the use of CDSS by multidisciplinary care members, reflecting positive clinical outcomes with shared decision-making.19 51 55

Unanswered questions and future research

More research is needed to establish the effectiveness of EHR-embedded CDSS on clinical outcomes of patients with chronic diseases treated in primary care settings. Further studies involving patients with chronic diseases are needed for robust evidence. Moreover, it may not be reasonable to evaluate the intervention’s efficacy based on older studies, given the exponential growth and rapid advancements in the field of CDSS.56 Future research could include practitioner-related cost effectiveness and patient-related outcomes to present an ampler perspective on the effect of this type of CDSS to inform policy-making and clinicians. Other PCSCs can be studied for a broader scope, including cancer, mental health illnesses and preventative interventions such as vaccination.57 Research around prolonged evaluations of CDSS is needed. Longer-term studies may account for potential implementation barriers that could take place. Additionally, it would be beneficial to synthesise findings on the effect of this particular type of CDSS on patient outcomes, when operated within a multidisciplinary team of non-clinician providers. It is worth noting that patient outcomes are not always addressed efficiently in trials, among the reasons is the possibility of having subjective, composite or surrogate outcomes, which do not necessarily translate into real-life patient outcome improvements.58 Hence, including observational studies in updated reviews may be useful. Additionally, no studies of registries embedded with CDSS fit the eligibility criteria, suggesting a potentially greater inclination to embed CDSS with EHRs currently. There also seems to be a dearth of studies evaluating non-cardiovascular risk-related chronic diseases post-implementation of CDSS linked with EHRs, as no studies evaluating patients with COPD, osteoporosis and arthritis were eligible.

Strengths and limitations

This systematic review focuses on interventions involving a specific type of CDSS—those that are knowledge-based, coupled with EHR systems, that provide electronic alerts and are used by physicians in primary care to support chronic disease management. To the authors’ knowledge, no such review exists considering this combined focus and given the attention these systems are increasingly gaining in primary care settings.29 As an example of the novelty of this review, compared with previous reviews with the most similar focus to our own by Bryan and Boren23 and Bright et al 19 this review included studies not featured in these previous reviews such as the studies by Schnipper et al, Gill et al, Heselmans et al and Hicks et al.38–40 44Application of the eligibility criteria identified eight studies for inclusion, which may be a small number given the recent growth of interest in CDSS in healthcare settings.59 Nevertheless, piloting the electronic search strategy and supplementing it with hand searching of reference lists of previous reviews in the field permitted a sufficient level of confidence that all relevant articles, against the eligibility criteria, were included for synthesis. The review also contributes to the knowledge provided by a recent review that focuses on process outcomes.24

Some limitations arose from methodological constraints. Studies were of an average of 1-year duration, which makes it difficult to deduce long-term effects on patient outcomes. Nine common chronic diseases were included in the syntax.28 However, there was an absence of eligible studies in five of them, which limited the number of studies for inclusion. Although a selection of nine chronic diseases was included in the syntax, these covered the most common chronic diseases, as done by Reynold et al.28 The review could have covered other PCSCs for a broader scope, including cancer and mental health illnesses, per se. The included studies were also of low-to-moderate quality of evidence. Additionally, our review does not report on process outcomes. The decision to exclude process outcomes such as practitioner-related outcomes, namely, clinician adherence to CPGs was made to channel the research question to focus on where the most significant knowledge gap lies, as the effect of CDSS on patient outcomes remains largely uncertain.19–21 23 24 Nevertheless, capturing those outcomes would have provided a more comprehensive assessment on the effect of CDSS for the management of patients with chronic disease in primary care. The heterogeneity in study intervention components, participant covariates, setting attributes and CDSS characteristics rendered outcome pooling in a meta-analysis not suitable.

Conclusion

This systematic review aimed to assess the impact of CDSS integrated with EHRs on patient-reported and clinical outcomes of chronic disease patients in primary care settings. A definite conclusion on the intervention effect on outcomes could not be drawn due to between-study heterogeneity and methodological risks, reflecting an immature research area. Intervention characteristics and implementation approaches may have an impact on outcomes from CDSS use and we make reference to the literature in this area that may guide and help evaluate successful implementations of CDSS in practice. Further studies are needed to formulate more robust inferences and validate the intervention’s effect on patient outcomes. With the continuous rise in the prevalence of chronic diseases, it is essential to understand how enhancing primary care management using IT support could support improved outcomes. Practical applications of CDSS evaluations in the healthcare workflow and setting should be sustained to suitably determine the actual effectiveness of CDSS and support improved implementation efforts into practice. Strengthening the clinical benefit of CDSS and determining their cost effectiveness could inform policy-making of enhanced primary healthcare of chronic disease management.