Computer decision support systems for asthma: a systematic review.

BACKGROUND: Increasing use of electronic health records offers the potential to incorporate computer decision support systems (CDSSs) to prompt evidence-based actions within routine consultations. AIM: To synthesise the evidence for the use of CDSSs by professionals managing people with asthma.
MATERIALS AND METHODS: We systematically searched Medline, Embase, Health Technology Assessment, Cochrane and Inspec databases (1990 to April 2012, no language restrictions) for trials, and four online repositories for unpublished studies. We also wrote to authors. Eligible studies were randomised controlled trials of CDSSs supporting professional management of asthma. Studies were appraised (Cochrane Risk of Bias Tool) and findings synthesised narratively.
RESULTS: A total of 5787 articles were screened, and eight trials were found eligible, with six at high risk of bias. Overall, CDSSs for professionals were ineffective. Usage of the systems was generally low: in the only trial at low risk of bias the CDSS was not used at all. When a CDSS was used, compliance with the advice offered was also low. However, if actually used, CDSSs could result in closer guideline adherence (improve investigating, prescribing and issuing of action plans) and could improve some clinical outcomes. The study at moderate risk of bias showed increased prescribing of inhaled steroids.
CONCLUSIONS: The current generation of CDSSs is unlikely to result in improvements in outcomes for patients with asthma because they are rarely used and the advice is not followed. Future decision support systems need to align better with professional workflows so that pertinent and timely advice is easily accessible within the consultation.

Introduction

The Global Initiative for Asthma estimates that 300 million people worldwide have asthma.[1] Prevalence rates as high as 32% have been recorded in the United Kingdom and Australia,[2] and the prevalence is increasing in many parts of the world.[3-5] Despite evidence-based guidelines,[1,6-9] there is consistent evidence that asthma is suboptimally controlled, resulting in unnecessary morbidity, loss of school and workdays, and high costs for countries.[9-11] There are 250,000 asthma-related deaths each year.[1]

There are many reasons why guidelines are poorly implemented, including physician’s lack of knowledge or inertia of practice.[12,13] As electronic health records are now the norm in many parts of the world,[14,15] it is feasible to provide professionals with computer decision support systems (CDSSs) to prompt evidence-based actions within routine consultations, potentially improving professional adherence to guidelines.

Our systematic review aimed to synthesise the evidence for the use of CDSSs by professionals managing people with asthma. We were primarily interested in the effectiveness of CDSSs in improving patient outcomes, but also sought to investigate process measures of guideline adherence and practical usage of the system.

Materials and Methods

Our protocol is registered with the PROSPERO international prospective register of systematic reviews (CRD 42012002412). We followed the methodology described in the Cochrane Handbook for Systematic Reviews of Interventions.[16]

Inclusion criteria

We used the PICOS (Participants, Intervention, Comparator, Outcomes, Study design) strategy for describing trials in which we were interested:

Participants

As this study is a review of the evidence, the study participants were de facto the health professionals using CDSSs who were caring for people with asthma—i.e., doctors, nurses and others (e.g., physiotherapists).

Intervention

We adopted Wyatt et al.’s definition of CDSs as ’active knowledge systems which use two or more items of patient data to generate case-specific advice.’[17] Haynes and Wilczynski similarly described such systems as ‘information technology which matches characteristics of individual patients to a computerised knowledge base’, with software algorithms generating patient-specific information in the form of recommendations.[18] There are various levels of sophistication for CDSSs, from reminders to enter specific data, prescribe certain drugs/vaccines or provide an asthma action plan, to a system retrieving patient asthma information from an electronic health-care record and providing a critique on the intended clinical action. Systems were included if they used patient data to generate case-specific asthma advice. Systems relating only to the task of asthma diagnosis or those exclusively providing patients with support for self-management were excluded.

Comparator

The comparator was ‘usual care’, specifically without the use of a CDSS.

Outcomes

Our primary interest was in the impact of CDSSs on clinical asthma control. In line with recommended guidelines,[19] we included outcomes that reflected current control (including asthma-related quality of life) and frequency of asthma exacerbations (including frequency of the general practitioner’s asthma visits, emergency department asthma visits and asthma hospitalisations).

We were also interested in the process by which CDSSs might impact asthma control, both practical usage issues (e.g. the proportion of professionals who actually used the CDSS, the numbers of alerts issued and the impact on time within the consultation) as well as process measures reflecting enhanced guideline adherence (e.g. changes in treatment, in tests ordered and in the proportion of patients with asthma action plans).

Study design

All reports of randomised controlled trials of CDSSs used by health-care professionals for patients with asthma, in any language, published and unpublished, were eligible for inclusion. No other study designs were included.

Information sources and search strategy

We searched Medline, Embase, Cochrane Central Register of Controlled Trials, Health Technology Assessment and Inspec (engineering) databases from 1990 to April 2012 with the terms listed in Supplementary Appendix 1. We wrote to experts and authors of all included studies requesting additional relevant studies. We searched for ongoing and unpublished trials on the following websites: https://portal.nihr.ac.uk/Pages/NRRArchive.aspx, www.clinicaltrials.gov, www.controlled-trials.com and www.anzctr.org.au.

Study selection

Two authors (PM and SM) independently screened titles and abstracts, assessing them against the inclusion criteria. The full text of each potentially eligible paper was reviewed by both authors to decide whether the study should be included. Disagreements were resolved by discussion and, if necessary, arbitration of a third researcher (HP, AS or JCW).

Data collection and abstraction

Using a piloted data extraction form, PM and SM independently extracted the following data from included trials: country, setting, funding, study design, health-care professionals, patient population, features of the CDSS intervention, description of the control group, outcome measures and any adverse effects. Extraction tables were compared, and discussed with a third researcher (HP, AS or JW) arbitrating in the event of unresolved disagreement.

Quality of reporting of trials

We assessed the risk of bias in each trial using the seven-criteria approach described in the Cochrane Handbook for Systematic Reviews of Interventions.[16] Overall, each study was rated as follows: A: low risk of bias—no bias found; B: moderate risk of bias—one criterion for risk of bias; C: high risk of bias—more than one criterion for risk of bias.

Synthesis of results

We anticipated considerable heterogeneity in the populations studied, and in the interventions and the outcomes reported in the trials precluding meta-analysis of data. Instead, we planned to undertake a narrative synthesis based on our theoretical model of how such computer systems are expected to exert their effects (see Figure 1). The expectation is that, in a linked causal chain, CDSSs will impact process outcomes, which, in turn, will impact clinical outcomes. The theory underpinning their effectiveness is that relevant reminders and recommendations during a consultation will influence clinicians’ behaviour and thereby improve guideline adherence as measured by process outcomes (e.g. more rational ordering of investigations, prescribing of treatment and use of asthma action plans). Implementation of evidence-based practices will consequently be measureable in clinical outcomes for asthma patients, such as fewer exacerbations, emergency department attendances and hospitalisations.

Figure 1

Theoretical model showing how a computer decision support system can improve asthma outcomes.

Results

Figure 2 is the PRISMA flow diagram. From 5,787 titles, eight studies were selected,[20-27] seven in English and one in Spanish.[26] One study had two reports.[24,28] None of the experts we contacted identified any additional eligible studies. We found nine ongoing and eight unpublished trials (Supplementary Appendix 2).

Figure 2

PRISMA flow diagram.

We excluded a small group of studies from the early 1990s of computerised theophylline dose calculators because they addressed a specific problem in emergency care and have already been evaluated in a Cochrane review.[29]

Study characteristics

See Table 1 for details of study characteristics. Most studies were cluster randomised controlled trials[20-26] in primary care in the UK[21,25] or the Netherlands.[23,24] Two studies randomised practices to receive a CDSS for asthma prescribing or a system for angina or cholesterol prescribing.[21,24] The practices providing data on angina and cholesterol prescribing were unaware that their (usual care) asthma prescribing data were control data for the parallel asthma study.

Table 1

Characteristics of studies

Author (country)	Study design	Participants and setting	Age (years)	Time scale	Intervention	Control
Bell et al.[20] (USA)	Cluster RCT	12 clusters: 12 primary care practices, 19,450 patients	0–18	12 months 6 months prior to study start clinicians participated in an educational programme, 12 months of intervention	CDSS embedded in an electronic health record (EHR) in the form of alerts and reminders based on expert asthma guidelines. This included a data entry tool, standardised documentation for asthma severity classification, standardised drug and spirometry order sets and an asthma control plan. There was also an educational programme for professionals.	The control group experienced educational programme for professionals. It also had access to the data entry and all documentation tools but only passively, without alerts and reminders.
Eccles et al.[21] (UK)	Cluster RCT with 2×2 incomplete block design	60 clusters: 60 primary care practices, 1,129 patients	⩾18	24 months 12 months baseline period, 12 months intervention	CDSS offered suggestions for management (including prescribing) depending on the chosen clinical scenario and requested the entry of relevant information.	Controls received intervention for angina, while the asthma intervention group was the control from the angina group as a strategy to balance the Hawthorne effect.
Fiks et al.[22] (USA)	Cluster RCT	20 clusters: 20 practices, 6,110 patients	5–19	6 months All intervention	CDSS was an EHR-based influenza vaccination alert system. Influenza vaccine alerts appeared prominently at the top of the computer screen in bold and highlighted text whenever the electronic health record was opened for a study subject who was due for this vaccine. Also a link was provided to simplify vaccine ordering.	Described as routine care.
Kuilboer et al.[23] (The Netherlands)	Cluster RCT	40 clusters: 32 primary care practices with a total of 40 GPs, each control practice with a mean of 4,933 control and 4,865 intervention patients	All	10 months 5 months baseline period, 5 months intervention	‘AsthmaCritic’, the CDSS, relied solely on the existing data in the EHR. Once data related to the visit was entered, the system evaluated whether the patient had asthma or COPD, reviewed the physician’s treatment of asthma and COPD, and generated feedback. In this way, the doctor made the decisions and the CDSS ‘critiqued’ these decisions.	Described as usual care.
Martens et al.[24,28] (The Netherlands)	Cluster RCT with an incomplete block design	53 clusters, 14 practices with a total of 53 GPs	All	12 months 6 months intervention, 6 months data collection	CDSS was part of a computer-reminder system integrated into the EHR as a prescribing module. When the GP prescribed a drug the decision support system was activated and provided information specific to the patient (e.g., age and gender) and the prescribed drug. The GP was obliged to enter a diagnosis code which the CDSS would check and use to issue relevant reminders.	One group that received prescription reminders for cholesterol-lowering drugs served as controls for the other group that received CDSS for antibiotics, asthma and COPD, and vice versa.
McCowan et al.[25] (UK)	Cluster RCT	40 clusters: 40 practices, 477 patients	All	6 months No baseline data	‘Asthma Crystal Byte’ was a stand-alone decision support system with management guidelines for asthma that aimed to improve the quality of the consultation. It included risk prediction software and printed asthma management plans.	The control group had no knowledge of the intervention and had to report parallel data on the same number of patients as were recruited to the intervention group.
Plaza et al.[26] (Spain)	Cluster RCT	20 clusters: 10 pulmonologists and 10 GPs, 198 patients	⩾14	12 months 6 months baseline and 2 sessions of educational programme for clinicians, 12 months intervention	CDSS providing patient-tailored recommendations based on the GINA guidelines enabled clinicians to establish the severity of asthma according to the GINA classification, from relevant inputs such as PEFR, symptom frequency, quantity of corticosteroids and the clinician’s professional opinion. Then the CDSS would recommend medications according to the GINA guidelines. There were also education programmes for clinician and patients, teaching inhaler technique and general information about the condition of asthma.	The control group worked as normal but recorded additional data for comparison.
Tierney et al.[27] (USA)	2×2 factorial randomisation of patients	4 clusters: 4 hospital-based academic practices with 25 faculty general internists and over 100 internal medicine residents, 1 full-time and 9 part-time pharmacists, 706 patients	⩾18	36 months 28 months recruitment and baseline, 8 months intervention	CDSS generated care suggestions based on agreed guidelines. These include performing pulmonary function tests, giving influenza and pneumococcal vaccinations, prescribing advice and encouraging smoking cessation. These suggestions were presented on doctors’ workstations or were printed under a list of active medications that doctors received along with the patient’s paper chart when he/she presented for usual care.	Care suggestions were still generated by the CDSS but were not displayed to the physician or pharmacists caring for patients in the control group.

Abbreviations: CDSS, computer decision support system; COPD, chronic obstructive pulmonary disease; GINA, The Global Initiative for Asthma; GP, general practitioner; RCT, randomised controlled trial.

Six of the systems were integrated into an electronic health record:[20,22-24,26,27] one was partly integrated[21] and one was a stand-alone system.[25] Five of the studies[20,21,23,26,27] explicitly reported that the system gave prescribing advice and reminders. One system concentrated solely on the prescribing of influenza vaccine for ‘at-risk’ children.[27] Four studies were based on asthma management guidelines.[21,25-27] One system included a complex risk prediction algorithm,[25] and one system ‘critiqued’ the doctor’s intended management plan and made recommendations.[23]

Risk of bias within studies

Table 2 lists the quality assessment: most studies were rated at high risk of bias. The study by Eccles et al.[21] was rated at low risk of bias and that by Martens et al.[24] at moderate risk of bias.

Table 2

Risk of bias summary table

Trial	Selection bias	Allocation concealment	Performance bias	Detection bias	Attrition bias	Selective reporting	Other bias	Quality
Bell et al.[20]	Yes—there were ethnic differences between suburban and rural practices; however, clustering would have helped to control for this	No allocation concealment	Yes—there was no blinding for users	Unclear—no mention of blinding of outcome assessors	Unclear as to how many of the patients enroled at each practice remained in the trial—pragmatic design, denominator quite flexible, withdrawals not reported.	Unclear—no pre-published protocol.	No	C—high risk
Eccles et al.[21]	No—minimised by computerised randomisation of practices in a cluster design	No allocation concealment	No—GPs were acting as controls for the other block	No—data collectors were blinded to the status of practice	No—attrition rates were presented and balanced; there were 31 intervention practices and 29 control practices who completed the study and two withdrawals.	No—a pre-published protocol-outlined plan for data analysis and embedded case study and economic evaluation.	No	A—low risk
Fiks et al.[22]	Unclear—no details of randomisation	No allocation concealment	Yes—no blinding, clinicians were aware that their software either did or did not have the alerts	Unclear—no mention of blinding of outcome assessors	No—attrition fairly balanced—no patients withdrew; however, there was fluctuation in the numbers of patients, as may be expected in such a large cohort.	Unclear—possibility of post hoc analysis of vaccination rates to achieve higher rates—no pre-published analysis protocol.	No	C—high risk
Kuilboer et al.[23]	No—randomisation performed with a table of random numbers by a researcher who was blinded to the identity of practices	No allocation concealment	Yes—there was no blinding for GP users	Unclear—no mention of blinding of outcome assessors	No—flow diagram explains why patients dropped out or withdrew. No attrition at practice level.	Unclear—no pre-published protocol.	No	C—high risk
Martens et al.[24,28]	Unclear—no details of randomisation	Yes—GPs blinded as to whether they were assessed on treatment of cholesterol or asthma and COPD	No—GPs did not know that they were acting as controls for the other block	Unclear—no mention of blinding of outcome assessors	No—attrition was fairly balanced but resulted in the study being underpowered. Reasons for attrition were given.	Unclear—no pre-published protocol.	No	B—moderate risk
McCowan et al.[25]	No—randomisation using random number sequence and performed independently of the project administration team	No allocation concealment	Yes—there was no blinding of GPs	Unclear—no mention of blinding of outcome assessors	No—attrition was unbalanced and although most practices gave some reasons this resulted in the study being underpowered and intention-to-treat analysis was impossible due to insufficient information.	Unclear—no pre-published protocol.	No	C—high risk
Plaza et al.[26]	No—randomisation using SAS (statistics programme). Patients were recruited as they came for consultation	No allocation concealment	Unclear, not reported	Unclear—no mention of blinding of outcome assessors	No—clinician withdrawals reported (2/22) due to administrative problems, patient withdrawals also reported in diagram.	Unclear—no pre-published protocol.	No	C—high risk
Tierney et al.[27]	No—randomisation by flip coin, then switching to equal numbers of consultations per arm by a researcher blinded to allocation.	No allocation concealment for professionals or patients	Yes—there was no blinding of GPs	Unclear—no mention of blinding of outcome assessors	No—flow diagram explains why patients dropped out or withdrew. Attrition appeared to be fairly balanced.	Yes—no pre-published protocol and post hoc analysis of power calculation.	No	C—high risk

Abbreviations: COPD, chronic obstructive pulmonary disease; GP, general practitioner.

Effectiveness of CDSSs

The impact of CDSS on process, usage and clinical outcomes is detailed in Table 3. It was anticipated that usage and process outcomes would influence clinical outcomes as reflected in our model (Figure 1).

Table 3

Effectiveness of CDSS: process outcomes—guideline adherence

Study	Risk of bias	Practical aspects of CDSS use	Process outccomes	Clinical outcomes	Interpretation
Eccles et al.[21]	Low	For both groups the median number of active interactions was zero. The number of alerts was approximately zero	No significant difference in drugs prescribed for asthma before and after introduction of CDSS. No significant difference in lung function assessment before and after OR 0.94 (0.67–1.33)	Overall effect of the CDSS on symptom score was non-significant: the parameter estimate from analysis of covariance of scale was −0.62 (95% CI is −2.12 to 0.88).[28] No effect on quality of life was measured on the validated Juniper’s Asthma Quality of Life Questionnaire (AQLQ).[30] No differences in GP visit rate; OR=0.94 (0.81–1.08)	The design of this British study incorporated two arms, each controlling for the other. The study was a cluster design, with practices as the unit of randomisation. Practices investigating CDSS-driven care for angina provided usual care control data for the asthma CDSS care practices, and vice versa. In addition, the study was very large, with 62 practices across the UK, and so results should have been robust. This trial demonstrated very clearly that CDSS will not be used by clinicians if it is not integrated with their usual workflow. The median usage of the CDSS in this study was zero and there were no differences in consultation rates, process outcomes or clinical outcomes, which were carefully measured.
Martens et al.[24,28]	Medium	GPs did not have a choice to decide if the CDSS was to be activated	44% of the intervention group were prescribed according to the recommendations compared with 27% of the control group among patients with mildly persistent asthma	No clinical outcomes reported	This Dutch study consisted of 14 general practices in a cluster randomised controlled trial. As in the Eccles study, two arms of the study acted as controls for each other. One arm was given a CDSS to guide on antibiotic, asthma and COPD prescribing, and the other received CDSS for cholesterol prescribing. This design minimises the impact of performance bias and the Hawthorne effect and has therefore contributed to it being rated as only at moderate risk of bias. The study was underpowered (the actual variation was larger than values used to estimate study power), which may have contributed to the non-significant results.
Bell et al.[20]	High	No difference between groups in the rate at which the CDSS was used (70% of the time during the intervention periods)	Controller medication prescribed more often in urban intervention practices compared with urban control practices (P=0.006) and NSD in suburban practices. Increase in spirometry in intervention sites from 15 to 24% and decrease in control sites from 8 to 1% (P=0.003). Number of asthma plans filed in suburban intervention practices increased compared with suburban controls (P=0.03). NSD in urban practices	No differences in GP visits	Although this US study was graded at high risk of bias, it did have a recognisable cluster design in which steps were taken to try to randomise the baseline differences of poverty and ethnicity in the different urban versus suburban practices. This study demonstrated that CDSS could improve the adherence to guidelines for prescribing, test ordering and use of asthma action plans. No clinical improvements were measured or reported in this trial and a major confounder was the introduction of asthma care-related pay-per-performance incentives during the time period of this trial (though this applied to both groups).
Fiks et al.[22]	High	Influenza vaccine alerts were active at only 27% of visits	Vaccination rates increased by 3.8% at control practices and by 4.8% at intervention sites	No differences in GP visits	This American study investigated the impact of CDSS for reminding clinicians to give children with asthma an influenza vaccination. The rate of increase in vaccination was not significantly different across the control and intervention groups as the rate increased in both groups. In interpreting this study it should be remembered that there are many influences on the uptake rate of vaccination, including whether a child is acutely unwell or not at the time they attend the clinic, and the health beliefs of the child and their parents.
Kuilboer et al.[23]	High	The doctor waited for the result of the CDSS analysis in 22% of 10,863 visits. 10,532 comments were produced and 32% of these were read by doctors. The CDSS took on average 31.7 s to analyse the record. The median time spent by the doctor reading comments was 9 s (25th percentile=4 s, 75th percentile=48 s)	Some evidence for a decrease in cromoglycate prescriptions in one of four age brackets, but no other significant changes. More tests were ordered among the CDSS group, but this difference was not always significant	No differences in GP visits except in one of the four age brackets, but risk of multiple testing	This trial provides some evidence of the effectiveness of CDSS in terms of its impact on guideline adherence. There were appreciable increases in the ordering of peak expiratory flow rates and spirometry. In addition, there was some evidence that doctors were more likely to change their prescribing of cromoglycate with the CDSS; however, there were no changes for the other drugs in the guideline (deptropine, antihistamines and oral bronchodilators)—probably because the general practitioners rarely prescribed these drugs anyway. Also measured were changes in the coding of the record: doctors recorded more data in a more structured fashion. It was reported that only a third of the comments were read by doctors. The explanation for this may be that the CDSS provided asthma-related comments irrespective of the reason for the visit.
McCowan et al.[25]	High	Usually less than 10 min to fill in the template and generate the advice according to a nested study	There was no difference in the proportions of patients in the different categories of maintenance prescribing according to the British asthma guidelines. No difference in PEFRs ordered. No difference in proportion with action plans	Reported no significant differences in asthma symptoms between the intervention and control groups (odds ratio 0.31, 95% CI, 0.03–3.32) In the CDSS intervention group, 12/147 patients had exacerbations and in the control group 57/330 patients had exacerbations; OR=0.43 (95% CI, 0.21–0.85) after adjusting for clustering. Therefore control patients were approximately twice as likely to experience an exacerbation as intervention patients Significantly fewer patients initiated GP consultations in the intervention group; OR 0.59 (0.37–0.95) No difference in emergency department visits: OR=0 (0–9.16) No difference in asthma hospitalisations; OR=0 (0–3.44)	From an initial 46 UK practices who registered to undertake the trial only 12 control practices and 5 intervention practices completed the trial. A significant number from the intervention practices had problems installing and using the software at the trial initiation. The CDSS was apparently partially effective in that there were significantly fewer exacerbations of asthma among intervention patients. However, the majority of outcomes (symptoms, inhaler technique and measurement of peak flow) were not statistically significantly different between control and intervention arms. This is on the basis of those who completed the trial; the data were not analysed by intention-to-treat analysis.
Plaza et al.[26]	High	Not reported	17.9 of control and 34% of intervention patients conformed to strict treatment guidelines (Wilcoxon P=0.0240). No difference in spirometry rates, X-rays allergy or blood tests	The number of patients with symptoms during the day in the intervention group was significantly less than that in the control group (Wilcoxon P<0.02). There was no difference between the groups in terms of nocturnal symptoms (Wilcoxon P=0.1). Exacerbation rates were not significantly different between the control and the intervention groups. Quality of life reported using the validated Spanish version of the St George’s Respiratory Questionnaire (SGRQ)[31] showed significant improvement by more than the minimally important difference of four points in all domains (activity P=0.002, symptoms P=0.003, impact P=0.001) No difference in GP visits (P>0.1) No difference in emergency department visits (P=0.0888) No difference in asthma hospitalisations (P>0.1)	This Spanish study reported randomising groups (clusters) to either the intervention or the control arm. It was a small study with only 10 doctors in each arm. There were two components to the intervention: the CDSS and an asthma education programme for nurses based on the GINA guidelines. This study produced significant improvements in the measures of the St George’s quality of life questionnaire. Daytime symptoms and exacerbations also improved but night-time symptoms did not. This study clearly demonstrated a link between significantly higher prescribing in the intervention arm of long-acting beta-agonists (especially formoterol) and leukotriene antagonists as per the guidelines and improved short-term outcomes (within 6 months). There was no significant difference in the rate of prescribing of inhaled steroids, oral steroids, anticholinergics or cromoglycate. This intervention was applied over a winter to spring period, which may have been a confounding factor in a seasonal condition such as asthma.
Tierney et al.[27]	High	87–95% of consultations resulted in the generation of a suggestion; doctors complied with only 32–37% of suggestions	5–9% of patients received the suggestion to ‘start inhaled corticosteroids.’ 11–0% of clinicians who received this suggestion adhered to it. Pulmonary function tests: 6% of the 39% in the control group and between 6 and 12% of 40–50% in the three intervention groups who received the suggestion adhered to it	The authors reported that patients with asthma in the pharmacist intervention arm of the trial had significantly (P<0.05) improved scores in the emotion subscale and that this was the only significant result following analysis of covariance of quality of life scores. It seems likely that this result may be significant only as a result of multiple testing No difference in emergency department vists or asthma hospitalisations	This study had four arms: one control and three intervention. The intervention arms consisted of physician CDSS intervention, pharmacist CDSS intervention and both physician and pharmacist intervention. There were no significant differences between the four study groups in adherence to the care suggestions. However, the care suggestions were also generated for the control patients—only that they were on paper, not on the computer. Adherence to care suggestions for the control arm varied from 9 to 71%. Adherence to care suggestions for the physician and pharmacist arm was from 12 to 91%. Overall, there was no clear pattern. It may be surmised that as the adherence to suggestions was very variable and frequently less than 50% this may explain why no significant differences were found in the quality of life and asthma control questionnaires.

Figures in brackets represent 95% confidence intervals.

Abbreviations: CDSS, computer decision support system; CI, confidence interval; COPD, chronic obstructive pulmonary disease; GINA, The Global Initiative for Asthma; GP, general practitioner; NR, not reported; NSD, no significant difference; OR, odds ratio.

Practical aspects of CDSS use

In the study by Eccles et al.,[21] the median number of activations of the system per practice was zero. In that by Kuilboer et al.,[23] 10,863 visits generated 10,532 decision support comments, but the doctor waited for the critique only 22% of the time, and then read only a third of them. In Tierney et al.’s study,[27] doctors complied with a third of the systems’ suggestions. Bell et al.[20] reported that the CDSS was used 70% of the time. In the study by Fiks et al.,[22] the vaccine alerts were only active during 27% of visits.

Process outcomes

Changes in tests ordered

Eccles et al.,[21] McCowan et al.[25] and Plaza et al.[26] all reported that the systems made no difference in the rates of ordering spirometry, X-rays, allergy tests or blood tests. Bell et al.[20] reported an increase in spirometry requests at intervention practices from 15 to 24%, whereas there was a decrease at control practices from 8 to 1%. In Kuilboer et al.,[23] peak expiratory flow rate and spirometry tests were ordered more often in the intervention group, in patients over 11 years of age. In a four-arm trial, Tierney et al.[27] reported that between 39 and 50% of patients received the suggestion to obtain pulmonary function tests.

Changes in treatments

Eccles et al.,[21] the only trial at low risk of bias, found no difference in asthma-related prescribing as a result of the intervention. Martens et al.[24] demonstrated an increase in the prescribing of inhaled corticosteroids to 44% of asthma patients (95% confidence interval (CI), 30–56%) in the intervention group, compared with 27% (95% CI, 14–47%) in the control group.

In the trial by Bell et al.,[20] there was a highly significant (P=0.006) difference between the rate of prescribing inhaled corticosteroids in the subgroup of urban intervention practices compared with urban control practices. Urban and suburban practices were analysed separately in the cluster controlled trial because of marked baseline differences in patient population: the urban practices had more severe asthma.

Kuilboer et al.[23] demonstrated a significant reduction in the prescribing of cromoglyate in a post hoc analysis. Plaza et al.[26] demonstrated a doubling of treatment conforming to guidelines, from 18 to 34% (P=0.02). Vaccination rates increased in both arms of the Fiks trial with no significant differences.[22] McCowan et al.[25] found no difference in asthma-related prescribing between the trial arms due to the intervention. Tierney et al.[27] reported on treatment suggestions for both asthma and chronic obstructive pulmonary disease. For example, across the four arms of the Tierney trial, between 5 and 9% of patients received the suggestion to ‘start inhaled corticosteroids.’ However, only 11–30% of the physicians or pharmacists complied with this suggestion.

Clinical outcomes

Asthma symptoms

Three studies reported asthma symptoms.[21,25,26] Eccles and coworkers[30] reported that the CDSS had no effect on the validated Newcastle Asthma Symptoms Questionnaire (mean difference −0.6 (95% CI, −2.1 to 0.9)).[21]

Plaza et al.[26] reported that asthma daytime symptoms, but not night-time symptoms, were significantly reduced in the intervention group compared with the control group (Wilcoxon P<0.02). McCowan et al.[25] reported no significant differences in asthma symptoms between the intervention and control groups (odds ratio 0.3, 95% CI, 0.03–3.3), although this study was underpowered.

Asthma-related quality of life

Three studies reported asthma-related quality of life.[21,26,27] The study by Eccles et al.,[21] a trial at low risk of bias, reported no effect on the validated Asthma Quality of Life Questionnaire.[31] Plaza and coworkers[32] reported quality of life using the Spanish version of the St George’s Respiratory Questionnaire and found significant improvement in all domains (activity P=0.002, symptoms P=0.003, impact P=0.001).[26] Tierney et al.[27]used two different quality-of-life scales,[33,34] but found a significant result only in one subdomain, possibly due to multiple testing.

Frequency of asthma exacerbations

Two studies reported exacerbation rates. In the study by Plaza et al.,[26] exacerbation rates were not significantly different between the control and intervention groups: mean exacerbations, 1.3 (s.e.)=1.2) in the control group and 0.5 (s.e.=0.3) in the intervention group (Wilcoxon P=0.2). McCowan et al.[25] reported that in the intervention group 12/147 patients had exacerbations compared with 57/330 in the control group: control patients were approximately twice as likely to experience an exacerbation as were intervention patients (odds ratio 0.4, 95% CI, 0.2–0.9, after adjustment for clustering). The denominators were different because of study dropouts.

Unscheduled health-care utilisation

McCowan et al.[25] reported significantly fewer unscheduled general practitioner consultations in the intervention group in comparison with the control group (odds ratio 0.6, 95% CI, 0.4–0.95). Four studies reported no differences in the frequency of asthma-related visits to the general practitioner.[20,22,23,26]

Two studies reported no significant difference between the intervention and control groups in emergency department visits or hospitalisations.[25,27] The absolute numbers were close to zero.

Discussion

Main findings

We found eight relevant trials, four of which reported clinical measures of asthma control.[21,25-27] The key finding was that CDSSs for health-care professionals were ineffective in improving patient outcomes because the systems were rarely used,[21-23] and there was low compliance with the advice when it was issued.[23,27] However, when systems are used, clinical outcomes can improve.[20,25]

Strengths and limitations of this study

A strength of this review is its robust search strategy. We used the Cochrane-suggested terminology for asthma and randomised controlled trials, and drew on our eHealth research group’s inclusive search terms for CDSS.[35] Nevertheless, we may have missed some relevant studies, and the list of ongoing trials suggests that more evidence may be available in due course.

In contrast to the methodology used by the recent McMaster group series of reviews in which improvement was considered to have occurred if >50% of the selected outcomes showed benefit,[18,36-41] we report specific clinical, usage and process outcomes from each trial to explain why the systems were having an effect or not.

We did not perform a meta-analysis as populations and outcomes across trials were too heterogeneous. Descriptions of interventions were often poorly described, which may have limited our interpretation of the findings.

Interpretation of findings in relation to previously published work

Our review focuses on asthma as a clinically important area for CDSSs. A crucial observation was that the systems were rarely used.[13,21,42] Usage was not considered in the recent McMaster group’s meta-regression,[43] although this is clearly fundamental to understanding the reasons for lack of effect, and should be a crucial focus for development if systems are to improve patient outcomes above the 15–31% impact on outcomes reported by the McMaster group in a series of reviews.[36-41] Usage rate of the systems should be a core standard for reporting trials of CDSSs.

The McMaster group’s meta-regression explored the features of CDSSs associated with system ‘effectiveness’. They found (1) stand-alone programs, (2) advice directed at both health-care practitioners and patients, (3) requiring users to enter an explanation for any overrides of system recommendations and (4) developers’ involvement in trials to be associated with better patient outcomes. Poor integration (as in a stand-alone program), however, risks clinicians avoiding using the system as in Eccles et al.[21] The issue, however, is complex as advice presented at the time of care does not always predict success, possibly because practitioners become overwhelmed by such integrated alerts that interrupt their workflow.[43]

Our recent analysis of recordings of general practice consultations emphasised the importance of the timing of alerts in the context of prescribing safety CDSSs.[44] The practitioner, negotiating with the patient, makes decisions regarding drugs and management throughout the consultation when information about allergies, sensitivities, interactions and guideline recommendations might be useful. Provision of information during the final computer-based task of generating the prescription can frustrate clinicians, who then override the alerts. Integration with workflow requires a detailed study of the consultation process.

Implications for future research, policy and practice

A detailed description of the CDSS intervention under investigation is essential to providing insight into what promotes a well-used and effective system that can inform future development. Taxonomies and frameworks such as those described by Kawamoto et al.,[45] Garg et al.[46] or Berlin et al.[47] may provide a suitable basis for a full description. Future research should substantiate our theoretical model (Figure 1), which we suggest as a possible useful framework. In terms of the logical chain from usage to process outcomes to clinical outcomes, Bell et al.[20] demonstrated that usage rates have an impact on process outcomes, and Plaza et al.[26] demonstrated the impact of process outcomes on health outcomes. However, we feel that further research is required to evidence this model more thoroughly.

Conclusions

Our review suggests that current CDSSs are unlikely to result in improved outcomes in asthma because they are rarely used and the advice not followed. A key challenge in the future design of decision support systems lies in the better integration and alignment with professional workflows such that they are adopted into routine practice.