Impact on process results of clinical decision support systems (CDSSs) applied to medication use: overview of systematic reviews.

OBJECTIVE: The purpose of this overview (systematic review of systematic reviews) is to evaluate the impact of clinical decision support systems (CDSS) applied to medication use in the care process.
METHODS: A search for systematic reviews that address CDSS was performed on Medline following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Cochrane recommendations. Terms related to CDSS and systematic reviews were used in combination with Boolean operators and search field tags to build the electronic search strategy. There was no limitation of date or language for inclusion. We included revisions that investigated, as a main or secondary objective, changes in process outcomes. The Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) score was used to evaluate the quality of the studies.
RESULTS: The search retrieved 954 articles. Five articles were added through manual search, totaling an initial sample of 959 articles. After screening and reading in full, 44 systematic reviews met the inclusion criteria. In the medication-use processes where CDSS was used, the most common stages were prescribing (n=38 (86.36%) and administering (n=12 (27.27%)). Most of the systematic reviews demonstrated improvement in the health care process (30/44 - 68.2%). The main positive results were related to improvement of the quality of prescription by the physicians (14/30 - 46.6%) and reduction of errors in prescribing (5/30 - 16.6%). However, the quality of the studies was poor, according to the score used.
CONCLUSION: CDSSs represent a promising technology to optimize the medication-use process, especially related to improvement in the quality of prescriptions and reduction of prescribing errors, although higher quality studies are needed to establish the predictors of success in these systems.

INTRODUCTION

The volume and complexity of information needed to care for sick patients are immeasurable, especially when the lability of the clinical condition of a hospitalized patient is considered.1 In this context, the medication-use process is considered a key point for the reestablishment and maintenance of health.

The medication-use process comprises five major steps: 1) Prescription of a medicine based on the needs of the patient; 2) Transcription, the transmission of prescription information; 3) Dispensing, providing the medication according to the prescription; 4) Administration or use of the drug; and 5) Monitoring, the continued therapy evaluation of its effects on patients (i.e., effectiveness, safety, therapeutic failures).2,3,4 This process is complex and can be permeated by quality deviations and failures, resulting in medication errors at any stage.5,6,7,8,9

The use of information technology (IT) in the health area is a potentially transformative strategy for the quality of care and patient safety and has been embraced as a public policy in many countries.10,11 In 2009, the American Recovery and Reinvestment Act allocated more than thirty billion dollars to the development of health-related information technology.10,11

In this context, computerized decision support systems (CDSSs) are software programs designed to assist health professionals with daily decisions using complex communication, analysis, synthesis, and presentation of clinical information selected from multiple sources.1,12 These systems can improve the medication-use process by providing information for standardizing clinical practice; clinical alerts regarding drug allergies, drug interactions, and dose ranges appropriate to the needs of the patient; clinical updates on the latest information relevant to pharmacotherapy; dose adjustment calculations based on patient characteristics; and communications on critical changes in the condition of a patient, facilitating appropriate adjustments.13,14

Despite the increasing number of published studies on CDSSs, the effect of these systems on clinical, humanistic and economic outcomes is still contradictory, which highlights the need to evaluate other quality mediator parameters.

Considering health service evaluation, Donabedian outlined three constructs from which inferences can be made regarding their quality: outcomes, process and structure.15 In this definition, the construct process refers to what is done and how the care itself is done. From this perspective, the objective of the present systematic review was to evaluate the impact of CDDSs in the care process, considering the possible mediators of the quality of health care.

METHODS

A search for systematic reviews that addressed clinical decision support systems was performed on Medline on December 1, 2015 following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)16,17 and Cochrane18 recommendations. Terms related to CDSS and systematic reviews were used in combination with Boolean operators and search field tags to build the electronic search strategy that is fully provided in Table 1. There was no limitation of date or language for inclusion. Medline was the search engine selected because of its extensive indexation of health journals and wide use by researchers and health professionals from around the world. A manual search was also performed via triage in the references of the included systematic reviews.

Table 1

Search strategy in Pubmed

((Decision Support Systems, Clinical[MH] OR “computerized decision support systems”[TIAB] OR “Decision Support System*”[TIAB] OR “decision support”[TIAB] OR “computer-assisted decision making”[TIAB] OR “clinical decision support system*”[TIAB] OR “clinical decision support tool*”[TIAB] OR “computerized decision support tool*”[TIAB] OR “clinical decision making tool*”[TIAB]) AND (systematic review[PT] OR meta-analysis[PT] OR systematic review[TIAB] OR meta-analysis[TIAB] OR systematic[SB] OR meta-analysis[MH]) NOT (randomized controlled trial[PT] OR controlled clinical trial[PT] OR random allocation[MH]) AND hasabstract))

First, an initial screening from titles and abstracts was performed in duplicate and independently considering, as the main focus, the selection of systematic reviews that applied CDSS to the use of drugs, which reported changes in process outcomes, as defined by Donabedian.15 CDSSs applied to the medication-use process consisted of those applied in the following steps: selection and prescribing, transcribing and verifying, dispensing, administering and therapeutic monitoring.

Second, eligibility of the systematic reviews was assessed by full-text evaluation. Reviews that investigated, as the main or secondary objective, changes in the quality of the medication-use process were included. Narrative revisions, synthesis of secondary sources of data, and descriptive reviews were excluded.

After this step, we proceeded with data extraction that was performed using a previously defined Excel® worksheet. Different process outcomes were grouped by similarity, their impact classified as positive, data insufficient for evaluation, neutral or negative, considering the authors’ report, and the significance statistics performed in each review. Each systematic review included was evaluated using the Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) scale.19 The screening, eligibility and quality evaluation phases were performed in duplicate by two independent reviewers; the disagreements were resolved with a third reviewer.

RESULTS

The search strategy retrieved 954 articles. Five articles were added from manual search, totaling an initial sample of 959 articles. Subsequently, the screening of titles and abstracts resulted in the exclusion of 647 articles. The 309 remaining systematic reviews were read in full; 44 met the inclusion criteria and 267 were excluded. This process is shown in the flow chart provided in Figure 1, which also provides the reasons for excluding the articles after reading in full.

Figure 1

PRISMA flow diagram for the review process

Most systematic reviews included experimental studies, especially randomized controlled trials. A wide variety of types of CDSSs was observed, integrated or not to other systems, such as computerized provider order entry (CPOE) or electronic health records (EHR) (Online appendix). Considering the setting, most of the reviews (23/44 - 52.27%) included CDSS applied to outpatients and/or inpatients.20-42

There are five stages of the medication-use process: prescribing; transcribing and verifying; dispensing and delivering; administering; and monitoring. As shown in Online appendix, the major process stage, in which the CDSS was used, was prescribing (n=38 (86.36%)), followed by administering (n=12 (27.27%)). The use of CDSS was not observed for transcribing and verifying or dispensing and delivering processes. Additionally, there were systematic reviews that included more than one stage of the medication-use process prescribing and administering (n=12 (18.2%)), prescribing and monitoring (n=6 (13.63%)), and only one study considered three stages: prescribing, administering, and monitoring.43

Quality evaluation

In general, the quality of the included systematic reviews was poor (Table 2), which makes the inference of robust conclusions complicated. The average R-AMSTAR score of the reviews was 24.39 (SD=5.28). Only eight studies performed meta-analysis (18.2%), so the domain 9 was not applicable for the majority of studies, reflecting the low scores found. Seventeen studies scored 22 points or lower, and just five reviews scored higher than 30 points.

Table 2

Quality evaluation of systematic reviews according to R-AMSTAR

Author, year	1	2	3	4	5	6	7	8	9	10	11	Total score
Bright, 2012	3	2	3	2	2	4	4	2	3	2	3	30
Carling, 2013	3	3	4	3	4	4	2	3	NA	1	4	31
Chaudhry, 2006	2	3	3	1	2	3	1	2	NA	1	3	21
Durieux, 2008	4	3	4	2	4	4	1	2	NA	1	3	26
Eslami, 2007	3	3	3	1	2	1	1	1	NA	1	1	17
Fathima, 2014	3	3	4	2	3	4	2	2	NA	2	3	28
Fraccaro, 2015	3	1	4	3	1	3	1	1	NA	2	3	26
Garg, 2005	3	3	3	1	1	2	2	4	NA	1	3	23
Georgiou, 2013	3	3	4	2	2	2	1	1	NA	2	3	23
Gillaizeau, 2013	3	4	4	3	4	4	4	4	4	2	2	38
Hemens, 2011	4	3	2	1	2	4	2	1	NA	2	3	24
Heselmans, 2009	3	3	4	2	4	4	1	4	NA	1	3	29
Holstiege, 2015	2	3	4	3	4	4	2	2	NA	2	3	29
Hunt, 1998	2	3	3	3	1	2	2	2	NA	1	2	21
Jamal, 2009	3	1	4	3	1	2	2	4	NA	1	1	22
Jeffery, 2013	4	4	3	1	1	4	2	2	2	1	4	28
Lainer, 2013	3	3	4	3	2	4	1	1	NA	1	3	25
Lobach, 2012	3	4	4	2	4	4	3	3	4	4	3	38
Marasinghe, 2015	3	1	3	1	0	2	3	2	NA	2	3	20
Marc, 2013	1	1	2	1	0	1	1	1	NA	1	1	10
Millery, 2010	3	3	4	1	1	4	1	1	NA	2	3	23
Montgomery, 1998	3	1	3	3	1	4	1	1	NA	1	3	21
Nieuwlaat, 2011	3	3	3	1	2	4	2	2	4	1	3	28
Njie, 2015	2	3	1	1	0	3	2	4	NA	2	3	21
Oluoch, 2012	3	3	4	2	2	4	1	2	NA	1	2	24
O’Reilly, 2012	2	2	3	3	0	3	1	1	NA	1	3	19
Pearson, 2009	3	3	4	2	2	4	2	2	NA	1	3	26
Randell, 2007	3	3	3	4	1	2	2	1	NA	1	1	21
Robertson, 2010	3	4	4	2	2	4	3	2	NA	1	3	28
Roshanov, 2011 (1)	4	4	4	3	2	4	2	2	NA	2	3	30
Roshanov, 2011 (2)	3	3	3	1	2	4	2	2	NA	2	3	25
Sahota, 2011	3	3	3	3	2	4	3	2	NA	2	3	28
Schedlbauer, 2009	3	3	3	3	2	1	2	1	NA	1	1	20
Shebl, 2007	2	1	3	3	1	4	1	1	NA	1	3	20
Shojania, 2009	3	3	3	4	2	2	1	1	1	1	3	24
Shojania, 2010	3	3	3	4	2	2	1	1	1	1	3	24
Sintchenko, 2007	3	1	3	2	1	4	1	1	NA	2	1	19
Souza, 2011	3	3	3	3	2	4	2	2	NA	2	3	27
Sturzlinger, 2009	2	1	3	2	1	1	1	1	NA	1	1	14
Tawadrous, 2011	3	4	3	4	2	4	2	1	NA	1	3	27
Vedel, 2013	3	3	3	3	0	2	2	1	NA	2	2	22
Welch, 2013	2	2	3	2	2	4	1	1	NA	2	3	22
Wong, 2010	3	3	3	2	2	4	2	2	4	2	2	29
Yourman, 2008	3	3	4	1	2	4	1	1	NA	2	1	22

NA: not applicable

Impact of the CDSS for the medication-use process

Most of the systematic reviews demonstrated improvement of the health care process (30/44 – 68.2%)27-56, as shown in Online appendix.

Quality of prescriptions

The main positive results were related to the improvement of the quality of prescriptions by the physicians (14/30 – 46.6%). The improvement of the quality of prescription included dose adjustments according to the clinical condition38,43,47 and the inclusion of the pharmacist in the medication review process.56

A review evaluating CDSSs applied to the prescription, guiding dose adjustments for different conditions, indicated benefits in increasing the initial dose (standardized mean difference 1.12), increasing serum concentrations (standardized mean difference 1.12), reducing the time to therapeutic stabilization (standardized mean difference -0.55) and reducing the length of hospital stay (standardized mean difference -0.35).38

A review evaluating the impact of CDSSs on addressing safety concerns (drug interactions, contraindications, dose monitoring and adjustment) and those focusing on medication use in line with guideline recommendations demonstrated that those applied to the safety of prescription had a high success rate, with 91% (10/11) of the studies reporting statistically significant improvements in favor of CDSSs on ≥50% of all outcomes reported.47

Combined analysis of ten studies included in a systematic review, evaluating systems applied to the therapeutic management of asthma and Chronic Obstructive Pulmonary Disease, indicated significant improvements in cases such as the cumulative sparing of prednisone dose adjusted weekly according to the internet-based CDSS, the percentage of children given at least one prescription of corticosteroid, the percentage of visits to the physician leading to medication step up of asthma medication and in the number of contacts with the patients’ physician.43

Garg (2005), in his review evaluating the impact of CDSSs on the performance of health professionals, indicated improvement in professional performance in 62 (64%) of the 97 studies that evaluated this result, including 4 (40%) of 10 health systems, 16 (76%) of the 21 warning systems, 23 (62%) of 37 disease management systems and 19 (66%) of 29 drug administration or prescription systems.55

From another perspective, in their review of CDSSs applied to multi-morbidity, Fracaro and colleagues (2015) emphasized the importance of these systems in the pharmaceutical review of medication, with a higher rate of identification of potential problems than the human experts.56

Lobach et al. (2012), in their review evaluating the impact of three types of CDSSs on clinical, humanistic, economic and process outcomes, indicated that these systems effectively improved health care process measures related to performing preventive services (OR=1.42), ordering clinical studies (OR=1.72), and prescribing therapies (OR=1.57).35

Reduction of prescribing errors

The reduction of prescription errors was another outcome of processes benefitted by the adoption of the CDSS, as reported in five reviews (5/30 - 16.6%).29,39,41,45,54

Georgiou (2013) demonstrated that the use of CPOE with CDSSs was related to significant decreases in prescribing errors (17-201 errors per 100 orders), potential adverse drug events (0.9 per 100 orders), and prescribing of excessive dosages (31%).54

Similarly, Schedlbauer (2009), in his review evaluating the impact of Computerized Alerts and Prompts to Improve Clinicians’ prescribing behavior, indicated that most studies (23 out of 27) demonstrated a benefit in improving prescription behavior and/or reducing the error rates of prescriptions.29 Three other reviews noted the benefits of CDSSs in prescription safety.39,41,45

Lainer (2013), in his review evaluating information technology interventions to improve medication safety in primary care, indicated that only 50% of the included studies revealed a reduction in error rates.51

Other results positively influenced by CDSSs related to the process included preventive care40,44; vaccination28,39,44; adherence to guideline-based care33,35,39,40; and monitoring.32,37,39,48

Chaundry and collaborators conducted a systematic review evaluating the impact of CDSS on the quality, efficiency, and costs of the care process and demonstrated absolute increases of 5 to 66 percentage points in outcomes related to process quality. The primary preventive measures most commonly examined were influenza vaccination rates (improvement from 12 to 18%), pneumococcal vaccines (improvement, 20 to 33%) and laboratory tests for evaluation of fecal occult blood (improvement, 12 to 33%). Benefits related to drug doses were also observed, varying from 12% to 21%. Considering the quality of care, the main benefits were greater adherence to evidence-based care, monitoring of processes and reduction of medication errors. In addition, the authors found benefits in time spent by health professionals, with absolute reductions ranging from 8.5 to 24%.39

Njie, 2015 et al., in their review evaluating the impact of CDSS on cardiovascular prevention, noted an improvement in the follow-up of guidelines for screening and other preventive care services (average global increase of 3.8%), requests for clinical exams (mean global increase of 4.0%) and treatment (overall mean increase of 2.0%).33

Twelve studies21-26,57-62 (27.3%) exhibited insufficient data to prove the efficacy of the medication-use process because some of them included primary studies with inconsistent results or results with no statistically significant difference. Only two reviews (4.5%) revealed no benefit in using CDSSs in process improvement.20,63

DISCUSSION

Most CDSS applied to the medication-use process have shown benefits in the process outcomes (62%), this situation is consistent with results noted in previous overviews.64-69

An overview performed by Cresswell (2012), with inclusion of 41 systematic reviews evaluating CDSSs, indicated that such tools may result in improvements in professional performance, promotion of preventive care, and adherence to clinical guidelines, particularly if specific information is available in real time and systems are effectively integrated into clinical workflows. However, the relationship with clinical outcomes was modest.66

Jasper (2011)70, in his overview, evaluated the impact of CDSSs on the performance of health professionals and patient outcomes. This author selected only systematic reviews with high methodological quality and demonstrated that clinical decision support systems significantly improved the professional’s performance in 52 of 91 primary studies included in 16 systematic reviews that examined this effect (57%). Despite the consonant results regarding benefits in the care process, we note that the overviews cited were not directed at the medication-use process, a critical stage in care.

The methodological difference between the selection of studies of the overview realized by Jasper and ours is noteworthy. Different from this author, who limited the methodological quality reviews to the selection of randomized clinical trial reviews, our intention was to obtain a broader sample that included observational studies and quasi-experimental analyses to simulate the process of using medications in the “real world”.

According to our results, the prescribing and administering stages particularly benefitted by the introduction of clinical decision support systems. These findings are justified, since these steps are associated with high rates of medication errors.

The main positive outcomes provided by CDSS were improvement in the quality of prescriptions and a reduction in the rate of medication errors. Adherence to clinical guidelines for screening, prevention, requests for clinical exams and treatment were also cited.

An overview of 2016, evaluating the impact of CDSS on medication safety69, with the inclusion of 20 systematic reviews, also mentioned that these systems significantly affected the care process by reducing medication errors and improving quality in 75% of the studies. However, the impact on patient outcomes was less consistent.

It is important to note that the change in process outcomes is an important mediator of quality of care; however, it does not always imply a change in clinical, humanistic and economic outcomes. The association between process outcome change and clinical impact has been little explored in the literature and deserves a detailed evaluation in different clinical settings with adequate descriptions of intervention characteristics to consolidate the clinical applicability of CDSSs.

As a limitation, we highlight the low methodological quality and the high heterogeneity reported in the reviews included in this study. This situation may compromise the inference of robust conclusions from the synthesized data. We believe that the broad mapping of the CDSS process outcomes applied to the use of medications allows for an impact survey, considering different settings and intervention methodologies for CDSSs application, which may help health professionals and institutions in the visualization of potential benefits and the implementation of decision making.

CONCLUSIONS

The adequacy of the medication-use process is fundamental to guarantee the quality and safety of care. CDSSs represent promising alternatives to improve prescription quality and reduce medication errors. However, the low methodological quality of the included reviews indicates the need for the standardization of studies and adequate descriptions of the components of each CDSS in order to stratify the determinant characteristics of the success of these systems in their process outcomes.