Erika Schneider1, Stacy Zelenka2, Paul Grooff2, Dan Alexa2, Jennifer Bullen3, Nancy A Obuchowski4. 1. Imaging Institute, Cleveland Clinic Foundation, Cleveland, Ohio. Electronic address: schneie1@ccf.org. 2. Imaging Institute, Cleveland Clinic Foundation, Cleveland, Ohio. 3. Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, Ohio. 4. Imaging Institute, Cleveland Clinic Foundation, Cleveland, Ohio; Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, Ohio.
Abstract
PURPOSE: The goal of the study was to determine the effects of guideline implementation strategy using 2 commercial radiology clinical decision support (CDS) systems. METHODS: The appropriateness and insurance dispositions of MRI and CT orders were evaluated using the Medicalis SmartReq and Nuance RadPort CDS systems during 2 different 3-month periods. Logistic regression was used to compare these outcomes between the 2 systems, after adjusting for patient-mix differences. RESULTS: Approximately 2,000 consecutive outpatient MRI and CT orders were evaluated over 2 periods of 3 months each. Medicalis scored 60% of exams as "indeterminate" (insufficient information) or "not validated" (no guidelines). Excluding these cases, Nuance scored significantly more exams as appropriate than did Medicalis (80% versus 51%, P < .001) and predicted insurance outcome significantly more often (76% versus 58%, P < .001). Only when the Medicalis "indeterminate" and "not validated" categories were combined with the high- or moderate-utility categories did the 2 CDS systems have similar performance. Overall, 19% of examinations with low-utility ratings were reimbursed. Conversely, 0.8% of examinations with high- or moderate-utility ratings were denied reimbursement. CONCLUSIONS: The chief difference between the 2 CDS systems, and the strongest influence on outcomes, was how exams without relevant guidelines or with insufficient information were handled. Nuance augmented published guidelines with clinical best practice; Medicalis requested additional information utilizing pop-up windows. Thus, guideline implementation choices contributed to decision making and outcomes. User interface, specifically, the number of screens and completeness of indication choices, controlled CDS interactions and, coupled with guidance implementation, influenced willingness to use the CDS system.
PURPOSE: The goal of the study was to determine the effects of guideline implementation strategy using 2 commercial radiology clinical decision support (CDS) systems. METHODS: The appropriateness and insurance dispositions of MRI and CT orders were evaluated using the Medicalis SmartReq and Nuance RadPort CDS systems during 2 different 3-month periods. Logistic regression was used to compare these outcomes between the 2 systems, after adjusting for patient-mix differences. RESULTS: Approximately 2,000 consecutive outpatient MRI and CT orders were evaluated over 2 periods of 3 months each. Medicalis scored 60% of exams as "indeterminate" (insufficient information) or "not validated" (no guidelines). Excluding these cases, Nuance scored significantly more exams as appropriate than did Medicalis (80% versus 51%, P < .001) and predicted insurance outcome significantly more often (76% versus 58%, P < .001). Only when the Medicalis "indeterminate" and "not validated" categories were combined with the high- or moderate-utility categories did the 2 CDS systems have similar performance. Overall, 19% of examinations with low-utility ratings were reimbursed. Conversely, 0.8% of examinations with high- or moderate-utility ratings were denied reimbursement. CONCLUSIONS: The chief difference between the 2 CDS systems, and the strongest influence on outcomes, was how exams without relevant guidelines or with insufficient information were handled. Nuance augmented published guidelines with clinical best practice; Medicalis requested additional information utilizing pop-up windows. Thus, guideline implementation choices contributed to decision making and outcomes. User interface, specifically, the number of screens and completeness of indication choices, controlled CDS interactions and, coupled with guidance implementation, influenced willingness to use the CDS system.
Authors: Madan M Rehani; Emily R Melick; Raza M Alvi; Ruhani Doda Khera; Salma Batool-Anwar; Tomas G Neilan; Michael Bettmann Journal: Eur Radiol Date: 2019-12-02 Impact factor: 5.315