Mor Saban1, Jacob Sosna2, Clara Singer3, Sharona Vaknin3, Vicki Myers3, Dorit Shaham2, Jacob Assaf4, Alon Hershko5, Paula Feder-Bubis6, Rachel Wilf-Miron3,7, Osnat Luxenburg8. 1. The Gertner Institute for Epidemiology and Health Policy Research, Sheba Medical Center, 526210, Ramat Gan, Israel. mors@gertner.health.gov.il. 2. Department of Radiology, Hadassah Hebrew University Medical Center, Jerusalem, Israel. 3. The Gertner Institute for Epidemiology and Health Policy Research, Sheba Medical Center, 526210, Ramat Gan, Israel. 4. Emergency Department, Hadassah Hebrew University Medical Center, Jerusalem, Israel. 5. Internal Department, Hadassah Hebrew University Medical Center, Jerusalem, Israel. 6. Department of Health Systems Management, Ben-Gurion University of the Negev, Beersheba, Israel. 7. School of Public Health, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel. 8. Medical Technology, Health Information and Research Directorate, Ministry of Health, Jerusalem, Israel.
Abstract
OBJECTIVE: To assess the acceptance and reliability of clinical decision support system (CDSS) imaging referral scores (ESR iGuide). METHODS: A pilot study was conducted in a tertiary hospital. Four different experts were invited to rate 40 simulated clinical cases on a 5-level scale, for the level of agreement with the ESR iGuide's recommended procedures. In cases of disagreement, physicians were asked to indicate the reason. Descriptive measures were calculated for the level of agreement. We also explored the degree of agreement between four different specialists, and examined the cases in which clinicians disagreed with ESR iGuide best practice recommendations. RESULTS: The mean rating of the four experts for the 40 clinical simulated cases was 4.17 ± 0.65, median 4.25 (on a scale of 1-5). All four raters totally agreed with the system recommendation in 75% of cases. No significant relationship was found between the degree of agreement and the number of indications and the patient's age or gender. In an optimistic scenario, using a binary agree/disagree variable, the Overall Percentage Agreement for the rating of the 40 simulated cases between the four experts was 77.28%. There were a total of 20 disagreements out of 160 cases with the ESR iGuide, of which 7 were among the two radiologists. CONCLUSIONS: CDSS can be an effective tool for guiding the selection of appropriate imaging examinations, thus cutting costs due to unnecessary imaging scans. Since this is a pilot study, further research on a larger scale, preferably at national level, is required. KEY POINTS: • The average of the mean rating of the four experts was 4.17 ± 0.65, median 4.25, on a scale of 1-5 where 5 represents total agreement with the CDSS tool. • In an optimistic scenario, using a binary agree/disagree variable, the Overall Percentage Agreement between the four experts was 77.28%. • Radiologists had fewer disagreements with the recommendations of the CDSS tool than other physicians, indicating a better fit of the support system to radiology experts' perspective.
OBJECTIVE: To assess the acceptance and reliability of clinical decision support system (CDSS) imaging referral scores (ESR iGuide). METHODS: A pilot study was conducted in a tertiary hospital. Four different experts were invited to rate 40 simulated clinical cases on a 5-level scale, for the level of agreement with the ESR iGuide's recommended procedures. In cases of disagreement, physicians were asked to indicate the reason. Descriptive measures were calculated for the level of agreement. We also explored the degree of agreement between four different specialists, and examined the cases in which clinicians disagreed with ESR iGuide best practice recommendations. RESULTS: The mean rating of the four experts for the 40 clinical simulated cases was 4.17 ± 0.65, median 4.25 (on a scale of 1-5). All four raters totally agreed with the system recommendation in 75% of cases. No significant relationship was found between the degree of agreement and the number of indications and the patient's age or gender. In an optimistic scenario, using a binary agree/disagree variable, the Overall Percentage Agreement for the rating of the 40 simulated cases between the four experts was 77.28%. There were a total of 20 disagreements out of 160 cases with the ESR iGuide, of which 7 were among the two radiologists. CONCLUSIONS: CDSS can be an effective tool for guiding the selection of appropriate imaging examinations, thus cutting costs due to unnecessary imaging scans. Since this is a pilot study, further research on a larger scale, preferably at national level, is required. KEY POINTS: • The average of the mean rating of the four experts was 4.17 ± 0.65, median 4.25, on a scale of 1-5 where 5 represents total agreement with the CDSS tool. • In an optimistic scenario, using a binary agree/disagree variable, the Overall Percentage Agreement between the four experts was 77.28%. • Radiologists had fewer disagreements with the recommendations of the CDSS tool than other physicians, indicating a better fit of the support system to radiology experts' perspective.
Authors: Aidin Abedi; Salar Tofighi; Sana Salehi; Paul T Latterman; Kyle D Basques; Ali Gholamrezanezhad Journal: Eur J Radiol Date: 2019-09-09 Impact factor: 3.528
Authors: Alan Rosenberg; Abiy Agiro; Marc Gottlieb; John Barron; Peter Brady; Ying Liu; Cindy Li; Andrea DeVries Journal: JAMA Intern Med Date: 2015-12 Impact factor: 21.873
Authors: Leonid L Chepelev; Xuan Wang; Benjamin Gold; Clara-Lea Bonzel; Frank Rybicki; Jennifer W Uyeda; Adnan Sheikh; Dan Anderson; Jared Lindaman; Greg Mogel; Dimitrios Mitsouras; Mary C Mahoney; Tianxi Cai; Frank J Rybicki Journal: J Digit Imaging Date: 2021-02-25 Impact factor: 4.056