Implementation of Korean Clinical Imaging Guidelines: A Mobile App-Based Decision Support System.

OBJECTIVE: The aims of this study were to develop a mobile app-based clinical decision support system (CDSS) for implementation of Korean clinical imaging guidelines (K-CIGs) and to assess future developments therein.
MATERIALS AND METHODS: K-CIGs were implemented in the form of a web-based application (http://cdss.or.kr/). The app containing K-CIGs consists of 53 information databases, including 10 medical subspecialties and 119 guidelines, developed by the Korean Society of Radiology (KSR) between 2015 and 2017. An email survey consisting of 18 questions on the implementation of K-CIGs and the mobile app-based CDSS was distributed to 43 members of the guideline working group (expert members of the KSR and Korean Academy of Oral and Maxillofacial Radiology) and 23 members of the consultant group (clinical experts belonging to related medical societies) to gauge opinion on the future developmental direction of K-CIGs.
RESULTS: The web-based mobile app can be downloaded from the Google Play Store. Detailed information on the grade of recommendation, evidence level, and radiation dose for each imaging modality in the K-CIGs can be accessed via the home page and side menus. In total, 32 of the 66 experts contacted completed the survey (response rate, 45%). Twenty-four of the 32 respondents were from the working group and eight were from the consulting group. Most (93.8%) of the respondents agreed on the need for ongoing development and implementation of K-CIGs.
CONCLUSION: This study describes the mobile app-based CDSS designed for implementation of K-CIGs in Korea. The results will allow physicians to have easy access to the K-CIGs and encourage appropriate use of imaging modalities.

INTRODUCTION

Evidence-based medicine (EBM) is characterized by conscientious, explicit, and judicious use of the best available evidence when making decisions about the care of individual patients (1). In clinical practice, EBM refers to the integration of clinical expertise with the best available external clinical evidence, as derived from systematic research, coupled with consideration of each patient's individual values, to inform the decision-making process regarding patient care (123). In the era of EBM, clinical practice guidelines, which are systematically developed statements used to assist practitioners and patients regarding decisions on appropriate treatments under specific clinical circumstances, are essential to optimize patient care and to assess the benefits and drawbacks of alternative treatment options (34).

In the field of radiology, overuse or inappropriate use of imaging modalities is a long-standing issue because of the potential harm caused by unnecessarily exposing patients to excess radiation, as well as increased medical costs (56789). However, ordering appropriate imaging tests can be challenging for many physicians, particularly under time constraints and when there is no clear guideline to support a decision. Therefore, many countries have started to develop clinical imaging guidelines (CIGs) to determine the most appropriate imaging modalities based on the best available evidence (101112). These CIGs are also publicly available as mobile applications in the form of clinical decision support systems (CDSSs), which can be easily accessed by ordering physicians and their patients (111213). Use of CIGs is believed to enhance quality of care and efficacious use of radiology (7).

The Korean Society of Radiology (KSR) started development of Korean CIGs (K-CIGs) in 2015, and work thereon is still in progress (1415). In response to the increasing number of guidelines, a mobile app-based CDSS was proposed via which K-CIGs could be implemented. The aims of this study were to develop a mobile app-based CDSS for implementation of K-CIGs and to assess future developments therein.

MATERIALS AND METHODS

Planning and Development of a Mobile App-Based CDSS

A web-based application was designed to promote clinical implementation of K-CIGs. Angular JS, which is a JavaScript-based, dynamic web application platform, and the Firebase database and Firebase-Auth frameworks, which constitute a mobile app platform and cloud-based, real-time database (Google, Inc., San Jose, CA, USA; https://firebase.google.com/), were used to create a data storage facility for the mobile web application interface and Android application (1617). This service includes a real-time NoSQL database management system with guaranteed scalability of data.

The app allows keyword searches in the K-CIGs, based on symptoms and diseases, thereby allowing easier access to guidelines for ordering physicians and patients. It also groups the guidelines according to 10 subspecialties (brain/head and neck, thyroid, chest, cardiovascular, breast, abdomen, genitourinary, musculoskeletal, pediatric, and interventional). A total of 119 guidelines for 53 key questions were integrated into a single JavaScript Object Notation (JSON) format using the jsonlite package of R software, which uses a tabular relationship instead of traditional relational databases (18). The K-CIGs can be downloaded as a JSON file via the “About CDSS” menu without authorization.

The Survey

An email survey comprising 18 questions on implementation of the K-CIGs was distributed to 43 expert members of the KSR and the Korean Academy of Oral and Maxillofacial Radiology and to 23 clinical experts belonging to related medical societies (consultant group) in January 2018. The survey aimed to gauge opinion on the current status and future developmental direction of K-CIGs (Table 1). Most of the questions were answered on a 7-point scale. The questions were organized into five domains: awareness of developments in K-CIGs; obstacles to application of K-CIGs; awareness of the development of a mobile app-based CDSS; the expected clinical utility of the app; and promotion of implementation of K-CIGs. Responses were received over a 2-week period and reminders were sent out twice during that time.

Table 1

Survey Questions and Distribution of Answers

Questions	Disagree ◄---------------------------Neutral---------------------------► Agree
	1	2	3	4	5	6	7
1. Awareness of developments in K-CIGs
1-1. Is continued development of K-CIGs worthwhile?	0 (0.0)	1 (3.1)	0 (0.0)	1 (3.1)	4 (12.5)	10 (31.3)	16 (50.0)
1-2. Do K-CIGs help justify diagnostic radiation use?	0 (0.0)	1 (3.1)	0 (0.0)	1 (3.1)	4 (12.5)	10 (31.3)	16 (50.0)
1-3. Will CIGs help clinicians to determine appropriate imaging modalities?	0 (0.0)	1 (3.1)	1 (3.1)	2 (6.3)	8 (25.0)	10 (31.3)	10 (31.3)
2. Obstacles to application of K-CIGs
2-1. Would clinician non-compliance or preferences for certain imaging tests impact K-CIGs use?	0 (0.0)	1 (3.1)	3 (9.4)	4 (12.5)	14 (43.8)	7 (21.9)	3 (9.4)
2-2. Is there lack of physical support for application of K-CIGs?	0 (0.0)	5 (15.6)	3 (9.4)	3 (9.4)	9 (28.1)	7 (21.9)	5 (15.6)
2-3. Is there lack of institutional support for application of K-CIGs?	0 (0.0)	0 (0.0)	1 (3.1)	3 (9.4)	7 (21.9)	10 (31.3)	11 (34.4)
2-4. Is there lack of education or public relations efforts supporting application of K-CIGs?	0 (0.0)	0 (0.0)	0 (0.0)	2 (6.3)	9 (28.1)	8 (25.0)	13 (40.6)
3. Awareness of development of mobile app-based CDSS
3-1. Would it be worthwhile to introduce CDSS to facilitate implementation of K-CIGs?	0 (0.0)	0 (0.0)	1 (3.1)	5 (15.6)	8 (25.0)	10 (31.3)	8 (25.0)
3-2. Would it be worthwhile to develop mobile app-based CDSS to facilitate implementation of K-CIGs?	0 (0.0)	2 (6.3)	1 (3.1)	3 (9.4)	8 (25.0)	10 (31.3)	8 (25.0)
4. Expected effect of K-CIGs implementation
4-1. Would applying K-CIGs reduce medical costs?	0 (0.0)	1 (3.1)	2 (6.3)	3 (9.4)	10 (31.3)	12 (37.5)	4 (12.5)
4-2. Would K-CIGs assist in efficient allocation of limited medical resources?	0 (0.0)	2 (6.3)	1 (3.1)	2 (6.3)	7 (21.9)	16 (50.0)	4 (12.5)
4-3. Would applying K-CIGs help improve quality of care?	1 (3.1)	1 (3.1)	1 (3.1)	1 (3.1)	4 (12.5)	15 (46.9)	9 (28.1)
5. Factors promoting application of K-CIGs
5-1. Are K-CIGs developed based on clear evidence?	0 (0.0)	0 (0.0)	1 (3.1)	1 (3.1)	5 (15.6)	13 (40.6)	12 (37.5)
5-2. Are sufficient organizational efforts and manpower being dedicated to continuous development of K-CIGs?	0 (0.0)	1 (3.1)	0 (0.0)	4 (12.5)	5 (15.6)	15 (46.9)	7 (21.9)
5-3. Are K-CIGs being continuously monitored and evaluated to ensure proper use?	0 (0.0)	0 (0.0)	1 (3.1)	1 (3.1)	5 (15.6)	12 (37.5)	13 (40.6)
5-4. Is CDSS being developed?	0 (0.0)	0 (0.0)	0 (0.0)	6 (18.8)	6 (18.8)	13 (40.6)	7 (21.9)
5-5. Is there link between insurance system and K-CIGs?	0 (0.0)	2 (6.3)	0 (0.0)	2 (6.3)	4 (12.5)	10 (31.3)	14 (43.8)
6. Free space for suggestions that could facilitate application of K-CIGs

Numbers in parentheses are percentages. CDSS = clinical decision support system, K-CIGs = Korean clinical imaging guidelines

RESULTS

Mobile App-Based CDSS

K-CIGs were implemented as a web application (http://cdss.or.kr/) and can be accessed by the public without the need for any special permissions. The web-based Android applications can also be downloaded from the Google Play Store.

Detailed information on the grade of recommendation, evidence level, and radiation dose for each imaging modality in the K-CIGs can be accessed via the home page and side menus (Fig. 1). Recommendation grades in the K-CIGs can be classified by reliability from the most (grade A) to the least reliable (i.e., not recommended; grade C). Grade I indicates no recommendation. The level of evidence when grading each K-CIG was categorized as high (I), moderate (II), low (III), or very low (IV). The radiation level of the different imaging modalities was divided into five categories according to the likely risk of radiation exposure, measured in mSv, and based on the methodology used to develop evidence-based CIGs (19) (Fig. 2). The values for all the information are visualized as a simple colored bar plot. The recommendation grades and levels of evidence are positive in the lower sense, so plots are visualized in the reverse direction with green color. Conversely, the radiation dose is drawn in the forward direction with red color in the same manner.

Fig. 1

Algorithm for mobile app-based CDSS.

K-CIGs can be searched for using keywords (symptoms or diseases) or by 10 subspecialty categories (via home page and side menus). CDSS = clinical decision support system, K-CIGs = Korean clinical imaging guidelines

Fig. 2

Information on recommendation grade, evidence level, and radiation dose provided by K-CIGs.

CT = computed tomography, CTA = computed tomography angiography, IVU = intravenous urography, MRI = magnetic resonance imaging, PA=posteroanterior, RRL = relative radiation level, UGIS = upper gastrointestinal imaging series

The 18-question survey was sent to the 66 experts detailed above and completed by 32 respondents (response rate, 45%). Twenty-four of the 32 respondents were from the working group and eight were from the consulting group (Fig. 3).

Fig. 3

Radiologic subspecialties represented by respondents (multiple responses possible).

Table 1 shows the survey questions and the distribution of answers. Regarding awareness of developments in the K-CIGs, most of the respondents (93.8%, 30/32) agreed on the need for continued development and implementation of K-CIGs; few respondents disagreed (3.1%, 1/32) or were undecided (3.1%, 1/32). Most (93.8%, 30/32) agreed that developing K-CIGs was important to justify the use of diagnostic radiation, and 87.6% (28/32) answered that K-CIGs would help ordering clinicians to determine the appropriate imaging tools for a given case. However, 24/32 respondents (75.0%) answered that clinician non-compliance or preferences for particular imaging modalities would affect the application of K-CIGs, while 4/32 (12.5%) considered that personal factors would have little effect on implementation of K-CIGs. In addition, 21/32 respondents (65.5%) noted a lack of any computerized system integrated with electronic medical records and 28/32 (87.5%) noted lack of an institutional environment, such as a health insurance or incentive system, to support application of K-CIGs. Meanwhile, 30/32 respondents (93.8%) noted a lack of provision of educational programs or public relations strategies by the KSR to support application of K-CIGs.

Twenty-six of 32 respondents (81.3%) agreed that introduction of a CDSS to support application of K-CIGs, including via a mobile app, would be worthwhile. Moreover, 26/32 respondents (81.3%) agreed that K-CIGs would contribute to reducing medical costs, while 27/32 (84.4%) agreed that such an application would assist in efficient allocation of limited medical resources. In addition, 28/32 respondents (87.5%) agreed that K-CIGs would ultimately help to improve the quality of health care. In terms of promoting the use of K-CIGs, clear evidence-based guidelines (93.8%, 30/32), continuous monitoring and evaluation of appropriate use (93.8%, 30/32), a link with the health insurance system (87.5%, 28/32), organizational efforts and manpower dedicated to development of K-CIGs (84.4%, 27/32), and development of CDSS (81.3%, 26/32) were all considered necessary by the majority of respondents. Continuous guideline updates to keep pace with emerging evidence, as well as promotional and educational activities aimed at clinicians and the general public, were also suggested by the respondents as directions for future development of K-CIGs.

DISCUSSION

Health care expenses are steadily increasing worldwide, and imaging studies account for a considerable proportion of all health care costs. However, although imaging is useful for accurate diagnosis and treatment, it may not be appropriate for certain conditions, and issues remain regarding the steady increase in exposure of the general population to medical radiation over the past 20 years (56789). These issues are largely attributable to the use of computed tomography and nuclear medicine, and several studies have suggested that a percentage of all future cancers will result from current imaging use (56). Therefore, judicious use of radiology is now a major issue, with a change in the prevailing concept from volume-based to value-based care in radiology (8). The American College of Radiology (ACR) Appropriateness Criteria are well-known evidence-based guidelines that assist referring physicians and other health care providers to make the most appropriate imaging decisions for specific clinical conditions (10). Since 1993, these guidelines have been continuously updated and have been integrated with electronic health/medical records within a CDSS known as the ACR Select™ (20). This resource covers more than 3000 clinical scenarios and includes 15000 clinical endpoints. A study by Huber et al. (21) reported a significant improvement in appropriateness scores, especially in imaging studies ordered by trainees, after implementation of a CDSS tool in the United States. Similarly, the Royal College of Radiologists have continually developed referral guidelines for imaging modalities (i.e., iRefer) since 1989 and has incorporated these guidelines into a CDSS known as OrderWise (MedCurrent Corp., Toronto, Canada), which affords simultaneous access to criteria for appropriate use and clinical guidelines from multiple sources (11). As part of these efforts, the European Society of Radiology (ESR) also provides the ESR iGuide (22).

In Korea, the development of K-CIGs began in 2015 based on the process of adapting existing imaging guidelines (19). In the protocol for development of K-CIGs, 53 guidelines pertaining to 25 key questions were initially developed during 2015–2016 with support from the KSR and the National Evidence-based Healthcare Collaborating Agency (14). Sixty-six guidelines for 28 key questions were then developed during 2016–2017, again with support from the KSR and Korea Centers for Disease Control and Prevention (15). However, although paper records and searchable PDFs are useful, they have limitations with respect to use of K-CIGs at the point of care. Therefore, a web-based app was considered a necessity for clinical use of K-CIGs. A mobile app-based CDSS is the first step toward such clinical use and has potential for widespread uptake by ordering physicians. Moreover, it is expected that mobile app-based CDSS will contribute to efficacious use of radiology (72324). In the present survey, most respondents agreed regarding the need to develop and implement K-CIGs and approved of the introduction of a mobile app-based CDSS. However, for the development of such an app, obstacles such as clinician noncompliance or specific preferences would need to be addressed, and improvements in physical, institutional, and educational support would be required. In-depth discussions are necessary to determine the environmental factors most in need of improvement, and how these improvements could be achieved in the future. These efforts will assist physicians in appropriate utilization of imaging modalities. In particular, increasing the range of diagnostic imaging scenarios via ongoing refinements in expert-developed guidelines is necessary to promote the use of K-CIGs.

A CDSS may be defined as a computerized system designed to improve clinical decision-making pertaining to individual patients (25). A CDSS that assists in prescribing, diagnosis, therapy, information retrieval, and image analysis could take a number of forms. CDSS confers a number of important benefits by promoting avoidance of errors and thus an increase in the quality of care and, potentially, superior patient outcomes. In the field of radiology, several studies have reported on the potential impact of adoption and use of a computerized physician order entry system integrated with CDSS for imaging (2124). Such systems, if widely accepted for clinical use, would influence ordering habits and potentially increase the appropriateness of imaging studies, especially in emergency department and inpatient settings. Although many examinations are ordered without involvement of radiologists, we believe that a mobile app-based CDSS will be useful to guide physicians toward appropriate imaging test requests. In ACR Select™, the CDSS provides an appropriateness score for every examination; these data can then be used as the basis for analytics and are expected to assist in understanding the reasons for inappropriate utilization of imaging modalities (20). Radiologists could also contribute to education designed to improve the performance of ordering physicians. Such measures could have a positive impact on creation of K-CIGs and their implementation in practice.

This study had some limitations. First, the proposed mobile app-based CDSS is not a final product, but rather the first step toward developing such a tool, which will ultimately be integrated into the medical records system. The system is expected to be continuously updated according to the newly developed guidelines. Second, although the survey was sent to all members of the guidelines working group and consulting group, the response rate was only 45%. Therefore, there may be some degree of selection bias in the results.

In conclusion, this study proposes the first mobile app-based CDSS to facilitate implementation of K-CIGs in Korea. The results will allow physicians to have easy access to the K-CIGs and encourage appropriate use of imaging modalities.