Preparing for a New World: Making Friends with Digital Health.

While digital health solutions have shown good outcomes in numerous studies, the adoption of digital health solutions in clinical practice faces numerous challenges. To prepare for widespread adoption of digital health, stakeholders in digital health will need to establish an objective evaluation process, consider uncertainty through critical evaluation, be aware of inequity, and consider patient engagement. By "making friends" with digital health, health care can be improved for patients. © Copyright: Yonsei University College of Medicine 2022.

Digital health refers to digital technologies used for health care.12 It includes diverse technologies, ranging from established electronic health records (EHRs) and clinical decision support systems to more recent promising technologies employing artificial intelligence (AI), applications to medicine, telemedicine, mobile health, wearable, and digital therapeutics. Usually, digital health is integrated into health care as a specific program or technical solution.

The phrase ‘digital health’ is seen and heard frequently in research articles and on the news. Although it has not yet been widely adopted in hospitals,34 digital health is gradually being adopted in clinical practice. Due to the recent COVID-19 outbreak, hospitals have been forced to offer non-face-to-face services. Various digital solutions have been used to this end.5 These newly adopted technologies provide an opportunity for physicians, patients, and medical IT workers to prepare for the upcoming era of digital health. The rise of the “digital-native physician” is expected to bring significant changes to health care delivery.6

Recently published studies have demonstrated the enormous potential of digital health; however, further challenges need to be addressed for it to be integrated into clinical practice, similar to how a new drug is put into use. When a new drug is approved, unexpected side effects may occur when it is used in clinical practice, despite extensive clinical trials. Furthermore, the drug may be used in an environment other than the intended setting (i.e., that tested in the original clinical trial). The same situation will likely occur with digital health solutions. Digital health solutions may show better therapeutic and diagnostic performance than conventional approaches in study environments, without notable adverse events. However, treatment outcomes in real-life patients may vary significantly depending on how digital health is integrated into actual clinical practice.

At this juncture, as digital health is posed to be adopted by hospitals, we would like to discuss several facets of preparing for this paradigm shift: 1) establishing an objective evaluation process, 2) considering uncertainty through critical evaluation, 3) awareness of inequity, 4) incorporating digital health into clinical practice, and 5) patient engagement.

First, it is important to establish an objective evaluation process that can be easily used by healthcare professionals. In recent times, considerable health care research has been published on digital health in the scientific journals and media outlets. However, the level of evidence of health care studies is variable. In clinical practice, it is challenging to achieve the level of performance reported in studies with low-level evidence. For this reason, uptake of digital health solutions has varied greatly.7 Traditional approaches to generating evidence, such as randomized controlled clinical trials, are impractical because of the short lifespan of digital solutions.3 Therefore, clinicians need objective criteria for evaluating medical AI. One paper published in JAMIA suggested that the following should be considered: the study population and setting, patient demographics, model architecture, and model evaluation.8 Regarding model evaluation, it is important to ascertain how internal and external validation was obtained and the degree of transparency. For internal validation, data from the same patients obtained at different time points can be used. However, in this scenario, it is possible that the model will solve classification tasks by learning about the individual characteristics of the patient (unintended pattern) instead of the disease (intended pattern). This could improve internal validation performance, but would present a challenge to external validation. Even if external validation is performed, the generalizability of the model will be limited to the population group described in the external validation dataset. In other words, the reported performance may not be realized in the actual clinical environment. Furthermore, reliable verification of models requires transparency and disclosure of the source code and data used for training the model. However, since digital health care solutions are usually commercialized, disclosure of the entire source code and raw data presents challenges.

Next, when using such critical evaluation, understanding and consideration of the uncertainty included in the digital health solutions are important. When employing digital health solutions, healthcare providers need to be cautious about unconditional dependence (i.e., overreliance or automation bias).9 While digital health solutions are expected to automate many repetitive tasks in clinical practice, they exhibit limited AI capabilities (i.e., pattern recognition).10 Many papers report that AI models perform better than medical experts (e.g., radiologists).11 However, as stated previously, results may differ for patients outside the validation set used by the model (Fig. 1A).1213 This is because a model forecasts an individual’s risk of a given outcome on the basis of a group of individuals with similar features. Therefore, we need to consider differences between the environment where a digital health solution was developed and that in which it intends to be implemented to determine whether it can be adopted successfully. In addition, there ought to be preparations for situations wherein the decision to implement a solution was ultimately inappropriate.

Fig. 1

Real-world environments that differ from settings where a digital health solution was developed. (A) The distribution of real patient data may differ from that of the dataset used in model development. By adopting several solutions (here, models A and B), model performance can be improved. (B) Differences in disease prevalence can affect model performance in different patient groups. In this example, recall was decreased when the model was adopted for a low-prevalence group, even though precision was maintained. (C) The implementation of health care solutions varies among hospitals. One hospital may adopt solutions for initial assessments, whereas others may use them for diagnosis or post-monitoring.

The above-described issues have prompted efforts to reduce errors by calculating uncertainty.12 In the future, when numerous digital solutions will be part of standard health care, it may be difficult for users to understand the specific features of individual solutions. Therefore, models that can judge whether a given clinical environment is compatible with a digital health solution will be helpful. Accordingly, using several digital health solutions together may be preferable to applying a single one for a specific purpose (Fig. 1A). However, this will require political and economic decisions about treatment effectiveness and resource value. Alternatively, it may be necessary to establish an AI-based clinical department that determines whether a digital solution is appropriate for the clinical environment, deploys health care solutions based on a hospital’s particular situation, and maintains the required environment.14

Awareness of inequity is also necessary, as it could be a further barrier to adoption of digital solutions. For this, consideration of the characteristics of a population for which a digital solution is designed is needed.15 Even if a patient’s data were in the distribution of the dataset used to develop the model, the prevalence of the disease in the target population could differ from that in the training and validation datasets. This could lead to biased predictions and decisions that differ among groups.15 For example, the performance of a digital solution may vary depending on the prevalence of the disease (Fig. 1B). As a result, there may be a group of patients who do not benefit from said solution. Rather than focusing solely on the positives of digital health solutions, we should also consider strategies that help patients who may not benefit from these solutions.

The actual incorporation of digital health into clinical practice is another important issue. Patient care can be more complex and diverse than expected. This is because patient care is optimized for an individual hospital and department. The characteristics of individuals in an organization and of a patient group, as well as aspects of management, may affect the situation in a hospital or department. For this reason, hospitals use a variety of information technologies (e.g., EHRs) or adapt external solutions for their specific environment.

Digital solutions will develop in a similar manner (Fig. 1C). A solution that does not fit into existing work processes will not be used by clinicians, regardless of its quality.1016 Because work processes differ among hospitals and departments, widespread deployment of an inflexible solution will be challenging. If a model is used in a clinical setting that differs from the environment in which it was designed, the results could fall far short of the reported performance.

Lastly, patient engagement in digital health and consideration of older individuals who are not familiar with digital technologies must be addressed. Current digital health solutions are driven mainly by medical staff and the IT industry. However, patients will ultimately benefit from these solutions.16 There are also many instances of programs being used by patients. Solutions preferred by patients will eventually become more popular, so patient opinions should be considered when developing and implementing solutions in hospitals.

In conclusion, widespread adoption of digital health appears to be inevitable. Our approach to the adoption and use of digital health will determine whether it improves patient health and makes hospitals more competitive by increasing their efficiency. In Korea, a government-level project called the Korean New Deal has started, and as part of that project, an investment of 200 billion won by 2025 has been planned to aid in the development of healthcare AI models and to support smart hospitals that adopt digital healthcare solutions.17 As a part of these current efforts, we should continue to strive to make friends with digital health.