Effectiveness of Mobile Health Application Use to Improve Health Behavior Changes: A Systematic Review of Randomized Controlled Trials.

OBJECTIVES: The purpose of this study was to examine the effectiveness of mobile health applications in changing health-related behaviors and clinical health outcomes.
METHODS: A systematic review was conducted in this study. We conducted a comprehensive bibliographic search of articles on health behavior changes related to the use of mobile health applications in peer-reviewed journals published between January 1, 2000 and May 31, 2017. We used databases including CHINAHL, Ovid-Medline, EMBASE, and PubMed. The risk of bias assessment of the retrieved articles was examined using the Scottish Intercollegiate Guidelines Network.
RESULTS: A total of 20 articles met the inclusion criteria. Sixteen among 20 studies reported that applications have a positive impact on the targeted health behaviors or clinical health outcomes. In addition, most of the studies, which examined the satisfaction of participants, showed health app users have a statistically significant higher satisfaction.
CONCLUSIONS: Despite the high risk of bias, such as selection, performance, and detection, this systematic review found that the use of mobile health applications has a positive impact on health-related behaviors and clinical health outcomes. Application users were more satisfied with using mobile health applications to manage their health in comparison to users of conventional care.

I. Introduction

The global mobile health (mHealth) application (app) market has been growing at a tremendous rate, and it is expected to continue to flourish [1]. These mHealth apps provide quick and easy access, transfer, and tracking of health information as well as interactive displays and interventions that can allow users to be highly engaged in promoting health outcomes and changing health-related behaviors [2]. Thus, health-related apps have a great potential to aid a wide range of target audiences with a variety of health issues [3].

Despite the evolution and widespread use of these mHealth apps, the factors involved in smartphone and health app use and their effectiveness are not yet fully understood and the field of research related to mHealth apps is still in a nascent stage [4]. Kitsiou et al. [5] mentioned that a wide range of mHealth apps have not been strictly evaluated. For this reason, most consumers use mHealth apps without any concrete information about their effectiveness or harm and evidence of the effectiveness of mHealth has been inconclusive and not fully understood [4].

Thus, research determining the effectiveness of health apps is urgently needed [2]. This study aimed to demonstrate the effectiveness of mHealth apps in changing health-related behaviors and clinical health outcomes through a systematic review of randomized controlled trials (RCTs).

II. Methods

1. Search Strategy

We searched the electronic literature of RCTs published from January 1, 2000 to May 31, 2017, using four databases: the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, Excerpta Medica dataBASE (EMBASE), and Ovid Medline.

A university librarian was consulted who was a subject expert in the field of teaching and learning of systematic review. Searches used the following medical subject headings terms and keywords in various combination. We derived three broad themes that were then combined using the Boolean operator ‘AND’. The first theme ‘mobile’ was created using the Boolean operator ‘OR’ to combine text words (mobile*, OR smartphone*). The second theme ‘application’ was created using the Boolean operator ‘OR’ to combine text words (app*, OR application*). The third theme ‘health behaviors’, was created using the Boolean operator ‘OR’ to combine text words (health behaviors*, OR health behaviors change*, OR behaviors change*).

In this study, to secure the quality of this systematic review, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used [6]. This is a tool developed to verify the quality of evidence obtained during systematic reviews.

2. Study Selection

Two investigators independently reviewed the titles first and then examined the abstracts. Data extraction was conducted by one reviewer (MH Han) and was rechecked for by another reviewer to confirm the accuracy (EJ Lee). The same investigators read and screened the full texts to make the final decision. The reasons for inclusion and exclusion were recorded.

We included articles with the following characteristics: (1) published in English, (2) published during the period from 2000 to 2017, (3) results related to changes in health behaviors, (4) RCTs designed for app-based interventions to improve any health-related behaviors. The exclusion criteria were being other kinds of study than RCTs; qualitative studies; books; conference proceedings; reviews; dissertations; protocols; or studies examining text messages, Web, emails, Twitter, social network services, or personal digital assistant-based health interventions. We also excluded studies lacking behavior change indicators or outcomes, not using apps as the primary intervention tools; or focusing primarily on app design and development. Conference abstracts, protocol papers, reviews, editorials, and commentary were also excluded.

References that clearly did not meet all criteria were excluded. Full-text articles that appeared to be relevant were retrieved and independently assessed by two reviewers. Disagreements were resolved through a meeting. The initial search revealed 1,247 articles: 66 in CINAHL, 481 in Ovid-Medline, 626 in EMBASE, and 74 in PubMed. Following the PRISMA guidelines (Figure 1), we removed duplicates and screened the titles and abstracts, which narrowed the list down to 57 relevant articles. Two investigators reviewed these 57 articles, 37 of which conducted other interventions with apps, compared two app-functions, examined protocols, or had unclear outcomes. Thus, finally, 20 articles were included for this review.

Figure 1

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.

3. Data Collection and Analysis

From the 20 included articles, the following information was retrieved and analyzed: first author, year of publication, country, study design, themes, participants' character, sample size, mean age, intervention tool, follow-up duration, intervention characteristics, outcome measurements, as well as reported outcomes and significant levels. The search was broad with no limited target health-related behaviors in the search strategy. In this study, apps were considered effective if statistically significant results of health-related behavior changes were reported for them.

4. Risk of Bias Assessment

Risk of bias assessment for included studies was conducted by two authors using a modified version of the Scottish Intercollegiate Guideline Network (SIGN) checklist for RCT [7]. Specifically, the SIGN checklists were applied to grade the level of evidence of each study. The evaluation items are divided into 7 categories as follows: random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection bias), incomplete outcome data (attrition bias), selective reporting (reporting bias), and other bias. Each domain was classified as having a low, high, or unclear risk of bias. This assessment was conducted by two researchers, and disagreements were resolved by discussion.

III. Results

1. Characteristics of Included Studies

The RCTs included in this review were published between 2014 and 2017. Most (n = 16) were simply RCTs; the rest were an open-labelled RCT, an unmasked RCT, a cluster RCT, and a single-blinded parallel 3-arm pilot cluster RCT. The longest study duration was 8 months [8].

Five studies had a large number of participants [89101112]. The cluster RCT study had the largest sample size (n = 1,192) [11]. Two studies had a moderate number of participants [1314]. The other 13 studies had a small number of participants. All 20 studies had between 80% and 100% retention rates; 18 (90%) studies achieved high (80–100%) retention rates in the intervention group, and only two (10%) studies [814] had a moderate retention rate.

The 20 selected studies were analyzed in this systematic review, and the following 16 themes related to health behaviors were created: physical activity (4), alcoholism (1), dietary change (1) adherence of medication or therapy (2), preparation of clinical procedure (1), PTSD management (1), weight loss (2), prenatal education and engagement (1), adherence to follow-up clinic appointments (1), improvement of CPR skill performance (2), suicide prevention (1), prevention of CHD (1), smoking cessation (1), and knowledge improvement of pap testing (1). The comprehensive characteristics of included articles are summarized in Table 1.

Table 1

Summary characteristics of articles included

RCT: randomized controlled trial, DSM-IV: Diagnostic and Statistical Manual of Mental Disorders 4th edition, PTST: posttraumatic stress disorder, SBP: systolic blood pressure, VAS: Visual Analogue Scale, NDI: Neck Disability Index, MVFS: maximal voluntary flexion strength, MVES: maximal voluntary extension strength, CPR: cardiopulmonary resuscitation, ART: antiretroviral therapy, HIV: human immunodeficiency virus, CHD: Coronary heart disease, BMI: body mass index.

2. Risk of Bias of Selected Studies

The evaluation of risk of bias for all 20 studies was conducted using the SIGN checklist for RCTs. The results were summarized using the risk of bias table of RevMan 5.3 software. A total of 15 studies properly reported random sequence generation. Only one article did not mention random sequence generation [15].

For allocation concealment, only 6 studies explicitly mentioned that allocation was concealed [810161718]. However, 8 studies did not discuss allocation concealment adequately. Participants were blinded in 4 studies [16171920]. However, due to the traits of mHealth apps, some studies could not be conducted with perfect blinding. For the remaining 16 studies, either they were not blinded or information on blinding was not clearly provided in the reporting.

Among the studies in this review, three reported the blinding of outcome assessment [161719]. For reporting bias, 9 studies had a low risk of bias, and 11 were evaluated as unclear on the presence of bias. A summary of Cochrane's risk of bias table is presented in Figure 2.

Figure 2

Summary of risk of bias.

3. Content Characteristics of Apps

Some app characteristics of contents were categorized according to the behavior change technique taxonomy by Abraham and Michie [21] that is, providing information, planning (goal setting), reminding, providing feedback, or monitoring. Furthermore, additional app characteristics, such as entering data, education/training, and communication were derived from this study. Ten apps have multiple functions to manage health-related behaviors. The most common function of mHealth apps is providing the opportunity for education or training.

All participants in the control groups underwent standardized or usual care. Table 2 shows a summary of app characteristics.

Table 2

Summary characteristics of application contents

IV. Discussion

These days, mHealth apps seem to be ubiquitous, and the body of research indicating their effectiveness has been growing rapidly. However, evidence for the effectiveness of mHealth apps has been uncertain, and much remains unknown in terms of health-related behavior changes and clinical results.

In this study, 20 RCTs were included to evaluate the effectiveness of mHealth apps for health-related behavior change. Seventeen studies among 20 showed a positive contribution to the enhancement of health-related behaviors. This result is similar to other previous evidence reviews [3]. Therefore, using mHealth apps could be an effective strategy to improve outcomes of users along with the high popularity of smartphone use in the everyday lives of users. However, three studies did not show positive effects of mHealth apps on health-related behavior changes. Laing et al. [10] found that there was no difference in weight loss between intervention and control groups. In addition, Nord et al. [11] reported that a DVD-based CPR education group had better performance than an app group. Tighe et al. [20] also reported that using an app had no effect on decreasing depression symptoms and impulsivity behaviors. However, it is difficult to assess the effectiveness of apps based on the results of a single study, and more studies with controlled research design are needed.

This study differs from other reviews in that we only included RCTs to examine the effectiveness of mHealth apps, because RCTs are considered the ‘gold standard’ in evaluating the effects of intervention and provide a valuable source of evidence in research and treated as a powerful experimental tool to examine the effectiveness of intervention [22]. Therefore, this systematic review, which only analyzed the results of RCTs, has provided more reliable evidence for the effectiveness of smartphone health apps.

More than half of the reviewed studies had small samples (<60). In addition, 11 reported that the duration of intervention was less than 2 months. According to Man-Son-Hing et al. [23], trials with larger samples and longer intervention durations or follow-up times are more reliable to appraise the effectiveness of intervention. Based on the results of this review, to demonstrate a certain effect using mHealth apps for health-related behavior changes, more research with long intervention durations and large samples is needed.

According to Zhao et al. [3], the retention rate is defined as the proportion of participants who remain to complete a study. The Cochrane Handbook for Systematic Reviews of Interventions reported that studies with retention rates over 80% are classified as having low attrition, and studies with retention between 60% and 79% are classified as having moderate attrition [24]. Eighteen studies achieved high (>80%) retention rates in the intervention group. In this study, over half of the studies had a moderate-high retention rate. It can be assumed that the reasons for the high retention rate were the high feasibility and acceptability of app use in users' everyday lives. Thus, mHealth apps could be effectively adopted for users to improve health-related outcomes by managing and supporting health-related behaviors of users.

In this review, some studies [8101315161718192526] considered multiple function apps, such as entering data and providing feedback, education, and reminders. In contrast, other studies [191112142027282930] considered apps that have only one function; most of these apps had an education function. One study reported that having multiple app functions is much better to manage health status and to improve health-related behaviors [4]. However, this result might not be concrete because applying apps to the different types of situations, health behaviors, and participants can yield different results and effectiveness. For example, Laing et al. [10] conducted a study regarding an app with multiple functions for weight loss, and the effect was not significant. However, Zhang et al. [29] reported a significant result in improvement of coronary heart disease (CHD) knowledge and awareness using a single-function app. This suggests that many unconditional features of apps might not work properly, and a customized app is needed that fits the purpose and intent of the user. In this aspect, mHealth apps could provide individualized information via feedback to users and benefit them.

With respect to the risk of bias of included studies, the categories of selection bias (allocation concealment), performance bias (blinding of participants and researchers), and detection bias (blinding of outcome assessment) indicated high risk of bias. Therefore, to enhance the quality of studies and ensure low risk of bias, researchers should consider rigorous study design and reporting. Based on the results of this review, further studies using meta-analysis are needed to identify the effects of mHealth apps with specific outcomes. In addition, the effectiveness of the apps should be verified with the effects or risks being used with verification.

In this review, there were several limitations. First, we used broad key words, such as ‘health’, ‘behaviors’, ‘smartphone’, and ‘mobile’. For this reason, many articles related to the use of mHealth apps on specific diseases or health conditions—for instance, diabetes, hypertension, or asthma—might not have been included in this study. Second, in this review, only RCTs were included to analyze the effectiveness of mHealth apps. However, some of the RCTs did not fully follow the form of the RCT or applied a modified form of RCT. However, well-structured RCT is needed to verify mHealth app effectiveness. Third, almost all of the studies considered in this review were conducted in developed countries. Hence, it is difficult to generalize our results to developing countries. Fourth, we did not conduct a meta-analysis because of interventions with different kinds of mHealth apps. However, with the results of meta-analysis, the effectiveness of mHealth apps can be verified more clearly.

This systematic review was conducted to examine the effectiveness of mHealth apps to lead to changes in their targeted health-related behavior. This study summarized the characteristics and changes in targeted health outcomes. To our knowledge, there has been no previous systematic review of RCTs for identifying the effectiveness of mHealth apps in improving health-related behaviors. Similar to previous studies, this systematic review also found that the use of mHealth apps has a positive impact on health-related behaviors, such as physical activity, diet change, adherence to medication or therapy, and knowledge enhancement related to clinical procedures. Moreover, most apps seem to promote better clinical health outcomes. Most app users are satisfied with the use of mHealth apps to manage their health in comparison to users of conventional care. Although most studies analyzed indicated statistically significant effects to improve health, more RCTs with larger samples and longer applied interventions are still needed to confirm the effectiveness of mHealth apps. To assess the efficacy of mHealth apps in greater detail, further research is needed.