Leveraging Implementation Science to Understand Factors Influencing Sustained Use of Mental Health Apps: a Narrative Review.

Mental health (MH) smartphone applications (apps), which can aid in self-management of conditions such as depression and anxiety, have demonstrated dramatic growth over the past decade. However, their effectiveness and potential for sustained use remain uncertain. This narrative review leverages implementation science theory to explore factors influencing MH app uptake. The review is guided by the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework and discusses the role of the innovation, its recipients, context, and facilitation in influencing successful implementation of MH apps. The review highlights critical literature published between 2015 and 2020 with a focus on depression and anxiety apps. Sources were identified via PubMed, Google Scholar, and Twitter using a range of keywords pertaining to MH apps. Findings suggest that for apps to be successful, they must be advantageous over alternative tools, relatively easy to navigate, and aligned with users' needs, skills, and resources. Significantly more attention must be paid to the complex contexts in which MH app implementation is occurring in order to refine facilitation strategies. The evidence base is still uncertain regarding the effectiveness and usability of MH apps, and much can be learned from the apps we use daily; namely, simpler is better and plans to integrate full behavioral treatments into smartphone form may be misguided. Non-traditional funding mechanisms that are nimble, responsive, and encouraging of industry partnerships will be necessary to move the course of MH app development in the right direction. © This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2020.

Introduction

The past decade has witnessed an explosion of mental health (MH) smartphone applications (apps). More than 10,000 MH apps are available for download, offering features such as symptom and behavior tracking, diagnostic screening, psychoeducation, and relaxation and mindfulness exercises (Torous and Roberts 2017; Torous et al. 2018a). The number of app studies funded by the National Institutes of Health (NIH) has nearly doubled from 60 in 2014 to 112 in 2018 (Hansen and Scheier 2019), and MH app companies received over $400 million in venture capital investment in 2019 (Day 2019).

While MH apps are receiving considerable attention from consumers, researchers, industry, and investors alike, their potential for long-term success is unclear. Meta-analyses have found MH apps to be effective in reducing anxiety and depression symptoms, but effects were considerably smaller when using a non-MH app as a control as opposed to an inactive control condition (Firth et al. 2017a, 2017b). This finding calls into question the effectiveness of MH apps as compared to any app at all and may be representative of a “digital placebo effect” (Firth et al. 2017a). Observed treatment effects are also smaller in real-world app studies without the follow-up and incentives common in highly controlled clinical trials (Arean et al. 2016; Roepke et al. 2015; Van Ameringen et al. 2017). A meta-analysis of depression app trials reported a dropout rate of almost 50% (Torous et al. 2019), and a large analysis of real-world MH app use reported a median daily engagement rate of 4% (Baumel et al. 2019). Collectively, these findings highlight a striking mismatch between the degree of enthusiasm, effort, and capital being dedicated to MH app development as compared to the extent of our knowledge regarding apps’ effectiveness and their potential for sustained use. There is a critical need to increase our understanding of these issues to ensure that investments in this fast-growing industry are not made in vain.

The field of implementation science is well-positioned to examine these issues, despite being underutilized within digital health research to date (Buis 2019). Implementation science involves the “study of methods to promote the systematic uptake of research findings and other evidence-based practices” (Bauer et al. 2015; Eccles and Mittman 2006). This highly interdisciplinary field has developed multiple frameworks and models to guide study design and analysis (Nilsen 2020). The current review is informed by the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework, which outlines factors contributing to the successful implementation of an innovation (Harvey and Kitson 2015; Helfrich et al. 2010; Lynch et al. 2018). I-PARIHS utilizes the following formula to define successful implementation:

According to i-PARIHS, factors contributing to successful implementation include qualities of the innovation itself (“what” is being implemented, e.g., MH apps), characteristics of the recipients of the given innovation (“who” are the intended users, e.g., patients and providers), and context (“where” implementation is occurring, e.g., within a healthcare organization or directly to consumers). The i-PARIHS framework also highlights the importance of facilitation (“how” implementation is being enacted). Facilitation involves using tailored strategies developed with the unique qualities of a given innovation, its recipients, and context in mind to work towards successful implementation. Successful implementation of MH apps refers to the uptake and embedding of the innovation into practice; it can be measured by the number of patients who have used an app or providers who have recommended an app during a given time period, the frequency of app use, or evidence of sustained use following the end of a research trial (Hermes et al. 2019). In this review, we leverage the i-PARIHS framework to characterize the current state of MH app use and provide recommendations to improve implementation.

Methods

This narrative review is not meant to be exhaustive but to highlight timely and relevant MH app research (Grant and Booth 2009). Narrative reviews are well suited to examine broader research questions that may span disciplines and source types; they differ from systematic reviews in that the search strategy is not fully comprehensive and the selection criteria are less restrictive, allowing a wider breadth of sources to be represented (Cook et al. 1997; see Lemon et al. (2020) and Hidalgo-Mazzei et al. (2020) for examples of additional MH app narrative reviews). Given the speed of app innovation, articles published more than 5 years ago could be outdated; therefore, we searched for studies published between 2015 and 2020, with a preference towards those published in the last 2 years. We focused on depression and anxiety apps, given their prevalence, but drew from the literature examining other behavioral health apps when instructive.

Peer-reviewed papers and additional sources (newspaper articles, press releases) were identified by the first author using the following strategies:

PubMed search of the English language peer-reviewed literature published from January 1, 2015, through February 1, 2020, with the following keywords in the title and/or abstract: app, mobile application, or smartphone application AND depression or anxiety

Review of Google Scholar alerts received over the last 2 years based on the following keywords: “eHealth”, “mHealth”, “mental health technology,” and “psychology mobile apps”

Review of the reference lists of already selected papers

Weekly review over the last 2 years of the first author’s Twitter feed for publications posted via the accounts of approximately 100 digital health researchers, reporters, academic departments, journals, institutes, and special interest groups. Twitter has become a widely used platform for real-time dissemination and discussion of research (Haustein 2019). Twitter use is increasing among the research community, with one study finding that 89% of science, technology, and medicine publishers have official Twitter accounts (Zedda and Barbaro 2015). Given its ability to increase the impact of scholarly work on a global scale, Twitter activity is included in a publication’s Altmetric score, which measures its online reach; higher Altmetrics have been correlated with higher numbers of citations (Thelwall et al. 2013). As such, Twitter is an important platform for identifying timely and high-impact MH app research.

The first author determined article inclusion. To be included, sources had to include information relevant to behavioral health app implementation as defined by i-PARIHS domains (features of the innovation, recipients, context, facilitation, and/or measurement of successful implementation). Eighty-eight sources were included. Twenty-eight of the total sources were original research, 23 were meta-analyses or reviews, 24 were conceptual or opinion pieces, and 13 were non–peer-reviewed sources (e.g., press releases, newspaper articles, or app store webpages). Forty-five of the sources were published in 2019 or 2020, 38 were published between 2015 and 2018, and 5 sources were published from 2010 to 2014. The first author categorized included sources based on the abovementioned i-PARIHS domains. The co-authors then provided critical feedback on this categorization process during multiple meetings held throughout the writing and editing of the manuscript.

Results

Findings were organized into the following categories based on the i-PARIHS framework: features of the innovation, recipients, context, facilitation, and successful implementation. See Table 1 for a summary of key findings and recommendations.

Table 1

Summary of key findings and recommendations

i-PARIHS construct	Key findings and recommendations for stakeholders
Innovation
Relative advantage	Apps must demonstrate an added value over and above preexisting care options to be used. While MH apps have a clear advantage over no treatment, it is uncertain how much of an advantage they have over apps that are not MH-specific
Usability	Providers, healthcare system technology managers, and app developers should first determine whether the complexity of an app is needed at all to achieve a desired outcome, or whether a simpler strategy (e.g., text messaging) may be sufficient
	App developers should leverage successful usability strategies championed by industry, including simple, clean designs, gamification, and dynamic, tailored responses
Recipients
Values, beliefs, and motivation	Negative recipient attitudes are one of the greatest barriers to MH app uptake; their opinions must be considered during app design and roll-out
	Purported interest in apps and actual use may be very different. Concerns are raised by patients and providers regarding apps’ credibility and privacy; increased app transparency and education may help address these concerns
Skills, knowledge, and resources	Ownership of a smartphone is not a sufficient predictor of app use; a patient’s data and Wi-Fi capabilities must be considered by providers, healthcare system technology managers, and app developers to determine feasibility of app use
	There is a need for more education, training, and support for both patients and providers in identifying effective and evidence-based MH apps. Curated app libraries may be helpful resources, provided that they are frequently updated to accurately reflect the fast-moving app landscape
Context
External context	App stores are the central context in which individuals find and download MH apps. They are largely unregulated, with rankings based on the number of user ratings versus credibility. Stronger regulation is needed, and caution should be exercised when choosing an app, recognizing that app store ratings are poor indicators of effectiveness and usability
	Business cases are variable: for instance, apps can be marketed directly to consumers to be paid for out-of-pocket, offered free of charge by healthcare systems, or reimbursed by insurers, among other scenarios
Organizational and local contexts	Incorporating MH apps into healthcare systems requires leadership support and long-term commitment of time and funds to ensure sustainability
Facilitation	There is little research regarding MH app facilitation. Some work has shown positive effects of increasing providers’ use of MH apps via training
	Facilitation will look very different for fully user-guided MH apps and could involve direct-to-consumer advertisement, education, or tailored coaching messages
Successful implementation	Successful implementation can be measured by the degree of integration of MH apps into provider workflows and healthcare system budgets. Other metrics include apps having undergone multiple versions or updates or the number of downloads and interactions with MH apps by users
	Implementation has not been a core focus of MH app research to date. Funding mechanisms must adapt to prioritize this research, so that strategies to ensure uptake and long-term sustainability can be identified and improved. Collaboration with industry partners is much needed

Features of the Innovation

The i-PARIHS framework draws from a rich history of implementation science theory, including Rogers’ diffusion of innovations theory (Rogers 2003), to define domains relevant to consider when evaluating features of the innovation. We focus here on relative advantage and usability, given their particularly critical roles in influencing app uptake.

Relative Advantage

A key question when evaluating MH apps is whether they demonstrate an advantage over preexisting modes of care. There is a growing concern that researchers and developers may be creating apps without first identifying the need that the app is intended to meet (Furner et al. 2018; Greenhalgh et al. 2017; Lattie et al. 2020; Powell et al. 2020; Sim 2019). If patients, providers, or healthcare systems do not see the added value of an app, they will not use it. MH apps can serve as an adjunct to ongoing care with a therapist or can be downloaded independently from an app store by someone not currently receiving MH care. The relative advantage of apps can differ widely based on the circumstances. For example, a patient undergoing cognitive behavioral therapy (CBT) who is completing daily activity logs may view a CBT app as more organized and discreet as compared to paper and pencil; a further benefit could include interoperability between the app and the patient’s electronic health record (EHR), allowing for direct data sharing with providers (Stawarz et al. 2018). However, someone who downloads the same app and is not in ongoing treatment may not see the value of logging their activity without subsequent discussion with a therapist, decreasing its relative advantage. Indeed, tracking one’s activities or symptoms may remind the individual of the extent of their impairment (Hidalgo-Mazzei et al. 2018; Torous et al. 2018d); without providing strategies to improve functioning, this tracking may feel overwhelming and negative, leading to “digital unengagement” (Laing 2019). Conversely, a simple but well-designed app offering daily motivational quotes may be exactly what an individual with depression was looking for in an app store, while another patient in ongoing treatment may not see a need for this tool over and above the cognitive strategies they are learning during in-person sessions. Therefore, it is critical to identify the population an app is intended for and the need the app is intended to fill.

A potential advantage of MH apps is their ability to reach larger populations due to their public availability in app stores. MH apps may help level the playing field by offering access to care for those with economic, geographic, or stigma-related barriers to receiving in-person treatment (Bakker et al. 2016). Apps may also serve as an important resource for patients who are either waiting to initiate in-person care or have declined a referral to MH services (Hoffman et al. 2019). For MH apps to be successful in these circumstances, they must demonstrate an advantage over the alternatives of (1) receiving no care at all, (2) seeking guidance from the Internet or self-help books, or (3) using other apps to manage or distract from symptoms, such as gaming apps.

An important gauge of MH apps’ relative advantage is their performance in controlled trials. Apps have shown significant effects in decreasing depression and anxiety symptoms (Linardon et al. 2019; Weisel et al. 2019), for example in remote trials in which participants have no in-person contact with the research team (Arean et al. 2016; Moberg et al. 2019). However, as was mentioned earlier, a meta-analysis of depression apps found moderate effect sizes in comparison to inactive controls, but small effect sizes when compared to active, app-based control conditions (Firth et al. 2017a). While MH apps have a clear advantage over no treatment, it is uncertain how much of an advantage they have over smartphone apps in general. Further research regarding this potential digital placebo effect is needed to isolate the active ingredients contributing to MH app effectiveness (Firth et al. 2017a).

Usability

Usability, or the degree to which users can effectively, efficiently, and satisfactorily interact with MH apps, is critical to their success (Brooke 1996). Perhaps the first question to ask is whether the complexity of an app is needed at all. It has been argued that many app functions can be achieved through simple text messaging that does not require smartphone ownership, data availability, or the frequent updates inherent to app use (Willcox et al. 2019). Such text-based designs may be ideal for sending medication or appointment reminders. Similarly, users may benefit most from receiving education about how to use apps already available on their phone; for example, using the clock app to set reminders to exercise or keep to a regular bedtime.

One of the greatest purported advantages of MH apps—that they are easy to use and are carried with us everywhere—may, in fact, be one of their greatest stumbling points. Introducing MH care to the world of apps means that the often complex concepts of behavioral change will have to compete against the visually appealing, easy-to-use apps that command our attention throughout the day. Checking the weather, reading a news alert, and liking a photo on social media are tasks that take a matter of seconds, can be completed in rapid succession, and consume little cognitive load.

Indeed, commercial app developers have leveraged a host of design strategies to make apps “sticky,” ensuring that users have a seamless and positive experience that keeps them coming back for more (Furner et al. 2018; Hartmans 2018). These include (1) using push notifications (e.g., alerts that appear on the smartphone’s screen, even when not actively in use), (2) providing tailored content recommendations based on the user’s prior app interactions, and (3) utilizing a variable ratio schedule of reinforcement, in which the user cannot predict when they may receive a reward (e.g., reaching a new level in a game). Other apps track and reward consecutive days of use, such as the social media app Snapchat’s popular “snapstreak” feature (Snapstreaks-Snapchat Support (n.d.). While successful apps may be ostensibly simple on the front end, they are powered by a host of deliberate usability strategies on the back end that contribute to impressive rates of sustained use.

Despite the obvious success of these techniques in attracting and maintaining users, they are absent from many MH apps. MH apps often feature dense sections of psychoeducational text, multiple fields requiring users to type in their thoughts or daily activities, and a lack of dynamic features that adapt based on users’ responses. MH researchers are attempting to squeeze complex behavioral therapies into patients’ pockets without adapting them to an app context, and it is often not working (Greenhalgh et al. 2017; Mohr et al. 2018; Mohr et al. 2017b; Torous et al. 2018b). As is inevitable when shifting the mode of care delivery, strategies that were successful in previous contexts (e.g., in-person or Web-based protocols) will be carried over; however, these strategies may need to be reimagined to create a successful user experience in app form (Mohr et al. 2018).

As the field matures, there have been increasing calls to incorporate design thinking strategies and user experience testing into MH app development (Lemon et al. 2020). Apps should be simple, allowing users to engage with them in short bursts versus requiring long stretches of sustained attention (Zhang et al. 2019); indeed, MH apps designed for briefer interactions demonstrated higher engagement rates (Firth et al. 2017a). The most frequently downloaded MH apps involve discrete activities such as relaxation and mindfulness practices as opposed to full treatment protocols (Carlo et al. 2019). The use of appealing graphics is also critical (Vaghefi and Tulu 2019). When a variety of apps were introduced to a primary care population, the app that proved most popular involved swiping to uncover a nature photo; patients reported liking it because it was fun, more visually pleasing, and less cognitively demanding than the other choices (Hoffman et al. 2019). In another study of individuals with depression, the disorder-agnostic app SuperBetter, intended to increase resilience, optimism, and self-efficacy by completing challenges, was compared to a modified version with added CBT modules (Roepke et al. 2015). The CBT version was no more effective; it was hypothesized that this version may have been less visually appealing and that the CBT principles were too complex to be communicated effectively via an app. The Intellicare suite of apps Mohr et al. (2017b) are another example; they are described as “more consistent with a frequent, dynamic style of real-world interactions rather than a weekly, didactic model drawn from face-to-face practices.” The apps are designed to be simple and intuitive; instead of starting with psychoeducation, users learn about the app as they go, with didactic material relegated to the help section. Intellicare has demonstrated high numbers of app launches and participant retention following trial completion (Mohr et al. 2019, 2017b). The mean app session length was 1 min, and the median session length was 17 s (Mohr et al. 2017b), emphasizing that app use patterns are categorically different than use of other modes of MH treatment.

Gamification may also improve app usage. As Bakker et al. (2016) review, gamification does not mean turning an app into a game, but rather to use “game-based mechanics, aesthetics, and game thinking to engage people, motivate action, promote learning, and solve problems” (e.g., SuperBetter, Roepke et al. 2015). This could take the form of use streaks or challenges to increase engagement (Torous et al. 2018b). Gamification may be an important strategy in treating depression, given that low motivation is a hallmark symptom. Patel et al. (2019) showed positive effects of a gamified app in increasing health behaviors; they drew from behavioral economics principles including precommitment to goals, loss aversion (e.g., not wanting to lose previously achieved gains), and the fresh start effect (e.g., giving users a clean slate after lapsing). The authors write, “most…digital health applications have not appropriately leveraged principles from theories of health behavior, which could be a major reason why recent evaluations suggest there has been little effect from them on health behaviors.”

Tailoring and adapting content based on patterns of app use and individual characteristics can help ensure that apps have value for a given user (Baumel and Kane 2018; Stawarz et al. 2018). For instance, while many apps feature meditation strategies, they are not designed to direct a user towards these strategies upon report of increased anxiety (Bakker et al. 2016). Similarly, crisis line information could appear directly upon report of suicide ideation. Push notifications may encourage sustained use of MH apps, although it is important to determine the ideal number and nature of notifications based on users’ unique app interaction characteristics, to prevent them from becoming annoyed and discontinuing use altogether (Bakker et al. 2016). Some work has suggested beneficial effects of personalized coaching via text messages or phone calls in increasing app use and decreasing anxiety and depression symptoms (Mohr et al. 2019). However, there is concern that the addition of coaching components will hurt app scalability. Alternatively, algorithm-based recommendations do not require human involvement but can provide users with suggestions tailored to their clinical presentation and usage patterns and may lead to increased app use (Cheung et al. 2018).

It is important to also assess usability from the provider perspective, as this will influence whether they incorporate apps into their clinical care (Gordon et al. 2020). For sustained uptake to occur in this setting, accessing patients’ app data must be simple and streamlined. For example, the Apple Health app has connectivity to certain EHR systems, which could conceivably make it easy for providers to view app activity from within the patient’s record (Sim 2019). Apps need to aggregate potentially large amounts of patient data into a quickly interpretable form for this information to be of any use to providers. If the process of accessing patient app information is not seamlessly integrated into providers’ hectic workflows, it will be difficult to see buy-in at this level (Gordon et al. 2020; Jacob et al. 2020; Lemon et al. 2020).

Recipients

The recipients of an innovation are central in determining successful implementation. Negative recipient attitudes are one of the greatest barriers to MH app implementation; soliciting recipient opinions, for example through participatory or user-centered design approaches, should therefore be a critical component of app development (Vis et al. 2018). Recipients can include individuals seeking MH apps outside of a healthcare setting, patients within a healthcare system, and providers recommending apps to their patients. We discuss recipient characteristics within two domains that strongly influence app uptake: values, beliefs, and motivation, and skills, knowledge, and resources.

Values, Beliefs, and Motivation

In multiple survey-based studies, the majority of patients with MH diagnoses reported interest in MH apps; however, rates of actual use are considerably lower than reported interest (Beard et al. 2019; Lipschitz et al. 2019, 2020; Torous et al. 2014). Furthermore, the percentage of people with MH needs reporting having a health app on their phone was much higher than the percentage who reported any use of those apps in the past week (Schueller et al. 2018). Such studies highlight the difference between an individual’s hypothetical interest in an app and the likelihood that they will actually use it. Patients may find apps less desirable than in-person care, self-help books, and Web-based information; concerns have been raised regarding apps’ credibility, efficacy, and data privacy (Carlo et al. 2019; Musiat et al. 2014). User interest in apps may also be limited to certain basic functions, such as appointment reminders, mood tracking, and mindfulness activities as opposed to full evidence-based therapies or more intrusive monitoring of GPS location or text messaging behaviors (Carlo et al. 2019; Torous et al. 2018a).

Some work has found no demographic differences in MH app interest, suggesting that apps may reach populations who are less likely to seek out MH care, including racial minorities and men (Lipschitz et al. 2020). However, additional work has reported greater interest among those with more education (Beard et al. 2019), as well as heightened skepticism among rural adults, who were more likely to describe technology-based care as being impersonal (Connolly et al. 2018). Educating patients about MH app features, data security, and intended use may help to improve attitudes (Ebert et al. 2015). An additional predictor of successful app uptake is the individual’s degree of motivation; someone with a strong drive to improve their functioning may be more likely to stick with a treatment protocol. This level of motivation may be difficult to sustain, particularly among those with symptoms of depression and those without the added support and accountability provided by a therapist (Hoffman et al. 2019).

Providers’ values, beliefs, and motivation are also critical to consider; providers must believe that an app will significantly benefit their patient for them to recommend one (Jacob et al. 2020). Providers may feel that apps can improve patient self-management and facilitate communication; however, they may also think that apps are impersonal, contain unreliable information, or do not have proper privacy regulations (Jacob et al. 2020; Lattie et al. 2020; Lemon et al. 2020). This skepticism may understandably impact providers’ motivation to recommend apps.

Skills, Knowledge, and Resources

MH apps serve no purpose if users do not own a smartphone or do not know how to download or use an app. Smartphone ownership is on the rise across individuals of all ages, races, and economic classes (Pew Research Group 2018). However, older adults may feel less comfortable navigating smartphones and may require more assistance using apps (Connolly et al. 2018). Importantly, ownership of a smartphone is not a sufficient predictor of successful app use. For those with limited data plans, the decision to download an app may mean sacrificing other functionalities on their phone and can therefore be a considerable barrier to use; lack of Wi-Fi access and service disruptions are additional obstacles (Lattie et al. 2020).

Beyond these more structural barriers to use lies a lack of knowledge regarding app effectiveness. Users generally learn about MH apps through Web or app store searches, social media, or word of mouth versus from a healthcare provider (Lipschitz et al. 2019; Schueller et al. 2018). This is concerning, as most MH apps in app stores have little to no empirical backing. Approximately 3% of available depression and anxiety apps provide information regarding testing or effectiveness (Marshall et al. 2019; Sucala et al. 2017); when restricted to the 29 highest-rated depression apps, this rate only increased to 7% (Qu et al. 2020). This has led to the development of multiple online libraries intending to curate effective MH apps (Marshall et al. 2019). However, these libraries may have trouble keeping up with apps’ unstable lifespans. Apps are constantly being added or removed from app stores, with one study finding that a depression app is deleted from the app store every 2.9 days (Larsen et al. 2019). This is further complicated by the need for regular updates to maintain compatibility with smartphone operating systems; without steady maintenance by app developers, a MH app can quickly become defunct.

There is a huge need for provider training in MH apps, with little if any instruction currently occurring within degree-granting programs or continuing education efforts (Gordon et al. 2020; Gratzer and Goldbloom 2020; Hilty et al. 2019; Lattie et al. 2020). Indeed, providers describe needing better education regarding effective MH apps, their functionalities, and how to introduce them to patients in a timely manner (Jacob et al. 2020); lack of familiarity with apps and lack of time to discuss them were noted as key barriers to recommending apps (Hoffman et al. 2019). There have been recent efforts to develop frameworks of clinician competencies and app evaluation criteria to help address this problem (Hilty et al. 2020; Torous et al. 2018c). However, to reduce provider burden, some have suggested the need for technology specialists or digital navigators tasked with introducing apps to patients (Ben-Zeev et al. 2015; Noel et al. 2019; Wisiewski and Torous 2020). Research is needed regarding how successful such arrangements are in practice.

Context

The context in which MH apps are being implemented is also critical to their success. We begin at the external level with a discussion of app stores and financial incentives, followed by a review of organizational and local-level contexts.

External Context

MH apps are unique from other treatments in that, regardless of whether a user finds an app independently or is counseled to use one by a provider, all publicly available apps are downloaded from centralized app stores (the App Store for iPhone users and Google Play for Android users). As such, app stores are a critical context in which users interact with MH apps. This is complicated by the fact that many researcher-developed apps have not been made publicly available within app stores (Hidalgo-Mazzei et al. 2020; Lattie et al. 2016); of the depression apps with identified developers, only 10% were medical centers, universities, or institutions (Shen et al. 2015).

The number of user ratings reigns supreme in determining the order in which apps are presented to users and likelihood of download (Optimizing for app store search n.d.; Walz 2015). However, consumer ratings of health apps are poor indicators of clinical effectiveness and usability, calling into question the value of app ratings altogether (BinDhim et al. 2015; Gordon et al. 2020; Singh et al. 2016). For example, 72% of the 29 most popular depression apps contained no information regarding suicide prevention, and some of these apps contained potentially harmful negative content (Qu et al. 2020). An additional study of top-ranked depression and smoking cessation apps found that while 92% of apps transmitted user data to third parties, the majority did not clearly disclose this information to users (Huckvale et al. 2019). Such findings have led to a growing call for app stores to better regulate MH apps. This could involve collaborating with health agencies to (1) adopt stricter screening and acceptance policies and (2) require clear labeling of an app’s developers, evidence base, indicated use, and privacy protections (Gordon et al. 2020; Hidalgo-Mazzei et al. 2020; Martinengo et al. 2019; Stawarz et al. 2018). In addition to benefiting the consumer, these changes may also benefit Apple and Google; modifying algorithms to elevate empirically supported MH apps to the front of the queue may lead to more downloads, as these apps may be more effective (Marshall et al. 2019).

The current lack of app store regulation is striking, and the market is in sore need of oversight to ensure that users gain access to effective and appropriate MH apps. Precedent exists in other spheres; in 2010, Google decided to display suicide crisis line information as the first result if a user types suicide-related content into their search engine (Cohen 2010), and a Google search for the term “depression” returns diagnostic information from the Mayo Clinic and NIH as its first hits. In an unprecedented regulatory response to the COVID-19 pandemic, the Apple app store limited distribution of COVID-19 apps to those created by recognized governmental or healthcare entities, demonstrating that such actions are indeed possible (Sherr 2020). App stores could similarly highlight critical MH contact information and empirically supported apps in response to common MH search terms.

Financial incentives and reimbursement are also important contextual factors to consider (Powell et al. 2020). Many MH apps aim to profit directly from consumers; some are offered free in the app store with the option to pay for additional features (i.e., freemium apps), while others require payment for initial download (Powell et al. 2019). In other cases, MH apps are offered free of charge with the intention of having broader public health impacts (e.g., Department of Veterans Affairs (VA) apps, Gould et al. 2019; Owen et al. 2018). A currently less common route aims for apps to be prescribed by psychiatrists and reimbursed by insurers (Gordon et al. 2020; Sim 2019). The FDA has developed a precertification regulatory process to facilitate this (Digital Health Software Precertification (Pre-Cert) Program 2019; Gordon et al. 2020), and Pear Therapeutics was the first company to obtain approval via this method for two substance use disorder apps (Pear Therapeutics 2017, 2018). App companies Pear Therapeutics, Akili, and Click have partnered with pharmaceutical companies in attempts to take this reimbursement route. However, the dissolution of the partnership between Pear and pharmaceutical company Sandoz gives pause regarding the viability of this pathway (Cairns 2019). The business case for MH apps can vary greatly across circumstances; implementation strategies, including marketing and advertisement, must be tailored accordingly to promote uptake among the appropriate stakeholders (e.g., non-patients, patients, providers, healthcare systems, pharmaceutical companies, or insurers, among others).

Organizational and Local Contexts

There is increasing interest in incorporating MH apps into patient care within larger healthcare systems. It is critical to understand the organizational and local context prior to implementing apps in these settings (Lattie et al. 2020). Implementation can be influenced by the opinions of healthcare leadership towards MH technologies and their willingness to prioritize the time and funds necessary; this can be further complicated by changes in leadership, staff turnover, and limitations of the system’s technology infrastructure (Mohr et al. 2017a).

Commitment to sustained MH app use within a healthcare system requires investment in time and staffing to (1) train staff and patients, (2) provide technical support, and (3) frequently monitor apps to ensure they remain updated and, in some cases, compatible with EHR systems (Hermes et al. 2019; Hidalgo-Mazzei et al. 2020; Sim 2019). Some healthcare systems have developed on-site tech “bars” where patients can try out apps and devices with guidance from staff (Gordon et al. 2020). VA has created a national Office of Connected Care in which funds are allocated towards staffing the office, providing technical support, and developing, maintaining, and advertising apps, among other technologies (Connected Care n.d.).

Greenhalgh et al. (2017) reviewed digital health implementation efforts within healthcare systems, highlighting the critical role of organizational and local context in determining success. Barriers to uptake included (1) inability to negotiate an acceptable business or reimbursement model; (2) lack of leadership interest; (3) inadequate resources to implement the innovation, adapt over time, and sustain use; and (4) lack of flexibility in adapting to new workflows. It is important to note that the degree of effort and cost to the healthcare system will vary depending on the app being implemented. A self-management app that does not require provider input will require more direct-to-consumer advertising but less provider training and effort, while a provider-intensive app would require the opposite (Hermes et al. 2019).

Facilitation

The literature is sparse regarding facilitation of MH app use, which could indicate (1) hesitation regarding apps’ readiness for implementation on a larger scale and/or (2) a lack of focus on implementation within funded MH app grants. One study conducted in an integrated primary care behavioral health setting involved development of a MH app toolkit that was introduced to providers via a series of meetings (Hoffman et al. 2019); feedback following this 2-month pilot study indicated that providers wanted more training to feel confident recommending apps to their patients.

The joint VA/Department of Defense (DoD) Practice-Based Implementation (PBI) Network Tech into Care pilot study is an example of an interorganizational effort to promote use of their collection of self-developed, publicly accessible MH apps (Creason et al. 2019; Pratt et al. 2019). This project utilized the i-PARIHS framework to develop an implementation and facilitation plan. Facilitators trained MH providers in app use and followed up with consultation calls, reminders, and feedback on progress after training completion. While most providers viewed the training positively and reported increased app use with their patients, there were multiple organization-level barriers, including lack of time, resources, and leadership support at certain sites. This Tech into Care initiative continues to grow, offering national app webinars and newsletters, as well as an ongoing nationwide study in which facilitators provide on-site MH app trainings followed by 3 months of remote facilitation support. The study is training a wide range of staff including primary care providers, peer support specialists, and chaplains.

It is also important to consider what facilitation might entail for fully user-guided MH apps that can be downloaded independent of any contact with a healthcare system and therefore do not require provider training. It will be critical to effectively disseminate educational information to consumers regarding the value of evidence-based treatments and ways to find them (Powell et al. 2015). Facilitation in these contexts may primarily take the form of direct-to-consumer advertisement and other strategies to increase app store ranking and visibility. These strategies appear to be much better mastered by industry versus academia, making the case for collaboration and knowledge sharing; the first ten apps displayed after searching “depression” in the App Store all have industry developers. The use of personalized coaching, including tailored supportive messages and recommendations of helpful app features, could also help to facilitate engagement; as mentioned earlier, this approach has been found to increase app use and improve outcomes among users with depression and anxiety (Cheung et al. 2018; Mohr et al. 2019). Further research is needed to determine how best to facilitate use via these more personalized and adaptive strategies.

Successful Implementation

As has been demonstrated throughout this review, the measurement of successful MH app implementation will vary based on unique features of the app itself, its recipients, and its context. While perhaps the least studied aspect of MH app development, there has been a growing body of work examining the sustainability of app use, which is a key indicator of implementation success (Hermes et al. 2019; Mohr et al. 2017a, 2018). Sustained use can be reflected in multiple ways, such as (1) apps being fully integrated into the EHR and provider workflows, (2) a steady budget for app advertising and maintenance, or (3) apps having undergone multiple updates or versions (Hermes et al. 2019; PTSD Coach n.d.). The number of app downloads and interactions over time may also capture sustained uptake. However, operationalizing meaningful app use is a challenge, as many downloaded apps are never used (Lattie et al. 2016; Owen et al. 2015), and multiple studies have found no relationship between the amount of time spent on an app and clinical outcomes (Kuhn et al. 2017; Moberg et al. 2019; Zhang et al. 2019). Additional research is needed to determine whether the number of log-ins, clicks, or modules completed may be more appropriate metrics of sustained use (Bakker et al. 2016; Donkin et al. 2011; Hermes et al. 2019; Mohr et al. 2019).

In order to identify factors contributing to successful implementation, funding mechanisms must adapt to meet the unique needs of MH app studies (Glasgow et al. 2014; Pham et al. 2016; Sim 2019). Grants take years from initial conception to publication of findings, and traditional RCTs require the innovation to remain static for the duration of the study period. These realities exist in stark opposition to the nature of apps, which must frequently change to retain users’ attention and remain compatible with smartphone technologies. Funding mechanisms must be nimble, dynamic, adaptive, and supportive of frequent iteration to ensure that the product being implemented is actually usable (Glasgow et al. 2014; Mohr et al. 2017a; Tønning et al. 2019). Grants that partner with industry, such as NIH Small Business Innovation Research grants, are critical in acknowledging that MH researchers will likely not also be experts in app design; such mechanisms can allow the strengths of both academia and industry to shine through (Hidalgo-Mazzei et al. 2020). Key stakeholders, including patients and providers, must be involved in app development to ensure that the product is meeting their needs (Brewer et al. 2020; Mohr et al. 2017a), and MH app grants should not be funded without including an implementation aim. Currently, a small minority of funded app studies have resulted in publicly available apps. This is a major implementation shortcoming and suggests that deliberate strategies are needed to support uptake of these innovations. Funding agencies should reflect carefully before allocating limited research dollars to the development of apps that will only gather virtual dust on institution shelves.

Discussion

The current narrative review utilized the i-PARIHS framework to highlight key factors influencing MH app uptake: features of the apps, their recipients, and the contexts in which they are being implemented, as well as the role of facilitation processes. Namely, for apps to be successful, they must be (1) advantageous over alternative tools, (2) relatively easy to navigate, and (3) aligned with users’ needs, skills, and resources. Significantly more attention must be paid to the complex contexts in which MH app implementation is occurring in order to refine facilitation strategies. Considerations can range from app store ranking algorithms to the openness of patients, providers, and healthcare leadership to new technologies, and to the financial priorities of pharmaceutical and insurance companies. As the field of MH apps continues to mature, there will ideally be a widening of the lens of inquiry from individual studies of app effectiveness to broader examinations of factors influencing successful app implementation, as well as a clearer identification of metrics demonstrating sustained app use; indeed, important work in these domains has already begun (Glasgow et al. 2014; Gordon et al. 2020; Hermes et al. 2019; Hoffman et al. 2019; Jacob et al. 2020; Lattie et al. 2020; Mohr et al. 2018; Vis et al. 2018). In turn, there will be a need for subsequent timely reviews that aim to aggregate this quickly growing wealth of information.

Strengths of this review include its synthesis of findings from a diverse range of sources and its leveraging of implementation science, a largely underutilized field with a wealth of research ready to benefit MH app development. There are several limitations worth noting. The current review situated findings within the i-PARIHS framework, given its consideration of multiple factors influencing successful implementation and its prominence within the field of implementation science (Helfrich et al. 2010; Lynch et al. 2018). However, it must be acknowledged that there are numerous other technology-specific frameworks that may have value when examining app implementation, including the unified theory of acceptance and use of technology (UTAUT; Venkatesh et al. 2003), the technology acceptance model (Davis 1989), and the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework (Greenhalgh et al. 2017). Furthermore, our search strategy favored findings from the peer-reviewed academic literature and therefore may have underrepresented lessons learned from industry, emphasizing the need for strong academic-industry partnerships for maximal knowledge sharing.

While our use of narrative review methodology allowed us to include a broader and more diverse range of sources, there are some limitations inherent to this strategy that are important to note. Given that narrative reviews do not involve a fully systematic and comprehensive search strategy, it is possible that additional relevant articles were not included in this work. While we drew from PubMed, Google Scholar, and Twitter, there are additional research databases that may have contained relevant articles but were not leveraged, such as CINAHL and Embase. However, as our review included 88 sources, 45 of which were published since 2019, we are confident that we have captured a meaningful portion of the relevant literature. Furthermore, narrative reviews do not involve the stringent selection and appraisal criteria characteristic of systematic reviews. This invites the possibility of bias with regard to the sources that the first author selected for inclusion within the review. However, all co-authors approved of the first author’s inclusion criteria and selection process during a series of meetings and multiple in-depth rounds of manuscript writing, editing, and review.

In sum, the evidence base is still uncertain regarding the effectiveness and usability of MH apps, and much can be learned from the function and design of the myriad successful apps we use daily; namely, simpler is better and plans to integrate full behavioral treatments into smartphone form may be misguided. Without a solid foundation, efforts to implement MH apps, whether directly to consumers or within healthcare systems, will suffer. Non-traditional funding mechanisms that are nimble, responsive, and encouraging of industry partnerships will be necessary to move the course of MH app development in the right direction. As the initial hype surrounding MH apps settles, there is an urgent need for reflection and humility regarding the current state of the field in order to develop strategies that are realistic, grounded in evidence, cognizant of context, and more likely to result in successful implementation outcomes.