Uptake and Usage of IntelliCare: A Publicly Available Suite of Mental Health and Well-Being Apps.

BACKGROUND: Treatments for depression and anxiety have several behavioral and psychological targets and rely on varied strategies. Digital mental health treatments often employ feature-rich approaches addressing several targets and strategies. These treatments, often optimized for desktop computer use, are at odds with the ways people use smartphone applications. Smartphone use tends to focus on singular functions with easy navigation to desired tools. The IntelliCare suite of apps was developed to address the discrepancy between need for diverse behavioral strategies and constraints imposed by typical app use. Each app focuses on one strategy for a limited subset of clinical aims all pertinent to depression and anxiety. This study presents the uptake and usage of apps from the IntelliCare suite following an open deployment on a large app marketplace.
METHODS: Thirteen lightweight apps, including 12 interactive apps and one Hub app that coordinates use across those interactive apps, were developed and made free to download on the Google Play store. De-identified app usage data from the first year of IntelliCare suite deployment were analyzed for this study.
RESULTS: In the first year of public availability, 5,210 individuals downloaded one or more of the IntelliCare apps, for a total of 10,131 downloads. Nearly a third of these individuals (31.8%) downloaded more than one of these apps. The modal number of launches for each of the apps was 1, however the mean number of app launches per app ranged from 3.10 to 16.98, reflecting considerable variability in the use of each app.
CONCLUSIONS: The use rate of the IntelliCare suite of apps is higher than public deployments of other comparable digital resources. Our findings suggest that people will use multiple apps and provides support for the concept of app suites as a useful strategy for providing diverse behavioral strategies.

Introduction

Despite effective treatments, depression and anxiety continue to be highly prevalent mental health issues, with a one-year prevalence rate of major depression estimated at 6.6% (Kessler et al., 2003) and a one-year prevalence rate of anxiety disorders estimated at 18% (Kessler et al., 2005) of the general population. Even more Americans report subclinical symptoms of depression and anxiety that impact their quality of life. Unfortunately, only a third of those in need actually receive services (Kessler et al., 2005), a problem that is in part due to substantial barriers to receiving traditionally delivered face-to-face mental health services (Mohr et al., 2010).

The development and utilization of behavioral intervention technologies (BITs), such as mobile apps, offers the potential to greatly expand the portfolio of available mental health resources (Kazdin and Blase, 2011). While not intended to replace face-to-face therapies, these modern adaptations have the potential to address the overwhelming need for and barriers to traditional services. Technologies delivered independent of hands-on clinician support may be particularly valuable if the public can appropriately utilize these technologies. As Muñoz (2010) noted, these types of interventions are non-consumable resources in that they can benefit a broad array of individuals without requiring additional therapeutic power. In order to substantially expand the treatment portfolio and serve the greatest number of people, further research must be done on Massive Open Online Interventions (MOOIs; Muñoz et al., 2015) or interventions that are free for anyone in the world to use.

Meta-analytic reviews have demonstrated the efficacy of web-based computer treatment programs for anxiety and depression, which are delivered in a manner that reduces many known barriers to traditionally-delivered, face to face services (Andrews et al., 2010, Richards and Richardson, 2012). Substantially less is known about the use of mobile apps for the treatment of anxiety and depression (Torous & Powell, 2015). The use of smartphones is rapidly increasing around the world with nearly two-thirds of Americans using smartphones in early 2015, up from just 35% in early 2011 (Smith, 2015). Furthermore, a growing number of Americans are “smartphone-dependent,” such that they rely on their smartphones for accessing online services because they either lack broadband Internet connections at home, or have limited options for Internet access apart from their mobile phones (Smith, 2015). People use their phones for a variety of functions including supporting their health. A recent national survey indicated that more than half of mobile phone users (58.23%) have downloaded at least one health-related mobile app (Krebs and Duncan, 2015). Mental health apps in particular appear to be of high interest among psychiatric outpatient populations (Torous et al., 2014).

Given the increasing acceptance of and capabilities for the delivery of mental health and wellness through mobile apps, many apps have been created to serve this purpose (IMS Institute for Healthcare Informatics, 2015, Torous and Powell, 2015). These apps have the potential to be effective but often lack scientific evidence about their efficacy (Donker et al., 2013). On the other hand, many apps that are presented in scientific journals are not publically available. Apps are generally downloaded directly by consumers through public marketplaces (e.g., Apple's App Store or the Google Play Store). Failure to account for this in empirical investigations decreases the generalizability of findings to likely end users.

In the service of app evaluation, we argue that the evaluation of uptake and usage is key to our understanding of public engagement especially using methods reflecting the traditional ways people find apps. As such it is critical to explore uptake and usage reflecting apps available in public app marketplaces. Muñoz et al. (2015) make a similar argument, noting that the relevant metrics for MOOIs are use, cost, and efficacy. Therefore, uptake and use are necessary requirements before exploring effectiveness. Furthermore, use is not a given for healthcare apps. As illustrated by Helander et al. (2014), many individuals who download a commercially developed health management app never use the app. Only 13.6% people who downloaded the app in Helander's study used the core functionality (taking a picture) more than once. Even the largest group of users deemed “semi-active” (11% of total users) only used the app for an average of 9.3 days. These brief interactions suggest that most people discontinue use prior to likely having received any clinically meaningful benefit.

A key question regarding design and engagement, however, is what behavior change principles an app should employ. Years of research have identified a multitude of behavioral and psychological components (including activity, cognition, and emotion regulation) that contribute to depression and anxiety and many effective strategies for treatment and prevention (e.g. activity tracking, cognitive restructuring, seeking social support). Chorpita et al. (2005) have described these diverse strategies found in evidence-based treatments as practice elements. In light of the multiple barriers for dissemination of complete evidence-based treatment protocols, they have advocated for distillation to specific practice elements and better examination of what works, what is used, and how this might vary among people and contexts (Chorpita et al., 2007). The desired integration of multiple strategies, or practice elements, into BITs has led to the development of feature-rich applications (Titov et al., 2011, van Straten et al., 2008, Whitton et al., 2015) designed mainly for use on a computer. While responsive websites can be accessed via multiple devices, including smartphones, the feature-rich nature of these sites is often counter to how individuals use apps. Typically, popular apps serve singular purposes, such as searching for restaurants/businesses, managing flights, or posting pictures. This creates a problem for app design for mental health. People respond to different components of treatment and are therefore likely to benefit from exposure to multiple practice elements, but they are accustomed to using apps that focus on singular functions with easy navigation to desired tools.

Based on methods shown to be efficacious at improving symptoms of depression and anxiety, a suite of mobile phone apps was developed by researchers at Northwestern University's Center for Behavioral Intervention Technologies (CBITs). These apps feature different methods of managing mental health and wellness, including practice elements from cognitive-behavioral therapy, positive psychology, and physical activity-based interventions. They feature a variety of types of user-app interactions, such as recording/logging, completing checklists, reviewing personal data, and reading didactic content. Each app focuses on one treatment strategy for a limited subset of clinical aims all pertinent to depression and anxiety. Once individuals have downloaded an app they like, they tend to visit the apps frequently in very shorts bursts of time (Oulasvirta et al., 2005, Vaish et al., 2014). Therefore the 12 interactive apps within the IntelliCare suite were designed to be used in this brief, but frequent, manner. While interest in mental health apps appears to be growing, most people do not know which apps could be helpful to them. When given directions, the majority of people (72%) follow through with downloading recommended apps (IMS Institute for Healthcare Informatics, 2015). Consistent with this knowledge, the IntelliCare Suite is composed of interactive apps, which are managed through a Hub app that coordinates user experience with the application suite. The ultimate goal for the Hub app is to harness data, including use data, user preference ratings, and periodic symptom assessments, to create an underlying analytic model that makes recommendations for further app use. Current deployments are aimed at developing the database for that recommender system. The lightweight, easy to use nature of these streamlined, yet integrated, apps may increase the accessibility and willingness to engage with different methods for mood management.

The aim of this initial in-the-wild deployment was to evaluate naturalistically the feasibility of a suite of behavior change apps. Feasibility and early testing commonly focuses on use of specific tools (Klasnja et al., 2011, Muñoz et al., 2015). This study examines the initial uptake and patterns of use of the IntelliCare suite of apps disseminated through the Google Play store. Specifically, use patterns are examined individually for each of the apps and for the apps as a group. Because the Hub app is a novel method of managing multiple apps, app use was compared between those who downloaded the Hub app, and those who did not.

Methods

Design and development

Each of the 12 apps within the IntelliCare suite was designed by a multidisciplinary team using the BIT Model (Mohr et al., 2014). The clinical aims were to reduce symptoms of depression and anxiety. Two clinical psychologists (DCM and SMS) defined a list of behavioral strategies with demonstrated efficacy. Brief design pitches were created by members of our multidisciplinary group consisting of mental health specialists and technologists. These design pitches included specifying the behavioral strategy, a patient's perspective of that strategy, the “app experience” that defined the basic app elements and workflow from the user's perspective, potential problems with the design, methods of identifying potential problems in intended functionality, and ease of use. Pitches were improved via an iterative process in which they were reviewed by a multidisciplinary team, feedback was provided, and a multidisciplinary team was created (consisting of one mental health specialist and one technologist) to revise the pitch and resubmit it. This continued until the pitch was accepted or rejected. Pitches were evaluated on fidelity to evidence-based behavior change strategies, diversity from other pitches, and adequate coverage of topics from the pre-defined list. Accepted pitches were forwarded for development, and then subjected to internal quality assurance testing.

Following deployment on the Google Play store, users were invited to provide feedback and use data were monitored to identify problems and make improvements in functionality, ease of use, and aesthetics. Bug fixes were made to address technical problems and updates to the Android operating systems throughout deployment. Table 1 provides descriptions of each app within the IntelliCare suite, and Fig. 1 provides a closer look at one of the clinical apps, Thought Challenger.

Table 1

Descriptions of IntelliCare Apps.

IntelliCare Hub	Manages messages and notifications from the other apps within the IntelliCare collection.
Aspire	Guides user to identify the values that guide one's life and the actions (or “paths”) that one does to live that value. Helps keep track of those actions throughout the day and supports the user in living a more purpose-driven and satisfying life.
Day to Day	Delivers a daily stream of tips, tricks, and other information throughout the day to boost the user's mood. Prompts the user to work on a particular theme each day, and every week; learn more about how to effectively cultivate gratitude, activate pleasure, increase connectedness, solve problems, and challenge one's thinking.
Daily Feats	Encourages the user to incorporate worthwhile and productive activities into the day. Users add accomplishments to the Feats calendar, where they can track their positive activity streaks and level up by completing more tasks. Helps motivate users to spend their days in more meaningful, rewarding ways to increase overall satisfaction in life.
Worry Knot	Teaches the user to manage worry with lessons, distractions and a worry management tool. Provides a guided tool to address specific problems that a user can't stop thinking about, and provides written text about how to cope with “tangled thinking.” Presents statistics about progress as the user practices coping with worry, gives daily tips and tricks about managing worry, and provides customizable suggestions for ways to distract oneself.
ME Locate	Provides a personal map for finding and saving user's mood-boosting locations. Assists the user in finding and remembering these places to help them make plans, maintain a positive mood, and stay on top of responsibilities.
Social Force	Prompts the user to identify supportive people in their lives, and provides encouragement for the user to get back in touch with those positive people.
My Mantra	Prompts the user to create mantras (or repeatable phrases that highlight personal strengths and values and can motivate one to do and feel good) and construct virtual photo albums to serve as encouragement and reminders of these mantras.
Thought Challenger	Guides the user through an interactive cognitive restructuring tool to examine thoughts that might exaggerate negative experiences, lead one to be overcritical and bring down one's mood. Teaches the user to get into the habit of changing perspective and moving toward a more balanced outlook on life.
iCope	Allows the user to send oneself inspirational messages and reassuring statements, written in their own words, to help the user get through tough spots or challenging situations.
Purple Chill	Provides users with a library of audio recordings to relax and unwind. Teaches a variety of relaxation and mindfulness practices to de-stress and worry less.
MoveMe	Helps the user select exercises to improve mood. Provides access to curated exercise videos and to written lessons about staying motivated to exercise. Allows the user to schedule motivational exercise time for oneself throughout the week.
Slumber Time	Prompts the user to complete sleep diaries to track sleep. Provides a bedtime checklist intended to clear one's mind before going to sleep. Provides audio recordings to facilitate rest and relaxation. Features an alarm clock function.

Fig. 1

Screenshots of Thought Challenger.

The Hub app was designed to coordinate users' experience with the IntelliCare app suite. The Hub app manages messages and notifications from the other apps within the IntelliCare suite. In addition, the Hub app was intended to promote exploration of new behavioral strategies over time by encouraging users to try new apps. To this end, the Hub provided recommendations each week for new IntelliCare apps that had not yet been used.

Dissemination

IntelliCare suite apps were made publicly available for free download on the Google Play store. Upon download of each app, users were presented with a user acknowledgement agreement. This agreement notified users that use information would be stored and analyzed for quality assurance purposes. These procedures were approved by the Northwestern University Institutional Review Board (IRB), and de-identified app usage data from the first year of IntelliCare suite was downloaded and analyzed for this report.

Outcomes

Number of app downloads and launches were used to examine uptake and usage of the IntelliCare suite. App launch sessions were defined as user-initiated action within the app, separated by < 5 min between events. If a user left an app open and did not take action for 5 min or more, then the next interaction with the app counted as a new session. Time between first and last use of each app was extracted to examine continued engagement. Based on Helander and colleagues' (2015) mobile app study in which users were deemed as “active” if they used their healthy eating app 10 or more times, users in this study were deemed as “active” if they used an app 10 or more times.

Results

Rate of download

We began placing apps on the Google Play store on September 22, 2014, and continued to add apps as they were completed up until recruitment efforts began. The first major recruitment effort began on April 7, 2015 when the Northwestern Media office issued a press release. As seen in Fig. 2, the early April press release generated the greatest number of downloads for the IntelliCare suite. By October 30, 2015, 5210 individuals downloaded one or more of the IntelliCare apps, for a total of 10,131 app downloads. Nearly a third of these individuals (31.8%) downloaded more than one of these apps (see Table 2). The modal number of apps downloaded per user was 1, and the mean number of apps downloaded was 1.94.

Fig. 2

Number of downloads by date.

Table 2

Number of apps downloaded by unique users.

# Apps downloaded	# Users	% Users
1	3601	69.12%
2	594	11.40%
3	388	7.45%
4	186	3.57%
5	105	2.01%
6	79	1.52%
7	58	1.11%
8	43	0.83%
9	38	0.73%
10	35	0.67%
11	30	0.58%
12	26	0.50%
13	27	0.52%

Of the 5210 users included in these analyses, 1613 (30.9%) users installed the IntelliCare Hub on their mobile phones. For 648 of these users, the Hub app was the only app installed. These left a total of 965 (18.5% of all users) who downloaded the Hub app in addition to at least one of the interactive apps. As seen in Table 3, individuals who downloaded the IntelliCare Hub typically downloaded more separate interactive apps within the IntelliCare suite than those who did not, χ2(11) = 1370.05, p < 0.001.

Table 3

Number of interactive apps downloaded by Hub use status.

# of apps downloaded	IntelliCare Hub Users (n = 965)	% of total Hub users	IntelliCare Hub non-users (n = 3597)	% of total Hub non-users
1	260	26.94	2953	82.10
2	219	22.69	334	9.29
3	124	12.85	169	4.70
4	70	7.25	62	1.72
5	62	6.42	35	0.97
6	46	4.77	17	0.47
7	35	3.63	12	0.33
8	36	3.73	8	0.22
9	32	3.32	2	0.06
10	30	3.11	3	0.08
11	24	2.49	0	0.00
12	27	2.80	2	0.06

Order and timing of app downloads

For each user, the order in which apps were downloaded and the time between app downloads was calculated. The most popular initial app to download was the IntelliCare Hub, with 25.9% of all users downloading it first. Among users who downloaded multiple apps, the mean time in between first and last app download was 3.84 days and the median time in first and last app download was 2.23 days. Results of a Mann-Whitney U test indicated that the length of time between downloads was significantly greater for those who downloaded the Hub app compared to those who did not (Z = 7.203, p < 0.001). Most users downloaded all of their apps within a short amount of time. More than half of users (57.10%) downloaded all of their apps within a 24-hour period. Users without the Hub (i.e., non-Hub users) were more likely to download all of their apps within a 24 hour period, χ2(1) = 29.52, p < 0.001, with approximately two-thirds (65.34%) of non-Hub users having downloaded all of their apps within a 24-hour period compared to approximately half (52.97%) of Hub users having downloaded all of their apps within this same period.

Number of sessions

The modal number of sessions for each of the apps was 1, but many of the users returned to use each app multiple times, with mean number of app launches per app ranging from a low of 3.10 for ME Locate to a high of 16.98 for Daily Feats (see Table 4). Table 4 shows that, for users who launched an app at least twice, the time interval between first and last use was significantly greater for Hub users than non-Hub users (p < 0.001). This effect was driven primarily by Aspire, Daily Feats, and Slumber Time (ps < 0.001).

Table 4

Descriptive statistics for app download and use.

App name	Total sample			IntelliCare Hub Users (n = 965)			IntelliCare Hub Non-Users (n = 3597)
	Total download n	App launches M(SD)	Usage period in days M(SD)	Total download n	App launches M(SD)	Usage period in days M(SD)	Total download n	AppLaunchesM(SD)	Usage Periodin DaysM(SD)
Aspire	1023	3.58 (6.63)	12.96 (29.64)	364	4.60 (9.21)⁎	17.76 (37.51)⁎	659	3.01 (4.53)⁎	10.31 (23.84)⁎
Daily Feats	661	16.98 (39.50)	25.33 (49.83)	292	22.49 (47.04)⁎	35.53 (60.76)⁎	369	12.62 (31.70)⁎	17.26 (37.23)⁎
iCope	486	4.24 (7.79)	18.50 (39.42)	265	4.68 (9.09)	20.41 (41.31)	221	3.73 (5.86)	16.20 (36.98)
MyMantra	484	10.25 (25.60)	21.80 (45.04)	252	12.10 (30.67)	25.46 (49.55)	232	8.25 (18.45)	17.83 (39.27)
ME Locate	256	3.10 (6.34)	14.95 (38.51)	166	3.52 (7.60)	17.42 (40.78)	90	2.33 (2.70)	10.38 (33.66)
Day to Day	696	11.09 (22.37)	22.10 (43.16)	377	10.66 (23.13)	21.55 (41.20)	319	11.6 (21.47)	22.75 (45.42)
MoveMe	358	3.24 (4.33)	19.23 (40.75)	229	3.41 (4.68)	18.9 (38.77)	129	2.95 (3.61)	19.82 (44.20)
Purple Chill	770	5.37 (19.52)	20.24 (43.29)	307	5.71 (13.57)	24.70 (45.40)	463	5.15 (22.63)	17.28 (41.62)
Slumber Time	634	8.48 (17.60)	20.12 (40.29)	281	11.75 (21.32)⁎	23.85 (42.59)	353	5.88 (13.42)⁎	17.15 (38.17)
Social Force	312	3.47 (4.77)	14.44 (34.46)	211	3.50 (4.71)	16.30 (37.20)	101	3.41 (4.92)	10.53 (27.67)
Thought Challenger	1560	3.46 (5.64)	16.10 (35.83)	434	4.01 (8.13)	19.40 (39.91)	1126	3.25 (4.30)	14.83 (34.06)
Worry Knot	1278	3.49 (5.49)	17.02 (34.85)	467	3.47 (4.64)	17.51 (36.10)	811	3.50 (5.92)	16.74 (34.12)
All interactive IntelliCare Apps	8518	6.11 (17.18)	18.24 (39.11)	3645	7.43(20.18)⁎	21.49 (42.96)⁎	4873	5.12(14.45)⁎	15.80 (35.77)*

Note:

p < 0.001.

Sustained engagement with apps

Repeated use varied considerably across apps. To examine this variability, the number of active users for each app (as defined by 10 or more app sessions) and the percentage of active users out of total users per app were calculated. As seen in Table 5, the percentage of active users ranged from 4.69% (for ME Locate) to 35.70% (for Daily Feats). To examine the persistent use of these apps, the percentage of individuals who continued to use each of the 12 apps over 1-, 3-, 7-, 14- and 28-day timeframes was calculated (by dividing number of users demonstrating use at each time point by the number of users who had initially downloaded that app). As seen in Table 6, approximately half of users (ranging from 38.67% for ME Locate and 70.19% for Daily Feats) continued to use the apps for more than one day. Seven days after download, approximately one-third of users continued to use the apps. While fewer individuals continued to use the apps over a 28-day period, Daily Feats remained the most continuously used app with 23.30% of initial users demonstrating sustained engagement at this time.

Table 5

Number of active users (≥ 10 sessions) per app.

App Name	Number of users	% of total users
Aspire	60	5.87
Daily Feats	236	35.70
iCope	41	8.44
MyMantra	111	22.93
ME Locate	12	4.69
Day to Day	178	25.57
MoveMe	23	6.42
Purple Chill	80	10.39
Slumber Time	128	20.19
Social Force	25	8.01
Thought Challenger	101	6.47
Worry Knot	82	6.42

Table 6

Use of each app over 1, 3, 7, 14 and 28-day timeframes.

App name	Total # of users	≥ 1 day		≥ 3 days		≥ 7 days		≥ 14 days		≥ 28 days
		# of users	% of remaining users	# of users	% of remaining users	# of users	% of remaining users	# of users	% of remaining users	# of users	% of remaining users
Aspire	1023	468	45.74%	378	36.95%	295	29.12%	220	21.50%	123	12.02%
Daily Feats	661	464	70.19%	392	59.30%	318	48.11%	241	36.46%	154	23.30%
iCope	486	251	51.65%	209	43.00%	166	34.16%	124	25.51%	83	17.08%
MyMantra	484	256	52.89%	203	41.94%	166	34.30%	132	27.27%	97	22.35%
ME Locate	256	99	38.67%	77	30.08%	58	22.66%	48	18.75%	35	13.67%
Day to Day	696	419	60.20%	346	49.71%	275	39.51%	205	29.45%	141	20.26%
MoveMe	358	167	46.65%	136	37.99%	118	32.96%	89	24.86%	62	17.32%
Purple Chill	770	349	45.32%	295	38.31%	250	32.47%	196	25.45%	143	18.57%
Slumber Time	634	374	58.99%	298	47.00%	239	37.69%	178	28.08%	115	18.14%
Social Force	312	144	46.15%	107	34.29%	83	26.60%	59	18.91%	41	13.14%
Thought Challenger	1560	744	47.69%	596	38.21%	486	31.15%	370	23.72%	237	15.19%
Worry Knot	1278	648	50.70%	524	41.00%	433	33.88%	341	26.60%	209	16.35%

Discussion

Results from our open deployment suggest that many people will use multiple mental health apps from an integrated app suite to meet their own needs. App suites have begun to be used in other commercial settings, and are likely to continue to be utilized in the coming years. For example, the Lexus Enform app suite connects Lexus owners' mobile phones with the vehicle's center display console and provides different entertainment-related tools. The Amazon app suite provides Amazon users with easy access to specific components of Amazon's services (e.g. music, books, shopping). To our knowledge, use of an integrated suite of mobile phone apps for behavior change and mental health care is a novel concept.

Usage patterns from this public deployment suggest that integrated app suites may have the potential to introduce multiple components of evidence-based mental health treatments to the general public in a format that matches prevailing app use trends (e.g., a focus on lightweight functionality with easy navigation to desired tools), and in a way that promotes both usage and self-tailoring. Despite the novelty of an app suite method for mental healthcare, substantial numbers of people downloaded and used multiple apps.

Usage of apps within the IntelliCare suite was higher than what has previously been reported in past public deployments of BITs. Helander and colleagues (2015) found that 2.6% of all downloads of a health eating app were active users. In contrast, active use of IntelliCare apps ranged from 4.7% to 35.7%. Furthermore, substantial numbers of users persisted in using the apps and engaged with the apps over extended periods of time. Use over 14 days ranged from 18.8% to 36.5% of users, while use over 28 or more days ranged from 13.1% to 23.3%. This is an important metric, given that effectiveness of the behavioral strategies depends in part on sustained engagement in the behaviors over time.

A suite of apps could be deployed as a collection unified simply by name and design. For example, searching “IntelliCare” will pull up the individual IntelliCare apps, which are distinguishable by the theme of the icon from other apps that also appear. However, this does not necessarily lead to an integrated experience. The Hub app was designed as a tool to coordinate the use of the IntelliCare app suite for the user, by coordinating notifications and providing recommendations. Even though a coordinating Hub app is a novel method of interfacing with users, 21.1% of users downloaded it. Those who downloaded the Hub app were more likely to download multiple apps from the suite, and use frequently and for longer periods of time. This is consistent with the concept of a central managing hub app method of coordinating user experience with a suite of apps. However, this was not a comparative trial and use of the Hub app was not randomly assigned to users. Therefore, the relationship between Hub use and use of other clinical apps should be subject to further empirical investigation.

Given the considerable variability in the use of the IntelliCare apps, further investigation and design iterations are warranted. Some of this variability may be due to variations in the frequency of the specific mental health and wellness concerns addressed by these apps, or due to the existence of other publically available apps that address said concerns. However, some of this variability is likely due to usability issues. Many of the apps, including Daily Feats, Day to Day, MyMantra and Slumber Time, appear to be performing well, as they are used consistently and frequently. Other apps may need to be refined, or removed from the IntelliCare suite as development of this system continues. We note that rather than engaging in extensive user-centered design processes for each app, we chose, due to time and cost limitations, to develop small, lightweight apps, release them to the general public quickly, and make iterative changes based on user feedback and observed usage patterns. We did not expect all of the apps to be successful, and these expectations appear to be borne out by the relatively low engagement among some of the apps.

Several limitations of the current study should be noted. First, while these initial engagement numbers are encouraging, they represent an early snapshot, in contrast to other reports that have analyzed 100,000′s of users over longer periods of time (Muñoz et al., 2015, Helander et al., 2014). The current set of users had to seek out IntelliCare, as the IntelliCare suite does not appear early in app store searches using common search terms such as “depression” or “anxiety.” It is possible that these early adopters are more motivated than later users will be. However, it is worth noting, that in previous large-scale deployments that have revisited their data over time, early trends tend to be retained in subsequent visitors even with changes in demographics of those who visit (Muñoz et al., 2012, Muñoz et al., 2015).

We also have no reliable data on efficacy at this point. Use and outcome may not be directly related (Donkin et al., 2011). It is possible that some individuals stop using these apps because they have benefitted from the apps. This potential subset of individuals may no longer feel a need to practice these mood management and wellness strategies via interactive tool use because they are no longer feeling distressed. It is also possible that some individuals stop using the apps because they have begun to apply the skills to their lives, or because they may not believe that the apps will be useful to them despite ongoing distress. App utilization, as presented in this study, is an important metric in evaluating the potential reach of the intervention, but it does not present a full picture of the effectiveness of the IntelliCare suite.

Finally, for those who downloaded multiple apps, all apps tended to be downloaded within a few days, which suggests that there is a brief period during which users of the IntelliCare suite explore and make app selections. While this period was significantly longer for Hub users, the difference did not appear to be meaningful. Thus, there was no evidence that the recommendation feature had any effect. There are several possibilities for failure. First, the recommendations were at random, and therefore not likely to be useful. It is also possible that people's motivation to engage in such self-help activities is episodic, and that such systems have only a brief period of time to engage them.

To address some of the problems described above, the IntelliCare suite is continuing to evolve through two lines of development. The failure of the Hub app's recommendation system to elicit new downloads may be improved if the suggestions are more likely to be perceived as useful. To address this, data from this public deployment are currently being used to develop analytics for a recommender system that can leverage use data to predict which of the remaining apps is most likely to be used and useful. Second, we are testing the value of integrating coaching into the app suite that can sustain users' motivation to explore new apps over time and to benefit from those apps (e.g., Mohr et al., 2011, Schueller et al., 2016). The use of support from coaches or therapists in Internet-based treatments for depression and anxiety have been found to have moderate to large outcome effect sizes (Andersson and Cuijpers, 2009, Cuijpers et al., 2009, Baumeister et al., 2014). While less work has been done on the use of coaching in mobile app-based interventions, a recent pilot randomized controlled trial examining an app for PTSD found that those users who received coach support had better outcomes than those who did not (Possemato et al., 2015).

Conclusions

The purpose of this study was to examine the initial uptake and patterns of usage of the IntelliCare suite of mental health and wellness apps, which was freely disseminated on the Google Play store. The IntelliCare app suite is novel in that it provides users with access to multiple apps, each of which targets a specific behavioral strategy through lightweight apps. The experience across apps is coordinated through a central Hub app. Uptake and continued usage of these apps were higher than has been seen in previous publicly available BITs, suggesting the feasibility of such an approach.