Coaching to Support Mental Health Apps: Exploratory Narrative Review.

BACKGROUND: The therapeutic alliance is crucial for the success of face-to-face therapies. Little is known about how coaching functions and fosters the therapeutic alliance in asynchronous treatment modalities such as smartphone apps.
OBJECTIVE: The aim of this paper was to assess how coaching functions and fosters the therapeutic alliance in asynchronous treatment modalities.
METHODS: We conducted a selected review to gather preliminary data about the role of coaching in mobile technology use for mental health care. We identified 26 trials using a 2019 review by Tønning et al and a 2021 scoping review by Tokgöz et al to assess how coaching is currently being used across different studies.
RESULTS: Our results showed a high level of heterogeneity as studies used varying types of coaching methods but provided little information about coaching protocols and training. Coaching was feasible by clinicians and nonclinicians, scheduled and on demand, and across all technologies ranging from phone calls to social media.
CONCLUSIONS: Further research is required to better understand the effects of coaching in mobile mental health treatments, but examples offered from reviewed papers suggest several options to implement coaching today. Coaching based on replicable protocols that are verifiable for fidelity will enable the scaling of this model and a better exploration of the digital therapeutic alliance. ©Ashley Meyer, Hannah Wisniewski, John Torous. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 08.03.2022.

Introduction

Smartphones and other mobile technologies are increasingly used in mental health care. The COVID-19 pandemic has highlighted the need for mobile treatments in providing access to and augmenting mental health care. However, fundamental questions remain around app engagement and efficacy as do concerns about technology use in the contact of coaching support and the therapeutic alliance. The therapeutic alliance, which is described as the alliance between a clinician and a patient [1], is considered crucial to the success of face-to-face therapies and associated with successful outcomes [2]. Numerous meta-analyses have confirmed the central role of the therapeutic alliance in driving both engagement and efficacy across both face-to-face care and even telehealth video visits. However, less is known about how this alliance functions in asynchronous treatment modalities such as smartphone apps. In both research studies as well as commercial apps, a rise in coaching to support engagement may be conceptualized in the context of adding an element of the therapeutic alliance into digital care. Preliminary research suggests that coaching is feasible and acceptable, with both clinicians and clients identifying benefits to this additional support, such as increased motivation and guidance as well as a new way to focus on clinical work [3]. Yet, little is known about how this app coaching is delivered or impacts outcomes.

This lack of knowledge on the therapeutic alliance and coaching around smartphone apps is in part related to a lack of consensus on coaching methods and training. Such a lack of protocols or training manuals has immediate consequences as well-trained clinicians are more prepared to foster the therapeutic alliance and support technological difficulties [4]. At the beginning of the COVID-19 pandemic, 21% of community health centers across the United States lacked training for telehealth [5]. Training for mobile apps is nascent, and evidence can often only be found in the research literature. For example, one study in Australia found that clinician training on mobile technologies and protocols for consistent messaging with the patient was necessary to increase engagement and knowledge of the apps [6]. A recent report from Kaiser Permanente explained that teaching clinical staff how to use and interact with apps in care settings was critical for implementation success [7]. With COVID-19, grassroots efforts to train medical students to support patients around app use have also emerged [8]. The need for proper training and new knowledge around engagement and alliance to support app use is thus an important new facet of care toward offering accessible mental health care in an increasingly digital world. The term “training” is used throughout this work and defined based on the above references as one or more of the following: the receipt of coaching protocols or manuals by the coach, instruction for the coach on how to conduct telehealth sessions, how to use and interact with mobile apps, and how to support patients around app use.

Several solutions have been proposed already. Many have examined a more traditional “coaching” model in which a member of the care team maintains contact with patients between visits to foster engagement with the mobile technology. This contact may be through the app, text messages, or even phone calls. A more engaged approach is conceptualized around the concept of a “digital navigator,” which seeks to integrate into more aspects of care with the goal of supporting both the patient and clinician [9]. Ben-Zeev et al [10] and Noel et al [11] have proposed related roles called clinical technology specialists or technology specialists, respectively. Each of these positions seeks to provide technology recommendations and support user engagement with digital technologies.

A related method for maintaining alliance with patients using digital technology uses a “social” model of coaching. This model focuses on interactions between peers as opposed to a member of the clinical or study team. Although this model of coaching appears less used, several studies have incorporated this method. McEnery et al [12] evaluated the feasibility of an online intervention, EMBRACE, where participants maintained contact with clinical moderators, a more traditional coach, as well as peer-to-peer moderators, who were young individuals with lived mental health experience who encouraged participant engagement and provided support. Further, Alvarez-Jimenez et al [13] assessed the feasibility and acceptability of the enhanced moderated online social therapy (MOST+), which allowed participants to interact with other participants using the platform, as well as peer moderators who facilitated engagement. However, neither of these protocols directly assessed the effect of moderation on outcomes, and, of note, neither involved smartphone apps.

To further understand the current knowledge of coaching effects, we conducted an investigation to gather preliminary data on coaching and to understand its effect on engagement and outcomes. We hypothesized that there would be substantial variability and little consensus on coaching protocols, inconsistent reporting measures, and a lack of protocols that directly assess coaching effects. However, understanding how coaching is reported and broad trends in its outcomes is useful for new efforts to align new research and implementation efforts with prior work. In turn, understanding and identifying best practices to facilitate coaching and support the digital therapeutic alliance is crucial as remote psychotherapy increases in popularity and necessity.

Methods

We conducted a selected review to gather preliminary data about the role of coaching in mobile technology use. Realizing there is no simple means to identify relevant papers as all will have some degree of coaching support, often unreported in research assistant help, we opted for an exploratory sampling approach. We chose to use a prior review of randomized controlled trials on smartphone-based treatment in psychiatry as well as a recent scoping review on digital health interventions for depression (featuring 6 smartphone-based interventions) as the samples for assessing how coaching is offered across different apps and studies. We did not attempt to conduct a qualitative analysis as we expected outcomes to be heterogeneous and diverse given the state of the literature.

We identified the trials using the 2019 review by Tønning et al [14] on the methodological challenges of randomized controlled trials on smartphone-based treatment in psychiatry and the 2021 scoping review by Tokgöz et al [15] on digital health interventions for depression. We selected these papers as they offered a recent and comprehensive sample of studies from which we could explore coaching. Each trial featured in this review was read and coded by the authors for method of contact with the patient, training coaches received, on demand vs scheduled interactions, clinical vs nonclinical interactions, evidence of dose effect, and social vs coaching model. Studies were excluded if the participants had no interaction while using the mobile technology.

After coding of each trial was completed, the trials were sorted into the following 4 categories based on the frequency (scheduled vs on demand) and nature of the coaching (nonclinical vs clinical): scheduled or clinical; scheduled or nonclinical; on demand or clinical; and on demand or nonclinical. Scheduled coaching included coaching delivered on a set time frame, such as once per week or per month, or after the completion of a certain assessment. On-demand coaching was delivered on an irregular schedule based on the needs of the study participant or clinician, such as clinician responses to high participant assessment score or a participant contacting study team for questions; however, we did not include on-demand crisis intervention in this category and will not be reviewing such interventions within this work. Clinical coaching focused on the participant’s symptomatology, whereas nonclinical coaching focused on technology or study protocol questions. Some trials that included a variety of methods for interaction were coded into multiple categories. We then assigned a hierarchy to the codes as follows from highest priority to lowest priority: (1) on demand or clinical; (2) on demand or nonclinical; (3) scheduled or clinical; and (4) scheduled or nonclinical. Dose effect was defined as a reported association between the time or intensity of coaching for the participants and a primary study outcome. While the digital therapeutic alliance is of critical interest, it is not yet possible to code given that the means to assess it are nascent, as discussed later in this paper.

Results

A total of 32 trials were reviewed as featured in the 2019 review by Tønning et al and 2021 scoping review by Tokgöz et al; 6 of these studies did not involve reported coaching interaction while the participants were using the smartphone technology and were excluded from review by the study team [16-21]. Therefore, a total of 26 trials were included [22-47]. A summary of the results can be seen in Multimedia Appendix 1 and Table 1.

Table 1

Summary of coding metrics.

Criteria and coding specifications		Number of studies, n (%)
Type of training
	Specified	9 (35)
	Unspecified	16 (62)
Mode of contact
	Email	7 (27)
	Phone call	14 (54)
	In-app message	5 (19)
	Text message	5 (19)
	In-person	6 (23)
	FaceTime or teleconference	2 (8)
	Notifications	1 (4)
	Facebook	1 (4)
On demand vs scheduled
	On demand	12 (46)
	Scheduled	21 (81)
Clinical vs nonclinical
	Clinical	19 (73)
	Nonclinical	14 (54)
Clinician vs nonclinician vs peer
	Clinician	18 (69)
	Nonclinician	3 (12)
	Peer	3 (12)
	Not specified	4 (15)
Time spent per interaction
	Specified	11 (42)
	Not specified	15 (58)
Evidence of dose effect
	No	25 (96)
	Yes	1 (4)
Social vs coach model
	Social	24 (92)
	Coach	5 (19)
Participants compensated
	Yes	17 (65)
	No	9 (45)
Participants received smartphone
	Yes	4 (15)
	Yes, if necessary	7 (27)
	No	16 (62)
Remote study
	Yes	12 (46)
	No	14 (54)

As seen in Table 1, there was high variability around coaching across each of the 26 studies in the type of coaching delivered and coach training. The majority of studies included a scheduled coaching component (14 scheduled or clinical) [22,23,25,26,28,29,31,33,36,38,40,45-47]; 12 scheduled or nonclinical [24,29,32,33,35,38,41,45,46]) compared to an on-demand coaching component (8 on demand or clinical [22,24,27,29,30,37-39]; 9 on demand or nonclinical [24,25,29,34,37,38,42,44,46]); 11 of the studies incorporated 2 or more kinds of coaching [18,20,21,25,29,33,34,38,44-46].

There was less variability as to who provided the coaching. Clinicians acted as coaches in 18 of the studies [22-31,36-41,45,47], nonclinicians in 3 studies [32,35,42], and peers in 3 studies [34,37,44]. Moreover, 4 studies did not specify who acted as coaches [33,34,44,46]. In addition, the majority of studies did not specify the type of training that the coaches completed for their role; 16 studies did not specify the type of training [22-25,27,30,32,33,35-45], 1 study specified that training was not conducted [34], and 7 studies specified that coaches underwent training of some kind [26,28,29,31,38,41,45]. Of the studies that did specify the training for the coaches, there was high variability; 1 study noted the coaches’ training was their standard training as a part of their clinical psychology program [29]; another noted the training was a “1-day workshop in using the self-help program and on how to write the weekly feedback, based on case material from earlier trials” [38], while another only stated their training was “based on the supportive accountability model” [41]. Only 1 study offered a protocol for the training offered [46].

Only 3 studies used the social coaching model [34,37,44]; 1 of these trials used only the social model [34], while 2 of them used the social model along with the coach model [37,44]. Boettcher et al [34], who used only the social model, examined the efficacy of a smartphone app called Challenger in reducing anxiety symptoms in individuals with social anxiety disorder. The participants were randomized to use Challenger and a self-help program simultaneously, the self-help program for 6 weeks followed by the Challenger app for 6 weeks, or a waitlist control. Challenger used cognitive behavioral therapy techniques to encourage its users to complete small exposure and behavioral challenges in everyday life. The skills gradually increased in difficulty. After each skill, the user is able to complete a reflection of the task, which is sent to another user who is able to respond with constructive feedback. The participants in Schlosser et al [37] and Roepke et al [44] were able to interact with other app users or use a forum and recruit social support from Facebook, respectively.

There was evidence of a dose effect in only 1 study. In particular, a pilot randomized controlled trial conducted by Pfeiffer et al [47], exploring psychotherapeutic text messaging for depression, found that change in behavioral activation was correlated with specifically 6 weeks of receiving acceptance and commitment therapy-based messages (ρ=-0.25; P<.05), as opposed to 12 weeks, at which point there was no correlation observed [47]. Studies used varying measures of engagement and efficacy of the respective smartphone technologies; 38.5% (n=10) reported percent completion of the program [21-23,28,29,32,34,35,40,46], 19.2% (n=5) reported app use per week or day [24,27,30,36,46], 19.2% (n=5 reported the retention or dropout rate [37,38,43,45,46], and 7.7% (n=2) reported the number of logins to the program [33,44]. However, only 10 studies reported the duration of time spent per coaching interaction [22-25,29-31,41,42,46], and many did not directly assess the influence of coaching on the results.

Finally, there was high variability in the mode of contact used across the studies; 13 (50%) of the studies used 2 or more means of contact [22,24,25,27,29,30,32,37,41,42,44-46]. Phone calls were most commonly used to contact participants (14, 53.8%) [22-24,27,29,30,32,33,35,37,41,42,45,46], followed by emails (7, 26.9%) [22,25,27,32,41,43,44], then in-person meetings (6, 23%) [26,28,31,36,42,45], in-app messaging (5, 19.2%) [24,29,30,34,37], text messaging (5, 19.2%) [25,27,38,46,47], FaceTime or teleconference (2, 7.7%) [37,39], app notifications (1, 3.8%) [39], and Facebook (1, 3.8%) [44].

Discussion

Coaching offers a solution to engagement challenges with digital mental health, but its interpretation and implementation remain heterogeneous, consistent with our hypothesis. While our results are not a comprehensive review, they offer a selected sample across the mental health app literature, which highlights the diversity of efforts and results when applying different models of coaching to support apps. A lack of consensus around coaching protocols and outcomes precludes discussion of whether coaching may be a covariate, confounder, moderator, or mediator for clinical improvement with apps. While we were not able to explicitly measure the therapeutic alliance construct within this work, the heterogeneity found across coaching modalities may suggest a lack of consensus regarding how to best foster a digital therapeutic alliance [48] between the patient and clinicians. Recent studies not captured in our sample have employed the Digital Working Alliance Inventory to measure alliance with apps and suggested that such an alliance may predict app engagement [49], highlighting the significance of future research and standardization around this concept.

The high degree of heterogeneity reflected in our results suggests the versatility of coaching and its ability to easily adapt to unique circumstances. Coaching was feasible across all platforms ranging from text messages to social media and for both on-demand and scheduled interactions. Coaches were also able to support completely remote studies (defined as specifically involving no synchronous interactions) as well as offer face-to-face services in meeting with participants in other studies. While a clinician served the role in 69.2% of studies, the role is also accessible to other people including those with no formal training.

One challenge around understanding the efficacy of coaching, beyond the heterogeneity of the role and studies, is that training protocols, fidelity to those protocols, and coaching specific outcomes are often not reported. Without understanding how coaches are trained and if they adhere to that training during the study, it is impossible to understand what support is actually being delivered. Study metrics reports by coach instead of participant and cohort may also offer productive data toward understanding the impact of this role. While no studies measured outcomes such as the Working Alliance Inventory, alliance-specific measures would offer information into potential mechanisms of action.

However, the results from this paper offer several paths forward. These results suggest that clinical vs nonclinical staff can serve in coaching roles, and scheduled vs on-demand support can also both be feasible. Crowdsourcing peer support via social networks or small internal networks also appears feasible. As the role and best practices evolve, clinics can implement the methods that best match their local needs and resources. The different models presented in this paper can serve as examples in building new coaching services and provide measures to consider during implementation. While beyond the immediate scope of this article, protocols around digital mental health coaching are emerging and can serve as further reference [50,51]. Of note, neither of these protocols or earlier versions of them was used in any paper reviewed.

Our results are in line with prior works that have examined coaching around mental health apps. In a 2020 paper, Callejas et al [51] reported on selected examples and noted a need for more data around engagement and mechanisms of action underlying coaching. A lack of consensus around app engagement measures has also been found in recent reviews [52,53].

A chief limitation of this work is that it draws a sample from only 2 reviews of mental health app studies. Given that nearly every digital health study involves some degree of coaching (even if they are informal support from research assistants, which may not be reported), it is infeasible to conduct a broader review. Therefore, our goal was not to include every relevant paper, but rather to conduct a preliminary investigation into coaching techniques used by recent studies and identify trends. Other studies have specifically explored coaching and mental health apps. For example, in their 2019 paper, Mohr et al [54] found that coaching was associated with more downloads of a mental health app but not long-term engagement with that app. Our results are thus best interpreted as exploratory signals that suggest productive avenues for exploring coaching as well as guidance for understanding the high degree of heterogeneity that must be unpacked in new research efforts. The classification scheme used in this study was created de novo by our team given the state of this literature and can serve as a useful scaffold to create new versions in the future.

Coaching for mental health apps will continue to expand in scope, necessitating an understanding of its therapeutic potential and implementation into care settings. While current efforts around the role remain diverse, they suggest a flexibility necessary to support the evolving digital mental health space and to work across diverse populations and technologies.