Evaluating the Feasibility and Effectiveness of an Australian Safety Planning Smartphone Application: A Pilot Study Within a Tertiary Mental Health Service.

OBJECTIVE: The purpose of this study was to examine the feasibility and effectiveness of a suicide prevention smartphone application.
METHOD: Thirty-six non-Aboriginal Australians aged between 16 and 42 years (67% female) were recruited from a tertiary mental health service where they were receiving treatment for suicide risk. Participants were asked to use the BeyondNow safety planning smartphone application to manage their suicide safety plan during a 2-month trial, as an adjunct to treatment as usual. A survey battery designed to measure feasibility and effectiveness of the smartphone app plus treatment as usual intervention was completed at baseline and follow-up.
RESULTS: A vast majority of participants used the app to view and edit their safety plans and reported that the app was easy to use. A reduction was observed in participant severity and intensity of suicide ideation, and suicide-related coping increased significantly. No significant changes were observed in suicide resilience.
CONCLUSIONS: The BeyondNow safety planning smartphone application was shown to be feasible and effective as an adjunct to mental health treatment among patients at risk of suicide.

In the last 10 years, substantial efforts have gone into suicide prevention strategies and activities in a bid to reduce the mortality rates and the morbidity associated with suicide attempts (Zalsman et al., 2016). Despite these efforts, suicide remains a leading cause of mortality and morbidity internationally, with suicide rates remaining largely stable in some Western countries such as Australia (Burns, 2016) and the United Kingdom (Coope et al., 2014) or increasing in others such as the United States (Curtin, Hedegaard, & Warner, 2017). The success of these prevention efforts may have been hampered by a range of factors including modest levels of help‐seeking following suicide attempts (De Leo, Cerin, Spathonis, & Burgis, 2005), limited evidence‐based interventions to manage suicide risk (Zalsman et al., 2016), and difficulty accessing services (WHO, 2010). As such, specific interventions and modes of service delivery that can overcome some of these barriers are needed.

One such intervention that was designed to support individuals experiencing suicidal thoughts, which is brief, low‐burden, and customizable, is the Safety Planning Intervention (SPI; Stanley & Brown, 2012). Although the general concept of developing a plan to follow during a suicidal crisis has been discussed for many years (e.g., Rudd, Mandrusiak, & Joiner, 2006), the SPI provides individuals with a structured and personalized resource to assist with identifying an impending crisis, and activating internal and external suicide‐related coping strategies. The SPI is a straight‐forward intervention that aims to increase self‐help and help‐seeking behaviors as well as supporting individuals who do not, or cannot, readily engage with mental health services.

Empirical research evaluating the SPI is emerging. Notwithstanding, safety planning has been identified as “best practice” by the US Suicide Prevention Resource Centre and is listed as a research informed intervention within the Zero Suicide in Health and Behavioral Health Systems initiative. Research on safety planning has largely taken place among samples of US military veterans, within a veteran emergency treatment project called Safe Vet (Knox et al., 2012). Participating suicidal veterans (N = 96) receiving Safety Planning Intervention and brief Structured Follow‐Up (SPI‐SFU) telephone calls were compared with those receiving treatment as usual (TAU; Stanley et al., 2015). The results showed that the SPI‐SFU group were significantly more likely to attend mental health or substance‐use outpatient treatments following discharge compared with TAU. Furthermore, hospital admissions for suicide risk were lower following the SPI‐SFU intervention compared to TAU, although this difference failed to reach statistical significance (Stanley et al., 2015). Safe Vet participants (N = 100) also reported that their safety plans are generally helpful (Stanley et al., 2016). Ninety‐nine percent reported at least one helpful section within their safety plan, including social contacts and places for distraction (52%), social support for help (47%), professional contacts (45%), internal coping strategies (27%), warning signs (8%), and ways to restrict access to lethal means (1%).

The Safe Vet project also gathered the opinions of service providers (N = 50) regarding the practice of SPI via interview (Chesin et al., 2016). Almost all (98%) participating staff reported that they were satisfied with the SPI‐SFU intervention for various reasons including their increased comfort in discharging patients who presented with suicidal behavior and increased monitoring of suicidal patients. Furthermore, staff reported that the intervention was helpful as it was perceived to decrease suicidal behavior and increase self‐efficacy in managing distress and suicidal thoughts (Chesin et al., 2016). The SPI is an efficient intervention in terms of its financial cost and time taken to complete and appears to be supported and accepted by clinicians and suicidal patients. Inherent in its design is its ability to be used by the individual away from psychiatric services. In these ways, the SPI might, in part, overcome some of the barriers to help‐seeking behavior for those at risk of suicide.

However, the SPI has its own barriers to use. A study by Kayman, Goldstein, Dixon, and Goodman (2015) examined qualitative opinions of 20 suicidal veterans’ regarding safety planning. The barriers reported by veterans included internal factors such as social withdrawal and a lack of motivation to use their plan. The veterans also reported barriers to use associated with the pen‐and‐paper safety plans, including difficulty locating the hard copy plan (Kayman et al., 2015). As such, it appears that although the SPI is a helpful suicide prevention intervention, the pen‐and‐paper mode of delivery has its own barriers to effective utilization.

The popularity of mobile technologies, such as smartphone applications, is growing quickly within the field of mental health. Smartphones provide a medium that has the potential to overcome some of the barriers associated with mental health interventions. Although there are limits to the scope of intervention which can be provided via mobile technologies, they potentially offer a platform for efficacious, readily accessible, and cost‐effective stand‐alone interventions as well as having the capacity to augment traditional face‐to‐face therapies. Specifically, smartphone technology provides a medium for documentation of safety plans via which to overcome some of the limitations of hard copy plans.

In this context, the BeyondNow safety planning smartphone application (app) was developed in a collaborative project between beyondblue (an independent not‐for‐profit organization working to reduce the impact of anxiety, depression, and suicide in Australia) and Monash University. BeyondNow is based on the Safety Planning Intervention (SPI; Stanley & Brown, 2012) and was modeled on the first app of its kind, Safety Net, developed in the United States by the same authors. The BeyondNow app provides a platform for individuals to create, edit, access, and share their personalized safety plan. Users are able to list warning signs, reasons to live, ways to limit access to lethal means, coping strategies, and personal and professional contacts. Information is entered into the app using free text or by selecting from a drop‐down list of suggestions. Personal and professional contacts can be directly imported from the user's phone directory. Once a safety plan has been created, it can be viewed easily in a single scroll‐down page, edited, or shared via email. The BeyondNow app also includes a red emergency phone button on each page, which allows quick access to preloaded Australian emergency service phone numbers, and also has an information section containing a video outlining the practice of safety planning as well as a link to further information.

BeyondNow was developed with input from individuals with lived experience of suicide ideation and behavior, who were from an Australian anxiety, depression, and suicide consumer reference group (blueVoices; n = 10), and mental health professionals from a variety of disciplines including psychology, general practice, mental health nursing, and occupational therapy (n = 9). Participants from both groups provided opinions on the US SafetyNet app, desirable features, and functionality, as well as advantages, disadvantages, and potential barriers to using mental health smartphone apps. The clinician group also reported their prior, and intended future usage, of mental health apps. In March 2016, the BeyondNow app was released in Australia. According to an app‐analytics service report (App Annie), by March 2018, the app had been downloaded more than 40,000 times.

Despite the high number of smartphone apps available that were designed for mental health difficulties, few have been examined empirically with regard to their effectiveness. Recent studies of smartphone‐ and computer‐based mental health treatments have described the software development and focused on examining usability and user satisfaction (Bush et al., 2015; Heffner, Vilardaga, Mercer, Kientz, & Bricker, 2015). Bush et al. (2015) compared an app‐based and a physical version of a therapeutic tool called a hope box, which allows an individual to collect and store a range of items to support coping, such as pictures, music, and relaxation tools. Participants found the virtual hope box easy to setup and helpful and used it more regularly when compared to the physical hope box. All participants stated that they would recommend the virtual hope box to a fellow veteran. A Danish safety planning smartphone app, based on SPI (Stanley & Brown, 2012), called MYPLAN, received positive anecdotal feedback from participants, primarily around the convenience of having their safety plan close at hand at all times. The authors reported that patients took ownership of individualizing their safety plans and found the app easy to use (Larsen, Frandsen, & Erlangsen, 2016). More recently, Boudreaux et al. (2017) reported the development and usability of an Internet‐based safety planning system based on the SPI. Participants had active suicide ideation (N = 30) and 40% also reported computer literacy difficulties. Around one‐quarter experienced difficulties with process of developing an online safety plan illustrating the importance of ensuring that end‐users are comfortable using apps and software.

The first aim of this pilot study was to examine the feasibility of integrating the BeyondNow safety planning smartphone app into an Australian tertiary mental health service, as well as the effectiveness of the app plus treatment as usual (App + TAU) intervention in reducing suicide risk. It was hypothesized that the App + TAU intervention would be feasible, based on participant uptake and use of the BeyondNow app, as well as positive opinions of app usability and functionality. It was also expected that the App + TAU intervention would be associated with increased suicide‐related coping and suicide‐related resilience, as well as reduced suicide ideation.

Method

Study design

An open‐label single‐group trial was conducted. All participants were allocated to a treatment condition which included developing a safety plan collaboratively with a clinician which was documented using the BeyondNow app, as an adjunct to existing interventions at the tertiary mental health service. A study design involving a nonsafety planning comparison group was not possible, in order to ensure adherence to best practice guidelines at Monash Health for suicidal patients.

Participant recruitment

Participants were recruited from a tertiary mental health service in Melbourne, Australia. Forty‐five participants were referred to the study by their treating clinicians. Six participants either declined to participate or were excluded due to having either an intellectual disability or a noncompatible smartphone. Three participants withdrew from the study during the baseline assessment. Hence, the sample comprised 36 individuals (female n = 24, 66.7%) aged between 16 and 42 years (M = 19.89 years, SD = 6.04 years; Figure 1).

Figure 1

Participant flow

Measures

Feasibility measures

Google Analytics software was used to collect aggregate app usage data for the entire sample. Frequency and total duration of app usage, as well as use of the sharing function and emergency button, were collected. The software was also used to collect data on the number of safety plan component entries via the “Give me suggestions” function, free‐text entry, and the number of contact detail entries added directly from the users’ phone directories.

An App Feedback Survey (AFS; Melvin & Gresham, 2016a) was developed for use within the current study. The survey included six quantitative items relating to the degree and difficulty of app usage, and one item asking whether the participant would recommend the app to a friend. The survey also included two free‐text qualitative items, “What are the best things about the app?” and “How would you improve the app?”

Effectiveness measures

The Columbia Suicide Severity Rating Scale (C‐SSRS; Posner, et al., 2011) is a semi‐structured clinical interview designed to assess the suicidal thoughts, suicidal behavior and nonsuicidal self‐injury. The C‐SSRS was used to capture participants’ severity of ideation (SI; past 2‐months), intensity of ideation (II; past 2‐months), suicidal behavior (SB; past 2‐months & lifetime), and nonsuicidal self‐injury (NSSI; past 2‐months & lifetime). The C‐SSRS has been validated for clinical use with adolescents and adults and has been shown to have strong psychometric properties including sensitivity to change over time, predictive validity, and internal consistency (Posner, et al., 2011).

The Suicide Related Coping Scale (SRCS; Stanley, Green, Ghahramanlou‐Holloway, Brenner, & Brown, 2017) is a 17‐item self‐report scale used to measure knowledge and confidence in using internal coping strategies and external resources to manage suicidal thoughts and urges. Initial investigation of psychometric properties showed high internal consistency as well as convergent validity with measures assessing attitude toward seeking healthcare services and perceived barriers to care (Stanley et al., 2017).

The Suicide Resilience Inventory‐25 (SRI‐25) is a 25‐item self‐report scale used to assess participants’ perceived ability, resources, and competence to regulate suicide‐related thoughts, feelings, and attitudes. The inventory measures protective factors for suicide via three subscales: Internal Protective (IP; e.g., “There are many things that I like about myself”), External Protective (EP; “People close to me would find the time to listen if I were to talk seriously about killing myself”), and Emotional Stability (ES; e.g., “I can resist the urge to kill myself when I feel depressed or sad”). The SRI‐25 has demonstrated sound psychometric properties among the target population (Gutierrez, Freedenthal, Wong, Osman, & Norizuki, 2011; Rutter, Freedenthal, & Osman, 2008).

The Coping Strategy Usage Questionnaire (CSUQ; Melvin & Gresham, 2016b), developed for use within the current study, was designed to measure the frequency of suicidal participants’ engagement with particular suicide‐related coping strategies within the previous month. The CSUQ was only completed by those who reported the presence of suicidal thoughts within the previous month. Participants rated the frequency with which they used specific coping strategies to help manage suicidal thoughts, using a four‐point Likert scale ranging from “not at all” to “a lot.” Coping strategies included examples of behaviors consistent with each section of the BeyondNow app.

Procedure

Approval to conduct the study was obtained from the university and health service human research ethics committees. Monash Health Mental Health Service treatment teams (N = 9) were contacted by phone or email to provide them with information regarding the study and all agreed to participate. Clinicians within each team were informed of the study protocols. Clinicians then identified potential participants, provided them with information about the study, and determined participant interest. All consenting participants completed the baseline assessment and collaboratively developed a safety plan that was documented using the BeyondNow smartphone application together with a researcher and treating clinician. Participants were provided information about safety planning and use of the BeyondNow app and were assisted in setting up a personalized safety plan. Postintervention assessments were planned for eight weeks after baseline; however, postintervention assessments ranged from 8.71 to 18.29 weeks (M = 10.42, SD = 2.20).

Data analysis plan

Data were analyzed using Stata version 13. A series of random‐effects regression analyses were performed using the generalized least squares estimation. This analysis technique allows the analysis of incomplete participant data without the use of the overly conservative last observation carried forward method. Given the range in duration between baseline and postintervention assessments, the variable “Time” reflects the number of days between baseline and postintervention for each participant. Although significant reductions in both severity and intensity of suicide ideation were observed over the trial period, these factors were highly correlated at baseline (r = .76). As such, the variable “Intensity of Ideation” (which involved frequency and controllability of suicidal thoughts) was used exclusively in regression analyses, as it was deemed to be more related to the primary outcome variables of suicide coping and suicide resilience. Means and standard deviations for baseline and postintervention variables were calculated. Longitudinal regression analyses were conducted to model primary variables. Severity and intensity of ideation were modeled as function of time, age, and gender, and SRCS and SRI variables were modeled as a function of time, intensity of ideation, age, and gender. t‐Tests were used to examine change in frequency of suicide‐related coping strategy use. Thematic analysis was undertaken on the qualitative responses within the App Feedback Survey, using the six‐step guidelines set out by Braun and Clarke (2006).

Results

Baseline characteristics

Participant characteristics of the sample (N = 36) at baseline can be seen in Table 1. Over 80% of participants presented with a depressive disorder and about a quarter experienced an anxiety disorder. Almost three‐quarters of the sample at baseline reported a lifetime suicide attempt, and just over half of the sample reported attempting suicide in the 2 months prior to baseline assessment.

Table 1

Demographic and characteristic information of the sample at baseline

	All participants N = 36
Gender, % female (n)	66.7 (24)
Age, mean (SD)	19.81 (6.02)
Psychiatric diagnosis
Depressive disorder, % (n)	80.5 (29)
Anxiety disorder, % (n)	27.7 (10)
Borderline PD/traits, % (n)	22.3 (7)
Eating disorder, % (n)	16.7 (6)
Trauma disorder, % (n)	8.3 (3)
Treatment team
Adolescent inpatient, % (n)	36.1 (13)
Short‐term residential, % (n)	36.1 (13)
Mixed adult/youth inpatient, % (n)	11.1 (4)
Adult outpatient, % (n)	8.3 (3)
Medium‐term youth outpatient, % (n)	5.6 (2)
Intensive youth outreach, % (n)	2.8 (1)
Suicide attempt
Lifetime, % (n)	69.4 (25)
2 months pretrial, % (n)	52.8 (19)
Nonsuicidal self‐injury
Lifetime, % (n)	80.6 (29)
2 months pretrial, % (n)	72.2 (26)
Phone type
Apple iPhone, % (n)	75 (27)
Android, % (n)	25 (9)

Fourteen of the 36 participants did not complete the postintervention assessment (attrition = 38.9%). One participant reported a suicide attempt within the previous 24 hours of the postintervention assessment (serious adverse event). The assessment was discontinued and the participant was immediately referred for crisis support. t‐Tests and Fisher's exact tests were used to examine baseline differences between those participants who completed the postintervention assessment and those who did not. Gender, baseline severity, and intensity of suicide ideation, and 2‐month and lifetime suicide attempt status were not associated with postintervention completion. However, participant age was associated with postintervention completion (completed mean age = 20.95, non‐completion mean age = 18.00).

Feasibility

App usage

Including the initial set up, Google Analytics data showed that BeyondNow was accessed a total of 203 times across all participants (N = 36, M = 5.64), with an average session duration of 4.81 min. The frequency of entry into each section of BeyondNow, via the “Give me suggestions” function, free text, and phone directory can be seen in Table 2. Overall, participants entered between 2.78 and 6.28 entries, on average, for the warning signs, reasons for living, making the environment safe, and internal coping strategy sections. However, fewer entries were observed for the external coping strategy sections (people and places, friends and family, and professional contacts), with the average number of strategies/contacts for these sections falling under two. Around one‐third of plan entries for each section were inputted from the preprogrammed suggestions; the remainder were free text or from personal phone directories. The safety planning sharing function was used by two participants (5.6%), and while 11 participants (30.6%) pressed the emergency button, data are not available on whether these calls were actually placed.

Table 2

Safety plan components: total and average entries per component and frequency of methods of entry

Safety plan section	Method of entry			Total	Average
	Suggestions n (%)	Free text n (%)	Imported from contacts n (%)
My warning signs	75 (33.2)	151 (66.8)	n/a	226	6.28
My reasons to live	58 (39.2)	90 (60.8)	n/a	148	4.11
Make my environment safe	37 (37.0)	63 (63.0)	n/a	100	2.78
Things I can do by myself	80 (36.2)	141 (63.8)	n/a	221	6.14
Connect with people and places	26 (37.1)	22 (31.4)	22 (31.4)	70	1.94
Friends and family I can talk to	18 (28.6)	18 (28.6)	27 (42.9)	63	1.75
Professional support	6 (24.0)	10 (40.0)	9 (36.0)	25	0.69

Frequencies based on all participants (N = 36).

App feedback

The majority of participants reported using the app “occasionally” (63.6%) or “a lot” (13.6%), as opposed to “not at all” (0%) or “rarely” (22.7%). Most participants used the app to make edits to their safety plan (77.3%), and a majority used the app to access their plan when experiencing suicidal thoughts (81.8%) and/or during a suicidal crisis (68.2%). All but two participants reported using the BeyondNow app either when experiencing suicidal thoughts or during a suicidal crisis. One of those two participants reported that they did not experience suicidal thoughts during the trial and therefore would not have been expected to use the app. The other participant, although reporting experiencing prominent suicidal thoughts during the trial, only referred to the app to edit their safety plan, not while experiencing suicidal thoughts. One participant found the app “quite tricky” to set up, while the remainder found it either “very easy” (90.9%) or “not too hard” (4.5%). Similarly, with regard to navigating within the app, one participant found it “quite tricky” (4.5%) while the remaining participants found using the app “very easy” (81.8%) or “not too hard” (13.6%). Every participant (100%) at postintervention reported that they would recommend the app to a friend.

In response to the question “What things did you like best about the app?” thematic analysis revealed three main themes: Hope, Connection, and Utility. Hope was demonstrated by quotes such as “Including reasons to live”; “It's good to look at when you feel as though there's no hope”; and “the things to live for”, and Connection was illustrated by “Something to talk about with friends”; “being able to send your safety plan to people”; and “When you can't think clearly, you can look at the app and see who's there to get help from”. The third theme, Utility, captured the apps accessibility (e.g., “It's easy to access and always with me”; “having a plan on you at all times”), customizability (e.g., “Customizing the safety plan”; “being able to change things when I needed”), and ease of use (“I love how everything is so easy to access”; “Simplicity”; “The straightforwardness of the layout”).

With regard to the question, “How would you improve the app?” six participants did not provide a response. Thematic analysis based on the remaining responses revealed two main themes; “Nothing to improve” (e.g., “Perfect”; “It's great”) and “Additional features” (e.g., “Medication reminder”; “Alerts”; “A game, a clip, meditation, music or puzzles”; “Add a breathing exercise”).

Effectiveness

Suicide ideation

Means and standard deviations for the severity and intensity of suicide ideation variables are presented in Table 3. Regression analyses (see Table 4) revealed significant reductions in severity of ideation, and intensity of ideation, from baseline to postintervention assessment.

Table 3

Means and standard deviations for baseline and follow‐up measures

Measure	Baseline (N = 36) M (SD)	Postintervention (n = 22) M (SD)
C‐SSRS—severity of ideation	4.33 (1.04)	2.29 (1.85)
C‐SSRS—intensity of ideation	19.64 (2.91)	17.44 (5.37)
SRCS—total coping	57.58 (11.35)	68.14 (11.61)
SRCS—internal coping	22.86 (5.39)	27.41 (5.22)
SRCS—external coping	25.58 (4.77)	29.45 (4.94)
SRI 25—total resilience	81.61 (26.21)	96.64 (25.66)
SRI 25—internal protective	25.53 (9.81)	30.27 (11.25)
SRI 25—external protective	30.39 (10.85)	35.32 (7.98)
SRI 25—emotional stability	25.69 (8.81)	31.05 (10.04)

C‐SSRS, Columbia Suicide Severity Rating Scale; SRCS, Suicide Related Coping Scale; SRI‐25, Suicide Resilience Inventory‐25.

Table 4

Summary of random‐effects regression model predicting change over time in severity and intensity of suicide ideation, controlling for gender and age (N = 22)

	Severity of ideation	Intensity of ideation
Time (days)	−.03 (.00)***	−.03 (.01)*
Male	−.47 (.48)	−.33 (1.34)
Age	.00 (.04)	.11 (.11)

***p < .001, *p < .05 (two‐tailed).

Suicide‐related coping

Mean scores on internal, external, and total suicide‐related coping scales increased from baseline to postintervention (see Table 3.). Regression analyses (see Table 5) revealed a statistically significant increase over time for both the total suicide‐related coping scale and subscales. Furthermore, older age was associated with higher scores on the total suicide‐related coping scale, while age was not associated with the internal or external suicide‐related coping subscales. Intensity of ideation was significantly and negatively associated with each of the suicide‐related coping scales. Gender was not associated with the suicide‐related coping scales.

Table 5

Summary of random‐effects regression model predicting change over time in total coping, internal coping, external coping, total resilience, internal resilience, external resilience, and emotional stability, controlling for gender, age, and intensity of suicide ideation (N = 22)

	Total coping	Internal coping	External coping	Total resilience	Internal resilience	External resilience	Emotional stability
Time (days)	.09 (.04)*	.04 (.02)*	.04 (.02)*	.06 (.06)	−.00 (.02)	.04 (.03)	.02 (.02)
Male	.73 (3.65)	1.77 (1.75)	−.83 (1.66)	3.37 (9.25)	−.56 (3.44)	−.07 (3.73)	4.51 (3.30)
Age	.68 (.29)*	.25 (.14)	.26 (.13)	.45 (.74)	.18 (.27)	.30 (.30)	−.03 (.26)
Intensity of ideation	−1.13 (.38)**	−.44 (.18)*	−.39 (.18)*	−2.30 (.75)***	−1.01 (.25)***	−.78 (.35)*	−.53 (.28)

***p < .001, **p < .01, *p < .05 (two‐tailed).

Suicide‐related resilience

Although mean scores for all suicide resilience scales increased from baseline to postintervention (see Table 3), regression analyses revealed that improvements in suicide resilience were not statistically significant (see Table 5). Intensity of ideation was found to be significantly and negatively associated with scores on total, internal, and external resilience scales, but not emotional stability. Age and gender were not associated with scores on suicide resilience scales.

Frequency of suicide‐related coping

Suicidal thoughts were reported by 14 participants (64%) during the one‐month period prior to postintervention assessment. As such, only 14 participants completed the CSUQ. A significant increase in the frequency of suicide‐related coping strategy use was observed between baseline (M = 22.29, SD = 4.61) and postintervention (M = 27.29, SD = 4.45) assessments; t (13) =−3.56, p < .01.

Discussion

The first aim of the current study was to examine the feasibility of integrating the BeyondNow safety planning smartphone application into an Australian tertiary mental health service. Overall, the results were very encouraging. The majority of participants reported accessing the BeyondNow app during the study, either to edit their safety plan or to view it to manage their suicidal crisis. Consistent with self‐report app use, Google Analytics data revealed that the app was accessed over 200 times by participants across the trial. It should be noted, however, that even a single viewing of the BeyondNow app could be sufficient to help an individual cope with a surge in suicidal thoughts. Indeed, one participant in the study reported during the postintervention assessment that they had only used the app on one occasion when they were intent on initiating a suicide attempt, which was enough to de‐escalate their intention. As such, the finding that the app was accessed by a majority of participants is a more pertinent indicator of app feasibility, as opposed to the sheer number of times the app was accessed.

Furthermore, Google Analytics data revealed that participants entered a range of entries into the sections of the app. The highest number of entries was observed within the warning signs and internal coping sections, and fewer entries were entered into the external coping strategy sections. Given the high suicide risk among participants, and the relationship between low levels of connectedness and suicide ideation (Klonsky & May, 2015; Kuramoto‐Crawford, Ali, & Wilcox, 2016), it is not surprising that fewer connections with social supports were registered. On the other hand, it is also not yet clear whether a higher number of entries within a particular section translate to greater utility or effectiveness. While some participants reported that the use of BeyondNow fostered their connection with others, it was noted that only two users (5.5%) shared their plans with other people via the app. This finding suggests BeyondNow might highlight the importance of social support; however, further steps are required to expand social contacts and develop supportive relationships for individuals at risk of suicide.

The BeyondNow app was also shown to be feasible given the high level of usability, with a vast majority of participants reporting that they were able to set up and navigate the BeyondNow app without difficulty. Furthermore, all participants at follow‐up reported that they would recommend the app to a friend. Qualitative analysis also revealed a high regard for the app's utility. Consistent with previous work in this area (Bush et al., 2015; Larsen et al., 2016), participants in the current study reported that the accessibility and customization that is possible with a smartphone‐based suicide prevention tool is highly regarded. The desire for customization is demonstrated by the high number of free‐text plan entries compared with preprogrammed suggestions. App feasibility was also supported by the finding that participants developed a sense of hope and connection from using the app. According to suicide theory (e.g., Klonsky & May, 2015), these are both important constructs which are inversely associated with suicide risk. The degree of hope and connection which appears to have been engendered by the app may have been facilitated by its ease of use and customizability. These findings provide support for the use of app‐based suicide prevention tools as a possible way to overcome some of the barriers to accessing help among people who are experiencing suicidal thinking.

With regard to what might be improved upon in BeyondNow, some participants indicated that they did not want the app to change, while others suggested additional features. BeyondNow was deliberately developed to have a single focus on documenting an accessible safety plan, which contrasts some user feedback that other tools could complement an individual's safety plan such as reminders, alerts, music, and relaxation tools. The simplicity of BeyondNow's design was intentional and aimed to provide a clear path to follow in the event of a suicidal crisis. While the inclusion of too many additional features may dilute or distract from the primary purpose of BeyondNow, feedback suggests that complementary strategies that may engage users ought to be considered.

The second aim of the current study was to examine the effectiveness of the App + TAU intervention. In line with the hypotheses, the results showed that the intervention was associated with increased knowledge of, and confidence in using, strategies to help cope with suicidal thoughts. Developing and using a safety plan in the context of tertiary mental health treatment appears to have an effect on suicide‐related coping over the duration of the current trial. Learning new practical ways to cope with suicidal thoughts could be expected to be a relatively quick process and speaks to the potential efficiency of the Safety Planning Intervention. The results also showed that the frequency of engagement with adaptive strategies to help cope with suicidal thoughts increased over the trial. However, considering that some patients were discharged on the day of study commencement, while others remained in treatment for the duration of the study, the degree to which safety planning alone is implicated in changes in suicide‐related coping observed is not known.

Contrary to expectations, no change in suicide resilience was observed across the trial period, which contrasts the related construct of suicide‐related coping. It is possible that suicide resilience might change slowly over time, perhaps with insufficient change detectable over the period of the current trial. To the authors’ knowledge, no studies have examined the rate of change of suicide‐related resilience factors. It was also hypothesized that the App + TAU intervention would be associated with reduced suicidal thinking. The results support this prediction, showing a significant reduction in the severity and intensity of suicide ideation across the trial period. This finding is encouraging and may point to the capacity of a plan to contain or de‐escalate thoughts of suicide via the use of coping strategies. The precise mechanism(s) via which safety plans have their effect might involve an increase in coping ability, although this requires further investigation.

The current study has several limitations. The study was an open trial of feasibility and effectiveness of BeyondNow plus TAU and therefore lacked a control group. Ideally, future studies would conduct randomized controlled trials to examine whether changes in suicide coping and suicide ideation associated with app use are above and beyond that of TAU alone. However, given the integration of SPI into best practice guidelines, ethical issues are raised regarding the appropriateness of withholding a safety plan from a suicidal patient. While app usage data were collected for all participants, the current study experienced a high degree of attrition post intervention assessment. Although age was the only baseline variable associated with study completion, it is possible that other factors relating to app use or opinions on the app were associated with participant dropout. Given the high attrition rate, it is possible that the current findings were impacted by a nonresponse bias with less satisfied participants perhaps being less likely to complete follow‐up assessment and report their dissatisfaction.

Future studies might go beyond aggregate‐level data and investigate individual‐level app usage. For example, an individual's app usage might be examined as a function of severity or intensity of suicidal thoughts, perhaps at shorter intervals than those used in the current study (e.g., weekly). Individual app usage might also be investigated in terms of a dose effect on suicide‐related coping or intensity and frequency of suicidal thoughts. Future research may be enhanced by the capacity of smartphone apps to automatically collect usage data and survey participants in a low‐burden fashion.

Overall, the BeyondNow safety planning smartphone app was shown to be feasible and effective as an adjunct to mental health treatment among patients at risk of suicide. The current findings are encouraging, although future research should attempt to disentangle the impact of safety planning from other treatment strategies.