Clinicians' perceptions of PTSD Coach Australia.

INTRODUCTION: Applications or "mobile apps" are a potentially important source of assistance for serving and ex-serving Defence members with mental health problems. PTSD Coach Australia is a modified version of an application developed by the US Department of Veteran Affairs. Clinician perceptions of mobile apps are important as they influence the dissemination and adoption of apps. This study aimed to explore the perceptions of PTSD Coach Australia by clinicians with experience in assisting Defence members with mental health problems.
METHOD: The study involved two samples of participants who were asked about their perceptions of PTSD Coach Australia. The first involved 33 clinicians who participated in one of five focus groups. The second comprised 30 clinicians who were individually interviewed by telephone. Qualitative responses to questions regarding PTSD Coach Australia were analysed to identify representative themes. Participants in the focus group sample also rated the app on the user version of the Mobile Apps Rating Scale (uMARS).
RESULTS: On the uMARS, clinicians rated the mobile app's subjective quality as 'average' to 'good'. Participants generally saw the app as a useful to help track symptoms, improve engagement and help implement strategies between sessions. However, they also expressed concerns with the app not being user-friendly (e.g. too wordy, poor layout/navigation) and having technical issues (freezing or crashing on Android devices). DISCUSSION: PTSD Coach Australia is generally seen as being acceptable and useful by mental health clinicians. However, it is important to include their concerns in future developments of PTSD Coach Australia and similar mobile apps in order to maximize their utilisation in Defence members.

Introduction

An estimated 22% of Australian Defence Force (ADF) members experienced a mental health disorder in the previous year (Van Hooff et al., 2014). This includes 9.5% with an affective disorder, 14.8% with an anxiety disorder and 5.2% with an alcohol use disorder. These estimates are also comparable to the rates of mental disorders in the military of other nations such as the USA (Kessler et al., 2014). One particularly common mental disorder within the ADF is post-traumatic stress disorder (PTSD) with estimate of prevalence ranges from 3% to 28% depending upon the number of traumatic events exposed to (Waller et al., 2012). Such estimates of the prevalence of PTSD in Australian Defence members are consistent with meta-analytic findings for other military populations (Xue et al., 2015). These high rates of PTSD and other mental health problems in Defence members highlight the need for effective interventions and support systems.

The combination of high prevalence and the geography of Australia pose significant challenges to the provision of equitable access to effective mental health treatment and support (Corrigan, 2004). A partial solution involves digital mental health resources. In the recent Mental Health Action Plan 2013–2020, the World Health Organisation (WHO) recommended an expansion of treatment modes and called for the inclusion of electronic and mobile health technologies to promote self-care and support greater treatment efficiency. Similarly, Australia's Fifth National Mental Health Plan (Council of Australian Governments' Health Councils, 2017) has digital mental health resources as a key component, especially for mental health conditions of mild to moderate severity. Smartphones provide a particularly accessible means of supporting self-management because of their high uptake and potentially constant accessibility (Anthes, 2016; Chan et al., 2017). Smartphone apps therefore are a promising option in helping to manage mental health problems such as PTSD in Defence members. While there is a wide range of mental health applications (i.e. apps) to support serving and ex-serving Defence members (Weingardt and Greene, 2015), there is limited evidence about their practicability and effectiveness.

In 2011, the United States Department of Veterans Affairs developed the PTSD Coach app. The Department of Veterans Affairs (DVA) in Australia subsequently received permission to replicate PTSD Coach and modify it for an Australian audience. The resultant PTSD Coach Australia was jointly released by DVA and the Australian Department of Defence in February 2013. This mobile app includes sections such as Learn (information about PTSD and professional care), Self-Assessment (monitoring symptoms of PTSD), Manage Symptoms (explanation of a range of symptom reduction strategies), Find Support (contact details for support) and Schedule Reminders (scheduling self-assessment, use of strategies and appointments). Since the release of the Australian app, some evaluations have been published on the original US version (Kuhn et al., 2014; Erbes et al., 2014; Miller et al., 2016). That research suggests that PTSD Coach is an acceptable resource for self-management of posttraumatic stress symptoms (Keen and Roberts, 2017; Miner et al., 2016). However, to date there is no published evidence on the acceptability and perceived utility of the PTSD Coach Australia app.

Perceptions of digital mental health resources by clinicians are important, both because their dissemination, adoption and use can be compromised if they do not support it (Kuhn et al., 2016; Kuhn et al., 2014), and because of the app's potential to be integrated with other treatment to increase its impact (Possemato et al., 2016; Possemato et al., 2017). There is also recent evidence that clinician-supported use of the US version of PTSD Coach improves engagement with the app and significantly increases its impact compared with purely self-guided use of the app (Possemato et al., 2016). Yet the research into clinician perceptions of smartphone apps is limited. The main two military-focused mental health apps that have clinician data are PE Coach, which supports exposure treatment of PTSD, and CBT-I (Kuhn et al., 2014; Kuhn et al., 2016; Kuhn et al., 2015) for treatment of insomnia.

This study therefore aimed to fill a gap in existing research, by examining the perceptions of the acceptability and utility of PTSD Coach Australia for serving and ex-serving Australian defence members with PTSD by mental health clinicians who frequently work with these Defence members. This study was conducted in parallel with a similar study examining the perceptions of PTSD Coach Australia by Australian Defence members (Shakespeare-Finch et al., 2020).

Methods

Participants

A total of 33 mental health clinicians were involved in focus groups, and a further distinct group of 30 mental health clinicians undertook individual telephone interviews. The participants in the focus groups were similar to the participants in the telephone sample in age (M = 47 years, SD = 10.5 years and M = 45.7 years, SD = 10.5 years respectively; t(61) = 0.47, p = .63), in years of professional experience (M = 15.7 years, SD = 11.6 years and 15.8 years, SD = 9.8 years respectively; t(61) = 0.033, p = .97), and percentage of clients experiencing PTSD symptoms (M = 43%, SD = 33% and M = 57%, SD = 27% respectively; t(61) = 1.83, p = .07). There was also no difference between the two samples in terms of gender, occupation, or percentage using an iPhone or Android smartphone, with most participants being female, psychologists, and using an iPhone (see Table 1). The two samples did however differ in the type of organisation they worked for and their work location, with those in the focus group being more likely to work for a government funded agency and to work in capital city (see Table 1). The higher percentage of focus group participants working in a capital city was as anticipated, given the need for participants to travel to a capital city location to participate in the groups.

Table 1

Participant demographics in focus groups and telephone interviews.

	Focus groupsn (%)	Telephone interviewsn (%)	Chi-square
Gender			0.46
Male	9 (27%)	6 (20%)
Female	24 (73%)	24 (80%)
Occupation			7.79
Psychologist	15 (46%)	21 (70%)
Social worker	7 (21%)	4 (13.3%)
Nurse	3 (9%)	2 (6.7%)
Counsellor/rehabilitation worker	1 (3%)	1 (3.3%)
Occupational therapist	1 (3%)	1 (3.3%)
General practitioners	4 (12%)	0
General medical officers	2 (6%)	0
Organisation			26.85⁎⁎
Private practice	2 (6%)	12 (40%)
Non-government agencies	4 (12%)	7 (23%)
Total government-funded agencies	24 (73%)	7 (23%)
Open Arms (VVCS)	14 (42%)	4 (13%)
Australian Defence Force	8 (24%)	1 (3%)
Other government department/agency	2 (6%)	2 (7%)
‘Other’ services	3 (9%)	6 (20%)
Work location			6.69⁎
Capital city	27 (82%)	16 (54%)
Other urban	6 (18%)	12 (40%)
Rural or remote	0	2 (7%)
Type of phone used by clinician			0.003
Apple phone	24 (73%)	22 (73%)
Android phones	9 (27%)	8 (27%)

Focus group total N = 33; individual telephone sample total N = 30; VVCS: Veterans and Veteran's Family Counselling Service, now Open Arms—Veterans & Families Counselling.

p < .05.

p < .001.

There was also a significant difference in the prior exposure to PTSD Coach Australia between these two samples, with more participants in the telephone interview sample tending to have downloaded the app for a longer period prior to the study (Table 2). While there was no difference between the samples in the percentage of participants recommending digital mental health programs to their clients in the past month, those in the individual telephone sample were more likely to have done so in the past 12 months, and to recommend PTSD Coach Australia to their clients in both the past month and year (see Table 2).

Table 2

Participant experience with PTSD Coach Australia in focus groups and telephone interview sample.

Past app experiences	Focus groupn (%)a	Telephone interviewsn (%)a	Chi-square
When first downloaded PTSD Coach Australiab			16.82⁎
More than 12-months previously	5 (16%)	15 (52%)
In the last 12-months	1 (3%)	5 (17%)
In the last 6-months	2 (6%)	2 (7%)
After they heard about the study	13 (42%)	7 (24%)
Never downloaded before interview/focus group	10 (32%)	1 (3%)
Clinician past recommendations to clients (yes/no)
PTSD Coach Australia in previous 12 months	11 (33%)	24 (80%)	13.86⁎⁎
PTSD Coach Australia in previous month	2 (6%)	17 (57%)	19.11⁎⁎
Other digital mental health programs in past 12 months	25 (76%)	29 (97%)	20.65⁎⁎
Other digital mental health programs in past month	20 (61%)	23 (77%)	1.87

Focus group total N = 33; individual telephone sample total N = 30.

31 focus group participants and 29 telephone interviewees responded to this question.

p < .01.

p < .001.

Design

The study involved two qualitative methodologies (focus groups and individual semi-structured telephone interviews) and a quantitative methodology (completion of the user version of the Mobile Apps Rating Scale, uMARS; Stoyanov et al., 2016). Given the relative strengths and limitations of both approaches, the use of both methodologies within the one study (method triangulation) can result in a broader understanding of the phenomenon of interest and an increase in trustworthiness (Carter et al., 2014).

Recruitment

Clinicians were recruited from multiple disciplines (including general practitioners, psychologists, allied mental health practitioners) and multiple services (including the government funded veteran counselling services, an army barracks health service, non-government organisations, private inpatient and outpatient psychiatric hospital services, and private practices). All clinicians had experience treating serving and ex-serving defence members with trauma symptoms. Information about the project was circulated through each of the above networks including by word of mouth, group emails and social media. If a participant contacted the research team and indicated an interest in the study, further information was provided via email. Potential participants were also contacted via phone or email prior to the focus group or individual telephone interview to answer any questions about it, and to provide details regarding its time and location. The same recruitment process was used for the telephone interviews, except that this methodology allowed the recruitment of participants across Australia rather than just those geographically close to the researchers thereby allowing the research team to examine the generalisability of the findings from clinicians able to attend the focus groups to clinicians throughout Australia.

Procedure

Each focus group included 4–10 participants and lasted approximately 2 h. Focus groups were facilitated by two members of the research team including a registered psychologist. Lunch was provided and participants were reimbursed for travel expenses. All sessions were audio recorded. The focus groups started with a brief ice-breaker activity and introduction to PTSD Coach Australia including its purpose and target population, followed by individual exploration of the app for 10–20 min. A focus group discussion was then facilitated by one of the research team members and involved addressing the following topics: 1) the participants' overall experience of using the app, 2) specific characteristics of the app that were most and least helpful, 3) what role (if any) the app may have in supporting people with trauma-related symptoms, 4) potential barriers and enablers to its use, and how these might be addressed, 5) how the app might be improved, 6) how the app might be used in clinical practice. At the end of the focus group, the participants completed a demographic survey.

The procedure for the telephone interviews was identical to workshops except that the session did not start with an ice breaker activity. In addition, telephone interview participants were asked to use the PTSD Coach Australia app for 2 weeks prior to the interview, as there was no opportunity for them to explore the app during the interview itself.

Ratings of app

Participants in the focus groups completed a user version of the MARS (uMARS; Stoyanov et al., 2016). The uMARS is a 23-item scale assessing objective smartphone app quality on four subscales (Engagement [how entertaining, interesting interactive the app is], Functionality [performance, ease of use, navigation, gestural design], Aesthetics [layout, graphics, visual appeal] and Information [quality and quantity of information provided and credibility of sources]) with items rated from 1, very poor to 5, excellent. Subjective quality was assessed using four questions (Would you recommend the app?—from 1, not to anyone, to 5, everyone; How many times would you use it?—from 1, 0 times to 5, >50 times; Would you pay for this app? No/Yes; Overall star rating, from 1 to 5 stars). All subscales of the uMARS have been shown to have acceptable internal consistency (Engagement α = 0.80; Functionality α = 0.70; Aesthetics α = 0.71; Information α = 0.78; Satisfaction α = 0.78; Stoyanov et al., 2015). The uMARS total score has good test-retest reliability with levels of ICC of 0.66 and.70 over 1- to 2-month and 3-month periods, respectively (Stoyanov et al., 2015). Levels for all subscales scores were also satisfactory. The participants rated the UMARS as how useful the app would be for the client from the clinician's perspective. In addition, participants reported their likelihood of recommending PTSD Coach Australia and other digital tools to clients within the next 6 months (each rated 0–100%).

Data analysis

Quantitative descriptive analyses on the uMARS, and demographic data were conducted using IBM SPSS™ Statistical version 25. The frequency of responses to questions was compared across samples using chi-square analysis. The samples were compared on continuous measures using t-tests. There were no missing data in this study. The same qualitative data analysis plan was used for both the focus groups and the individual telephone interviews. Audio recordings of all sessions were professionally transcribed. Themes were extracted from the data and grouped under the six questions posed adhering to Braun and Clarke's (2012) Thematic Analysis approach. The analysis within each question allowed exploration of the clinician's perceptions for each question. The analysis was conducted by a single coder with many years of conducting and publishing qualitative research. Analysis involved initially becoming familiar with the data through reading and re-reading transcripts and taking notes in the margins. Subsequent readings involved a process of constant comparison within the transcript and then between transcripts, combining notes into lower order themes that were eventually grouped into higher order themes. This approach focused on patterns of meaning across the participants. Validity strategies included keeping meticulous notes on all decision making, triangulating interpretations and descriptions of codes and themes across the research team, and triangulating the results from multiple data collection methods (e.g., the focus groups were audio recorded, while the facilitators also took notice of comments, and photographs were taken of group feedback to the questions that was summarised on the whiteboard).

Results

uMARS outcomes

The app received ‘average’ to ‘good’ ratings on all four uMARS objective app quality subscales (see Table 3). A comparison of their ratings with those of 25 Defence members in the companion paper (Shakespeare-Finch et al., 2020) showed that ratings were comparable except for Aesthetics, which was rated somewhat lower by clinicians than by Defence members (M = 3.91, SD = 0.67; F(1, 56) = 3.89, p = .016; η2 = 0.100).

Table 3

Clinician ratings of PTSD Coach Australia on the uMARS.

		Mean (SD)	Range
Objective subscalesa	Engagement	3.43 (0.62)	1.80–4.60
	Functionality	3.95 (0.77)	2.25–5.00
	Aesthetics	3.38 (0.87)	1.00–4.67
	Information	4.04 (0.62)	2.67–5.00
	Overall mean	3.69 (0.59)	2.20–4.75
Subjective subcalesa	Subjective quality	3.27 (0.90)	1.00–4.75
	Perceived impact	2.88 (1.18)	1.00–5.00

All uMARS items are rated on a 5-point scale from 1—very poor, through 2—poor, 3—average and 4—good, to 5—excellent. The subscales average ratings across items.

At the end of the focus groups or the individual telephone interviews, the two samples did not differ in their likelihood to recommend or use other digital mental health programs or tools in the next 6 months (Focus Group M = 74.2%, SD = 22.4%; Individual Telephone Sample M = 83.1%, SD = 25.8%; t(1,61) = 1.46, p = .15). However, the samples did differ in their likelihood to recommend or use the PTSD Coach Australia app in the next 6 months (Focus Group M = 59.8%, SD = 30.5%; Individual Telephone Sample M = 81.5%, SD = 24.1%; t(1,61) = 3.15, p < .01).

Themes of clinician perceptions of app

The main themes for each of the questions asked of both samples are summarised in Table 4 below and presented in the sections below.

Table 4

Thematic map for qualitative analysis of clinician perceptions.

Question topic	Themes
Clinicians' knowledge and experience of the app
Focus groups	Technical issues with the app, e.g. crashing
Individual telephone sample	Found helpful adjunct to therapy in educating client to techniques and increasing engagement
Characteristics of the app that were likely to be most helpful
Common across samples	Helpful functionality: personalisation, self-assessment symptom history tracking, information sharing with physician, available free of cost, breathing and guided imagery strategies, scheduling reminders, discrepancy on providing rationale for strategy before starting.
Characteristics of the app that were likely to be least helpful
Common across samples	Americanised inspirational quotes; aesthetics: boring, text heavy; poor layout/organisation; name of app indicates only for those with PTSD; least helpful sections were learn, self-assessment and sleep management.
How the PTSD Coach Australia might be improved
Common across samples	Improve navigation; improve aesthetics; add section for family; change language/tone; add more strategies for sleep management
Enablers and barriers to using PTSD Coach Australia
Common across samples	Enablers: FreeBarriers: technical glitches e.g. crashing; poor technical literacy of users; concerns about confidentiality; attributes of users; difficulty of using during acute distress; need for different language for serving and ex-serving Defence members
What role might PTSD Coach Australia have in supporting people with trauma-related symptoms
Common across groups	Educate users about mental health problems and psychological therapy
Focus groups	Reservations about use with acute distressed clients
Individual telephone sample	Symptom management; Normalise symptoms
Under what circumstances would clinicians use PTSD Coach Australia in treatment
Common across groups	To facilitate between session engagement
Focus groups	Only use if client has sufficient technological literacy
Individual telephone sample	Use in initial stages of therapy

Clinicians' knowledge and experience of the app

Clinicians in the focus groups who were more experienced using apps tended to make negative comparative statements between PTSD Coach Australia and other apps. When asked about their overall experience using PTSD Coach Australia, focus group participants expressed concerns about technical issues: “…the app actually froze, and if I was someone in the process of being triggered, I probably would have smashed the phone.” Another participant said, “when the original PTSD Coach [USA] came out, I got a little feedback from clients saying they liked it. The Australian one not as much so because of the glitches in it.”

In contrast, clinicians who were interviewed individually tended to have a more consistently positive view of the app, for example: “I really, really liked it. I thought it was a nice, all-in-one component”. Within the telephone interview sample, participants who had used PTSD Coach Australia in their clinical practice (60%), discussed their experiences of using the app as an adjunct tool to their individual therapeutic sessions, with some also introducing it in group therapy. The app was generally used to educate about therapy techniques, particularly different types of relaxation for arousal reduction, and to demonstrate exercises during their session. They used the app to increase engagement with clients, to consolidate the learning from the session and to set reminders for their clients to practice their strategies as homework. For example, one clinician shared “So, the app, is a handy way to teach people symptom management strategies… everyone has a phone, including me. I can show them the app, and download it, and then we can practise some of the symptom management stuff together.”

Characteristics of the app that were most helpful

A common theme across both focus group and interview participants was the functionality of the app. and the ability to individualise it: “what I like is that you can add your own music, you can add your own pictures”. In addition, both groups reported that the ability to track wellbeing/symptoms and being able to share that information in therapy sessions was helpful in increasing self-awareness. For example, one participant said “a person may be able to reflect on their progress or areas for improvement and say yes, I'm steadily getting better, or hey, look, I've had a bit of a tough month but, to be able to use that in therapy…would be great, to challenge the client”.

The focus group reported a unique theme that an advantage of the app was the option allowing the sharing of the assessment results with clinicians. There was also an almost unanimous agreement amongst the focus group participants that a key strength of the app was that it was free, easy to access on a client's phone and combined a variety of techniques in one place. Other helpful aspects included the ability to set reminders.

Participants who were interviewed over the phone provided more specific positive feedback about the functionality of the app being the most helpful characteristic. Manage Symptoms, Learning, Subjective Units of Distress (SUDs), Self-Assessment, and Support contacts were identified as the most helpful modules.

Characteristics of the app that were least helpful

A common theme across both samples was that the Americanised inspirational quotations were a drawback, since they may not be as relevant to Australian users. For example, “You only stand with your hand on your heart, stars and stripes flying in the background if you are American …I stopped after 3 quotes”. Other comments included, “yeah some of the inspirational quotes are pretty silly and they lack inspiration”.

The aesthetics of the app were also commonly criticised by clinicians in both groups. Its appearance was predominantly seen as boring and its content was outdated. Many clinicians suggested that the app was too text-heavy.

“I found the interface kind of clunky…it didn't feel particularly user friendly…it was too wordy. There were lots and lots of words everywhere. The techniques that are in there are fine but I didn't feel that it added value in terms of encouraging me to use it over other apps that I do use”.

In addition, concerns were expressed about the layout or organisation of the app, resulting in users having to go through so many screens when they wanted a specific tool. Further, the fact that each management tool began with the Subjective Unit of Distress scale was perceived as potentially confusing, perhaps causing users to think that they were being taken back to the same page.

There was also a general perception that the name of the app might suggest that it is only for people with a diagnosis of PTSD rather than for all serving and ex-serving Defence members. For some clinicians the term coach was also problematic, “I guess the idea of you trying the app of PTSD Coach is to coach people through how to manage their symptoms. I personally didn't feel the app did that”.

Regarding sections of the app, some said that Learn section was least helpful, due to simplified psychoeducation. Some nominated Self-assessment as least helpful, due to the potential for patients becoming overly focussed on self-assessment. Others found the sleep management techniques introduced in the app, together with some of the pictures and positive affirmations unhelpful.

There were no unique themes for the two samples with this question.

How the PTSD Coach Australia might be improved

A common recommendation from both samples was the need for an improvement in the app's navigation. For example, there was a frequent request for the ability to shortcut to favourites, instead of having to trawl through many screens, or to have the option to scroll. Reduced text would also be preferred. For example, “I'm thinking having a cleaner design. Fewer words. Maybe more pictures that help you to understand what's behind each section and simplifying what's in the app as well...” Further improvements included the addition of a progress bar so that users know how long each task is. While information about the length of each exercise is stated at the beginning, many clinicians skipped over that because they considered it as more text.

The clinicians recommended improvement for the app's aesthetics. As noted already, most thought it was boring, stating that it was “bland or grey and white or pedestrian or, all the text looks the same”. It should also be tailored to the audience. “Even the icons…I think if they were tailored to the specific audience, people would have a sense that this is important to them”. Common suggestions were, “I would love for that information to be in videos rather than text” and “to have that image as big as possible”. Suggestions for reducing the amount of text included adding voice activation and voice recordings for sight impairment. In addition, suggestions for improving the readability included changing the font, increasing font size and using different background colours.

Several participants noticed a lack of references to the family, which they regarded as a serious omission. While users were asked if they were a family member, there was no helpful information offered to them. Both participant groups said that a section with education for the partner and family could be useful.

The language or tone of the app was sometimes seen as problematic. For example, a clinician expressed, “Well I think it's better off to say you might have PTSD-like symptoms, but they usually disappear after a couple of weeks… normalising those reactions instead of calling them a disorder in the first period.”

Most participants also had suggestions for including more comprehensive strategies in various sections including sleep management.

There were no unique themes for each of the two samples.

Enablers and barriers to using PTSD Coach Australia

Consistent with previous responses, the main enabler to the greater use of PTSD Coach Australia was its availability at no cost. Common barriers to using the app that were identified by both groups were its technical problems and the technical literacy required of users. Technical problems reported included the app crashing, particularly on Android devices. For example, “once I added three photos and the whole app crashed, and I had to start all over”. Most said that the app—as with other interventions utilising smartphones—would be useful to technologically minded people only and would not be useful for persons who “were not technology savvy, people who don't use smartphones or don't want to buy a smartphone or those who can't afford to buy a smartphone”. It especially could pose potential barriers for older ex-serving Defence members and to people with disabilities such as sight or hearing difficulties or literacy issues.

Another important barrier reported by both samples was user concerns with confidentiality. That is, there was a perception that many Defence members would be concerned about whether or what data were being shared with the ADF or Department of Veteran Affairs. One clinician gave an example that when they asked a client with severe PTSD to download the app between sessions, they came back and said: “no way, because they ask questions at the beginning…he was convinced the information is stored for the government”.

A further common barrier was a concern about how the attributes of Defence members may interfere with their use of the app. For example, one clinician said: “We've got people with attention problems …there's so much information for people who are struggling to concentrate anyway, it's just information overload”. Approximately half of the participants said their clients were “pretty impatient” or had “low frustration tolerance”, so the content needs to “be relevant. It's gotta fit me”. The clinicians saw this as critical because a failure to work for a client with PTSD the first time means: “this doesn't work for me, I'm never doing it again”. One participant succinctly summarised the sentiment by saying: “…for someone with those issues that they're dealing with, then they get less frustration tolerance. They could well pelt a phone across the room, just trying to find things.”

There was also common agreement that PTSD Coach Australia was difficult for a client to use when they were experiencing acute distress.

And he felt that, when he was in a crisis, he just didn't feel that he was able to adequately manage or, utilise the app because he, would just shut down all together and, didn't feel that he was able to even have the forward thinking or the foresight to access the app.

Other perceived barriers included the need for a different style of language for the serving versus ex-serving Defence members. For example, “when you are serving you follow orders, however when you are not, you baulk at orders”. The clinicians perceived the tone of the app as somewhat authoritarian and recommended that it be used with caution in veterans. Participants suggested that if the app was to be used by veterans it should start with something to encourage them to feel supported such as, “you've provided a wonderful service for your country…”.

There were no unique themes for the two samples.

What role might PTSD Coach Australia have in supporting people with trauma-related symptoms?

Consistent with responses to the previous question, most focus group clinicians expressed reservations in using the app with acutely distressed clients: “if you're in distress...I would not recommend this app to someone who was elevated because of difficulties in how to navigate through it, how to find what you need.”

Focus group participants also suggested that the app could have reminders at specific times of the day to cue self-monitoring, coping strategies or homework tasks. Users could chart well-being between sessions, and this information might then be used in therapy sessions.

Overall, clinicians who were interviewed by telephone agreed that PTSD Coach Australia could be beneficial in assisting people who experienced trauma, reporting that many of the strategies available in the app are helpful for symptom management. Participants in this sample tended to say that PTSD Coach Australia had the potential to normalise symptoms and experiences of people with PTSD as well as reduce stigma, especially if Defence members were still serving and wanted to be deployed.

Finally, those interviewed by telephone thought that PTSD Coach Australia could potentially educate users about the nature of psychology sessions and encourage them to seek professional help. Many suggested that it would be beneficial to educate Defence members about the existence of the app along with including information about it into briefing talks before deployment.

It could be used, perhaps pre-deployment, by making, medical professionals more aware of the app and- and its availability as a resource to support people at that point.

Under what circumstances would clinicians use PTSD Coach Australia in treatment

There was a general perception that the app might be used to help facilitate between-session engagement in the therapeutic process: “You know, ‘cause they only come one hour a week then, and that's all they've done, then it's a long therapeutic process”.

Clinicians in the focus groups stated that they would only use PTSD Coach Australia in treatment if they thought that the client had enough technological literacy to benefit from the app. With clients who were not fans of technology or lacked confidence in its use, they thought it would add to their stress levels. Some clinicians who had used the app with less app-literate clients said “they get lost in it”. Overall, few focus group participants stated that they would use PTSD Coach Australia, preferring other apps that are more engaging and perceived to be more useful and user-friendly.

The telephone interviewees tended to express fewer concerns about using the app with their clients. They said that they educated their clients about the existence of the app, tending to use it in initial stages of therapy as an extra resource, to engage clients in therapeutic work, reinforce strategies learnt, and consolidate education or provide more information about PTSD.

I think yeah, as an educational tool and an extra component to demonstrate some of the ideas and concepts. Rather than you know, trying to explain it or walk them through something and then they walk out the door and then you're not sure if they're going to be able to integrate it.

Discussion

This study aimed to explore the perceptions of acceptability and utility of PTSD Coach Australia by clinicians who regularly work with the mental health problems of Australian Defence members.

Clinician perceptions of acceptability

An understanding of clinician's perceptions of PTSD Coach Australia was gained by examining both clinician ratings of the app using uMARS as well qualitative themes of responses by clinicians. The clinicians' moderately positive view of PTSD Coach Australia was indicated by their mean ratings on uMARS being either equivalent to or higher than the scale's “Average” level, and in two areas (Functionality and Information) being at or near “Good”. These moderately positive ratings parallel the generally positive qualitative comments provided by the clinicians.

On average the participants in the individual telephone sample generally had a more positive attitude towards PTSD Coach Australia than the focus group participants. While the two samples were identical on most demographic variables, apart from their location and work context, it appeared that the individual telephone interview sample tended to have greater previous experience with PTSD Coach Australia. In addition, interviewees were asked to download and use the app for 2 weeks, whereas the first experience of the app by some focus group participants was during the focus group. It is possible that the greater previous experience reduced concerns about the app.

Clinician perception of utility

A common theme reported by the clinicians was that the app may assist with improving engagement in therapy and with the implementation of strategies between sessions. Consistent with their ratings on uMARS, participants expressed a liking for the level of information of the app. This included helpful information and techniques in the Manage Symptoms, Learning, SUDS and Self-Assessment sections.

A significant concern regarding utility related to the experience of technical problems, mainly involved the app freezing or crashing on Android phones. There were frequent expressions from participants that such technical glitches would not be easily tolerated in this population and would significantly reduce usability. There was also a concern about the perceived confidentiality of stored data, especially since many clients expressed a lack of trust about information being shared with government departments. Many participants did not view the app as aesthetically appealing, although the average uMARS rating for aesthetics was above the scale's “average” midpoint. The apparent incongruence of these two results is likely due to uMARS asking participants to rate layout, graphics and visual appeal. While the participants did express some concern about these attributes, their greatest expressed concern was about the app's wordiness. This was seen as particularly problematic given their perception that many Defence members experience attentional problems or low levels of frustration tolerance.

Both groups felt that the user-friendliness of PTSD Coach Australia would be very limited in specific client sub-populations. In particular, there was a common perception that the app was not useful for clients who were experiencing acute distress, including its sequential organisational structure, resulting it in being difficult for clients to quickly access soothing strategies. There was also a perception that use of the app would be challenging for those with less technical literacy or experience, such as older clients. While some clients may struggle with most apps, there was a perception that the app's layout and navigation may exacerbate the problem. This is an important issue given that perceived complexity is a significant predictor of low levels of intention by US Department of Veteran Affairs clinicians to use PE Coach (a smartphone app for prolonged exposure therapy in adults with PTSD) and CBT-I (a smartphone app for insomnia; Kuhn et al., 2014, Kuhn et al., 2016, Kuhn et al., 2015).

Comparison with perceptions of Defence members

This study was conducted in parallel with a study with a similar methodology that examined the perceptions of PTSD Coach Australia by 53 Australian Defence members (Shakespeare-Finch et al., 2020). Both clinicians and Defence members saw the app as primarily an adjunct to psychotherapy and viewed the fact that it was free as the main enabler to its use. Both valued the ability to track wellbeing and symptoms, while clinicians emphasised the ability to share this information in sessions. There was consensus that the app was too text heavy, and both groups identified technical problems when accessing through Android devices. Other similarities included the recognition of issues with the app's expression being a little authoritarian, which particularly may not resonate with ex-serving Defence members. Both groups also noted that users with limited technological facility may have difficulty using it, and that the current app may not be suitable for Defence members who were in crisis or had a low frustration tolerance. Both groups criticised the name of the app, although clinicians noted that it was too limiting, while Defence members tended to emphasise the risk of stigma. Both also saw a need for increased marketing.

An interesting difference in perception was that Defence members were somewhat more positive about the app's information and ease of use than clinicians, who had more critical—a difference that was also seen in the quantitative uMARS ratings of Aesthetics. Some Defence members liked the inspirational quotations, although others, together with most clinicians—were highly critical of their American tone. Some clinicians saw the information in the app as needing updating, while Defence members generally valued that aspect of the app. Conversely, clinicians generally mentioned the ability to individualise aspects of the app as a strength, whereas several Defence members criticised the limited degree to which this was possible. Overall, the combined voices of clinicians and Defence members suggested that PTSD Coach Australia had substantial strengths but needed significant updating and improvement.

Strengths and limitations

In order to maximize the methodological rigor of the study, the design involved the methodological triangulation of combining data from focus groups, telephone interviews, and uMARS ratings. Such a design incorporates the strengths and balances the biases of each method (Carter et al., 2014). For example, the individual uMARS ratings provided individual perspectives on the app but lacked the richness of the qualitative experiences. Within the focus groups, louder and more passionate members appeared to influence the tone and direction of discussions, whereas comments by participants in the individual interviews were not subject to that potential issue; conversely, the interviewees were not exposed to alternative perspectives on each question, which may have reminded them about similar or contrasting experiences and opinions. Differences in recruitment processes and location of interviewees and focus group members, the requirement that interviewees use the app for 2 weeks and the differing social influences within individual interviews and focus groups may have been responsible for some differences in the favourability of responses to the app, and it is possible that the sample as a whole was more positively disposed to the app than those who did not volunteer. However, substantial commonalities in responses were seen both within this study and with responses by Defence members in Shakespeare-Finch et al. (2020), which increases confidence in the results. Finally, another possible limitation with this study is the possibility of sampling bias. That is, it is possible that the participants who agreed to participate in the study may have been more motivated or held a more positive pre-existing view of PTSD Coach Australia than those who chose to not participate.

Implications

Findings from this study indicate that many mental health clinicians will find PTSD Coach Australia acceptable and useful in their therapeutic work serving and ex-serving Defence members. They also provided suggestions on how both this app, and other similar apps, may be improved to increase their perceived acceptability and utility in clinical practice. These results are important, given that the perception of clinicians is critical in the adoption and dissemination of health apps (Greenhalgh et al., 2004).A key implication of this study is the need to fix technical problems that were identified in use of the app on Android devices. Another implication is the need to improve the app's aesthetics by decreasing the amount of text, increasing the use of images or videos, and improving its visual design. A third identified need is to improve the app's layout and navigation. Participants of this study saw these issues as particularly important for Defence members with mental health problems, limited frustration tolerance or poor computer literacy.

Conclusions

While the Australian mental health clinicians in this study saw PTSD Coach Australia as a helpful adjunct to therapy with serving and ex-serving Defence members, they also identified a range of issues with the app, and ways it could be improved. While the focus of the study was on a specific app, aspects that these clinicians valued or criticised have wide potential application to other digital mental health resources for Defence members, and many are also relevant to other user groups.

Given the influence clinicians have on the use of digital tools, especially within therapy, studies of clinicians' views deserve additional attention. Just as research on co-development and acceptability of resources by end users may increase the acceptability and uptake of digital health products, the inclusion of clinicians' views has substantial potential benefit for their clinical content and utility, as well as offering indirect input from a wider range of potential end users than is typically included in standard co-development. As is shown in the current research, this extension to the definition of a resource ‘user’ can both strengthen confidence in conclusions from research with patients or community members and provide important additional perspectives.

Funding

This work was supported by the Department of Veterans' Affairs (tender MH EVAL 01/2016, 2016–18).

Acknowledgements and disclaimer

This project was funded by the Department of Veterans' Affairs (DVA MH EVAL 01/2016). Material in paper is the responsibility of the authors and does not necessarily reflect the views of the Australian Government. The authors acknowledge the guidance of other steering committee members, including Michael Correll, L-Col Kathryn Henderson, Maureen Montalban, Tracey Noble and Brian Riches, and the contributions of QUT staff, including Professor Ross Young, Dr. Madhavan Mani, Stoyan Stoyanov, Susan Lingard and Daniel Krosch. The assistance of the Open Arms (Veterans and Veterans Families Counselling Services), Mater Health Services North Queensland, Greenslopes Private Hospital, Garrison Health Services, Part Two, Career Shift, Go2 Human Performance, Mates4Mates, White Cloud, the Gallipoli Medical Research Foundation, Defence Families Australia and the Defence Community Organisation is also acknowledged, as is the time and contributions of participating clinicians.

Declaration of competing interest

None