The feasibility and effectiveness of Catch It, an innovative CBT smartphone app.

BACKGROUND: The widespread use of smartphones makes effective therapies such as cognitive-behavioural therapy (CBT) potentially accessible to large numbers of people. AIMS: This paper reports the usage data of the first trial of Catch It, a new CBT smartphone app.
METHOD: Uptake and usage rates, fidelity of user responses to CBT principles, and impact on reported negative and positive moods were assessed.
RESULTS: A relatively modest proportion of people chose to download the app. Once used, the app tended to be used more than once, and 84% of the user-generated content was consistent with the basic concepts of CBT. There were statistically significant reductions in negative mood intensity and increases in positive mood intensity.
CONCLUSIONS: Smartphone apps have potential beneficial effects in mental health through the application of basic CBT principles. More research with randomised controlled trial designs should be conducted. DECLARATION OF INTEREST: None. COPYRIGHT AND USAGE: © The Royal College of Psychiatrists 2016. This is an open access article distributed under the terms of the Creative Commons Non-Commercial, No Derivatives (CC BY-NC-ND) licence.

The use of mobile devices and technology has been a part of medical care.[1] In the UK, 93% of the population owns a mobile phone, and 61% owns a smartphone device that enables them to download apps and access the internet.[2] Cognitive–behavioural therapy (CBT) has good evidence as a psychological intervention for a wide range of mental health problems,[3] and considerable effort has been expended in increasing the accessibility of such therapies – through commissioning additional services and training more therapists[4] and by developing novel approaches such as computerised forms of therapy, available on a number of platforms such as webpages, DVDs and smartphone apps.[5]

There are around 14 000 healthcare apps available, of which around 6% are devoted to mental health.[5] The use of smartphone apps in physical healthcare has been seen as very successful,[6] and the National Health Service (NHS) currently has a health apps library of recommended and assessed apps covering a wide range of conditions, healthy living options and healthcare information sources such as mole tracking for cancer patients, a health mapper for long-term health problems, and symptom checkers.[7] Through randomised controlled trials, health apps have been found to benefit weight loss,[8] aid patient care of chronic disease,[9,10] increase positive health outcomes and communicate with patients with diabetes[11] and even increase monogamy and safe sex practice through HIV education.[12] Donker and colleagues[5] found evidence of significant benefits for users of a small selection of apps targeting depression, anxiety and substance misuse, and concluded that mental health apps have great potential in improving the accessibility of effective interventions. However, they also expressed concern over the amount of apps that claim to be mental health apps but are not based on robust evidence.

Method

Participants

Participants were an unselected, opportunistic sample taken from the population of people participating in an online course (MOOC; Massive Open Online Course) on the topic of mental health: Psychology and Mental Health: Beyond Nature and Nurture (www.futurelearn.com/courses/mental-health-and-well-being) available through the FutureLearn online learning platform and accessible to any member of the public, inside and outside of the UK. As part of the course, learners were able to download and use a new smartphone app – Catch It.

This study complied with the guidelines of the 1964 Declaration of Helsinki. Ethical approval was obtained from the University of Liverpool’s IPHS Research Ethics Committee in June 2014, reference number IPHS-1314-310.

The app

Catch It (http://appstore.liv.ac.uk/catch-it/) is an innovative smartphone app which uses the basic principles of CBT and has been developed jointly by the universities of Liverpool and Manchester. The app takes users through a process referred to as ‘catch it, check it, change it’. ‘Catch it’ aims to help the user identify thoughts and thinking styles associated with a shift in mood or a particular emotion. ‘Check it’ helps them reflect on and identify whether the thought processes are helpful in pursuit of general life goals they hold to be important. ‘Change it’ encourages them to generate other ways of thinking if the current state is not helpful, fully considered or proportionate.

The aim of the current study was to test the feasibility of the Catch It app as an aid to achieving well-being and mental health. To investigate this, we assessed usage rates, including how many people downloaded the app and how often it was used. The appropriateness of users’ entries was also assessed, exploring whether they were able to identify and change their thinking styles in the manner required for successful CBT. Finally, ‘before and after’ ratings of each user’s mood were analysed to assess whether the app is a feasible way of improving mental well-being or addressing mental health problems in a cost-effective manner for a large population.

App overview and procedure

The Catch It app ran on all Apple iOS or Google/Android smartphones and could be downloaded from the University of Liverpool app store, the Apple App Store and Google Play Store (http://appstore.liv.ac.uk/catch-it/). Selected screenshots of the app are presented in Fig. 1.

Fig. 1

Illustration of the Catch It app user interface.

On first using the app, the person selected a personal identification number (PIN) and was then taken through the relevant components of the app. Data generated by the app could only be shared with university researchers if the user actively opted into that choice (which means there are an unknown number of anonymous users of the app). In the first substantive element of the app – ‘Catch it’ – the user recorded their mood by selecting the emotion experienced from a drop-down menu (pre-loaded with ‘anxiety’, ‘happiness’, ‘depression’, ‘anger’, ‘relaxation’, ‘confusion’ and ‘other’, when the user could enter their own data), the strength or intensity of the mood (from a drop-down menu ranging from 1 to 5), the time that the emotion was felt (either by entering the date manually or by clicking ‘just now’ and automatically inserting the contemporaneous time and date), and the location (a drop-down menu of ‘home’, ‘work’, ‘shopping’, ‘socialising’ and ‘other’, when the user could insert their own choice of location).

Pressing ‘next’ led the user to the next substantive screen, ‘What happened?’, where they were informed that ‘the things that happen to us can affect our moods, so first, briefly note down what was happening when you felt this way’. Users entered their own free responses into a text box which used partially obscured (greyscale) text to clarify this by prompting, ‘What was happening when, and immediately before, you felt this way?’.

Pressing ‘next’ then took the user to the next substantive screen, ‘What were you thinking at the time?’, with information that ‘what you’re thinking can also affect your mood, so try to identify what thoughts you were having at the time’. Users entered their own free responses into a text box which used partially obscured (greyscale) text to clarify this by prompting, ‘What automatic thoughts were you having?’. Pressing ‘next’ then took them to the next substantive screen, ‘Take a moment to reflect on what you’re thinking’. Here, the user was presented with their data from the immediately preceding step (their ‘automatic thoughts’) with the prompt ‘Look at it a different way:’ and then pressing ‘next’ took them to the penultimate screen for that entry, ‘what has changed?’. Here, the user saw their original mood and the rating of intensity, their ‘automatic thoughts’ (identified as ‘you said…’) and a prompt for the user’s more thoughtful responses, in a text box which used partially obscured (greyscale) text repeating the prompt from the previous screen. After entering this text, the user was prompted to rate their emotion again, with the question, ‘Has the strength of the mood changed?’ and the 1–5 drop-down menu of emotion intensity. Users were then prompted to add that mood event into a diary and given brief feedback on their emotions.

Results

Usage

In total, 285 individual users downloaded the app and consented for their user data to be shared with the University of Liverpool. The online course from which the bulk of users is believed to have originated had 27 701 ‘joiners’ (people registering to study), with 15 330 active learners starting the course. Although the app was made available on the Apple App Store and Google Play Store throughout the 6-week course (from 8 September to 19 October 2014), users were only notified of this at week 5 of the course, at which point there were 3369 active learners. Specific demographics of the anonymous users who downloaded the app and consented to their data being shared are not available, but demographics of the MOOC participants at week 4 are shown in Table 1. As can be seen, the general demographic for the participants was female, university educated, married, retired or earning £20 000–30 000, with a mean age of 48 years (range 18 to 85). The app was also available on the two major app stores, but the vast majority of users are likely to have found it through the MOOC as this was the only platform through which it was advertised during the period of data collection. This means that about 7% of the target population (285 out of 3369) downloaded the app and agreed to share their data.

Table 1

Demographic information on MOOC participants – potential respondents[a] (week 4 of online course)

	n (%)
Gender
Male	499 (20.4)
Female	1947 (79.6)
Information missing	11
Highest level of schooling achieved
None	13 (0.5)
Primary	9 (0.4)
Secondary	170 (7.0)
Further A levels	231 (9.6)
Further HNVQ	315 (13.0)
University	1679 (69.5)
Information missing	40
Occupational status
Still at school	30 (1.2)
University	105 (4.3)
Full-time employment	749 (31.0)
Part-time employment	399 (16.5)
Self-employed	237 (9.8)
Unemployed	194 (8.0)
Retired	498 (20.6)
Other	205 (8.5)
Information missing	40
Total gross annual or weekly household income
Up to 10k	208 (8.6)
Up to 20k	390 (16.1)
Up to 30k	403 (16.7)
Up to 40k	295 (12.2)
Up to 50k	235 (9.7)
Over 75k	164 (6.8)
Don’t know	176 (7.3)
Prefer not to say	272 (11.3)
Information missing	41
Hours a week work
15 or below	937 (40.4)
16 to 25	303 (13.1)
26 to 35	266 (11.5)
36 to 40	479 (20.7)
41 to 48	225 (9.7)
49 or more	107 (4.5)
Information missing	140
Parents’ income when growing up
Much lower than others (bottom 25% population)	379 (15.7)
Slightly lower than others (lower 50% population)	801 (33.2)
Slightly higher than others (upper 50% population)	781 (32.3)
Much higher than others (top 25% population)	88 (3.6)
Don’t know	301 (12.3)
Prefer not to say	66 (2.7)
Information missing	41
Relationship status
Single	518 (21.1)
Divorced	288 (11.7)
Cohabiting	316 (12.9)
Married	1220 (49.7)
Number of children
None	876 (35.7)
1	379 (15.4)
2	738 (30.0)
3	314 (12.8)
4	107 (4.4)
5	18 (0.7)
6 or more	25 (1.0)

HNVQ, Higher National Vocational Qualifications.

Total responses were n=2457. Mean age was 48.16 years (range 18–85 years).

The majority of participants only used the app one or two times: 65% (186/285) used it once with 17% (49/285) returning to use it a second time. This figure dropped to 7% (21/285) for three entries. The maximum return usage was 30 entries, with only one participant using the app more than 13 times.

Fidelity to CBT principles

Because the app, and indeed CBT itself, relies on the systematic analysis of emotions, their situational circumstances, and the thoughts of the individual, a key element of evaluation was the degree to which participants were able to differentially identify these elements. Users’ text entered into the app was therefore coded using specific criteria from the Mind Over Mood CBT self-help book.[13] Entries were coded as ‘situational comment’, ‘emotional comment’, ‘thoughtful comment’ together with necessary ‘nothing entered’ and ‘unclassifiable material’. A total of 676 entries (from 285 users) were analysed, after removal of entries in foreign languages (9 entries). If participants were appropriately following CBT principles, then it would be anticipated that ‘situational comment’ responses should be entered to the prompt; ‘What was happening when, and immediately before, you felt this way?’, and ‘thoughtful comment’ responses should be entered to the prompts: ‘What were you thinking at the time?’ and later, ‘what has changed?’. It was anticipated that the most likely inappropriate comment would be ‘emotional comment’, as these are the most common forms of response found in clinical settings, where it may prove difficult for people to distinguish their thoughts from their emotions.

An example of an appropriate ‘situational comment’ in response to the prompt ‘What was happening when, and immediately before, you felt this way?’ was: ‘In math and panicking because I feel useless. I don’t understand this concept and if I ask for help I’ll get in trouble because I should know this. Also the teacher isn’t very sympathetic’. An inappropriate comment (coded as ‘emotional comment’) for that prompt was: ‘I just feel really sad and useless’.

An example of an appropriate ‘thoughtful comment’ in response to the prompt ‘What were you thinking at the time?’ was: ‘She is talking about me and not being nice. Or saying something which I am not supposed to hear’, with the alternative: ‘Maybe she is taking about something personal but nothing to do with me. And anyway I don’t much like her – do I care what she thinks?’ after the prompt: ‘What has changed?’. An inappropriate comment (coded as ‘emotional comment’) for that prompt was: ‘Scared. Anxious. Stressed. Panicked’, because – while perfectly valid as a comment – it did not represent thoughts as defined in the traditional approaches to CBT.[13]

In response to the prompt ‘What was happening when, and immediately before, you felt this way?’, 83% (559/676) of the user responses were ‘situational comments’, 5% (31/676) were ‘thoughtful comments’, 10% (35/676) were ‘emotional comments’ and the remainder were non-codable or nothing was entered. In response to the prompt ‘What were you thinking at the time?’, 67% (450/676) of the user responses were ‘thoughtful comments’, 7% (47/676) were ‘situational comments’ and 13% (85/676) were ‘emotional comments’; the remainder were non-codable or with nothing entered. In response to the prompt ‘What has changed?’, 68% (462/676) of the user responses were ‘thoughtful comments’, 10% (66/676) were ‘situational comments’, and 2% (16/676) were ‘emotional comments’; the remainder were non-codable or with nothing entered.

Mood severity

A clear aim of the Catch It app was to improve users’ mood. Because users could select positive moods (‘happy’ and ‘relaxed’) and use their own terms to describe their mood, positive moods were excluded from this analysis. That is not to maintain that it would be impossible or inappropriate for an app to improve positive moods, but the majority of CBT studies have examined a positive impact on negative moods. After removing emotions classed as positive there were 504 valid entries. Combining all negative emotions, there was a significant reduction in users’ severity ratings over the use of the app (mean initial severity rating 3.28, s.d.=1.69; mean final severity rating 2.15, s.d.=1.21). A repeated-measures analysis of variance (ANOVA) was conducted to investigate the relationship further, taking account of multiple data entries by participants by using ‘participant’ as a ‘within subjects’ random effect. This revealed a significant main effect of user (F(1,246) = 2.16, P <0.001), indicating that participants differed consistently one from another (which is to be expected) but also a significant main effect of time (F(1,281)= 330.49, P<0.001), demonstrating that there was a significant decrease in the strength of the negative moods from first entry to second entry.

One design element of the app, however, was that the default entry setting for emotional intensity was set to 1. While appropriate for users, this could lead to false statistical results if users were to have entered a rating other than 1 on first use, and then failed to actively select a second rating – the default setting of 1 could have led to the spurious conclusion that there had been a positive change in mood. To correct for this, a second (very conservative) analysis was performed after removing all the data sets where a second mood rating of 1 was entered. This analysis revealed that there was still a consistent difference between mean first mood rating (mean=3.64, s.d.=0.97) and the second mood rating (mean 2.97, s.d.=0.95). The repeated-measures ANOVA was also conducted again, taking account of the multiple data entries by participants; this was again consistent with the first analysis showing a significant main effect for user, F(1,160)=1.64, P<0.001, and a significant main effect for time, F(1,158)=153.02, P<0.001. From the second analysis, we can positively infer that even when possible default entries influencing the average decrease were accounted for, there was still a significant decrease in the strength of the negative moods from first entry to second entry (Table 2).

Table 2

Negative and positive mood ratings before and after the use of the app[a]

	Negative moods	Positive moods
Participants, n	247	93
Mood ratings, n	504	139
First rating, mean (s.d.)	3.64 (0.97)	3.39 (0.88)
Second rating, mean (s.d.)	2.97 (0.95)	3.54 (0.91)

Scores have been calculated after the removal of potentially confounding default values.

A similar analysis was conducted on the 139 positive entries, after removing items for which the second entry was 1 (the potentially confounded default setting). This revealed a small increase in positive mood ratings (from mean 3.39, s.d.=0.88 to mean 3.54, s.d.=0.91). A repeated-measures ANOVA, again taking account of the multiple data entries by participants, showed no significant main effect for user (F(1,92)=1.17, P=0.278) but a statistically significant main effect for time (F(1,46)=4.28, P<0.05), demonstrating that there was a small but significant increase in the strength of the positive moods from first entry to second entry.

Discussion

Data from 285 individual users were available for analysis. This represents a relatively small proportion of the population who were aware of the app. Moreover, the majority of participants used the app only once. A minority of users, however, appeared to find the app helpful, with highly appropriate entries, consistent with the principles of CBT, and a significant change in mood intensity.

Although absolute uptake was low and most users made few entries, the nature of the entries made showed that users found the app easy to engage with and the significant decrease in negative mood intensity following use of the app are encouraging findings. The fact that the majority of users were able to identify appropriate situations, thoughts and alternative thoughts is the key outcome. It suggests that members of the public are able to benefit from smartphone apps, but that considerable additional research and development is required to understand the differences between interventions aimed at improving resilience or well-being in the general population, what might be termed ‘public mental health’, and apps based on CBT principles addressing more clearly identified problems. Many people report that psychological therapy is beneficial in that it facilitates focusing their attention on a problem, while the very act of talking about the problem necessarily involves deliberate broadening of perspective;[14] this mechanism may be facilitated effectively via apps as well as face-to-face therapy. The use of apps in mental healthcare may have other benefits. Psychiatric medication has recognised adverse effects and could even be counterproductive in helping people reflect on their emotional state and generate alternative perspectives.[15] If effective, apps such as Catch It may be valuable first-line interventions as alternatives to prescribed medication. At present, there is no research examining this possibility, but this would represent a valuable potential avenue for future randomised controlled studies.

It is significant that Catch It had a significant positive impact on positive moods. Traditionally, therapies such as CBT have tended to focus on negative moods and resolving problems through reducing supposed ‘negative cognitions’. The fact that a positive shift in positive moods was observed suggests that smartphone apps such as Catch It could offer ways to improve well-being, as well as addressing specific problems. Further research could investigate the relationship between changes in positive and negative moods, and their relationship with both specific mental health problems and subjective well-being.[16]

Given the apparent potential effectiveness of the Catch It app, one possible approach could be to use smartphone apps in primary care or other community settings where people are requesting more conventional therapy or seeking help with problems such as anxiety or low mood. As an adjunct to care plans, apps such as Catch It, if proved effective in more robust research designs, could, for instance, avoid inappropriate prescription of medication during inevitable waiting periods. Moreover, although the absolute uptake of the Catch It app was low (7% of the available population), this would, for the wider general population, represent a highly cost-effective intervention.

There were, of course, some other limitations to the present study. We were unable to access the specific demographic data for the consenting users, and available demographic data (for the target population) showed disproportionately high numbers of females, university graduates and an average age of 48. This was a self-selecting sample of learners in an online course and hence highly familiar with online media. This population was also presumably particularly interested in mental health issues. Our data must therefore be interpreted with caution, as the cohort is unlikely to be representative of the general population. There is a difference, however, between the use of smartphone apps by people with a general interest in mental health and people with specific problems and seeking help. There is a growing body of evidence to suggest that smartphone apps can play a limited but useful role in a range of psychological problems.[17]

Minor bugs in the app software (for instance, the default setting for mood rating) need to be addressed, although, as detailed above, we appropriately controlled for this issue in our data analysis. Clearly, the next logical step should be to conduct a randomised controlled trial with appropriate intention-to-treat analysis and employing masked, independent and objective measures of psychological distress as well as the app’s internal mood rating system. A suitable population could be individuals seeking help from primary care, especially as interventions such as this could complement, rather than replace, ‘treatment as usual’.