A study protocol for a non-randomised comparison trial evaluating the feasibility and effectiveness of a mobile cognitive-behavioural programme with integrated coaching for anxious adults in primary care.

INTRODUCTION: Generalised anxiety disorder (GAD) and subclinical GAD are highly prevalent in primary care. Unmanaged anxiety worsens quality of life in patients seen in primary care practices and leads to increased medical utilisation and costs. Programmes that teach patients cognitive-behavioural therapy (CBT) techniques have been shown to improve anxiety and to prevent the evolution of anxiety symptoms to disorders, but access and engagement have hampered integration of CBT into medical settings. METHODS AND ANALYSIS: This pragmatic study takes place in University of Pittsburgh Medical Center primary care practices to evaluate a coach-supported mobile cognitive- behavioural programme (Lantern) on anxiety symptoms and quality of life. Clinics were non-randomly assigned to either enhanced treatment as usual or Lantern. All clinics provide electronic screening for anxiety and, within clinics assigned to Lantern, patients meeting a threshold level of mild anxiety (ie, >5 on Generalised Anxiety Disorder 7-Item Questionnaire (GAD-7)) are referred to Lantern. The first study phase is aimed at establishing feasibility, acceptability and effectiveness. The second phase focuses on long-term impact on psychosocial outcomes, healthcare utilisation and clinic/provider adoption/sustainable implementation using a propensity score matched parallel group study design. Primary outcomes are changes in anxiety symptoms (GAD-7) and quality of life (Short-Form Health Survey) between baseline and 6-month follow-ups, comparing control and intervention. Secondary outcomes include provider and patient satisfaction, patient engagement, durability of changes in anxiety symptoms and quality of life over 12 months and the impact of Lantern on healthcare utilisation over 12 months. Patients from control sites will be matched to the patients who use the mobile app. ETHICS AND DISSEMINATION: Ethics and human subject research approval were obtained. A data safety monitoring board is overseeing trial data and ethics. Results will be communicated to participating primary care practices, published and presented at clinical and scientific conferences. TRIAL REGISTRATION NUMBER: NCT03035019. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

RCT Entities:   Population Interventions Outcomes

First study evaluating a coach-guided cognitive–behavioural programme delivered via mobile app in primary care.

Pragmatic two-arm parallel comparison effectiveness study design to allow for the evaluation of a digital behavioural intervention in primary care settings with minimal research infrastructure in place.

Detailed characterisation of a large primary care population to understand who engages and uses mobile app-based cognitive–behavioural program.

Lack of randomisation of the sample.

With the absence of research-related facilitators of study recruitment (eg, subject payments and research assistant facilitated recruitment), there may be high rates of missing data, and completer rates may be smaller than in typical efficacy trials.

Introduction

Behavioural health conditions are among the most prevalent health problems in the US population with a lifetime prevalence of 46.4%.1 Depression and anxiety disorders are the most common psychiatric disorders in the general population and major drivers of healthcare costs.2 3 In fact, it is in the primary care setting that most mental health disorders are addressed and treated.4 5 In the subset of patients with chronic medical conditions, behavioural health issues are even more common and costly.6–8 These rates are likely even higher since less acute patients may not seek care outright, seeing themselves only as ‘stressed,’ leaving them undercounted.9 10

Over the past several years, there have been two broad shifts in behavioural health: integrating behavioural health into broader medical care and focusing on upfront prevention rather than solely on treatment for large populations. Primary care and specialist medical providers are busy, do not have straightforward access to behavioural treatment for their patients and patients are often not compliant even if referred for behavioural health services.5 While behavioural health issues result in massive direct and indirect costs, there is a relative shortage in the mental health workforce to treat them. In primary care settings, this shortage is even greater.11 12 Most current behavioural treatment models involve face-to-face sessions and are delivered individually, making it difficult to scale them adequately to address increased demand.

Cognitive–behavioural therapy (CBT) is the most studied type of behavioural health intervention approach for anxiety, focused on modifying ‘maladaptive patterns of thinking and behavior to improve mood and coping.’13 CBT is time-limited, problem-focused and involves between-session practice of new skills. As a learning-based intervention, practice of CBT skills improves symptoms and is associated with changes in brain metabolism.14 15 CBT is the standard of care for many psychiatric disorders including depression, anxiety disorders, eating disorders, addiction, chronic pain and sleeping disturbance (insomnia) across the spectrum of acuity.16–20

In the last 15 years, there has been significant effort to provide CBT to be delivered via interactive online programmes. The interest in digital CBT is intended to overcome the cost and access burdens of one-to-one, in-person therapy. It was also seen as a better fit for individuals who, in the midst of emotional distress, may not be motivated to seek or remain in high burden treatments. Computerised or digital CBT programmes have been shown to be effective in treating anxiety and depression in primary care settings in some21–24 but not all studies.25 26 Digital CBT offers the advantage of convenience, accessibility, less stigma and being less labour intensive than face-to-face therapy.27 28 While outcomes are promising with small trial populations for anxiety and depression, existing digital programmes have struggled with engagement and adherence, making it difficult to scale to large populations effectively.25 Digital CBT can be delivered as either guided or unguided with the highest retention rates and best outcomes involving some type of collaboration with a therapist or coach.29–33 Taking advantage of technological advances for delivery of CBT via a mobile app offers advantages.34 This study compares a novel-guided cognitive–behavioural programme delivered via mobile app to medical treatment as usual in reducing anxiety and improving quality of life in adults in primary care practices.

Objective

The primary aims of this study are to evaluate the feasibility, acceptability and effectiveness of a mobile coach-facilitated cognitive–behavioural programme, Lantern, in adults with generalised anxiety in primary care settings. We hypothesise that integration of Lantern, a mobile cognitive–behavioural programme, for anxiety within primary care is more effective at reducing anxiety and improving quality of life than enhanced treatment as usual.

To show feasibility and acceptability of implementing Lantern in primary care settings in patients aged 20–65 years who meet threshold anxiety criteria.

To evaluate effectiveness of Lantern for anxiety and quality of life at primary care practices compared with propensity score matched controls at practices without the availability of this programme over a 6-month period.

To evaluate the effectiveness of Lantern for improving quality of life and reducing medical utilisation driven by behavioural health symptoms over a 12-month period.

To evaluate moderators of programme effectiveness.

Methods

Trial design

This study is a prospective pragmatic two-arm parallel comparison trial of Lantern (active) to treatment as usual (control) at four primary care sites (see figure 1). As this is a non-randomised trial, propensity score matching is used at the level of primary care practices and by individual patients at control sites using Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) criteria. This trial will employ a mixed-methods approach to include both quantitative and qualitative analytic methods for a richer contextual understanding of our study aims. Quantitative analyses will be conducted both within and between subjects. For qualitative analyses, semistructured interviews will be administered with equal proportion of Lantern completers and non-completers.

Figure 1

Trial design, intervention and end points. CBT, cognitive–behavioural therapy; GAD-7, Generalised Anxiety Disorder 7-Item Questionnaire; TAU, treatment as usual.

Participants

Patients are enrolled in the study from four primary care practices affiliated with a major academic institution. Patients between 20 and 65 years of age are screened for anxiety using the Generalised Anxiety Disorder 7-Item Questionnaire (GAD-7) and quality of life using the 12-Item Short-Form Health Survey (SF-12) as part of their routine medical care. Patients who score ≥5 on the GAD-7 will be invited to participate in the study during their primary care clinic visit (see figure 1). Participants will not be reimbursed or incentivised for their study participation but will be provided access to Lantern, the mobile-delivered cognitive–behavioural programme, at no cost.

Eligibility criteria

Patients are eligible for this study if they are aged 20–65 years, endorse ≥5 on GAD-7 and have access to a smartphone or tablet.

Clinic selection and patient recruitment

The four clinic sites were chosen based on a convenience sample having similar patient population characteristics and similar clinician willingness to participate.

At active sites, an electronic best-practice alert is generated to prompt clinic staff to obtain informed consent and to offer Lantern. Active site patients with acute back pain who are participating in a co-occurring study are excluded. Active site patients who are scheduled for a pregnancy appointment are not screened for this study. If the patient is eligible for the study based on above criteria, then the primary care provider will further ascertain whether the patient is appropriate for Lantern. Primary care providers will be provided with guidelines that patients with current suicidal ideation, current psychosis or psychotic disorders or current post-traumatic stress disorder are not appropriate for this evaluation of Lantern’s anxiety programme. If the patient consents, the physician orders Lantern through the electronic medical record. Participants are able to sign up for Lantern (by downloading the mobile app) at the time of the primary care clinic visit if they choose. Each participant will have access to Lantern for 2 years after enrolment.

At control sites, patients will be screened in the same way as those in the active sites. Informed consent process will not be required as the GAD-7 and SF-12 are part of routine clinical screening. These two measures will be obtained in a deidentified format from the control sites at baseline and over the next year. The mean number of visits at the control sites is 2.2 per year. Each control practice was provided with an educational brochure from National Institute of Mental Health about GAD and its treatment and a list of three psychiatric practices in the area with availability to see patients within 3 months of referral which constitutes enhanced treatment as usual.

Intervention

Lantern is based on empirically supported cognitive–behavioural protocols for GAD,35–38 developed in partnership with academic investigators to target symptoms of stress and anxiety for the general population. The content was developed based on the empirically supported CBT model but optimised to drive engagement on the mobile platform (eg, high-quality audio, bite-sized content delivery and seamless user experience) and includes motivational behavioural coaching.

There are six core components within the Lantern anxiety programme:

Education and awareness about anxiety and how thoughts, emotions and behaviours are inter-related. This section provides the rationale for the cognitive–behavioural model.

Relaxation: participants learn several empirically supported relaxation techniques to manage anxiety.

Thoughts: participants are guided on how to challenge their assumptions about these thoughts/stories and to create new stories.

Behavioural change and exposure: participants will learn the behavioural cycle of avoidance and how this maintains anxiety. Through learning the principle of exposure, patients learn how systematic exposure to anxiety provoking situations can help to overcome anxiety in the long term.

Mindfulness: this component is about learning to observe one’s thoughts and feelings without judgement or attachment, which helps to interrupt the fused thought–feelings–behaviours cycle to promote a more flexible behavioural repertoire beyond anxiety-driven behaviours.

Habit formation/maintaining skills: patients will reflect on what they have learnt and work on making a habit out of the most effective techniques.

Coaching model

While the programme is largely self-directed, coaches are integrated into the programme to increase adherence using motivational techniques, answer questions and humanise the experience. Coaches exchange short written messages with users through the app. The coaches have access to an internal coaching portal where they have a dashboard for each of their users. The dashboard includes all information the user inputs into the programme including both their direct messages to coaches and all the content they have completed in the programme.

Lantern primarily employs Master’s-level coaches with backgrounds in health and wellness coaching or mental health treatment. All coaches are provided with training and supervision by doctoral-level licensed clinical psychologists in CBT techniques, digital mental health, coaching methods and risk management strategies. Coaches are trained to adhere to a standard risk protocol which includes recognising signs of potential risk, expressing concern to the user and referring to appropriate and study-specific additional services. Risk review is conducted daily whereby all user messages are read/reviewed for potential risk and appropriate steps are taken as per protocol.

Study risk management protocol

In addition to the above Lantern risk management protocol managed by coaches, additional procedures were established in accordance with the medical centre, study principal investigator (PI) and the IRB. If any participant endorses ≥15 on the GAD-7 during follow-up study or clinic assessments, primary care clinic staff will be notified via an electronic medical record alert and will also be alerted by the PI. The PI will also be notified when coaches activate the standard risk management protocol. Additionally, on study entry, participants are encouraged to provide an emergency contact who the research team may contact in the event of worsening anxiety/psychiatric symptoms. Patients with worsening anxiety or those who experience other severe psychopathology will be offered appropriate medical management but not be removed from the study.

Communication of Lantern progress to primary care providers

Deidentified usage data (eg, progression through the programme, frequency of messages to coaches, frequency of session access and techniques completed) is collected as part of the programme. Lantern internally has a database that tracks user progress through the programme. Using the unique study identifier assigned to each participant at the time of consent, this information can be combined with clinical measures tracked to the study coordinators who then generate progress reports sent to the physicians at each practice at regular intervals.

Feasibility/acceptability,

Anxiety symptom severity,

Quality of life.

Usability, satisfaction, helpfulness of Lantern programme,

Medical utilisation.

Data collection/measures

At the active sites, primary outcome assessments (GAD-7 and SF-12) will occur at baseline, 6 and 12 months (see table 1). At the active sites, participants will be sent these measures electronically through secure email. For those patients who do not respond via email, research assistants blinded to their progress through the programme will contact participants to capture these assessments via telephone.

Table 1

Primary outcomes, secondary outcomes and time points of collection

	Category	Measures	Baseline	2 months	6 months	12 months
Demographics	Predictor		✓
Chronic disease characterisation	Predictor	Diagnoses based on ICD-10 coding in the electronic medical record	✓
Anxiety	Primary outcome	GAD-7	✓	✓*	✓	✓
Quality of life	Primary outcome	SF-12	✓		✓	✓
DSM-V psychiatric cross-cutting questionnaire	Predictor (Lantern only)		✓
Helpfulness/satisfaction questionnaire	Secondary outcome (Lantern only)				✓
Medical/behavioural health utilisation questionnaire	Secondary outcome (Lantern only)				✓	✓

*The 2-month GAD-7 assessment only occurs for active sites.

DSM-V, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; GAD-7, Generalised Anxiety Disorder 7-Item Questionnaire; ICD-10, International Statistical Classification of Disease and Related Health Problems, Tenth Revision; SF-12, Short-Form Health Survey.

At the control sites, the GAD-7 and SF-12 questionnaires were provided to patients as part of routine care at each subsequent visit. The GAD-7 and SF-12 assessments that are completed in clinic at 6 and 12 months will be used for this study (table 1).

Measures

Demographics

Date of birth, gender, ethnicity, race, insurance type and zip code as proxy for socioeconomic status will be extracted from the medical record.

Generalised anxiety symptom severity

The GAD-7 is a 7-item validated self-report questionnaire used to identify probable cases of GAD and screening for the diagnosis of GAD according to Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria.39 The questionnaire asks respondents to rate the degree to which they have experienced seven core diagnostic features of GAD during the previous 2 weeks. Items are scored on a four-point Likert-type scale with a minimum score of 0 and a maximum score of 3. Total scores (ranging from 0 to 21) are commonly categorised into four severity groups: minimal/no anxiety (0–4), mild (5–9), moderate (10–14) or severe (15–21). It is one of the most widely used anxiety measures in the US in primary care settings.40

Quality of life

The SF-12 is a 12-item validated self-report measure assessing one’s perceived quality of life (adapted from SF-36).41 The SF-12 comprised eight subscales describing health functioning: physical functioning, role limitations due to physical health problems, bodily pain, general health, energy/fatigue, social functioning, role limitations due to emotional problems and mental health. Results are derived from two component summary scales: the Physical Component Summary (PCS-12) and Mental Component Summary (MCS-12) and are scored using norm-based methods. Both the PCS-12 and MCS-12 summary scores range from 0 to 100 with a mean of 50 and SD of 10 in the general US population. Thus, scores greater than 50 represent above average health status.

DSM-V level 1 cross-cutting symptom measure

This is an informant-rated measure that assesses mental health domains across psychiatric diagnoses.42 43 The measure consists of 23 questions. We modified the questionnaire to only include 19 questions assessing depression, anger, mania, anxiety, somatic symptoms, sleep problems, repetitive thoughts and behaviours, dissociation, personality functioning and substance use. The measure was developed by consensus of experts by the American Psychiatric Association and was found to be clinically useful and to have good test–retest reliability in Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V) field trials that were conducted in adult clinical samples. Each item rates how much a subject was bothered by a specific symptom during the past 2 weeks with items being rated on a 5-point scale (0=none to 4=severe or nearly every day). This measure is only administered to patients with baseline GAD-7 ≥10 at the Lantern sites.

Medical diagnoses and utilisation

International Statistical Classification of Disease and Related Health Problems, Tenth Revision code diagnoses, visit problem lists, smoking history, surgical history, medication history and medical utilisation. More specifically, medical utilisation will include number of primary and specialty care outpatient visits, emergency room visits and hospitalisations (number and days). Utilisation information will be requested 1 year prior to enrolment and 2 years poststudy enrolment.

Lantern Helpfulness and Satisfaction Scale

This is a 14-item scale (Likert scale ranging from 1=not at all to 7=extremely) asking respondents to rate the helpfulness of the programme, satisfaction with the programme, evaluation of length of Lantern programme, evaluation of the helpfulness of the Lantern coach, level of difficulty/effort to do the Lantern skills/assignments and likelihood of recommending Lantern to family/friends. It is adapted from the Client Satisfaction Questionnaire.44

Behavioural health utilisation

This is a 14-item self-reported measure of past 6-month use of psychological treatments, psychotropic and pain management medications, hospitalisation and emergency services collected at 6 and 12 months. It was adapted from the Cornell Services Index.45 This measure is only administered to the Lantern group.

Lantern utilisation

Usage metrics such as number of log-ins per day/week, number of sessions completed and frequency of messages sent to coach will be collected within the Lantern mobile app.

Qualitative interview process (active sites only)

A subset of 24 participants who were offered Lantern will be asked to participate in an in-depth qualitative telephone interview at 6 months after enrolment. Participants will be randomly selected and be balanced to include both those who completed and did not complete the programme. The interviews will be conducted by research staff contacting the randomly selected participants. The semistructured qualitative interview will explore participants’ experience with and expectations of the Lantern programme, including their experience with coaching and areas of suggested improvement. All interviews will be audio-recorded and transcribed verbatim for analysis.

Statistical methods

Sample size and power calculation

We expect 1200 patients will meet entry criteria, with 600 from control and 600 from Lantern sites. Of these, we expect approximately 50% attrition over 6 months (n=600, with 300 from control and 300 from Lantern sites). With n=1200 (n=300 at each of four primary care practices) and an intracluster correlation (ICC) of 0.20 within practices,46 we will have 0.80 power to detect a between-group difference of d=0.43 and 0.90 power to detect a between-group difference of d=0.49. With only n=150 at each of four sites (ie, those with 6-month outcome data), we expect to have 0.80 power to detect a between-group difference of d=0.45 and 0.90 power to detect a between-group difference of d=0.52. These effect sizes are within the moderate range (d=0.40–0.60), we hypothesise for Lantern versus enhanced care. All power analyses were performed using PASS V.13.0.8.

Data analysis

Feasibility and acceptability will be assessed using two criteria: (1) among the first 50 patients, at least 50% of eligible participants meeting inclusion criteria should accept participation in the study and (2) at least 50% of participants who initiate Lantern will complete at least three techniques.

Prior to testing a priori hypotheses, we will use descriptive statistics and effect sizes to characterise and compare the Lantern and control samples overall and by clinic, including reporting the ICCs. Graphical displays will be used to visualise distributions and trajectories of change over time in order to inform model specification. We will examine missing data frequencies and mechanisms at baseline and follow-up and use multiple imputation to retain the full sample when possible. Because this is a non-randomised design, propensity score matching will be used to develop a matched sample of eligible Lantern and control participants with 6-month follow-up outcome data. Our strategy will be to consider all baseline characteristics that are potentially related to treatment for use in the propensity score model. We expect this will include age, gender, SF-12 and PCS-12 score, in addition to any other clinic-level and individual-level characteristics we observe to be important. To account for the small number of clinics, we will use propensity score cross-cluster matching as proposed by Leon.47 The parameter specifications (eg, calliper width) and specific variables to be included in the propensity score model will be determined based on an iterative approach aimed at finding the best balance between finding matches and reducing bias. If the number of control patients is substantially smaller than the number of Lantern patients (or vice versa), we will consider using 1–2 matching to retain a larger sample.

Because treatment was provided at the clinic level, we will consider the impact of clinic in all analyses. For a priori hypothesis testing, we will use mixed-effects models for repeatedly measured outcomes. These models will include time, treatment and the time by treatment interaction. A significant interaction will indicate that the trajectory of change in the outcome differs by treatment. To appropriately model the covariance structure, we will include a random subject effect nested within the random clinic effect. We will also test whether the effect of treatment on change over time differs by baseline symptom score by adding further interaction terms to the model. Both intent to treat and per-protocol analyses based on only those Lantern participants who completed an adequate dose of the treatment (50% of the programme at 4 months) will be performed. Statistical programmes SPSS V.21 and R will be used for all analyses.

For qualitative analysis, a thematic analytic approach will be used. Interview transcripts will be uploaded into Atlas.ti, a programme for qualitative analysis, and thematic codes will be developed inductively. Trained coders will create codes based on content, relevance and prevalence of themes and use codes to develop analytic categories. Altas.ti’s query tools will be reviewed for common themes.

Data collection, management and confidentiality

A secure database maintained within the Institution’s firewall will store all the data collected in a deidentified way. All analyses will be completed by a study-independent statistician. Data at the practices will be collected in two ways: (1) aggregate deidentified reports of all screened patients at each site (active and control) by an honest broker and (2) deidentified data for individual consented patients at each active site.

Safety

Several steps were taken to minimise breach in participant confidentiality including the use of deidentified study identifiers, keeping all data behind the UPMC firewall, securing all hard copy data in locked file cabinets and electronic data in password protected files. Participant medical record information will be stored in an honest broker database. Several steps will be taken to monitor for adverse events. Patient progress will be monitored by clinical staff and the PI. Lantern coaches follow a risk management plan and communicate with the PI and the research team for any participant concerns. If at any time in the study a participant is judged to be experiencing adverse events such as severe depression or suicidality, the event will be recorded, and a treatment escalation plan implemented. For patients with significantly worsening anxiety (GAD-7 score >15), the PI will be notified and communicated with the clinical team. For Lantern sites, the clinical team will receive regular updates in the electronic medical record of participants’ progression in Lantern and GAD-7 scores at baseline, 2 months and 6 months.

Ethical considerations

This study obtained ethics and human subject research approval. Participants provided IRB-approved consent before taking part in this study. A data safety monitoring board is overseeing the trial data and ethics. There are weekly meetings with the research team and biweekly meetings with the participating primary care practices.

Results

This study began in July 2016 and is expected to end in July 2018.

Dissemination

The results of this study will be communicated to the participating primary care practices and published in peer-reviewed publications and presented at national and international clinical and scientific conferences/meetings.

Discussion

Common behavioural health disorders, such as depression and anxiety, remain inadequately addressed with evidence-based care and treatments. Less than half of patients with common behavioural health conditions receive evidence-based treatment, such as cognitive–behavioural interventions.48 Self-directed and coach-supported online delivery of a cognitive–behavioural programme for anxiety via a mobile app allows for the scalable provision of and access to evidence-based care. This paper presents a description of the pragmatic research methods to evaluate the integration of a mobile-delivered cognitive– behavioural programme into primary care and the impact on anxiety symptoms and quality of life among adult patients. Much research in this field is currently evaluating the application of online-guided cognitive–behavioural programme into routine care settings.48 As such, the findings from this study will add value to the evidence base through translation of research into a real-world practice setting with an optimal balance of internal and external validity.

The findings will need to be evaluated and interpreted in light of the following limitations of the study design. Given the non-randomised design, the potential influence of factors such as clinic setting and provider differences cannot be controlled for thorough randomisation. However, the propensity score matching methods will allow for minimising these differences at baseline to distil the effects of Lantern, and results will be interpreted in the context of these limitations. Another limitation is the limited standardisation of how primary care providers prescribe/recommend Lantern to their patients. Uptake of Lantern by primary care patients is likely driven in part by the interaction with their primary care provider, and we will have limited data to systematically evaluate these potential effects. Overall, this study will provide evidence for the effectiveness of Lantern on anxiety symptoms and quality of life while also informing future improvements to scale and implement Lantern in primary care.