Engagement with online psychosocial interventions for psychosis: A review and synthesis of relevant factors.

BACKGROUND: Little is known about factors associated with engagement with online interventions for psychosis. This review aimed to synthesise existing data from relevant literature to develop a working model of potential variables that may impact on engagement with online interventions for psychosis.
METHODS: Online databases were searched for studies relevant to predictors of engagement with online interventions for psychosis; predictors of Internet use amongst individuals with psychosis; and predictors of engagement with traditional psychosocial treatments for psychosis. Data were synthesised into a conceptual model highlighting factors relevant to engagement with online interventions for psychosis.
RESULTS: Sixty-one studies were identified. Factors relevant to engagement related directly to the impact of psychosis, response to psychosis, integration of technology into daily lives and intervention aspects.
CONCLUSION: While several candidate predictors were identified, there is minimal research specifically investigated predictors of engagement with online interventions for psychosis. Further investigation examining both individual- and intervention-related factors is required to inform effective design and dissemination of online interventions for psychosis.

Introduction

Despite a previous digital divide, individuals with psychotic disorders have increasing access to the Internet (Robotham et al., 2016). In recent surveys of Australian and Spanish outpatient service users, 76 and 85% of participants respectively reported having access to the Internet (Bonet et al., 2018; Thomas et al., 2017). In addition to increased access, a large proportion of individuals with psychosis are positive towards and interested in utilising digital technologies for mental health (Bonet et al., 2018; Firth et al., 2016). As access to the Internet increases amongst individuals with psychosis, so does the opportunity to develop and provide low-intensity, evidence-based, interventions online.

To date, various Internet- and mobile- based interventions (‘online interventions’) have been developed for psychosis. Internet-based interventions include psychoeducational websites (Rotondi et al., 2010); self-guided cognitive behavioural therapy (CBT) courses (Gottlieb et al., 2017); recovery-oriented web-based programs (Thomas et al., 2016); and peer support/social networking tools (Alvarez-Jimenez et al., 2013). Mobile apps have also been increasingly utilised for ecological momentary assessment (EMA), collecting repeated, real-time data in the context of daily life (Shiffman et al., 2008). EMA apps enable real-time symptom monitoring and in-the-moment clinical intervention (e.g., for treatment adherence) for experiences of psychosis (Bell et al., 2017). A growing evidence base suggests that online interventions are both feasible and acceptable to individuals with psychotic disorders and may be effective in assisting with clinical and social outcomes amongst this population (Ben-Zeev et al., 2018; Berry et al., 2016; Schlosser et al., 2018). For example, web-based interventions have been associated with symptom reduction of small effect size (Gottlieb et al., 2017) and mobile apps have been associated with medium effects for improving auditory hallucinations, depressive symptoms, self-efficacy, motivation and functioning in randomised controlled trials (Granholm et al., 2012; Schlosser et al., 2018).

Engagement with online interventions

Despite demonstrated efficacy for various mental health difficulties (Rogers et al., 2017), issues with engagement such as low levels of use have presented as a substantial challenge in the implementation of online interventions for mental health difficulties broadly (Christensen et al., 2009). The term ‘engagement’ refers to a user's involvement and interaction with an intervention (Baltierra et al., 2016). This includes both the extent of intervention use, as well as the user's subjective experiences, including attention, affect, and interest (Perski et al., 2017). Engagement overlaps with the term ‘adherence’, which denotes the extent of use of the intervention as intended but does not require prescriptive use (Donkin et al., 2011). Various measures have been utilised to examine engagement: the number of modules completed, the number of clicks or logons, length of time spent online or proportion of individuals completing an intervention (Short et al., 2018). A further often examined index of engagement is unintended dropout or disengagement from a service – indicating a lack, or cessation, of engagement with an intervention (Melville et al., 2010). Qualitative reports can also provide insights into the subjective components of engagement with online interventions (Short et al., 2018).

As increased engagement is associated with improved outcomes, furthering understanding of associated factors becomes an important research target (Donkin et al., 2011; Fuhr et al., 2018). Individual, environmental and intervention factors may influence engagement with online interventions (Perski et al., 2017). In populations other than psychosis, higher levels of engagement have been associated with adjunct support (e.g., telephone or email contact), female gender, higher treatment expectations, personalised interventions and having sufficient time to utilise the intervention (Baumeister et al., 2014; Beatty and Binnion, 2016). Factors such as motivations for treatment, the impact of symptoms (mood, anxiety) and demographic factors (age, education, employment, computer literacy) have also been identified as potential predictors of increased engagement (Christensen et al., 2009; Perski et al., 2017).

While individuals with psychosis may have overlapping experiences with other clinical populations, their often complex needs may be unique factors relevant to engagement with online interventions. For example, individuals may have reduced awareness of illness, potentially inhibiting treatment-seeking behaviour (Buckley et al., 2007). Cognitive deficits, as well as disorganisation, limited insight into their symptoms and illness, or negative symptoms, are additional experiences that may interfere with individuals' treatment decision-making capacity and potentially engagement with treatments (Lakeman, 2006; Larkin and Hutton, 2017; Thomas et al., 2011). Additionally, some individuals with persisting disability may have less familiarity with technology as a result of reduced daily exposure from work or study (Baup and Verdoux, 2017; Robotham et al., 2016).

Killikelly et al. (2017) reviewed rates of adherence to interventions utilising digital technology for people with psychosis. Out of 20 identified studies (randomised controlled trials, feasibility studies, observational studies, experience sampling method studies), only two found significant associations between examined variables and adherence to the interventions: van der Krieke et al. (2013) found younger age (t(213) = −2.120, p = .03) and male gender (χ21 = 5.6, p = .02) were significantly associated with drop out from a study evaluating a web-based shared decision-making tool. Palmier-Claus et al. (2012) found that positive symptom severity was significantly associated with non-compliance (completing <33% of potential data entry points) with Clintouch, a mobile assessment application in the context of ambulatory symptom monitoring (OR = 0.68, p = .033).

The review by Killikelly et al. (2017) highlighted the limited existing empirical data on factors associated with adherence with online interventions for psychosis. Many studies to date have included an uncontrolled study design and investigated feasibility or acceptability of interventions, rather than engagement specifically (Killikelly et al., 2017). Identifying factors relevant to engagement will assist in directing research efforts and inform tailored design of future interventions. Currently, understanding of relevant factors remains limited.

To address this knowledge gap, we sought to identify factors relevant to engagement with online interventions for psychosis. Given the formative stage of this research area, a broad approach was considered appropriate to identify factors not determinable through examination of predictors of engagement with online interventions alone. As online interventions require Internet access and use, predictors of Internet use may overlap with predictors of engagement with these interventions. Predictors of engagement with traditional face-to-face psychosocial services and psychological treatments may also translate to interventions provided via an online format. As such, this review, which was of a descriptive and exploratory nature, aimed to synthesise relevant data from these intersecting bodies of literature to develop a working model of potential variables that may impact on engagement in online interventions in psychosis.

Methods

We considered the following intersecting literature:

Studies directly reporting on patterns of engagement with Internet- or mobile-based interventions for psychosis.

Studies directly reporting on patterns of Internet use amongst people with psychosis.

Studies on predictors of engagement in non-digital, psychosocial interventions for psychosis.

We sought to identify both quantitative studies with predictors of any index of engagement (including drop out, disengagement, and clinician-rated engagement) and qualitative studies capturing the subjective experience of engagement.

We focused on psychosocial and psychological interventions delivered via an online format. Studies utilising technology to promote medication adherence were considered a distinct type of intervention from psychosocial treatments and thus, outside the scope of this review.

Search strategy

We adopted a pragmatic approach to synthesising the literature. First, we identified published reviews of the domains of interest to identify primary studies: Doyle et al. (2014); Killikelly et al. (2017); Leclerc et al. (2015); Nosé et al. (2003); Staring et al. (2006); Villeneuve et al. (2010). We then supplemented these studies using literature searches for more recent publications. We searched the PsycInfo, PubMed, and Web of Science databases utilising a combination of search terms from the identified systematic reviews. Search results were limited to studies published from the year 2000 onwards. To identify studies of online interventions for psychosis or relating to Internet use in psychosis search terms pertained to psychosis (psychosis OR psychotic OR schizo* OR “severe mental illness” OR “serious mental illness”) and an online format (online OR mobile OR website OR digital OR Internet OR web-based OR computer). To identify any studies reporting on predictors of engagement in face-to-face interventions we used the same search terms pertaining to psychosis, terms relating to treatment (treatment OR program OR intervention OR CBT OR “cognitive behaviour therapy” OR service OR therapy), and also relating to engagement (engage* OR adher* OR attrition OR dropout). Searches were conducted on 29 May 2019 by the first author (CA). An inclusive approach to selection was taken and articles were included if they contained any results (qualitative or quantitative) that were relevant to engagement with online interventions to psychosis, engagement with face-to-face services for psychosis, or internet use amongst individuals with psychosis.

Data extraction

The following data were extracted by the first author (CA): (a) study identification items (author, year of publication); (b) study design; (c) participant sample characteristics (i.e., diagnostic group); (d) intervention type or relevant Internet-related behaviour; (e) engagement (or Internet use) construct; (f) operationalisation of engagement; (g) significant and non-significant associations with engagement indices or Internet use; (h) primary aim of the study; (i) main outcome measure; (j) results in terms of effectiveness For qualitative studies, information relating to consumers' or staff members' experiences of engagement was extracted in the form of relevant subthemes or descriptors.

Data analysis

Following extraction, data were narratively synthesised to establish factors relevant to engagement with online interventions for psychosis. This involved tabulation of significant and non-significant associations between investigated variables and engagement indices or internet use. Variables were considered as relevant factors when examined in multiple studies and a demonstrated association. The similarities, importance, and potential role of identified variables were discussed between authors and considered in the context of relevant literature and previous conceptual models of engagement (Perski et al., 2017; Short et al., 2015). Conceptual mapping was additionally utilised to consider the potential interplay between variables. This led to the synthesis of identified variables into a conceptual model.

Results

We identified a total of 60 studies from the three relevant literatures (Fig. 1). For engagement with online interventions for psychosis, we identified six relevant studies from the Killikelly et al. (2017) review, plus a further ten studies from our supplementary literature search (see Table 1 for study details. Additional study details are included in the Supplementary table). For use of the Internet by people with psychosis, we identified 12 studies (see Table 2). For engagement with face-to-face treatments in psychosis, from previous reviews we identified eight studies from Doyle et al. (2014), two studies from Leclerc et al. (2015), two studies from Staring et al. (2006) and we incorporated the meta-analytic results from Villeneuve et al. (2010), we supplemented this with a further 19 studies from our own literature search (see Table 3).

Fig. 1

Flow diagram.

Table 1

Studies including factors associated with engagement with online and mobile assessment and intervention studies.

Study and origin	Sample	N	Intervention	Engagement construct	Engagement operationalisation	Variables
						Significant/associated	Non-significant/associated
Ainsworth et al. (2013)UK	Sz, SA	24	Smartphone assessment app/SMS only	Using the intervention	No. of data point entries	Intervention modality: smartphone app w. graphical touch user interface > SMS text-only implementation
Ben-Zeev et al. (2014)USA	Sz, SA	33	Smartphone app	Using the intervention	% of days using the intervention, No. of interactions with the intervention on those days		Baseline cognitive functioning, negative symptoms, persecutory ideation, reading level
Ben-Zeev et al. (2016)USA	PDa	342	Smartphone app	(Longitudinal) engagement	No. days of intervention use/week, No. days responding to prompts/week, No. days initiating use/week, Average initiations/day	Females > males (all but no. days initiated use/week), white participants > ethnic minorities (all but average initiations/day), age: 30-45 years > 18-29 years (no. days initiated use/week); 46-60 years > 18-29 years (no. days initiated use/week), <18-29 years (no. days responding to prompts), +7previous hospitalisations<−7 hospitalisations (No. of days using the intervention)
Bucci et al. (2018)UK	EP	21	Digital health interventions	Subjective views	N/A (qualitative investigation)	Barriers: practical (forgetting to turn on or charge phone, losing or breaking phone), digital divide (e.g., poor data allowance)
Eisner et al. (2019)UK	Sz, SAb	16	Smartphone app	Engagement	N/A (qualitative investigation)	Barriers: mental health (positive symptoms, mood symptoms), phone-related barriers (lack of smartphone experience), other (scepticism, lack of literacy, physical illness, being busy, being away)
Granholm et al. (2012)USA	Sz, SA	55	Text-message intervention	Completion	Sending texts >2 weeks	Completers > self-reported living skills, <severe negative symptoms, >estimated premorbid verbal IQ than non-completers	Age, education, positive symptoms, depression
Kumar et al. (2018)USA	EP	61	Web-based intervention + smartphone app	Completion	Completing the study intervention, length of time in study, survey completion rates (daily, weekly)	Amongst completers: more severe baseline negative, agitation/mania symptoms associated with lower weekly survey completion rates	Overall sample: baseline positive, negative, depression/anxiety, agitation/mania symptoms
Maijala et al. (2015)Finland	Sz, SAc	100	Internet-delivered question & answer column	Using the intervention	Using the intervention	Age: 18–24 > 55-65 years, occupation: students > unemployed	Gender, education
Moitra et al. (2017)USA	PD	65	EMA	Completion	Completing >1 survey, no. completed surveys	Gender: females>males, cannabis use negatively associated with completion rates. Qualitative reasons for non-completion of surveys: did not understand procedure/could not get the device to work properly, inconvenient survey times. Reasons for withdrawal included: lack of interest, feeling overwhelmed, losing contact	Age, education, ethnicity, alcohol use, positive, negative, and affective symptoms, cognitive functioning
Niendam et al. (2018)USA	EP	76	EMA	Survey completion	Survey completion rates (daily, weekly)		Baseline positive, negative, depression/anxiety, agitation/mania symptoms
Palmier-Claus et al. (2012) UK	PD	44	EMA	Compliance, adherence	Completing >33% of surveys, no. of survey entries	More severe positive symptoms predicted non-compliance	Age, gender, negative or general symptoms, depression
Palmier-Claus et al. (2013)UK	NAP	24	EMA	Subjective views	N/A (qualitative investigation)	Repetitiveness of the questions led to disinterest. Need for variation in the content, number and order of items to avoid boredom and fatigue effect.
Rotondi et al. (2010)USA	Sz	31	Web-based intervention	Engagement	Time spent on the intervention, frequency of page visits	Positive symptoms positively associated with engagement indices
Schlosser et al. (2016)USA	EP/Sz	10	Mobile app	Engagement	Login frequency, average no. of challenges completed, challenge completion percentage, average no. of peer and coach interactions, active versus passive use, degree of social reciprocity from peer-to-peer and coach-to-peer interactions.	Increased frequency and personalisation and decreased content in coach interactions associated with increased engagement
So et al. (2013)UK	Sz, BPADd	26	PDA EMA	Compliance	Completing >33% of surveys, no. of survey entries	Reasons for non-completion: beeps were annoying, PDA or eye glasses went missing, physical illness, being paranoid about the PDA	Age, duration and dose of medication, number of admissions, and general, positive, negative symptoms.
Williams et al. (2018)Australia	Experiences of psychosis	36	Web-based intervention	Subjective views	N/A (qualitative investigation)	Using with a worker improved experience, limited access to Internet access at home (reception, accommodation, financial difficulties) limited use

BPAD: bipolar affective disorder, EMA: ecological momentary assessment, EP: early psychosis, N/A: not applicable, NAP: non-affective psychosis, PD: psychotic disorder, Sz: schizophrenia, SA: schizoaffective disorder; UK: United Kingdom, USA: United States of America.

Recently discharged from hospital.

Relapse in previous year.

Inpatients.

With acute delusions.

Table 2

Studies including factors associated with Internet use.

Study	Sample	N	Internet-related behaviour(s)	Variables
				Significant/associated	Non-significant/not associated
Aref-Adib et al. (2016)UK	PD	22	Internet use for MH related information	Barriers: financial (concerns about data, not having a smart-phone), being unwell (feeling unmotivated, finding resources too difficult to engage with), insufficient time, not wanting to think about MH (when well), difficulty processing information online
Baup and Verdoux (2017)France	Sz, SA, or BPAD	100	Internet use for medical information	Internet use associated with: younger age, higher educational level, having a computer at home. Reasons for not seeking medical information on the Internet were: lack of Internet skills, discussing health only with doctors or pharmacists, not being interested, not having Internet access
Bonet et al. (2018)Spain	EP, Chronic PD	105	Internet use (general and for mental health), Internet access, experience with technology, interest in eHealth	EP (younger, higher level of education) > Chronic PD (older, mostly unemployed) in frequency of Internet use (general), Internet use for social and mental health, access to devices
Fernández-Sotos et al. (2019)Spain	EP	180	Internet and technology use	Higher education related to more frequent computer Internet use	Age, gender
Lal et al. (2016)Canada	FEP	17	Internet use for MH related information	Intervention aspects promoting use: access to peers with lived experience, professional moderation. Concerns expressed regarding: content, source & impact of information
Miller et al. (2015)USA	Sz	80	Internet and technology use	Younger age positively associated with Internet and technology use. Females more likely to endorse paranoia associated with computer use. Males more likely to endorse worsening of voices associated with computer use.
Robotham et al. (2016)UK	Psychosis	121	Digital exclusion: Internet use, Internet access, confidence using the Internet	Barriers to Internet use: security concerns, lack of credit/money, lack of knowledge, lack of places to access the Internet, lack of availability, not wanting to use the Internet. Older age and longer duration of illness associated with digital exclusion.
Schrank et al. (2010)Vienna	Sz, SA	26	Internet use for health-related purposes	Reasons for not using: lack of access to a computer, financial problems, difficulties using technology, fear of computer viruses, fear of Internet addiction, preference for other sources of information and the expectation of low quality information, need for information already satisfied, lack of interest and the wish to rely on a doctor. Illness-related reasons: stimulus overflow, the inability to deal with the abundance of information, problems with concentration, lack of energy, depressive symptoms, paranoid ideas, fear of symptom provocation and the wish to distance oneself from illness-related topics as part of the recovery process
Spinzy et al. (2012)Israel	PD	143	Internet use, Internet access	Higher severity of illness associated with lower Internet use and access
Thomas et al. (2017)Australia	SMI	100	Internet use (general and for mental health), Internet access	Internet access associated with higher levels of education and younger age. Higher Internet use associated with younger age. Amongst those with regular access: female gender associated with higher general use, younger age and higher education level associated with use of the Internet for mental health	Internet access and Internet use for MH (amongst regular users): gender. Internet use (general): education
Välimäki et al. (2017)Finland	Sz spectrum	297	Internet use, Internet access	Age: participants 18–24 had greater Internet access than older age groups. Higher education associated with greater Internet access and use	General functioning
Villagonzalo et al. (2019)Australia	Sz-related, BPAD or MDD with psychotic features in the past 2 years	189	Internet use (general and for mental health)	General Internet use: positively associated with younger age, higher education level, higher executive functioning and processing speed scores; positive symptoms (highest amongst medium level Internet users); negative symptoms (highest amongst low level Internet users). Internet use for mental health: younger age, employed, higher occupational attainment, higher levels of loneliness	General Internet use: gender, current employment status, occupational attainment, attitudes towards utilising the Internet for MH treatment; working memory, premorbid IQ, symptoms (disorganisation, excitement, emotional distress). Internet use for MH: attitudes towards utilising the Internet for MH treatment, cognitive variables, self-efficacy, recovery style, internalised stigma, or psychotic symptom severity

BPAD: bipolar affective disorder, EP: early psychosis, FEP: first-episode psychosis, MDD: major depressive disorder, MH: mental health, PD: psychotic disorder, SA: schizoaffective disorder, SMI: severe mental illness, Sz: schizophrenia, UK: United Kingdom, USA: United States of America.

Table 3

Studies including factors associated with face-to-face treatments/services.

Study	Sample	N	Intervention	Engagement construct	Engagement operationalisation	Variables
						Significant/associated	Non-significant/not associated
Alvarez-Jimenez et al. (2009)Australia	FEP	41	CBTp	Adherence	Completion of the intervention (as indicated by the treating psychologist) and/or completing 80% of the planned intervention	Longer DUP and poorer level of insight associated with lower levels of adherence	Age, gender, marital status, education level, employment status, living with family, general illness/symptom severity, currently on antipsychotic medication, functioning level
Anderson et al. (2013)Canada	FEP	324	EPP	(Service) disengagement	No contact for 3 months	Older age and ethnic minority (black compared to white) were associated with an increased risk of disengagement. Individuals living alone had a reduced likelihood of disengagement	Gender, material deprivation, social deprivation, duration of untreated illness, substance abuse, symptom severity, police/ambulance contact
Casey et al. (2016)UK	FEP	103	EIS	(Service) engagement	SOLES score	No educational qualification>further education; living with others> living alone; positive association with duration of untreated illness (values >1220 days); positive association with beliefs that MH associated with social stress and odd thinking	Age, gender, ethnicity, socio-economic status, marital status. Beliefs about MH (family, expressed emotion, genetics, supernatural), diagnosis, DUP
Conus et al. (2010)Australia	FEP	660	EPP	(Service) disengagement	Case notes suggest active refusal of contact or untraceable	Previous forensic history, lower severity of illness at baseline, lower baseline functioning, persisting SUD, living without a family member at discharge predicted disengagement	Gender, family history of psychosis, severity of illness at discharge, global functioning at discharge
Currell et al. (2016)UK	CBTp therapists	21	CBTp	Completing CBTp	N/A (qualitative investigation)	Positive components: acceptance and application of the cognitive model, attending to the present - adequate concentration and memory, secure base - shorter duration of untreated illness and history of secure attachment, meaningful active collaboration - ability to tolerate new experiences/information/change and ability to collaborate to produce the formulation
Fanning et al. (2012)Ireland	FEP	88	Group CBTp	Attendance, adherence	Attending >1 session, attending >5/12 sessions	Attendance associated with greater education, lower negative symptoms, more insight into illness. Adherence associated with less positive & negative symptoms, less social self-consciousness	Living arrangement, marital status, socio-economic status, diagnosis, DUP, employment status, recovery style, premorbid adjustment, drug attitudes
Fung et al. (2008)Hong Kong	Sz	86	Psychiatric hospitals, community services	(Psychosocial treatment) adherence	PTCS: attendance and participation scales	Higher attendance associated with lower levels of internalised-stigma, better current insight on the social consequences of mental illness and living alone. Better participation associated with higher levels of self-esteem and better current insight on the social consequences of mental illness
Fung et al. (2010)Hong Kong	Sz	105	Psychiatric hospitals, community services	(Psychosocial treatment) adherence	PTCS: attendance and participation scales	Better treatment adherence associated with greater readiness for changing own mental health and less psychiatric symptoms, lower levels of internalised stigma
Gurak et al. (2017)USA	Sz and families	64	Culturally-informed family therapy	Adherence	No. of sessions attended	Adherence negatively associated with greater symptoms severity, ethnic minority of participants (black<white), higher levels of religious coping	Gender, education
Hazell et al. (2017)UK	People with experiences of hearing voices, MH clinicians	21, 201	Brief CBTp	Engagement	N/A (qualitative investigation)	Consumers reported experiencing cognitive difficulties could make engagement difficult. Positive symptoms (voice hearing) could sabotage or distract from treatment = potential barrier for engagement. Clinicians viewed positive symptoms or lack of insight, cognitive difficulties, and motivational issues viewed as potential hindrance to engagement
Johansen et al. (2011)Norway	FEP	148	Psychiatric treatment	(Service) engagement	SES total score	Lower engagement associated with lower neurocognitive functioning (conceptualization, verbal fluency, and semantic set shift) and more clinical symptoms (positive, negative, depressive/anxious, disorganized, and excitative components)	Age, education, no. years on medication
Kilbride et al. (2013)UK	Psychosis	9	CBTp	Subjective experiences, engagement	N/A (qualitative investigation)	Personal engagement and trust, partnership and collaboration considered important for engagement. Flexibility was highlighted as an important attribute of therapy that improved participants' ability to remain engaged. Personal motivation and agency were required to achieve progress and continue with CBT. CBT considered effortful due to concentration on specific cognitive processes; Barriers to engagement also include readiness for therapy
Kim et al. (2019)Australia	FEP	700	EIS	(Service) disengagement, re-engagement	Active refusal of contact or untraceable, making face-to-face contact with clinical staff following disengagement	Disengagement associated with: not being in employment/education/training; not having a second-degree relative with psychosis; concurrent substance abuse (cannabis & amphetamine)	Age (when controlling for confounding variables), gender, family history of psychosis, diagnosis (affective v non-affective) and DUP. No variables associated with re-engaging following disengagement
Kukla et al. (2014)USA	Sz spectrum	50	CBTp	Engagement (engagers v. non-engagers)	Attending >7 sessions	Engagers had a higher educational attainment than non-engagers. Higher cognitive functioning (working memory) and less severe negative symptoms at baseline were associated with engagement	Age, gender, ethnicity, psychiatric diagnosis, income level, history of psychiatric hospitalisations, self-esteem or work history as assessed at baseline
Lecomte et al. (2008)Canada	EP	118	Community services	(Service) engagement	SES total score	Poor service engagement associated with childhood physical abuse, male gender, history of legal issues. Good service engagement associated with strong alliance with the therapist, knowledge about consumer rights, personality traits of high neuroticism and low agreeableness	Age at baseline, substance abuse, symptoms, insight, social & daily living skills, history of physical/sexual abuse, age of onset, diagnosis
Lincoln et al. (2014)Germany	PD	80	CBTp	Treatment dropout	Dropping out from therapy during waiting phase or therapy	Participants who dropped out had been hospitalised less often, had lower levels of insight, lower social functioning and more negative symptoms	Age, gender, years of education, duration of illness, comorbid Axis I or II disorder, positive symptoms, cognitive flexibility, (working) memory
Ma et al. (2012)China	Psychosis (Sz, SA, BPAD, DD)	1386	Community services	Treatment dropout	Lost to follow up for 3 consecutive months	Dropout associated with: Poor socio-economic status, poor medication use, decline in clinical functioning, longer duration of illness, lower social functioning, lower levels of violence	Age, gender, education level, marital status, family history of any mental disorder, diagnosis
Macbeth et al. (2013)UK	FEP	64	EIS	(Service) engagement	SES total score and availability, collaboration, help-seeking and treatment adherence subscales	Poorer engagement associated with more positive symptoms, negative symptoms, general psychopathology, late adolescent academic adjustment, poor premorbid adjustment, female gender	DUP
Miller et al. (2009)US	FESz	112	Community services	Treatment dropout	Leaving treatment for >1 month and not following up with treatment elsewhere	Cannabis use	Age, socio-economic status, ethnicity.
Perivoliotis et al. (2010)UK	Psychosis outpatients	141	CBTp	Completion	Completing the course of CBT & completing follow up Qs.		Pre-treatment levels of overall cognitive insight, positive symptoms (severity of delusions or voice hearing experiences)
Perry et al. (2019)UK	FEP	98	EIS	Engagement, high/low engagement	SOLES total score, ≥/≤5 SOLES total score	Positive associations between engagement (continuous and binary) and biological, psychological and community treatment beliefs
Schimmelmann et al. (2006)Australia	FEP adolescents	134	EIS	(Service) disengagement	Active refusal of contact or untraceable	Disengagement associated with lower baseline severity of illness, higher baseline global functioning, living without family at baseline, a diagnosis other than schizophrenia, persistent substance use during treatment	Age, gender, family history of mental illness, past suicide attempts, psychiatric history, DUP, insight at baseline, comorbid diagnosis at baseline (including SUD), severity of illness or global functioning at discharge
Spidel et al. (2015)Canada	EP, PD (Forensic inpatients)	117	EPP, inpatient forensic hospital	Treatment adherence	SES total score and availability, collaboration, help-seeking and treatment adherence subscales	FEP group: poor service engagement associated with childhood physical abuse, more psychopathic traits, history of physical violent behaviour, more severe symptomatology. In Forensic: poor service engagement associated with a history of childhood abuse, more sever symptomatology	Regular alcohol use, regular drug use.
Startup et al. (2006)Australia	Sz spectrum	20	CBTp	Treatment dropout	Dropping out of treatment before the final session	Patients who dropped out of treatment, were less engaged in treatment, showed less agreement with their therapists, and had a sealing-over recovery style before they dropped out compared to those who stayed in the treatment	Therapeutic bond
Stewart (2013)Australia	EP	30	EPP	Engagement during initial stages of treatment	N/A (qualitative investigation)	Engagement support by positive relationship & experiences with treating team (accepting, genuine, optimistic, confident, flexible, and communicative characteristics of community staff). Successful engagement attributable to relationships in which clinicians taught individuals about the illness, guided them through treatment, identified and supported their personal strengths, and instilled an optimistic view of the future. They described these clinicians as genuine, unconditionally accepting, and comfortable with personal closeness. Many participants added that the introduction of the peer-group culture found at the EPP served to solidify the process of engagement
Stowkowy et al. (2012)Canada	FEP	286	EPP	(Service) disengagement	Leaving the program before 30 months: uncontactable or non-attendance for 3 months	Disengagement associated with lower negative symptom, shorter DUP, not having a family member involved in the program	Positive symptoms, general psychopathology, level of insight, depression, general functioning, premorbid functioning, quality of life, cannabis or other drug use, alcohol use, cognition
Tait et al. (2003)UK	Sz	50	Mental health treatment	(Service) engagement	SES total score and availability, collaboration, help-seeking and treatment adherence subscales	Lower engagement associated with a sealing-over recovery style relative to integration recovery style	General, positive and negative symptoms, insight
Thomas et al. (2018)USA	Sz, SA	46	Cognitive training	(Psychosocial treatment) engagement	Number of hours of attended group therapies, number of completed activities of daily living and number of hours of structured social or vocational rehabilitation activities	Higher engagement in psychosocial treatment associated with receiving the cognitive training intervention and greater cognitive abilities	Age, gender, illness duration, positive and negative symptoms
Tindall et al. (2018)Australia	Psychosis	14	EIS	Initial engagement	N/A (qualitative investigation)	An important aspect of initial engagement was the relationship between the young person and their case manager (trust, relaxed & approachable style)
Tsang et al. (2010)Hong Kong	Sz	105	Psychiatric hospitals, community services	(Psychosocial treatment) adherence	PTCS: attendance and participation scales	Better participation associated with higher global functioning, better readiness for action, and lower level of internalised stigma. Better attendance associated with less severe psychiatric symptoms and female gender
Turner et al. (2007)New Zealand	FEP	232	EPP	(Service) disengagement	Termination of treatment despite therapeutic need within 12 months of entry	Disengagement associated with current alcohol and/or cannabis abuse/dependence, initial diagnosis other than mood disorder, longer DUP, lower total symptomatology, lower negative symptoms, lower insight at referral, ethnic minority (Maori), unemployment	Age, gender, living with parents, positive symptoms, police contact before entry, percentage of inpatient and compulsory admissions, quality of life
Villeneuve et al. (2010)Canada	Sz spectrum	4372	Psychosocial treatment	Treatment dropout	Loss of participants, either prior or during treatment amongst persons who had agreed to undergo psychosocial treatment	Higher drop-out rates associated with older age, longer illness duration, longer treatment duration, male gender, lower impact factor journal, outpatient sample group	Severity of illness, treatment modality

BPAD: bipolar affective disorder, CBTp: cognitive behaviour therapy for psychosis, DD: depressive disorder, DUI: duration of untreated illness, DUP: duration of untreated psychosis, EIS: early intervention service, EPP: early psychosis program, FEP: first-episode psychosis, FESz: first-episode schizophrenia, MH: mental health, PTCS: Psychosocial Treatment Compliance Scale (Tsang et al., 2006), SA: schizoaffective disorder, SES: Service Engagement Scale (Tait et al., 2002), SOLES: Singh O'Brien Level of Engagement Scale (O'Brien et al., 2009), Sz: schizophrenia, UK: United Kingdom, USA: United States of America.

There was significant heterogeneity between studies in terms of the type of intervention, methodology, and definition and measurement of engagement. Several variables relevant to engagement with online interventions for psychosis were identified based on the synthesis of data from these identified studies Variables included factors relevant to both usage of the intervention and qualitative reports of users' subjective experience. These variables comprised illness-related, psychosocial, psychological, demographic, and intervention factors. Identified factors are summarised in Table 4 and detailed below.

Table 4

Summary of studies supporting factors identified as relevant to engagement.

Predictor	Face-to-face services	Internet use	Online interventions
Illness factors
Illness severity	3+, 6–, 6 not associated	2–	1 not associated
Positive symptoms	4–, 6 not associated	1 mixed, 1–	1+, 2 –, 6 not associated
Negative symptoms	2+, 4–, 2 not associated	2–	2–, 5 not associated
Cognitive functioning	6+, 2 not associated	3+	1+, 2 not associated
Duration of illness	2–, 2 not associated	1 earlier phase of illness	.
Insight into illness	6+, 5 not associated	.	.
Psychological factors
Recovery style	2 integrated, 1 not associated	2 integrated, 1 not associated	.
Internalised stigma	3–	1 not associated	.
Psychosocial factors
Functioning	3+, 2–, 3 not associated	1 not associated	1+
Substance use	5–, 4 not associated	.	1–
Internet access	.	4+	2+
Demographics
Age	2–, 12 not associated	6–, 1 not associated	1 mixed, 1–, 4 not associated
Gender	2 female, 1 male, 12 not associated	1 female, 2 not associated	2 female, 2 not associated
Education	2+, 1–, 5 not associated		3 not associated
Employment	2 employed, 3 not associated	1 productive employment	1 student
Computer literacy	.	3+	1+
Intervention aspects
Support	Therapeutic relationship 4+, 1 not associated	.	2+
Program format	1 treatment modality not associated	.	1 smartphone app>SMS, 1

+ positive association, − negative association.

Illness factors

Illness severity

Findings for traditional psychosocial interventions in psychotic disorders were mixed: several studies found no association between severity of general illness and the examined engagement construct (Alvarez-Jimenez et al., 2009; Anderson et al., 2013; Lecomte et al., 2008; Stowkowy et al., 2012; Tait et al., 2003; Villeneuve et al., 2010). A more severe illness presentation has been negatively associated with engagement with face-to-face treatments for first episode psychosis (FEP) and schizophrenia (Fung et al., 2008; Gurak et al., 2017; MacBeth et al., 2013; Spidel et al., 2015; Tsang et al., 2010). Conversely, lower baseline severity of illness predicted disengagement from early psychosis services in several studies (Conus et al., 2010; Schimmelmann et al., 2006; Turner et al., 2007).

Illness severity also appears relevant to Internet use: Spinzy et al. (2012) found that a more severe illness was associated with decreased Internet use amongst individuals with psychotic disorders.

Only one study reported on overall illness severity and online intervention engagement (Palmier-Claus et al., 2012). This study used ecological momentary assessment (EMA) completion as the dependent variable and found no significant association with assessment completion.

These mixed findings suggest a potentially complex relationship between symptom severity and engagement. Severe psychiatric symptoms may prevent individuals from participating in psychosocial treatment and accessing the Internet. Concurrently, individuals with low levels of symptoms, especially during early psychosis, may have less motivation or perceived need for treatment. Qualitative reports support this interpretation and highlight subjective experiences: some individuals with psychosis find Internet resources challenging to engage with when unwell, while not wanting to access mental health information when feeling well (Aref-Adib et al., 2016).

Positive symptoms

Positive symptoms of psychosis include paranoid ideation, delusional beliefs, hallucinations and disorganized thinking (Gaebel and Zielasek, 2015). Several studies demonstrated no association between severity of positive symptoms and engagement indices with traditional services (Lincoln et al., 2014; Perivoliotis et al., 2010; Stowkowy et al., 2012; Tait et al., 2003; Thomas et al., 2018; Turner et al., 2007). Conversely, more severe positive symptoms have been associated with lower levels of engagement amongst treatment for individuals with FEP (Fanning et al., 2012; Johansen et al., 2011; MacBeth et al., 2013).

One study examined severity of positive symptoms in relation to Internet use: Villagonzalo et al. (2019) found that individuals who used the Internet moderately (less than daily but at least weekly) had more severe positive symptoms than those who were high (at least daily) or low (less than weekly) users.

The majority of studies examining the association between positive symptoms and engagement with online interventions have found no association (Ben-Zeev et al., 2014; Granholm et al., 2012; Kumar et al., 2018; Moitra et al., 2017; Niendam et al., 2018; So et al., 2013). One study found a positive association between positive symptoms and use of an online psychoeducational website for schizophrenia (Rotondi et al., 2010). Conversely, one study found more severe positive symptoms significantly predicted non-compliance (completing <33% of entries) with an EMA intervention amongst individuals with schizophrenia-related disorders over one week (Palmier-Claus et al., 2012).

As with overall illness severity, these findings may reflect a complex relationship between engagement and severity of positive symptoms, potentially occurring upon a perceived needs basis (Rotondi et al., 2010). Qualitative reports suggest that positive symptoms may interfere with engagement with interventions and Internet use (Eisner et al., 2019; Hazell et al., 2017; Schrank et al., 2010). Conversely, individuals with low levels of positive symptoms, especially during an early phase of illness, may have limited engagement potentially relating to low perceived needs for treatment. Therefore, individuals with moderate positive symptoms may be more motivated and equipt to use a relevant online resource.

Negative symptoms

Negative symptoms in schizophrenia and other psychotic disorders include avolition, anhedonia, and asociality (Messinger et al., 2011). Relating to traditional psychosocial services, several studies demonstrated an association between negative symptoms and poorer engagement with treatment (Fanning et al., 2012; Johansen et al., 2011; Lincoln et al., 2014; MacBeth et al., 2013). Other studies found lower levels of negative symptoms were associated with disengagement from FEP services (Stowkowy et al., 2012; Turner et al., 2007) or were not associated (Tait et al., 2003; Thomas et al., 2018).

Only one study investigated the association between negative symptoms and Internet use: Villagonzalo et al. (2019) found that low Internet users had significantly more negative symptoms in comparison to high users.

Several studies have found no association between negative symptoms and usage or completion rates of mobile apps (Ben-Zeev et al., 2014; Moitra et al., 2017; Niendam et al., 2018; Palmier-Claus et al., 2012; So et al., 2013). However, more severe negative symptoms were associated with non-completion of a mobile CBT intervention (Granholm et al., 2012) and low completion rates for a mobile app for early psychosis (Kumar et al., 2018).

Although findings with regards to face-to-face services have been mixed, severe negative symptoms may present as a barrier to using the Internet and online interventions. As such, negative symptoms are likely to be associated with decreased engagement with online interventions.

Cognitive functioning

Psychotic disorders such as schizophrenia are associated with cognitive deficits across a broad variety of domains including processing speed, working and episodic memory, executive functioning, verbal ability, fluency, sustained attention and visuospatial capacities (Schaefer et al., 2013). Although not a unanimous finding (Lincoln et al., 2014; Stowkowy et al., 2012), better cognitive functioning has been associated with greater psychosocial treatment engagement in traditional settings (Currell et al., 2016; Johansen et al., 2011; Kukla et al., 2014; Thomas et al., 2018).

Better cognitive capacities were associated with higher levels of Internet use in one study: Villagonzalo et al. (2019) found higher levels of Internet use were associated with better executive functioning and processing speed capabilities.

Two studies found that cognitive functioning was not related to use of the clinical EMA mobile app FOCUS (Ben-Zeev et al., 2014), or EMA completion rates amongst individuals who had recently been discharged from hospital for psychosis (Moitra et al., 2017). However, lower estimated premorbid IQ was associated with non-completion rates of an EMA app amongst individuals with schizophrenia or schizoaffective disorder (Granholm et al., 2012).

Qualitative reports highlight cognitive difficulties as a barrier to Internet use and engagement with psychosocial treatments. Individuals with psychotic disorders have reported difficulties processing information online and not using the Internet as a result of stimulus overload and concentration difficulties (Aref-Adib et al., 2016; Schrank et al., 2010). Cognitive capacity has been highlighted as an important factor for engagement with therapies such as CBT for psychosis (CBTp) by both consumers and clinicians (Currell et al., 2016; Hazell et al., 2017; Kilbride et al., 2013).

Although cognitive functioning was not associated with completion rates of two online interventions, these studies were both limited by small sample sizes (Ben-Zeev et al., 2014; Moitra et al., 2017). As individuals with better cognitive functioning are more likely to use the Internet and potentially be better equipped to engage in psychosocial treatments, cognitive functioning is likely to be positively associated with engagement with online interventions.

Duration of illness

Following a first episode of psychosis, some individuals may have minimal or no further psychotic experiences (Alvarez-Jimenez et al., 2012). Conversely, other individuals will experience a more chronic course of illness with ongoing symptoms and associated disability (Morgan et al., 2014). A longer duration of illness has been associated with dropout from psychosocial treatment in a meta-analytic examination of individuals with schizophrenia spectrum disorders (Villeneuve et al., 2010) and a large scale (N = 1386) examination of individuals with psychosis (Ma et al., 2012). Two identified studies did not find an association between duration of illness and engagement with psychosocial treatments (Lincoln et al., 2014; Thomas et al., 2018).

Relating to Internet use, one study demonstrated that individuals with early-stage psychosis used the Internet more frequently and for mental health related purposes than individuals with a more chronic stage of psychosis (Bonet et al., 2018). A longer duration of illness (>3 years) has similarly been associated with greater risk of ‘digital exclusion’ – lacking access and confidence in using the Internet (Robotham et al., 2016).

No studies identified in this review investigated the association between duration of psychotic illness and engagement with online interventions. However, individuals with a longer duration or later stage of psychotic illness may have lower levels of engagement as a result of their lower Internet use and higher risk of dropout from psychosocial treatments. This association may potentially be explained by other potentially relevant stage-related factors such as age, symptom severity, or cognitive capacities (Bagney et al., 2013).

Insight

Individuals with psychosis may lack awareness regarding their symptoms or mental illness (Pini et al., 2001). Although not all studies have found an association (Lecomte et al., 2008; Perivoliotis et al., 2010; Schimmelmann et al., 2006; Stowkowy et al., 2012; Tait et al., 2003), limited insight into illness has been associated with non-attendance of treatments amongst individuals with FEP (Alvarez-Jimenez et al., 2009; Fanning et al., 2012; Turner et al., 2007) and persisting psychosis (Fung et al., 2008; Lincoln et al., 2014). Additionally, clinicians have qualitatively identified a lack of insight into illness as a barrier to engagement with CBTp (Hazell et al., 2017).

No identified studies examined the association between insight and either Internet use or engagement with online interventions. It is proposed the demonstrated association in face-to-face interventions may indicate that those with lower levels of insight into illness could also be less likely to engage in online interventions due to a lack of perceived need for treatment.

Psychological factors

Recovery style

Recovery style is distinguished from insight by its focus on attitudes towards ones' mental illness and is conceptualised as either ‘integrating’ or ‘sealing over’ (McGlashan et al., 1975). Those with an integrating recovery style generally acknowledge the significance of their psychosis, demonstrate curiosity towards their experiences and make active attempts to manage their illness. Conversely, a sealing over recovery style is characterized by minimising the significance of psychosis and a more negative view of one's mental illness (McGlashan et al., 1975).

In face-to-face psychosocial treatment, a sealing-over recovery style has been associated with significantly lower levels of service engagement (Tait et al., 2003) and greater likelihood of drop out from CBTp (Startup et al., 2006). One study found no association between recovery style and engagement (Fanning et al., 2012).

One study investigated the impact of recovery style on Internet use amongst individuals with psychosis but found no association with general or mental-health related Internet use (Villagonzalo et al., 2019). Qualitative reports highlight attitudes towards psychosis as potentially relevant to using the Internet. Individuals with a diagnosis of schizophrenia have reported wanting to distance themselves from illness-related topics on the Internet during their recovery (Schrank et al., 2010). Additionally, some individuals with psychosis have reported not wanting to think about their mental health when well (Aref-Adib et al., 2016).

Internalised stigma

Individuals with mental illness may internalise negative stereotypes against oneself (Corrigan and Watson, 2002). This internalised stigma may result in diminished self-esteem and self-efficacy for individuals with mental illness such as psychosis (Corrigan et al., 2006; Hill and Startup, 2013; Karakaş et al., 2016).

Each of the three identified studies examining the impact of internalised stigma in regard to psychosocial treatments found higher levels of internalised stigma was associated with poorer clinician-related treatment engagement (Fung et al., 2010; Fung et al., 2008; Tsang et al., 2010).

One study investigated the association between internalised stigma and Internet use and found no association with either general or mental-health related use (Villagonzalo et al., 2019).

No studies identified for this review investigated the association between internalised stigma and engagement with online interventions. As evinced by lower psychosocial treatment engagement, internalised stigma may limit the pursuit of goal-directed behaviour (Corrigan et al., 2009).

Psychosocial factors

Functioning

Many individuals with psychotic disorders have impaired occupational and daily functioning (Morgan et al., 2012). In face-to-face treatment, lower social and global functioning has predicted decreased participation and dropout from psychosocial treatment for schizophrenia (Lincoln et al., 2014; Ma et al., 2012; Tsang et al., 2010). Conversely, higher levels of baseline global functioning have been associated with disengagement from a specialised service for early psychosis (Conus et al., 2010; Schimmelmann et al., 2006). Other studies found no association (Alvarez-Jimenez et al., 2009; Lecomte et al., 2008; Stowkowy et al., 2012).

Global functioning was not associated with Internet use amongst Finish individuals with schizophrenia spectrum disorders in a large cross-sectional study (Välimäki et al., 2017).

One study investigated functioning in relation to engagement with an online intervention: non-completers of a clinical EMA intervention had lower levels of independent living skills compared to individuals who did complete the intervention (Granholm et al., 2012).

Substance use

Substance and alcohol use are common amongst individuals with psychotic disorders (Koskinen et al., 2009a, Koskinen et al., 2009b; Sara et al., 2014). In face-to-face therapies, alcohol, cannabis and other substance use have been associated with disengagement from early psychosis programs (Conus et al., 2010; Kim et al., 2019; Miller et al., 2009; Schimmelmann et al., 2006; Turner et al., 2007). Other investigations found no association between early psychosis service engagement and alcohol or substance use (Anderson et al., 2013; Fanning et al., 2012; Lecomte et al., 2008; Spidel et al., 2015; Stowkowy et al., 2012).

No studies examining the association between substance use and Internet use amongst individuals with psychosis were located.

One study examined substance use in relation to engagement with an online intervention: Moitra et al. (2017) examined the feasibility and acceptability of utilising EMA for individuals with psychotic disorders following discharge from hospitalisation. They found that individuals who did not complete any of the EMA surveys had significantly higher levels of recent cannabis consumption than individuals who completed any entries. The number of entries also decreased as cannabis consumption became more frequent.

Although preliminary in nature, these findings suggest that ongoing substance use may be a barrier to engagement with psychosocial interventions provided online.

Internet access

While not necessarily relevant to traditional psychosocial services, access to the Internet is a necessary precursor to engagement with online interventions. Several studies relating to Internet use amongst individuals with psychosis have highlighted limited access as a relevant barrier. Robotham et al. (2016) identified a lack of credit/money, lack of places to access the Internet, and lack of availability as the most commonly reported barriers. A lack of Internet access at home has also been identified as a barrier to utilising the Internet for medical-related information (Baup and Verdoux, 2017). Financial retrains and insufficient data are the most commonly reported qualitative barriers to accessing digital mental health information online and using online interventions (Aref-Adib et al., 2016; Bucci et al., 2018; Schrank et al., 2010; Williams et al., 2018).

Although accessibility is increasing, individuals with psychotic disorders may still represent a group of individuals with less access to the Internet than the general population (Robotham et al., 2016). Limited Internet access appears to present as a barrier to engagement for a subgroup of individuals with psychotic disorders.

Demographics

Several broad level demographic variables (age, gender, education, employment and computer literacy) were identified as candidate predictors in this review. We considered these factors unlikely to be directly associated with engagement and instead likely mediated by potential mechanisms of action and other factors.

Age

Regarding traditional psychosocial services, the majority of identified studies found no association between age and engagement (Alvarez-Jimenez et al., 2009; Johansen et al., 2011; Kim et al., 2019; Kukla et al., 2014; Lecomte et al., 2008; Lincoln et al., 2014; Ma et al., 2012; Miller et al., 2009; Schimmelmann et al., 2006; Thomas et al., 2018; Turner et al., 2007). However, older age has been associated with higher rates of dropout and disengagement from psychosocial treatments for schizophrenia and early psychosis (Anderson et al., 2013; Villeneuve et al., 2010).

Consistent with the general population (Poushter, 2016), a negative association has been demonstrated between age and Internet use amongst individuals with psychosis (Baup and Verdoux, 2017; Miller et al., 2015; Robotham et al., 2016; Thomas et al., 2017; Välimäki et al., 2017; Villagonzalo et al., 2019). Although, one study did not find an association amongst individuals with early psychosis (Fernández-Sotos et al., 2019).

The majority of studies examining age in relation to engagement with online interventions found no significant association with completion of EMA in psychosis (Granholm et al., 2012; Moitra et al., 2017; Palmier-Claus et al., 2012; So et al., 2013). One study found that use of an online question and answer column was most common amongst individuals aged 18–24 and least common amongst individuals aged 55–65 years old (Maijala et al., 2015). In contrast, Ben-Zeev et al. (2016) found that engagement (on-demand use) with the FOCUS app was significantly higher amongst individuals aged between 30 and 60 years compared to those aged 18–29 years old. However, in this study, participants younger than 30 years were significantly more responsive to prompts than older individuals (aged 46–60 years).

Given the potential for lower use of the Internet and higher risk of disengagement from psychosocial interventions, older age may also be associated with lower levels of engagement with online interventions. However, the relationship between age and engagement may not necessarily be linear: potentially specific age groups may be more or less likely to engage with online interventions, and different interventions (and intervention aspects) may be more or less appealing to different age groups with varying needs and interests (Pak et al., 2009).

Gender

While females are typically more help-seeking than males, findings between gender and engagement with psychosocial treatments for psychosis have been mixed (Thompson et al., 2016). Many studies found gender was not associated with service engagement (Alvarez-Jimenez et al., 2009; Anderson et al., 2013; Casey et al., 2016; Conus et al., 2010; Gurak et al., 2017; Kim et al., 2019; Kukla et al., 2014; Lincoln et al., 2014; Ma et al., 2012; Schimmelmann et al., 2006; Thomas et al., 2018; Turner et al., 2007). However, male gender was associated with increased risk of dropout in meta-analytic examination (Villeneuve et al., 2010), and lower rates of engagement amongst individuals with both early psychosis and schizophrenia (Lecomte et al., 2008; Tsang et al., 2010). Conversely, female gender was associated with lower levels of clinician-rated adherence with an early psychosis program (MacBeth et al., 2013).

Relating to Internet use, Thomas et al. (2017) found in a population of consumers predominantly diagnosed with schizophrenia-related disorders or mood disorders, amongst those with regular internet access (n = 66) females were more frequent users. However, in this study, gender was not associated with confidence using the Internet of the likelihood of utilising the Internet for mental-health related information amongst regular Internet users. Gender has not been associated with Internet use in two other investigations (Fernández-Sotos et al., 2019; Villagonzalo et al., 2019).

Four studies investigated the relationship between gender and engagement with online interventions. Gender was not associated with EMA schedule completion in one study (Palmier-Claus et al., 2012), or use of an online question and answer column (Maijala et al., 2015). Conversely, females used the FOCUS application more than males in a longitudinal examination over a six-month period (Ben-Zeev et al., 2016) and completed significantly more EMA entries following hospital discharge (Moitra et al., 2017).

There is limited indication that males are more likely to dropout from psychosocial treatments and use the Internet less. According to a systematic review by Beatty and Binnion (2016), preliminary results from EMA studies suggest that, consistent with other clinical populations, males appear likely to be less likely to engage in online interventions for psychosis than females.

Education

The findings between level of education and engagement with traditional psychosocial treatments have been mixed: A large number of studies found no association (Alvarez-Jimenez et al., 2009; Gurak et al., 2017; Johansen et al., 2011; Lincoln et al., 2014; Ma et al., 2012). Higher levels of education were associated with CBTp attendance in two studies (Fanning et al., 2012; Kukla et al., 2014), and one study found individuals with no educational qualification were more engaged within FEP services than individuals with further education (Casey et al., 2016).

Consistent with the general population (Anderson et al., 2019), higher educational attainment is associated with increased general (Fernández-Sotos et al., 2019; Välimäki et al., 2017; Villagonzalo et al., 2019) and mental health or medical-related Internet use amongst individuals with psychotic disorders (Baup and Verdoux, 2017; Thomas et al., 2017).

Educational attainment was not associated with use of an online question and answer column (Maijala et al., 2015) or EMA completion in two studies (Granholm et al., 2012; Moitra et al., 2017). However, these studies had small overall sample sizes and low numbers of individuals not completing the intervention.

Employment

While some studies found no association (Alvarez-Jimenez et al., 2009; Fanning et al., 2012; Kukla et al., 2014), some findings suggest a lack of current employment/vocation is linked to an increased risk of disengagement from early psychosis services (Kim et al., 2019; Turner et al., 2007).

One study examined the association between employment and Internet use: Villagonzalo et al. (2019) found that individuals using the Internet for mental health related information were significantly more likely to be currently engaged in productive employment than those who did not, with a small-medium difference.

Relating to online interventions, one study examined employment and found students were more likely than unemployed individuals to use an online question and answer column during psychiatric care (Maijala et al., 2015).

Individuals who are employed or actively engaged in equivalent activities appear to be more likely to use the Internet for their mental health. Additionally, these individuals may have reduced risk of disengagement from psychosocial treatment.

Computer literacy

Computer literacy is proposed to be an important factor in the uptake and continued use of technology (Venkatesh, 2000) and may be pertinent to engagement with online interventions. In surveys relating to Internet use amongst individuals with psychosis, commonly reported barriers have included both a lack of knowledge and skills (Baup and Verdoux, 2017; Robotham et al., 2016). Qualitatively, individuals with schizophrenia have reported not using the Internet due to difficulty using technology (Schrank et al., 2010).

While computer literacy has not been quantitatively examined in relation to engagement with online interventions; qualitative reports echo experiences relating to Internet use. In a post-hospital discharge EMA study, all four individuals who did not complete the intervention highlighted a lack of understanding and ability to work the mobile device as a reason for non-completion of items (Moitra et al., 2017). Similarly, a lack of familiarity with smartphones was reported as a barrier to engagement with a mobile monitoring app for psychosis relapse prevention (Eisner et al., 2019).

On the basis of qualitative reports relating to difficulties using the Internet and some online interventions, low computer literacy may be a barrier to engagement with online interventions for some individuals with psychosis.

Intervention aspects

Support

A positive relationship with a support person incorporating trust and guidance is an important factor promoting engagement with traditional services for people with psychosis (Kilbride et al., 2013; Lecomte et al., 2008; Stewart, 2013; Tindall et al., 2018).

While support has not been investigated in relation to Internet use amongst individuals with psychosis, preliminary evidence suggests support may facilitate engagement with online interventions. In Killikelly et al. (2017)'s review, interventions (mobile-basedassessment and intervention, websites and web-based interventions, virtual-reality, and peer-to-peer networking) with very high contact (20+/week) had the highest rates of adherence. However, adherence rates also remained high amongst interventions with less or no support. Schlosser et al. (2016) also examined differing frequencies of adjunct text-message support in their evaluation of a mobile application targeted at increasing motivation and functioning in people with schizophrenia. Following limited initial utilisation of the app, attempts were made to increase interactions: shorter and more casual text messages were sent to participants more frequently. This resulted in increased use of the app with participants completing significantly more activities (increasing from 3.9 to 11.2 active uses per week) and more participant-initiated interactions with coaches (increasing from 29.9 to 118.4). These results suggest that more frequent and brief support may best promote engagement.

Qualitative reports also highlight that additional contact support may facilitate practical assistance, further understanding of intervention content and increase motivation (Williams et al., 2018). In an examination of online mental health seeking behaviours, a small number (n = 2/22) of participants reported they would require clinicians support prior to utilising any future mental health applications (Williams et al., 2018).

The preliminary findings suggest that consistent with other mental health groups, adjunct support is likely to promote engagement with online interventions for individuals with psychosis (Baumeister et al., 2014). Additional contact support may facilitate further understanding of intervention content and increase motivation.

Program format

How the format of online interventions for psychosis may influence engagement has received relatively limited investigation. A number of included studies utilised EMA as an intervention focused on monitoring symptoms and treatment adherence as opposed to symptom reduction. In one study, participants with schizophrenia and schizoaffective disorder reported a preference for an EMA graphical smartphone app over text message only as it was easier to use and less time consuming (Ainsworth et al., 2013). Participants also completed significantly more EMA entries when using the smartphone app. Difficulties getting EMA to work and the inconvenient times of assessments have been identified as reasons for non-completion of assessments (Moitra et al., 2017). Thus, online interventions that are easy to use, intuitive, convenient and non-burdensome are likely to be utilised more amongst individuals with psychosis.

Conceptual model

This review aimed to synthesise relevant data and develop a working model of potential variables that may impact on engagement in online interventions in psychosis. Therefore, despite need for specific examination in future studies, we considered all identified variables as candidate predictors which may prompt further investigation into the relationship between these factors and engagement.

In considering how the identified variables could be synthesised into specific domains, we drew on previous conceptual models of engagement with online interventions (Perski et al., 2017; Short et al., 2015). These previous models have highlighted the varying impact of the individual user's characteristics, the intervention itself and psychosocial factors on engagement (Perski et al., 2017; Short et al., 2015). In synthesising the variables identified in this review, they may be considered as relating to four key domains: the direct impact of psychotic symptoms and illness; individuals' response to psychosis; the extent of prior and current day-to-day exposure and integration of technology; and intervention aspects (see Fig. 2). The direct impact of psychosis may interfere with one's motivation or ability to engage with an online intervention. Additionally, individuals with psychosis may have unique experiences that directly impact engagement. For example, positive symptoms may make individuals suspicious of technology or interfere with use of the intervention (Eisner et al., 2019; Miller et al., 2015). Negative symptoms or limited insight may impact motivation or perception of the intervention and accordingly decrease engagement (Aref-Adib et al., 2016; Hazell et al., 2017). Individuals with limited exposure to technology may lack the requisite skills or be unable to access online interventions, which may turn impede on utilisation of the Internet and also online interventions (Baup and Verdoux, 2017; Eisner et al., 2019). Individuals' response to psychosis, including attitudes and coping styles may also impact their interest in- and motivation to engage with online interventions. High levels of internalised stigma may impede goal-directed behaviour, a necessary requisite for engagement with self-guided online interventions (Corrigan et al., 2009; Mohr et al., 2013). Finally, intervention aspects such as adjunct support and program design may moderate the impact of the other individual-related domains. For example, adjunct support may improve motivation to engage with the resource (Mohr et al., 2011). Consistent with other disorders, technical support and simplified and intuitive intervention design could aid with difficulties using an online intervention associated with potential limited skills (Ainsworth et al., 2013; Williams et al., 2018).

Fig. 2

Conceptual model of factors influencing engagement with online interventions for psychosis.

The three user-related domains may hypothetically mediate the association between identified demographic factors and engagement. For example, the impact of psychotic illness may be greater for men than women on average: Men may have more negative and general symptoms of psychosis and poorer psychosocial outcomes than women with psychotic disorders (Grossman et al., 2008; Lang et al., 2013). There may also be gender differences in relation to individuals' response to illness: In the general population, women are more likely both to be help-seeking and to utilise the Internet for mental-health related information and therefore may be more likely to engage in online interventions (Thompson et al., 2016; Yousaf et al., 2015). While women and men are likely equally exposed to technology, women generally demonstrate more positive attitudes towards the Internet and are more likely to adopt online interventions more broadly (Bidmon and Terlutter, 2015; Carroll et al., 2017). As such, there may be gender differences in how individuals with psychosis engage with online interventions.

The relationship between age and engagement may also be partially mediated by these factors. Older adults are less likely in general, to seek assistance for mental health (Thompson et al., 2016). Middle-aged and older adults are less likely to adopt technologies, and older adults, in general, may be less enthusiastic or confident utilising technology (Carroll et al., 2017; van Deursen and Helsper, 2015). However, it is important not to assume that older adults should be excluded from utilising online interventions, as they demonstrate more positive than negative attitudes towards utilisation of technology (Mitzner et al., 2010). Further investigation is required to examine the relationship between age and engagement, as the relationship may not be linear.

It is likely that the association between engagement and education and employment is mediated by exposure to technology. Individuals who have limited education or long-term unemployment are not likely to be as familiar with utilising technology in day-to-day life and may have lower levels of computer literacy. Importantly, unemployment may also be reflective of the severity of illness: Individuals with psychosis with long-term unemployment may have a more severe illness with lower cognitive and general functioning and more negative symptoms (Goldberg et al., 2001; Ramsay et al., 2012; Waghorn et al., 2012). Together, individuals with lower levels of education and employment may, therefore, have additional barriers to engaging with online interventions.

Discussion

Our review identified 18 variables of potential importance in understanding engagement with digital interventions amongst people with psychosis. Factors identified in this review as relevant to engagement with psychosis were categorised in to four key domains: the direct impact of psychotic symptoms and illness; individuals' response to psychosis; the extent of prior and current day-to-day exposure and integration of technology; and intervention aspects. This model builds on previous conceptual models (e.g., Perski et al., 2017; Short et al., 2015) by focusing on factors specific to this population. However, due to the limited number of studies investigating engagement with online interventions, small sample sizes and presence of non-significant findings, these findings should be considered tentative. Overall, the current state of the evidence for their contribution to engagement is uncertain. For most factors, the evidence was equivocal, limited or potentially dependent on other factors. Most studies examined the variables separately, with the interdependence of factors rarely being considered. Although a conceptual model was developed, at this stage, the identified factors and interrelationships between factors in the proposed model remains hypothetical due to the limited research. However, this model may aid in furthering research attention to the examination of the potential importance of relevant factors and relationships between the identified factors and engagement. Further elucidation of these associations can aid design and dissemination of future online interventions for individuals with psychosis.

Design implications

Consistent with other clinical populations, incorporating some form of support appears beneficial in promoting engagement amongst individuals with psychosis (Baumeister et al., 2014). Ongoing support, such as emails or messaging may maintain motivation and engagement with otherwise self-guided interventions. The preliminary results from Schlosser et al. (2016) suggest that frequent and brief messages may be optimal to promote engagement. Where possible, personalised support may be optimal to increase engagement, although generic reminder alerts may also be sufficient where personalisation is not possible (Titov et al., 2013). Availability of technical support may be an especially important design element for some individuals with lower levels of computer literacy or other difficulties using the intervention (Robotham et al., 2016).

One potential method of providing adjunct support may be via some initial in-person contact. For example, clinicians could provide training in how to use the intervention to improve users' confidence and skills, and also work to identify personally relevant goals to build motivation. This type of blended treatment has recently been shown to be feasible, acceptable and clinically beneficial for individuals with serious mental illness including psychosis (Bell et al., 2018; Depp et al., 2018).

In further considering the role of online interventions, results suggest that individuals with severe psychotic symptoms may be able to utilise online interventions successfully. However, extra assistance or guidance may be required to overcome the impact of symptoms. Additionally, individuals may have limited capacity to engage with this type of resource during a psychotic episode. Clinicians may therefore wish to consider postponing promotion or use of online interventions until a reduction in acuity. To afford for fluctuations in symptoms and access, interventions should incorporate a more flexible design that can accommodate periods of non-use. Given several of the factors identified in this review are not static, the role of online interventions is likely also to change over time. A flexible approach to incorporating online interventions into psychosocial services may therefore be required. Services should enable clients to choose when and how they utilise online interventions.

Limitations and future research

The field of engagement research has some clear limitations. While the broad search approach enabled identification of additional relevant factors (including duration of illness, insight into illness, recovery style, and internalised stigma), there is currently a lack of research specifically investigating factors predictive of engagement with online interventions for psychosis. Studies that did examine relevant factors, tended to examine variables separately, limiting understanding of potential interrelationships. Many studies in this review had small sample sizes, limiting statistical power and ability to detect significant effectsIt remains unclear how engagement may vary for different types of online interventions (e.g., symptom monitoring versus web-based interventions). Given the varying study types and purposes, we did not conduct an assessment of study quality or statistical analyses. Finally, the large majority of studies originated from western countries, limiting the understanding of how these factors may operate in developing countries. In light of these limitations, the relevant predictors, mediators and conceptual framework provided in this review should be considered preliminary in nature. The specific associations between proposed factors remain largely unknown. Additionally, due to delays and resource limitations, the original search may have missed more recent relevant studies. Future research should aim to further investigate the associations identified in this review as well as examine interrelationships between identified factors and engagement.

To date, there has been minimal investigation into the impact of intervention manipulation on engagement. In addition to examining what individual-related variables are associated with engagement with online interventions for psychosis future research should also investigate the impact of design and intervention elements on engagement and associated benefits. Rather than focusing on a single index of engagement, research would benefit from a multi-dimensional examination of engagement. Mixed methods research incorporating both quantitative and qualitative methods will be especially informative in this emerging area of research.

Conclusion

Individuals with psychosis present with various challenges in relation to engagement with online interventions. Engagement may be directly impacted by psychotic illness, as well as individuals' response to experiencing psychosis. Individuals with psychosis may face additional barriers to utilising technology, including limited accessibility and computer literacy. Clinicians and developers of future interventions should aim to tailor such interventions to fit the relevant needs of this client group. Interventions should be flexible, easy to use, aim to promote motivation and incorporate some form of support. Further research specifically investigating predictors of engagement with online interventions and examining the proposed models is required.

The following is the supplementary data related to this article.

Supplementary Table 1

Declaration of competing interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property.

We further confirm that any aspect of the work covered in this manuscript that has involved either experimental animals or human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript.

We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author and which has been configured to accept email from the journal.