Can cognitive remediation therapy be delivered remotely? A review examining feasibility and acceptability of remote interventions.

Cognitive remediation (CR) is an effective treatment for schizophrenia. However, issues such as motivational impairments, geographic limitations, and limited availability of specialized clinicians to deliver CR, can impede dissemination. Remote delivery of CR provides an opportunity to implement CR on a broader scale. While empirical support for the efficacy of in-person CR is robust, the evidence-base for virtual delivery of CR is limited. Thus, in this review we aimed to evaluate the feasibility and acceptability of remote CR interventions. Nine (n = 847) fully remote and one hybrid CR intervention were included in this review. Attrition rates for remote CR were generally high compared to control groups. Acceptability rates for remote CR interventions were high and responses from caregivers were positive. Further research using more methodologically rigorous designs is required to evaluate appropriate adaptations for remote treatment and determine which populations may benefit more from remote CR.

Introduction

Schizophrenia-spectrum disorders (SSDs) affect over 20 million people worldwide (World Health Organization, 2019). They are associated with significant functional impairments and are one of the top 15 leading causes of disability (GBD 2016 Disease and Injury Incidence and Prevalence Collaborators, 2017). Furthermore, only about 19% of individuals with schizophrenia-spectrum disorders achieve “functional recovery” (Jääskeläinen et al., 2013), which refers to recovering the ability to perform daily activities in domains such as vocation, self-care, and social activities (Harvey and Bellack, 2009).

Impairments in neurocognitive abilities are the best predictor of community functioning in schizophrenia-spectrum disorders, and even when individuals possess functional skills, they continue to display impairments in real world functioning (Best et al., 2014). This is concerning because more than 70% of individuals with schizophrenia-spectrum disorders experience neurocognitive impairment (Keefe and Fenton, 2007; Reichenberg, 2010). Furthermore, neurocognitive impairment is present prior to disorder onset (Fusar-Poli et al., 2012) and is observed in unaffected first-degree relatives (Horan et al., 2008; Ma et al., 2007), possibly representing an endophenotypic marker for schizophrenia-spectrum disorders (Barrett et al., 2018).

Cognitive Remediation (CR) is an evidence-based treatment that improves neurocognition (Best and Bowie, 2017), including both global neurocognition and functioning (Vita et al., 2021; Wykes et al., 2011). CR is effective for individuals in both early (Revell et al., 2015) and late (Cella et al., 2017) stages of schizophrenia. It is typically conducted in-person, however, attrition rates from in-person CR (approximately 25%) exceed the average drop-out rates of other complex interventions for schizophrenia (Szymczynska et al., 2017).

There are numerous client and therapist-related factors which may affect treatment adherence in CR. Namely, there are certain clinical characteristics which may predict lower adherence during treatment. Motivational impairments may be one factor affecting attrition from CR. Motivational impairments are associated with general treatment disengagement (Drieschner et al., 2004), and may make it challenging for clients to attend appointments (Thomas et al., 2018). Remote treatment delivery can be easily accessed from clients' homes and may encourage treatment adherence for individuals who have motivational impairments (Jimenez et al., 2019). Conversely, while specific motivational impairments are associated with treatment disengagement, past research has also demonstrated that those with higher negative symptom profiles may have greater adherence to CR programs (Dillon et al., 2016), supporting the notion that there may be a specific symptom profile of individuals who may benefit most from CR. Furthermore, cognitive impairments themselves may also impede therapeutic engagement and attendance while participating in CR (e.g. forgetting to practice cognitive exercises). Additionally, with regards to therapist-related factors, therapeutic alliance has also been demonstrated to predict adherence for computerized CR interventions (Hargreaves et al., 2018).

Adherence to CR treatment may also be addressed through addressing beliefs individuals with schizophrenia have about their illness and accessibility of CR as a treatment strategy. According to the Health Beliefs Model, addressing an individual's perceptions of barriers towards participating in treatment interventions, can improve overall adherence to clinical treatment interventions (Jones et al., 2015).

Thus, on a broader scale, despite the established efficacy of CR, implementation has remained a barrier to widespread access to treatment. Remote delivery methods may help expand delivery of CR to remote geographic areas that are typically underserviced by specialty mental health resources (Iyer et al., 2015), and provide treatment access to historically underserved communities. For example, people living in rural Indigenous communities must often relocate to urban centers to access specialist mental health services that are not available within their community (McGrath et al., 2006; Nagel et al., 2009). More generally, the COVID-19 pandemic resulted in social restrictions that made in-person treatment inaccessible and encouraged a movement towards remote delivery (Medalia et al., 2020). Remote delivery of CR is a potential method of overcoming some of these challenges, however, the feasibility and acceptability of remotely delivered CR are currently unclear.

Methods

This review was pre-registered in the PROSPERO database (CRD42020189460) as part of a larger effort to examine the evidence base for remote delivery of psychological treatments for schizophrenia.

Data sources

A literature search was conducted for articles published between January 1990 to January 2022 in Ovid, MEDLINE, EMBASE, and PsychINFO.

Study selection

The literature search consisted of a title and abstract screening phase, and a full text screening phase. Three reviewers were involved in all screening phases and supervised by the principal investigator (MB). Discrepancies in ratings were resolved by the principal investigator. The Cochrane Central Registry Of Controlled Trials central registry of controlled trials was also searched to determine whether any clinical trials were registered that may have been missed in the database search. Searches were restricted to papers published in English. The following search terms were used: (“psychotic” OR “psychosis” OR “schizo*”) AND (“computer” OR “phone” OR “tablet” OR “mobile” OR “internet” OR “online” OR “web” OR “app” OR “virtual” OR “telehealth” OR “remote”) AND “cognit*” AND (“training” OR “remediation” OR “rehabilitation” OR “enhancement”). Furthermore, the inclusion criteria were as follows: (1) at least 50% of the participants were diagnosed with schizophrenia spectrum disorders; (2) the treatment delivered during the study was CR; (3) CR must have been delivered by some form of remote method that was not in-person and (4) only full papers (i.e., no conference abstracts) were selected.

Data extraction

During the data extraction phase, we collected information on: (a) study methodology, including sample size, participant diagnosis, study design, type of control group used, and length of follow-up, and (b) in which studies researchers trained higher-order processes (i.e., executive functioning) versus lower-order processes (i.e., perceptual functioning; Best and Bowie, 2017). To assess feasibility of remote CR, total rates of attrition were calculated for all participants in the study (see Table 2). Attrition was calculated as the proportion of participants who dropped out of the study after being assigned to complete the CR intervention. Information of acceptability of remote CR was collected for studies that reported satisfaction and/or improvement from remote CR. For studies that included responses from caregivers, information was also collected on caregiver response and their perceptions of acceptability of remote CR.

Table 2

Study feasibility.

Authors	Participant age	Total study attrition rate	Attrition rate for cr group	Attrition rate for control group
Biagianti et al., 2017	44.8	36.5%	36.5%	N/A
Donohoe et al., 2018	43.3	38.9%	47.9%	32.4%
Fisher et al., 2015	21.2	27.2%	30.2%	24.1%
Hargreaves et al., 2015	43.0	27.0%	27.0%	Not reported
Ventura et al., 2013	Not reported	37.5%	37.5%	N/A
Harris et al., 2017	39.6	39.6%	34.9%	44.1%
Melo Moura et al., 2019	23.6	29.4%	29.4%	N/A
Loewy et al., 2021	21.1	27.9%	29.6%	25.8%
Krzystanek et al., 2020	32.1	30.3%	28.6%	34.4%
Total	29.9	31.8%	32.3%	30.4%

Data synthesis

Information on study methodology, attrition, acceptability, and caregiver response was synthesized to evaluate overall feasibility of remote CR and factors associated with increased feasibility and study adherence.

Results

Included studies

Nine studies, with a total of 847 participants (M age = 29.9), met inclusion criteria for virtual-CR (see Table 1). These studies used a variety of training approaches to CR including auditory/verbal working memory training (Fisher et al., 2015; Hargreaves et al., 2015; Krzystanek et al., 2020; Loewy et al., 2021; Ventura et al., 2013) and auditory processing training (Ventura et al., 2013). A variety of online cognitive training programs were used including Posit Science (Fisher et al., 2015; Loewy et al., 2021), Scientific Brain Training Pro (Harris et al., 2017), COGWEB, Luminosity (Harris et al., 2017), MyBrainSolution (Harris et al., 2017), Brain HQ (Biagianti et al., 2017; Harris et al., 2017), as well as the novel smartphone training application MONEO (Krzystanek et al., 2020). These programs targeted a range of cognitive processes (Best and Bowie, 2017), including higher order executive processes such as working memory training (Donohoe et al., 2018; Hargreaves et al., 2015; Krzystanek et al., 2020), lower-order perceptual processes such as auditory processing (Biagianti et al., 2017; Fisher et al., 2015; Loewy et al., 2021; Ventura et al., 2013), and non-targeted processes (Harris et al., 2017; Melo Moura et al., 2019). Only one study directly compared at-home iPad-based cognitive remediation to cognitive remediation delivered in-person in a laboratory setting and this was a non-randomized trial (Biagianti et al., 2017). In addition to the online cognitive training exercises, some studies also incorporated other therapeutic components of CR such as strategy monitoring (Donohoe et al., 2018), psychoeducation (Donohoe et al., 2018) and goal setting (Fisher et al., 2015; Loewy et al., 2021).

Table 1

Study characteristics of all included studies.

Authors	n for CR condition	n for control condition	Type of remote CR	Study design	Diagnosis	Training type	Type of training approach	Length of follow-up	Type of control group	Primary outcomes assessed	Training length
Biagianti et al., 2017	74	N/A	Fully remote	Non-randomized study: CR using laptop vs. CR on iPad	Schizophrenia	Posit Science; auditory processing and auditory/verbal working memory	Lower order	No follow-up	N/Aa	Neurocognition and functioning	5 days a week; 60 min a day; 8 weeks
Donohoe et al., 2018	48	42	Fully remote	RCT; Remote CR vs. active control condition	Psychosis	Working Memory Training + CR techniques (Psychoeducation, Strategy Monitoring); targeting both audio and visual working memory.	Higher order	2 week, 3–6 months	Active Control Condition	Neurocognition and functioning	5 days a week; 30–40 min a day; 8 weeks
Fisher et al., 2015	43	43	Fully remote	RCT; Remote CR vs. computer games	Recent-onset schizophrenia	Posit Science; auditory processing and auditory/verbal working memory	Lower order	No follow-up	Computer games	Neurocognition and functioning	5 days a week; 1 h a day; 8 weeks
Hargreaves et al., 2015	22	26	Fully remote	Non-randomized study; Remote CR vs. TAU	Psychosis	Working Memory Training; ecologically valid auditory and visual working memory training	Higher order	No follow-up	Treatment as usual	Neurocognition	5 days a week; 30–40 min of training; 40 days within 12-weeks
Ventura et al., 2013	9	N/A	Fully remote	Non-randomized study; no control group	Schizophrenia	Posit Science; auditory processing and auditory/verbal working memory	Lower order	No follow-up	No control condition	Neurocognition and functioning	2 h long sessions per week; 6 weeks
Harris et al., 2017	43	43	Fully remote	RCT; Remote CR vs. supported employment vs. Internet-based control condition	Severe Mental Illness	Lumosity, Posit Science, My BrainSolution, Scientific Brain Training Pro	Non-targeted	No follow-up	Internet-based control condition	Functioning	At least twice a week; 10 h over a 4-month period
Melo Moura et al., 2019	17	N/A	Fully remote	Non-randomized study;	First Episode Psychosis	COGWEB; computerized exercises focused on enhancing attention, memory, and executive functions	Non-targeted	No follow-up	No control condition	Neurocognition and functioning	5 days a week; 30–35 min; 6 months
Loewy et al., 2021b	81	66	Fully Remote	RCT; Remote CR vs. computer games	Recent-onset schizophrenia	Posit Science; auditory processing and auditory/verbal working memory	Lower order	6-month follow-up	Computer games	Neurocognition and functioning	5 days a week; 1 h a day; 8 weeks
Krzystanek et al., 2020	199	91	Fully remote	RCT; Remote smartphone-based CR vs. placebo	Paranoid Schizophrenia	MONEO smartphone application; visual working memory	Higher order	6-month, 12-month	Inactive/Limited version of cognitive training application	Neurocognition	Twice a week; around 10 min per session

This study did not have any control groups (all participants received CR), however the modality in which participants received CR varied by comparison group (laptop computer vs. iPad).

This study presents follow-up data from Fisher et al.'s (2015) initial investigation.

Of the nine studies discussed in this review, five studies (including one follow-up study) were randomized controlled trials (Donohoe et al., 2018; Fisher et al., 2015; Harris et al., 2017; Loewy et al., 2021; Krzystanek et al., 2020). The remaining four studies were non-randomized studies (Biagianti et al., 2017; Hargreaves et al., 2015; Ventura et al., 2013; Melo Moura et al., 2019). Two studies did not incorporate a control group in their design (Ventura et al., 2013; Melo Moura et al., 2019). Control groups for the remaining studies ranged from treatment as usual comprising of regular multidisciplinary treatment and supportive group interventions (Hargreaves et al., 2015), playing commercially available computer games (Fisher et al., 2015; Loewy et al., 2021), participating in an internet-based supported employment program (Harris et al., 2017), as well as being given access to a limited version of the training application (Krzystanek et al., 2020). Two of our included studies included a follow-up assessment (Donohoe et al., 2018; Loewy et al., 2021). A study by Donohoe et al. (2018) assessed for improvements in task performance and functioning at both a two-week and a three-month follow-up, while a study by Loewy et al. (2021) assessed for improvements in neurocognition and functioning at a six-month follow-up of another included study (Fisher et al., 2015).

Of the nine included in this review, the primary outcome measures assessed were neurocognition and functioning, with all studies except one assessing neurocognition as an outcome (Harris et al., 2017), and all except two assessing functioning as an outcome (Hargreaves et al., 2015: Krzystanek et al., 2020). Two of the studies were feasibility and acceptability studies (Biagianti et al., 2017; Melo Moura et al., 2019).

Three studies had less intensive training requirements of only one or two days per week (Harris et al., 2017; Krzystanek et al., 2020; Ventura et al., 2013), and all other studies required cognitive training to be completed five days per week. Training sessions lasted approximately 30 to 60 min per day, with interventions being offered over a range from six weeks to six months. The included studies utilized weekly phone call check-ins to allow study personnel to provide coaching and keep up with participant progress. The majority of the studies had participants complete the cognitive training on either loaned or personal computers, however, one study conducted the remote cognitive training using iPads (Biagianti et al., 2017).

Partially remote CR

In addition to the included studies comprising of fully remote CR interventions, we also found one study examining partially remote CR (Medalia et al., 2021). In this study, a hybrid model of CR was examined, which involved completing two 60-minute sessions of remote CR weekly plus one in-person session with a therapist. In addition to completing CR training exercises, this study also utilized bridging as a therapeutic technique.

Adherence/feasibility of remote CR

Overall attrition rates across the studies for participants assigned to complete CR training ranged from 27.0% to 47.9%, with the average attrition rate for participants who took part in a remote CR intervention being 32.3%. See Table 2 for a summary of study adherence rates for all nine of our included studies. Factors contributing to attrition rates as noted by participants included lack of interest in the study (Melo Moura et al., 2019), clinical instability (Melo Moura et al., 2019), mental fatigue (Melo Moura et al., 2019), serious medical illness (Ventura et al., 2013), and a change in availability for study participation/not having enough time to complete training (Biagianti et al., 2017; Melo Moura et al., 2019).

Overall, among all nine included studies, the most consistently reported reason for participant drop-out was that the CR was perceived to be too demanding, or participants were not available to complete the training (Biagianti et al., 2017; Hargreaves et al., 2015; Melo Moura et al., 2019). Another significant factor was that some participants were not completely comfortable with using a computer (Hargreaves et al., 2015; Medalia et al., 2021). Furthermore, one study that reported an attrition rate of 35% suggested that participants who dropped out were significantly less likely to have strong social networks compared to participants who completed treatment (Harris et al., 2017). Many of the included studies did not report what stage of the cognitive training participants tended to drop out, however, in one study with a high attrition rate of 48%, researchers found that most participants tended to drop out of the study during the first two weeks of training (Donohoe et al., 2018).

Of the five studies that included a control group (Donohoe et al., 2018; Fisher et al., 2015; Hargreaves et al., 2015; Harris et al., 2017; Krzystanek et al., 2020), one study reported no significant difference in attrition rates between study conditions (Harris et al., 2017) and four studies did report an increased attrition rate for the remote CR condition relative to the control condition (Donohoe et al., 2018; Hargreaves et al., 2015; Krzystanek et al., 2020; Loewy et al., 2021). In particular, the drop-out rate following the treatment period in Donohoe et al.' (2018) study was substantially higher for participants in the remote CR condition (48%) than for participants in the control condition (32%). This study utilized a novel web-based CR program, targeting both auditory and visual memory processes (Donohoe et al., 2018).

Furthermore, while most of the studies relied on computers to deliver virtual-CR, one study examined CR delivered through both laptops and iPads (Biagianti et al., 2017). There were no differences in total training time or training intensity for participants who completed remote cognitive training on iPads or in-person cognitive training through a computer, providing preliminary evidence that both methods support good adherence. Additionally, one other study also utilized a smartphone application to deliver the virtual-CR application (Krzystanek et al., 2020). This study reported low to moderate attrition rates across therapeutic conditions (28.6%–34.4%) compared to other studies we reviewed, suggesting delivery of remote-CR through smartphones may be a viable option for clinicians.

Acceptability of remote CR

Of the nine studies included in our review, two specifically examined acceptability of remote CR through semi-structured interviews (Melo Moura et al., 2019), as well as through self-reported ratings of enjoyment (Fisher et al., 2015). In addition to reporting overall satisfaction with a remote CR intervention, participants expressed self-reported improvements with memory and concentration due to the training and reported more participation in daily activities after the treatment (Melo Moura et al., 2019). However, 33.3% of study participants who dropped out due to loss of interest in the study also reported experiencing mental fatigue in response to the training, which was identified as being one of the prominent reasons for participant attrition. The second study, conducted by Fisher et al. (2015) found that those who participated in the active CR condition overall “slightly enjoyed” participating in the intervention, however, self-reported ratings of enjoyment did not significantly differ between the active CR condition and the control condition.

Response from caregivers

Two studies discussed the role of caregivers in the intervention. Ventura and colleagues (2013) recruited both outpatients with schizophrenia and their relatives to participate together in a home-based CR study. Relatives participated alongside participants in cognitive training sessions assisted in scheduling cognitive practice sessions, and motivated participants and provided emotional support. Following the study, both caregivers and participants demonstrated increased knowledge about the role of cognition in daily living and caregivers reported noticing subjective improvements in cognition in the participants (Ventura et al., 2013). Despite the inclusion of caregivers alongside clinical participants, the attrition rate in this study was among the highest of all included studies, with 37.5% of participants dropping out after starting the remote training. However, this large rate of attrition could have been due, in part, to the very small sample size of the study.

In addition to examining clinical participants' subjective appraisal of remote CR, Melo Moura et al.' (2019) also gathered feedback from participants' caregivers. Caregivers were asked to provide feedback on the intervention and any changes they noticed in their relatives over the course of the study. Caregivers reported that their relatives demonstrated both objective and subjective improvements in cognitive functioning. In addition to clinical participants reporting satisfaction with remote CR, caregivers also supported the acceptability of the intervention and denied any negative consequences of taking part in the study.

Hybrid cognitive remediation

Medalia et al.' (2021) examined hybrid CR, which comprised a combination of both (a) in-person clinician-led group CR training and (b) independent practice of cognitive exercises completed at home. The in-person sessions incorporated discussions in which, participants linked their completed (a) 3–4 cognitive training exercises to daily activities selected by the clinician for 45-min, and b) a 15-min “bridging” sessions. In addition to completing these weekly sessions, participants also completed 60-min of computerized cognitive practice “homework” remotely.

Feasibility of hybrid CR

Participants who partook in a hybrid delivery model of CR reported discomfort/lack of skill with using a computer (Medalia et al., 2021), connectivity issues (Medalia et al., 2021), and not having access to a computer (Medalia et al., 2021) as being the primary reasons for study withdrawal. Seventeen participants withdrew from this study following randomization.

Acceptability of Hybrid CR

In the additional study we reviewed by Medalia et al. (2021), the researchers compared an in-person version of CR to a partially remote (hybrid) version of CR delivered to individuals with schizophrenia and schizoaffective disorder, Participants reported no difference in satisfaction between completing the exercises independently at home versus in -person in the clinic. Acceptability ratings for this study were also gathered through semi-structured interviews. Although participants in this study reported that independently completing computerized exercises was challenging at times, they also reported that the hybrid version of CR enabled them to feel a greater sense of independence. The majority of participants reported that the hybrid model still allowed them to receive sufficient support and guidance from clinicians.

Additional studies

In addition to the aforementioned studies, our literature search uncovered two additional studies which in which participants were provided the option in which to receive remote CR. A recent study by Miley et al. (2019) provided six-month follow-up data on one of our included remote-CR studies (Biagianti et al., 2017). However, the study investigators did not determine whether treatment outcome, treatment adherence, or treatment acceptability differed based on remote treatment status. Furthermore, a study by Scoriels et al. (2020) examined the efficacy of auditory and visual cognitive training on neurocognition, psychotic symptoms, and quality of life in a sample of individuals with schizophrenia. In this study participants were given the option to conduct the training sessions remotely, however, results on whether treatment outcomes differed based on remote treatment status were not provided.

Discussion

The present study sought to examine the current evidence-base for remote delivery of cognitive remediation for schizophrenia. Although few studies have examined remote CR, preliminary evidence suggests that it may be feasible and acceptable. Nine studies evaluated fully remote CR in individuals with SSDs and one recent study examined a hybrid version of CR (Medalia et al., 2021). We also found two additional studies which examined further remote applications of CR, however, these studies were not fully remote and/or utilized mixed models of delivery. Overall, the methodological rigour of the studies was poor, with only four studies conducting randomized trials, only three studies including follow-up assessments, and no studies conducting a full randomized trial comparing in-person to virtual treatment.

The average attrition rates for participants who took part in a remote CR intervention was 32.3%. This is qualitatively higher than previously reported attrition rates (25%) for in-person CR received by participants with schizophrenia (Szymczynska et al., 2017). The high attrition rate for remote CR was contrary to our hypotheses that remote CR may encourage greater treatment attendance. Developing methods to further promote motivation and engagement during cognitive training may be especially important for virtual CR. The most common reason for dropping out of the included studies was that participants found the training to be too demanding. However, the length and frequency of sessions of remote CR interventions included in this review are consistent with current CR treatment standards (Bowie et al., 2020), suggesting that this may be an issue for CR more broadly. Furthermore, to help offset the intensive training schedule, making the training more gamified or “fun” may motivate participants more to complete the course of treatment. Greater attendance of CR sessions and more time spent on completing homework for CR is also associated with better outcomes for neurocognition and functioning (Best et al., 2019, Best et al., 2019). Indeed, gamification of cognitive training tasks has been shown to increase motivation and engagement during treatment, across a range of psychiatric disorders including schizophrenia (Lumsden et al., 2016). It may be especially important for the computerized exercises to be inherently motivating in remote CR, since other factors which may contribute to motivation and accountability, such as the physical presence of a therapist and other participants, are omitted in this treatment model. This could be accomplished by incorporating elements of gamification in computerized exercises such as the use of leaderboards, levels, and narrative structure (Deterding et al., 2011). In addition to increased gamification, increased study check-ins and therapist availability may also be beneficial. Even though all included studies relied on remote delivery of CR, all studies incorporated some form of weekly check-ins.

Another factor that contributed towards feasibility and acceptability outcomes was access to technology. While most studies provided technological devices to participants, one study required participants to access their own computers from home (Medalia et al., 2021). As a result, 64% of participants reported not having access to a computer as a challenge to completing CR remotely (Medalia et al., 2021). This finding is in line with the health beliefs model, which states that individuals are less likely to participate in interventions if they perceive barriers towards participation (Jones et al., 2015). Over 45% of individuals with psychotic disorders do not have access to a computer (Miller et al., 2015), which poses a significant barrier to remote treatment. In instances where participants do not have access to their own personal computers, it may be important for institutions to provide computers to access CR, however, in some cases it may not be economically viable for institutions to provide these devices, thus, limiting the feasibility of remote CR.

Family support may be an additional factor that increases retention in remotely delivered CR programs (Harris et al., 2017). Greater emotional support from one's social network is associated with greater treatment engagement in psychosis (Conus et al., 2010), and this may be especially important to consider when asking clients to access a remote treatment. Where possible, it may be effective to integrate family members into the remote treatment delivery, however, further research will be required to evaluate this. Two articles included in our review lent support to caregiver acceptability of remote CR, with both articles stating that caregivers of individuals participating in remote CR supported the acceptability of the remote intervention and reported subjective cognitive improvements in the study participants. However, the majority of our included articles did not include information on these perspectives.

Additionally, it is possible that age may also be a contributing factor towards growing attrition rates. Our results generally demonstrated that studies where the average age of participants was over the age of 35, tended to have higher attrition rates (35.5%), than studies where the average age of participants was under the age of 35 (29.3%). This finding is discordant with existing literature which suggests that older individuals with psychosis may be more engaged in online interventions (Arnold et al., 2019). With previous research demonstrating that motivational impairments do not tend to get more pronounced over the course of illness in those with schizophrenia (Schlosser et al., 2014) and that those with higher negative symptom profiles have greater adherence to CR programs (Dillon et al., 2016), it may be worth examining whether other symptom profiles may be associated with greater attrition rates for remote CR. Furthermore, it is important to investigate if individuals who have chronic treatment-resistant schizophrenia maybe also be more likely to be less likely to attend remote CR interventions.

Finally, there does not seem to be a relationship between the level of attrition and the dose of CR received, despite the variation in training schedules across our included studies. For instance, of the three studies with the least rigorous training requirements (training needed to only be completed twice a week; Harris et al., 2017; Krzystanek et al., 2020; Ventura et al., 2013) two studies had a similar attrition rate to a study with one of the most intensive training schedules included, requiring training to completed five days a week for one hour per day (36.5%–39.6%; Biagianti et al., 2017). However, the third study had a lower attrition rate (Krzystanek et al., 2020; 30.3%), similar to other studies that that required training to be completed five days a week for a minimum of 30 min a day (Fisher et al., 2015; Hargreaves et al., 2015; Loewy et al., 2021; Melo Moura et al., 2019). Attrition rates for these studies ranged from 27% to 30.3%. However, another study which required training to be completed five days a week for a minimum of 30 min per day had a far higher attrition rate of 38.9% (Donohoe et al., 2018). Furthermore, two studies had identical training schedules of requiring participants to train five days a week for an hour per day, but vastly different attrition rates ranging from 27.2% (Fisher et al., 2015) to 36.5% (Biagianti et al., 2017), even with both targeting lower order processes and similar populations.

Limitations

These findings should be interpreted with consideration of several limitations. Firstly, there are very few studies examining remote CR and the methodological rigour of these studies is poor. Furthermore, only two studies discussed in our present review had a follow-up visit. Thus, the longitudinal effects of remote CR are still largely unclear. Additionally, we found only one study comparing partially remote CR to in-person CR, and no study directly comparing in-person CR to fully remote CR, to fully evaluate any differences in treatment effects between modalities. Furthermore, only one study evaluated differences between two different technological modalities (i.e., remote CR administered over desktop computer versus remote CR administered over an electronic tablet), and only one study examined feasibility of remote CR administered specifically over a smartphone. Given the feasibility and acceptability of prior smartphone interventions for individuals with schizophrenia (Ben-Zeev et al., 2014), as well as the increase in mobile ownership for this population (Firth et al., 2016), further examining the feasibility of remote mobile health applications of CR is necessary.

Future directions

Given the relative success of these preliminary studies, it is critical that future research examine large adequately powered trials to examine the efficacy of remotely delivered CR. The inclusion of active control conditions will be an especially important methodological consideration, and the direct comparison of remote CR to in-person CR is required to determine non-inferiority of the remote option. Longitudinal follow-up will also be important to determine whether remote-delivery methods result in the same continued improvements following the end of treatment that have been reported during in-person trials (Best et al., 2019, Best et al., 2019). Further studies should also incorporate more smartphone-based applications of CR, in light of increasing smartphone ownership for those with severe mental health conditions (Firth et al., 2016), to evaluate the feasibility of remote CR across different technological applications and to ensure that remote CR can be more accessible to a wider range of individuals. To this end, future interventions should also aim to incorporate more personalized approaches to remote CR, that support more blended and flexible forms of delivery. CR delivery modality may benefit from greater input from patients on their desired method of completing CR. It may also be possible to integrate remote and in-person modalities to better meet the needs of patients as they change. Furthermore, in light of the preliminary evidence for increased attrition during remote CR, it will also be important for future studies to examine at what point, and for what reasons, individuals discontinue remote treatment.

Additionally, understanding the symptom and demographic profiles of individuals who are at risk for discontinuing treatment may provide opportunities to develop specifically tailored treatments for these populations to reduce attrition. It is also possible that certain personal characteristics or cultural factors may predispose someone from benefitting more greatly from remote CR. For example, individuals from certain collectivistic cultures who may prefer to conduct training with greater family support may prefer to access CR remotely. Additionally, factors such as therapeutic alliance have been demonstrated to predict adherence to computerized CR interventions (Hargreaves et al., 2018). However, none of our included studies have examined this factor in relation to treatment adherence and treatment outcomes. Future research should work on examining the quality of treatment alliance in remote CR interventions in relation to these outcomes. Future research should continue to examine remote delivery of CR, focusing on incorporating gamified cognitive training methods to promote sustained engagement, and increasing measures of support for participants, including the incorporation of family support and regular check-ins from therapists.

Conclusion

Compared to the overall evidence-base for cognitive remediation for schizophrenia-spectrum disorders, the evidence-base for remote delivery of CR is limited. Only nine studies to date have examined fully remote delivery of CR and the methodological rigour of these studies has been low. These initial reports suggest that remote CR may be associated with higher rates of attrition than in-person delivery, however, for individuals who do complete treatment it appears to be an acceptable approach. Given the opportunity to dramatically increase access to care, future research should focus on more methodologically rigorous designs to determine appropriate adaptations for remote treatment and identify who is most likely to benefit from remote CR.

CRediT authorship contribution statement

All authors contributed to and approved the final manuscript. The study was designed by SJ and MWB. Data extraction was conducted by SR, TL, and KAO. All authors contributed to the writing of the final manuscript.

Ethical approval

This study was approved by the University of Toronto Research Ethics Board.

Funding

This study was funded by the (#171738) awarded to M. Best.

Declaration of competing interest

No authors have any conflicts of interest.