Improving social functioning in people with schizophrenia-spectrum disorders via mobile experimental interventions: Results from the CLIMB pilot trial.

BACKGROUND: Patients with Schizophrenia Spectrum Disorders (SSD) demonstrate poor social functioning. While group-based approaches show long-term improvements, access to treatments is limited. Digital platforms hold promise to overcome barriers to treatment delivery and improve outcomes.
OBJECTIVE: In a parallel arm, double-blind RCT, we tested CLIMB, a clinician-assisted, adjunct to treatment that includes computerized social cognition training (SCT), ecological momentary assessments (EMAs), group tele-therapy, and moderated messaging. CLIMB was compared to an active control that includes computerized general cognitive training (GCT), unstructured support groups, and unmoderated messaging.
METHODS: The primary outcome was social functioning. Secondary outcomes were negative symptoms and quality of life (QoL). Given the sample size, Propensity Score Models were used to ensure balanced baseline covariates. Mixed-effects models examined change over time.
RESULTS: 24 participants completed the study (12 per arm). No significant between-group differences emerged in engagement. CLIMB participants engaged in a median of 8 sessions (IQR = 2), 2.8 h of SCT (IQR = 7.5), and 2710 EMAs; control participants engaged in a median of 9 sessions (IQR = 3) and 2.2 h of GCT (IQR = 7.9). As a group, participants showed significant improvements in social functioning (p = .046), with no between-group differences. Intent-to-treat analyses indicated greater improvements in QoL (p = .025) for the active control.
CONCLUSIONS: Delivering group-based mobile interventions to individuals with SSD is feasible. EMAs allow clinicians to maintain inter-session engagement, build participant self-awareness, and tailor treatment delivery. In this treatment model, whether SCT or GCT is more effective remains unclear. Further research will evaluate group-based mobile interventions to improve outcomes in SSD.

Introduction

Patients with Schizophrenia-Spectrum Disorders (SSD) have difficulties engaging in meaningful and constructive social interactions (Couture et al., 2006). These difficulties originate from aberrant and uncoordinated activity in the neural systems underlying the perception, interpretation and processing of socially-relevant information (Adolphs, 2003; Crossley et al., 2009; Savla et al., 2013). Remediating cognitive deficits via targeted training approaches, however, is likely to translate into successful engagement in real-world social situations if treatment takes place in a meaningful social context, allowing patients to practice the trained social cognitive abilities in supervised real-world social situations (Bowie et al., 2017; Kukla et al., 2018; Nahum et al., 2021). In fact, evidence proves that the prediction of social performance by cognitive functioning is almost completely explained by the mediation of knowledge related to the social tasks and the user's functioning on these tasks (Brown et al., 2006).

To date, several integrated, behavioral approaches have shown the ability to induce long-term improvements in social, vocational, and occupational functioning (Hogarty et al., 2004; Hogarty and Flesher, 1999; Nahum et al., 2021). However, access to these treatments is often very limited, with only 10% of those with SSD seeking this type of treatment, and even fewer ultimately receiving it (Essau et al., 1999; Grant et al., 2005; Gross et al., 2005; Merikangas et al., 2011; Wittchen et al., 1999). The nature of SSD contributes to the delay in seeking treatment, as in-person social interactions are required to access professional services (Miers et al., 2014). For those seeking treatment, barriers to receiving it include geographic location, availability of trained therapists, long waiting lists, and the requirement to take time off from school for clinic visits (Cavanagh, 2014; Shafran et al., 2009).

Using digital platforms to remotely deliver individual treatments to those with SSD, including treatments that directly target social functioning, has been proven feasible, efficient, and cost-effective (El Alaoui et al., 2017; Gershkovich et al., 2017; Gruber et al., 2001; Hedman et al., 2011; Lawlor and Kirakowski, 2014; Nahum et al., 2021; Nordh et al., 2017). However, a gap exists in our current understanding of the feasibility of delivering group-based interventions via mobile technology for SSD.

Our research study explored how recent advances in mobile and communication technology can be harnessed to improve social functioning and negative symptoms among individuals with SSD. With evidence suggesting that several technology-based solutions can be used to improve the quality of treatment delivery and support the retention and successful generalization of trained skills in real-world settings (Schlosser et al., 2018), we designed and compared two experimental mobile interventions offering individuals with SSD the opportunity to mitigate social isolation and connect with peers through digital technology.

Group-therapy can be feasibly administered to SSD patients using mobile devices (Biagianti et al., 2016a). Additionally, computerized cognitive training targeting the perception, salience, and understanding of socially-relevant information has been recently shown to produce a meaningful impact in the social wellbeing of SSD patients (Nahum et al., 2021). Ecological Momentary Assessments (EMAs), i.e. repeated sampling of subjects' current behaviors and experiences in real time, in subjects' natural environments, can be feasibly administered in SSD (Granholm et al., 2008, Granholm et al., 2013), and have been shown to increase accuracy, minimize retrospective bias and highlight context-specific relationships of symptoms or behaviors (Heron and Smyth, 2010). Nonetheless, only a few research studies have used these experience sampling methods in SSD to capture patients' emotional status and social role performance, and leveraged this information to guide targeted delivery of psychological evidence-based treatments (Heron and Smyth, 2010; So et al., 2013). Clinicians monitoring clinical and social functioning with EMAs during the intervention have, in fact, the opportunity to tailor the content of psychological therapies, thereby promoting the generalization of trained skills in real-world settings, which could, in turn, augment the efficacy of such treatments (Kashdan and Collins, 2010; Stone et al., 2019; Forbes et al., 2012). In short, EMAs have the potential to radically change how providers and patients relate to their healthcare by delineating exactly when and how to intervene.

Finally, navigating the intricacies of today's social world requires the ability to communicate across various technology platforms, including Instant Messaging (IM). Developing competence and ease with IM seems essential if the goal is to enhance social functioning in this population. Extending the reach of peer-support beyond the traditional clinical setting, by means of direct peer-to-peer 1:1 or group IM has been posited to translate into better social connectedness in the everyday lives of patients (Biagianti et al., 2016a; Schlosser et al., 2018; Tillfors et al., 2012). While embedding IM into experimental interventions was recently demonstrated to be an effective means to provide patients with opportunities, encouragement, and reinforcement for using the behaviors and skills learned and receiving the appropriate rewards (Biagianti et al., 2016a; Schlosser et al., 2018), and, simultaneously, to maintain inter-session continuity, with important implications for treatment engagement (Aguilera and Muñoz, 2011; Biagianti et al., 2017; Furber et al., 2014), the detriments on social wellbeing of excessive digital interactions, especially for vulnerable populations, are well known (Lawlor and Kirakowski, 2014). As a matter of fact, unhealthy thought patterns, perceptual distortions, and communication styles triggered by technology-based peer-based communication have been posited to inhibit people with SSD from successfully forming meaningful social connections (Berry et al., 2018).

Leveraging our expertise in the development of digital platforms that provide individualized monitoring and treatment strategies (Biagianti et al., 2016a; Nahum et al., 2017), the effects of two experimental mobile approaches were tested in people with SSD in hopes to reduce negative symptoms and enhance social functioning.

Experimental/materials and methods

Experimental interventions

Participants in the CLIMB and active control conditions used iOS devices and engaged in their assigned intervention for nine weeks—attending structured or unstructured weekly remote group sessions, connecting with peers via moderated or unmoderated IM, and accessing up to 18 h of Social Cognitive Training and EMAs or General Cognitive Training, respectively. See Fig. 1 for the Intervention Design Diagram and details below.

Fig. 1

Intervention design diagram.

CLIMB

CLIMB integrates neuroplasticity-based Social Cognitive Training (SCT) with an experimental intervention that pairs EMAs of emotional and cognitive status with group therapy video-sessions and IM, the feasibility of which was recently demonstrated in a six-week proof-of-concept trial (Biagianti et al., 2016a).

Social cognitive training

The nine SCT exercises used in CLIMB and developed by Posit Science aim to treat social cognition deficits targeting the impaired brain systems underlying social cognition and cognitive control, spanning the domains of target gaze perception, visual emotion perception, prosody, theory of mind, affective memory and attribution bias, the rationale for which have been reported elsewhere (Biagianti et al., 2016b). The computerized exercises are completed individually and harness the principles of brain plasticity, employing speeded, accurate and increasingly more challenging discriminations of socially relevant information (e.g., eye gazes, emotional faces, prosody, social situations), adapting to each individual's performance. Emotionally laden stimuli are also used to train various cognitive control functions, including attentional control, cognitive inhibition, inhibitory control, working memory, and cognitive flexibility. See Multimedia Appendix 1 for a full description of the exercises.

Ecological momentary assessments

User-Centered Design (UCD) principles were used to create EMAs systematically investigating individual thought, feeling and goal-setting patterns. Over the course of three iterative focus groups, participants expressed a need to navigate extreme fluctuations in daily thoughts, affect and goal-oriented behaviors within personal and social situations. The questionnaires were designed based on categories and sub-categories assessed as key components of cognitive, affective and social functioning and include both behavioral indexes measured with a Likert rating scale (0-7) and self-journaling analytics. See Supplementary Table 1 for selected variables; Supplementary Fig. 1, Supplementary Fig. 2 illustrate the design process. During the trial, CLIMB EMAs are presented in three phases, each lasting three weeks, focused on exploring thoughts, feelings and goals. The first phase focuses on self-reflection on 10 key cognitive thought categories in both positive and negative domains via ‘thoughts’ questionnaires. The second phase focuses on self-journaling on key positive and negative emotional outcomes as well as inputted behavioral outcomes via ‘feelings’ questionnaires. The last phase incorporates ‘goal’ questionnaires that build on the first two phases to reinforce self-awareness and encourages goals that take both obstacles and opportunities into account—including positive and negative feelings around an upcoming goal, confidence surrounding goal completion, motivation to complete a goal and journaling of fears or anxieties surrounding obstacles to goal completion. Participants were offered the opportunity to complete EMAs on a daily basis during the nine-week intervention.

Supplementary Fig. 1

Process of EMA delivery throughout the nine-week CLIMB Intervention.

Supplementary Fig. 2

The user-interface examples of the EMAs.

Structured group therapy sessions

CLIMB weekly one-hour group therapy sessions are modeled after Cognitive Behavioral Therapy for psychosis (CBTp). During the first group session, the clinician facilitates introductions, reviews procedures and guidelines for group therapy and sets expectations. In subsequent sessions, participants reflect on their general goals and are introduced to the value of monitoring both their negative and positive day-to-day experiences. They explore utilizing cognitive and meditative techniques across positive and negative social and personal situations. Participants are encouraged to investigate the dichotomies of negative and positive elements within their actions and concepts of self and to discover a mindset or framework that can help them navigate the dualities that they experience throughout their life. Thus, from the very beginning, CLIMB helps participants consider how their cognitive and affective patterns affect their sense of social belonging, meaning and purpose. The clinician uses a data-driven and personalized approach, leveraging data collected from participant EMAs, to send weekly analytic reports to inform participants of any patterns observed across thought, feeling and goal categories—particularly in response to social and personal stressors—to encourage participants to recognize and proactively change patterns. See Multimedia Appendix 2 for an excerpt of the CLIMB manual and an example of weekly analytic reports.

Moderated peer-to-peer IM

Between group sessions, the CLIMB clinician and participants can maintain contact through direct peer-to-peer 1:1 or group IM features embedded in the videoconferencing platform. Embedding IM into experimental interventions has recently been demonstrated to be an effective means to provide patients with opportunities, encouragement, and reinforcement for constructive behaviors and skills (Biagianti et al., 2016a; Schlosser et al., 2018), as well as the maintenance of inter-session continuity and treatment engagement (Aguilera and Muñoz, 2011; Biagianti et al., 2017; Furber et al., 2014). See Multimedia Appendix 2 for an example of a CLIMB group-chat.

Active control

The active control included a non-social General Cognitive Training (GCT) program, nine remote unstructured support group sessions, and unmoderated peer-to-peer IM—providing an experience that is matched to the experimental treatment program in intensity and duration, without systematically targeting social functioning and negative symptoms, while maintaining overall engagement (mobile device use, contact with the research team, monetary payments) and ensuring that participants remained blind to group assignment.

The five GCT exercises developed by Posit Science engage attention, executive functions, and motivation and target auditory, verbal and visual working memory, spatial memory, navigation, and mental manipulation. A full description of the exercises is provided in Multimedia Appendix 1.

During weekly unstructured group meetings, the clinician supervises and facilitates the group discussion but does not provide guidance for the discussion nor follows CBTp guidelines. Finally, while participants have the opportunity to engage in social interactions via IM, the clinician supervises content for safety reasons, but does not encourage, initiate, or moderate discussions.

Study participants

Over the course of the study, 125 individuals expressed interest in CLIMB, of which 88 completed the preliminary screening questionnaire, 49 completed the consent form, and 31 were eligible for enrolment following a formal assessment of inclusion and exclusion criteria. Participants were recruited through the Internet, primarily in the United States and Canada. Three participants joined the study from France, India and the UK, respectively. See Fig. 2 for the Consolidated Standards of Reporting Trials (CONSORT) Diagram.

Fig. 2

Consolidated standards of reporting trials (CONSORT) flow diagram for creating live interactions to mitigate barriers (CLIMB).

Inclusion and exclusion criteria

Inclusion and exclusion criteria were chosen to identify participants with a clinical diagnosis of schizophrenia or schizoaffective disorder, as confirmed by the Structured Clinical Interview for DSM-5 (SCID). Participants were on a stable psychiatric medication regimen for at least four weeks prior to screening, age of 18 and 60 (inclusive), fluent English speakers, and had intact sensorimotor capacity to use the computerized intervention. Participants were excluded if they had a medical history of or a chronic condition with known cognitive consequences (e.g. traumatic brain injury, bipolar disorder), a psychiatric hospitalization within eight weeks of randomization, evidence of active suicidal ideation or behavior within two months of consent on the Columbia-Suicide Severity Rating Scale, if they were enrolled in a concurrent clinical trial that could affect the outcome of this trial or used computer-based cognitive training programs within one month of consent. Participation in standard treatments, such as therapy or use of prescribed medications (e.g., antipsychotics) were not exclusion criteria.

Procedures

A standard prospective, parallel arm, double-blind, randomized, controlled clinical trial was employed in 24 participants. Participants meeting preliminary screening were sent an electronic consent form, completed the informed consent discussion and provided electronic consent. Participants then completed a formal, remote screening assessment of inclusion and exclusion criteria. Eligible participants underwent a structured remote assessment battery to capture baseline social functioning, negative symptoms and quality of life, and secondary outcome measures, followed by block-randomization. The Study Coordinator oriented all participants to the study applications (BrainHQ, Wire Secure Messenger) and associated activities and introduced participants to the clinician prior to the first group session. Participants that did not own a mobile device were loaned one for the duration of the study. Participants that attended their weekly group session were compensated with an Amazon eGift card, while those that were absent were contacted to discuss their absence and maintain engagement. Following the intervention, participants completed a remote post-intervention assessment with a blinded clinical psychologist. Changes to medications and suicidal ideation and adverse effects during the intervention period were assessed. To increase validity and reliability, every effort was made to ensure the same clinical psychologist completed both the baseline and post assessment for a given participant. Finally, participants were asked to complete an exit survey, compensated, and formally exited from the study. Participants loaned an iPad were provided a prepaid return shipping label and materials for shipment.

Measures

Clinical psychologists administered standardized clinical assessments that measured primary and secondary clinical outcomes. The primary outcome of social functioning specific to the psychotic population was investigated using the Social Functioning Scale (SFS) (Birchwood et al., 1990). Quality of life and negative symptoms were assessed as secondary outcomes. Life satisfaction was assessed across general domains including physical health, employment, social support and education with the abbreviated Quality of Life Scale (aQLS) (Bilker et al., 2003). The experience of positive and negative symptoms that are specific to SSD through the Positive and Negative Syndrome Scale (PANSS) (Kay et al., 1987) was used to evaluate the experience of positive and negative symptoms specific to SSD.

Participant engagement (group session attendance rate, hours of cognitive training completed, number of questionnaires completed) were analyzed for the intent-to-treat population and used to assess if such metrics mediated primary and secondary outcomes. SCT and GCT exercises include tens to hundreds of trials across multiple levels, each level lasting approximately 2.5 min. The total levels completed by each participant were computed and multiplied by 2.5 min to obtain total training time for each participant, after which, group means were calculated.

Data processing and statistical analyses

Balancing baseline covariates

For smaller sample sizes like ours, there is a greater likelihood that there will be larger differences at baseline between groups simply due to sampling error. One way to mitigate this is through a Propensity Score Model (PSM), which is typically used in quasi-experimental designs (Hirano and Imbens, 2001). The PSM allows the development of a sample that is balanced on baseline covariates. The approach used here is referred to as a stabilized inverse probability weight, where participants in the treatment group are given a weight of (where p is the proportion of participants in the intervention group, and π is the individual participant's propensity score), and participants in the control group are given a weight of . These weights are then normalized such that the sum of weights is equal to the total sample size. Altogether, this approach has been shown to create a matched sample that is balanced on baseline covariates (Hirano and Imbens, 2001). For the present study, the propensity score was developed via logistic regression. Variables were included in the propensity score model if the univariate relationship between the variable of interest and group status (per logistic regression) had a standardized effect size of ≥0.10 (Austin, 2011), which corresponds to an absolute Odds Ratio (OR) of 1.20. Effect sizes were emphasized in lieu of statistically significant group differences due to the smaller sample size.

Group differences over time

Mixed-effects models were used to examine change over time (linear for continuous variables, and logistic for one binary variable). Mixed-effects models are ideal for the analysis of longitudinal data, due to the fact that it tolerates missing observations–assuming that it is missing at random (MAR). First, simple growth models (change from baseline to post) were conducted to examine change over time (for all participants). Following this, mixed-effects models testing group x time interactions were conducted using only participants with complete data (i.e., listwise deletion, or complete case analysis). Because complete case analyses have been shown to provide biased estimates in the presence of missing data (Enders, 2010), data were then modeled using the intent-to-treat (ITT) framework, where all participants were included in the model, and missing data were estimated via full information maximum likelihood (FIML). This method for handling missing data is considered a gold standard approach, as it gives relatively unbiased estimates (Enders, 2010), and is the standard for the analysis of randomized trials. Finally, because the efficacy of ITT models is predicated on the assumption that data is MAR, a sensitivity analysis was conducted to assess for potential bias from missing data. This was done via a Heckman selection model, which combines two regression equations that assume there is a joint distribution (that is, that data informs the probability of missingness, and vice versa (Enders, 2010). The model includes 1) the regression model of interest and 2) an additional regression that models the probability of missing data. Because longitudinal Heckman models are not readily equipped to deal with propensity score weights, a cross-sectional Heckman model was used. Because of this, a slightly different approach was used. For the standard regression model (part 1), the change score of the variable of interest was estimated for each participant based on predicted values from mixed-effects models and served as the dependent variable. For the second regression (part 2), in which the probability of missing data was modeled, variables were included if there was an absolute standardized difference of ≥0.10 between subjects who dropped out and those who did not. This sensitivity analysis was only conducted for variables in which there was a significant group x time interaction in the ITT model.

Results

Protocol adherence

A total of 31 volunteer participants were eligible for enrolment. Over the course of 12 months, 10 cohorts were assembled—all were initially comprised of a cohort size of three. Three cohorts experienced dropouts or withdrawals prior to the conclusion of the nine-week intervention. Mean age was 36.6 ± 10.8. Mean years of education was 14.0 ± 4.7. 15 were randomized to the experimental condition; 3 of them dropped out, leaving the final sample size at 12. The reasons provided for dropping out included lack of interest in study activities (n = 1), time commitment (n = 1), and non-compliance (n = 1). 16 were randomized to the active control condition; 4 of them dropped out, leaving the final sample size at 12. The reasons provided for dropping out included lack of interest in study activities (n = 3) and unknown reasons (n = 1). See CONSORT diagram shown in Fig. 2.

Engagement

Engagement variables were not normally distributed, thus their results are reported as medians ± interquartile ranges. CLIMB participants attended 8 ± 2 group therapy sessions and completed 2.83 ± 7.5 h of SCT. Active control participants attended 9 ± 3 group sessions and completed 2.2 ± 7.9 h of GCT. No significant between-group differences in weekly group session attendance rates and time spent on cognitive training emerged. During the nine-week intervention, the 12 CLIMB participants filled out a total of 2710 EMAs (~226 EMAs/person, ~25 EMAs/person/week). Engagement with IM was extremely variable in both treatment conditions. Over nine weeks, a total of 11,027 messages were posted in the IM group chat by all cohort users (moderator and participants). The median number of messages posted by each participant per week across all participants was 23.1, with a semi-interquartile range of 17.8. Across groups, the average ratio of clinician to participant messages was 0.48:1, indicating that an average participant contributed roughly twice as much as the clinician. There were no significant between-group differences in IM use.

Balancing baseline covariates

Group differences via univariate logistic regressions can be found in Table 1. Several medication types differed between groups per effect sizes—because of this, and to avoid problems with cell sizes and/or multicollinearity, a composite score of “total non-antipsychotics” and “total antipsychotics” were created (for non-psychotics, it was the simple sum of non-antipsychotic medications, for antipsychotics, it was the sum of 1st and 2nd generation antipsychotics taken). Variables included in the propensity score model included gender, number of antipsychotics, number of non-antipsychotics, and aQLS. Prior to weighting, the absolute standardized differences (converting from odds ratios to Cohen's d) between groups ranged from 0 to 1.15; after weighting, the absolute standardized differences ranged from 0 to 0.22.

Table 1

Descriptive statistics and univariate logistic regressions predicting group status.

Variable	TAU	CLIMB	|z|	p-Value	OR	ORadj
Age (M, SD)	35.13 (9.13)	39.47 (12.06)	1.13	0.258	1.04	1.06
Education (Years; M, SD)	13.31 (4.60)	14.67 (4.94)	0.80	0.426	1.07	1.06
Gender (Male reference)
Male	8 (50.0%)	3 (20.0%)	–	–	–	–
Female	7 (43.8%)	9 (60.0%)	1.46	0.144	3.43	1.28
Non-binary	1 (6.3%)	3 (20.0%)	1.55	0.120	8.00	1.34
Schizoaffective Disorder	5 (31.3%)	9 (60.0%)	1.58	0.113	3.30	1.48
Taking medications	15 (93.8%)	11 (73.3%)	1.43	0.153	0.18	–
Total medications (M, SD)	2.13 (1.45)	3.00 (2.59)	1.16	0.246	1.24	–
Taking antipsychotics	14 (87.5%)	10 (66.7%)	1.34	0.180	0.29	–
Multiple antipsychotics	4 (25.0%)	3 (20.0%)	0.33	0.740	0.75	–
Number of antipsychotics (M, SD)	1.13 (0.62)	0.93 (0.88)	0.71	0.475	0.70	0.80
Number of non-antipsychotics (M, SD)	1.00 (1.15)	2.07 (2.05)	1.68	0.094	1.51	1.20
Taking antidepressants	6 (37.5%)	5 (33.3%)	0.24	0.809	0.83	–
Multiple antidepressants	1 (6.3%)	2 (13.3%)	0.12	0.903	1.07	–
Taking mood stabilizers	1 (6.3%)	5 (33.3%)	1.72	0.085	7.50	–
Taking anti-anxiety (with benzo)	3 (18.8%)	5 (33.3%)	0.92	0.359	2.17	–
Taking benzodiazepine only	2 (12.5%)	3 (20.0%)	0.56	0.573	1.75	–
Total CPZ daily dose (M, SD)	380.94 (582.14)	723.33 (1261.50)	0.95	0.344	1.00	1.00
PANSS – P (M, SD)	12.56 (4.88)	13.53 (4.47)	0.59	0.555	1.05	1.10
PANSS – N (M, SD)	14.63 (4.53)	12.33 (4.25)	1.41	0.157	0.88	1.00
PANSS – G (M, SD)	25.88 (6.47)	27.67 (6.70)	0.76	0.445	1.05	1.04
PANSS – Total (M, SD)	53.06 (13.16)	53.53 (12.56)	0.11	0.916	1.00	1.02
aQLS (M, SD)	3.37 (1.09)	3.95 (1.43)	1.26	0.206	1.46	1.12
SFS (M, SD)	17.42 (2.39)	17.99 (2.94)	0.61	0.545	1.09	0.97
PANSS – PNGA (M, SD)	57.31 (14.35)	57.33 (12.65)	0.00	0.996	1.00	1.02
C-SSRS (presence/absence)	4 (25.0%)	7 (46.7%)	0.21	0.833	0.94	0.84

Legend. TAU = Treatment as Usual, OR = Odds Ratio, ORadj = Odds Ratio adjusted by propensity score weighting, CPZ = Chlorpromazine, PANSS = Positive and Negative Syndrome Scale, PANSS – P = PANSS, Positive Scale; PANSS – N = PANSS, Negative Scale; PANSS – G = PANSS, General Psychopathology Scale; aQLS = abbreviated Quality of Life Scale, SFS = Social Functioning Scale, C-SSRS = Columbia-Suicide Severity Rating Scale.

Overall trajectories of change

Simple change over time on each of the outcome measures for all participants can be found in Table 2. Only one outcome variable was statistically significant at p < .05 (SFS); however, several other variables had similar effect sizes that were in the “moderate” range.

Table 2

Change over time for all participants.

Variable	b(SE)	|z|	p-Value	95% CI	resta
PANSS-P	−0.96 (0.73)	1.31	0.189	−2.40, 0.47	0.24
PANSS-N	−0.98 (0.89)	1.10	0.272	−2.72, 0.77	0.20
PANSS-G	−2.42 (1.38)	1.76	0.078	−5.12, 0.27	0.32
PANSS-PNG	−4.40 (2.41)	1.82	0.068	−9.13, 0.33	0.33
aQLS	0.33 (0.19)	1.80	0.072	−0.03, 0.70	0.32
SFS	0.92 (0.46)	2.00	0.046	0.02, 1.82	0.36
PANSS-PNGA	−4.86 (2.56)	1.90	0.058	−9.89, 0.16	0.34
C-SSRS (binary)	−0.27 (0.41)	0.68	0.496	−1.08, 0.52	0.12

Legend. PANSS = Positive and Negative Syndrome Scale, PANSS – P = PANSS, Positive Scale; PANSS – N = PANSS, Negative Scale; PANSS – G = PANSS, General Psychopathology Scale, aQLS = abbreviated Quality of Life Scale, SFS = Social Functioning Scale, C-SSRS = Columbia-Suicide Severity Rating Scale.

Note. Models are linear mixed effects models, with the exception of C-SSRS, which is a logistic mixed effects model. aSimple estimate of effect size r, defined as .

Complete case analysis

Results can be found in Table 3. There was only one group x time interaction that was statistically significant at p < .05; aQLS, with an effect size in the “moderate” range. Active control participants exhibited a sharper increase than those in CLIMB (see Supplementary Fig. 3).

Table 3

Completers only.

Variable	b(SE)	|z|	p-Value	95% CI	resta
PANSS-P
Time	−0.54 (0.97)	0.55	0.580	−2.45, 1.37	0.10
Group	5.13 (2.43)	2.11	0.035	0.37, 9.89	0.38
Group x Time	−1.95 (1.29)	1.51	0.130	−4.47, 0.57	0.27
PANSS-N
Time	−0.34 (1.35)	0.25	0.802	−2.98, 2.31	0.04
Group	0.70 (2.36)	0.30	0.766	−3.93, 5.34	0.05
Group x Time	−0.47 (1.55)	0.30	0.761	−3.51, 2.57	0.05
PANSS-G
Time	−1.60 (1.57)	1.02	0.307	−0.468, 1.47	0.18
Group	3.07 (3.64)	0.84	0.400	−4.07, 10.21	0.15
Group x Time	−0.40 (2.59)	0.15	0.877	−5.48, 4.68	0.03
PANSS-PNG
Time	−2.23 (3.29)	0.68	0.499	−8.69, 4.23	0.12
Group	9.51 (6.04)	1.57	0.115	−2.33, 21.36	0.28
Group x Time	−3.18 (4.01)	0.79	0.427	−11.04, 4.67	0.14
aQLS
Time	0.59 (0.24)	2.43	0.015	0.11, 1.07	0.44
Group	0.85 (0.62)	1.36	0.172	−0.37, 2.06	0.24
Group x Time	−0.67 (0.34)	1.98	0.048	−1.34, −0.01	0.36
SFS
Time	0.92 (0.68)	1.35	0.176	−0.41, 2.25	0.24
Group	−0.06 (1.86)	0.03	0.973	−3.70, 3.57	0.01
Group x Time	0.02 (1.24)	0.02	0.985	−2.42, 2.46	0.00
PANSS-PNGA
Time	−2.36 (3.47)	0.68	0.495	−9.6, 4.43	0.12
Group	9.35 (6.26)	1.49	0.135	−2.91, 21.62	0.27
Group x Time	−2.93 (4.20)	0.70	0.486	−11.17, 5.30	0.13
C-SSRS (binary)
Time	−0.21 (0.82)	0.26	0.795	−1.81, 1.39	0.05
Group	3.26 (2.56)	1.27	0.203	−1.76, 8.28	0.23
Group x Time	−0.82 (1.25)	0.65	0.513	−3.27, 1.63	0.12

Legend. PANSS = Positive and Negative Syndrome Scale, PANSS – P = PANSS, Positive Scale; PANSS – N = PANSS, Negative Scale; PANSS – G = PANSS, General Psychopathology Scale, aQLS = abbreviated Quality of Life Scale, SFS = Social Functioning Scale, C-SSRS = Columbia-Suicide Severity Rating Scale.

Note. All models adjusted via propensity score weighting. Models are linear mixed effects models, with the exception of C-SSRS, which is a logistic mixed effects model. aSimple estimate of effect size r, defined as.

Supplementary Fig. 3

Intent-to-treat analysis of abbreviate Quality of Life Scale scores over time.

Intent to treat analysis

Results are shown in Table 4. Results were the same as those in the complete case analysis, with a “moderate to large” effect for aQLS scores.

Table 4

Intent to treat analysis.

Variable	b(SE)	|z|	p-Value	95% CI	resta
PANSS-P
Time	−0.97 (0.94)	1.04	0.298	−2.81, 0.86	0.19
Group	2.44 (2.52)	0.97	0.334	−2.51, 7.38	0.17
Group x Time	−1.35 (1.26)	1.07	0.285	−3.83, 1.13	0.19
PANSS-N
Time	−0.82 (1.33)	0.61	0.540	−3.43, 1.80	0.11
Group	−2.00 (2.44)	0.82	0.411	−6.78, 2.78	0.15
Group x Time	0.33 (1.55)	0.21	0.830	−2.71, 3.37	0.04
PANSS-G
Time	−2.41 (1.58)	1.52	0.128	−5.51, 0.69	0.27
Group	−0.34 (3.82)	0.09	0.929	−7.83, 7.15	0.02
Group x Time	0.86 (2.57)	0.33	0.739	−4.17, 5.88	0.06
PANSS-PNG
Time	−4.43 (1.37)	1.37	0.171	−10.77, 1.91	0.25
Group	−0.88 (6.81)	0.13	0.897	−14.23, 12.47	0.02
Group x Time	0.56 (4.02)	0.14	0.890	−7.31, 8.43	0.03
aQLS
Time	0.65 (0.24)	2.73	0.006	0.18, 1.12	0.49
Group	1.33 (0.60)	2.21	0.027	0.15, 2.52	0.40
Group x Time	−0.75 (0.33)	2.25	0.025	−1.40, −0.10	0.40
SFS
Time	0.98 (0.66)	1.48	0.139	−0.32, 2.28	0.27
Group	0.50 (1.71)	0.29	0.768	−2.86, 3.87	0.05
Group x Time	−0.04 (1.23)	0.03	0.975	−2.45, 2.37	0.01
PANSS-PNGA
Time	−4.93 (3.45)	1.43	0.152	−11.69, 1.82	0.26
Group	−2.28 (7.26)	0.31	0.754	−16.50, 11.94	0.06
Group x Time	1.43 (4.27)	0.33	0.738	−6.94, 9.80	0.06
C-SSRS (binary)
Time	−0.62 (1.00)	0.62	0.537	−2.57, 1.34	0.11
Group	1.83 (1.95)	0.94	0.346	−1.98, 5.65	0.17
Group x Time	−0.19 (1.79)	0.16	0.872	−2.46, 2.08	0.03

Legend. PANSS = Positive and Negative Syndrome Scale, PANSS – P = PANSS, Positive Scale; PANSS – N = PANSS, Negative Scale; PANSS – G = PANSS, General Psychopathology Scale, aQLS = abbreviated Quality of Life Scale, SFS = Social Functioning Scale, C-SSRS = Columbia-Suicide Severity Rating Scale.

Note. All models adjusted via propensity score weighting. Models are linear mixed effects models, with the exception of C-SSRS, which is a logistic mixed effects model. aSimple estimate of effect size r, defined as.

Sensitivity analysis

Group differences between dropouts and non-dropouts can be found in Table 5. Treatment status, gender, schizoaffective disorder, total number of antipsychotics, total number of non-antipsychotics, baseline aQLS, and baseline SFS met the univariate threshold for inclusion into the Heckman model. The model estimated that selection bias did not influence the differential effect of treatment status on aQLS change, as it remained statistically significant (b = -0.16, |z| = 2.98, p = .003, 95% CI = -0.26, -0.05).

Table 5

Univariate differences between dropouts and non-dropouts.

Variable	|z|	p-Value	OR
Group	0.33	0.740	0.75
Age	0.06	0.954	1.00
Education	0.11	0.910	1.01
Gender (Male reference)
Male	–	–	–
Female	1.46	0.145	4.35
Non-binary	0.15	0.884	0.78
Schizoaffective Disorder	0.98	0.325	0.40
Taking medications	0.99	0.322	0.36
Total medications	1.18	0.239	0.74
Taking antipsychotics	1.41	0.160	0.27
Multiple antipsychotics	1.22	0.222	0.16
Number of antipsychotics	1.79	0.073	0.26
Number of non-antipsychotics	0.66	0.510	0.83
Taking antidepressants	0.43	0.665	0.67
Multiple antidepressants	0.10	0.917	0.93
Taking mood stabilizers	0.38	0.701	0.63
Taking anti-anxiety (with benzo)	0.77	0.439	0.40
Taking benzodiazepine only	0.20	0.845	0.82
Total CPZ daily dose	0.82	0.41	1.00
PANSS – P	1.17	0.240	1.12
PANSS – N	1.09	0.277	1.12
PANSS – G	0.45	0.651	1.03
PANSS – Total	1.05	0.294	1.04
aQLS	1.74	0.081	0.48
SFS	1.00	0.317	0.84
PANSS – PNGA	1.15	0.252	1.04
C-SSRS (presence/absence)	0.18	0.857	1.07

Legend. OR = Odds Ratio, PANSS = Positive and Negative Syndrome Scale, PANSS – P = PANSS, Positive Scale; PANSS – N = PANSS, Negative Scale; PANSS – G = PANSS, General Psychopathology Scale, aQLS = abbreviated Quality of Life Scale, SFS = Social Functioning Scale, C-SSRS = Columbia-Suicide Severity Rating Scale.

Discussion

This study compared the feasibility and efficacy of two group-based mobile interventions designed to enhance social functioning, quality of life and symptoms in patients with SSD.

In both experimental approaches, group session attendance and EMAs completion rates aligned with those found in the proof-of-concept trial and by other mobile health studies conducted in SSD (Barbeito et al., 2019; Biagianti et al., 2016a; Fulford et al., 2020; Granholm et al., 2013; Schlosser et al., 2018); high enrolment, retention, and remote assessment completion rates were comparable to those reported by studies testing mobile apps in patients with psychotic disorders (Barbeito et al., 2019; Bucci et al., 2018; Fulford et al., 2020; Kumar et al., 2018; Meyer et al., 2018; Niendam et al., 2018; Schlosser et al., 2018)—which corroborates the feasibility of remotely delivering assessments and treatments to people with SSD. Engagement with SCT or GCT exercises, however, was lower than anticipated across both groups, likely attributed to being encouraged, but not required, to complete 2 h of weekly training.

The high degree of interindividual variability in engagement patterns with intervention features continues to represent a challenge in the field—while the majority of enrolled participants met intervention requirements, a subgroup completed less than 1 h of training a week, did not engage in IM, and attended the weekly group sessions quite passively, despite strenuous efforts to monitor, support and motivate study participation. In line with Birnbaum et al., IM, allowing for continuity of group communication in between sessions elicited divergent responses in research participants across groups: while some barely contributed, thus losing the opportunity to befriend peers with relatable lived experiences (Allott et al., 2011), others quickly developed addictive tendencies towards the medium and showed unhealthy communication styles (oversharing, poor mentalization, primitive defense mechanisms, boundary crossing) (Birnbaum et al., 2017).

As a group, participants showed significant improvements in the primary outcome, social functioning, over the course of the interventions (p = .046), with no between-group differences. Using an intent-to-treat analysis, we found a significant group-by-time effect (p = .025) in the secondary outcome assessing quality of life, with participants in the active control intervention showing slightly greater improvements on this outcome.

At first glance, this could suggest that, in our effort to develop a balanced design with a control that was well-matched with the CLIMB intervention, the control intervention had an underlying therapeutic effect that benefited participants. The weekly group sessions attended by participants in the control intervention, albeit unstructured in nature, likely contributed to an increase in socialization that could have had a generalized effect on quality of life. Additionally, the non-social cognitive training assigned to the control group may have induced non-social cognitive improvements, which are known to enhance social functioning in SSD (Gard et al., 2009), although recently published trials of SCT in SSD suggest the unique benefits of targeting socially relevant stimuli to enhance social functioning (Fisher et al., 2017; Nahum et al., 2021).

At the same time, we cannot rule out three other reasons for these underwhelming effects of social functioning and quality of life. First, in absence of a no-treatment control group, we cannot rule out non-specific effects of study participation and medication effects on the observed improvements. Second, while use of EMAs indicates a promising ability to deliver more focused and tailored care based on the thoughts and emotions of users as well as promote more self-reflection and affectively regulate behavior, we realize that, for participants randomized to CLIMB, concurrently engaging in cognitive training, EMAs, manual-based group sessions and IM for nine weeks was perhaps too intensive and could have had a negative impact on our outcomes of interest. More relevantly, it is our clinical impression that the signature lack of socialization in SSD originates from intrapsychic and interpersonal impairments that our mobile communication technology features could unpredictably attenuate or exacerbate. While some patients, arguably those with better perception and understanding of socially-relevant information, can benefit from online social interactions and mitigate their social isolation, others may be triggered by the undetermined, loose and sometimes ambiguous boundaries of a group-based digital environment, experiencing unhealthy thought patterns and perceptual distortions that could easily manifest as poor, if not negative, response to the intervention. This would confirm existing literature suggesting mixed effects on social wellbeing for SSD patients engaging with online experiences (Bjornestad et al., 2019).

A rich agenda of work spawns from these preliminary findings. First, it is critical to increase sample size to exclude that lack of power jeopardizes the possibility to detect between-group differences. A larger sample size would also reduce the risk of random between-group differences at baseline, which in this study contributed to variability and introduced bias that influenced our results. Second, identifying predictors of engagement and response to digital interventions in this vulnerable population is essential, given the striking differences in cognition, symptoms, and functioning that characterize SSD. Studies are currently underway to explore such predictors (Arnold et al., 2019; Biagianti et al., 2021). Third, mobile interventions for SSD, especially those that aim to be delivered in a group-based format, must take into account the stress that this patient population may experience in navigating virtual social interactions, and structure intervention features accordingly.

Conclusions

During the implementation of a NIMH-funded randomized clinical trial, we developed an understanding of the complex challenges that individuals with SSD experience as they attempt to overcome social isolation and connect with peers. We demonstrated that leveraging digital platforms for group-based mobile interventions that are remotely delivered to people with SSD is feasible. CLIMB was not associated with greater adherence rates and engagement patterns compared to the active control condition. Across groups, participants successfully engaged with weekly group video-sessions, attending 80% of sessions. EMAs in CLIMB prove to be an innovative feature that provide a unique opportunity to engage participants outside of their group-therapy sessions by sharing weekly, personalized reports with participants that both build self-awareness and help the clinician tailor the content of group-therapy sessions. Results on the usefulness of integrating neuroplasticity-based cognitive training and IM with this remote, digitally-enhanced, group-based approach are inconclusive. It remains unclear whether using computerized training to target social cognitive processes vs general cognitive processes yields a better outcome. Additionally, although the presence of a clinician is warmly recommended (Biagianti et al., 2018), our preliminary data indicate attending unstructured support groups may be as effective for enhancing social functioning as manualized group therapy. Finally, instant messaging, inasmuch it showed great potential in offering people with SSD the opportunity to connect with peers and mitigate social isolation outside the weekly group session, may have an unpredictable and controversial impact in this patient population, and should be rigorously studied and moderated. Further research is required with larger sample sizes and more balanced baseline characteristics to evaluate the potential of mobile interventions for social functioning and better understand the implications for improving quality of life in SSD.

The following are the supplementary data related to this article.Multimedia Appendix 1

Computerized training program manuals.

Multimedia Appendix 2

CLIMB intervention manual, weekly analytic reports and group chat.

Supplementary Table 1

CLIMB ecological momentary assessments: categories captured in thoughts, feelings, and goals daily questionnaires.

Ethical approval and funding source

The CLIMB protocol was reviewed and approved by the Western Institutional Review board (IRB Protocol 20172401) and this research was supported by the (NIMH) Grant 1R43MH114765-01A1. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIMH. Trial Registration: ClinicalTrials.gov NCT03317769; http://clinicaltrials.gov/ct2/show/NCT03317769

CRediT authorship contribution statement

BB and SQ contributed to the study conception and design. SD, JJ, SQ and BB contributed to the drafting of the manuscript and figs. SD, SQ, JJ and BB contributed to the acquisition and analysis of the data.

Declaration of competing interest

Posit Science was the sponsor of this trial, and is the developer of the BrainHQ cognitive training programs used in this study. Posit Science holds patents for and a proprietary interest in this software. SD and BB are employees of Posit Science, and SD holds equity in Posit Science. Other authors have no conflicts to report.