Participant Engagement and Symptom Improvement: Aripiprazole Tablets with Sensor for the Treatment of Schizophrenia.

Purpose: A recent, phase 3b, mirror-image clinical trial of outpatients with schizophrenia found that use of aripiprazole tablets with sensor (AS; Abilify MyCite®, comprising an ingestible event-marker sensor embedded in aripiprazole tablets, wearable sensor patches, and a smartphone application) reduced the incidence of psychiatric hospitalizations relative to oral standard-of-care antipsychotics. This analysis explored the relationship between AS engagement by participants and changes in participant performance and symptom-severity measures assessed by clinical raters. Participants and
Methods: This post hoc analysis used prospectively collected clinical data from a phase 3b clinical trial (NCT03892889). Outpatients had schizophrenia, were aged 18-65 years, and had ≥ 1 psychiatric hospitalization in the previous 48 months. Participants were grouped by study completion status and a k-means clustering algorithm based on AS utilization, resulting in 3 groups: discontinued (discontinued AS before month 3 of the study); moderate engagement (completed 3 months, used AS intermittently); and high engagement (completed 3 months, used AS regularly). Baseline to end-of-study differences for the Clinical Global Impression Scale (Severity of Illness and Improvement of Illness scales), Personal and Social Performance Scale, and Positive and Negative Syndrome Scale were calculated.
Results: A total of 277 outpatients were enrolled (discontinued, n = 164; moderate engagement, n = 63; high engagement, n = 50). All groups experienced symptom improvement from baseline to end-of-study, with significant changes in the more-engaged groups. Highly engaged participants showed significant improvement for all clinical scores and subscores (all P < 0.05) and demonstrated significantly more improvement in symptoms than participants with less engagement.
Conclusion: Participants who completed 3 months of the study and had higher AS engagement experienced significantly greater improvement in their end-of-study clinical assessments versus participants who did not complete 3 months. Improvement may be related to more-consistent medication intake and better engagement with a digital health system.

Plain Language Summary

People with schizophrenia have an increased risk of suicide attempts and death. They also have high healthcare expenses due to prescription medications and more hospital visits than people without schizophrenia. People with schizophrenia have fewer symptoms and need fewer visits to the hospital when they take their medications regularly, but this is often difficult for them.

Technology can help people with schizophrenia stay on top of their treatment timing—aripiprazole tablets with sensor is a digital medicine system that uses tablets of aripiprazole (an antipsychotic used to treat schizophrenia) embedded with a sensor that sends a signal when the pill has been taken. The signal is picked up by a patch stuck on the skin, which connects through Bluetooth to a smartphone application (app). Both the person with schizophrenia and their healthcare provider can see the app information.

This study looked at participant engagement—how frequently participants wore the patch, took their medication, or opened the app, and how long they used the system—and whether it improved their schizophrenia symptoms. Participants were sorted by a computer program into 3 groups (low, medium, and high engagement with the system).

All 3 engagement groups showed symptom improvement, with higher engagement linked to more improvement. Because many participants were taking aripiprazole before the start of the study, this change could be from using the digital medicine system. However, more studies are needed to understand how digital medicine systems and participant engagement are linked to improved symptoms of schizophrenia.

Introduction

Schizophrenia is a chronic mental disorder with significant health burdens, and is among the top 20 causes of disability worldwide.1,2 People with schizophrenia have a greater risk of premature mortality than the general population, including an increased risk of death by suicide.3,4 Symptoms of schizophrenia include both positive symptoms (eg, hallucinations, delusions, and disorganized speech and behavior) and negative symptoms (eg, social withdrawal and lack of emotion).5 Because schizophrenia has a complex presentation and is difficult to diagnose, current estimated prevalence of schizophrenia in the United States (US) alone ranges from 821,000 to over 3 million people.1,6,7

Schizophrenia leads to a high economic burden.1,7,8 People with schizophrenia in the US incur mean monthly costs that are over 4 times the amount incurred by a demographically matched population, with costs driven by inpatient admissions (42%), outpatient treatment (33%), and prescription drugs (25%).9 Of the over $155 billion in costs associated with patients with schizophrenia in the US in 2013, inpatient costs were responsible for over $15 billion of the total.8 However, a systematic review found that indirect costs are the biggest contributor to overall schizophrenia costs, making it likely that the economic burden of schizophrenia is higher.10 Globally, costs related to schizophrenia are between $94 million–102 billion annually.10

Hospitalization rates for people in the US with schizophrenia increased significantly from 2005 to 2014 (from 453,020 to 722,415 hospitalizations, respectively).7 Prior hospitalization was associated with relapse and increased risk of rehospitalization in people with schizophrenia.11,12 A 2017 study of people with schizophrenia or schizoaffective disorder who had been discharged from a psychiatric hospital found that 15% were readmitted within 3 months and 33% were readmitted within 1 year of discharge.11 Psychiatric hospitalization rates are also inversely associated with medication adherence.13,14 A systematic review and meta-analysis found that nonadherence to psychiatric medication increased the chance of relapse by 400% and was the biggest factor associated with relapse.15,16

Long-acting injectable (LAI) antipsychotics can help improve medication adherence. A systematic review and meta-analysis of 25 studies comparing LAI antipsychotics and oral antipsychotics for people with schizophrenia found that people with schizophrenia on an LAI antipsychotic were 89% more likely to continue taking their treatment than those on an oral antipsychotic.17 However, LAI antipsychotics may be perceived by people with schizophrenia as having a stigma or as being coercive.18,19 People with schizophrenia may also have difficulty adjusting to the dose, have delayed resolution of side effects, have a fear of injection pain or needles, and have pain or irritation at the injection site.18,19

Because current treatment methods may not always be good options for all people with schizophrenia, there is a growing interest in the use of digital health tools to help with managing schizophrenia.20 Three preliminary studies have used passive data collection, or digital phenotyping, through a smartphone or wearable device to determine which behaviors may be correlated with relapse.21–23 All 3 studies found that passive monitoring of behavior could be used to develop technologies to identify potential early warning signs for schizophrenia relapse. A study collecting both passive smartphone data and sleep data with a wearable device, as well as using a brief daily smartphone-based symptom diary, found that people with schizophrenia were willing to use digital technology to predict relapse.24 However, adherence to the diary declined over the 8-week study period, suggesting that passive monitoring tools may be more reliable. Another study attempted to detect early psychosis by using a smartphone application (app) to administer a quick daily questionnaire to participants; the app provided physicians with access to the survey data and alerted participants and physicians to low engagement or abrupt survey score changes.25 The study found that most participants had a ≥ 85% app compliance.25 Participants who used the app had a significantly lower relapse rate than control participants receiving treatment-as-usual (20% vs 58%; P = 0.001).25

Aripiprazole tablets with sensor (AS; Otsuka America Pharmaceutical, Inc.) consist of aripiprazole tablets embedded with an ingestible event-marker sensor, wearable sensor patches, and a smartphone app. AS is indicated for the treatment of adults with schizophrenia or bipolar disorder I, and as adjunct treatment for adults with major depressive disorder.26,27 Patients can track their treatment ingestion data via a smartphone app, and clinicians and caregivers can also access these data using internet-based dashboards.26,27 For people with schizophrenia that have difficulty maintaining regular ingestion with oral antipsychotics, but for whom LAI antipsychotics are not a good option, AS may help improve medication ingestion.27 The objective data on medication ingestion provided by AS may allow patients and providers to have more candid discussions on their treatment, and allow for shared decision-making regarding treatment.28,29

A recent, phase 3b, mirror-image clinical trial of outpatients with schizophrenia compared the difference between psychiatric hospitalization rates of participants receiving oral standard-of-care antipsychotic treatment(s) for a period of 6 months by psychiatric history (lookback period) followed by a switch to AS for a period of 3 months.27 The study found that participants had significantly fewer psychiatric hospitalizations using AS during prospective months 1–3 versus the retrospective months 1–3 (as recorded by psychiatric history) using standard-of-care oral antipsychotics (0% vs 9.7%).27

While there has been some exploration of the relationship between digital technology adherence and outcomes for people with schizophrenia,24,27 little is known about the relationship between participant engagement with digital technologies and outcomes. This post hoc analysis explored the relationship between AS engagement by participants and concurrent changes in clinically collected participant performance and symptom-severity measures assessed by clinical raters.

Materials and Methods

This post hoc study used prospectively collected clinical data from the phase 3b clinical trial (NCT0389288930). The full study methods have been previously described.27 The trial was conducted in accordance with local laws, the International Conference on Harmonization Good Clinical Practice guidelines, and the Declaration of Helsinki. Before study initiation, written informed consent covering retrospective, screening, and prospective trial phases was obtained electronically from participants. The protocol and consent forms were approved by the relevant Institutional Review Board (IRB) or independent ethics committee at each site.

Clinical Trial Design

The clinical trial was designed to assess the difference between psychiatric hospitalization rates of participants receiving oral standard-of-care antipsychotic treatment(s) for a period of 6 months collected by history followed by a switch to AS for a period of 3 months. At the month 3 visit, study investigators decided together with research participants if the participants would continue on AS or switch to a standard-of-care treatment for an additional 3 months. Participants completed the Participant Usability and Satisfaction Scale (PUSS) at month 3 and month 6 (or early termination) visits. The PUSS was a questionnaire that evaluated how participants felt about the ease of use and their satisfaction with using AS. The study was terminated by the sponsor at the interim analysis for meeting efficacy criteria.

Outpatients were aged 18–65 years, were required to have 1 or more psychiatric hospitalizations in the past 48 months, were stable on an oral antipsychotic that had been prescribed for 6 months or longer, and had a clinical diagnosis of schizophrenia (as defined by the Diagnostic and Statistical Manual of Mental Disorders, 5th edition), with a Positive and Negative Syndrome Scale (PANSS) total score ≥ 60 and ≤ 90. Patients were excluded if they were currently treated with a long-acting injectable (LAI), were diagnosed with a mental disorder other than schizophrenia or had a comorbid mental disorder, and if they were unwilling to use a smart phone.

Post Hoc Analysis

This post hoc analysis used data from the first 3 months of AS use. Participants were grouped by study completion status and a k-means clustering algorithm, which was implemented using the scikit-learn package in Python 3.7,31 based on 4 features measuring utilization of AS: (1) fraction of expected time on AS, defined as the actual number of days on AS (last day on AS minus the first day on AS) divided by the expected number of days on AS; (2) patch wear rate, defined as the number of days with patch data divided by the actual number of days on AS; (3) patch-normalized ingestion rate, defined as the number of days with aripiprazole ingestion divided by the number of days with patch data; and (4) app log-in rate, defined as the number of days the participant logged into the app divided by the actual number of days on AS (Figure 1A). This resulted in 3 groups with different levels of AS engagement (Figure 1B). The first group, discontinued (D/C) participants, discontinued AS use prior to month 3 of the study. The second group, moderate-engagement participants, completed the first 3 months of the study (ie, remained enrolled at least through the month 3 visit or recorded ≥ 80% of the expected number of ingestions through the first 3 months) and used AS intermittently, as determined by the k-means clustering algorithm. The third group, high-engagement participants, completed the first 3 months of the study and used AS consistently throughout, as determined by the k-means clustering algorithm.

Figure 1

(A) Definitions used for the AS utilization framework and (B) decision tree used to sort participants into 3 groups. aLast day on AS minus the first day on AS.

Distributions of baseline to end-of-study differences were calculated within and across groups for the Clinical Global Impression (CGI)–Severity of Illness (CGI-S) scale, CGI–Improvement of Illness (CGI-I) scale, Personal and Social Performance (PSP) scale, and PANSS (including subscale scores and Marder factors).32–34 Total and mean PUSS scores were calculated for the moderate- and high-engagement groups.

Statistical Analysis

The significance of these changes for each group and the pairwise differences between groups was characterized using the Wilcoxon signed-rank test and Mann–Whitney U test, respectively. PUSS scores were also compared across engagement groups using the Mann–Whitney U test. Statistical significance was set at P < 0.05. Effect sizes for within-group changes in the clinical scales were calculated using the matched pairs rank-biserial correlation, and effect sizes for pairwise differences were calculated using rank-biserial correlation.

Results

Participant Characteristics

A total of 277 outpatients enrolled in the trial. Briefly, participants had a mean (SD) age of 44.2 (12.4) years; 65.7% were male; 60.3% were Black, 35.4% were White, and 4.3% were other races; and 18.8% of participants identified as Hispanic. As defined by the clustering algorithm, there were 164 D/C participants, 63 moderate-engagement participants, and 50 high-engagement participants. In the D/C group, 130 participants discontinued due to participant withdrawal (n = 43), loss to follow-up (n = 39), noncompliance with the study treatment (n = 20), adverse events (n = 13), physician decision (n = 7), deviation from the protocol (n = 2), withdrawal by parent/guardian (n = 1), or other reasons (n = 5). When the study reached interim analysis criteria and was terminated early, the remaining 34 of the 164 D/C participants were discontinued from the trial by the sponsor.

There were no significant differences in participant demographics between the 3 engagement groups (Table 1). Participants had a mean baseline PANSS score of 71.5, and 88.1% of participants had taken aripiprazole in the past (though were not necessarily taking it at the study start). Baseline PANSS scores were comparable between engagement groups.

Table 1

Baseline Demographics of Participants by Engagement Group

	Participant-Engagement Group
	D/C n = 164	Moderate n = 63	High n = 50
Characteristic
Age, years, mean (SD)	43.2 (12.0)	44.7 (13.7)	46.7 (11.5)
Sex, n (%)
Male	113 (68.9)	36 (57.1)	33 (66.0)
Female	51 (31.1)	27 (42.9)	17 (34.0)
Race, n (%)
White	59 (36.0)	21 (33.3)	18 (36.0)
Black	100 (61.0)	39 (61.9)	28 (56.0)
American Indian/Alaska Native	1 (0.6)	2 (3.2)	1 (2.0)
Asian	3 (1.8)	1 (1.6)	1 (2.0)
Other	1 (0.6)	0 (0)	2 (4.0)
Ethnicity, n (%)
Hispanic or Latino	30 (18.3)	13 (20.6)	9 (18.0)
Not Hispanic or Latino	134 (81.7)	49 (77.8)	40 (80.0)
Unknown	0 (0)	1 (1.6)	1 (2.0)
Clinical scale score, mean (SD)
CGI-S	3.8 (0.7)	3.8 (0.7)	3.8 (0.6)
PSP	62.7 (11.7)	62.8 (14.2)	61.1 (13.7)
PANSS total	71.2 (8.9)	71.9 (8.2)	72.0 (8.3)
Positive	18.4 (3.8)	18.6 (3.9)	18.1 (3.9)
Negative	18.0 (4.1)	17.9 (3.3)	18.7 (4.2)
General psychopathology	34.8 (4.9)	35.4 (5.2)	35.1 (5.8)

Abbreviations: D/C, discontinued; CGI-S, Clinical Global Impression–Severity of Illness Scale; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance Scale; SD, standard deviation.

AS Feature Group Means

Although the patch-normalized ingestion rate for AS was high across groups, means for the other aspects varied; only the high-engagement group had high mean values for all 4 utilization aspects (fraction of expected time on AS, patch wear rate, patch-normalized ingestion rate, and app log-in rate) (Figure 2).

Figure 2

Radar chart showing distribution of the mean and standard deviation for AS utilization aspects by participant engagement group.

Within-Group Changes from Baseline to End-of-Study

Participants in all 3 engagement groups experienced symptom improvement from baseline to end-of-study in most clinical scale measures, with the moderate- and high-engagement groups showing significant score improvement for more measures versus the D/C group (Table 2, Table 3, Figure 3). Participants with high or moderate engagement experienced significant improvements (effect sizes are absolute values) in the CGI-S (high: r = 0.93, P < 0.0001; moderate: r = 0.66, P = 0.00083), CGI-I (high: r = 1, P < 0.0001; moderate: r = 0.80, P < 0.0001), PSP (high: r = 0.59, P = 0.00026; moderate: r = 0.43, P = 0.0045), and Total PANSS (high: r = 0.81, P < 0.0001; moderate: r = 0.54, P = 0.00022) assessments. D/C participants had significant improvements in the CGI-S (r = 0.67, P = 0.00011), CGI-I (r = 0.52, P = 0.00017), and Total PANSS (r = 0.31, P = 0.0043) assessments.

Table 2

Endpoint Clinical Scale Scores by Engagement Group

Clinical Scale	Participant-Engagement Group
	D/C		Moderate		High
	na	Mean (SD)	na	Mean (SD)	na	Mean (SD)
CGI-S	96	3.6 (0.8)	60	3.4 (0.9)	50	3.3 (0.8)
CGI-I	94	3.7 (0.9)	60	3.2 (1.0)	50	3.1 (0.9)
PSP	96	62.7 (14.0)	54	67.1 (13.1)	45	65.6 (12.6)
PANSS total	99	69.4 (11.9)	60	66.8 (12.4)	50	64.0 (10.2)
Positive	99	17.6 (4.6)	60	16.3 (4.4)	50	15.5 (3.9)
Negative	99	17.6 (3.8)	60	17.1 (3.7)	50	17.4 (4.3)
General psychopathology	99	34.2 (7.0)	60	33.4 (6.7)	50	31.1 (5.9)

Notes: aNot all scores were collected for each participant at the end of the study. The D/C group had 164 participants total; the moderate-engagement group had 63 participants total; and the high-engagement group had 50 participants total.

Abbreviations: D/C, discontinued; CGI-I, Clinical Global Impression–Improvement of Illness Scale; CGI-S, Clinical Global Impression–Severity of Illness Scale; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance; SD, standard deviation.

Table 3

Absolute values of effect sizea of clinical change in scores from baseline to last visit within groups

Scale used to assess change from baseline to last visit scores (|r|b)		Participant-engagement group
		D/C(n = 164)	Moderate(n = 63)	High(n = 50)
CGI-S		0.67***	0.66***	0.93****
CGI-I		0.52***	0.80****	1****
PSP		0.13	0.43**	0.59***
PANSS	Total	0.31**	0.54***	0.81****
	Positive	0.39***	0.66****	0.75****
	Negative	0.18	0.39**	0.40**
	General psychopathology	0.21	0.33*	0.46****
PANSS Marder factors	Positive symptoms	0.46***	0.67****	0.64***
	Negative symptoms	0.17	0.28	0.32*
	Disorganized thoughts	0.18	0.38*	0.36*
	Hostility / excitement	0.11	0.28	0.49**
	Anxiety / depression	0.17	0.23	0.81****

Notes:*P < 0.05; **P < 0.01; ***P < 0.001; ****P < 0.0001.aP-values were calculated using the Wilcoxon signed-rank test. P < 0.05 indicates that the difference between baseline and endpoint for a given group is significant.bEffect size (r) was calculated using the matched pairs rank-biserial correlation. Darker shading indicates larger absolute effect size.

Abbreviations: D/C, discontinued; CGI-I, Clinical Global Impression–Improvement of Illness Scale; CGI-S, Clinical Global Impression–Severity of Illness Scale; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance Scale; r, effect size.

Figure 3

Withina and acrossb group comparison of change from baseline to end-of-study by clinical scale. P-values for within-group comparisons were calculated using the Wilcoxon signed-rank test and are indicated by asterisks (*). P-values for between-group comparisons were calculated using the Mann–Whitney U test and are indicated by daggers (†). *P < 0.05 vs baseline; **P < 0.01 vs baseline; ***P < 0.001 vs baseline; ****P < 0.0001 vs baseline. †P < 0.05; ††P < 0.01; †††P < 0.001.

Only high-engagement participants had significant improvement for all PANSS subscales and Marder factor subscales, including those for negative symptoms (all P < 0.05). Moderate-engagement participants had significant improvement for the PANSS negative subscale (r = 0.39, P = 0.0099), but no significant change for the Marder PANSS negative factor subscale. The change in the D/C-group scores for the PANSS negative symptoms subscale and Marder PANSS negative factor subscale were not statistically significant.

Across-Group Change from Baseline to End-of-Study Comparison

Participants with high-engagement levels demonstrated significantly more improvement than the D/C group for the CGI-S (r = 0.23, P = 0.0039), CGI-I (r = 0.34, P = 0.00014), PSP (r = 0.28, P = 0.0034), and Total PANSS (r = 0.33, P = 0.00052) (Table 4; Figure 3). There were few significant differences between the high- and moderate-engagement groups or between the D/C and moderate-engagement groups. High-engagement participants demonstrated significantly more improvement than moderate engagement participants for the total PANSS (r = 0.20, P = 0.04). Moderate-engagement participants demonstrated significantly more improvement than D/C participants for the CGI-I (r = 0.29, P = 0.0007).

Table 4

Effect size of differencea between groups of change from baseline to end-of-study in clinical scale

Scale used to assess change from baseline to last visit score (rb)		Participant-engagement group
		D/C vs moderatec	Moderate vs highc	D/C vs highc
CGI-S		0.07	0.14	0.23**
CGI-I		0.29***	0.04	0.34***
PSP		0.15	0.14	0.28**
PANSS	Total	0.14	0.20*	0.33***
	Positive	0.14	0.09	0.25**
	Negative	0.10	0.08	0.17*
	General psychopathology	0.10	0.23*	0.33***
PANSS Marder factors	Positive symptoms	0.17*	0.02	0.17*
	Negative symptoms	0.06	0.10	0.16
	Disorganized thoughts	0.10	0.002	0.11
	Hostility / excitement	0.16*	0.09	0.25**
	Anxiety / depression	0.03	0.26**	0.31**

Notes: *P < 0.05; **P < 0.01; ***P < 0.001. aP-values were calculated using the Mann-Whitney U test. P < 0.05 indicates that the difference between baseline and endpoint for a given group is significant. bEffect size (r) was calculated using the rank-biserial correlation. Darker shading indicates larger effect size. cD/C, n = 164; moderate engagement, n = 63; high engagement, n = 50.

Abbreviations: D/C, discontinued; CGI-I, Clinical Global Impression–Improvement of Illness Scale; CGI-S, Clinical Global Impression–Severity of Illness Scale; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance Scale; r, effect size; vs, versus.

PUSS Results

Most participants in the moderate- and high-engagement groups were satisfied with AS and found it easy to use (Figure 4; ). There were no significant differences in PUSS responses when compared by age, sex, race/ethnicity, or baseline PANSS score. High-engagement participants had more favorable PUSS responses than did moderate-engagement engagement participants. Except for the question related to the ease of applying the patch, where there was no significant difference between groups, the mean values of the high-engagement group were statistically higher (all P < 0.05) than those of the moderate-engagement group for all questions in the PUSS (Figure 4; ).

Figure 4

Select results of moderate- and high-engagement participant responses from the Participant Usability and Satisfaction Scale. See and for the full survey results and all means comparisons. aP-values were calculated using the Mann–Whitney U test. P < 0.05 indicates that the difference between baseline and endpoint for a given group is significant. Effect size (r) was calculated using the rank-biserial correlation.

Discussion

Participant engagement was characterized by 4 features based on utilization of different aspects of AS: (1) fraction of expected time on AS, (2) patch wear rate, (3) patch-normalized ingestion rate, and (4) app log-in rate. A gradient of improvement in clinical scores was observed: participants who completed 3 months of the study and were more engaged with AS (as measured by the 4 utilization features of AS) experienced significantly greater improvements in their end-of-study clinical assessments compared with participants who did not complete 3 months. There was also a correlation between increased engagement and greater effect sizes for clinical score change from baseline to end-of-study, with highly engaged participants having the greatest effect size for most clinical scores (Table 3).

There were no significant differences among engagement groups in baseline clinical scales, retrospective period hospitalization rates, prior aripiprazole use, or demographics. The relative similarity of baseline demographics between engagement groups is especially interesting considering research finding that older age may be detrimental to the adoption of newer healthcare technology.35 Though the participant mean age across engagement groups in this study was relatively young (approximately 43–47 years of age), these findings are still promising for the ability to engage a wide age range of people with schizophrenia. Additionally, 88.1% of participants had taken aripiprazole in the past, and the baseline PANSS total mean score was 71.5. Therefore, the improvements observed are probably because of the addition of AS rather than aripiprazole use. This greater improvement may be related to more-consistent medication intake and to better engagement with a digital health system; the contributions of research follow-up to outcome improvements were likely small, as touch points were minimized to reflect the real-world scenario of monthly visits with the healthcare provider.

Participant engagement with AS was correlated with improvement of negative symptom scores in all groups, with significant changes from baseline in the moderate- and high-engagement groups. Though participants in the D/C group did not have significant improvement of negative symptom scores, their scores also did not worsen. Assessing the improvement of negative symptom scores is difficult, and there is not a consensus on how to determine clinically meaningful improvement.36 Though the overall changes in score were low, our results suggest that participant engagement with a digital health system could potentially impact negative symptoms. However, given the difficulties in evaluating negative symptoms and that interventions for negative symptoms were used in this study, these results may not be clinically significant and must be interpreted with caution.37

In addition to the clinical outcomes, participants also had positive views on the utility and usability of AS, regardless of baseline demographics, disease severity, or symptom profile. More-frequent use of AS was significantly correlated with favorable responses regarding their ability to use, and satisfaction with, AS. Given that clinical scores were also correlated with increased AS use, these results support increased participant engagement with improved treatment outcomes and participant acceptance of AS.

The data in the phase 3b clinical trial were not collected with the intent of analyzing participant engagement, limiting this post hoc analysis. Because the primary trial was designed around study hospitalization, it did not have a longer follow-up, and this post hoc analysis is limited by the length of the primary trial. The follow-up time of the primary study was only 6 months. Though the 3-month time point was used to group participants into the 3 engagement groups, engagement was assessed and analyzed for a participant’s entire time in the trial (which could be up to 6 months). Additionally, though the primary trial was designed to mimic clinical practice as much as possible (eg, monthly visits with no mandated contact between healthcare providers and participants between visits; see the primary publication for more details on the trial design27), any post hoc analysis performed on data from participants in a clinical setting necessitates further research into the transferability of the results into real-world practice. Another limitation is that aripiprazole ingestion data could not be recorded if participants were not wearing the sensor patch. While participants are highly likely to take their pill if they are wearing the patch, they may also be taking their medication without patch use. In this regard, it cannot be assumed that AS use is the only reason for improved clinical scores.

Another caveat is that the patch-normalized ingestion rate was designed to measure ingestion only on days that the participant was wearing a patch. Some participants in the D/C group may have used the patch only for a few days, but may have taken their medication on most or all of those days. Thus, the mean ingestion rate for the D/C group is relatively high, but their overall lack of engagement with the system can still be observed in the fraction of expected time on AS and patch wear-rate metrics.

Given that this is an unplanned post hoc analysis using multiple scales and comparisons, the problem of multiple testing needs to be addressed. Running multiple tests of significance on the same data set can increase the chances of finding differences by chance, especially when looking at multiple, unplanned outcome measures.38 The authors are aware of this potential problem and, as such, have presented the results of this analysis as needing to be explored further, rather than as a definitive relationship.38 Additionally, there are relationships between the clinical scales used in this study. The PANSS and the PSP are not independent and have been shown to have correlated scores.39–41 Associations have also been found between the PANSS and the CGI-I, the PANSS and the CGI-S, and the PSP and the CGI-S when evaluating clinical outcomes in people with schizophrenia.40,42,43 This analysis found similar differences within and between groups for these scales, supporting that the results found here may be due to more than chance and are worth future consideration.

The results from this post hoc analysis show that level of engagement with AS is likely important for improving outcomes in this participant population. Future research should apply engagement modeling in other participant populations to validate the correlation between varying engagement levels and outcomes for people with schizophrenia.

Conclusion

While adherence to pharmacological treatment is known to improve outcomes for people with schizophrenia, research into how different levels of engagement with digital health technologies relates to clinical outcomes for schizophrenia treatment is limited. In this post hoc analysis, greater participant engagement with AS was correlated with improved clinical scores. Use of AS has the potential to improve clinical symptoms beyond the effects of oral pharmacotherapy use alone. Digital health technologies may help improve health outcomes for people with schizophrenia. More research is needed to understand the relationship between participant engagement with these technologies and improved outcomes.