Usability of a novel digital medicine system in adults with schizophrenia treated with sensor-embedded tablets of aripiprazole.

OBJECTIVE: Digital medicine system (DMS) is a novel drug-device combination that objectively measures and reports medication ingestion. The DMS consists of medication embedded with an ingestible sensor (digital medicine), a wearable sensor, and software applications. This study evaluated usability of the DMS in adults with schizophrenia rated by both patients and their health care providers (HCPs) during 8-week treatment with prescribed doses of digital aripiprazole.
METHODS: Six US sites enrolled outpatients into this Phase IIa, open-label study (NCT02219009). The study comprised a screening phase, a training phase (three weekly site visits), and a 5-week independent phase. Patients and HCPs independently rated usability of and satisfaction with the DMS.
RESULTS: Sixty-seven patients were enrolled, and 49 (73.1%) patients completed the study. The mean age (SD) of the patients was 46.6 years (9.7 years); the majority of them were male (74.6%), black (76.1%), and rated mildly ill on the Clinical Global Impression - Severity scale (70.1%). By the end of week 8 or early termination, 82.1% (55/67) of patients had replaced the wearable sensor independently or with minimal assistance, based on HCP rating. The patients used the wearable sensor for a mean (SD) of 70.7% (24.7%) and a median of 77.8% of their time in the trial. The patients contacted a call center most frequently at week 1. At the last visit, 78% (47/60) of patients were somewhat satisfied/satisfied/extremely satisfied with the DMS.
CONCLUSION: A high proportion of patients with schizophrenia were able to use the DMS and reported satisfaction with the DMS. These data support the potential utility of the DMS in clinical practice.

Introduction

Suboptimal response to treatment is common for patients with schizophrenia despite pharmacologic treatment interventions that can reduce the severity of symptoms and improve outcomes.1,2 One of the factors contributing to suboptimal response is poor adherence to medication, a frequently observed problem in schizophrenia.3–5 For example, the results from a large, cross-sectional study that examined adherence retrospectively over 4 years showed that 36% of patients with schizophrenia were poorly adherent each year and 61% had adherence problems at some point during the study.6 Poor outcomes associated with nonadherence in schizophrenia include symptom exacerbation, relapse, and rehospitalization7–9 that result in increased treatment costs.10 Assessment of nonadherence is difficult in clinical practice and most commonly relies on subjective patient self-report or impression of a health care provider (HCP).11 These subjective methods often underestimate the extent of patient nonadherence.12,13 The currently available objective methods to assess adherence include pill counts, pharmacy refill records, and electronic devices that capture pill container openings such as Medication Event Monitoring System’s bottle caps.14 All of these objective methods have limitations and do not provide an accurate assessment of actual medication ingestion.14 These modalities also do not provide functions useful for patients to better manage their condition, such as medication reminders and feedback on medication taking.

A novel digital medicine system (DMS) has been developed for patients with serious mental illness to objectively measure and report ingestion of aripiprazole, an atypical antipsychotic.15 Aripiprazole is indicated in adults for the treatment of schizophrenia, for acute treatment of manic and mixed episodes associated with bipolar I disorder (both monotherapy and adjunctive therapy), and for adjunctive treatment of major depressive disorder.16 The DMS is an integrated product with three distinct components (medication embedded with an ingestible sensor, wearable sensor, and software applications) that enable secure collection and patient-controlled sharing of objective medication ingestion information using a patient mobile interface (ie, patient smartphone application) and corresponding web-based interface for HCPs and caregivers. Figure 1 describes functions of the individual components and information flow through the system. In brief, after the patient ingests a digital medicine, the wearable sensor receives a signal from the ingestible sensor that is embedded within the medication and activated in the stomach. The signal is communicated from the wearable sensor to the mobile application, which transmits the data to the Otsuka cloud-based server.

Figure 1

DMS information components and data communication.

Abbreviations: DMS, digital medicine system; MDDS, Medical Device Data System.

The process of developing a drug–device combination such as the DMS requires usability testing of each individual component and of the whole system, in addition to assessment of safety and tolerability.17 The International Organization for Standardization defines usability as:

the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.18

Usability testing requires evaluation of users’ experience with the product, conducted with a representative sample of target users. The safe and effective use of the DMS has been assessed in human factor studies that iteratively mitigated usability-associated, user-related risks and informed design modifications of the product.19 In addition to conducting short-term human factor studies, the expected long-term use of the DMS with aripiprazole in clinical practice requires evaluation of the product’s usability over a longer period of time in patients with serious mental illness. Long-term usability is particularly critical for patients with schizophrenia who often exhibit cognitive impairments20 associated with poor functional skills.21

To ensure optimal performance of the DMS, the patients were asked to pair the wearable sensor with a mobile application on the smartphone, wear the sensor continuously, and replace it after 7 days or as needed. A multi-center, 8-week clinical trial 316-14-220 (NCT02219009) conducted in patients with schizophrenia was designed to assess the system’s usability, provide input for future product development, and inform the use of the DMS in clinical practice. This was the first study that used digital medicine (medication embedded with an ingestible sensor) for the treatment of patients with schizophrenia over an extended period of time. The test of such a drug–device combination represents a significant advancement in the field of psychiatry.

Methods

Objectives

The primary objective of this study was to assess the ability of adult patients with schizophrenia stabilized on a maintenance dose of oral aripiprazole to apply a wearable sensor of the DMS and pair it with the smartphone application independently or with minimal assistance during an 8-week study period. Secondary objectives were to assess the patient’s ability to use the wearable sensor regularly and determine proportion of time the sensor was worn over the 8-week period. Other objectives included assessment of patient- and HCP-reported satisfaction with the system, perceived usability of the DMS by patients and HCPs, support sought from the call center, the proportion of detected versus expected ingestible sensors, and safety and tolerability of the DMS assessed as device- and medication-associated adverse events (AEs) using standard AE reporting.

Usability end points

The primary end point was the proportion of patients able to apply a wearable sensor successfully and independently on their torso and pair the wearable sensor with a smartphone application by the end of 8 weeks. Patient ability to complete these tasks was assessed using the 100-point Subject Ability to Use System Scale – Healthcare Professional Version (SAUSS-HCP). “Successful and independent” was defined by a score of 91–100. The proportion of patients completing the task successfully and independently or with minimal assistance by the end of week 8, based on a score of 71–100 on the SAUSS-HCP, was a main secondary end point. The SAUSS-HCP is a Global Assessment of Functioning-like scale developed for the present study to adequately assess the patient’s ability to successfully use and manage the DMS. The SAUSS-HCP is included in the Supplementary material. The team of investigators assigned to rate patient skills received training on using the SAUSS-HCP scale before the study started and met to discuss scoring results and ensure the scale was used with reliability and consistency during the study. Additional usability end points were patient satisfaction with the system, HCP-perceived usability of the DMS, and the frequency of calls to the call center. Patient satisfaction and HCP-perceived usability of the DMS were measured by the five-question Subject Satisfaction Scale and the 16-question Healthcare Professional Usability Assessment Scale, respectively. The two scales were developed specifically for the present study. All patients who entered the study and used the DMS were included in the usability assessment.

The trial was exploratory in nature, and the sample size was determined based on practical considerations. However, 32 patients in a cohort would produce a two-sided 95% confidence interval (CI) with a width of 0.298 and a lower limit of 0.621 for the proportion of patients who were able to pair and apply a wearable sensor independently and successfully when the sample proportion was 0.8. The total number for the combined cohorts 1 and 2 would be 64 patients.

Safety end points

The safety and tolerability of the DMS were assessed by investigators and reported as the frequency and severity of device-associated treatment-emergent adverse events (TEAEs), separately from medication-associated TEAEs. Serious TEAEs and AEs leading to discontinuation were also reported. AEs were coded by system organ class and Medical Dictionary for Regulatory Activities (Version 17.0). Suicidality was monitored throughout the study using the Columbia-Suicide Severity Rating Scale.22 All patients who entered the study and used the DMS were included in safety assessment.

Study design

This open-label study enrolled adult patients with schizophrenia, currently treated with oral aripiprazole at six US sites, into two sequential cohorts that each included a single treatment group but differed in the iterative versions of the DMS components, specifically, the wearable sensor and the Otsuka medical software. All investigators received training to use the DMS before the study initiation. The trial included a screening period (≤2 weeks), a treatment period (8 weeks), and a safety follow-up period (2 weeks). The treatment period comprised a 3-week training phase and a 5-week independent phase. The patients received structured training at baseline (BL) and additional direct weekly support/remedial training as needed by trial staff at the clinical site during the training phase. During the independent phase, the patients were asked to use the system and replace and pair the wearable sensor with their smartphone application independently or with the assistance of a caregiver once weekly until returning to the trial site at week 8. Throughout the treatment period, the patients were taking a single daily tablet of prescribed doses of aripiprazole (10 mg, 15 mg, 20 mg, or 30 mg) that contained an embedded ingestible sensor (digital medicine). The patients used only aripiprazole digital medicine during the study. Because the participants were stable patients, the aripiprazole dose was unchanged from BL. Each patient was scheduled to visit the study site six times (screening, BL, and weeks 1, 2, 3, and 8). In addition, a call center and trial site staff were available for assistance throughout the study period. The patients could contact the call center directly at any time, and the trial site staff could call on behalf of patients. Disease severity was assessed at BL with the Clinical Global Impression – Severity (CGI-S) scale,23 Personal and Social Performance scale,24 Positive and Negative Syndrome Scale,25 and Instrumental Activities of Daily Living scale.26 The protocol, amendments, and informed consent form were reviewed and approved by the governing institutional review board or independent ethics committee of each investigational site prior to the starting of the trial (Copernicus Group).

Eligibility criteria

Patients aged 18–65 years with a primary diagnosis of schizophrenia based on the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5) criteria, stabilized on oral aripiprazole at once-daily doses, willing to give a written informed consent, and capable of using a DMS smartphone with the DMS application were eligible for enrollment into the trial. Patients with diagnoses other than DSM-5 schizophrenia, acutely psychotic or requiring hospitalization, at risk of suicide, receiving long-acting injectable antipsychotics, abusing substances, with a history or evidence of a medical condition imposing a risk of AEs, or considered by the study staff to be incapable of using technology were excluded.

Data analysis

The estimates for the primary and main secondary end points were expressed as proportions of patients who successfully completed the task based on the specified criteria; the 95% CI of the estimate was calculated using the exact Clopper–Pearson binomial method for each cohort and for the overall patient population. Descriptive statistics for the SAUSS-HCP scores were calculated by cohort and visit. Descriptive statistics for the proportion of sensor-wearing time during the trial was calculated using data from the DMS.

Results

In cohorts 1 and 2, 71 patients were screened, 67 enrolled and treated with the DMS, and 49 completed the 8-week study (73.1%; Table 1). Patient demographic characteristics and BL disease severity were similar in cohorts 1 and 2 (Table 1). A majority of enrolled patients were male (74.6%), black or African American (76.1%), mildly ill (70.1%) based on the CGI-S score of 3, and in a higher range of function as indicated by relatively high mean scores on the Instrumental Activities of Daily Living and Personal and Social Performance scales.

Table 1

Patient disposition and BL characteristics

Disposition	Cohort 1(n=37)	Cohort 2(n=30)	Total(N=67)
Screened, n	38	33	71
Enrolled, n	37	30	67
Treated, n (%)	37 (100)	30 (100)	67 (100)
Completed	27 (73.0)	22 (73.3)	49 (73.1)
Discontinued	10 (27.0)	8 (26.7)	18 (26.9)
Consent withdrawal	5 (13.5)	1 (3.3)	6 (9.0)
AEs	2 (5.4)	4 (13.3)	6 (9.0)
Lost to follow-up	2 (5.4)	2 (6.7)	4 (6.0)
Protocol deviation	1 (2.7)	0	1 (1.5)
Withdrawal by investigator	0	1 (3.3)	1 (1.5)
BL characteristic
Sex, n (%)
Male	27 (73.0)	23 (76.7)	50 (74.6)
Female	10 (27.0)	7 (23.3)	17 (25.4)
Race, n (%)
White	9 (24.3)	3 (10.0)	12 (17.9)
Black	26 (70.3)	25 (83.3)	51 (76.1)
Asian	2 (5.4)	1 (3.3)	3 (4.5)
Other	0	1 (3.3)	1 (1.5)
Ethnicity, n (%)
Hispanic	3 (8.1)	0	3 (4.5)
BMI, kg/m2
Mean (SD)	31.2 (6.8)	32.6 (8.1)	31.9 (7.4)
Range (min, max)	(20.4, 52.1)	(22.8, 58.1)	(20.4, 58.1)
Diagnosis, years
Mean (SD)	19.4 (11.4)	19.1 (9.3)	19.3 (10.4)
Range (min, max)	(2.2, 38.2)	(4.5, 34.8)	(2.2, 38.2)
PANSS total score
Mean (SD)	62.0 (9.6)	57.0 (11.3)	59.7 (10.6)
Range (min, max)	(39, 92)	(36, 88)	(36, 92)
PSP score
Mean (SD)	61.9 (12.6)	63.6 (10.8)	62.7 (11.8)
Range (min, max)	(30, 85)	(40, 80)	(30, 85)
IADL score
Mean (SD)	7.3 (1.0)	7.5 (1.0)	7.4 (1.0)
Range (min, max)	(5, 8)	(5, 8)	(5, 8)
CGI-S scale,a n (%)
Borderline ill	1 (2.7)	2 (6.7)	3 (4.5)
Mildly ill	25 (67.6)	22 (73.3)	47 (70.1)
Moderately ill	10 (27.0)	5 (16.7)	15 (22.4)
Markedly ill	1 (2.7)	1 (3.3)	2 (3.0)

Notes:

Scoring on the CGI-S 7-point scale: 1, normal – not at all ill; 2, borderline mentally ill; 3, mildly ill; 4, moderately ill; 5, markedly ill; 6, severely ill; and 7, among the most extremely ill patients.

Abbreviations: AE, adverse event; BL, baseline; BMI, body-mass index; CGI-S, Clinical Global Impression – Severity; IADL, Instrumental Activities of Daily Living; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance.

Pairing and application of the wearable sensor

Assessment of the ability to apply and pair a wearable sensor with a smartphone application by week 8 including early termination showed that 82.1% (55/67) of patients were able to complete the tasks independently or with minimal assistance by receiving a score of 71–100 on the SAUSS-HCP scale; 55.2% (37/67) of patients were able to complete the tasks independently by receiving a score of 91–100 (Figure 2). The results were similar between cohorts 1 and 2 (not shown). Relative to BL, the increase in the proportion of patients who were able to complete the task independently by week 8 was accompanied by a decrease in the proportion of patients requiring minimal assistance (Figure 2). The proportional shifts in usability were also reflected by the mean (SD) scores on the SAUSS-HCP 100-point scale, which increased from 68.9 (20.1) at BL to 85.5 (17.9) at week 8 or at early termination evaluation. The number of patients who were able to pair and apply the wearable sensor independently or with minimal assistance increased over time from 32/66 (48.5%) at BL to 50/59 (84.7%) at the end of training period (week 3).

Figure 2

Assessment of patient ability to apply and pair a wearable sensor with a smartphone application by week 8.

Notes: Two patients completed the task successfully and independently at BL but did not continue in the study. The performance of these two patients was included into the results at BL but was rated as unsuccessful by week 8. One patient whose performance was not assessed at the BL completed the task with minimal assistance by week 8. Independent: score of 91–100 on the SAUSS-HCP scale; minimal assistance: score of 71–90 on the SAUSS-HCP scale.

Abbreviations: BL, baseline; SAUSS-HCP, Subject Ability to Use System Scale – Healthcare Professional Version.

Wearable sensor use time

The patients wore their sensors for a mean (SD) of 70.7% (24.7%) and a median of 77.8% of their time while in the study. The mean (SD) time of continuous use of the wearable sensor was 3.6 days (2.7 days). However, this number is affected by the study design that required applying and removing the wearable sensor at scheduled visits. When data related to these exchanges were excluded from analysis, the mean (SD) time of continuous use of the wearable sensor was 4.0 days (2.7 days), which is more reflective of real-world use.

Adherence metric over course of the study

The number of detected/expected ingestion events was 1,824/3,072 (59.4%), where the denominator refers to all ingestions from the entire study period regardless of the time spent wearing the wearable sensor. Because ingestion can be detected only when the patient wears the wearable sensor, a modified adherence metric (expressed as a number of detected ingestions divided by the number of days with >80% data coverage or with the ingestible sensor detected within a 24-hour period) may be a more meaningful parameter to assess. The mean (SD) and median of the modified, likely more accurate estimate of adherence over the study period were 73.9% (23.3%) and 78.8%, respectively. The DMS identified six patients who each took a single aripiprazole tablet in error on one occasion in addition to the regular daily dose, out of the overall 1,824 medication ingestions detected during the study (<1% of the total number of ingestions).

Satisfaction with the DMS

Patients

At the last visit, which included evaluation at early termination, 60 patients rated ease of use and helpfulness of the DMS and overall satisfaction with the system using the Subject Satisfaction Scale. Overall, 78% (47/60) of patients were somewhat satisfied, satisfied, or extremely satisfied with the system, which aligns well with the 82.1% of patients who were able to replace the wearable sensor with no or minimal assistance by the end of the study. The majority of patients rated the DMS as somewhat helpful, helpful, or extremely helpful in management of their condition (70%, 42/60) and for improving discussions with their HCP (77%, 46/60). The DMS was somewhat easy, easy, or extremely easy to use for 65% (39/60) of patients.

HCPs

At the last visit, HCPs and site staff at six trial sites together answered the Healthcare Professional Usability Scale for each patient. Medication ingestion data were selected as a helpful function of the DMS in managing the condition of 87% (52/60) of patients. Overall, HCPs were satisfied with the system for 72% (43/60) of patients.

Call center support

The largest proportion of patients seeking assistance was observed at week 1 (60%, 39/65) and decreased over the study period (Figure 3). Those who contacted the call center did so most often once a week, although several patients called multiple times per week (Figure 3). It is important to note that clinical sites also advised the patients directly and the call center data only partially illustrate the extent of the provided assistance.

Figure 3

Proportion of patients who contacted the call center weekly during the study period by number of calls.

Note: Calls at BL were received prior to the first day of treatment with study medication.

Abbreviation: BL, baseline.

Adverse events

Overall, 37 of 67 (55.2%) patients experienced TEAEs during the study, and six patients discontinued due to TEAEs; 22 of 67 (32.8%) patients experienced device-associated TEAEs, while 21 of 67 (31.3%) patients experienced medication-associated TEAEs.

Device-associated TEAEs could have been related to any component of the DMS except for the medication agent, aripiprazole. Table 2 shows device-associated TEAEs experienced by >1 patient; there were five TEAEs (rash, papular rash, rash pruritic, pruritus, skin discoloration) that led to study discontinuation of four patients. Most TEAEs were mild (24/30, 80%), and all were within expectations associated with the use of an adhesive medical device. One patient had a severe rash. With the exception of the one event, all TEAEs were self-limited and resolved by the end of the study.

Table 2

Device-associated TEAEs experienced by >1 patient

Parameter	Cohort 1f(n=37), n (%)	Cohort 2(n=30), n (%)	Total(N=67), n (%)
Patients with TEAEs	13 (35.1)	9 (30.0)	22 (32.8)
Pruritus	5 (13.5)	4 (13.3)	9 (13.4)
Erythema	2 (5.4)	2 (6.7)	4 (6.0)
Rash	3 (8.1)	0 (0)	3 (4.5)
Dermatitis contact	2 (5.4)	1 (3.3)	3 (4.5)
Rash erythematous	2 (5.4)	0 (0)	2 (3.0)
Severe TEAEs (rash)	1 (2.7)	0 (0)	1 (1.5)

Abbreviation: TEAE, treatment-emergent adverse event.

The most common medication-associated TEAEs were upper respiratory tract infection and hypertension, each reported by three patients (4.5%). Two patients experienced a serious TEAE (transient ischemic attack and agitation). In general, all reported TEAEs were consistent with the known safety profile of aripiprazole. Two patients discontinued the study because of a medication-associated TEAE (drooling and serious agitation). None of the TEAEs were considered related to over or under dosing of aripiprazole.

Discussion

Optimal performance of the DMS depends on continuous use of the wearable sensor, which requires a patient’s ability to regularly replace the sensor and pair it each time with a smartphone application. The results of this study show that 82.1% of patients with schizophrenia completed the task independently or with minimal assistance within 8 weeks. Note that although the patients met diagnostic criteria for schizophrenia, they were mildly ill and higher functioning patients previously stabilized on oral aripiprazole and capable of operating a smartphone. The proportion of patients who completed the task without any assistance was 55.2%, which indicates that providing minimal assistance substantially increased usability.

Applying and pairing the wearable sensor requires practice. At the end of the 3-week training period, 84.7% of patients completed the task independently or with minimal assistance. The increase from BL in the number of patients who completed the task independently or with minimal assistance at the end of the training period and overall by the end of the study at week 8 demonstrates that the patients with schizophrenia are able to improve and eventually acquire the needed skills with practice.

As shown in this study, medication adherence metric can be calculated using digital data reporting medication ingestion. Expressing a simple ratio of DMS-detected versus expected ingestions may be sufficient for calculating adherence of patients who continuously use the wearable sensor. In this study, patient training on the use of the DMS led to gaps in the use of the wearable sensor, thus underestimating adherence and requiring calculation of a modified adherence metric that includes only time periods when the patient wears the sensor.

Availability of the call center to assist patients during the study proved useful. The patients used the call center as a resource in addition to that provided by the clinical site. As expected, the frequency of contacts peaked during the first week and steadily decreased over the study period, which indicates patients’ improving proficiency and confidence in using the system over time. Patient support provided by a call center may be beneficial for clinical practice when the DMS becomes available. A currently ongoing study (NCT02722967) assesses the functionality of the call center in optimizing the use of the DMS by patients with serious mental illness.

Most patients expressed satisfaction with the DMS by responding favorably to questions about ease of use and helpfulness of the system in managing their condition. HCPs were also satisfied with using the system for a majority of their patients and recognized utility of the DMS for measuring medication ingestion. Continued efforts to make the system easy to use and to educate about the value of the information provided by the system will enhance adoption of the DMS in clinical practice.

The reported TEAEs were consistent with the known safety profile of aripiprazole and that of the device. The device-associated TEAEs were mostly mild and self-limiting, and all were related to the wearable sensor as skin reactions typical for use of any medical adhesive. AEs related to overdosing were not reported although the DMS identified several patients who took an additional dose of the medication in error.

The result that 82.1% of patients with schizophrenia were able to complete the task of pairing/applying the wearable sensor with minimal or no assistance is promising for the clinical utility of the system. It is also notable that nearly every screened patient agreed to participate in the study, showing that stable patients with schizophrenia are willing to try a new digital medicine product. When available, the results of an ongoing study will further enhance our understanding of how to optimize the utility of the system in clinical practice.

This study has two limitations. First, assessment of the usability end points, except for the wearable sensor use time and proportion of the ingestible sensor detection, was based on subjective ratings. Second, most of the enrolled patients were male and black and were rated as mildly ill on the CGI-S scale and all were capable of using the smartphone; therefore, the current results may not be generalizable to a more typical population of patients with schizophrenia. The DMS has the potential to change current management of patients with stable schizophrenia who are willing to engage with a digital medicine product. The system may not work well for some patients with schizophrenia being treated as usual in clinical practice (ie, more severe or acutely ill patients, those from lower socioeconomic backgrounds, or those with high levels of delusional symptoms or suspiciousness), just as not every antipsychotic medication is suitable for all patients.

Conclusion

In this first study of the usability of an innovative drug–device combination in schizophrenia, the majority of the enrolled patients had either the skills needed or the ability to acquire the skills to successfully use the DMS over an 8-week period. Both patients and HCPs considered the DMS a useful tool for helping to manage their condition.