Mobile phone-based e-diary for assessment and enhancement of medications adherence among patients with multiple sclerosis.

BACKGROUND: Adherence to multiple sclerosis (MS) disease-modifying drugs (DMDs) is essential for realization of their optimal effectiveness and benefits.
OBJECTIVE: To evaluate the usefulness and validity of a smartphone-based e-diary as a tool for adherence assessment as well as its effectiveness as a promoter of adherence to DMDs.
METHODS: An MS tailored e-diary (MyMS&Me) reminded patients to take their DMDs on time. DMD intake was self-recorded in the e-diary by the participants. Three methods of adherence evaluation were compared: e-diary derived, retrospective self-reported, and the medication possession rate (MPR). The proportion of patients with poor adherence to DMDs (defined as MPR <80%) among e-diary users was compared with a control group without intervention.
RESULTS: Sixty-two patients downloaded the e-diary (Female: 41 (66%), Expanded Disability Status Scale 3.2 ± 2.2) and 55 controls were enrolled. The median difference between e-diary-derived adherence and the MPR was -3% (95% limits of agreement: -53% to 12%). The median difference between retrospective self-reported adherence and the MPR was 0.3% (95% limits of agreement: -20% to 42%). The proportion of participants with poor adherence to DMDs was similar in the e-diary and control groups (10% vs. 13%, p = 0.6).
CONCLUSIONS: Substantial and clinically important disagreement between methods of medication adherence evaluation was noted. Smartphone reminders did not significantly improve the MPR of DMDs.

Introduction

Disease-modifying drugs (DMDs) for treatment of multiple sclerosis (MS) have the proven efficacy to prevent disease exacerbations and to slow disability progression.[1] Adherence to DMDs is defined as the extent to which a patient acts in accordance with the prescribed interval and dose of a dosing regimen.[2] Treatment interruptions due to non-adherence have been associated with reappearance or worsening of MS activity, increased risk of MS-related hospitalization and higher MS-related medical costs.[3-5] Prior studies have suggested that up to 51% of people with MS (PwMS) missed at least one dose of their DMDs in a month,[6] while about 15–20% of PwMS were failing to take more than 20% of their prescribed doses.[7,8] Though thought to be improved by the introduction of oral medications, adherence was found to be similar in both injectable and oral medications.[9] It follows that both detection of non-adherence to DMDs and interventions to increase adherence are important elements in the clinical care of PwMS.

The advent of mobile electronic platforms, as part of the implementation of patient-centric approaches and participatory medicine,[10,11] opens up the possibility for interactive diaries that remind patients to take their medications and prospectively collect information about adherence to treatment. In a review of currently available electronic tools related to medical use, the most prevalent functionalities were promotion of adherence to medications and monitoring of patient-reported outcomes (PROs).[12] Despite the abundance of healthcare-related applications, evidence for their validity and efficacy are scarce.[13] In a systematic review of 681 smartphone applications designed to enhance medication adherence, only 8 (1.2%) had an evidence base in relation to the development process and only 3 were clinically tested for efficacy.[14]

Multiple factors are responsible for patients not taking their DMDs as prescribed. Those factors may be related to the patient, treatment, and healthcare provider.[15] Failure to remember to take the medication (unintentional non-adherence) is the most common explanation for non-adherence that patients provide.[16,17] Therefore, it is reasonable to premise that a smartphone application that reminds patients to take their medications may help to improve adherence. However, in a review of 13 studies among adults with diabetes or hypertension, there was no conclusive demonstration of improved medication adherence using digital health interventions such as text messages, interactive voice responses, or telemonitoring of medication intake.[18] Similarly, four randomized clinical trials evaluating digital text reminders showed no statistically significant effect on treatment adherence, when compared with standard care of patients with tuberculosis.[19] Nevertheless, memory impairment is common among PwMS as a result of their neurological disease,[20] therefore we hypothesized that digital reminders may prove beneficial for this specific population, even if of limited effectiveness for patients with other chronic diseases.

A common method of adherence evaluation in clinical practice is by directly asking patients to estimate how many doses of medications were missed. However, patient-reported adherence may be inaccurate, as patients generally tend to overestimate their degree of adherence, or their report is influenced by the most recent period.[8] Pharmacy data regarding medication possession, may be more accurate, but are not readily available for all patients during medical encounters. Furthermore, pharmacy data indicate how much of the medication was supplied, but not whether it was taken properly. In a review of studies measuring medication adherence using both monitoring devices and self-reported questionnaires, a moderate to high correlation was found between these two measures, however 60% of articles appeared to report better adherence with the self-reported questionnaires than with the electronic monitoring devices.[21] A smartphone-based electronic diary (e-diary) offers the opportunity to keep a prospective documentation of DMD intake, although patient cooperation is necessary, which may be less than perfect, especially in the long run. Furthermore, both e-diary and retrospective self-report may be influenced by the desire to please healthcare providers.

In this study we aimed to evaluate the validity of a smartphone-based e-diary as a tool for adherence assessment, as well as appraising its effectiveness as a promoter of adherence to DMDs among PwMS.

Methods

Participants

One-hundred seventeen PwMS according to the revised McDonald criteria were recruited for this study. Inclusion criteria were age 18 to 70 years, able to browse the internet and use a smartphone, willing and able to give informed consent, Expanded Disability Status Scale (EDSS) ≤7, treated with self-administered DMDs, and a member of the Health Maintenance Organization Clalit Health Services (in order to obtain pharmacy data regarding medication supply). Patients with neurological conditions involving the central nervous system other than MS were excluded. Sixty-two PwMS downloaded the e-diary application and 55 patients were followed clinically without e-diary intervention. Patients were recruited sequentially during their routine clinic visits. Allocation to study groups was randomly determined by day of attendance. In each given clinic day, all recruited patients were enrolled to one of the two distinct study groups. Of the patients approached, only five declined to participate. The study protocol was approved by the institutional review board (IRB), and all participants gave informed consent (IRB approval number: CMC-0065-13).

The e-diary application

An e-diary, an internet smartphone application tailored for persons with MS, was developed (MyMS&Me). The application sent reminders to take DMDs and asked users to mark their actual intake in the e-diary. To record medication intake, patients had to click on the reminder notification, and then to confirm actual intake with another click. The e-diary also periodically collected PROs, but this component is not part of the current report. Since the application was considered experimental, adherence data that were collected by the e-diary were not consistently reviewed with the patients during medical encounters, and did not influence clinical decisions.

Study measures and procedures

Patients were clinically evaluated at baseline, after 6 months, and at the end of a 12-month follow-up period. On each visit, relapses were recorded, a detailed neurological examination was performed and an EDSS score was obtained.

At the 6-month and 12-month visits, patients were asked to estimate how many doses of medication they had taken in the month prior to the medical encounter. Self-reported adherence was calculated by dividing patients’ reported intake by the instructed dose, as assessed by the previously validated Multiple Sclerosis Treatment Adherence Questionnaire (MS-TAQ).[22] The average of the 6-month and 12-month self-reported adherence was used for comparison with other methods of adherence evaluation. E-diary-derived adherence was calculated by dividing the total number of diary entries indicating medication intake by the instructed number of doses between the first and last dates of adherence, reported by means of the e-diary. The medication possession rate (MPR) was calculated as the ratio of the total days’ supply of medication dispensed during the period evaluated by the e-diary, to the expected number of days covered.[2,23] Using a commonly accepted threshold[23] patients with MPRs ≥80% were considered adherent to their index DMDs, whereas patients with MPRs <80% were considered non-adherent.

Statistical analysis

Data analysis was performed using SPSS, version 23.0 (IBM Corp., Armonk, NY, USA). Data distribution was inspected for normality. Continuous variables were analyzed using the between-groups t-test. Categorical variables were analyzed with chi-square or Fisher’s exact test. Agreement between the MPR and e-diary-derived adherence, as well as between the MPR and self-report adherence were evaluated by Bland–Altman plots. In these graphs the MPR was plotted on the x-axis, while the difference between adherence estimation from the e-diary and self-report and the MPR were plotted on the y-axis. Since the distribution of the differences was not normal, limits of agreement were derived from the 2.5 and 97.5 percentiles of the differences. Positive- and negative percent agreement (PPA and NPA respectively) as well as the overall rate of agreement (ORA) were calculated as described in Figure 1. Measures of agreement, rather than of diagnostic accuracy, were chosen in the absence of an absolute gold standard for adherence evaluation.[24]

Figure 1.

Calculation of positive (PPA), negative percent agreement (NPA) and overall rates of agreement (ORA). PPA is the percentage of patients who had poor adherence by medication possession rate (MPR) and also reported that they were non-adherent. NPA is the percentage of patients who had adequate adherence by MPR and also reported that they were non-adherent. ORA is the percentage of patients with concordant adherence evaluation by both MPR and self-report.

Results

Adherence to the e-diary

Demographic and clinical characteristics of the 62 patients who downloaded the e-diary application, and the 55 controls who were followed regularly, are given in Table 1. Fifty-nine (95%) patients in the e-diary group and 54 (98%) controls completed the 12-month follow-up period. Drop out was defined as complete cessation of e-diary activity or a no-show to clinical follow-up. Reasons for discontinuation included time-consuming/not interested (n = 3, e-diary group) and loss to follow-up (n = 1, control group). DMDs were changed during the study from injectable to oral medication for three patients in the e-diary group and for five patients in the control group. In the control group an additional five patients switched from one oral medication to another and one patient switched from an oral to an injectable medication.

Table 1.

Patients’ characteristics.

	E-diary (n = 62)	Control (n = 55)	P value[a]
Age	40.3 ± 11.4	42.3 ± 13.9	0.38
EDSS	3.2 ± 2.2	2.8 ± 2.0	0.34
Disease duration (from diagnosis, years)	8.2 ± 8.4	6.9 ± 8.0	0.42
Female	41 (66%)	42 (76%)	0.2
Ethnicity	Jewish: 54 (87%)Arab: 8 (13%)	Jewish: 37 (67%)Arab: 18 (33%)	0.1
Active disease at enrolment (relapse or MRI new lesion in previous year)	20 (32%)	26 (47%)	0.1
DMD at baseline	Fingolimod: 25 (40%)Dimethyl fumarate: 18 (29%)Interferon beta: 8 (13%) Teriflunomide: 6 (10%)Glatiramer acetate: 5 (8%)	Fingolimod: 3 (5%)Dimethyl fumarate: 24 (44%)Interferon beta: 18 (33%)Glatiramer acetate: 10 (18%)	<0.001

EDSS: Expanded Disability Status Scale; DMD: disease-modifying medication.

aP values are from independent group t-test for continuous variables and from chi-square or Fisher’s exact test for categorical variables.

Agreement between methods of adherence evaluation

The difference between e-diary-derived adherence and the MPR for various MPRs is plotted in Figure 2(a). The median difference between e-diary-derived adherence and the MPR was –3% (95% limits of agreement: –53% to 12%). It follows that in cases of disagreement between the e-diary and the MPR, e-diary-derived adherence tended to be lower than the MPR. The difference between self-reported adherence and the MPR for various MPRs is given in Figure 2(b). The median difference between self-report adherence and the MPR was 0.3% (95% limits of agreements: –20% to 42%). That is to say that in cases of disagreement between the questionnaire and the MPR, questionnaire-derived adherence tended to be higher than the MPR.

Figure 2.

Bland–Altman plots for the agreement of adherence evaluation between the e-diary, self-report and the medication possession rate (MPR). The median and 95% limits of difference in adherence evaluations are represented by the dotted lines.

Measures of agreement between the three methods of evaluation using a commonly accepted cutoff for non-adherence (<80%) are shown in Table 2. PPA, NPA, and ORA between the e-diary and MPR were 33%, 82%, and 77%, respectively. PPA, NPA, and ORA between retrospective self-report and MPR were 16%, 95%, and 87%, respectively.

Table 2.

Agreement between methods of adherence evaluation.

	MPR <0.8Positive“non-adherence”	MPR ≥0.8Negative“adequate adherence”	PPA	NPA	ORA
E-diary adherence <0.8Positive: “non-adherence”	2	10	33%	82%	77%
E-diary adherence ≥0.8Negative: “adequate adherence”	4	46
Self-report adherence <0.8Positive: “non-adherence”	2	5	16%	95%	87%
Self-report adherence ≥0.8Negative: “adequate adherence”	10	99

MPR: medication possession rate; PPA: positive percent agreement; NPA: negative percent agreement; ORA: overall rate of agreement.

Effect of the e-diary on adherence to DMDs

MPR below 80% was found in 6/62 (10%) patients in the e-diary group and in 7/55 (13%) controls. The proportion of patients with poor adherence to DMDs was similar in both groups (χ2(1) = 0.27, p = 0.6). The same comparison was repeated separately for participants who were using injectable DMDs and for participants who were using oral DMDs. E-diary reminders did not have a significant effect on the non-adherence rate in either subgroup (data not shown).

Discussion

Assessment of adherence to DMDs is crucial for appropriate treatment decisions. Practice guidelines emphasize the importance of taking into account non-adherence as a possible cause of breakthrough disease activity.[25] Yet, there is no gold standard method for medication adherence evaluation. In this study, we found substantial, clinically important disagreement between three measures of adherence evaluation: prospective self-report using an e-diary, retrospective self-report by means of a questionnaire, and the MPR. We also found that smartphone reminders did not improve the MPR of DMDs.

Requesting patients to retrospectively report how often they failed to take their medication is a common way to evaluate adherence in clinical practice. Differences between self-reported adherence and the MPR were larger when self-reported adherence was higher than the MPR, compared with instances in which self-reported adherence was lower than the MPR (Figure 2(b)). Only 16% of patients with MPR <80% self-reported non-adherence (Table 2). These findings imply that adherence may be over-estimated by retrospective self-report, as suggested by previous studies.[8,16,26,27] Possible explanations may be that patients do not remember accurately the number of missed doses, or that their report is more representative of the most recent time period, whereas adherence may change over time. It is also possible that patients are reluctant to admit non-adherence to avoid a rebuke by the care team. This was demonstrated in a study of adherence to inhaled bronchodilator for chronic obstructive pulmonary disease: 30% of participants in that study activated their inhaler intensively in the 3 h prior to their clinic visits, implying dumping medications to appear adherent.[28]

Differences between e-diary-derived adherence and the MPR were larger when e-diary-derived adherence was lower than the MPR (Figure 2(a)). Eighty-two percent of patients with adequate MPR (≥80%) were classified as adherent by the e-diary, compared with 95% that were classified as adherent by retrospective self-report (Table 2). Taken together, the e-diary seems to underestimate adherence compared with the MPR and retrospective self-report by questionnaire. This may be due to inconsistent reporting of medication use in the e-diary. However, since in Israel patients do not pay for DMDs, the possibility of medication hoarding exists as an alternative explanation for the discrepancy between adherence evaluation by the e-diary and by the MPR. Agreement on non-adherence between the e-diary and MPR was higher than between retrospective self-report and the MPR (33% vs. 16%, respectively) but still low. It follows that over-reporting of medication consumption in the e-diary could happen, although to a lesser degree than by retrospective self-report.

Importantly, disagreement between the three methods evaluated here was mainly in the detection of non-adherent patients, as depicted by the low positive percent agreement (Table 2). ORAs were misleadingly high, because they were closer to the much more prevalent agreement on adequate adherence. Likewise, the misleadingly low median differences between adherence rates of the various measures should not be taken as evidence of agreement, because the spread of actual differences around this median value was high and clinically important (Figure 2).

Taken together, substantial disagreement casts doubt on the reliability of assessment by the three methods that were compared. The ideal method of adherence evaluation should not require patient cooperation and be unequivocally linked to the act of consuming the medication. Examples could include electronic injectors of DMDs, which keep record of actual injections[29] or medication event monitoring systems (MEMS) that count needle disposals.[8] Notably, even with these methods, the possibility of medication dumping remains, as electronic injections do not report the passage of chemical material into the body; rather, they report the amount and timing of the liquid leaving the needle. Adequate agreement between MEMS and a medication diary has been reported,[8] but the evaluation period in that particular study was limited to 8 weeks, while our study lasted 1 year. It is reasonable to premise that events of non-recording of medication intake accumulate over time.

The proportion of non-adherence by the MPR did not differ between the group that received e-diary reminders and the group that did not receive them in this study, in line with previous research that demonstrated the ineffectiveness of reminding tools to promote adherence to chronic medications.[30] Interestingly, PwMS rank forgetfulness as the main reason for non-adherence to DMDs.[16,17] The ineffectiveness of reminders found in the present study raise questions on the contribution of forgetfulness to non-adherence. Non-adherence to chronic medications probably has deeper roots including personal and cultural beliefs, with implications for patient–physician communication.[31] Indeed, it has been shown that effective interventions to promote adherence were generally complex and included elements of providing more convenient care, information, counseling, reinforcement, family therapy and other forms of additional supervision and attention.[32,33] Reminders may have a role in such comprehensive programs, but they are probably ineffective on their own.

This study has several limitations that must be mentioned. There was an imbalance in the composition of DMDs that were used by patients of the e-diary and the control groups (Table 1) that could have influenced the adherence of the studied groups in an unpredictable way. Adherence was somewhat unexpectedly very high in both groups. Only 13 (11%) out of 117 PwMS in this study had an MPR <80%. This greatly decreased the power of the study to detect any effect of the e-diary reminders. One possible explanation for the high adherence rate may be the Hawthorne effect (change in behavior due to awareness of being observed).[34] Indeed, our participants knew that they were being monitored and that their pharmacy records were analyzed. It is also unknown whether participants from the control group were using unsolicited reminders, for example, alarm clocks. Furthermore, low prevalence of non-adherence results in unstable rates of positive agreement between the various methods of non-adherence evaluation (Table 2), since every individual has a profound effect on the positive percent agreement.

In conclusion, the present study employing an e-diary for PwMS demonstrated the substantial disagreement between methods of adherence evaluation, and that smartphone reminders have limited effect in improving medication adherence. In light of the importance of accurate estimation of adherence for informed medical decisions, new methods of unequivocal adherence detection are needed, such as digital adherence technologies being developed.[19,35] Implementation of these improved medication adherence assessment tools will also allow development of novel innovative interventions to improve therapy adherence as part of MS care.