Adherence to monthly online self-assessments for short-term monitoring: a 1-year study in relapsing-remitting multiple sclerosis patients after start of disease modifying treatment.

BACKGROUND: The participation of neurologists and patients in studies on the effectiveness and safety of newly authorized drugs in multiple sclerosis (MS) is insufficient. Monthly online self-assessments using patient-reported outcomes may help in short-term monitoring of neurological changes and side effects.
OBJECTIVE: Investigate in relapsing-remitting (RR) MS patients the adherence to monthly online self-assessments after the start of disease modifying treatment.
METHODS: Observational study in 39 neurological departments in The Netherlands. Patients starting glatiramer acetate treatment were instructed to complete online the Modified Fatigue Impact Scale 5-item version and the 8-item Leeds Multiple Sclerosis Quality of Life scale every month during 1 year (T0 toT12).
RESULTS: Sixty-three investigators included 163 analyzable patients. At T3, 148 (90.8%) patients had completed all questionnaires; at T6, 142 (87.1%); at T9, 133 (81.6%); and at T12, 123 (75.5%). Eight (4.9%) patients did not complete any questionnaire. Median values for inter-assessment intervals ranged from 32 to 34 days (first quartile [Q1] 30 days, third quartile [Q3] 41 days), and the final assessment was at 417 days (median: Q1 385 days, Q3 480 days). Forty-three (26.3%) patients completed the questionnaires at all time points (completion adherent) with their final assessment within 30 days after the scheduled T12 (interval adherent). Eighty (49.1%) patients were completion adherent, but not interval adherent. Forty (24.5%) patients were not completion adherent, as they discontinued assessments prematurely. Men were more interval adherent than women (47.5% vs 20.0%; P = 0.001).
CONCLUSION: The observation that three out of four (75.5%) RRMS patients completed two short questionnaires at all monthly time points during 1 year after the start of disease modifying treatment suggests that intensive online monitoring in this patient group is feasible. As only one in five (19.6%) patients adhered to the time intervals between self-assessments, measures are needed that improve the timely completion of questionnaires.

Introduction

In recent years, new drugs have become available for the treatment of relapsing– remitting (RR) multiple sclerosis (MS) and MS-related walking disability.1–4 After authorization the effectiveness of these drugs has to be evaluated in large numbers of patients, treated in various settings of daily practice. However, most effectiveness studies in MS involve only a small number of hospitals with an underrepresentation of general neurological practices, as busy schedules and limited resources make it difficult for neurologists to participate. In RRMS a clinical worsening in the first year of disease modifying drug (DMD) treatment predicts a worse long-term disability.5 To be able to consider treatment alternatives in a timely fashion it is therefore necessary to closely monitor clinical disease activity. Since the detection of relapses is patient-driven, a relapse might not be a sufficiently reliable sign of clinical disease activity. For, during DMD treatment, as well as in an advanced stage of RRMS, relapses may be atypical by developing gradually and therefore be noticed less readily by patients. Moreover, a relapse-induced increase in disability and the degree of recovery are relevant aspects as well that need to be quantified when assessing disease activity. In clinical trials relapse-related disability is measured by the Expanded Disability Status Scale (EDSS).6 However, this measure is time consuming, fails to adequately assess fatigue and cognitive impairment, and does not quantify the disease burden from the patient’s perspective. Patient-reported outcomes (PROs) reflecting various aspects of disability may counterbalance these shortcomings.7 Interestingly, the second-line DMDs not only prevent relapses but may also lead to clinical improvements, that should be adequately documented for treatment evaluation. As to safety monitoring of DMDs, pre-defined frequent assessments may detect safety issues more comprehensively and earlier than patient-initiated reporting. Finally, from a methodological viewpoint it can be argued that repeated measurements provide more reliable information on disability and side effects than single pre- and posttreatment assessments.

Given these considerations, we imagined that the requirements for optimal monitoring of disease activity and safety after the start of a DMD treatment in RRMS might not be fully met by regular outpatient visits using doctor-centered measures and that more frequent assessments using PROs could have an added value.

Online questionnaires and diaries for self-assessment can be completed at home at convenient time points; errors and missing data are minimized by automated checks of completeness and consistency; electronic data capture prevents transmission errors; and scores are calculated automatically.8–10 As the online use of PROs does not depend on out-patient visits, monitoring may be more intensive. Moreover, online use of PROs could provide information that complements or substitutes doctor-reported data, rendering monitoring less burdensome for neurologists.11

In view of the above, we investigated whether patients with RRMS would adhere to a monthly online self-assessment schedule to monitor aspects of effectiveness after the start of disease modifying treatment. The choice of the variables was based on the availability of short validated questionnaires assessing the past month, and on their potential clinical relevance. So we chose fatigue – a frequent and disabling symptom in MS - and health-related quality of life – an overall measure of well-being from a patient’s perspective.

Methods

Study design and procedures

This was a prospective, observational, multicenter study in which 54 neurologists and nine MS nurses participated from 39 neurological outpatient departments in The Netherlands. The information on the study was given by the neurologist or the nurse. Patients who agreed to participate signed an informed consent form. Because of the observational design of the study, a review by an ethical committee was not required, as the study did not meet the criteria stated in the Dutch Medical Research Involving Human Subjects Act of 1999.12 The study was carried out in compliance with the Declaration of Helsinki.

After having received a personal code, patients logged on to the study website to choose a username and password. To protect the personal data from unauthorized access, various mechanisms were used to ensure compliance with European Union regulations concerning online medical data. Patients were instructed to visit the website http://www.gezondheidsmeter.nl at the start of treatment, or as soon as possible in case they had already started treatment, and at monthly intervals thereafter during a period of 12 months (T1 to T12), in order to complete two short questionnaires: the Modified Fatigue Impact Scale 5-item version (MFIS-5) and the Leeds Multiple Sclerosis Quality of Life (LMSQoL) scale. The MFIS-5 is a validated, short, 5-item questionnaire examining a patient’s perception of fatigue over the past month.13,14 The LMSQoL is a validated, short questionnaire consisting of eight questions, examining MS-related aspects of health-related quality of life over the past month.15,16

Patients were sent an email informing them of the fact that the questionnaires were available online for completion. Patients who had not completed the questionnaires within 1 week were sent a reminder email. Those who had not completed the questionnaires within 1 week after this latter email were given a reminder phone call by the study help desk.

The questionnaires ran on the Curavista Health platform, a fully operational online business application for patient–health care provider communication that was already being used in various settings. The responses were automatically captured. The data collection started October 1, 2004 (first patient in), recruitment stopped July 31, 2007 (last patient in), and the closure of the database was December 31, 2008 (last patient out). The items of the two questionnaires were fixed, without randomization or alternation. Adaptive questioning was not used. The MFIS-5 consisted of five items that were presented on one page (Figure 1), and the LMSQoL consisted of eight items presented on another page. Consistency and completeness checks were done before a questionnaire was submitted (Oracle Database [Oracle, Redwood Shores, CA, USA] with JAVA scripting). The patients saw an overview of all questions and answers before submitting and could change the answers before submitting. After submission changes were no longer possible. Users were registered by their doctors, using date of birth and email address. Via graphs, patients were given online feedback on their scores and the respective changes over time (Figure 2). To this end, the maximum scores of both scales were transformed to 100%, and the fatigue score was inversed to represent “fitness.”

Figure 1

Screenshot of the five items of the MFIS-5 questionnaire. Copyright © 2012, Curavista bv, Geertruidenberg, Netherlands. Reprinted with permission.

Notes: Questions: (1) Due to my fatigue I have been less alert. (2) Due to my fatigue I have been limited in my ability to do things away from home. (3) Due to my fatigue I have had trouble maintaining physical efforts for long periods. (4) Due to my fatigue I have been less able to complete tasks that require physical effort. (5) Due to my fatigue I have had trouble concentrating. Possible answers: never, seldom, sometimes, often, almost always.

Abbreviation: MFIS-5, Modified Fatigue Impact Scale 5-item version.

Figure 2

Screenshot of graphic feedback of changes in fitness score (black line) and quality of life score (red line). Copyright © 2012, Curavista bv, Geertruidenberg, Netherlands. Reprinted with permission.

Patients

Inclusion criteria were: (1) diagnosis of RRMS; (2) at least 18 years of age; (3) relapse- and steroid-free for at least 30 days; and (4) willing and able to complete online questionnaires. Exclusion criteria were: (1) hypersensitivity to glatiramer acetate or mannitol; (2) pregnancy or lactation; (3) glatiramer acetate treatment for more than 3 months; and (4) previous glatiramer acetate treatment. One-hundred-seventy-six patients were included. In 13 patients the inclusion was a protocol violation since the disease course was not RR or was unknown (n = 9), or the patients had started treatment more than 3 months before study entry (n = 4). The data analysis set consisted of 163 patients. Mean value for age at baseline was 40.9 years (95% confidence interval [CI] 39.3–42.4 years), for disease duration was 5.2 years (95% CI 4.4–6.0 years), and for EDSS score was 2.8 (95% CI 2.5–3.1).

Outcome measures

Study outcome was the adherence to the monthly self-assessment schedule. Measures for adherence were (1) the percentages of patients that had completed all questionnaires at the time points T3, T6, T9 and T12, and (2) the intervals between two consecutive self-assessments and between the baseline and T12 assessment.

Patients were classified as completion adherent if they had completed both questionnaires at all time points. Completion-adherent patients who had completed the study within 30 days after the scheduled T12 date were classified as interval adherent.

Statistical analysis

The numbers and percentages of patients who completed the questionnaires at the various time points were calculated, as well as the intervals (days) between two consecutive self-assessments and between the baseline and T12 assessment.

In order to test for significance, for categorical variables we used the Chi-square test (Pearson’s Chi-square test) and, for ordinal variables, the Chi-square test for trend (linear-by-linear association). The difference in continuous variables between two groups was tested by means of the nonparametric Mann–Whitney U-test. We performed analyses of adherence by sex, age, and duration of disease (Chi-square test). For statistical analysis we used SAS (v 9.1.3; SAS Institute Inc, Cary, NC, USA) and SPSS (v 19 or higher; IBM Corporation, Armonk, NY, USA).

Results

Completion of questionnaires

Figure 3 presents the numbers and percentages of patients who completed the MFIS-5 and LMSQoL questionnaires at the consecutive time points. Eight (4.9%) of the 163 patients did not complete any questionnaire at any time point. In 155 (95.1%) patients the MFIS-5 and LMSQoL questionnaires were always completed in combination. Patients who stopped self-assessments had completed all questionnaires until stopping. At T3 148 patients, being 90.8% of all patients and 95.5% of the patients who had performed their baseline assessment, had completed all questionnaires. At T6 this number was 142 (87.1%, 91.6%), at T9 it was 133 (81.6%, 85.8%), and at T12 it was 123 (75.5%, 79.4%).

Figure 3

Numbers and percentages of patients who completed the MFIS-5 and LMSQoL questionnaires at the various time points from T0 to T12.

Abbreviations: LMSQoL, Leeds Multiple Sclerosis Quality of Life; MFIS-5, Modified Fatigue Impact Scale 5-item version.

Time intervals

Intervals (days) between two consecutive self-assessments and between the baseline and final self-assessment (T12) are given in Table 1. Median values for intervals between two consecutive assessments ranged from 32 to 34 days. The median time between T0 and T12 was 417 days.

Table 1

Intervals (days) between two consecutive time points (T) of self-assessment and between baseline and final self-assessment

Interval	N	Mean	SD	Q1	Median	Q3
T0–T1	155	39.1	16.4	31	33	37
T1–T2	152	36.4	12.7	30	32	37
T2–T3	148	37.4	13.4	30	32	37
T3–T4	148	36.7	13.3	30	32	37
T4–T5	146	38.7	22.8	31	33	37
T5–T6	142	36.4	11.9	30	32	36
T6–T7	140	37.1	13.8	30	32	37
T7–T8	136	39.2	19.4	30	32	39
T8–T9	133	38.5	18.5	30	33	38
T9–T10	131	41.7	21.9	30	34	41
T10–T11	131	41.7	28.3	30	32	40
T11–T12	123	40.1	22.8	30	33	39
T0–T12	123	451.5	102.3	385	417	480

Abbreviations: N, number of patients; Q1, first quartile (25%); Q3, third quartile (75%); SD, standard deviation.

Completion adherence and interval adherence

One-hundred-twenty-three (75.5%) patients were completion adherent, whereas 40 (24.5%) were not completion adherent (premature discontinuation of self-assessments). Of the completion adherent patients, 43 (26.3% of the total group) were interval adherent and 80 (49.1% of the total group) patients were not interval adherent (final self-assessment later than 30 days after scheduled T12).

There was no difference between the completion adherence rate in men and in women (75.0% vs 77.5%; P = 0.75). However, men were more often interval adherent than women (47.5% vs 20.0%; P = 0.001). For age, disease duration, and pretreatment, there were no significant differences between adherent and nonadherent patients. The numbers of patient per center were too low to include centers in the analyses.

Discussion

In a prospective, observational, multicenter study we investigated the adherence to monthly online self-assessments in RRMS patients during 12 months after the start of disease modifying treatment. Adherence to the completion of two short questionnaires at all time points (completion adherence) and adherence to the scheduled monthly intervals (interval adherence) were evaluated. It was found that 75.5% of the patients were completion adherent, but only 19.6% were interval adherent. The intervals between two consecutive scheduled assessments were systematically exceeded by 2 to 4 days (median), leading to an extension of the 12-month study period by 1.7 months (median). One in four patients (26.3%) was both completion adherent and interval adherent. Almost half of the patients (49.1%) were completion adherent, but failed to comply with the predetermined monthly intervals. Interestingly, male patients were more likely to be interval adherent than women.

Post-authorization research is important to evaluate the effectiveness, side effects, and safety of newly authorized drugs in daily practice.11 Internet usage is well accepted by MS patients.17 Academic MS centers, national networks of MS centers, and patient communities have developed tools for patient-centered web-based research.8–10 Web-based studies can collect “real-world” information on the therapeutic use of drugs from the patient’s perspective using appropriate PROs.18,19 In patients who start a treatment, the intensive online monitoring of predefined outcomes may facilitate the collection of data on the short-term effects.20

A major factor determining the practicability of intensive online monitoring is the adherence to the self-assessment schedule. Whether the completion and interval adherence rates in monthly assessments that we found in RRMS patients are adequate depends on the variable that is monitored, the psychometric characteristics of the PRO, and the goal of the monitoring. In MS, the potential applications of intensive online monitoring by self-assessments include the evaluation of the effectiveness of disease modifying treatments and the follow-up of the recovery from a relapse (disability, impact of disability, and quality of life questionnaires), the evaluation of the effectiveness of symptomatic treatments (symptom-specific and quality of life questionnaires), the detection of side effects (drug-specific questionnaires), the detection of insidious relapses, and of the progression from RRMS to secondary progressive MS (disability questionnaire). As we studied RRMS patients with relatively short disease duration and fairly limited (motor) impairment, our findings cannot be generalized to later stages of RRMS or progressive MS. The optimal duration of the monitoring process as well as the frequency and intervals of self-assessments has to be established per indication. For early detection of a typical relapse, a predefined schedule of assessments is not suitable as relapses may occur at any point in time, and a patient-initiated completion of a questionnaire seems a better option.

The 75.5% completion adherence in our study is in line with the existing literature. Bush et al reported a completion rate of monthly online brief questionnaires during 1 year for measuring quality of life in patients after hematopoietic stem cell transplantation of 78.3%.20 In the recent study by Bhinder et al, most patients complied with intensive online self-assessments, although a substantial minority was unwilling or unable to take part in home assessments.21 These findings in patients contrast with those in healthy persons. In young, healthy adults who had received a preventive training and exercise program, a monthly self-assessment schedule over a 12-month period had a response rate of only 14% after 1 month, which further decreased to 5% after 12 months.22

The less-than-optimal adherence in our patient group may relate to various factors. The web-based approach does not necessarily play a role. In a study of elderly patients with uncontrolled hypertension who were visited at monthly intervals during a 10-month period for pharmaceutical care, only 69% of the patients completed the study.23 In our study, the fact that the outcomes were not used for treatment evaluation by the neurologists may have negatively affected motivation and adherence. Basch et al reported that without clinician feedback patients demonstrated limited voluntary interest in self-reporting between outpatient visits.24 The overall patient burden of our monthly monitoring schedule was conceivably low, as the interference with daily life was limited in frequency and time. Yet, the study could have constituted a mental pressure, as the repeated self-assessments may have heightened the patients’ awareness of their fatigue or impaired quality of life and the questions may have confronted them with the potential negative consequences of their MS. If the adherence was negatively affected by the (psychological) intensity of the monitoring process, one would expect, however, that patients would become less punctual with increasing study duration (ie, a worsening interval adherence). Yet, the length of the intervals between consecutive assessments in the fourth 3-month period (T9 to T12) did not significantly differ from that in the first 3-month period (baseline to T3) (data not shown). This rather suggests that systematic factors have interfered with the interval adherence right from the start. We think that the busy life of many young people may explain the delay between the receiving of the email and the completion of the questionnaires, and that this factor might be more prominent in women, who often combine work, household activities, and care for children. In fact, in the study by Bhinder et al, reasons for non-completion were lack of time or an inability to find time to complete the questionnaire.21 Our interval data suggest that a way to prevent delays in self-assessments could be to send the emails 3 to 5 days before the assessment is due.

To promote adherence we used short and relevant questionnaires, sent reminder emails, and provided graphic feedback on the changes in personal scores over time.25,26 Other factors that may improve adherence are the use of smart phones and short message services.27 It has been known that in patient care, web-based monitoring should be integrated into the care process, whereby relations and communication with health care providers play a critical role.28,29 A study of diabetes patients found that the frequency of monitoring blood sugar was increased by more personalized interactions, notably a higher number of person-centered messages from the health care provider.29

In MS patient care the start of a disease modifying treatment is followed by monitoring to check whether the response is as expected and the therapeutic goal is being met. Given the predictive value of a short-term clinical worsening after the start of treatment with respect to long-term disability, monitoring of clinical features can provide a signal for a switch to a more potent drug or provide motivation to adhere to the existing treatment.30 In daily practice, the monitoring process coincides with outpatient visits, during which patients are assessed clinically and by magnetic resonance imaging. Intensive online monitoring using PROs provides unique and complementary data, as these instruments comprehensively measure the benefits and adverse effects of a treatment on both the physical, emotional, and social health status.7 A recent study in amyotrophic lateral sclerosis patients showed that the online use of the Amyotrophic Lateral Sclerosis Functional Rating Scale compared well to the in-clinic evaluation and complements the onsite assessments.31 Intensive online monitoring in MS using PROs might also help to quantify the effect of symptomatic treatments for walking disability, fatigue, depression, anxiety, or bladder dysfunction, whereas the Multiple Sclerosis Impact profile, a self-report tool that measures the objectified and perceived impact of MS, might complement the EDSS.32

In this study, we assessed in RRMS patients the adherence to a monthly online self-assessment schedule for intensive monitoring aspects of their disease by using short questionnaires. As the scores were not made available to the neurologists or MS-nurses, our data do not inform on the clinical relevance of this approach. Future studies are needed to investigate PRO findings in correlation with objective measures like EDSS and magnetic resonance imaging, the reliability of the various PROs, what constitutes a clinically relevant change, and the optimal frequency of assessments.

In all, our data suggest that intensive web-based monitoring in RRMS patients is feasible for research purposes in terms of adherence. However, measures need to be taken to improve the timely completion of questionnaires, like sending the emails 3 to 5 days before the assessment is due, enabling the use of smartphones for completion, and integrating the online assessments into the care process.