Using telehealth in motor neuron disease to increase access to specialist multidisciplinary care: a UK-based pilot and feasibility study.

OBJECTIVES: Care of patients with motor neuron disease (MND) in a specialist, multidisciplinary clinic is associated with improved survival, but access is not universal. We wanted to pilot and establish the feasibility of a definitive trial of a novel telehealth system (Telehealth in Motor neuron disease, TiM) in patients with MND.
DESIGN: An 18-month, single-centre, mixed-methods, randomised, controlled pilot and feasibility study. INTERVENTION: TiM telehealth plus usual care versus usual care.
SETTING: A specialist MND care centre in the UK. PARTICIPANTS: Patients with MND and their primary informal carers. PRIMARY AND SECONDARY OUTCOME MEASURES: Recruitment, retention and data collection rates, clinical outcomes including participant quality of life and anxiety and depression.
RESULTS: Recruitment achieved the target of 40 patients and 37 carers. Participant characteristics reflected those attending the specialist clinic and included those with severe disability and those with limited experience of technology. Retention and data collection was good. Eighty per cent of patients and 82% of carer participants reported outcome measures were completed at 6 months. Using a longitudinal analysis with repeated measures of quality of life (QoL), a sample size of 131 per arm is recommended in a definitive trial. The methods and intervention were acceptable to participants who were highly motivated to participate to research. The low burden of participation and accessibility of the intervention meant barriers to participation were minimal. However, the study highlighted difficulties assessing the associated costs of the intervention, the challenge of recruitment in such a rare disease and the difficulties of producing rigorous evidence of impact in such a complex intervention.
CONCLUSION: A definitive trial of TiM is feasible but challenging. The complexity of the intervention and heterogeneity of the patient population means that a randomised controlled trial may not be the best way to evaluate the further development and implementation of the TiM. TRIAL REGISTRATION NUMBER: ISRCTN26675465. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

This is the first study of the feasibility of this digitally enabled care system for patients and carers living with motor neuron disease.

The trial methods and intervention enabled patients with significant disabilities to participate.

The qualitative data collection aimed to identify key barriers and enablers to participation in clinical trials of telehealth and motor neuron disease from the perspective of patients, carers and nurses.

This was a study with a small number of patients in a single centre.

It was not possible to fully assess the impact of the intervention on the clinical service’s staffing or healthcare resource use.

Background

Motor neuron disease (MND) is an incurable, neurodegenerative disease that causes progressive muscle paralysis, limb weakness, breathing, speech and swallowing difficulties leading to death after, on average, 2–4 years from symptom onset.1 Riluzole, non-invasive ventilation (NIV) and possibly edaravone only offer small survival benefits.2–4 Attendance at a specialist MND multidisciplinary clinic (MDC) is associated with improved survival and increased use of proven therapies.5–10 The aim of specialist MND care is to maximise survival and QoL by providing coordinated, patient-centred care to address the biopsychosocial needs of patients and their families.11 It is recommended that the MDC should monitor patients regularly to detect and treat complications quickly.12 13 However, patients become progressively more disabled and travelling to clinic becomes difficult or impossible. Even in developed countries, attendance at MDCs varies (between 43% and 85%) and many of those who do see a specialist are unable to return.14 The disease is rare (a worldwide prevalence of 5.40 per 100 00015) meaning there are few specialist centres and general clinicians have limited experience of caring for patients with MND. This makes accessing the right care at the right time difficult leading to significant distress.16–21 There is therefore a great need to improve access to specialist MND care.

We developed the Telehealth in Motor neuron disease (TiM) system: a digitally enabled care system using telehealth to enable patients and their informal carers to report their progress and symptoms from their homes.22 We hypothesised that improving access to specialist care may result in earlier identification and management of complications thus improving QoL and survival. We thought the TiM system could improve care coordination and result in better prioritisation of health resource use, thereby reducing costs or increasing service capacity. The TiM was developed using a process of user-centred codesign22 but further piloting of such a complex intervention was required.23

Telehealth is a complex intervention that consists of different component parts (such as the software, the context and behaviours of those who use it) whose success depends on these interacting factors.23 Clinical trials of telehealth face various challenges including difficulties with recruitment, staff engagement and difficulties capturing the important impacts of the intervention.24–26 Furthermore, MND trials tend to recruit an unrepresentative sample of patients (on average younger, male patients with longer survival).27 This may be explained, in part, due to the many barriers to participation including the need to travel to study centres, and the small number of geographically dispersed patients who may be frail or deteriorating rapidly. The Medical Research Council (MRC) Framework for Developing and Evaluating Complex Interventions highlights the importance of feasibility and piloting of the study methods to ensure the definitive trial will overcome these barriers.23

Aims of the study

We wanted to pilot the methods and evaluate the feasibility of conducting a definitive randomised controlled trial (RCT) of the TiM in patients and carers versus usual care. We aimed to use low-burden, pragmatic study methods that could recruit and retain a representative sample of all patients with MND. The feasibility outcomes examine recruitment, retention and data collection. The study also aimed to provide an understanding of the resources required to conduct a definitive trial including staff burden and an estimation of variation in outcomes in order to provide a more accurate predictor of sample size. Online supplementary file 1 describes the feasibility questions using A process for Decision-making after Pilot and feasibility Trials (ADePT) framework28. We also explored factors that may influence these outcomes and also whether the outcomes measures effectively assessed aspects of life with MND and the impact of the TiM.

We also conducted a process evaluation of the TiM to understand how the system was used and to identify some of the potential impacts of the TiM on participants, carers and the MND service. This is reported in a parallel publication.29

Methods

Study design

This was a single-centre, unblinded, randomised, controlled pilot and feasibility trial of usual MND care versus the TiM plus usual care. Usual care involves invitations to the MDC clinic every 2–6 months and access to the MDC between times via the specialist nurse. The protocol and statistical analysis plan are available in online supplementary file 2.

Patient and public involvement

In addition to the user-centred design process used to develop the TiM22 during development of the intervention and the protocol we consulted patients, carers and the Sheffield MND Research Advisory group (a patient and public involvement group). They reviewed the intervention, principles of the trial, trial design, outcome measures and participant information leaflets and provided comments on their feasibility and accessibility. They were not involved in recruitment. Results of the study have been communicated at various public meetings through the Sheffield MND Research Advisory group and local branch of the MND Association and a lay summary will be circulated. Members of this group attended the trial steering and trial management groups. AQ was a member of the trial management group, provided advice on the research methods, interpretation of the data and dissemination and is a coauthor on this paper.

Recruitment

We prescreened patients using a clinical database of patients attending the Sheffield MDC. We determined the order of invitation using a list of random numbers generated using Excel. When the pool of prevalent patients was exhausted, we invited all newly diagnosed patients. We invited patients and their primary informal carer to participate by letter including a prepaid return slip. Eligibility was confirmed and written/witnessed verbal consent was obtained at all research visits and data collection was conducted in participants’ homes. This approach meant patients who were not currently attending the MDC (due to frailty or geography) were able to participate.

Eligibility criteria

We included adult patients receiving care from the Sheffield MDC, living within 2 hours’ drive from Sheffield. Initially the diagnostic inclusion criteria were those with a diagnosis of clinically definite or probable amyotrophic lateral sclerosis (ALS) according to the El Escorial Criteria.30 However, prior to recruitment commencing, a review of 200 patients on the MDC database found that 58% of patients would be excluded based on these criteria: 38% of patients had ALS but did not fulfil the El Escorial Criteria at their last assessment30 and 21% of patients had atypical MND (primary lateral sclerosis (PLS), progressive muscular atrophy (PMA) or an uncategorised progressive ALS/MND illness). We felt many of these patients not fulfilling the strict criteria would benefit from better MDC care and so we modified the criteria to include all patients with ALS with symptom onset within the last 3 years. We also included all patients diagnosed with any type of MND (ALS, PLS, PMA) who had evidence of progression in their condition (an indicator that they may require MDC monitoring and care) as evidenced by a deterioration in the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R)31 of at least two points during the previous 18 months (a small but meaningful change).

We excluded patients attending another MDC, those unwilling to allow their carer to operate the TiM on their behalf if they could not do it themselves and those with no form of telephone or internet access. Patients had the option of identifying their primary informal carer who was then invited to participate as a carer and consented. Eligible informal carers were adults who were the patients’ main provider of unpaid care. Initially, patients could participate only if their carer consented to be involved. Later we changed the criteria to allow patients to participate without a carer.

Randomisation

Patients were randomised 1:1 after recruitment to receive usual care or TiM using www.sealedenvelope.com which employs permuted block randomisation with a mixture of block sizes (block size concealed). Stratification was not employed.

Intervention

A detailed description of the TiM has been published.22 The use of the TiM in this trial is described in detail in the parallel publication describing the process evaluation.29 In brief patients and carers were asked to complete weekly sessions answering questions about their condition (such as functional ability, symptoms, depression and anxiety symptoms, carer strain) and patients recorded their weight and balance. The results could be viewed on a website by the MND nurse at the specialist care centre. She could take actions including telephoning the patient/carer, expediting clinic appointments or liaising with the multidiscipliary team. She could not delay appointments or make clinical decisions without checking the accuracy of the information with the patient, carer or clinical team. All participants were shown how to use the TiM app during recruitment and the presence of any difficulties using the system was recorded.

Data collection

Patient/carer-reported outcome measures (PROM) were completed at home during the baseline study visit and at 3, 6, 12 and 18 months using postal questionnaires. Generic and MND-specific PROMs captured QoL, MND clinical outcomes, survival and health resource use. Adverse events were recorded using PROMs and during MDC visits. We created a ‘Shadow Monitoring Protocol’ where the MND physician would review the TiM data and complete a questionnaire 2 weeks prior to MDC appointments and again at each appointment. Physicians were asked to respond to agree/disagree statements assessing their opinion on the accuracy and acceptability of the TiM answers and whether it may be possible to use the information to make decisions without seeing the patient (see online supplementary file 2; p 71). Telehealth nurse activity was also collected using a 2-week diary, twice in the trial. After the trial we downloaded all data in TiM system into Excel.

We conducted 56 semistructured participant interviews (characteristics are described in full in online supplementary file 3). Control participants were interviewed at baseline (17 interviews) and intervention participants were interviewed at 1 and 6 months (20 at 1 month and 19 interviews at 6 months). Most interviews were face to face at home with patients and carers together, but telephone and email were also used. Interviews were also conducted with the telehealth nurse (at months 4 and 14) and a community nurse (month 18). Topic guides were used (see online supplementary file 2 pp 41–44) and field notes taken during and after the interview. Interviews were audio recorded and transcribed verbatim. Early results and observations during the trial informed later interviews. As new themes were being identified later in the research study, we attempted to interview all participants and by the end of the study no new themes were identified. To evaluate the feasibility of the study, participants were asked about their attitudes towards research. To evaluate the validity and acceptability of the PROMs, control participants also performed a ‘think-aloud’ task during which they were asked to complete the baseline PROMs and describe their reactions. To explore the feasibility and acceptability of digitally enabled care, participants and clinicians were asked about their attitudes towards technology, the TiM system and MND care.

Outcomes

The trial assessed the following feasibility outcomes:

Trial processes: rates of eligibility, recruitment, retention and completion of postal questionnaires.

Use of the intervention: frequency of use of TiM by participants (collected automatically by the TiM system), participant satisfaction with TiM (questionnaires) and telehealth nurse time using TiM using two fortnight diaries (see parallel paper for additional results).29

In addition, clinical outcomes (listed in tables 1 and 2) were collected to test trial procedures and as indicative parameters to inform the sample size of a full-scale trial.

Blinding, bias and study conduct

It was not possible to blind the patients or investigators to treatment allocation and EVH had involvement in some patients’ clinical MDC visits. Measures to reduce bias were used. The role of EVH as an investigator and her involvement in the TiM development was explained at each visit and any clinical queries were passed onto the telehealth nurse. Follow-up PROMs were completed by participants independently at home and entered by an independent study nurse. Quantitative analysis was overseen by the study statistician using a prespecified analysis plan. Data triangulation (qualitative, TiM use, trial data) and methodological triangulation (patients, carers, staff) were employed. EVH conducted all the interviews except the 14-month interview with the telehealth nurse which was conducted by coauthor WOB (an experienced qualitative researcher independent of the clinical team) who also oversaw the interview planning, conduct and analysis.

Analysis

Quantitative data were analysed using descriptive statistics and qualitative data were organised using NVivo32 and analysed using thematic analysis.33 A triangulation process compared the quantitative and qualitative data to further understand and explain important, incongruent and unexpected observed phenomenon. Early results provided some insight into how the TiM was being used and informed changes to the intervention and study methods.

Sample size

A target of 40 patients and 40 carers was selected to enable an estimation of the SD of potential outcome measures to within a precision of ±20% of its true underlying value with 90% confidence.34 The sample size was also based on guidance that a minimum of 12 evaluable patients per trial arm is required (ie, after allowing for death, withdrawal or dropout) in order to inform a sample size calculation for a definitive trial.34 35

Role of the funding sources

The TiM was developed through a collaboration between the University of Sheffield, Sheffield Teaching Hospitals NHS Trust, Mylan and Abbott Healthcare Products Ltd. This trial was funded by the National Institute for Health Research and the Motor Neurone Disease Association. Mylan supplied software, hardware and some technical expertise. The telehealth nurse took on the additional duties as part of her current role. The study design, conduct, analysis, and interpretation of data, writing of the report, and the decision to submit the paper for publication were conducted by the authors independently of the funders with the exception of a requirement to report adverse events the investigator deemed to be related to Abbott Healthcare’s pharmaceuticals.

Results

A summary of the feasibility questions using the ADePT framework28 is presented in the online supplementary file 1. Qualitative data are reported within each section to explain the findings. Online supplementary file 3 contains the results of each outcome measure and supporting qualitative quotes (online supplementary file 3).

Screening and eligibility

Three hundred and six patients were prescreened (figure 1). One hundred and twenty-three patients (40%) were excluded because the Sheffield MDC was not their main or current care centre. Of the remaining 183, eighty-eight patients were excluded on clinical grounds, mainly due to lack of disease progression or cognitive impairment. This left 95 eligible patients (52% of the patients attending the MDC).

Figure 1

Consolidated Standards of Reporting Trials (CONSORT) flow diagram. Follow-up varied depending on when patients entered the study ranging from 18 months (for those recruited at the start of recruitment period) to 6 months for those recruited at the end of the recruitment period. At each time point patients are either reported as reached the time point (analysed) or had died, withdrawn or did not reach that time point due to being recruited later in the study. ARC database, the database of living patients with MND receiving/having received care at the Sheffield MDC; MCD, multidisciplinary clinic.

Forty-two patients (44%) expressed an interest in participating. Forty (42%) were eligible: 28 prevalent and 12 incident cases. All 40 were consented, randomised and received the intervention between October 2014 and November 2015. Thirty-seven eligible carers were recruited. Three patients were recruited who did not have a primary carer.

Participant characteristics

Age, gender, phenotype and site of onset were similar to a much larger cohort of patients at the Sheffield MDC7 (table 3). Age ranged from 30 to 78 years and participants had disabilities ranging from mild to severe, including 13 (33%) who used either NIV or a gastrostomy tube (King’s stage 436).

Table 3

Participant characteristics

	Telehealthn=20	Controln=20
Gender male	14 (70%)	14 (70%)
Age (years)Mean (SD), range	60.4 (11.7), 30–78	60.0 (10.0), 39–73
Phenotype
Amyotrophic lateral sclerosis	17 (85%)	18 (90%)
Primary lateral sclerosis	2 (10%)	2 (10%)
Progressive muscular atrophy	1 (5%)	0 (0%)
Disease duration (months)
Mean (SD), range	53 (48), 12–197	46 (35), 7–123
Duration since diagnosis (months)
Mean (SD), range	32 (34), 3–137	21 (19), 1–58
King’s ALS clinical stage*
1	3 (15%)	2 (10%)
2	4 (20%)	5 (25%)
3	5 (25%)	8 (40%)
4	8 (40%)	5 (25%)
Use of the TiM app
Independently	17 (85%)	17 (85%)
Assistance from carer	1 (5%)	1 (5%)
Patient instructs carer	2 (10%)	2 (10%)
Technology use†
Daily	14 (70%)	18 (90%)
A few times per week	3 (15%)	1 (5%)
Once a week	1 (5%)	1 (5%)
Every few weeks	0 (0%)	0 (0%)
Never	2 (10%)	0 (0%)
Home technology
Broadband	18 (90%)	20 (100%)
3G mobile reception	18 (90%)	15 (75%)

*King’s stage 1 refers to patients with functional deficit in one domain, stage 2: two domains, stage 3: three domains, stage 4: patients requiring non-invasive ventilation (NIV) and/or gastrostomy.36 King’s stage was calculated using the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) scale at baseline.

†Technology: computer, smartphone, tablet.

ALS, amyotrophic lateral sclerosis; TiM, Telehealth in Motor neuron disease.

Barriers and enablers to recruitment

Interviews indicated that recruitment and randomisation were acceptable and participants understood the principles of the study and were willing to be randomised. Some participants thought that the TiM would simply function to collect research data, rather than to facilitate MDC care. Participants wanted to participate because they liked the concept of telehealth and because they were highly motivated to participate in research. They expressed a strong altruistic desire to help others and to help the clinical team. Participation made patients feel they still had a valuable contribution to make, even when severely disabled:

I love being part of something worthwhile. (Patient 229)

Participants thought they might gain benefit by learning more about the condition, to improve their chances of taking part in a treatment trial and to have increased contact with the MND team. Some kept up to date with information on the internet and many expressed frustration about the speed of research and felt time was running out for them to be cured. They all wanted to see treatments that had tangible benefits.

…it doesn’t have to cure you it just has to make things better. (Carer 232)

Participants wanted to learn about research as this provided hope. Patients thought trials would be ‘safe’ if they involved a doctor whom they trusted but also recognised that information could be unreliable and offer ‘false hope’. A small number were willing to use unproven therapies or take part in trials even if they had potential for significant harm in order to gain an opportunity to be cured as they felt they had ‘have nothing to lose’.

Retention

To maximise the experience gained using the TiM, all participants remained in the study until they died or the study finished in April 2016. Follow-up ranged from 6 to 18 months. Two patients (5%) who were severely disabled at recruitment felt too ill to continue. No carers withdrew and no participants were lost to follow-up.

The factors that facilitated study participation were the low burden of the study and the intervention (completed at home, minimal visits and a clear understanding of what would be expected of them). Participants identified barriers posed by other clinical trials such as fatigue, burden or disruption of family life but stated these were not experienced in this study. Carers did not want research that required them to be removed from their caring duties.

My care’s here. I can’t have anything that takes it away from what I’m doing with P. It’s got to be very simple things. I can sit with my iPad and I can fill in a questionnaire. (Carer 184)

Adherence to the intervention

Detailed description of the acceptability, use of and adherence to the intervention is described in the parallel publication.29 In brief: compliance was high with 14 (70%) patients completing a TiM session, on average, fortnightly and 13 (70%) carers completing at least three weekly sessions.

Feasibility and validity of the participant-reported outcome measures

Adherence to the postal questionnaires was good. At 6/12 months 80%/71% of patients and 82%/67% of carer questionnaires were returned. Both treatment groups had similar completion rates. Participants felt the PROMs were accessible and not burdensome. They welcomed a thorough assessment of all aspects of life with MND and identified PROMs examining their emotional health and carer strain as the best assessment of their experiences of MND.

To be quite frank, doctor, I wouldn't care a monkey's what you ask …I have no hang-ups about any questions, however personal, the team think it's necessary to ask. (Carer 229)

Most PROMs were returned complete but 2% of the Rand 36-Item Health Survey (RAND-36) questionnaires were incomplete and participants felt the statements posed in the RAND-36 were too subjective and did not reflect the experiences of life as a carer or patient with MND. Patients and carers found it difficult to answer questions referring to ‘health’: some thought they were entirely healthy or did not perceive MND to be a ‘health’ problem. Participants favoured the MND-specific QoL questionnaire (40-item Amyotrophic Lateral Sclerosis Assessment Questionnaire, ALSAQ-40), preferring the format and content. It was observed that the language used in the ALSAQ-40 closely reflected the language patients used to describe their experiences. Patients found it difficult to report the number of informal hours of care they received per week. Nine per cent of these questions were blank. There was a large variation in informal carer hours required (0–168 hours/week) with no clear relationship between a patient’s disability and hours of care. Couples’ roles had gradually changed as carers took over many of the domestic jobs that were usually shared. This made it difficult to quantify how much of their role was ‘caregiving’. Some carers explained that even if they were not directly providing care they always had to be alert to the needs of their loved one and so many patients wrote that they required care ‘24× 7’. In addition, the questionnaire did not record multiple carers or where professional carers took over the role of an informal carer.

Feasibility of other data collection methods

The telehealth nurse diaries were returned incomplete. The nurse reported that it was difficult to assess her time using the TiM because she was often doing multiple tasks making it difficult to determine whether the time spent on an activity was part of ‘usual care’ or triggered by a TiM session. The Shadow Monitoring Protocol planned for clinicians to review information on the TiM and provide feedback prior to clinic. This was found to be infeasible because clinic appointments were frequently rescheduled or not booked sufficiently far in advance. There were administrative difficulties accessing the paper records (electronic records were not used during the trial). However, after the MDC clinic, physicians completed 38 Shadow Monitoring feedback forms about the TiM, although total clinic numbers were not collected so compliance with this form could not be measured.

Outcome assessment

All clinical outcomes are reported in the online supplementary file 3. Adverse events were low and none directly caused by the TiM system. We did not compare treatment arms in this feasibility study. However, we examined the data to determine whether it was possible to capture the impact of the TiM on MND outcomes. It is expected that over 6–12 months most patients with MND deteriorate in a meaningful way and therefore this change should be captured by our outcome measures. The ALSFRS-R confirms this as scores declined at a similar rate to the MDC population indicating disease progression (0.39 points per month compared with 0.34 per month recorded in the Sheffield MDC clinic).7 Physical QoL showed a trend towards deterioration in both the RAND-36 and the physical subscores of the participants’ preferred QoL measure the ALSAQ-40. The incidence of severe anxiety in carers also increased. Mostly these changes were small and did not reach significance but the sample sizes were too small to draw firm conclusions. Measuring health economic data identified a number of difficulties: only four hospital admissions were reported during the whole trial and other health resource use was highly variable. For example, between the third and sixth months of the trial healthcare visits ranged from 0 to 121 (intervention: median 8, range 0–121; control: median 4, range 2–17). The number of encounters did not appear to be related to patient satisfaction or access to MND services: some patients who reported to receive excellent, coordinated care had very few appointments whereas other patients reporting good care had many appointments.

Sample size for a full RCT

The sample size for a full-scale trial depends on the type I and II errors, the level of missing data due either to death or withdrawal and the anticipated size of effect. Assuming a 5% level of statistical significance, 90% power and 75% follow-up (based on the number completing 12-month follow-up in this study) and an effect size of 0.3 SD, a standard sample size calculation requires a prohibitive 312 patients per trial arm. This number can be reduced by employing a longitudinal approach to the analysis which the repeated measures (in this study the baseline, and 3, 6 and 12 months) are used in a repeated measures regression.37 Adopting this approach to the above scenario and assuming a correlation of 0.5 between measures leads to a more achievable sample size of 131 per arm. The sample size could be reduced further if a larger effect size was used (a 0.4 SD effect size requires 74 patients per arm), although larger effect sizes seem unlikely for a non-disease-modifying intervention. On the other hand, smaller effect sizes would be unjustified given the cost and service redesign requirements associated with implementation. We also note the present study is limited by its size, thus precluding a reliable estimate of whether the postulated effect sizes are reasonable.

Discussion

The aim of the study was to determine the feasibility and acceptability of conducting an RCT of TiM. Recruitment and retention were successful and rates similar to those in trials of disease-modifying treatments in MND (such as diaphragmatic pacing).38 In addition, we recruited patients who were more representative of the typical MND population than in other clinical trials.27 39 40 This included patients with severe disability and those living at a distance from hospital. Involvement of patient groups in the trial design, the low burden of the study and participants’ motivations to participate in research were the key facilitators of study success. The potential barriers to research participation identified (time, fatigue and the impact on research on day-to-day life) were not a problem in this trial. We recommend that our methods and findings be adopted in other trials in MND in order to improve recruitment and equality of access. Future clinical trials could even use the TiM as a cost-effective, low-burden research tool to collect outcome measures.

This was a small study in a single centre using motivated patients and staff involved in the development of the TiM. Larger trials at other centres may not experience the high levels of recruitment and retention seen here. Problems with recruitment are not unique to MND: less than a third of trials manage to meet recruitment targets and half require an extension.41 A sample size of 260 is feasible but challenging. The Sheffield MND clinic is one of the largest in the UK, and even using very broad inclusion criteria, only approximately half of patients recently attending the MDC were eligible to participate. Of these, only approximately half responded to an invitation. Therefore, for a sample size of 260 and a realistic estimate of recruiting 10%–25% of all the patients under the care of MND centre, the involvement of MND centres with a total caseload of between 1000 and 2500 patients would be required (this accounts for one quarter and a half of UK centres). Recruitment might be improved with face-to-face invitations, advertising and the use of national MND registries (eg, ref 42) that could even allow patients to identify themselves for research even when they cannot travel to a research centre.

The main limitation of this study was the difficulty estimating the impact that the TiM system would have on a service and healthcare resources. Hardware costs are likely to be minimal, particularly if patients used their own devices and software costs will depend on factors such as the uptake of the service, the capability of the system and data storage costs. The nurses’ time diary was not completed and it was difficult to differentiate time spent providing usual care from the additional work generated by the TiM system. Any additional work or time saved will vary depending on the MDC set-up and how the individual reacts to alerts (discussed in the parallel paper 29) but reassuringly the telehealth nurse felt the additional work was minimal and she could use the system within her current role. Many trials of telehealth employ additional staff to use the system but this fails to reflect how a new system would be used when embedded within an established service. Like other trials of telehealth (eg, ref 43), it would also be challenging to demonstrate a reduction in health resource use. In this case it is the low levels of the most costly encounters (hospital admissions) and the complexity of MDC care with highly variable levels of health resource use which do not appear to be directly linked to quality of care that pose challenges. In addition, our parallel paper suggests that any potential impact of the TiM on individual patients may vary depending on the stage and severity of the disease, meaning single outcome measures may fail to capture all relevant impacts (eg, improved communication may improve emotional QoL for patients early in the disease whereas earlier identification of physical complications may prolong survival in the later stages).29

While it may be feasible to conduct a larger trial of telehealth in MND, a traditional RCT may not always be the best way to evaluate such a complex intervention.44 An RCT aims to determine whether, all other factors being equal, a specific intervention works at the population level. However, a multicentre study would involve different MND services and while all MND care centres do adhere to the same guidelines.12 13 Their structures differ, meaning the impact of the TiM may differ with results from one site unlikely to fully predict whether it would work in other services. Service-level evaluations using non-randomised studies are inevitably less costly, quicker and able to recruit in larger cohorts than RCTs (all important factors in rare, terminal diseases such as MND). However, the limitations of such studies have been extensively documented with several published examples reminding us of the need to undertake assessments of both benefits and risks of new interventions. These include studies in MND, where apparently beneficial therapies subsequently were reported as ineffective or even harmful (eg, ref 38). However, there is conflicting evidence on the extent by which RCTs and well-designed controlled studies differ in this regard.45–48

While the role of non-RCTs in evidence-based medicine remains controversial, they are appealing in this type of situation where the intervention is perceived as being low risk and having modest clinical impact. In terms of efficacy, given MDC has been demonstrated to improve outcomes, it may be preferable to simply determine whether the TiM system can deliver an equivalent service to the current usual care, and/or widen access to MDC services that are already proven to be beneficial. In addition, the MRC framework recommends that developing and implementing technology is an iterative processes.23 An RCT would not provide sufficient opportunity to change the intervention substantially during the trial in response to feedback from centres, advances in technology and changes to the way MND care is delivered. Implementation studies may be better placed to demonstrate this while also providing the opportunity to demonstrate some of the complex clinical, professional and institutional factors that influence the success or failure of such an intervention.44 49 These also enable clinical services to test and modify the TiM to increase the likelihood of local buy-in, promoting local ‘champions’ who witness the successes of new services can deliver persuasive arguments to support commissioning of services.50 In the case of digitally enabled technology, at both a methodological and health service level it remains uncertain what represents a good trial and what evidence is needed to enable interventions to be funded and adopted.

Conclusion

The study suggests that a large-scale evaluation of the TiM system would be possible but challenging. This study suggests that it could be possible to overcome some challenges seen in other MND trials (such as recruitment and retention). With such diverse clinical settings with complex groups of patients, alternative methods of evaluating may be more appropriate in generating practical and generalisable data and support TiM development and implementation.

Table 1

Patient outcome measures collected

	Baseline*	Postal questionnaires					Clinic visits
		Baseline	3 months	6 months	12 months	18 months
Patient characteristics
Age, gender	X
Frequency of technology use	X
Broadband/mobile internet access	X
Presence of difficulties using TiM	X
Need for help using TiM	X
Medical history
Diagnosis		X
Disease duration		X
Comorbidities		X
Drug history		X
Quality of life
ALSAQ-40		X	X	X	X	X
RAND-36		X	X	X	X	X
EQ-5D+D		X	X	X	X	X
Clinical measures
ALSFRS-R		X	X	X	X	X
Pain score (current and worst)†		X	X	X	X	X
CSS-MND		X	X	X	X	X
Hospital Anxiety and Depression score		X	X	X	X	X
Survival							X
Adverse events			X	X	X	X	X
Health resource use
Clinician encounters†		X	X	X	X	X	X
Hospital admissions†		X	X	X	X	X	X
Informal care use†		X	X	X	X	X
Formal care use†		X	X	X	X	X
Satisfaction
MND care satisfaction†		X	X	X	X	X
TiM satisfaction†			X‡	X‡	X‡	X‡

*Collected by investigator.

†Questionnaires designed for the trial.

‡Intervention arm only.

ALSAQ-40, 40-item Amyotrophic Lateral Sclerosis Assessment Questionnaire; ALSFRS-R, Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised; CSS-MND, Clinical Saliva Scale for Motor Neuron Disease; EQ-5D+D, EQ-5D + dignity bolt-on; MND, motor neuron disease; RAND-36, Rand 36-Item Health Survey; TiM, Telehealth in Motor neuron disease.

Table 2

Carer outcome measures collected

	Baseline*	Postal questionnaires					Clinic visits
		Baseline	3 months	6 months	12 months	18 months
Carer characteristics
Age, gender	X
Relationship to patient	X
Frequency of technology use	X
Presence of difficulties using TiM	X
Quality of life
RAND-36	X		X	X	X	X
Clinical measures
Hospital Anxiety and Depression score	X		X	X	X	X
Zarit Burden Interview	X		X	X	X	X
Adverse events			X	X	X	X	X
MND care satisfaction†	X		X	X	X	X
TiM satisfaction†			X‡	X‡	X‡	X‡

*Investigator completed.

†Questionnaires designed for the trial.

‡Intervention arm only.

MND, motor neuron disease; RAND-36, Rand 36-Item Health Survey; TiM, Telehealth in Motor neuron disease.