Learning from remote decentralised clinical trial experiences: A qualitative analysis of interviews with trial personnel, patient representatives and other stakeholders.

AIMS: The aim of the study was to identify actionable learning points from stakeholders in remote decentralised clinical trials (RDCTs) to inform their future design and conduct.
METHODS: Semistructured interviews were carried out with a purposive sample of stakeholders, including senior managers, trial managers, technology experts, principal investigators, clinical investigators, research scientists, research nurses, vendors, patient representatives and project assistants. The interview data were coded using a thematic approach, identifying similarities, differences and clustering to generate descriptive themes. Further refinement of themes was guided by empirical phenomenology, grounding explanation in the meanings that interviewees gave to their experiences.
RESULTS: Forty-eight stakeholders were interviewed. Actionable learning points were generated from the thematic analysis. Patient involvement and participant engagement were seen as critical to the success of RDCTs where in-person contact is minimal or nonexistent. Involving patients in identifying the research question, creating recruitment materials, apps and websites, and providing ongoing feedback to trial participants were regarded as facilitating recruitment and engagement. Building strong relationships early with trial partners was thought to support RDCT conduct. Multiple modes of capturing information, including patient-reported outcomes (PROs) and routinely collected data, were felt to contribute to data completeness. However, RDCTs may transfer trial activity burden onto participants and remote-working research staff, therefore additional support may be needed.
CONCLUSION: RDCTs will continue to face challenges in implementing novel technologies. However, maximising patient and partner involvement, reducing participant and staff burden, and simplifying how participants and staff interact with the RDCT may facilitate their implementation.

What is already known about this subject

Technological developments, and the COVID‐19 pandemic, have increased the use of remote decentralised clinical trial (RDCT) methods.

Some information on the challenges of RDCTs has been published, but we can learn more from people who have been directly involved in RDCTs.

Some challenges are generic to all clinical trials; others are specific to RDCTs.

What this study adds

Attention to participant involvement and engagement is crucial for RDCTs, where in‐person contact is limited or absent.

Although RDCTs may lessen the logistical burden of trial participation, they can increase/introduce other burdens.

Employing mature, well‐tested technology and simplifying how participants and trial personnel experience RDCT methods can contribute to success.

INTRODUCTION

Clinical trials are required to test medicines. Traditionally, trials have involved participants attending hospitals or research sites for face‐to‐face interaction with trial personnel. This approach is costly (the median cost for pivotal efficacy trials in 2015–16 was estimated at $19 million), slow and inefficient. Moreover, it is estimated that half of trials fail to meet recruitment targets, , resulting in delays, insufficient data or early termination. In addition to financial costs, there is also the burden that participation places on trial volunteers. Difficulties in scheduling and attending in‐person visits are significant reasons for low recruitment. , A site‐based clinical trial approach may discourage people in full‐time employment, with disabilities or caring responsibilities, or who live in rural areas from participating; by excluding these populations, trials may produce less generalisable results.

Remote decentralised clinical trials (RDCTs) are proposed as one way to improve clinical trials. , Employing digital and other innovations to move clinical trial activities to a participant's home or local setting may reduce or eliminate physical visits to a clinical centre. This change should make trials more accessible and participant‐centred, and produce results more applicable to the wider population and easier to translate into usual practice. Indeed, in a world influenced by a recent pandemic, remote trials may be the preferred option for many participants.

Trials without face‐to‐face interactions are not new; large, streamlined mail‐based randomised trials have been conducted since the 1980s. , , Recent advances in communications technology have permitted new approaches to trial activities, such as telemedicine visits, data capture using mobile devices and online participant‐reported outcomes. Pfizer's REMOTE study was the first large‐scale attempt at exploiting the potential of internet‐enabled technology to run a completely web‐based trial for an investigational new drug application. Despite the REMOTE study being terminated early, there has since been sustained interest in developing RDCT methods. However, this is not without its challenges.

The Trials@Home project (https://trialsathome.com) is a multistakeholder project, supported by the European Federation of Pharmaceutical Industries and Associations (EU/EFPIA) Innovative Medicines Initiative. As part of Work Package 1 (BEST) of the Trials@Home project, we set out to learn from academic institutions, pharmaceutical companies and small‐medium enterprises about their experiences in developing and implementing RDCT methods. The objective was to identify actionable learning points to inform the future design and conduct of RDCTs.

METHODS

Design and study setting

We conducted one‐to‐one semistructured interviews with trial personnel and other stakeholders to explore their experiences of conducting RDCTs, including the challenges they faced and how they responded. This interview method was chosen because of its flexibility and ability to achieve extensive coverage of the topic under investigation. Guided by empirical phenomenology, we explored meaning and context to understand RDCTs through the specific human experience. We followed the consolidated criteria for reporting qualitative studies (COREQ) guidelines.

Participants and sample

Case study identification and selection

RDCT case studies were purposefully selected to represent a diversity of methods and therapeutic areas. A Microsoft Excel data collection tool was developed in collaboration with the Trials@Home BEST work package to facilitate the collection of structured information on case studies proposed by consortium partners. We encouraged partners to submit RDCTs and used the tool to summarise relevant external clinical trials identified during an initial scoping review. The completed data collection tool facilitated the consensus identification of 20 case studies representing a range of RDCT activities (Table 1).

TABLE 1

Remote decentralised clinical trials (RDCT) case studies (n = 20)

Case study	Therapeutic area	Study features	Status at time of interview	Participants' location
1	Cardiovascular	Fully remote, including PROs and record linkage to routinely collected data	Ongoing	UK
2	Rheumatology	Hybrid with direct IMP supply, outcome reports from participants, healthcare providers and routinely collected data	Ongoing	UK and European countries
3	Cardiovascular	Hybrid, IMP prescribed by usual care provider, outcome reports from participants, healthcare providers and routinely collected data	Ongoing	UK
4	Diabetes	Fully remote (Europe), online clinical platform, medicinal device, social media recruitment, eConsent, participant feedback through online questionnaires	Completed	UK and European countries
5	Neurology	Telemedicine, direct patient recruitment, direct to participant IMPs, nurse home visit for samples, participant feedback of trial experience explored	Completed	USA and European countries
6	Neurology	Comparison of remote vs traditional, telemedicine, app, nurse home visit, ECG device, PROs	Ongoing	USA
7	Diabetes	Comparison of remote vs traditional, home nursing, direct to participant IMP, app, Bluetooth device, participant feedback of trial experience explored	Completed	USA
8	Diabetes	Comparison of remote vs traditional, direct to participant IMPs, virtual visits, medicinal devices	Completed	USA
9	Rare disease	Interventional, complex set up: home infusion with a nurse, patient involvement	Ongoing	USA and International
10	Rheumatology	Hybrid and traditional, three groups: participants visited by nurses, participants visited by nurses and attending traditional sites, participants only attending traditional sites Recruitment using social media and patient advocacy	Completed	USA and International
11	Rheumatology	Fully remote, adolescents, social media recruitment, iPhone and app provided, direct to participant IMPs, home nursing, feedback collected via device	Ongoing	USA
12	Neurology	Hybrid, paediatric, interventional adaptive design, patients' organisation involvement before protocol finalisation, telemedicine, home nursing, eConsent, wearable use for 24‐h ambulatory EEG	Setting up	USA
13	Cardiovascular	Hybrid, wearable device and transmitter for data collection, eConsent	Ongoing	International
14	Women's health	Interventional, eConsent, daily questionnaires input to study supplied hand‐held device	Ongoing	International
15	Women's health	International, pregnancy, community‐based complex intervention, apps and devices for community healthcare workers	Ongoing	India
16	Cardiovascular	Comparison between remote and traditional, complex intervention, Bluetooth‐connected device, tablet, app	Completed	UK
17	Asthma	Fully remote, interventional with Bluetooth‐connected devices, app, environmental data collected, direct to participant shipment	Completed	USA
18	Cardiovascular	Fully remote, comparing doses, extensive patient involvement in investigator meetings, steering committee and executive committee, eConsent	Ongoing	USA
19	Diabetes	Hybrid, interventional, recruitment through a national screening program	Ongoing	UK
20	Cardiovascular	Fully remote, interventional, smartphones and wearable devices	Setting up	USA

IMP, investigational medicinal product; PRO, patient‐reported outcome.

Selection of participants

Case study contributors were asked to generate a list of personnel and stakeholders with experience of specific aspects of trial conduct (named “basic building blocks” within the Trials@Home project) (Figure 1). Interviewees were consecutively and purposefully selected to capture a diversity of opinions and experiences.

FIGURE 1

Building blocks – specific aspects of trial conduct. Source: Rogers and Mackenzie8 (https://trialsathome.com)

Recruitment

Participants were recruited by email between January and June 2020. We invited 60 potential participants, with 35 initially agreeing to take part. The first and second reminder emails resulted in 10 and three additional participants, respectively. Reasons for declining participation included trials still in early set‐up phases and diversion of staff to COVID‐19 work. Participants did not receive any remuneration.

Ethics statement

The project was granted ethical approval by the University of Dundee School of Medicine Research Ethics Committee (SMED REC number 20/07, 27 January 2020).

Instrument development

Empirical phenomenology argues that explanation must be grounded in the first‐order constructs of the actors, that is, the meanings they give to their experiences. With this in mind, we developed a semistructured interview schedule, piloted in five interviews (Supporting Information Data S1). Open‐ended questions were derived from topics identified in a scoping review of RDCT literature and from the informational needs of the Trials@Home project. Additional items and modifications were added iteratively as interviewees raised new relevant issues unforeseen by the study team.

Interview procedure

The interviews were conducted by a single researcher (J.C.) based in the UK between the 31 January 2020 and 26 June 2020. Twelve interviews were carried out in‐person before the UK COVID‐19 lockdown (23 March 2020), 35 of the remaining interviews took place using videoconferencing and one was carried out by telephone. Interviews lasted 45–75 minutes, averaging approximately 1 hour.

Data analysis

With interviewee consent, all interviews were recorded using a digital recorder and fully transcribed. Thematic analysis was employed to identify themes in the data, including familiarisation (reading the transcripts to gain an overall understanding of the meanings conveyed), generating initial codes (identifying significant phrases, sentences and words, and organising them into categories), searching for preliminary descriptive themes, and reviewing and modifying themes. The final refinement stage involved finding associations between themes to generate explanations for them. ,

Two co‐authors (J.C. and A.R.) read the transcripts and generated initial codes and categories independently. Similarities, differences and clustering were noted, and agreement reached on initial descriptive themes. These were shared with the remaining authors, and the resulting themes were presented to the research team for further refinement until consensus was achieved.

Role of the funding source

The funder (EU/EFPIA Innovative Medicines Initiative grant no. 831458) had no role in the design and conduct of the qualitative research or interpretation of the data. Trials@Home consortium partners helped identify case studies and provided feedback on interim findings. The manuscript was approved for submission by the Trials@Home Partner Assembly.

RESULTS

We interviewed 48 stakeholders from 20 case studies, including trial staff, vendors and patient representatives (Table 2). The qualitative findings are presented as themes influencing the implementation of an RDCT (Table 3). These include facilitators and barriers to recruitment and engagement, technology‐related challenges and proposed solutions, transferred burden, data‐flow challenges and proposed solutions, and COVID‐19 restrictions.

TABLE 2

Participant characteristics

Role	Value, n
Trial staff (clinical/research): Principal investigator, clinical investigator, research scientist, research nurse	19
Trial staff (management/administration): Senior management, directors/global leaders, trial manager, project assistant	17
Trial staff (technology/data)): Software developer, data manager	6
Vendor	4
Patient representative	2
Total	48

TABLE 3

Themes representing factors influencing the implementation of an RDCT

Themes and subthemes	Illustrative quotes
Facilitator of recruitment
Ease of participation	“We make it easy for them to stay in, they do not have to go and pick up prescriptions, they do not have to attend appointments …. we cannot be making it easier for them to take the drug.” Interviewee 0049 Trial Staff (clinical/research)
Perceived patient value	“… it was clearly a study looking at hypertension, and I suffer from hypertension … I started being more interested in self‐management, self‐care. Hypertension was part of this … I thought this was very necessary. It also was aiming at recruiting 20 000 patients … so it was kind of serious.” Interviewee 0058 Patient Representative
Relatability	“They [patient partners] wrote letters about their personal journey and many of the sites included the personalised letters with their recruitment material. I think it helps you humanise what we were doing and to have patients like them tell why the study was important to them.”. “Interviewee 0016 Trial Staff (management/administration)
Barriers to recruitment
Identifying eligible participants	“The broad‐brush approach of targeting the whole type 2 diabetes population was a waste of time. We could not get the number of patients that we were hoping … we had very large numbers at the beginning … But then very small numbers at the end.” Interviewee 0029 Trial Staff (management/administration)
Restrictive regulations	“The complication is … the data belongs to the person who is responsible for the patient's care … you have only got a right to access it and use it if your data generated is to the care of the patient. You need to have a good strong clinical reason for knowing that person exists, and that person has …” Interviewee 0049 Trial Staff (clinical/research)
	“So, the challenge that we had … we were not able to recruit from [sic] every state in the union … Wherever we had doctors that were licensed … we had 15 states that we were able to operate in … [But] There were patients who contacted us who were not living in the states that we were licensed in.” Interviewee 0021 Trial Staff (clinical/research)
Facilitators of engagement
Familiar environment	“The concentration is going to be very important. And the concentration can be affected by the environment … which room do you do your RCT in? Your kitchen? Your bathroom? Your bedroom?… Where do you keep either the tools or devices, or the medicine?” Interviewee 0018 Patient Representative
Simple interfaces	“Oh, the study website, it's very clear. It works well. It's a joy to use…[it] is very good at asking specific questions. Or saying do this, click there…not too many words. It's simple. It's focused. It's easy to comprehend.” Interviewee 0058 Patient Representative
Feedback	“We measured what features are important to them (the participants) and how well we did in that category. So, we discovered that being able to see your own measures was really highly valued.” Interviewee 0015 Trial Staff (technology/data)
Collaboration	“Oftentimes it is a collaboration between one of the clinicians … and our patient partners …. The cool feature of the newsletter that the patient came up with was a way for patients to communicate back with the study team. So, we actually have a Share your Story section … where the patient can … share … their experience in the study … And then we give them control over how and when we share their story … in a future newsletter or on social media, or on our web.” Interviewee 0016 Trial Staff (management/administration)
Barriers to engagement
Overburdening participants	“It was simply too ambitious to use so many devices. I think the study would have been more successful had they been more selective and had a lower number of devices because the burden was perceived by the patients to be very high.” Interviewee 0053 (Vendor)
Kitchen sink mindset	“We said why do not we try throwing the kitchen sink at it … So let us do remote tele‐health monitoring with some virtual devices. We had the patients take their blood pressure remotely, take their weight, their pulse, the glucose meter and you know not that we had not used those technologies before but using them all at once.” Interviewee 0044 Trial Staff (management/administration)
Lack of shared understandings	“The other thing we learned is that the registration link originally was expired in 24 hours for security reasons … but then we learned that this population does not read their email every day. So, they would sign up and then they would kind of forget about it and then they would open their emails 3 days later and the link had expired and lots of people had trouble with that … we started getting so many calls from that.” Interviewee 0015 Trial Staff (technology/data)
Fewer clarification opportunities	“People can forget they are in a trial, if you are taking your tablets in the morning and you have always taken them in the morning and you are randomised to the morning and 2 years later, you get an email, am I in a trial, I cannot remember, am I?” Interviewee 0011 Trial Staff (clinical/research)
Technology related challenges and proposed solutions
Immature technology	“… these technical issues started cropping up … With connectivity … certain pages not loading up … Functions not appearing on the app … when push came to shove, it wasn't really that mature. It still needed testing.’ Interviewee 0029 Trial Staff (management/administration)
Lack of shared meanings	“I looked at a storyboard of the app and I was talking about individual patients and I said okay so if an individual patient starts on this day, how are we going to trigger the procedures that they are supposed to do before their next visit? … take my glucose, do my weight, my pulse … The person I was talking to did not understand we need the notifications. He said you mean everybody does not start at the same time?” Interviewee 0044 Trial Staff (management/administration)
Simplify	“Small pre‐studies solely for testing the technology that will be used in the main trial. … have this study with a smaller population of patients. Something that makes it simpler so that you can really only focus on the digital aspect that you are trying to test.” Interviewee 0029 Trial Staff (management/administration)
Commitment	“…making sure that you select a vendor for technology that's willing to partner with you. A lot of these studies are not out of the box type studies and you need to customise the visit, you need to customise the look and feel of the portal …”. Interviewee 0016 Trial Staff (management/administration)
Transferred burden (staff)
Logistical	“The weather conditions in the wintertime, for driving to the different places, the safety of the staff could cause a problem.” Interviewee 0010 Trial Staff (clinical/research)
Digital overload	“A little bit of digital overload in some sites in that there is already a government app for pregnancy registration … so essentially we were asking workers to sort of duplicate enter these things … I think this is something must be graded into the existing government platform that they are using. You cannot really expect health workers to have separate apps. We even heard stories from health workers where there are some areas where they have got 9 different apps.” (interviewee 0042 trial staff (clinical/research)
Isolation	“If we are out remotely at practices, it can feel like you are on your own. You know, I mean, who do you contact if you have an IT issue? There's nobody just near to hand.” Interviewee 0008 Trial Staff (clinical/research)
Data flow challenges and proposed solutions
Accessing routinely collected data	“Well, we have only been able to get one for England. Well, they started in 2012 too … So yes, we have only had the one, so it's been frustrating, 2018, we must have got it. Scotland worked okay, overall. Denmark used to work okay but then with GDPR, they now aren't giving us any data from the end of 2018 onwards and Sweden, it theoretically should be possible, but we have not been able to get anyone to help us there.” Interviewee 0012 Trial Staff (management/administration)
Interpretation of data protection regulations	“Near the end of the trial, with how the authorities interpreted the GDPR … They were not entirely clear about how the GDPR should manifest and tended to err on the more restrictive side. Kind of, lawyers are even more cautious than doctors in terms of not making mistakes.” Interviewee 0017 Trial Staff (clinical/research)
Early engagement	“Start applying for record linkage from NHS Digital as soon as you have your ethics approval. Instead of waiting for 2 years into the study and going let us try and get some record linkage data and finding out that after 3 years you still do not get it ‐ start applying for it as soon as you have ethical approval.” Interviewee 0013 Trial Staff (technology/data)
Multiple ways of capturing data	“… probably better ways of measuring the effect of a medication that are less dependent upon subtle physical exam characteristics and more dependent upon patient quality of life and ability to function … rather than changes in tone that the patient does not notice anyway.” Interviewee 0021 Trial Staff (clinical/research)
	“In some ways the more you move towards the patient and you are able to leverage real data, that may actually improve the quality over time because you are having a direct source.” Interviewee 0016 Trial Staff (management/administration)
Surrogates	“We piloted a concept of having surrogates … So, patients were asked to nominate surrogates that we could contact, if and when they did not reply … Patients gave us the surrogates email address or phone number. So, we would contact them and say, can you tell us what happened, is Jimmy still alive?…and so you can often get a lot of information … And we consented each surrogate …” Interviewee 0011 Trial Staff (clinical/research)
Additional challenges raised by COVID‐19 restrictions
Staffing	“We've got behind with the phone call questionnaires at 6 months and things because nurses have been redeployed onto COVID studies or into ward work.” Interviewee 0049 Trial Staff (clinical/research)
Social distancing	“It's a randomised placebo‐controlled trial … with the initial screening visit and then the randomisation visit face‐to‐face with the nurse … we have got a group of people paused awaiting bloods for randomisation.” Interviewee 0050 Trial Staff (management/administration)
	“We had decided in our protocol the endpoint was going to be their blood pressure and their haemoglobin at sort of 6–12 weeks after the baby was born … The whole COVID thing this has been a little bit of a spanner in the works because researchers cannot go into the communities…We do not know whether that will be a month later or 6 months later … So how do we know what the endpoint is?” Interviewee 0042 (Clinician Investigator)
	“… so, the trial will inevitably have to be longer because it is going to take a longer time to recruit the events we were looking for … And that carries with it other issues … we need more funding, quite possibly, will we need more drugs because we may have to treat patients longer, so there are a number of knock‐on effects.” Interviewee 0041 Trial Staff (clinical/research)
Approval delays	“I think it had some impact on our start‐up because … Some of the IRBs had so many COVID studies, so they are backed up to even give a waiver.” Interviewee 0063 Trial Staff (clinical/research)

Facilitators and barriers to recruitment

Interviewees identified several facilitators to recruitment, including ease of participation, perceived participant value, relatability and patient involvement. Many interviewees believed enabling participants to take part in their own homes, and sending the Investigational Medicinal Product there, reduced barriers such as taking time off work and travelling. An interviewee participating in a fully remote trial explained how she appreciated it was easy to fit into her routine: “There was something very real world about it … Every quarter I would take my blood pressure measurements … at 8 o'clock in the morning and 8 pm ‐ every day for three days. It was very good … I could do it at home.” Interviewee 0058 Patient Representative.

Furthermore, this convenience, some felt, encouraged broader geographical participation and greater socio‐demographical diversity: “It also allows for diversifying patients that participate … we've had patients share back with us they've never been asked to do a research study, and this was really exciting for them.” Interviewee 0016 Trial Staff (management/administration).

Interviewees also believed they secured a high recruitment rate because they investigated an issue of high value to participants, as this interviewee explains: “ … it's because it's retinopathy, blindness is the number one fear of people with diabetes, and these are people that have been told they've got changes to their eyes and there's nothing we can do about it.” Interviewee 0049 Trial Staff (clinical/research).

Other recruitment strategies deemed to work well sought to make the trial relatable by including real patient/partner stories in their recruitment literature. These interviewees felt it was essential to involve participants in the set‐up and design of the trial, including drafting the protocol and recruitment materials. One case study made maximising patient involvement a key objective, educating the patients in trial methodology and enabling them to participate more fully in trial team meetings.

Case studies aiming to recruit a specific subgroup of the participant population reported that social media advertising had been ineffective. Another approach was to use routinely collected healthcare data to identify potential participants. Although this could be a complex and lengthy process, it was reported to be highly successful in one RDCT: “We got a response rate of over 20%… from proposing a clinical trial where you take a new tablet for five years, continuously … “ This contrasted with their previous experiences: “Our cardiovascular trials [where] for every 100 letters we sent out we might hear back from 7 or 8 people.” Interviewee 0041 Trial Staff (clinical/research).

Facilitators and barriers to engagement

Many interviewees raised the challenge of keeping participants engaged with trial activities.

Facilitators of good adherence included familiar environments and technologies, simple user interfaces for technologies and feedback to participants.

Interviewees emphasised the importance of quiet environments where participants could concentrate and have their devices at hand. “Bring your own device” (BYOD) was used in several RDCTs, for example participants using their own blood pressure monitors. Additionally, devices giving patients immediate feedback, such as glucose meters, were highly valued by trial participants. Moreover, patient representatives commented that the responsibility to take simple measurements made them feel like collaborators: “The study did not treat patients as passive …. It said you … you report your blood pressure … whether you're taking new medication … if anything else changes, you're in charge, you tell us … real collaboration and partnership with the professionals.” Interviewee 0058 Patient Representative.

Other interviewees reported that their participants particularly welcomed regular feedback on the progress of the trial.

Many interviewees felt that simple websites with limited text and clear directions were easier to use and encouraged participants to engage regularly and, if required, complete online questionnaires. Involving participants in drafting the content also enabled trial teams to get the look and feel right.

Some trials encountered missing data resulting from incomplete questionnaire responses, missed scheduled measurements or failure to connect devices. Interviewees suggested this was due to overburdening participants, a lack of shared understanding and fewer opportunities to clarify trial requirements.

Several interviewees describe their participants being overburdened with digital technology, as an interviewee involved in an asthma trial explains: “There was a sleep monitor, a spirometer, a biosensor that tracked temperature, respiration, heart rate … there was a daily questionnaire … if they were on a maintenance inhaler, they'd got a Bluetooth device … it was a big ask … “ Interviewee 0053 Vendor. Although the total time taken to perform the tasks, once set up, was relatively short, the perception of burden, the vendor continued, remained high: “Those of us who knew it inside and out, knew that once you were up and running it was less than 10 minutes a day … But it was overwhelming to the patients. Regardless of how reassuring we were.” Interviewee 0053 Vendor.

Overburdening was attributed to a “kitchen sink” mindset within the study team to test multiple new technologies and devices in a single trial.

Some interviewees believed comprehension was also problematic. Trial teams sometimes falsely assumed participants had understood the information they were given. The following interviewee illustrates the confusion caused by a simple screening question in a questionnaire: “We asked “do you have regular meal schedule?” … then we learned that this one question was excluding 75%… Because… like, normally, it's regular but no, “sometimes I skip breakfast, so I don't know how to answer this question.” Interviewee 0015 Trial Staff (technology/data).

This confusion generated frustration in participants and increased the workloads of trial staff who had to respond to queries: “So when you signed up … the system told you … we will send you the registration key into your email … it was just a link. And the word ‘key’ was confusing to some people; they are like “What key?” And then they called for help. “What is this key that you're going to send?” Interviewee 0013 Trial Staff (technology/data).

Moreover, many interviewees felt there were fewer opportunities to explain and to check participant understanding of the trial because in‐person interactions were minimal or nonexistent.

Technology‐related challenges and proposed solutions

Several interviewees expressed frustration with technology not functioning as anticipated, concluding it had not been thoroughly tested in a clinical trial environment. Critically, they reported that clinicians and participants subsequently lost confidence in their trial: “It would encounter an error at every juncture … little failures where either the data wouldn't transfer, a menu wasn't accessible … When you're dealing with the patient's and investigator's tolerance for having these types of errors … it was death by a thousand papercuts.” Interviewee 0044 Trial Staff (management/administration).

Technological immaturity and overburdening participants with digital technology were cited as the main reasons for the abandonment of two case‐study trials. One interviewee reflected: “At the end of the day, we had to stop both clinical trials. We weren't able to proceed anymore. So maybe had these been smaller studies, simpler ones, maybe we would have been able to continue.” Interviewee 0029 Trial Staff (management/administration).

These interviewees stressed that clinical trials were already complex activities, and adding remote technology complicated them further. “There is a temptation with increasing technology to increase complexity. That would be a huge mistake. Keep it simple. Get the basics right. Then add, if you need to.” Interviewee 0061 Trial Staff (management/administration).

Although there may be times when complexity is needed, such interviewees suggested that the reasons should be to simplify recruitment processes, unburden patients or reduce staff workload. They consequently advised using mature, well‐tested, validated technology or conducting smaller, simpler, pre‐pilot (or feasibility) studies to test novel hardware and software.

Just as some interviewees falsely assumed participants understood the intended meanings of trial information, so too did they incorrectly believe that technology vendors understood the clinical trial environment, as the following interviewee explains: “The lack of knowledge that they had of clinical trials itself was a problem because we felt we would tell them one thing and that they understood exactly what we were asking them to do, but until later some of these things didn't become apparent.” Interviewee 0044 Trial Staff (management/administration).

Such misunderstandings could result in trial delays. “The person I was talking to didn't understand we need the notifications. He said, “you mean everybody doesn't start [the trial] at the same time?” … because they already started coding … that was a big step back where they had to … recode a lot of stuff.” Interviewee 0044 Trial Staff (management/administration).

It was therefore deemed essential to engage technology partners with experience of clinical trials and associated terminology. Furthermore, technology partners needed to be committed to the project because RDCTs often need to customise software after launch due to participant or staff feedback.

Transferred burden

Reducing the burden of trial activities on participants may unintentionally transfer it to individual research staff. Nurse interviewees described stressful new logistical challenges in performing activities nearer to participants. For example, using local healthcare facilities required negotiating access to consulting rooms and patient data, travelling to unfamiliar locations and keeping track of equipment. “If you're going to a practice … getting a room can be very difficult … sometimes you're in a room, and then somebody will come to the door and say, ‘But we need this room, you'll need to get out.” Interviewee 0010 Trial Staff (clinical/research).

Additionally, receiving emails, messages or calls in busy, unfamiliar environments could be disorientating: “Our administrators were quite good at sending us an email if you were out at a [primary care] practice to deal with something. But sometimes you would be busy, and there would be a lot of things coming through, and it's quite difficult keeping track.” Interviewee 0008 Trial Staff (clinical/research).

Research staff could also be overburdened with digital technology. One clinician overseeing research involving educating and training community health workers explained how the trial duplicated apps and devices already used by state governments, necessitating repeated data entry.

Some nurses also reported feeling isolated, especially when experiencing technical problems or professional dilemmas. Moreover, remote means of contact did not always provide the type of professional support required: “When the follow‐ups were getting done, there would be serious adverse events … If you're in the office, you can check … would it be an SAE [reportable serious adverse event]?… Whereas it's harder if you're going through all the technology.” Interviewee 0008 Trial Staff (clinical/research).

Data flow challenges and proposed solutions

Studies relying on routinely collected healthcare data could be affected by lengthy application processes, delays in data provision and incomplete datasets. Consequently, some had to seek additional funding to support extensions despite having data‐sharing agreements in place. Changing legislation was especially problematic for longer RDCTs. “At regular intervals, we had record linkage whereby we received from central registries their [participant's] records of hospitalisation…Unfortunately, this was not possible for the last year of follow up, because the local authorities did not allow us to retrieve that information …” Interviewee 0017 Trial Staff (clinical/research).

A common perception among respondents in Europe was that the recently introduced General Data Protection Regulations (GDPR) created confusion and reinforced a pre‐existing culture of caution. The problem was that organisations holding healthcare data tended to conceptualise individual clinical care and clinical trials as separate entities, limiting access to patient data for trial teams, as this interviewee explains. “They were just a bit nervous about the use of it for this. They felt that using data that was collected for clinical care for clinical research was somehow different.” Interviewee 0049 Trial Staff (clinical/research).

However, COVID‐19 may have changed this culture because treating patients was now clearly linked to knowledge gained through clinical trials, as the interviewee further comments: “They know we don't know the answer with COVID … So, a lot more people are getting involved in the trials … Entire COVID wards here were structured around the trials.” Interviewee 0049 Trial Staff (clinical/research).

Interviewees suggested several strategies to offset the problems of delays in data provision and incomplete datasets. One was to develop multiple ways of capturing data to ensure the timely reporting of relevant events. When participants contacted teams and disclosed events, these needed to be captured, coded and added to a database. This process would require detailed protocol provisions for dealing with such data, with participant engagement facilitating this process: “Patient‐reported outcomes. Record linkage obviously works to an extent … but if you can do patient engagement well … [and] get participants to accurately give you the events, the combination of the two you will probably [give] a very clear picture of events.” Interviewee 0013 Trial Staff (technology/data).

Additional challenges raised by COVID‐19 restrictions

The COVID‐19 pandemic restrictions created challenges for several case studies relating to staffing issues, social distancing requirements and delays in approvals.

Some case studies were adversely affected by clinical staff being redeployed to COVID‐related duties. For example, an interviewee reported being behind with their telephone questionnaires because nurses were redeployed to ward work. Hybrid trials were particularly vulnerable to restrictions on face‐to‐face research activity, and several interviewees reported having to pause trials because screening or randomisation visits required these nurses.

Even restrictions on usual clinical activities were having knock‐on effects on research. In one case study, community health workers screened pregnant women and visited them 6–12 weeks after the birth of their babies to measure blood pressure and haemoglobin. However, COVID‐19 restrictions meant that planned post‐birth visits were postponed. Such trials needed additional funding given the additional time required to capture necessary endpoints. Several interviewees also described review boards prioritising COVID‐related research, causing delays for non‐COVID RDCTs.

DISCUSSION

We aimed to discover the experiences of personnel and stakeholders in academic institutions, pharmaceutical companies and small‐medium enterprises in developing and implementing RDCT methods. Our objective was to identify actionable learning points to inform their future design and conduct.

We have identified several learning points, which can be broadly categorised into two groups: participant‐focused and trial‐focused. Concerning RDCT participants, maximising participant involvement, reducing burden and minimising complexity of participation are vital. For the trial‐focused aspects, early partner involvement, enabling multiple modes of data capture and mitigating transferred burden are essential.

Although participant involvement and engagement are crucial for all trials, they assume even greater importance in RDCTs where in‐person contact is reduced or eliminated. For example, involving patients in identifying the research question ensured it was of high value to participants; input in designing recruitment materials made trials more relatable, facilitating recruitment. However, maximising diversity may mean involving groups of participants and developing strategies to build their confidence to participate fully in trial meetings.

Participant input in creating websites and portals, ensuring they are easy to navigate with clear calls to action, was seen as especially important to engagement. Some trials, such as an asthma medication trial in adolescents, used patient involvement and extensive user acceptance testing to design web content. , Providing participants with ongoing feedback about trial progress can make them feel more like collaborators, while feedback from devices can facilitate self‐management.

These findings are consistent with research into maintaining patient engagement in remotely delivered healthcare interventions such as rehabilitation; this suggests an opportunity to learn from related research fields. For example, a systematic review of teleneurorehabilitation found that a range of cognitive, behavioural and emotional strategies, some of which may be relevant to RDCT engagement, could be delivered remotely to support motivation, adherence and decision‐making in people with neurological conditions.

Although it is assumed that RDCTs reduce the logistical burden on participants by enabling at‐home participation, interviewee references to participant incomprehension and observations about “digital overload” indicate the presence of other cognitive and psychological burdens. While further research is needed, it is known that each piece of data or supporting documentation a participant must supply increases the burden of enrolment and risks loss of engagement. Indeed, a multistep screening process is thought to have been a significant factor in the early drop‐out from the REMOTE trial. Adopting technologies familiar to participants, such as BYOD, may help offset some of these encumbrances. ,

While validity testing of digital biomarkers is essential, validity does not ensure that a selected technology will perform as expected in a specific remote clinical trial context. We recommend that trialists consider small‐scale feasibility testing of any new technologies or combinations of technologies planned to be used in a trial. These pilot studies should include, as far as possible, participants representative of the proposed trial cohort. Consideration should also be given to incorporating qualitative methods to deepen understanding of technology implementation.

“Digital overload” could be exacerbated by reduced access to digital technologies in some groups (eg, older adults, the economically deprived and people with disabilities). This “digital divide” may also account for some of the missing data reported in several RDCTs due to the failure to connect devices and missed scheduled measurements. For example, in 2018, Age UK reported that 36% (4.1 million) of people aged 65 plus were offline, lapsed or never used the internet. Although the COVID‐19 pandemic may have increased older people's engagement with digital technology to access services and facilitate social connection, there are still likely to be significant numbers who remain digitally excluded. Research has identified critical cognitive, social, cultural and physiological barriers for this, including low self‐confidence, perceiving themselves as novices or lacking patience, fear of breaking devices and text or buttons being too small. , Therefore, supplying devices with mobile data provided, with bigger keypads and text magnification, and simplifying interfaces could support the participation of older adults in RDCTs. Additionally, RDCTs may seek to leverage existing social networks, for example community and support groups, to facilitate technological engagement.

RDCTs also require a different approach to trial activity by staff and stakeholders. Teams must not assume that all partners share the same understandings of trial requirements. Involving partners such as vendors and regulators early and clearly describing the anticipated participant journey should avoid misunderstandings later in the trial. Strong relationships with organisations holding routinely collected data may also assist in gaining access and anticipating changes in requirements.

Using multiple modes of data capture can mitigate missing data and unexpected problems with data access, ensuring timely reporting of relevant events. Collecting participant‐centric endpoints, such as quality of life, to supplement conventional measurements, can improve the usefulness of the data by capturing information important to participants.

Reducing the logistical burden on participants may inadvertently transfer it to research staff. Therefore, researchers must consider the cognitive and psychological burdens of remote working caused by logistics, technological problems and isolation, and plan appropriate support for trial staff.

Finally, COVID‐19 restrictions highlighted the value of contingency planning in the event of staff redeployment, delays in clinical activities and approvals.

Limitations

The case studies were predominantly selected from organisations already interested in improving RDCT conduct as part of the Trials@Home consortium. However, we also included some external case studies to ensure more representative results.

The significant effect of COVID‐19 on global clinical trial operations during our study period meant that we could not as fully as originally hoped meet our aim of capturing a diversity of opinions and experiences. We interviewed proportionately more trial staff and fewer vendors and patient representatives than planned; this may limit the transferability of our findings.

However, our purposive sampling of stakeholders involved in the day‐to‐day running of an RDCT, or with oversight of it or clinical input, ensured our findings represented the critical factors influencing implementation.

CONCLUSION

RDCTs remain a relatively new approach to conducting clinical trials, and trial teams face challenges implementing novel technologies to engage participants and collect clinical data. However, the findings of this study suggest that by developing strategies to maximise participant and partner involvement and reduce participant and staff burden by simplifying participant experiences and staff workflows, RDCTs could maximise recruitment, engagement and retention.

DISCLAIMER

This communication reflects the views of the Trials@Home consortium, and neither I.M.I. nor the European Union and EFPIA are liable for any use that may be made of the information contained herein.

CONTRIBUTORS

All authors have made a significant contribution to the concept, design, analysis, writing and revision of the manuscript, and have agreed to be listed as authors.

COMPETING INTEREST

The authors declare research income to their institution from Menarini, IMI, EMA, NIHR HTA, BHF, Amgen, RTI, CSO Scotland, Tenovus Scotland, George Clinical, Sanofi and HDR UK, and consultancy income to their institution from AstraZeneca. I.S.M. declares personal consultancy income from AstraZeneca.

SUPPORTING INFORMATION DATA S1 Interview schedule (with minor refinements following pilot and COVID‐19 see below)

Click here for additional data file.

SUPPORTING INFORMATION DATA S2 WP1 BEST case study interviews protocol

Click here for additional data file.