Optimised patient information materials and recruitment to a study of behavioural activation in older adults: an embedded study within a trial.

Background: Printed participant information about randomised controlled trials is often long, technical and difficult to navigate. Improving information materials is possible through optimisation and user-testing, and may impact on participant understanding and rates of recruitment.
Methods: A study within a trial (SWAT) was undertaken within the CASPER trial. Potential CASPER participants were randomised to receive either the standard trial information or revised information that had been optimised through information design and user testing.
Results: A total of 11,531 patients were randomised in the SWAT. Rates of recruitment to the CASPER trial were 2.0% in the optimised information group and 1.9% in the standard information group (odds ratio 1.027; 95% CI 0.79 to 1.33; p=0.202). Conclusions: Participant information that had been optimised through information design and user testing did not result in any change to rate of recruitment to the host trial. Registration: ISRCTN ID ISRCTN02202951; registered on 3 June 2009. Copyright:

RCT Entities:   Population Interventions Outcomes

Introduction

Potential participants in randomised controlled trials are given information that is often long, technical and difficult to navigate [1– 3]. Consequently, they may lack understanding of important details about the trial [1, 4, 5], which limits their ability to make an informed decision about consent.

Improving information materials is possible through optimisation and user-testing. This involves making changes to the design and text based on good practice in information design and people’s ability to find and understand information during testing [6]. Materials revised after user-testing have been shown to be preferred [7, 8], although a recent review concluded that optimised information has little or no impact on trial recruitment [9]. However, the evidence base remains limited [10– 13], and a recent ‘review of reviews’ reported that information for patients can be a facilitator of research participation [14].

Study aims

This embedded study within a trial (SWAT) assesses whether optimisation of patient information materials through user testing could increase participant recruitment to the CASPER study [15].

Methods

Design

The SWAT was conducted within CASPER, which investigated the effectiveness of behavioural activation in patients aged 65 years or older with sub-threshold levels of depression [15]. CASPER used a cohort multiple randomised controlled trial design [16].

Participants

Participants were registered patients at one of six UK medical practices in Durham, Harrogate, Leeds and York. They were included if they were potentially eligible for CASPER.

Intervention

All participants in the SWAT were posted an invitation letter, participant information sheet (PIS), screening questionnaire and consent form for the CASPER trial. The control group received the standard CASPER developed PIS (see Extended data) [17] whilst the intervention group were sent an optimised version (see Extended data) [18] developed through three rounds of user testing and revision.

Patients returned the questionnaire and a consent form indicating a willingness to participate, after which they were recruited to the CASPER cohort. Following a telephone diagnostic interview, eligible patients were recruited to the CASPER intervention trial.

User testing

User testing involved 30 people reflecting the CASPER target population. In the first round of testing, 10 participants read the standard invitation letter and PIS. They were then asked to locate and demonstrate their understanding of 18 items of information within the PIS (on the study’s nature and purpose; process and meaning of consent; study procedures; nature of the CASPER trial intervention). The PIS was then revised based on participant responses. A second round of testing was completed, in which 10 new participants read the invitation letter and a revised PIS and were asked to find and show understanding of the same 18 information items. The PIS was further revised and tested on 10 new participants through the same 18 information items.

Through testing, changes to the PIS included adding a title page, a summary of key points and a contents page, highlighting headings using colour and larger font, and simplifying wording. The final optimised PIS was printed as an A4 booklet (Figure 3).

Outcomes

The primary outcome measure was the proportion of patients in each group who were recruited to the CASPER trial. The secondary outcomes were (i) the proportion of patients recruited to the CASPER cohort, and (ii) the proportion of invited patients returning forms to express interest in participation in CASPER.

Sample size

It was predicted that 30% of invited patients would return the consent form and indicate interest in CASPER participation, of whom 20% (600) would be eligible to take part in the CASPER trial. An improvement in response rate of 10% (i.e. from 30% to 33% participants) would be a significant increase in uptake. A sample size of 8,000 potential participants would be sufficient at 80% power to detect a difference of 10% in recruitment rate.

Randomisation

Individual patients were allocated randomly (1:1) to receive either the standard or optimised PIS by an independent statistician at York Trials Unit.

Statistical analysis

Odds ratios (ORs) and corresponding 95% confidence intervals (CIs) were calculated to compare the proportion of patients from each group that were recruited to the CASPER trial; recruited to the CASPER cohort; or expressed interest in participation. Analyses were conducted in Stata version 14.2.

Approvals

CASPER and the SWAT were approved by the NHS Leeds North-East Research Ethics Committee (10/H1306/61).

Results

Overall, 11,531 patients were invited to participate [19]; 5,765 (50.0%) were randomised to the optimised PIS and 5,766 (50.0%) to the standard PIS ( Figure 1).

Figure 1.

Flow diagram of recruitment to the CASPER trial.

A total of 2,169 patients returned the consent form indicating a willingness to take part: 1,102 (19.1%) in the optimised PIS group and 1,067 (18.5%) in the standard PIS group (odds ratio (OR) 1.04; 95% confidence interval (CI) 0.95 to 1.14; p=0.402).

A total of 229 patients were recruited to the CASPER trial: 116 (2.0% of those invited) in the optimised PIS group and 113 (1.9%) in the standard PIS group (OR 1.027; 95% CI 0.79 to 1.33; p=0.202).

In total, 1,667 patients expressed interest in participating but were ineligible for the CASPER trial and were recruited to the CASPER cohort: 851 (14.8% of those invited) in the optimised PIS group, and 816 (14.1%) in the standard PIS group (OR 1.05; 95% CI 0.95 to 1.16).

Discussion

Optimisation of the PIS resulted in no statistically significant difference in the rates of recruitment to the CASPER trial or CASPER cohort, or rates of consent form returns. This is consistent with previous research [9], including other embedded trials within the MRC START programme, which have observed little or no effect on recruitment [11– 13, 20].

Whilst there was no impact on recruitment, the optimised materials may have improved understanding of the trial thus enabling patients to make a more informed decision. Improved comprehension could also increase retention, due to greater understanding of the trial prior to recruitment. These outcomes were not assessed and further research examining this is warranted.

Conclusion

Optimised patient information materials did not increase recruitment to the host trial or expressions of interest in participation.

Data availability

Underlying data

Figshare: CASPER SWAT data.csvCASPER SWAT recruitment data and evaluated information sheets. https://doi.org/10.6084/m9.figshare.12302672 [20].

This project contains the underlying data

Extended data

Figshare: Figure 2 CASPER PIS (original). https://doi.org/10.6084/m9.figshare.12302675 [17].

This file is the original CASPER participant information sheet.

Figshare: Figure 3 CASPER PIS (revised). https://doi.org/10.6084/m9.figshare.12302678 [18].

This file is the revised CASPER participant information sheet.

Reporting guidelines

Figshare: CONSORT checklist for ‘Optimised patient information materials and recruitment to a study of behavioural activation in older adults: an embedded study within a trial’. https://doi.org/10.6084/m9.figshare.12312206.v1 [21].

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

This is a well-written article describing a large, well done SWAT. I only have a few comments, which are listed below.

Study aims

The authors might want to add a sentence (and possibly a reference) that explains in brief what a SWAT is.

Methods

Design– linked to the host trial ‘cohort multiple randomised controlled trial design’. This design is sometimes called a ‘Trials Within Cohort (TwiCs)' design. The authors might want to mention this name too, just in case readers are more familiar with it.

User testing– the authors mention users were asked to locate and demonstrate understanding of 18 items of information within the participant information leaflet. I wasn’t sure what the 18 items were. Would it be possible to list them? Apologies if I have missed them within the article or extended data.

Randomisation– could the authors give a little more information on how the randomisation sequence was generated?

Discussion

The authors mention that their results are consistent with previous research, which is my understanding of the literature too. Do the authors think further evaluation of this SWAT evaluation is required, and if so, could they give some pointers to what sorts of evaluations would add most value to the evidence base?

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Randomised trial methodology, including SWATs.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

This is an impressive and important piece of methodology research, which has been reported in a clear and succinct manner. I am pleased to have been asked to review it.

The reported randomized trial is an example of a very large SWAT, that was embedded in the CASPER trial of behavioural activation therapy for older patients with sub-threshold levels of depression, to test the impact on recruitment of an optimized information materials for people being invited to consider participation in the clinical trial.

It might be helpful to readers if the authors could include a little more information about the SWAT concept, perhaps by citing one or both of the relevant Trial Forge papers: (a) Treweek S, Bevan S, Bower P, et al. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials 2018; 19: 139; and (b) Treweek S, Bevan S, Bower P, et al. Trial Forge Guidance 2: how to decide if a further Study Within A Trial (SWAT) is needed. Trials 2020; 21: 33.

Although the optimized information materials did not improve the proportion of invitees who were recruited into CASPER, it would be interesting to know if there were other benefits, such as a better understanding of the clinical trial. The paper concludes with some comments about this but the authors' comment that “further research examining this is warranted” seems to suggest that this type of qualitative research might not be done with the CASPER trial patients. However, given that they will be collecting retention data for CASPER, I hope they will revisit this SWAT later in the CASPER trial to answer their own question about the potential for an impact on retention.

If they have not already done so, I suggest that the authors register this SWAT in the SWAT repository. It might also help readers if they referred to two similar SWAT that are already registered there (SWAT 101 and SWAT 105).

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Health services research. Randomized trials. Systematic reviews. Methodology research.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.