Development of an online research platform for use in a large-scale multicentre study.

BACKGROUND: Participation in research can be beneficial for patients and healthcare providers, but may prove demanding at patient, clinician and organizational levels. Patient representatives are supportive of online research to overcome these challenges. The aim of this pilot study was to develop an online recruitment platform and test its feasibility and acceptability while evaluating the accuracy of participant-reported data.
METHODS: The online research platform was developed in a 1-day 'hackathon' with a digital design company. Women who underwent implant-based breast reconstruction in 2011-2016 were invited by letter containing the web address (URL) of the study site and their unique study number. Once online, participants learned about the study, consented, entered data on demographics, treatment received and patient-reported outcome measures (BREAST-Q™), and booked an appointment for a single hospital visit for three-dimensional surface imaging (3D-SI). Real-time process evaluation was performed. The primary endpoint was recruitment rate.
RESULTS: The recruitment rate was 40 per cent. Of the 100 women, 50 logged on to the platform and 40 completed the process through to 3D-SI. The majority of discontinuations after logging on occurred between consenting and entering demographics (3 women, 6 per cent), and between completing the BREAST-Q and booking an appointment for 3D-SI using the online calendar (3 women, 6 per cent). All women completed the online BREAST-Q™ once started. Participants took a median of 23 minutes to complete the online process. Patient-reported clinical data were accurate in 12 of 13 domains compared with electronic records (95 per cent concordance). Process evaluation demonstrated acceptability.
CONCLUSION: The results of this pilot demonstrate the online platform to be acceptable, feasible, and accurate for this population from a single institution. The low-burden design may enable participation from centres with less research support and participants from hard-to-reach groups or dispersed geographical locations, but with online access.

Introduction

Online research is commonplace in epidemiological studies, and has reported advantages of being cost-effective, accessing hard-to-reach populations and providing a safe space in which participants are more likely to give a candid response to questions of a sensitive nature. Ninety per cent of households have internet access, with 89 per cent of adults in the UK using the internet at least weekly. The age group over 65 years has seen the biggest growth in internet use since 2008, with 87 per cent of households having at least one adult in this age group who had internet access in 2018. Despite this, surgical and oncological research has relied on face-to-face consultations for consent and questionnaire completion. The COVID-19 pandemic has placed the onus on clinical and research teams to limit face-to-face consultations, highlighting the need for more research to be carried out by telephone or online.

Breast reconstruction is evolving rapidly. Aesthetic outcome from breast cancer surgery is important, influencing long-term psychosocial wellbeing along with a patient-centred focus on survivorship. Currently there is no widely accepted standard to measure aesthetic outcome. Panel assessment is where a group of individuals (usually doctors and allied health professionals, sometimes with representation from patients or lay members of the public) review photographs and score aesthetic outcome based on a predefined scale. This is the most widely accepted measure, but has inherent bias, is costly, time consuming, and non-standardized. Patient-reported outcome measures (PROMs) are becoming more popular, but also have multiple external influencing factors, are discordant from expert opinion, and often reported more favourably. An objective method is required to evaluate the technical success of aesthetic outcome for the robust communication and comparison of results and quality assurance.

Three-dimensional surface imaging (3D-SI) has been used to create an objective method of aesthetic outcome assessment for breast-conserving treatment that could replace panel assessment and complement PROMs. Reconstructive surgery involves a number of operative approaches with a broad spectrum of aesthetic goals not limited only to the maintenance of symmetry. To develop an objective method for assessment of aesthetic outcome of breast reconstruction that can capture the diversity of practice, a large multicentre study is required.

The aim of this work was to develop an online research platform for use in a large multicentre study to derive a method of assessment of the aesthetic outcome of breast reconstruction using objective measures from 3D-SI. The research platform was designed to improve accessibility to research and reduce the burden of participation on both participants and investigators. The rationale for this was to facilitate recruitment from centres with less research support, thereby capturing representative data from the UK population rather than major research centres alone.

The aim of this pilot study was to investigate the feasibility, acceptability and accuracy of this bespoke research platform in women undergoing implant-based breast reconstruction.

Methods

The pilot study protocol was reviewed and passed by the London–Surrey Research Ethics Committee (17/LO/0763). A summary is available at clinical trials.gov (NCT03203252).

All procedures performed in studies involving human participants were done in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standard.

The aim of the study was to establish the feasibility, acceptability and accuracy of a bespoke online research platform for recruitment, consent and data collection, ultimately for use in a multicentre study to develop an objective method for assessment of the aesthetic outcome of breast reconstruction. The primary endpoint was assessment of the recruitment rate. Secondary endpoints included discontinuation rates, time burden for participants, accuracy of participant-reported clinical information, and the feasibility of online PROMs in the form of the BREAST-Q™ postoperative reconstruction module.

Women over the age of 18 years who had undergone mastectomy and implant-based breast reconstruction at the Royal Marsden Hospital in Sutton, UK, between 2012 and 2017 were identified through operation records. The Royal Marsden is a Comprehensive Cancer Centre providing treatment and research, working in partnership with the Institute of Cancer Research. It is a National Institute for Health Research (NIHR) Biomedical Research Centre for cancer. The Sutton branch, where this study was carried out, provides secondary breast care, treating women from the local population with symptomatic and screen-detected breast disease. Eligibility was cross-checked with electronic patient records, and women were invited to participate by postal letter. The potential participants were invited in reverse chronological order of operation date (most to least recent). Unilateral or bilateral, nipple-sparing or nipple-sacrificing, therapeutic or risk-reducing, and immediate or delayed reconstructions with or without radiotherapy were included. Exclusion criteria were implant loss at any stage to a flat chest wall, metastatic disease, or an autologous component to the reconstruction (nipple reconstruction using autologous tissue, and fat transfer to improve aesthetics were allowed).

The letter of invitation contained a participant information sheet, a unique study identity number (ID) (the link to which was held securely), and information on how to access the study website. Participants visited the website, and from the homepage could access study information with options to follow links to further information, read biographies of the study team, browse patient testimonials from previous studies using 3D-SI, and read about the institutions involved (). Once satisfied that they had enough information and would like to participate, they were invited to follow the consent link.

Homepage of the study website

Consent was completed at the outset of the online data collection process with two checkpoints to ask whether any further information was required before consent and to explain withdrawal policy. The online data collection was by study ID alone, and included demographics, clinical data, PROMs and a real-time process evaluation. Participants visited the hospital once for a 3D-SI (10-min appointment), which was booked via the online platform. The data collection pages and calendar were not made available until the consent process had been completed in its entirety.

3D-SIs were captured using a Vectra XT® body scanner (Canfield Scientific, Parsippany, NJ, USA). Women were photographed using a standard protocol: hands on hips, elbows positioned behind the mid-axillary line, at the end inspiratory pause of quiet breathing. The patient was aligned to a predefined grid visible on the preview screen before image capture. The images did not contain the patient’s face and were stored under study ID only.

The primary endpoint was the proportion of women invited who completed the online process and attended for 3D-SI. The discontinuation rate was assessed at each stage of the process. Relevant domains from the BREAST-Q™ postoperative reconstruction module (1–6 and 10) were embedded into a Survey Monkey questionnaire within the website. Completion rate and ease of analysis were used to gauge feasibility.

Patient-reported data (demographics and clinical data) were compared with electronic records to establish accuracy. A predefined threshold of 95 per cent agreement was set for binary variables (correct or incorrect). For continuous variables, individual thresholds were set: height within 5 cm and weight within 5 kg (weight was measured at the 3D-SI appointment and height was taken from electronic records). A binary outcome (yes or no) was used for analysis depending on whether the patient-reported data met the criteria or not.

Sample size

This was a feasibility study, so no formal power calculation was undertaken. An estimated sample size of 50 participants was based on the need to gauge uptake and acceptability of the study design. A recruitment rate of 50 per cent was predicted based upon previous research and set as the target for the pilot. Letters of invitation were sent to 100 patients on this premise.

A protocol amendment to permit a telephone call to potential participants was made when the expected recruitment target was not reached. From then on, a telephone call was made to the potential participants who had not engaged with the first letter of invitation (not consented and not declined to participate) in order to endorse the study, answer any questions, and provide assistance. The impact of the telephone call on recruitment rate was reported.

Statistical analysis

SPSS® version 24 (IBM, Armonk, NY, USA) was used. Continuous data were summarized as mean(s.d.) or median (i.q.r.) values, with appropriate transformations applied. Categorical data were expressed as frequencies of counts with associated percentages.

Results

Demographic and clinical data for the pilot, amendment, and total study population are summarized in . The median time from surgery to study participation was 29 (i.q.r.14–41) months. The mean age was 52 (range 28–77) years. The most common operation was unilateral mastectomy (19 of 44 patients, 43 per cent), with the majority having had immediate definitive fixed-volume implant reconstruction (30 of 44, 68 per cent).

Of 100 women invited by letter, 38 started the online process and 36 consented to the study. Two potential participants actively declined by e-mail, and 60 did not respond. Thirty (79 per cent of the 38 women who started) completed the online process and attended for 3D-SI (). The recruitment rate for letter-only invitation was therefore 30 per cent.

Recruitment

Primary endpoint is the recruitment rate, defined as the proportion of those completing the online process and attending for three-dimensional surface imaging compared with those invited.

The amendment, permitting a follow-up telephone call to potential participants who had not previously engaged, was applicable to 62 women (36 had previously consented (2 discontinued during the consent phase) and 2 had declined to participate). Fifty three women (85 per cent) remained eligible (4 had since undergone autologous reconstruction, 4 had moved out of area and 1 had metastatic disease). Eligible women were contacted by telephone. Thirty-six participants (68 per cent) were contactable and 17 (32 per cent) were not contactable on two separate occasions. All women successfully contacted expressed interest in participation. Twenty four (67 per cent) did not engage further and 12 started the online process, of whom 10 completed the primary endpoint (). The additional recruitment rate with telephone endorsement was 19 per cent (10 of 53), improving the overall recruitment rate from 30 to 40 per cent.

Completion rate and time burden

Completion rates for the online steps are shown in . Of the 50 women who started the online process, 40 (80 per cent) attended for 3D-SI. The majority of dropouts occurred between consenting and entering demographics (3 women, 6 per cent), and between completing the BREAST-Q™ and booking an appointment for 3D-SI using the online calendar (3 women, 6 per cent). The median time taken from starting the consent process to completing the BREAST-Q™ was 23 (i.q.r. 14–28) min for the total study population.

Cumulative completion rate for the total study population by online stage.

3D-SI, three-dimensional surface imaging.

Accuracy of participant-reported clinical data

Participant-reported clinical data met the predefined criteria for acceptable accuracy of 95 per cent concordance with medical records in 12 of the 13 domains for the total study population (). The domain ‘date of reconstruction’ did not meet the predefined threshold of 95 per cent (91 per cent). Height was reported to within 5 cm for 96 per cent of patients, with a median error of 1 (i.q.r. 0–2) cm. Of the 28 women in the pilot cohort who had their weight measured at the time of photography, 27 (96 per cent) were accurate to within 5 kg with a median error of 2 (i.q.r. 1–2) kg. Weight was not measured at the time of photography for the amendment cohort.

Accuracy of participant-reported clinical data compared with electronic patient records for pilot, amendment and total study population

A predefined threshold of 95 per cent concordance was set (dotted black horizontal line). EPR, electronic patient records.

Online patient-reported outcomes measures

Of 44 participants (100 per cent) who started the online BREAST-Q™, all completed it, demonstrating acceptability (). There was a high response rate to the sexual well-being domain (41 of 44, 93 per cent).

Real-time user evaluation

The optional real-time user evaluation was completed by 30 participants. All said the website was easy to navigate, and 28 (93 per cent) found the questions clear and easy to understand. Twelve participants made suggestions for additional questions pertaining to aesthetics and well-being that included satisfaction with the contralateral symmetrization, satisfaction with prosthetic nipples, implant versus native breast (specifically temperature, how they move, and how they feel to touch), availability of preoperative information on aesthetic outcome, and changes in aesthetics over time.

Cross-checking the real-time user evaluation with the participant-reported clinical data drew attention to important areas to modify for the main study. For example, there was no option for women to say that they had declined sentinel lymph node biopsy despite having invasive cancer, so participants chose a ‘best fit’ option, not because they were unclear of their situation, but owing to limitations in the format of the questionnaire.

Discussion

This pilot study has demonstrated the feasibility of an online platform, acceptability to participants, and accuracy of participant-reported data in this population. The target recruitment rate of 50 per cent was based on previous studies using 3D-SI technology conducted at this institution. Differences in study design including the addition of an online component, the removal of a study endorsement telephone call, no face-to-face contact with a member of the study team, an additional trip to the hospital (previously studies were coordinated with the surveillance mammography schedule, so participants were already at the hospital), and different demographics for participants meant that the target was not met. Recruitment rates with and without telephone endorsement were 40 and 30 per cent respectively, in line with other letter-only invitation studies (for example PROCAS 20 per cent). Although a response rate of 55 per cent for online PROMs was reported for patients with potentially curable breast, colorectal and prostate cancer, a varied approach was used and considerably more people participated when approached face-to-face (61 per cent) compared with telephone (48 per cent) or letter (41 per cent). A meta-analysis demonstrated an average recruitment rate of 39 per cent for 68 internet-based surveys in 49 studies, although more recently a response rate was only 18 per cent with a completion rate of 5 per cent for the CUPID study (Contraceptive Use, Pregnancy Intention, and Decisions), which consisted of online consent and a survey, using a mailed invitation, stratified sampling and incentivized participation.

One-to-one contact time with a member of the research team to endorse the study is reported to be the most effective way to enhance understanding and optimize recruitment,. The burden placed on clinical or research teams to provide this contact may preclude participation from units with little or no research support. Several elements of the online method described in this study have been designed to overcome this lack of ‘face time’, including study team biographies, photographs of the research team, patient testimonials, and an easily accessible ‘contact us’ link, even though these are dependent on initial engagement with the website. A further consideration to augment the recruitment process could be the inclusion of a short video with the local principal investigator embedded within the website.

This pilot study enrolled women who had undergone implant-based reconstruction at a single hospital site. Patients with breast cancer at the Royal Marsden Hospital in Sutton are drawn predominantly from the local area and include screening and symptomatic presentations, with relatively few patients transferring care from a wider geography. The findings from this study are therefore more generalizable than those from other parts of the organization’s practice. The rate of participation demonstrated is sufficient to support the view that a highly scalable online platform will recruit many participants when offered through multiple hospitals, without requiring significant research staff input from local clinical teams.

There is concern about internet-mediated research and non-response bias. Are the collected data representative of the population or is the sample skewed by the method of collection? This needs to be borne in mind, given that 60 per cent of the target population did not respond in this pilot study. Several studies have provided reassurance that these issues are not major sources of bias. An Australian study recruited 3795 women aged 18–23 years to an internet-based research project on contraceptive use and pregnancy, and found the population to be broadly representative of the general population aside from an over-representation of tertiary-educated women (89 per cent in the study versus 73 per cent in the general population). A study that compared results from an internet-based survey with well known statistics for perinatal health (low birthweight and smoking status) across six domains found similar results in a cohort of women of reproductive age.

In the present study, the 3D-SI patient-steering group at this institution was consulted during protocol development and after pilot study completion. The group concluded that the potential selection bias of women more familiar with internet capabilities was acceptable given the ability for improved accessibility to potential participants from centres with less research support and for women who find hospital visits difficult. The advantage of providing the ability to take part in research in a more comfortable environment, especially when answering questions of a sensitive nature, may be important. Indeed, only 7 per cent (3 participants) in this studychose the ‘not applicable’ option for the sexual well-being domain, compared with 37.5 per cent of 200 women in a study in which the BREAST-Q™ was completed in a private room at the hospital. This observation is concordant with the literature, which reports less non-response and an increase in disclosure of sensitive information in web-based surveys.

It has been reported that participants of online research often do not read the consent form before consenting. The consent form used in this study was designed in line with the British Psychological Society Ethics Guidelines for Internet-Mediated Research to optimize the chances of participants reading it thoroughly. The consent form was simply laid out, tick boxes were used next to each statement of consent to encourage participants to address each point, a check point was built into the system to ascertain whether the potential participant had enough information to consent and, if not, there were links to more information or an opportunity to make contact with the study team. The withdrawal policy was clearly defined before completion of the consent process, and, finally, the data collection pages were not accessible unless the consent form had been completed in its entirety.

With online research some control of participant activity is lost. The data collection pages were designed, where possible, to be multiple-choice questions, with each question requiring an answer or at least a ‘not applicable’ box to be ticked, in order to facilitate ease of data analysis, reduce misinterpretation of prose during data analysis, bridge literacy/language gaps (medical versus lay), and ensure complete data sets.

Feedback on the website design from patient representatives from the 3D-SI research steering group endorsed the use of Royal Marsden Hospital and National Health Service (NHS) branding to engender trust. This is in line with NHS England’s NHS identity research published in 2016, stating that the NHS logo was associated with trust, respect, service quality, expertise, and accountability to the public. It is proposed that, for the future multicentre study, each participating centre will have a branded subdomain of the website under the umbrella of the NHS.

Participant-reported clinical information was accurate in 12 of 13 domains to a minimum standard of 95 per cent concordance with electronic records in this pilot study. Closer analysis of the errors in data reporting, together with cross-checking the answers to the real-time user survey, showed that data were often incomplete rather than inaccurate, reflecting the terminology or options available in the multiple-choice answers. For example, missing multiple operations or historical reconstructions were responsible for failure to reach the predefined threshold for accuracy of date of reconstruction. This is rectifiable by clarifying vocabulary and adding further options or free-text boxes where required. The accuracy of patient-reported data could greatly reduce the burden on investigators. The data collected in this trial were relatively simple, but prospective patient-reported data collection could be used in future studies to enable more complex data entry in real-time. The reported accuracy is in line with that of a large study investigating the internal validity of demographic data entered online involving more than 84 000 participants that reported 94 per cent consistency with database records.

This small single-centre pilot study has demonstrated feasibility, accuracy and acceptability to participants. It has been designed to be scalable for use in a multicentre study, carries a low burden for investigators, and has enabled data collection from a broad, representative sample of the population. A wider use of these resources should encourage more patients and centres to participate in clinical trials.

Table 1

Demographics and clinical data for the pilot, amendment and total study populations

	Pilot (n = 34)	Amendment (n = 10)	Total population (n = 44)
Age (years)*	51 (28–69)	54 (40–77)	52 (28–77)
Height (m)†	1.64(0.08)	1.61(0.06)	1.63(0.08)
Weight (kg)†	67.1(9.7)	67.4(7.2)	67.1(9.1)
Indication for initial surgery
Unilateral mastectomy + implant reconstruction for cancer	15	4	19
Bilateral mastectomy + implant reconstruction (cancer 1, symmetry/risk reduction 1)	11	1	12
Bilateral risk-reducing mastectomy	5	3	8
Bilateral mastectomy + implant reconstruction for bilateral cancer	3	2	5
Timing of reconstructive surgery
Mastectomy + immediate definitive implant reconstruction	24	6	30
Mastectomy, expander + delayed definitive implant	10	4	14
Date of most recent operation
2012	3	0	3
2013	4	3	7
2014	5	0	5
2015	9	3	12
2016	12	4	16
2017	1	0	1
Symmetrization surgery
Bilateral mastectomy at first surgery	18	6	24
Contralateral reduction	1	2	4
Symmetrizing mastectomy + reconstruction of a different type	1	0	1
None	14	2	15
Nipple surgery
Nipple-sparing mastectomy	18	6	24
Nipple removed, not reconstructed	12	3	15
Nipple removed, reconstructed	4	1	5
Chemotherapy
Adjuvant	10	2	12
Neoadjuvant	9	2	11
Cancer, but chemotherapy not indicated	10	3	13
No radiotherapy (risk-reducing surgery)	5	3	8
Radiotherapy
Postmastectomy to reconstruction	12	4	16
Adjuvant after previous BCS	2	0	2
Cancer, but radiotherapy not indicated	15	3	18
No radiotherapy (risk-reducing surgery)	5	3	8
Axillary surgery
SLNB	18	4	22
ALNC	8	1	9
Unilateral SLNB + contralateral ALNC	1	1	2
None (risk-reducing surgery)	5	3	8
None (patient choice)	2	1	3

Values are mean (range);

values are mean(s.d.). BCS, breast-conserving surgery; SLNB, sentinel lymph node biopsy; ALNC, axillary lymph node clearance.