The use of volunteers to implement electronic patient reported outcomes in lung cancer outpatient clinics.

BACKGROUND: Treatment related toxicity is common after chemotherapy and radiotherapy. Our group has developed and validated an electronic Patient Reported Outcome questionnaire (ePRO) to assess symptoms and toxicity in lung cancer patients receiving (chemo)radiotherapy treatment. We assessed the need for volunteer support in clinics to assist patients in completing ePROs.
METHODS: Lung Cancer patients attending outpatient or radiotherapy clinics at The Christie NHS Foundation Trust, Manchester were consented and asked to complete a Patient Reported Outcomes questionnaire using an electronic device (a touchscreen). Trained volunteers were available if patients required help such as verbal or physical assistance. The primary objective was to determine the need for volunteers to assist lung cancer patients in completing ePROs.
RESULTS: 27/86 (31.4%) of patients who consented to this study required assistance to complete the ePRO. After questioning, we found that only 7/86 (8.1%) would have relied on volunteers for assistance as the majority of patients had a companion that could have provided help. 81/86 (94.2%) of patients were satisfied with the use of a touchscreen tablet to complete the ePRO.
CONCLUSION: Our results demonstrate that the introduction of ePROs in lung cancer outpatient clinics is feasible, even without the use of volunteers for the majority of patients. The implementation of ePROs would allow large volumes of high quality (chemo)radiotherapy toxicity data to be collected accurately and quickly. This is essential for the development of predictive models of outcome using population-based data, which could allow the personalisation of (chemo)radiotherapy treatment for lung cancer patients.

Introduction

Chemotherapy and radiotherapy play a major role in the treatment of lung cancer patients. Treatment-related toxicity is common in patients treated with radiotherapy with or without chemotherapy ((chemo)radiotherapy) [1], [2]. The standard for grading treatment-related toxicity in the context of clinical trials is clinician-led, using the Common Toxicity Criteria for Adverse Events (CTCAE) [3]. In the routine setting toxicity data is generally not recorded in a structured or consistent way and data is often missing, possibly due to time constraints during busy oncology clinics. The use of Patient Reported Outcome questionnaires (PROs) is a solution to collect such data in a more efficient manner. PROs have been shown to be more accurate, highlight more symptoms, and provide more details than traditional clinician-based reporting. It has been demonstrated that clinician graded toxicity tends to underestimate symptom severity, and is influenced by patient-clinician dynamics and inter-rater variability [4]. PRO data collection eliminate these factors by allowing patients to prospectively describe and grade their own symptoms using a validated questionnaire derived from the CTCAE system [5], [6]. Recent randomised controlled trials have shown that cancer patients followed up with the help of PRO tools have a significantly better survival compared to patients followed up in a standard way [4]. These trials highlight the importance of the introduction of such tools in the clinic.

Electronic data collection has many practical advantages over paper-based PROs including ease of data collection and storage. It also allows more efficient data analysis without compromising data validity [7], [8]. This platform could allow large volumes of high-quality large scale prospective toxicity data to be gathered for the development of predictive toxicity models following treatment for lung cancer with (chemo)radiotherapy [9]. Treatment decisions are often based on clinical trials that involve younger, fitter individuals without comorbidities. Given that the median age of patient diagnosed with lung cancer is 70 years [10], elderly patients would benefit from individualised treatment based on the use of predictive models.

Previous literature shows that the use of ePROs in cancer outpatient consultations is feasible and acceptable to patients [7], [11], [12]. There is currently very little data to understand the feasibility of implementing ePROs in the lung cancer population. This group of patients are generally elderly and come from lower socio-economic backgrounds [13]. The primary aim of this study was to understand if volunteers were necessary to implement ePROs in lung cancer patients. Further aims were to identify which particular patient groups are more likely to require assistance and if help is given whether this has an impact on completion rate at subsequent visits. We also aimed to understand patients’ satisfaction with the introduction of ePROs.

Materials and methods

Patients

This study recruited patients between May and July 2016 at a large UK cancer centre. Eligible patients were aged 18 and over with a histological or clinical diagnosis of lung cancer, attending lung cancer or radiotherapy treatment clinics at The Christie NHS Foundation Trust. Patients unable to give informed consent, attending lung cancer clinics for the first time, or had previously completed ePROs were excluded from the study. The study protocol gained ethical approval by the North of Scotland Research Ethics Committee.

Study design

This was a prospective open questionnaire based study. Enrolled patients were asked to complete three questionnaires consecutively during a single hospital visit (as shown in Fig. 1) and if possible were asked to complete the questionnaires again at a subsequent visit.

Fig. 1

Questionnaires used in the study, completed in the order shown.

Two of the questionnaires (Q1&3) were completed on paper and one electronically (Q2). Questionnaire (Q1) (see Appendix A) collected patient demographic information. A patient satisfaction questionnaire (Q3) (see Appendix B) was also completed on paper before and after completion of the ePRO questionnaire and provided feedback regarding the use of ePROs. The rationale for using paper for Q1&3 was to encourage participation in patients hesitant towards using electronic devices.

The ePRO questionnaire (Q2) (see Appendix C) was completed on a web tool using a touchscreen tablet that was cleaned between uses. It was an electronic adaptation of a previously validated paper PRO used to collect data on acute toxicities and performance status (PS) for lung cancer patients receiving (chemo)radiotherapy [14]. During our study, completed ePROs were uploaded to The Christie’s electronic patient record to allow doctors to access them in real-time before the consultations.

Patients who consented to the study completed the three questionnaires at up to two time points. The first time-point was at baseline (the first clinical visit) and the second at their subsequent clinic visit if this was possible during the timeframe of the study. All data was collected over a 6-week period.

Patients were given no training on using touchscreen devices before being asked to complete the ePRO questionnaire. Patients received the tablet with a new ePRO form ready to complete. They were asked to attempt to complete ePRO unaided if possible. If help was required, the volunteer/researcher was available to provide assistance. This included both verbal and physical help in order to complete ePRO using the touchscreen tablet. The key reason we asked companions not to assist was because we wanted to find out what proportion of patients required help completing an ePRO. We recorded the main difficulties patients encountered when using the touchscreen tablet to complete ePRO. The study also investigated if a companion attending clinic with the patient could have helped with ePRO completion to determine volunteer necessity. The volunteers were recruited using The Christie NHS Foundation Trust regulated and vetted volunteer service, and had received training regarding ePROs and the study process.

There are a number of challenges associated with creating a platform to collect ePRO securely within a hospital’s electronic record. It is essential that volunteers and patients did not have access to other patients’ confidential information. For eligible patients attending clinic, a link to a web address was created that would open a new ePRO form within the patient’s record. These links were verified by the research student and stored on a web page only accessible by the research student. Each patient’s link was identifiable by their clinic date, time, and their hospital number. A trained volunteer or research student would open the link on the patient’s behalf, before asking the patient to complete ePRO. Access to any other webpages during or after completion of the ePRO questionnaire was denied by design, preventing inadvertent access other patients’ electronic record.

Patients’ clinical data regarding disease and treatment were obtained from the patient electronic record. Patients’ postcodes were also extracted to calculate an Index of Multiple Deprivation (IMD). Areas in the UK are ranked from 1 (most deprived) to 32,844 (least deprived), with each area representing a small piece of the country containing an average of 1500 people. The calculation is based on seven domains such as income, employment, and health, each given different a weighting [15].

Statistical analysis

Sample size calculation determined that a minimum of 86 patients would be required to test whether the proportion of patients requiring volunteer assistance with ePROs was significantly different to 10%. It should be noted that 10% is the threshold above which the study researchers judged that the implementation of volunteers would be necessary. The sample size calculation was based on our estimation that approximately 20% of patients would require assistance; power was set to 80% and 2-sided alpha to 5%. All 86 patients had volunteer help available to them if required, hence a control group was not necessary.

A 1-sample chi-squared test (2-sided) was used to determine whether the observed proportion of patients needing help from a volunteer was significantly different to 10%. T-tests, trend tests and Fisher’s exact tests (2-sided) were used to test for association between patient characteristics and whether volunteer help was required. A McNemar’s test (2-sided) was used to determine whether volunteer help had an impact on the completion rate at subsequent visits. Data was analysed using Stata version 13.

Results

From May to July 2016 a total of 104 eligible lung cancer patients were approached during outpatient and radiotherapy treatment clinics. 86 patients consented to take part in the study and 18 declined for reasons displayed in Table 1. All 86 patients completed the three questionnaires consecutively in a single siting on at least one occasion. 15 individuals completed the questionnaires a second time on their subsequent visit during the study period.

Table 1

Reasons given by eligible patients declining to participate.

Reasons given for declining participation in the study	Number of patients, n
Not comfortable using electronic devices	7
Feeling unwell/anxious about appointment	5
Prefer to wait undistracted	5
Patient reported physical disability	1

Study population

The individuals enrolled in the study formed a small sample of a general lung cancer patient population (Table 2). The median age was 68 years (49–88 years) and 58.1% of patients were female. The most common diagnosis was non-small cell lung cancer (NSCLC) (86.0%), with adenocarcinoma being the most common histological subtype (36.5%), and stage III the most common stage of disease (31.4%). The modal therapy was radical radiotherapy alone (26.7%), followed by concurrent chemoradiotherapy (16.3%). Most patients had both a PS and adult comorbidity evaluation (ACE) score of 1 (53.5%, 32.6%). The median Index of Multiple Deprivation for the group was 12, 032.

Table 2

Study population characteristics.

Patient, Tumour and Treatment Characteristics n = 94
Characteristic	Median	Range
Age, years	68	49–88
IMD, rank	12,032	407–32,234
	Number of patients, n	Percentage (%)
Gender
Male	36	41.9
Female	50	58.1
Diagnosis
NSCLC	74	86.0
SCLC	12	14.0
Clinical stage
NSCLC	74	86.0
I	15	17.4
II	9	10.5
III	27	31.4
IV	23	26.7
SCLC	12	14.0
Limited	11	12.8
Extensive	1	1.2
Histology
NSCLC (n = 74)
Adenocarcinoma	27	36.5
Squamous	21	28.4
Large cell	1	1.4
Mixed	4	5.4
Not otherwise specified	19	25.7
Awaiting confirmation	2	2.7
SCLC (n = 12)	12	14.0
Treatment
Concurrent CRT	14	16.3
Sequential CRT	9	10.5
Radical RT alone	23	26.7
SABR	5	5.8
Palliative RT	7	8.1
Adjuvant RT	1	1.2
Palliative CT alone	4	4.7
Neoadjuvant CT	5	5.8
Palliative CT	11	12.8
Other	4	4.7
Awaiting decision	3	3.5
ECOG PS
0	16	18.6
1	46	53.5
2	16	18.6
3	7	8.1
Unknown	1	1.2
ACE comorbidity score
0	16	18.6
1	28	32.6
2	24	27.9
3	9	10.5
Unknown	9	10.5
Smoking status
Ex	53	61.6
Current	28	32.6
Never	4	4.7
Unknown	1	1.2

Abbreviations: IMD, Index of Multiple Deprivation; NSCLC, Non-Small Cell Lung Cancer; SCLC, Small Cell Lung Cancer; CRT, Chemoradiotherapy; RT, Radiotherapy; CT, Chemotherapy; SABR, Stereotactic Ablative Body Radiotherapy; ECOG, Eastern Cooperative Oncology Group; PS, Performance Status; ACE, Adult Comorbidity Evaluation.

ePRO questionnaire completion

Fifty-nine (68.6%) patients completed the first ePRO questionnaire without requiring assistance. The remaining 27 (31.4%) patients received help from either a volunteer or the researcher. Of the 27 patients that required assistance, the majority usually attend clinic with a companion (for example, family member or friend). Twenty (74.1%) patients that received volunteer help had a companion that could have helped the patient if necessary (Fig. 2). If we exclude these individuals, only seven (8.1%) patients would have relied on volunteers for assistance in order to complete ePRO.

Fig. 2

Requirement for help to complete the ePRO questionnaire.

Comparing the effect of patient characteristics on completion of ePRO

We tested for association between patient characteristics and completion of ePROs within our study group. Only age (p = 0.0056) (Fig. 3) and internet access at home (p = 0.015) (Table 3) were statistically significant; older patients and those with no internet access at home were more likely to require help. Disease stage, PS, ACE score, gender, and Index of Multiple Deprivation were not significantly associated with the need for volunteer assistance.

Fig. 3

Patient age and ePRO completion. Older patients were more likely to require help (p = 0.0056).

Table 3

Home internet access of the study population, comparing help versus no help groups. Patients without internet access at home were more likely to require help (p = 0.015).

	Internet access	No internet access	No response	Total
Help group	19	8	0	27
No help group	54	4	1	59
Total	73	12	1	86

Eight (29.6%) patients out of 27 who needed help at baseline were able to complete the questionnaire a second time at their next hospital visit. Half of these individuals required help again, the remaining 4 patients were able to complete ePRO without assistance (p = 0.125).

Patient satisfaction and preference

Before the initial attempt at ePRO completion, 60 (69.8%) patients agreed or strongly agreed that they had experience using an electronic device, and more patients preferred paper questionnaires (n = 37, 43%) rather than electronic collection (n = 16, 18.6%) if given a choice (Fig. 4).

Fig. 4

Patients preference between electronic and paper questionnaires before and after completion of the ePRO questionnaire. Patient response to the question: ‘Would you prefer to complete the questionnaire on paper?’

Following the completion of the questionnaires at baseline, the majority of patients found that the ePRO questionnaire was easy to follow (n = 85, 98.8%) and understand (n = 81, 94.2%). 81 patients strongly agreed or agreed that they were happy to continue using the touchscreen tablet to complete ePRO. 79 (91.9%) patients agreed or strongly agreed that they are confident in completing the ePRO questionnaire next time without help. After using the tablet to complete the ePRO form, there was a shift in patient preference from paper-based collection towards the electronic format. 39 (45.3%) patients either disagreed or strongly disagreed that they preferred paper-based questionnaires after completing ePRO questionnaire, compared to only 16 (18.6%) before attempting to fill in the ePRO questionnaire.

Discussion

This study was specifically designed to understand how ePROs can be implemented in routine lung cancer care. For this group of oncology patients, there is a paucity of data regarding the feasibility of introducing ePROs in outpatient clinics. The study was well received by patients, this was reflected by the high enrolment rate (82.7%) of patients approached, and if implemented in routine practise we may expect that number to increase. From a total of 86 consented patients, 27 (31%) required help from a volunteer to complete the ePRO on a touchscreen tablet. This proportion was much greater than the 10% (p < 0.0001) threshold at which we agreed that volunteers should be provided when implementing ePRO in lung cancer outpatient clinics. However, after questioning we found that the majority of patients attended clinic with their partner or other family member, whom in most cases were capable of using the touchscreen tablet on the patient’s behalf. If we exclude these individuals, only seven (8.1%) patients remained who would have relied on volunteers for assistance in order to complete ePRO; falling below our 10% threshold (p = 0.565) and suggesting that, per outpatient clinic, only a few individuals would rely on assistance from a volunteer or member of staff.

When a patient had difficulty using the touchscreen tablet to complete the ePRO questionnaire, the volunteer or researcher would offer to help the patient with verbal prompts or physical assistance. Most patients were able to complete the rest of the ePRO after receiving assistance with a couple of questions. In a few instances the ePRO questionnaire was completed on the patient’s behalf if attempts to train them were unsuccessful. Amongst patients that required help with ePRO at baseline, half did not need help at their subsequent visit, demonstrating that the need for volunteers is likely to be less with time. When implementing ePROs in clinics it may still be worth considering using volunteers at least during the initial stages of implementation to enable a smoother introduction. Once the collection of ePROs during clinics becomes standard practice, existing clinic-based healthcare assistants could be asked to take on the role of volunteers since few patients per clinic may need help. With regards to the effect of patient characteristics on ePRO completion, we found that advanced age and lack of internet access were associated with the need for assistance. Other factors including ACE comorbidity score, disease stage, PS and IMD were not predictive of assistance being required. These findings could support clinics by helping to anticipate the need for volunteers to assist with ePRO collection. Overall patients’ experiences were very positive and most participants were happy to continue using touchscreen tablets to complete ePROs. This was reflected by the shift in patient preference from paper-based collection toward electronic questionnaires after using a touchscreen tablet.

This study has a number of limitations. The sample was defined in a pragmatic way to allow the recruitment of patients during the study period of six weeks. It is therefore unlikely to be fully representative of the disease population and this could have introduced bias into the study. The majority of the study population were attending outpatient clinics and did not return within the study period; thus few individuals completed the questionnaires at a second time point. This reduced the reliability of our findings regarding the rate of ePRO completion on subsequent visits. Seven individuals declined to take part in the study due to hesitance using electronic devices. If they had consented it is likely that the number of patients requiring help would have been higher. The patient satisfaction questionnaire was designed by the study group and had not been formally validated, unlike the ePRO. It contained questions with Likert scales, which occasionally led to confusion by participants.

Our expectation was that as lung cancer patients are commonly elderly and from a lower socio-economic background, implementing ePROs during outpatient consultations may be challenging. However, our findings are consistent with literature in other oncology outpatient clinics which show that the use of ePROs is feasible and acceptable [7], [11], [12]. This is significant, as ePRO implementation would enable the collection of large volumes of high-quality toxicity data essential to developing predictive models of outcome following treatment for lung cancer [9]. Current treatment decisions for lung cancer patients are usually based on younger, fitter individuals, therefore elderly patients would benefit from the use of predictive models developed from large population-based data.

We can make a number of practical suggestions that may facilitate the implementation of ePROs in the routine clinical setting. Screening questions may aid planning by anticipating the number of volunteers that will be required, if at all. For example: ‘Do you use a device with a touchscreen at least twice a week?’ and ‘Will you be attending clinic with someone who can use a tablet? ‘. Collecting this information ahead of clinics would be valuable. Another important consideration is to provide adequate privacy within the outpatient setting. The use of booths where patients can complete the ePRO questionnaire within clinic waiting areas is an alternative to a private room which may not be available during a clinic. Posters and leaflets with ‘frequently asked questions’ for using the tablet is likely to reduce the need for assistance. For many patients the main difficulty with using the touchscreen tablet involved pressing the screen correctly to navigate the form and to select the desired responses. A possibility of improving ePRO completion is to provide a touchscreen stylus pen which may make the touchscreen tablet easier to use. Future possibilities to make ePRO collection more streamlined could involve completion on home computers and smartphones either before clinic or at regular intervals during treatment.

Conclusion

Routine collection of high quality prospective toxicity data using ePROs is crucial to collect robust data on treatment-related toxicity. Such toxicity following (chemo)radiotherapy is common, and treatment decisions for lung cancer patients are usually based on young, fitter individuals rather than on routine clinical data.

This study demonstrates that a high completion rate can be achieved during outpatient clinics with good planning. In the long term, volunteers should not be necessary as the number of patients who are unfamiliar with electronic devices such as touchscreen tablets or smart phones will decrease. Moving forward, the implementation of ePROs will allow large volumes of high quality (chemo)radiotherapy toxicity data to be collected quickly and efficiently. This is essential for the development of valid predictive models of outcome using large scale population-based data that could lead to more personalised treatments [16].

Declarations of interest

None.

Funding sources

None.