Insights into the experiences of patients with cancer in London: framework analysis of free-text data from the National Cancer Patient Experience Survey 2012/2013 from the two London Integrated Cancer Systems.

OBJECTIVE: To shed light on experiences of patients with cancer in London National Health Service (NHS) trusts that may not be fully captured in national survey data, to inform improvement action plans by these trusts.
DESIGN: Framework analysis of free-text data from 2012/2013 National Cancer Patient Experience Survey (NCPES) from the 2 London Integrated Cancer Systems. SETTING AND PARTICIPANTS: Patients with a cancer diagnosis treated by the NHS across 27 trusts in London. MAIN OUTCOME MEASURES: Free-text data received from patients categorised into what patients found good about their cancer care and what could be improved.
METHODS: Using Framework analysis, a thematic framework was created for 15,403 comments from over 6500 patients. Themes were identified across the London data set, by tumour group and by trust.
RESULTS: Two-thirds of free-text comments from patients in London were positive and one-third of those related to the good quality of care those patients received. However, the majority of comments for improvement related to quality of care, with a focus on poor care, poor communication and waiting times in outpatient departments. Additionally, 577 patients (9% of those who returned free-text data in London) commented on issues pertaining to the questionnaire itself. Some patients who experienced care from multiple trusts were unclear on how to complete the questionnaire for the single trust whose care they were asked to comment on, others said the questions did not fit their experiences.
CONCLUSIONS: NCPES free-text analysis can shed light on the experiences of patients that closed questions might not reveal. It further indicates that there are issues with the survey itself, in terms of ambiguities in the questionnaire and difficulties in identifying patients within specific trusts. Both of these issues have the potential to contribute to knowledge and understanding of the uses and limitations of free-text data in improving cancer services. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

The data come from a large national survey, with over 15 403 free-text comments for London from more than 6500 patients with cancer. There was a 75% increase in free-text comments within the London trusts’ data since the last survey in 2011/2012. Comments identify aspects of why and how cancer care was good, and whether or not it required improvement, from the patients’ perspective.

Free-text comments provided information beyond what was measured. They helped to increase understanding of experiences that were not covered by closed multiple choice questions within the National Cancer Patient Experience Survey (NCPES).

The research could be strengthened by understanding more about the demographics of patients who completed the free-text questions and how individual free-text responses related to relevant survey responses. Free-text data were provided to the authors separately from other data sets.

While free-text data are useful in understanding some aspects of patients’ experience, they may not be able to evidence the depth of response that other forms of qualitative data, namely, interviews and ethnographic studies, might be able to provide.

Some patients may have completed the questionnaire (and free text) with more than one trust in mind. Some patients commented that these hospitals were outside the London area. Data could be strengthened with a full understanding of patients’ pathways.

Background

In the UK, there are currently (2015) an estimated 2.5 million people who have had a cancer diagnosis.1 One in two people will develop cancer at some point in their lives.2 People living with cancer have a variety of support needs, some of which are not being fully met at the moment.3 Improved patient experience affects patients’ well-being, their ability to relate to professionals and to self-manage. Indeed, it has been shown that having a good experience of care matters as much to most patients as clinical outcomes and safety do.3 4 Moreover, in recent years, patient experience has become an important indicator of healthcare performance.5 Patients’ views are gathered alongside clinical quality and safety when assessing quality of care.6

Within the UK, there has been great emphasis on understanding and improving patient experience of cancer care,7 and on addressing the variations within that experience occurring across tumour groups and different regions of the country.8 One way of accessing patients’ views, tracking performance over time, and examining where and why patients are having poorer experiences, is via the National Cancer Patient Experience Survey (NCPES)—an annual survey that asks patients about their cancer care and treatment. The NCPES began in 2010, after the 2007 ‘Cancer Reform Strategy’ detailed the need for a new survey programme.9 The NCPES poses 70 questions on patients’ experience of their cancer treatment ranging from topics such as whether patients were told the name of the Clinical Nurse Specialist in charge of their care, to how decisions on their treatment were made.10 Since 2010, the NCPES has been administered by Quality Health on behalf of the Department for Health.11

The 2012/2013 survey included all adult patients who were treated for cancer between 1 September and 30 November 2012 in the 155 National Health Service (NHS) trusts across England that provided adult acute cancer services. The survey covered both inpatients and day case patients.9 It was administered by post and sent to patients’ home addresses. Over 68 000 patients with a primary diagnosis of cancer took part. Results from the survey formed a national report as well as reports for each of the trusts involved.12

Across London, cancer services are organised within two Integrated Cancer Systems, London Cancer (LC) and London Cancer Alliance (LCA). These integrated systems come together to streamline cancer services and pathways across London trusts. In 2012/2013, the NCPES data were used to show that, for the third consecutive year, overall patient experience for London was poorer than for the rest of England.13 The 10 poorest performing NHS trusts were ranked by Macmillan, based on the number of questions (out of 63) for which patient scores fell within the bottom 20% of results for all trusts. Nine out of the 10 lowest-scoring NHS trusts in England were in the capital.14

A number of hypotheses have been put forward to explain this ‘London effect’,14 including patient characteristics, environment and organisation, and staffing and treatment issues. Evidence suggests that younger patients or patients from ethnic minority groups report worse experiences of care,15–18 and in this way, the demographic make-up of London (with more younger people and a higher proportion of ethnic minority populations) might affect overall scores. A Canadian survey further indicates that urban settings evoke less positive assessments of care experience than rural environments.19 If this is the case, then, as the capital city of England, London might be unfavourably compared with smaller English towns or more rural areas. A further hypothesis is that London's higher concentration of teaching hospitals means that its trusts treat patients with more complex problems and that these patients are more critical of their experiences, or that the focus on teaching is at the expense of patient care.20 Alternately, an analysis of the relationship between NCPES data and the NHS staff survey suggests that poor performance might connect to issues with staffing levels, staff turnover and poor staff experience.14 21 However, secondary analysis of the 2011/2012 NCPES data set suggested that the impact of patient case-mix on the results was small, and that the higher proportion of teaching hospitals was unlikely to be an important source of geographical variations in patient experience of cancer16 22 leading to further questions about the specific reasons for the London results.

One potential way of answering these questions is to consider how the survey itself might have some relationship with the variation in results. Although national surveys such as the NCPES help to measure healthcare services over time, to benchmark and to identify how and where quality of services differ,8 they also rely on standardisation—posing questions that are general enough for all respondents to answer. What may therefore be missed are the questions that are most appropriate to individual respondents’ concerns and issues.

The NCPES also includes qualitative free-text questions at the end of the survey. Previous analyses of the NCPES data have not tended to make use of these free-text data. However, free text—or open comments—can start to access those individual responses that may be excluded in survey data. Indeed, they may get a little closer to the type of data provided in qualitative interviews, where patients can provide more detailed descriptions of experiences than is possible to convey in closed questions, and a space to communicate the issues that are important to them is provided.6 In the absence of interview data, which may be time-consuming to collect and are not generalisable, free-text data may shed light on specific aspects where quality improvement initiatives need to be targeted.6 18 This paper therefore reports on analysis of free-text comments from the 2012/2013 NCPES to focus on particular facets of individual patients’ experiences that were used to inform action plans and improvement strategies for London.

Methods

Data

Free-text data from the NCPES 2012/2013 were provided to the research team by all trusts within LC and LCA. Free-text questions at the end of the survey ask what patients found good about their cancer care and what could be improved, and a third question invites ‘other’ comments (box 1).

Was there anything particularly good about your National Health Service (NHS) cancer care?

Was there anything that could be improved?

Any other comments?

In response to the three free-text questions in the survey, over 6500 patients in London made 15 403 comments about their cancer care, and added further comments about the questionnaire itself.

Analysis

In total, 15 403 comments were analysed using Framework analysis (Ritchie and Spencer23). The initial framework was developed for the 2011/2012 free-text analysis by two researchers—TW and AR—based on a review of the patient experience literature and a preliminary analysis of the data. In this process, both researchers independently looked at comments from three trusts. Following identification of potential themes, the researchers discussed and compared the themes and TW devised the framework using tables of data in Microsoft Excel. The framework was then piloted by the research group with data from the first trust. A few minor changes were then made before using the framework as a basis for analysing all the data. Data from the first question, ‘Was there anything particularly good about your NHS cancer care?’, were inserted into a framework for ‘positive’ responses while data from the second question populated a framework containing ‘comments for improvement’. Data from the third question ‘Any other comments?’ were found to fall into either a positive comment about care or a comment for improvement; hence, all the comments from this question were subsumed into either the ‘positive’ or ‘improvement’ framework. This positive/negative binary is consistent with the rest of the survey data, which are also reported in this way.18 Within the framework, researchers also coded findings to be able to look at services by tumour group as well as by trust. Data were cleaned to check that the comments assigned in the positive or improvement boxes did actually belong to those categories and, if they did not, they were correctly re-assigned.

Findings from the 2011/12 free-text analysis were fed back and subsequently used to inform the work plans of the Pan London Patient Experience Work Group for 2013.

LC Integrated Systems requested a similar analysis on the 2012/2013 data set. To ensure the 2011/2012 framework was fit for the purpose of the 2012/2013 data set, the research team independently conducted preliminary analysis on the data from one trust. The research team then regrouped to agree on a slightly adjusted framework. Each trust's data were put on to a ‘positive’ framework, which consisted of seven themes across the horizontal axis and different tumour groups within that trust on the vertical axis. The same was done with the ‘improvement’ framework; this time with 17 themes on the horizontal axis. There was some cross-over and natural relationship between the themes—in the themes for improvement between poor care and poor communication. To make sure there was consistency of approach, each data set was examined by two researchers to ensure a consensus was reached as to how comments were attributed to each theme.

At the outset of the analysis, whether to count the free-text data by number of comments or by number of respondents was discussed. Across the three free-text questions, one patient would often make several different comments about his/her experience. Owing to limited resources and a large data set, it was not possible to count for subthemes. The research team therefore decided to try to retain a sense of each theme's relative importance by indicating the diversity of comments within each theme by counting comments as opposed to respondents. If a participant made two very different comments for question 1: ‘Was there anything particularly good about your NHS cancer care?’, these comments would be separately attributed to the relevant themes. If the comments were on the same issue, they would be treated as ‘one comment’. As with the previous year, data were cleaned to ensure that comments were correctly distributed to the positive or improvement framework. The relationship between the number of free-text comments and the number of respondents can be seen in table 1. An average of two-thirds of patients (66%) who returned their NCPES provided free-text responses across the 27 trusts; some questionnaires included several comments written across the three free-text questions and others were left blank.

Table 1

Free-text respondents and comments

Trusts across LCA and LC that provided data	Patients sent the NCPES (n)	Respondents completing NCPES (n)	Respondents providing free-text comments (n)	NCPES respondents providing free-text comments (%)	Free-text comments about cancer care—positive and for improvement (n)	Free-text respondents saying questionnaire unsuitable (n)	Free-text respondents saying questionnaire unsuitable (%)
1	61	41	27	65.9	70	7	25.9
2	106	43	29	67.4	69	3	10.3
3	1655	744	491	66.0	1167	15	3.1
4	66	44	36	81.8	103	6	16.7
5	763	390	254	65.1	605	33	13.0
6	907	474	311	65.6	689	27	8.7
7	244	128	76	59.4	154	5	6.6
8	1139	616	424	68.8	976	30	7.1
9	145	65	60	92.3	115	2	3.3
10	1555	873	589	67.5	1100	58	9.8
11	3144	1815	1244	68.5	3165	101	8.1
12	353	198	123	62.1	198	22	17.9
13	161	85	57	67.1	167	4	7.0
14	294	149	104	70.0	279	11	10.6
15	486	244	154	63.1	376	11	7.1
16	258	155	112	72.3	347	9	8.0
17	1003	563	125	22.2	296	11	8.8
18	1784	867	616	71.0	1413	61	9.9
19	495	267	156	58.4	435	16	10.3
20	146	71	49	69.0	139	12	24.5
21	1474	705	440	62.4	961	51	11.6
22	246	148	102	68.9	221	14	13.7
23	199	99	64	64.6	235	7	10.9
24	494	289	208	72.0	496	19	9.1
25	685	363	256	70.5	602	17	6.6
26	1184	626	390	62.3	963	24	6.2
27	98	41	29	70.7	62	1	3.4
Total	19 145	10 103	6526		15 403	577

LCA, London Cancer Alliance; NCPES, National Cancer Patient Experience Survey.

Alongside the overall framework and analysis, data were also analysed for each of the 27 trusts and a one page report for each was produced detailing the most prominent themes within their data. Data were also analysed for each tumour group across the sites to inform tumour working groups’ action plans across the LC Integrated Systems.

Findings

Of the 15 403 free-text comments from patients for the London trusts within the framework, 10 232 were positive comments about patient experience and 5171 were comments for improvement. Positive comments therefore made up 66.4%, or two-thirds, of total comments. Comments for improvement contributed 33.6%, or one-third, of the total. This equates to a 6.2% point increase towards the positive direction in the free-text comments from 2011/2012 to 2012/2013 survey—reflecting an overall national picture of improvement in cancer care.4 24

Positive comments

Across the London trusts, there was broad consensus among patients as to what was good about the service. The framework for positive comments had seven themes encompassing all the comments relating to cancer care within the free-text data. Consistent with previous studies, positive comments tended to be more generic than those for improvement, focusing on overall experience rather than specific elements.25 However, after piloting the data it was felt that there was a clear distinction between the seven key themes (table 2).

Table 2

Positive themes

Themes	Number of comments made/percentage of positive comments	Sample comments
1. The quality of care	3343 (33)	“NHS cancer care is excellent”“We have always received first class treatment”
2. Particular services/teams	2765 (27)	“Both chemotherapy and radiotherapy teams were amazing”
3. The quality of professionals	2700 (26)	“They were so thorough. Everyone treated me like a person and not a number”
4. Speed and efficiency	688 (7)	“Everyone acted so quick: appointment, scans, operation”
5. Support and attention	627 (6)	“Everyone was sensitive, articulate and emotionally supportive”
6. Being part of clinical trials	67 (0.6)	“Speedy assessment and good monitoring during clinical research”
7. Food	42 (0.4)	“The food menus—explaining which choices would ‘build you up’ or ‘gluten free’, were very good”
Total	10 232 (100)

NHS, National Health Service.

Within the free-text responses, the quality of care patients received made up the greatest number of positive comments (33%). ‘Quality of care’ related to care at an overall level—where patients referred to the ‘care’ they received and how it had positively impacted on their experience. This theme was differentiated from ‘particular services/teams’; this is where individual departments or services were highlighted in terms of their contribution. Comments related to services ranging from chemotherapy and radiotherapy units, to complementary therapies and ambulance drivers:

“The aftercare service, lymphoedema, acupuncture and psychology services are important to name. I attended ‘Living After Cancer’ workshop which was a great day; informative and therapeutic.”

A further identifiable theme was that of the ‘quality of professionals’ who were mentioned in 26% of the positive comments. Professionals made a difference to patients’ experience, helping them feel secure in their treatment, and managing and coping with the cancer diagnosis:

“I was treated with dignity and respect, and was told everything in a way I could understand.”

Indeed, there were over 5500 positive comments made about staff and their work within the free-text data. The free-text responses also indicate how and why staff made such a positive contribution in terms of their manner, kindness and attentiveness:

“Doctors, nurses, radiotherapy, chemo staff, receptionists, auxiliary personnel have been kind, caring, compassionate and polite. This made a stressful time more bearable.”

This kindness was accompanied by comments on the knowledge and expertise of professionals:

“A centre of excellence. All people involved with me remain (sic) patient, kind and compassionate; yet also efficient and knowledgeable.”

Two further specific themes were identified on the framework from 2011/2012 and were seen again in the 2012/2013 data. ‘Speed and efficiency’ related to where referrals and treatment had been noticeably and vitally quick for patients, and ‘support and attention’ related to the specific words patients used to define how they had felt personally supported and treated as an individual. There were two small adjustments to the ‘positive’ framework, with the addition of new themes from the 2012/2013 data: ‘being part of clinical trials’—where patients mentioned that they valued and had benefited from being on a clinical trial, as well as a smaller number of positive comments about the hospital food patients received.

Comments for improvement

Although positive patient comments comprised two-thirds of all the free-text data, one-third of the comments were less encouraging and suggested areas where cancer care in London needs to be improved (table 3). These comments were included within a framework of 17 themes, demonstrating a wider range of issues for improvement than the more generically framed positive comments. While there was much cross-over and many relationships between these themes, it was felt that these 17 areas best captured the content of the comments for improvement.

Table 3

Themes for improvement

Themes	Number of comments made/percentage of comments for improvement	Sample comments
1. Poor care	866 (17)	“Nobody wanted to help me to bed. Two ladies were arguing in front of me because neither wanted to look after me. It was hard to witness that”
2. Poor communication	518 (10)	“Often the level of English of some doctors and nurses was very poor. They could not explain my treatment, the medicines they were giving me or tell me why—very frightening”
3. Waiting times: in OPD	512 (10)	“When you are feeling dreadful, you've got a lot of pain, waiting up to 3 h is excruciating”
4. Information	423 (8)	“Explaining more on the side effects and duration [could be improved]”
5. Understaffed	398 (8)	“The nurses were not lazy just stretched. There should be more regular nurses than bank nurses and agency nurses”
6. Liaison between departments	303 (6)	“Transfer of information between departments within the hospital can be improved”
7. Environment/hospital site	295 (6)	“The oncology clinic needs to be a brighter and more cheerful place to wait”
8. Support	283 (6)	“More help to cope with depression caused by the return of my cancer after a remission of 6 years would have been a great help”
9. GP	265 (5)	“GP practice needs a cancer expert as I don't feel they understand treatment prescriptions and requirements”
10. Delays	218 (4)	“Referral waiting period, for example, physiotherapy goes on for months”
11. Waiting in CDU	214 (4)	“It has been a regular feature of my treatment that my chemotherapy medication was not available at the appointed time, and was delivered as much as 2/3 hours late”
12. Access to doctors	212 (4)	“Completely impossible to contact my consultant”
13. Food	211 (4)	“If you stay long term, the food leaves a lot to be desired”
14. Resources	172 (3)	“MRI services are not adequately available. Once I waited as an inpatient 7 days for a scan”
15. Parking	118 (2)	“Parking…at least £2 an hour. I have been an outpatient for 17 months visiting the hospital twice a month at least. Us pensioners can't afford it”
16. Pharmacy	112 (2)	“When waiting as an outpatient for medication is unacceptable in the amount of time you have to wait for it to be dispensed. Always an hour or longer”
17. Discharge	51 (1)	“It took a significant amount of time to be assessed and discharged from hospital post operation”
Total	5171 (100)

CDU, Chemotherapy Day Unit; GP, general practitioner; OPD, Outpatients Department.

The detail of these comments reveals a range of issues and some clear insights into how patients feel they are being let down. In this way, free text helps to identify specific actions for trusts that can be acted on and that can feel ‘owned’ by staff in the particular services concerned.6 In the free-text comments about communication, for example, communication is not just negatively rated, instead, focused comments are made:

“The doctor should look at the patient first and then the scan.”

This kind of comment suggests how and where small changes can be focused to make a considerable difference to patient experience.

Poor care contributed to the greatest number of negative comments in the 2012/2013 data set (17%). These comments related to issues including: patients being treated with a lack of dignity, medical staff demonstrating a lack of attention with regard to knowing the case, patients being made to wait for essential needs and depersonalisation:

“I felt degraded and humiliated; chewed up and spat out! Emotionally insecure and I'm still suffering the consequences.”

Poor care recurred as an issue at night and at weekends. This subtheme of poor care was also identified in analysis of the free-text data from the Welsh Cancer Patient Experience Survey, which highlighted the need for improvements in out of hours care.25 In the London data, this issue was linked in some comments to understaffing and/or complaints about agency staff.

“It's a different place at night and on the weekend. It's full of agency staff, some do not understand or speak English, they are rude and talk over you in their native tongue to their friends.”

Indeed, comments about understaffing particularly related to a lack of permanent staff as well as the perceived poor quality of agency staff, and comprised 8% of comments for improvement.

As there were many comments about waiting, these were split within the framework to help identify departments where issues with waiting were occurring; this was particularly useful when the data were looked at for each individual trust. Outpatient departments emerged as the areas most often cited for improvement in terms of waiting times.

Poor communication—as the second most prominent theme for improvement, and forming 10% of negative comments—was linked closely with poor care and staffing issues especially in terms of the number of staff who were reported to have a poor level of spoken English. Although many of the communication comments focused on diagnosis, examples were given across the pathway in terms of the manner and content of communication with patients. One patient reported:

“I was told I had cancer by the chemotherapy nurse who didn't realise I hadn't been told.”

Another explained: “Medical terminology needs to be translated so the ordinary person can understand.”

The details of how and why patients experienced poor communication emerge clearly in the free-text data. Although a further detailed full sub-analysis of each theme has not been conducted on the data, some clear areas arose within the communication theme. These included insensitive communication such as bad news broken inappropriately and patients not being informed about their diagnosis or treatment. Other communication issues stemmed from poor administration such as letters not being sent; patients being sent details of the wrong appointment; conflicting information being given by healthcare professionals and information not being given in plain English.

The communication theme also related to two other overlapping themes: ‘information’ and ‘liaison between departments’. Within both of these themes, poor communication played a role in patients’ sense that information was confusing or lacking, and that departments within the hospitals were not working in an integrated manner.

Furthermore, there were some themes that focused on more practical and functional issues, including issues with parking and problems with the hospital site or environment, which negatively impinged on patients’ experience of their care and emerged as distinct themes within the analysis.

As the analysis was being fed back to hospital trusts for their improvement action plans, the theme around general practitioner care was not a focal point of the free-text analysis. However, an analysis of all data on primary care was subsequently conducted and will be reported separately.

Ambiguity of questionnaire

While the overall Macmillan analysis using NCPES data rightly identifies key issues that need addressing to improve cancer care and highlights, in particular, the ‘London effect’,14 more work is needed to understand exactly how patients reported their experiences. This free-text analysis shows that out of the total 6526 patients who returned free-text data on the questionnaire across London, 577 patients (9%) made comments relating to the ambiguity of the questionnaire as a whole or had concerns about completing it. Across the London trusts, the percentage of people unsure about elements of the questionnaire ranges from 3% to 25.9% of those people who returned free-text data per trust (see table 1). These comments related to two main areas of concern: first, that patients are asked to complete the questionnaire for one trust when they may have been diagnosed and subsequently treated in more than one place; second, that the questionnaire lacks clarity in some areas, making it difficult for patients to complete.

The complex patient pathway

A theme coming through strongly in the data was that some patients had attended different trusts, from diagnosis through to treatment, and so filled out the questionnaire with different places in mind. Previous research has identified that the complexity of care pathways might mean that patients experience fragmented care across different hospitals.15 This is exemplified in the text written by a patient who was completing the survey for one of the London trusts:

“I consider the 2–3 hour wait, to see a specialist at the [hospital A], unacceptable. Surely patient management can be improved. Staff at the breast clinic and subsequently [hospital B] were brilliant and I am grateful for the care that they provided. Unfortunately, the [hospital C] administration staff and pre-op admissions were not of the same calibre.”

These types of comments were seen throughout the data, with patients often reporting of treatment in more than one place or of being transferred between hospitals:

“It was interesting to be diagnosed at one hospital and then have to attend another for treatment (chemo) and then a third for radiotherapy.”

In many cases, patients tried to detail those questions they had answered in relation to the trusts as a corrective to the closed questions the survey had asked them: “My replies to Q10, 11 and 12 refer to my local hospital not to [London] trust D”.

“I was sent to Trust E by [Trust outside of London]. Trust E was outstanding. The negative answers relate to [“Trust outside of London”] not Trust E.”

Free-text data further indicate that some patients may have received care from other trusts in England and were subsequently referred to a London trust to carry out certain specialist treatments or diagnoses. This indicates that some of the data in the questionnaire have potentially been attributed to London when patients have completed the survey in relation to their earlier experience in another trust based elsewhere in England:

“My association with Trust F was short. I was diagnosed there and spent a week having tests then transferred to Trust G. 90% these answers are about Trust G not Trust F which is the only hospital in the letter.”

Lack of clarity in the questionnaire

Analysis of the free-text comments relating to the questionnaire identified some potential areas where the quantitative measures might need correction. These responses were assessed outside of the framework about patients’ positive and negative experiences of cancer care. For some patients, the NCPES questions did not reflect their experience and the appropriate answer was not available:

“I don't feel this survey is well thought through and doesn't give the necessary options to some questions for my case.”

“The questionnaire is not really suitable for my experience; most of it doesn't apply to me.”

As has been identified in the previous literature looking at surveys and their relationship with free-text comments, a survey may not be able to address the whole patient pathway and therefore free text might identify areas that could be expanded in future surveys.6

For other patients, the survey was confusing, which meant that they were not sure if they had filled it out correctly: “I may have filled out too many boxes, sorry.”

Other patients remarked that the questionnaire might not best represent their opinions: “I have been hesitant about completing it, these tick boxes could be misleading and the wrong conclusions drawn.”

These comments, which centred on the questionnaire not being fit for purpose, led some patients to explain why they had left some of the boxes unticked:

“I have not answered some of the questions as they do not fit my care”, and “I'm sorry not to fill this out properly. I seemed to get lost with repetitiveness.”

These comments did not tend to detail specific problematic questions but instead indicated a general sense of unease with the questionnaire as a whole.

Discussion

The NHS Outcomes Framework 2014/2015—which seeks to drive NHS quality improvement—outlines that the NHS should collect patient experience information and use it to improve services.26 Trusts are advised to use NCPES data to inform improvement initiatives.4 A potential benefit of survey data is that they can compare experiences of different patient groups and assess changes over time.6 The NCPES has been seen as a national driver for quality improvement leading to better experience for patients with cancer.4

However, survey data may well not be able to articulate how patients’ experiences can be improved, and patients suggest that it is not always possible to fit their experiences into ‘predetermined tick box survey questions’.27 Where patient experience is being measured, the evidence base suggests that collecting a combination of both qualitative and quantitative data should be considered.28

When qualitative interviews are not possible due to scale and resources, free-text data can complement quantitative analysis29 as well as providing a more nuanced type of data closer to that provided in patient narrative accounts.6 18 The open nature of the free-text comments means that patients can address the issues most pertinent to them—patients lead how they communicate their experiences across the whole journey—as opposed to answering to general, standardised measures of experience.

It has been identified that the most important reason for collecting, analysing and managing patient experience data is how that information can lead to change.30 In order to facilitate change, free-text data should be analysed and fed back to staff in the relevant services.4 6 Free-text data can help to reveal patients’ individual voices, which provides motivation to improve and suggests how services can be developed. Positive comments help staff to appreciate what they are doing well, how they are valued and the impact positive patient experience can have.4 Two-thirds of the free-text data relating to experiences of London patients with cancer consisted of positive comments, and, in comparison to the 2011/2012 free-text data, there was a shift in the positive direction. Where care is good, and staff communicate well and provide support for patients, the often-difficult experience of cancer is made to feel more manageable.

Comments for improvement help clinicians to understand the particular aspects of services that need to change. They tend to be more detailed and specific than positive comments. This is useful in understanding where and why things have gone wrong as well as providing ideas to guide the next steps in service improvement.4 Findings from this analysis of free-text data for London were fed back to inform work plans for the individual tumour groups within LC Integrated Systems. Such an approach brings organisations together to address issues, devise strategies and work across complex cancer pathways.4 These findings translated into targeted improvements creating real change in practice, for example, clinics were asked to make sure they were starting on time as it had been identified that waiting times were often impacted by clinics starting late. The Picker Institute's report for Macmillan highlights how issues with staffing might relate to poorer patient experience.21 This was substantiated by some of the comments for improvement in the London free-text data for 2012/2013, which specifically focused on: understaffing, staff with poor English language skills and problems with agency staff. On many occasions, this was linked to poor care—the theme that appeared most prominently in comments for improvement. As a result of these findings, working groups set targets to reduce overall use of agency staff with a focus on reducing weekend or out of hours agency staffing in particular.

If the NCPES does not cover all parts of the patient experience, free-text data might suggest where the survey can be refined or expanded in the future and therefore how the survey can be better developed for quality improvement purposes.6 31 The design of the 2012/2013 survey (and carried into the 2013/2014 survey) is limiting in terms of the difficulty in identifying patients with specific sites. This seems to be particularly problematic for London, with patients reporting a complex care pathway with variable experiences across different trusts. Future research could be strengthened with the inclusion of questions about pathways of patients with cancer through different hospitals to allow for the variation in fragmented versus consolidated care pathways.15 Given that patients completing the NCPES for London hospitals report a less positive experience compared with hospitals elsewhere in England, the free-text data suggest it is critical to evaluate the whole trajectory of patient care from diagnosis through treatment, and to understand when and what role London hospitals have played in that care. Within this set of free-text data, some patients reported they had excellent care in London—but had experienced poor care before their referral to a London trust; hence, the survey that only asked them about their experience of a London trust, was unsuitable. Indeed, the analysis from London suggests that there are a significant number of people who indicate that their comments may not match the areas with which they are identified in the data. This is of particular concern when data from the NCPES are being used both to inform policy and to plan service improvement. It is important that patients are identified with their correct areas of care, in order to avoid wasting resources responding to data that may be incorrect or misattributed. It further suggests that the whole picture for London might be more nuanced and complex than the survey data reveal.

The NCPES free-text questions were well used by patients in London, with over two-thirds of those responding to the survey also providing free-text responses. The number of comments for London also increased by 75% from the 2011/2012 survey, indicating the value of including free-text questions within the questionnaire and providing a large and rich data set of more than 15 400 individual comments.

In order to increase the usefulness of free-text data, an analysis could be more closely aligned and related to the relevant closed questions data, so that all parts of the survey work together to suggest how improvements can be best targeted. As this piece of research was based on free-text data provided to trusts and detached from the survey data, this type of analysis was not possible; however, such an analysis would help to create a fuller picture of patient experience for patients with cancer in London and across the rest of the country, to target improvements and inform action plans to turn patient experience feedback into positive change.