Write2me: using patient feedback to improve postconsultation urology clinic letters.

Letters between hospital clinicians and general practitioners following an outpatient clinic (OPD) consultation have generally not been shared with patients. Recent guidelines from the Academy of Medical Royal Colleges recommend that all OPD letters should be written directly to the patient. While the benefits of this approach are recognised, additional attention is required to ensure readability, accuracy and acceptability. Our aim was to improve urology OPD letters in a university teaching hospital to ensure suitability for sharing with patients over a 3-month period as measured by patient feedback. In one OPD, 71% of patients stated that they wished to receive a copy of their letter. We designed, tested and implemented two paper-based, self-explanatory prompts to ensure doctors used paragraphs and a structured letter format when dictating OPD letters. This was achieved using a 90-day improvement cycle supported by a quality improvement learning collaborative and evaluated by measurement of Flesch Reading Ease Score, use of paragraphs, use of letter structure and patient feedback. Following the implementation of the intervention, letters were sent to 120 patients and feedback was obtained from 63 patients with either a feedback postcard or telephone interview. Of the 53 patients who agreed to participate in the telephone feedback, 39 (74%) found the letter easy to understand, 49 (92%) reported it was accurate and summarised the consultation as they remembered it and 38 (72%) reported that reading the letter improved their understanding of their OPD visit. All patients said they would like to receive similar letters from future OPD consultations. This improvement report describes the implementation of an intervention to improve the quality of OPD letters and the acceptability and value of these letters to patients. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Problem

There are 3.3 million outpatient clinic (OPD) visits each year in Ireland, making OPD letters the most common form of written communication in the Irish health service.1 The primary purpose of this letter is to summarise an OPD clinical encounter for the referring doctor. Copying OPD letters to patients requires additional attention to readability, accuracy and acceptability, which may not be necessarily considered when writing OPD letters to medical practitioners and may represent a barrier to implementation.2

The context of this study was a general urology clinic in a university teaching hospital staffed by a consultant urologist and two registrars which sees approximately 1300–1400 patients annually. Within the hospital, the radiology and laboratory healthcare records are electronic, but the bulk of the patient’s healthcare record is a paper-based chart. The current process for letter production consists of a post-OPD consultation unstructured letter dictated ad libitum via an electronic dictation system to the referring doctor. The dictating doctor reviews, edits and approves the dictation via a web-based portal 48 hours later followed by postal distribution by the administrative staff.

The aim of this study was to improve urology clinic letters to ensure readability, accuracy and acceptability for sharing with patients over a 3-month period as measured by patient feedback.

Background

The 2001 Institute of Medicine’s report, Crossing the Quality Chasm: A New Health System for the 21st Century, stated that ‘patients should have unfettered access to their own medical information’ as an essential element of high-quality care.3 Traditionally, correspondence on the outcome of a patient’s consultation between hospital clinicians and general practitioners (GP) has not been shared with patients themselves. A 1991 report challenged this paradigm with an audit of OPD letters, suggesting that patients wanted to receive copies of their OPD correspondence.4 The Academy of Medical Royal Colleges recently recommended that all OPD letters written to GPs should be written directly to patients with a copy to their GP.5

The benefits of sending OPD letters to patients have long been recognised, most notably in clinical genetics.2 In a randomised trial in a haematology clinic, patients and referring doctors were very positive about letters written directly to patients.6 There is less agreement regarding patient comprehension of the content7–9 though studies have reported that patients who have received letters wished to continue to do so.9–12 Concerns, principally from doctors, that copying letters to patients would cause problems with consent, confidentiality, cost and anxiety are largely unfounded.11

Measurement

The measures selected for evaluating OPD letters were (1) Flesch Reading Ease Score (FRES), (2) use of paragraphs in letters, (3) use of predetermined letter structure, and (4) patient feedback.

Readability is a measurement of the grammatical complexity of a document and the ease with which it can be decoded. Readability calculations are based on measurements of semantic difficulty (ie, the number of syllables per word) and syntactic difficulty (ie, the number of words in a sentence). There are a number of well-validated indices which assess the readability of documents, including the FRES and the Flesch-Kincaid grade.13 The FRES assessment produces a score from 0 to 100, with a higher score indicating that the text is easier to read. A score of >60 is classified as ‘Plain English’. A specific intervention to improve readability was not tested in this study; however, readability was measured throughout as a balancing measure to ensure that changes tested did not lead to a degradation in the overall readability of letters.

The best structural device available for developing an effective OPD letter is the paragraph. The paragraph allows complex information to be broken down into more understandable sections.14 Ideas should flow in a logical manner from one sentence to another and from one paragraph to the next, resulting in a letter that is readable and complete.

The extent to which OPD letters contain the information needed by letter recipients is uncertain and a challenging measurement. The use of standardised letter formats or structures has been shown to enhance information content and communication in OPD clinic letters.15 16

OPD letters were assessed using the above measures on a weekly basis and the run charts shared with the improvement team. After each clinic, all letters were transferred to a single Microsoft Word document and FRES measured using the inbuilt capabilities of the programme. The presence or absence of paragraphs and letter structure was also determined from this document by a member of the improvement team. The measurement process took between 15 and 20 min each week.

Patient feedback was measured in two ways: (1) feedback from a four-point postcard questionnaire included with the patient’s copy of their OPD letter and (2) feedback from an eight-point structured telephone interview with patients who had received at least one OPD letter. The questions were based on a previous study of patient’s perceptions on how they would like to receive information following an OPD consultation.12

Design

An effective OPD letter suitable for sharing with patients was predefined as one with a readability FRES >60% (ie, plain English), paragraphs used >80% of the time and adherence to a structure in >80% of letters. We designed and tested a series of prompts to encourage doctors to improve the structure of their clinic letters and to use paragraphs. Interventions were iteratively designed to be self-explanatory and memorable.

The two successful interventions to improve the structure of letters were:

‘Effective letters have the: aragraphs,lain English andlan.’

‘up clinic letters in 3 points:eason for referral/review;ssessment;lan.’

These interventions were designed to be user-friendly, memorable and easy to implement in a busy clinic with the existing digital dictation system and with minimal disruption to the established clinic work flow. The improvement team consisted of a consultant urologist (PEL) and two registrars (SG, EJR). PEL designed the interventions and all team members tested them in clinic and participated in measurement. Informal feedback on the performance of the interventions was obtained after each clinic and measurements were performed weekly and shared with all team members. The use of quality improvement (QI) methodology and feedback on the tests of change was supported by the ‘betterbeaumont’ QI learning collaborative17 and the two senior authors (DAMc, FF). The change was planned as a 90-day QI cycle.

Strategy

Plan-Do-Study-Act cycle 1: establishing the demand for patients receiving their OPD letters

The aim of this Plan-Do-Study-Act (PDSA) cycle was to assess patient demand for receiving OPD letters and assess baseline measures of readability, presence of paragraphs and letter structure in existing letters. Patients receiving OPD letters directly was a departure from established practice in the hospital, however a driver for the study, based on a review of the literature, we believed patients were interesting in accessing more information about their own health.18 Of 24 patients who attended one OPD, 17 (71%) stated that they wished to receive a copy of their OPD letter when asked by the doctor at the end of the consultation. Baseline measurements of the use of paragraphs, letter structure and readability were performed on existing letters from four weekly clinics from 24 September to 22 October. The existing practice of letter dictation is an unstructured dictation ad libitum, therefore, we predicted there would be infrequent use of paragraphs and a letter structure. The baseline use of paragraphs and a letter structure ranged from 10% to 27% (figure 1) and from 18% to 35% (figure 2), respectively. The learning from this PDSA cycle prompted the team to design and test an intervention to promote the use of paragraphs and a structured letter in clinic at the time of letter dictation.

Figure 1

Run chart showing the percentage of clinic letters containing paragraphs before and after the intervention was introduced (denoted by the broken vertical line). The inset shows the 3P’s dictation intervention.

Figure 2

Run chart showing the percentage of clinic letters adhering to a structure before and after the intervention was introduced (denoted by the broken vertical line). The inset shows the RAP up dictation intervention. 3P’s, Paragraphs, Plain English and Plan.

PDSA cycle 2: designing, testing and measuring an initial intervention

The purpose of this PDSA cycle was to test an initial prompt for paragraphs (3P’s) and letter structure (WISE UP).

The initial six-point letter structure prompt was based on the acronym WISE UP:

‘to Improving Clinic Letters:hat is the reason for referral/review; htory of complaint, medical history, medications;xamination; stdies/investigations/labs;lan for treatment/followup.’

The prompt to encourage the use of paragraphs was termed the 3P’s:

‘Effective letters have the: aragraphs,lain English andlan.’

Both prompts were written documents designed to be self-explanatory. They were printed as an aide memoire for doctors to use as they were dictating OPD letters and placed prominently in each consultation room. These interventions were trialled for three clinics (12 November to 26 November) and there was an improvement in the use of paragraphs and letter structure ranging from 48% to 81% (figure 1) and from 61% to 73% (figure 2), respectively.

There was an initial improvement in the use of paragraphs in the first clinic, however, this was not sustained in the second clinic. Careful review of the measures from that particular clinic identified that one of the doctors was unsure of how to introduce paragraphs on the dictation system. This was remedied before the next clinic with some additional training on the use of the electronic dictation system. The most valuable learning from this PDSA cycle was that we needed a shorter, more intuitive letter structure prompt that was more generalisable to the variety of clinic presentations seen in a general urology clinic. The 3P’s prompt was retained and a new, shorter iteration of a letter structure prompt was designed.

PDSA cycle 3: initial patient feedback on receiving postclinic letters

The aim of this PDSA cycle was to send OPD letters to a small number of patients and obtain feedback. Anonymous postcards with four binary feedback questions were included with the OPD letter sent to the patient.

Was the letter easy to understand? Yes or No.

Was the letter accurate? Yes or No.

Did you share the letter? Yes or No.

After reading the letter, did you have a better understanding of your clinic visit? Yes or No.

This means of feedback was quick to perform but results were slow with responses taking from 1 to 5 weeks to be returned by post. From the 17 postcards sent, 10 (59%) were returned. From this PDSA cycle we learnt that patients were positive about receiving their OPD letters and prepared to give feedback. However, the postcard feedback mechanism was time consuming and unsuitable to gain feedback from a larger number of patients or to assess a greater number of questions. At this point we were confident about sending all patients their letters from future clinics and began to do so from 3 December.

PDSA cycle 4: testing an improved letter structure intervention

This PDSA cycle tested the second, shorter iteration of the letter structure prompt—‘up clinic letters in 3 points: ’ This shorter letter structure prompt was designed based on feedback obtained from two practising GPs on the elements they valued the most in clinic letters as well as feedback obtained from the QI learning collaborative members. This intervention was first tested in clinic on 3 December. Eighty-two per cent of letters from this clinic adhered to this format and this improvement was sustained throughout the next three clinics.

PDSA cycle 5: sending modified clinic letters to all patients and obtaining patient feedback

Cycle 5 aimed to assess patient feedback from a larger cohort of patients using a structured telephone interview of eight questions. The questions were based on a previous study of patient’s perceptions on how they would like to receive information following an OPD consultation.12

Did you receive a copy of the letter sent by your hospital doctor to your GP?

Was the letter easy to understand?

When you opened the letter what did you think?

Did you agree with what the letter said (did it summarise the conversation as you remembered it)?

Did you share the letter with anyone else?

Would you like to get letters from all your doctor’s appointments in the future?

Have you ever received a copy of a doctor’s letter before?

Would you like to receive copies of your doctors’ letters?

All patients from four consecutive clinics (n=120) from 3 December through to 7 January received a copy of their GP letter in the post. At this point, we had demonstrated and sustained improvements in the use of paragraphs and letter structure over a number of weeks. In addition, all doctors were made aware that their letters would be copied to patients. Patient feedback was measured using a structured telephone interview of eight questions at least 2 weeks after the letters had been posted to patients in January 2019 and was sought from 78 patients.

Results

The practice of sending patients a copy of their GP letter was not previously routine in this urology clinic or the hospital and little was known about patient preferences. All patients (n=24) in one clinic were asked at the conclusion of their consultation if they would like to receive a copy of their GP letter. In total, 17 (71%) were in favour, 4 (17%) were not interested and 3 (12%) were not asked. This led to the design of the intervention tested in this study, namely the definition and implementation of the characteristics of an effective clinic letter using prompts in combination with timely feedback to the doctors performing the dictations. The purpose was to ensure the use of paragraphs and a letter structure to improve the readability and clarity of clinic letters from the patient’s perspective.

Four measures were used in this study: percentage of letters that contained paragraphs (figure 1), percentage of letters that followed a structure (figure 2) and readability as measured by FRES (figure 3) and patient feedback.

Figure 3

Run chart showing the readability of clinic letters measured using the Flesch Reading Ease Score, before and after the intervention was introduced (denoted by the broken vertical line). 3P’s, Paragraphs, Plain English and Plan; RAP, Reason for referral/review; Assessment; Plan.

Prior to the introduction of the intervention, the percentage of existing OPD letters that used paragraphs ranged from 10% to 27% measured over four successive clinics (figure 1) in PDSA cycle 1. Following the introduction of the clinic letter prompt intervention in PDSA cycle 2, a target of 80% of clinic letters with paragraphs was set. In the first clinic following introduction of the 3P’s intervention, an initial improvement to 66% was found, however this reduced to 48% in the following clinic on 17 November. An improvement to 81% was measured at the next clinic, an improvement that was sustained over the next four weekly measures.

Figure 2 shows the use of structured letters before and after the introduction of the letter prompt intervention (represented by the broken vertical line). Prior to the introduction of the intervention, the percentage of existing clinic letters with a structure ranged from 18% to 35%. The initial letter structure intervention tested (WISE UP) in PDSA cycle 2 showed an improvement to 73% and 74% after the first two clinics, respectively. However, this reduced to 61% the following week. Learning from the initial testing of the WISE UP intervention led to a further iteration being tested in PDSA cycle 4 (the ‘RAP up’ letter prompt) which led to improvements ranging from 81% to 87% of letters with a structure measured over successive clinics from 3 December to 7 January. Examples of preintervention and postintervention clinic letters are found in online supplementary file 1.

Although a specific intervention to improve readability was not tested in this initiative, readability was measured throughout using FRES as a balancing measure (figure 3). Prior to the intervention, FRES ranged from 50 to 57 which equates to reading level of a 15–17 year-old. Following the introduction of the interventions, the FRES improved slightly initially to 63 (reading level of a 13–15 year-old) and overall remained stable throughout the PDSA cycles.

In PDSA cycle 3, patient feedback was obtained from 63 patients using two separate methods, first a postcard feedback questionnaire of four questions, and second a structured telephone interview consisting of eight questions. In total 17 patients received a feedback postcard with their clinic letter and 10 (59%) of the patients returned the postcards. All patients found the letter easy to understand and reported that it was accurate. Six (60%) patients shared the letter with a family member and all found that reading the letter gave them a better understanding of their clinic visit.

A larger patient feedback exercise was performed in PDSA cycle 5 using a structured telephone interview. In total, 120 patients received their clinic letter from four different clinics after the introduction of the interventions. Fifty-three (68%) of 78 patients telephoned agreed to participate. Thirty-nine (74%) patients found the letter easy to understand and 12 (23%) found it mostly easy to understand. Forty-nine (92%) patients felt it was accurate and summarised the consultation as they remembered it. Thirty-one (58%) patients shared the clinic letter they received and 38 (72%) patients felt reading the letter improved their understanding of their clinic visit. Only 3 (6%) patients had received a copy of a clinic letter before. All patients reported they would like to receive similar letters from future clinics.

Lessons and limitations

Sharing OPD letters with patients enhances communication, openness and trust between patients and doctors but is a departure from long-standing practice. This was also an innovation for our patients and only a very small number had received OPD letters previously. Prior to making a change we established that there was an unmet need from patients who wanted more information in the form of their OPD letters. There is ample evidence from the literature that this practice is both acceptable and preferable to patients and GPs alike.6 9–12 Although it appears an intuitive and straightforward innovation to deliver, it is important that patients derive meaningful information from letters without causing unnecessary anxiety or confusion. Writing letters directly to patients should prompt doctors to improve the way they communicate but also to rethink the way they carry out the consultation. Some may benefit from retraining and new learning to improve their skills.

There are few established guidelines for content, format or wording of clinic letters and most doctors develop clinical letter writing skills by trial and error over time.2 Having to gather one’s thoughts to construct a letter that is readily understood, informative and educational reinforces the role of the doctor as educator and helps focus the discussion on the issues that matter most to the patient. In this study, we have designed, tested, iterated and implemented two simple, self-explanatory innovations to improve fundamental aspects of clinic letters, namely letter structure and paragraphs.

The interventions designed and tested in this study needed to be useable and compatible with a busy urology clinic, without disruption to the existing clinic workflow. Similarly, measurements had to be simple, easy and straightforward to be completed weekly by the improvement team. Finally, apart from the additional cost of postage, the financial impact of this improvement initiative had to be minimal.

Over the course of this 90-day improvement cycle, we designed and implemented a useable intervention that improved the structure of clinic letters. The key strength of this study is patient feedback that confirms that patients found the resulting letters acceptable, easy to understand and accurate. Furthermore, we found that patients had a better understanding of their clinic visit having received a copy of their clinic letter. All patients we received feedback from wanted to continue to receive their clinic letters after each consultation. In the future, we plan to test writing directly to the patient and copying the letter to their GP.

There were a number of key drivers of success in this study. The study was supported by an established QI learning collaborative (‘betterbeaumont’)17 which provided access to a number of subject matter experts from a variety of disciplines but also allowed easy access to improvement experts who could provide guidance on QI methodology. As the study progressed, the learning collaborative was a means to review progress, share and synthesise learning, solidify understandings and determine next steps. The use of QI methodology, especially timely sharing of measures, encouraged the improvement team to persevere and encourage learning.

There are a number of limitations to our study. Our interventions were tested in a single site and in a general urology clinic and may not meet the needs of different specialties, a more specialised clinic or other healthcare settings. The patient feedback was primarily quantitative in nature and may miss valuable qualitative feedback. The interventions are ‘people-focused’ as opposed to ‘system-focused’ and within the hierarchy of effectiveness represent relatively weak interventions. Within time and financial constraints, we were able to quickly design, test and implement a straightforward intervention. Incorporation of these interventions into electronic healthcare records may allow forcing functions that supports sustainability. At present, the extent to which these interventions are sustainable or can be replicated in other areas is unclear.

Conclusion

This improvement study designed, tested and implemented two interventions to improve the structure and content of GP clinic letters suitable for sending to patients. Patient feedback found these letters were acceptable, improved their overall understanding of the clinic consultation and wanted to continue to receive their clinic letters in the future. The sustainability and spreadability of these interventions to other healthcare settings and specialties remain to be determined.