E-referrals: improving the routine interspecialty inpatient referral system.

Interspecialty referrals are an essential part of most inpatient stays. With over 130 referrals occurring per week at the Royal Devon and Exeter Hospital, the process must be efficient and safe. The current paper-based 'white card' system was felt to be inefficient, and a Trust incident highlighted patient safety concerns. Questionnaires reinforced the need for improvement, with concerns such as a lack of referral traceability and delays in the referral delivery due to workload. The aims of the project were to improve patient safety and junior doctor efficiency in the referral process. Through appreciative enquiry and the PDSA (Plan-Do-Study-Act) model, an electronic referral system was developed, piloted within two specialties and later expanded to others with improvements made along the way based on user feedback. The system includes novel features including specialties 'acknowledging' a referral to allow referral progress to be tracked. The system stores all referrals, creating a fully auditable inpatient referral pathway. Qualitative data indicated improvement to patient safety and user experience (n=31). Timings for referrals were measured over a 6-month period; referrals became faster with the electronic system, with average time from decision to refer to referral submission improving from 2.1 hours to 1.9 hours, with a noted statistically significant improvement in timings on a statistical process control chart. An unexpected benefit was that patients were also reviewed faster by specialties. Measuring these changes presented a significant challenge due to the complexity of the referral process, and this was a big limitation. Overall, the re-design of a paper-based referral system into an electronic system has been proven to be more efficient and felt to be safer for patients. This is a sustainable change which is being rolled out Trust-wide. We hope that the reporting of this project will help others considering reviewing their inpatient referral pathways.

Problem

Inpatients often require reviews by doctors from different specialty teams. At the Royal Devon and Exeter Hospital (a large district general hospital in South-West England delivering over 115 000 day case or inpatient admissions per year),1 the system for making non-urgent referrals consisted of writing on small pieces of white card (hence the name ‘white card referrals’) and then leaving these cards in a variety of places for the specialty team to pick up. Urgent referrals are made via phone call. There were concerns that this routine paper-based system was both a patient safety issue and inefficient.

The potential for patient safety incidents can be easily seen. On several occasions white cards were lost which effectively meant the referral had not been made, delaying patient review and potentially affecting patient care in a significant manner. A Trust incident confirmed the vulnerability of the system and demonstrated the need for change; a patient was referred from one surgical specialty to another via white card. The referring team removed the patient from their inpatient list assuming that the receiving specialty would now be responsible for their care; however, the receiving team had not yet seen or accepted the patient. The patient was then not reviewed for 2 days over a weekend and sadly died later on the Sunday. During the inquest following this case, the referral process was flagged as one of the contributing factors to this tragic incident.2

The white card system was inefficient because junior doctors had to physically take the cards to locations across the hospital, including to a building away from the main site. Additionally, there was no way of knowing whether the referral had been received or acted on. This created a great deal of extra work as the junior doctors would have to repeatedly check the referred patients' notes to see if the patient had been reviewed.

The primary aim of this project was to improve patient safety relating to routine referrals; though this would be difficult to measure specifically. A secondary aim was to improve efficiency by 20% over a 6-month period.

Background

Referrals to seek advice from or transfer care to other specialties form a large part of inpatient care. Though the decision to refer is made by a senior doctor, the actual referral is usually made by the ward junior doctor. Making an appropriate referral forms part of the Foundation Programme Curriculum as a core skill for a junior doctor.3 Via anecdotal evidence and a Trust incident, the current referral process was found to be problematic. This related to wide variation between the method of referrals (some specialties not accepting white cards) and the specific location to deliver a white card for each specialty; information which is largely unknown to new junior doctors starting at the Trust. This time-consuming process was thought to increase overall patient length of stay due to delays in obtaining specialty opinions. A literature search revealed a project aiming to tackle this issue in another Trust, where a ‘Referral Finder’ page was introduced on the Trust intranet which explained how to make referrals to different specialties.4 However, this does not streamline the process and could therefore still lead to errors as each specialty may have a different referral method. Our project does not necessarily completely fill the knowledge gap in this area, especially because we used a computer system that is unique to our Trust, however it does illustrate an original referral system developed within a Trust and raises further questions regarding patient safety and routine referrals.

Junior doctors are currently managing ever-increasing workloads—in the 2017 General Medical Council (GMC) National Training Survey, 40.84% of junior doctors in the UK rated the intensity of their work as ’Heavy/very heavy'.5 As a result of this workload, 53.56% of UK junior doctors report that they are working above their rostered hours on a daily or weekly basis.5 We felt that this complicated and time-consuming referral process only served to increase workload, and having to spend time off the ward delivering a white card means less time delivering patient care. The referral would often be left until later in the day when it was possible to leave the ward, resulting in delays in obtaining a specialty opinion. There is no published literature relating to the patient safety implications of delayed reviews by specialities; however, we believe it is a reasonable theory that delayed reviews would affect patient care in a negative fashion.

To see how other junior doctors felt about this issue we sent a questionnaire to receivers of referrals. (n=17).

How do you currently receive referrals?

What works well for you about this process?

What would you like to see changed about this process?

Have you used other processes (in other trusts) that you liked or disliked and why?

What are the core things you need when you receive a referral?

In terms of responses, we received seven variations as answers for how referrals are received (eg, white card in box by phone, gastroenterology of the day phone, etc), confirming that this process is far from standardised. When asked what could be changed there was criticism of the white card referral system: felt to be unsafe and inefficient, no way of confirming delivery/receipt of white card, and often the specialty are not given enough information. For the gastroenterology team receiving phone calls for both routine and urgent referrals, they felt it was ‘difficult for registrars to complete their ward round/clinic/endoscopy list due to the volume of phone calls’ and one registrar stated ‘absolutely everything (should be changed about the system). It should be online, this is the 21 st century’.

Another big variation noted between specialties was the intention of the white card referral. Within medicine the white card generally meant ‘We would like your advice please’; a specialty would review the patient and give their advice in the notes. In surgery however, the white card was generally used for ‘Will you take over this patient’s care’, for example, referring patients from the general acute take to a specific specialty (in medicine a different system is used for specialty allocation). The intention of the referral was not specifically stated on the referral, further adding to this ambiguity. This difference in opinion of the white card intention could lead to difficulties in knowing which team the patient actually remains under (as in the previously mentioned Trust incident). We felt, therefore, that the intent of the referral needs to be specifically stated. Under ’Delegation and Referral' in the GMC’s Good Medical Practice guidance, it is stated that when referring, the health professional must pass on ’relevant information about the patient’s condition and history' and ’the purpose of transferring care and/or the investigation, care or treatment the patient needs'.6 This suggested the need for a standard layout meaning that the referring doctor must fill in the patient’s current condition, past history, and importantly, the purpose of the referral.

Baseline measurement

We collected two types of data; quantitative relating to the time from referral to review, and qualitative data in the form of a questionnaire to assess the usability of the systems and the perception of safety (as we were not able to directly measure the impact on safety).

In order to establish the need for this project and confirm anecdotal concerns, we devised two questionnaires; one to the senders and the other to receivers of referrals. The questionnaire that we sent to referrers is shown with the modal answer below: (n=17)

How long does it take, on average, to complete a white card referral? ‘10-20mins’.

What time frame would you expect a patient to be reviewed following referral? ‘Next day’.

Is the patient reviewed within the expected time frame? ‘Most of the time/Some of the time’.

Statement: The current white card referral system is reliable. ‘Disagree’.

Statement: The current white card referral system is a risk to patient safety. ‘Agree’.

What is your overall satisfaction with the current white card referral system? ‘Dissatisfied’.

How do you think the system could be improved? Answers included ‘Hard to work out if being followed up’, ‘Unclear if referral has been made unless you make it yourself’, ‘Sometimes due to time on ward difficult to leave to hand them in’, ‘Risk to patient safety if go missing’.

Although this was somewhat limited by a small number of responders, it aligned with our own concerns about the system.

We also collected quantitative data to demonstrate the inefficiency of the system. Our measures included the time of the decision to refer the patient (for example, as documented on ward round), the time the referral was made and the time of review by the receiving specialty. These data were collected by all members of our project team on the specialties that they were currently working on; every time a referral was made, these data points were recorded on a shared spreadsheet and continuously plotted onto run charts. These baseline measures formed the basis for the outcome measures; time of decision to referral made (referral time), and time of referral made to time patient reviewed (review time). We aimed to continuously record these data throughout the project, however the data collection changed slightly after the electronic system was introduced, as instead of being limited to the specialties that we were currently working on, the electronic system recorded all the timings for all referrals which we were able to extract and plot.

A further questionnaire was sent electronically to senders and receivers of e-referrals following our main intervention. This was used as a surrogate marker to measure the effect on patient safety.

Please state which specialty/specialties you have made an electronic referral to.

Do you feel the electronic referral process is easier to use compared with the previous white card system?

Approximately how long does it take you to complete an electronic referral?

Do you feel that the electronic acknowledgement of the referral by the specialty is an improvement compared with the previous white card process?

Do you feel the electronic referral process, using the SBAR (situation, background, assessment, response) format is appropriate?

Do you feel that the electronic referral process improves patient safety?

Would you like the electronic referral system to be available across all specialties?

How do you feel the electronic referral process could be improved?

Do you feel the electronic referral process is easier to use compared with the previous white card system?

Do you feel the electronic referral process, using the SBAR format is appropriate?

Do you feel that the ability to acknowledge the referral electronically is beneficial compared with the previous white card system?

Once completing the referral task, how long would you like to be able to access/view the original electronic referral?

Do you feel the electronic referral process improves patient safety?

Do you feel that the electronic referral system would be appropriate across all specialties?

How do you feel the electronic referral process could be improved?

Design

We conducted a process flow analysis of the referral process and identified its pitfalls (figure 1). We brainstormed with staff involved in all parts of the referral process; including using appreciative inquiry to ‘dream’ of the perfect referral system. In line with other hospital systems, it was felt that using an electronic system would be preferable. This would remove the need to physically transport the referral, and also provided traceability. To determine whether this would be possible, the team met with the Trust IT department to discuss our ideas and create a pilot version of the e-referral system. The IT department had recently created an inhouse-designed computer system for out-of-hours handover which had already been implemented in the Trust, so we decided to incorporate the e-referral system into this. Key features included an SBAR format for better quality referral information. Each box contained a prompt regarding what information should be inputted, so for the ’Response' section of the referral we could prompt the referrer to state whether they wanted advice, a review or for the specialty to take over care of the patient (figure 2). The referrer must input a telephone/bleep number so they can be contacted should the specialty need further information, and each referral shows the most up-to-date blood results which auto updates via a link to the pathology system. Other features included an ‘acknowledged’ button which would be clicked by the reviewer, helping to track the referral’s progress. Each specialty has their own page which forms a list of their received referrals for ease of use. An advice box was created on this page for each referral so the reviewing team could make suggestions such as ‘order an up to date CXR and we will review’ with the referrer expected to check for any advice updates. This specialty referral screen is demonstrated in figure 3.

Figure 1

A process map demonstrating the old paper-based white card referral system, with it’s main pitfalls identified.

Figure 2

A screenshot demonstrating the referral screen with the SBAR (situation, background, assessment, response) format.

Figure 3

A screen shot demonstrating the referral list as it appears to the receiving specialty. Clicking the red ’Requested' option changes this to the orange ‘Acknowledged’. A further click once the patient has been seen changes this to a green ‘Reviewed’.

Alongside this pilot system we drew up a new process map that outlined the new, more streamlined referral process. The system was piloted and adjusted as required via multiple PDSA (Plan-Do-Study-Act) cycles. The intervention would remain sustainable through becoming the main referral system across specialties. The project was started in November 2015. Data were collected over a 6-month period from December 2015 to June 2016, and the roll-out of the electronic system is ongoing still.

PDSA cycle 1

The pilot version of the e-referral system was reviewed by our team before going live. A mock referral was made through the system offline to model the e-referral process and was studied by the project and IT teams. Improvements were made based on the appearance and some features of the system, for example, changes were made to ensure the referral was not editable and when advice was written into the advice box, it could not be changed.

PDSA cycle 2

The e-referral system went live as a pilot for two specialties: healthcare for older people and urology. Training was given by the project team on a one-to-one basis across the Trust and also via an information booklet on the Trust intranet. Contact details were given for troubleshooting or problems via regular Trust-wide emails. To study the impact of the pilot, qualitative interview feedback was gathered from these specialities on their experience in using the system. This was very positive and they were keen to continue using the system. We also reviewed our efficiency measures and were able to make improvements based on all this information (eg, changing the layout to make viewing referrals clearer).

PDSA cycle 3 onwards

Following a successful pilot of the e-referral system, further specialties were added one by one. This was done in a stepwise fashion with support for the team reviewing the referrals and Trust-wide email updates to inform users of the new specialties receiving e-referrals. Each of these specialty implementations acted as a further PDSA cycle, as each specialty was met with to plan how to introduce e-referrals, and then each implementation was studied in terms of feedback and efficiency in order to make any further changes. A formal questionnaire was distributed to evaluate user experience and thoughts regarding the system’s safety.

Examples of actions from the ‘study’ phase of PDSA cycle 4 onwards included:

Addition of a notification system: the receiving specialty can now receive an alert via email, SMS or bleep to let them know that a referral has been received.

Addition of a ’delete referral' function. The user must specify the reason for deletion (eg, wrong patient was selected for referral or referral no longer required).

Addition of the ’Reviewed between' function where the specialty can filter down to referrals in a specified time period, for example, looking at all referrals made over a 6-month period for audit purposes.

Results

Referral time and review time were calculated for each referral and plotted on run charts as shown (figures 4 and 5), with the point of initial introduction of the electronic system marked. The data were analysed using statistical process control charts, and an I-chart was used.

A run chart demonstrating the time taken from decision to refer to referral made (referral time).

A run chart demonstrating the time taken from the referral being made to the patient being reviewed (review time).

Our primary aim was to improve patient safety which was measured using the surrogate marker of a questionnaire to doctors. When asked ’Do you feel that the electronic referrals process improves patient safety?', 71% responded ’Strongly agree' (n=31). Also, the use of the ‘acknowledged’ and ‘reviewed’ functions means that we can see that none of the referrals made so far on the system have been missed by the receiving specialty.

Our secondary aim was to improve efficiency. The average referral time initially showed a statistically significant increase from 1.5 hours to 4.09 hours for the first 10 referrals after introduction, with a subsequent statistically significant decrease to 0.45 hours. This initial increase may have been a slight ‘adjustment period’ and also hopefully aided by some of the changes that we made to the system. The overall average measured referral time decreased from 2.1 hours to 1.9 hours. In the questionnaire to the senders of e-referrals, the most popular answer to ’Approximately how long does it take you to complete an electronic referral?' was '<5 min' (70%) followed by '5–10 min' (26%), n=27. We incidentally noted a statistically significant decrease in review time, from 21.75 hours to 18.55 hours.

Lessons and limitations

The main lesson for us was that it is possible to initiate change from a junior level to standardise an important and essential hospital pathway. There is the potential for a cost saving for the Trust, as a faster review by a specialty could result in a decreased length of stay for that patient.

Before embarking on this new referral system, it was hard to estimate the scale of this project. There was no reliable way to measure the number of white card referrals made at the Trust per week. At the point of having 12 specialties on the referral system, we were averaging 130 referrals per week, which illustrates the significant impact of this project.

One of the key limitations was trying to gather enough data in order to truly understand the problems associated with the old system, and to gain feedback on the new system. The measurements made regarding timings of referrals were a very crude measure of a complicated process. It should also be noted that the amount of data collected was small given the size of the hospital and relied on people volunteering to give feedback and therefore the conclusions drawn from this may not represent the true impact of the referral system. In terms of collecting the qualitative data, we received lots of anecdotal comments regarding the system in person rather than our questionnaires being filled in. If these data had been captured in a questionnaire, it could have strengthened our findings. From the perspective of the project team, we are full-time junior doctors, therefore it was difficult to give this project the significant amount of time that it required. As we have been trying to draw conclusions from this project, we have learnt the real importance of collecting a large amount of good quality data, and this is a limitation of this project despite the successful implementation of a new referral system.

As demonstrated on the run charts, there was huge variability in the length of time taken for referrals to be made and for patients to be reviewed, which we believe is multifactorial due to the complex nature of interspecialty referrals. First, the referral time can be broken down into a number of factors. This time is affected by the delivery of white card/writing of the e-referral, and when the decision was made. For example, if the decision to refer is made at the start of the ward round then often the request won’t be made until after the ward round finishes which can cause a delay of hours. The review time will be affected by the receiving specialty; each specialty has different workloads and staffing levels affecting how soon they can review a patient. In some specialties, referrals are reviewed as part of the ward round so they are checked first thing in the morning, and then not again for another 24 hours. Referrals over the weekend also differ between specialties; most teams will only pick up referrals from Monday to Friday as they are not separately staffed over the weekend. Most of the surgical teams however will pick up referrals over the weekend as each surgical specialty has its own consultant ward round. To account for this ’weekend effect' in the data we have subtracted 48 hours from all referrals made on a Friday and the patient reviewed on the Monday to try and improve consistency in the data.

Conclusion

In conclusion, we have successfully implemented an effective method of routine interspecialty referrals which is perceived to be a safer system by the doctors using it. We have proven e-referrals to be more efficient in terms of both time taken to make a referral and also for the patient to be reviewed. The improved junior doctor efficiency and possible reduced length of stay for the patient should have an associated cost reduction for the Trust.

This project does have significant limitations attributed to the complexity of the referral processes. Finding parts of the referral process that are measurable and accurately represent patient safety in particular is a challenge and therefore limits the measurement of improvement. If this project were repeated, more specific measures for a more direct comparison of the referral processes would be used.

The sustainability of the project has been assured by involvement of the Associate Medical Directors at the Trust to make e-referrals the standard process of routine referrals. Future steps to further improve the e-referral system would include specialty-specific referral pages rather than the standardised SBAR format to improve the information transmitted to each specialty.

If others were to undertake a similar piece of work, then the involvement of IT and the early engagement of stakeholders at the Trust would be key. Any team who wishes to improve their interspecialty referral system will hopefully learn from our project which presents an original referral system design which has been successfully implemented with positive reviews.