Lessons learned from the introduction of an electronic safety net to enhance test result management in an Australian mothers' hospital.

This study describes the implementation and impact of an electronic test result acknowledgement (RA) system in the Mater Mothers' Hospital in Brisbane, Australia. The Verdi application electronically records clinicians' acknowledgement of the review of results. Hospital data (August 2011-August 2012) were extracted to measure clinicians' acknowledgement practices. There were 27,354 inpatient test results for 6855 patients. All test results were acknowledged. 60% (95% CI 59% to 61%) of laboratory and 44% (95% CI 40% to 48%) of imaging results were acknowledged within 24 h. The median time between report availability and acknowledgement was 18.1 h for laboratory and 1 day 18 h for imaging results. The median time from when a result was first viewed to its acknowledgement was 7 min for laboratory and 1 min for imaging results. The longest recorded time to acknowledgement was 38 days. Electronic RA provides a safety net to enhance test result management. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

The efficient management of laboratory and imaging test results by clinicians has been an area of concern across healthcare systems internationally.1 The failure to review and follow-up test results can have significant consequences for the quality of care, including missed diagnoses and suboptimal patient outcomes.2 When measured as a proportion of tests the failure to follow-up test results has been shown to range from 20%3 to 62%4 for inpatients, and up to 75% for patients treated in the emergency department.5

Many of the problems associated with test follow-up have been related to paper-based systems where the tendency for test results to be delayed, misplaced, or lost can be attributed to pressures of time, change of staff, or as a consequence of patient transfer.2 6 Often an existing mechanism does not exist for confirming that a result has been seen.2 Health information technology (IT) systems are seen as a means for improving the safety and quality of test management.7–9 There are electronic applications which can track pending test results at hospital discharge,10 deliver result alerts,11 or employ tracking systems to document acknowledgement and subsequent clinical actions.12

The introduction of IT into hospital settings has been neither rapid nor uniform, and evidence of its effectiveness in improving test result follow-up remains inconsistent.2 9 13 In many instances IT implementation has resulted in the establishment of hybrid systems involving both paper and electronic systems,14 which require clinicians to coordinate activities across paper (eg, hand written requests) and electronic systems (eg, computerized results reporting), thereby adding to the risk of missed test results.15 16 This study describes the introduction and impact of an innovative, electronic results acknowledgement (RA) system in the Mater Mothers’ Hospital in Australia.

Methods

Setting

The study was conducted in the 249-bed Mater Mothers’ Hospital, Brisbane, Australia, which in 2011 recorded 9525 births, 15 246 inpatient discharges, and 66 667 outpatient encounters.17 In 2010 the implementation of the IP Health Verdi software18 enabled the development of an RA system which allowed clinicians to electronically document the review of test results.

Intervention

The implementation of RA involved a number of strategic and organizational considerations to secure clinical acceptance across the hospital.19 On a strategic level the health service established a governance structure (including an executive-level steering committee led by the hospital's executive director) to ensure that the new system complemented the requirements of the health service. On an operational level it required decisions about how the new RA system would work within existing IT structures and clinical work practices.20 A decision was made to establish a clinical portal system (Verdi) with the capacity to integrate multiple sources of information into a single point of access for clinicians. The electronic request form (ERF) (see figure 1) for ordering tests and the RA functionality were developed in-house to integrate with Verdi, and were introduced in January 2011 after a series of trials. The primary RA interface consisted of an electronic tick box requiring clinicians to indicate that a result had been reviewed. It was accompanied by an ‘unacknowledged test’ list of results awaiting review and a corresponding electronic alert indicating that the patient had unacknowledged test results for over 3 days.

Figure 1

Screen shot of the electronic request form (ERF).

The RA solution was built around a set of procedures based on the health service's test management governance model (shown in figure 2) which begins when a medical officer completes an ERF, a hard copy of which is then printed and transferred to the laboratory or imaging departments, whose systems do not interface with the RA system. If the result is not electronically acknowledged within 3 days, the notification/escalation process is set in motion so that as each day passes, email or pager alerts are sent, initially to the clinical unit's medical officer designated to undertake RA (day 4), and then escalated to the clinical unit support (administration or medical supervisor) to check that the ordering clinician is still correctly listed and/or is not unexpectedly absent) (day 5), clinical unit director (day 7), and the divisional director (day 10). The RA system complements existing medical imaging and laboratory department procedures which require direct phone notification to the referring clinician of a life threatening or significantly abnormal result.

Figure 2

Depiction of the process involving test generation, result acknowledgement, and the governance and escalation process.

Initial pilots of the RA system highlighted issues related to workflows in different clinical settings, which led to specific alterations to the system. For instance, prior to the introduction of RA, screening tests at the hospital were ordered by midwives and doctors during routine consultations and the results vetted by midwives. Most of these results were normal and filed in patients’ records for communication at the next appointment, while abnormal results were referred to the responsible registrar. The initial design of the RA system did not include this midwives’ ‘triaging’ step and would have involved a huge increase in workload for doctors. This prompted the inclusion of a special midwife ‘triage’ RA function to account for the previous vetting task. The system also incorporated an automatic acknowledgement of normal results (ie, numeric only results that fall within a specified range considered ‘normal’) through the application of a decision support algorithm.

Statistical analysis

Imaging and laboratory test results were extracted for all inpatient episodes between August 2011 and August 2012. A total of 28 077 records were extracted (689 for imaging (2.5%) and 27 388 for laboratory (97.5%)). Automatically acknowledged results were not included. Records were excluded from analysis in cases of multiple acknowledgements (involving any change in results after initial acknowledgement) (580 records), missing episode identifier (90 records), missing dates (7 records), and time stamp inconsistencies which occurred in situations where a previous time stamp had been manually overwritten (51 records). Analysis used SAS V.9.3 and was performed separately for laboratory and imaging results across the major Australian-refined diagnosis-related groups (DRGs).21 Descriptive and survival analysis techniques were used to compare time distributions.

Results

There were 27 354 inpatient test results (679 for imaging and 26 675 for laboratory) relating to 6855 patients (involving 7647 episodes of care) across the August 2011 to August 2012 period. The findings showed that all results in the hospital had been acknowledged, with 60% of laboratory tests (n=24 458; 95% CI 59% to 61%) and 44% of imaging tests (n=586; 95% CI 40% to 48%) acknowledged within 24 h (figure 3). The longest time to acknowledgement was for one imaging result which took 37 days and 20 h. The average time between report availability and acknowledgement for laboratory results was 1 day 15 h, with a median of 18.1 h. Of the laboratory results, 19.2% were acknowledged outside of 3 days and 1.1% outside of 10 days. The average time between report availability and acknowledgement for imaging results was 2 days 19 h, with a median of 1 day 18 h. For imaging results, 34.9% were acknowledged outside of 3 days and 3.7% outside of 10 days.

Figure 3

Distribution of time from report to acknowledgement by service.

The time distribution between test result availability and acknowledgement was significantly different between laboratory and imaging (log-rank χ2=117.7; p<0.0001). Within laboratory results, there was a significant difference in the time distribution of the DRG (table 1; log-rank χ2=58.0; p<0.0001). Similarly for imaging results, there was a significant difference in the time distribution of the DRG categories as shown in table 1 (log-rank χ2=145.8; p<0.0001).

Table 1

Percentage of test results by diagnosis-related group (DRG) reviewed and acknowledged outside three days

Source	Category	% Time to acknowledge >3 days	95% CI
Imaging	Cancer DRG	33.7	19.4 to 48.0
	Gynecological	31.9	24.2 to 39.6
	Obstetric	25.7	19.8 to 31.7
	Neonate	41.5	33.9 to 49.0
	Overall	33.0	29.2 to 36.8
Laboratory	Cancer DRG	12.3	10.5 to 14.2
	Gynecological	20.5	19.2 to 21.8
	Obstetric	17.5	17.0 to 18.1
	Neonate	28.9	26.9 to 30.8
	Overall	18.8	18.3 to 19.3

There were significant differences in test acknowledgement rates within the three-day benchmark by patient DRG category. Laboratory test results for patients with an obstetric or cancer related DRG were significantly more likely to be acknowledged within 3 days compared to tests results for other DRGs. Patients with a neonate DRG were significantly less likely to have laboratory test results acknowledged within 3 days compared to the other patient groups (see table 1).

The percentage of pathology reports with an acknowledgment outside of 3 days was significantly higher on Fridays at 34.4% (95% CI 33% to 36%). The other days ranged from 10.1% (95% CI 9% to 11%) on Mondays, to 24.5% (95% CI 23% to 26%) on Thursdays. For imaging reports the highest percentage occurred on Fridays at 63.4% (95% CI 56% to 71%) but was not significantly different to Saturdays at 40% (95% CI 22% to 58%) or Sundays at 46.2% (95% CI 19% to 73%). Other days ranged from 14.4% (95% CI 8% to 21%) for Tuesdays, to 43.4% (95% CI 35% to 52%) for Thursdays.

Results acknowledgement was primarily undertaken by medical staff, particularly registrars (table 2). Average time from results availability to first viewing of laboratory test results was 19.1 h, with a median of 3 hours. The longest time was 19 days 22.1 h. Imaging results were on average first viewed 1 day and 23.7 h after report availability, with a median time of 20.8 h. The longest time was 37 days 20 h. For 56.5% of all tests (56.1% of laboratory results and 69.6% of imaging results), the first person to view the result also acknowledged the result. The average time between first viewing and acknowledgement was 19.3 h for imaging and 20.1 h for laboratory results. The median times from first viewing to acknowledgement were 7 min for laboratory test results and 1 min for imaging results.

Table 2

Percentage of test acknowledgements by clinical position

Position	Imaging%	Laboratory%
Registrar	33.3	25.3
Intern	21.4	24
Junior house officer	18	28.9
Senior house officer	11.8	6.3
Senior medical officer	9	5.7
Surgical assistant	3.5	1.8
Principal house officer	0.3	1.3
Other (eg, staff specialist, GP, visiting medical officer)	0.2	0.3
Midwife	0	2.7
Unknown	2.5	3.7
Total	100	100

Discussion

The findings of this study show that after the implementation of the RA system, all inpatient test results within the hospital were recorded as acknowledged, a result never previously reported from reviews of test follow-up using paper-based systems.2 The electronic RA system provides a safety net to ensure that all results can be monitored in real-time, representing a significant advance on previous test management practices, which often relied on time consuming retrospective paper-based audits to monitor, identify, and rectify problems.2 13 22 23

The decision to introduce the RA system involved strategic and operational deliberations about how to ensure it functioned seamlessly as part of the existing health IT infrastructure, alongside consideration of the costs and potential risks of work practice changes.19 The experience of implementing the RA system highlights three interrelated factors for the implementation of safe electronic test management systems. The first is the need for clear levels of clinical responsibility for test follow-up and escalation.2 24 25 At this site it included the reporting of test results directly to the responsible medical officer able to take the required clinical action,26 along with an escalation process in the case of that officer's unavailability, or if the result was not acknowledged within the required time frame.27 Second, the implementation plan was based on an assessment of the capacity to integrate clinical data electronically across information systems. Poor integration of electronic systems in healthcare settings is a potential hazard,28 with important implications for the effective transfer of clinical information.29 30 Third, the Mater incorporated trials of the system which were designed to foster staff engagement and gather valuable feedback about the new IT system.10 11 30–32

Limitations

This study did not include data on the prevalence of unacknowledged test results prior to the establishment of the RA system which would have provided a valuable baseline comparison. The study was also unable to investigate doctors’ clinical actions after viewing results.27 This is a developmental area which the Mater Health Service is intent on addressing in the future.

Conclusion

The problem of delayed or incomplete test result follow-up is an area of continuing patient safety concern for hospitals in Australia and internationally.33 The scope of the problem was underscored by a 2011 report of the Clinical Excellence Commission in New South Wales, Australia, which reported that 11% (3/27) of incidents with serious outcome (eg, patient death) and 32% (24/75) of clinical incidents with major patient-related consequences (eg, loss of bodily function) were associated with results that were either delayed or not reviewed.34 The Mater Mothers’ Hospital has demonstrated the potential of health IT to contribute to innovation in work practices when accompanied by resilient management and clear clinical governance.26 35 In the future, across all hospitals, it should be reasonable to expect that clinicians, healthcare managers, and administrators will be able to immediately identify any existing problem related to a delay in test follow-up and to undertake the required clinical action at the time it is most needed to the patient.36