External Quality Assessment beyond the analytical phase: an Australian perspective.

External Quality Assessment (EQA) is the verification, on a recurring basis, that laboratory results conform to expectations for the quality required for patient care. It is now widely recognised that both the pre- and post-laboratory phase of testing, termed the diagnostic phases, are a significant source of laboratory errors. These errors have a direct impact on both the effectiveness of the laboratory and patient safety. Despite this, Australian laboratories tend to be focussed on very narrow concepts of EQA, primarily surrounding test accuracy, with little in the way of EQA programs for the diagnostic phases. There is a wide range of possibilities for the development of EQA for the diagnostic phases in Australia, such as the utilisation of scenarios and health informatics. Such programs can also be supported through advances in health information and communications technology, including electronic test ordering and clinical decision support systems. While the development of such programs will require consultation and support from the referring doctors, and their format will need careful construction to ensure that the data collected is de-identified and provides education as well as useful and informative data, we believe that there is high value in the development of such programs. Therefore, it is our opinion that all pathology laboratories should strive to be involved in an EQA program in the diagnostic phases to both monitor the diagnostic process and to identify, learn from and reduce errors and near misses in these phases in a timely fashion.

Introduction

Pathology is a crucial clinical tool, estimated to contribute to 60-70% of all critical decisions involving patient treatment (). Despite this potential value, the Carter review estimated that 25% of pathology requests are unnecessary or inappropriate (). Furthermore, CareTrack Australia examined the appropriateness of care provided in Australia for 22 common conditions and demonstrated that only 57% of patients received what was regarded as appropriate care (). The cost of diagnostic test services in Australia rose to $5.25 billion in 2013. For pathology services, this represented an increase of 81% in the decade to 2013 (). This rise has led to major concerns about the substantial costs and risks associated with unnecessary tests and incorrect result interpretation. For pathology services to be of value, the correct ordering of tests and interpretation of results is crucial. This is the responsibility of the treating clinician, and as such, can be considered as the pre- and post-laboratory or diagnostics phase ().

External Quality Assessment (EQA) is the verification, on a recurring basis, that laboratory results conform to expectations for the quality required for patient care (). However, Australian laboratories tend to be focussed on very narrow concepts of EQA, even though the significance of pre and post laboratory errors is now widely recognized. This can be partly attributed to the fact that laboratories are largely sample, as opposed to patient, oriented.

If the broader concept of Quality Management Systems (QMS) is primarily aimed at meeting customer requirements and enhancing customer satisfaction, then it is clear that the quality of the product of a laboratory is taken as a given by referrers, and enhancing clinician, patient and payer satisfaction extends far beyond the traditional boundaries of the laboratory. One way to improve the quality of service laboratories provide is to extend laboratory based EQA programs to the requesting and reporting phases, which are outside the current scope of pre-analytical, analytical and post-analytical EQA programs (). Whilst it is recognised that some countries have made steps in this direction, this is far from widespread, and currently lacking in Australia. The pre-pre-analytical phase, which is primarily composed of test ordering, and the post-post-analytical phase, which is primarily composed of test result interpretation, can be regarded as the diagnostic phases (as opposed to the analytical phases) and sub-divide it into a pre-laboratory and post-laboratory phase (Figure 1). This terminology has been chosen to remove the laboratory as the focus of the process and shift it back to the referring clinician.

Figure 1

The phases of laboratory testing

In addition to introducing the diagnostic phase terminology, this opinion paper aims to set out the reasons why pathology laboratories and diagnostic medicine needs a way of monitoring these phases more effectively, with particular reference to the Australian situation. It is the belief of the authors that an EQA program could be developed to identify, learn from and reduce these errors and near misses in a timely fashion, and this will be explored in this article.

Pre and post-laboratory errors

A study conducted by the American Academy of Family Physicians reported that participants submitted 590 event reports with 966 pre- and post-laboratory errors. Pre-laboratory errors occurred in ordering tests (12.9%) and implementing tests (17.9%), while post-laboratory errors occurred in reporting results to clinicians (24.6%), clinicians responding to results (6.6%), notifying patient of results (6.8%), general administration (17.6%), communication (5.7%) and other categories (7.8%). Charting or filing errors alone accounted for 14.5% of errors. While patients were unharmed in 54% of events, 18% resulted in some harm, and harm status was unknown for 28%. Furthermore, these errors led to a range of other adverse consequences including time and financial consequences (22%), delays in care (24%), pain/suffering (11%) and adverse clinical consequence (2%) (). Therefore, the impact of these pre- and post-laboratory errors demonstrates a pressing need to identify the sources of these errors to facilitate the development of interventions that can reduce the error rate.

While there has been a vast amount of research to identify pre-laboratory error quality indicators (), there are also significant pre-laboratory errors that we believe have not been included in these indicators. One of these areas of omission is the proportion of patients who are not adherent to a pathology request. It has been estimated that in Australia, approximately 20-30% of patients who are given a pathology request form in the community do not have this request completed (). There are multiple reasons for this non-compliance including language barriers in communication, low socioeconomic status and poor health literacy such as forgetting important appointments, losing pathology forms and not showing up to or attempting to reschedule the appointment. This has potentially far greater impact on patient treatment than the analytical phase.

Primary care in Australia is the responsibility of General Practitioners (GPs). The impact of the aforementioned non-compliance to test requests has required GPs to adopt complex workflows to remind patients of the need to have an appropriate test before the next appointment. Anecdotally, the non-adherence rates are of a similar range in hospital outpatient clinics. The cost to the community of these wasted appointments is significant. This is one reason why point of care testing, which enables laboratory tests to be performed at the patient location as opposed to a laboratory, may have significant benefits for both patients and health professionals.

Hickner et al. reported that GPs described uncertainty in ordering laboratory tests in approximately 15% of diagnostic encounters (). The task of selecting appropriate diagnostic testing is challenging for clinicians, in part because of the sheer volume of choices. For example, there are currently over 850 different pathology tests for which the government will reimburse patients in Australia. Therefore, methods to improve this workflow could lead to a significant improvement in the quality of pathology services.

Sikaris has identified the importance of the post-laboratory phase and how it is subject to error, such as the misapplication of appropriate and accurate test results through cognitive failure (). Laboratory tests and their misinterpretation are still an important contributor to misdiagnosis because of the emphasis put on laboratory testing for diagnosis and monitoring decisions. In the post-laboratory phase the quality of the final report, including its reference intervals, clinical interpretations and notifications based on knowledge from laboratory specialists, should support clinical decision-making. It has been reported that incorrect interpretation of diagnostic tests accounts for up to 37% of malpractice claims in primary care and emergency departments ().

Audit and dissemination of best practice plays an important role in managing the quality of results interpretation. While audits are not the preferred option for the Australian situation, primarily due to the great distances required to make on-site visits, a number of international studies have examined the quality of results interpretation in general practice. Skeie et al. found that 22% of Norwegian GPs misclassified changes in haemoglobin A1c (HbA1c) for patients with diabetes mellitus (DM) and that the vast majority of GPs assumed that analytical quality was better than it really was (). The finding of this study are supported by that of Thue and Sandberg (), who analysed clinician expectation of analytical performance in relation to current analytical performance specifications, finding that clinicians are generally uninformed of the capability of analytical performance. Three subsequent Norwegian studies performed EQA of GP’s interpretation of pathology results and showed general agreement in critical differences (CDs) for blood glucose and HbA1c, with variation in the perceived risk to patients of a severe bleed (, ). There was also a large variation in CDs for uric acid and in International Normalized Ratio (INR) interpretation for warfarin monitoring (, ). Kristoffersen et al. found that GPs across 13 countries overestimated the risk of ischemic stroke and bleeding in people treated with vitamin K antagonists (VKA) by 2-3 times (). The results of these studies suggest that guidelines for these conditions may be either unknown or impractical. Hellemons et al., who found that guidelines around the use and interpretation of albuminuria in patients with DM were poorly followed in general practice further support this ().

EQA programs for the pre- and post-laboratory phases

The Institute of Medicine (IOM) report entitled “To Err is human” identified the types of error that arise in the diagnostic process, namely: failure to employ indicated tests; use of outmoded tests or therapy; failure to act on results of monitoring or testing; treatment error in the performance of an operation, procedure, or test; error in administering the treatment; error in the dose or method of using a drug; avoidable delay in treatment or in responding to an abnormal test; inappropriate (not indicated) care; preventive failure to provide prophylactic treatment; inadequate monitoring or follow-up of treatment; other failures of communication; equipment failure; other system failure (). Many of these errors, such as failure to order a test, wrong test ordered and failure to recognise urgency, are amenable to the type of EQA programs that are used in laboratory medicine. However, developing EQA programs for the pre- and post-laboratory phases will require consultation and support from the referring doctors. The format of the programs will also need careful construction to ensure that the data collected is de-identified and provides education as well as useful and meaningful data.

The EQA program we propose could take the form of a series of patient scenarios where a response would be required. The Interpretative Comment programs of The United Kingdom National External Quality Assessment Service (UKNEQAS) or Royal College of Pathologists of Australasia Quality Assurance Programs (RCPAQAP) are two such examples; however, the referring doctor would be the participant. The results would be analysed and reported back with a guideline-based response, concordance, and group performance (see Appendix 1 for an example of such a report). Sikaris has described these concepts in terms of a medical laboratory, but they are translatable to a referring doctor model (). The cases will need to be carefully chosen, however, so that the suggested interpretations in terms of what tests to order based on a given clinical scenario or what treatment to suggest has a strong evidence base in both current literature and current clinical guidelines.

EQA programs for information technology

The purpose of a pathology report is to communicate the results of the test in a clear and unambiguous manner. It is clearly a patient safety issue if a report is misread in a way that may lead to an incorrect understanding of the results. Hickner et al. found that 8.3% of GPs had uncertainty towards interpreting results (). Challenges included different names for the same test, tests not available except as part of a test panel and different tests included in panels with the same names. While this has been addressed in some countries, it remains a prominent issue in the Australian setting. Additionally if a report is difficult to read, there can be valuable time lost in trying to correctly identify the key elements of the results.

In the modern era, doctors commonly receive pathology reports from a range of different laboratories. Examples include tests requested by a specialist, results from a hospital, results obtained while travelling interstate or overseas or results from a different laboratory attended by the patient for convenience or other reasons. Clearly, uniformity of reporting formats amongst laboratories can be beneficial in making the review of pathology reports easier and safer, irrespective of the testing laboratory (). Clear and consistent reporting is vital to support safe pathology interpretation. Guidelines aimed at improving the effectiveness of testing have been the subject of standardisation between medical groups for a significant period of time (). While there has been focus on communication using electronic systems (, ), paper reports remain in common use and rendered reports (e.g. portable document formats (PDF) or Pathology Information Transfer protocol (PIT) formats) are still widely used in practice. In 2013, the Royal College of Pathologists Australasia (RCPA) published an initial Standard. A group known as the Australian Pathology Units and Terminology Standardisation Project (APUTS) wrote the draft, and after public feedback, edited and finalised comments. The Standards and Guidelines were released in 2014 to assist in the requesting and reporting of pathology ().

It is now important that conformity to the aforementioned guidelines and standards be monitored. This can be done through a form of EQA for reports. An EQA organisation is part of the request-result cycle and hence is in a position to perform quality assurance on the laboratory result when the laboratory sends a result back to the EQA. The units, format, reference interval and comment are all a part of the EQA result and hence can be treated as part of the EQA program (Figure 2).

Figure 2

Laboratory messaging in context
The figure illustrates the responsibilities of the laboratory and its interaction with external parties within the pathology test-request-report cycle.

The Pathology Information, Terminology and Units Standardisation (PITUS) Informatics EQA Project aims to build a system to enable electronic requesting and reporting for an existing RCPAQAP EQA program. The electronic messages involved in the process will be assessed for compliance and conformance to relevant Standards from National Pathology Accreditation Advisory Council (NPAAC), Australian Standards and the RCPA. A rendered PDF version of the report will also be generated and assessed against the format, rules and rendering conformance requirements of the APUTS Standard.

Use of ICT to support EQA in diagnostic phase

Advances in Health Information and Communications Technology (ICT) means that ICT can be used to support EQA and assist clinicians when both ordering and interpreting pathology test results. This combination has the potential to significantly reduce errors in the diagnostic phase of pathology testing. In the pre-laboratory phase, Computerised Physician Order Entry (CPOE) systems allow clinicians to enter laboratory orders directly into a computer system. This can support EQA systems that aim to reduce the chance of errors associated with illegible handwriting, patient identification and specimen collection and labelling, key sources of error in the pre-analytical phase. CPOE systems can also be coupled with clinical decision support, assisting the clinician to choose the most appropriate tests for their patient. However, there is still scant evidence around the impact of such systems on patient outcomes (). In one of the few studies to date, Georgiou et al. demonstrated that the implementation of a CPOE system led to a reduction in errors associated with mislabelled, mismatched and unlabelled specimens (). CPOE also led to a reduction in both the number of tests being ordered per episode of patient care and laboratory turnaround time. These findings have a direct impact on patient safety and quality of care as a subsequent study showed that for every five additional tests, emergency department length of stay increased by 10 minutes and that each 30-minute increase in turnaround time was associated with a 17 minute increase in emergency department length of stay ().

In the post-laboratory phase, ICT can be used to support EQA systems that aim to standardise result reporting, reduce the number of missed test results and improve the quality of pathology result interpretation. The use of ICT to generate standardised pathology result reports, such as through mobile applications, may decrease the risk of incorrect result interpretation due to the clinician being unfamiliar with the report layout. However, the impact of such systems remains to be fully explored. Electronic test acknowledgement systems, which require the clinician to acknowledge that they have viewed a pathology result, can also be used to reduce the number of missed test results (). Finally, electronic decision support systems can also be used in the post-laboratory phase to assist clinicians with adhering to guideline or protocol based care. However, in Australia, evidence surrounding the impact of such systems on patient outcomes remains weak (). Therefore, while further patient centric studies are required to fully assess the impact of Health ICT on patient safety, ICT combined with EQA has the potential to reduce errors in the diagnostic phase of the pathology process.

Conclusions

For pathology services to be of value, the correct ordering and interpretation of results is crucial. Both of these factors are the responsibility of the treating clinician, and as such, can be considered as the pre- and post-laboratory phase (). Errors occur throughout the testing process, most commonly involving test implementation and reporting results to clinicians. While significant physical harm caused by these errors is rare, adverse consequences for patients are common.

It is a recommendation of the IOM that accreditation organizations have programs in place to ensure competencies in the diagnostic phase, and to identify and learn from diagnostic errors and near misses with an aim to reduce these errors in a timely fashion (). EQA programs are proven ways of achieving these goals and have the experience and processes in place to provide the required platforms. To this end, we believe that widespread implementation of such programs, supported by ICT, is the next stage of identifying and reducing error in the diagnostic phase of the request-result cycle.