Use of a laboratory information system driven tool for pre-signout quality assurance of random cytopathology reports.

BACKGROUND: Quality assurance (QA) programs in cytopathology laboratories in the USA currently primarily involve the review of Pap tests per clinical laboratory improvement amendments of 1988 federal regulations. A pre-signout quality assurance tool (PQAT) at our institution allows the laboratory information system (LIS) to also automatically and randomly select an adjustable percentage of non-gynecological cytopathology cases for review before release of the final report. The aim of this study was to review our experience and the effectiveness of this novel PQAT tool in cytology.
MATERIALS AND METHODS: Software modifications in the existing LIS application (CoPathPlus, Cerner) allow for the random QA of 8% of cases prior to signout. Selected cases are assigned to a second QA cytopathologist for review and all agreement and disagreements tracked. Detected errors are rectified before the case is signed out. Data from cases selected for PQAT over an 18-month period were collected and analyzed.
RESULTS: The total number of non-gynecological cases selected for QA review was 1339 (7.45%) out of 17,967 cases signed out during this time period. Most (1304) cases (97.4%) had an agreement in diagnosis. In 2.6% of cases, there were disagreements, including 34 minor and only 1 major disagreement. Average turnaround time of cases selected for review was not significantly altered.
CONCLUSION: The PQAT provides a prospective QA mechanism in non-gynecological cytopathology to prevent diagnostic errors from occurring. This LIS-driven tool allows for peer review and corrective action to be taken prior to reporting without delaying turnaround time, thereby improving patient safety.

INTRODUCTION

Quality assurance (QA) in recent years has gained much attention in the field of anatomical pathology.[1] Numerous approaches have been attempted to improve practice, including audit systems based upon retrospective and prospective reviews.[23] In cytopathology, mandated QA programs currently involve the review of gynecologic cases per clinical laboratory improvement amendments (CLIA) of 1988 federal regulations.[4] Per the CLIA 88 Final Rule, cytology laboratories in the USA are required to prospectively rescreen 10% of their negative Pap test cases that are examined by each cytotechnologist. Some of these Pap tests must be selected randomly and others by targeting high-risk women (e.g. those with a history of a squamous intraepithelial lesion of the cervix). Published results have shown that 10% rescreen of negative Pap tests does detect a few false-negative cases.[5] The rescreen must be carried out prospectively, so that diagnostic errors can be corrected before a report is issued. A similar rescreening mechanism of normal Pap tests is also a quality control (QC) requirement in cytology laboratories in other countries.[6] For non-gynecology cases in cytopathology, retrospective cytologic–histologic correlation is the mainstay of QC.[7] Based on this cytologic–histologic correlation process, a non-gynecologic error frequency of 11% has been reported.[8]

The advantage of prospective reviews is that it can immediately impact patient care if a diagnostic error can be averted. In such cases, corrective measures are possible before a case gets signed out with a misdiagnosis. If an error is uncovered once a pathology case has already been signed out, issuing an amended report is often necessary. The department of pathology at the university of pittsburgh medical center (UPMC) recently developed and tested a laboratory information system (LIS) driven pre-signout quality assurance tool (PQAT).[9] The PQAT allows for the random review of a proportion of cases by a second pathologist before case verification and release of the final report. This tool replaced the retrospective audit system being used and provided a prospective mechanism for preventing diagnostic errors from occurring. Initial experience with surgical pathology cases demonstrated that disagreement numbers and levels were similar to those identified using the retrospective system, case turnaround time was not significantly affected, and the number of case amendments generated decreased.[10]

The PQAT was subsequently applied as a prospective QA mechanism for non-gynecology cytopathology cases at UPMC. We describe herein our experience with this novel PQAT at our institution in non-gynecology cytopathology practice and evaluate its effectiveness in our cytopathology laboratory.

MATERIALS AND METHODS

Vendor-assisted software modifications were made to the existing LIS application (CoPathPlus, Cerner Corporation, Kansas City, MO, USA) to allow for QA peer review of random cases prior to signout. PQAT was implemented in February 2009 at two medical centers (UPMC-Shadyside and UPMC-Presbyterian). After the PQAT was implemented and tested, a cytopathologist had around 8% of their case load for the day randomly selected at the time of electronic signout for peer review by a second cytopathologist. The tool automatically informs the case cytopathologist that a particular case has been selected for QA when the electronic signature is entered for signout. This occurs in a non-biased fashion as the pathologist has no inclination as to which case is to be selected by the LIS for a review before signout. The case then gets moved to a different worklist in the LIS where it can be accessed and reviewed by the second cytopathologist who is on assigned QA service for that day. The selected case (glass slides and accompanying paperwork) is forwarded to the reviewing cytopathologist who has 24 hours to complete their review and return the case to the primary cytopathologist. The assigned QA cytopathologist reviews the selected cases and enters their interpretation with comments directly into the LIS. The reviewing cytopathologist has four choices for agreement level [Figure 1]: agree (diagnosis appropriate as written), disagree minor (subtle findings of interest to pathologists only; no clinical importance), disagree moderate (disagreement may have clinical importance, but of little importance to the patient) and disagree major (disagreement with major impact on patient care; requires significant change to the report) which is recorded by the LIS. The QA pathologist also has a free text field available to enter any comments. After review, the case is returned to the primary cytopathologist for final verification with the level of agreement displayed on the worklist [Figure 2]. The original pathologist is provided with the opportunity to rectify any detected errors before the case finally gets signed out. The reviewing pathologist is entered into the LIS as a consultant in the final report, without their comments or review results being included in the final report.

Figure 1

Screenshot from the LIS showing the QC pathologist activity tab. After reviewing a randomly selected case, the reviewing pathologist can enter their level of agreement (agree or disagree) along with any review comments in the appropriate field

Figure 2

Screenshot from the LIS worklist for the primary pathologist. The bronchoalveolar case in this example seen within the primary pathologist's worklist displays (in green) the agreement level of the QA pathologist. In this example, the case can now be signed out

In this study, data from non-gynecologic cases selected for PQAT over an 18-month period (Feb 2009-Aug 2010) were collected and analyzed using descriptive statistics. The PQAT was used in conjunction with the regular QA program (e.g., retrospective surgical-cytopathology review) performed by most cytopathology laboratories. We did not compare retrospective and prospective QA error rates.

RESULTS AND DISCUSSION

The total number of non-gynecological cases subjected to pre-signout QA review during this time period was 1339 (7.45%) out of a total of 17,967 cases signed out. For the vast majority of cases (1304 cases; 97.4%), cytopathologists agreed with the entered diagnosis [Table 1]. There was a disagreement in 2.6% of cases. This included 34 (2.5%) cases with minor disagreements, such as 14 thyroid fine needle aspirations involving atypical (so-called follicular lesion of undetermined significance or FLUS) cases and 3 bronchoalveolar lavages where fungus was missed. Only one major disagreement was identified, which involved a CSF specimen in a patient who had a history of myelomonocytic leukemia and examination of the CSF showed atypical monocytes. The primary pathologist called this case positive for involvement of CSF by the leukemic process based on cytomorphology. The reviewing pathologist requested flow cytometry on the fluid which was reported as negative for leukemia. The final report was rectified before the case was signed out as negative for malignant cells. The average turnaround time of cases selected for the PQAT was 1.56 days, compared to an average of 2 days for all non-gynecological cases signed out.

Table 1

PQAT outcome data for non-gynecologic cases reviewed during 18 months

CONCLUSIONS

The LIS-driven PQAT is a novel mechanism in non-gynecological cytopathology that provides an automated prospective QA method for preventing potential diagnostic errors from occurring. In our laboratory, it is used to supplement the traditional retrospective QA program we have in place. This prospective QA process allows for a randomized, second peer review and potential corrective action to be taken prior to the reporting of a diagnosis, without delaying turnaround time. This tool allows for real-time prospective QA activity rather than relying only retrospective review. It is likely that more errors would be uncovered with the PQAT if the proportion of cases selected for review were increased. While double slide viewing of pulmonary cytology cases in a prior study did not lower the frequency of cytologic–histologic correlation false-negative errors,[11] in our experience this mechanism allowed for departmental process improvements. For example, awareness of this tool among pathologists increased their vigilance prior to signing out cases (e.g. double checking diagnoses and case details) as well as the number of formal intradepartmental consultations. This tool may also be strengthened by enriching cases selected for review from sub-populations of cases with known higher rates of disagreements, and perhaps correlating these to histologic diagnoses. Some of the potential drawbacks of instituting a PQAT are the fear of prolonged turnaround time of cases, the need for additional pathologists available to review cases, and the anxiety of a colleague disagreeing with a diagnosis. Our study showed that turnaround time was not significantly affected, and in fact, the cases selected for PQAT actually had an unsuspected slightly faster turnaround time than the overall turnaround time for non-gynecology cases because they were handled with immediate attention. In addition, we have not had an issue finding a second pathologist to review PQAT cases. Finally, despite the anxiety induced over a potential disagreement when a case gets selected for second review, pathologists in our experience welcome the benefits of this tool versus the alternative which would involve amending a previously misdiagnosed case with potential serious consequences for patient care and the associated medicolegal ramifications. Thus, in our experience, the PQAT has been an effective tool which has helped to improve our practice without significantly changing the workflow for case signout. Many of the cases slated to have minor disagreements were related to specimens known to exhibit increased interobserver variability, such as a diagnosis of FLUS with thyroid fine needle aspirations which varies substantially among pathologists and institutions.[12]

By leveraging informatics tools, potential errors in pathology practice can be reduced. With advances in information technology solutions (e.g. automation, autoverification, electronic medical records, digital imaging, etc.), it is possible for laboratories to improve quality in the entire workflow process.[1314] In clinical pathology, QC programs have become increasingly sophisticated with algorithms built into analytical instruments, middleware systems and/or within the primary LIS.[15] Laboratory errors can be divided into pre-analytical (e.g., patient misidentification), analytical (e.g., interpretation errors) and post-analytical (e.g., pathology report transmission failure) errors.[16] In anatomical pathology, informatics tools such as barcoding are increasingly used to help minimize pre-analytical errors. At present, very few information technology tools have been described for reducing analytical errors. The LIS-driven QA process used for cytopathology provides an example of such a tool that allows for real-time, prospective peer review and corrective action to be taken in an attempt to improve patient safety.