Impact of reference change value (RCV) based autoverification on turnaround time and physician satisfaction.

BACKGROUND: For a quicker delivery of laboratory test results to the hospital emergency department (ED), we implemented an autoverification system based on the reference change value (RCV). The aim of this study was to assess how the RCV based autoverification reflected on turnaround time (TAT) and on physician satisfaction.
MATERIALS AND METHODS: The laboratory information system (LIS) was programmed to autoverify the results as long as they were within the range settled by RCV, so that the autoverified results were reported to the physician as soon as the tests were carried out, without any further intervention. We analyzed the same three-month periods' TAT and verification time (VFT) from the years prior to and following the implementation of RCV autoverification. The change in physicians' satisfaction levels was assessed using the hospital's Annual Physician Satisfaction Survey (APSS). Over sixty percent of physicians completed the questionnaire, and the amount of daily ED test requests (nearly three hundred) did not vary throughout the duration of this study.
RESULTS: Mann-Whitney U test showed that the VFT was significantly reduced in all the test but troponin I. There were substantial reductions in TAT medians (haemogram, 75%; fibrinogen, 41%; prothrombin time, 40%; sodium, 27%). The percentage of physicians satisfied with the haematological and biochemical tests´ TAT increased from 84% to 93% and from 86% to 91% respectively.
CONCLUSIONS: Our results reveal that VFT and TAT were severely reduced in most emergency tests, greatly improving physicians' satisfaction with TAT.

Introduction

Emergency department (ED) crowding represents an international problem that may affect the quality and access of health care. Common causes of ED crowding include non urgent visits, “frequent-flyer” patients, influenza season, inadequate staffing, inpatient boarding, and hospital bed shortages (, ). The rise in the number of ED patients has a negative impact on waiting times, being thence closely related to the patients’ dissatisfaction (), and most importantly, to the diminution in the quality of the care provided (, ). Therefore, it is essential to limit the effects of any factor that might interfere with the output of admitted patients.

Among many other factors that contribute to the bottleneck effect in the ED, delays in laboratory have always been considered very influential. In fact, the increase in laboratory TAT has been reported to cause delays in treatments, as well as a lengthening of the patients’ time of stay by more than 50% (-). Moreover, the aforementioned rise in the volume of visits results in an increase of the number of specimens that are sent to the emergency laboratories, which aggravates the situation and worsens the delay in turnaround times. That is the reason why laboratories need to provide strategies to minimize these lags, thus relieving the pressure put on the ED and contributing to a quicker diagnosis and a better treatment of the patients.

Turnaround time covers the entire process that ranges from sample registration to the reporting of the results, that is, the three phases of the handling of any specimen sent to the laboratory: 1) pre-analytic phase (requesting, collection and transport of specimens); 2) analytic phase (analysis of specimens); 3) post-analytic phase (time to verify and report the test result to physicians). Laboratories nowadays have optimized their organization thanks to the automation of the pre-analytic and analytic phases, thus managing to reduce their workload (, ). The post-analytic phase, and particularly its most important process, verification (the laboratory staff’s approval of results that match the clinical features and physiopathological status of patients) is a dull manual review task that demands a great deal of the laboratory staff’s effort and time, to the point of preventing them from focusing on the small number of test results that are actually controversial and need greater attention (). Therefore, the automation of verification is a new, strategic opportunity to optimize the laboratory staff´s efforts and tests’ turnaround times, allowing for the immediate release of the results, provided that they meet the required rules. Automatic verification, i.e. autoverification, is performed through an expert system within the Laboratory Information System (LIS), or other middleware software. The test results that fulfil the established requirements are handed out to the ED as soon as they emerge from the analyzers. These requirements may be chosen based on population reference ranges, analytical measurement ranges, critical values, delta checks, instrument error flags, interference indices and any other predetermined set of criteria settled by the laboratory ().

The autoverification criterion that has recently been implemented in our emergency laboratory is that the reference change value (RCV) limits must not be surpassed. RCV is a very objective and altogether ideal strategy for the assessment and verification of serial results, because the evaluation of these results is based on the unavoidable variation generated by the analytical imprecision and within-individual biological variation (). Although certain disadvantages of using biological variation for reference change values have been pointed out () and RCV may need some improvement, it is important to consider that it is currently an invaluable tool in laboratory medicine ().

Any change in consecutive measurements within the limits defined by RCV is judged as non-significant, and the results are autoverified. However, when the range is surpassed, test results are held for manual verification by laboratory staff, so that the significance of this alteration can be evaluated. Autoverification has been proven to be an enormous advantage to the laboratory because it decreases verification workload, increases productivity, through important savings in Full-time equivalents (FTE) to result-review functions and reduces overall error rates as well as TAT (, -). Clinicians have always viewed the timeliness of the reporting of results as a key criterion when judging the performance of laboratories. They also consider that a reduction in laboratory TAT is essential for proper patient care and treatment (, ). This prompted us to investigate whether the improvement noted on TAT after the introduction of RCV-based autoverification was perceived by ED physicians. Since our hospital’s quality unit conducts an annual physician satisfaction survey, we compared the resulting level of satisfaction obtained regarding the emergency tests report turnaround time with that of the year before the implementation of RCV-based autoverification.

Material and methods

The reference value that we used to determine significant changes in each test’s serial results was calculated following the term introduced by Harris and Yasaka, RCV (). Three years before this study was started, our laboratory implemented a quality management system which fulfils the requirements of the ISO 9001:2008. Since then, it is guaranteed that the standard operating procedures are well established and respected by our technical staff, so that the pre-analytical conditions are unvarying and the RCV formula becomes:

RCV = 21/2 x Z x (CVA2 + CVI 2) 1/2where Z indicates the number of standard deviations appropriate to the desired probability, 1.96 for P < 0.05; CVA, analytical imprecision; and CVI, within subject biological variation. The CVA of each test was provided by imprecision testing in laboratory and the estimates of CVI are available for many biological compounds ().

Autoverification was activated in every test file of our LIS (SIGLO, Horus Hardware S.A., Las Rozas de Madrid, Spain) after introducing the corresponding RCV figure in the “delta-check” box and selecting a period of three years for the time back applied in the “delta-time” box. Tests whose results had been analyzed less than three years ago, and whose present result is within the limits defined by RCV, are autoverified. However, when the range defined by RCV is surpassed, test results are held for manual verification by laboratory staff, so that the significance of this alteration can be evaluated.

Tests with no previous results, or those whose last result had been obtained more than three years ago were held for manual validation.

RCV for sodium in serum, for example, equals 3.8% in our laboratory (as our analytical imprecision is 1.2% and the within subject biological variation is 0.7%), so when the autoverification according to RCV is activated in the sodium file, any patient with a sodium test result included within the interval (last sodium result within three years back ± 0.0385 x last sodium result within three years back), would be autoverified for this test. Patients whose sodium results were out of this interval, as well as patients without a previous result for sodium, or those with previous results from more than three years ago, would be held for manual verification.

Creatin kinase-MB, mass and troponin I were run on UniCel DxI 800 Immunoassay System (Beckman Coulter Inc, Brea, USA). Coagulation tests were performed on ACL TOP 500 CTS (Instrumentation Laboratory Werfen Co, Bedford, USA). The biochemical tests were performed on UniCel DxC 800 Synchron Clinical System (Beckman Coulter Inc, Brea, USA). Hemograms were run on COULTER LH 750 Hematology Analyzer (Beckman Coulter Inc, Brea, USA). All analyzers and the associated reagents were used according to the manufacturer´s instructions.

For each test, TAT was the period from when the specimen was received in the laboratory until the result was reported. VFT was the period from when the test was completed until the result was reported. TAT and VFT medians and autoverification percentages (%VT) for each test were taken from the statistical module of the LIS. In this regard, two equivalent three-month periods, from the year prior to and the one following the implementation of RCV-based autoverification, were examined.

To measure the physicians’ satisfaction we used the Annual Physician Satisfaction Survey (APSS), carried out by the Quality of Care Unit of the hospital. Every year, the APSS requests the clinicians’ assessment of the emergency turnaround time. To do so, it uses the Likert scale, which classifies the participants in five different categories: very dissatisfied, rather dissatisfied, indifferent, satisfied and very satisfied. Thus, we compared results from the years before and after implementation of the RCV based autoverification. The survey was launched using the online tool LimeSurvey version 2.05 + Build 140520 () which feeds back the percentage of responses and the total numbers, but not the raw data.

All the statistical analyses used were performed using SPSS Statistics v15 (IBM, Chicago, IL).

Results

During the three-month period prior to the introduction of RCV-based autoverification, 252,767 tests were carried out for 10,769 ED patients. All of these tests were manually validated. After the introduction of the new autoverification method, 252,225 tests were performed for 10,930 ED patients throughout an equivalent three-month period. This time, 140,294 tests (56%) were autoverified. The improvement was applied to 59% of the population, as 6437 out of 10,930 ED patients had at least one test autoverified.

The TAT and VFT medians of the third trimester of the years before and after RCV autoverification implementation, for each test, are shown in Table 1. When conducting the comparisons, as in no case the data in both groups were sampled from a normally distributed population (Kolmogorov-Smirnov tests), the nonparametric Mann-Whitney U tests were done. Mann-Whitney U tests revealed significant differences (P < 0.001) between the two terms for TAT and VFT in all the tests, with the exception of troponin I. Troponin I showed no significant changes in VFT and its %VT was very small (25%).

Table 1

Laboratory turnaround time and verification time before and after RCV-based autoverification implementation

Test	TATB	TATA	VFTB	VFTA	%TATR	%VT
Total bilirubin	41.2 (30.2–48.2)	30.9 (22.1–42.5)	11.2 (6.2–14.0)	3.5 (0–8.4)	25	57
Creatinine	40.3 (24.2–47.4)	31.7 (18.5–42.0)	10.7 (5.1–12.2)	4.9 (0–7.5)	21	46
Glucose	40.2 (24.5–47.1)	32.4 (18.5–41.8)	10.6 (5.2- 12.1)	5.6 (0–7.4)	19	38
Total protein	41.2 (29.7–48.3)	33.2 (20.2–42.5.)	11.1 (8.0–13.5)	5.7 (2.2–8.6)	19	34
Urea	40.2 (24.2–46.3)	30.7 (17.8–37.5)	10.6 (5.3–12.1)	4.0 (0–6.5)	24	55
Troponin I	47.4 (34.3–54.6)	41.9 (32.5–53.8)	9.6 (6.3–12.2)	8.7 (6.2–12.2)*	12	25
Creatine Kinase	44.7 (32.6–50.1)	35.5 (24.5–42.2)	11.1 (6.3–13.8)	5.0 (0–6.2)	21	52
Creatin Kinase MB, mass	53.7 (35.9–57.5)	45.7 (32.2–52.8)	9.0 (6.3–12.2)	3.7 (1.5–6.5)	15	41
Hemogram	26.6 (8.2–35.5)	6.7 (4.2–15.4)	25.2 (8.4–34.3)	5.0 (0–10.1)	75	51
Prothrombin time	36.9 (26.2–38.3)	22.0 (20.1–25.1)	16.7 (6.2–19.3)	6.6 (0–7.5)	40	52
Fibrinogen	36.8 (26.2–38.3)	21.6 (20.2–25.1)	16.7 (6.2–19.4)	6.2 (0–7.8)	41	55
Activated partial thromboplastin time	37.2 (28.5–42.3)	27.4 (22.8–35.2)	15.4 (9.3–19.5)	9.2(2.2–13.2)	26	30
Sodium	40.5 (24.5–47.1)	29.7 (18.5–40.8)	10.9 (5.2–12.1)	2.7 (0–5.0)	27	64
Potassium	39.7 (24.4–46.9)	30.6 (18.5–40.9)	10.9 (4.9–12.0)	3.1 (0–6.8)	26	53
Chloride	40.5 (24.5–47.1)	29.9 (18.5–41.8)	11.8 (5.2–12.1)	5.6 (0–7.3)	20	60
Calcium	42.2 (29.7–48.3)	32.2 (22.1–42.5)	11.7 (6.2–14.0)	5.0 (0–8.5)	24	47
Phosphate	43.1 (29.7–48.3)	32.4 (22.1–42.5)	12.2 (6.2–14.7)	4.9 (0–8.5)	25	51
Alanine aminotransferase	41.2 (28.6–48.1)	31.0 (23.5–41.2)	11.2 (6.4–13.9)	3.6 (0–6.8)	25	54
Aspartate aminotransferase	41.2 (28.6–48.1)	32.2 (23.5–41.2)	11.2 (6.4–13.9)	4.6 (0–6.2)	22	46
α-Amylase	41.6 (30.1–49.6)	34.9 (24.7–42.6)	11.0 (6.3–13.7)	6.6 (1.9–9.2)	16	32
Cholinesterase	40.4 (30.2–49.4)	34.2 (25.5–43.2)	9.0 (5.8–13.8)	6.0 (2.7–9.6)	15	28
Data are expressed as median (interquartile range) in minutes. TATB–turnaround time before autoverification implementation; TATA–turnaround time after autoverification implementation; VFTB–verification time before autoverification implementation; VFTA–verification time after autoverification implementation; %TATR–percentage of turnaround time reduction: 100 (1- (TATA / TATB)); %VT–percentage of autoverified tests.*Mann-Whitney U test showed no significant difference vs. VFTB, P = 0.399.

Table 1 also shows the remarkable reduction of TAT for most of the tests studied, displayed as percentage of turnaround time reduction (%TATR). For troponin I this reduction was only 12%. It is worth noting that every haematological test showed a much larger %TATR than any biochemical one.

Concerning physicians´ satisfaction, 89 out of 142 physicians (61%) completed the annual survey after the introduction of RCV-based autoverification, and 118 physicians (70%) had filled out the questionnaire the previous year.

Figure 1 shows physician satisfaction with TAT for biochemical (BTAT) and haematological (HTAT) tests before and after the implementation of RCV based autoverification. The percentage of physicians who felt satisfied with the HTAT (combining both “satisfied” and “very satisfied” categories) was increased from 84 to 93%, and from 86 to 91% with BTAT; the percentage of dissatisfied clinicians (unifying the “rather dissatisfied” and “very dissatisfied” categories) showed no reduction with HTAT, from 3.5 to 3.7%. However, it showed an important increment (more than twice the initial value) from 1.7 to 3.8% in BTAT. Interestingly enough, the percentage of physicians in the “indifferent” category showed an important reduction from 12 to 2.8% in HTAT and from 12 to 5.1% in BTAT. Thus, we observed an overall improvement in satisfaction with a clear reduction in the percentage of the “indifferent” category (BTAT 7.6% and HTAT 9.5% reduction) and a notable increment in the percentage of the “satisfied” or “very satisfied” categories (BTAT 5.5% and HTAT 9.3% increment). It is worth noting that the “very satisfied” category was importantly increased on both haematological and biochemical tests (7.1% and 8.6% respectively).

Figure 1

Categorised percentage of ED physician’s satisfaction on turnaround time before and after RCV autoverification implementation. (A) Haematological tests. (B) Biochemical tests.

Discussion

We activated the built-in RCV based automatic verification software of our LIS in the emergency laboratory to optimize the laboratory staff´s efforts and tests’ turnaround times.

Verification workload decreased importantly. A three-month period after the software activation, 140,294 tests (56%) were autoverified. This event probably permitted a more efficient staff´s manual verification. There was not any reduction in Full-time equivalents because in our laboratory there is only one physician on call.

In this study, we explored above all whether autoverification based on RCV had a positive impact on laboratory turnaround time for emergency tests, and more importantly, if this improvement impacted ED physician satisfaction.

Papers on autoverification of acute-care patient tests but also providing TAT data are rather scanty in the literature. In spite of that, we found some articles to compare our results with. Torke et al. reported that TAT for stat chemistries (sodium, potassium, chloride, glucose, blood urea nitrogen and creatinine) was reduced by 10 minutes (48 minutes before vs 38 minutes after autoverification) (). Table 1 shows for these tests similar differences between TAT before and after autoverification. Cardiac test TAT, creatine kinase-MB and troponin I, decreased 11 minutes (from 45 minutes to 34 minutes). Our results were similar for creatine kinase-MB but only decreased 5.5 minutes for troponin I. The small percentage of autoverified results for troponin I might explain this. Cheng et al. showed very significant decreases in TAT both for the metabolic panel and for troponin I (). The median after autoverification was 30.3 and 44.5 minutes respectively, close to ours. The TAT before in both cases was reported very high.

Blick study, providing services to critical care areas through middleware software autoverification and Six Sigma initiatives, describes 93% of hemoglobin results under 30 minutes and TAT for potassium of 27 ± 7.2 minutes (mean ± SD) (). Potassium TAT was similar to ours. For hemoglobin, we delivered results more rapidly, 75% in 15.4 minutes (Q3) after autoverification (93th percentile was 23.5 minutes, data not shown in Table 1).

Onelöv et al. reported coagulation assay data from laboratories performing tests on a 24/7 basis (). They activated the delta check function on their LIS software for autoverification (delta range for prothrombin time 40%, for the rest of assays 20%; delta time 90 days) and compared two similar weeks. The median TAT for all routine coagulation assays was 37.0 and 32.0 min, during the weeks before and after, respectively. The introduction of an autoverification protocol decreased the median overall TAT by 5 min. Our TATs were decreased to a greater extent (Table 1, %TATR from 26% for activated partial thromboplastin to 41% for fibrinogen).

We achieved smaller percentages of autoverified tests (25% - 64%) in comparison to Torke et al. (62% - 73%), an analogous study but where a set of multiple rules was used (). This could be due both to the fact that the RCV based autoverification requires the existence of previous values to compare the obtained results with and that it was the only rule that we applied in autoverification. Concerning the percentages of autoverification, RCV based autoverification as the sole rule in our emergency laboratory yielded appealing results, as according to Cava-Valenciano report (), while in outpatient laboratories up to 70% of the results are autoverified, in centers of complex pathologies the percentage may be only 10%.

The lack of significant differences for VFT in troponin I was probably due to the very small percentage of autoverified results (%VT) and was consistent with the low %TATR shown for this test. Regarding the very small %VT for troponin I, it is important to point out that in addition to the previous value that RCV autoverification requires, the troponin I test used was conventional, so there were many results under the limit of detection (10 ng/L). Therefore, a very large amount of results handed out by the analyzer to the LIS as alphanumeric results, i.e. < 10 ng/L, were excluded from the formula for autoverification.

Despite this technical drawback concerning troponin I, we have introduced a significant innovation for TAT reduction that saves time objectively and securely where it is most needed: the use of RCV as a tool for autoverification in ED population.

This tool for autoverification is a very simple method based just on a physiological and unbiased rule. As it does not depend on a set of rules or algorithms that might be accidentally changed or inactivated, and as it requires hardly any maintenance, it is a very safe mean of generating important reductions in TATs while ensuring analytical and post analytical quality. This tool can contribute to the great benefits that others standard softwares have shown with regard to autoverification ().

Moreover, our results showed that these significant reductions in TATs were accompanied by an overall improvement in satisfaction of the ED physicians. Due to the method employed in the APSS we did not have access to the raw data. Therefore, an inferential statistical test was unavailable. This is probably the main limitation of the survey as a tool for investigation, but it can also be regarded as a double blind experiment, since neither the data analyst nor the physicians were aware of the fact that this study was being carried out. Double-blinding was relatively easy to achieve as, in spite that our emergency laboratory and the ED are very well connected for patient´s sake at large, many times internal protocols are not diligently communicated to each other. The high workload that both departments have to put up with may be responsible of this. Besides, we were not really testing or introducing any new kind of expert system or middleware that would rapidly spread from the laboratory and reach the ED. It was just that the software of our LIS could be activated for RCV based autoverification, so after a short pre-production phase which was entirely satisfactory, we launched it. Our first purpose in activating this kind of autoverification was to alleviate the increased workload that each day the only laboratory physician on call had to deal with. After noticing the important test TAT reductions resulting from the implementation of this autoverification, we decided to explore the impact that it had had on ED physician satisfaction.

As we described in the results section, there were significant changes worth discussing (Figure 1); and considering that no other action involving the TAT was taken during the realization of this study, this improvement in satisfaction may be regarded as a direct effect of the implemented RCV based autoverification.

As many authors have considered, reduction of TAT can be essential to improve the user’s satisfaction with the laboratory (, , , ). In fact, TAT is an indicator used to evaluate the quality of care and the perceived quality of the laboratories; it is conceivable and it has been assumed that quicker TATs will concur with a more satisfied user. Our study seems to support this association between faster TAT and user satisfaction.

TAT is a good quality indicator, since an improvement in TAT implies an improvement in satisfaction, evidenced by the reduction of the “indifferent” category. This is particularly clear because the improvements in HTAT were larger than in BTAT, as the %TATR shows in Table 1, and consequently the satisfaction with HTAT was also bigger, with less “indifferent”, more “satisfied” and less “dissatisfied” physicians. The highest reduction of %TAT noticed between haematological tests was probably due to the fact that their verification time was reduced most markedly. For haematological tests it is more frequent, in view of a pathological result, to recheck the sample quality than for biochemical tests. Samples yielding long clotting times or low platelet counts or low red blood cell counts are routinely checked for the presence of a clot by inserting applicator sticks into the whole blood sample. This is a time-consuming practice carried out frequently before the manual verification is done. Meanwhile, the BTAT also showed less “indifferent” and more “satisfied” users, but the number of “dissatisfied” users had increased as well. We interpret this as a non-intentional comparison: the satisfied users may be under the impression that BTAT has worsened, because they perceive a substantial raise in HTAT quality compared to BTAT.

The implementation of an expert system of autoverification based on RCV to speed up the delivery of results, even of those out of range of normality, has been a first step of a process of improvement of the emergency laboratory that has increased ED physician satisfaction. Further steps in this direction, as reporting patient results which surpass RCV, will be taken after the necessary technical modifications of our LIS.

In summary, we have introduced the autoverification based on RCV in our laboratory as a simple and objective way to autoverify emergency test results. This significant innovation has led to better TATs and higher physician satisfaction in the emergency room, a place where timeliness of test result reports is critical.