Timeliness “at a glance”: assessing the turnaround time through the six sigma metrics.

Almost thirty years of systematic analysis have proven the turnaround time to be a fundamental dimension for the clinical laboratory. Several indicators are to date available to assess and report quality with respect to timeliness, but they sometimes lack the communicative immediacy and accuracy. The six sigma is a paradigm developed within the industrial domain for assessing quality and addressing goal and issues. The sigma level computed through the Z-score method is a simple and straightforward tool which delivers quality by a universal dimensionless scale and allows to handle non-normal data. Herein we report our preliminary experience in using the sigma level to assess the change in urgent (STAT) test turnaround time due to the implementation of total automation. We found that the Z-score method is a valuable and easy to use method for assessing and communicating the quality level of laboratory timeliness, providing a good correspondence with the actual change in efficiency which was retrospectively observed.

Introduction

The turnaround time (TAT) is indeed a “fundamental dimension” within the clinical laboratory paradigm, since it represents a benchmark as well as a means to pursue improvements (). In this regard, the Working Group for Laboratory Errors and Patient Safety of the International Federation of Clinical Chemistry and Laboratory Medicine has recognized the TAT as a mandatory quality indicator (level 1) for the post-analytical phase ().

The TAT analysis is aimed to assess the tendency of the laboratory system to meet a certain goal of timeliness. This in turn assumes quality as a natural consequence of speed (“the faster, the better”), so that efficiency and speed are thought synonyms (). However, quality is also a matter of variability, in that the service reliability relies on the capability to grant almost the same time of completion to any incoming request. (). In this regard, several studies have shown that regularity of laboratory service is as relevant as speed in improving the efficiency of hospital departments (, ).

Thus, in order to pursue real improvements, a quality assessment should show whether the laboratory process is capable to near a precise target of timeliness (the “appropriate” TAT”), while respecting certain limits of tolerability in delaying the results (the “acceptable TAT”). To deal with such a trade-off between outer demand (customer) and inner capabilities (producer), the manufacturing domain has developed several quantitative techniques, among which there is the so-called “six sigma” (usually abbreviated in 6σ) (). The six sigma has the uncommon quality to be as easy to compute as immediate to understand. In this respect, our aim is to give a contribution in showing how well this industry-born concept can suit the TAT quality level assessment of a core laboratory.

Materials and methods

Under a practical standpoint, the sigma level corresponds to the spreading between the tendency of the process output, namely the mean (m), and the boundary of tolerance for the same output, namely the specification limit (SL), standardized by the process variability (SD). For a normally distributed output, this corresponds to calculating the Z-score of the SL using the parameters m and SD as a modification of the common process capability indices (, ):

For a quite large sample and considering a short-time period of observation, the formula above can be used without any particular correction, with the exception of checking the data normality and applying the appropriate transformation (like the Box-Cox power family) accordingly (). A process rating at a level of “six sigma” (also indicated as “world class quality”) is expected to miss suitability with a probability of 0.00034% (about 1 every 300,000 tries), even considering a 1.5 sigma drift from its target output (Table 1) ().

Table 1

Z-score and sigma level.
For each Z-score it is given the corresponding cumulative probability with respect to the standard normal curve, alongside with the rate of missed suitability expressed in percentage (%) and parts-per-million (ppm) and the Sigma Level with respect to the short term and long term. The solid grey line highlights the Z-score which corresponds to what is commonly considered the “world class” or “six sigma” quality level.

Z-score	Cumulative probability		Missed suitability		Sigma Level
	≤ Z-score	> Z-score	%	ppm	Short term	Long term
0.5	0.69	0.31	30.85	308537.5	0.5	2.0
1	0.84	0.16	15.87	158655.3	1.0	2.5
2	0.98	0.02	2.275	22750.1	2.0	3.5
3	0.99865	0.00135	0.135	1349.9	3.0	4.5
4	0.999968	0.000032	0.0032	31.7	4.0	5.5
4.5	0.9999966	0.0000034	0.00034	3.4	4.5	6.0
5	0.99999971	0.00000029	0.000029	0.287	5.0	-
6	0.999999999	0.000000001	0.0000001	0.00099	6.0	-

Regarding our experience, in May 2011 the core laboratory of Tor Vergata University Hospital of Rome implemented a total automation system for routine and urgent (STAT) clinical chemistry (). In January 2011 (dataset 1, N = 399), with no automation but having all the analysers operating stand-alone, the median and the 90th percentile TAT of STAT cardiac troponin-I (CTNI) tests ordered by the Emergency Department (ED) in the regular morning shift were 55 and 95 minutes respectively. Thereafter, in January 2015 (dataset 2, N = 413) after almost 4 years of total automation, these scores resulted to be 35 and 60 minutes respectively. In order to evaluate the actual quality achieved through automation with respect to the ED service, the American College of Cardiology Foundation and American Heart Association (ACCF/AHA) guidelines for the management of myocardial infarction were considered as the reference for STAT CTNI timeliness, and thus the acceptability or tolerance limit for the TAT of STAT CTNI test was set at 60 minutes (). Before performing the sigma level calculation, data were checked for normality by means of the Kolmogorov-Smirnov test which resulted to be statistically significant in both cases (P < 0.01). Thus, the Box-Cox power transformation was applied to each dataset, finding that the appropriate values were λ = -0.4 for dataset 1 and λ = 0.2 for dataset 2. The Kolmogorov-Smirnov test repeated thereafter showed that both transformed datasets finally met normality (P = 0.907 and P = 0.556 for dataset 1 and dataset 2 respectively). In order to compute the Z-score method, the tolerance limit of 60 minutes underwent the Box-Cox power transformation applying the same λ value which was used for the corresponding dataset. Therefore we had:

Taking m and SD for each transformed dataset we computed the Z-score as following:

All the calculations were performed with MedCalc 12.2.1.0 (MedCalc Software bvba, Ostend, Belgium). The probability of expected not compliance to the SL was obtained as the complementary to the cumulative probability corresponding to the calculated Z-score (Figure 1).

Figure 1

Calculation of the sigma level with the z-score method.
The dataset of January 2011 STAT CTNI turnaround time (A) showed an evident right skewed distribution which disappeared after the appropriate Box-Cox power transformation (B) (λ = -0.4); the standardized cumulative probability plot (C) of the frequency distribution in panel B shows the expected percentage of uncompliant TAT (1-P) respect to the computed Z-score for the tolerance limit set (60 minutes).

Results

As it can be seen, in 2011 the Z-score was 0.38, which corresponds to a probability of 35.20% of uncompliant timeliness, whereas in 2015 it was 7.93%. Although this change seemed to have no effect over the number of incoming orders, we observed some other substantial changes within the laboratory regarding the management of the ED orders ().

Indeed, in January 2011 the laboratory logbook reported 102 communications between ED and core laboratory concerning delayed STAT CTNI orders, corresponding to 1 call every 4 submitted orders. For each communication, the estimated average time spent by the lab technologist to verify the status of the request was 4.8 minutes, with a maximum of up to 18 minutes. Therefore, the time spent to accomplish the task of checking the order status amounted to about 8.5 hours per month, which corresponded to the duration of a single working shift. Conversely, in January 2015 laboratory logbook reported just 17 communications for delayed STAT CTNI results (4%), which amounted to 1.36 hours per month. Noticeably, due to the possibility of tracking the sample position by means of the total automation middleware, the time required to perform such a task was also considerably less than in the past. Therefore, we found the sigma level a reliable metrics which accurately and appropriately showed the leaning in laboratory procedures and the improvement in intra-laboratory TAT due to the total automation. Noteworthy, the change in sigma level also corresponded well to the perception which the ED personnel had of the laboratory service. An internal survey carried out independently by the hospital in 2014/2015 showed that the degree of satisfaction with the STAT laboratory service was rated 8.5/10, whereas similar information gathered between 2010 and 2011 rated the same service at 3/5. Unfortunately, as no data were available regarding the ED length of stay in 2011, we cannot assess whether the sigma level reached by our laboratory had an impact on the critical cares efficiency.

Discussion

There are some points that deserve attention regarding the use of six sigma. First, it is mandatory to check data for normality and eventually transform them before applying the Z-score method in order to obtain a reliable sigma level (Figure 1). Furthermore, as the sigma level/Z-score is dimensionless, there is no need of any back transformation to interpret the results afterwards. Second, the sigma level should not be confused with the outliers percentage of TAT, although both quantify the tendency of the system to exceed a certain acceptability limit. In fact, the outliers percentage does not take into consideration the output dispersion, and thus it gives only a gross estimate of the probability of unsuitability. Third, it should be always specified if the sigma level reported is for short term or long term, in that the long term level is computed with an offset of 1.5 sigma in order to account for the possible drift of the process average (Table 1). For instance, even though the literature reported a 2.95 sigma level for the cytology specimen adequacy, the rate of unsuitability shown therein was 7.32%, which corresponds instead to a short term level of 1.45 sigma (). Lastly, it should be remarked that the sigma level strictly depends on the tolerance limit set, and therefore the appropriate SL value should be established with respect to analytes and operative conditions through consensus conferences in order to grant standardization.

In conclusion the six sigma metrics is a valuable tool for delivering “at a glance” the quality level of the clinical laboratory TAT, due to its immediacy and the correspondence with the perceived level of proficiency.