Application of sigma metrics for the assessment of quality control in clinical chemistry laboratory in Ghana: A pilot study.

BACKGROUND: Sigma metrics provide a uniquely defined scale with which we can assess the performance of a laboratory. The objective of this study was to assess the internal quality control (QC) in the clinical chemistry laboratory of the University of Cape Cost Hospital (UCC) using the six sigma metrics application.
MATERIALS AND METHODS: We used commercial control serum [normal (L1) and pathological (L2)] for validation of quality control. Metabolites (glucose, urea, and creatinine), lipids [triglycerides (TG), total cholesterol, high-density lipoprotein cholesterol (HDL-C)], enzymes [alkaline phosphatase (ALP), alanine aminotransferase (AST)], electrolytes (sodium, potassium, chloride) and total protein were assessed. Between-day imprecision (CVs), inaccuracy (Bias) and sigma values were calculated for each control level.
RESULTS: Apart from sodium (2.40%, 3.83%), chloride (2.52% and 2.51%) for both L1 and L2 respectively, and glucose (4.82%), cholesterol (4.86%) for L2, CVs for all other parameters (both L1 and L2) were >5%. Four parameters (HDL-C, urea, creatinine and potassium) achieved sigma levels >1 for both controls. Chloride and sodium achieved sigma levels >1 for L1 but <1 for L2. In contrast, cholesterol, total protein and AST achieved sigma levels <1 for L1 but >1 for L2. Glucose and ALP achieved a sigma level >1 for both control levels whereas TG achieved a sigma level >2 for both control levels.
CONCLUSION: Unsatisfactory sigma levels (<3) where achieved for all parameters using both control levels, this shows instability and low consistency of results. There is the need for detailed assessment of the analytical procedures and the strengthening of the laboratory control systems in order to achieve effective six sigma levels for the laboratory.

INTRODUCTION

Quality control (QC) is important for any installation where a critical end results is essential in in the determination of the final product. The use of QC validation to determine the statistical procedures appropriate for distinguishing variations critical for clinical interpretation has been established.1 The level of regulation of various serum analytes will determine the variation that is critical for clinical interpretation.2 Six sigma is a quality management procedure with the aim to improve assay quality by identifying inaccurate and/or imprecise assays so that suitable quality monitoring schemes will be employed to address assay performance. It is important that an analytical procedure achieves a good sigma level if a high reliability is to be attached to the results. A sigma level <3 is an indication of a poor performance procedure, whilst a good performance is indicated by a sigma level >3.3 Above six sigma level is a world class performance. Various laboratories that have been assessed with the sigma scale have achieved mixed results for different analytes.24

Errors in laboratory analysis are mostly divided into pre-analytical, analytical and post-analytical.5 However even though the analytical phase is the least prone to errors,6 there is still room for improvement.7 Westgard reports that, as at 2006, 5-10% of laboratories were deficient in QC practices by inspecting data from The Centers for Medicare and Medicaid Services' (CMS's).3 Various quality control studies both in clinical chemistry8910 and hematology laboratories11 have been carried out in Ghana, but the use of the six sigma metrics as a scale for the assessment of quality control has not yet been explored. This study therefore used the six sigma metrics application to assess the quality control in the clinical chemistry laboratory of the University of Cape Coast (UCC) hospital.

MATERIALS AND METHODS

This hospital based study was conducted at the clinical chemistry laboratory of the UCC hospital from January to March 2014. The laboratory uses the Mindray BS 120 chemistry analyzer to assess it clinical chemistry parameters and the FT-300 electrolytes analyzer for electrolytes assessment. Commercial control samples, (Human assayed control-Fortress Diagnostics Ltd, unit Antrim Technology Park, Antrim BT41 IQS) one with normal and one with pathological values were analyzed each day over a 20 day period.

Analytes representing metabolites (glucose, urea, creatinine), lipids [triglycerides (TG), total cholesterol, high-density lipoprotein cholesterol (HDL-C)], enzymes [alkaline phosphatase (ALP), alanine aminotransferase (ALT)], electrolytes (sodium, potassium, chloride), and total protein were tested to validate quality control in our laboratory.

Statistical analysis

Between-day imprecision (CV) and inaccuracy (Bias) were calculated. We calculated percentage bias according to the equation:

Inaccuracy was considered to be the measure for systemic analytical error.12 Designated means (Dm) for selected analytes in commercial controls were provided by the reagent manufacturer. The CVs and standard deviations (SDs) were calculated using the GraphPad Prism version 5.0. The daily quality control (QC) vector charts were constructed using the multiQC software version 5.

The sigma metrics σ (for the various analytes) were calculated by the following equation

(TEa-total allowable error, CV-coefficient of variation). TEa values of various parameters were taken from the Clinical Laboratories Improvement Act (CLIA) guidelines.213

RESULTS

Results for the daily variation of QC vector and the comparison of means between the Dm and Lm are indicated on Figure 1. The electrolytes had between 30-50% (average of 38.33%) of the daily QC points exceeding the tolerance limits for normal (L1). Conversely an average of 21% (range 10-35%) of the daily QC points exceeded the tolerance limits. Generally for both L1 and pathological (L2), the Lm was higher than the Dm, however, when the two were compared using one sample t test, the electrolytes showed no difference (P > 0.05) for L1 whilst there was a difference (P < 0.01) between Dm and Lm for L2.

Figure 1

Daily variation of QC vectors and the comparison of laboratory means with designated means for electrolytes. L1 – normal control serum; L2 – abnormal control serum; Lm – laboratory mean (green horizontal line); Dm – designated mean (green broken horizontal line). Red line describes the general trend of the daily fluctuations of the QC vectors. Two red horizontal lines – lower and upper tolerance limits; Dm and Lm are measured in mm/L

Figure 2 indicates the results of the daily variation of QC vectors and the data for the comparison of means for the lipid metabolites cholesterol, triglycerides and HDL-cholesterol. TG had no variation between Lm and Dm for L1 whilst the two means were similar (<1.5% variation) for L2. Cholesterol recorded a lower variation (<5%) for both L1 and L2, whilst a higher variation (>10%) was recorded for HDL-C for both normal and pathological controls. A comparison of Dm and Lm for the lipid metabolites revealed no difference (P > 0.05) for the three parameters (L1) and TG (P = 0.782) (L2) whereas a difference was shown for both cholesterol (P < 0.001) and HDL-C (P < 0.001) for L2.

Figure 2

Daily variation of QC vectors and the comparison of laboratory means with designated means for. TG, Cholesterol and HDL-C TG – triglyceride; HDL-C – High density lipoprotein cholesterol

Figure 3 shows the results of the daily variations of QC vectors for the enzymes and total protein. The Lm and Dm varied (>10%), for both levels of QC for the two enzymes (AST and ALP). The Dm and Lm for total protein for L1 varied by 10.73%. Conversely, Dm and Lm for total protein were similar (<2%) for the pathological control. All three parameters showed differences (P < 0.01) between the Lm and the Dm for both normal and pathological controls, except for total protein which indicated no differences (P = 0.201) for L2.

Figure 3

Daily variation of QC vectors and the comparison of laboratory means with designated means for AST, ALP and Total Protein AST – alanine aminotransferase; ALP – alkaline phosphatase;T. Protein – Total Protein

Results for the daily variation of QC values for urea were similar (<1% variation) for L1 but varied widely for L2 (>17%). On the contrary glucose had a wide variation (22.5%) and a lower variation (2.3%) for L1 and L2 respectively. A wide variation (>10%) was shown by creatinine for both controls. A comparison of means showed a difference (P < 0.05) for the three parameters for both controls except urea (L1) which showed no difference (P = 0.794).

Table 1 highlights TEa, bias, coefficient of variation and sigma values for the two control levels. Sodium and potassium have relatively smaller TEa. The CVs ranged from 2.4% (sodium) to 25.12% (HDL-C) for quality control L1 and 2.52% (chloride) to 28.27% (HDL-C) for quality control L2. Only sodium and chloride obtained CVs <5% for both controls, however glucose and cholesterol for L2 also obtained CVs <5%. Sigma values <1 was achieved for 7 analytes (cholesterol, HDL-C, urea, creatinine, total protein, AST and ALT) for the normal quality control. Conversely, glucose, TG, sodium and chloride achieved sigma values between 1.6 and 2.05. The abnormal control, L2, generated sigma values between 0.59-2.92 for AST, total protein, glucose, cholesterol, ALP and TG. The highest sigma value for both quality controls was recorded by TG for L2 (2.92) while creatinine for L1 (0.10) generated the lowest sigma value.

Table 1

%Bias, TEa, SD, %CV and Sigma values for the replication studies

DISCUSSION

Six sigma is a management strategy with the main focus of improving the quality of process outputs by the establishment, and removal of the causes of defects (errors) and decreasing the variation that occur in a manufacturing and business processes. Attainment of six sigma is envisaged as the gold standard for defining world class measure of quality in the clinical laboratory.2 This study used the six sigma metrics approach to assess the internal QC of the clinical chemistry laboratory of UCC hospital. Our findings indicated low sigma levels for all the parameters assessed. None of the parameters achieved a sigma level >3, indicating poor QC of the parameters that were assessed. High imprecision (CVs >5%) was observed for all parameters except sodium and chloride for both the normal and pathological controls and also glucose and cholesterol for the pathological control. The high imprecision is a function of an unstable analytical process with wide fluctuations around the true concentration of the analytes as shown by the daily QC plots [Figures 1–4]. In clinical chemistry QC, imprecise QC points may be viewed as defects in the product outcome.14 A high defect indicates a low quality and a high cost. When defect is high there is the need for re-testing, follow-up testing and usually a high number of customer complaints. Situations like this require a lot of time and effort needed to service those complaints, and in the case of serum analytes, it may involve several repetitions of a test to ascertain the true concentration. Non-conformities in laboratory testing are caused basically by excessive process variation and mistakes.15 Compliance with acceptable specification is therefore very important to the clinical chemist as this make use of the medically allowable tolerance limits for each of the parameters. The level of variability within a particular analyte determines its quality requirement and this is based on how strict the analyte is physiologically regulated. This justifies the main objective of the six sigma metrics with respect to minimizing both variance and quality control processes to guarantee compliance with the critical specifications.2 Singh et al., (2010) achieved a sigma metrics above six for creatinine, triglycerides, SGOT, CPK-Total and amylase indicating a high quality as compared to what was achieved in our laboratory. Achieving a sigma level >6 must be encouraged in all laboratories, however, in doing so, measures must be put in place to reduce the probability of false rejection whiles increasing the probability of error detection. The findings in this study may be limited by the short period (20 days) in which the replication study was carried out.

Figure 4

Daily variation of QC vectors and the comparison of laboratory means with designated means for Urea, creatinine and Glucose

CONCLUSION

Unsatisfactory sigma levels where achieved for all parameters using both control levels, this shows instability and low consistency of results being delivered. This is an indication of poor quality control based on the sigma model applied in this study. There is therefore the need for detailed assessment of the analytical procedures and the strengthening of the laboratory control systems in order to achieve effective six sigma levels for our laboratory.