Comparative analysis of calculating sigma metrics by a trueness verification proficiency testing-based approach and an internal quality control data inter-laboratory comparison-based approach.

INTRODUCTION: Two methods were compared for evaluating the sigma metrics of clinical biochemistry tests using two different allowable total error (TEa) specifications.
MATERIALS AND METHODS: The imprecision (CV%) and bias (bias%) of 19 clinical biochemistry analytes were calculated using a trueness verification proficiency testing (TPT)-based approach and an internal quality control data inter-laboratory comparison (IQC)-based approach, respectively. Two sources of total allowable error (TEa), the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) and the People's Republic of China Health Industry Standard (WS/T 403-2012), were used to calculate the sigma metrics (σCLIA, σWS/T ). Sigma metrics were calculated to provide a single value for assessing the quality of each test based on a single concentration level.
RESULTS: For both approaches, σCLIA  > σWS/T in 18 out of 19 assays. For the TPT-based approach, 16 assays showed σCLIA  > 3, and 12 assays showed σWS/T  > 3. For the IQC-based approach, 19 and 16 assays showed σCLIA  > 3 and σWS/T  > 3, respectively.
CONCLUSIONS: Both methods can be used as references for calculating sigma metrics and designing QC schedules in clinical laboratories. Sigma metrics should be evaluated comprehensively by different approaches.

INTRODUCTION

Sigma (σ) metric was first introduced into clinical laboratories by David Nevalainen1 in 2000. The practice of using the σ to improve clinical laboratory quality has been in place for nearly two decades.2 Sigma metric has become a useful tool to monitor quality indicators,1 to assess the analytical quality of assays,3, 4 to set quality control rules,5, 6, 7, 8 to describe assay analytical performance for external quality assessment participants,9 and to help manufacturers choose product requirements.10 Analytical quality of assays is quantitatively estimated as a sigma metric based on 3 parameters: allowable total error (TEa), bias, and imprecision. The TEa from various sources, such as the Clinical Laboratory Improvement Amendments (CLIA), the College of American Pathologists (CAP), Reference Institute for Bioanalytics (Rilibak), the Royal College of Pathologists of Australasia (RCPA), and the China National Center for Clinical Laboratories (NCCL) external quality assessment goals, is associated with significantly different Sigma metrics for the same assay.8, 11 Thus, laboratories have to take into account which TEa specification is most appropriate initially.

Imprecision is usually presented as the standard deviation (SD) or coefficient of variation (CV). CV (SD) and bias often vary with the concentration of analytes, so the σ computed at different QC concentration levels could be quite different from one another.12 Previous studies have reported that individual assays' biases were calculated by comparing their mean with the statistical target mean in external quality assessment (EQA) survey reports,13, 14 but the statistical target mean was derived from statistical results, which did not have measurement traceability. Additionally, if the concentration of IQC materials differed significantly from that of EQA samples, the σ calculation may not represent the optimal quality of the analytes. On the other hand, the target means in trueness verification proficiency testing was determined by the reference methods. Therefore, the bias and imprecision from this approach were more convincing.12 Laboratories using commercial quality control could upload the IQC data to the inter‐laboratory comparison database and compare these data with the target mean of the peer group to calculate bias and CV. The internal quality control data inter‐laboratory comparison‐based approach has been shown to be a convenient and reliable method among clinical laboratories. Xinqi Cheng et al15 introduced two approaches to compute bias and CV for σ evaluation, using a proficiency testing (PT) or an internal quality control (IQC)‐based approach. Herein, we take similar approaches.1 For the trueness verification proficiency testing (TPT)‐based approach, bias was calculated by comparing the measured mean with the target mean derived from the measured results of the reference measurement procedure provided by the China National Center for Clinical Laboratories (NCCL), and the CV was calculated by testing the analytes five times daily for 3 days for each TPT sample according to the instructions from the Chinese NCCL2; for the internal quality control data inter‐laboratory comparison (IQC)‐based approach, bias was calculated by comparing the measured mean with the target mean of the global peer group, and CV was calculated with the IQC results. Based on both approaches, the bias and CV of the analyte could be acquired simultaneously at the same concentration, rather than being synthetically calculated at different concentrations. We introduced this study to compare both approaches for bias and CV calculations and to determine the effects of these approaches on σ evaluation at two TEa specifications.

MATERIALS AND METHODS

Materials

Nineteen assays were tested using the manufacturer's original reagents and calibrators on the Siemens Advia 2400 system (Siemens Healthcare Diagnostics Inc) in Beijing Tsinghua Changgung Hospital. The reference method/material traceability of the assays is shown in Table 1. The trueness verification proficiency testing (TPT) samples (including Metabolites and Total Protein/Electrolyte/Enzymes/Lipids trueness verification proficiency testing samples) for nineteen analytes were provided by China National Center for Clinical Laboratories (NCCL) in 2018. The nineteen analytes of Metabolites and Total Protein/Electrolyte/Enzymes/Lipids TPT samples were GLU, UN, CREA, UA, TP/Na, K, Ca, Mg, Cl/ALT, AST, GGT, LDH, CK, AMY, ALP/TC, and TG. The internal quality controls were the Bio‐Rad (Bio‐Rad Laboratories, Inc) Liquid Assayed Multiqual controls (Lot No. 45792/45793).

Table 1

Methods, units, traceability of the analytes on the Siemens Advia 2400 analyzer and number of laboratories included in each instrument peer group for the trueness verification proficiency testing (TPT) program of the Chinese NCCL, and the number of laboratories included in the Bio‐Rad IQC global peer group reports

Analytesa	GLU	UN	CREA	UA	TP	Na	K	Ca	Mg	Cl
Methods	Hexokinase	Urease, UV	Enzymatic	Uricase, colorimetric	Biuret	ISE indirect	ISE indirect	Arsenazo III	Xylidyl blue	ISE indirect
Units	mmol/L	mmol/L	µmol/L	µmol/L	g/L	mmol/L	mmol/L	mmol/L	mmol/L	mmol/L
Reference method traceability	NA	CDC‐R eference method	IFCC‐Reference method	CDC candidate Reference method (uricase)	Biuret Reference method	CDC Flame photometry reference method	CDC Flame photometry reference method	Inductively coupled atomic emission method	Atomic absorption	NIST coulometric reference
Reference Material Traceability	NIST SRM 965a	NIST SRM 912a, NIST SRM 909b	IRMM/IFCC‐4	NIST SRM 913a, NIST SRM 909	NIST SRM 927	NIST SRM 909b	NIST SRM 909b	NIST SRM 915, NIST SRM 90	NIST SRM 929	Method NIST SRM 919
Number of laboratories for NCCL TPT	241	241	276	246	318	320	320	318	292	319
Number of laboratories for Bio‐Rad IQC	35	17	13	40	40	50	50	29	44	49
Number of IQC data points	1695	507	653	2096	2190	2538	2529	1086	2052	2404

Abbreviations: ALP, alkaline phosphatase; ALT, alanine aminotransferase; AMY, amylase; AST, aspartate aminotransferase; Ca, calcium; CK, creatine kinase; Cl, chloride; CREA, creatinine; GGT, gamma‐glutamyltransferase; GLU, glucose; IQC, internal quality control; K, potassium; LDH, lactate dehydrogenase; LDL, low density lipoprotein cholesterol; Mg, magnesium; NA, none available; Na, sodium; NCCL, China National Center for Clinical Laboratories, Peking, China; TC, total cholesterol; TG, triglycerides; TP, total protein; TPT, trueness verification proficiency testing; UA, uric acid; UN, urea nitrogen.

All analytes were tested on the Siemens Advia 2400 analyzer using manufacturer's original reagents.

Methods

Sample preparation

All the TPT samples from the Chinese NCCL were prepared from pooled fresh human serum. Each level, with 3 separate aliquots, was frozen and stored at −70°C until shipment on dry ice to our laboratory. The samples were stored in the freezer at −70°C as soon as they were received until use. When testing, the samples were placed at room temperature for 20 minutes and then gently mixed until completely dissolved and assayed in our laboratory within 4 hours.

Bias calculation

For the TPT‐based approach, the target value of each level of nineteen analytes of TPT samples was assigned by the reference laboratories organized by the Chinese NCCL, following the recommended reference measurement procedures.16, 17, 18 The aliquot of each level was measured on 3 specific days, and each aliquot was tested under repeatability condition five times referring to the procedure of CLSI EP15‐A2.19 Thus, a total of 15 results were obtained for each concentration level. The average of the 15 results was considered as the laboratory‐tested value. All the results were reported to the online EQA platform developed by the Chinese NCCL (www.nccl.org.cn), and a summary report of the evaluation results can also be acquired on the online EQA platform.16, 18 Bias was determined as (our mean − target mean)/target mean × 100%.

For the IQC‐based approach, all IQC data of nineteen analytes were included in the Bio‐Rad global comparison program and the data were collected in the clinical laboratory of Beijing Tsinghua Changgung Hospital from July 1, 2018, to December 30, 2018. The monthly (December 2018) mean, with SD and CV, was calculated. Bias was calculated as (our mean − mean of peer group)/ mean of peer group × 100%.15

Imprecision evaluation

Imprecision in this study was determined for each level of 19 analytes of TPT samples using the 15 results according to the formula defined in CLSI guideline EP15‐A219 for the TPT‐based approach.18

Bio‐Rad Liquid Assayed Multiqual controls were assayed daily for the nineteen analytes and calculated with the IQC results for the IQC‐based approach.

Sigma calculation

The two TEa requirements, USA Clinical Laboratory Improvement Amendments of 1988—CLIA '88 (TEaCLIA) and People's Republic of China Health Industry Standard—WS/T 403‐2012 (TEaWS/T), were selected to calculate σ for each assay, using the equation σ = (TEa − bias)/ SD (for concentration units) and σ = (TEa% − bias%)/ CV% (for percentage units). Both TEa requirements of the 19 assays are listed in Table 2. Excel 2016 software (Microsoft Corporation) was used for data analysis and graphing.

Table 2

Sigma metrics computed by the trueness verification proficiency testing (TPT)‐based approach and the internal quality control data inter‐laboratory comparison (IQC)‐based approach

Item	Sample number	Mean	Target	Bias (%)	CV (%)	TEaCLIA (% [absolute value])	TEaws/t (%)	σ CLIA	σ ws/t
GLU	201911	3.66	3.70	−1.08	0.55	10 (0.33)	7.00	16.22	10.76
	201912	9.24	9.43	−2.01	0.87	10 (0.33)	7.00	9.18	5.74
	45792	6.19	6.22	−0.49	1.41	10 (0.33)	7.00	6.74	4.62
	45793	19.04	19.35	−1.63	1.18	10 (0.33)	7.00	7.09	4.55
UN	201911	5.06	4.85	4.33	1.58	9 (0.71)	8.00	2.96b	2.32b
	201912c	13.94	13.68	1.90	0.57	9 (0.71)	8.00	12.46	10.70
	45792c	14.02	14.22	−1.35	3.61	9 (0.71)	8.00	4.01	3.48
	45793	24.51	25.07	−2.24	1.97	9 (0.71)	8.00	3.43	2.92b
CREA	201911	65.20	66.41	−1.82	0.80	15 (26.5)	12.00	16.48	12.73
	201912	728.60	703.38	3.59	0.60	15 (26.5)	12.00	19.02	14.02
	45792	159.02	161.67	−1.64	1.86	15 (26.5)	12.00	7.18	5.57
	45793	569.52	571.09	−0.28	1.47	15 (26.5)	12.00	10.01	7.97
UA	201911	247.00	258.30	−4.37	3.00	17	12.00	3.54	2.54b
	201912	519.00	519.30	−0.06	1.00	17	12.00	14.94	11.94
	45792	366.99	371.93	−1.33	1.45	17	12.00	10.81	7.36
	45793	570.12	578.20	−1.40	1.17	17	12.00	13.33	9.06
TP	201911c	57.00	57.48	−0.84	1.60	10	6.00	5.73	3.23
	201912	76.50	79.66	−3.97	1.60	10	6.00	3.77	1.27b
	45792c	55.83	54.51	2.43	1.90	10	6.00	3.98	1.88b
	45793	71.83	70.40	2.03	1.70	10	6.00	4.69	2.34b
Na	201811	127.30	126.30	0.79	0.50	(4)	4.00	5.00a	6.42a
	201812c	140.80	140.36	0.31	0.40	(4)	4.00	7.12a	9.23a
	45792c	141.81	141.89	−0.06	0.40	(4)	4.00	6.88a	9.85a
	45793	159.95	159.56	0.24	0.54	(4)	4.00	4.15a	6.96a
K	201811	5.82	5.72	1.84	0.52	(0.5)	6.00	13.17a	8.00a
	201812	5.97	5.93	0.62	0.34	(0.5)	6.00	23.15a	15.82a
	45792	4.20	4.18	0.47	0.67	(0.5)	6.00	16.00a	8.25a
	45793	7.77	7.70	0.91	0.82	(0.5)	6.00	7.17a	6.21a
Ca	201811	2.10	2.08	0.82	2.38	(0.25)	5.00	4.65a	1.76a, b
	201812c	2.34	2.33	0.65	2.14	(0.25)	5.00	4.69a	2.03a, b
	45792c	2.47	2.50	−1.39	1.66	(0.25)	5.00	5.49a	2.17a, b
	45793	3.17	3.20	−1.00	1.55	(0.25)	5.00	4.39a	2.58a, b
Mg	201811	0.77	0.80	−3.99	3.90	25	15.00	5.39	2.82b
	201812c	1.05	1.07	−1.96	1.90	25	15.00	12.13	6.86
	45792c	1.11	1.09	2.12	1.98	25	15.00	11.56	6.51
	45793	1.75	1.74	0.96	1.82	25	15.00	13.21	7.71
Cl	201811c	95.20	95.19	0.01	0.40	5	4.00	12.48	9.98
	201812	107.10	105.24	1.77	0.30	5	4.00	10.77	7.43
	45792c	99.46	99.24	0.22	1.05	5	4.00	4.55	3.60
	45793	120.55	120.97	−0.34	1.19	5	4.00	3.92	3.08
ALT	201801	35.70	37.70	−5.31	5.30	20	16.00	2.77b	2.02b
	201802	62.90	67.40	−6.68	2.20	20	16.00	6.05	4.24
	201803c	99.40	109.30	−9.06	1.70	20	16.00	6.44	4.08
	201804	153.60	169.40	−9.33	1.00	20	16.00	10.67	6.67
	201805	191.60	205.30	−6.67	1.00	20	16.00	13.33	9.33
	45792c	93.17	95.55	−2.48	2.48	20	16.00	7.06	5.45
	45793	200.51	207.21	−3.24	1.71	20	16.00	9.80	7.46
AST	201801	36.70	33.90	8.26	2.20	20	15.00	5.34	3.06
	201802	73.00	70.10	4.14	1.00	20	15.00	15.86	10.86
	201803c	108.40	105.20	3.04	1.30	20	15.00	13.05	9.20
	201804	173.50	167.40	3.64	0.60	20	15.00	27.27	18.93
	201805	202.20	199.70	1.25	0.80	20	15.00	23.44	17.19
	45792c	111.36	114.25	−2.53	1.59	20	15.00	10.99	7.84
	45793	272.14	279.18	−2.52	1.43	20	15.00	12.22	8.73
GGT	201801	44.50	49.00	−9.18	2.20	15	11.00	2.65b	0.83b
	201802	74.00	77.32	−4.29	1.40	15	11.00	7.65	4.79
	201803c	147.70	150.67	−1.97	0.60	15	11.00	21.72	15.05
	201804	194.00	194.55	−0.28	0.60	15	11.00	24.53	17.87
	201805	239.90	243.32	−1.41	0.80	15	11.00	16.99	11.99
	45792	85.37	87.23	−2.13	1.71	15	11.00	7.53	5.19
	45793c	133.32	137.50	−3.04	1.52	15	11.00	7.87	5.24
LDH	201801	155.30	157.16	−1.18	0.80	20	11.00	23.53	12.28
	201802	196.90	197.55	−0.33	0.60	20	11.00	32.78	17.78
	201803	291.90	292.56	−0.23	0.70	20	11.00	28.24	15.39
	201804c	411.90	421.00	−2.16	0.40	20	11.00	44.60	22.10
	201805	517.20	526.98	−1.86	0.70	20	11.00	25.91	13.06
	45792	176.88	177.43	−0.31	1.47	20	11.00	13.39	7.27
	45793c	427.66	431.08	−0.79	1.09	20	11.00	17.62	9.37
CK	201801	139.30	139.76	−0.33	0.90	30	15.00	32.97	16.30
	201802	351.70	345.19	1.89	0.60	30	15.00	46.85	21.85
	201803c	659.30	632.56	4.23	0.50	30	15.00	51.54	21.54
	201804	864.40	842.35	2.62	0.50	30	15.00	54.76	24.76
	201805	1147.20	112.84	3.09	0.70	30	15.00	38.44	17.01
	45792	254.45	257.41	−1.15	1.44	30	15.00	20.03	9.62
	45793c	629.34	645.79	−2.55	2.52	30	15.00	10.89	4.94
AMY	201801c	124.00	118.00	5.08	0.90	30	15.00	27.69	11.02
	201802	253.70	240.31	5.57	0.80	30	15.00	30.54	11.79
	201803	436.80	409.93	6.55	0.40	30	15.00	58.63	21.13
	201804	611.20	575.36	6.23	0.40	30	15.00	59.43	21.93
	201805	722.10	676.74	6.70	0.50	30	15.00	46.60	16.60
	45792c	133.96	134.37	−0.31	1.81	30	18.00	16.40	8.12
	45793	285.64	289.69	−1.40	1.65	30	18.00	17.33	8.24
ALP	201801	105.00	94.73	10.84	0.80	30	18.00	23.95	8.95
	201802	260.90	241.75	7.92	0.60	30	18.00	36.80	16.80
	201803	454.80	402.41	13.02	0.60	30	18.00	28.30	8.30
	201804	541.60	487.27	11.15	0.40	30	18.00	47.13	17.13
	201805	635.90	580.86	9.48	0.70	30	18.00	29.31	12.17
	45792	138.54	141.77	−2.28	3.77	30	18.00	7.35	4.17
	45793	284.20	291.80	−2.60	2.40	30	18.00	11.42	6.42
TC	201811	5.78	5.63	2.72	1.21	10	9.00	6.02	5.19
	201812	3.72	3.61	2.96	1.08	10	9.00	6.52	5.59
	45792	4.32	4.21	2.51	1.41	10	9.00	5.31	4.60
	45793	7.72	7.69	0.32	1.10	10	9.00	8.80	7.89
TG	201811	1.91	1.84	4.09	1.05	25	14.00	19.91	9.44
	201812c	1.30	1.23	5.69	1.54	25	14.00	12.54	5.40
	45792c	1.54	1.49	3.50	2.06	25	14.00	10.44	5.10
	45793	4.92	4.81	2.17	1.10	25	14.00	20.75	10.75

Abbreviations: ALP, alkaline phosphatase, U/L; ALT, alanine aminotransferase, U/L; AMY, amylase, U/L; AST, aspartate aminotransferase, U/L; Ca, calcium, mmol/L; CK, creatine kinase, U/L; Cl, chloride, mmol/L; CREA, creatinine, µmol/L; CV, coefficient of variation; GGT, gamma‐glutamyltransferase, U/L; GLU, glucose, mmol/L; K, potassium, mmol/L; LDH, lactate dehydrogenase, U/L; Mg, magnesium, mmol/L; Na, sodium, mmol/L; TC, total cholesterol, mmol/L; TEaCLIA, allowable total error derived from USA Clinical Laboratory Improvement Amendments of 1988 (CLIA '88); TEaWS/T, allowable total error derived from the People's Republic of China Health Industry Standard (WS/T 403‐2012); TG, triglycerides, mmol/L; TP, total protein, g/L; UA, uric acid, µmol/L; UN, urea nitrogen, mmol/L.

σ is calculated by using absolute bias as the allowable total error (TEa), not marked indicates using percentage bias to calculate the σ.

σ < 3.

σCLIA and σWS/T were compared at similar concentration levels for each assay using TPT‐based method and IQC‐based method.

RESULTS

Sigma metrics from TPT‐based approach

Sigma metrics of 19 assays computed by the TPT‐based approach and IQC‐based approach using two TEa requirements (σCLIA and σWS/T) are listed in Table 2. The TEaCLIA used absolute bias for K, Na, and Ca assays, whereas TEaWS/T used percentage bias and was stricter than TEaCLIA. Among the 19 assays by a TPT‐based approach, 16 assays showed σCLIA > 3, 10 assays (GLU, CREA, K, Cl, LDH, CK, AMY, ALP, TC, and TG) showed σCLIA > 6, and 3 assays (UN, ALT, and GGT) showed σCLIA < 3. On the other hand, 12 assays showed σWS/T > 3, 8 assays (CREA, Na, K, Cl, LDH, CK, AMY, and ALP) showed σWS/T > 6, and 6 assays (UN, UA, Ca, Mg, ALT, and GGT) showed σWS/T < 3. σCLIA > σWS/T in 18 out of 19 assays (all except Na).

Sigma metrics from IQC‐based approach

Sigma metrics of 19 assays computed by the IQC‐based approach using two TEa requirements (σCLIA, σWS/T) are shown in Table 2. For the IQC‐based approach, 19 and 16 assays showed σCLIA > 3 (GLU, CREA, UA, K, Mg, Cl, ALT, AST, GGT, LDH, CK, AMY, ALP, and TG exceeded 6σ levels) and σWS/T > 3 (UA, Na, K, Mg, Cl, AST, LDH, AMY, ALP, TC, and TG exceeded 6σ levels), respectively. Similar to the TPT‐based approach, σCLIA > σWS/T for all assays except Na.

Comparative analysis of sigma metrics between both methods

Nine analytes (GLU, CREA, K, Cl, LDH, CK, AMY, ALP, and TG) had a σCLIA above 6, and 6 analytes (Na, K, Cl, LDH, AMY, and ALP) had a σWS/T above 6 from both methods, which is actually effectively in agreement.

As shown in Figure 1, the σCLIA and σWS/T derived from both methods for nineteen analytes were significantly different. For instance, the σCLIA for the TPT‐based approach vs the IQC‐based approach of UN at similar concentration levels was 12.46 (201912) versus 4.01 (45792), respectively, and similarly, the σWS/T was 10.70 (201912) vs 3.48 (45792), respectively. The σCLIA and σWS/T comparisons at similar concentration levels for both methods are shown in Table 2.

Figure 1

Comparison of σCLIA and σWS/T calculated using the trueness verification proficiency testing (TPT)‐based approach and the internal quality control data inter‐laboratory comparison (IQC)‐based approach for the same analyte. Note: solid lines are used for σCLIA, and dashed lines are used for σWS/T. 201801‐201805, 201811‐201812, and 201911‐201912 represent the lot numbers of trueness verification proficiency testing materials, and 45792 and 45793 represent the lot number of Bio‐Rad biochemistry quality control materials. ALP, alkaline phosphatase, U/L; ALT, alanine aminotransferase, U/L; AMY, amylase, U/L; AST, aspartate aminotransferase, U/L; Ca, calcium, mmol/L; CK, creatine kinase, U/L; Cl, chloride, mmol/L; CREA, creatinine, µmol/L; GGT, gamma‐glutamyltransferase, U/L; GLU, glucose, mmol/L; K, potassium, mmol/L; LDH, lactate dehydrogenase, U/L; Mg, magnesium, mmol/L; Na, sodium, mmol/L; TC, total cholesterol, mmol/L; TG, triglycerides, mmol/L; TP, total protein, g/L; UA, uric acid, µmol/L; UN, urea nitrogen, mmol/L

DISCUSSION

Six Sigma and analytical sigma metrics are a widely accepted measure of assessing method quality, optimizing QC procedures, and processing improvement.2, 7 The sigma metrics computed at different concentration levels can be quite different from one another. Another challenge in calculating sigma metrics is how to acquire the appropriate TEa, bias, and CV (SD). Therefore, this study was introduced to explore the comparative analysis of sigma metrics computed by both methods.

There are different TEa quality requirements for routine chemistry,20 and the optimal TEa quality requirements should be established depending on the conditions and requirements of the individual laboratory. At present, the first choice of TEa selection in China is the WS/T 403‐2012, which is the standard of the Ministry of Health published analytical quality specifications for routine clinical biochemistry in 2012. If the standard is too strict, other requirements, such as the CLIA'88, the College of American Pathologists (CAP), guidelines of the German medical association for the quality assurance of laboratory medical examinations (RiliBÄK), and The Royal College of Pathologists of Australasia quality assurance programs (RCPA), could also be used.2, 20 If the TEa is too strict, the laboratory should assess the suitability of the TEa for clinical use or patient care and determine if that would allow a larger TEa choice. The European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) published a recommended hierarchy for choosing an appropriate TEa whereby setting‐required performance specifications are based on 3 possible models: clinical outcomes, biological variabilities, and state‐of‐the‐art.21 It provides a feasible model for standardizing protocols and is being introduced to encourage all laboratories to use the same quality goals for their sigma metrics benchmarks.22 Because TEaWS/T was more stringent than TEaCLIA for the most analytes, the σWS/T was significantly lower than that of σCLIA in 18 out of 19 assays (all except Na). Therefore, the limitation of selecting an appropriate TEa requirement for σ calculation should be considered.

Bias is an estimate of systematic measurement error.23 Assessing bias can be challenging, and bias evaluation can significantly impact the sigma metrics. Bias is always evaluated using the following methods.8 (a) The optimal method is to compare results obtained from fresh human specimens using the measurement procedure and a reference measurement procedure. In this study, we used fresh frozen human serum samples provided by trueness verification proficiency testing program from the Chinese NCCL. The target means of nineteen analytes were determined using the corresponding reference measurement procedure from the Chinese NCCL. Samples derived from human serum could avoid the matrix effect, which was caused by the interaction of processed material and the measurement procedure and may suggest that erroneous results are being generated when in fact the results are acceptable.24 The use of fresh frozen human specimens could avoid the matrix‐related bias.25 (b) Another way to assess relative bias is by comparing laboratory results with the statistical mean of the peer group using the same instrument and method from inter‐laboratory QC data or proficiency testing (PT)/external quality assessment (EQA) reports. Because the target means in PT/EQA programs were derived from statistical results without measurement traceability,15, 26, 27 those using this approach should be aware of the possible limitations, including statistical methods used to generate the data and the number of laboratories that participate.23

Imprecision is typically expressed as an SD or CV. The imprecision estimation for sigma metrics should be based on results from a sufficient duration of time to adequately represent the types of influences, such as a periodic recalibration, changes in bottles of reagents, changes in lots of reagents or calibrators, and maintenance procedures. We used six months of IQC data to obtain relatively stable results. By contrast, the TPT‐based approach may lead to a lower CV and an overestimated σ value because of the short‐term assessment.15

We further studied the QC rule design based on the sigma metrics of nineteen analytes designed based on sigma metrics by both methods using two TEa standards (data not shown in this study). The differences in QC rules, QC numbers, and QC run sizes as calculated using both approaches and two TEa standards were analyzed. The QC setting of nineteen analytes was compared test by test, that of seven analytes were the same and twelve analytes were different. QC rules based on TEaws/t were more stringent than that based on TEaCLIA. We are working on the adjustment of the individualized QC plan of the analyte based on its sigma metric evaluation.

One weakness of this study is that the imprecision evaluation following the Chinese NCCL instruction by the TPT approach (analyze five replicate samples at each of different concentrations daily for 3 days to obtain 15 results) was not conducted strictly according to the method described in the CLSI guideline EP15‐A2 document (analyze one run per day with 3 replicate samples at each of different concentrations daily for 5 days to obtain 15 results) and CLSI guideline EP5‐A3 (analyze two replicate samples at each of different concentrations daily for 20 days to obtain 40 results).28 The TPT approach only lasted 3 days, which may have underestimated the imprecision.18 The numbers of laboratories for the Bio‐Rad IQC method from peer group comparison ranged between 5 and 50, and there may be increased imprecision due to the small number of laboratories. The concentration of analytes of the IQC materials differed significantly from that of the TPT samples, which may contribute to the differences in the calculation of σ based on both methods. Moreover, the research results may encourage the application of different methods for sigma evaluation, without the cost of high maintenance of commercial QC materials.

In conclusion, a combination of the TPT‐based and IQC‐based approach using NCCL and CLIA TEa goals may be adopted as the useful approaches for sigma metrics evaluation in clinical laboratories. When applying sigma metrics for Six Sigma management or quality control, sigma metrics should be evaluated comprehensively by different approaches.

CONFLICT OF INTEREST

None declared.