Recent Trends in Creatinine Assays in Korea: Long-Term Accuracy-Based Proficiency Testing Survey Data by the Korean Association of External Quality Assessment Service (2011-2019).

BACKGROUND: Accurate serum creatinine (Cr) concentration measurement is essential for evaluating kidney function. In 2011, the Korean Association of External Quality Assessment Service (KEQAS) launched an accuracy-based Cr proficiency testing (ABCr PT) survey. We analyzed long-term data of the KEQAS ABCr PT survey collected between 2011 and 2019 to assess recent trends in Cr assays in Korea.
METHODS: The ABCr PT survey including three commutable fresh-frozen serum samples was performed twice a year. The target Cr concentration was assigned using isotope-dilution mass spectrometry. We analyzed data obtained from the participating laboratories, calculated the yearly bias, and evaluated bias trends for the major reagents and instruments. Outliers were excluded from all analysis.
RESULTS: The mean percentage bias based on the total data of all participating laboratories was 10.8% in the 2011-A survey and 0.2% in 2019-B survey. Bias for the major reagents and instruments differed depending on the manufacturer. Enzymatic assays generally showed desirable bias ranging from -3.9% to 3.2% at all Cr concentrations and lower interlaboratory variability than non-enzymatic assays (enzymatic vs. non-enzymatic, 3.3%-7.2% vs. 6.3%-9.1%).
CONCLUSIONS: Although the mean percentage bias of Cr assays tends to decrease over time, it is necessary to continuously strive to improve Cr assay accuracy, especially at low concentrations.

INTRODUCTION

The estimated glomerular filtration rate (eGFR) is the single best indicator for evaluating kidney function [1]. eGFR calculations recommended by organizations such as the Kidney Disease Improving Global Outcomes (KDIGO) and National Kidney Disease Education Program (NKDEP) are mainly based on serum creatinine (Cr) and cystatin C, and Cr is widely used in current clinical practice [2]. Cr-based eGFR values form the basis for medical decision making, including diagnosis, classification, prognosis, and determination of treatment policies, for chronic kidney disease and/or acute kidney injury [2-5]. Accurate Cr concentration measurement is essential to maximize the value of the laboratory test data. Clinical laboratories should use a calibrator that is traceable by an isotope dilution mass spectrometry (IDMS) measurement procedure, and the precision and accuracy of Cr concentration measurement should be continuously monitored through internal quality control and proficiency testing (PT) [6-8].

The Korean Association of External Quality Assessment Service (KEQAS) has been conducting the accuracy-based Cr PT (ABCr PT) survey since 2011; we previously reported the results of this survey conducted between 2011 and 2017 [9]. The mean percentage bias for all participating laboratories significantly decreased from 11.1% in 2011 to 2.4% in 2017 [9]. However, the previous study included outliers, and detailed analyses according to the assay principle (i.e., enzymatic vs. Jaffe method) or target Cr concentration (i.e., low vs. high) were not performed and their effects on bias were not evaluated. Until 2019, participation in the KEQAS ABCr PT survey was voluntary; however, 2020 onward, all laboratories participating in KEQAS PT have to participate in the ABCr PT survey. More small- to medium-sized laboratories may participate from 2020; however, to date, mainly medium- to large-sized laboratories were included. As laboratory size can affect analytical performance [10], we thought it would be meaningful to analyze bias trends based on KEQAS ABCr PT survey data collected until 2019. We analyzed KEQAS ABCr PT survey data from 2011 to 2019 to identify trends in Cr assay standardization across participating laboratories and to explore ways to improve the quality of Cr assays based on long-term ABCr PT survey data.

MATERIALS AND METHODS

KEQAS ABCr PT survey

We retrospectively reviewed the KEQAS ABCr PT survey data. This study did not involve human subjects and was exempted from the approval of Institutional Review Board of Ewha Womans University Seoul Hospital. The KEQAS ABCr PT survey was performed twice a year (A and B trials in sequential order) from 2011 to 2019. Three samples of fresh frozen serum were used in each survey and were processed according to the Clinical & Laboratory Standards Institute document C37-A, with modification [11]. Venous blood was drawn from donors into plastic blood bags without anticoagulant. Within 5 minutes of blood collection, plasma was separated by centrifugation at 1,500×g, 4°C for 8 minutes. The plasma was transferred to a clean and sterile borosilicate glass bottle and allowed to clot at room temperature for 4 hours. The serum was separated by centrifugation at 2,700 ×g for 20 minutes. Cr was spiked to prepare high Cr concentration serum pools. Serum pools were incubated at 4°C for 14 hours under constant, low-speed, magnetic stirrer mixing to ensure homogeneity. The serum pools were then filtered through a 0.22-µm hydrophilic membrane and aliquoted into glass vials at 4°C. The samples were stored at –70°C until shipment to ABCr PT survey participants. The target Cr concentration was measured by IDMS in the Reference Material Institute for Clinical Chemistry Standards (Kawasaki, Japan).

Outlier elimination

The distribution of Cr data from each participating laboratory was visually observed, and any difference greater than ±50% from the target Cr concentration was considered an outlier and excluded from further analyses. Detailed participant responses and the number of outliers are described in Table 1.

Table 1

Bias based on KEQAS ABCr PT survey data from 2011 to 2019

Trial	Total N of participating labs	N of response results[*]	N of outliers[†]	Bias (%)		Bias (mg/dL)
				Mean (95% CI)	SD	Mean (95% CI)	SD
2011-A	54	161	1	10.82 (9.45–12.19)	12.15	0.096 (0.080–0.112)	0.102
2011-B	56	166	0	7.12 (5.77–8.46)	11.02	0.075 (0.058–0.091)	0.108
2012-A	103	303	2	9.33 (8.33–10.33)	10.79	0.082 (0.072–0.092)	0.090
2012-B	108	323	0	6.18 (5.22–7.15)	9.86	0.062 (0.050–0.073)	0.105
2013-A	140	420	1	5.78 (4.93–6.62)	10.10	0.049 (0.036–0.061)	0.133
2013-B	139	416	0	7.63 (6.78–8.48)	10.81	0.066 (0.057–0.074)	0.091
2014-A	176	527	4	4.04 (3.29–4.80)	11.29	0.024 (0.013–0.036)	0.132
2014-B	167	501	2	–0.01 (–0.79–0.76)	7.07	–0.004 (–0.012–0.004)	0.091
2015-A	175	525	1	1.36 (0.60–2.11)	6.67	0.018 (0.011–0.026)	0.090
2015-B	178	534	1	3.64 (2.89–4.39)	7.97	0.036 (0.028–0.045)	0.100
2016-A	146	437	3	2.37 (1.54–3.20)	7.69	0.02 (0.011–0.029)	0.098
2016-B	150	450	2	4.01 (3.20–4.83)	10.04	0.027 (0.019–0.036)	0.091
2017-A	153	459	3	1.77 (0.96–2.58)	8.82	0.008 (0.000–0.015)	0.080
2017-B	167	501	3	2.05 (1.27–2.82)	8.95	0.014 (0.008–0.02)	0.068
2018-A	197	591	1	2.36 (1.65–3.07)	8.57	0.008 (0.000–0.015)	0.093
2018-B	200	600	0	3.87 (3.17–4.58)	8.11	0.033 (0.027–0.039)	0.074
2019-A	227	681	2	0.45 (–0.21–1.11)	6.40	–0.003 (–0.009–0.003)	0.077
2019-B	227	681	2	0.20 (–0.47–0.86)	7.36	–0.018 (–0.026––0.009)	0.111

*N=3 samples per trial; †An outlier was defined as a result with an absolute percentage bias >50% of the target Cr concentration.

Abbreviations: ABCr PT, accuracy-based creatinine proficiency testing; Cr, creatinine; KEQAS, Korean Association of External Quality Assessment Service.

Statistical analysis

To investigate bias (measured Cr concentration–target Cr concentration) trends in Cr assays for all participants from 2011 to 2019, the mean percentage bias and 95% confidence interval (CI) for the mean were calculated after outlier elimination. For the bias analysis, samples were classified into three groups based on the target Cr concentrations (<88.40 µmol/L [1.00 mg/dL], 88.40–175.92 µmol/L [1.00–1.99 mg/dL], and 176.80–264.32 µmol/L [2.00–2.99 mg/dL]), and into two groups based on the assay principle (enzymatic vs. non-enzymatic methods) of Cr concentration measurement. The non-enzymatic assays were subdivided into kinetic Jaffe with compensation, kinetic Jaffe without compensation, and rate-blanked and compensated kinetic Jaffe. The assay principles reported may differ from those used in actual laboratory practice, as they were simply divided into enzymatic and non-enzymatic methods. The interlaboratory coefficient of variation (CV) depending on the Cr assay principle was calculated. To evaluate the cumulative ABCr PT survey participation effect, bias was analyzed by grouping the responses in order of survey participation. In total, 328 laboratories participated in the ABCr PT survey over nine years. Nine of these laboratories participated only once, and 32 laboratories participated in all 18 surveys. Based on the first bias results, post-hoc analysis was conducted using Dunnett’s test. Forest plots were used to identify bias trends for major reagents and instruments that were used by more than 10 participants per trial of the ABCr PT survey.

For statistical analysis, MedCalc Statistical Software version 18.10.2 (MedCalc Software Bvba, Ostend, Belgium) and Analyse-it for Microsoft Excel 5.51.1 (Analyse-it Software, Leeds, UK) were used. P<0.05 was considered statistically significant.

RESULTS

The mean percentage bias (95% CIs) for all participants was 10.8% (9.5%–12.2%) for 2011-A, and 0.2% (–0.5%–0.9%) for 2019-B. Between 2011 and 2019, the mean percentage bias significantly decreased (P for trend <0.001). Detailed mean percentage bias and 95% CIs for all ABCr PT survey data are presented in Fig. 1 and Table 1.

Fig. 1

Trend in mean percentage bias in the ABCr PT survey by the KEQAS (2011–2019). N represents the number of participating laboratories in the first (A) and second (B) ABCr PT survey of the year.

Abbreviations: ABCr PT, accuracy-based creatinine proficiency testing; KEQAS, Korean Association of External Quality Assessment Service.

The bias differed depending on the Cr concentration. For the group with Cr concentrations <1.00 mg/dL, the mean percentage bias was 12.8% in 2011 and 3.5% in 2019. The group with Cr concentrations of 1.00–1.99 mg/dL showed 2.5%–6.3% positive bias between 2011 and 2016, and a ±1.5% decrease after 2017. The group with Cr concentrations of 2.00–2.99 mg/dL showed a bias of ±3% regardless of the year (Fig. 2A).

Fig. 2

Mean percentage bias in the ABCr PT survey stratified by (A) Cr concentrations and (B) assay methods.

Abbreviations: ABCr PT, accuracy-based creatinine proficiency testing; Cr, creatinine; NT, not tested.

The mean percentage bias for enzymatic assays was within ±4% from 2011 to 2019, and the interlaboratory %CV was 3.3%–7.2%. The mean percentage bias for non-enzymatic methods was 9.8% in 2011 and within ±3.5% since 2014. The interlaboratory %CV of non-enzymatic assays was 6.3%–9.1% (Fig. 2B and Table 2).

Table 2

Interlaboratory percentage coefficient of variations

Year	Interlaboratory coefficient of variation (%)
	All	Enzymatic methods	Non-enzymatic methods
2011	9.5	7.2	9.1
2012	8.3	4.1	8.3
2013	8.6	4.6	8.4
2014	8.0	5.2	8.1
2015	6.7	6.1	6.5
2016	7.7	7.2	7.7
2017	8.2	4.8	8.3
2018	6.9	4.7	6.9
2019	6.1	3.3	6.3

Compared with that in the first participation, the bias significantly decreased after more than four participations (1st vs. 4th, difference in mean percentage bias –2.06%, 95% CI –3.35%– –0.77%, P<0.001) (Table 3).

Table 3

Percentage bias of ABCr PT survey by cumulative trial

Cumulative trial	N of participating laboratories	Bias (%)		Difference in mean percentage bias compared with the 1st trial[*]
		Mean (95% CI)	SD	Mean (95% CI)	P
1st	328	5.51 (4.94–6.08)	10.87	Reference
2nd	319	4.74 (4.16–5.31)	9.95	−0.77 (−1.98–0.44)	0.584
3rd	268	4.31 (3.68–4.93)	10.61	−1.20 (−2.47–0.07)	0.081
4th	258	3.45 (2.80–4.09)	10.24	−2.06 (−3.35– −0.77)	< 0.001
5th	206	3.29 (2.56–4.01)	9.44	−2.22 (−3.60– −0.84)	< 0.001
6th	196	2.66 (1.91–3.40)	9.20	−2.85 (−4.25– −1.45)	< 0.001
7th	172	2.60 (1.80–3.39)	8.68	−2.91 (−4.37– −1.45)	< 0.001
8th	163	1.51 (0.70–2.33)	7.85	−3.99 (−5.48– −2.50)	< 0.001
9th	133	2.32 (1.42–3.22)	7.14	−3.18 (−4.78– −1.59)	< 0.001
10th	128	3.08 (2.17–4.00)	7.78	−2.42 (−4.04– −0.81)	< 0.001
11th	111	1.82 (0.84–2.79)	6.72	−3.69 (−5.38– −1.99)	< 0.001
12th	107	2.40 (1.40–3.41)	8.03	−3.10 (−4.83– −1.37)	< 0.001
13th	88	1.86 (0.75–2.96)	7.42	−3.65 (−5.51– −1.78)	< 0.001
14th	85	1.68 (0.55–2.81)	7.33	−3.83 (−5.72– −1.93)	< 0.001
15th	67	1.65 (0.37–2.92)	6.27	−3.86 (−5.95– −1.77)	< 0.001
16th	60	2.16 (0.81–3.50)	6.61	−3.35 (−5.53– −1.16)	< 0.001
17th	38	0.02 (−1.67–1.72)	5.36	−5.48 (−8.15– −2.81)	< 0.001
18th	32	0.24 (−1.58–2.07)	7.18	−5.26 (−8.13– −2.40)	< 0.001

*Dunnett’s test.

Abbreviations: ABCr PT, accuracy-based creatinine proficiency testing; CI, confidence interval.

Beckman Coulter reagents showed 6.8%–7.2% positive bias in 2011–2013, <1% positive bias in 2014–2018, and –2.5% negative bias in 2019. Roche reagents had an overall –0.1%–4.1% positive bias. Sekisui reagents showed 6.0%–12.7% positive bias between 2011 and 2017, which decreased to 2.7% in 2019. Siemens reagents had 2.1%–6.1% positive bias between 2011 and 2013, and –5.5% negative bias as of 2014 (Fig. 3A). The bias for Beckman Coulter, Roche, and Siemens instruments was similar to that for the assay reagents. The Hitachi instrument had 4.1%–9.7% positive bias between 2011 and 2017 and 2.6%–5.1% in 2018–2019. The Toshiba instrument had a 13.9% positive bias in 2011, which gradually decreased to –1.2% in 2019 (Fig. 3B).

Fig. 3

Forest plot of mean percentage bias in the ABCr PT survey stratified by year. (A) Reagent manufacturer group. (B) Instrument manufacturer group. A and B in parenthesis are the number of participating laboratories in the first and second ABCr PT survey of the year, respectively.

Abbreviations: ABCr PT, accuracy-based creatinine proficiency testing; CI, confidence interval.

DISCUSSION

The strength of this study was that the bias trend analysis for Cr assays was based on KEQAS ABCr PT survey data collected over nine years. The mean percentage bias for all participants, excluding outliers, significantly decreased from 10.8% for 2011-A to 0.2% for 2019-B. The decrease in bias was the most prominent for non-enzymatic assays, with Cr concentrations <1.0 mg/dL. This trend was similar to that in previous studies [10, 12, 13].

For the Beckman Coulter and Siemens platforms, the annual bias had decreased significantly since 2014. This is mainly because an IDMS-traceable calibrator was used in the participating laboratories. Standardized Cr assays with an IDMS-traceable calibrator yielded Cr concentrations 0.1–0.2 mg/dL lower than non-IDMS-traceable assays, indicating that only changes in the calibrator can be expected to improve positive bias in Cr assays [14, 15].

Constant positive bias, probably due to a systematic error, was observed for the Sekisui reagents. The bias especially increased in 2016–2017 and resulted in more than a half of the participants using these reagents reporting “unacceptable” results for low-concentration samples. The main reason for the increased bias in this period was the relatively large number of low-concentration (Cr<1.0 mg/dL) samples. Some laboratories discussed this issue with the manufacturer to decrease the bias and used compensated kinetic Jaffe assays to minimize the constant positive bias. As a result, the bias has decreased as of 2018 (i.e., mean %bias: 11.7% in 2016; 12.7% in 2017; 5.9% in 2018; and 2.7% in 2019). This indicates that assay standardization can be greatly improved when clinical laboratories, manufacturers, and external quality assessment (EQA) agencies combine efforts.

The optimum, desirable, and minimum acceptable bias considering biological variation for Cr assays are 1.7%, 3.4%, and 5.1%, respectively, by the NKDEP and 1.9%, 3.7%, and 5.6%, respectively, by the latest European Federation of Clinical Chemistry and Laboratory Medicine data [16, 17]. ABCr PT survey participants using an enzymatic method had a mean percentage bias of –3.9%–3.2% from 2011 to 2019, satisfying the desirable bias criterion regardless of year and Cr concentration. Enzymatic methods avoid interference by Jaffe-like chromogens, such as glucose, protein, and acetone, allowing more accurate Cr concentration measurements than the Jaffe method [18, 19].

Participants who used non-enzymatic methods exceeded the minimum acceptable bias criterion in 2011, with a 9.8% mean percentage bias, but generally met the desirable bias criterion after 2014. However, unlike the enzymatic method, bias significantly differed depending on the Cr concentration (i.e.,<1.0 mg/dL vs. ≥1.0 mg/dL) for the non-enzymatic method. As of 2014, samples with a Cr concentration ≥1.0 mg/dL showed a –1.3%–2.0% bias and usually met the optimum bias criterion, whereas samples with a Cr concentration <1.0 mg/dL showed 3.6%–7.2% mean percentage bias, resulting in many cases not meeting the minimum acceptable bias criterion. Although the Cr bias for ABCr PT survey participants gradually decreases over time, the non-enzymatic method still overestimates Cr concentration and consequently underestimates eGFR, especially in low-concentration samples. Relatively larger bias in the Jaffe method than in the enzymatic method at low Cr concentrations has been reported [10, 13, 20].

Conversely, a negative bias tendency was observed in high-concentration samples. There was –2% to –3% negative bias in samples with a concentration >2.0 mg/dL. In the Italian study, the bias distribution in 227.2 μmol/L (2.57 mg/dL) samples was –8.4% to –1.0% in 2011 [12]. Similarly, in the 2011 College of American Pathologists survey, the bias for 2.77 mg/dL samples was –4.9%–0.9%, and that of 4.09 mg/dL samples was –4.7%–0.4%, revealing an overall negative bias [10]. The different bias tendencies between low- and high-concentration samples are assumed to be a calibration effect. Currently, the surest way to decrease bias in low-concentration samples and inter-laboratory variation is to use enzymatic Cr assays. However, this method has practical limitations, including high cost.

This study had some limitations. First, the Cr assay status in small laboratories was not represented. Second, although outliers were removed, the definition of an outlier lacks scientific rationale; the mean percentage bias in Cr concentration measurement for the participating laboratories in the KEQAS ABCr PT survey may vary depending on how the outlier is defined. Third, the bias for the detailed Jaffe method was not analyzed.

In conclusion, the long-term KEQAS ABCr PT survey demonstrated the Cr assay standardization status across participants and provided information for improving assay quality. Overall, the bias in Cr assay tends to decrease with participation in the KEQAS ABCr PT survey; however, continuous efforts are needed to decrease the bias for low-concentration samples. Clinical laboratories should consistently monitor Cr assay accuracy by using standardized calibrators and reagents that maintain metrological traceability, and by participating in an ABCr PT program. Coordinated efforts of clinical laboratories, manufacturers, and EQA agencies may further standardize Cr assay.