Concordance of Three Automated Procalcitonin Immunoassays at Medical Decision Points.

Procalcitonin (PCT) is a useful bacterial infection biomarker with the potential for guiding antibiotic therapy. We evaluated the concordance of three automated PCT immunoassays: Kryptor (BRAHMS GmbH, Hennigsdorf, Germany), Atellica IM 1600 (Siemens Healthcare Diagnostics, Munich, Germany), and Cobas e801 (Roche Diagnostics, Mannheim, Germany). In 119 serum samples with a PCT concentration <5.00 μg/L, Kryptor (reference assay) was compared with the other two immunoassays by Spearman's rank correlation, regression analysis, and concordance at two antibiotic stewardship medical decision points: 0.25 and 0.50 μg/L. The Atellica IM 1600 and Cobas e801 results showed high correlations with those of Kryptor, with correlation coefficient (ρ) values of 0.97 and 0.99, respectively. However, negative biases were observed in both immunoassays (slope/y-intercept: 0.75/-0.00 for Atellica IM 1600; 0.88/-0.01 for Cobas e801). Atellica IM 1600 and Cobas e801 demonstrated excellent concordance with Kryptor at both medical decision points, with linearly weighted κ values of 0.90 and 0.92, respectively, despite discrepancies, which were more prominent at the 0.25 μg/L medical decision point. Based on these biases and discrepancies, the alternate use of different PCT immunoassays in repeat examinations is inadvisable. Standardization is required before comparing the results of different PCT immunoassays.

Procalcitonin (PCT) is a useful biomarker of bacterial infection and its resolution [1-3]. Repeated PCT concentration assessment can be used to monitor the therapeutic response to guide medical decisions on early discontinuation of antibiotics and support personalized antibiotic management [2, 3]. Multiple studies have shown that PCT-guided antibiotic initiation and discontinuation can shorten the duration of antibiotic treatment, which can in turn reduce the associated side effects and improve clinical outcomes [4-7]. Recently, two medical decision points have been suggested for PCT-guided antibiotic stewardship: 0.25 and 0.50 μg/L for non-intensive care unit (ICU) and ICU patients, respectively [2].

Sensitive and accurate determination of PCT concentration is required for its use as an antibiotic stewardship guide. Numerous fully automated PCT immunoassays are available [8]. However, because PCT reference material and methods are lacking, the comparability of these immunoassays is not guaranteed, necessitating comparative studies. Kryptor (BRAHMS GmbH, Hennigsdorf, Germany) was the first fully automated PCT immunoassay [9] and it was used in several clinically important studies to measure PCT concentrations [4, 6, 10]. Thus, most published comparative studies have used Kryptor as the reference method [11-20].

We examined whether PCT concentrations measured using Atellica IM 1600 (Siemens Healthcare Diagnostics, Munich, Germany) and Cobas e801 (Roche Diagnostics, Mannheim, Germany) are equivalent to those measured using Kryptor in terms of PCT-guided antibiotic stewardship employing the medical decision points of 0.25 and 0.50 μg/L. To the best of our knowledge, this is the first study to evaluate Atellica IM 1600 in this regard. Our results would improve the understanding of how PCT concentrations can be influenced by different immunoassay principles employing the same BRAHMS PCT antibody.

For this comparison study, clinical serum samples were collected from patients at Severance Hospital, Seoul, Korea, from March to April 2019. A routine PCT immunoassay was performed using Cobas e601 upon collection. Immediately thereafter, 119 serum samples with a PCT concentration <5.00 μg/L were selected to increase the number of samples with PCT concentrations near the medical decision points, a strategy not applied previously. Selected samples were divided into three identical aliquots and stored at –70°C. All aliquots were thawed at 20°C for an hour and immediately analyzed simultaneously for PCT concentration using three different immunoassays: Kryptor (BRAHMS PCT-sensitive KRYPTOR), Atellica IM 1600 (Atellica IM BRAHMS PCT), and Cobas e801 (Elecsys BRAHMS PCT). Detailed descriptions and technical data of these three immunoassays are summarized in Supplemental Data Table S1. This study was reviewed and the need for informed consent for reviewing medical records of the study population has been waived by the Institutional Review Board of Yonsei University of Medicine (4-2020-0013).

The PCT concentrations measured by Atellica IM 1600 and Cobas e801 were compared with those measured by Kryptor (used as the reference assay). Spearman’s rank correlation coefficient (ρ) values, Passing-Bablok regression, Bland–Altman difference plots, and Cohen’s kappa coefficient (κ) were used for statistical analyses. The linearly weighted kappa coefficient across two PCT medical decision points, 0.25 and 0.50 μg/L, was used to assess overall concordance. All data were analyzed using Analyse-it (Analyse-it Software Ltd., Leeds, UK). The statistical significance threshold was set at P<0.05.

All PCT concentrations measured by Kryptor were within the analytical measurement range of 0.03–4.63 μg/L (median: 0.30 μg/L; interquartile range: 1.26). Atellica IM 1600 and Cobas e801 failed to detect PCT concentrations below the limit of detection (Supplemental Data Table S1) in 12 and six samples, respectively. These samples were excluded from the correlation and comparison analyses. Correlation analyses of the Atellica IM 1600 and Cobas e801 PCT concentrations with the Kryptor PCT concentrations yielded strong ρ values of 0.97 and 0.99, respectively (P<0.05 for both assays). However, both Atellica IM 1600 and Cobas e801 showed a negative bias compared with Kryptor (Fig. 1A, B). The negative bias of Cobas e801 compared with Kryptor was more obvious at lower PCT concentrations, as the –0.01 negative constant bias was too high to be disregarded at lower PCT concentrations (e.g., a –0.01 constant bias only affects approximately –1.0% at 1.00 μg/L vs –10.0% at 0.10 μg/L). Atellica IM 1600 and Cobas e801 PCT concentrations showed a strong correlation (ρ=0.98; P<0.05) (Fig. 1C).

Fig. 1

Comparative analysis of PCT concentrations measured by Atellica IM 1600, Cobas e801, and Kryptor by Passing–Bablok regression (left) and Bland–Altman plots (right). (A) Atellica IM 1600 vs. Kryptor (n=107). (B) Cobas e801 vs. Kryptor (n=113). (C) Atellica IM 1600 vs. Cobas e801 (n=107).

Abbreviations: PCT, procalcitonin; CI, confidence interval.

Next, we investigated the influence of these systemic biases on the clinical interpretation of the PCT concentrations. To this end, the PCT concentrations of all 119 samples measured by all three immunoassays were classified into three categories based on the two medical decision points (0.25 and 0.50 μg/L). The overall concordance across the two PCT medical decision points (0.25 and 0.50 μg/L) of Atellica IM 1600 vs. Kryptor and Cobas e801 vs. Kryptor was 89.9 and 92.4%, respectively, with excellent linearly weighted κ values (κ=0.90 for Atellica IM 1600; κ=0.92 for Cobas e801; Table 1). However, compared with Kryptor, Atellica IM 1600 underestimated the PCT concentration in 11 samples and overestimated it in one sample, whereas Cobas e801 underestimated the PCT concentration in nine samples (no overestimation; Table 2). Notably, the discordance did not surpass either medical decision point in the Atellica IM 1,600 vs Kryptor or Cobas e801 vs Kryptor comparison, i.e., no sample was <0.25 µg/L by one method and >0.50 by the other (Table 2). Thirteen of the 119 samples (10.9%) measured by all three immunoassays were classified discordantly by at least one immunoassay; 10 at the 0.25 µg/L and three at the 0.50 μg/L medical decision point.

Table 1

Concordance and κ value analysis of PCT concentrations measured by Atellica IM 1600 and Cobas e801 compared with Kryptor (n=119)

Medical decision point (µg/L)	Atellica IM 1600 vs. Kryptor	Cobas e801 vs. Kryptor
0.25[*]	91.6% (κ= 0.83)	95.0% (κ= 0.90)
0.50[*]	98.3% (κ= 0.97)	97.5% (κ= 0.95)
0.25 and 0.50[†]	89.9% (linear weighted κ= 0.90)	92.4% (linear weighted κ= 0.92)

*One medical decision point was used to classify the PCT concentrations into two categories. †Two medical decision points were used simultaneously to classify the PCT concentrations into three categories. Abbreviation: PCT, procalcitonin.

Table 2

PCT concentration classification based on two medical decision points, 0.25 and 0.50 μg/L (n=119)

		Kryptor (µg/L)
		< 0.25	0.25–0.50	> 0.50
Atellica IM 1600 (µg/L)	< 0.25	54	9	-
	0.25–0.50	1	3	2
	> 0.50	-	-	50
Cobas e801 (µg/L)	< 0.25	55	6	-
	0.25–0.50	-	6	3
	> 0.50	-	-	49

Abbreviation: PCT, procalcitonin.

These results indicate that despite excellent concordance, both Atellica IM 1600 and Cobas e801 still risk leading to premature cessation of antibiotics and treatment failure due to underestimation of the PCT concentration compared with Kryptor. These discrepancies are more pronounced at the lower 0.25 μg/L medical decision point. Thus, cautious interpretation of PCT concentrations measured by these immunoassays is needed, especially when they are near the lower 0.25 μg/L medical decision point. Atellica IM 1600 could not be compared with other instruments because of a lack of published data. The Cobas e801 vs Kryptor comparison results are largely consistent with previous data [19]. Although Lippi, et al. [18] did not place a 5.00 μg/L limit on sample collection, they reported a similar correlation (ρ: 0.99; P<0.05) compared with our data; however, they reported a lower negative bias (slope: 0.89; y-intercept: –0.01; mean percentage difference: –14.9% [95% CI: –18.5 to –11.2%]) when comparing Cobas e801 with Kryptor. In addition, they obtained similar concordance results, with more discrepancies at the lower medical decision point of 0.25 μg/L than at 0.50 μg/L (96% at 0.25 μg/L and 99% at 0.50 μg/L) [19]. As all three immunoassays utilize the BRAHMS PCT monoclonal antibody, these discordances are likely to be related to the use of different calibrators by the manufacturers and/or lack of common reference materials, in addition to the innate differences in the immunoassay principles.

Our study has some limitations. First, we did not include immunoassays employing antibodies other than BRAHMS PCT. However, the evaluated immunoassays showed biases and discrepancies, supporting the necessity of reference materials and standardization methods [15, 17, 19, 21]. Second, duplicated sample measurements were not available because of volume limitations. Considering the number of samples used in this study, the possibility of random errors owing to the use of a single replicate would have minimally affected the overall results. The strengths of this study are that it was the first to evaluate Atellica IM 1600 for PCT concentration measurement and that a large number of samples with concentrations close to the medical decision points were analyzed.

In conclusion, this study demonstrated that the Atellica IM 1600 and Cobas e801 immunoassays provide strongly correlated and largely concordant PCT concentrations compared with Kryptor, despite the different assay principles. However, both immunoassays manifested modest negative bias compared with Kryptor, which could, in some cases, result in discordant medical decisions at the 0.25 and 0.50 μg/L medical decision points. Therefore, to successfully implement PCT-guided antibiotic stewardship, rigorous evaluation of immunoassay comparability is essential on the implementation of alternative PCT immunoassays. In addition, alternate use of different PCT immunoassays in repeat examinations should be avoided, because it can hamper the interpretation of PCT concentrations during longitudinal patient assessment. Furthermore, efforts should be made to develop and establish reference materials and methods for PCT concentration measurement, which would improve comparability and ultimately achieve standardization of the various PCT immunoassays.