Comparison of Four Automated Carcinoembryonic Antigen Immunoassays: ADVIA Centaur XP, ARCHITECT I2000sr, Elecsys E170, and Unicel Dxi800.

BACKGROUND: Carcinoembryonic antigen (CEA) is one of the tumor markers available for evaluating disease progression status after initial therapy and monitoring subsequent treatment modalities in colorectal, gastrointestinal, lung, and breast carcinoma. We evaluated the correlations and differences between widely used, automated CEA immunoassays at four different medical laboratories.
METHODS: In total, 393 serum samples with CEA ranging from 3.0 to 1,000 ng/mL were analyzed on ADVIA Centaur XP (Siemens Diagnostics, Tarrytown, NY, USA), ARCHITECT i2000sr (Abbott Diagnostics, Abbott Park, IL, USA), Elecsys E170 (Roche Diagnostics, Indianapolis, IN, USA), and Unicel DxI800 (Beckman Coulter, Fullerton, CA, USA), and the results were compared. Deming regression, Passing-Bablok regression, and Bland-Altman analyses were performed to evaluate the data correlation and % differences among these assays.
RESULTS: Deming regression analysis of data from Elecsys E170 and UniCel DxI800 showed good correlation (y=3.1615+0.8970x). According to Bland-Altman plot, no statistically significant bias (-1.78 ng/mL [95% confidence interval: -4.02 to 0.46]) was observed between Elecsys E170 and UniCel DxI800. However, the relative differences of CEA concentrations between assays exceeded the acceptable limit of 30%. Regarding the agreement of positivity with cut-off value 5.0 ng/mL, ARCHITECT i2000sr and Elecsys E170 showed the highest agreement (95.2%), whereas ADVIA Centaur XP and ARCHITECT i2000sr showed the lowest agreement (70.7%).
CONCLUSIONS: Agreements between automated CEA immunoassays are variable, and individual CEA concentrations may differ significantly between assays. Standardization of serum CEA concentrations and further harmonization are needed. © The Korean Society for Laboratory Medicine.

INTRODUCTION

Tumor markers are useful for evaluating disease progression status after initial therapy and monitoring subsequent treatment modalities [12345]. Carcinoembryonic antigen (CEA) is one of the longest known tumor antigens [6], and is a marker for colorectal, gastrointestinal, lung, and breast carcinoma [7]. With increasing incidence and prevalence of cancers, the CEA immunoassay workload in medical laboratories has increased. Because CEA concentration is used to monitor treatment responses and recurrences of various cancers, more sensitive, specific, reproducible, and interchangeable assays are needed to manage cancer patients.

Immunoassays quantify biologically relevant molecules based on the specificity and selectivity of antibody reagents [89]. Significant variability in results can result from the statistical model used for the calibration curve, which is used for quantification. Therefore, it is important to choose an appropriate curve-fitting model for calibration curves and to consider all calibration curve-related factors, including quality and stability of reference standards, quality and stability of reagents, and statistical validity of the calibration curve [1011]. Although various CEA assays with different principles, including chemiluminescence immunoassay (CLIA), enzyme immunoassay, radioimmunoassay, fluorescence immunoassay, and lateral flow immunoassay have been introduced, currently, automated CLIA analyzers with high sensitivity and high throughput are the most widely used [121314]. Despite ongoing standardization efforts, CEA concentrations from different manufacturers can vary owing to the lack of accurate calibration as well as differences in assay principle, the epitope used, antibody specificities, and the reagents used. Previous studies using individual samples and standard materials have reported that harmonization of CEA assays is far from being realized [1516].

This study aimed to comparatively evaluate the four automated CEA immunoassays and to estimate the harmonization of these four analyzers.

METHODS

1. Sample preparation

In total, 393 serum samples with high CEA concentrations were obtained from four laboratories. The samples were subjected to routine CEA quantification at all four laboratories using different CLIAs. Sera with CEA concentrations of 3.0–1,000 ng/mL were randomly collected between March 2014 and February 2015. The leftover samples after routine CEA tests were aliquoted into 6–10 new tubes and stored immediately at−70℃ until analysis according to the manufacturer's recommendations for sample management as per which samples should be frozen at or below −20℃ if they are not assayed within 2–7 days. Samples were transported in the frozen state. The declared sample stability and storage conditions are given in Table 1. This study was approved by the Institutional Review Board of The Catholic University of Korea (XC14SIMI0069K).

Table 1

Summary of performance characteristics of four carcinoembryonic antigen (CEA) immunoassays according to information provided by the manufacturers

Characteristics	ADVIA Centaur XP	ARCHITECT i2000sr	Elecsys E170	UniCel DxI 800
Method principle	Chemiluminescence immunoassay (CLIA)	Chemiluminescent Microparticle Immunoassay (CMIA)	Electrochemiluminescence immunoassay (ECLIA)	Chemiluminescence immunoassay (CLIA)
Assay principle	Two-site sandwich immunoassay	Two-step immunoassay	Sandwich immunoassay	Two-site immunoenzymatic “sandwich” assay
Sample volume (μL)	50	60	10	35
Sample stability	< 8 hours at 24℃	< 24 hours at 24℃	< 2 hours at 24℃	< 8 hours at 24℃
Sample storage	< 48 hours at 2–8℃	< 7 days at 2–8℃	< 7 days at 2–8℃	< 48 hours at 2–8℃
	> 48 hours at –20℃	> 7 days at –20℃	< 6 months at –20℃	> 48 hours at –20℃
Reagent	Purified polyclonal rabbit anti-CEA antibody labeled with acridinium ester, monoclonal mouse anti-CEA antibody covalently coupled to paramagnetic particles	Anti-CEA (mouse, monoclonal) coated microparticles, anti-CEA (mouse, monoclonal) acridinium-labeled conjugate	Streptavidin-coated microparticles, Biotinylated monoclonal anti-CEA antibody (mouse/human), monoclonal anti-CEA antibody (mouse) labeled with ruthenium complex	Anti-CEA MAb-alkaline phosphatase conjugate, anti-CEA MAb bound to paramagnetic particles
Calibration	Traceable to an internal standard manufactured using highly purified material	Traceable to an internal standard manufactured using highly purified material	Standardized against the 1st IRP WHO Reference Standard 73/601	Traceable to an internal standard manufactured using highly purified material
Measuring range (ng/mL)	0.5–100	0.5–500 (can be extended to 1,500)	0.2–1,000	0.1–1,000
Limit of detection (ng/mL)	0.5	0.5	0.2	0.1
Precision (total % CV)	3.4–5.5	2.7–4.0	4.6–5.1	3.8–4.5
Reference value (ng/mL)	95th percentile; 5.0	93.5th percentile; < 5.0	95th percentile; 4.7, 5.2	95th percentile; 3.1–5.0

2. CEA immunoassays

The automated immunoassays used at the medical laboratories were ADVIA Centaur XP (Siemens Diagnostics, Tarrytown, NY, USA) at Seoul St. Mary's Hospital (Seoul, Korea), ARCHITECT i2000sr (Abbott Diagnostics, Abbott Park, IL, USA) at St. Vincent's Hospital (Suwon, Korea), Elecsys E170 (Roche Diagnostics, Indianapolis, IN, USA) at Daejeon St. Mary's Hospital (Daejeon, Korea), and Unicel DxI800 (Beckman Coulter, Fullerton, CA, USA) at Inchon St. Mary's Hospital (Inchon, Korea). The measuring ranges of the four assays were as follows: 0.5–100 ng/mL for ADVIA Centaur XP, 0.5–1,500 ng/mL for ARCHITECT i2000sr, 0.2–1,000 ng/mL for Elecsys E170, and 0.1–1,000 ng/mL for Unicel DxI800. To evaluate the effect of dilution on the CEA results, comparisons between assays were performed separately for samples having concentrations <100 ng/mL and within the measurement range of all four immunoassays. Serum samples were thawed immediately before analysis, mixed thoroughly, and checked for clots. CEA concentrations were quantified concurrently in the same batch between 7 and 14 days by using each immunoassay based on the principle of electrochemiluminescence detection. Samples with measured concentrations exceeding the analytic measurement range were diluted on-board according to the manufacturers' recommendations. Performance characteristics of the four automated CEA immunoassay analyzers according to information provided by the manufacturers are summarized in Table 1.

3. Statistical analyses

Normality was assessed using Kolmogorov-Smirnov and Shapiro-Wilk tests. Deming regression with a constant CV of 5% and Passing-Bablok regression analyses were performed to identify proportional and systematic bias [17]. As the CEA data were not normally distributed as indicated by Kolmogorov-Smirnov and Shapiro-Wilk tests, Bland-Altman plots were displayed as relative difference plots with clinically acceptable bias limits of 30% according to a previous recommendation [17]. CEA concentrations of 0.5–100 ng/mL are within the overlapping analytical measuring range, and the results were regarded valid without further sample dilution. Therefore, the results from these samples were compared separately. MedCalc Statistical Software Version 17.6 (MedCalc software, Ostend, Belgium) was used for statistical investigation. Statistical significance was accepted at P <0.05.

RESULTS

1. Method comparison of four CEA immunoassays

The CEA concentrations in the 393 serum samples obtained from the four automated immunoassays were as follows: ADVIA Centaur XP (median, 7.4; range, 1.4–636.6 ng/mL), ARCHITECT i2000sr (12.8; 4–1,134), Elecsys E170 (12.1; 3–913.8), and UniCel DxI800 (10.2; 2.1–885.8). CEA concentrations measured by ARCHITECT i2000sr were found to be the highest, followed by those measured by Elecsys E170, Unicel DxI800, and ADVIA Centaur XP. Results of between-assay comparisons are shown in Table 2. Deming regression coefficients for these CEA assays varied from 0.6335 to 1.2895 (Fig. 1). There was no linear relationship for ARCHITECT i2000 vs Elecsys E170 and Elecsys E170 vs Unicel DxI800 (Cusum test for linearity, P <0.05). Therefore, Passing-Bablok analysis was not applicable to these comparisons. According to the Bland-Altman plot, no statistically significant bias (−1.78 ng/mL [95% confidence interval: −4.02 to 0.46]) was observed between Elecsys E170 and UniCel DxI800. The mean % difference in CEA concentrations by Bland-Altman analysis ranged from −54.5 to 21.3%. The mean % difference between ARCHITECT i2000sr and Elecsys E170, and that between Elecsys E170 and UniCel DxI800 was as low as 10.5% and 10.9%, respectively (Table 2). On the other hand, all six pairwise comparisons demonstrated % differences exceeding the acceptable limit of <30%. When the median difference (%) between assays was employed, the 2.5th to 97.5th percentile of median difference (%) also exceeded the acceptable limit of 30% (Fig. 2).

Table 2

Comparison data of four carcinoembryonic antigen (CEA) immunoassays using 393 serum samples with CEA<1,000 ng/mL

Statistics	Centaur XP (y) vs i2000sr (x)	Centaur XP (y) vs E170 (x)	Centaur XP (y) vs DxI800 (x)	i2000sr (y) vs E170 (x)	i2000sr (y) vs DxI800 (x)	E170 (y) vs DxI800 (x)
Deming regression
Regression equation	y=−1.1579+0.6335x	y=−3.8739+0.8197x	y=−1.1501+0.7326x	y=−4.0789+1.2895x	y=0.0232+1.1562 x	y=3.1615+0.8970x
Intercept, 95% CI	−3.1532 to 0.8373	−7.7759 to 0.02813	−3.2721 to 0.9719	−8.1134 to −0.04427	−2.9394 to 2.9858	0.9961 to 5.3269
Slope B, 95% CI	0.5679 to 0.6991	0.6744 to 0.9650	0.6474 to 0.8177	1.1479 to 1.4310	1.0485 to 1.2639	0.8245 to 0.9696
Bland-Altman analysis
Mean difference (ng/mL) (95% CI)	−21.5 (−27.1 to −15.8)	−12.2 (−16 to −8.5)	−14.0 (−18 to −9.9)	9.3 (5.3 to 13.3)	7.5 (4.4 to 10.6)	−1.8 (−4.02 to 0.46)
P value	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	0.1196

Abbreviations: y, y axis; x, x axis; 95% CI, 95% confidence interval.

Fig. 1

Method comparison of four carcinoembryonic antigen (CEA) immunoassays by Deming regression analysis. Results are in ng/mL. Black solid lines are from Deming regression. Black dashed lines are x=y. Comparison of CEA concentrations between ARCHITECT i2000sr and ADVIA Centaur XP (A), Elecsys E170 and ADVIA Centaur XP (B), UniCel DxI800 and ADVIA Centaur XP (C), Elecsys E170 and ARCHITECT i2000sr (D), UniCel DxI800 and ARCHITECT i2000sr (E), and UniCel DxI800 and Elecsys E170 (F).

Fig. 2

Method comparison of four carcinoembryonic antigen (CEA) immunoassays using Bland-Altman plots in 393 serum samples. The solid line indicates the % difference from the averages of two assays. Dashed lines indicate the upper and lower 95% confidence limits of % difference between two analyzers. The red lines are the acceptable bias limits of 30.0%. Comparison of CEA concentrations between ADVIA Centaur XP and ARCHITECT i2000sr (A), ADVIA Centaur XP and Elecsys E170 (B), ADVIA Centaur XP and UniCel DxI800 (C), ARCHITECT i2000sr and Elecsys E170 (D), ARCHITECT i2000sr and UniCel DxI800 (E), and Elecsys E170 and UniCel DxI800 (F).

2. Evaluation of the dilution effect on CEA results

Twenty-four (6.1%) serum samples that had CEA concentrations over 100 ng/mL by ADVIA Centaur XP were diluted by a factor of 10. When we divided the samples into two subgroups with CEA concentrations <100 ng/mL and ≥100 ng/mL, mean differences (%) of ADVIA Centaur XP against mean CEA concentrations were similar in both subgroups (−31.1% and −26.5%, respectively). In Bland-Altman and Passing-Bablok regression analyses, the differences of CEA concentrations among the six pairwise comparisons exceeded the acceptable limit of 30%, but diluent matrix effects were not detected (Table 3).

Table 3

Comparison data of four carcinoembryonic antigen (CEA) immunoassays using 355 serum samples with CEA<100 ng/mL

Statistics	Centaur XP (y) vs i2000sr (x)	Centaur XP (y) vs E170 (x)	Centaur XP (y) vs DxI800 (x)	i2000sr (y) vs E170 (x)	i2000sr (y) vs DxI800 (x)	E170 (y) vs DxI800
Passing-Bablok regression
Regression equation	y=−0.629+0.628x	y=−0.377+0.671x	y=−0.038+0.709x	y=0.665+1.038x	y=1.214+1.114x	y=0.557+1.045x
Intercept (95% CI)	−0.910 to −0.386	−0.595 to −0.181	−0.199 to 0.104	0.269 to 1.048	0.640 to 1.727	0.240 to 0.904
Slope B (95% CI)	0.597 to 0.657	0.648 to 0.693	0.691 to 0.729	0.996 to 1.074	1.049 to 1.175	1.002 to 1.085
Cusum test for linearity	ns (P=0.97)	ns (P=0.88)	ns (P=0.20)	ns (P=0.31)	ns (P=0.46)	ns (P=0.38)
Bland-Altman analysis
Mean difference (ng/mL) (95% CI)	−8.1 (−9.1 to −7.2)	−6.5 (−7.2 to −5.7)	−5.1 (−6.0 to −4.3)	1.6 (1.1 to 2.2)	3.0 (2.2 to 3.8)	1.3 (0.6 to 2.1)
P value	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001	0.0003

Abbreviations: y, y axis; x, x axis; 95% CI, 95% confidence interval.

3. Agreement of categorical CEA data generated by using the same cut-off value of 5.0 ng/mL

We used a cut-off value of 5.0 ng/mL to categorize CEA data. Of all samples, 69.7% (274/393), 98.9% (389/393), 95.2% (374/393), and 87.3% (343/393) had CEA concentrations above 5.0 ng/mL when they were tested with the ADVIA Centaur XP, ARCHITECT i2000sr, Elecsys E170 and UniCel DxI800 analyzers, respectively. When we analyzed the agreement between assays based on categorical data, we observed the highest concentration of agreement between ARCHITECT i2000sr and Elecsys E170 (95.2%, 374/393) and the lowest concentration between ADVIA Centaur XP and ARCHITECT i2000sr (70.7%, 278/393) (Table 4).

Table 4

Agreement of positivity between CEA assays using a cutoff value of 5.0 ng/mL

	ADVIA Centaur XP	ARCHITECT i2000sr	Elecsys E170
ARCHITECT i2000sr	70.7%
Elecsys E170	74.6%	95.2%
UniCel DxI800	81.9%	87.3%	92.1%

DISCUSSION

This study aimed to compare four widely used automated CEA assays. Approximately 80% of healthy subjects have a CEA concentration lower than 3 ng/mL [14]. Therefore, serum samples with a CEA concentration >3.0 ng/mL as measured by any of the four immunoassays were randomly selected for analysis. The Bland-Altman plots showed that many samples were largely outside the linearity limits, and CEA concentrations varied between assays. In the absence of a reference method for CEA measurement, the clinically acceptable significant percentage difference between assays was defined as 30%. The maximum differences (%) between all six assays exceeded this limit. On comparing mean CEA concentrations, ARCHITECT i2000sr overestimated while ADVIA Centaur XP underestimated CEA concentrations, mainly for low-concentration samples (<10 ng/mL). Elecsys E170 and Unicel DxI800 showed good correlation by Deming regression analysis and in Bland-Altman plots. This might be due to the fact that these two immunoassay analyzers use a single monoclonal antibody for two-step sandwich immunoassay, while the other two assays use one or more different antibodies. Several comparative studies of CEA assays have been reported [1215181920]. In one of these studies, CEA data obtained by Unicel DxI800 showed the highest degree of correlation with those measured by ADVIA Centaur XP [slope (95% CI), 0.910 (0.883 to 0.947); intercept (95% CI), −0.240 (−0.362 to −0.171)] [19]. On the other hand, CEA concentrations from Unicel DxI800 were found to be the highest in one study [15], which was different from the results of the present study. CEA concentrations from ARCHITECT i2000sr and Elecsys E170 are reportedly higher than those from Siemens ADVIA Centaur XP, in agreement with the results of this study.

Differences in results from comparative studies might be due to the diluent matrix effects or interaction between components from blood collection tubes and blood samples [21]. Concentrating samples with CEA concentrations beyond the maximum detection limit might be highly subjective because of dilution effects. In the present study, the dilution of samples did not seem to influence the data, and the matrix effect of the diluent for ADVIA centaur was minimal. Therefore, variability in CEA measurements might be mainly due to harmonization problems.

To harmonize CEA concentrations, an international reference standard for CEA (code 73/601) was established by the World Health Organization in 1975 [22]. However, instrument-specific calibration and working standards in current immunoassays are less traceable to the international standard, resulting in inconsistent CEA results between assays [1523]. In the present study, the calibrators provided by the four manufacturers, except for Elecsys E170, were not standardized against the WHO 1st international reference preparation 73/601.

In general, a disadvantage of CEA measurement is the high rate of false positives. In the current study, the highest discrepancies between assays were noted for samples with low CEA concentrations. When we used 5.0 ng/mL as cut-off value for serum CEA, the agreement of positivity ranged from 70.7% to 95.2%. The agreement between ADVIA Centaur XP and ARCHITECT i2000sr was the lowest. Reference intervals of CEA can vary by ethnicity, assay method, and many other factors [14242526]. Therefore, different reference ranges for each immunoassay need to be established; and for follow-up of CEA variations, using the same immunoassay is recommended. Precautions should be taken when changing CEA assay because CEA concentrations from automated immunoassay are not comparable. In addition, clinicians should be aware of changes in analyzers and techniques used for CEA measurement and consider between-method agreement and CV as evidenced by external quality assessment data.

Potential limitations of this study include relatively small number of studied samples, non-normally distributed samples, and lack of information on the patients and pre-analytical errors in relation to the sample tube, sample storage, or transportation. These limitations should be considered when interpreting the present results. Despite these limitations, our results demonstrated that CEA concentrations might vary among the four immunoassays currently in use, and standardization and further harmonization for CEA testing are needed.

In conclusion, agreements between automated CEA immunoassays are variable and individual CEA concentrations can differ significantly between assays. Therefore, reference ranges should be established for each immunoassay or the widely used cut-off value of 5.0 ng/mL should be employed, and the reference range should be validated in laboratories to decrease the false positive rate.