Analytical evaluation of the Radiometer AQT90 FLEX βhCG assay.

OBJECTIVES: Many hospitals cannot afford an hCG assay on a central lab analyzer and turn to point of care testing (POCT) solutions. The Radiometer AQT90 FLEX is a small benchtop immunoareement between the AQT90 and comparator methods for samples with hCG ssay analyzer for use in the laboratory or at the patient bedside. This study evaluated the analytical performance of the AQT90's βhCG assay.
METHODS: Precision was assessed using whole blood patient samples and two levels of quality control. Linearity was assessed by dilution of a high hCG plasma sample. Carryover and hook effect were assessed using high and low hCG samples. Method comparisons were done against Abbott i-STAT Total βhCG, Beckman Coulter Total βhCG (5th IS), and Roche hCG+β. Sample concentrations ranged from<2 IU/L to 4,973 IU/L.
RESULTS: Repeatability and within-laboratory precision passed most manufacturer's claims and allowable error criteria. Linearity was validated from<2 IU/L to 4,741 IU/L. Hook effect was not observed up to 2,446,448 IU/L. Carryover was<4.0 ppm. A linear relationship was observed with i-STAT, Beckman and Roche methods. At>20 IU/L, biases were apparent against all three comparator assays (i-STAT: +20%, Roche: +30%, Beckman: +5 to 15%). At ≤20 IU/L, the acceptability of agreement varied according to TAE specifications. Concordance between AQT90 and comparator assays using 5 IU/L as the medical decision level ranged from 69% to 81%.
CONCLUSIONS: Overall, the AQT90 hCG assay performed well and would be suitable for smaller suburban or rural hospitals. Some limitations have been noted and should be kept in mind during clinical testing.

Introduction

Rapid and accurate diagnosis of pregnancy complications that require acute care is frequently augmented by combining clinical assessment, medical history, and one or more quantitative human chorionic gonadotropin (hCG) results. Informal feedback from emergency medicine physicians throughout the province of Alberta in Canada has indicated that acceptable turnaround times for such hCG results are no more than 45–60 min. This is feasible if hCG testing is available locally, but it is almost impossible to meet if hCG testing has to be referred to a laboratory in another city. Many of Alberta's smaller suburban or rural hospitals have been unable to maintain an hCG assay on an automated, random-access immunoanalyzer in the laboratory. For one, the capital cost of such analyzers has been frequently prohibitive. Moreover, laboratories have not always had the space to accommodate such large analyzers. Low testing volume has posed an additional challenge and has led to high overall cost-per-reportable test. Point of care testing (POCT) solutions have the potential to address many of these restrictions. In recent years only two POCT quantitative hCG assays have been commercially available in Canada: the Siemens Stratus CS Acute Care βHCG assay [1], and the Abbott i-STAT Total Beta-Human Chorionic Gonadotropin (β-hCG) assay [2]. In 2014, a third assay became available when Radiometer launched an hCG assay on its new random access point of care immunoassay analyzer, the AQT90 FLEX.

Previous publications reporting on the AQT90 FLEX βhCG assay do not exist, but one conference abstract that reported on its analytical performance has been published [3]. The authors stated that they verified linearity between 2 and 5000 IU/L and did not see a hook effect up to an hCG of 250,000 IU/L. Their precision studies, details of which are not provided, indicated a coefficient of variation (CV) of 10% at 8 IU/L. Excellent linear correlation was noted between the AQT90 and the Siemens Stratus CS (r = 0.99), but a bias was also apparent with the AQT90 generally yielding higher values than the Stratus (linear regression slope = 1.291, y-intercept = −7.2 IU/L).

This study evaluated the analytical performance of the AQT90 hCG assay and compared it to a POCT assay and two central laboratory assays that are currently in clinical use in Edmonton, Canada. To date, this is the first peer-reviewed publication on the analytical performance of the AQT90 hCG assay.

Material and methods

Ethics

This study was part of a quality improvement project and did not constitute clinical research. Review by the University of Alberta Human Research Ethics Board was, therefore, not required.

AQT90 FLEX βhCG immunoassay

The measurement principle of all AQT90 assays is based on an all-in-one dry-reagent concept [4]. A test cartridge contains 16 reaction cups and is stable onboard the analyzer for 20 days. All required assay-specific reagents, including europium chelate-labelled tracer antibodies, biotinylated capture antibodies, and stabilizing reagents, are dry-coated into the reaction cups. A sample (whole blood, plasma, or aqueous quality control/linearity material) and assay buffer are dispensed into a reaction cup by the analyzer. This is followed by an automated incubation step, washing and drying of the cup, and measurement of an emitted signal at 616 nm. The strength of the emitted signal is directly proportional to the concentration of the measured analyte in the sample.

The AQT90 hCG assay is intended for use as an aid in the early detection of pregnancy. It takes 18 min to complete, and its antibodies recognize both intact hCG and the free β subunit. Radiometer's analytical performance claims for this assay are described in Supplemental Tables 1 and 2, and also in the manufacturer's package insert [4].

Overview of study design

The primary aim of this study was to assess key analytical performance indicators for the AQT90 hCG assay including repeatability, within-laboratory precision, hook effect, carryover and accuracy. A secondary aim was to get a preliminary appreciation of the variability in results that may be seen in an institution or an organization that has multiple analyzers and reagent lots in use simultaneously. To achieve the secondary aim, sample analysis was completed using two different lots of Solution Packs, with the change from the first to the second lot having been made mid-way during the study. Limited reagent availability precluded the use of two different lots of Test Kits for the study. To achieve the secondary aim, sample analysis was also performed using a combination of two different AQT90 analyzers whenever possible. Experiments that only used a single analyzer are noted in the text as having done so. For the method comparison study, samples were randomly run on one or the other analyzer with roughly 20 samples being run on each.

Patient samples

All patients whose samples were included in this study had an hCG measured on them for clinical purposes. The remnant samples from these patients were used for the experiments described herein. All AQT90 experiments used EDTA whole blood with the exception of the linearity experiment which used EDTA plasma. EDTA plasma was obtained by centrifuging EDTA whole blood at 2479g for 10 min and transferring the resultant plasma into an aliquot tube. All samples analyzed on the comparator methods were lithium heparin plasma. These samples had been collected concurrently with the EDTA samples that were tested on the AQT90, thereby yielding paired EDTA and lithium heparin samples (same patient, same collection date, same collection time).

Sample types for this study were dictated by a number of considerations. While EDTA or lithium heparin samples are equally acceptable for the AQT90, i-STAT and Roche assays, the recommended sample type for the Beckman assay is serum or lithium heparin plasma. The Beckman package insert does not have any claims for EDTA plasma, nor have Edmonton laboratories ever validated this sample type. Because of this, lithium heparin samples were used with all of the comparator methods. A second consideration was that the lithium heparin vacutainers used in Edmonton for clinical samples contain a gel barrier and, therefore, are incompatible with the AQT90. The AQT90 package insert explicitly states to not use any tubes that contain a gel. Because of this, EDTA samples were used with the AQT90.

All experiments on the AQT90 except for repeatability, within-laboratory precision and linearity used whole blood to best mimic the sample matrix conditions that would be present in routine clinical operation. The 18-min assay time would not allow for completion of the entire linearity experiment within the 3-h whole blood stability window specified by Radiometer. Because of this, plasma samples were used instead. Radiometer's stability claim for plasma samples stored at 2–8 °C is 24 h and the linearity experiment was completed within 6 h.

Precision

Repeatability

Repeatability was assessed using seven patient samples. Each sample was analyzed twenty times within a single run. Mean, standard deviation (SD), CV were calculated for each set of results.

Within-laboratory precision

Within-laboratory precision was determined using two levels of external liquid quality control (QC) from Radiometer (AQT90 FLEX LQC hCG-CHECK Level 1 and 2). Each level was analyzed over 10 days, with one run per day and five replicates per run. Mean, SD, and CV were calculated for each set of results.

Linearity

A negative patient pool was first created by combining nine plasma samples with hCG concentrations below the lower end of the AQT90 measuring range (<2 IU/L). A patient sample was then identified with an hCG concentration above the upper end of the AQT90 measuring range (>5,000 IU/L). This “high” sample was then combined in different ratios with the negative patient pool to create seven different dilutions. The negative patient pool and each dilution were measured in triplicate. This linearity study was performed using a single AQT90 analyzer.

Hook effect

Each of three samples with a high hCG concentration was run in triplicate on one of two AQT90 analyzers. The hCG concentrations of the lithium heparin plasma samples that had been collected concurrently with the EDTA samples (see Section 2.4) were measured by the Beckman Coulter Total βhCG (5th IS) assay [5] and yielded the following results: 148,370, 282,262 and 2,446,448 IU/L.

Carryover

A sample with hCG<2 IU/L (“negative” or “N”) and a sample with hCG>5,000 IU/L (corresponding paired lithium heparin plasma Beckman result = 282,262 or 2,446,448 IU/L; “high” or “H”) were run in the following sequence: N1, N2, N3, N4, H, N5, N6, N7, N8, N9. Carryover in parts per million (ppm) was calculated according to the following equation: carryover (in ppm) = [(N5-N4)/H]*1,000,000. If N4 and/or N5 were<2 IU/L, the values were converted to 2 IU/L for the purposes of the carryover calculations.

Method comparison

Thirty six patient samples were compared between the AQT90 [4], i-STAT [2], Beckman [5], and Roche hCG+β assays [6]. Supplementary Tables 1 and 2 provide methodology information and manufacturers’ analytical performance claims for each assay. Samples were analyzed with the AQT90, i-STAT, and Beckman assays within 3 h of collection. An aliquot of each lithium heparin plasma sample was then frozen and stored at −30 °C. The frozen aliquots were shipped to a laboratory with the Roche assay where they were subsequently thawed and analyzed. AQT90 hCG concentrations for the patient sample comparison ranged from<2 IU/L to 4,973 IU/L (corresponding paired lithium heparin plasma Beckman results = 0.1–5,985 IU/L, respectively).

Performance goals

Analytical performance goals are not widely available for hCG. Short- and long-term physiological variation has been poorly characterized in states outside of pregnancy, making it impossible to develop biological variation-based performance goals. Goals based on other considerations, such as inter-laboratory performance, have only been made widely available by three professional groups: Royal College of Pathologists of Australasia (RCPA) [7], Institute for Quality Management in Healthcare (IQMH) [8], and RiliBÄK [9]. Supplemental Table 3 summarizes the recommendations of each group. This study assessed the performance of the AQT90 using all three sets of recommendations. Precision criteria were set at one third of the total allowable error (TAE).

Data analysis

Ordinary least squares regression was used to analyze the linearity data. Method comparison data were analyzed with weighted Deming regression. The comparator method was set as the “x” data set, and AQT90 was set as the “y” data set. CVs were set at 4% for AQT90, 4% for i-STAT, 4% for Beckman, and 3% for Roche. These values were obtained from the current study for the AQT90 and from previous in-house method validation studies for the other assays [unpublished data]. Concordance between two methods was calculated by summing all result pairs that showed agreement and dividing this number by the total result pairs for a comparison. Microsoft Excel + Analyse‐it 2010 (Microsoft, Redmon, MA) was used to perform data analysis and to prepare figures and tables for publication.

Results

Repeatability was assessed at hCG concentrations ranging from 7 to 3,652 IU/L (Supplementary Table 4). At the lowest concentration examined (∼7 IU/L), SDs ranged from 0.4 to 0.8 IU/L and CVs ranged from 6.7 to 9.7%. At intermediate concentrations (399‐2,918 IU/L), SDs ranged from 7 to 68 IU/L and CVs ranged from 1.8 to 3.3%. At the highest concentration of 3,652 IU/L, the SD was 53 IU/L while the CV was 1.5%. The SDs and CVs of one analyzer were similar to the manufacturer's claims of 3 ± 0.5 IU/L (15.4%), 13 ± 0.7 IU/L (5.2%), and 4,173±96 IU/L (2.3%). The SDs and CVs of the other analyzer slightly exceeded the manufacturer's claims at the lower hCG concentration. At all concentrations, the SDs and CVs of both analyzers were less than one third of IQMH and RiliBÄK TAE criteria. SDs and CVs of both analyzers were higher than one third of RCPA TAE criteria at the lowest concentration examined.

Within-laboratory precision was assessed at two different hCG concentrations (Table 1). The level 1 QC material mean ranged from 18 to 19 IU/L with SDs of 0.6–0.7 IU/L and CVs of 3.5–3.9%. The level 2 QC material mean ranged from 299 to 307 IU/L with SDs of 9–15 IU/L and CVs of 2.8–5.1%. SDs and CVs of both analyzers at the lower concentration were similar to the manufacturer's stated claims for total precision [3 ± 0.5 IU/L (17.2%) and 13 ± 1 IU/L (8.6%)]. The manufacturer did not have a stated claim for a concentration close to 300 IU/L. Both analyzers met IQMH and RiliBÄK TAE criteria at all concentration. However, neither analyzer met the RCPA criteria at the lower concentration while both only partially met the RCPA criteria at the higher concentration.

Table 1

Within-laboratory precision studies using the AQT90 hCG assay.a

	AQT90 #1				AQT90 #2
QC level	1		2		1		2
Lot	1	2	1	2	1	2	1	2
Mean (IU/L)	18	18	300	299	19	19	307	304
SD (IU/L)	0.7	0.7	9	11	0.6	0.7	9	15
%CV	3.9	3.9	3.0	3.8	3.5	3.8	2.8	5.1
SD or %CV is ≤1/3rd of RCPA TAE?	No	No	Yes	No	No	No	Yes	No
SD or %CV is ≤1/3rd of IQMH TAE?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
SD or %CV is ≤1/3rd of RiliBÄK TAE?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes

Abbreviations: RCPA, Royal College of Pathologists of Australasia; IQMH, Institute for Quality Management in Healthcare.

Linearity was verified across the measuring range of 2–4741 IU/L (Supplementary Table 5, Supplementary Fig. 1). The slope, y-intercept, and Pearson correlation coefficient (r) for the regression were 1.003 (95% confidence interval or 95% CI: 0.979–1.026), 25.868 IU/L (95% CI: −23.479 to 75.214 IU/L) and 0.9998 respectively.

Each of three whole blood samples with hCG> 100,000 IU/L yielded the expected result of>5,000 IU/L with the AQT90 hCG assay. There was no evidence of a hook effect even at an hCG concentration of 2,446,448 IU/L. Our results confirm the manufacturer's claim of no hook effect for hCG ≤ 250,000 IU/L.

Carryover was not observed (i.e. 0 ppm) for the sample with hCG = 2,446,448 IU/L. Interestingly, a small amount of carryover (3.9 ppm) did occur for the sample with hCG = 282,262 IU/L, but this was below the manufacturer's carryover claim for the AQT90 hCG assay (<50 ppm for a sample with hCG = 100,000 IU/L). In this instance, the negative sample result went from<2 IU/L (preceding high sample analysis) to 3 IU/L (immediately following high sample analysis). The manufacturer's claim for the 95th percentile of a reference population is<2 IU/L (male),<2 IU/L (pre-menopausal female), and<6 IU/L (all females), so it is unlikely that the small amount of carryover observed in our experiment would have an impact on clinical patient care.

hCG was measured with four different methods in 36 pairs of patient samples (Supplementary Table 6). Overall, the AQT90 demonstrated a linear relationship with all three comparator methods (Fig. 1A). The slope, y-intercept, and 95% Confidence Intervals (95% Cis) of the Deming regression line for each comparison are listed in Table 2. The AQT90 showed a proportional bias against all three comparator methods as evidenced by the 95% CIs of all three slopes being above (i-STAT, Roche) or below (Beckman) 1.000. Constant bias appeared negligible for the Roche and Beckman methods but not for the i-STAT. Fig. 1B and C illustrate that AQT90 results were, on average, approximately 20% higher than those of i-STAT and approximately 30% higher than those of Roche. The relationship between AQT90 and Beckman results appeared more complex (Fig. 1D). At hCG<1,000 IU/L, the average bias was approximately 5%, but differences ranged from −8% to +25%. At concentrations ≥1,000 IU/L, the difference between the two methods stabilized with the AQT90 giving lower results by ∼15%.

Fig. 1

Results of method comparison studies using four different hCG assays. (A) Weighted Deming regression plot. Black circles (•) represent AQT versus Beckman; grey triangles () represent AQT versus Roche; white squares () represent AQT versus i-STAT. The solid grey line denotes the line of identity. (B)–(D) Bland-Altman percent difference plots. Grey horizontal lines denote TAE criteria of the Royal College of Pathologists of Australasia (), Institute for Quality Management in Healthcare (), and RiliBÄK ().

Table 2

Results of method comparison studies using four different hCG assays. Weighted Deming regressions were performed with the comparator method set as the “x” data set and AQT90 set as the “y” data set.

Comparator	Sample type	Slope (95% CI)	Y-intercept (95% CI) (IU/L)
i-STAT	Fresh lithium Heparin plasma	1.260 (1.061–1.459)	−6.323 (−43.910 to 31.260)
Beckman	Fresh lithium Heparin plasma	0.937 (0.869–1.005)	0.411 (−0.004 to 0.827)
Roche	Frozen lithium Heparin plasma	1.263 (1.215–1.312)	0.568 (0.078–1.058)

Agreement between the four methods was further investigated at hCG concentrations ≤20 IU/L. First, the difference between paired results (AQT90 versus comparator) was compared against RCPA, IQMH and RiliBÄK TAE criteria (Table 3). The AQT90 performed well against the i-STAT and Beckman methods when using TAE criteria from RCPA and IQMH, with results agreeing 88–100% of the time. Performance against the Roche method was worse, with results agreeing only 69–94% of the time. Performance against all three comparator methods was suboptimal (81–94%) when using RiliBÄK TAE criteria.

Table 3

Agreement between the AQT90 and comparator methods for samples with hCG<20 IU/L. The table indicates the absolute number of result pairs that agreed within specified TAE criteria. The percent agreement (out of a total of 16 paired results) is indicated in brackets.a

	TAE criteria
Comparator	RCPA	IQMH	RiliBÄK
i-STAT	14 (88%)	15 (94%)	15 (94%)
Beckman	14 (88%)	16 (100%)	13 (81%)
Roche	11 (69%)	15 (94%)	13 (81%)

Abbreviations: RCPA, Royal College of Pathologists of Australasia; IQMH, Institute for Quality Management in Healthcare.

Agreement of the AQT90 with the three comparator methods was also examined using 5 IU/L as the medical decision level (Table 4). This concentration was chosen because a large majority of healthy, non-pregnant individuals have hCG<5 IU/L, and also because it best aligned with the slightly different reference intervals provided by each manufacturer for their hCG assay [2], [4], [5], [6]. Concordance was 69% with i-STAT, 81% with Beckman, and 75% with Roche. The largest discordances were observed between the AQT90 and i-STAT (<5 versus 7 IU/L, respectively), and the AQT90 and Roche (<2 versus 9 IU/L, 5 versus 4 IU/L, 5 versus 3 IU/L) (Supplementary Table 6). All other discordant result pairs were within±1 IU/L of each other and, therefore, unlikely to have any clinical significance.

Table 4

Concordance analysis for the AQT90 hCG assay against comparator assays.

		AQT90
		Elevated (≥5 IU/L)	Negative (<5 IU/L)	Concordance (%)
i-STAT	Elevated (≥5 IU/L)	1	0	69
	Negative (<5 IU/L)	5	10
Beckman	Elevated (≥5 IU/L)	3	0	81
	Negative (<5 IU/L)	3	10
Roche	Elevated (≥5 IU/L)	2	0	75
	Negative (<5 IU/L)	4	10

Discussion

This study set out to evaluate the analytical performance of the Radiometer AQT90 FLEX βhCG assay. Study components included repeatability, within-laboratory precision, linearity, hook effect, carryover, and a method comparison with three assays that are currently used in Edmonton, Canada. Overall, the AQT90 hCG assay performed well and met most of the manufacturer's claims and TAE criteria for repeatability, within-laboratory precision, linearity, hook effect and carryover.

The results of this study suggest that the AQT90 performs comparably, or even better, to some other POCT hCG assays available in Canada. For example, the i-STAT assay demonstrated a clinically acceptable precision profile and excellent correlation with central laboratory hCG assays in two different studies [10], [11]. However, both studies noted a hook effect when hCG concentrations exceeded certain thresholds: 218,712 IU/L in one study [10] and 400,000–600,000 IU/L in the other [11]. The present study did not observe a hook effect with the AQT90 even at an hCG concentration of 2,446,448 IU/L. The analytical measurement range of the AQT90 also appears more suitable to the investigation of early pregnancy complications than that of the i-STAT. While both Radiometer and Abbot claim that their assays are for the detection of early pregnancy, clinical providers also use such assays to help investigate early pregnancy complications such as ectopic pregnancy. Expected hCG concentrations in such conditions [12], [13] typically are close to or exceed the upper end of the i-STAT reportable range, but they fall within the measurement range that was verified in this study for the AQT90.

A couple of limitations in AQT90 hCG assay performance were noted in this study. While the AQT90 hCG assay showed good linear correlation with comparator assays, a large degree of bias was evident against each comparator. This is not unexpected because each assay uses different antibodies, detection methods and/or is traceable to different reference materials. However, such large biases could lead to clinical confusion when a single patient ends up with hCG measurements from more than one assay as is often the case for high risk pregnancies that are followed in outpatient settings but require hospital admission for new-onset complications. Any health care system considering implementation of the AQT90 alongside other hCG assays should carefully consider how it will reduce patient safety risks arising from lack of assay standardization. Examples of mitigation strategies include appending methodology information to the hCG result, or providing interpretive comments for the change in hCG concentration relative to a previous result.

An additional limitation that was noted for the AQT90 in this study was its performance at low hCG concentrations. Repeatability data for one analyzer at 8 IU/L slightly exceeded the manufacturer's claims, and neither analyzer was able to meet at<20 IU/L the TAE goals issued for repeatability or within-laboratory precision by one of three professional groups. At hCG concentrations ∼5 IU/L, AQT90 results agreed with comparator results to variable degrees. The largest number of discordances (5 out of 16) were observed between the AQT90 and i-STAT. However, for all but one of these discordances, the results were within±1 IU/L of each other and also within TAE criteria specified by RCPA, IQMH and RiliBÄK. Therefore, these discrepancies are unlikely to be clinically significant. The largest discrepancy was observed between the AQT90 and Roche assays with one pair of results showing a difference of 7 IU/L. This difference exceeded RCPA, IQMH and RiliBÄK TAE limits and was also large enough to warrant clinical concern. However, i-STAT and Beckman results supported the validity of the AQT90 result, so it is likely that the Roche result was falsely elevated. The use of freeze-thawed samples with the Roche assay may have contributed to this false elevation, but additional contributory factors cannot be ruled out at this time.

This study has several limitations. First, limited access to different Solution Pack and Test Kit lot numbers did not allow investigation of lot-to-lot differences. Reagent lot-to-lot differences have previously been identified as an important source of variability in laboratory test results [14], [15], [16], with the Clinical and Laboratory Standards Institute even publishing a guideline on new reagent lot validation in 2013 [17]. Further evaluation of the AQT90 hCG assay using multiple reagent lots will, therefore, be an important step in fully characterizing its analytical performance.

The second limitation of this study is that it did not investigate the impact of biotin interference. Biotin has become a popular over-the-counter dietary supplement, and it is also being prescribed in mega-doses to a subset of multiple sclerosis patients owing to its beneficial effects on clinical outcomes and quality of life [18], [19]. Biotin-mediated interference has been documented in several immunoassays, leading to misdiagnosis and inappropriate treatment in some instances [20], [21], [22], [23], [24], [25], [26]. All of the AQT90 assays, including the hCG assay, rely on a biotinylated capture antibody that is immobilized on the reaction cup surface with the help of streptavidin [4]. While the AQT90 hCG assay package insert does not include any claims for biotin interference, the AQT90 troponin I package insert indicates that biotin does not interfere up to a concentration of 3 μg/L [27]. This appears promising as 3 μg/L approaches the mean day-7 peak serum biotin concentration that was measured for six healthy volunteers taking 10 mg/day of over-the-counter biotin supplements for seven days (>3.6 μg/L) [25]. However, additional studies that document interference, or lack thereof, for all other AQT90 assays and also at higher biotin concentrations are still needed.

Conclusions

Overall, the AQT90 hCG assay performed well in this study, and the analytical data suggest that it would be suitable for use at smaller suburban or rural hospitals that are unable to maintain hCG assays on automated, random-access immunoanalyzers in the laboratory. As with all assays, some limitations have been noted and should be kept in mind during implementation and clinical testing.