A practical approach for the validation and clinical implementation of a high-sensitivity cardiac troponin I assay across a North American city.

OBJECTIVES: Despite several publications on the analytical performance of high-sensitivity cardiac troponin (hs-cTn) assays, there has been little information on how laboratories should validate and implement these assays into clinical service. Our study provides a practical approach for the validation and implementation of a hs-cTn assay across a large North American City. DESIGN AND METHODS: Validation for the Abbott ARCHITECT hs-cTnI assay (across 5 analyzers) consisted of verification of limit of blank (LoB), precision (i.e., coefficient of variation; CV) testing at the reported limit of detection (LoD) and within and outside the 99th percentile, linearity testing, cTnI versus hs-cTnI patient comparison within and between analyzers (Passing and Bablok and non-parametric analyses). Education, clinical communications, and memorandums were issued in advance to inform all staff across the city as well as a selected reminder the day before live-date to important users. All hospitals switched to the hs-cTnI assay concurrently (the contemporary cTnI assay removed) with laboratory staff instructed to repeat samples previously measured with the contemporary cTnI assay with the hs-cTnI assay only by physician request.
RESULTS: Across the 5 analyzers and 6 reagent packs the overall LoB was 0.6 ng/L (n=60) with a CV of 33% at an overall mean of 1.2 ng/L (n=60; reported LoD=1.0 ng/L), with linearity demonstrated from 45,005 ng/L to 1.1 ng/L. Precision testing with a normal patient-pool QC material (mean range across 5 analyzers was 3.9-4.4 ng/L) yielded a range of CVs from 7% to 10% (within-run) and CVs from 7% to 18% (between-run) with the high patient-pool QC material (mean range across 5 analyzers was 29.6-36.3 ng/L) yielding a range of CVs from 2% to 5% (within-run) and CVs from 4% to 8% (between-run). There was agreement between hs-cTnI versus cTnI with the patient samples (slope ranges: 0.89-1.03; intercept ranges: 1.9-3.8 ng/L), however, the median CV on patient samples <100 ng/L across the analyzers was 5.6% for hs-cTnI versus 18.7% for the contemporary assay (p<0.001). Following the switch to hs-cTnI testing, no requests for repeat measurements were received.
CONCLUSIONS: Validation and implementation of hs-cTnI testing across multiple sites requires collaboration within the laboratories and between hospital laboratories and clinical staff.

Introduction

The introduction of high-sensitivity cardiac troponin (hs-cTn) assays into clinical use has had varied success within Canada [1], [2], [3]. This partly may be explained by insufficient analytical, clinical and educational material widely available to efficiently and effectively institute such a change within a North American city. There are, however, national and international materials published on the analytical aspects [4], [5], [6], and there has been excellent evaluation studies, both single center [7], [8] and multicenter studies [9], [10] on the two clinically approved hs-cTn assays in Canada (Roche Diagnostics hs-cTnT and Abbott Diagnostics hs-cTnI).

So why has there been problems, considering the reported analytical and clinical performance of the hs-cTn assays has always been demonstrated to be superior to the contemporary assays [11], [12], [13], [14]? One contributing factor could be the lack of consensus on the most appropriate cutoff and interpretation for hs-cTn concentrations [3]. Another, more practical aspect for clinical laboratories, could be the lack of published data on the key elements required for internal validation and implementation of hs-cTn assays. Locally, within the city of Hamilton (population>500,000) after clinical consultation, consensus and support, all four acute-care hospital core laboratories proceeded to implement hs-cTn testing. Detailed below is the step-by-step procedure followed by all laboratories for the validation and implantation of a hs-cTnI assay with materials and supplies readily available in all the laboratories.

Material and methods

Hospital laboratories, quality control, reagents

The four hospital core laboratories were the Juravinski Hospital and Cancer Centre (JHCC; Abbott ARCHITECT i2000SR analyzer), the Hamilton General Hospital (HGH; two Abbott ARCHITECT i2000SR analyzers), St. Joseph׳s Hospital (SJH; Abbott ARCHITECT i2000SR analyzer) and McMaster Children׳s Hospital (MCH; Abbott ARCHITECT i1000 analyzer). The laboratory practice for all four core hospital laboratories has been to prepare patient-pool quality control (QC) material (e.g., citrate phosphate dextrose plasma pool from Canadian Blood Services spiked with cTn) [15], [16] to monitor cTn at the 99th percentile. In preparation for transitioning to a hs-cTn assay, a “normal” low patient-pool QC material was also prepared. Two different size hs-cTnI reagent packs (100-test and 500-test) were evaluated on the JHCC analyzer as this analyzer was the first to identify the first replication outlier effect present with the 500-test pack size for the contemporary Abbott cTnI assay on certain analyzers [17]. MCH was the only other site which evaluated the hs-cTnI 100-test pack (i1000 only supports 100-test packs), with HGH and SJH laboratories both evaluating the 500-test packs.

Limit of the blank and precision testing

Each site was instructed to run water as a patient 10 times on each instrument and reagent pack size (n=60 in total, as 10 water tested on 500-test pack and 100-test pack at JHCC). The limit of the blank (LoB) was determined by the mean concentration of the water (n=60)+3SD. Patient-pool EDTA plasma with a measured concentration of approximately 1 ng/L (to reflect the reported limit of the detection (LoD) for this assay) [8] at the JHCC was aliquoted, frozen (−20 C) and distributed for testing on each analyzer and reagent pack size (n=60 in total measurements; performed as within-run). The normal patient-pool QC and high patient-pool QC material (frozen aliquots below −70 C) were measured for within-run (n=10 tests) and for between-run precision (over 4 weeks) on the 5 different analyzers.

Linearity and patient comparison testing

An extremely high cTnI concentration EDTA patient-pool (approximately 50,000 ng/L, frozen below −20 C) was distributed to each site for linearity testing. Each site performed 16 serial dilutions with the Abbott Diagnostic multi-assay diluent (manufacturer recommended diluent for the hs-cTnI assay). Each site performed duplicate testing on each serial dilution (exception JHCC with the 100-pack). Linearity was achieved if the site׳s measured concentrations at all 17 levels were within 2SD of the average measurement for each level. Forty frozen (below −20 C) EDTA patient-pools (n=8: <10 ng/L, n=10: 10–30 ng/L, n=13: 31–300 ng/L; n=9: >300 ng/L) were distributed to each site to run for hs-cTnI and cTnI. The JHCC measured the 40 samples on the hs-cTnI 500-test pack, 100-test pack and the cTnI 100-test pack; SJH measured the 40 samples on the hs-cTnI 500-test pack and the cTnI 100-test pack; MCH measured the 40 samples on the hs-cTnI 100-test pack and the cTnI 100-test pack; and the HGH measured the 40 samples on analyzer 1 in duplicate: hs-cTnI 500-test pack and cTnI 500-test pack and on analyzer 2 in duplicate for the hs-cTnI 500-test pack. Passing and Bablok regression analyses were performed with Analyse-it software with proportional or absolute differences noted only if the 95%CI of the slope or intercept did not include 1.00 or 0 ng/L, respectively. The average concentration of each sample across all analyzers for both hs-cTnI and cTnI were calculated as well as the CV, with differences between hs-cTnI and cTnI measurements and imprecision assessed by Mann–Whiney non-parametric testing (Statsdirect software, with two-sided p<0.050 considered significant).

Implementation of hs-cTnI

After clinical agreement and support for the hospitals’ core laboratories to proceed to switch cTnI testing to hs-cTnI testing, extensive education and dissemination of this forthcoming change occurred via various educational activities and documents (targeted to all hospital staff, trainees, and students). The day before the switch a targeted communication to the emergency departments, internal medicine and cardiology services at the hospitals also occurred as a reminder. Each laboratory concurrently stop the contemporary cTnI testing (reported in µg/L to two decimal places) and proceeded with hs-cTnI testing (reported in ng/L, and in whole numbers). The practice of reporting hs-cTnI in whole numbers has been previously advocated and supported [6], [18] and the following comment “⁎Units (ng/L) as high-sensitivity assay” was appended to the hs-cTnI results for further clarity.

As the stability of cTnI as measured by the Abbott hs-cTnI assay has already been documented [19], the core laboratory staff at all hospitals were instructed after the switch to hs-cTnI testing to repeat samples previously measured with the contemporary cTnI assay with the hs-cTnI assay only by physician request. To assess the impact on repeat testing; queries in the laboratory information system databases were performed to identify if a previous specimen number with a cTnI result also had a hs-cTnI result. Finally, to assess overall how the implementation proceeded, the Biochemist (PK) the following day asked each laboratory as well as directly following up with the targeted services (emergency medicine, internal medicine, and cardiology) if there were any concerns or problems with the implementation.

Results

The Abbott hs-cTnI assay across 5 instruments and 6 reagent packs yielded a LoB of 0.6 ng/L. The average concentration in water was 0.12 ng/L with a CV=141% (n=60 measurements). The LoD has been reported as 1.0 ng/L [8], and a patient-pool targeted at this concentration yielded an overall CV of 33% at an average concentration of 1.2 ng/L [n=60]. The within-run precision of the normal patient-pool QC material (n=10) was as follows: SJH mean=4.1 ng/L, CV=10%; HGH-1 mean=4.3 ng/L, CV=10%; HGH-2 mean=4.3 ng/L, CV=7%, JHCC mean=4.1 ng/L, CV=10%; MCH mean=4.2, CV=7%; with the high patient-pool QC material yielding the following within-run precision: SJH mean=33.7 ng/L, CV=5%; HGH-1 mean=29.6 ng/L, CV=3%; HGH-2 mean=34.9 ng/L, CV=2%; JHCC mean=32.7 ng/L, CV=4%; MCH mean=34.7, CV=3%. The between day performance over 4 weeks for each of these materials across the sites were as follows: normal patient-pool QC mean range: 3.9–4.4 ng/L; CV range: 7–18%; high patient-pool QC mean range: 32.2–36.3 ng/L; CV range: 4–8% (n>20 data points for each material on each analyzer).

Each of the analyzers demonstrated acceptable linearity with the average measurement for each of the 17 levels falling within the allowable limits (±2SD from the overall level average). Table 1 displays the overall average concentrations for the 17 different levels and the upper and lower limits as well as the precision obtained from the 11 measurements per level. The last dilution was set to yield a result below the LoD, and the absolute % difference between the overall average concentration and the predicted concentration was 1.2% on this last dilution. Overall, there was ≤30% absolute difference between the average concentration and the predicted concentration. The linearity regression analyses for the serial dilutions are illustrated in Fig. 1.

Table 1

Linearity, precision and acceptable limits for 5 analyzers and the 500- and 100-test packs for hs-cTnI (ng/L).

Serial dilution	Overall hs-cTnI assignment average (ng/L) (All measurements, n=11)	Standard deviation (SD)	Coefficient variation (CV) (%)	Low (mean−2SD)	High (mean+2SD)
Concentrate	45,004.6	1733.7	4	41,537.1	48,472.0
1 in 2	24,897.4	992.1	4	22,913.3	26,881.5
1 in 4	11,809.4	554.2	5	10,701.0	12,917.7
1 in 8	5355.1	263.9	5	4827.2	5883.0
1 in 16	2445.7	138.3	6	2169.0	2722.3
1 in 32	1153.3	89.8	8	973.8	1332.8
1 in 64	565.8	34.8	6	496.1	635.5
1 in 128	268.8	23.8	9	221.2	316.3
1 in 256	135.7	15.4	11	105.0	166.5
1 in 512	67.1	5.9	9	55.3	78.9
1 in 1024	33.2	4.3	13	24.6	41.8
1 in 2048	16.2	2.4	15	11.5	21.0
1 in 4096	8.0	1.1	14	5.8	10.2
1 in 8192	3.8	0.6	15	2.7	4.9
1 in 16,384	2.1	0.5	27	1.0	3.2
1 in 32,768	1.1	0.3	31	0.4	1.7
1 in 65,536	0.7	0.4	57	−0.1	1.5

Fig. 1

Linear regression analysis for the serial dilutions for the hs-cTnI assay.

There was no significant proportional difference (i.e., slope 95%CI included 1.00) between hs-cTnI versus cTnI by Passing and Bablok regression analyses on the 5 analyzers assessing the different reagent combinations with the 40 patient-pool EDTA plasma samples; however, there appeared to be a slight absolute bias (y-intercept). To address this, the average hs-cTnI concentrations (n=8 measurements for the 40 samples) was compared to the average cTnI concentrations (n=5 measurements for the 40 samples). Passing and Bablok regression analyses did indicate a small absolute bias (3.4 ng/L; 95%CI: 1.2–4.6) but no significant proportional bias (see Fig. 2). Overall, however, there was no significant difference between the average concentrations measured with the hs-cTnI assay versus the contemporary cTnI assay on these 40 samples (hs-cTnI median (25th–75th)=41 ng/L (13–263) versus cTnI median (25th–75th)=38 ng/L (8–366), p=0.59).

Fig. 2

Passing and Bablok regression analyses on the average concentrations of 40 EDTA patient-pool plasma samples (range: 1.6–7016 ng/L) between hs-cTnI and cTnI.

A comparison between the CVs of the two assays on these 40 patient-pool EDTA plasma samples, did indicate superior precision for the hs-cTnI assay (hs-cTnI median CV=5.6% versus cTnI median CV=10.5%, p=048). A closer examination indicated that samples (n=15) with concentrations >100 ng/L there was no statistical difference between the CVs (hs-cTnI median CV=5.7% versus cTnI median CV=3.4%; p=0.074). However, below 100 ng/L (n=25 samples) the hs-cTnI assay had superior precision (hs-cTnI median=5.6% versus cTnI median=18.7%; p<0.001) (see Fig. 3). Despite the excellent analytical precision of the hs-cTnI assay on the 40 patient-pool EDTA plasma samples, there were still samples that gave different results on the different analyzers (see Table 2).

Fig. 3

Precision profile for hs-cTnI and cTnI with patient-pool EDTA plasma samples with concentrations <100 ng/L.

Table 2

Comparability testing on 40 patient-pool EDTA plasma samples for hs-cTnI (ng/L).

Abbott hs-cTnI (n=8 measures on same samples)
Average hs-cTnI (ng/L)	Minimum result	Maximum result	Absolute difference between maximum and minimum results	Coefficient variation (CV) (%)
1.6	1.3	2.2	0.8	18
2.3	2.0	2.8	0.8	12
4.1	3.9	4.3	0.4	4
4.7	3.6	5.7	2.1	15
7.1	6.2	8.0	1.8	8
7.9	7.5	8.3	0.7	4
9.7	8.7	10.6	1.9	7
9.7	8.8	10.9	2.1	8
10	9.3	10.6	1.2	5
12	11	21	10	31
13	12	14	2	6
15	13	18	5	12
16	15	17	2	4
17	13	20	7	15
18	16	19	3	5
19	17	20	3	5
20	18	23	5	9
27	24	28	4	5
31	29	33	4	5
37	33	45	12	10
44	40	50	10	7
46	44	50	6	4
53	49	56	7	5
64	62	67	5	3
64	60	70	10	5
117	110	125	15	5
137	131	147	17	4
147	138	156	19	6
167	158	185	27	6
242	207	299	92	15
284	270	300	30	4
419	399	458	59	5
527	489	574	85	6
1145	936	1317	381	12
1349	1250	1412	162	5
1506	1380	1639	259	7
1546	1321	1747	426	11
1641	1534	1853	319	7
3350	3126	3483	357	5
7016	6597	7409	812	4

On the day of the implementation for the hs-cTnI assay, there were no physician requests to measure hs-cTnI on previous samples reported with the contemporary cTnI assay. Laboratory and clinical staff within the hospitals reported no major issues.

Discussion

The hs-cTnI assay has superior precision at lower concentrations as compared to the contemporary cTnI assay and is more analytically sensitivity (hs-cTnI LoD is 1 ng/L versus 10 ng/L or 0.01 µg/L for the contemporary cTnI assay). This study, which used available resources and supplies, with a set protocol verified the LoB and the precision estimates previously reported for the Abbott ARCHITECT's hs-cTnI assay [8]. The Clinical and Laboratory Standards Institute has very detailed and prescribed methods to determine the LoB and LoD; however, the necessity to re-establish these estimates when literature provides these parameters may need to be re-evaluated. Another mechanism that could be proposed for laboratories that may not have the resources or expertise to perform such extensive testing may be of great value. Our study proposes such an alternative mechanism to verify the LoB, to test precision at the reported LoD, and assess the linearity range. Evaluating all three variables across 5 different analyzers and 6 reagents packs, we have established the lower limit of reporting for the hs-cTnI assay as 1 ng/L, with concentrations below this limit reported as <1 ng/L. This mechanism may also prove to be useful for subsequent longitudinal monitoring of the hs-cTnI assay for these important variables (i.e., periodically assessing if precision at 1 ng/L is ≤33% and LoB ≤0.6 ng/L).

Moreover, the whole study protocol should be of interest to others wishing to validate and implement hs-cTnI testing across several different hospitals using either the i2000SR or i1000 analyzers on either the 500-test or 100-test reagent packs. Also, the linearity testing was performed by simple serial dilution using the multi-assay diluent. The percent deviation was a little higher than other reports [8], [10], which might be explained using the multi-assay diluent (rather than an undetectable hs-cTnI pool) and the multiple dilutions that were performed (16 in total). However, considering imprecision estimates into the linearity testing and performing dilutions through the reference interval and below the reported LoD for the hs-cTnI assay, the practical approach of using the multi-assay diluent may be of particular importance for laboratories investigating potential interferences, which may be identified with serial dilutions.

The introduction of a “normal” hs-cTnI patient-pool QC material for validation, implementation and on-going quality assurance is also unique to this study and for hs-cTn testing. Myocardial injury is being assessed in various acute hospitalized populations [20] with interest into primary and secondary prevention [21], [22], [23], [24], [25], [26], [27], [28], so there is a greater need for the clinical laboratory to monitor this assay below the 99th percentile. Even for the physicians within emergency departments who want a precise assay at the 99th percentile cutoff, there is also much interest with some intriguing publications suggesting an undetectable and/or low hs-cTn result may identify patients at low risk for acute coronary syndrome and possibly early discharge [29], [30], [31]. Therefore, monitoring the hs-cTn assays at these decisions points may become essential [32]. In our educational activities and laboratory reports we always state that cardiac troponin reflects myocardial injury and not necrosis. We also reiterated in activities leading to and after the implementation of the hs-cTnI assay that measurable quantities of cardiac troponin I will now be more pervasive in patients being evaluated for acute coronary syndrome. The dialog with clinical users before, during and after the transition to hs-cTnI has been essential to help mitigate confusion around this new assay.

Another aspect not typically addressed for hs-cTn testing (using the same assay) is comparability testing, which is important for institutions with multiple analyzers or who work in a network and/or laboratory program. For instance, within a city there may be times when analyzers are down and samples sent to another site for testing or more commonly there will be patients transferred from one hospital to another for specific procedures. Therefore, comparability testing for hs-cTnI across hospital core laboratories is essential. The findings from this study indicates that agreement, as assessed by CV, is superior for hs-cTnI as compared to cTnI; however there are still some samples that yield very discrepant results. Specifically, analytical differences up to 12 ng/L between analyzers testing the same samples at concentrations below 65 ng/L were observed for the hs-cTnI assay in this study, consistent with another study suggesting up to 10 ng/L as variation [33]. These data lend support that greater absolute changes in hs-cTnI concentrations over time are needed for ruling-in events [34].

Finally, the implementation was successful, due to the fact that there was already clinical agreement and support for this change, that there was core laboratory staff available to make the quality control material, to do the analytical testing and the method comparisons (which in this study showed closer agreement than our previous publication, most likely due to fewer high concentration samples [19]), and that there was ample education/dissemination activities involving the hs-cTnI change. The good communication and collaboration within and between laboratory and clinical staff were keys to a successful validation and clinical implementation.