A comparison of the analytical level of agreement of nine treponemal assays for syphilis and possible implications for screening algorithms.

OBJECTIVE: The serological diagnosis of syphilis requires the detection of two distinct antibodies, the non-treponemal and trepomenal. Center for Disease Control and Prevention (CDC) recommends screening first with a non-treponemal test such as (Rapid Plasma Reagin/Venereal Disease Research Laboratory), and then confirming those results with one of the several treponemal tests (Fluorescent Treponemal Antibody-Absorption (FTA-ABS), Enzyme Immunoassay, chemiluminescence, treponema pallidum particle agglutination (TP-PA) or Point of Care). Owing to the high volume of samples processed by some laboratories using automated systems, the screening with treponemal assays and confirming with non-treponemal tests is becoming the established norm. The purpose of this study was to evaluate eight treponemal assays using TP-PA as the predicate assay.
METHODS: 290 stored serum samples were tested qualitatively according to the manufacturer's directions.
RESULTS: Concordance with specimens tested as reactive or non-reactive using TP-PA was: FTA-ABS 94.5-100%, Trep-Sure 100-98.9%, BioELISA 100-98.9%, INNO-LIA 99.1-99.4%, BIOLINE 100-98.9%, CAPTIA IgG 100-97.2%, Trep-ID 100-100% and LIAISON 100-99.4%. In order to properly evaluate the performance of these assays, the analytical sensitivity was determined by endpoint titration of serial dilutions of the reactive serum samples in normal sera. The median endpoint titre varied from 1:4 for FTA-ABS to 1:512 for Trep-Sure.
CONCLUSIONS: The performance of the treponemal serological assays was comparable while using medium and high-titre sera. However, the varying performance on specimen dilutions suggests that there may be differences in sensitivity with low-titre sera that are more prevalent in primary and late syphilis cases.

The study demonstrates that selecting the reverse algorithm in syphilis serology could have an adverse effect on the serological diagnosis of syphilis if a less analytical sensitive test is used.

The study suggests that the reverse algorithm in syphilis serology is conducive to variability of results due to the differences in analytical sensitivity of the assays.

The finding of the study was based on the results obtained with stored sera. Further studies with documented clinical samples are necessary to demonstrate correlation between antibody titre and the stage of the disease.

Introduction

Syphilis is a sexually transmitted disease caused by the bacterium Treponema pallidum. The clinical diagnosis is difficult due to the complexity of manifestations which requires the aid of serological interpretation. The serological diagnosis of syphilis depends on the detection of two distinct antibodies, the non-treponemal or heterophile antibodies (reagin) directed against cardiolipin released from damaged host cells and from the treponemes themselves. These antibodies can also be present in other diseases and human conditions such as (lupus, malaria, HIV, intravenous drug users, etc).1 The presence of non-treponemal antibodies is indicative of active infection, thus a reduction in titre can suggest a successful antibiotic therapy, and a significant increase can indicate a possible relapse or reinfection.2 The treponemal antibodies are primarily directed against specific lipoprotein antigens of the bacterium such as 15, 17 and 47 kDa. Even after treatment or time, these treponemal antibodies usually remain present for life. A positive treponemal test cannot distinguish between active, treated and old cases of syphilis.

The traditional algorithm is to screen with a non-treponemal assay such as Venereal Disease Research Laboratory (VDRL) or Rapid Plasma Reagin (RPR) and those serum samples found reactive are then confirmed using a treponemal test. With the introduction of automated systems which are appropriate for high output volumes, the reverse algorithm is gaining acceptability and its usefulness is arguably justified.3 However, the adoption of this reverse algorithm has increased the number of discrepant results between the screening and confirmatory tests.4 5 One reason the traditional algorithm was established was to avoid the detection of previously treated cases, especially in low prevalence settings where the majority of sera will test negative and not require further evaluation, unless a patient has been reinfected or in rare circumstances, their non-treponemal antibody titres remain serofast. With the application of the reverse algorithm even in low prevalence settings all previously treated cases will likely be detected. When the screening treponemal test is positive and the confirmatory non-treponemal test is negative, the Center for Disease Control and Prevention (CDC) guidelines recommend using a second treponemal test to validate the results of the screening test. The selection of a second confirmatory test introduces the possibility of having an analytically less-sensitive test which would suggest a false positive screening or a less-sensitive second confirmatory test. Then it is important to determine if all available treponemal assays are comparable in analytical sensitivity to avoid uncertainty in their selection.6

Methods

Two hundred ninety stored serum samples randomly selected from our serum bank were used for this study. Originally the serum samples were obtained from the Georgia Public Health Laboratory with all identifiers removed. Owing to the prevalence of high lipid contents of the sera, the samples were treated with Cleanascite HC (PureBiotech, Middlesex, New Jersey, USA).7 The patterns of reactivity of the test panel were determined at CDC by a quantitative RPR (Becton Dickinson, Baltimore, Maryland, USA) and a qualitative treponema pallidum particle agglutination (TP-PA; Fujirebio Diagnostics, Inc, Malvern, Pennsylvania, USA). All serum samples were divided into aliquots and frozen at −20°C so that the number of freeze–thaw cycles was consistent for each sera tested. Eight of the treponemal assays: fluorescent treponemal antibody-absorption (FTA-ABS; Zeus Scientific, Raritan, New Jersey, USA), LIAISON Treponema Assay (DiaSorin, Inc, Stillwater, Minnesota, USA), SD BIOLINE Syphilis 3.0 Rapid Point of Care (POC) test (Standard Diagnostics, Inc, Korea), INNO-LIA immunoblot (Innogenetics, Gent, Belgium), BioELISA (BioKit, Barcelona, Spain), and the CAPTIA IgG, Trep-ID and Trep-Sure (Trinity Biotech, Jamestown, New Jersey, USA) were tested qualitatively, then quantitatively with reactive samples. In the quantitative test, twofold serial dilutions (from 1:2 to 1:16,384) were prepared using normal non-reactive human plasma converted to serum8 as a diluent. All dilutions were treated as neat sera and tested with the nine different assays according to the manufacturer's package insert. The test was performed blind; the enzyme immunoassay (EIA) results were collated and compared to the TP-PA used as the predicate for the qualitative test.

Statistical methods

The endpoint titre data were converted to dilution data from the exponential scale (titre) to a linear scale (dilution) by taking the logarithm of the endpoint titre, that is, log2(titre)=dilution. This transformation reduces the variance within the test and between the tests. A generalised linear model was fitted to these data by the generalised estimating equations (GEE) method to test for the differences between the mean dilutions of these tests. Adjustments for multiple comparisons between the mean dilutions of these tests were made by Tukey-Kramer method. A p value of <0.05 was used as a significance level for the differences. All statistical analyses were conducted using SAS, V.9.3 (SAS Institute, Cary, North Carolina, USA).

Results

All treponemal syphilis assays were performed according to the manufacturer's direction circulars using 290 stored serum samples of which 109 were reactive and 181 were non-reactive for TP-PA. Concordance with specimens tested as reactive or non-reactive using TP-PA was: FTA-ABS 94.5–100%, INNO-LIA 99.1–99.4%, LIAISON 100–99.4%, Trep-Sure 100–98.9%, BioELISA 100–98.9%, BIOLINE 100–98.9%, CAPTIA IgG 100–97.2% and Trep-ID 100–100%. These results indicated that most of the assays were comparable to TP-PA when the serum samples were tested undiluted. However, FTA-ABS failed to detect six TP-PA reactive samples and one INNO-LIA, while the other four assays missed none. One hundred and nine stored serum samples reactive by TP-PA were serially diluted in non-reactive human plasma converted to serum,8 with the purpose of demonstrating the endpoint titre of each sample with the nine different assays. With a simple comparison the endpoint titres indicated that Trep-Sure had a median value 1:512, BioELISA 1:128, LIAISON 1:64, Trep-ID 1:64, BIOLINE 1:32, INNO-LIA 1:16, TP-PA 1:16, CAPTIA IgG 1:8 and FTA-ABS 1:4 (table 2).

Table 2

Mean dilutions (quantitative endpoint titres) for nine treponemal syphilis assays of TP-PA reactive sera

Test	Median titre	Estimates of mean dilutions and 95% CL				Mean dilution difference from FTA-ABS
		Mean dilution	SE	Lower95% CL	Upper95% CL	Mean difference	Std error
FTA-ABS*	4	1.9266	0.1655	1.6022	2.2510	NA	NA
CAPTIA IgG*	8	3.1560	0.1552	2.8517	3.4602	1.229	0.082
TP-PA†	16	4.0734	0.1624	3.7551	4.3917	2.147	0.102
INNO-LIA*	16	4.0917	0.1927	3.7141	4.4694	2.165	0.123
BIOLINE†	64	5.6606	0.1886	5.2910	6.0301	3.734	0.135
Trep-ID†	64	6.2752	0.1906	5.9016	6.6489	4.349	0.123
LIAISON†	128	6.3119	0.1603	5.9977	6.6262	4.385	0.105
BioELISA†	128	7.1560	0.1789	6.8053	7.5066	5.229	0.112
Trep-Sure†	512	8.8899	0.1747	8.5475	9.2323	6.963	0.1182

*IgG only.

†IgG/IgM detection.

CL, confidence limits; FTA-ABS, fluorescent treponemal antibody-absorption; NA, not applicable; TP-PA, treponema pallidum particle agglutination.

Of the 109 TP-PA reactive sera, 95 (85%) were non-reactive for the confirmatory RPR test. Fourteen samples were reactive for TP-PA and RPR. All other assays were able to detect treponemal antibody in those undiluted sera that were reactive by the TP-PA. However, five sera found to be reactive by the CAPTIA IgG test and one by the BIOLINE were found to be non-reactive and all other tests suggest false positives. All other sera found to be non-reactive by the TP-PA but reactive by another test could be considered as true positives since the sera were found to be reactive by at least two other tests (see table 1). If the TP-PA would have been selected as the screening test for the reverse algorithm then eight samples reactive by the other assays would have been missed resulting in a false serological result.

Table 1

Per cent concordance of eight qualitative syphilis treponemal tests compared to the predicate TP-PA.

TP-PA predicate	Total number of samples	FTA-ABS		INNO-LIA		LIAISON		Trep-Sure		BioELISA		BIOLINE		CAPTIA (IgG)		Trep-ID
		R	NR	R	NR	R	NR	R	NR	R	NR	R	NR	R	NR	R	NR
R	109	103	6 (94.4)	108	1 (99.1)	109	0 (100)	109	0 (100)	109	0 (100)	109	0 (100)	109	0 (100)	109	0 (100)
NR	181	0	181 (100)	1	180 (99.4)	1	180 (99.4)	2	179 (98.9)	2	179 (98.9)	2	179 (98.9)	5	176 (97.2)	0	181 (100)

FTA-ABS, fluorescent treponemal antibody-absorption; NR, non-reactive; R, reactive; TP-PA, treponema pallidum particle agglutination.

Mean dilutions and their SEs of eight quantitative treponemal assays and the TP-PA estimated by the generalised linear model are shown in table 2. There were significant (p<0.0001) differences between the mean dilutions of all tests, except between TP-PA and INNO-LIA (p=1.000) and Trep-ID and LIAISON (p=1.000). This analysis was repeated after subtracting the FTA-ABS titre from the titre of the corresponding test. In this analysis also there was no significant difference between TP-PA and INNO-LIA (p=1.000) and Trep-ID and LIAISON (p=1.000), and the differences between the rest of the pairs were significant (p<0.0001; table 2).

Discussion

The reverse sequence syphilis screening algorithm is being used because it takes advantage of automated systems. However, the selection of these assays to screen serum samples with suspected cases of syphilis presents problems that were not clearly anticipated.6 In cases where the treponemal test is positive and the confirmatory RPR test is negative, the CDC recommends a second treponemal test be used to determine the serological validity of the screening test.5 If the second treponemal test is negative, then it is inconclusive whether the first screening test is a false positive or a more sensitive test. This situation presents a dilemma for physicians and understandably they will be asking for clarification as to whether it is or not a case of syphilis in asymptomatic situations. Consequently, it would be advisable for the laboratorian to consider the selection of more sensitive screening and confirmatory treponemal assays to avoid having discrepant results. This is not an easy task because of the different assay platforms presently used for screening in the reverse algorithm system ranging from FTA-ABS, EIA, chemiluminescence (CIA), TP-PA and POC. In this study an attempt was made to determine the analytical level of agreement of the assays with the purpose of better understanding the abilities of the assays for detection of low-level antibodies. When the nine assays were tested quantitatively, the FTA-ABS was the treponemal test with the lowest median endpoint titre (1:4). In contrast, Trep-Sure had a median endpoint titre 1:512; table 2). Since the FTA-ABS was the least reactive, its re-evaluation should be considered.

Further studies with documented clinical samples are necessary to demonstrate the correlation between the antibody titre and the stage of the disease. With these comparison results we then could infer that there are marked differences between the analytical sensitivities of these assays especially with low titre sera such as those found in primary or latent cases. The variability of results with the different assays may be due in part to the composition of their systems. Those assays using the double antigen or sandwich configuration such as Trep-Sure, BioELISA, Trep-ID and LIAISON where the recombinant antigen such as Tp15, Tp17, Tp47 KD or combination of them are both attached to the ELISA plate or latex particle and are directly conjugated to horse radish peroxidase or CIA. This configuration tends to be more sensitive because they are able to detect IgG as well as IgM antibodies in the sera of patients with syphilis. In addition, the detection of IgM antibodies amplifies the signal due to its multiple binding sites. On the other hand, normally those assays using the indirect or antihuman system such as CAPTIA (IgG) and FTA-ABS are limited to that they only detect one single immunoglobulin.

When selecting the reverse algorithm system for syphilis diagnostic serology and in cases when the treponemal test is reactive and the RPR non-reactive, it would be advisable to consider the selection of more analytically sensitive screening and confirmatory treponemal assays preferably those based on the double antigen or sandwich method, such as Trep-Sure, BioELISA, Trep-ID and LIAISON to avoid the necessity of using a less sensitive second confirmatory treponemal assay.