Large differences between test strategies for the detection of anti-Borrelia antibodies are revealed by comparing eight ELISAs and five immunoblots.

We investigated the influence of assay choice on the results in a two-tier testing algorithm for the detection of anti-Borrelia antibodies. Eighty-nine serum samples from clinically well-defined patients were tested in eight different enzyme-linked immunosorbent assay (ELISA) systems based on whole-cell antigens, whole-cell antigens supplemented with VlsE and assays using exclusively recombinant proteins. A subset of samples was tested in five immunoblots: one whole-cell blot, one whole-cell blot supplemented with VlsE and three recombinant blots. The number of IgM- and/or IgG-positive ELISA results in the group of patients suspected of Borrelia infection ranged from 34 to 59%. The percentage of positives in cross-reactivity controls ranged from 0 to 38%. Comparison of immunoblots yielded large differences in inter-test agreement and showed, at best, a moderate agreement between tests. Remarkably, some immunoblots gave positive results in samples that had been tested negative by all eight ELISAs. The percentage of positive blots following a positive ELISA result depended heavily on the choice of ELISA-immunoblot combination. We conclude that the assays used to detect anti-Borrelia antibodies have widely divergent sensitivity and specificity. The choice of ELISA-immunoblot combination severely influences the number of positive results, making the exchange of test results between laboratories with different methodologies hazardous.

Introduction

Lyme disease is caused by Borrelia spp. In Europe, infection is mostly caused by B. afzelii and B. garinii, while in the United States, B. burgdorferi sensu stricto is the causative agent [1]. Lyme disease manifests in a myriad of clinical ways, including erythema migrans, arthritis, carditis and neuroborreliosis [1]. Extracutaneous Lyme disease requires laboratory confirmation by culture, polymerase chain reaction (PCR) or antibody determination [2, 3]. Culture is only available in a limited number of laboratories, and the value of PCR in the diagnosis of various forms of Lyme disease is of limited use [2, 3]. Therefore, serological assays are the main method used to diagnose extracutaneous forms of Lyme disease.

Current guidelines for the diagnosis of Lyme disease include a two-tier testing algorithm [2, 3]. First, an enzyme-linked immunosorbent assay (ELISA) is performed, followed by the confirmation of positive ELISA results with an immunoblot. This two-step procedure was initiated because first-generation ELISAs for the detection of anti-Borrelia antibodies lacked specificity. The inclusion of a second, more specific, serological method made it possible to exclude false-positive ELISA samples [2, 4].

Many diagnostic assays are currently commercially available, and manufacturers have developed them to increase their sensitivity and specificity. During the last decade, assays using a peptide from the sixth invariant region (C6) of the variable major protein-like sequence-expressed (VlsE) of B. burgdorferi have been shown to be promising [5, 6]. Laboratories can choose between ELISAs and immunoblots using sonicated whole-cell antigens, whole-cell antigens combined with recombinant antigens (VlsE C6 peptide) and exclusively recombinant antigens. Due to this array of serological tests, there are an almost indefinite number of possible combinations between ELISA and immunoblot in a two-tier testing scheme. Comparing anti-Borrelia test results between laboratories and studies may be impossible if tests with widely diverging sensitivities and specificities are used [7].

The aim of the present study was to compare a wide range of ELISA assays and immunoblots, based on either whole-cell or recombinant antigens, for detecting anti-Borrelia antibodies. We also aimed to investigate the influence of assay choice on results in a two-tier testing algorithm (ELISA followed by immunoblot). Therefore, we tested serum samples in eight ELISA systems and five immunoblots, covering the entire spectrum of native and recombinant antigens.

Patients and methods

Patients

Serum samples were selected from 89 clinically well-defined individuals. Fifty-nine samples were from patients suspected of Borrelia infection (skin manifestations, n = 8; neurological symptoms, n = 26; arthritic symptoms, n = 11; ocular symptoms, n = 4; other, n = 10). Fourteen samples were from healthy controls and 16 came from patients with a high possibility for cross-reacting antibodies (syphilis patients, n = 10; Mycoplasma pneumonia-infected patients based on symptoms consistent with M. pneumoniae infection and a positive result for anti-M. pneumoniae IgM and IgG with a Virion/Serion ELISA , n = 6).

Methods

Serum samples were tested in eight different ELISA systems. Three assays were based on sonicated whole-cell antigens (Diacheck/Moran anti-Borrelia, VIDAS and Virion/Serion ELISA Classic Borrelia burgdorferi), three assays with sonicate whole-cell antigens supplemented with VlsE for IgG anti-Borrelia antibodies (Dade Behring Enzygnost Lyme link VlsE, Euroimmun Anti-Borrelia plus VlsE ELISA and Genzyme Virotech Borrelia afzelii + VlsE ELISA) and two assays using recombinant proteins (Immunetics C6 Lyme ELISA Kit and Mikrogen recomWell Borrelia). A subset of samples from 31 patients suspected of Borrelia infection were also tested in five different immunoblots. This group consisted of the following patients: skin manifestations, n = 3; neurological symptoms, n = 15; arthritic symptoms, n = 6; ocular symptoms, n =2; other, n = 5. One whole-cell blot (home-made using B. afzelii strain A39 cell sonicate, RIVM), one whole-cell blot supplemented with VlsE (Viramed Borrelia “MiQ” + VlsE ViraBlot) and three recombinant blots (Euroimmun Euroline-RN-AT, Mikrogen recom Line Borrelia and Genzyme Virotech Borrelia Europe Line). A total of 31 samples were tested in all immunoblots.

Manufacturer-suggested cut-off levels and interpretation criteria were used for the ELISAs and immunoblots. Statistical analysis was performed using SPSS version 16.0 (SPSS Inc., Chicago, IL, USA).

Results

As expected, there was considerable discordance between the eight ELISAs. We tested 89 samples from patients and controls on all eight ELISAs. Of the complete set of serum samples, 35/89 (39%) were negative in all assays, while 16/89 (18%) were positive in all assays. The remaining 38/89 (43%) samples were positive in one to seven ELISAs.

In the 59 patients that were suspected of Borrelia infection, we observed a wide range of positive results, with percentages of positive ELISAs varying between 34 and 61% (Table 1). We did not observe a relation between the fraction of positive results and the nature of antigen used for the ELISA. The specificity of the ELISAs also varied widely. Although we had only small numbers of positive tests in healthy controls, some ELISAs produced up to 38% of positive tests in the cross-reactivity group (syphilis and M. pneumonia-infected patients).

Table 1

Performance of eight enzyme-linked immunosorbent assay (ELISAs) in the three patient groups

ELISA manufacturer	Antigen used for ELISA	Number of positive samples (%)			Total number of tested samples
		Patients suspected for Borrelia infection	Cross-reactivity controls	Healthy controls
Diacheck/Moran	Whole-cell	20/59 (34%)	2/16 (13%)	1/14 (7%)	89
VIDAS	Whole-cell	31/59 (53%)	4/16 (25%)	1/14 (7%)	89
Virion/Serion	Whole-cell	24/59 (41%)	1/16 (6%)	0/14	89
Enzygnost	Whole-cell + VlsE	23/59 (39%)	0/16	0/14	89
Euroimmun	Whole-cell + VlsE	29/59 (49%)	3/16 (19%)	0/14	89
Virotech	Whole-cell + VlsE	35/59 (59%)	6/16 (38%)	0/14	89
Immunetics	Recombinant	22/59 (37%)	0/16	0/14	89
Mikrogen	Recombinant	24/59 (41%)	3/16 (19%)	0/14	89

We aggregated results from the IgM and IgG tests and assessed them using a kappa statistic to determine agreement between the ELISAs. The kappa values ranged from 0.41 (moderate agreement) to 0.79 (substantial to good agreement), emphasising the differences between the ELISAs (Table 2). The choice of antigen does not seem to influence the level of agreement. Even the lowest kappa values were observed between two ‘whole-cell + VlsE’ ELISAs (0.43).

Table 2

Agreement between ELISAs for detecting IgM and/or IgG anti-Borrelia antibodies (kappa values)

ELISA manufacturer	Antigen used for ELISA	Diacheck/Moran	VIDAS	Virion/Serion	Enzygnost	Euroimmun	Virotech	Immunetics
Diacheck/Moran	Whole-cell	-	-	-	-	-	-	-
VIDAS	Whole-cell	0.53	-	-	-	-	-	-
Virion/Serion	Whole-cell	0.67	0.69	-	-	-	-	-
Enzygnost	Whole-cell + VlsE	0.71	0.62	0.78	-	-	-	-
Euroimmun	Whole-cell + VlsE	0.71	0.45	0.56	0.56	-	-	-
Virotech	Whole-cell + VlsE	0.44	0.65	0.57	0.43	0.47	-	-
Immunetics	Recombinant	0.74	0.60	0.64	0.86	0.53	0.41	-
Mikrogen	Recombinant	0.79	0.53	0.63	0.68	0.67	0.44	0.65

We tested a subset of 31 serum samples from patients suspected of Borrelia infection in all five immunoblots. Samples were from patients with positive and negative ELISA results, allowing us to investigate the specificity of the immunoblots. In general, we observed a much lower agreement for the immunoblots than for the ELISAs. Kappa values ranged from 0 (poor agreement) to 0.84 (good agreement), indicating that, for many samples, the outcome of the immunoblot is highly dependent on the choice of manufacturer (Table 3). Inter-blot agreement was disappointingly low for IgM and much higher for IgG (Table 3). Interestingly, recombinant blots did not have a higher agreement than whole-cell blots, and there was limited agreement even between recombinant blots. The highest agreement was for the home-made whole-cell blot with the Mikrogen recombinant blot. Additional analysis on the individual band level revealed similarly poor agreement, even in immunoblots containing recombinant antigens.

Table 3

Agreement between immunoblots for detecting anti-Borrelia antibodies (kappa values)

Blot	Blot type	Home-made	Virablot	Euroimmun	Mikrogen	Virotech
IgM and IgG combined
Home-made	Whole-cell	-	-	-	-	-
Virablot	Whole-cell + VlsE	0.55	-	-	-	-
Euroimmun	Recombinant	0.45	0.24	-	-	-
Mikrogen	Recombinant	0.74	0.42	0.29	-	-
Virotech	Recombinant	0.66	0.60	0.25	0.55	-
IgM
Home-made	Whole-cell	-	-	-	-	-
Virablot	Whole-cell + VlsE	−1.57	-	-	-	-
Euroimmun	Recombinant	0.04	0.20	-	-	-
Mikrogen	Recombinant	0.42	0	0.26	-	-
Virotech	Recombinant	0.20	0.46	0.39	0.34	-
IgG
Home-made	Whole-cell	-	-	-	-	-
Virablot	Whole-cell + VlsE	0.43	-	-	-	-
Euroimmun	Recombinant	0.43	0.24	-	-	-
Mikrogen	Recombinant	0.84	0.27	0.43	-	-
Virotech	Recombinant	0.71	0.63	0.30	0.56	-

When performing eight different ELISAs and five different blots, there are 40 possible ELISA–blot combinations. Thirty-one samples were tested in all 40 combinations. A score of 0 indicates a negative result in all ELISAs and all blots, while a score of 40 indicates a positive result in all ELISAs and all blots. A score between 0 and 40 indicates that not all possible combinations yielded a positive result (i.e. disagreement between various ELISA–blot combinations). Of this small sample cohort, 20/31 (65%) had either a score of 0 or 40, indicating perfect agreement, irrespective of the ELISA–blot combination used. Discordant interpretations were generated in the other 35% of samples.

The influence of assay choice is further illustrated by investigation of the relationship between each ELISA and the fraction of positive blots. Surprisingly, we found anti-Borrelia immunoblot reactivity in samples that were negative in all eight ELISAs. These are samples that normally would not have been tested in immunoblots. Again, this was not dependent on the nature of the antigen used for the immunoblot. For the Euroimmun immunoblot, 4/11 (36%) of the ELISA-negative samples were blot-positive. Some immunoblots also seem to lack sensitivity, since samples that were positive in six to all eight of the tested ELISAs remained negative in all immunoblots. Some of these samples were from Lyme disease patients with a short duration of symptoms, confirming that ELISAs may have a higher sensitivity than immunoblots during the early phase of a Borrelia infection.

For some ELISA–blot combinations, only about half of the ELISA-positive samples could be confirmed by immunoblot (e.g. VIDAS ELISA–Virotech immunoblot, Table 4). The quality of the other ELISAs was so high that the majority of ELISA-positive samples were confirmed with immunoblots (e.g. Diacheck/Moran and Enzygnost ELISAs). When taking into account the lack of specificity of a number of the immunoblots, it is clear that the combination of a non-specific ELISA with a non-specific blot will lead to a high fraction of presumably false-positive test results.

Table 4

Fractions of blot-confirmed samples for 40 ELISA–immunoblot combinations

ELISA manufacturer	Antigen used for ELISA	Number of positive samples in ELISA/total number of samples	Blot
			Whole-cell	Whole-cell + VlsE	Recombinant
			Home-made	Virablot	Euroimmun	Mikrogen	Virotech
Diacheck/Moran	Whole-cell	12/31	11/12 (92%)	9/12 (75%)	11/12 (92%)	12/12 (100%)	9/12 (75%)
VIDAS	Whole-cell	19/31	11/19 (58%)	12/19 (63%)	13/19 (68%)	14/19 (74%)	10/19 (53%)
Virion/Serion	Whole-cell	15/31	11/15 (73%)	11/15 (73%)	13/15 (87%)	12/15 (80%)	9/15 (60%)
Enzygnost	Whole-cell + VlsE	12/31	11/12 (92%)	10/12 (83%)	10/12 (83%)	12/12 (100%)	10/12 (83%)
Euroimmun	Whole-cell + VlsE	14/31	11/14 (79%)	11/14 (79%)	12/14 (86%)	12/14 (86%)	9/14 (64%)
Virotech	Whole-cell + VlsE	17/31	11/17 (65%)	11/17 (65%)	13/17 (77%)	13/17 (77%)	9/17 (53%)
Immunetics	Recombinant	13/31	11/13 (85%)	10/13 (77%)	10/13 (77%)	13/13 (100%)	10/13 (77%)
Mikrogen	Recombinant	13/31	11/13 (85%)	9/13 (69%)	11/13 (85%)	12/13 (92%)	9/13 (69%)

The ELISA test value is the final factor influencing the fraction of positive confirmatory blots. Figure 1 depicts an example—values for the VIDAS and Immunetics C6 Lyme ELISA according to the immunoblot results of a whole-cell blot (home-made) and a recombinant blot (Mikrogen). For the VIDAS–home-made blot combination, it is difficult to indicate a cut-off value for the VIDAS ELISA with a good separation between blot-positives and blot-negatives. When using the Immunetics ELISA as a screening tool, it becomes clear that, irrespective of the blot method used, samples with an index >4 are almost always blot-positive. These characteristics make it possible to define groups of ELISA-positive serum samples that do not need immunoblot confirmation.

Fig. 1

Enzyme-linked immunosorbent assay (ELISA) test values in relation to immunoblot results for the detection of anti-Borrelia antibodies

Discussion

We studied the influence of the choice of detection method on the results of Borrelia serology. We found that Borrelia ELISAs and immunoblots for detecting anti-Borrelia antibodies have widely divergent sensitivity and specificity, and that immunoblots generally show limited agreement. Analysis of a large number of ELISA–immunoblot combinations revealed large differences between various test strategies in a two-tier testing algorithm. Although we only studied a limited number of serum samples, our extensive approach allowed us to draw several conclusion based on our observations.

Theoretically, the use of recombinant antigens should lead to increased specificity and, possibly, increased sensitivity as well. This does not seem to be true for the currently available ELISAs and immunoblots for the detection of anti-Borrelia antibodies. We could not find a clear relationship between the fraction of positive tests, the specificity and the nature of the antigen used for the serological tests. ELISAs using sonicated whole-cell antigens can be sensitive and specific, while recombinant ELISAs may lack specificity. Therefore, manufacturer claims for the superior performance of assays using recombinant antigens for the detection of Borrelia antibodies must be interpreted with caution.

A two-tier testing algorithm for the detection of anti-Borrelia antibodies is recommended world-wide [2, 3, 6]. However, there are several reasons to reappraise the additional value of an immunoblot confirmatory test in a two-tier testing scheme.

First, the lack of specificity of some immunoblots is counter-intuitive. The immunoblot is used as a confirmatory test, although it can be argued that it is merely a supplemental test due to the inter-dependence of ELISAs and immunoblots [8]. Theoretically, the use of recombinant antigens should allow discrimination between a specific antibody reactivity, cross-reactive antibodies and true anti-Borrelia antibodies [4]. The presence of commercially available immunoblots with low specificity diminishes the value of the immunoblot as a confirmatory test [8]. Furthermore, the two-tier testing scheme was originally proposed to overcome the lack of specificity of Borrelia ELISAs. This study has shown that not all of the newer generation ELISAs using recombinant Borrelia antigens have improved specificity compared to older serological assays [9, 10].

Second, the low level of agreement between the different immunoblots is very disappointing, especially for IgM. This low level of agreement, even at the individual band level, makes it hard to compare immunoblot results from different manufacturers.

Third, a mismatch between immunoblot and ELISA may occur during the early phase of infection. There are numerous examples—from this and other studies—in which patients with early Lyme disease were initially ELISA-positive and blot-negative [11]. In such cases, immunoblot seroconversion can only be documented in a follow-up sample, and, sometimes, even this option is blocked because antibiotic treatment may interfere with the development of the anti-Borrelia antibody response [12]. This is an example of better sensitivity in the ELISAs compared to the immunoblots. Without detailed knowledge of the clinical manifestations and illness duration, reporting these cases as ‘negative’ could lead to erroneous conclusions.

Finally, several groups can be discriminated based on the ELISA value [10]: a ‘high positive’ group exhibiting clinical symptoms consistent with a diagnosis of Lyme disease and which can be reported as ‘positive’ without confirmatory testing, a ‘low positive’ group in which confirmatory testing may be helpful and, lastly, a negative group that does not require any further investigation. We do not advocate abandoning the use of immunoblots to confirm anti-Borrelia antibodies, but we do think that only a selection of samples needs confirmatory blotting. Furthermore, knowledge about the lower sensitivity of immunoblots compared to some of the ELISAs is indispensable in interpreting results.

In conclusion, ELISAs and immunoblots for detecting anti-Borrelia antibodies have widely divergent sensitivity and specificity, and immunoblots for detecting anti-Borrelia antibodies have only limited agreement. Therefore, the choice of ELISA–immunoblot combination severely influences the number of positive results, making the exchange of test results between laboratories with different methodologies hazardous. The widespread availability of more specific and sensitive assays for the detection of anti-Borrelia antibodies will open the way for a reappraisal of the two-tier testing system.