Accuracy of a commercial IgM ELISA for the diagnosis of human leptospirosis in Thailand.

The Leptospira immunoglobulin M enzyme-linked immunosorbent assay (IgM ELISA) has been recommended for the rapid diagnosis of leptospirosis in endemic areas. We conducted a retrospective case-control study of 218 patients (109 leptospirosis cases confirmed by Leptospira culture and/or microscopic agglutination test and 109 control patients with acute febrile illness) to evaluate the diagnostic accuracy of a commercial IgM ELISA (Panbio) in northeast Thailand. Paired serum samples taken on admission and at least 10 days after the onset of symptoms were tested. Using the cutoff value recommended by the manufacturer (11 Panbio units), sensitivity and specificity of IgM ELISA on paired sera were 90.8% and 55.1%. A receiver operating characteristic curve was used to determine the optimal cutoff value. This was 20 Panbio units, which gave a sensitivity and specificity of 76.1% and 82.6%, respectively, on paired sera. We conclude that using either cutoff value, the accuracy of IgM ELISA is limited in our setting.

Introduction

Leptospirosis is considered an emerging infectious disease in Thailand.1–3 Regular contact with the environment is a major risk factor, with a study in Thailand reporting that leptospirosis accounted for around one-third of undifferentiated fevers in agricultural workers.3 Such infection usually results from contact with water or soil contaminated with pathogenic Leptospira originating from the urine of maintenance (carrier) hosts such as rodents. The clinical features of leptospirosis are often nonspecific and severity ranges from a mild flu-like illness to severe sepsis and septic shock. Signs and symptoms of leptospirosis may resemble those of other diseases in the tropics including dengue, scrub typhus, and malaria, resulting in an inaccurate clinical diagnosis.4 Diagnostic confirmation is not attempted across much of the world, however, leading to sub-optimal patient management and a poor understanding of disease epidemiology.

In Thailand, the definitive diagnosis of leptospirosis is made by culture of Leptospira spp. from clinical samples such as blood or urine, or by the reference serological assay, the microscopic agglutination test (MAT). These tests require specialist expertise, are labor-intensive, and provide a retrospective diagnosis.5 Several rapid serological diagnostic tests have been developed as alternatives to MAT, of which a commercial immunoglobulin M enzyme-linked immunosorbent assay (IgM ELISA) is promising because it can be performed in a greater number of laboratories throughout the tropics, and is inexpensive compared with MAT. The IgM ELISA has been recommended by the World Health Organization (WHO) as a diagnostic test for the serodiagnosis of leptospirosis where healthcare resources are limited,4 although its reported accuracy is variable. A number of studies have reported that IgM ELISA has high sensitivity and specificity for the diagnosis of acute leptospirosis.6–8 However, recent studies evaluating this test in Laos, Vietnam, and Hawaii found that its diagnostic accuracy was very low (sensitivity ranged from 36.0% to 60.9% and specificity ranged from 65.1% to 98.0%).9–11 The reason for poor performance in the latter studies may be that in countries where leptospirosis is highly endemic, a variable proportion of the apparently healthy population may be seropositive. This would result in variability in the optimal cutoff value in different geographic regions. In this study, we tested the hypothesis that the optimal cutoff for this assay in Thailand is higher than that stated by the manufacturer, and determined the optimal cutoff value and diagnostic accuracy of the IgM ELISA in this setting.

Materials and Methods

The Standards for the Reporting of Diagnostic accuracy testing (STARD) were followed in this study.

Study patients

Patients with laboratory-confirmed leptospirosis (cases) or without leptospirosis (controls) were identified from a prospective cohort study of consecutive patients presenting to Udon Thani hospital, northeast Thailand with an acute febrile illness between 2001 and 2002.12 In brief, patients were recruited into the study during twice daily ward rounds. Inclusion criteria were patients who were ≥ 15 years of age with fever (> 37.8°C) of unknown cause who agreed to participate and to attend outpatient follow-up for a convalescent serum sample. Exclusion criteria were patients with a definable source of infection on admission. Blood samples for Leptospira culture and serological testing were taken on admission, and a second serum sample taken 10 or more days after the onset of symptoms. Leptospira culture and MAT were performed as described previously.5,12 Serum was stored at −80°C until analysis for IgM ELISA and MAT.

Cases of leptospirosis were defined by isolation of pathogenic Leptospira spp., and/or a 4-fold or greater rise or a single serum MAT titer of ≥ 1:400. Controls (ratio 1:1) were randomly selected from those patients who had convalescent sample and did not meet the diagnostic criteria for leptospirosis. Overall, 109 cases and 109 controls were recruited in our study. A database was constructed in which cases and controls were entered, randomized, and blinded to the investigators before performing IgM ELISA.

Leptospira IgM ELISA

Detection of IgM antibodies to Leptospira species was determined using a commercially available Leptospira IgM ELISA (Panbio Pty., Ltd., Queensland, Australia). The assay was performed according to the manufacturer’s instructions. Briefly, test sera, cutoff calibrator, and positive and negative control sera were diluted 1:100 in serum diluent, and 100 μL added to Leptospira antigen-coated microwells and incubated for 30 minutes at 37°C. After washing with phosphate-buffered saline containing 0.05% Tween 20, 100 μL of HRP conjugated anti-human IgM was added and incubated for another 30 minutes at 37°C. After further washing, 100 μL of tetramethylbenzidine substrate was added and incubated at room temperature for 10 minutes, after which the reaction was stopped with 100 μL of 1 M phosphoric acid. The absorbance of each well was read at a wavelength of 450 nm with a Bio-Tek ELX 808 plate reader (Bio-Tek Instruments, Winooski, VT). The results were expressed as Panbio units calculated by the ratio of sample absorbance to the mean cutoff absorbance multiplied by 10. The recommended cutoff for a positive result is a value of ≥ 11 Panbio units, and is interpreted by the manufacturer to indicate recent infection of leptospirosis.

Evaluation of IgM ELISA in healthy individuals

Serum samples were obtained from 218 healthy blood donors residing in northeast Thailand and tested using the IgM ELISA to evaluate background seropositivity. To determine the reference range for this population, IgM Panbio units were log transformed so that data was normally distributed, and the reference range then calculated using the geometric mean and range values (geometric mean ± 2 SD).

Statistical analysis

Statistical analyses were performed using STATA/SE version 11.2 (College Station, TX). Diagnostic sensitivity and specificity of the IgM ELISA was defined against the combined result for culture and MAT (a positive result of either or both being interpreted as diagnostic for leptospirosis), and expressed as a proportion with exact 95% confidence interval (CI). The Fisher’s exact test was used to compare categorical variables, and the Mann-Whitney test was used to compare continuous variables. The optimal cutoff value of IgM ELISA was determined using receiver operating characteristic (ROC) curves.13 Median time to positivity of IgM ELISA in leptospirosis cases was estimated by survival analysis. This analysis was censored for loss to follow-up.

Ethical statement

The study protocol was approved by the Ethics Committee of the Ministry of Public Health, Royal Government of Thailand. Each patient was recruited to the study after written informed consent was obtained.

Results

A case-control study consisting of 109 leptospirosis cases and 109 controls was used to evaluate the accuracy of the IgM ELISA. Of 109 leptospirosis cases, laboratory confirmation was made on the basis of being positive by culture and MAT in 19 patients (17%), culture positive/MAT negative in 15 patients (14%), and culture negative/MAT positive in 75 patients (69%). The proportion of study patients who were male was 82% and 57% for cases and controls, respectively (P < 0.001). Three cases (2%) and four controls (4%) died during hospital admission (P > 0.1). All patients in the control group had a convalescent serum sample taken a median of 17 days (interquartile range [IQR] 13–21 days, range 10–43 days) after the onset of symptoms. The discharge diagnoses of controls were as follows: scrub typhus (N = 51), bacterial septicemia (N = 8) (Escherichia coli [N = 2], Klebsiella pneumoniae [N = 2], Acinetobacter baumannii [N = 1], Corynebacterium jeikeium [N = 1], Enterococcus spp. [N = 1], and Streptococcus pneumoniae [N = 1]), dengue fever (N = 4), murine typhus (N = 4), melioidosis (N = 2), HIV-related infections (N = 2), other diagnoses (N = 6), and unknown diagnosis (N = 32).

The sensitivity and specificity of IgM ELISA were determined using culture and/or MAT positivity as the gold standard. The diagnostic accuracy of the ELISA based on testing paired (acute and convalescent) sera and using a positive result on admission and/or during follow-up was determined using the manufacturer’s cutoff of > 11 Panbio units. The sensitivity and specificity of paired serum samples were 90.8% (95% CI = 83.8–95.5) and 55.1% (95% CI = 45.2–64.6), respectively (Table 1). Recalculation of these using results from the admission sample only gave a sensitivity and specificity of 52.3% (95% CI = 42.5–62.1) and 66.4% (95% CI = 56.6–75.2), respectively.

Table 1

Sensitivities and specificities of Panbio IgM ELISA for the diagnosis of leptospirosis*

Cutoff value	Sample timing	Positive (+) or negative (−) result	Gold standard result†		% (95% CI)
			+	−	Sensitivity	Specificity
≥ 11 Panbio units	On admission‡	+	56	36	52.3 (42.5–62.1)	66.4 (56.6–75.2)
		−	51	71
	Paired sera	+	99	49	90.8 (83.8–95.5)	55.1 (45.2–64.6)
		−	10	60
≥ 20 Panbio units	On admission‡	+	33	10	30.8 (22.2–40.5)	90.7 (83.5–95.4)
		−	74	97
	Paired sera	+	83	19	76.1 (67.0–83.8)	82.6 (74.1–89.2)
		−	26	90

IgM ELISA = immunoglobulin M enzyme-linked immunosorbent assay; CI = confidence interval.

Positive gold standard result was isolation of pathogenic Leptospira spp., and/or a 4-fold or greater rise or a single serum MAT titer of ≥ 1:400.

Sera on admission of two leptospirosis cases and two controls were not available to test by the IgM ELISA.

A ROC curve analysis was performed using results based on paired sera, which demonstrated an area under the curve of 0.85 (95% CI = 0.79–0.89). From this, an optimal cutoff was estimated to be ≥ 20 Panbio units. This gave an accurate classification in 79.4% of patients compared with 72.5% if the low cutoff value was used (Figure 1). Using this higher cutoff value, the sensitivity and specificity of the ELISA on paired sera were 76.1% (95% CI = 67.0–83.8) and 82.6% (95% CI = 74.1–89.2), respectively, and those on admission were 30.8% (95% CI = 22.2–40.5) and 90.7% (95% CI = 83.5–95.4), respectively. Median duration between onset of symptoms and primary specimen collection were 7 days (IQR 4–10 days) and 10 days (IQR 7–16 days) for cases and controls, respectively. Using survival analysis, the median time from onset of symptoms to positivity of IgM ELISA in leptospirosis cases was estimated to be 7 and 14 days for the low cutoff value (≥ 11 Panbio units) and high cutoff value (> 20 Panbio units), respectively.

Figure 1.

Receiver-operator characteristic curve of the Leptospira immunoglobulin M enzyme-linked immunosorbent assay (IgM ELISA) versus gold standard test (a combination of culture and MAT) on paired sera.

The optimal cutoff value defined here was almost twice the cutoff value recommended by the manufacturer, suggesting high background seropositivity. This was evaluated by testing 218 serum samples from healthy individuals residing in northeast Thailand. The ELISA results for these samples demonstrated a geometric mean value of 5.7 Panbio units, with a normal range (geometric mean ± 2 SD) of 1.7 to 19.3 Panbio units. Of 218 healthy individuals, 28 (12.8%) had an IgM ELISA level equal to or more than 11 Panbio units, and 6 (2.8%) had an IgM ELISA level equal to or more than 20 Panbio units.

Discussion

Reassessment of diagnostic sensitivity and specificity of the IgM ELISA showed that using either the low or high cutoff value, the accuracy of this test is poor in Thailand. Using the cutoff value recommended by the manufacturer (11 Panbio units), the sensitivity on paired sera is high (90.8%) and may be used to rule in patients with suspected leptospirosis. However, the poor specificity of the test (55.1%) suggests that using this test in the clinical setting could lead to over diagnosis of leptospirosis and a high frequency of false positivity. Using the high cutoff value generated by ROC curve analysis (20 Panbio units), the overall accuracy of the test was improved. However, both sensitivity and specificity on paired sera were 76.1% and 82.6%, respectively, indicating that the utility of the test using the higher cutoff value is also limited. In addition, using the test on admission is neither sensitive nor specific enough for the diagnosis of leptospirosis.

Our finding that IgM ELISA had a poor diagnostic accuracy is consistent with previous reports from Laos, Vietnam, and Hawaii.9–11 Plausible explanations for poor sensitivity include short duration of follow-up and inadequacy of an IgM response in some leptospirosis patients. Taking a second convalescent sample or sampling after a longer duration of follow-up may be required. IgM may also fail to be elevated in patients presenting with a second or subsequent episode of infection, when the antibody response would be predicted to be largely IgG. Poor specificity of the IgM ELISA is also likely to be caused by a high background seropositivity rate in the general population in our setting. Positivity of IgM ELISA could also represent past exposure of leptospirosis. This is strongly supported by the high seropositivity rate in healthy individuals if the low cutoff value is chosen.

We conclude that the Leptospira IgM ELISA is not sufficiently accurate in Thailand. The recommendation by WHO to use such an assay for the diagnosis of leptospirosis if financial resources permit requires country-specific evaluation to determine its clinical use before implementation.