Evaluation of antigen-detecting and antibody-detecting diagnostic test combinations for diagnosing melioidosis.

BACKGROUND: Melioidosis, an infectious disease caused by Burkholderia pseudomallei, is endemic in many tropical developing countries and has a high mortality. Here we evaluated combinations of a lateral flow immunoassay (LFI) detecting B. pseudomallei capsular polysaccharide (CPS) and enzyme-linked immunosorbent assays (ELISA) detecting antibodies against hemolysin co-regulated protein (Hcp1) or O-polysaccharide (OPS) for diagnosing melioidosis. METHODOLOGY/PRINCIPAL
FINDINGS: We conducted a cohort-based case-control study. Both cases and controls were derived from a prospective observational study of patients presenting with community-acquired infections and sepsis in northeast Thailand (Ubon-sepsis). Cases included 192 patients with a clinical specimen culture positive for B. pseudomallei. Controls included 502 patients who were blood culture positive for Staphylococcus aureus, Escherichia coli or Klebsiella pneumoniae or were polymerase chain reaction assay positive for malaria or dengue. Serum samples collected within 24 hours of admission were stored and tested using a CPS-LFI, Hcp1-ELISA and OPS-ELISA. When assessing diagnostic tests in combination, results were considered positive if either test was positive. We selected ELISA cut-offs corresponding to a specificity of 95%. Using a positive cut-off OD of 2.912 for Hcp1-ELISA, the combination of the CPS-LFI and Hcp1-ELISA had a sensitivity of 67.7% (130/192 case patients) and a specificity of 95.0% (477/502 control patients). The sensitivity of the combination (67.7%) was higher than that of the CPS-LFI alone (31.3%, p<0.001) and that of Hcp1-ELISA alone (53.6%, p<0.001). A similar phenomenon was also observed for the combination of CPS-LFI and OPS-ELISA. In case patients, positivity of the CPS-LFI was associated with a short duration of symptoms, high modified Sequential (sepsis-related) Organ Failure Assessment (SOFA) score, bacteraemia and mortality outcome, while positivity of Hcp1-ELISA was associated with a longer duration of symptoms, low modified SOFA score, non-bacteraemia and survival outcome.
CONCLUSIONS/SIGNIFICANCE: A combination of antigen-antibody diagnostic tests increased the sensitivity of melioidosis diagnosis over individual tests while preserving high specificity. Point-of-care tests for melioidosis based on the use of combination assays should be further developed and evaluated.

Introduction

Melioidosis, an infectious disease caused by the Gram-negative bacterium Burkholderia pseudomallei, is endemic to and has high mortality in tropical developing countries [1]. The disease is estimated to affect 165,000 people and account for 89,000 deaths per year worldwide [2]. Naturally acquired infections result from exposure through skin inoculation, inhalation or ingestion of B. pseudomallei, which is commonly present in soils and surface water in tropical countries [1,2]. The disease is difficult to diagnose and treat. Patients commonly present with sepsis, which is a syndrome defined by a dysregulated host response to infection resulting in significant organ dysfunction and death. Sepsis can be caused by any variety of agents, including bacteria, fungi, and viruses [3]. A recent study in Thailand showed that there were 7,126 culture-confirmed melioidosis patients diagnosed from 2012 to 2015 in 70 hospitals countrywide, and that 39% of them died [4].

Culture remains the mainstay and gold standard for melioidosis diagnosis. Culture positivity for B. pseudomallei from any clinical sample is a definitive diagnosis for melioidosis since the organism is never part of the normal human flora (i.e. 100% specificity). Unfortunately, culture takes from 2 to 7 days and has a sensitivity of only about 60% based on a model estimate [5], and requires both experienced microbiologists and strict laboratory safety procedures [6]. Serological tests using crude antigen preparations, such as the indirect hemagglutination assay (IHA), are neither sensitive nor specific, and have no role in the diagnosis of melioidosis in melioidosis-endemic regions [7]. Specificity of the IHA ranges from 68% to 72% in Thailand using a cut-off of 1:160 according to the Thai standard [8,9] and from 75% to 91% in Australia and Papua New Guinea using a cutoff of 1:40 according to the Australian standard [10,11].

Misuse of diagnostic tests with low specificity, such as the IHA can lead to misdiagnoses, and thus, a lack of public health responses against melioidosis in Thailand [12]. Melioidosis has been a notifiable disease in Thailand since 2001, and a positive IHA result has been one of the criteria used to diagnose and report melioidosis cases in the country. This has led to a high number of IHA false-positive melioidosis cases with close to zero mortality being reported to the national notifiable disease surveillance system [12]. The incorrect mortality (ranging from 0.1 to 0.5%) has led to a false sense of security among people and policy makers, and limited prioritization by Ministry of Public Health [12]. In addition, misdiagnosis of melioidosis by using IHA results alone may lead to overuse of antibiotics that are effective against B. pseudomallei and place patients at risk of avoidable adverse drug reactions.

An increasing number of non-culture-based diagnostic tests for melioidosis are being developed and evaluated. Numerous PCR assays have been developed, but none are routinely used for clinical diagnosis in endemic areas because they are not cost-effective and their sensitivity is limited [1,7]. Recently, a lateral flow immunoassay (LFI) that detects the B. pseudomallei capsular polysaccharide (CPS) has been developed [13]. The test seems to have high specificity but limited sensitivity, particularly for blood samples [14-18]. It is possible that using a CPS-LFI in both serum and all non-blood specimens collected systematically from melioidosis suspected patients could increase the sensitivity of the combination further. Additionally, enzyme-linked immunosorbent assays (ELISA) that detect specific IgG antibodies against hemolysin co-regulated protein (Hcp1) or Type A O-polysaccharide (OPS) have been developed. An evaluation in northeast Thailand found that both Hcp1 and OPS-ELISA exhibited sensitivities and specificities ranging from 72% to 83% and 95% to 100%, respectively [19]. Assessment of a rapid immunochromatography test (ICT) that detects specific antibodies against Hcp1 has also reported a sensitivity of 88.3% in melioidosis patients in northeast Thailand and a specificity of 86.1% in Thai healthy donors [20].

We previously demonstrated that the CPS-LFI had limited sensitivity (31.3%) but high specificity (98.8%) for diagnosing melioidosis among patients presenting with community-acquired infections or sepsis in northeast Thailand when stored serum were tested [15]. Here, we evaluated combinations of the CPS-LFI and Hcp1- or OPS-ELISA for diagnosis of melioidosis. We hypothesized that a combination of antigen-antibody diagnostic tests may increase the sensitivity of melioidosis diagnosis over individual tests while preserving high specificity. The CPS-LFI was selected as an antigen-detecting diagnostic test under evaluation because it is in a point-of-care test that can be used to diagnose melioidosis in 15 minutes, has high specificity, and appears promising for resource-limited melioidosis-endemic settings [14-18]. Hcp1 and OPS-ELISA was selected as an antibody-detecting diagnostic test under evaluation because of its promising sensitivity and specificity [19].

Material and methods

Ethics statement

This study was conducted in full compliance with the principles of good clinical practice (GCP), and the ethical principles of the Declaration of Helsinki. The study protocol and related documents were approved by Sunpasitthiprasong Hospital Ethics Committee (039/2556), the Ethics Committee of the Faculty of Tropical Medicine, Mahidol University (MUTM2012-024-01), the University of Washington Institutional Review Board (42988) and the Oxford Tropical Research Ethics Committee at the University of Oxford (OXTREC172-12). Written, informed consent was obtained from participants prior to enrollment.

Study population

We conducted a prospective observational (non-interventional) study of community-acquired infection and sepsis in Sunpasitthiprasong Hospital, Ubon Ratchathani province, northeast Thailand. From March 2013 to February 2017, we enrolled 5,001 adult patients (≥18 yr of age) who were admitted with a primary diagnosis of suspected or documented infections as determined by the attending physician, were within 24 h of hospital admission and had at least three sepsis diagnostic criteria documented in their medical record [21]. We excluded patients who were suspected of having hospital-acquired infections determined by the attending physician, had a hospital stay within 30 days prior to this admission or were transferred from another hospital with a total duration of hospitalization >72 hours. Organ dysfunction was determined by a modified Sequential (sepsis-based) Organ Failure Assessment (SOFA) score on admission as previously described [21]. 28-day mortality data were collected via telephone contact if subjects were no longer hospitalized and had been discharged alive [21]. Blood was drawn from all patients at the time of enrolment for culture and polymerase chain reaction (PCR) and serum samples were frozen at -80°C.

Patients who were culture positive for B. pseudomallei from any clinical specimens were selected as cases. Patients with blood cultures positive for Staphylococcus aureus, Escherichia coli or Klebsiella pneumoniae, or those patients testing positive for malaria or dengue via PCR assays, were selected as controls. Dengue and malaria were diagnosed by a nested PCR assay as described previously [22,23].

CPS-LFI

The Active Melioidosis Detect LFI used in this study was developed by InBios (Seattle, WA, USA; lot no. WJ1222) as a research use only device [13]. Results from a study using this CPS-LFI alone were previously published [15], and used in this study. The result of this test was dichotomous; all weakly positive results were considered as positive because most weakly positive results were from culture-confirmed melioidosis cases [15].

Hcp1-ELISA and OPS-ELISA

ELISA using Hcp1 or OPS antigens were performed essentially as previously described [19, 24]. Briefly, B. pseudomallei LPS Type A was extracted from the select agent excluded strain RR2808 (capsule mutant) using a modified hot aqueous-phenol method and purified O-polysaccharide (OPS) was then obtained via acid hydrolysis and gel permeation chromatography as previously described [25]. Hcp1 was obtained using recombinant DNA techniques and purified as previously described [19]. For ELISA procedures, the microtiter plates were prepared by using optimized antigen concentrations of 2.5 μg/mL for Hcp1 and 1 μg/mL for OPS (50 μl per well for coating) and plates were incubated at 4°C for overnight. The plates were then washed 4 times with wash buffer using a Hydrospeed microplate washer followed by blocking with 5% skim milk in PBS at 37°C for 2 hours and were further washed as described above. Fifty microliters of serum diluted (1:2000) was added to wells and incubated at room temperature for 30 min. After washing, 50 μl of 1:2000 horseradish peroxidase-conjugated rabbit antihuman IgG (Dako, Denmark) was added and incubated at room temperature for 30 min. After last washing, 50 μl of TMB substrate solution (Invitrogen, USA) was added and incubated at room temperature for 15 min. The reaction was then stopped with 50 μl of 1N HCl and followed by the optical density (OD) measurement. All ELISAs were conducted in duplicate, and the absorbance values (optical density [OD]) were determined at a wavelength of 450 nm using a microtiter plate reader [19]. Pooled melioidosis sera (5 patients with culture-confirmed melioidosis) and pooled healthy sera (5 healthy Thai donors) were used as positive and negative sera controls for ELISAs. All ELISAs were conducted on specimens frozen at the time of patient enrollment [21], not on freeze-thaw specimens from any previous studies.

Statistical analysis

Data were summarized with medians and interquartile ranges (IQR) for continuous measures, and proportions for discrete measures. IQRs are presented in terms of 25th and 75th percentiles. Continuous variables and proportions were compared between groups using Kruskal Wallis tests and Chi-square tests, respectively. All data in box plots are presented as 25th and 75th percentile boundaries in the box with the median line within the box; the whiskers indicate the 10th and 90th percentiles. A receiver operating characteristic (ROC) curve was created to monitor the shifting of the positive cut-off value of true-positive (sensitivity) and false positive (1-specificity) rates. Areas under the ROC curves (AUROCC) were compared using a nonparametric method. The lowest OD cut-off values that gave a specificity of 95% were selected. The sensitivity of diagnostic tests was defined as the proportion of melioidosis case patients who had positive test results. The specificity of diagnostic tests was defined as the proportion of control patients who had negative test results. The McNemar exact test was used to compare the sensitivity and specificity between tests.

We also explored the performance of these tests among case and control patients with different durations of symptoms, modified SOFA score, blood culture results and mortality outcome. The p values for trends were generated using the non-parametric test for trend across ordered groups. The Spearman correlation coefficient was used to explore the association between the OD value of ELISA and continuous variables. All analyses were performed using Stata version 14 (Stata Corp LP, College Station, TX, USA) and Prism 8 Statistics (GraphPad Software Inc, La Jolla, CA).

Results

Study participants

From March 2013 to February 2017, 5,001 adult patients presenting with community-acquired infections or sepsis were enrolled and followed for 28 days. A total of 193 patients were culture positive for B. pseudomallei and thus included as cases. Another 544 patients who were blood culture positive for E. coli (n = 189), K. pneumoniae (n = 27) and S. aureus (n = 53), or PCR positive for malaria (n = 152) and dengue (n = 123) were included in this study as controls. Serum was not available for one culture-confirmed melioidosis case. Serum of 42 patients with other confirmed diagnoses were not previously tested for CPS-LFI and were excluded from the study. Therefore, a total of 192 culture-confirmed melioidosis cases and 502 controls were included in the analyses.

Accuracy of the CPS-LFI, Hcp1-ELISA and OPS-ELISA

As previously reported [15], the CPS-LFI had a sensitivity of 31.3% (60/192 case patients were positive [95%CI 24.8 to 38.3]) and a specificity of 98.6% (495/502 control patients were negative [95%CI 97.1–99.4]) (Table 1). The median OD value of Hcp1-ELISA for the case patients was higher compared to control patients (median OD 2.922 [IQR 0.864–3.454]) vs. 0.299 [IQR 0.116–0.831], p<0.001) (Fig 1). The median OD value of OPS-ELISA for the case patients was also higher compared to control patients (median OD 2.695 [IQR 0.847–3.451]) vs. 0.604 [IQR 0.251–1.348], p<0.001). The AUROCCs of Hcp1-ELISA and OPS-ELISA were not significantly different (0.80 vs. 0.78, p = 0.12) (Fig 2).

Table 1

Sensitivity and specificity of the CPS-LFI, Hcp1-ELISA, OPS-ELISA and the combinations of the CPS-LFI and Hcp1-ELISA or OPS-ELISA.

Assay	OD cut-off (of ELISA)	% Sensitivity (95% CI, N = 192)	% Specificity (95% CI, N = 502)
CPS-LFI	-	31.1 (24.8–38.3)	98.6 (97.1–99.4)
Hcp1-ELISA	2.758	53.6 (46.3–60.9)	95.0 (92.7–96.8)
OPS-ELISA	2.839	48.4 (41.2–55.7)	95.0 (92.7–96.8)
A combination of CPS-LFI and Hcp1-ELISA	2.912	67.7 (60.6–74.3)	95.0 (92.7–96.8)
A combination of CPS-LFI and OPS-ELISA	3.100	63.0 (55.8–69.9)	95.0 (92.7–96.8)

Fig 1

Results of Hcp1-ELISA and OPS-ELISA from culture-confirmed melioidosis cases and controls*.

* Controls were patients whose blood culture were positive for Staphylococcus aureus, Escherichia coli or Klebsiella pneumoniae or those who were malaria or dengue polymerase chain reaction assay positive. Box plots represent 25th and 75th percentile boundaries in the box with the median line within box; the whiskers indicate the 10th and 90th percentiles. The plots show optimal density (OD) 450 of each antigen from serum samples collected within 24 hours of hospital admission from culture-confirmed melioidosis cases and controls.

Fig 2

Receiver operating characteristic curves (ROC) plots of Hcp1-ELISA and OPS-ELISA*.

*Areas under the ROC curves (AUROCC) for Hcp1-ELISA and OPS-ELISA were calculated from the optimal density (OD) from serum samples collected within 24 hours of hospital admission from culture-confirmed melioidosis cases and controls.

Using a positive cut-off OD of 2.758 for Hcp1-ELISA to achieve a specificity of 95.0% (477/502 control patients were negative), Hcp1-ELISA had a sensitivity of 53.6% (103/192 case patients were positive; Tables 1 and S1). Using a positive cut-off OD of 2.839 for OPS-ELISA to achieve a specificity of 95.0% (477/502 control patients were negative), the OPS-ELISA had a sensitivity of 48.4% (93/192 case patients were positive).

Accuracy of a combination of the CPS-LFI and Hcp1-ELISA or OPS-ELISA

To achieve a specificity equal to or higher 95% for the combination of antigen-detection and antibody-detection diagnostic tests, we found that the positive cut-off OD selected for the ELISAs needed to be higher than using the ELISAs alone (Tables 1, S1 and S2). Using a positive cut-off OD of 2.912 for Hcp1-ELISA to achieve a specificity of 95.0% (477/502 control patients were negative), the combination of the CPS-LFI and Hcp1-ELISA had a sensitivity of 67.7% (130/192 case patients were positive). Of 130 case patients with a positive test result, 33 (25.4%) were positive with the CPS-LFI alone, 70 (53.9%) were positive with Hcp1-ELISA alone, and 27 (20.8%) were positive with both tests.

Sensitivity of the CPS-LFI and Hcp1-ELISA combination was higher than that of CPS-LFI alone (67.7% vs. 31.3%, p<0.001) and that of Hcp1-ELISA alone (67.7% vs. 53.6%, p<0.001). A similar phenomenon was also observed for the CPS-LFI and OPS-ELISA combination. Sensitivity of the CPS-LFI and Hcp1-ELISA combination was not significantly different than that of CPS-LFI and OPS-ELISA combination (67.7% vs. 63.0%, p = 0.16).

Sensitivity of diagnostic tests in melioidosis case patients by the duration of symptoms

Of 192 case patients, 44, 64, 44 and 40 reported having symptoms prior to admission for 1–2 days, 3–6 days, 7–13 days and ≥14 days, respectively. There was a trend showing that the sensitivity of the CPS-LFI decreased from 36.4% (16/44 case patients were positive) in patients with duration of symptoms for 1–2 days to 17.5% (7/40 case patients were positive, p = 0.004) in patients with symptoms for ≥14 days (Fig 3 and S3 Table). The sensitivity of Hcp1-ELISA correspondingly increased from 45.5% (20/44 case patients with symptoms for 1–2 days were positive) to 77.5% (31/40 case patients with symptoms for ≥14 days were positive, p<0.001). We did not observe a trend in the sensitivity of the CPS-LFI and Hcp1-ELISA combination by the duration of symptoms (p = 0.15); the sensitivity ranged from 65.9% (29/44 cases patients were positive) in patients with duration of symptoms for 1–2 days to 80.0% (32/40 cases patients were positive) in patients with symptoms for ≥14 days. A similar pattern was also observed for the OPS-ELISA (Fig 4).

Fig 3

Sensitivity of the combination of the CPS-LFI and Hcp1-ELISA, Hcp1-ELISA alone and the CPS-LFI alone.

(A) by duration of symptoms prior to admission and (B) by blood culture results in 192 melioidosis cases. * using OD cut-off value at a specificity of 95% (OD 2.912) ** using OD cut-off value at a specificity of 95% (OD 2.758) *** Blood culture positive for B. pseudomallei.

Fig 4

Sensitivity of the combination of the CPS-LFI and OPS-ELISA, OPS-ELISA alone and the CPS-LFI alone.

(A) by duration of symptoms prior to admission and (B) by blood culture results in 192 melioidosis cases. * using OD cut-off value at a specificity of 95% (OD 3.100) ** using OD cut-off value at a specificity of 95% (OD 2.839) *** Blood culture positive for B. pseudomallei.

Sensitivity of diagnostic tests in melioidosis case patients by modified SOFA score

Of 192 case patients, 42, 22, 41 and 87 had a modified SOFA score of 0–1, 2–3, 4–5 and ≥6, respectively. There was a trend showing that the sensitivity of the CPS-LFI increased from 7% (3/42 case patients were positive) in patients with a modified SOFA score of 0–1 to 49% (43/87 case patients were positive) in patients with a modified SOFA score of ≥6 (p<0.001) (S3 Table). The sensitivity of Hcp1-ELISA alone correspondingly decreased from 66.7% (28/42 case patients with modified SOFA score of 0–1 were positive) to 47.1% (41/87 case patients with modified SOFA score ≥6 were positive, p = 0.03). We did not observe a trend in the sensitivity of the CPS-LFI and Hcp1-ELISA combination by modified SOFA score (p = 0.62). A similar pattern was also observed for the OPS-ELISA.

Sensitivity of diagnostic tests in melioidosis case patients by bacteraemia status

Of 192 case patients, 150 (78.1%) were blood culture positive for B. pseudomallei. As previously reported [15], the sensitivity of the CPS-LFI was higher in patients with positive blood culture (38.0% [57/150]) than in those with negative blood culture (7.1% [3/42], p<0.001) (Fig 3). The sensitivity of Hcp1-ELISA alone was lower in patients with positive blood culture (49.3% [74/150]) than in those with negative blood culture (69.1% [29/42], p = 0.02). Overall, the sensitivity of the CPS-LFI and Hcp1-ELISA combination was not associated with blood culture results (p = 0.87), ranging from 68.0% [102/150] in patients with positive blood culture to 66.7% [28/42] in patients with negative blood cultures. A similar pattern was also observed for OPS-ELISA (Fig 4).

Association between diagnostic test results and 28-days mortality in melioidosis case patients

Of 192 melioidosis case patients, 99 (51.6%) died within 28 days of hospital admission. Patients with positive CPS-LFI were more likely to die than those with negative CPS-LFI results (72% [43/60] vs. 42% [56/132], p<0.001) (S3 Table). Patients with positive Hcp1-ELISA results were associated with lower mortality than those with negative Hcp1-ELISA results (43.7% [45/103] vs. 60.7% [54/89], p = 0.02). More specifically, the median OD value for Hcp1-ELISA in the 99 case patients who died was significantly lower than that of the 93 case patients who survived up to 28 days after hospital admission (2.448 [IQR 0.729–3.300] vs. 3.273 [IQR 1.342–3.494], p = 0.009).

Positive OPS-ELISA results were not significantly associated with lower mortality than those with negative OPS-ELISA results (47.3% [44/93] vs. 55.6% [55/99], p = 0.25). The median OD value for OPS-ELISA in 99 cases patients who died was not significantly lower than that of 93 case patients who survived up to 28 days after hospital admission (2.342 [IQR 0.801–3.331] vs. 2.966 [IQR 1.052–3.488], p = 0.18).

Specificity of diagnostic tests in different groups of control patients

Specificity of the CPS-LFI was not significantly different among groups of control patients (p = 0.86, S4 Table). However, specificity of Hcp1-ELISA was highest among patients with PCR positive results for dengue (99.2%; 122/123) and lowest among patients with positive blood cultures for S. aureus (90.0%; 18/20) or PCR positive results for malaria (90.1%; 136/151, p = 0.003). This pattern was also observed with OPS-ELISA (p = 0.04). Specifically, the median OD value of Hcp1-ELISA and OPS-ELISA were highest among patients who were PCR positive for malaria (p = 0.001 and p = 0.002, respectively, S5 Table), while the low specificity among patients who were blood culture positive for S. aureus were likely caused by two outliers with high OD values. Additionally, there was some evidence that the specificity of Hcp1-ELISA and OPS-ELISA decreased in control patients with higher modified SOFA score (p = 0.08 and p = 0.02, respectively, S4 Table). A very weak positive correlation between modified SOFA scores and the OD values of Hcp1-ELISA (rho 0.14, p = 0.002) or OPS-ELISA (rho 0.15, p<0.001) was also observed.

Of 502 control patients, 51 (10.2%) died within 28 days of hospital admission. An association between diagnostic test results and 28-day mortality of control patients was not observed (S4 Table).

Discussion

This study of patients hospitalized within 24 hours with community-acquired infection and sepsis at a referral hospital in northeast of Thailand demonstrates that a combination of the CPS-LFI and Hcp1-ELISA or CPS-LFI and OPS-ELISA increased the sensitivity of melioidosis diagnosis over any of the three tests alone while maintaining high specificity. This is an important advancement over current insensitive or protracted diagnostic strategies for disease with a nearly 40% case fatality rate in northeast of Thailand.

Although the CPS-LFI is in a format that can be readily used as a rapid diagnostic test, it is still in the development and evaluation phase and not yet commercially available. A rapid immunochromatography test (ICT) using Hcp1 as the target antigen has been developed and has a strong agreement with Hcp1-ELISA results [20]. The Hcp1-ICT is currently available for research use only from Mahidol University. The increased diagnostic accuracy of the combination of antigen and antibody detection tests indicates that these tests will be valuable in optimizing the care of melioidosis patients.

Additional findings are that in case patients, positivity of CPS-LFI is associated with patients with shorter duration of symptoms, higher modified SOFA score, bacteraemia and 28-day mortality outcome, while positivity of Hcp1-ELISA is associated with longer duration of symptoms, lower modified SOFA score, non-bacteraemia and 28-day survival outcome. These features of the tests may further support physicians in making triage decisions, resource allocation and antibiotic stewardship in resource-limited melioidosis-endemic regions.

The benefits of combining antigen and antibody-detecting diagnostic tests for melioidosis are consistent with combination diagnostic tests that are recommended for some infectious diseases such as dengue infection. Currently, point-of-care tests for dengue diagnosis include NS1 and IgM-based tests [26]. NS1 antigen can be detected in dengue patients within the first few days after the onset of illness, while IgM is still not detectable. By day 5 after the onset of illness, however IgM is detectable in 80% of dengue patients while NS1 may only be detectable in some patients [27]. This phenomenon is also observed in our study, in which the sensitivity of the antigen-detecting (CPS-LFI) test also declines in patients with longer duration of symptoms, while the sensitivity of the antibody-detecting (Hcp1- and OPS-ELISA) test increases.

Our defined cut-off values for Hcp1-ELISA and OPS-ELISA were higher than those used in the previous study [19]. This may be due to differences in study populations as well as timing of the specimen collections. Patient samples used in this study were obtained from a cohort of patients presenting with community-acquired infection and sepsis in northeast Thailand (Ubon-sepsis) whose blood was sampled within 24 hours of study hospital admission, while patient samples used in the previous study were obtained from multiple populations and different time points during illness (e.g. samples were obtained from case patients at a median of 5 days after hospital admission) [19]. Control patients in the previous study included Thai healthy donors, U.S. healthy donors, tuberculosis patients, scrub typhus patients and leptospirosis patients [19], and the OD values from those control patients were lower than what we observed from patients in this study who were blood culture positive for E. coli, K. pneumoniae or S. aureus, or PCR positive for malaria and dengue. Control patients enrolled in our study died within 28 days of hospital admission at a rate of 10%. Collecting blood specimens within 24 hours of admission can reduce the possibility of survival bias [28,29], in which only patients who survive up to a specific time point can be evaluated and enrolled into a study. For example, only patients who survive up to when results of bacterial culture are reported can be evaluated and enrolled into a study. The sensitivity of Hcp1-ELISA in this study (53.6%) was lower than that previously reported (83.0%) [19], which was due to the higher cut-off OD value required to give a specificity of 95.0% in the cohort setting.

The sensitivity of the CPS-LFI in this study using stored sera (31.3%) was higher than that recently reported from a study in India using whole blood (25.0% [2/8]) [17] and a study in Laos using stored sera (13.9% [5/36]) [18]. These differences could be caused by the use of different study populations, timing of specimen collection and generation of CPS-LFI test. It is also possible that melioidosis patients presenting in northeast Thailand have higher levels of CPS in their blood on admission and enrollment. If this is the case, these high CPS levels may be associated with higher mortality outcomes of melioidosis in northeast Thailand compared to those observed in India and Laos [17,18]. A previous study also found that the CPS-LFI appeared to perform better with blood that have been collected a few days prior to the collection of blood cultures that subsequently yield positive culture [14]. Further studies on factors associated with sensitivity and specificity of diagnostic tests for melioidosis are required.

This study has four strengths. First, cases and controls were drawn from a large prospective observational study of patients presenting with community-acquired infection and sepsis in northeast Thailand which represented a real-world setting. Second, all serum samples were drawn within 24 hours of admission to the study hospital which is ideal for evaluating point-of-care diagnostic tests. This approach would also avoid a survival bias [28,29]. Third, the prospective study collected blood and other relevant clinical specimens for bacterial culture from every patient enrolled systematically. This allows us to evaluate the accuracy of new diagnostic tests based on culture positivity for B. pseudomallei from both blood and non-blood specimens with low sample selection bias [21]. Fourth, the study evaluated modified SOFA score on admission and followed all patients for 28-day mortality outcome.

A limitation of this study is that positive predictive and negative predictive values could not be estimated because of the case-control study design. While preserving high specificity of the combination, the sensitivity of the combination was only 67.7%. This means that this combination may still miss a moderate proportion of melioidosis patients. The CPS-LFI can detect the B. pseudomallei CPS in a wide range of clinical specimens, including sera, sputum, urine, pus and sterile fluid [17,18]. However, we could not evaluate the accuracy of the CPS-LFI in non-blood specimens as they were not systematically stored during the prospective study. It is possible that using a CPS-LFI in both serum and all non-blood specimens collected systematically from melioidosis suspected patients could increase the sensitivity of the combination further. Our study did not evaluate LPS type of B. pseudomallei isolated from the case patients. B. pseudomallei LPS can be categorized into typical type A, atypical types B1 and B2, and rough type to represent the difference of OPS. The OPS-ELISA used in the previous and this study was prepared from B. pseudomallei LPS type A [19,25], and may have limited sensitivity among patients infected with B. pseudomallei with atypical or rough LPS type. A high proportion of patients enrolled into the Ubon-sepsis cohort (71%) were transferred from district hospitals, smaller hospitals in the province and hospitals in other provinces. The performance of diagnostic tests could vary in different settings, and, therefore, generalizability of these findings may be limited. The sera were tested for CPS-LFI in 2017 [15] and tested for ELISAs in 2020. We acknowledge that different durations of specimen storage could affect the performance of diagnostic tests. We also acknowledge that ELISAs under evaluation are not point-of-care tests as we need to define optimal cutoffs for ELISA when used in combination.

In conclusion, our results indicate that the combination of antigen and antibody-detection diagnostic tests significantly improved the sensitivity of melioidosis diagnosis, compared to when the tests were used individually, while also maintaining high specificity. Retaining high specificity while achieving good sensitivity of diagnostic tests in the real-world setting is essential. We propose that rapid diagnostic tests for melioidosis based on the combination of antigen-antibody detection should be further studied and developed as implementation of these tests is likely to benefit tropical developing countries where melioidosis is endemic.

Sensitivity and specificity of Hcp1-ELISA and OPS-ELISA using different OD cut-off values.

(DOCX)

Click here for additional data file.

Sensitivity and specificity of a combination of the CPS-LFI and Hcp1-ELISA and a combination of the CPS-LFI and OPS-ELISA using different OD cut-off values.

(DOCX)

Click here for additional data file.

Diagnostic test results in different groups of melioidosis case patients.

(DOCX)

Click here for additional data file.

Diagnostic test results in different groups of control patients.

(DOCX)

Click here for additional data file.

OD values of ELISA in different groups of control patients.

(DOCX)

Click here for additional data file.

4 Jul 2021

Dear Amornchai,

Thank you very much for submitting your manuscript "Evaluation of antigen-detecting and antibody-detecting diagnostic test combinations for diagnosing melioidosis" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Husain Poonawala

Associate Editor

PLOS Neglected Tropical Diseases

Anna Ralph

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: No changes needed

Reviewer #2: Some experiments lack appropriate controls

Some of the methodology is written very briefly and requires the reader to look for the referenced protocol. Not all journals are easily accessible especially for those in countries where melioidosis is endemic.

The sample size is adequate but I question the comparison that was done on the same samples but 3 years (?) apart with no statistical analysis between tests.

Reviewer #3: (No Response)

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: Yes

Reviewer #2: Yes but the authors should refer to comments within the attachment.

Reviewer #3: (No Response)

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: Yes

Reviewer #2: The limitations are identified but taken together and including other limitations not specifically mentioned in this submission makes it somewhat difficult to appreciate the novelty and applicability of the data in a public health setting, particularly in melioidosis endemic countries.

Reviewer #3: (No Response)

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: None needed

Reviewer #2: (No Response)

Reviewer #3: (No Response)

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: This paper uses a subset of a cohort of patients from a sepsis study in NE Thailand to evaluate a combination of antibody and antigen detection in serum for the diagnosis of melioidosis and demonstrates the superior sensitivity of this combined approach compared with individual tests, although this was still only 67.7% even when compared with the imperfect gold standard of culture. Whilst I agree that this is an important development, the fact that it will still miss many patients with melioidosis shopuld be given greater prominence.

It is implicit that the tests were only conducted on serum, although the CPS antigen detection test can also be used on other samples, such as pus, sputum and urine, and in such samples it may have higher sensitivity, but this is not made explicit until the discussion, so this should be made clear earlier in the manuscript.

There are a few minor typos that need correcting:

1. Line 93: change 'accounts' to 'account'.

2. Line 123: change 'placing' to 'place'.

3. Line 131: change 'detects' to 'detect'.

4. Line 432: change 'features' to 'has'.

Reviewer #2: Please refer to the attachment

Reviewer #3: Dear Editor,

This manuscript describes the use of 3 immunological based assays to diagnose melioidosis, a fatal tropical disease, in patients in Northeast Thailand. Their findings have suggested that the assays would be more effective if they were used in combinations: CPS-LFI and Hcp1-ELISA, or CPS-LFI and OPS-ELISA. These combinations improved the sensitivity, while the specificity of the combinations remained at 95% or more. However, the adjusted OD cut-off values of ELISA need to be used to achieve the acceptable sensitivity and specificity when the assays were used in a combination. This is a good finding since these 3 assays have been validated in multiple studies earlier. However, the manuscript is not well written, and the calculations may be inaccurate. Here are some comments:

Major comments:

1. It was not mentioned how the sensitivity or the specificity was calculated. Based on their findings, it is unlikely that the authors used the false positive value to calculate the sensitivity, and the false negative value to calculate the specificity. Please note that the sensitivity of an assay is the proportion of the diseased people correctly identified, while the specificity of same assay is the proportion of non-diseased people correctly identified.

% Sensitivity = {a / (a + b)} × 100

% Specificity = {d / (d + c)} × 100

a, true positive (assay positive, culture positive)

b, false positive (assay positive, culture negative)

c, false negative (assay positive, culture negative)

d, true negative (assay negative, culture negative)

Did the authors observe the false positive or false negative results at all? If they did, these values need to be included in formulas above. The authors will need to mention in the manuscript if there were false positive or false negative results.

2. It has been known that B. pseudomallei has diverse LPS types. Is OPS-ELISA able to detect immunoglobulin responses in sera from patients who were infected by B. pseudomallei strains with other LPS/O-antigen types? Has this been tested? Since the atypical LPS types (B, or rough) have been detected in B. pseudomallei strains from Thailand, targeting only the LPS type A by this assay may limit the efficacy of this assay. The authors will need to discuss this limitation.

3. Are both Hcp1-ELISA and OPS-ELISA detecting igG, IgM, or total IgG? It is worth mentioning it, otherwise the readers have to check with ref. 19 and 20 for more details. Since both assays have been compared in ref. 19, and the Hcp1-ELISA was the best one, why did the authors compare both assays again in this study? Were the patients' sera used in this study from the same cohort used in ref. 19?

4) It was not clear if the authors tested all these 3 assays at the same time in this study, or only used the results from previous studies for statistical analysis? Please clarify this since the authors cited ref. 15 and 19 in most parts of their methods.

Other comments:

1. In the last statement of the Author Summary, the authors should suggest whether the combination, CPS-LFI and Hcp1-ELISA, or CPS-LFI and OPS-ELISA, is further evaluated for the point-of-care diagnosis of melioidosis rather than saying in general that "using combinations of antigen- and antibody-detection should be further developed and evaluated".

2.Line: 126-128: Did the authors mean the sensitivity of PCR was limited in detecting B. pseudomallei in clinical specimens?

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Prof. David Dance

Reviewer #2: No

Reviewer #3: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Submitted filename: PNTD review June 2021.docx

Click here for additional data file.

26 Aug 2021

Submitted filename: Response to reviewers.docx

Click here for additional data file.

27 Sep 2021

Dear Ms Amornchai,

We are pleased to inform you that your manuscript 'Evaluation of antigen-detecting  and antibody-detecting diagnostic test combinations for diagnosing melioidosis' has been provisionally accepted for publication in PLOS Neglected Tropical Diseases.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Husain Poonawala

Associate Editor

PLOS Neglected Tropical Diseases

Paul J. Brindley

Co-Editor-in-Chief

PLOS Neglected Tropical Diseases

***********************************************************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: (No Response)

Reviewer #2: Yes, the previous comments have been addressed.

Reviewer #3: (No Response)

**********

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: (No Response)

Reviewer #2: Yes

Reviewer #3: (No Response)

**********

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: (No Response)

Reviewer #2: The limitations can be re-emphasised with caveats on the samples used

Reviewer #3: (No Response)

**********

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: (No Response)

Reviewer #2: Grammatical errors that need to be corrected

The figshare link on line 228 is not available

Reviewer #3: (No Response)

**********

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: (No Response)

Reviewer #2: Most of my comments and suggestions from the previous review have been sufficiently addressed.

However, the improvement in sensitivity demonstrated by the combination tests is still subjective because the individual test data was obtained at much earlier dates. How the cut-offs for controls and samples were determined is still not clear.

I suggest that the authors have to emphasise the limitations of the study design and samples and provide caveats on the feasibility of the combination tests for routine clinical diagnosis, particularly in rural hospitals, as the interpretation in association with when the patient presents at the hospital can be somewhat subjective.

Reviewer #3: Dear Editor,

The revised manuscript has most improvements and is well discussed. The authors have addressed my comments and concerns.

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Apichai Tuanyok

20 Oct 2021

Dear Ms Amornchai,

We are delighted to inform you that your manuscript, "Evaluation of antigen-detecting  and antibody-detecting diagnostic test combinations for diagnosing melioidosis," has been formally accepted for publication in PLOS Neglected Tropical Diseases.

We have now passed your article onto the PLOS Production Department who will complete the rest of the publication process. All authors will receive a confirmation email upon publication.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any scientific or type-setting errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript. Note: Proofs for Front Matter articles (Editorial, Viewpoint, Symposium, Review, etc...) are generated on a different schedule and may not be made available as quickly.

Soon after your final files are uploaded, the early version of your manuscript will be published online unless you opted out of this process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Shaden Kamhawi

co-Editor-in-Chief

PLOS Neglected Tropical Diseases

Paul Brindley

co-Editor-in-Chief

PLOS Neglected Tropical Diseases