Diagnostic accuracy of a novel tuberculosis point-of-care urine lipoarabinomannan assay for people living with HIV: A meta-analysis of individual in- and outpatient data.

BACKGROUND: Tuberculosis (TB) is the most common cause of death in people living with HIV (PLHIV), yet TB often goes undiagnosed since many patients are not able to produce a sputum specimen, and traditional diagnostics are costly or unavailable. A novel, rapid lateral flow assay, Fujifilm SILVAMP TB LAM (SILVAMP-LAM), detects the presence of TB lipoarabinomannan (LAM) in urine, and is substantially more sensitive for diagnosing TB in PLHIV than an earlier LAM assay (Alere Determine TB LAM lateral flow assay [LF-LAM]). Here, we present an individual participant data meta-analysis of the diagnostic accuracy of SILVAMP-LAM in adult PLHIV, including both published and unpublished data. METHODS AND
FINDINGS: Adult PLHIV (≥18 years) were assessed in 5 prospective cohort studies in South Africa (3 cohorts), Vietnam, and Ghana, carried out during 2012 to 2017. Of the 1,595 PLHIV who met eligibility criteria, the majority (61%) were inpatients, median age was 37 years (IQR 30-43), 43% had a CD4 count ≤ 100 cells/μl, and 35% were receiving antiretroviral therapy. Most participants (94%) had a positive WHO symptom screen for TB on enrollment, and 45% were diagnosed with microbiologically confirmed TB, using mycobacterial culture or Xpert MTB/RIF testing of sputum, urine, or blood. Previously published data from inpatients were combined with unpublished data from outpatients. Biobanked urine samples were tested, using blinded double reading, with SILVAMP-LAM and LF-LAM. Applying a microbiological reference standard for assessment of sensitivity, the overall sensitivity for TB detection was 70.7% (95% CI 59.0%-80.8%) for SILVAMP-LAM compared to 34.9% (95% CI 19.5%-50.9%) for LF-LAM. Using a composite reference standard (which included patients with both microbiologically confirmed as well as clinically diagnosed TB), SILVAMP-LAM sensitivity was 65.8% (95% CI 55.9%-74.6%), and that of LF-LAM 31.4% (95% CI 19.1%-43.7%). In patients with CD4 count ≤ 100 cells/μl, SILVAMP-LAM sensitivity was 87.1% (95% CI 79.3%-93.6%), compared to 56.0% (95% CI 43.9%-64.9%) for LF-LAM. In patients with CD4 count 101-200 cells/μl, SILVAMP-LAM sensitivity was 62.7% (95% CI 52.4%-71.9%), compared to 25.3% (95% CI 15.8%-34.9%) for LF-LAM. In those with CD4 count > 200 cells/μl, SILVAMP-LAM sensitivity was 43.9% (95% CI 34.3%-53.9%), compared to 10.9% (95% CI 5.2%-18.4%) for LF-LAM. Using a microbiological reference standard, the specificity of SILVAMP-LAM was 90.9% (95% CI 87.2%-93.7%), and that of LF-LAM 95.3% (95% CI 92.2%-97.7%). Limitations of this study include the use of biobanked, rather than fresh urine samples, and testing by skilled laboratory technicians in research laboratories, rather than at the point of care.
CONCLUSIONS: In this study, we found that SILVAMP-LAM identified a substantially higher proportion of TB patients in PLHIV than LF-LAM. The sensitivity of SILVAMP-LAM was highest in patients with CD4 count ≤ 100 cells/μl. Further work is needed to demonstrate accuracy when implemented as a point-of-care test.

See S1 Translation for the Japanese language Abstract.

Introduction

Tuberculosis (TB) is the most common cause of death in people living with HIV (PLHIV), whose risk of developing active TB is estimated to be approximately 30 times greater than in people without HIV [1]. Evidence from randomized diagnostic studies shows that early diagnosis of TB among PLHIV reduces mortality [2,3].

Traditional diagnostic methods, such as culture or smear microscopy, are slow or low in sensitivity. More sensitive modern techniques, such as Xpert MTB/RIF (Xpert), require a certain infrastructure, are costly, and are not widely accessible. Moreover, TB is harder to diagnose in PLHIV, since many of the patients have extrapulmonary TB (approximately 25%), produce paucibacillary sputum samples, or cannot reliably produce a sputum specimen [4,5]. TB in PLHIV is often fatal if undiagnosed or left untreated. New, rapid, non-sputum-based point-of-care (POC) diagnostic solutions to detect TB, especially in vulnerable groups, are urgently needed [6].

The commercially available Alere Determine TB LAM lateral flow assay (LF-LAM; Abbott, Chicago, US; in previous studies also called AlereLAM) is a rapid, inexpensive POC TB test [7]. While its use is associated with a mortality benefit in severely ill and immunocompromised PLHIV [2,3], it has only moderate sensitivity in patients with a low CD4 count and has had low programmatic uptake [8-10]. We have already reported on the novel Fujifilm SILVAMP TB LAM (SILVAMP-LAM; Fujifilm, Tokyo, Japan; in previous studies also called FujiLAM) assay. SILVAMP-LAM, similarly to LF-LAM, detects the presence of lipoarabinomannan (LAM) in urine using a visually read lateral flow test, but unlike LF-LAM, SILVAMP-LAM utilizes silver amplification. For inpatients with HIV, it offers on average an increase in sensitivity of approximately 25%–30% compared to LF-LAM across CD4 strata, while maintaining a high specificity when a composite reference standard is used [11].

Here, we present all available individual patient data (IPD), both published [11,12] and unpublished, from SILVAMP-LAM testing (“intervention” per PRISMA guidelines [13]) in adult inpatients and outpatients with HIV (“patients”) in comparison to LF-LAM (“comparator”) across 5 prospective cohorts (“studies”) and analyze its diagnostic accuracy (“outcome”). These data contributed to a guidance development group consultation of the World Health Organization in May 2019.

Methods

Study population

Biobanked urine samples from adult PLHIV (≥18 years), collected in 5 prospective cohort studies in South Africa, Vietnam, and Ghana, were assessed (S1 Table). Study protocols and statistical analysis plans for the different studies are available upon request. A study that combines inpatient cohorts from South Africa (cohorts 1A, 2, and 3) [11] and the study from Ghana (cohort 5) [12] have been published.

For the first cohort from South Africa, adults with TB symptoms able to produce sputum were enrolled consecutively, independently of their HIV status. Inpatients were enrolled on admission to Khayelitsha Hospital (cohort 1A), while outpatients were enrolled at the Town Two and Nolungile primary healthcare facilities in the Khayelitsha township (cohort 1B), between February 2017 and August 2017. Those in whom the disease was thought to be only extrapulmonary were excluded.

The second cohort from South Africa (cohort 2) enrolled adult inpatients with HIV consecutively, independently of their CD4 count, as they were admitted to adult medical wards at GF Jooste Hospital between June 2012 and October 2013, regardless of their ability to produce sputum or whether they reported TB symptoms [14].

The third cohort from South Africa (cohort 3) enrolled inpatient PLHIV at Khayelitsha Hospital with a CD4 ≤ 350 cells/μl in whom TB was considered the most likely diagnosis at presentation between January 2014 and October 2016 [15]. A list of all potentially eligible patients was compiled daily, and a random selection procedure (using a die after all potentially eligible patients had been identified) was followed to enroll 2–4 patients daily.

The fourth cohort (cohort 4), from Vietnam, used samples from consecutively enrolled patients presenting to the outpatient clinics of a public sector district hospital (Pham Ngoc Thach Hospital, Ho Chi Minh City) with symptoms suggestive of TB, independently of HIV status, between September 2016 and July 2017.

For cohort 5, HIV-infected adults eligible for antiretroviral therapy were consecutively enrolled irrespective of whether they reported TB signs and symptoms from an outpatient clinic in the Fevers Unit of Korle-Bu Teaching Hospital, a public referral hospital in Accra, the capital city of Ghana, between January 2013 and March 2014 [12,16].

All cohorts excluded patients who were already receiving anti-TB therapy. Where patients were enrolled independently of HIV status, the present study included only PLHIV. More details on the individual cohorts can be found in S1 Table and S2 Table. Relevant data are available in S1 Data. Study protocols are available upon request.

All study-related activities were approved by the human research ethics committees of the respective sites including City of Cape Town (Ref. 10364a) and the University of Cape Town Human Research Ethics Committee (UCT HREC, Ref. 250/2018) for cohort 1, UCT HREC (Ref. 001/2012) for cohort 2, UCT HREC (Ref. 057/2013) for cohort 3, Ministry of Health Vietnam (Ref. 2493/QĐ-BYT) for cohort 4, and the Institutional Review Board of University of Ghana Medical School (Ref. MS-Et/M.4-P3.3/2012-13) and the Danish National Committee (Ref. 1302133/Doc No. 1206169) for cohort 5. Written informed consent was obtained from patients, as per the study protocols. Study participation did not affect standard of care. All reporting follows STARD and PRISMA guidelines [17,18] (S1 STARD Checklist and S1 PRISMA IPD Checklist), and no analysis plan was prespecified.

Laboratory methods

Samples

Urine specimens were stored at −80°C (cohort 1, cohort 3, cohort 4) and −20°C (cohort 2, cohort 5) (S3 Table). Aliquots of frozen, unprocessed urine were thawed to ambient temperature and mixed manually prior to testing with SILVAMP-LAM and LF-LAM. Samples that were not immediately used for testing were stored at 4°C for a maximum of 4 hours.

Users and training

SILVAMP-LAM and LF-LAM testing was performed by skilled laboratory technicians in research laboratories at the University of Cape Town and the Research Institute of Tuberculosis of the Japan Anti-Tuberculosis Association. The technicians were newly trained to perform SILVAMP-LAM. Training included review of the English instructions for use, explanation and demonstration of both assays by a trainer, and conductance of up to 3 tests by the users followed by a proficiency test questionnaire. The user’s ability to correctly interpret the LF-LAM band intensity was assessed using 20 scanned LF-LAM test strips with different results next to the 4-grade reference scale card. The total training time for both tests was 4–6 hours.

Index testing (SILVAMP-LAM)

Testing with SILVAMP-LAM was performed according to the manufacturer’s instructions using urine from the same aliquot as that used with LF-LAM. The 5-step test procedure is illustrated in an online video [19] and takes 50–60 minutes from start to end result. In brief, urine is added to the reagent tube up to the indicator line (approximately 200 μl), mixed, and incubated for 40 minutes at ambient temperature. After mixing again, 2 drops of urine/reagent are added to the test strip at position 1. Following this, button 2 is pressed immediately. After the “go-next” color indicator mark turns orange (within 3–10 minutes), button 3 is pressed. The result is then read within 10 minutes. The SILVAMP-LAM assay does not use a reference scale card and any visible test line is considered positive.

Comparator testing (LF-LAM)

LF-LAM is a commercially available lateral flow assay that detects LAM with polyclonal antibodies. It is currently recommended by WHO to assist in the diagnosis of active TB in PLHIV [20]. LF-LAM was used according to the test’s package insert. To sum up, 60 μl of urine is applied to the sample pad. After 25 minutes, the test strip is interpreted using the 4-grade reference scale card, with the grade 1 cut-off point as the positivity threshold.

Blinding

SILVAMP-LAM was read independently by 2 readers blinded to the results of one another (blinded double reading). After the initial test interpretation, the 2 readers compared results and, in the event of discordance, re-inspected the test to establish a final consensus result (by mutual agreement) that was then used for analysis. The same procedure was used for LF-LAM. SILVAMP-LAM and LF-LAM reading occurred blinded to the test results of the other LAM-based test, patient diagnosis, and all other TB test results. SILVAMP-LAM and LF-LAM results were not available to the assessors of the reference standard.

Test failure

In case of SILVAMP-LAM or LF-LAM failure, the test was repeated once. Accuracy calculations were performed from the valid result (first or second attempt).

Reference standard testing

For reference standard testing, the specimens were processed using standardized protocols from centralized accredited laboratories of the different sites. The testing flow for each cohort is shown in S2 Table. Sputum, blood, and urine specimens for Mycobacterium tuberculosis (M.tb) reference standard testing were collected at enrollment, and additional clinical samples were obtained during hospital admission and at follow-up. Sputum collection across cohorts was done by an experienced nurse or trained clinical research worker, and sputum induction was performed (except for cohort 5) when required. Reference standard testing was performed on all available sputum specimens and included Xpert MTB/RIF (Xpert, Cepheid, Sunnyvale, CA, US; testing predated rollout of Xpert Ultra MTB/RIF), sputum smear microscopy (fluorescence microscopy using Auramine O staining and/or Ziehl–Neelsen staining), Mycobacteria Growth Indicator Tube (MGIT) liquid culture (Becton Dickinson, Franklin Lakes, NJ, US), and solid culture on Löwenstein–Jensen medium. The presence of M.tb complex in solid or liquid culture was confirmed with MPT64 antigen detection and/or MTBDRplus, MTBC, or CM/AS line probe assays (Bruker Hain [formerly Hain Lifesciences], Nehren, Germany). Blood culture from all participants was done in BACTEC Myco/F Lytic culture vials (Becton Dickinson). The exception was that no blood cultures were done for cohorts 4 and 5. WHO-prequalified in vitro diagnostic tests were used for HIV testing (rapid diagnostic tests) and CD4 cell counting (flow cytometry). For urine Xpert testing, 30–40 ml of urine (in cohort 5, only 6 ml) was centrifuged, and, following removal of the supernatant, the pellet was re-suspended in the residual urine volume, then 0.75 ml was tested using Xpert. No urinary Xpert testing was done for cohort 4. For cohorts 2 and 3, additional respiratory and non-respiratory samples such as pleural fluid, cerebrospinal fluid, and tissue fine needle aspirates were obtained, where clinically indicated, and tested using MGIT liquid culture and/or microscopy and/or Xpert. Details of testing according to cohort are reported in S2 Table.

Case definitions

Patients were assigned to 1 of 4 diagnostic categories using a combination of clinical and laboratory findings. S4 Table indicates the categorization per cohort. Briefly, “definite TB” included patients with microbiologically confirmed M.tb (any culture or any Xpert positive for M.tb). “Not TB” included patients with all microscopy, culture, and Xpert tests negative for M.tb (including at least 1 negative noncontaminated culture result), who were not started on anti-TB treatment and were alive or who improved at 2 to 3 months’ follow-up. “Possible TB” was diagnosed in patients who did not satisfy the criteria for “definite TB” but had clinical/radiological features suggestive of TB and were started on TB treatment by non-study clinicians. Patients who did not fall into any of these categories were considered “unclassifiable” and removed from the main analyses but included in a sensitivity analysis. Definition and examples of the “unclassifiable” category can be found in S5 Table.

Statistical methods

Simple descriptive statistics were used to characterize cohorts. Sensitivity and specificity of the index test were estimated against a microbiological reference standard (MRS) or a composite reference standard (CRS). The “definite TB” and “not TB” categories were used to allocate patients into positive and negative, respectively. The “possible TB” group was considered negative by MRS but positive by CRS, as previously proposed in a study guidance publication [21].

Diagnostic accuracy was determined separately for each cohort, and 95% confidence intervals (95% CIs) were computed using Wilson’s score method. Sensitivity and specificity of LF-LAM and SILVAMP-LAM for each cohort were compared using the McNemar test. To estimate sensitivity and specificity across cohorts and CD4 strata, we performed a 2-stage IPD meta-analysis; aggregate data (true positives, false negatives, false positives, true negatives) were extracted from the individual studies and combined using a Bayesian bivariate random-effects model using the meta4diag package [22]. Results are presented with 95% CIs. In a sensitivity analysis, we assessed the effect on performance when the “unclassifiable” cases were included (a) as MRS negative or (b) as CRS positive. Cohen’s kappa coefficient was used to determine inter-reader agreement for SILVAMP-LAM and LF-LAM. The data analysis was performed with R (version 3.5.1) and STATA 15.

Results

Overall, 3,062 potentially eligible participants were screened across the 5 cohorts, of which 1,132 were ineligible according to exclusion criteria predefined in the cohort protocols (Fig 1). HIV-negative participants were excluded and will be reported separately. As a result, 1,930 patients were considered for urine LAM testing on biobanked samples. For the primary analysis, an additional 335 participants were excluded, either due to unavailability of a urine sample (n = 129), failed index test (n = 6), or being “unclassifiable” (n = 200).

Fig 1

Flow diagram showing the study populations and the number of patients included overall, per cohort, per hospitalization status, and per TB case definition.

CRS, composite reference standard; LAM, lipoarabinomannan; MRS, microbiological reference standard; TB, tuberculosis; w or w/o, with or without.

Consequently, 1,595 PLHIV across all 5 cohorts were combined for the primary analysis. The majority were inpatients (968; 61%), and 627 (39%) were outpatients. All inpatients came from South African sites, while outpatient data originated primarily from Ghana (63%), with South Africa contributing 28%, and Vietnam the remaining 9%. The characteristics across all PLHIV (and across cohorts) are reported in Table 1.

Table 1

Demographic and clinical characteristics of all PLHIV.

Characteristic	All PLHIV (n = 1,595)	Inpatient PLHIV (n = 968)	Outpatient PLHIV (n = 627)
Demographic and clinical characteristics
Age—years	36.6 (30; 43)	35 (30; 42)	38 (31; 44)
Female sex	939 (59%)	523 (54%)	416 (66%)
Positive WHO TB symptom screen	1,498 (94%)	933 (96%)	565 (90%)
History of TB	536 (34%)	439 (45%)	97 (16%)
Antiretroviral therapy	558 (35%)	394 (41%)	164 (26%)
CD4 count—cells/μl	126 (42; 302)	86 (33; 190)	249 (91; 477)
Distribution in diagnostic categories
Definite TB	724 (45%)	600 (62%)	124 (20%)
Possible TB	110 (7%)	91 (9%)	19 (3%)
Not TB	761 (48%)	277 (29%)	484 (77%)
CD4 count (cells/μl)
0 to 100	677 (43%)	516 (53%)	161 (26%)
101 to 200	319 (20%)	216 (22%)	103 (16%)
>200	581 (36%)	231 (24%)	350 (56%)
Unknown	18 (1%)	5 (1%)	13 (2%)
Outcome
Died within 2–3 months	119 (7%)	105 (11%)	14 (2%)
Alive	1,354 (85%)	810 (84%)	544 (87%)
Lost to follow-up	35 (2%)	16 (2%)	19 (3%)
No follow-up done/required*	87 (6%)	37 (4%)	50 (8%)

Data are given as median (interquartile range) or number (percent).

*No follow-up done/required: empty follow-up field in case report form of cohort 4 or no follow-up required at other sites.

PLHIV, people living with HIV; TB, tuberculosis; WHO, World Health Organization.

Participants were typically young adults (median age 37 years [IQR 30–43]), and 59% were female. Forty-three percent had a CD4 count below or equal to 100 cells/μl and 36% above 200 cells/μl. Most participants (94%) had a positive WHO symptom screen for TB upon enrollment. Forty-five percent (n = 724) were diagnosed with definite TB, and 119 (7%) died within 2–3 months after enrollment.

Comparison of diagnostic sensitivity of SILVAMP-LAM and LF-LAM

The meta-analysis of all PLHIV across cohorts showed an overall sensitivity for active TB detection of 70.7% (95% CI 59.0%–80.8%) for SILVAMP-LAM compared to 34.9% (95% CI 19.5%–50.9%) for LF-LAM against the MRS, with a difference of 35.8 percentage points between the 2 tests (Fig 2A). When using the CRS, the sensitivity difference between the assays was 34.4 percentage points: the overall SILVAMP-LAM point estimate was 65.8% (95% CI 55.9%–74.6%) and that of LF-LAM, 31.4% (95% CI 19.1%–43.7%). Amongst inpatients, SILVAMP-LAM sensitivity was 28.1 percentage points higher compared to LF-LAM, and confidence intervals did not overlap (Fig 2B). In outpatients, SILVAMP-LAM sensitivity was 42.7 percentage points higher compared to LF-LAM, but confidence intervals overlapped (Fig 2B). An analysis per cohort, and tables comparing LF-LAM to SILVAMP-LAM results, can be found in S1 Fig, S6 Table and S7 Table.

Fig 2

Accuracy of SILVAMP-LAM and LF-LAM against different reference standards and in in- and outpatients.

Forest plots of sensitivity and specificity and differences between SILVAMP-LAM and LF-LAM (A) against the microbiological and composite reference standards for all cohorts combined and (B) against the MRS by in- and outpatients. The axis for sensitivity ranges from 0% to 100%, while the axis for specificity ranges from 50% to 100%. CRS, composite reference standard; FN, false negative; FP, false positive; LF-LAM, Alere Determine TB LAM lateral flow assay; MRS, microbiological reference standard; SILVAMP-LAM, Fujifilm SILVAMP TB LAM; Sn, sensitivity; Sp, specificity; TN, true negative; TP, true positive.

When patients were stratified by CD4 count, we observed an inverse relationship between sensitivity and CD4 count for both assays (Fig 3A): the sensitivity was higher for lower CD4 counts and systematically decreased for higher CD4 counts. In patients with a CD4 count ≤ 100 cells/μl, SILVAMP-LAM had a sensitivity of 87.1% (95% CI 79.3%–93.6%), compared to 56.0% (95% CI 43.9%–64.9%) for LF-LAM (Fig 3A). A similar difference in sensitivity was observed in patients with less severe immunosuppression (CD4 > 200 cells/μl), but overall sensitivity was lower for both assays in this group: 43.9% (95% CI 34.3%–53.9%) for SILVAMP-LAM and 10.9% (95% CI 5.2%–18.4%) for LF-LAM (Fig 3A). The difference in performance in inpatients versus outpatients was largely explained by the differences in the distribution of the populations across CD4 strata, as outlined in Fig 3B.

Fig 3

Accuracy of SILVAMP-LAM and LF-LAM across CD4 strata.

Forest plots of sensitivity and specificity of (A) SILVAMP-LAM and LF-LAM in CD4 strata ≤100, 101–200, and >200 cells/μl against the microbiological reference standard for all cohorts combined and (B) SILVAMP-LAM only against the microbiological reference standard in in- and outpatients in the CD4 strata. The axis for sensitivity ranges from 0% to 100%, while the axis for specificity ranges from 50% to 100%. FN, false negative; FP, false positive; LF-LAM, Alere Determine TB LAM lateral flow assay; SILVAMP-LAM, Fujifilm SILVAMP TB LAM; TN, true negative; TP, true positive.

Additional information on the distribution of patients across cohorts by CD4 group and smear status is provided in S8 Table and S9 Table. An analysis of accuracy by smear status is provided in S2 Fig. A sensitivity analysis including those falling in the “unclassifiable” category is reported in S10 Table. When the “unclassifiable” cases were included as CRS positive, the sensitivity decreased for both tests, but more for SILVAMP-LAM (12.2%) than for LF-LAM (4.7%).

Comparison of diagnostic specificity of SILVAMP-LAM and LF-LAM

The specificity estimates, when using the MRS for the meta-analysis, were 90.9% (95% CI 87.2%–93.7%) and 95.3% (95% CI 92.2%–97.7%) for SILVAMP-LAM and LF-LAM, respectively (Fig 2A); however, the confidence intervals overlapped. When using the CRS, the overall specificity estimates were higher: 93.4% (95% CI 89.3%–96.2%) for SILVAMP-LAM and 97.3% (95% CI 95.1%–98.9%) for LF-LAM (Fig 2A). Here too, the confidence intervals overlapped. The specificity of SILVAMP-LAM was lower amongst those with CD4 ≤ 100 cells/μl (80.5% using the MRS and 85.2% using the CRS) compared to those in the higher CD4 count strata (Fig 3A). Of the 47 patients with SILVAMP-LAM false-positive results, 34 (72%) came from patients with CD4 count ≤ 100 cells/μl, while overall this CD4 group included 43% of patients. Nontuberculous mycobacteria (NTM) were cultured from sputum (M. avium/intracellulare complex) in 2 of the patients classified as “not TB” but positive by SILVAMP-LAM. A more detailed analysis of all false-positive results, with additional information on the clinical and laboratory workup of the individual patients, is provided in S11 Table.

When the “unclassifiable” patients were included in a sensitivity analysis as MRS negative, the specificity remained largely the same (with a difference of −0.4 percentage points for SILVAMP-LAM and −0.8 percentage points for LF-LAM).

Invalid results and inter-reader agreement

Out of 1,801 initial test runs with SILVAMP-LAM, 26 tests (1.4%) failed (by comparison to 6 that failed for LF-LAM; 0.3%). Reasons for failure are reported in S12 Table. The most common reason for failure was that no control line was present (12/26). A repeat was possible for 23 out of 26 samples (3 had insufficient sample), and 20 yielded a valid result on repeat. Thus, there was an overall invalid rate of 0.16% (3/1,824 runs). Interrater agreement was very high both for SILVAMP-LAM and LF-LAM (kappa 0.95 and 0.92, respectively; S13 Table).

Discussion

In this assessment of 1,595 hospitalized and non-hospitalized PLHIV from settings with high TB burden, the SILVAMP-LAM assay identified a substantially higher proportion of TB patients than LF-LAM. While the specificity was lower for SILVAMP-LAM than for LF-LAM, the 95% confidence intervals of specificity for these tests overlapped. In all sub-analyses by CD4 count (Fig 3), the sensitivity of SILVAMP-LAM was higher (range 23.8–37.4 percentage points) than that of LF-LAM, with non-overlapping confidence intervals for the inpatient cohort (likely due to the higher case number, which allowed more accurate estimates).

As in previous reports [11,23], we found an inverse relationship between the sensitivity of LAM tests and CD4 count (Fig 3), and sensitivities of the tests were lowest in cohort 1B, where the majority of patients had CD4 count > 200 cells/μl (S1 Fig; S9 Table). The sensitivity of SILVAMP-LAM was the highest (87.1%, 95% CI 79.3%–93.6%) in patients with the highest risk of having disseminated TB and of death from TB associated with severe immunosuppression (patients with CD4 ≤ 100 cells/μl) [24]. SILVAMP-LAM could have diagnosed TB in up to 89% of patients who died [25], and testing, particularly in immunocompromised patient populations, is expected to have a mortality benefit, as has been shown for LF-LAM [2,3], although this still needs to be evaluated. The moderate sensitivity that was maintained even in the highest CD4 stratum (44.4%) will result in a higher diagnostic yield for SILVAMP-LAM and will offer an expanded opportunity of rapid diagnosis on presentation in this patient population.

The increase in sensitivity that had been observed in inpatients was confirmed in outpatients, and differences in performance of SILVAMP-LAM between in- and outpatients were explained by differences in CD4 group distributions [11]. For LF-LAM, the CD4 count distribution did not fully explain the sensitivity difference observed between in- and outpatients. This finding is in line with the results published in the updated Cochrane meta-analysis for LF-LAM [23].

The point estimates of specificity for SILVAMP-LAM were lower than those for LF-LAM using both the MRS and the CRS. However, the lower specificity of both LF-LAM and SILVAMP-LAM could be explained in part by an imperfect reference standard that lacks complete sensitivity (i.e., reference-standard-negative, LAM-positive results that represent true TB) [26]. It is possible that an imperfect reference standard could disproportionally affect a more sensitive test. The lower specificity seen with a decreased CD4 count in this study, and the improved specificity seen with the CRS compared to the MRS, further support this explanation. Also, in higher CD4 count strata, the specificities of the 2 tests approximate each other, which provides further supporting evidence. In addition, cohorts with no or with only limited culture or Xpert testing in blood or urine (S2 Table and S1 Fig; cohorts 4 and 5) reported lower specificities, which again might point towards underdiagnosis by the reference standard, as shown in previous studies [26,27].

An alternative explanation for the reduced specificity of SILVAMP-LAM could be cross-reactivity to other pathogens. This was a problem with LF-LAM, as the polyclonal antibodies used in the test are known to react with urinary tract pathogens and fast-growing NTM [28,29]. Cross-reactivity to common urinary tract pathogens and fast-growing NTM has been excluded in studies assessing the antibodies used in SILVAMP-LAM, but some cross-reactivity has been observed with slow-growing NTM [28,30]. Our data suggest that cross-reactivity is a small problem, if at all, as slow-growing NTM were observed in only 2 out of 47 patients with false-positive results on SILVAMP-LAM (S11 Table).

This meta-analysis has some limitations, pointing to areas needing further research. SILVAMP-LAM testing was done using biobanked specimens from hospitalized and non-hospitalized patients. However, there is evidence to suggest that data from retrospective LAM testing based on frozen samples are comparable to data from testing on fresh samples [31,32]. The operators in the current study were highly skilled laboratory personnel in reference centers. SILVAMP-LAM may have the potential to be implemented as a POC assay in primary care clinics, HIV clinics, or TB microscopy centers. The accuracy, feasibility, and acceptability in these settings, with often less-skilled workers, needs to be evaluated in a separate study, particularly considering the slightly higher complexity of SILVAMP-LAM over LF-LAM. The inclusion of different cohorts with different study designs resulted in heterogeneity and different exclusion rates; however, differences in diagnostic sensitivity between SILVAMP-LAM and LF-LAM persisted in a sensitivity analysis including the “unclassifiable” category (S10 Table) and in the analyses by CD4 subgroup (Fig 3). Further, antiretroviral treatment may have influenced patient outcome and thus the reference standard categories “possible TB” and “unclassifiable.”

SILVAMP-LAM has the potential to have similar or improved favorable effects on patient outcomes compared to LF-LAM, although a prospective study in relevant clinical settings is needed to evaluate this.

Neither LF-LAM nor SILVAMP-LAM can differentiate drug-resistant from drug-sensitive TB, and therefore it is important that these rapid diagnostic tools are supplemented with drug susceptibility testing. Evaluation of SILVAMP-LAM in diagnostic algorithms therefore should be considered.

In 2019, over 5 years after the establishment of the WHO policy on LF-LAM, a survey of 31 countries with high TB/HIV burden, with responses obtained from 24, showed that only 11 countries had LF-LAM policies in place, with only 5 countries currently using LF-LAM [10]. Limited budgets, lack of country-specific data, administrative hurdles such as local regulatory approval, lack of coordination between national TB and HIV programs, and small perceived patient population size were the most commonly cited constraints on LF-LAM adoption [10]. There is real potential for a broader and simpler WHO recommendation for SILVAMP-LAM given its higher sensitivity, and this could help to overcome some of these implementation barriers.

Collectively, these results suggest that, if implemented in clinical practice and linked with appropriate treatment, the SILVAMP-LAM assay would allow for earlier diagnosis of HIV-associated TB in a larger proportion of hospitalized and non-hospitalized patients compared to the current LF-LAM test.

(XLSX)

Click here for additional data file.

Forest plots of sensitivity and specificity for SILVAMP-LAM and LF-LAM by cohort against the microbiological reference standard (MRS) and the composite reference standard (CRS).

(TIFF)

Click here for additional data file.

Accuracy by smear status.

(TIFF)

Click here for additional data file.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

Study population, setting and location, eligibility, and inclusion and exclusion criteria used for the studies.

(DOCX)

Click here for additional data file.

Specimen collection and testing flow by cohort.

(DOCX)

Click here for additional data file.

Specimen storage.

(DOCX)

Click here for additional data file.

Diagnostic categories by cohort.

(DOCX)

Click here for additional data file.

Definition and examples of “unclassifiable” category.

(DOCX)

Click here for additional data file.

Two-by-two table of SILVAMP-LAM versus LF-LAM among “definite TB” patients.

(DOCX)

Click here for additional data file.

Two-by-two table of SILVAMP-LAM versus LF-LAM among “not TB” patients.

(DOCX)

Click here for additional data file.

Analysis by cohort, smear status, and CD4 group for all HIV-positive inpatients.

(DOCX)

Click here for additional data file.

Analysis by cohort, smear status, and CD4 group for all HIV-positive outpatients.

(DOCX)

Click here for additional data file.

Sensitivity analysis of diagnostic accuracy for all PLHIV (including “unclassifiable” patients) by MRS and CRS.

(DOCX)

Click here for additional data file.

Further Information on patients categorized as “not TB” with positive SILVAMP-LAM results.

(DOCX)

Click here for additional data file.

SILVAMP-LAM failure rates and errors for all samples tested.

(DOCX)

Click here for additional data file.

Agreement of 2 independent test readers for all samples tested.

(DOCX)

Click here for additional data file.

Japanese translation of the abstract by Satoshi Mitarai.

(DOCX)

Click here for additional data file.

31 Jan 2020

Dear Dr. Denkinger,

Thank you very much for submitting your manuscript "Diagnostic accuracy of a novel point-of-care urine lipoarabinomannan assay for people living with HIV – a meta-analysis of in- and outpatient data" (PMEDICINE-D-19-03864) for consideration at PLOS Medicine.

Your paper was evaluated by an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to invite you to submit a revised version that fully addresses the reviewers' and editors' comments. You will appreciate that we cannot make a decision about publication until we have seen the revised manuscript and your response, and we expect to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We hope to receive your revised manuscript by Feb 21 2020 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosmedicine/s/submission-guidelines#loc-methods.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

Please let me know if you have any questions. Otherwise, we look forward to receiving your revised manuscript in due course.

Sincerely,

Richard Turner PhD, for Thomas McBride, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

-----------------------------------------------------------

Requests from the editors:

Our academic editor commented: "Using brands in scientific articles is something it is better to avoid. Could the authors instead specify the technology used in each assay to differentiate the two?"

We suggest amending the data statement (to be presented in the metadata in the event of publication) to "Yes - all data available" reflecting that the data for the analyses presented in the current study are available. Please include these data in the form of supplementary files or, if this is not possible, supply a non-author contact for readers interested in inquiring about data access. Please let me know if it would be helpful to discuss this further.

Please substitute a colon in your title after "HIV".

At line 78, please start the sentence "In this study, we found that ..." or similar.

In your abstract and elsewhere in the paper, please quote p values alongside 95% CI where available.

After your abstract, we will need to ask you to add a new and accessible "author summary" section in non-identical prose. You may find it helpful to consult one or two recent research papers published in PLOS Medicine to get a sense of the preferred style.

Early in the methods section of your main text, please state whether the current analysis had a specific protocol or prespecified analysis plan, and if so attach the document(s) as a supplementary file (referred to in the text). Please highlight analyses that were not prespecified.

Please update the reference at line 408.

Throughout the text, please format reference call-outs as follows: "... immunocompromised PLHIV [7,8], ...".

Please remove trade marks throughout your text.

Please spell out the group author name for references 9, 17 and 18.

Please add full access details for references 11 and 23.

Please ensure that all journal names are abbreviated as appropriate in your reference list, e.g., reference 19.

We ask you to adapt your PRISMA checklist so that individual items are referred to by section (e.g., "Methods") and paragraph number rather than by page or line numbers, as the latter generally change in the event of publication.

Please use the PRISMA IPD document (http://www.prisma-statement.org/documents/PRISMA%20IPD%20checklist.pdf).

Please refer to the document in the methods section, around line 156.

Comments from the reviewers:

*** Reviewer #1:

[See attachment]

Michael Dewey

*** Reviewer #2:

A very interesting study with several strengths:

1. Clear and well written

2. Findings are important. The new Fuji test appears to be a major improvement over the old Alere test.

3. Multiple cohorts from different settings strengthens the findings

4. Stratified analyses are useful to position the findings.

Weaknesses/major comments:

1. My major concern revolves around the practical utility of the test. The sensitivity is adequate only in persons with advanced HIV. In HIV infected with CD4 >100 sensitivity drops, and if CD4>200 the test is of questionable clinical utility. We need to know more about when the test functions best, and when it is sub-optimal. A major piece of information, that I could not find anywhere (which is puzzling - not in Tables, nor text) is ARV treatment. How many were on ARV, at time of testing, how many were started within 2 weeks, or 1 month, etc - this would influence the clinical reference standard a lot.

2. I strongly recommend the addition of the findings from the HIV negative participants in the different cohorts. This will help inform what settings this test will be useful to implement. If sensitivity is very low in HIV negative, this wil be useful to decision makers whether to invest in this test.

3. I suggest the sensitivity/specificity at higher cut-points, or thresholds of CD4 (>300, >400, etc)

4. The authors acknowledge that for the study the test was conducted by skilled technicians. Can they provide some information on initial and on-going training and supervision required. And, indeed, what were the qualifications of the technicians involved. This is an often overlooked point in diagnostic studies. Reading the description, the test seems a little tricky, so more info on the training and qualifications of the techs involved would be useful to be able to gauge whether this really can be a POC test.

5. Results - abstract: The authors mention the stratified analysis that favours the test (CD4<100). They should also mention the very important finding of 63% sensitivity if CD4 100-200, and 44% if CD>200.

6. Conclusions - abstract: I find some of the conclusions, especially in the abstract, not supported by the data presented. "In patients with the greatest risk of death from TB (patients with CD4 ≤100 cells/μl), the sensitivity of FujiLAM was highest. Thus, FujiLAM is expected to have at least a similar mortality benefit to that previously shown for AlereLAM". I did not see results of sensitivity or specificity among those who died vs survived, so I suggest they simply state what they did show - sensitivity was greatest if CD4<100. The "expected mortality benefit" is purely speculative. Similarly, "and to establish whether reduced specificity may be due to missed detection of TB by other diagnostic tests currently in use" This seems speculative to me; they did see that when the clinical standard truly excluded TB (although the requirement for improvement may have selected for the more healthy/ milder disease) the specificity was higher - this finding can be mentioned.

Minor points:

1. I find the data access to be not in conformity with PLOS Med policy.

2. The COI statement reads more like publicity for FIND. More importantly, the nature of the agreement between FIND and the test developers is unclear and must be clarified in terms of financial considerations, ownership of rights to the test, and future access to the test. The comments I have made about the abstract results and conclusions reflect a concern over potential bias. No doubt the investigators are enthusiastic about this test; this 'optimism bias' afflicts us all. But this makes it all the more important to understand the exact financial relationship between FIND and the test developers. And to guarantee public access to the data. (ie these are related concerns)

*** Reviewer #3:

This is a very well written individual patient data level meta-analysis of the diagnostic accuracy of FujiLAM, with a comparison to AlereLAM. The data presented are important in the field of HIV-associated TB, and I believe the methodology used is appropriate. My only major concern is the overlap between this article, and the original manuscript published in the Lancet ID (ref 11 in the manuscript) which already combines the inpatient data from cohorts 1-3 (includes all the in-patient data presented in this manuscript). I do not think the authors have adequately acknowledged or justified this duplication in the current manuscript, and why this manuscript did not simply present the outpatient data. The discussion and conclusions are very similar between both papers.

Minor points

-Line 90, I disagree with the statement that MOST deaths would be preventable if TB were diagnosed earlier, the evidence to support this statement does not exist

-Line 106 mentions low programmatic uptake of AlereLAM, could the authors add to the discussion if/why they think uptake of FujiLAM might be better

-Please add heterogeneity of reference TB tests in the different studies to the limitations in the discussion section

-Lines 390-394, please clarify that the CIs for outpatients did overlap

-Please add to appendix details of how urine specimens were biobanked, including temperature and duration etc, as this could impact accuracy

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

Submitted filename: broger.pdf

Click here for additional data file.

20 Mar 2020

Dear Dr. Denkinger,

Thank you very much for re-submitting your manuscript "Diagnostic accuracy of a novel tuberculosis point-of-care urine lipoarabinomannan assay for people living with HIV: a meta-analysis of individual in- and outpatient data" (PMEDICINE-D-19-03864R1) for consideration at PLOS Medicine.

I have discussed the paper with our academic editor and it was also seen again by two reviewers. I am pleased to tell you that, provided the remaining editorial and production issues are dealt with, we expect to be able to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

Our publications team (plosmedicine@plos.org) will be in touch shortly about the production requirements for your paper, and the link and deadline for resubmission. DO NOT RESUBMIT BEFORE YOU'VE RECEIVED THE PRODUCTION REQUIREMENTS.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We hope to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

Please let me know if you have any questions. Otherwise, we look forward to receiving the revised manuscript shortly.

Kind regards,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

------------------------------------------------------------

Requests from Editors:

Please note in your competing interests statement that CMD is a member of PLOS Medicine's Editorial Board.

Noting your first point of response, please do adopt the terminology "LfLAM" and "LfSilvAmpLAM" throughout the paper.

Early in the "methods and findings" subsection of your abstract, please quote the range of years over which the studies were conducted (e.g., "... and Ghana, carried out during 2012-17").

At line 82, "... proportion of TB cases"?

In the abstract and throughout the paper, please provide p values alongside 95% CI where available.

You mention twice that the meta-analysis follows PRISMA guidelines (lines 155 and 197), and once will suffice. Please refer to the attached checklist (e.g., "see S1_PRISMA_Checklist") at this mention.

In your reference list, please remove the academic editors' names from references 4 and 30, and any others.

Please remove "Lancet Infectious Diseases" from reference 19 (we believe the relevant paper is reference 11) and substitute "YouTube".

We ask you to also provide a completed STARD checklist as a supplementary document, again referred to in your methods section.

Comments from academic editor:

(1) I do think we should avoid brand names and refer to the technology. This is what we always do for medicines so think it is appropriate for diagnostics as well. Using LfLAM for the assay that relies on lateral flow alone and LfSilvAmpLAM to refer to the assay that uses both lateral flow and silver amplification seems to be a good alternative.

(2) The highest risk for misinterpretation occurs when p-values are presented without 95% CIs. I understand the authors' concerns about p-values, but we are asking that they provide both the 95% CI and p-value.

(3) The authors should state that the data for HIV negative persons are being analysed and will be published separately somewhere in the MS (this was in their response but did not see it the text)?

(4) Similarly, the issues with performance across CD4 strata and with respect to ART status should be crystal clear in the abstract and the main text.

Comments from Reviewers:

*** Reviewer #1:

The authors have met my points.

I have just one minor comment. I would leave in the statement about the use of the die for selection, it is what was done and although perhaps sub-optimal it could not be called wrong in the field.

Michael Dewey

*** Reviewer #3:

I am happy that all the reviewers comments have been adequately addressed

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

9 Apr 2020

Dear Dr. Denkinger,

On behalf of my colleagues and the academic editor, Dr. Amitabh Bipin Suthar, I am delighted to inform you that your manuscript entitled "Diagnostic accuracy of a novel tuberculosis point-of-care urine lipoarabinomannan assay for people living with HIV: a meta-analysis of individual in- and outpatient data" (PMEDICINE-D-19-03864R2) has been accepted for publication in PLOS Medicine.

PRODUCTION PROCESS

Before publication you will see the copyedited word document (in around 1-2 weeks from now) and a PDF galley proof shortly after that. The copyeditor will be in touch shortly before sending you the copyedited Word document. We will make some revisions at the copyediting stage to conform to our general style, and for clarification. When you receive this version you should check and revise it very carefully, including figures, tables, references, and supporting information, because corrections at the next stage (proofs) will be strictly limited to (1) errors in author names or affiliations, (2) errors of scientific fact that would cause misunderstandings to readers, and (3) printer's (introduced) errors.

If you are likely to be away when either this document or the proof is sent, please ensure we have contact information of a second person, as we will need you to respond quickly at each point.

PRESS

A selection of our articles each week are press released by the journal. You will be contacted nearer the time if we are press releasing your article in order to approve the content and check the contact information for journalists is correct. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact.

PROFILE INFORMATION

Now that your manuscript has been accepted, please log into EM and update your profile. Go to https://www.editorialmanager.com/pmedicine, log in, and click on the "Update My Information" link at the top of the page. Please update your user information to ensure an efficient production and billing process.

Thank you again for submitting the manuscript to PLOS Medicine. We look forward to publishing it.

Best wishes,

Richard Turner, PhD

Senior Editor

PLOS Medicine

plosmedicine.org