Comparative evaluation of Xpert MTB/RIF and the new Xpert MTB/RIF ultra with respiratory and extra-pulmonary specimens for tuberculosis case detection in a low incidence setting.

BACKGROUND: The Xpert MTB/RIF assay (Xpert) is an automated molecular test for the detection of tuberculosis and rifampin resistance (RIF-R), but it lacks sensitivity in smear-negative samples and some limitations in determination of RIF-R have also been reported. The new Xpert MTB/RIF Ultra (Ultra) was developed to overcome these limitations. We aimed to compare Ultra and Xpert diagnostic accuracy setting culture and drug susceptibility testing as reference standards.
METHODS: A retrospective analysis was performed on 359 consecutive, respiratory (269) and extrapulmonary (90) specimens collected from 340 patients investigated for TB along a two-year period. Patients presenting at primary health-care centres and hospitals were recruited on the basis of symptoms and abnormal X-ray imaging. One-hundred seventy-four subjects were identified to have active tuberculosis by culture and 2 were MDR.
FINDINGS: Sensitivities of Ultra and Xpert were 87% and 75% for the 48 individuals with smear-negative and culture-positive respiratory TB (difference of 12%, 95% CI 3 to 21); 95% and 72% for the 40 individuals with smear-negative and culture-positive extrapulmonary disease (22%, 95% CI 10 to 34); and 95% and 86%, respectively, across all 174 individuals with culture-positive samples (8.5%, 95% CI 4.5 to 12.5). Specificities of Ultra and Xpert for tuberculosis case detection were 98% and 100% (-2.0%, 95% CI -4.3 to +0.3). Ultra and Xpert performed equal in detecting RIF-R.
INTERPRETATION: Sensitivity of Ultra was superior to that of Xpert in all categories of clinical samples. However, improved sensitivity was associated with a modest reduction in specificity.

Introduction

Rapid diagnosis of tuberculosis (TB) and determination of resistance is essential to adequately manage the disease and control its spread. Since the introduction of molecular methods, several kit-based nucleic acid amplification tests (NAAT) able to detect and identify M. tuberculosis complex (MTB) from clinical specimens within few hours were developed. Subsequently, a number of selected NAATs have been validated for use with smear-positive and (to a lesser extent) smear-negative respiratory and extrapulmonary samples [1]. In general, the routine application of these amplification systems allows an accurate diagnosis of TB but requires laborious hands-on processing and a dedicated biosafety setting [2].

The Xpert MTB/RIF (Xpert) assay (Cepheid, Sunnyvale, California) is an automated, cartridge-based, real-time heminested PCR test that works in conjunction with a software-driven modular platform. Simultaneous detection of MTB and rifampin resistance (RIF-R) is obtained in less than 2 h by amplification of the rifampin resistance-determining region of the rpoB gene [3] and subsequent probing of this region for resistance-associated mutations. From a practical point of view, Xpert has some key advantages over conventional NAATs: a) it is simple to perform, b) may be used on-demand rather than by the conventional batched testing, c) needs little training, d) requires minimal biosafety facilities and e) is not prone to cross-contamination. Since the very first appearance in the Literature, Xpert was introduced as the non plus ultra molecular test for the diagnosis of TB and RMP-R. When tested in high-incidence settings with respiratory samples whose ratio of smear-positive to smear-negative specimens was frequently ≥ 2, Xpert demonstrated to be highly accurate exhibiting a pooled sensitivity and specificity of 88 and 99% respectively [4], [5], [6], [7], [8]. However, the impact of this new technology in high-resource, low-incidence setting with a prevalence of paucibacillary disease and full mycobacteriology capabilities was shown to be quite different. Although the results for smear-positive samples were within the range observed in high-incidence settings, sensitivity of the Xpert for smear-negative samples was found to be considerably lower than that previously reported ranging from 50% to 75% [9], [10], [11] and substantially comparable with other commercially available NAA assays [12], [13]. Moreover, the assay was noted to have a limited capacity to detect RIF-R associated mutations in mixed samples [14] and false-positive results were also observed [15], [16].

To overcome the above limitations a new generation assay for TB and RIF-R detection has been developed and commercialized as Xpert MTB/RIF Ultra assay (Ultra). Ultra incorporates two different multicopy amplification targets (IS6110 and IS1081) and uses a renewed assay chemistry coupled with a larger DNA reaction tube. These changes have enabled the assay to detect 16 CFU/ml, in contrast to the 114-CFU/ml limit of detection exhibited by Xpert [17]. In addition, to improve the accuracy of RIF-R detection Ultra incorporates a melting temperature-based analysis instead of real-time PCR [18]. Preliminary evaluations of Ultra suggest that it seems to offer a greatly improved sensitivity compared with that of Xpert in smear-negative, culture-positive TB and HIV-associated TB meningitis [17], [19], [20], [21]. However, the improved sensitivity appears to be associated with a reduction in specificity which requires further evaluation. We performed a retrospective study aimed to assess the performance of both assays in the setting of a routine tertiary care centre and reference TB laboratory particularly addressing sensitivity in paucibacillary, smear-negative, culture-positive specimens and specificity in non-TB patients.

Materials and methods

Study design

We aimed to estimate and compare the sensitivity of a single Ultra test with that of a single Xpert test of the same specimen for detection of smear-negative TB and RIF-R, and to evaluate also differences in specificity.

A retrospective analysis was carried out at the United Hospitals, Ancona – Italy, which provide general medical and surgical services and act as a tertiary referral centre for many medical specialties including infectious diseases and TB. The local protocol specifies use of Xpert to test clinical samples that have stained smear-positive, investigate patients with soundly suspected TB, and on a ‘case-by-case’ basis where there is a justifiable rationale. Demographic information, the patient's medical history, signs, symptoms, chest X-ray/Computed tomography, histology and microbiology results were recorded aiming to set culture and drug susceptibility testing as reference standards. In addition, our mycobacteriology lab is the only one in the Marche region serving a population of one million, five-hundred thousand people and receiving clinical samples of suspected TB cases from different hospitals. All the above selected samples were marked as priority and underwent amplification within 24 h of specimen receipt. Samples collected from patients receiving antituberculous therapy or previously diagnosed as having TB were excluded from the analysis. Three-hundred, fifty-nine consecutive respiratory (269) and extrapulmonary (90) specimens collected from 340 patients (216 males and 124 females) submitted between January 2016 and December 2017 were included in this study. This study was retrospective without interaction with patients and all patients’ information was de-identified prior to analysis. Therefore, institutional ethics committee approval was not to be required.

Specimen collection and processing

Investigated specimens included 269 respiratory specimens collected from 254 patients (124 bronchial aspirates, 87 sputa, 58 bronchoalveolar lavage fluids) and 90 extrapulmonary collected from 86 patients (5 gastric aspirates, 2 urine, 5 sterile body fluids (2 pleural, 2 ascites, and 1 synovial) 6 cerebrospinal [CSF] fluids, 13 fine needle aspirates, 16 pus, and 40 biopsies). Urine, gastric aspirates (which were neutralized upon receipt with 0.067 M phosphate buffered saline of pH 6.8[PBS]), pleural and other similar body fluids (pericardial, synovial and ascites), were centrifuged at 3500 x g for 15 min at 4 °C. The supernatant was discarded and the pellet resuspended in 5 ml of sterile saline. Lymphnode and biopsy samples were homogenized by a tissue grinder and resuspended in 5 ml of sterile saline. After the above procedures, all specimens were liquefied and decontaminated by the standard N-acetyl-L-cysteine, Sodium Hydroxide (NALC-NaOH) method [22]. CSF was centrifuged at 12,000 x g for 10 min. The pellet was resuspended in PBS and treated without prior decontamination. Part of the sediment from each specimen was inoculated onto the culture media and used for acid-fast staining, while 0.5 ml was aliquoted and stored at 4 °C until Xpert test was performed. A further aliquot sample with a total remaining volume of 0.8–1 ml was stored at -80 °C for future use.

Microscopy

To detect AFB, smears from clinical samples were stained by the Z-N stain and examined under the oil immersion objective lens of the microscope (x 1000).

Culture

A 0.5-ml portion of the sediment was inoculated into a MGIT tube (Becton-Dickinson Diagnostic Instrument Systems, USA) to which an enrichment supplement (OADC) and an antibiotic mixture (PANTA) were added and onto Löwenstein–Jensen (LJ) solid medium. The MGIT tubes were placed into the BACTEC MGIT 960 system, which is a fluorescence-based detection instrument [23]. Liquid cultures were continuously monitored for bacterial growth for 6 weeks or until flagged as positive by the instrument. Any sample identified as positive was removed from the instrument and a smear was prepared and examined for AFB. LJ slants were incubated for 8 weeks at 36 ± 1 °C and inspected weekly for growth. Acid-fastness from suspect colonies was confirmed by the Ziehl-Neelsen (Z-N) staining.

Identification of mycobacteria and drug susceptibility testing

Isolates were identified by specific DNA probes assays (Accuprobe Hologic Inc., San Diego, USA and Genotype Mycobacteria, Hain Lifescience, Nehren, Germany) [22]. Phenotypic drug susceptibility testing was done from the first positive MTB culture using the BACTEC MGIT 960 system and a rifampicin critical concentration of 1 μg/mL. [22].

Xpert and ultra assays

Starting from a 0.5 ml sample portion, the procedure consisted of two steps: specimen preparation and combined, fully automated amplification and detection. During preparation, sample was diluted with a sample reagent (SR) at a ratio of 3:1 [3], vortexed and incubated for 15 min at room temperature. Two milliliters of the mixture was then transferred into the Xpert cartridge and loaded into the four-module GeneXpert instrument with automated readouts for MTB detection (invalid; not detected; or detected [with semiquantitation]) and rifampicin resistance (detected, not detected, or indeterminate). To compare the sensitivity of Ultra versus Xpert frozen samples were thawed to room temperature and vortexed for 15 s. According to the manufacturer's package insert, 0.5 mL of the total resuspended pellet and 1.5 ml of SR were mixed and incubated for 15 min at room temperature. Then the mixture was transferred into the Ultra cartridge and loaded into the Xpert instrument. The semiquantitative scale for Xpert results was: very low, low, medium, or high. The semiquantitative scale for Xpert Ultra results was: trace, very low, low, medium, or high [18]. Staff doing Ultra tests were blinded to Xpert results.

Statistical analysis

Sensitivity was defined as the proportion of patients testing positive with the reference standard who tested positive by the index test (Ultra) or comparator test (Xpert). Specificity was the proportion of patients testing negative with the reference standard who tested negative by the index test or comparator test. Sensitivity and specificity of the assays with 95% confidence intervals (95% CI) were determined according to the method described by Newcombe [24]. Tests without interpretable results (eg. Ultra/Xpert error and invalid) were counted as negative results. In case of discrepant results between NAATs and culture, a repeat testing was carried out.

Results

Respiratory samples

Respiratory samples included 123 - 75 (61%) AFB smear-positive and 48 (39%) smear-negative - MTB culture-positive samples (Table 1). One hundred forty-six were non-TB samples. Of these, 131 (90%) were smear-negative, culture negative and from 15 smear-positive samples (10%) NTM were grown. Among the 75 smear- and culture-positive samples, all of them were positive by Ultra and Xpert assays. Semiquantitative readouts showed a moderate shift toward higher categories in those specimens tested with Ultra (Table 2). Within the group of 48 smear-negative, culture-positive samples, Ultra detected MTB in 41, while 32 were identified as positive as by Xpert. Sensitivities of Ultra and Xpert for this category of samples were 87% and 75%, respectively (difference of 12%, 95% CI 3 to 21). Out of the 16 samples flagged as not detected by Xpert, 7 confirmed this readout with Ultra while 3 flagged trace and 6 showed a readout of very low. Among the 131 smear and culture-negative samples, Ultra identified 128 as negative (specificity: 98%) while Xpert yielded a specificity value of 100%. All false-positive readouts (n = 3) fell into the category of trace. Within the subgroup of 15 smear and culture-positive samples with NTM, the following species were isolated: M. avium (10), M. intracellulare (2), M. abscessus (1), M. kansasii (1), and M. xenopi (1). No false-positive MTB result was produced by any assay. In 2 samples RIF-R was detected by both assays. Resistance was subsequently confirmed by the phenotypic drug susceptibility testing. Finally, non-determinate readouts (invalid, error, no result) were obtained for 2 (0.7%) specimens with Xpert and for 3 (1.1%) specimens with Ultra.

Table 1

Respiratory specimens: comparison of Ultra vs. Xpert assays with AFB smear and culture results.

Type of specimen	No. of specimens	No. of specimens positive with
		Ultra	Xpert
Smear positive and MTB culture positive	75	75	75
Smear negative and MTB culture positive	48	41	32
Smear positive and NTM culture positive	15	0	0
Smear and culture negative	131	3	0
All categories	269	119	107

Table 2

Respiratory specimens: comparison of Ultra vs. Xpert semiquantitative readouts with AFB smear and culture results.

No. of specimens with ultra		Specimens with	No. of specimens with Xpert
Not detected		Smear positive growth of MTB (no. = 75)		Not detected
Trace
Very low				Very low
Low	1		8	Low
Medium	41		32	Medium
High	33		35	High
Total	75		75	Total
Not detected	7	Smear negative growth of MTB (no. = 48)	16	Not detected
Trace	5
Very low	20		8	Very low
Low	11		22	Low
Medium	5		2	Medium
High				High
Total	48		48	Total
Not detected	15	Smear positive growth of NTM (no. = 15)	15	Not detected
Trace
Very low				Very low
Low				Low
Medium				Medium
High				High
Total	15		15	Total
Invalid/Error	3	Smear negative no growth of AFB (no. = 131)	2	Invalid/Error
Not detected	125		129	Not detected
Trace	3
Very low				Very low
Low				Low
Medium				Medium
High				High
Total	131		131	Total

Extrapulmonary samples

For the 90 extrapulmonary specimens, all those smear and culture positive (n = 11) were positive by both assays. Of the 40 smear-negative and culture-positive samples, 38 were detected by Ultra, and 29 were picked up by Xpert (Tables 3 and 4). Distribution of specimens by clinical source and the results of Ultra vs. Xpert assays compared with MTB culture are shown in table 5. Sensitivities of Ultra and Xpert for this category of samples were 95% and 72%, respectively, (difference of 22%, 95% CI 10 to 34). Out of the 11 samples tagged as not detected by Xpert, 2 confirmed this readout with Ultra while 2 flagged trace, 6 showed a readout of very low and 1 the readout of low. Four smear-positive samples grew NTM (2 M. avium, 1 M. intracellulare, and 1 M. celatum). All these specimens were negative by both assays. Among the 35 smear- and culture-negative samples, 1 was positive by the Ultra assay, while no one was positive by Xpert. There was one invalid result using the Ultra assay, while 3 invalid results were found among samples tested with Xpert. When respiratory and extrapulmonary culture-positive samples were considered altogether, sensitivities of Ultra and Xpert were and 95% and 86%, respectively, while specificities for case detection were 98% and 100% (difference of –2.0%, 95% CI –4.3 to +0.3) for smear-negative culture positive TB and overall (Table 6).

Table 3

Extrapulmonary specimens: comparison of Ultra vs. Xpert assays with AFB smear and culture results.

Type of specimen	No. of specimens	No. of specimens positive with
		Ultra	Xpert
Smear positive and MTB culture positive	11	11	11
Smear negative and MTB culture positive	40	38	29
Smear positive and NTM culture positive	4	0	0
Smear and culture negative	35	1	0
All categories	90	50	40

Table 4

Extrapulmonary specimens: comparison of Ultra vs. Xpert semiquantitative readouts with AFB smear and culture results.

No. of specimens with ultra		Specimens with	No. of specimens with Xpert
Not detected		Smear positive growth of MTB (no. = 11)		Not detected
Trace
Very low				Very low
Low	1		2	Low
Medium	8		6	Medium
High	2		3	High
Total	11		11	Total
Not detected	2	Smear negative growth of MTB (no. = 40)	11	Not detected
Trace	5
Very low	14		6	Very low
Low	6		15	Low
Medium	13		6	Medium
High			2	High
Total	40		40	Total
Not detected	4	Smear positive growth of NTM (no. = 4)	4	Not detected
Trace
Very low				Very low
Low				Low
Medium				Medium
High				High
Total	4		4	Total
Invalid/Error	1	Smear negative no growth of AFB (no. = 35)	3	Invalid/Error
Not detected	33		32	Not detected
Trace	1
Very low				Very low
Low				Low
Medium				Medium
High				High
Total	35		35	Total

Table 5

Distribution of extrapulmonary specimens and comparison of Ultra vs. Xpert assays with MTB culture results.

Type of specimen	No. of specimens	No. of specimens positive with
		MTB culture	Ultra	Xpert
Biopsy samples	40	18	17	9
Pus samples	16	13	13	12
Fine needle aspirates	13	9	9	9
CSF samples	6	2	2	2
Gastric aspirates	5	3	3	2
Lymph nodes	3	1	1	1
Other samples	7	5	5	5
All specimens	90	51	50	40

Table 6

Test sensitivity and specificity depending on repeat testing of semiquantitative trace-positive results for MTB detection by Ultra.

	Sensitivity		Specificity
	All culture positive (95% CI, n/N)	Smear-negative, culture-positive (95% CI, n/N)	All culture negative (95% CI, n/N)
Xpert	86% (81–91, 147/174)	76% (67–85, 61/88)	100% (NA, 185/185)
Ultra	95% (92–98, 165/174)	90% (84–96, 79/88)	98% (96–100, 181/185)
Ultra (trace repeat)	95% (98–92, 165/174)	90% (84–96, 79/88)	100% (NA, 185/185)

Discussion

This paper focuses on the comparative evaluation of two amplification tests and their diagnostic accuracy under routine clinical conditions in a low-prevalence high-resource setting. Results of this study show that the sensitivity of Ultra was superior to that of Xpert for TB case detection in patients with smear-negative respiratory and extrapulmonary tuberculosis [25]. The sensitivities of 87 and 95% found for Ultra with smear-negative, culture-positive respiratory and extrapulmonary TB samples respectively were considerably higher than 63% reported by Dorman [19] and in the upper range of values reported by Chakravorty [17]. In a recently published paper, Perez-Risco et al. [26] have applied Ultra to 168 smear-negative extrapulmonary samples collected over a time-lag of 18-years. They obtained an average sensitivity of 75.9% and 100% specificity. Sensitivity was especially notable for lymph nodes, tissue samples and also body fluids which are generally regarded as suboptimal samples unfit to detect MTB – DNA. In our study, selection of patients driven by a sound clinical suspicion and the prevalence of specimens requiring invasive investigations which permitted to sample as close as possible to the site of disease (sputa represented less than 25% of the whole set of specimens), may be the likely explanation for this discrepancy. From a clinical standpoint, the high sensitivity of Ultra can facilitate diagnosis of pulmonary tuberculosis at earlier stages of disease and also facilitates diagnosis of extrapulmonary forms featured by an almost complete negativity of smear microscopy. It should be noted that the results obtained with tissues (biopsy) samples and abscess (fine needle) aspirates are similar and even better than those reported by Perez-Risco et al. [26] (Table 5). Although an empirical anti-tuberculosis treatment may be started on the basis of a positive smear microscopy or a sound clinical suspicion, nevertheless the use of Ultra test in suspect TB cases according to the ‘ruling-in’ strategy is likely to bring a considerable impact on disease management. Similarly, negative assay results while not excluding the diagnosis of TB also provided rapid support for clinical decision-making. This targeted approach used on a case-by-case basis after discussion with a consultant microbiologist is further supported by the evidence that showed a prohibitive expense for routine use of Ultra in place of microscopy and culture [27].

Unfortunately, the increase in sensitivity provided by Ultra came at the expense of a loss of specificity. To minimize the number of false-positive results, we excluded from the study individuals with a history of TB treatment [15] or recently diagnosed as having TB. We also excluded other causes responsible for a positive NAAT result in MTB culture-negative samples such as laboratory environment contamination or inappropriate (too harsh) decontamination. Although we were unable to detect which step along the lab workflow may have predisposed Ultra to an increase of false-positive results, it was observed that these results (4 out of 4) showed the semiquantitative result of trace. According to the residual pellet available, we performed a repeat testing for as many trace readout as possible including both true-positive and false-positive samples. Out of 7 repeat tests (3 true-positive and 4 false-positive), all true-positive samples confirmed positive results (2 trace and 1 very low readouts), while false-positive samples turned out as not detected. These data, albeit too small, suggest a reclassification of these samples as tuberculosis-negative according to the trace repeat test results on the same specimen or a new test from another specimen [17]. This procedure could reduce some loss of Ultra specificity while maintaining its considerable upgrade in sensitivity (Table 6). This approach seems to be supported by a recent paper by Kendall et al. [28], in which the number of inappropriate antituberculosis treatments would be greatly reduced if trace results are repeated and treated as positive only if the repeat result is trace or fully positive. We recorded excellent specificity (100%) of Ultra test within the sub-set of NTM samples. In countries with a growing percentage of NTM isolates, a high specificity value is indispensable in order to discriminate tuberculous from non-tuberculous bacteria in smear-positive samples. For detection of RIF-R in clinical samples, Ultra and Xpert showed comparable accuracies. However, the limited number of RIF-R isolates present in this study hindered a proper assessment of the efficacy of both assay in detecting resistance.

Finally, both assays can be run in the same self-contained, integrated instrument after a simple software upgrade. Thus, Ultra can be implemented with little additional training in sites that already use the Xpert assay.

In conclusion, our results suggest that Ultra test does appear to have a pivotal role in tuberculosis case detection and simultaneous RIF-R detection. Its excellent upgrade in sensitivity is particularly relevant in subjects with paucibacillary TB such as respiratory TB in HIV-coinfected, pediatric patients and those with extrapulmonary TB. In our opinion, this is an ‘add-on’ rather than a ‘replacement’ investigation and should be used as a test to ‘rule-in’ TB and screen for MDR-TB. Further studies are required to investigate the frequency and clinical relevance of false-positive Ultra results.