Diagnostic accuracy of metagenomic next-generation sequencing for active tuberculosis in clinical practice at a tertiary general hospital.

BACKGROUND: To evaluate the diagnostic accuracy of metagenomic next-generation sequencing (mNGS) for active tuberculosis (TB).
METHODS: We retrospectively collected 820 samples at Zhongshan Hospital, Fudan University in Shanghai, China, between 1 April 2017 and 31 March 2018. They were classified into TB cases (125, 15.2%) and NOT TB cases (695, 84.8%) according to the clinical diagnosis. Specimens were evaluated by a regular clinical microbiological assay and mNGS performed in parallel.
RESULTS: Sixty-one confirmed TB cases and 64 clinical TB cases were included. The overall sensitivity of mNGS was 49.6% [95% confidence interval (95% CI), 40.6-58.6%], and the specificity was 98.3% (95% CI, 96.9-99.1%), with peak sensitivities of 88.9% (95% CI, 50.7-99.4%) for lung tissue, 55.0% (95% CI, 32.0-76.2%) for bronchoalveolar lavage fluid (BALF), and 50.0% (95% CI, 32.8-67.2%) for serous fluids. The overall sensitivity of mNGS was superior to that of the culture assay (35.2%, 95% CI, 27.0-44.3), but no superior sensitivity for sputum was observed in mNGS compared with the culture assay (mNGS: 52.3%, 95% CI, 31.1-72.6%; culture: 60.9%, 95% CI, 38.8-79.5%). In clinical TB cases, mNGS detected additional positive results (40.6%, 26/64). mNGS reduced the turnaround time from 2-6 weeks to 32-36 hours.
CONCLUSIONS: mNGS may be a promising technology for the early auxiliary diagnosis of active TB, especially sputum-negative pulmonary TB (PTB) and tuberculous serous effusion. 2020 Annals of Translational Medicine. All rights reserved.

Introduction

Tuberculosis (TB) is the tenth leading cause of death worldwide and the leading cause of death from a single infectious agent, ranking above HIV/AIDS. In 2017, approximately 10.0 million new TB patients, including an estimated 1.3 million HIV-negative and 0.3 million HIV-positive individuals, died of the disease worldwide. Due to its large population, China had a total of 889,000 new cases in 2017, although the incidence rate was only 63/100,000, ranking as one of the top three countries for the total TB incidence (1).

Early and accurate detection of Mycobacterium tuberculosis (MTB) is important for TB diagnosis, immediate treatment and prevention of TB transmission (2). Despite advances in diagnostics, a considerable proportion of TB cases reported to the World Health Organization (WHO) are still clinically diagnosed rather than bacteriologically confirmed. Classic diagnostic tests for TB include acid-fast bacilli (AFB) smears and mycobacterial culture. AFB tests have limitations in terms of low sensitivity and confusion with nontuberculous Mycobacteria (3). Although liquid mycobacterial culture can reduce the median turnaround time and increase sensitivity, up to 2–6 weeks is still required to provide results (4). Because microscopy fails to allow species identification, and culture-based workflows for the diagnosis of TB remain slow, technically demanding. Therefore, we need fast and accurate methods for diagnosing TB. Xpert MTB/RIF is a nucleic acid amplification test recommended by the WHO to diagnose TB that can identify MTB and rifampin (RIF) resistance simultaneously in 2 hours through probes complementary to the rpoB gene (5). Previous studies have shown the high sensitivity and specificity of the Xpert MTB/RIF assay for the detection of TB and RIF resistance; nonetheless, it has limited detection efficacy in some cases of extrapulmonary TB (EPTB) (6-8).

In general hospitals, many difficult miscellaneous diseases can affect various systems and organs, which creates difficulty in diagnosis and differential diagnosis. Rapid and accurate diagnostic techniques for pathogens are urgently needed. Metagenomic next-generation sequencing (mNGS) is an emerging rapid test for the diagnosis of different pathogens, including MTB, at the same time (9,10). However, the clinical application of mNGS in the diagnosis of TB has mostly been reported in case reports (11,12). Our study aimed to evaluate the diagnostic accuracy of mNGS in pulmonary TB (PTB) and EPTB in clinical practice. We present the following article in accordance with the STARD reporting checklist (available at http://dx.doi.org/10.21037/atm-20-2274).

Methods

Study subjects

We retrospectively reviewed 820 samples from patients with different diseases, including infectious and non-infectious diseases, at Zhongshan Hospital in Shanghai, China, between 1 April 2017 and 31 March 2018. Specimens were subjected to regular clinical microbiological assays and mNGS testing (BGI China) in parallel. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). An ethical review application was validated by the Ethical Review Committee of Zhongshan Hospital, Fudan University, Shanghai, China (approval No. B2017-193R). Written informed consent was obtained for all patients or their legal designees.

Case definitions of TB

A case of TB refers to a patient who presents with symptoms, signs, images, microbial results or pathology suggestive of TB, where a clinician has diagnosed TB and decided to treat the patient with a full course of TB therapy (13).

Bacteriologically confirmed TB (confirmed TB) refers to a case of TB with MTB identified from a clinical specimen, either by culture or by a newer method such as Xpert MTB/RIF or TB nucleic acid PCR.

Clinical TB refers to a case of TB without the M. tuberculosis complex (MTBC) identified.

PTB refers to a case of TB involving the lung parenchyma (13).

EPTB refers to a case of TB involving organs other than the lungs, e.g., the pleura, lymph nodes, abdomen, genitourinary tract, skin, joints and bones, or meninges. Intrathoracic TB lymphadenopathy (mediastinal and/or hilar) or TB pleural effusion, without radiographic abnormalities in the lungs, constitutes a case of EPTB.

Not tuberculosis (NOT TB) refers to a case with an established alternative diagnosis.

mNGS and analysis

Different samples [sputum, bronchoalveolar lavage fluid (BALF), serous fluid, blood, tissue homogenates, etc.] were collected from patients according to standard procedures. The process of mNGS consisted of sample processing, nucleic acid extraction, library generation, and bioinformatic pipeline analysis as described previously (14). Extracted DNA was fragmented to yield 150- to 200-bp fragments, and DNA fragments were constructed through an end-repair method. Quality-qualified libraries were sequenced on the BGISEQ-100 platform. High-quality sequencing data were generated by removing low-quality and short (length <35 bp) reads, followed by computational subtraction of human host sequences mapped to a human reference genome (hg19) using Burrows-Wheeler alignment. Nonhuman sequences were mapped to classification reference databases downloaded from NCBI (ftp://ftp.ncbi.nlm.nih.gov/genomes/). RefSeq contains 2,700 whole-genome sequences of viral taxa, 1,494 bacterial genomes, 73 genomes of fungi related to human infection, and 47 genomes of parasites associated with human diseases.

Criteria for a positive MTB mNGS result

The MTBC includes M. tuberculosis, M. canettii, M. africanum, and M. bovis in the mNGS database. TB was considered positive with at least one read mapped to the MTBC (number of sequences strictly mapped at the genus level, SMRNG ≥1). We evaluated the accuracy of mNGS for TB diagnosis.

Statistical analysis

Comparative analysis was conducted by Pearson’s test, Fisher’s exact test or the McNemar test for discrete variables where appropriate. Data analysis was performed with SPSS 22.0. P values less than 0.05 were considered significant, and all tests were two tailed.

Results

Sample and patient characteristics

A total of 820 samples were included in the analysis, with 125 samples (15.2%) from TB cases and 695 samples (84.8%) from NOT TB cases (). The distribution of all sample types is listed in , and the distribution of TB cases is shown in . Of all the TB cases, the number of confirmed TB cases was 61, and the number of clinical TB cases was 64. Pulmonary samples including sputum, BALF and lung tissue accounted for 58.2% of all the samples (477/820). Other common sample types included blood, pleural fluid, pus, and extrapulmonary tissue. In TB cases, extrapulmonary samples accounted for most samples [57.6% (72/125)]; specifically, 34 serous fluid samples (27.2%), 22 extrapulmonary tissue samples (17.6%), 10 pus samples (8.0%), and 6 other samples [3 blood samples, 2 secretion samples, and 1 cerebrospinal fluid (CSF) sample] were included. The pulmonary samples included 23 sputum samples (18.4%), 21 BALF samples (16.8%), and 9 lung tissue samples (7.2%). The characteristics of the TB cases are listed in .

Figure 1

Flowchart of sample selection and classification. In total, 820 samples were divided into TB and NOT TB on the retrospective diagnosis of the corresponding patients. All samples were examined for the concordance analysis of mNGS and culture technique. mNGS+ refers to at least one read mapped to genus level (SMRNG ≥1). TB, tuberculosis; mNGS, metagenomic next-generation sequencing.

Figure 2

The distribution of total samples of present study and distribution of samples of TB cases. BALF, bronchoalveolar lavage fluid; CSF, cerebrospinal fluid.

Table 1

Characteristics of tuberculosis cases

Characteristics	Confirmed TB	Clinical TB
Age, year, mean [range]	45.7 [15–73]	45.1 [14–79]
Sex, female, No. (%)	17 (27.9)	17 (26.6)
Site of TB, No. (%)
Only PTB	21 (34.4)	15 (23.4)
Only EPTB	16 (26.2)	30 (46.9)
PTB + EPTB	24 (39.3)	19 (29.7)
ESR, mm/H (reference range, 0–22)	48.5 [5–120]	41.8 [2–103]
CRP, mg/L (reference range, 0–3.0)	44.9 [0.3–222.1]	38.6 [0.3–122.6]
T-SPOT.TB A	30 [0–82]	30 [1–71]
T-SPOT.TB B	47 [0–150]	34 [0–114]

TB, tuberculosis; ESR, erythrocyte sedimentation; CRP, C-reactive protein.

Turnaround times for different test methods

The turnaround time for mNGS was compared with the times for AFB smears and cultures on liquid medium (the MGIT 960 system). The mNGS turnaround time in our study was 32–36 hours, which was significantly shorter than that of culture on liquid medium (median 34 days, 14–55 days). Although this turnaround time was not shorter than that of AFB smears (median 1 day, 0–1 days), AFB smears had very low sensitivity (positive for only 7 of 125 samples).

Diagnostic performance comparison of mNGS and a culture assay

As shown in , the overall sensitivity of mNGS was 49.6% [95% confidence interval (95% CI), 40.6–58.6%], which was superior to that of the culture assay (35.2%, 95% CI, 27.0–44.3), and the overall specificity was 98.3% (95% CI, 96.9–99.1%). Although the sensitivity for pulmonary samples was much higher than that for extrapulmonary samples (58.5%, 95% CI, 44.2–71.6% versus 43.1%, 95% CI, 31.6–55.2%, respectively), no superiority was noted for the sputum between mNGS and the culture assay (52.3%, 95% CI, 31.1–72.6% versus 60.9%, 95% CI, 38.8–79.5%, respectively). Peak sensitivities were observed in lung tissues (88.9%; 95% CI, 50.7–99.4%), BALF (55.0%; 95% CI, 32.0–76.2%), and serous fluids (50.0%; 95% CI, 32.8–67.2%). The sensitivities of pus and non-lung tissues were similar for the culture assay (50.0%; 95% CI, 20.1–79.9% and 40.9%; 95% CI, 21.4–63.3%, respectively). MTB was detected in 12 samples from NOT TB cases (listed in ), in which active TB at the relevant site was excluded.

Table 2

The diagnosis performance of mNGS and culture for diagnosis of tuberculosis

Sample type	Test	Sensitivity				Specificity
		Value	95% CI	n/N		Value	95% CI	n/N
Total (n=820)	mNGS	49.6%	40.6–58.6	62/125		98.3%	96.9–99.1	683/695
	Culture	35.2%	27.0–44.3	44/125		100%	99.3–100	695/695
Pulmonary samples (n=477)	mNGS	58.5%	44.2–71.6	31/53		98.3%	96.5–99.3	417/424
	Culture	41.5%	28.4–55.8	22/53		100%	98.9–100	424/424
Sputum (n=292)	mNGS	52.3%	31.1–72.6	12/23		98.5%	96.0–99.5	265/269
	Culture	60.9%	38.8–79.5	14/23		100%	98.2–100	269/269
BALF (n=131)	mNGS	55.0%	32.0–76.2	11/21		98.2%	93.0–99.7	108/110
	Culture	28.6%	12.2–52.3	6/21		100%	95.8–100	110/110
Lung tissue (n=54)	mNGS	88.9%	50.7–99.4	8/9		97.8%	86.7–99.9	44/45
	Culture	22.2%	3.0–59.8	2/9		100%	90.2–100	45/45
Extrapulmonary samples (n=343)	mNGS	43.1%	31.6–55.2	31/72		98.2%	95.5–99.3	266/171
	Culture	30.6%	20.5–42.7	22/72		100%	98.3–100	271/271
Serous fluid (n=71)	mNGS	50.0%	32.8–67.2	17/34		97.3%	84.2–99.9	36/37
	Culture	23.5%	11.4–41.6	8/34		100%	88.3–100	37/37
Pus (n=61)	mNGS	50.0%	20.1–79.9	5/10		98.0%	88.2–99.9	50/51
	Culture	50.0%	20.1–79.9	5/10		100%	91.3–100	51/51
Extrapulmonary tissue (n=45)	mNGS	40.9%	21.4–63.3	9/22		100%	82.2–100	23/23
	Culture	40.9%	21.4–63.3	9/22		100%	82.2–100	23/23
Blood (n=102)	mNGS	0.0%	0–69.0	0/3		100%	95.3–100	99/99
	Culture	0.0%	0–69.0	0/3		100%	95.3–100	99/99
Others (n=64)	mNGS	0.0%	0–69.0	0/3		95.1%	85.4–98.7	58/61
	Culture	0.0%	0–69.0	0/3		100%	92.6–100	61/61

Others: including 17 CSF, 16 swab, 10 secretion, 10 bone marrow, 4 bile, 4 urine, 2 paraffin section and 1 saliva. mNGS, metagenomic next-generation sequencing.

Table 3

List of “false positive” in NOT TB cases

Sample No.	Diagnosis	Sample type	T-SPOT A/B	ESR (mm/H)	CRP (mg/L)	SMRN	SMRNG
S145	Lung adenocarcinoma	BALF	0/0	30	0.4	3	1
S139	Bronchial cyst combined with infection	BALF	4/0	72	40.4	1	1
S233	Fever of unknown origin: amyopathic dermatomyositis	Throat swab	0/0	30	0.4	1	1
S316	Organized pneumonia	Sputum	7/0	73	118.8	6	1
S425	Hydropneumothorax, lung infection	Sputum	0/0	66	85.7	6	1
S445	Lung infection	Sputum	0/0	29	1.2	3	1
S483	Organized pneumonia, postoperative lung cancer	Sputum	10/0	93	76.7	25	1
S501	Pulmonary cryptococcosis	Throat swab	0/0	19	0.3	5	1
S643	Chronic lung infection after run-over injury	Lung tissue	0/0	10	23.1	1	1
S720	Plasmacytoma	Pleural fluid	0/4	93	84.1	1	1
S757	Nocardiosis (lung, brain)	CSF	0/0	113	87	6	2
S767	Fever of unknown origin, EBV− DNA+	Bone marrow	0/2	118	59.2	3	1

TB, tuberculosis; ESR, erythrocyte sedimentation; CRP, C-reactive protein; SMRN, number of sequences strictly mapped at species level; SMRNG, number of sequences strictly mapped at the genus level; BALF, bronchoalveolar lavage fluid; CSF, cerebrospinal fluid.

Concordance between mNGS and the culture assay

mNGS and the culture assay were both positive for 26/125 samples (20.8%) and both negative for 45/125 samples (36.0%), while 36 samples (28.8%) were positive by mNGS only and 18 samples (14.4%) were positive by the culture assay only. Positivity differed among sample types, with more mNGS positivity found in BALF, lung tissue and serous fluid ().

Figure 3

The concordance rate of mNGS and culture was 56.8%, and additional 28.8% mNGS positive were detected in culture negative samples and it differed in sample types. mNGS, metagenomic next-generation sequencing. Others included 17 CSF, 16 swab, 10 secretion, 10 bone marrow, 4 bile, 4 urine, 2 paraffin section and 1 saliva. BALF, bronchoalveolar lavage fluid; mNGS+, metagenomic next-generation sequencing positive for Mycobacterium tuberculosis complex; P, pulmonary sample; EP, extrapulmonary samples; tissue-P, lung tissue; tissue-EP, extrapulmonary tissue; BALF, bronchoalveolar lavage fluid; mNGS, metagenomic next-generation sequencing.

Comparison of mNGS analysis between confirmed TB and clinical TB

We had 61 confirmed TB cases and 64 clinical TB cases. The MTB positivity of mNGS was higher in confirmed TB than in clinical TB (59.0% versus 40.6%, respectively, P<0.05), and a similar trend was observed among various sample types, although without a significant difference. In clinical TB without conclusive aetiological evidence, mNGS substantially improved the detection of positive cases by 40.6%, performing exceptionally in tissue and pus (50.0%) ().

Figure 4

Comparison of mNGS in confirmed TB and clinical TB. The MTB positivity of mNGS in confirmed TB versus in clinical TB were 59.0% and 40.6% respectively, and similar trend was detected in sputum serous fluid, BALF and tissue. *, P<0.05 statistic difference. BALF, bronchoalveolar lavage fluid; TB, tuberculosis; mNGS, metagenomic next-generation sequencing; MTB, Mycobacterium tuberculosis.

The number of unique reads of the MTBC by mNGS ranged from 1 to 3,966, and the coverage ranged from 0.01% to 99.4%. Because of the difficulty of DNA extraction, the read numbers mapped to the MTBC (SMRNG) were mostly small, and almost 50% of cases had fewer than 5 reads. No significant difference in SMRNG number was observed between confirmed TB and clinical TB ().

Figure 5

Specific reads of MTBC in TB cases: reads number mapped to the MTBC (SMRNG) were mostly low, while almost 50% of cases were less than 5 reads. There was no significant difference of reads number between confirmed TB and clinical TB. TB, tuberculosis; MTBC, Mycobacterium tuberculosis complex.

Discussion

In the present study, we retrospectively reviewed TB cases with both mNGS and a culture assay performed in a pairwise manner with different sample types to illustrate the feasibility of mNGS for early diagnosis of PTB and EPTB. mNGS produced a higher sensitivity for MTB than the culture assay (49.6% and 35.2%, respectively), and a high specificity of 98.3%. The turnaround time could be shortened from 2–6 weeks to 32–36 hours and is currently reduced to almost 24 hours due to the localization of sequencing in our hospital.

Moreover, our results showed that mNGS was not advantageous with sputum, but a relatively higher sensitivity was detected for lung tissue and BALF, implying that mNGS performed better in culture-negative PTB than in culture-positive PTB, which is relatively difficult to diagnose. The overall sensitivity of mNGS in detecting MTB in EPTB appeared to be superior to that of the culture assay and not inferior to that of Xpert MTB/RIF (6-8). However, mNGS was superior to Xpert MTB/RIF in serous fluid evaluation in most studies (5). Due to the low sensitivity of the culture assay itself, the concordance between mNGS and the culture assay was not high, but mNGS could detect MTB more frequently in culture-negative specimens, which can provide more aetiological evidence in clinical practice.

MTB was not detected in 47 CSF samples, probably because of the low proportion of definite/probable tubercular meningitis. Wang et al. reported that mNGS is a good method to detect MTB in CSF samples from patients with tubercular meningitis (15). Unfortunately, in our study, the diagnostic value of mNGS in tubercular meningitis was difficult to evaluate, as we need more meningitis cases. The MTBC was not detected in blood samples. Detection of the MTBC in the blood of a recently diagnosed disseminated TB patient in our department suggested the value of peripheral blood mNGS for disseminated TB.

The MTBC was detected at the species level in 12/695 (1.5%) NOT TB cases; however, no previous data for mNGS in latent tuberculosis infection (LTBI) or other infections have been reported. Moreover, the possibility of strong positive pollution in the same batch on our NGS platform complicates explaining whether this result is a false positive or indicative of contamination.

The concordance between culture and mNGS was 56.8%, and mNGS additionally detected sequences of the MTBC in some culture-negative samples. A positive mycobacterial culture requires approximately 100 organisms per mL, and culture has reduced sensitivity in paucibacillary samples (such as serous fluid, BALF, and tissue homogenate) and samples from patients who previously received anti-TB therapy (14,16,17).

As MTB is an intracellular mycobacterial pathogen, the read number of the MTBC detected was very small, with fewer than 5 reads in half of the cases due to the difficulty of DNA extraction. Theoretically, the read number of mNGS should be positively correlated with the amount of bacteria in a specimen; while no correlation was found between MTB positivity by mNGS and bacterial confirmation, positive AFB smears, or the MTB culture period. The preprocessing procedure should be optimized to improve the application of mNGS for intracellular pathogens such as Mycobacterium, Brucella, and fungi (9). Even if only one MTBC sequence is detected, TB still needs to be highly suspected (12,14).

Some limitations exit in this study. First, this was a retrospective study with inappropriate patient selection. Second, bacterially confirmed cases were mostly evaluated by culture, and only a few cases were PCR positive or Xpert MTB/RIF positive. We could not simultaneously compare mNGS with other rapid detection methods, especially Xpert MTB/RIF, which the WHO recommends as a confirmatory test. Third, our tests were delivered to a centralized laboratory, and DNA degradation may have occurred because of inappropriate specimen delivery.

Overall, most of the cases in our study were atypical PTB or difficult-to-diagnose cases affecting various systems, and mNGS determination of MTB positivity may provide a prompt clinical diagnosis and a direction for clinical strategies. In conclusion, mNGS may be a promising technology for early diagnosis of active TB, especially sputum culture-negative PTB and tuberculous serous effusions.

The article’s supplementary files as