Performance of Wayne assay for detection of pyrazinamide resistance in Mycobacterium tuberculosis: a meta-analysis study.

Conventional culture-based drug susceptibility testing (DST) of Mycobacterium tuberculosis to pyrazinamide (PZA) is time-consuming and difficult to perform. The current systematic review and meta-analysis was aimed to evaluate the diagnostic accuracy of Wayne assay against culture-based DSTs as the reference standard. We searched the MEDLINE/Pubmed, Embase, and Web of Science databases for the relevant records. The QUADAS-2 tool was used to assess the quality of the studies. Diagnostic accuracy measures (i.e., sensitivity and specificity) were pooled with a random-effects model. Statistical analyses were performed with STATA (version 14, Stata Corporation, College Station, TX, USA), RevMan (version 5.3; The Nordic Cochrane Centre, the Cochrane Collaboration, Copenhagen, Denmark), and Meta-DiSc (version 1.4, Cochrane Colloquium, Barcelona, Spain). A total of 31 articles comprising data for 2457 isolates of M. tuberculosis were included in the final analysis. The pooled sensitivity and specificity of the Wayne assay against all reference tests (the combination of BACTEC MGIT 960, BACTEC 460, and proportion method) were 86.6% (95% CI: 84.3-88.7) and 96.0% (95% CI: 94.8-97). The positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR), and the area under the curve (AUC) estimates were found to be 17.6 (95% CI: 10.5-29.3), 0.11 (95% CI: 0.06-0.20), 164 (95% CI: 83-320) and 97%, respectively. Deek's test result indicated no evidence for publication bias (P > 0.05). Although the current study shows that the Wayne test is sensitive and specific for detecting PZA resistance, it may be used in combination with conventional DSTs to diagnose PZA resistance accurately.

Introduction

Tuberculosis (TB) caused by Mycobacterium tuberculosis is still an endemic disease in many low-income countries, and in 2019, an estimated 10 million people fell ill with TB [1]. The increasing rate of multidrug-resistant TB (MDR-TB) is one of the most critical threats to TB control [2]. Pyrazinamide (PZA) is an essential anti-TB drug that is frequently used in both first- and second-line treatment regimens [[3], [4], [5], [6]]. Although PZA is a cornerstone in the treatment of TB, its drug susceptibility testing (DST) is not routinely performed due to technical limitations (i.e., inoculum size, long turnaround time, an acidic culture medium, etc.) [[7], [8], [9], [10]].

The Bactec Mycobacterial Growth Indicator Tube 960 PZA (MGIT-PZA) is the most recently endorsed assay by the World Health Organization (WHO) for DST of PZA [11,12]. However, the reproducibility of this method is lacking [11]. The sensitivity and specificity of molecular tests are also varied (45% to 95%), and using this method also has been limited [11,13].

The Wayne test is a biochemical colorimetric test, which is inexpensive and simple to perform in low-income countries. The classic Wayne test has a 75.6 to 95.7% sensitivity and specificity of 88.7 to 97% [[14], [15], [16]].

Since a comprehensive evaluation of the Wayne test's accuracy is not available, the current study aimed to investigate the diagnostic accuracy of the Wayne test against culture-based susceptibility testing methods as the reference standard for DST of PZA.

Methods

This study was conducted and reported according to the PRISMA guidelines [17].

Search strategy and selection criteria

The MEDLINE/Pubmed, Embase, and Web of Science were searched for relevant studies published from January 2000 to December 2019. The combination of the following keywords was used: tuberculosis AND Pyrazinamide.

Bibliographies of all included articles were also reviewed to ensure that no related papers have been missed. The references of previous meta-analyses and systematic reviews were checked as well to supplement the databases. Only English studies were included.

Studies were included if they compare a Wayne assay against a conventional culture-based susceptibility testing and provide data necessary for the computation of both sensitivity and specificity. Conventional culture-based susceptibility testing was; Lowenstein-Jensen proportion method, the BACTEC radiometric method, and the BACTEC mycobacteria growth indicator tube 960 (MGIT) method. Duplicate publications of the same study, articles with no full text, reviews, meta-analysis, and papers containing no relevant information were also excluded.

Extraction of data

The following items were extracted from each article: the name of the first author, year of publication, study location, conventional culture-based methods used, number of confirmed TB cases, number of susceptible PZA isolates, and resistant PZA isolates. Two reviewers independently extracted the data, and differences were resolved by consensus.

Quality assessment

The methodological quality of the studies was assessed using the QUADAS-2 tool [18].

Statistical analyses

Statistical analyses were performed with STATA (version 14, Stata Corporation, College Station, TX, USA), RevMan (version 5.3; The Nordic Cochrane Centre, the Cochrane Collaboration, Copenhagen, Denmark), and Meta-DiSc (version 1.4, Cochrane Colloquium, Barcelona, Spain) software. The pooled sensitivity, specificity, and diagnostic odds ratio (DOR) with 95% confidence intervals between Wayne assay and the reference standard were assessed. A random-effects model was used to pool the estimated effects. Diagnostic accuracy measures (i.e., the summary receiver operating characteristic [SROC] curve and the summary positive likelihood ratios [PLR], negative likelihood ratios [NLR], and DOR) were calculated [19].

Sensitivity is the proportion of positive test results among those with the target disease. Specificity is the proportion of negative test results among those without the disease. The PLR measures how much more frequent a positive test is found in diseased versus non-diseased individuals in a clinical setting. On the other hand, the NLR measures how likely a negative result is found in diseased versus non-diseased individuals. Tests with pooled PLR values greater than ten and a pooled NLR value of less than 0.1 have the greater discriminating ability [20,21].

The DOR or the odds of a positive result in diseased individuals compared to the odds of a positive result in non-diseased individuals. It is calculated according to the formula: DOR = (TP/FN)/(FP/TN). DOR depends significantly on the sensitivity and specificity of a test. A high specificity and sensitivity test with a low rate of false positives and false negatives have high DOR [21].

The area under the curve (AUC) serves as a global measure of test performance; a value of 1 indicates perfect accuracy [19,21].

The heterogeneity among the studies was assessed using the chi-square test and I-square statistics. To identify the risk of publication bias, Deek's test was used, based on parametric linear regression methods.

Results

Study selection

Fig. 1 summarizes the study selection process. We retrieved data from 31 selected articles comprising data for 2457 isolates of M. tuberculosis. The characteristics of the included studies are described in Table 1.

Fig. 1

Flow chart of study selection for inclusion in the study.

Table 1

Characterization of included studies

Study	Country	Published year	Diagnostic methods	Reference standard
Alcántara [25]	Peru	2019	Wayne assay	BACTEC MGIT 960
Bouzouita [26]	Tunisia	2018	Wayne assay	BACTEC MGIT 960
Calderón [15]	Peru	2017	Wayne assay	BACTEC MGIT 960
Bhuju [27]	Brazil	2013	Wayne assay	Proportion method
Cui [16]	China	2013	Wayne assay	BACTEC MGIT 960
Huang-2013 [28]	China	2013	Wayne assay	Proportion method
Barros Ribeiro [29]	Brazil	2012	Wayne assay	Proportion method
Campanerut [30]	Brazil	2011	Wayne assay	Proportion method
Ghirald [31]	Brazil	2011	Wayne assay	Proportion method
Muthaiah [32]	India	2010	Wayne assay	Proportion method
Sharma [33]	India	2010	Wayne assay	BACTEC MGIT 960
Ando [34]	Japan	2009	Wayne assay	BACTEC MGIT 960
Shenai [35]	India	2009	Wayne assay	BACTEC MGIT 960
Yüksel [36]	Turkey	2009	Wayne assay	BACTEC 460
Zhang [37]	China	2009	Wayne assay	Proportion method
Aragon [38]	Spain	2007	Wayne assay	BACTEC 460
Singh [39]	India	2007	Wayne assay	Proportion method
Baco [40]	Brazil	2006	Wayne assay	Proportion method
Martin [41]	Belgium	2006	Wayne assay	BACTEC 460
Sekiguchi [42]	Japan and Poland	2006	Wayne assay	BACTEC 460
Huang-2003 [43]	Taiwan	2003	Wayne assay	BACTEC 460
Bamaga-2002 [44]	London	2002	Wayne assay	BACTEC 460
Endoh [45]	Japan	2002	Wayne assay	Proportion method
Suzuki [46]	Japan	2002	Wayne assay	Proportion method
Bamaga-2001 [47]	London	2001	Wayne assay	BACTEC 460
Bishop [48]	South Africa	2001	Wayne assay	BACTEC 460
Kew Park [49]	Korea	2001	Wayne assay	Proportion method
Davies [50]	South Africa	2000	Wayne assay	BACTEC 460
Martilla [51]	Russia	1990	Wayne assay	BACTEC 460
Mestdagh [52]	Belgium	1990	Wayne assay	BACTEC 460
Mcclaytchy [53]	Israel	1981	Wayne assay	Proportion method

Quality of including studies

The quality assessment of studies is shown in Fig. 2 with the QUADAS-2 tool. The studies revealed a low risk of bias in the index test categories, reference standard, flow, and timing. There was little concern regarding the patient's selection.

Fig. 2

Overall quality assessment of included studies (QUADAS-2 tool); the proportion of studies with a low, unclear, and high risk of bias (left), and proportion of studies with low, unclear, and high concerns regarding applicability (right). Patient selection: describes methods of patient selection; index text: describes the index test and how it was conducted and interpreted; reference standard: describes the reference standard (gold standard test) and how it was conducted and interpreted; flow and timing: describes any patients who did not receive the index tests or reference standard or who were excluded from the 2 × 2 tables, and describes the interval and any interventions between index tests and the reference standard.

Diagnostic accuracy of Wayne assay against combined reference tests

The pooled sensitivity and specificity of the Wayne assay against all reference tests (the combination of BACTEC MGIT 960, BACTEC 460, and proportion method) were 86.6% (95% CI: 84.3-88.7) and 96.0% (95% CI: 94.8-97). The PLR, NLR, DOR and the AUC estimates were found to be 17.6 (95% CI: 10.5-29.3), 0.11 (95% CI: 0.06-0.20), 164 (95% CI: 83-320) and 97%, respectively (Fig. 3, Fig. 4, Fig. 5). A PLR of 17.6 suggests that patients with PZA resistance have a 17.6-fold higher chance of being Wayne test-positive than patients without PZA resistance. This ratio suggests a potential role for Wayne's assay in confirming (ruling in) PZA resistance. An NLR of 0.11 means that isolates with PZA resistance are 0.11 times as likely to have a negative Wayne test as isolates without PZA resistance. DOR and the AUC estimates in this report also represented a high level of test accuracy. Deek's test result indicated no evidence for publication bias (P > 0.05).

Fig. 3

Forest plot of pooled sensitivity of Wayne assay for the diagnosis of PZA resistance. The point estimates of sensitivity from each study are indicated as a circle and a 95% confidence interval is shown with a horizontal line.

Fig. 4

Forest plot of pooled specificity of Wayne assay for the diagnosis of PZA resistance. The point estimates of specificity from each study are indicated as a circle and a 95% confidence interval is shown with a horizontal line.

Fig. 5

Summary receiver operating characteristic (SROC) plot. The area under the curve (AUC), acts as an overall measure for test performance. Particularly, when AUC would be between 0.9 and 1, the accuracy is high. AUC was 0.97 in this report which represented a high level of accuracy.

Discussion

PZA is an essential drug in current anti-TB regimens. The practical use of PZA in patients with PZA resistance is associated with treatment failure. Therefore, knowing the PZA resistance patterns would significantly improve TB patients' management [22].

Culture-based phenotypic tests are slow, with a high rate of false resistance due to the culture medium's acidity needed for PZA activity [23,24]. Likewise, molecular PZA DSTs based on the detection of a pncA mutation have been widely used, the diagnostic accuracy of molecular tests is varied [11,13].

Wayne assay is simple to perform, inexpensive, and requires minimal laboratory staff training, which facilitates its usability in many low-income countries. In this study, the Wayne assay results were assessed against different conventional PZA DSTs (i.e., Proportion method, BACTEC MGIT 960, etc.).

Our systematic review shows that the Wayne test can determine M. tuberculosis resistance to PZA and achieve this in a short period compared to other assays such as BACTEC MGIT 960, BACTEC 460, and the proportion method.

A similar observation was reported by Chang et al. in which the sensitivity and specificity of the Wayne assay, was found to be 91% and 97%, respectively [22]. Although the present study suggests that the Wayne test might be sensitive and specific for detecting PZA resistance, molecular assays are probably the way forward.

Our study has some limitations. First, given the number of published data included in the current systematic review, we may have overestimated the Wayne assay's actual test performance. Second, the exclusion of studies in other languages could have introduced publication bias, and thus limitations associated with potential publication bias should be considered. Third, heterogeneity exists among the included studies.

In conclusion, although the current study shows that the Wayne test is sensitive and specific for detecting PZA resistance, it may be used in combination with conventional DSTs to diagnose PZA resistance accurately.

Conflict of interest

None.

CRediT author statement

MJN: conception and design of study. FF, MA, BD, FKH, MD, and MG: acquisition of data. MJN and FF: analysis and/or interpretation of data. MJN, and MM: drafting and revision of manuscript. All authors contributed to the article and approved the submitted version