Diagnostic Accuracy and Agreement between Four Phenotypic Carbapenemase Detection Tests among Enterobacterales.

INTRODUCTION: Carbapenem-resistant Enterobacterales (CREs) are becoming increasingly popular as a cause of hospital-acquired infections that are difficult to treat and are frequently reported as causes of outbreaks in various hospitals. Conventional culturing techniques take at least 2 days to report a case as carbapenem resistant, and it is therefore important to detect such resistance mechanisms as early as possible.
METHODS: This study aimed to compare the diagnostic performance of Carba NP, modified Hodge test (MHT), ethylenediaminetetraacetic acid (EDTA) disk synergy test (DST), and the modified carbapenem inactivation method (mCIM). This study was done at Microbiology Laboratory, Aga Khan University Hospital, Karachi. It was an observational study. Carba NP, MHT, EDTA DST, and the mCIM were performed on consecutive isolates of Enterobacterales. Sensitivity, specificity, and agreement between the four tests were calculated.
RESULTS: Of 207 Enterobacterales isolated, 127 were resistant to carbapenems. One hundred and fourteen of these were tested by a polymerase chain reaction, and the sensitivities of the Carba NP, MHT, EDTA DST, and the mCIM were found to be 94.34%, 75.47%, 79.25%, and 98.11%, respectively.
CONCLUSIONS: Due to increased rates of carbapenem resistance, there is a need to employ mechanisms in hospitals that can identify such organisms as early as possible, both from clinical and epidemiological standpoints. The Carba NP test is a rapid, cost-effective, and reliable method and mCIM is more accurate but time consuming and both can be safely used for the screening of CREs. Copyright:

INTRODUCTION

In recent years, resistance to antibiotics has become a great concern for microbiologists and physicians alike.[1234567] In response to this broad-spectrum resistance, antibiotics from the carbapenem class have been established as agents of last resort in treating these infections and are increasingly being used in recent decades as the only effective therapy. During the past decade, carbapenem resistance in Enterobacterales has emerged, and in recent years, widespread outbreaks of carbapenem-resistant Enterobacterales (CREs) have been increasingly reported.[8] Since the breakpoints of carbapenem have been revised, decreased rates of susceptibility have been observed by the SENTRY Antimicrobial Surveillance Program from 2008 to 2012 in the United States.[9] According to laboratory-based data collected over 3 years at a tertiary care in Pakistan, a change from 0% to 2% was noted in the resistance rates of CRE in 2010.[10]

Certain Gram-negative bacilli acquire resistance to carbapenems by plasmid-borne and easily transmissible carbapenemases,[8] and since timely control needs to be employed for CRE infections, it is imperative that such resistance mechanisms are detected as early as possible. The Clinical Laboratory Standards Institute (CLSI) recommends to use such screening methods for the detection of CRE for epidemiological and infection control purposes.[11]

Although molecular tests are confirmatory, due to cost and equipment issues, various phenotypic methods have been developed for the detection of carbapenemases, which, despite their simplicity, take 18–24 h to yield results.[1112]

The mCIM was included in the CLSI in 2017[13] and is a very simplistic method of carbapenemase detection employing the concept of carbapenem inactivation by immersing a carbapenem disk in a solution of suspected carbapenemase producers, and later using the same disk for susceptibility testing against a known sensitive strain. The Carba NP test is a rapid chromogenic test and there have been several studies to determine its diagnostic accuracy, and most have shown excellent results in terms of diagnostic accuracy.[141516] The modified Hodge test (MHT) and the ethylenediaminetetraacetic acid disk synergy test (EDTA DST) are some phenotypic tests that were previously used for carbapenemase detection in Enterobacterales.

In this study, we aim to compare the sensitivity and specificity and diagnostic accuracy of the modified Carbapenem inactivation method (mCIM), the Carba NP test, the MHT, and the EDTA EDTA DST for the detection of carbapenemases in Enterobacterales. In addition, other parameters like turnaround time and technical practicability were evaluated as well.

METHODS

Setting

This study was carried out at the Aga Khan University Hospital (AKUH), Karachi, from June 2014 to January 2016, nonprobability consecutive sampling technique was applied. Enterobacterales isolated from clinical samples such as urine, blood, pus, body fluids, and respiratory specimens were included. Duplicate samples from the same patient were excluded.

In addition, 114 previously saved Enterobacterales isolates were included. These isolates used to be part of a previous study performed at our institute[17] and were tested for NDM-1, KPC, IMP, and VIM carbapenemase genes.

Identification

Conventional microbiological tests such as gram stain, growth on MacConkey agar, sulfide production, indole ring formation, motility, citrate utilization, urease production, and reaction on triple sugar iron were used for species-level identification, and when in doubt, API® 20E (BioMérieux, Paris, France) was used for confirmation of identification.

Tests for susceptibility

Disk-diffusion method: We performed disk diffusion for susceptibility testing of the Enterobacterales against carbapenems. As per the CLSI recommendations, a zone diameter of ≥23 mm was taken as sensitive and ≤19 mm as resistant.

Confirmation of carbapenem resistance: For isolates that were tested resistant on disk-diffusion method, minimal inhibitory concentrations (MIC) were determined against meropenem using an automated system (VITEK 2® GN81) which works on the principle of broth microdilution. An MIC of ≥4 μg/ml was taken as resistant.

Testing by phenotypic methods

The organisms selected for the study were saved at –80°C and mCIM, Carba NP, MHT, and EDTA DST were performed in batches at an appropriate time later.

Carba NP test

This test was performed on the basis of following CLSI recommendations.[11] Briefly, a loop full of bacterial colonies was emulsified in cell lysis reagent, and to this added a solution containing imipenem and a pH-based indicator (phenol red). A color change from red to yellow was taken as indicative of carbapenemase production.

Modified Hodge test

We performed the MHT as per CLSI guidelines.[11] Briefly, a carbapenem disk was placed in the center of a plate on which a lawn of Escherichia coli was made. Test organisms were streaked from the center to the periphery of the plate, a clover-shaped indentation in the zone of inhibition formed by the growth of CRE at 18–24 h was considered positive [Figure 1].[11]

Figure 1

Modified Hodge test. Block arrow – carbapenemase nonproducer, thin arrow – carbapenemase producer

Ethylenediaminetetraacetic acid disk synergy test

We adjusted the test strains to the McFarland 0.5 standard and used them to make lawns on Mueller-Hinton agar (MHA) plates. We then placed a 10-μg IPM disk and a blank filter paper disk at a distance of 10 mm on each lawn and 10 μl of a 0.5 M EDTA solution was added to the blank disk. After overnight incubation, the presence of even a small synergistic inhibition zone was interpreted as positive [Figure 2].[12]

Figure 2

Ethylenediaminetetraacetic acid disk synergy test. 1 – carbapenemase producer, 2 – carbapenemase nonproducer

Modified carbapenem inactivation method

mCIM was performed as per the CLSI guidelines. Briefly, 18–24 h growths of test organisms were emulsified in Trypticase Soy Broth, a meropenem disk was added and incubated for 4 h. This disk was placed on a lawn of 0.5 Mc Farland ATCC E. coli 25922 on MHA and incubated for 18–24 h. After this time, zone sizes around the meropenem disks were interpreted as carbapenemase positive or negative as per the CLSI recommendations.

Molecular methods

Molecularly tested isolates were borrowed from a previous study as mentioned earlier. In that study, presence of gene for carbapenemases class A KPC enzymes and class B metallo-beta-lactamase (blaNDM-1, blaIMP-1, and blaVIM-1) was detected using polymerase chain reaction (PCR). Different primers were used for the above-mentioned genes, blaNDM-1 genes were detected through conventional PCR, using the following primer sequence: F GGG CAG TCG CTT CCA ACG GT; R GTA GTG CTC AGT GTC GGC AT. Deoxyribonucleic acid (DNA) was extracted by Qiagen DNA extraction kit (Qiagen, CA, USA). A 475 bp product was amplified by NDM primers and visualized on 3% agarose gel.

Statistical analysis

The sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic accuracy for Carba NP, MHT, EDTA DST, mCIM, and Vitek2 MIC were calculated using molecular method of carbapenemase gene detection. Agreement between the methods was also checked, and kappa scores were calculated. Finally, IBM SPSS statistics (IBM Corp. Released 2013. IBM SPSS Statistics for Windows, Version 22.0. Armonk, New York: IBM Corp., United States of America), was utilized for data entry and analysis.

RESULTS

At the end of the study duration, there were a total of 207 Enterobacterales. Details of susceptibility and tests for carbapenemase detection are shown in Figure 3. Klebsiella species and E. coli were the most common organisms that were isolated. Figures 4 and 5 describe the identification and sources of these isolates in relation to their carbapenem susceptibility as per the defined standards, respectively.

Figure 3

Schematic representation of results

Figure 4

Carbapenem resistance and susceptible Enterobacterales

Figure 5

Specimen sources of resistant and susceptible isolates

The results of all the performed tests are shown in Table 1.

Table 1

Results of Carba NP test, modified Hodge test, EDTA disk synergy test, and modified carbapenem inactivation method for those isolates which were also tested for blaNDM-1

Test	NDM PCR (n=114)
	Positive	Negative
Carba NP
Positive	100	6
Negative	6	2
MHT
Positive	80	5
Negative	26	3
EDTA DST
Positive	84	6
Negative	22	2
mCIM interpretation
Positive	104	8
Negative	2	0
Indeterminate	0	0

DST: Disk synergy test, mCIM: Modified carbapenem inactivation method, PCR: Polymerase chain reaction, MHT: Modified Hodge test, NDM: New Delhi Metallo-β-Lactamase, EDTA: Ethylenediaminetetraacetic acid

As per our reference (MIC) standard, 127 Enterobacterales were found to be resistant to carbapenems. Of these, 114 were also tested for the presence of bla-NDM-1, the results are shown in Table 2.

Table 2

Sensitivity, specificity, positive and negative likelihood ratios, and diagnostic accuracy for Carba NP test, modified Hodge test, EDTA disk synergy test, modified carbapenem inactivation method using NDM-1 Polymerase chain reaction as a gold standard

Test name	Sensitivity % (CI)	Specificity % (CI)	Positive likelihood ratio % (CI)	Negative likelihood ratio % (CI)	Diagnostic accuracy %
Carba NP	94.34 (88.2-97.38)	25 (7.15-59.07)	1.258 (0.841-1.882)	0.226 (0.054-0.946)	89.4
EDTA DST	79.25 (70.57-85.888)	25 (7.15-59.07)	1.057 (0.7-1.595)	0.830 (0.236-2.917)	75.4
MHT	75.47 (66.49-82.68)	37.50 (13.68-69.43)	1.208 (0.698-2.088)	0.654 (0.252-1.7)	72.4
mCIM	98.11 (93.38-99.48)	00 (0.00-32.44)	0.981 (0.956-1.007)	-	91.22

CI: Confidence interval, DST: Disk synergy test, mCIM: Modified carbapenem inactivation method, MHT: Modified Hodge Test, EDTA: Ethylenediaminetetraacetic acid

Eight carbapenem-resistant isolates were negative for bla-NDM-1, thus they were also tested for IMP, KPC, and VIM, which were not found in these isolates.

We tested all of these (207) isolates for carbapenemase production by using four phenotypic methods, i.e., mCIM, Carba NP, MHT, and EDTA DST. However, the calculated sensitivity, specificity, positive and negative likelihood ratio, and diagnostic accuracy were possible in only 114 isolates tested by carbapenemase gene, as shown in Table 2. A receiver operating characteristic curve was made to assess the diagnostic accuracy [Figure 6]. Table 3 shows agreement between the four phenotypic tests that were performed.

Figure 6

Receiver operating characteristic curve comparing the Carba NP, Modified Hodge test, ethylenediaminetetraacetic acid disk synergy test and modified carbapenem inactivation method

Table 3

Agreement between Carba NP test, Modified Hodge Test, EDTA disk synergy test, modified carbapenem inactivation method

Methods	Cohen’s Kappa
Carba NP and MHT	0.682
Carba NP and EDTA DST	0.721
Carba NP and mCIM	0.677
MHT and EDTA DST	0.575
MHT and mCIM	0.471
EDTA DST and mCIM	0.529

DST: Disk synergy test, mCIM: Modified carbapenem inactivation method, MHT: Modified Hodge test, EDTA: Ethylenediaminetetraacetic acid

The Carba NP had a substantial agreement with the MHT, EDTA DST, and mCIM, while there was moderate agreement between the other three methods.

DISCUSSION

To the best of our knowledge, this is the first study from Pakistan that has compared modern phenotypic techniques of carbapenemase detection with other previously known phenotypic methods of carbapenemase detection and has confirmed the positive results with PCR.

The mCIM has been shown previously to be highly sensitive and specific,[1819] and the current study also demonstrates it to be a highly sensitive test. Similarly, our study results validate the previously reported high sensitivity[141516] of Carba NP method. The other two methods (MHT, EDTA DST) showed comparatively lower sensitivities.

The Carba NP and mCIM had very good diagnostic accuracies of 89% and 91%, respectively,[19] and there was substantial agreement between the two. A limitation regarding the specificity of the four phenotypic tests found in this study is that a very small number of isolates were PCR negative, even though they were carbenem resistant according to their MICs. The low specificity of all the tests may therefore be attributable to the small number of NDM-1 negative isolates, and not a true reflection of the diagnostic accuracy.

If we compare the turnaround time of four tests, the Carba NP is the most rapid as it takes 2 h for performance and obtaining results. Although preexisting phenotypic methods of carbapenemase detection such as the MHT and EDTA DST are simpler to perform and perhaps more cost-effective than the Carba NP, they have longer turnaround times and are not as accurate.[121720] The mCIM has good diagnostic accuracy, but like the MHT and EDTA DST, it takes 18–24 h for the results to be read, and thus, the Carba NP seems like an attractive option for microbiologists and clinicians considering its shorter turnaround time.

On the other hand, the Carba NP is the most meticulous, while the mCIM, MHT, and EDTA DST are considerably simpler to perform. While the MHT, EDTA DST, and mCIM all involved steps like making lawns of standard solutions of organisms and placing disks and/or making streaks, the Carba NP needed careful measurement by weight of imipenem powder, adjusting the pH of solutions, and cautious pipetting to mention a few. It may, therefore, need staff well trained in procedures involving pipetting and media preparation, and laboratories must keep in mind all these factors when deciding on which test to select as a screening tool.

Increase in carbapenem resistance has become a major problem both worldwide[7] and locally. Data from AKUH, Karachi, Pakistan,[21] show that carbapenem resistance in Enterobacterales rose from an average of 9.6% in 2013–13.5% in 2016, emphasizing the need for the implementation of rapid tests for carbapenemase detection in the laboratory.

In our study, most of the isolates were identified as E. coli (51.2%), followed by Klebsiella pneumoniae (41.5%). However, of the resistant isolates, the breakup was 59% for Klebsiella and 32% for E. coli. This correlates with previous studies, in which most resistant organisms were identified as Klebsiella species.[20222324] Klebsiella species are known to be the most common carbapenemase-producing Enterobacterales in the health-care settings and have become a significant cause of morbidity and mortality in the admitted patient.[25]

This study has some limitations. First, the sample size was low. Second, only a subset of carbapenem-resistant isolates was tested for commonly found resistance genes, and therefore, diagnostic accuracy parameters, specificity to be more exact, gave unexpected results. Although MIC is considered to be confirmatory for the purpose of reporting resistance,[11] the presence of carbapenemase cannot be attributed to MIC alone, and a genotypic analysis is necessary to imply that an organism harbors carbapenemase as the resistance mechanism, which is why diagnostic accuracy could not be calculated using MIC as a gold standard. Third, the result of this study cannot be generalized to carbapenemase enzymes other than NDM-1 metallo-beta-lactamase group as all PCR-proven isolates tested for these phenotypic methods belonged to this carbapenemase group.

CONCLUSIONS

Due to increased rates of carbapenem resistance, there is a need to employ mechanisms in hospitals that can identify such organisms as early as possible, both from clinical and epidemiological standpoint. The Carba NP test is a rapid, cost–effective, and reliable method that can be safely used for the screening of CREs in case of outbreak or infection control in the hospital setting.

Research quality and ethics statement

This study was approved by the Aga Khan university Karachi Ethical review committee (IRB 2519-Pat-ERC-13). The authors followed applicable EQUATOR Network (”http://www.equator-network.org/) guidelines during the conduct of this research project.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.