Detection of carbapenem-resistant Klebsiella pneumoniae on the basis of matrix-assisted laser desorption ionization time-of-flight mass spectrometry by using supervised machine learning approach.

BACKGROUND: Carbapenem-resistant Klebsiella pneumoniae (CRKP) is emerging as a significant pathogen causing healthcare-associated infections. Matrix-assisted laser desorption/ionisation mass spectrometry time-of-flight mass spectrometry (MALDI-TOF MS) is used by clinical microbiology laboratories to address the need for rapid, cost-effective and accurate identification of microorganisms. We evaluated application of machine learning methods for differentiation of drug resistant bacteria from susceptible ones directly using the profile spectra of whole cells MALDI-TOF MS in 46 CRKP and 49 CSKP isolates.
METHODS: We developed a two-step strategy for data preprocessing consisting of peak matching and a feature selection step before supervised machine learning analysis. Subsequently, five machine learning algorithms were used for classification.
RESULTS: Random forest (RF) outperformed other four algorithms. Using RF algorithm, we correctly identified 93% of the CRKP and 100% of the CSKP isolates with an overall classification accuracy rate of 97% when 80 peaks were selected as input features.
CONCLUSIONS: We conclude that CRKPs can be differentiated from CSKPs through RF analysis. We used direct colony method, and only one spectrum for an isolate for analysis, without modification of current protocol. This allows the technique to be easily incorporated into clinical practice in the future.

Introduction

Carbapenem-resistant Enterobacteriaceae (CRE) is listed as one of the most urgent antibiotic resistance threats by the Centers for Disease Control and Prevention (CDC) and The World Health Organization (WHO) [1, 2]. Carbapenem resistance can be mediated by production of carbapenemases or by the combination of outer membrane porin expression disruption and production of various β-lactamases [3, 4]. Carbapenemases may confer resistance to virtually all available beta-lactam antibiotics. The most frequent carbapenemases are the Klebsiella pneumoniae carbapenemase (KPC) enzymes. KPC-producing bacteria are mostly in the form of K. pneumoniae. Since the first report of KPC-producing K. pneumoniae in the United States in 1996[5], KPC-producing organisms are now found in many other countries. [6-9]. In 2014, a global study reported 6.0%, 15.7%, 16.0%, 3.8% and 14.9% of meropenem nonsusceptible K. pneumoniae occurred in the United States, Europe, Latin American, APAC, and China [10]. The European Centre for Disease Prevention and Control (ECDC) report showed that between 2015 and 2018, there was a small but significant increasing trend of population-weighted mean percentages for carbapenem resistant K. pneumoniae (CRKP), from 6.8% to 7.5%, in the European Union (EU)/European Economic Area (EEA)[11]. The study of ECDC on the health burden of antimicrobial resistance estimated that the number of deaths attributed to infections with CRKP increased six-fold between 2007 and 2015. Even in countries with lower levels of CRKP, the impact of antimicrobial resistance on national burden is significant because of the high attributable mortality of these infections [12]. Therefore, rapid detection of carbapenem-resistance is important to guide the initial choice of antimicrobial treatment, as well as to prompt initiation of effective infection control measures. However, detection of carbapenem resistance using susceptibility testing methods are time consuming. Polymerase chain reaction (PCR) is a rapid, accurate and reliable method to detect resistant genes. However, it is limited to reference laboratories and can only detect known enzymes.

MALDI-TOF MS is now widely used to identify clinical microorganisms because of lowered cost and faster turn-around times [13, 14]. In recent years, four approaches have been reported for detection of certain drug resistant bacteria [15, 16] based on the postulation that any or combination of the polypeptides related to drug resistance could possibly alter the spectra obtained by MALDI-TOF MS and possibly enable differentiation from the drug-susceptible bacteria. One approach concerns the analysis of antibiotics and their degradation products [15, 17–19]. The hydrolysis degradation product of the β-lactam antibiotic shows a molecular mass different from that of the native molecule. The detection of β-lactamase activity was performed by analyzing the spectra containing peaks representing the β-lactam molecule, its salts (usually sodium salts), and/or its degradation products. For example, with MALDI-TOF MS, the presence of carbapenemases was confirmed by the detection of carbapenem hydrolysis (loss of molecular peaks: 476 5 m/z for ertapenem and its sodium salts, 498 5 m/z, 520 5 m/z; 383.0 m/z for meropenem and its sodium salts, 405.2 m/z and 427.4; 300.0 m/z for imipenem) and presence of their degradation products (450 m/z for ertapenem; 401.1 m/z for meropenem and its sodium salts, 423.3 m/z, 445.6 m/z, 467.839 m/z; and 254.0 m/z for imipenem). Moreover, the combination the β-lactam antibiotic with β-lactamase inhibitors has the potential of identification of the type of β-lactamase [20, 21]. The second approach directly detect β-lactamase [22] A peak of approximately 29 kDa that represented β-lactamase was detected in the spectra. The third approach detects the specific peaks in the resistant group [23]. Some peaks were invariably present in, and unique to drug resistant organism or susceptible ones. Edwards-Jones V et al. reported some peaks unique to methicillin-resistant Staphylococcus aureus (MRSA)(e.g., 891, 1140, 1165, 1229 and 2127) or methicillin-susceptible S.aureus (MSSA)(e.g., 2548 and 2647). The m/z peaks 1774.1 and 1792.1 reported by Lu et al. were able to successfully discriminate between hospital-acquired MRSA and community-acquired MRSA isolates. The MALDI-TOF instruments acquire spectra in a linear positive-ion mode at a laser frequency of 60 Hz across a mass/charge (m/z) ratio of 2,000 to 20,000. Therefore, the first three approaches can detect molecules beyond the detection limit.

The fourth approach involves the analysis of ribosomal DNA methylations [24]. The methylation of rRNA confers resistance to antibiotics that inhibit protein synthesis. Methyltransferase activity was investigated by detection of digested rRNA to yield smaller products that were subsequently analyzed by MALDI-TOF MS. This approach requires specific sample preparations and purified ribosomes and enzymes, which also impedes its use in routine clinical laboratory. In addition, all these applications are limited by the need to know the specific target peak as a biomarker.

Some studies describe application of machine learning methods for differentiation of drug resistant from susceptible bacteria by directly using the profile spectra of whole cells MALDI-TOF MS. Griffin et al [25] successfully used a support vector machine (SVM) algorithm to accurately identify vanB positive, vancomycin-resistant Enterococcus faecium from vancomycin-susceptible isolates, directly using the profile spectra of whole cells. Although these preliminary results were promising, clonal relatedness of these strains was not clarified. It is therefore impossible to exclude the chance that the findings reflect only a favorable epidemiological condition in the geographical region of the authors. In addition, ethanol-formic acid extraction was used for sample preparation. It involves several centrifugation steps and is more labor-intensive than direct analysis of bacteria without additional protein extraction, which is usually done in routine work flow.

Mather et al. [26] successfully differentiated vancomycin-intermediate S. aureus (VISA) and heterogeneous VISA (hVISA) from vancomycin-susceptible S. aureus (VSSA) by using a SVM algorithm. A total of 20 spectra from biological duplicate experiments were acquired for each isolate. Only peaks present in 80% of the spectra for a given isolate (≧16/20) were selected and merged into one representative spectrum for further analysis. The peak selection criteria required multiple spectra for a given isolate and preprocessing for superspectrum made it difficult for use in clinical laboratory. Furthermore, the reproducibility of these findings has not been widely applied and published.

Supervised machine learning is a powerful tool for classification. However, MALDI-TOF mass spectrometry data consist of hundreds or thousands of mass to charge (m/z) ratio per specimen and an intensity level for each m/z ratio. The dimensionality is usually much larger than the sample size. This makes many standard pattern classification algorithms fail. Therefore, it is critical to reduce dimensions before discrimination using such data.

Here we describe an application of machine learning methods for differentiation of drug resistant bacteria from susceptible ones directly using the profile spectra of whole cells MALDI-TOF MS in 46 CRKP and 49 CSKP isolates. Data were preprocessed, consisting of peak matching and a feature selection step. Subsequently, five supervised machine learning algorithms were compared. We demonstrated that the RF algorithm outperformed other methods including logistic regression, naïve Bayes, nearest neighbors, and SVM. Using the RF algorithm, we correctly identified 93% of the CRKP and 100% of the CSKP isolates with an overall classification accuracy of 97% when 80 peaks were selected as input features.

Materials and methods

Bacterial strains

This study was approved by the Institutional Review Board (VGHKS18-CT8-07). All bacteria strains used in this study were from frozen stocks that was stored in 50% glycerol and/or Mueller-Hinton (MH) broth at −80°C. Minimum inhibitory concentrations (MICs) of carbapenems were determined by Vitek 2 (bioMérieux, Marcy l'Etoile, France) test using the AST N320 card (bioMérieux, Durham, NC), which included imipenem (range, 1 to 16 μg/ml) and meropenem (range, 0.25 to 16 μg/ml), according to the manufacturers' recommendations. Study strains included in this study were K. pneumoniae isolates that were resistant to imipenem or meropenem according to the updated 2018 CLSI breakpoints [27], and had a “carbapenemase” phenotype detected by the advanced expert system (AES) of VITEK 2 system.

MALDI-TOF MS analysis

MALDI-TOF MS

A portion of a colony was picked up from the blood agar plate and spotted onto a MALDI-TOF target plate. Each deposit on the target plate was overlaid with 2 μL of matrix solution (α‐cyano‐4‐hydroxycinnamic acid) (VITEK® MS CHCA) and air dried.

Following sample preparation, samples were analyzed with the Vitek MS system (bioMérieux SA) in linear positive-ion mode, at a laser frequency of 50 Hz across the mass-to-charge ratio range of 2,000 to 20,000 Da. For each target slide, the E. coli reference strain ATCC 8739 [28] was used for instrument calibration according to the manufacturer's specifications. Quality of protein extraction was assessed by the data count (DC), defined by the interpretable number of peaks considered in the algorithm. These isolates with a score less than 90% were repeated once.

Spectral analysis. To recognize well-defined peaks, each spectrum was processed by baseline correction, denouncing, and peak detection. The data was transferred from the Vitek MS acquisition station to the Vitek MS RUO v4.12 after spectrum acquisition. The data were presented as a spectrum of intensity versus mass, in Daltons (Da).

Peak matching

Mass-Up software (http://sing.ei.uvigo.es/mass-up) was used to preprocess the data [29]. The list of peaks and intensities were imported to Mass-Up as CSV files. An inter-sample peak matching was performed with the Mass-Up software configured to use a tolerance of 300 ppm for forward algorithm. This step unifies the peak values among samples.

Data analysis

All of the following data analysis was performed using aligned spectra output from Mass-up software in mathematica software v. 12 (Wolfram Research, Champaign, IL). The performance of the machine learning prediction procedure was evaluated using a leave-one-out (L1O) cross-validation procedure. The overall flowchart and the representative steps of procedures of dimension reduction and cross validation steps were shown in Fig 1.

Fig 1

Overall flowchart (A) and the representative steps of procedures of dimension reduction and cross validation steps (B).

Split data

Each spectrum was ‘left out’ in turn as testing set, all the other spectra were training set. The testing set was not used in the training set. The dimension reduction process and building classifier was then executed using all the remaining data. The resulting machine learning structure was then tested using the left-out spectrum in the beginning. The process repeats, each time omitting a different spectrum, until all spectra have been omitted once.

Dimension reduction

The Student’s t-test was used to compare the distribution of each m/z ratio (the null hypothesis H0 is that CRKP and CSKP have the same distribution) and generate a p value for the t-statistic. Ranking the p value from the smallest (most significant between two groups) to the largest. Select a list of top ranked k peaks to machine learning algorithms to build classifier.

Class prediction

Five machine learning algorithms were performed for construction of classifiers: SVM, RF, nearest neighbors, naïve Bayes, and logistic regression. For each model, the default settings were left unaltered; for example, with the RF model, automatic detection was selected for the fraction of features to be randomly selected to train each tree, the maximum number of examples in each leaf, and the number of trees in the forest. In addition, the regularization parameter is set as 0.5.

In these analyses, different combinations of the most significant peaks with smaller p value by the Students’s t-test were used to build the models. The testing spectrum that was “left out” in the beginning was then repeated with the same parameters in feature selection and feature extraction of the training set and input to classifiers. The classifier output was then compared to the known group of the spectrum, and a performance tally is recorded. The process repeats, each time omitting a different spectrum, until all spectra have been omitted once. Classifier performance was assessed by calculating the accuracy, sensitivity and specificity with which test observations were classified over all iterations of the L1O cross-validation process.

Sensitivity of the methods was assessed by calculating the percentage of CRKP that were correctly identified. Specificity was assessed by calculating the percentage of CSKP that were correctly identified. Accuracy was assessed by calculating the percentage of CRKP and CSKP that were correctly identified. The machine learning algorithm predictions were output as probabilities and exported to a comma-separated file.

Random Amplified Polymorphic DNA (RAPD)

Bacteria were grown overnight on Columbia agar medium supplemented with 5% sheep blood. The bacterial DNA was prepared by QIAamp DNA Mini Kit (Qiagen Inc., Valencia, Calif., USA) according to the manufacturer’s instructions. The primer used for PCR was AP4 [30].

The reaction mixture contained 10 mM Tris-HCl (pH 8.3), 1.8 mM MgCl2, 2 μM primer, 400 μM each deoxynucleoside triphosphate, 2.5 U of Super-Therm Gold DNA polymerase (JMR Holdings), and 2μL of DNA in a final volume of 25μL. Amplification was performed in a GeneAmp PCR 9700 thermal cycler (Perkin-Elmer). Initial denaturation was for 4 min at 94°C. Amplification was performed with 50 cycles of 1 min at 94°C, 2 min at 30°C, and 2 min at 72°C, with a single final extension step of 10 min at 72°C. PCR products were separated by electrophoresis in a 1.5% agarose gel and stained with ethidium bromide.A CRKP strain, S, is included in all runs to ensure comparability between different runs.

Results

Bacteria strains

A total of 46 CRKP strains isolated consecutively from January 2016 to October 2017 were selected from frozen stock, one isolate for each patient. Bacteria isolates included in this study were isolated from clinical specimens including 7 (7.4%) pus/wound, 5 (5.3%) sputum, 20 (21.1%) urine, 54 (56.8%) blood, and 5 (5.3%) body fluids like ascitic fluid, bile, pleural fluid and 4 (4.2%) other specimens like catheter tips. There was no nosocomial outbreak of CRKP during the study period. The distribution of cases by month ranged from 0 to 5, with a mean of 2.1 cases per month (standard deviation, 1.4). Fifty CSKP isolates were randomly selected over the same period, 49 of them successfully sub-cultured and included in this study.

Among the 46 CRKP, 29 (63.0%) were detected as “carbapenemase (+ or–ESBL)”, 17 (37.0%) were carbapenemase (+ or–ESBL) plus “impermeability CARBA (+ESBL or +HL AmpC)”. Phenotypes of the 49 non-CRKP isolates were one or combinations of wild (penicillinase), acquired penicillinase + impermeability (cephamycins), acquired cephalosporinase (except ACC-1), acquired penicillinase, ESBL (CTX-M Like), ESBL+ impermeability (cephamycins), extended spectrum β-lactamase, or inhibitor resistant penicillinase (IRT or OXA).

Analysis of CRKP and CSKP proteomic profiles

Protein extraction was assessed with data count (DC) rated by Saramis v4.12. None of these peaks was distinctive to the CRKP or CSKP group. There was one peak (9478.87 Da) present in 82.6% of the CRKP isolates and only 2.2% of the CSKP isolates, one peak (7705.009 Da) was present in 80.4% of the CRKP isolates and only 2.2% of the CSKP isolates, and the other peak (9541.41 Da) was present in 76.1% of the CRKP isolates and only 2.2% of the CSKP isolates (Table 1).

Table 1

Top 10 peaks with least sum of 95 ranks produced each time one different spectrum omitted and their percentage of presence in each group.

m/z	Present in no. (%) of:		No. of times the peak ranked as:
	CRKP	CSKP	1	2	3	4	5	6	7	8	9	10	>10
9478.866	38 (82.6%)	1 (2.2%)	95	0	0	0	0	0	0	0	0	0	0
9541.405	35 (76.1%)	1 (2.2%)	0	91	2	1	0	0	0	0	0	0	1
6288.794	35 (76.1%)	12 (26.1%)	0	2	67	26	0	0	0	0	0	0	0
7705.009	37 (80.4%)	1 (2.2%)	0	1	25	66	1	0	0	0	0	0	2
7158.634	41 (89.1%)	12 (26.1%)	0	1	1	1	91	0	0	0	0	0	1
10287.76	35 (76.1%)	2 (4.3%)	0	0	0	1	1	92	0	0	0	0	1
4768.279	25 (54.3%)	10 (21.7%)	0	0	0	0	2	1	89	3	0	0	0
2636.88	24 (52.2%)	0	0	0	0	0	0	2	2	70	15	6	0
4362.217	21 (45.7%)	10 (21.7%)	0	0	0	0	0	0	2	8	58	18	9
5379.418	25 (54.3%)	10 (21.7%)	0	0	0	0	0	0	0	5	7	54	29

The Student’s t-test was used as the feature selection method on the basis of leave-one-out cross-validation. Student’s t-test was used to compare the distribution of each m/z ratio and generate a p value for the t-statistic for each peak. The low p values obtained for each peak indicate that the observed difference in intensity of the individual peaks is highly statistically significant and not coincidence-based (the lower the p value, the better a respective peak signal is suitable for separating the two classes).

After removing one sample at a time, peaks were ranked according to the p-value calculated by the Students t-test on the remaining samples. The top k peaks were selected to construct the classifier. The process repeats, each time omitting a different spectrum, until all spectra have been omitted once. Therefore, 95 ranks were produced for each peak, equating to the total number of samples. As shown in Table 1, the peak of 7705.009 Da ranked first, no matter which sample was left out. This peak was highly significantly different between CRKP and CSKP isolates.

We evaluated the classification performance of the five machine learning algorithms for detection of CRKP with leave-one-out cross-validation, where the classifier was constructed after serial removal of a sample from the dataset with subsequent prediction for the removed sample with different feature subset when k = 1 to 100. As criteria, precision, sensitivity, and specificity were used and compared to estimate model validation. As shown in Fig 2, panel A, except using naïve Bayes algorithm, the maximum classification accuracy ranged from 89% to 97% when up to100 peaks were selected.

Fig 2

Performance of five machine learning algorithms with leave-one-out cross validation using 46 CRKP and 49 CSKP mass spectra, in terms of accuracy, sensitivity and specificity for differentiate CRKP from CSKP.

Accuracy (Accu), sensitivity (Sen) and specificity (Spe) are used to evaluate prediction systems. Panel A, Boxplots (25th to 75th percentiles, Min to max) of accuracy, sensitivity, and specificity for differentiation of CRKP from CSKP when 1–100 peaks selected. In Panel B, Values of accuracy, sensitivity and specificity with number of ranked peaks (k = 1–100) with increasing p-value (X axis) selected for machine learning algorithms. k: number of ranked peaks selected with increasing p-value for classification by using all five machine learning algorithms.

Maximal values for all three statistical parameters were greater than 90% except results obtained using naïve Bayes algorithm. RF outperformed other four algorithms with all three statistical parameters greater than 90% and their minimal values not less than 75% when 1–100 peaks were selected as input variables. The ranges of accuracy, sensitivity, and specificity were 81%-97%, 76%-96%, and 82%-100%, respectively, when 1–100 peaks selected. Classification results were influenced by not only the machine learning algorithm, but also by the number of peaks selected. We found that sensitivity was improved and variability decreased when 50 to 80 peaks were selected as features for building classifier, irrespective of which algorithm was used.

In Fig 2, panel B, while increasing the number of peaks selected as input variables, the classifier robustness increased. RF exhibited satisfactory performance. Results of classification performance were shown in Table 2. The maximal overall performance achieved when 80 peaks was selected by using RF. When 80 peaks selected as input features, classification sensitivity, specificity and accuracy achieved 93%, 100% and 97%, respectively.

Table 2

Performance of five machine learning algorithms with L1O cross validation using 46 CRKP and 49 CSKP mass spectra, in terms of accuracy, sensitivity and specificity for differentiate CRKP from CSKP.

Algorithm	Metric	No. of ranked peaks selected with increasing p-value
		50	60	70	80	90	100
Random Forest	Accuracy	94%	94%	95%	97%	95%	94%
	Sensitivity	91%	91%	91%	93%	89%	87%
	Specificity	96%	96%	98%	100%	100%	100%
Logistic Regression	Accuracy	93%	92%	92%	93%	91%	91%
	Sensitivity	93%	93%	96%	98%	96%	98%
	Specificity	92%	90%	88%	88%	86%	84%
Naïve Bayes	Accuracy	86%	88%	87%	89%	88%	89%
	Sensitivity	74%	76%	74%	78%	76%	78%
	Specificity	98%	100%	100%	100%	100%	100%
Nearest Neighbors	Accuracy	91%	84%	87%	87%	83%	85%
	Sensitivity	87%	83%	87%	89%	87%	91%
	Specificity	94%	86%	88%	86%	80%	80%
Support Vector Machine	Accuracy	87%	84%	84%	86%	85%	87%
	Sensitivity	96%	96%	96%	100%	98%	98%
	Specificity	80%	73%	73%	73%	73%	78%

There was one CRKP repeatedly misclassified by RF algorithm despite varying the number of peaks selected as input features. However, when Logistic Regression, Nearest Neighbors or SVM algorithm was used, it can be classified correctly all the time.

Strain relatedness

Clonal relatedness of the study strains were clarified using RAPD fingerprinting method (Fig 3). The distribution of CRKP strains by month ranged from 0 to 5, with a mean of 2.1 cases per month. We found that RAPD profiles of CRKP strains showed different patterns. Thus, our machine learning model's classification of the CRKP and CSKP does not seem to be due to variations in strain type.

Fig 3

Random amplified polymorph in DNA fingerprinting (RAPD) types of 46 carbapenem-resistant K. pneumoniae generated by arbitrarily primed PCR.

Lanes S, standard strain included in every experiment as a control. Results for the study strains were shown in the order of the date of isolation. Lane M shows the 1-kb DNA ladder.

Discussion

In this study, we demonstrated that MALDI-TOF MS can reliably differentiate CRKP from CSKP. We were able to achieve 97% classification accuracy of CRKP and CSKP isolates by MALDI-TOF MS spectral files using RF algorithm when 80 top ranked peaks were selected to input into machine learning algorithms to build the classifier.

MALDI-TOF MS has focused on the existence or lack of particular spectral peaks to recognize antibiotic-resistant bacteria in the previous reports [15-23]. Unlike most studies in which MALDI-TOF MS was used to predict antibiotic resistance, we did not focus on target peaks from enzymes or metabolites related to drug resistance. In this study, there were only three peaks in CRKP and CSKP isolates, which were not exclusive to either group. Whether these target peaks are potential biomarkers require further study which analyze and process large series of bacterial strains. The proposed method herein can be adapted to other classification problems by using MALDI-TOF data. This technology has the potential for broad application to detection of other drug resistance, genotyping, or any classification problems.

Many bioinformatics modeling tasks, such as sequence analysis over microarray analysis or spectral analysis, have high dimensional characteristics. Computation of hundreds of peaks from such tasks is a challenge for applying machine learning methods. To benefit the predictability of the machine learning process as well as the computation speed, it is essential to reduce dimensionality through feature selection. We use Student t-test to compare the distribution of each peak in CRKP and CSKP groups and generate a p value for the t-statistics on the basis of L1O cross validation. Significant peaks with top ranking according to the p-value were selected as input features to construct classifier. In addition to reducing the dimensionality, it is also beneficial for reducing the effect of interference peaks obtained from culture or MALDI-TOF process. Also, the basal level of signal that is fundamentally not related with drug resistance will be excluded.

Due to limited availability of CRKP isolates, our study has the limitation of a relatively small sample size. We used a L1O cross validation procedure to assess the predictive ability of different learning algorithms. The tested data is not part of the training set used in the building model. The training set in each L1O iteration is different. The set of peaks selected in feature selection may vary from one iteration to another. We tested all available data with this method and used the largest possible training set in each trial. Furthermore, due to the re-sampling procedure in RF, the use of external samples may not be required to validate the prediction model [31].

If in each of the five algorithms the same isolates were consistently incorrectly classified, then this would indicate that they were most likely misidentified. This was not the case in our study, the isolates that were incorrectly classified in one algorithm were correctly identified using at least one other algorithms. This would implicate that this is a reflection of the varying power between the different algorithms.

The combination of two methods, bagging (a contraction of bootstrap-aggregating) and the Classification And Regression Trees (CART)-split criterion [32,33], makes the RF a very effective way to build models of classification that are highly predictive. Bagging generates bootstrap samples from the original data set, constructs a predictor from each sample, and decides by voting for classifications [34, 35]. The bootstrap-aggregating procedure yields better model performance as it reduces the model's variance without increasing the bias. This means that while a single tree's predictions in its training set are sensitive to noise, the vote of many trees is not, as long as the trees are not correlated. Training many trees on a single training set would give strongly correlated trees; bootstrap sampling is a way to de-correlate trees by showing them different training sets. In RF, thousands of tree-like models are grown on bootstrapped data samples. Tree-like models split the data into groups repeatedly by the predictor variable and value leading to the most homogeneous post-split groups. RF further de-correlate the tree-type models by allowing each tree to choose on each split only from a small sub-selection of predictors. This way, each tree will fit some of the true patterns in the data, and some noise, that is unique to its bootstrap sample. The noise should balance out when the projections of all trees are averaged, and only the true signal remains. Therefore, RF can used when the number of variables is much larger than the number of observations, and returns measures of variable importance and is suitable for spectral analyses.

We also ensured that our findings were not simply due to clonal relatedness in the study strains. Random amplified polymorphic DNA (RAPD) method is used for the molecular epidemiology of ESBL-producing K. pneumoniae strains in a previous study (31). RAPD analysis is significantly simpler to perform and produces results more rapidly. Interassay reproducibility has been criticized with RAPD typing because it can be affected by minor variation in factors influencing regular PCR. Therefore, a control strain was included in each run to ensure inter-assay reproducibility.

We used the direct colony method which involves spotting a small amount of bacteria directly onto the target plate without extraction procedures. The same spot can be analyzed not only for identification purpose, but also add-on information of drug resistance. Furthermore, only one spectrum for an isolate was use to analysis. These benefits make it possible to incorporate this method into clinical practice without altering the current protocol.

Early detection of multiple drug-resistant microorganisms has the ability to enhance clinical decision making and patient outcomes. Although additional studies are required before this test is integrated into the normal workflow in clinical environments, we demonstrated the ability of machine learning methods to identify CRKP.

Original gel images contained in the manuscript’s Fig 3.

(PDF)

Click here for additional data file.

20 Nov 2019

PONE-D-19-26377

Detection of carbapenem-resistant Klebsiella pneumonia on the basis of matrix-assisted laser desorption ionization time-of-flight mass spectrometry by using supervised machine learning approach

PLOS ONE

Dear Professor Chang,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Jan 04 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Joseph Banoub, Ph,D., D. Sc.

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We suggest you thoroughly copyedit your manuscript for language usage, spelling, and grammar. If you do not know anyone who can help you do this, you may wish to consider employing a professional scientific editing service.

Whilst you may use any professional scientific editing service of your choice, PLOS has partnered with both American Journal Experts (AJE) and Editage to provide discounted services to PLOS authors. Both organizations have experience helping authors meet PLOS guidelines and can provide language editing, translation, manuscript formatting, and figure formatting to ensure your manuscript meets our submission guidelines. To take advantage of our partnership with AJE, visit the AJE website (http://learn.aje.com/plos/) for a 15% discount off AJE services. To take advantage of our partnership with Editage, visit the Editage website (www.editage.com) and enter referral code PLOSEDIT for a 15% discount off Editage services.  If the PLOS editorial team finds any language issues in text that either AJE or Editage has edited, the service provider will re-edit the text for free.

Upon resubmission, please provide the following:

The name of the colleague or the details of the professional service that edited your manuscript

A copy of your manuscript showing your changes by either highlighting them or using track changes (uploaded as a *supporting information* file)

A clean copy of the edited manuscript (uploaded as the new *manuscript* file)

3. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

4. PLOS ONE now requires that authors provide the original uncropped and unadjusted images underlying all blot or gel results reported in a submission’s figures or Supporting Information files. This policy and the journal’s other requirements for blot/gel reporting and figure preparation are described in detail at https://journals.plos.org/plosone/s/figures#loc-blot-and-gel-reporting-requirements and https://journals.plos.org/plosone/s/figures#loc-preparing-figures-from-image-files. When you submit your revised manuscript, please ensure that your figures adhere fully to these guidelines and provide the original underlying images for all blot or gel data reported in your submission. See the following link for instructions on providing the original image data: https://journals.plos.org/plosone/s/figures#loc-original-images-for-blots-and-gels.

In your cover letter, please note whether your blot/gel image data are in Supporting Information or posted at a public data repository, provide the repository URL if relevant, and provide specific details as to which raw blot/gel images, if any, are not available. Email us at plosone@plos.org if you have any questions.

Additional Editor Comments:

Your manuscript will be reconsider after you complete the major revision suggested by referee 1.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Review of: Detection of carbapenem-resistant Klebsiella pneumonia on the basis of matrix assisted laser desorption ionization time-of-flight mass spectrometry by using supervised machine learning approach

The authors describe a machine learning algorithm that can differentiate between carbapenem resistant K. pneumonia and carbapenem sensitive K. pneumonia. Whole cells were analyzed with MALDI-ToF. While the manuscript could be of potential interest to readers, it needs major revisions to put it in its final form as outlined below.

Major comments:

1- The conclusion in the abstract section is vague. What do the authors mean by “analyzed spectra with the same MALDI-TOF MS settings used for bacterial identification. Furthermore, only one spectrum for an isolate was use to analysis.”

2- The introduction section is superficial and requires thorough revision in terms of the clarity of used language. Sentences are hard to read, although they describe mostly general information. The depth of the content, in my opinion, is not suitable for the audience. Some sections in the introduction also requires the addition of citations to support the authors’ claims. Furthermore, the compilation of the literature review is very confusing. I am not sure what the authors were trying to explain with the four approaches that identify drug resistant bacteria. There was no adequate description of what these approaches do in terms of drug resistant bacteria and how is the approach comparable to the presented work. Also no clear description of the used techniques in these approaches and the associated problems.

3- The authors did not attempt any peak identification at this stage, neither did they correct for isotope, adducts and salts during peak annotation. As such, I don’t see the validity in assessing the significance of the difference in the number of peaks between CSKP and CRKP. As such, figure 2 and discussions in the manuscript on that topic should be removed.

4- The authors selected 80 top ranked unidentified peaks to differentiate between CRKP and CSKP? Can the authors comment on the uniqueness of these peaks to solely carbapenem resistance and whether these peaks are also found in other penicillins that klebsiella is also resistant to such as amoxicillin and ampicillin?

5- Why didn’t the authors attempt for the identification of the peaks, especially with the m/z 9478.87 and 7705.009 as being mostly present in CRKP. Can the authors comment on the availability of data bases.

6- How was the feature selection using t-test conducted, i.e. what was the software used for it? Why did the authors use a univariate statistical test for feature selection? Why multivariate statistical testing such as PCA was not used?

Minor comments:

1- Line 26: “healthcare-associated pathogen” do the authors mean it infect healthcare professionals?

2- Authors should state the full name of the acronym at its first appearance throughout the document, e.g. ToF, CDC, WHO, PCR, MIC, CHCA and RF

3- Line 34: description of the utility of MALDI-ToF should be a part of the introduction and not the methods section of the abstract.

4- Line 40: authors already defined CSKP

5- Line 54: could the authors explain more on the urgency of carbapenem resistance as a threat to healthcare?

6- Line 56: do the authors mean that usually involve a combination of mechanisms at a time, the authors stated once mechanism? Also what is the role of B-Lactamase inhibitors as a combination therapy in addressing this resistance?

7- Line 58: what is KPC?

8- Line 58? What do the authors mean by “KPC-producing bacteria have increasingly been isolated worldwide” this is a very general term and not suitable for a scientific audience.

9- Line 68: the sentence needs to be thoroughly revised to indicate that polypepetides related to drug resistance could alter the spectra usually obtained from susceptible strains.

10- Line 74: what do the authors mean by: “The first three approaches usually detect small molecules (<74 1,000 Da) beyond the detection limit of the MALDI-TOF instruments now commonly employed in clinical microbiology, which spectra were acquired in a linear positive-ion mode at a laser frequency of 60 Hz across a ass/charge (m/z) ratio of 2,000 to 20,000.”

11- Line 89: can the authors comment on the advantages of analyzing whole cells over solvent extraction?

12- Line 109 and line 157: please use SVM abbreviation instead.

13- Line 122: can the authors comment on the possibility of carabapenemase phenotype susceptibility to meropenem or imipenem?

14- Line 125: authors need to be more specific in describing their specimen size. How much is a portion?

15- Line 127: did the authors do any optimization in regards to the MALDI matrix? Was there any matrices tried?

16- Line 131: can the authors reference the manufacturer specification that describes the use of E. Coli as a reference strain?

17- Line 133: I am not clear on the selection criteria for ensuring the quality of the protein extraction. Was 90% a score that was set in-house for assessing the quality of protein extraction or it is a common practice? When isolates were re-analyzed, were the scores from the repeats only used in the analysis or the scores before and after repeating were averaged? In general, was it an n=1 that was used throughout the study?

18- Line 134: did the authors correct for isotopes and salts in their spectral analysis?

19- Line 135: what was the software used for spectral analysis?

20- Line 151: did the authors apply any corrections for the multiple student t-testing?

21- Line 176: was the temperature set at 208 C?

22- Line 188: where were the isolates collected from the patient, are they respiratory fluids? Urine?

23- is there an ethics approval for collecting patient samples?

24- Line 202? What is the unit of dimensionality?

25- Line 245: the authors stated that random forest algorithm outperformed the other techniques, however there was one CRKP that was constantly misclassified using the random forest algorithm while was correctly classified using the other techniques? Can the authors comment on this finding? Did they run any blinded samples to confirm the superior performance of one algorithm over the other?

26- Line 264: the authors mentioned that they did not focus on targeted peaks from enzymes or metabolites related to drug resistance. However, they detected 3 peaks that were differentially present in CRKP and CSKP. Can the authors comment on the significance of their findings in regards to biomarker discovery?

27- Line 306: the authors explicitly detailed the advantages of random forest testing, however, they did not justify the inclusion of other algorithms in this study.

28- Figure 1 and figure 4 were not cited in text

29- Can the authors make a clear discrimination between the specification of the training set and the test set?

30- Figure 3. Can the authors specify what is on the x and y axis in panel A and B?

31- Reference formatting should be corrected for errors

Reviewer #2: In this Manuscript, the authors used MALDI-TOF-MS to differentiate between Carbapenem-resistant and Carbapenem-susceptible Klebsiella pneumonia pathogens (CRKB and CSKB). The authors used statistical techniques to analyze the MALDI-MS data of 46 CRKB and 49 CSKB and to build classifiers for differentiation purposes between these two types of pathogens.

The classifiers MS peaks are shown in table 1. From this list, they found that “ there were none of these peaks distinctive to CRKP or CSKP, although one peak (9478.87 206 Da) was present in 82.6% of the CRKP isolates and only 2.2% of the CSKP isolates, one peak (7705.009 Da) was present in 80.4% of the CRKP 207 isolates and only 2.2% of the CSKP isolates, 208 and the other peak (9541.41 Da) was present in 76.1% of the CRKP isolates and only 2.2% of 209 the CSKP isolates.”

Overall, the work sounds technically good, but it needs statistics experts to assess these statistical techniques used to build these classifiers peaks.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

14 Jan 2020

To the Editor

Plos ONE Editorial Office

Re: PONE-D-19-26377

Thank you for your constructive review of this manuscript. My co-authors appreciate the opportunity to address the reviewer's comments and to revise the manuscript accordingly.

Reviewer #1

Major comments:

1. The conclusion in the abstract section was rewritten.

2. The introduction was revised to state the problem more clearly and explain why it is important and why we chose the study method. We also made a brief review of the previous studies, including what these approaches do in terms of drug resistant bacteria, the techniques they used, and how these approaches compares to the presented work.

3. Figure 2 and discussions in the manuscript on that topic was removed, as suggested by the reviewer.

4. Klebsiella spp. are intrinsically resistant to penicillins, i.e. all of the CRKP as well as CSKP are penicillin resistant. Therefore, the peaks that differentiate between CRKP and CSKP are not related to penicillins, since both CRKP and CSKP are resistant to penicillin.

5. It is not able to find potential peptides/proteins of interest from just m/z ratio alone. We demonstrated that CRKPs can be differentiated from CSKPs through supervised machine learning method. It is possible to consider the specific attributes (such as m/z ratio) as biomarkers for the defined classes. Whether these target peaks are potential biomarkers require further study which analyze and process large series of bacterial strains (Line 296-298).

6. Feature selection using t-test was performed using Mathematica software v. 12 (Wolfram Research, Champaign, IL) (Line 173-174). Both t-test and PCA essentially reduce the dimensionality of the data, however, the overall performance by using PCA was inferior to t-test (accuracy was 85-92%, with 74%-83% of sensitivity and 92%-100% of specificity).

Minor comments:

1. The sentence was corrected as “a significant pathogen causing healthcare-associated infections”. (Line 26).

2. The full name of the acronyms at its first appearance were added: ToF (Line 27) CDC WHO (Line 49-50) PCR (Line 70) MIC (Line 146-147) CHCA (Line 157) RF (Line 109).

3. The abstract was rewritten.

4. Full name of CSKP Deleted.

5. The urgency of carbapenem resistance as a threat to healthcare was described in Introduction (Line 52-66).

6. This sentence was corrected as “Carbapenem resistance can be mediated by production of carbapenemases or by the combination of outer membrane porin expression disruption and production of various β-lactamases” (Line 50-52). The role of β-lactamase inhibitors combined with β-lactam antibiotic to address the resistance was added. (Line 88-89).

7. The full name of KPC was added. (Line 55)

8. The epidemiology of carbapenem-producing K. pneumoniae was described in Introduction (Line 55-63).

9. We rewrote the paragraph to indicate why spectral differences between resistant and sensitive isolates are expected (Line 78-97).

10. The first three approaches usually detect molecules beyond the detection limit of the MALDI-TOF instruments now commonly employed in clinical microbiology were explained more clearly (Line 78-100).

11. Solvent extraction involves several washing and centrifugation steps and is more labor-intensive than direct analysis of bacteria without additional protein extraction, which is usually done in routine work flow (Line 116-118).

12. SVM abbreviation was used.

13. The carbapenemase phenotype in the Vitek2 AES in our hospital predicts imipenem and ertapenem susceptibility, but not meropenem. Meropenem was not included in our panel (Line 145-149).

14. Sample size was specified as approximately three millimeters in diameter, according to the User Manual VITEK MS Clinical Use-Workflow) (Line 154).

15. All the procedures and reagents were conducted according to the manufacture’s recommendations for identification of bacteria in a routine laboratory without any modification.

16. The Reference of E. coli as reference strain was cited (Line 161).

17. The score 90% is a common practice. When isolates were re-analyzed, the scores from the repeats only were used in the analysis, not averaged. N=1 was used throughout this study (Line 164).

18. No isotopes or salts were used in the spectral analysis.

19. The data transferred from the Vitek MS acquisition station was analyzed by Vitek MS RUO v4.12 (Line 167).

20. In this study, each peak was treated as an independent observation (Line 183). It was not necessary to apply any corrections for the multiple student t-testing to adjust the individual p value for each peak to keep the overall error rate to less than or equal to the user-specified p cutoff value.

21. RAPD procedures were revised (Line 207-216).

22. Bacteria isolates included in this study were isolated from clinical specimens including 7 (7.4%) pus/wound, 5 (5.3%)sputum, 20 (21.1%) urine, 54 (56.8%) blood, and 5 (5.3%) body fluids like ascitic fluid, bile, pleural fluid and 4 (4.2%) other specimens like catheter tips.

23. This study was approved by the Institutional Review Board (VGHKS18-CT8-07) (Line 143-144).

24. Dimensionality is the number of peaks and is count data.

25. The conclusion that Random forest algorithm outperformed than the other algorithm was based on the ability to correct classification. Comments discussing the repeatedly misclassified isolate are written in discussion (Line 320-324). The L1O cross validation procedure was used to assess the predictive ability of different learning algorithms. The tested data is not part of the training set used in the building model and served as a blinded sample (Line 312-319).

26. These three differentially expressed peaks needs to be analyzed and processed using a large series of bacterial strains in the future to see if they are potential biomarkers (Line 296-298).

27. We used all the five machine learning algorithms that were offered in the Mathematica software v. 12 (Wolfram Research, Champaign, IL).

28. Figures and Tables were cited in the text.

29. The training set and testing set was described in Materials and method (Line 178-179).

30. The X and Y axis in Figure 3 were specified.

31. Reference formatting was corrected.

Reviewer #2

1. The corresponding author is a statistics expert and performed analysis and interpretation.

The revised manuscript is attached. We hope the revised manuscript is in satisfactory form and we look forward to hearing from you again soon.

Sincerely yours,

Fu-Chuen Chang

Professor

Department of Applied Mathematics

National Sun Yat-sen University

70 Lien-hai Rd.

Kaohsiung 804, Taiwan

Submitted filename: Response to reviewers-20191225R1.doc

Click here for additional data file.

16 Jan 2020

Detection of carbapenem-resistant Klebsiella pneumonia on the basis of matrix-assisted laser desorption ionization time-of-flight mass spectrometry by using supervised machine learning approach

PONE-D-19-26377R1

Dear Dr. Chang,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Joseph Banoub, Ph,D., D. Sc.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

This is to inform you that your manuscript has now been accepted for publication in PLOS one.

Reviewers' comments:

24 Jan 2020

PONE-D-19-26377R1

Detection of carbapenem-resistant Klebsiella pneumonia on the basis of matrix-assisted laser desorption ionization time-of-flight mass spectrometry by using supervised machine learning approach

Dear Dr. Chang:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Joseph Banoub

Academic Editor

PLOS ONE