Next-generation sequencing of microbial cell-free DNA for rapid noninvasive diagnosis of infectious diseases in immunocompromised hosts.

Background: Cell-free DNA (cfDNA) sequencing has emerged as an effective laboratory method for rapid and noninvasive diagnosis in prenatal screening testing, organ transplant rejection screening, and oncology liquid biopsies but clinical experience for use of this technology in diagnostic evaluation of infections in immunocompromised hosts is limited. 
Methods: We conducted an exploratory study using next-generation sequencing (NGS) for detection of microbial cfDNA in a cohort of ten immunocompromised patients with febrile neutropenia, pneumonia or intra-abdominal infection. 
Results: Pathogen identification by cfDNA NGS demonstrated positive agreement with conventional diagnostic laboratory methods in 7 (70%) cases, including patients with proven/probable invasive aspergillosis, Pneumocystis jirovecii pneumonia, Stenotrophomonas maltophilia bacteremia, Cytomegalovirus and Adenovirus viremia. NGS results were discordant in 3 (30%) cases including two patients with culture negative sepsis who had undergone hematopoietic stem cell transplant in whom cfDNA testing identified the potential etiological agent of sepsis; and one kidney transplant recipient with invasive aspergillosis who had received >6 months of antifungal therapy prior to NGS testing.
Conclusion: These observations support the clinical utility of measurement of microbial cfDNA sequencing from peripheral blood for rapid noninvasive diagnosis of infections in immunocompromised hosts. Larger studies are needed. Copyright:

Introduction

Infections are a leading cause of morbidity and mortality among immunocompromised individuals [1– 4]. Bacteremia occurs in up to 25% of all patients with neutropenia and fever [5]. Infection is a leading cause of non-relapse mortality among hematopoietic cell transplantation (HCT) recipients [6]. The incidence of bacteremia [7– 9] and double-stranded DNA viral reactivation [10] is higher than 40% and 90%, respectively, within the first 100 days post-transplant. The cumulative incidence rates of proven/probable invasive fungal infections during the first year after allogeneic HCT with non-myeloablative conditioning is 19% [11]. Infection is also a common complication of chimeric antigen receptor-modified T (CAR-T)-cell immunotherapy with 28-day cumulative incidence of 23% after CAR-T-cell infusion [12].

Establishing a microbiological diagnosis of infectious diseases in this vulnerable population is often challenging for a number of reasons. i) Prior exposure to antibiotics and antifungals which confounds the yield of blood cultures; indeed, most patients with neutropenia and fever will have no infectious etiology documented [5]. ii) Low sensitivity of mycobacterial and fungal cultures; some microorganisms, such as fastidious bacteria, mycobacteria and dimorphic fungi require longer incubation periods; and blood cultures in almost half of patients with candidemia are negative [13, 14]. iii) Tissue biopsies are often precluded due to the risk of bleeding in the setting of thrombocytopenia, coagulopathy in those with liver disease or hemodynamic instability in critically ill patients. A delay in diagnosis in patients with invasive fungal infection results in higher mortality [15, 16]. Thus, there is an unmet need for novel, rapid, cost-effective, noninvasive diagnostic methods in the field.

Cell-free DNA (cfDNA) technology has been used successfully in noninvasive prenatal testing, organ transplant rejection screening, and oncology liquid biopsies [17– 22]. In recent years, this technology has been developed for use in infectious disease diagnostics [23, 24]. Detection of microbial cfDNA by next generation sequencing (NGS) is an accurate and precise way of identifying and quantifying pathogens [25]. The Karius ® Test relies on sequencing of microbial cfDNA circulating in plasma to identify over 1,000 pathogens, including bacteria, viruses and fungi, from a 5 ml blood sample [25]. This novel diagnostic tool has been recently validated in a study showing that microbial cfDNA NGS identified 94% of microbes identified by conventional blood culture in patients with sepsis [25] and has excellent correlation with quantitative PCR testing in patients with cytomegalovirus (CMV) [23, 25].

Recent reports indicate that NGS measuring microbial cfDNA is useful in the diagnosis of cases of Streptococcus pneumoniae-related hemolytic uremic syndrome, Coxiella burnetii endocarditis, invasive Mycobacterium chimaera infection, Nocardia cyriacigeorgica pneumonia, Capnocytophaga canimorsus sepsis, M. tuberculosis complex and M. haemophilum infections, M. bovis aortitis; Candida spp., Aspergillus spp ., non -Aspergillus molds invasive infections; Pneumocystis jirovecii pneumonia (PJP) , Toxoplasma gondii infection and chorioamnionitis, among others [24, 26– 33]. Among 21 patients with culture-positive infective endocarditis, cfDNA NGS identified the same organism as blood cultures in 20 patients (95% sensitivity) and additionally identified Enterococcus faecalis in one out of the three patients with definitive culture-negative endocarditis [34]. Of note, in this study the cfDNA NGS test identified pathogens causing endocarditis in patients pre-treated with antibiotics up to 30 days prior to initial sample collection.

Here we evaluated the clinical utility of NGS for detection of microbial cfDNA in plasma in a cohort of ten patients receiving chemotherapy or transplants with episodes of febrile neutropenia, sepsis or documented infection.

Methods

Study design and study subjects

This was an exploratory study sponsored by Karius, Inc. A total of ten cfDNA kits were provided to the investigators free of charge to be used during a 60-day period. The main goal of this pilot study was to assess the performance of the cfDNA NGS test, compared to standard microbiological evaluation, in immunocompromised patients with documented infection and those undergoing diagnostic evaluation for febrile illness. Patients were enrolled if they had a clinical scenario (e.g., such as fever or pulmonary nodules) suspected or confirmed to be infectious in origin. Half of the patients enrolled in this pilot study had an established diagnosis of infection prior to NGS testing. Our goal in such patients who had documented infection prior to enrollment was to evaluate the positive agreement between NGS and conventional diagnostic testing results. Adult patients followed at the Sylvester Comprehensive Cancer Center were enrolled between July 31 and October 2, 2018. Inclusion criteria were: i) age >18 years old; ii) patients must have received chemotherapy or transplant; and iii) must have had a febrile illness or documented infection (e.g., positive blood cultures, clinical/radiographic evidence of pneumonia). There were no exclusion criteria. The study was approved by the University of Miami Institutional Review Board (IRB approval #20080899), consistent with principles in the Declaration of Helsinki. Each participant provided written informed consent for their inclusion in the study. No sample size calculation was done; instead the number of patients enrolled was entirely dependent on the number of cfDNA kits made available for the pilot study.

Sample collection and processing

Blood samples (5 mL) were collected in BD vacutainer plasma preparation tubes. Samples were collected at the time of suspected or confirmed infection diagnosis. Within 1 hour of sample collection, tubes were spun down at 1,100 RCF for 10 min at room temperature. Samples were shipped overnight to Karius, Inc. (Redwood City, CA).

Measurement of cfDNA using NGS

Cell-free DNA was extracted from plasma, NGS libraries were prepared, and sequencing was performed on an Illumina NextSeq ®500. Sequencing reads identified as human were removed, and remaining sequences were aligned to a curated pathogen database. Any of over 1,000 organisms in the Karius clinical reportable range found to be present above a predefined statistical threshold were reported as previously described [24]. The quantity for each organism identified was expressed in Molecules Per Microliter (MPM), the number of DNA sequencing reads from the reported organism present per microliter of plasma.

The Karius ® Test

Reference genomes for Homo sapiens and microorganisms (bacteria, viruses, fungi/molds, and other eukaryotic pathogens) were retrieved from the National Center for Biotechnology Information (NCBI) ftp site ( NCBI, U.S. National Library of Medicine (NLM), Human Genome, release GRCh38.p7, and NCBI, U.S. NLM, Microbial Genomes, respectively). Sequence similarities between microorganism references were inspected to identify taxonomic mislabeling and sequence contamination. From the reference genomes passing these quality controls, a subset was selected to maximize sequence diversity. As part of the selection process, NCBI BioSample data were used to ensure the inclusion of reference genomes from both clinical and non-clinical isolates. The final reference genome dataset included over 21,000 reference genomes, containing over 2.7 million sequences. Selected sequences were collected into a single FASTA file and used to generate our microorganism reference BLAST database. A subset of these taxa, including 1251 clinically significant microorganisms, was used as the clinical reportable range.

The selection of organisms in the clinical reportable range (CRR) was performed as follows. A candidate list was generated by two board-certified infectious disease physicians by including (a) DNA viruses, culturable bacteria, additional fastidious and unculturable bacteria, mycobacteria, and eukaryotic pathogens from a clinical infectious diseases reference textbook [35] and a number of infectious disease references, (b) organisms in the pathogen database referenced in published case reports, and (c) reference genomes sequenced from human clinical isolates (as indicated by the NCBI BioSample resource) with publications supporting pathogenicity. Organisms from the above list that were associated with high-quality reference genomes, as determined by our reference database QC process (see above), were used to further narrow the range. Finally, organisms observed as sporadic environmental contamination were excluded from the CRR in order to prevent false-positive calls, e.g., Propionibacterium acnes, Acinetobacter lwoffii, and several Methylobacterium spp. The full list of pathogens detected can be found online: kariusdx.com/pathogenlist/3.3 (where 3.3 is the Karius Test version used in this study). The sequence database is continuously curated to minimize human cross-reactivity as well as cross-reactivity between pathogens and is screened to mitigate contamination with sequences from human or other organisms.

Plasma samples were thawed, centrifuged at 16,000 RCF for 10 min, and spiked with a known concentration of synthetic DNA molecules for quality control purposes. Cell-free DNA was extracted from 0.5 mL plasma using a magnetic bead-based method (Omega Bio-tek Mag-Bind® cfDNA kit; catalog number M3298-01, Norcross, GA). DNA libraries for sequencing are constructed using a modified Ovation® Ultralow System V2 library preparation kit (NuGEN, San Carlos, CA). Negative controls (buffer only instead of plasma) and positive controls (healthy plasma spiked with a known mixture of microbial DNA fragments) were processed alongside patient samples in every batch. Samples were multiplexed with other samples and sequenced on an Illumina NextSeq® 500.

Primary sequencing output files were processed using bcl2fastq (v2.17.1.14) to generate the demultiplexed sequencing reads files. Reads were filtered based on sequencing quality and trimmed based on partial or full adapter sequence. The bowtie2 (version 2.2.4) tool was used to align the remaining reads against Karius’ human and synthetic-molecules references. Sequencing reads exhibiting strong alignment against the human references or the synthetic molecule references were collected and excluded from further analysis. Remaining reads were aligned against Karius’ proprietary microorganism reference database using NCBI-blast (version 2.2.30+). A mixture model was used to assign a likelihood to the complete collection of sequencing reads that included the read sequence probabilities and the (unknown) abundances of each taxon in the sample. An expectation-maximization algorithm was applied to compute the maximum likelihood estimate of each taxon abundance. Only taxa whose abundances rejected the null hypothesis of originating from environmental contamination (as calculated from the negative controls) at high significance levels were reported. The quantity for each organism identified was expressed in molecules per microliter (MPM), the number of DNA sequencing reads from the reported organism present per microliter of plasma. MPM values are calculated from the ratio between the number of sequencing reads assigned to an organism and to an internal control (see Methods in Blauwkamp et al. [25]). Depending on both the concentration of the microbe as well as its genome length, sequencing coverage can range from a few reads and up to >10x for high-concentration shorter viral genomes. Importantly, the MPM value is not affected by sequencing depth or human cell-free DNA concentration in the sample. The entire process from DNA extraction through analysis was typically completed within 28 hours.

Results

Background patient information

The characteristics of the patients studied are presented in Table 1. The median age was 56 years (range, 20–65) with 60% of participants being males. Except for a kidney transplant recipient, all other patients had underlying hematological malignancy and/or had received an HCT. All but one (patient #2) were admitted in the hospital at the time of clinical evaluation. All the patients were receiving antimicrobials at the time of plasma sample collection. Three patients had neutropenia (absolute neutrophil count <500/µL) at the time of febrile illness. All febrile patients had blood cultures collected within 24 hours of plasma sample collection for NGS.

Table 1.

Clinical characteristics of study subjects and results of next-generation sequencing of cell-free DNA.

Patient	Age, gender	Underlying disease	Clinical scenario	Sample from CVC	Days of antibiotics/ antifungals prior to blood draw [a]	Conventional diagnostic method results [b]	Microbial cfDNA pathogen results	MPM	Reference values [c]	Correlation [d]
1 [*]	65F	Kidney transplant	Pyogenic intra-abdominal infection	No	18/182	Aspergillus fumigatus detected by PCR and culture in abdominal fluid	Negative ( Aspergillus fumigatus [e])	15 [e]	<10	No [e]
2	21M	NHL, HCT day +342	Mediastinal lymphadenopathy	No	0/8	Negative fungal serologies and antigens BAL and lymph node tissue cultures negative	Negative			Yes
3 [*]	20M	AML, HCT day +9	Neutropenic fever, diarrhea	Yes	8/2	CMV detected <137 IU/mL (subsequently peaked at 2,621 IU/mL) Blood cultures and C. difficile PCR negative	Cytomegalovirus	108	<10	Yes
4 [*]	64F	B-ALL MMUD day +291	Fever, cough, lung mass	Yes	6/5	Pneumocystis jirovecii BAL PCR+	Pneumocystis jirovecii	263	<10	Yes
5	37M	Relapsed DLBCL after CAR-T	Neutropenic fever, weakness, diarrhea, cough	Yes	21/5	Adenovirus 480 copies/mL (subsequently peaked at 2,600 copies/mL)	Adenovirus	845	<10	Yes
6 [*]	56M	AML, MMUD day +290	Pulmonary nodules (recently diagnosed IA) admitted with SOB	Yes	6/21	CMV detected <137 IU/mL (subsequently peaked at 440 IU/mL) Repeat BAL negative	Cytomegalovirus	93	<10	Yes
7	44M	DLBCL	Fevers, pulmonary nodules	Yes	3/3	Blood cultures negative	Rothia mucilaginosa	20	<10	No
8	60F	MDS, HCT day+160, GI-GVHD	Septic shock, multi-organ failure	Yes	15/10	Blood cultures negative	Escherichia coli Lactobacillus rhamnosus Torque teno virus	2,492 308 91	<17 <10 <10	No
9	55F	Multiple myeloma	Pneumonia	Yes	2/0	Negative BAL studies	Negative			Yes
10 [*f]	58M	AML	Neutropenic fever, pulmonary nodules, sepsis	Yes	120/129	S. maltophilia in blood cultures Pan- Aspergillus PCR+ in BAL Serum galactomannan+	Stenotrophomonas maltophilia Aspergillus oryzae Staphylococcus epidermidis	236,594 11,533 9,673	<83 <10 <17	Yes

*These patients had documented infection by standard laboratory methods prior to Karius ® Test

a Refers to empiric or targeted therapy only. It does not include days of antimicrobial prophylaxis.

b Blood cultures were obtained within 24h of plasma sample for NGS in all patients and resulted as negative unless specified otherwise in the table.

c Reference value is the 97.5th percentile in self-reported healthy adults for whom the Karius ® Test was performed

dCorrelation between Karius ® Test and standard laboratory methods

e Aspergillus fumigatus reads were present in the raw data but below the statistical threshold for a positive test result. Kidney transplant complicated with perinephric abscess due to Aspergillus fumigatus requiring multiple abdominal washouts. The patient had received >6 months of voriconazole and few days of combination therapy with micafungin prior to NGS testing

f Initial cfDNA testing performed 7 weeks prior had only identified S. epidermidis and EBV. At that time, BAL and transbronchial biopsy results were unrevealing.

ALL, acute lymphoblastic leukemia; AML, acute myeloid leukemia; BAL, bronchoalveolar lavage; CAR-T, chimeric antigen receptor-modified T-cell immunotherapy; cfDNA, cell-free DNA; CMV, cytomegalovirus; CVC, central venous catheter; DLBCL, diffuse large B cell lymphoma; GI-GVHD, gastrointestinal graft-versus-host disease; HCT, hematopoietic cell transplantation; F, female; M, male; MPM, molecules per microliter; NGS, next-generation sequencing; NLH, Non-Hodgkin lymphoma; SOB, shortness of breath.

Results of NGS for detecting microbial cfDNA

In this cohort of immunocompromised hosts, pathogen identification by cfDNA NGS demonstrated positive agreement with conventional diagnostic laboratory methods in 7 (70%) cases including positive concordant results in 5 cases and negative concordant results in 2 cases ( Table 1). The kidney transplant recipient had an Aspergillus fumigatus perinephric abscess, and Aspergillus cfDNA levels, although detected in plasma, were below the positive reporting threshold. However, among patients with hematological malignancy in whom a microbiological diagnosis was established (n=5), cfDNA NGS testing correlated with other methods in all cases. This included patients with proven/probable invasive aspergillosis, PJP, Stenotrophomonas maltophilia bacteremia, CMV and adenovirus viremia. Among four patients with hematological malignancy with negative standard laboratory testing, the NGS test identified the potential cause of bacterial sepsis in two patients ( Rhotia mucilaginosa in patient #7 and Escherichia coli in patient #8; Table 1), both of whom had a compatible clinical scenario and experienced good clinical response to antibiotic therapy with resolution of fever and hypotension.

Five patients (#1, 3, 4, 6, and 10) had documented infection diagnosed by conventional diagnostic methods prior to NGS testing. Notably, the Karius test detected circulating cfDNA of all the organisms identified by conventional diagnostic methods in these five patients, but as mentioned above for patient #1 levels were below the positive reporting threshold ( Table 1). This is, the reported NGS results were concordant with the results of conventional diagnostic laboratory methods in 4 out of 5 patients with documented infection prior to NGS.

After excluding the five patients in whom a microbiological diagnosis was established prior to NGS testing, and the two patients in whom there was negative agreement between conventional testing and NGS (i.e., a diagnosis could not be established), there were only three cases in whom we could assess the impact of NGS results on clinical decision making. In patient #5 adenovirus viremia was detected via NGS, which triggered assessment of adenovirus DNA levels in blood by PCR and ultimately led to initiation of antiviral therapy. In patients #7 and #8 with culture-negative sepsis, NGS did not change management per se in terms of escalation or de-escalation of therapy but it supported the diagnosis of bacterial sepsis and both patients completed a course of antibiotic therapy with clinical improvement.

Discussion

Here we report our experience using cfDNA NGS in the evaluation of immunocompromised patients—predominantly those with hematological malignancy—with febrile illness or documented invasive infections. The study cohort included a heterogeneous group of clinical scenarios, including intra-abdominal infection, pulmonary nodules/pneumonia, neutropenic fever, and septic shock. The results of this proof-of-concept study, where most of the patients had an established diagnosis of infection prior to NGS testing, complement recent reports studying the use of cfDNA NGS in immunocompromised hosts. In a recent study of 55 patients with neutropenic fever, cfDNA testing had positive agreement with conventional blood cultures in 9 of 10 patients in whom blood cultures identified a causative organism of sepsis. Using clinical adjudication by three infectious diseases specialists, cfDNA NGS had a sensitivity of 85.4% (41/48) and specificity of 100% (7/7) [36]. Thus, this test is a promising diagnostic tool in neutropenic fever, a clinical scenario where conventional work up fails to identify an etiological agent in a majority of cases [5]. Another study evaluated 40 pediatric patients with prolonged neutropenia and fever (>96h) despite administration of antibiotics for suspected fungal infection (the authors excluded patients who had received antifungal therapy for >4 days); in this study cfDNA NGS identified fungal pathogens including Aspergillus fumigatus, Rhizopus spp., Candida albicans, Candida glabrata and Pneumocystis jirovecii [37].

Except for patients diagnosed with viral infections (e.g., patients #3, #5 and #6 with adenovirus or CMV viremia), all other patients were receiving antimicrobial therapies that were active against the organism(s) identified ( Table 1) suggesting that NGS may be able to detect organisms in the setting of effective treatment. For example, patient #4 who was diagnosed with PJP, had detectable levels of Pneumocystis jirovecii DNA in blood despite receiving three days of trimethoprim/sulfamethoxazole treatment dose at the time of NGS testing; and patient #10 had positive NGS testing for Aspergillus oryzae despite having received >120 days of anti-mold therapy including triple antifungal regimen (isavuconazole, micafungin and liposomal amphotericin B) at the time of NGS testing.

There is limited data on Karius test performance for invasive mold infections. In a retrospective case-control study of 57 HCT recipients with proven/probable pulmonary invasive mold infections, the cfDNA NGS test identified 83% (5/6) of molds among patients with non- Aspergillus infections; but among those with Aspergillus proven/probable disease, Aspergillus fumigatus was only identified in 13.7% (7/51) of cases [38]. In the report by Armstrong et al. [37], in a cohort of 40 pediatric hematology-oncology and HCT patients, sequencing of circulating cfDNA detected fungal pathogens in five of seven cases with proven and probable invasive fungal disease, and correlated with microbiological diagnosis in four of six proven cases. In a recent report by Hong et al. [24], in seven out of nine subjects (including seven immunocompromised hosts) with proven invasive fungal infection, plasma NGS testing detected the same fungus identified from the biopsy tissue at the genus level. The fungi identified by plasma NGS included Aspergillus spp. and non- Aspergillus molds such as Scedosporium, Rhizopus, and Cunninghamella [24]. In that report, there was one case where the plasma sample was obtained after at least 15 days of anti- Aspergillus therapy, and NGS testing did not identify the causal organism of invasive fungal infection. Similarly, for the kidney transplant patient reported here with invasive aspergillosis, in whom Aspergillus fumigatus cfDNA levels in plasma were detected below the reporting threshold, six months of anti- Aspergillus therapy (including combination of voriconazole plus micafungin at the time of NGS testing) had been administered prior to the time of plasma collection. Thus, prolonged antifungal therapy prior to sample collection (e.g., >7–14 days) might interfere with detection of fungal DNA. One exception to this might be patients with profound prolonged neutropenia (e.g., absolute neutrophil count <100 cells/mL for more than 7 days) and those with refractory acute leukemia, in whom NGS might detect Aspergillus spp. DNA in peripheral blood despite significant exposure to antifungal therapy like it occurred with patient #10.

Although NGS has been used for screening of allograft rejection in solid organ transplant recipients [17– 19, 23], there are limited data with the use of NGS for diagnosis of infections in this population. A recent study demonstrated strong correlation between clinical test results and cfDNA derived from CMV in a cohort of lung transplant recipients [23]. In addition, cfDNA revealed undiagnosed cases of infection with microsporidia and pathogenic viruses, including adenovirus and human herpesvirus 6 among lung transplant patients [23].

Recently, Fung et al. reported three patients who received allogeneic HCT transplant in whom NGS cfDNA facilitated the diagnosis of an uncommon presentation of Chlamydia trachomatis and recurrent and metastatic complications of Staphylococcus aureus bacteremia before standard microbiology [39].

The fact that in our cohort cfDNA NGS testing identified the potential cause of febrile illness in two patients with culture-negative sepsis who had a compatible clinical syndrome and responded well to antibiotic therapy supports the notion that NGS testing can be a useful diagnostic tool, particularly when conventional blood cultures are negative. The Karius ® Test pathogen-specific reference ranges have been established using cfDNA levels from healthy donors. Patient #8 had detectable levels of Torque teno virus, which belongs to Anelloviridae family and is considered to lack pathogenic potential; this patient also had detection of Lactobacillus spp, which is part of normal gastrointestinal flora and usually interpreted as a contaminant when isolated from blood cultures. Similarly, for patient #10, in addition to pathogenic organisms such as Aspergillus sp. and Stenotrophomonas maltophilia, cfDNA of Staphylococcus epidermidis, of unclear clinical significance in this patient and likely contaminant, was also detected. This suggests the possibility that cfDNA NSG might on occasion yield detection of members of the commensal microbiota or viroma. Thus, the results of cfDNA NGS technology need to be interpreted with caution and in conjunction with other laboratory, radiological and clinical findings.

To our surprise, however, even though many of the patients tested had mucosal barrier damage (e.g., mucositis) allowing for bacterial translocation from the gut, the Karius ® Test did not show a non-specific gut flora signal. The test was negative in patients in whom we failed to establish a microbiological diagnosis for their febrile illness, and when positive, typically correlated with conventional laboratory testing. Whether the currently defined cfDNA thresholds are optimal for identifying and quantifying pathogens of clinical relevance in highly vulnerable immunocompromised hosts will require further study. Importantly, the turnaround time for results was consistently within 48 hours, which is quite rapid considering that samples were shipped overnight from our institution located in Florida to the Karius Inc. laboratory in California.

Lack of control group, small number of patients and the heterogeneity of the cohort in terms of underlying diseases and causes of immunosuppression represent major limitations of this report. Larger clinical trials evaluating plasma NGS in patients with cancer and undergoing transplant are ongoing ( NCT03226158, NCT03262584, NCT02912117, NCT02804464). Until larger cohort data becomes available, our observations suggest that detection of microbial cfDNA using NGS is valuable for the rapid noninvasive diagnosis of infectious complications following chemotherapy or transplantation.

Conclusion

In this small cohort of immunocompromised hosts, the NGS correlated with standard microbiological testing in 70% of cases suggesting this technology might be useful in this clinical setting, particularly for patients in whom bronchoscopy or biopsy for tissue diagnosis is not feasible. As with other novel laboratory diagnostics used in clinical practice, the results of cfDNA NGS technology need to be interpreted with caution and in conjunction with other laboratory, radiological and clinical findings. Larger studies are needed to validate these findings.

Data availability

Underlying data

Microbial cfDNA NGS for Rapid Noninvasive Diagnosis of Infectious Diseases in Immunocompromised Hosts, BioProject accession number PRJNA554271

I will approve version 4 of the article.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

My queries have been adequately addressed, I have no further comments.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

This submission (v3) by Camargo et al. describes the use of plasma cell-free DNA metagenomic sequencing (cf-mNGS) for pathogen detection performed by Karius in a limited number of immunocompromised patients. Results showed that the cf-mNGS results detected the same organism as conventional microbiological methods in 5 patients (one patient had two organisms detected by both conventional and cf-mNGS methods). One patient with a known diagnosis of Aspergillus fumigatus had negative cf-mNGS results, although organism reads were detected below reporting thresholds. There were four patients with no conventional diagnosis; two of these had negative cf-mNGS results, and 2 had additional organisms detected by cf-mNGS. There were 5 additional organisms detected by cf-mNGS in 3 patients, of these 2 were adjudicated as clinically significant (not stated which in paper, but I’m presuming E. coli and R. mucilaginosa) that may have caused clinical sepsis, as the patients responded to therapy which covered these species. In addition to these organisms adjudicated as clinically significant, cf-mNGS detected 3 organisms in 2 patients that were clinically adjudicated to be not significant.

The paper is generally well written and understandable, although the small number of patients enrolled limits the conclusions that can be drawn, and it would be useful to see the authors expand the study group. For the study design, it is particularly important to know whether these patients were enrolled due to their interesting conventional results and clinical course or whether all patients admitted during this time frame were enrolled. This sort of clinical trial is highly susceptible to bias due to “cherry-picking” patients with severe infections that are highly likely to yield positive cf-mNGS findings.

Most of the clinical need for diagnostic utility comes in patients that are undiagnosed at the time testing is sent (often despite extensive workup), and the test performance in this population may be very different. Most of the patients had an established diagnosis prior to sending cf-mNGS, and it would be best to show in the tables whether the diagnosis was already known at the time cf-mNGS testing was sent.

This version improves on the prior description of cf-mNGS test sensitivity, but the concordant results should be broken down into positive concordant results (5 cases) and negative concordant results (2 cases), along with additional organism detections.

The statement made in the abstract and throughout the article that cf-mNGS identified the etiological agent in two patients with culture negative sepsis is misleading and overly interpreted by the authors. The data show that cf-mNGS detected 5 additional organisms in 3 patients. Of these, two were clinically interpreted as potentially consistent with the patient’s suspected infection. I don’t think the article actually says which cases, but from the table I’m presuming that they intend the E. coli and Rothia findings. However, without additional discrepancy testing, one cannot say that this has made the diagnosis. It does seem that the E. coli finding with high MPM is likely clinically significant. The case with Rothia mucilaginosa is more suspect, it was detected with relatively low MPM, and Rothia is a reasonably common contaminant organism that can be seen in mNGS data.

The authors don’t discuss how certain organisms were interpreted to be clinically insignificant other than TTV which is well established as a non-pathogen. Presumably the Lactobacillus rhamnosus was thought to be a likely contaminant. This is a reasonable clinical interpretation, but again is not definitive. The finding of Staphylococcus epidermidis is even more problematic as this can certainly cause clinical sepsis, and it was found at fairly high MPM. It could also be a sample contaminant, and so is very difficult to clinically adjudicate. Unfortunately, clinical response while on therapy that should cover the organism is still not proof that the organism caused the infection or was a true positive finding. In order to make a definitive statement about diagnosis, additional discrepancy testing via orthogonal methods would be required.

In the discussion, the authors state sensitivity and specificity after clinical adjudication, and also state that the cfDNA NGS testing “identified the cause of febrile illness in two patients with culture-negative sepsis”, which is not accurate as described above. Without true gold standard results, sensitivity and specificity are not the appropriate terms. Given the issues with clinical interpretation, the authors should provide positive and negative agreement before and after clinical adjudication, and the reasons for their interpretations should be shown. The detection of additional organisms that were not found on other tests is a potential strength of cf-mNGS testing, but is also very difficult to interpret, and there is potential for many false positives. Clinical interpretation in these cases is important, and the authors should provide more detail on their approach to this and how to deal with potential false-positive results.

The interpretation of cf-mNGS test utility needs to be done using objective standards, and requires a prospective study design (without inclusion of cases with known diagnosis). The authors should show whether patients were already on empiric therapy that covered the organism detected or not, and whether any change was made to therapy decisions based on results, along with clinical outcomes of therapy. This ability (or at least potential) to modify treatment decisions based on cf-mNGS is an important outcome metric, especially since definitive outcome studies (clinical cure) require large numbers of patients to show effectiveness. However, if the cf-mNGS testing was sent retrospectively or on known diagnosis cases already under treatment, that should be stated for these cases since there is not a potential to change therapy in those situations. Essentially the reader will want to know how test results changed patient management, which is not shown here.

Patient #3 (CMV viremia) seems excluded from the discussion of viral infection in the second paragraph of the discussion.

The Aspergillus fumigatus case with reads detected below thresholds is interesting and could have more discussion about whether this finding would be communicated to providers for potential followup testing or correlation with other data. The presentation in Table 1 is confusing as the MPM is shown as 15 with reference value < 10, seemingly a positive result above threshold, but the text states that it is below threshold.

The technical description of the Karius test is fairly general, which is understandable given that it is proprietary, but it does not allow other researchers to assess it independently. More detail could be provided regarding the thresholds for reporting. For example, in the Analysis Pipeline section, the actual quantitation for “high significance” needed to report organisms that are not environmental contaminants as shown by negative controls should be shown. Adequate sequencing depth should be stated as well as the number of base pairs sequenced. The selection of organisms for reporting and organisms excluded from reporting should be referenced or potentially shown in an appendix.

The methods state that MPM value is not affected by human cell-free DNA concentration, but others have shown that the human DNA portion does affect the number of reads recovered. The authors should state of reference how normalization is done to adjust for samples with differing amounts of human cell-free DNA.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

This submission (v3) by Camargo et al. describes the use of plasma cell-free DNA metagenomic sequencing (cf-mNGS) for pathogen detection performed by Karius in a limited number of immunocompromised patients. Results showed that the cf-mNGS results detected the same organism as conventional microbiological methods in 5 patients (one patient had two organisms detected by both conventional and cf-mNGS methods). One patient with a known diagnosis of Aspergillus fumigatus had negative cf-mNGS results, although organism reads were detected below reporting thresholds. There were four patients with no conventional diagnosis; two of these had negative cf-mNGS results, and 2 had additional organisms detected by cf-mNGS. There were 5 additional organisms detected by cf-mNGS in 3 patients, of these 2 were adjudicated as clinically significant (not stated which in paper, but I’m presuming E. coli and R. mucilaginosa) that may have caused clinical sepsis, as the patients responded to therapy which covered these species. In addition to these organisms adjudicated as clinically significant, cf-mNGS detected 3 organisms in 2 patients that were clinically adjudicated to be not significant.

Authors’ response: we thank the reviewer for his insightful comments/suggestions that have improved the quality of the manuscript.

The paper is generally well written and understandable, although the small number of patients enrolled limits the conclusions that can be drawn, and it would be useful to see the authors expand the study group. For the study design, it is particularly important to know whether these patients were enrolled due to their interesting conventional results and clinical course or whether all patients admitted during this time frame were enrolled. This sort of clinical trial is highly susceptible to bias due to “cherry-picking” patients with severe infections that are highly likely to yield positive cf-mNGS findings.

Authors’ response:  We have edited the methods and results to improve clarity.

We agree with the reviewer that the small cohort size is a limitation and for this reason we acknowledge that this data should be considered preliminary and results validated in larger cohorts. However, we feel our observations suggest potential clinical utility of NGS in immunocompromised hosts with febrile illness, particularly those in whom invasive diagnostic methods (e.g. tissue biopsy) are precluded, and as such these observations set the foundation for larger studies.

With regards to potential selection bias, this pilot project was designed as proof-of-concept study to assess the performance of the NGS in the “real-world” clinical setting and the agreement rate with conventional diagnostic testing.  The Karius kits were provided to the investigators free of charge for a 60-day period so efforts were made to use all the kits prior to such deadline. As such, the patient population was not completely homogenous. Patients were enrolled if they had a clinical scenario (e.g, fever, pulmonary nodules)  suspected or confirmed to be infectious in origin, and indeed most (4/7) patients in whom Karius was positive (e.g., such as patient #4 with PJP and patient #10 with invasive aspergillosis) had documented infection prior to Karius testing. Our goal in such patients who had documented infection was to evaluate the positive agreement between NGS and conventional diagnostic testing results.   The other three patients correspond to the two cases of culture negative sepsis where Karius identified etiological agent of septicemia (patients #7 and #8), and one patient (patient #5) in whom Adenovirus (AdV) viremia detected via NGS triggered serial assessment of AdV DNA levels by PCR and ultimately led to initiation of antiviral therapy.  In patient #1 diagnosis of invasive aspergillosis was known prior to Karius testing but NGS was negative, likely due to chronic antifungal therapy prior to NGS negatively impacting yield of the test.  Five patients with no documented infection at the time of NGS, underwent NGS testing concurrent with standard microbiological evaluation for febrile illness which is often due to infection in immunocompromised hosts, and so clearly –as stated by reviewer- pre-test probability of a positive result was high. As stated in the conclusion of the manuscript, larger cohort studies are needed.

Most of the clinical need for diagnostic utility comes in patients that are undiagnosed at the time testing is sent (often despite extensive workup), and the test performance in this population may be very different. Most of the patients had an established diagnosis prior to sending cf-mNGS, and it would be best to show in the tables whether the diagnosis was already known at the time cf-mNGS testing was sent.

Authors’ response:  We agree with reviewer. cf-mNGS is most useful in patients in whom previous work up has been non diagnostic and those in whom invasive procedures such as bronchoscopy or tissue biopsy are precluded due to low cell counts or hemodynamic instability. It is in these settings where cf-mNGS can provide information that conventional testing would not.  Thus, your point is well taken and we have added information regarding whether the diagnosis was already known at the time cf-mNGS testing in the revised methods, results and Table.

This version improves on the prior description of cf-mNGS test sensitivity, but the concordant results should be broken down into positive concordant results (5 cases) and negative concordant results (2 cases), along with additional organism detections.

Authors’ response:  This break down of positive and negative concordant results has been added to the revised version. Specific organism detections are outlined in Table 1.

The statement made in the abstract and throughout the article that cf-mNGS identified the etiological agent in two patients with culture negative sepsis is misleading and overly interpreted by the authors. The data show that cf-mNGS detected 5 additional organisms in 3 patients. Of these, two were clinically interpreted as potentially consistent with the patient’s suspected infection. I don’t think the article actually says which cases, but from the table I’m presuming that they intend the E. coli and Rothia findings. However, without additional discrepancy testing, one cannot say that this has made the diagnosis. It does seem that the E. coli finding with high MPM is likely clinically significant. The case with Rothia mucilaginosa is more suspect, it was detected with relatively low MPM, and Rothia is a reasonably common contaminant organism that can be seen in mNGS data.

Authors’ response:  We have edited manuscript (both abstract and results section) to avoid strong statements and emphasize suspected culprit of sepsis in these two patients. For patient #8 Diagn Microbiol Infect Dis. 2016 May;85(1):116-20.)

The authors don’t discuss how certain organisms were interpreted to be clinically insignificant other than TTV which is well established as a non-pathogen. Presumably the Lactobacillus rhamnosus was thought to be a likely contaminant. This is a reasonable clinical interpretation, but again is not definitive. The finding of Staphylococcus epidermidis is even more problematic as this can certainly cause clinical sepsis, and it was found at fairly high MPM. It could also be a sample contaminant, and so is very difficult to clinically adjudicate. Unfortunately, clinical response while on therapy that should cover the organism is still not proof that the organism caused the infection or was a true positive finding. In order to make a definitive statement about diagnosis, additional discrepancy testing via orthogonal methods would be required.

Authors’ response: As mentioned above it is difficult to establish if

In the discussion, the authors state sensitivity and specificity after clinical adjudication, and also state that the cfDNA NGS testing “identified the cause of febrile illness in two patients with culture-negative sepsis”, which is not accurate as described above. Without true gold standard results, sensitivity and specificity are not the appropriate terms. Given the issues with clinical interpretation, the authors should provide positive and negative agreement before and after clinical adjudication, and the reasons for their interpretations should be shown. The detection of additional organisms that were not found on other tests is a potential strength of cf-mNGS testing, but is also very difficult to interpret, and there is potential for many false positives. Clinical interpretation in these cases is important, and the authors should provide more detail on their approach to this and how to deal with potential false-positive results.

Authors’ response: We have edited manuscript to avoid strong statements. We also edited the discussion to emphasize that detection of commensal organisms can occur and results need to be interpreted with caution and in the context of other clinical, laboratory and radiological findings.

The interpretation of cf-mNGS test utility needs to be done using objective standards, and requires a prospective study design (without inclusion of cases with known diagnosis). The authors should show whether patients were already on empiric therapy that covered the organism detected or not, and whether any change was made to therapy decisions based on results, along with clinical outcomes of therapy. This ability (or at least potential) to modify treatment decisions based on cf-mNGS is an important outcome metric, especially since definitive outcome studies (clinical cure) require large numbers of patients to show effectiveness. However, if the cf-mNGS testing was sent retrospectively or on known diagnosis cases already under treatment, that should be stated for these cases since there is not a potential to change therapy in those situations. Essentially the reader will want to know how test results changed patient management, which is not shown here.

Authors’ response:  Table 1 includes the days of antimicrobial (empiric and targeted, not prophylaxis) that each patient received prior to blood sample collection for NSG, and in the second paragraph of the discussion we mention that “except for patients diagnosed with viral infections, all other patients were receiving antimicrobial therapies that were active against the organism(s) identified”.

Given the limitations with study design, after excluding the five patients in whom a diagnosis was known prior to NGS testing, and the two patients in whom both conventional testing and NGS were negative, there are only three cases in whom we could assess modifications in treatment decisions based on NGS.   In patient #5 Adenovirus (AdV) viremia was detected via NGS which triggered serial assessment of AdV DNA levels by PCR and ultimately led to initiation of antiviral therapy. In patients #7 and 8 with culture-negative sepsis, NGS did not change management per se in terms of escalation or de-escalation of therapy but it supported the diagnosis of bacterial sepsis and both patients completed courses of antibiotic therapy with clinical improvement.  This information has been added to the discussion in the revised version.

Patient #3 (CMV viremia) seems excluded from the discussion of viral infection in the second paragraph of the discussion.

Authors’ response: Patient #3 was added to this sentence.

The Aspergillus fumigatus case with reads detected below thresholds is interesting and could have more discussion about whether this finding would be communicated to providers for potential followup testing or correlation with other data.

Authors’ response: Evidence supporting the presence of microorganisms not reaching statistical significance thresholds may be observed. These observations may provide additional value given the clinical context, but with the caveats related to the reduced analytical specificity accompanying them. Such observations can be communicated in the form of a clinical consultation to providers with the associated caveats and understanding that the observations are not part of the CAP-accredited, CLIA-certified, validated test.

The presentation in Table 1 is confusing as the MPM is shown as 15 with reference value < 10, seemingly a positive result above threshold, but the text states that it is below threshold.

Authors’ response:  There are many different comparisons and analyses that are performed in order to determine whether a microbe is “Reported” by Karius. The text stating that the

The reference interval is the 97.5% highest MPM value observed in a cohort of 167 plasma samples from healthy blood donors.  Since

The technical description of the Karius test is fairly general, which is understandable given that it is proprietary, but it does not allow other researchers to assess it independently. More detail could be provided regarding the thresholds for reporting. For example, in the Analysis Pipeline section, the actual quantitation for “high significance” needed to report organisms that are not environmental contaminants as shown by negative controls should be shown. Adequate sequencing depth should be stated as well as the number of base pairs sequenced.

Authors’ response: As described previously (Blauwkamp et al. Nat Microbiol. 2019 Apr;4(4):663-674), the number of reads required to pass QC is dependent on the total amount of plasma DNA in each sample. Samples that contain higher amounts of cfDNA get more sequencing reads. The number of base pairs sequenced has been highly optimized for the application and remains proprietary.

The selection of organisms for reporting and organisms excluded from reporting should be referenced or potentially shown in an appendix.

Authors’ response:  The full list of organisms for reporting is available online (kariusdx.com/pathogenlist). Please note that all samples were processed with version 3.3 of the Karius Test. The list of organisms specific to version 3.3 can be found here: kariusdx.com/pathogenlist/3.3. For clarification, we added the Karius Test version as well as the link to the pathogen list to the manuscript.

The methods state that MPM value is not affected by human cell-free DNA concentration, but others have shown that the human DNA portion does affect the number of reads recovered.

Authors’ response: The amount of human DNA does indeed affect the number of microbial reads recovered.  That is why we designed our quantitation strategy differently, in a way that is 100% independent of the number of human DNA reads.  See Figure 3d and Supplementary Figure 1 in Blauwkamp et al (Nat Microbiol. 2019 Apr;4(4):663-674) for data addressing performance in light of this issue. There is a 10-fold difference in the amount of human DNA in low versus high, but the MPM quantity is not affected.

The authors should state of reference how normalization is done to adjust for samples with differing amounts of human cell-free DNA.

Authors’ response: As described above, input normalization is not required for accurate quantification. Some additional detail is provided in Blauwkamp et al. Nat Microbiol. 2019 Apr;4(4):663-674.

In the article revision, I think the authors responded to my comments, except I would like to see an explanation why case 1 is considered a negative result with 15 MPM where the reference value is < 10, normally a value above the reference range is considered positive.

We would like to apologize for not having clarified this aspect previously. The Karius Test determines presence/absence of a pathogen based on several technical characteristics, such as number of reads aligned and abundance in negative control samples. The reference value based on the asymptomatic/normal population is only provided as annotation and context of what to expect in the "normal population" and is never used in determining whether to filter out or report a pathogen.

This is an interesting, single-center small case series of the Karius test performance in 10 immunocompromised patients. Agreement with standard microbiological methods was 70% (7 of 10), but with only one "false negative" case of Aspergillus abscess while on treatment. I agree with the authors this is a promising assay that we have all been using in TOID, and as such support its clinical utility, but a scientific manuscript should stick to standardized methodology and report the results unbiased to contribute to our knowledge and set the ground for future studies.

As such:

The authors need to provide more details about the study design. It seems to be a prospective study, but inclusion criteria are not clearly defined. The au mention "availability of kits". Was this a prospective study funded by the sponsor? Were they consecutive pts? More specifically, it seems strange they would enroll a patient who had been on antifungals for so long. Were the tests ordered as part of routine clinical care? The study design needs clarification.

Instead of reporting 100% sensitivity "among the patients with HM and confirmed infections", I would prefer to (also) see the "overall agreement" of 70% with standard micro methods.

There is limited data on Karius performance for invasive mold infections, and the authors should acknowledge that, referring to additional studies, specifically DMID 2018;92:210 [1] (7 of 9 pts correctly identified), Pediatr Blood Cancer 2018;66:e27734 [2] (4 of 6 pts correctly identified).

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

This is an interesting, single-center small case series of the Karius test performance in 10 immunocompromised patients. Agreement with standard microbiological methods was 70% (7 of 10), but with only one "false negative" case of Aspergillus abscess while on treatment. I agree with the authors this is a promising assay that we have all been using in TOID, and as such support its clinical utility, but a scientific manuscript should stick to standardized methodology and report the results unbiased to contribute to our knowledge and set the ground for future studies.

Authors’ response: We thank the reviewer for his insightful comments which have improved the quality of the manuscript. You have bought up very valid points. Text has been modified accordingly.  Please find response to your specific comments below.

As such:

1. The authors need to provide more details about the study design. It seems to be a prospective study, but inclusion criteria are not clearly defined. The au mention "availability of kits". Was this a prospective study funded by the sponsor? Were they consecutive pts? More specifically, it seems strange they would enroll a patient who had been on antifungals for so long. Were the tests ordered as part of routine clinical care? The study design needs clarification.

Authors’ response: Study design and study subject section have been edited to improve clarity.

2. Instead of reporting 100% sensitivity "among the patients with HM and confirmed infections", I would prefer to (also) see the "overall agreement" of 70% with standard micro methods.

Authors’ response: We agree with the reviewer. We have removed this strong statement and abstract and results section have been edited accordingly.

3. There is limited data on Karius performance for invasive mold infections, and the authors should acknowledge that, referring to additional studies, specifically DMID 2018;92:210 (7 of 9 pts correctly identified), Pediatr Blood Cancer 2018;66:e27734 (4 of 6 pts correctly identified).

Authors’ response:  Hong et al. and Armstrong et al. reports were already included in the discussion.  We have expanded this paragraph in the discussion and added the study by Hill et al.

Camargo et al. report on results of plasma next-generation sequencing (NGS) for infectious diagnosis in a series of ten patients at risk for infection due to underlying immunocompromise. In addition to demonstrating concordance of NGS testing with conventional microbiologic diagnostic testing methods in seven of the ten patients, NGS testing identified a possibly causative organism in two cases in which conventional test results were negative. In one of the ten patients, PCR and culture detected Aspergillus; while NGS was able to detect this organism, it was below the assay detection limit. The authors conclude that this pilot study supports the conclusion that there may be clinical benefit for using this test in this population of patients, warranting more rigorous studies of test performance.

Overall, the manuscript does not contain significant flaws that would preclude indexing. The authors appropriately acknowledge the limitations of this small pilot study, which by design is not able to determine detailed test performance characteristics such as sensitivity or specificity. Accordingly, my comments are fairly minor and include:

The methods and results sections report the NGS data in “molecules per microliter” (MPM), which is defined as “the number of DNA sequencing reads from the reported organism present per microliter of plasma.” Although this is a fairly straightforward concept, in the context of sequencing it is not totally clear how the more commonly used concepts of sequencing “breadth” and depth” (e.g., as discussed by Sims et al., 2014 [1]) apply to MPM. Is it possible to explain MPM in a little more detail? E.g., is there a minimum depth of sequencing that needs to be satisfied for there to be "one" MPM? Does every base in a sequence need to have a certain number of reads?

The methods also refer to the removal of possible “false-positive calls” from common environmental contaminants – can any examples of organisms that would fit these criteria be provided? Since immunocompromised patients may be at risk of infection from uncommon organisms associated with their environment (particularly uncommon moulds), it is important to understand what may not be reported.

It’s not completely clear from the text whether results of NGS testing were used for clinical care. Although in most cases it is stated that an established diagnosis of infection was made prior to NGS testing, at least two patients had negative conventional testing and were reported to respond to therapy directed at the organisms identified by NGS. Were these patients responding to empiric treatment, or did NGS results direct the treatment?

Several places in the manuscript refer to the abdominal infection in the kidney transplant patient as “deep-seated,” but only once is the more medically precise term “abscess” used to describe this infection. Also, can more details about this infection be provided? This is important as the patient appears to have had negative testing (unless the detection limit was lowered below that which the assay typically uses), providing additional clues to situations in which NGS testing may provide false negative results (more on this in the next comment).

All of the patients had received some antibiotic (antibacterial and/or antifungal) treatment at the time of NGS testing, and one of the more impressive and useful aspects of the test is that it may be able to detect organisms in the setting of effective treatment. However, although a duration of treatment is reported in Table 1, whether this treatment would have been effective against the organism identified is not clear, complicating the interpretation of the data. For example, the Discussion notes that “several months” of treatment had been given to the patient with the Aspergillus abscess prior to NGS testing, but it is not totally clear if it was all directed against Aspergillus – if so, the negative result is less concerning (and the ability to detect the organism below the reportable threshold is still impressive). Similarly, did patient 10 receive 129 days of antifungal treatment with anti- Aspergillus activity prior to his positive test? Although it is understandable why prophylactic treatment would not generally be reported for all patients, was patient 4 receiving any prophylaxis against Pneumocystis?

There are a handful of fairly minor editorial corrections also, including:

On page 4 the “clinically reportable range” is referred to as “our” clinically reportable range – although it is acknowledged that three of the authors are from the company which performed the NGS testing, given that certain details of the testing are ultimately proprietary perhaps a more generic statement (as simple as “the clinically reportable range”) would be preferred.

In that same paragraph, although the Mandell textbook is considered by many to be the “go-to” reference for clinical infectious diseases, perhaps it could be referred to as “a clinical infectious diseases reference textbook” or some less subjective phrase? Also, it appears the 8 th edition is being referenced, which I believe is actually from 2015 (not 2014)?

The third paragraph in the left column on page 7 contains the phrase “there is limited data…” – as data is plural this should be corrected to “there are limited data…”.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Reviewer’s comment:  Camargo et al. report on results of plasma next-generation sequencing (NGS) for infectious diagnosis in a series of ten patients at risk for infection due to underlying immunocompromise. In addition to demonstrating concordance of NGS testing with conventional microbiologic diagnostic testing methods in seven of the ten patients, NGS testing identified a possibly causative organism in two cases in which conventional test results were negative. In one of the ten patients, PCR and culture detected Aspergillus; while NGS was able to detect this organism, it was below the assay detection limit. The authors conclude that this pilot study supports the conclusion that there may be clinical benefit for using this test in this population of patients, warranting more rigorous studies of test performance.

Overall, the manuscript does not contain significant flaws that would preclude indexing. The authors appropriately acknowledge the limitations of this small pilot study, which by design is not able to determine detailed test performance characteristics such as sensitivity or specificity. Accordingly, my comments are fairly minor and include:

Author’s response: Thank you for this thorough review of the manuscript and your insightful comments that have improved the clarity and quality of the paper. Below there is a point-by-point response your inquiries.

Reviewer’s comment: Author’s response: We revised the text to clarify the connection between MPM and sequencing depth. In the Karius Test, the total number of reads observed per organism is often orders of magnitude lower than in applications where the concepts of sequencing breadth and depth are typically used (such as genome assembly). There is indeed a minimum amount of sequencing information that must be obtained in order for a sample to pass quality control criteria which is proportional to the concentration of cell-free DNA in that patient’s sample.  Importantly, the MPM value is not affected by sequencing depth or human cell-free DNA concentration in the sample.

The methods and results sections report the NGS data in “molecules per microliter” (MPM), which is defined as “the number of DNA sequencing reads from the reported organism present per microliter of plasma.” Although this is a fairly straightforward concept, in the context of sequencing it is not totally clear how the more commonly used concepts of sequencing “breadth” and depth” (e.g., as discussed by Sims et al., 2014 ) apply to MPM. Is it possible to explain MPM in a little more detail? E.g., is there a minimum depth of sequencing that needs to be satisfied for there to be "one" MPM? Does every base in a sequence need to have a certain number of reads?

Reviewer’s comment: Author’s response:  The manuscript was revised to provide further clarity on process to guarantee that the Karius Test does not include frequently observed environmental contaminants. Here, it is important to mention that those taxa are not removed on a case-by-case basis but are completely out of scope for the whole test.

The methods also refer to the removal of possible “false-positive calls” from common environmental contaminants – can any examples of organisms that would fit these criteria be provided? Since immunocompromised patients may be at risk of infection from uncommon organisms associated with their environment (particularly uncommon moulds), it is important to understand what may not be reported.

Reviewer’s comment: Author’s response:  Two aspects of the study should be considered here. i) This was a pilot, proof-of-concept, study in which the majority of patients had an established diagnosis prior to Karius test. ii) We found that NGS correlated well with results from standard diagnostic evaluation. Consequently, the NGS results did not largely influence clinical decision making in this cohort except perhaps in patient #5 where adenovirus PCR in blood was ordered (to confirm DNAemia and monitor viral kinetics which often guide initiation of antiviral therapy) after NGS yielded adenovirus; and in patient #8 in whom clinical presentation was very suggestive of gastrointestinal sepsis due to bacterial translocation in the setting of GVHD, and although blood cultures were negative decision was made to continue empiric antibiotic therapy to complete a course in view of clinical improvement and results of NGS testing.

It’s not completely clear from the text whether results of NGS testing were used for clinical care. Although in most cases it is stated that an established diagnosis of infection was made prior to NGS testing, at least two patients had negative conventional testing and were reported to respond to therapy directed at the organisms identified by NGS. Were these patients responding to empiric treatment, or did NGS results direct the treatment?

Reviewer’s comment: Author’s response:  We edited the abstract and manuscript to avoid the term “deep-seated”.  This was a 65-year-old patient who presented with a 1-month history of fever, generalized body aches, malaise and abdominal pain.  Ten months prior, the patient received a kidney transplant complicated with perinephric abscess due to Aspergillus fumigatus that required multiple abdominal washouts and several months of antifungal therapy.  Namely, at the time of admission when NGS was sent, the patient had received >6 months of voriconazole (serum levels: 0.5-6.1 mcg/ml; target: 1.5-5 mcg/ml), and micafungin was added for persistent fungal infection. We have added some of this clinical information to the footnote of Table 1. At the time of blood sample for NGS the patient was receiving dual antifungal therapy. We suspect this is the reason why the Karius test was negative. We describe the possibility of false negative results in patients receiving antifungal therapy in the discussion: “prolonged antifungal therapy prior to sample collection (e.g., >7-14 days) might interfere with detection of fungal DNA.”

Several places in the manuscript refer to the abdominal infection in the kidney transplant patient as “deep-seated,” but only once is the more medically precise term “abscess” used to describe this infection. Also, can more details about this infection be provided? This is important as the patient appears to have had negative testing (unless the detection limit was lowered below that which the assay typically uses), providing additional clues to situations in which NGS testing may provide false negative results (more on this in the next comment).

Reviewer’s comment: Author’s response: Your point is well taken. The kidney transplant patient received >6 months of anti- Aspergillus therapy and was receiving combination of voriconazole plus micafungin at the time of sample collection for NGS testing. Patient 10 had indeed received 129 days of anti-mold prophylaxis/empiric treatment with various agents (including posaconazole, isavuconazole, liposomal amphotericin B [L-AmB]), and was receiving combination of isavuconazole, L-AmB and micafungin at the time of NGS testing. We believe significant fungal burden in the setting of refractory leukemia and prolonged neutropenia facilitated the detection of fungal DNA despite triple antifungal therapy. Patient 4 was not receiving prophylaxis against Pneumocystis (atovaquone had been discontinued 3 months prior presentation since CD4>400) but had received 3 days of TMP-SMX treatment dose at the time of NGS testing.  We have edited the discussion to clarify that all patients (except those with viral infections) were on active antimicrobial therapy at the time of NGS testing.

All of the patients had received some antibiotic (antibacterial and/or antifungal) treatment at the time of NGS testing, and one of the more impressive and useful aspects of the test is that it may be able to detect organisms in the setting of effective treatment. However, although a duration of treatment is reported in Table 1, whether this treatment would have been effective against the organism identified is not clear, complicating the interpretation of the data. For example, the Discussion notes that “several months” of treatment had been given to the patient with the Aspergillus abscess prior to NGS testing, but it is not totally clear if it was all directed against Aspergillus – if so, the negative result is less concerning (and the ability to detect the organism below the reportable threshold is still impressive). Similarly, did patient 10 receive 129 days of antifungal treatment with anti- Aspergillus activity prior to his positive test? Although it is understandable why prophylactic treatment would not generally be reported for all patients, was patient 4 receiving any prophylaxis against Pneumocystis?

Reviewer’s comment:

There are a handful of fairly minor editorial corrections also, including:

Author’s response: This has been modified.

On page 4 the “clinically reportable range” is referred to as “our” clinically reportable range – although it is acknowledged that three of the authors are from the company which performed the NGS testing, given that certain details of the testing are ultimately proprietary perhaps a more generic statement (as simple as “the clinically reportable range”) would be preferred.

Reviewer’s comment: Author’s response: Text has been modified. 2014 is correct.

In that same paragraph, although the Mandell textbook is considered by many to be the “go-to” reference for clinical infectious diseases, perhaps it could be referred to as “a clinical infectious diseases reference textbook” or some less subjective phrase? Also, it appears the 8 th edition is being referenced, which I believe is actually from 2015 (not 2014)?

Reviewer’s comment: Author’s response: Thank you. This grammar mistake has been corrected in the revised version.

The third paragraph in the left column on page 7 contains the phrase “there is limited data…” – as data is plural this should be corrected to “there are limited data…”.