Laboratory evaluation of the BioFire FilmArray Pneumonia plus panel compared to conventional methods for the identification of bacteria in lower respiratory tract specimens: a prospective cross-sectional study from South Africa.

Lower respiratory tract infections are important causes of morbidity and mortality. The global increase in antimicrobial resistance necessitates rapid diagnostic assays. The BioFire FilmArray Pneumonia plus (FAPP) panel is a Food and Drug Administration-approved multiplex polymerase chain reaction assay that detects the most important etiological agents of pneumonia and associated antibiotic resistance genes, in approximately 1 hour. This study assessed the diagnostic performance of this assay by comparing it to conventional culture methods in the analysis of 59 lower respiratory tract specimens. The sensitivity and specificity of the FAPP panel for bacterial detection were 92.0% (95% confidence interval [CI], 80.8% to 97.8%) and 93.8% (95% CI, 91.1% to 95.3%) respectively. For detecting antibiotic resistance, the positive- and negative percent agreement were 100% (95% CI, 81.5% to 100.0%) and 98.5% (95% CI, 216 96.7% to 99.4%) respectively. The FAPP panel was found to be highly accurate in evaluating tracheal aspirate specimens from hospitalized patients.

Introduction

Globally, lower respiratory tract infections (LRTIs) are the leading infectious cause of death (Troeger et al., 2017, Noviello and Huang, 2019). Pneumonia guidelines now recommend that antibiotics be initiated as early as possible, based on evidence from several studies (Mandell et al., 2007, Lim and Woodhead, 2011, Kao et al., 2019, Iregui et al., 2002, Jain et al., 2015). With the worldwide increase in antibiotic-resistant bacteria, rapid and accurate diagnosis of the etiological agent(s) of LRTI is essential for appropriate management (Noviello and Huang, 2019). The current gold standard for diagnosing bacterial LRTIs is based on culture, with long turnaround times and is no longer fit-for-purpose when dealing with acute, serious infections (Trotter et al., 2019). Rapid diagnostic technologies that detect antimicrobial resistance genes allow for timely implementation of infection prevention and control measures (Sullivan and Bard, 2019). Consequently, rapid multiplex polymerase chain reaction (PCR) assays are fast becoming valuable diagnostic tools (Alby and Mitchell, 2018). A number of commercially available PCR assays have recently been approved for the diagnosis of pneumonia by the US Food and Drug Administration (US FDA, Washington, DC, USA) (Alby and Mitchell, 2018, Ramanan et al., 2018). One such assay, the BioFire FilmArray Pneumonia plus (FAPP) panel (BioFire Diagnostics LLC, Salt Lake City, UT, USA) uses a nested PCR technique with melting curve analysis to detect 15 common bacteria, 3 atypical bacteria, and 9 viruses that cause both community- and hospital-acquired LRTI, as well as 7 antibiotic resistance markers, in approximately 1 hour (BioMérieux Diagnostics). To facilitate clinical decision making, genomic material of certain bacteria that are detected in the assay are then also reported semiquantitatively with bins representing 104-, 105-, 106-, and ≥107 copies per milliliter (copies/mL) (BioFire Diagnostics). The atypical bacteria and viruses are reported qualitatively.

Bacterial organisms semiquantitatively detectable by the FAPP panel include Acinetobacter calcoaceticus-baumannii complex, Enterobacter cloacae complex, Escherichia coli, Haemophilus influenzae, Klebsiella aerogenes, Klebsiella oxytoca, Klebsiella pneumoniae group, Moraxella catarrhalis, Proteus species, Pseudomonas aeruginosa, Serratia marcescens, Staphylococcus aureus, Streptococcus agalactiae, Streptococcus pneumoniae, and Streptococcus pyogenes (BioMérieux Diagnostics). Atypical bacteria that are qualitatively detected include: Chlamydia pneumoniae, Mycoplasma pneumoniae and Legionella pneumophila (BioMérieux Diagnostics). Antibiotic resistance genes detectable by the panel include CTX-M, IMP, KPC, NDM, VIM and OXA-48-like, mecA/C and MREJ (BioMérieux Diagnostics). The panel also has the ability to detect 9 viruses that commonly cause respiratory tract infections, including, Adenovirus, Coronavirus, Middle East Respiratory Coronavirus, Human Metapneumovirus, Human Rhinovirus/Enterovirus, Influenza A, Influenza B, Parainfluenza virus, and Respiratory Syncytial virus (BioMérieux Diagnostics). Although data are limited, the FAPP panel has demonstrated good overall performance (Alby and Mitchell, 2018, Yoo et al., 2020, Lee et al., 2019, Faron et al., 2018, Kerr et al., 2018, Furukawa et al., 2019, Huang et al., 2018).

This study aimed to establish the usefulness of the FAPP panel, compared to culture, to detect bacterial agents in lower respiratory tract specimens in a prospective, real world setting in South Africa. We further endeavored to compare the turnaround time between the FAPP panel and culture to determine the theoretical impact on infection prevention and control measures. This study is unique because it focused on the utility of the FAPP panel in hospitalized and ventilated patients. This is also the first evaluation of the FAPP panel performed in an African setting.

Materials and methods

Study design and setting

This cross-sectional study was conducted at the National Health Laboratory Service, Tshwane Academic Division, Microbiology Laboratory (Pretoria, South Africa) from November 2019 to March 2020. This study was approved by the Faculty of Health Sciences Research Ethics Committee of the University of Pretoria (Ethics Reference Number: 133/2020). As this was a laboratory-based study using anonymized data, informed consent requirements were waived.

Lower respiratory tract specimens

The study analyzed lower respiratory tract specimens received from Steve Biko Academic Hospital (Pretoria, South Africa), collected from persons with suspected LRTI. The sampling strategy was to include all specimens that meet the inclusion criteria consecutively as a case series. The inclusion criteria comprised endotracheal aspirate (ETA) or bronchoalveolar lavage (BAL) specimens from patients admitted to Steve Biko Academic Hospital, sent to the National Health Laboratory Service Tshwane Academic Division Microbiology laboratory with a request for microscopy, culture and susceptibility testing. Repeat specimens from patients of whom a previous specimen was already included in the study were excluded from evaluation. The sample thus represents 59 unique clinical patient specimens.

FilmArray Pneumonia plus panel

The FAPP panel testing method was done in accordance with the manufacturer's instructions. Briefly, a sterile disposable sample transfer device (Copan Flock Technologies, Brescia, Italy) was used to transfer approximately 200 µL of the specimen to the sample injection vial. It was then mixed with the provided sample buffer. Next, this solution was loaded into the FilmArray pouch, which in turn was loaded into the FIlmArray instrument (Release Version 2, Software Module Version: BioFire.FilmArray.FALink.UI 2.1.273.0) where automated nucleic acid extraction, multiplex PCR and postamplification analysis were performed. Upon completion of each run, a report was generated that documented detected organisms and antibiotic resistance genes.

Conventional methods used as reference standard

Specimen processing and evaluation were performed according to standard laboratory procedures to detect respiratory pathogens (Leber, 2016). Briefly, a direct Gram stain was performed followed by subculture onto solid agar media. All specimens were inoculated, using a 1 µL disposable loop (Davies Diagnostics, Randburg, South Africa), onto 5% sheep blood agar, Haemophilus isolation agar and MacConkey agar (Diagnostic Media Products, Johannesburg, South Africa) and streaked for single colonies and semiquantification. After inoculation the agar plates were incubated at 35°C in 5% CO2 for 18 to 24 hours. Plates were observed after 18- to 24 -hour incubation and incubated for an additional 2 days before being reported as no bacterial growth. Evaluation, interpretation, isolation, and subsequent identification of colonies on culture plates were performed by qualified laboratory technologists, and reviewed by the researchers, according to the guidelines of standard laboratory procedures (Leber, 2016). Quantification and identification of each unique morphological type of colony cultured was done after 18- to 24-hour incubation, according to the following scale: 1+ (0 to 10 colony forming units [CFU]/µL), 2+ (11 to 100 CFU/µL), 3+ (>100 CFU/µL). Identification and antibiotic susceptibility testing was performed using VITEK 2 (bioMérieux, Marcy I'Etoile, France) according to the manufacturer's instructions. To confirm the presence of carbapenemase production the modified carbapenem inactivation method was performed on all Enterobacterales and P. aeruginosa isolates that had carbapenem nonsusceptible results on VITEK 2 (CLSI 2020). To confirm the presence of mecA/C the cefoxitin screen and oxacillin susceptibility results from VITEK 2 were used. To confirm the presence of extended spectrum beta-lactamase encoding genes, susceptibility reports from VITEK 2 were used. All antimicrobial susceptibility testing data were interpreted according to CLSI 30th edition M-100 2020 breakpoints (CLSI 2020). The date and time of each specimen's collection, arrival in the laboratory and release of results were documented.

Statistical analysis

Statistical analysis employed MedCalc, Version 19.2.1 (MedCalc Software Ltd, Ostend, Belgium).The identification of bacterial targets by the FAPP panel was compared to the reference standard to calculate sensitivity, specificity, positive predictive value and negative predictive value along with the 95% confidence interval (CI) for each variable. The antibiotic resistance genes detected by the FAPP panel was compared to phenotypic methods described above. Since the reference standard used is imperfect, only positive percent agreement and negative percent agreement were calculated for these targets. The results from the FAPP panel were considered concordant when they were consistent with the results of conventional methods and discordant when they were inconsistent with the results of conventional methods. A true positive result means both methods detected the target organism or resistance mechanism. A true negative result means neither method detected the target organism or resistance mechanism. A false positive result means the FAPP panel detected the target organism or resistance mechanism when conventional methods did not. A false negative result means the FAPP panel did not detect the target organism or resistance mechanism when conventional method did. The FilmArray semiquantification bin category of detected organisms was also compared to the culture semiquantification category using the Pearson correlation coefficient. The turnaround time of each method was compared using the Student's 2-sample t test. Testing was done at the 0.05 level of significance.

Results

The study evaluated 59 lower respiratory tract specimens, 58 (98.3%) were ETA specimens and 1 was a BAL specimen. The majority of specimens (n = 52, 88.1%) were collected from patients in intensive care units (ICU). Approximately half (n = 30, 50.8%) of the specimens were from male patients. The mean age of patients from which specimens were collected was 46 years (range, 2 months to 86 years). Neutrophils were observed on microscopy for all specimens included.

Both the FAPP panel and culture detected at least 1 organism on 42 (71.2%) of specimens. The mean number of bacterial targets detected by the FAPP panel per specimen was 2 (range, 0 to 8). Culture detected a mean of 1 (range, 0 to 5) organism per specimen. All of the bacterial (excluding the atypical bacteria) targets present on the FAPP panel were detected. A. baumannii complex were detected most frequently (18/42, 42.9%), followed by K. pneumoniae group (15/42, 35.7%) and S. aureus (12/42, 28.6%). The least frequently detected targets were K. aerogenes (1/42, 2.4%), S. agalactiae (1/42, 2.4%), and S. pyogenes (1/42, 2.4%). Culture generally detected fewer organisms compared to the FAPP panel and failed to detect any M. catarrhalis, Proteus species, S. agalactiae, and S. pyogenes. Table 1 provides a comparison between the targets detected by FAPP panel and culture. However, culture was able to detect organisms that are not included on the FAPP panel. Of these, 4 (6.8%) specimens potentially harbored clinically relevant organisms, including 1 Citrobacter freundii, 1 Providencia stuartii, and 2 Stenotrophomonas maltophilia. Table 2 provides a list of organisms detected by culture that are not included on the FAPP panel.

Table 1

Comparison of FilmArray Pneumonia plus panel and culture for the detection of targets included on the FilmArray Pneumonia plus panel.

Target	FilmArray Pneumonia plusn (%)	Culturen (%)
Bacteria
Acinetobacter baumannii complex	18 (30.5)	13 (22.0)
Enterobacter cloacae complex	7 (11.9)	1 (1.7)
Escherichia coli	8 (13.6)	4 (6.8)
Haemophilus influenzae	8 (13.6)	2 (3.4)
Klebsiella aerogenes	1 (1.7)	2 (3.4)
Klebsiella oxytoca	2 (3.4)	1 (1.7)
Klebsiella pneumoniae group	15 (25.4)	8 (13.6)
Moraxella catarrhalis	2 (3.4)	0 (0.0)
Proteus species	5 (8.5)	0 (0.0)
Pseudomonas aeruginosa	10 (16.9)	9 (15.3)
Serratia marcescens	5 (8.5)	3 (5.1)
Staphylococcus aureus	12 (20.3)	6 (10.2)
Streptococcus agalactiae	1 (1.7)	0 (0.0)
Streptococcus pneumoniae	3 (5.1)	1 (1.7)
Streptococcus pyogenes	1 (1.7)	0 (0.0)
Antibiotic resistance genes
CTX-M	7 (11.9)	5 (8.5)
IMP	0 (0.0)	0 (0.0)
KPC	0 (0.0)	0 (0.0)
mecA/C and MREJ	1 (1.7)	1 (1.7)
NDM	8 (13.6)	7 (11.9)
OXA-48-like	7 (11.9)	4 (6.8)
VIM	1 (1.7)	1 (1.7)
Atypical bacteria
Chlamydia pneumoniae	0 (0.0)	Not tested
Legionella pneumophila	1 (1.7)	Not tested
Mycoplasma pneumoniae	0 (0.0)	Not tested
Viruses
Adenovirus	0 (0.0)	Not tested
Coronavirus	1 (1.7)	Not tested
Human Metapneumovirus	0 (0.0)	Not tested
Human Rhinovirus/Enterovirus	4 (6.8)	Not tested
Influenzae A	0 (0.0)	Not tested
Influenzae B	0 (0.0)	Not tested
MERS-CoV	0 (0.0)	Not tested
Parainfluenza Virus	1 (1.7)	Not tested
Respiratory Syncytial Virus	0 (0.0)	Not tested

Table 2

Comparison of FilmArray Pneumonia plus panel and culture for the detection of targets not included on the FilmArray Pneumonia plus panel.

Target	FilmArray Pneumonia plusn (%)	Culturen (%)
Off panel
Acinetobacter haemolyticus	Not tested	1 (1.7)
Aspergillus fumigatus	Not tested	1 (1.7)
Bacillus species	Not tested	1 (1.7)
Candida albicans	Not tested	2 (3.4)
Citrobacter freundii	Not tested	1 (1.7)
Coagulase negative Staphylococcus	Not tested	3 (5.1)
Corynebacterium species	Not tested	1 (1.7)
Haemophilus parainfluenzae	Not tested	1 (1.7)
Moraxella species	Not tested	3 (5.1)
Pantoea species	Not tested	1 (1.7)
Providencia stuartii	Not tested	1 (1.7)
Pseudomonas putida	Not tested	1 (1.7)
Raoultella planticola	Not tested	1 (1.7)
Stenotrophomonas maltophilia	Not tested	2 (3.4)
Streptococcus species	Not tested	2 (3.4)
Yeast not Candida albicans	Not tested	1 (1.7)

The overall sensitivity of the FAPP panel for identifying bacteria is 92.0% (95% CI, 80.8% to 97.8%), the overall specificity is 93.8% (95% CI, 91.1% to 95.3%), the overall positive predictive value is 46.9% (95% CI, 40.2% to 53.8%) and the overall negative predictive value is 99.5% (95% CI, 98.7% to 99.8%). Four false negative results were obtained with the FAPP panel. The false negative results involved 1 A. baumannii complex, 1 K. aerogenes, 1 K. pneumoniae group, and 1 S. pneumoniae. There were 52 false positive results obtained. Each individual bacterial target had at least 1 false positive result, the organism with the most false positive results (n = 8, 13.6%) was K. pneumoniae group. Atypical bacteria were detected in only 1 (1.7%) specimen and involved Legionella pneumophila. Viruses were detected in 5 (8.5%) specimens and involved 4 specimens with Human Rhinovirus/Enterovirus and 1 specimen with both coronavirus and parainfluenza virus.

The following antibiotic resistance genes were detected, CTX-M (n = 7, 11.9%), mecA/C and MREJ (n = 1, 1.7%), NDM (n = 8, 13.6%), OXA-48-like (n = 7, 11.9%), and VIM (n = 1, 1.7%). No IMP or KPC genes were detected. For detecting antibiotic resistance, the overall positive percent agreement is 100% (95% CI, 81.5% to 100.0%) and the overall negative percent agreement is 98.5% (95% CI, 96.7% to 99.4%). No false negative results were obtained in this category. Six false positive results were obtained. These involved 2 CTX-M genes, 1 NDM gene, and 3 OXA-48-like genes. Table 3 summarizes the accuracy of the FAPP panel compared to culture as the reference standard for each individual target and overall across all targets.

Table 3

Performance summary of the FilmArray Pneumonia plus panel compared to culture as the reference standard.

Target	Cult 1 FA 1T Pos	Cult 1 FA 0F Neg	Cult 0 FA 1F Pos	Cult 0 FA 0T Neg	FA Sens	95% CI	FA Spec	95% CI	FA PPV	95% CI	FA NPV	95% CI
Bacteria	(n)	(n)	(n)	(n)	(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)
Acinetobacter baumannii complex	12	1	6	40	92.3	64.0–99.8	87.0	73.7–95.1	66.7	48.3–81.1	97.6	85.8–99.6
Enterobacter cloacae complex	1	0	6	52	100.0	2.5–100.0	89.7	78.8–96.1	14.3	7.2–26.2	100.0
Escherichia coli	4	0	4	51	100.0	39.8–100.0	92.7	82.4–98.0	50.0	28.0–72.0	100.0
Haemophilus influenzae	2	0	6	51	100.0	15.8–100.0	89.5	78.5–96.0	25.0	13.5–41.5	100.0
Klebsiella aerogenes	1	1	0	57	50.0	1.3–98.7	100.0	93.7–100.0	100.0		98.3	93.4–99.6
Klebsiella oxytoca	1	0	1	57	100.0	2.5–100.0	98.3	90.8–100.0	50.0	12.5–87.5	100.0
Klebsiella pneumoniae group	7	1	8	43	87.5	47.3–99.7	84.3	71.4–93.0	46.7	30.5–63.5	46.7	30.5–63.5
Moraxella catarrhalis	0	0	2	57			96.6	88.3–99.6	0.0		100.0
Proteus species	0	0	5	54			91.5	81.3–97.2	0.0		100.0
Pseudomonas aeruginosa	9	0	1	49	100.0	66.4–100.0	98.0	89.4–100.0	90.0	56.4–98.4	100.0
Serratia marcescens	3	0	2	54	100.0	29.2–100.0	96.4	87.7–99.6	60.0	27.8–85.4	100.0
Staphylococcus aureus	6	0	6	47	100.0	54.1–100.0	88.7	77.0–95.7	50.0	32.0–68.0	100.0
Streptococcus agalactiae	0	0	1	58			98.3	90.9–100.0	0.0		100.0
Streptococcus pneumoniae	0	1	3	55	0.0	0.0–97.5	94.8	85.6–98.9	0.0		98.2	98.1–98.3
Streptococcus pyogenes	0	0	1	58			98.3	90.9–100.0	0.0		100.0
Bacteria cumulative	46	4	52	783	92.0	80.8–97.8	93.8	91.9–95.3	46.9	40.2–53.8	99.5	98.7–99.8
	T Pos	F Neg	F Pos	T Neg	PPA	95% CI	NPA	95% CI
Antibiotic Resistance Genes	(n)	(n)	(n)	(n)	(%)	(%)	(%)	(%)
CTX-M	5	0	2	52	100.0	47.8–100.0	96.3	87.3–99.5
IMP	0	0	0	59			100.0	93.9–100.0
KPC	0	0	0	59			100.0	93.9–100.0
mecA/C and MREJ	1	0	0	58	100.0	2.5–100.0	100.0	93.8–100.0
NDM	7	0	1	51	100.0	59.0–100.0	98.1	89.7–100.0
OXA-48-like	4	0	3	52	100.0	39.8–100.0	94.5	84.9–98.9
VIM	1	0	0	58	100.0	2.5–100.0	100.0	93.8–100.0
ABR genes cumulative	18	0	6	389	100.0	81.5–100.0	98.5	96.7–99.4

Key: Cult = Culture, FA = FilmArray Pneumonia plus panel, T Pos = True positive, F Neg =False negative, F Pos = False positive, T Neg = True negative, Sens = Sensitivity, CI = Confidence interval.

Spec = Specificity, PPV = Positive predictive value, NPV = Negative predictive value, ABR = Antibiotic resistance, PPA = Positive percent agreement, NPA = Negative percent agreement.

Regarding quantification, the FAPP panel's bin semiquantification category was compared to the culture semiquantification category. The Pearson correlation coefficient for the 2 methods was calculated as 0.50 (95% CI, 0.24 to 0.69, P = .0004). Table 4 shows the comparison of the FAPP panel's bin semiquantification category to the culture colony count semiquantification category.

Table 4

Comparison of semiquantification bin category of FilmArray Pneumonia plus panel results and bacterial culture colony count category results.

			Culture semiquantification
		CFU/mL	0–10	11–100	> 100
?>FAPP bin semiquantification	Copies/mL	Category	1+	2+	3+	Total
	104	4	2	1	0	3
	105	5	0	2	1	3
	106	6	4	6	4	14
	107	7	2	3	21	26
		Total	8	12	26	46

FAPP = FilmArray Pneumonia plus.

Concerning the turnaround time, the mean difference in turnaround time between the FAPP panel and culture was 3705 minutes (95% CI, 3191 to 4219 minutes, P <.0001). A separate turnaround time analysis was performed for specimens from which antibiotic resistance genes were detected (n = 18, 30.5%), as these results have infection prevention and control implications. In this analysis the mean difference in turnaround time was 3805 minutes (95% CI, 3038 to 4573 minutes, P< .0001).

Discussion

This study prospectively evaluated the FAPP panel by comparing it to conventional culture methods in the evaluation of 59 LRTI specimens. The specimens were collected mainly from ICU patients which constitute 88.1% of the sample. A. baumannii complex, K. pneumoniae group and S. aureus were the 3 most common organisms detected at a prevalence of 30.5%, 25.4%, and 20.3% respectively. Only 1 atypical bacterium, L. pneumophila, and 6 viruses were detected. This is in keeping with the etiology of hospital acquired pneumonia (HAP) and ventilator associated pneumonia (VAP) (Jones, 2010, Herkel et al., 2016).

The study finds that the performance characteristics of the FAPP panel are comparable to other multiplex respiratory platforms such as the Verigene Respiratory Pathogens Flex test (Luminex Corporation, Austin, TX, USA), and the ePlex respiratory pathogen panel (GenMark Diagnostics, Carlsbad, CA, USA) with a high overall sensitivity and specificity in identifying bacterial pathogens of 92.0% and 93.8% respectively (Ramanan et al., 2018). The findings are also generally in keeping with other studies that evaluated the FAPP panel (Yoo et al., 2020, Lee et al., 2019, Faron et al., 2018, Kerr et al., 2018). Yoo et al. (2020) evaluated a total of 31 sputa and 69 ETA specimens against culture from patients at the Samsung Medical Center in Seoul, Republic of Korea (Yoo et al., 2020). They found an overall sensitivity and specificity for organism detection of 98.5% and 76.5% respectively (Yoo et al., 2020). Lee et al. (2019) evaluated 59 ETA and BAL specimens from 51 adult patients from the National Taiwan University Hospital in Taipei, Taiwan (Lee et al., 2019). They also used culture as the reference method and found an overall positive percentage agreement of 90.0% and an overall negative percentage agreement of 97.4% (Lee et al., 2019). Faron et al. (2018) evaluated 57 BAL and 48 sputum specimens in patients from 8 hospitals in the United States of America with symptoms of respiratory tract infection against standard of care methods (culture or PCR, based on clinician request) (Faron et al., 2018). They found a positive percentage agreement of 94.7% and 95.8% for bacterial targets in BAL and sputum specimens, respectively (Faron et al., 2018). Of note, a large number of false positive results were found (52.9%) (Faron et al., 2018). However, many of these were observed in patients who had antibiotic exposure more than 12 hours before specimen collection, which may have negatively affected the yield of culture in these specimens (Faron et al., 2018). These findings are consistent with findings demonstrated by Kerr et al. (2018). Similar to previous studies, we found a high number (n = 52) of false positive results for identifying bacterial pathogens when comparing the FAPP panel to culture (Faron et al., 2018, Kerr et al., 2018). This negatively impacts on the specificity and positive predictive value of the assay, calculated at 93.8% and 46.9% respectively. This may be due to one of the following reasons. Firstly, the ability of the assay to detect small quantities of bacteria that fail to grow in culture as they are below the limit of detection of culture. Of the 98 bacterial targets detected by the FAPP panel in this study, 30 (30.6%) were quantified at the lowest bin category (104 copies/mL). Secondly, the improved sensitivity of PCR in the detection of fastidious organisms compared to culture. In this study Haemophilus influenzae, which is known for its fastidious nature, was responsible for 6 (11.5%) of the false positive results. Lastly, the detection of nonviable genomic material found in respiratory specimens by the assay. A previous study suggested that the high false positive rate may be due to specimens collected from patients with prior exposure to antibiotics (Faron et al., 2018).

In terms of accuracy of detecting antibiotic resistance genes, we find a high correlation between culture-based phenotypic antibiotic susceptibility methods and the FAPP panel, with an overall positive percent agreement and negative percent agreement of 100% and 98.5% respectively. Similar to the findings with regards to bacterial identification, the number of false positive results (n = 6) negatively impacts on the concordance with culture. This may be due to resistance genes not being expressed phenotypically or the detection of nonviable genomic material. Another possibility is that the antibiotic resistance genes detected were from bacteria that were present in very low numbers that culture failed to detect.

The quantification of respiratory pathogens assists in determining the clinical significance of the result (Leber, 2016). When comparing the FAPP panel's bin semiquantification category to the culture semiquantification category we find a correlation coefficient of 0.5 (95% CI, 0.24 to 0.69). This suggests that the FAPP panel's bin semiquantification system may be helpful in the determination of the clinical importance of a specific organism that is detected. Other studies have reached similar conclusions in their comparisons (Yoo et al., 2020, Lee et al., 2019).

The major advantage multiplex PCR panels have over conventional culture methods is the significant improvement in the turnaround time. This is shown in our results with an average difference in turnaround time of 3705 minutes (P <.0001). This equates to an average time saving per specimen of 2 days, 13 hours and 45 minutes when using the FAPP panel compared to culture. This significant time saving can have a major impact on antimicrobial stewardship and especially on infection prevention and control. Other studies have demonstrated the positive impact of rapid diagnostics on antimicrobial stewardship in terms of early antibiotic de-escalation and discontinuation (Furukawa et al., 2019, Huang et al., 2018). Antibiotic resistance genes were detected in 30.5% of the specimens evaluated in this study. For these specimens the average difference in turnaround time was 3805 minutes (P <.0001).This equates to a time saving per specimen of 2 days, 15 hours and 25 minutes when using the FAPP panel compared to culture. This significant difference can have a substantial impact on the spread of antibiotic resistant organisms in an ICU or hospital by allowing for early isolation and implementation of transmission-based precautions to prevent the spread of these organisms.

This assay has proven to be a valuable tool in antimicrobial stewardship (Furukawa et al., 2019, Huang et al., 2018). Furukawa et al. (2019) demonstrated a 100% intervention rate with early discontinuation of broad spectrum antimicrobials, prevention of inappropriate antimicrobial initiation/escalation, as well as early intravenous to oral transition of antibiotics (Furukawa et al., 2019). Huang et al. (2018) also demonstrated a high rate of appropriate antimicrobial de-escalation (47.9%) with the use of the FilmArray panel (Huang et al., 2018). The authors concluded that an average of 159.1 hours of antimicrobial therapy per patient may have been avoided when compared to standard of care methods (Huang et al., 2018).

The most recent guidelines of the American Thoracic Society and the Infectious Diseases Society of America on the diagnosis and treatment of adults with community acquired pneumonia (CAP) recommend not obtaining sputum Gram stain and culture on patients managed for CAP in the outpatient setting, and only requesting these investigations on selected patients in the hospital setting (Metlay et al., 2019). In contrast, the latest guidelines of the American Thoracic Society and the Infectious Diseases Society of America on the management of adults with HAP and VAP recommend that patients with suspected HAP be treated according to the results of microbiological studies performed on noninvasively obtained respiratory specimens, rather than with empiric therapy (Kalil et al., 2016). Based on these recommendations the main utility of the FAPP panel is in severe CAP, HAP, and VAP.

Although the use of the FAPP panel has obvious benefits for the diagnosis and management of patients with suspected pneumonia, its implementation is not without challenges (Alby and Mitchell, 2018). Firstly, the sensitivity of detection of each target may be compromised due to the diverse number of targets included in the panel (Alby and Mitchell, 2018). This was demonstrated in the study by Kerr et al. (2018) that reported lower sensitivities for the detection of a number of targets in the FAPP panel (Kerr et al., 2018). In our analysis we also found low sensitivity for some targets. Secondly, results generated by the FAPP panel may be difficult to interpret in a clinical context. Detection of some of the organisms included on the panel (e.g., S. pneumoniae, H. influenzae, and M catarrhalis) may represent colonization as opposed to infection. Traditional culture methods have the advantage of comparing the quantities of these organisms to those of the normal flora to facilitate the interpretation and identification of the causative agents. The FAPP panel does not include targets for the organisms comprising commensal respiratory flora. Although the implementation of semiquantitative bacterial results aims to aid in this distinction, it is still not possible to make a definitive conclusion of colonization with the FAPP panel (Alby and Mitchell, 2018). Thirdly, because the FAPP panel does not contain all possible clinically relevant organisms as targets, the panel may fail to detect some important pathogens. This was demonstrated in our study with the culture detection of 2 S. maltophilia isolates that may have required special consideration with regards to treatment. We recommend that manufacturers of multiplex PCR panels consider the inclusion of such pathogens for future respiratory panels. Lastly, although the panel includes a broad range of antibiotic resistance determinants, it does not link the detected antibiotic resistance gene to a specific organism, making the interpretation difficult. This is especially troublesome when more than 1 organism is detected. It remains necessary to perform culture to obtain full susceptibilities on the organisms identified, rendering the FAPP panel still only an adjunct in the diagnostic process (Alby and Mitchell, 2018).

This study has some limitations. The sample size is relatively small and focused mainly on adult ICU patients from a single center. Culture is an imperfect “gold standard.” Many of the false positive results obtained with the FAPP panel may be due to the suboptimal sensitivity of culture. Ideally the FAPP panel should be compared not only to culture, but also to validated molecular techniques in order to overcome the limitations of culture methods. This may be done in a future study. This study also has some strengths. It evaluated the FAPP panel prospectively in a real world setting with actual patient specimens. It not only assessed the detection of bacterial agents and antibiotic resistance genes, but also evaluated quantification, turnaround time and potential impact on infection prevention and control measures.

In conclusion, the FAPP panel is a rapid, multiplex PCR-based assay that is highly sensitive and specific in identifying the most important bacterial causes of LRTIs, as well as commonly associated antibiotic resistance genes, from lower respiratory tract specimens. Its quantification system correlates well with that of conventional culture methods and may be helpful in determining the clinical significance of bacteria detected. The significantly reduced turnaround time of this assay compared to conventional culture methods has important implications for antimicrobial stewardship and infection prevention and control.