Assessment of the FilmArray® multiplex PCR system and associated meningitis/encephalitis panel in the diagnostic service of a tertiary hospital.

Rapid and accurate diagnosis of meningitis/encephalitis (M/E) is essential for successful patient outcomes. The FilmArray® meningitis/encephalitis Panel (MEP) is a multiplexed PCR test for simultaneous, rapid detection of pathogens directly from cerebrospinal fluid (CSF) samples. 94 prospectively collected CSF specimens from patients with clinical suspicion of infective M/E underwent testing for 14 pathogens simultaneously, including Escherichia coli, Haemophilus influenzae, Neisseria meningitidis, and Varicella zoster. MEP demonstrated 95% agreement with current PCR methods, resulting in 16 diagnosed cases of M/E. Typically, the FilmArray® MEP results were delivered within approximately one hour, contrasting with current practices taking up to 5.6 days. Given the significant morbidity and mortality associated with delayed diagnosis of central nervous system infections, the FilmArray® MEP is a useful addition to the diagnostic capabilities of a clinical microbiology department.

Introduction

Despite the introduction of vaccine immunisation programs worldwide during the 1970s, morbidity and mortality associated with meningitis or encephalitis (M/E) continue to remain exceptionally high. Potentially caused by bacteria, viruses, mycobacteria or, on occasion, fungi [1,2] those at increased risk of infection include babies <18 months, pre-school children under the age of 5, teenagers/young adults and also immunocompromised patients. Unfortunately, 30–50% of M/E cases progress to serious complications including hearing and/or speech loss, blindness, permanent brain or nerve damage, seizures and loss of limbs [3]. Streptococcus pneumoniae, Neisseria meningitidis and Haemophilus influenzae have in the past been reported as the most frequent causative agents [4]. Over 1.2 million cases of bacterial meningitis are estimated to occur annually worldwide [2] with, for instance, the Meningitis Research Foundation UK [5] reporting an estimated 3,200 cases of bacterial meningitis annually between years 2000–2012. However, viral meningitis is more common, albeit that many cases go unrecognised and undiagnosed due to mild, sometimes flu like symptoms.

A diagnosis of M/E is dependent upon both clinical presentation of the patient and laboratory analysis of cerebrospinal fluid (CSF). However, according to Hanson [6] reaching an accurate diagnosis can be complex as clinical signs and symptoms associated with M/E are not organism specific and, therefore, differential diagnosis can be broad. Culturing and Gram stain have been described as the optimal tests (currently available) for the diagnosis of bacterial and fungal meningitis, with PCR methods most accurate for detecting viral meningitis [4,[7], [8], [9]]. However for bacterial and fungal meningitis, both conventional microbiology and PCR tests are prone to confounders whereby antimicrobial treatment prior to CSF sampling and slow growth rate of some pathogens can jeopardise or delay accurate diagnosis. Prior to introduction of the FilmArray® MEP, CSF samples had been referred for viral PCR where any suspicion of viral M/E existed, even in the absence of CSF pleocytosis. Additionally, when polymorphonuclear cells represented >50% WBCs, bacterial polymerase chain reaction (PCR) was performed on the CSF samples. Viral PCR was performed when mononuclear cell dominated. Although 600–700 CSF samples are processed annually in our diagnostic service, such PCR testing is outsourced to the central Irish Meningococcal and Sepsis Reference Laboratory (IMSRL) or the central National Virus Reference Laboratory (NVRL). Concern arose in our service due to delays in obtaining results of referred PCR tests; specifically, in the context of timely diagnosis being required to enable targeted antimicrobial therapy for confirmed bacterial M/E (rather than empiric intravenous/intramuscular benzyl penicillin or cephalosporin) or, more commonly, discharge from hospital and discontinuation (where appropriate) of therapy for viral M/E patients.

Ireland has the highest rate of meningococcal meningitis in Europe with almost 200 cases reported annually [5]. Therefore, our study aimed to assess a rapid molecular method of diagnosis that could be implemented locally and lessening reliance on centralised national laboratories. More precisely, we performed a comprehensive evaluation of the FilmArray® multiplex PCR system (FA) with FilmArray® Meningitis/Encephalitis panel (MEP) [10] to determine its impact in the diagnosis of M/E, when introduced routinely in our microbiology laboratory. Emphasis was placed on: [1] validation of the FilmArray® MEP for detection of 14 common CSF pathogens including bacteria, viruses and yeast; [2] assessment of the sensitivity of the FilmArray compared to conventional methods performed in house, in conjunction with qualitative PCR assays completed at central national laboratories; [3] determination of M/E diagnosis turnaround time (TAT); and [4] retrospective chart review-based calculation of cost-effectiveness of FilmArray introduction.

Methods

Setting and inclusions

The study was performed at University Hospital Limerick in Ireland's mid-West; a 480-bed tertiary hospital for a population of > 600,000. The microbiology diagnostic service processes between 600 and 700 CSF samples annually. This assessment was performed over a 5 month period (March–July; due to availability of FilmArray® MEP products for that period) during which a minimum of 97 samples would be required due to a calculated margin of error of 8% and confidence level of 95%. CSF samples were eligible for inclusion if submitted for testing due to clinical suspicion of an infective M/E, a raised Leucocyte count (>5/μl) was noted on microscopy, or viral PCR was requested. To establish the sensitivity of the FilmArray® MEP, CSF samples that had previously been confirmed positive by the reference laboratories were frozen and stored at – 20oC prior to the six-month testing period (Dec 2016–March 2017). Those samples were analysed subsequently by FilmArray® MEP and outcomes compared to the PCR results obtained when the same samples were tested by the accredited reference laboratory.

Non-FilmArray® MEP CSF protocol

Macroscopic appearance of each CSF sample was described (clear, cloudy, bloodstained, yellow, turbid or clotted). Leucocytes and erythrocytes were quantified by manual microscopy using a KOVA Glasstic Slide counting chamber in which 1μl of CSF was assessed using manual light microscopy. Un-centrifuged samples were inoculated on both 5% sheep blood agar (Oxoid) and chocolate agar (Oxoid) and incubated in 5% CO2 at 35–37°. An additional blood agar plate was cultured and incubated anaerobically. All plates were incubated for 48 hours and also reviewed subsequent to overnight incubation. All CSF samples with a white cell count >5 leucocytes/μl underwent Gram stain and a differential white blood cell count. Where microbes were observed, a Meningitis Latex Agglutination Test (Pasteurex) was performed for identification of N. meningitidis A, B, C, Y and W135; E. coli K1; H. influenzae Type b, S. pneumoniae or S. agalactiae. Differential slides were prepared by spinning 200 μl of CSF sample at 1000 rpm for 5 minutes using an Aerospray cytocentrifugation slide stainer. The slides were processed with Wright-Giemsa stain using a Hematek slide stainer (Siemens) and the ratio of polymorphonuclear cells to mononuclear cells recorded. CSF Samples with a cell count of >5/μl and >50% polymorphonuclear leucocytes were referred to the IMSRL for bacterial PCR testing targeting; N. meningitidis, S. pneumonia, S. agalactiae, H. influenzae and E. coli K. CSF samples with a cell count >5/μl and >50% mononuclear leucocytes, or where specifically requested, were referred to the National Viral Reference Laboratory for viral PCR targeting Herpes simplex virus 1, Herpes simplex virus 2, Varicella zoster virus and Human herpes virus 6. In cases where Cryptococcus neoformans was tested for, the Cryptococcal Antigen Lateral Flow Assay (LFA) was used. For our prospective comparison, all CSF samples referred for external PCR underwent parallel processing using the FilmArray® MEP.

FilmArray® MEP protocol

Processing was performed as per manufacturer's instructions. In summary, approx. 200 μl CSF is lysed using buffers provided prior to qualitative PCR using proprietary primers on the in vitro FilmArray® Multiplex PCR analyser. The entire PCR process occurs within one pouch and takes approx. 1 hour from start to completion. The process involves nucleic acid purification, reverse transcription, 1st stage multiplex PCR, 2nd stage nested PCR and DNA Melting Analysis 10).

Quality procedures

For validation, 94 prospective patient CSF samples were run on both the FilmArray® MEPl and the current routine methods of microscopy/Gram stain culture. Samples were also referred to reference laboratories for both bacterial and viral PCR as part of the quality control programme. Retrospective samples that had been frozen were also run on the FilmArray MEP and the results compared to reference reports. All targets on the FilmArray® MEP were detected by 3 methods: 1) true positive samples; 2) control samples received from the NVRL arising from other hospital sites; and 3) the recommended ZeptoMetrix® validation panel controls. Positive/negative controls could not be run each time as the platform has a low sample throughput. However, the assay includes an internal quality control including a RNA process control. The latter targets an RNA transcript from the yeast Schizosaccharomyces pombe. The yeast was present in the pouch in a freeze-dried form and becomes rehydrated when sample is loaded. The control material is carried through all stages of the test process, including lysis, nucleic acid purification, reverse transcription, 1st stage PCR, dilution, 2nd stage PCR and DNA melting. A positive control result indicates that all steps performed in the FilmArray ME pouch were successful. Therefore, if there is an RNA process control failure, all steps were not completed successfully and a result cannot be accepted.

Retrospective chart review and economic analysis

A retrospective chart review of paediatric patients who had CSF samples analysed in the six months prior to the availability of FilmArray® MEP was performed to evaluate potential improved turnaround time (TAT) of results and diagnosis. This was compared with actual length of stay and treatment of patients following introduction of the FilmArray®.

Potential cost-savings were deduced from potential reduction in duration of hospitalisation and reduction in duration of empirical antimicrobial/antiviral therapy. The overall cost of the FilmArray® MEP multiplex PCR in terms of material and labour was based on actual cost for 94 tests. The scientist rate of pay was established from the median of the basic grade medical scientist rate in Ireland, so again is based on actual costs.

Statistical analysis

All statistical analysis was performed using IBM SPSS Statistics 24.

Levene's test for equality of variance was used when determining significant difference in length of stay (LOS) in the paediatric cohort pre- and post-introduction of the FilmArray® MEP. An independent samples t-test for equality of means was utilised to determine significance in variation between length of hospital stay before and following the intervention.

The FilmArray® MEP test run–time was compared to the time taken for return of reference laboratory reports to establish the difference in test TAT and to determine the usefulness of the FilmArray® MEP for timely diagnosis. One sample

Results

Comparison of testing pre- and post-FilmArray use

The manufacturer of the BioFire FilmArray® MEP state that the limit of detection (LOD) is 100–1,000 CFU/ml for all bacteria & yeast, and 5–500 Tissue Culture Infective Dose (TCID) 50/ml or 100–1,000 copies/ml for all viruses, which was utilised as the LOD for this study [11]. Prospective analysis using the FilmArray® MEP was attempted in 100 CSF samples. However, 6 were discarded due to RNA process control failure (1 sample), insufficient sample volume for referral to reference laboratories and therefore no comparison was possible (3 samples), and false positive contamination (2 samples). The latter were confirmed as negative when re-run on the FilmArray® MEP and found to be PCR negative when referred to the Reference Lab. Overall, the FilmArray demonstrated sensitivity and specificity analogous to that of the PCR-based bacterial reference laboratory, but was less sensitive with regard to virus sensitivity (86%).

Turnaround time and impact on length of stay

When compared, the mean TAT using the FilmArray® MEPas 70 mins versus a mean time of 135 hrs 40 mins for communication of results from the reference laboratories. The average time taken for the return of reference laboratory reports was about 134 hours longer (5.6 days) than the average time taken to run a FilmArray analysis (1.167 hours). The one sample t–test demonstrated (p <0.05) rejection of the null hypothesis indicating that the introduction of the FilmArray® MEP improved the TAT for diagnosis of M/E, due to reduction in mean sample test processing time.

Comparison of 18 retrospective charts and 16 prospective charts was performed. Improvement in availability of test results was associated with statistically significant shortening of hospitalization. Specifically, a 15.5 day mean reduction in length of stay (0.046 t-test) with studies undertaken by Nabower et al. and Didiodato et al. showing similar reduction in LOS [12,13].

Economic analysis

The costs for performance of laboratory testing comprised materials and the attributable pro rata medical scientist salaries. The direct costs of analysis of 94 CSF samples before introduction of the FilmArray was €2,800 (comprising consumables costings of approximately €750 with the remained being labour), while adoption of the FilmArray resulted in an increase in actual costs of €12,480 (which was primarily consumables). However, this increase is offset by potential reduction of length of hospital stay and reduced duration of empiric treatment. Specifically, for instance, based on the paediatric patient cohort alone there would be a net saving of over €114,000, while extrapolating reductions in duration of stay and empiric treatment collectively in relation to all patients for whom CSF analysis was requested the potentially accruable savings exceed €2 Million.

Discussion

In the context of requirements for rapid and accurate diagnosis of meningitis/encephalitis (M/E) to promote successful patient outcomes, this report describes an evaluation of the FilmArray® meningitis/encephalitis Panel (MEP), a multiplexed PCR test for simultaneous, rapid detection of pathogens directly from cerebrospinal fluid (CSF) samples. To summarise, 94 prospectively collected CSF specimens from patients with clinical suspicion of infective M/E underwent testing for 14 pathogens simultaneously, including Escherichia coli, Haemophilus influenzae, Neisseria meningitidis, and Varicella zoster. MEP demonstrated 95% agreement with current PCR methods, resulting in 16 diagnosed cases of M/E. Typically, the FilmArray® MEP results were delivered within approximately one hour, contrasting with current practices taking up to 5.6 days. Given the significant morbidity and mortality associated with delayed diagnosis of central nervous system infections, the FilmArray® MEP is a useful addition to the diagnostic capabilities of a clinical microbiology department.

All causes of M/E can present with similar symptoms; yet bacterial, viral or fungal involvement requires distinct treatment strategies. Not surprisingly then, rapid and accurate diagnosis continues to attract considerable attention, with investment in molecular technologies and antimicrobial stewardship targeting reduction in duration of empiric treatment while enhancing patient outcomes. Through that lens, emphasis has been placed on the fact that although not especially labour intensive or time consuming, conventional processes of microscopy and culture for microbiological analysis of CSF has relatively poor diagnostic accuracy and reliability. For example, in the United Kingdom a study of 103 patients with clinically defined meningococcal meningitis resulted in only 13% positive CSF cultures [4]. This inaccuracy has been attributed somewhat to administration of empiric antimicrobials prior to lumbar puncture sampling, effectively sterilising the CSF sample or at least reducing the microbial load to levels below detection possible using conventional culturing techniques [4,14]. This observation has been confirmed in reports such as that by Khater & Elabd [14], whereby positive PCR results were demonstrated in 36 of 40 CSF samples, which had been found to be culture negative. With a recent history of adoption of molecular diagnostic technologies and their effective implementation in outbreak management [15,16] the objective of this new study was to evaluate multiplex PCR, the FilmArray® MEP, with regard to diagnostic accuracy compared with current routine methods of culture/microscopy/Gram stain and compared with singleplex bacterial and viral PCR services provided by Irish centralised reference laboratories.

Notably, there was no statistical variation between sensitivity results achievable through current conventional techniques and the FilmArray® MEP. In summary, there were 19 positive pathogen detections using the FilmArray® MEP while 18 pathogen detections were recorded using the comparative reference methods, with viral pathogens accounting for 68% of positive FilmArray® MEP detections. The FilmArray® MEP detected at least one pathogen in 20% (19/94) of CSF specimens. These included enterovirus, N. meningitidis, Varicella zoster virus and Herpes simplex virus 1. Among the multiplex PCR positive cases, enterovirus was detected at the highest incidence of 7 cases (44%). This result is similar to that found by Leber et al. [11] at 38%, along with Desmond et al. [17] who have reported enterovirus as the most common cause of viral meningitis in the United States with an estimated 75,000 cases annually. The second highest occurring agent detected by multiplex PCR was N. meningitidis in 4 cases (25%). All 4 positive cases were from an adult cohort and all were identified in the reference laboratory as N. meningitidis serogroup C. Of interest, the introduction in Ireland of the MenC vaccine for infants and adolescents in the late 1990s can account for the higher rate of pathogen positivity in the older cohort with no positive cases identified in those <18 years [18]. However, we have reported elsewhere that the FilmArray® MEP can be used effectively in the analysis of paediatric samples, albeit not CSF-derived, for effective detection of N. meningitidis [19]. Overall, in our hands, the multiplex PCR demonstrated sensitivity and specificity lower than found by both Leber et al. (100%) [11] and von Allmen et al. (100%, 99%) [20]. Despite this, the FilmArray® MEP and PCR-based results obtained from the national bacterial reference laboratory correlated well. Indeed, the sensitivity results were identical, albeit that the specificity of FilmArray® MEP was lower than anticipated at 80%.

Conversely though, the FilmArray® MEP was less sensitive in detection of viral nucleic material than the Irish National Viral Reference Laboratory, but where detected the identification proved as accurate. More specifically, two CSF samples referred to the National Virus Reference Laboratory were positive for Human herpes virus 6, while not detected by the FilmArray® MEP. This highlights the limitation of the assay for detection of HHV-6 compared with the reference laboratory. Analogously, the FilmArray® MEP identified Human herpes virus 6 in one sample that was not detected at the central laboratory. However, the positive result did not correlate with clinical presentation. The clinical significance of HHV-6 in particular requires close clinical correlation to determine its significance. HHV-6 detection in the CSF may indicate primary infection, secondary reactivation, or the presence of latent virus. Also, integration of HHV-6 DNA into human chromosomes has been identified in approximately 1% of the human population [21]. Results should always be interpreted in conjunction with other clinical, laboratory, and epidemiological information. According to Slenker et al. caution should be taken before attributing CNS disease to HHV6 when a positive result is identified [22].

Given acceptable sensitivity and specificity, the most notable result related to turnaround time for tests; the FilmArray® MEP was found to enable actionable results within ∼one hour compared to 5.6 days for return of results from the reference services. At the time of this study, positive results only were communicated (verbally) to the duty clinical microbiologist on the same day as the test result was authorised at the reference laboratories. Negative results were mailed in hardcopy form. There now exists a secure electronic information transport service for the prompt return of all authorised results; both positive and negative.

Whether positive or negative, FilmArray results represent a potential for clinicians to make relatively quicker decisions regarding patients' clinical management, including appropriate antimicrobial stewardship. Timely diagnostics can influence whether a patient merits admission from the ED, for example in the setting of a Enteroviral-positive result. However, there is a need to perform longitudinal studies to determine the full impact of the FilmArray® MEP on cost savings encompassing antimicrobial savings, other diagnostic tests as well patient outcomes and patient satisfaction. The outcomes of both retrospective chart reviews and prospective patient management were promising, with considerable reduction of avoidable hospitalisation (mean >15 days) with associated actual direct economic savings exceeding €100,000 that can be extrapolated, based on whole cohort care, to potential savings of more than €2 Million in unnecessary expenditure.

One limitation of this study is the relatively low number of pathogens identified from clinical CSF samples allowing for direct comparison with the reference method. It does, however, highlight the potential, as with any conventional PCR based technique, for false positive results due to possible sample contamination. This would reinforce the requirement for strict laboratory procedures with “bench-top” molecular platforms, particularly if the instrument becomes a point-of-care test in an emergency departmental setting. Regardless of the clinical significance of a positive result for HHV-6, in this study the FilmArray MEP panel demonstrated inferior sensitivity compared to the reference laboratory method. The inclusion of data on estimated cost-savings is a strength of this study, although limited to the paediatric cohort only and potentially under-estimated overall cost savings. The retrospective nature of the study enabled collation of turnaround times, which helped facilitate development of a business case for implementation of the apparatus in routine diagnostic service. Anecdotally, since its introduction locally into routine CSF diagnostics, the feedback from medical staff has been very positive, especially pertaining to turnaround times and impact on antimicrobial stewardship. It is also noteworthy that actual utilisation of the FilmArray® MEP in practice has since changed. Rather than reflex testing based on WCCs/differential, the FilmArray® MEP is now employed based on clinical suspicion of meningitis/encephalitis.

In summary, our findings suggest that there is merit in considering use of the FilmArray® MEP in routine CSF analysis. However, its adoption should parallel continuation of conventional culture-based testing as the new instrument does not allow antimicrobial sensitivity testing or, indeed, high-throughput testing due to its capacity being limited to one CSF sample.

Funding

There was no external financial support for this work.

Disclosures

The authors declare no competing personal or financial interests and have nothing to disclose.

Conflicts of interest

None declared.

CRediT authorship contribution statement

Amanda Mostyn: Conceptualization, Writing - review & editing, Writing - original draft. Marie Lenihan: Formal analysis, Writing - review & editing. Donnchadh O'Sullivan: Formal analysis, Writing - review & editing. Sara Woods: Formal analysis, Writing - review & editing. Maureen O'Hara: Formal analysis, Writing - review & editing. James Powell: Formal analysis, Writing - review & editing. Lorraine Power: Formal analysis, Writing - review & editing. Nuala H. O'Connell: Conceptualization, Writing - original draft, Writing - review & editing. Colum P. Dunne: Conceptualization, Writing - original draft, Writing - review & editing.