Health service impact of testing for respiratory pathogens using cartridge-based multiplex array versus molecular batch testing.

There is increasing demand for access to rapid microbiological testing, with a view to improving clinical outcomes. The possibility of rapid testing has been facilitated by development of cartridge-based random access molecular technologies that are now widely available. Whether the expense of cartridge-based assays is justified in terms of clinical or laboratory cost savings is controversial. This prospective study evaluated the impact of the Biofire FilmArray Respiratory Panel ('FilmArray'), a cartridge-based random access molecular test, compared with standard batched molecular testing using an 'in-house' respiratory polymerase chain reaction (PCR) on laboratory and health service outcomes for adult patients at a tertiary-level adult hospital in Melbourne, Australia. Laboratory result turnaround time was significantly reduced with the FilmArray (median 4.4 h) compared to a standard validated in-house respiratory PCR assay (median 21.6 h, p < 0.0001) and there was a significant increase in diagnostic yield with the Filmarray (71/124, 57.3%) compared to in-house PCR (79/200; 39.5%; p = 0.002). Despite improved result turnaround time and increased diagnostic yield from testing, there was no corresponding reduction in hospital length of stay or use of isolation beds. Although cartridge-based molecular testing reduced turnaround time to result for respiratory pathogen testing, it did not impact on health service outcomes such as hospital length of stay. Further work is warranted to determine whether cartridge-based tests at the point of care can improve clinical and health service impacts. Crown

Introduction

Cartridge-based random access molecular technologies, with the ability to rapidly detect multiple targets in a single, low complexity test format, are becoming widely available for diagnostic microbiology laboratories. The costs and benefits of implementing these new technologies for rapid diagnostic testing are not fully established, and may depend significantly on patient, laboratory and hospital characteristics.

The cost per test for random access molecular platforms (usually cartridge-based) is significantly higher than the cost of batched molecular testing, where multiple specimens are held in the laboratory before being processed in a batch. Batched testing minimises costs by efficient use of reagents, equipment and staff time; however, it may also result in slower laboratory turnaround time (TAT), leading to delays in clinical decision-making for patients. Batched testing also usually requires a higher level of laboratory scientific expertise and oversight, potentially increasing overall laboratory costs.

Delayed results from the laboratory may have flow on effects for the hospital, in terms of antibiotic use, duration of inpatient stay and use of isolation rooms, resulting in increased hospital costs. Timely identification of a specific pathogen can lead to either cessation of antibiotics or directed antibiotic therapy, delivered in a timely manner. Rapid and accurate diagnosis can play a key role in optimising antimicrobial stewardship outcomes. Rapid testing may also improve use of respiratory isolation rooms, either by appropriate isolation of patients with a respiratory pathogen, or early discharge from isolation for patients with a negative test result.

There is some evidence that rapid testing for respiratory viruses decreases duration of antibiotic use,2, 4, 5, 6, 7 hospital length of stay (LOS),8, 9 and isolation bed LOS,2, 3, 10 particularly for patients with influenza viruses. However, these findings have not been uniform and may be impacted by the type of population (e.g., paediatric vs adult), underlying comorbidities, clinical setting, prevalence of influenza versus other respiratory pathogens and whether other respiratory infections (e.g., bacterial pneumonia) are present.9, 11, 12 There are a few studies that have specifically examined the impact of broad respiratory pathogen panels, finding favourably in terms of hospital resource utilisation and improved clinical management;2, 6, 13, 14 however, most of these studies have been performed in the paediatric hospital setting. There is a relative paucity of evidence regarding the impact of random access respiratory testing for adults.

The objective of this study was to determine whether deployment of a random access cartridge-based molecular assay (Biofire FilmArray Respiratory Panel), in a medium-sized diagnostic laboratory serving a tertiary adult hospital, impacts laboratory TAT or workflow when compared to standard batched molecular testing for respiratory targets, and to prospectively measure clinical resource utilisation including hospital and isolation bed LOS.

Materials and methods

This was a prospective study at the Royal Melbourne Hospital (RMH, Parkville, Australia) comparing clinical, hospital and laboratory parameters for 6 weeks before (2 July–30 August 2016) and after (21 September–20 October 2016) implementation of the Biofire FilmArray Respiratory Panel (Biomerieux, France) for diagnosis of respiratory pathogens. The Biofire FilmArray Respiratory panel is a cartridge-based multiplex molecular assay capable of detecting 20 targets including influenza A (H1, H1-2009 and H3 strains), influenza B, parainfluenza 1, 2, 3 and 4, adenovirus, coronavirus (HKU1, NL63, 229E and OC43), human metapneumovirus, respiratory syncytial virus, rhinovirus/enterovirus, Bordetella pertussis, Chlamydophila pneumonia and Mycoplasma pneumoniae. There have been several studies demonstrating high clinical and analytical sensitivity and specificity compared to standard diagnostic testing algorithms.1, 15

Prior to the introduction of the FilmArray panel, a National Association of Testing Authorities (NATA) approved in-house real-time polymerase chain reaction (PCR) detecting influenza A and B, respiratory syncytial virus (RSV) and rhinovirus was in routine use for detection of respiratory viruses. Primers and probes for influenza A were supplied by Biosearch Technologies (USA). RNA was extracted using a commercial kit (EZ1 Advanced XL BioRobot; Qiagen, Germany) according to the manufacturer's instructions, and run on the ABI7500 (Applied Biosystems, USA). Up to eight samples can be processed in a single run.

Following implementation of the FilmArray, both assays were run on each sample in order to facilitate a direct comparison between the assays for verification purposes.

RMH is a 350-bed major tertiary hospital in Melbourne, Australia, with a busy (>60,000 admissions/year) Emergency Department (ED) serving the Melbourne metropolitan area. The microbiology laboratory predominantly serves RMH (approximately 80% of workload) and operates 24 hours per day, 7 days per week. After hours, there is only one scientist available in the laboratory and work is prioritised according to clinical urgency.

For this study, nasopharyngeal swabs were tested from patients admitted to RMH for <48 h with a request for ‘respiratory PCR’. Patients from the Haematology, Bone Marrow Transplant and Medical Oncology units were excluded as they are often treated via febrile neutropenia guidelines and other protocols, which are likely to affect hospital stay.

Clinical and demographic data were recorded on a standardised case report form (CRF), including admission and discharge date and time, use of respiratory isolation rooms, time of specimen receipt in laboratory, processing start and finish time, time of result release and time that the respiratory PCR results were viewed by a clinician in the hospital's electronic pathology record. A time and motion study was done to record hands-on laboratory work time, with timing of each step of both assays recorded on a standardised CRF. For the time and motion study, laboratory staff members used a timer or stopwatch to record the time in minutes and seconds for each step. For the in-house PCR this included recording the total time for accession, setting up specimens for extraction, preparing samples for amplification, reading and reporting results. Since the in-house PCR assay was batched, the total time for each individual specimen in the study was calculated by dividing the overall time for the entire batch by the number of samples in the batch. For the FilmArray assay, the time taken for accession and set up and loading of the test and reporting of results was recorded individually for each specimen.

In the pre-implementation phase, nasopharyngeal specimens were batched for accession, extraction and amplification stages, which were all performed by dedicated molecular scientists. The in-house assay was run once per weekday with the results released in the late afternoon. In the post-implementation phase, the specimen was split with the in-house PCR assay performed as above and the FilmArray processed as soon as practicable (24 h per day by any available microbiologist). All results from both pre-implementation and post-implementation periods were reported directly to the hospital's electronic pathology record and notified to the hospital's Infection Prevention team by fax and pager within 15 min. Discrepant results from the two assays were sent for testing at an external laboratory.

Outcome measures including laboratory TAT (time from arrival of specimen in the laboratory to release of electronic result to clinician), staff hands-on time, time to result being checked by treating clinician, hospital LOS and isolation bed LOS were calculated and were compared between the pre-implementation and post-implementation groups. The study was powered to detect a difference in laboratory TAT of at least 7 h and differences in hospital LOS and isolation bed LOS of at least 10 h with 80% power and 95% confidence based on previous estimates of these parameters.

Due to the non-parametric distribution of the continuous outcome measures collected, median values were calculated and Wilcoxon-rank sum tests used to determine clinically significant differences. For categorical variables, chi-squared tests were used. Visual tests for linearity were performed with removal of data outliers before running univariate linear regression models for factors predicting both hospital LOS and isolation bed LOS. All factors with p < 0.2 along with clinically relevant factors determined a priori were incorporated into a final multivariate model. All statistical calculations were done in STATA 12 (Statacorp, USA).

The study was approved by the RMH Human Research Ethics Committee (HREC approval number 2016.011).

Results

Over the pre-implementation study period (pre-implementation period), 321 nasopharyngeal specimens with a request for respiratory testing were received. Of these, 121 (37.7%) were excluded from the study (Fig. 1 ). During the post-implementation study period (post-implementation period), 211 patient specimens were received and 87 (41.2%) of these were excluded. Thus, for the final analysis, 200 patients were included in the pre-implementation group and 124 were included in the post-implementation group. Both groups were similar across a range of baseline characteristics that are presented in Table 1 .

Fig. 1

Flow diagram describing the process of selection of appropriate specimens for the study. Specimens received with a request for ‘respiratory PCR’ were assessed for eligibility during both the pre-implementation and post-implementation periods. Specimens were excluded if the patient was admitted under the Oncology, Haematology or Bone Marrow Transplant unit, had a hospital length of stay (LOS) greater than 48 h at the time of specimen request, were private patients or outpatients, or had duplicate or inappropriate specimens (only nasopharyngeal swabs or aspirates were accepted). After applying exclusions, 200 specimens were included for the pre-implementation phase. These specimens had the in-house respiratory PCR performed. During the post-implementation phase, 124 specimens were included following exclusion. These specimens had both in-house respiratory PCR and FilmArray performed.

Table 1

Baseline characteristics of pre-implementation and post-implementation groups

Characteristic	Pre-implementation n = 200	Post-implementation n = 124	p value
Age, median (range)	64 (18–95)	65 (16–94)	0.65
No. female (%)	101 (50.5)	59 (47.6)	0.61
Treating unit (%)
Emergency Department	38 (19)	29 (23)	0.34
Acute Medical Unit	56 (28)	36 (29)	0.84
General Medicine	15 (7.5)	13 (10.5)	0.35
Infectious Diseases	33 (16.5)	23 (18.5)	0.64
Respiratory	29 (14.5)	10 (8.1)	0.08
Other	29 (14.5)	13 (10.5)	0.30
Time of day arrived (%)a
Day	96 (48)	50 (40.3)	0.18
Evening	54 (27)	40 (32.3)	0.31
Night	50 (25)	34 (27.4)	0.63
No. admitted to isolation bed (%)	74 (37)	47 (37.9)	0.87

Times for different shifts as follows: Day = 08:00–17:00; Evening = 17:00–23:00; Night = 23:00–08:00.

Target detection

Over the two combined study periods, the in-house respiratory PCR yielded a positive result in 126/324 cases (38.9%). The proportion of influenza A cases detected was higher in the pre-implementation than the post-implementation period (31% versus 19.4%, respectively, p = 0.02). In the post-implementation period the FilmArray respiratory panel detected a target in 71/124 cases (57.3%), compared to the in-house PCR which detected a target in 47/124 cases (37.9%). The increased yield was largely due to detection of targets not included in the in-house PCR panel; however, there were four specimens where discrepant results occurred involving pathogens included in the in-house panel (influenza A, rhinovirus and RSV) (Supplementary Table 1, Appendix A). Additional testing supported the FilmArray result in two of these four specimens, the in-house PCR result in one, and one specimen was indeterminate.

Turnaround time

The median laboratory TAT was 4.4 h using the FilmArray and 21.6 h with the in-house PCR (p < 0.0001). The FilmArray required approximately the same staff hands-on time as the batched in-house assay (10 min vs 11.5 min, respectively, p = 0.07). The overall laboratory TAT was most impacted by earlier processing. FilmArray processing commenced after a median of 2.8 h from arrival in the laboratory, compared with 15.4 h for the in-house assay (p < 0.0001).

The reduced laboratory TAT was reflected in earlier availability of results to clinicians (Table 2 ). In the pre-implementation period, 141 (70.5%) specimens had a result available before the patient was discharged from hospital and the median duration between laboratory receipt of the specimen and the clinician checking the result on the pathology viewer (clinical TAT) was 27 h (range 3.9–261.2 h). In comparison, 109 (88%) specimens had a result available before discharge during the post-implementation period (p < 0.001) and the clinical TAT was a median of 9.4 h (range 2.2–34.6 h, p < 0.0001).

Table 2

Health service outcomes for patients in the pre-implementation vs post-implementation period

Characteristic	Pre-implementation n = 200	Post-implementation n = 124	p value
No. with positive result (%)	79 (39.5)	71 (57.3)	0.002
No. result released before discharge (%)	141 (70.5)	109 (87.9)	<0.001
Time to when result observed by clinician, hours, median (range)	27.0 (3.9–261.2)	9.4 (2.2–34.6)	<0.0001
Hospital LOS, hours, median (range)	52.8 (0.8–1361.7)	48.1 (2–855.9)	0.50
Isolation bed LOS, hours, median (range)	62.2 (1.1–455.5)	44.7 (0.32–267.2)	0.26

LOS, length of stay.

Health service impact

Despite significant reductions in laboratory and clinical TAT, there were no significant changes in hospital LOS or isolation bed LOS between the pre-implementation and post-implementation periods (Table 2). There were also no differences in hospital or isolation bed LOS in those with a hospital LOS <7 days, those admitted to ED only, those with a positive test result or those with a positive test for influenza. Univariable and multivariable regression models were performed to identify other factors predicting both hospital LOS and isolation bed LOS (Table 3 ). ED and General Medicine admissions were associated with shorter and longer hospital LOS, respectively. ED admission was also associated with a reduction in isolation bed LOS, although admission to other units did not impact this outcome. Having a positive influenza result was also associated with both a shorter hospital and isolation LOS.

Table 3

Linear regression analysis of factors associated with hospital length of stay and isolation bed length of stay

	Unadjusted coefficient (95% CI)	p value	Adjusted coefficient (95% CI)	p value
Factors associated with hospital LOS a
Age	1.09 (0.47, 1.71)	0.001	0.44 (–0.21, 1.09)	0.18
Groupb	–14.62 (–44.32, 15.07)	0.33	–20.79 (–50.16, 8.58)	0.17
Positive result	–20.88 (–49.78, 8.02)	0.16	19.30 (–19.97, 58.57)	0.33
Positive influenza result	–42.87 (–74.47, –11.26)	0.008	–48.74 (–91.31, –6.17)	0.03
Admission to ED only	–92.61 (–126.82, -58.40)	<0.0001	–70.25 (–105.98, –34.52)	<0.0001
General Medicine admission	35.50 (18.80, 52.20)	<0.0001	26.80 (9.99, 43.62)	0.002
Factors associated with isolation bed LOS c
Age	0.30 (–0.22, 0.82)	0.25	0.15 (–0.38, 0.67)	0.58
Groupb	–13.01 (–36.68, 10.65)	0.28	13.38 (–14.18, 40.94)	0.34
Positive result	–9.99 (–34.90, 14.91)	0.43	23.12 (–9.45, 55.68)	0.16
Positive influenza result	–24.27 (–47.04, –1.51)	0.04	–28.03 (–57.80, 1.74)	0.07
Admission to ED only	–56.76 (–95.74, –17.77)	0.005	–37.99 (–79.52, 3.54)	0.07

CI, confidence interval; ED, Emergency Department; LOS, length of stay.

Other factors found to be non-significant on univariable analysis included: gender, admission to Acute Medical Unit, admission to Infectious Diseases Unit, admission to Respiratory Unit, and shift specimen received in laboratory.

Pre-implementation vs post-implementation.

Other factors found to be non-significant on univariable analysis included: gender, admission to Acute Medical Unit, admission to Infectious Diseases Unit, admission to Respiratory Unit, admission to General Medicine Unit, and shift specimen received in laboratory.

Discussion

To our knowledge, this is one of the first prospective studies measuring laboratory and health service outcomes after FilmArray implementation in a diagnostic laboratory in an adult hospital. We found that use of the FilmArray significantly increased target detections compared to the existing in-house respiratory PCR method and resulted in a decrease in laboratory and clinical TAT. However, we did not find that this translated to reduced hospital or isolation bed LOS.

FilmArray detected a target in 57% of samples compared to 39.5% of samples using the in-house respiratory PCR, which is similar to the positivity rate reported in other studies in adults after FilmArray implementation. This was predominantly due to the broader range of respiratory targets detected by the FilmArray assay.

The pre-implementation period corresponded to the earlier part of the Australian winter season (July–August) and the post-implementation period corresponded to the latter part of the season (September–October). We found that detection of influenza A, as a total proportion of respiratory target detections, was lower in the post-implementation compared to the pre-implementation period. Along with other studies that have shown early detection of influenza A is most likely to influence healthcare resource utilisation (such as earlier discharge or use of isolation rooms),4, 6 we noted a decreased hospital and isolation bed LOS for patients with a positive influenza diagnosis (Table 3). The reduced number of influenza A detections in the post-implementation period may have impacted our ability to detect the health service outcomes of reduced laboratory and clinical TAT. However, we still noted no change in LOS for the post-implementation group, even after accounting for this potential confounder.

This study found a significant decrease in laboratory TAT for respiratory pathogen testing from a median of 21.6 h in the pre-implementation period to 4.4 h in the post-implementation FilmArray period. There was also significantly reduced clinical TAT (from 27.0 h to 9.4 h). In common with other studies that evaluated FilmArray implementation, we found that a significantly larger percentage of test results were available to the clinician prior to patient discharge from the hospital (87.9% compared to 70.5%), increasing the chance that these laboratory results could have an impact on patient care or hospital outcomes.

Interestingly, the TAT for FilmArray in our hands was longer than that reported by some other laboratories, which may have reduced the clinical impact of rapid testing.4, 16, 17 Our median TAT was 4.4 h from time of specimen arrival in laboratory to result report, compared to 1.7 h reported by Rappo et al. This may reflect the specific conditions of our laboratory, such as lack of dedicated scientist time to run the assays (out of working hours we have a single scientist in the microbiology laboratory and specimens are processed in order of clinical priority). Given the actual running time for the test (approximately 1 h), there is scope to further decrease laboratory TAT for the FilmArray in our laboratory.

Laboratory hands-on work time

To the best of our knowledge, ours is the first study to report a time and motion study of laboratory hands-on work time with the FilmArray. We unexpectedly found very little difference in hands-on work time per specimen with the in-house PCR compared to the highly automated cartridge-based FilmArray assay (11.5 vs 10 min per sample). This is likely because the batched nature of in-house PCR testing increases workflow efficiency. It should also be noted that the measured median 10 min hands-on work time for FilmArray is very different from the 2 min claimed by the manufacturer. Although FilmArray is low complexity, the individual cartridge-based nature of testing meant that a single specimen had to be attended to several times by a scientist (for accession, set up, PCR loading, PCR result reading, LIS reporting and clinical results notification). During the present study the cost per individual cartridge for the FilmArray assay was AU$150. In comparison, the approximate cost for the in-house PCR was AU$25 per sample. Given this cost difference, it is unlikely that the FilmArray respiratory panel testing could be justified on cost grounds for routine clinical samples, in the absence of evidence that the testing results in improved patient care or clinical outcomes.

Our finding that reduced clinical TAT in the post-implementation period did not have a statistically significant impact on hospital LOS is different to that reported in some studies after implementation of the FilmArray at other institutions. In a larger retrospective study in children conducted at Children's Healthcare of Atlanta, FilmArray implementation was associated with a significant decrease (3.5 vs 3.2 days) in inpatient LOS for patients with a positive test result, although it is not clear that this minor difference in LOS (approximately 7 h) would be significant from a health systems perspective. Our present study was powered to detect a clinically significant difference in both hospital and isolation LOS which was determined a priori as a minimum of 10 h. Another retrospective study in adults, conducted at the New York Presbyterian Hospital/Weill Cornell Medical Centre, reported reduced hospital LOS in patients testing positive for influenza A after controlling for confounding factors. The median TAT in this study for reporting positive results from the FilmArray results was 1.5–1.7 h; this shorter TAT (compared to the TAT in our current study of 4.4 h) may have increased its impact on health service outcomes. It is possible that there may be a critical laboratory TAT that needs to be reached in order to positively impact on hospital LOS.

We hypothesised that there would be a reduction in isolation bed LOS with the faster availability of respiratory PCR results; however, we did not find a statistically significant impact on isolation bed usage during the post-implementation period. There was unexpectedly low use of symptoms-based isolation in our group of patients being tested for respiratory pathogens (<40%, Table 1), which may have reduced our power to detect a significant change in isolation bed-days. Brendish et al. also reported no change in isolation bed LOS in their randomised controlled trial, although there was a shorter duration to de-isolation for patients empirically isolated on admission who subsequently tested negative for influenza virus.

Hospital isolation bed policies will influence whether rapid microbiological testing has a positive or negative impact on isolation bed LOS. Interestingly, in a retrospective study in children at Cohen Children's Medical Centre (NY), FilmArray implementation resulted in a significant increase in use of isolation precautions (34 per 100 patient days compared to 15 per 100 patient days in the pre-implementation period), suggesting that implementation resulted in more appropriate use of patient isolation rooms, potentially preventing hospital transmission of respiratory targets.

We did not assess the impact of reduced laboratory TAT on antimicrobial use in our hospital, although others have reported some benefit.9, 16 Interestingly, a recent study on the impact of FilmArray testing amongst adult outpatients at a tertiary hospital found that influenza A diagnosis (but not other respiratory pathogen detection) had a positive impact on oseltamivir prescription and reduced antimicrobial prescriptions, suggesting that a rapid molecular test detecting a narrower spectrum of pathogens might be more cost-effective.

Strengths and limitations

A major strength of this study was its prospective design, with real time recording of data throughout the study period over a single winter season. We also did a formal time-and-motion study time to accurately measure hands-on laboratory scientist work time for each assay. Exclusion of patients admitted under the Oncology units and patients who had been admitted to hospital for more than 48 h may have decreased the hospital-wide applicability of our results. Another limitation of the study is that samples were not randomised to FilmArray testing versus ‘standard’ in-house testing, so that there may have been minor unrecognised differences in patient cohorts between the two sequential study periods.

Future directions

We have demonstrated that introduction of a highly automated cartridge-based molecular assay for respiratory targets had a significant impact on laboratory TAT, clinical TAT and availability of results prior to patient discharge from hospital. We did not find that decreased laboratory or clinical TAT directly resulted in reduced hospital LOS or changes in use of isolation beds. Given the increased cost of testing, unchanged laboratory hands-on work time, and lack of health service impact, it is difficult to justify implementation of a relatively expensive test without further evidence of improved clinical outcomes.

Whilst from a simplistic point of view, shorter laboratory TAT should result in shorter hospital LOS, there are other factors that contribute to LOS; for example, the natural history of disease, availability of senior clinicians to make a final decision about patient discharge and discharge planning requirements. Elucidating the relative contributions of different factors to LOS will be important for better allocation of resources, as well as realisation of the need to deploy new assays in conjunction with increased bedside resources.

Future work could focus on further reducing laboratory TAT, and working more closely with the hospital antimicrobial stewardship team and infection prevention team to ensure that earlier availability of laboratory results can be translated into a direct clinical impact.

Conflicts of interest and sources of funding

This study was supported by a Victor Hurley Research grant in aid through the Royal Melbourne Hospital, Parkville, Australia. The authors state that there are no conflicts of interest to disclose.