Comparison of two broadly multiplexed PCR systems for viral detection in clinical respiratory tract specimens from immunocompromised children.

BACKGROUND: The detection of viral respiratory tract infections has evolved greatly with the development of PCR based commercial systems capable of simultaneously detecting a wide variety of pathogens.
OBJECTIVES: Evaluate the relative performance of two commercial broad range systems for the detection of viral agents in clinical respiratory tract specimens from immunocompromised children. STUDY
DESIGN: A total of 176 patient samples were included in the analysis, representing only the first sample collected for each patient, and excluding failed reactions. Samples were de-identified and assayed in parallel using two different, broadly multiplexed PCR systems: ResPlex™ II Panel v2.0 (ResPlex), Qiagen, Hilden, Germany and FilmArray(®) Respiratory Panel (FilmArray), Idaho Technology Inc., Salt Lake City, UT. Method comparison was based upon pair-wise concordance of results according to patient age, viral target and number of targets detected.
RESULTS: The two systems showed an overall concordance, by patient, of 83.8% (p=0.0001). FilmArray detected at least one target in 68.8% of samples, while ResPlex detected at least one target in 56.8%. ResPlex failed to detect 20.7% of FilmArray positives, and FilmArray failed to detect 4% of ResPlex positives. The relative performance of each system (including which system detected a higher number of positive samples) varied when stratified by target viral pathogen.
CONCLUSIONS: Broadly multiplexed PCR is an effective means of detecting large numbers of clinically relevant respiratory viral pathogens. Copyright Â

Background

Viral respiratory tract infections can cause serious morbidity and increased mortality in immunocompromised pediatric patients. The rapid and sensitive detection of such agents has implications for treatment decisions, clinical care, and infection control practices. As in other areas of diagnostic virology, molecular diagnostic methods have shown promise in markedly improving diagnostic sensitivity when compared to culture or antigen detection assays. While such favorable performance characteristics have made molecular methods appealing, their introduction in the clinical laboratory has been slowed by issues related to cost and technical expertise required to perform this testing.

Early versions of such tests have focused on a limited number of pathogens, and typically detected only one or two viruses (or groups of viruses) at a time.1, 2, 3, 4, 5, 6, 7 Thus, detection of all clinically relevant entities has required running an entire panel of tests, compounding costs, and staffing requirements. The advent of broadly multiplexed assays has sought to address some of these issues.8, 9 Front-end multiplexed amplification (typically PCR) followed by detection is capable of identifying over 20 different targets. Thus, a single assay utilizing the sensitivity of PCR can mimic the diagnostic spectrum of culture creating the potential for increasing routine detection beyond cultivable viruses, reducing processing costs, staffing requirements, and turn-around time. This technology also offers the possibility of increased ability to detect multi-viral infection. While the clinical importance of such capabilities is presently uncertain, this may carry important prognostic or infection control related implications, particularly among immunocompromised patients, such as those studied here.

A number of different methodologies have now been proposed or marketed that use such an approach; however, limited published studies have compared different broadly multiplexed systems to one-another. In the current study, we compared two such systems to each other and to a panel of real-time PCR assays targeting individual viruses. Technologies evaluated included the ResPlex II Panel v2.0 (ResPlex), Qiagen, Hilden, Germany and the FilmArray Respiratory Panel (FilmArray), Idaho Technology, Inc., Salt Lake City, UT. Both products used for this study were for research use only (RUO). The FilmArray product has only recently become available as an FDA-cleared assay, and at the time of this submission few studies have been published examining performance of this method using clinical respiratory tract specimens.

Objectives

Two broadly multiplexed PCR systems were compared to each other and to a panel of laboratory developed tests for the detection of respiratory viral pathogens in clinical respiratory tract specimens from pediatric immunocompromised children.

Study design

Study population and samples

Samples were collected prospectively, as part of routine clinical care at St. Jude Children's Research Hospital (SJCRH) between January 13 and May 4, 2010, from children presenting with signs and symptoms of upper respiratory tract infection. Results from patients 19 years of age and above were excluded from the analysis. The SJCRH Institutional Review Board (IRB) classified this study as non-human research; the study was exempted from IRB approval and informed consent requirements were waived.

Samples consisted of bronchoalveolar lavage (BAL) specimens, nasopharyngeal swabs (NPS), nasopharyngeal washes (NPW), and tracheal aspirates (TA). Unused samples remaining after routine diagnostic testing were de-identified and blinded prior to study inclusion. Unique patient and sample identifiers were assigned to each sample in order to match the results obtained from the various methods being compared. Samples were divided into two 0.5 ml aliquots where one was tested by the laboratory developed test panel(LDTP), which included the Pro hMPV Assay Kit (Gen-Probe, San Diego, CA) for detection of hMPV, at SJRCH and the other was tested using the FilmArray assay. Left over sample from the clinical aliquot was then tested by the ResPlexII assay at Diagnostic Laboratory Services (DLS), Aiea, HI.

Sample preparation and testing

Sample extraction for LDTP and ResPlex

Extraction was performed at SJCRH and nucleic acid was sent and stored frozen to DLS until testing at this remote site. This collection, transport, and storage method was used to allow preanalytical standardization between the testing methods. A 50 μl nucleic acid solution was extracted from 250 μl of respiratory sample together with prior addition of RNA and DNA controls for LDTP test; a second 50 μl nucleic acid solution was extracted from 250 μl aliquots of the same respiratory sample without addition of the RNA and DNA controls, for RespPlexII testing. Extractions were otherwise identical and were performed using the Nuclisens® MINIMAG Magnetic Extraction System (bioMérieux Inc., Durham, NC), per manufacturer's instructions. The RNA control was purchased as a ready-to-use lyophilized beadcontaining a 1 kb armored synthetic nucleic acid (Cepheid). The DNA control was a plasmid constructed by inserting a 357-bp DNA fragment of Phocid Herpesvirus type 1 gB gene into vector pUC57 (designed in-house at SJCRH and manufactured at GenScript Corp., Piscataway, NJ). Both controls were detected only in LDTP test.

LDTP PCR

A panel of real-time PCR assays was developed to detect INFA and INFB, RSV, adenovirus, and parainfluenza viruses 1, 2, and 3 from respiratory specimens. The Pro hMPV Assay Kit was also run as part of this panel. The assays utilized TaqMan chemistry and real-time PCR technology and were carried out on the SmartCycler II platform (Cepheid, Sunnyvale, CA). The molecular detection targets, primer and probe sequences are listed in Table S1.

ResPlex II Panel v2.0 (ResPlex)

The ResPlex™ II Panel v2.0 uses a multiplex RT-PCR analysis to amplify and detect 18 respiratory viruses (Table 2) using the QIAplex and xMAP technologies on the QIAGEN LiquiChip System. Controls were included on each test run and consisted of a positive control (in vitro transcribed RNA corresponding to a human genomic DNA sequence), an internal control to check for viral RNA isolation and PCR inhibition, and a sample control to detect traces of human genomic DNA present in all specimens. Viral RNA and DNA were extracted using the Nuclisens® MINIMAG Magnetic Extraction System and 10 μl was amplified by RT-PCR on the GeneAmp 9700 PCR System. The final step consisted of amplified product detection using the LiquiChip 200 Workstation with the QIAplex MDD Software. Total run time was approximately 4 h (1 h hands-on-time) for 24 specimens. Virus detected by ResPlex consisted of respiratory syncytial viruses A and B (RSV), influenza virus types A and B, parainfluenza viruses 1, 2, 3, and 4, human metapneumovirus (hMPV), coxsackieviruses/echovirus, rhinovirus, adenoviruses B and E, human coronaviruses NL63, HKU1, 229E, and OC43, and bocavirus.

Table 2

Detection results for FilmArray® and LDTP.

Frequency (%) (n = 176)	Results by FilmArray®/LDTP				p value*
	+/+	−/−	+/−	−/+
Overall results	55 (31.3)	107 (60.8)	8 (4.6)	6 (3.4)	0.7905
Respiratory pathogen
Adenovirus	3 (1.7)	168 (95.5)	3 (1.7)	2 (1.1)	1
hMPV	12 (6.8)	159 (90.3)	4 (2.3)	1 (0.6)	0.3750
Influenza A	11 (6.3)	163 (92.6)	1 (0.6)	1 (0.6)	1
Parainfluenza	5 (2.84)	170 (96.6)	1 (0.6)	0 (0)	1
RSV	27 (15.3)	144 (81.8)	2 (1.1)	3 (1.7)	1

Exact p-values.

FilmArray

FilmArray utilizes a prefabricated pouch containing lyophilized reagents. The sample is added directly to the pouch wherein specimen preparation, amplification, and detection all take place without further offline sample manipulation. 300 μl of original sample was mixed with 500 μl of FilmArray lysis buffer. Approximately 1 ml of hydration solution was added by syringe to the FilmArray pouch through hydration solution inlet port and 300 μl of sample/lysis buffer mix was added to the pouch through the sample inlet port. The pouch was then loaded on the FilmArray instrument, with automated extraction, amplification, and data analysis; total run time, approximately 1 h (5 min hands-on time). Quality of testing was assured by the inclusion of two RNA process controls(PC) in each pouch (proprietary sequences, Idaho Technology). The RNA process controls were carried through all stages of the test process from samples lysis to PCR and DNA melt analysis. Both controls had to produce positive results for validation of test results. FilmArray detected viral targets: adenovirus, bocavirus, coronavirus 229E, HKU1, NL63, OC43, enterovirus, hMPV, human rhinovirus, influenza virus types A and B, parainfluenza viruses 1, 2, 3 and 4, and RSV.

Statistical methods

The McNemar test was used to compare the accuracy of all three tests; comparison was made in pair-wise fashion and exact methods were used when discordant cells had less than 20 observations. To reduce bias, only the first sample of each patient with multiple samples was included in the analysis. For purposes of analysis, patient age was divided into four different age groups based on years of age: ≤2, 3–5, 6–13, and 14–18. This was a paired study, pair-wise comparisons being made between performance of FilmArray and ResPlex, FilmArray and LDTP, and ResPlex and LDTP.

When targets differed between the systems under comparison, results were collapsed into the narrowest taxonomic category that encompassed both systems of a pair. Results from individual samples in which the first run failed were excluded from the analysis. However, results from second run from consecutive samples in which the entire run failed were included in the analysis as it was assumed that a systematic or technologist error was most likely the cause for the failure of the entire run For the ResPlex system, a positive result was determined using a cut-off mean fluorescent intensity (MFI) of 200.

Results

A total of 440 samples were collected from 210 children during the study period. After excluding samples from patients aged 19 and above, samples obtained after the first collection, and failed runs, the total number of samples analyzed was 176; only a single sample was tested per patient. While there were no failures with the LDTP, the total number of failed runs by FilmArray was 11 and with ResPlex was 15.

The mean age of children with samples included in the analysis was 6.9 years, ranging from 2 months to 18 years. The vast majority of samples were NPW with 167 samples (94.9%), followed by TA with 6 samples (3.4%), BAL with 2 samples (1.1%) and one NPS (0.6%). Over 80% of tested samples came from children who were 13 or younger: samples from children two years or less accounted for 27.3% (48) of samples, between three and five years inclusive were 24.4% (43), between six and 13 were 32.4% (57), and 15.9% (28) of samples came from children aged between 14 and 18, inclusive.

FilmArray and ResPlex

Overall difference in detection of targets between FilmArray and ResPlex was found to be significant (p  < 0.0001). Out of the 121 samples detected positive by FilmArray, ResPlex did not detect a target for 20.7% (25) of samples, while FilmArray missed 4% (4) of samples detected positive by ResPlex (Table 1 ). The total number of samples for which both systems agreed was 147 samples; 96 samples testing positive and 51 samples testing negative. After stratifying by different targets and target groups, differences were observed in the detection of coronavirus 229E (exact p  = 0.0313), INFA (exact p  = 0.0313), enterovirus (p  < 0.0001), and rhinovirus (p  < 0.0001).

Table 1

Detection results for FilmArray® and ResPlex™.

Frequency (%) (n = 176)	Results by FilmArray®/ResPlex™				p value
	+/+	−/−	+/−	−/+
Overall results	96 (54.5)	51 (29)	25 (14.2)	4 (2.3)	0.0001
Respiratory pathogen
Adenovirusa	2 (1.1)	170 (96.6)	4 (2.3)	0 (0)	0.1250*
Bocavirus	0	0	0	0	–
Coronavirus, 229E	3 (1.7)	167 (94.9)	0 (0)	6 (3.4)	0.0313*
Coronavirus, HKU1	5 (2.8)	164 (93.2)	6 (3.4)	1 (0.6)	0.1250*
Coronavirus, NL63	6 (3.4)	169 (96)	0 (0)	1 (0.6)	1*
Coronavirus, OC43	0 (0)	171 (97.2)	5 (2.8)	0 (0)	–
hMPV	11 (6.3)	160 (90.9)	5 (2.8)	0 (0)	0.0625*
Influenza A	6 (3.4)	164 (93.2)	6 (3.4)	0 (0)	0.0313*
Influenza B	0	0	0	0	–
Parainfluenza, 1	2 (1.1)	174 (98.9)	0 (0)	0 (0)	–
Parainfluenza, 2	1 (0.6)	174 (98.9)	1 (0.6)	0 (0)	1*
Parainfluenza, 3	1 (0.6)	173 (98.3)	1 (0.6)	1 (0.6)	1*
Parainfluenza, 4	2 (1.1)	173 (98.3)	1 (0.6)	0 (0)	1*
Enterovirus	9 (5.1)	128 (72.7)	1 (0.6)	38 (21.6)	<0.0001
Rhinovirus	24 (13.6)	107 (60.8)	44 (25)	1 (0.6)	<0.0001
RSV	19 (10.8)	143 (81.3)	10 (5.7)	4 (2.3)	0.1796

ResPlex detects only adenovirus types B and E.

Exact p values.

Analysis stratified by patient age showed that in ages 3–5, 83.7% of samples were positive by FilmArray and of these, ResPlex missed eight samples (22.2%), p  = 0.0391; likewise, age group 6–13, 63.2% of samples were positive by FilmArray but ResPlex missed 19.4% (7), p  = 0.0156 (Table S2).

When positive results were stratified by the number of targets detected by FilmArray, differences in detection were found to be significant in cases where the FilmArray system detected one or two targets (Table S3). In the 76 cases where FilmArray detected one target, ResPlex did not detect 18 of those cases, p  < 0.0001. ResPlex did not detect any targets in six samples of the 34 samples for which FilmArray detected the presence of two targets, p  = 0.0313. In these six samples, the targets not detected by ResPlex were rhinovirus (4), coronavirus HKU1 (2), RSV (2), adenovirus (1), Bocavirus (1), coronavirus OC43 (1), and enterovirus (1).

FilmArray and LDTP

Analysis for this comparison included only viruses targeted by both systems. These target groups were: adenovirus, hMPV, INFA, parainfluenza types 1–3, and RSV. There were no differences in the number of positives detected by either system over the other, either overall or when stratified by target (Table 2 ). Differences in detection rate were not observed when data were stratified by age groups (Table S2). After stratifying by number of targets detected by FilmArray, LDTP detected viruses in six samples that FilmArray called negative (p  = 0.0313); but LDTP did not detect targets in seven samples in which FilmArray detected one target, p  = 0.0156 (Table S4).

ResPlex and LDTP

The percentage of samples detected positive by ResPlex (24.4%) was lower than that for the LDTP (34.7%). Among the samples detected positive by ResPlex, LDTP did not detect 1.1% (2); ResPlex did not detect targets in 20 samples (11.4%) that were detected positive by LDTP, p  = 0.0001. The only target group for which significant differences were observed was influenza A; LDTP detected 12 samples as positive for influenza A but ResPlex did not detect 50% of these p  = 0.0313 (Table 3 ).

Table 3

Detection results for ResPlex™ and LDTP.

Frequency (%) (n = 176)	Results by ResPlex™/LDTP				p value
	+/+	−/−	+/−	−/+
Overall results	41 (23.3)	113 (64.2)	2 (1.1)	20 (11.4)	0.0001
Respiratory pathogen
Adenovirus	2 (1.1)	171 (97.2)	0	3 (1.7)	0.2500*
hMPV	11 (6.3)	163 (92.6)	0	2 (1.1)	0.5000*
Influenza A	6 (3.4)	164 (93.2)	0	6 (3.4)	0.0313*
Parainfluenza	4 (2.3)	170 (96.6)	1 (0.6)	1 (0.6)	1*
RSV	21 (11.9)	144 (81.8)	2 (1.1)	9 (5.1)	0.0654*

Exact p-values.

When results were stratified by age group, significant differences were observed in samples taken from children aged 3 to 13 (Table S2). ResPlex detected 50% less positives compared to LDTP in samples from children aged between 3 and 13; and 25% less in the remaining age groups, though these differences were not significant. Differences in detection between ResPlex and LDTP were significant in samples where ResPlex did not detect any targets (Table S4).

Discussion

The introduction of multiplex molecular amplification assays for the detection of viral respiratory pathogens has improved the sensitivity of routine viral detection methods. The range of agents detected10, 11, 12 has been expanded to include newly discovered and emerging respiratory viruses such as coronaviruses NL63 and HKU1, human metapneumovirus (hMPV), and bocavirus.13, 14, 15, 16, 17, 18, 19 The broadly multiplexed molecular approach studied here has the capability to detect and identify more viruses simultaneously than traditional methods (i.e. culture and DFA) and shows an increased detection rate for co-infections.8, 9, 20, 21, 22, 23, 24 In the current study, coinfections were detected in the range of 1–26%, which is similar to other investigators. Sanghavi et al. investigated both adult and pediatric patients, including organ transplants and found dual infections at 10.4% and triple at 1.3%. In a 2003 study, Guittet et al. reported a range of 5–40% in hospitalized children. The clinical significance of detecting more than one virus is not clear, but Calvo et al. reported multi-viral infections more frequently in hospitalized infants with respiratory tract disease (17.4%). These infections were also linked to higher fever, longer hospital stays, and more frequent use of antibiotics than single RSV infections. In another study by Semper et al. co-infection with hMPV and RSV revealed a tenfold increase in the relative risk for admission into the pediatric intensive-care unit for mechanical ventilation as a result of severe bronchiolitis. There is little published data beyond this on the clinical significance of multi-viral respiratory tract infection, and no work specific to the importance of detecting >2 viruses. The findings here suggest that as broadly multiplexed PCR systems become more common, a new body of literature will need to be developed to address the relevance and implications of such findings for prognosis, treatment, and infection control.

The current study, to our knowledge, is the first reported that compares the FilmArray with the ResPlex II v2.0 for the direct detection of viral agents in clinical respiratory tract specimens from immunocompromised children. The viral targets for the two systems were similar with previously noted exceptions. Overall concordance, by patient, for the two systems was 83.8%, p  = 0.0001. In general, the FilmArray detected a higher number of positive samples. FilmArray showed a detection rate of 68.8% compared to 56.8% for ResPlex. ResPlex failed to detect 20.7% of FilmArray positives, and FilmArray failed to detect 4% of ResPlex positives. The increased sensitivity of the FilmArray may be due to the incorporation of a nested PCR approach in this system compared to one PCR for the Resplex. In a recent retrospective study by Rand et al., the FilmArray was compared to the xTAG RVP (Luminex Corporation, Toronto, Canada) following viral culture and DFA in a set of 200 patient samples. Even though both systems detected 40–50% more viruses than traditional methods, the FilmArray detected significantly more viruses and mixed infections than the xTAG RVP.

In the current study, ResPlex detected less RSV, influenza type A, hMPV, and adenovirus than the FilmArray. Increased RSV detection by the FilmArray was also noted by Rand et al. compared to the xTAG RVP. Adenoviruses were not typed in this study, but the reduced rate of detection by ResPlex may relate to the fact that it detects only subtypes B and E, compared to the FilmArray which detects all subtypes. It was noteworthy that the detection rate for enterovirus and rhinovirus were inversely related when the two systems were compared to each other. Furthermore, if the FilmArray and ResPlex enterovirus/rhinovirus were added together in each case, they would be approximately equal in the total number of detections (78 vs. 72, respectively). This could be the result of crossover detection related to differences in the nucleic acid sequences between the two assays. Rhinoviruses and enteroviruses show extensive genomic similarity, but yet can be very different in their phenotypic characteristics. The self-contained nature of the pouch system used by the FilmArray, where sample preparation, amplification, and detection take place in a closed system markedly reduces any risk for carry-over contamination between specimens. Both the Resplex II v2.0 and the xTAG RVP require external nucleic acid extraction followed by multiplexed amplification and detection using a liquid-phase array technology. This approach may carry a risk of carry-over contamination due to both additional sample manipulation and the lack of a closed system. The difference in nucleic acid extraction prior to amplification and detection, along with other variables, such as nucleic acid sequence variations and chemistry may also contribute to the differences in test performance. Specimen collection procedures can also contribute to a difference in the overall test performance for any molecular test. The use of oropharyngeal swabs, nasopharyngeal swabs, or nasopharyngeal washings showed multiplex real-time RT PCR detection rates of 54.2%, 73.3%, and 84.9%, respectively in a study by Lieberman et al. Other investigators have shown similar differences with the use of different swabs and collection procedures.33, 34

The performance of both traditional and molecular-based tests varies according to the age group studied. However, generally the younger children tend to show a higher viral load than older individuals. In general, this observation may account for a more favorable performance between traditional and molecular-based testing in the younger age group. In the present study, both the FilmArray and the Resplex exhibited similar viral detection in the ≤2 years age group. However, the FilmArray outperformed the ResPlex in the 3–5 year and 6–13 year groups. The viral load may have been lower in the >2 years group, accounting for the overall increase in sensitivity for the FilmArray.

The material costs for most molecular testing are greater than for DFA and culture, but the overall benefit may depend upon the protocol involved and could result in an overall cost reduction as described by Mahony et al. The other aspect of the cost analysis is the actual labor time involved for the FilmArray and ResPlex. In the current study, the hands-on-time for the ResPlex was approximately 1 h with a total time of 4 h for testing 24 samples. The FilmArray involved only 5 min for hands-on-time with a total time of 1 h/sample. Based on the one cartridge per machine design of the FilmArray instrument, the throughput for FilmArray is integrally related to the number of machines available in a given laboratory. So while laboratories that process less than one sample per hour would be able to take advantage of apparently improved turnaround time with FilmArray, ResPlex is more accommodating of larger specimen loads. It would therefore take six FilmArray instruments to meet the 4-h turnaround time of a full Resplex run. Likewise, the apparent labor-saving benefits of the FilmArray system apply primarily to low volume settings. Although with practice, processing time might be less than the above stated 5 min per sample, total hands-on time for more than 15–20 samples might be expected to meet or exceed that of Resplex.

This study demonstrates the promise of automated, broadly multiplexed PCR systems for the detection of viral respiratory pathogens. Such methods combine the exquisite sensitivity of molecular amplification with a breath of targets previously attainable only through the use of culture-based techniques. These operating characteristics, together with increasingly self-contained, automated, and easy-to-use methods, bring us closer to the mainstream adaptation of molecular diagnostic testing, no longer limited to academic centers or reference laboratory settings. The widespread use of these powerful techniques will have implications for diagnosis, treatment, infection control, and resource utilization in the healthcare setting.

Funding

Funding for this study was supported in part by the American Lebanese Syrian Associated Charities, ALSAC and the Anderson Charitable Foundation. Reagents were kindly supplied by QIAGEN and Idaho Technology, Inc. Instrumentation for the FilmArray process was provided by Idaho Technology, Inc.

Competing interests

R. Hayden has served as a consultant for Idaho Technology.

Other authors have no competing interests to report.

Ethical approval

The study was classified as non-human research; study was exempted from IRB approval and informed consent requirements were waived.