Comparison of the performance of 2 commercial multiplex PCR platforms for detection of respiratory viruses in upper and lower tract respiratory specimens.

The performance of the CLART® PneumoVir system with that of the Luminex xTAG RVP Fast v1 assay for detection of most common respiratory viruses in upper and lower tract respiratory specimens (n=183) from unique patients with influenza-like syndrome or lower tract respiratory infection. Nested PCR coupled to automated sequencing was used for resolution of discrepancies. Fully concordant results were obtained for a total of 122 specimens, whereas 56 specimens gave partially (n=21) or fully discordant (n=35) results (Kappa coefficient, 0.62). The overall specificity of the Luminex xTAG RVP Fast v1 assay was slightly higher than that of the CLART® PneumoVir assay for human bocavirus, influenza A virus/H3N2, influenza B virus, human metapneumovirus, and parainfluenza virus, whereas the sensitivity of the latter was higher for most targeted viruses except, notably, for picornaviruses. This was irrespective of either the origin of the respiratory specimen or the age group to which the patients belonged.

Introduction

Acute respiratory tract infections caused by respiratory viruses (RVs) are the most common infections attended in hospitals and primary care centers. They can range from mild self-limiting illness to severe disease, the latter particularly in pediatric and severely immunosuppressed patients (Ison and Hayden, 2002, Vallières and Renaud, 2013). A great diversity of RVs produces clinically indistinguishable symptoms; thus, laboratory diagnosis based on the simultaneous detection of multiple targets has become the best option for etiological diagnosis. Molecular methods are being increasingly used for the diagnosis of respiratory viral infections due to their fine sensitivity, specificity, and timely turnover and are progressively replacing conventional methods (Caliendo, 2011). The decision to choose one or other molecular system is complex and requires systematic and comparative evaluations. In this context, the Luminex xTAG RVP Fast assay (Luminex Molecular Diagnostics, Austin, TX, USA) has been extensively evaluated in recent years, and comparative studies have been also published showing high reliability for detection of 19 RVs in different clinical settings (Babady et al., 2012, Dabisch-Ruthe et al., 2012, Gadsby et al., 2010, Hwang et al., 2014, Jokela et al., 2012, Krunic et al., 2007, Merante et al., 2007, Pabbaraju et al., 2008, Pabbaraju et al., 2011, Pillet et al., 2013, Popowitch et al., 2013, Rand and Houck, 2011, Raymaekers et al., 2011). The CLART® PneumoVir assay (Genomica, Coslada, Spain) is a reverse transcription–polymerase chain reaction (RT-PCR) DNA microarray method that makes it possible to detect simultaneously 17 RVs. The reliability of this procedure for the detection of RVs in children and adults has been previously shown (Culebras et al., 2013, Frobert et al., 2011, Pillet et al., 2013, Renois et al., 2010, Tokman et al., 2014). Nevertheless, little is known as to how this system compares with other multiplex platforms. In the current study, the performance characteristics of the CLART® PneumoVir assay was compared to that of the xTAG RVP Fast v1 assay for detection of RVs in clinical specimens from children and adults with influenza-like syndromes or lower tract respiratory infections.

Materials and methods

Patients and samples

This was a retrospective study including a total of 183 nonconsecutive upper (n = 125) or lower tract (n = 58) respiratory specimens obtained from unique patients received at the Microbiology Services of the Hospital Ramon y Cajal from Madrid and the Hospital Clínico Universitario from Valencia, between January 2007 and December 2010. The clinical indications for RVs testing included the occurrence of influenza-like clinical symptoms or x-ray–documented lower respiratory tract infection (bronchiolitis or pneumonia). Relevant clinical and demographic data from patients and the type of specimens collected are shown in Table 1 . Throat and nasopharyngeal swabs were collected with flocked swabs in universal transport medium (Beckton Dickinson, Sparks, MD, USA, or Copan Diagnostics, Murrieta, CA, USA). The remaining types of specimens were transported undiluted. In all cases, specimens were received at the laboratory within 30 min of collection and were conserved at 4 ° C until processed (within 18 h of reception). Nucleic acid extraction was performed using the Qiagen EZ-1 Viral extraction kit on the EZ1 Robot instrument (Qiagen, Valencia, CA, USA) at Hospital Clínico Universitario or the NucliSENS® easyMAG™ method (BioMérieux, Madrid, Spain) at the Hospital Ramón y Cajal, following routine diagnostic protocols established at each center and according to the manufacturer's instructions. Both nucleic acid extraction platforms have been validated for the Luminex xTAG RVP Fast assay by the manufacturer (package insert; www.luminexcorp.com/Assays/xTAGRVP). Likewise, the NucliSENS® easyMAG™ method has been previously validated for its use coupled to the CLART® PneumoVir assay (Frobert et al., 2011, Pillet et al., 2013). In turn, the Qiagen EZ-1 Viral extraction kit has been validated by the manufacturer (personal communication) and by our group (unpublished results) against the manual nucleic acid extraction and purification method recommended by the manufacturer in the package insert. Sample volumes of 200 and 250 μL were used for nucleic acid extraction by the EZ1 and the NucliSENS® easyMAG™ platforms, respectively. Both methods use isopropanol as a solvent. The nucleic acids were eluted in a volume of 60 and 55 μL for the EZ-1 and the NucliSENS® easyMAG™ kits, respectively. Both leftover specimens and nucleic acid extracts were then conserved at −70 °C for further investigations. Initial testing at both laboratories was performed with the Luminex xTAG RVP Fast v1 assay. Frozen nucleic acid extracts that had been stored for a maximum of 6 months were thawed for testing with the CLART® PneumoVir assay. Either original specimens stored at −70 °C (new extraction) or frozen nucleic acid extracts (when available) were used to analyze the discrepancies. In order to determine whether long-term storage and repeated freezing and thawing (maximum, 3 cycles) had any effect on RVs detectability, 10 randomly selected nucleic acid extracts that were subjected to nested PCR sequencing for analysis of discrepancies were reanalyzed by both the Luminex xTAG RVP Fast v1 assay and the CLART® PneumoVir assay. Reanalyses gave similar results to those obtained previously.

Table 1

Clinical and demographic data from patients and types of specimens included in the study.

Clinical feature	Number of samples (%)
Patient's gender
Males	107 (58.4)
Females	76 (41.6)
Patient's age
Adults	100 (54.6)
Median age (range)	56 (18–84)
Children	83 (45.4)
Median age (range)	1 (1 month–16 years)
Patient's clinic admission
Emergency unit	6 adults/23 pediatrics (3/10)
Intensive care unit	41 adults/13 pediatrics (17/5)
Hematology	45 adults/1 pediatrics (25/1)
Others, inpatient	8 adults/26 pediatrics (4/14)
Cystic fibrosis, outpatient	0 adults/20 pediatrics (0/11)
Upper respiratory samples
Throat swabs	46 (25)
Nasopharyngeal swabs	8 (4)
Nasopharyngeal aspirates	59 (32)
Respiratory secretions	12 (7)
Lower respiratory samples
Endotracheal aspirates	10 (5)
Bronchial brushings	22 (12)
Bronchoalveolar lavage fluid	22 (12)
Sputum	4 (2)

Molecular detection of respiratory viruses

The Luminex xTAG RVP Fast v1 assay was used in the current study. In this assay, nucleic acids from the sample are converted to complementary DNA and mixed with short sequences (TAG primers) of DNA specific to each viral target. If the target is present, the primer will bind and will be lengthened through a process called target specific primer extension. During this extension, a label is incorporated. Color-coded beads are added to identify the tagged primers. Attached to each differently colored bead is an anti-TAG sequence specific to 1 of the extended TAG primers. Each anti-TAG only binds to the complementary TAG sequence on the primer. Samples are then placed in a Luminex instrument where beads are read and analyzed by lasers. The lasers identify the color of the bead (specific to a virus of subtype) and the presence or absence of the labeled primer. The MS2 phage genome (ssRNA, size 3569 nt.) is added to the specimens and serves as a control for nucleic acid extraction efficiency (internal positive control). The bacteriophage Lambda is added to the amplification reactions as a control for RT-PCR efficiency. RT-PCR was performed according to the xTAG RVP Fast assay product insert instructions (10-μL template volume) on a UnoCycler thermocycler (VWR International BVBA, Leuve, Belgium). RT-PCR was followed by a single-step hybridization of PCR products to the fluorescent bead array and incubation with reporter reagents. The plate was then analyzed using the xMAP 200 IS instrument (Luminex Molecular Diagnostics, Toronto, Canada) using the xPONENT software (v3.1).

The Luminex xTAG RVP Fast v1 assay allows the detection of adenovirus (Adv); human bocavirus (hBoV); human coronavirus (hCov) E-229, HKU1, NL63, and OC43; seasonal influenza A virus (InfA) A/H1N1, InfA/H3N2, and other InfA viruses (non-subtypificable); influenza B virus (InfB); human metapneumovirus (hMPV) A and B; parainfluenza virus (PIV) 1, 2, 3, and 4A-4B; respiratory syncytial virus (RSV) A-B; and enterovirus/rhinovirus (EvRh).

The CLART® PneumoVir DNA array assay (Genomica, Coslada, Spain) was performed and interpreted following the manufacturer's recommendations. This assay is based on the amplification of specific fragments (120–330 bp) of the viral genome by means of 2 multiplex PCRs (RT-PCR or PCR). During a 5-h RT-PCR/PCR amplification, the amplified products were labeled with biotin. Following amplification, hybridization with specific probes immobilized sites of the microarray was performed. After incubation with a streptavidin–peroxidise conjugate, the addition of tetramethylbenzidine resulted in the appearance of an insoluble product, which precipitated at the hybridization sites on the microarray. The hybridization profile was read on the clinical array reader and interpreted by means of the CLART® pneumoVir Software. Amplification reactions were performed on a UnoCycler thermocycler (VWR International BVBA) using a template volume of 5 μL. An internal control was added to the amplification reactions, as specified by the manufacturer.

The CLART® PneumoVir DNA array assay differs from the Luminex xTAG RVP Fast assay in that it detects influenza C virus but does not allow the detection of the alphacoronavirus NL63 virus and the betacoronaviruses HKU1 and OC43. The CLART® PneumoVir is able to discriminate between rhinovirus and enterovirus genus, and it permits the identification of the new influenza A/H1N1v.

Discrepancies between both methods were resolved by means of an “in-house”–developed nested PCR assays and direct sequencing of amplicons following previously published protocols (Coiras et al., 2003, Coiras et al., 2004, Coiras et al., 2005, López-Huertas et al., 2005). Only viral agents missed by either one or the other assay were targeted in the analysis of discordances. Hence, the presence of viral agents detected by both systems was not confirmed by nested PCR coupled to sequencing. We distinguished between fully and partial discordant results. The former referred to qualitative discrepancies (positive by one method, but negative by the other assay, or when the virus detected was different in each assay). The latter were those in which both assays were coincident in at least 1 virus, although 1 assay could identify more viruses than the other. Full concordance was defined as obtaining identical results by both methods (including negative results).

Settlement of results and statistical analysis

A true positive was defined as either being positive (identical result) by the 2 assays or by a single assay if confirmed by the corresponding “in house” RT-PCR assay coupled to direct sequencing. Agents not included in CLART® PneumoVir panel and detected by the Luminex xTAG RVP Fast v1 (hCov HKU1, NL63, and OC43) were excluded for comparative analyses. Pairwise differences in sensitivity were determined by the McNemar test, with P values <0.05 being considered significant. The level of agreement between both assays was calculated by means of Kappa statistics.

Results

Among the 183 respiratory specimens initially included in the study, 5 were positive by the Luminex xTAG RVP v1 assay for human CoVs not included in the CLART® PneumoVir panel (CoV NL63, n = 2; CoV HKU1, n = 1; and CoV OC43, n = 2). These were excluded from the analyses described below. Thus, the final number of samples included for comparative purposes was 178. The total number of viruses detected by the Luminex xTAG RVP Fast assay and the CLART® PneumoVir both was comparable (n = 140 versus n = 138, respectively; P  = 0.92). Likewise, both methods performed comparably for detection of multiple infections (n = 29 versus n = 27, respectively; P  = 0.94).

Fully concordance was obtained for a total of 122/178 (68.5%) specimens (65 identical positive results and 57 samples yielding negative results in both assays). Thus, the overall rate of concordance was 80% (Kappa coefficient, 0.62). A single RV was detected in 54 out of the 65 specimens (83.0%) yielding concordant positive results. Two and 3 RVs were detected in 9 (13.8%) and 2 (3.0%) of these samples.

Partially or fully discordant results were observed in 21/178 samples (11.8%) and 35/178 samples (19.6%), respectively. The latter discrepant results corresponded to 28 specimens testing negative by one method and positive by the other and 7 specimens testing positive by both methods but yielding different viral agents. It is of note that InfA H1N1/v was detected by the CLART® PneumoVir in 5 specimens. These specimens tested either negative (n = 2) or positive for InfA H1/N1 (n = 3) by the Luminex xTAG RVP Fast v1 assay. A detailed description of discrepant results is shown in Table 2 .

Table 2

Viral agents detected in partially or fully discordant specimens.

	Fully discordant samples (n = 35)		Partially discordant samples (n = 21)
Discrepant virus	CLART® PneumoVir, no./confirmation by nested PCR coupled to sequencing (no.a)	Luminex xTAG RVP Fast, no./confirmation by nested PCR coupled to sequencing (no.a)	CLART® PneumoVir, no./confirmation by nested PCR coupled to sequencing (no.a)	Luminex xTAG RVP Fast, no./confirmation by nested PCR coupled to sequencing (no.a)
Adv	3/2 (2)	0	1/0 (0)	0
hBoV	4/1 (4)	1/1 (1)	2/1 (1)	0
EvRh	4/3 (3)	19/16 (16)	3/3 (3)	11/8 (8)
InfA A/H1N1	3/0 (2)	4/0 (1)	2/1 (2)	4/0 (3)
InfA A/H1N1v	2/2 (2)	-	3/3 (3)	-
InfA A/H3N2	1/0 (1)	0	2/0 (0)	0
hMPV	1/1 (1)	0	1/1 (1)	0
PIV-1	1/0 (1)	1/0 (1)	0	0
PIV-2	1/1 (1)	0	0	0
PIV-3	1/0 (1)	0	1/0 (0)	0
RSV A-B	2/1 (1)	0	1/0 (0)	0

Number of available specimens for confirmation by nested PCR coupled to sequencing.

Table 3 shows the number of viral agents detected by each method. An excellent concordance (Kappa index >0.90) was found for hMPV, RSV, PIV-3, and PIV-4. Conversely, a suboptimal concordance (Kappa index <0.60) was found for InfA A/H1/N1 and picornaviruses (EV/Rh). The data clearly proved the superiority of the Luminex xTAG RVP Fast v1 assay over the CLART® PneumoVir assay for detection of picornaviruses (P  ≤ 0.001). In contrast, the CLART® PneumoVir method appeared to perform slightly better for detection of most of the other RVs included in the panels, although the differences did not reach statistical significance (Table 3). The performance of both assays was irrespective of either the origin of the respiratory specimen or the age group to which the patients belonged (data not shown).

Table 3

Viral agents detected by the CLART® PneumoVir and the Luminex xTAG RVP Fast assays.

Virus	CLART® PneumoVir, n (%)	Luminex xTAG RVP Fast, n (%)	P valuea	Kappa index
Adv	13 (7.3)	9 (5.1)	0.13	0.81
hBoV	13 (7.3)	8 (4.5)	0.13	0.65
hCov	1 (0.6)	2 (1.1)	1.0	0.66
EvRh	36 (20.2)	59 (33.1)	<0.001	0.48
InfA A/H1N1	6 (3.4)	9 (5.1)	0.58	0.1
InfA A/H1N1v	5 (2.8)
InfA A/H3N2	6 (3.4)	3 (1.7)	0.25	0.66
InfB	3 (1.7)	2 (1.1)	1.0	0.80
hMPV	14 (7.9)	12 (6.7)	0.50	0.92
PIV-1	3 (1.7)	3 (1.7)	1.0	0.66
PIV-2	2 (1.1)	1 (0.6)	1.0	0.66
PIV-3	7 (3.9)	6 (3.4)	1.0	0.92
PIV-4	3 (1.7)	3 (1.7)	1.0	1.0
RSV A-B	26 (14.6)	23 (12.9)	0.25	0.93

The data refer to RVs presumably present in respiratory specimens (prior to analysis of discrepancies by nested PCR couplet to direct sequencing).

McNemar test. P values <0.05 were considered significant.

A total of 47 out of 56 specimens yielding discordant results were further analyzed by nested PCR coupled with automated sequencing. No sufficient volume for analysis was available from the remaining 9 specimens. The viral agent missed by either one or the other multiplex PCR assay was confirmed by the reference method in 41/47 specimens. In the remaining 6 specimens, the presence of 1 or more viral agents in respiratory specimens, as detected by one or the other method, could not be confirmed by the reference method and thus were considered false-positive results (Table 2). Following resolution of discrepancies, the data indicated that the sensitivity of the Luminex xTAG RVP Fast v1 assay was higher than of the CLART® PneumoVir assay for detection of picornaviruses (89.8% versus 59.3%). The opposite was true for Adv (90.0% vs. 100%), hBoV (80% versus 90%), InfB (66.7% versus 100%), hMPV (92.3% versus 100%), PIV-2 (50% versus 100%), and RSV A-B (92.0% versus 100%). Both assays were equally sensitive for CoV (100%), InfA H1/N1 (100%), InfA H3/N2 (100%), PIV-1, PIV-3, and PIV-4 (100%). In turn, the specificity of the Luminex xTAG RVP Fast v1 assay was 100% for all targeted viruses except for InfA H1/N1 (97.7%) and PIV-1 (99.4%). The specificity of the CLART® PneumoVir assay was 100% for Adv, CoV, picornaviruses, InfA H1/N1v, InfB, PIV-2, PIV-4, and RSV A-B; 99.4% for InfA H3/N2, hMPV, PIV-1, and PIV-3; and 98.2% for hBoV and InfA H1/N1.

Discussion

In the current study, the performance characteristics of 2 commercially available multiplex PCR assays for detection of RVs, the CLART® Pneumovir assay and Luminex xTAG RVP Fast v1 assay, were compared. It is of note that this version is no longer available in the market. Both assays have a comparable hands-on time and time to result (slightly longer for the CLART® PneumoVir assay; approximately 5.5 h versus 4 h). Our data indicated the following: i) the overall degree of concordance between both assays was 80% (Kappa coefficient, 0.62). As discussed below, this was mostly due to discrepancies in the detection of picornaviruses. In this sense, the Luminex xTAG RVP Fast v1 assay was clearly superior (P  < 0.001) over the CLART® PneumoVir assay. ii) The sensitivity of the CLART® PneumoVir assay was higher than that of the Luminex xTAG RVP Fast v1 assay for most targeted agents, except notably for picornaviruses. In contrast, the overall specificity of the latter assay was slightly higher than that of the CLART® PneumoVir assay for most RVs. The results were highly concordant (Kappa index >0.90) for hMPV, RSV, and PIV-3 and PIV-4, whereas striking discrepancies were observed for InfA H1/N1 and picornaviruses. iii) The performance of both assays was comparable irrespective of the origin of the specimen (upper versus lower respiratory tract) and the age of patients (pediatric versus adult patients).

In line with our data, Pillet et al. (2013) found the CLART® Pneumovir assay to display a better overall sensitivity than the Luminex xTAG RVP Fast v1 assay, mostly due to an increased rate of detection of influenza viruses (specially InfB), Adv, and hBoV. A suboptimal sensitivity of the Luminex xTAG RVP Fast v1 assay for detection of Adv, hBoV, InfA viruses, InfB, and RSV A-B in comparison to that of other commercial platforms (eSensor RVP and the FilmArray RVP) has also been reported (Babady et al., 2012, Popowitch et al., 2013).

On the contrary, the CLART® PneumoVir was poorly sensitive for detection of picornaviruses. This has also been reported in other studies (Pillet et al., 2013, Tokman et al., 2014) and may likely be due to the fact that the assay only detects enterovirus type B (Pillet et al., 2013). False-positive results were scarce, and most frequently obtained with the CLART® PneumoVir assay, especially for hBoV and some PIVs, and notably in the context of mixed infections. Mispriming is a likely reason accounting for these false-positive results. Nevertheless, given the high overall sensitivity of CLART® PneumoVir, we cannot rule out the possibility of these being true positives that remained undetected by the nested PCR assay employed for resolution of discrepancies. This is in contrast to previously published data (Pillet et al., 2013, Tokman et al., 2014). The use of different criteria for resolving discordant results among the studies may account for these discrepancies.

Limitations of the current study are the relatively scarce number of positive specimens for several viral agents, and its retrospective design, which implied the use of thawed instead of freshly obtained specimens for testing with the CLART® PneumoVir assay. In this context, there is also inherent bias associated with selecting archived specimens for comparative studies. In addition, due to financial constraints, detection of a given target by both methods was considered a true-positive result without further testing by the reference method. Despite the above limitations, our data proved the reliability of the CLART® PneumoVir assay as compared to the Luminex xTAG RVP Fast v1 assay for the diagnosis of upper and lower tract respiratory infections, both in adult and pediatric patients.