Evaluation of three commercial multiplex assays for the detection of respiratory viral infections.

BACKGROUND: Timely identification of respiratory virus infection is essential to mitigate inappropriate antibiotic use and to implement appropriate treatment and/or infection control procedures. As such, multiplexed PCR assays have become standard in many virology laboratories.
OBJECTIVES: To compare the Seeplex RV15 (test of record) with two newer generation multiplex assays, the Anyplex II RV16 and the xTAG respiratory virus panels. STUDY
DESIGN: Two hundred and three retrospective and 36 prospective respiratory samples were tested by all three assays. Samples were deemed to be positive if they tested positive for a virus by at least two of the three respective assays. Negative samples also had to test negative by at least two of the three assays. Inconclusive samples were those that showed band signal intensity between 0 and 100 on the RV15, but had not been previously tested on the RV16 or xTAG. RESULTS AND
CONCLUSIONS: Overall sensitivity and specificity of all three assays were similar (∼85% and 100%, respectively). Given each assay can identify multiple different viruses, the targets reported by one assay did not always agree with each target from another assay. Partial discordant rates were 47% and 21% for positive and negative samples, respectively. These higher than expected partial discordant rates may be due to primer or chemistry differences amongst the three multiplex assays.

Background

Respiratory viral infections are a significant cause of morbidity and mortality globally (Esposito et al., 2013, Huijskens et al., 2013). Early identification of respiratory pathogens permits rapid implementation of appropriate infection control precautions, decreased antibiotic use and where appropriate, initiation of antiviral therapies (Barenfanger et al., 2000, Heinonen et al., 2011). Traditional laboratory methods such as viral culture and direct fluorescent-antibody testing are time consuming and lack sensitivity, and are no longer the method of choice (Gharabaghi et al., 2011, She et al., 2010). Currently, molecular methods are now standard and are employed in most virology laboratories. Multiplexed assays have enabled the detection of several viral targets and permit the simultaneous identification of co-infections in patient specimens (Esper et al., 2011). There are currently several multiplex assays available commercially. Three such Health Canada approved assays include the Seeplex RV15 ACE Detection Kit (RV15) (Seegene, South Korea), the Anyplex II RV16 (RV16) (Seegene, South Korea) and the xTAG Respiratory Viral Panel (xTAG) (Luminex, United States)

The RV15 is a multiplex assay based on dual priming oligonucleotide (DPO) technology (Bruijnesteijn van Coppenraet et al., 2010). The list of fifteen detectable viruses include: influenza A virus (INF A), influenza B virus (INF B), respiratory syncytial viruses A and B (RSVA and RSVB), adenovirus (ADV), human metapneumovirus (hMPV), coronavirus OC43 (CoV OC43), parainfluenza viruses (PIV) 1–4, rhinovirus (RhV) A to C, enterovirus (EV), and Bocaviruses (BoV).

The RV16 is based on Tagged Oligo Cleavage Extension (TOCE™) technology, which makes it possible to detect multiple pathogens in a single fluorescence channel using real time PCR (Kim et al., 2013). The RV16 can detect a total of 16 viruses including serotypes of each virus. The RV16 viral panel is identical to the RV15 viral panel with the additional detection of CoV 229E and CoV NL63 viruses.

The Luminex xTag system is a liquid-bead-suspension-array that is based on multiplex PCR (Jokela et al., 2012). Its viral panel includes influenza A virus (INF A) H1, H3, H1N1, influenza B virus (INF B), respiratory syncytial viruses (RSV), adenovirus (ADV), human metapneumovirus (hMPV), coronavirus (CoV) 229E, CoV NL63, CoV OC43, HKU1, parainfluenza viruses (PIV) 1–4, rhinovirus (RhV)/enterovirus (EV), and Bocaviruses (BoV). It is important to note that, bocaviruses were previously not included on the viral panel for xTAG; however, version 2 being analyzed in this study has this viral target as part of its testing panel.

Thus, this study aims to compare the diagnostic performance of the RV16 and xTAG assays to that of the RV15, as the test assay of record.

Study design

Specimens

Patient respiratory specimens including nasopharyngeal swabs (NPS) and bronchoalveolar lavage (BAL) fluid samples submitted for RV15 testing were collected between November 2012 and June 2013. NPS samples were collected using flocked swabs (Starwab Multitrans Collection and Transport System). A total of 239 samples were collected. Of these samples, 203 were retrospective and 36 were prospective. Retrospective samples were RV15-test-positive sample aliquots that were stored at −80 °C. After undergoing one freeze-thaw cycle, the retrospective samples were tested simultaneously on the Anyplex II RV16 and xTAG. Prospective samples were tested as they were received by the laboratory and were simultaneously tested on all three assays. The patient demographics in this study were as follows: 104 female patients ranging from 0 to 95 years old and 135 male patients ranging from 0 to 94 years old. The mean ages of the female and male patients in this study were 42.8 and 33.6 years old, respectively.

Nucleic acid extraction and internal control

Nucleic acid extraction for all assays was performed on the MagnaPur Compact (Roche, Switzerland). The initial input volume for all Seegene RV15 and RV16 extractions was 700 μL (300 μL Lysis Buffer and 400 μL patient sample) with a final elution volume of 100 μL. Samples for the Luminex MagPix assay had an initial input volume of 400 μL (200 μL Lysis Buffer and 200 μL patient sample) with a final elution volume of 50 μL. MS2 bacteriophage was added as an internal control prior to extraction for both the Luminex MagPix and Seegene RV16 assays as per manufacturer’s instructions. The Seegene RV15 internal control and specific primer were added following extraction, according to manufacturer’s instructions.

RV15 testing

Samples were tested using the Seegene Seeplex RV15 One Step ACE Detection Kit. Each sample was simultaneously amplified in three separate reactions with corresponding primer sets specified as “A”, “B” and “C”. The final volume of the PCR reaction mixture was 50 μL, containing 40 μL of One-Step RT-PCR Master Mix and 10 μL of the sample’s eluate. Master Mix composition was as per manufacturer’s instructions, as were thermocycling conditions using the SeeAmp thermocycler. Reaction mixtures were vortexed prior to thermocycling, to allow for thorough reaction mixing. The amplified PCR products were analyzed by agarose gel electrophoresis using the Caliper gel based detection platform (Life Sciences, United States). Following analysis, the Seegene Viewer software assigned a positive or negative result for each virus present in the sample tested against the viral panel.

RV16 testing

Samples were tested using the Seegene RV16 Detection kit. Reverse transcription was performed, followed by cDNA synthesis on the SeeAmp thermocycler using manufacturer specifications.

PCR was then conducted in a final volume of 20 μL and reactions were analyzed on the CFX96 Real-Time PCR System (BioRad) using manufacturer’s instructions. The Catcher Melting Temperature Analysis (CMTA) was achieved by cooling the samples down to 55 °C for 30 s and then heating the mixtures from 55 °C to 85 °C. During the heating period, fluorescence (F) and Temperature (T) were measured continuously. Curves of the negative derivative of the fluorescence over temperature versus temperature (-dF/dT versus T) were generated by the CFX96 to determine the samples melting point. The melting points were interpreted by using Seegene Viewer software as either “ + ” or “−” for each virus and its subtype. The RV 15 and RV16 limit of detection is 50 copies for each virus type, as reported by the manufacturer. The manufacturer product insert does indicate that for the RV15 some strain variations can cause the sensitivity for Enterovirus and Rhinovirus to vary 10–100 fold.

xTAG RVP fast v2

Reverse transcription and cDNA amplification were conducted on the Luminex MagPix as per manufacturer instructions, using an Eppendorf thermocycler with ramp speeds of 1.2 °C/sec followed by reverse transcription, cDNA amplification, and bead hybridization.

After hybridization, plates were transferred to the Luminex MagPix for detection. Each well was analyzed for bead hybridization using TDAS RVP FAST 2.20 software and cutoff thresholds for each virus were determined, based on previously determined thresholds by the manufacturer. Samples were considered positive for a virus or its serotype if the threshold was met or exceeded.

Definitions and data analysis

Samples were considered positive if the sample tested positive for the same virus by at least two of the three assays. Samples were deemed to be negative if they tested negative by at least two of the three assays. Inconclusive results were previously defined by the RV15 as positive specimens if the band signal intensity was between 0 and 100. In specimens where the virus or viruses identified were not concordant between all the three assays, conflicting results were classified based on discordant virus identities and analyzed accordingly. Agreement, sensitivity and specificity were calculated for each assay.

Results

Sample classification

The distribution of viruses detected is shown in Table 1 and using our definition of positive or negative result being agreement in two out of three assays, a total of 161 positive specimens were identified. Of these, 32 (19%) specimens were co-infected with more than one type of virus with 26 and 6 samples having dual and triple co-infections, respectively.

Table 1

Virus Distribution.

Virus	Number of Positive Specimens
Adenovirus	12
Bocavirus	10
Coronavirus	23
Human Metapneumovirus	15
Inflluenza A	20
Influenza B	11
Parainfluenza 1	1
Parainfluenza 2	2
Parainfluenza 3	14
Parainfluenza 4	2
Rhino/Enterovirus	50
RSV A	31
RSV B	10
Total	201

Eighty-six positive samples (53%) showed complete agreement among all three assays. Seventy-five (47%) positive specimens had partial discordant results (where a virus was only identified by two of three assays). Of these, 57 samples had a single discordance, 16 samples had two discordances present and one sample each, had three and four discordances, respectively Table 2 provides a representative list of 15 specimens with their respective results by each assay. Noteworthy is that all 3 assays identify one common virus but one of the other assays also identified a different second or third virus, resulting in a patient co-infection. This profile of results was reported as a partial discordance.

Table 2

Table of Discordant Results Amongst Positive Specimens.

Specimen #	Number of Discordances	RV15 Result	xTAG Result	RV16 Result
1	1	Rhino	Entero/Rhino	Rhino, RSV
2	1	PIV1	Negative	PIV1
3	1	RSVA	RSV, Boca	RSV
4	1	RSVB	RSV, Boca	RSVB
5	1	Influenza B	Boca	Influenza B
6	1	Corona	Corona NL63, Boca	Corona NL63
7	1	Rhino	Entero/Rhino, Boca	Rhino
8	1	RSVB	RSV, Boca	RSVB
9	1	RSVA	PIV4	RSVA
10	1	PIV2	Negative	PIV2
11	1	PIV4	Influenza B, Entero/Rhino	Influenza B, Rhino
12	1	PIV2	PIV2	PIV2, Rhino
13	1	RSVB	RSV, Entero/Rhino	RSVB, Rhino
14	1	Entero	Entero/Rhino	Entero, Corona OC43
15	1	RSVB	RSV, Boca	RSVB

One of 20 influenza A and 4 of 11 influenza B positive samples, respectively had discordances where influenza was not detected by at least one of the assays. For RSV A and B, 3 of 31 and 2 of 10 samples, respectively had discordances where virus was not detected by one of the assays.

A total of 78 samples were classified as negative for viruses (at least two assays having a negative result). Table 3 outlines sixteen (21%) of the negative samples displayed discordant results with 11, 4 and 1 samples having single, double and triple discordant, respectively.

Table 3

Table of Discordant Results Amongst Negative Specimens.

Specimen #	Number of Discordances	RV15 Result	xTAG Result	RV16 Result
1	1	Negative	Entero/Rhino	Negative
2	1	Negative	RSV	Negative
3	1	Negative	Negative	Boca
4	1	Negative	Negative	PIV4
5	1	Negative	RSV	Negative
6	1	Corona	Negative	Negative
7	1	Entero	Negative	Negative
8	1	PIV2	Negative	Negative
9	1	RSVB	Negative	Negative
10	1	RSVB	Negative	Negative
11	1	Negative	Entero/Rhino	Negative
12	2	PIV3, RSVB	Negative	Negative
13	2	Negative	FluB, Boca	Negative
14	2	Negative	Influenza B, Boca	Negative
15	2	RSVA, Corona	RSV	FluB
16	3	PIV4	FluB, Entero/Rhino	FluB, Rhino

Twenty of twenty-seven samples that were inconclusive for a virus by RV15 testing were found to be positive when tested by the other multiplex assays. Weak signals were found by RV15 for 48 virus-types among these twenty-seven samples. A breakdown for the inconclusive results is shown in Table 4 with their respective xTAG and RV16 assay results.

Table 4

Inconclusive Results by Virus Type for RV15, xTAG and RV16.

Respiratory Virus Type	# of RV15 Inconclusive Specimens	XTAG Results		RV16 Results
		Positive	Negative	Positive	Negative
Respiratory Syncytial Virus (RSV)	20	7	13	6	14
Influenza A	1	0	1	1	0
Influenza B	5	4	1	5	0
Adenovirus (AdV)	0	0	0	0	0
Metapneumovirus (MPV)	6	3	3	3	3
Coronavirus (Corona)	1	0	1	0	1
Parainfluenza (PIV)	1	0	1	0	1
Enterovirus (Entero)	0	0	0	0	0
Rhinovirus (Rhino)	5	5	0	4	1
Bocavirus (Boca)	8	0	8	1	7
Total Viruses Detected	48	19	29	20	28

Assay performance

Overall sensitivity and specificity of the three assays were, 84% and 99% for the RV15, 87% and 100% for RV16 and 84% and 100% for xTAG. Multiple virus types were more commonly detected by xTAG compared to the other two assays. Bocavirus signals were more common with the xTAG system than either of the RV panels (11 vs. 5 and 6). One RSV A positive sample that was tested by the xTAG system displayed signals for RSV, influenza B, rhino/enterovirus, metapneumovirus and bocavirus. Table 5 includes 2 × 2 representations of RSV and Influenza A and B tested by each assay, with sensitivity and specificity calculations.

Table 5

Assay Sensitivity and Specificity for RV15, xTAG and RV16 for RSV and Influenza.

RV15 RSV	Test +	Test −
Condition +	38	0	Sens: 100
Condition −	8	201	Spec: 96

Discussion

The performance of all three assays in this evaluation was equivalent. Small numbers of some virus types were expected to skew some of the sensitivity calculations. For example, the sensitivity of xTAG for influenza B was reported as 75% as the assay only detected 7 out of 11 positive samples. All three assays had overall sensitivities that were similar (∼85% and ∼100%, respectively). Sensitivity of all three platforms was generally less than those reported by Kim et al. while specificities were identical (Kim et al., 2013).

Co-infections rates (13%) in the present study were similar to those described by Kim et al. (Kim et al., 2013). This is interesting, given the geographic and temporal differences between the two studies.

Our study design was largely retrospective with the majority of samples being RV15-test-positive. Both of these factors may have biased our results in favor of the RV15. As the RV15 is an earlier generation assay, one may have expected it to perform less effectively than its comparators in this study. In addition to this, the effect of a freeze-thaw cycle on samples done on the RV16 and xTag for virus detection and accuracy was not analyzed and is a limitation of this study. Sample input among the three assays also varied, with an input of 400 μL for xTAG and 700 μL for RV15 and RV16 respectively. There were no issues identified between the assays using the MS2 internal controls, and based on the results obtained from this study, alterations in sample input do not affect sensitivity of the assays. However, a lower sample input may be beneficial if patient sample volumes are limited. In this respect the xTAG assay may be more favorable.

The discordancy rates amongst positive samples were high (47%). This may reflect preferential chemistry or primer sets to virus-type matching in one of the assays versus another. Upon closer inspection of the data, one can see variation between the different assays with respect to certain virus type sensitivities. A limitation of this evaluation to further investigate these differences is that sequencing and monoplex PCR testing was not conducted on discordant samples. In addition, conventional methods such as cell culture or direct fluorescence antibody testing were not performed.

Sensitivity for the detection of influenza A by all three systems was generally good and was equal between the xTAG and RV16 but less for the RV15. The poor performance of the xTAG for influenza B has already been addressed. Discordant results amongst influenza A and RSV were low.

The sensitivity of the RV15 for rhinovirus and enterovirus was poorer than that of the other two assays. This finding is similar to what Kim et al. reported previously (Kim et al., 2013).

It was assumed that all samples were from individuals who had clinical symptoms. However, chart reviews were not done to confirm this was the case. Given the high sensitivity of all of these multiplex assays, it is probable that asymptomatic or convalescent individuals may shed virus genomes for considerable periods of time. This would be especially true for adenovirus infected patients and may account for the levels of co-infections seen.

Any of the three aforementioned assays would be acceptable for use in our laboratory for the detection of respiratory viruses. Ultimately, the ease of use, time to completion and cost of consumable materials will dictate which test methodology is preferable. Following this study, the RV16 replaced the RV15 in house, as the RV16 did not have any inconclusive results. There are however also many other commercial multiplex assays on the market. Some examples include FTD Respiratory (Fast-track diagnostics), EP Respiratory (Ausdiagnostics), FilmArray Respiratory Panel (Biofire Diagnostics/Biomerieux), Verigene® RV test (Nanosphere) and Prodesse Respiratory assay (Hologic/Gen-Probe) that have in-vitro diagnostic (IVD) indications by various licensing bodies (i.e. FDA, Health Canada, CE). Choosing an assay may be influenced by which jurisdiction it has approval for IVD. In conclusion, the RV16 was implemented in the laboratory because the inconclusive results previously obtained by the RV15 were resolved with the RV16 next generation platform. The RV16 platform also improved workflow compared to the RV15 gel-based detection to RT-PCR, which was deemed easier to interpret.

Conflicts of interest

None.

Funding

None.

Competing interests

None.

Ethical approval

Local research ethics board advised this study did not fall within the scope of an REB review.