Comparison of the AdvanSure™ real-time RT-PCR and Seeplex(®) RV12 ACE assay for the detection of respiratory viruses.

The AdvanSure™ RV real-time PCR kit (AdvanSure; LG Life Sciences, Korea) is based on multiplex real-time PCR and can simultaneously detect 14 respiratory viruses. We compared the performance of the AdvanSure assay with the Seeplex RV 12 ACE detection kit (Seeplex; Seegene, Seoul, South Korea), a multiplex end-point PCR assay. A total of 454 consecutive respiratory specimens were tested with both AdvanSure and Seeplex assays; AdvanSure detected 153 (33.7%) positive cases and Seeplex detected 145 (31.9%) positive cases. The positive percent agreement, negative percent agreement, and kappa value for the two assays were 87.2% (95% CI, 80.3-92.1), 91.1% (95% CI, 87.2-93.9), and 0.77 (95% CI, 0.70-0.83), respectively. Compared with the Seeplex assay, the AdvanSure assay had a shorter turnaround time (3h vs. 8h) and a shorter hands-on time (<1h vs 2h). In conclusion, the AdvanSure assay demonstrated comparable performance to the Seeplex assay.

Introduction

Respiratory viral infections are common causes of substantial morbidity and mortality in both pediatric and adult populations. Timely and accurate diagnosis is crucial for patients of all ages, especially those with underlying cardiopulmonary disease or compromised immune systems (Glezen et al., 2000, Weinberg et al., 2004). Early detection of respiratory virus infections allows clinicians to initiate immediate therapeutic interventions that can reduce complications, antibiotic use, and unnecessary laboratory testing (Adcock et al., 1997, Jernigan et al., 2011, Mahony et al., 2009, Renaud et al., 2012).

Traditionally, non-molecular approaches such as direct immunofluorescence and culture-based methods using cell culture and antigen detection are used to identify respiratory viral pathogens (Glezen et al., 2000, Weinberg et al., 2004). While these conventional assays are still considered to be the reference methods for respiratory virus detection, they are labor-intensive, time-consuming, and inferior to nucleic acid amplification tests (in both sensitivity and specificity) (Falsey et al., 2006, Vallieres and Renaud, 2013, Zhang et al., 2012). By screening for a variety of pathogens at the same time, nucleic acid-based multiplex assays are able to simultaneously detect a panel of viruses and shorten the hands-on time (HOT), turnaround time (TAT), and length of hospital stay compared to those of traditional methods (Boivin et al., 2004, Cho et al., 2013, Gharabaghi et al., 2011, Kim et al., 2013, Renaud et al., 2012).

The Seeplex® RV12 ACE detection kit (Seeplex; Seegene, South Korea) allows for concurrent detection of 12 respiratory viruses (RVs) in two reactions per sample using Dual Priming Oligonucleotides as PCR primers (Yoo et al., 2007). Previous studies evaluated the performance of the Seeplex assay and determined that it can save time and resources compared to conventional methods (Bibby et al., 2011, Drews et al., 2008, Kim et al., 2009, Weinberg et al., 2004, Yoo et al., 2007). Recently, the AdvanSure™ RV real-time PCR kit (AdvanSure; LG Life Sciences, Korea) has been introduced. This kit is comprised of five reaction tubes (Rheem et al., 2012) and can detect 14 different RVs: human coronavirus (CoV) 229E, CoV NL63, CoV OC43, parainfluenza virus (PIV) 1, PIV 2, PIV 3, influenza viruses (INF) type A and B, human respiratory syncytial viruses (RSV) type A and B, human rhinovirus (HRV), human metapneumovirus (HMPV), human adenovirus (ADV) and human bocavirus (BoV). This profile is similar to that of the Seeplex assay but also includes BoV and can distinguish between CoV 229E and CoV NL63.

The aim of this study was to evaluate the analytical performance and clinical applicability of the AdvanSure assay and compare the results with those of a conventional multiplex PCR method, the Seeplex assay. These tests were carried out in a routine diagnostic laboratory setting and used nonselective clinical specimens from Korean patients.

Materials and methods

Clinical specimens

This prospective study was approved by the Institutional Review Board of Samsung Medical Center. A total of 454 nonselective consecutive clinical respiratory specimens were obtained. These samples were submitted for the Seeplex assay from July 2013 to September 2014 and included 421 nasopharyngeal swabs or aspirates, 29 samples of bronchoalveolar lavage fluid, and 4 sputum samples. These samples were simultaneously analyzed by the AdvanSure assay.

Seeplex® RV 12 ACE kit

Nucleic acids were extracted from 100 μl specimens by MagNa Pure LC Total Nucleic Acid Isolation Kit (Roche, Mannheim, Germany), and the final elution volume of each sample was 50 μl. Random hexamer-primed cDNA synthesis products were generated using the Revertaid First Strand cDNA synthesis kit (Fermentas, Ontario, Canada) according to the manufacturer's instructions. Each cDNA preparation was subjected to the Seeplex PCR procedure (Seegene, Seoul, South Korea) according to the manufacturer's instructions. Parallel 20 μl reactions were set up, with Seeplex Master Mix containing the internal control, 8-methoxypsoralen as a contamination control reagent, and 3 μl cDNA. One reaction from each pair was supplemented with 4 ml of primer mix A, and the other with 4 ml of primer mix B. Thermal cycling conditions were as follows: 15 min at 95 °C, followed by 30 cycles of 95 °C for 30 s, 60 °C for 90 s, and 72 °C for 90 s, ending with a single incubation at 72 °C for 10 min. Amplification products were detected using capillary electrophoresis technology (Lab901 Screen Tape System; Lab901 Ltd, Loanhead, UK).

AdvanSure RV real-time PCR

As recommended by the manufacturer of AdvanSure, nucleic acids were extracted from 200 μl of clinical samples using the TANBead® Smart LabAssist-32 extraction system (Taiwan Advanced Nanotech Inc., Taoyuan City, Taiwan) and the final elution volume was 100 μl for each sample. Extracted nucleic acids were then amplified and probed for RVs with the AdvanSure kit according to the manufacturer's instructions. Briefly, 5 μl of extracted DNA was added to a tube containing 5 μl of primer probe mix and 10 μl of one-step RT-PCR premix. For the reverse transcription step, this mixture was incubated at 50 °C for 10 min. Denaturation followed at 95 °C for 30 s, then 10 cycles of PCR (15 s at 95 °C, 30 s at 53 °C, and 30 s at 60 °C). Subsequently, 30 additional cycles of PCR were completed for the detection of fluorescence signals (15 s at 95 °C, 30 s at 53 °C, 30 s at 60 °C). The AdvanSure assay was designed to simultaneously perform the reverse-transcription reaction with the extracted RNA along with the multiplex PCR reaction and using an endogenous internal control, RNase P (RNP).

Analytical accuracy of the Seeplex and AdvanSure assays

Concordant results between the Seeplex and AdvanSure assays were regarded as true positive without further verification. In cases where multiple viruses were detected, concordance was determined if at least one of the viruses detected was the same in both assays when the agreement rate was analyzed. The specimens with discrepant results between the Seeplex and AdvanSure assays were further assessed by monoplex PCR and sequencing in a blind manner. The discrepant results that were negative from monoplex PCR and the sequencing method were considered false positives. Because the Seeplex assay did not detect BoV and the AdvanSure assay did not detect CoV HKU, we excluded these results from the analytical accuracy analyses.

Analytical sensitivity and specificity of the AdvanSure assay

Serially diluted plasmids containing the target gene were used for determination of analytical sensitivity. pGEM®-t easy vector (Promega, Madison, WI, USA) was used for plasmid DNA preparation. Serial dilutions of the prepared plasmid DNA were made from 107 to 100 copies per reaction to determine the analytical sensitivity of the assay. The plasmid DNAs of RSV A (Korea Bank for Pathogenic Viruses, KBPV-VR-41), PIV 1 (KBPV-VR-44), HRV (KBPV-VR-39), CoV 229E (KBPV-VR-9), CoV OC43 (KBPV-VR-8), and BoV (clinical isolate) were used. Three replicates of each dilution step were performed. The lower detection limit was defined as the lowest concentration detected by the assay in a series of three replicates.

The cross-reactivity of the AdvanSure assay was evaluated using nine different types of bacteria. Streptococcus pneumonia (ATCC 49619), Streptococcus pyogenes (ATCC 19615), Staphylococcus epidermidis (ATCC 12228), Moraxella catarrhalis (clinical isolate), Pseudomonas aeruginosa (ATCC 27853), Escherichia coli (ATCC 25922), Klebsiella pneumonia (ATCC 700603), Staphylococcus aureus (ATCC 25923), and Haemophilus influenza (ATCC 9007) were obtained from the American Type Culture Collection (ATCC, Manassas, VA, USA). The RNA or DNA of supplied samples was extracted and assayed with the AdvanSure assay abiding by the same procedures applied for sample processing.

Workflow analysis

The total TAT and HOT between the AdvanSure and Seeplex assays were evaluated and compared. TAT was defined as the time interval between sample reception in laboratory to the final result. HOT was defined as the time spent by a technician for sample preparation.

Statistical analysis

Statistical analysis was performed using SPSS software, version 21.0 (SPSS Inc., Chicago, IL, USA) and the VassarStats website (http://vassarstats.net/). Inter-rater agreement statistics (the Kappa values) were used to compare the detection of respiratory viruses between the Seeplex and AdvanSure assays. P-values less than 0.05 were considered statistically significant.

Results

Distribution of respiratory viruses

A total of 454 respiratory specimens were tested by the Seeplex and AdvanSure assays. Of these, 23 samples (5.1%) had more than one virus. Of the 23 co-infected samples, both the Seeplex and the AdvanSure assay identified 16 samples (69.6%) as having a multiple agent infection. Twenty-one specimens had two viruses, and two specimens had three viruses as determined by either the AdvanSure or Seeplex assay. HRV was most frequently associated with coinfection, followed by ADV and RSV B. A total of 153 (33.7%) and 145 (31.9%) samples were AdvanSure- and Seeplex-positive, respectively, among which 123 (27.1%) samples were positive in both assays.

Comparison of the AdvanSure assay with the Seeplex Assay

As shown in Table 1 , the positive percent agreement between the Seeplex and AdvanSure assays was 87.2% (95% CI, 80.3 to 92.1), and the negative percent agreement was 91.1% (95% CI, 87.2 to 93.9). The kappa value for the two methods was 0.77 (95% CI, 0.70 to 0.83). The lowest positive percent agreement between the two assays was observed for PIV 2 (50.0%), HRV (60.3%), and CoV 229E/NL63 (60.0%).

Table 1

Detection of respiratory viruses by AdvanSure™ RV real-time RT-PCR and Seeplex® RV12 ACE assays.

Virus	Agreement				Kappa value
	Positive		Negative		Observed kappa	95% CI
	%	95% CI	%	95% CI
Total	87.2	80.3–92.1	91.1	87.2–93.9	0.77	0.70–0.83
ADV	100	71.7–100	91.1	97.2–99.6	0.83	0.69–0.98
INF A	100	59.8–100	98.9	98.6–99.9	0.94	0.82–1.00
INF B	100	56.1–100	99.8	98.6–99.9	0.93	0.80–1.00
PIV 1	100	19.8–100	99.8	99.0–100	1.00	1.00
PIV 2	50.0	2.7–97.3	100	98.2–99.9	0.40	0.00–0.94
PIV 3	100	5.4–100	99.8	98.6–99.9	0.67	0.05–1.00
HRV	60.3	47.2–72.2	95.9	93.3–97.6	0.60	0.49–0.71
RSV A	100	19.8–100	99.6	98.2–99.9	0.66	0.23–1.00
RSV B	100	91.9–100	98.5	96.6–99.4	0.94	0.89–0.99
HMPV	100	19.8–100	100	99.0–100	1.00	1.00
CoV 229E/NL63	60.0	17.0–92.7	99.8	98.6–99.9	0.66	0.31–1.00

ADV, adenovirus; INF A, influenza A virus; INF B, influenza B virus; PIV, parainfluenza virus; HRV, human rhinovirus; RSV A, respiratory syncytial virus; HMPV, human metapneumovirus; CoV, coronavirus.

Thirty-four specimens identified as positive by the AdvanSure assay were negative by the Seeplex assay: One sample had two discrepant viruses, so there were a total of 35 discrepancies observed. Meanwhile, 28 results in 28 specimens identified as negative in the AdvanSure assay were positive by the Seeplex assay. The results of monoplex PCR and sequencing revealed that 68.6% (24/35) of the results that were identified as positive by the AdvanSure assay but not the Seeplex assay were true positives. Of note, the results for ADV, INF A, INF B, PIV 2, and PIV 3 were all confirmed as true positives by the monoplex PCR and sequencing (Table 2 ). For these 28 AdvanSure-negative and Seeplex-positive results, the reference methods indicated that 60.7% (17/28) were true positives.

Table 2

Analysis of samples with discrepant results in AdvanSure™ RV real-time RT-PCR and Seeplex® RV12 ACE assays.

Virus	Discrepant results		No. of positive results in monoplex PCR and sequencing
	AdvanSure™ RV real-time RT-PCR/Seeplex® RV12 ACE detection result	No. of samples
ADV	+/−	5	5
INF A	+/−	1	1
INF B	+/−	1	1
PIV 2	+/−	2	2
	−/+	1	0
PIV 3	+/−	1	1
HRV	+/−	16	10
	−/+	25	17
RSV A	+/−	2	1
RSV B	+/−	6	3
CoV 229E/NL63	+/−	1	0
	−/+	2	0

ADV, adenovirus; INF, influenza virus; PIV, parainfluenza virus; HRV, human rhinovirus; RSV, respiratory syncytial virus; HMPV, human metapneumovirus; CoV, coronavirus.

The sensitivity and specificity were calculated for each target and assay. The overall sensitivity of the AdvanSure assay [91.6% (95% CI, 85.8 to 95.3)] was comparable to that of the Seeplex assay [87.1% (95% CI, 80.5 to 91.8)]. The overall specificities of the AdvanSure and Seeplex assays were 97.3% (95% CI, 94.6 to 98.8) and 98.0% (95% CI, 95.5 to 99.2), respectively. Analytical sensitivity, specificity, positive predictive value, and negative predictive value of the AdvanSure and Seeplex assays for each virus are presented in Table 3 .

Table 3

Percent sensitivity, specificity, PPV and NPV of two assays for detection of respiratory viruses.

Virus	AdvanSure™ RV real-time RT-PCR				Seeplex® RV12 ACE
	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
Total	142/15591.6 (85.8–95.3)	291/29997.3 (94.6–98.8)	142/15094.7 (89.4 –97.5)	291/30495.7 (92.6 -97.6)	135/15587.1 (80.5–91.8)	293/29998.0 (95.5–99.2)	135/14195.7 (90.6–98.3)	293/31393.6 (90.1 –96.0)
ADV	18/18100 (78.1–100)	436/436100 (98.9–100)	18/18100 (78. 1 –100)	436/436100 (98.9–100)	13/1872.2 (46.4–89.3)	436/436100 (98.9–100)	13/13100 (71.7–100)	436/44198.9 (97.2–99.6)
INF A	9/9100 (62.9–100)	445/445100 (98.9–100)	9/9100 (62.9–100)	445/445100 (98.9–100)	8/988.9 (50.7–99.4)	445/445100 (98.9–100)	8/8100 (59.8–100)	445/44699.8 (98.6–99.9)
INF B	8/8100 (59.8–100)	446/446100 (98.9–100)	8/8100 (59.8–100)	446/446100 (98.9–100)	7/887.5 (46.7–99.3)	446/446100 (98.9–100)	7/7100 (56.1–100)	446/44799.8 (98.6–99.9)
PIV 1	2/2NE	452/452100 (98.9–100)	2/2NE	452/452100 (98.9–100)	2/2NE	451/451100 (98.9–100)	2/2NE	451/451100 (98.9–100)
PIV 2	3/3NE	451/451100 (98.9–100)	3/3NE	451/451100 (98.9–100)	1/3NE	450/45199.8 (98.6–99.9)	1/2NE	450/45299.6 (98.2–99.9)
PIV 3	2/2NE	452/452100 (98.9–100)	2/2NE	452/452100 (98.9–100)	1/2NE	452/452100 (98.9–100)	1/1NE	452/45399.8 (98.6–99.9)
HRV	48/6573.8 (61.2–83.6)	383/38998.5 (96.5–99.4)	48/5488.9 (76.7–95.4)	383/40095.8 (93.1–97.4)	55/6584.6 (73.1–92.0)	381/38997.9 (95.8–99.0)	55/6387.3 (76.0–94.0)	381/39197.4 (95.2–98.7)
RSV A	3/3NE	450/45199.8 (98.6–99.9)	3/4NE	450/450100 (98.9–100)	2/3NE	451/451100 (98.9–100)	2/2NE	451/45299.8 (98.6–99.9)
RSV B	58/58100 (92.3–100)	393/39699.2 (97.6–99.8)	58/6195.1 (85.4–98.7)	393/393100 (98.8–100)	55/5894.8 (84.7–98.7)	395/395100 (98.8–100)	55/55100 (91.9–100)	395/39899.2 (97.6–99.8)
HMPV	2/2NE	452/452100 (98.9–100)	2/2NE	452/452100 (98.9–100)	2/2NE	452/452100 (98.9–100)	2/2NE	452/452100 (98.9–100)
CoV229E/NL63	3/3NE	450/45199.8 (98.6–99.9)	3/4NE	450/450100 (98.9–100)	3/3NE	449/45199.6(98.2–99.9)	3/5NE	449/449100 (98.9–100)

PPV, predictive positive value; NPV, negative predictive value; ADV, adenovirus; INF, influenza virus; PIV, parainfluenza virus; HRV, human rhinovirus; RSV, respiratory syncytial virus; HMPV, human metapneumovirus; CoV, coronavirus; NE, not estimated (number of positive results was less than five).

The detection limits of the AdvanSure assay for 6 respiratory viruses were 2.92, 29.2, 27.9, 29.5, 2.95, and 29.1 copies/μl for RSV A, PIV 1, HRV, CoV 229E, CoV OC43, and BoV, respectively.

To assess the cross-reactivity and detection specificity, nine different bacterial reference strains were tested using the same assay procedure applied for the clinical samples for RV detection with the AdvanSure assay. The assay results were all negative, and non-specific positive reactions were not observed.

Workload analysis between the AdvanSure and Seeplex assays

The total TATs and HOTs between the AdvanSure and Seeplex assays for 16 samples were evaluated and compared. The total TAT (3 h) and HOT (<1 h) required to perform the AdvanSure assay were both shorter when compared with those of the Seeplex assay (8 h and 2 h, respectively). The approximate times needed for each step are shown in Fig. 1 .

Fig. 1

Flowchart showing the turnaround time and hands-on time of Seeplex® RV12 ACE and AdvanSure™ RV assays.

Discussion

This study compared the performance of two multiplex PCR kits, the AdvanSure and Seeplex assays, for respiratory virus detection using clinical respiratory samples. Along with the comparison, monoplex PCR and sequencing were performed in a blind manner as reference methods. The AdvanSure assay demonstrated equivalent results to the Seeplex assay in detecting all virus targets, and showed a slightly higher sensitivity (91.6% vs. 81.7%) but the difference was not statistically significant. The AdvanSure assay had high PPVs and NPVs with the samples overall.

A previous study by Rheem et al. (2012) reported that the AdvanSure assay had comparable sensitivity to the Seeplex assay, and our study demonstrated similar findings. We analyzed a larger amount of clinical samples among which 10.1% (46/454) of samples revealed discrepant results, whereas Rheem et al. reported only 1.9% (6/320) of samples with discrepancies. This difference might be partly due to the differences in the DNA extraction system, and the different study periods used for the studies. We used the TANBead® Smart LabAssist-32 DNA extraction system for the AdvanSure assay, which is recommended by its manufacturer. On the other hand, Rheem et al. used the QIA-cube platform (Qiagen, Hilden, Germany).

The lowest positive percent agreement between the two assays was observed for PIV 2 (50.0%), HRV (60.3%), and CoV 229E/NL63 (60.0%) as opposed to all other viruses that showed 100% agreement. There were 3, 65, and 3 true positive samples for PIV 2, HRV, and CoV 229E/NL63, respectively. For PIV 2 and CoV 229E/NL63, the lowest agreement was likely due to the low number of target virus-positive specimens. For HRV, the sensitivity and the negative predictive value of the AdvanSure assay (73.8% and 95.8%) were lower than those of the Seeplex assay (84.6% and 97.4%), while vice versa for the specificity and the positive predictive value: 98.5% and 88.9% for the AdvanSure assay vs. 97.9% and 87.3% for the Seeplex assay, albeit without statistical significance.

Cho et al. (2014) reported that the AdvanSure assay had exquisite performance for RV detection when compared with the RV15 assay. For the AdvanSure assay, they used the TANBead® Smart LabAssist-32 DNA extraction system as in this study, and reported a sensitivity of 95.1%, which was higher than the result of this study (91.6%).

The Seeplex assay system had a significant drawback regarding its internal control facility. The artificial targets included in each PCR mastermix can only validate the PCR step, not the RNA extraction or reverse transcription step (Bibby et al., 2011). On the contrary, the AdvanSure assay targets the human RNP gene as an internal control and quantifies it in each sample, which enables monitoring of the RNA extraction and the reverse transcription steps along with the specimen quality (Cho et al., 2013, Dreier et al., 2005). Monitoring the specimen quality by using an endogenous internal control is important because it could prevent misjudgment from sampling error. In other words, with the results presented as negative in the Seeplex assay, it is impossible to distinguish negative from invalid as opposed to the AdvanSure assay.

Previously, Kim et al. reported that the AdvanSure assay provides a faster turnaround time because it avoids additional nested amplification or hybridization steps needed for identification of viral products (Kim et al., 2013). The AdvanSure assay is a one-step real-time PCR assay in which the reverse transcription and multiplex PCR steps are both conducted in the same reaction well. The advantages of the one-step assay over the two-step assay include improved workflow, reduction in assay preparation time, and elimination of cross contamination from the transfer of cDNA from the reverse transcription reaction into the PCR reaction. These factors can play a role in improving the workflow speed for the AdvanSure assay. In our study, TAT and HOT were calculated on the condition that we perform both AdvanSure and Seeplex assays with 16 clinical samples (80 reactions for AdvanSure assay) in one batch. The AdvanSure assay, which is comprised of five reaction tubes, has definitive strengths in workload when compared with the Seeplex assay comprised of two reaction tubes. However, by reducing the number of reaction tubes to four in the AdvanSure assay, it would be possible to lessen the workload as well as facilitate medium to high throughput workloads more effectively.

One of the limitations of this study is that a shortage of specific virus-positive specimens prevented robust comparisons of some respiratory viruses. Another potential shortcoming is that neither the AdvanSure nor the Seeplex assay was compared with viral cultures or direct fluorescent-antibody assays. This fact could be offset, however, because nucleic acid amplification tests are known to have greater sensitivity than conventional methods including viral culture and antigenic detection (Mahony et al., 2007, Weinberg et al., 2004). To minimize the limitation of this particular study design, monoplex PCR and sequencing were performed under code. Lastly, the analytical sensitivity was not evaluated for the Seeplex assay herein. Since the main purpose was to evaluate the analytical performance of the AdvanSure assay, a complete battery of tests was performed to characterize the AdvanSure assay.

In conclusion, the AdvanSure assay produces results comparable to the Seeplex assay for RV detection, exhibiting a tendency to have better sensitivity. Further analysis of samples that yielded discrepant results demonstrated that the AdvanSure assay exhibited a higher concordance with the results of monoplex PCR and sequencing when compared with the Seeplex assay.