Performance of a novel microarray multiplex PCR for the detection of 23 respiratory pathogens (SYMP-ARI study).

Symptoms of acute febrile respiratory tract infection are often unspecific, but the rapid identification of pathogens allows optimised patient management. The objective of this study was to evaluate a novel multiplex polymerase chain reaction (PCR) suspension microarray which detects 19 viral and four atypical bacterial targets. A comprehensive set of sensitive monoplex real-time PCR assays was used for each pathogen as the gold standard. A panel of archived as well as 300 prospectively collected clinical samples was analysed by both methods. At least one target was detected in 165/300 (55 %) samples by monoplex PCR and in 140/300 (46 %) samples by multiplex PCR, respectively. The positivity rate was significantly higher in paediatric patients compared to adults [126/154 (82 %) vs. 39/146 (27 %) by monoplex and 114/154 (74 %) vs. 26/146 (18 %) by multiplex PCR, respectively]. Among all samples, 17/300 (5.6 %) were positive for atypical bacteria by monoplex and 8/300 (2.6 %) by multiplex PCR, respectively. Multiple detections were recorded in 35/300 (11.6 %) samples by monoplex and 26/300 (8.7 %) by multiplex PCR. For the most common pathogens, the sensitivity ranged from 57 to 93 % and the specificity ranged from 95 to 100 %. The overall concordance between both methods was 77 % [95 % confidence interval (CI) 72-81 %]. False-negative results by multiplex PCR were mainly due to the low target concentration. Compared to monoplex PCR, the novel microarray assay proved its principle but displayed overall lower sensitivities, potentially restricting its use to paediatric patients. For some targets, only small numbers of positive samples were available, requiring larger studies to firmly assess the sensitivity and specificity.

Introduction

The introduction of nucleic acid amplification tests (NAT) has significantly improved the diagnosis of acute respiratory tract infection (ARI) [13, 20]. The time-to-result has decreased considerably, allowing timely patient management and infection control. In addition, the availability of NAT can reduce costs in selected situations and possibly reduce antibiotic usage in the case of viral infection [21]. Real-time polymerase chain reaction (PCR) in particular demonstrated its usefulness and reliability not only during the 2009 influenza pandemic [14, 26]. However, the increasing number of respiratory pathogens to diagnose puts a tremendous challenge on NAT methodology. In recent years, new molecular technologies have fostered the shift from single-pathogen detection to a multiplex approach to simultaneously detect a range of respiratory agents [24]. Indeed, it has been shown that real-time PCR can be performed in a multiplex fashion, but the multiplexing of real-time PCR assays can have a detrimental influence on sensitivity due to complex primer/probe interactions. As an alternative, suspension microarray assays on Luminex platforms allow multiple target detection in a single reaction tube [1, 25]. Importantly, most currently available assays include either viral or bacterial targets only, rendering a combination hereof an attractive option for laboratories. Since viral and bacterial infections are often indistinguishable on clinical grounds, definite aetiological diagnosis retains the utmost importance for optimal patient care. In addition, complex interactions between viral and bacterial co-infections have been demonstrated, but there is little information on the clinical relevance of this finding [4]. To streamline diagnostics and to elucidate the possible relevance of viral/bacterial co-infections, including bacterial targets is, therefore, highly desirable. In particular, atypical bacteria represent reasonable targets, since they are refractory to standard culture techniques and are non-colonising bacteria. Within a network of academic as well as industry partners (Qiagen), we have, therefore, developed a novel multiplex PCR using suspension microarray technology for the combined detection of viral as well as atypical bacterial targets (SYMP-ARI study). Previous studies have shown that multiplex suspension microarrays outperform traditional techniques, e.g. direct immunofluorescene or virus culture [1, 20]. However, in most studies, a combination of different diagnostic methods has been used as the reference and not all were of prospective nature. The aim of this study was to evaluate a new microarray assay for 23 pathogens in comparison to individual real-time PCR. The real-time PCR assays were optimised to reach ultimate sensitivity and reduce hands-on time. In a first step, both methods have been validated on a panel of known positive archived samples. Hereafter, both assays were deployed on prospectively collected samples including children and adults to assess assay performance.

Materials and methods

Nucleic acids extraction

Nucleic acids were extracted using the QIAamp MinElute Virus Spin Kit (Qiagen, Hilden, Germany) on a QIACube robot (Qiagen), according to the manufacturer’s instructions. A quantity of 200 μl of specimen spiked with 30 μl MS2 phage lysate (105 PFU per ml) was extracted into an elution volume of 100 μl and analysed by monoplex PCR. Effectively, 90 copies/PCR reaction of the MS2 phage were used. In a similar fashion, 200 μl of specimen spiked with the internal control of the novel multiplex PCR was extracted into an elution volume of 100 μl and analysed by multiplex PCR.

Multiplex PCR

Extracts were tested by the novel multiplex assay (Qiagen, Hamburg, Germany), according to the manufacturer’s instructions. The assay comprised the amplification of pathogen DNA with labelled primers, followed by hybridisation of the PCR products to target-specific capture probes, which were covalently linked to a specific set of colour-coded beads. Reverse transcription PCR (RT-PCR) was performed using the OneStep RT-PCR Kit (Qiagen, Hilden, Germany). In brief, 10 μl of extracted nucleic acids were used in a total reaction volume of 50 μl, including 10 μl 5× Qiagen OneStep RT-PCR buffer, 6 μl of target-specific multiplex SYMP-ARI primer mix, 2 μl of dNTP mix (Qiagen), 0.75 μl amplification enhancer (Qiagen) and 2 μl Qiagen OneStep RT-PCR enzyme mix. Cycling was done on a GeneAmp 9700 cycler (Applied Biosystems, Weiterstadt, Germany) with the following protocol: 50 °C for 35 min, followed by 95 °C for 15 min, 15 cycles of enrichment cycling (94 °C for 30 s, 52 °C for 90 s, 72 °C for 60 s), 10 cycles of two-step cycling (94 °C for 15 s and 70 °C for 90 s) and three-step cycling (94 °C for 25 s, 52 °C for 15 s, 72 °C for 15 s). A final extension step was done at 72 °C for 3 min. After the cycling protocol, 5 μl of RT-PCR product, 10 μL of SYMP-ARI bead mix and 35 μL of detection buffer were incubated at 52 °C for 10 min. Subsequently, 7.5 μl of streptavidin–phycoerythrin conjugate mixed with 7.5 μl detection buffer was added to each reaction well. Hereafter, the reaction was incubated at 52 °C for 5 min. To terminate the reaction, 180 μL of stopping buffer was added to each well. Detection was performed in a 96-well plate format on the Luminex 100 machine (Luminex Molecular Diagnostics Inc., Toronto, Canada) using Luminex 100 IS software (Luminex). The median fluorescent intensity (MFI) was determined and MFI values above a threshold level of 150 were regarded as positive. The multiplex assay simultaneously detects 19 viral and four bacterial pathogens: adenovirus, human bocavirus (hBoV), human coronaviruses (CoV) 229E, NL63, OC43 and HKU1, enterovirus, human metapneumovirus (HMPV), universal influenza virus A, influenza virus A/H1N1, influenza virus A/H3N2, influenza virus B, parainfluenza virus 1–4, rhinovirus, respiratory syncytial virus A (RSV-A), respiratory syncytial virus B (RSV-B), Bordetella pertussis, Chlamydia pneumoniae, Legionella pneumophila and Mycoplasma pneumoniae. Results of the monoplex and multiplex PCR were analysed independently in a blinded manner by two individuals.

Monoplex PCR

All monoplex real-time PCR assays were adapted from published methods [5, 6, 8, 15–17, 23, 26, 27, 29, 32, 33]. To streamline diagnostics, all assays including those for DNA targets were carried out in a 25-μl reaction volume using the Qiagen OneStep RT-PCR Kit (Qiagen). For each assay, a primer concentration of 1 μM for each primer and a probe concentration of 0.5 μM were used. Reverse transcription and amplification was performed using the Roche LightCycler 480 II System (Roche, Mannheim, Germany) with the following cycling profile: 30 min at 50 °C, 15 min at 95 °C, 45 cycles at 95 °C for 15 s and 60 °C for 30 s. The RNA bacteriophage MS2 was used as an internal extraction/inhibition control at a concentration of 90 copies/PCR reaction [7]. All assays were optimised with respect to the maximum sensitivity and specificity. Positive (in vitro RNA transcripts or DNA plasmids of the respective target) and negative controls were used throughout for each individual assay. Ct-values <45 were regarded as positive.

Archived samples

A panel of 50 archived respiratory tract samples (nasopharyngeal aspirates and pharyngeal swabs) submitted to the Department of Virology for routine testing were used for pre-clinical validation. The panel comprised 1–3 individual samples of 22 respiratory pathogens, which were available in the repository. All influenza virus A-positive samples in this panel were collected before the influenza pandemic of 2009 and comprised seasonal influenza virus A/H3N2 and A/H1N1. No clinical samples with avian influenza virus A/H5N1 were available to us. Coronavirus HKU1 was not available due to its very low prevalence. All samples had been pre-tested by ResPlex I PCR (Qiagen, Hilden, Germany) or individual real-time monoplex PCR (bacterial targets only) and original patient material had been stored at −70 °C hereafter. All samples were re-extracted using methods described below and analysed by real-time monoplex and the novel multiplex PCR, respectively.

Clinical specimens

Respiratory specimens from paediatric as well as adult patients submitted to the Department of Virology between 30.11.2009 and 30.06.2010 were included in this prospective study. Original patient samples were aliquoted and stored at −70 °C until extraction. Inclusion criteria were acute febrile lower respiratory tract infection for immunocompetent patients and acute upper and/or lower respiratory tract infection for immunocompromised patients, respectively. Specimens comprised pharyngeal secretions collected by flocked swabs in viral transport medium (Copan, Brescia, Italy) and nasopharyngeal aspirates in saline solution (NA). Informed consent was obtained from each participant or his/her legal guardian. Ethical approval was obtained from Freiburg University Medical Center and the participating study centers, respectively.

Statistics

Data were analysed using SPSS software version 19 (SPSS, Chicago, IL, USA) and the openly available statistical software environment R, version 2.10.1. Data were compared by Fisher’s exact test and the Mann–Whitney U-test. p-values were deemed significant at the 0.05 level. Cohen’s kappa was used to assess the agreement between the two methods [10].

Results

A novel target-enriched multiplex PCR using Qiagen proprietary technology (liquid-phase bead-based array; Luminex xMAP) was developed. The assay included 19 viral as well as four bacterial pathogens. Technically, the assay was optimised with respect to the reaction conditions and ultimate analytical sensitivity (data not shown).

Pre-clinical validation

In a first step, 50 archived patient samples, which had yielded positive results by routine molecular diagnostics, were used for a pre-clinical validation of the novel multiplex assay. Overall, 43/50 (86 %) samples yielded concordant results (Table 1). Monoplex PCR-positive samples displayed a median Ct-value of 30 (range 16–40), indicating a broad range of pathogen concentrations. Multiplex-positive samples yielded a median MFI value of 1,050 (range 188–4,334). Six of seven samples, which were negative by multiplex PCR (one human metapneumovirus, parainfluenza virus 1, parainfluenza virus 4, rhinovirus, Legionella pneumophila, Mycoplasma pneumoniae, respectively) yielded monoplex Ct-values ≥35 (median 39, range 35–40), indicating low concentrated targets (Fig. 1). Only 1/7 multiplex-negative samples (one human bocavirus) displayed a higher target gene concentration, with a Ct-value of 30 by monoplex PCR. Of note, this sample yielded a corresponding MFI value of 149, which is slightly below the positivity threshold of the multiplex PCR (150 MFI). Exclusion of weakly positive samples (monoplex PCR Ct-value >35) from the analysis would increase the overall detection rate of the multiplex PCR to 37/40 (93 %).

Table 1

Results of archived samples. Ct-values of monoplex and MFI values of novel multiplex PCR were compared. For monoplex PCR Ct-values ≤41 and for multiplex PCR MFI values ≥150 were regarded as positive

Pathogen	Monoplex PCR, Ct-value	Multiplex PCR, MFI value
Adenovirus	21	1,183
Bocavirus	27	889
Bocavirus	30	149
Coronavirus NL63	31	313
Coronavirus OC43	16	3,682
Coronavirus OC43	17	3,549
Coronavirus OC43	24	815
Coronavirus 229E	22	4,334
Coronavirus 229E	32	1,633
Coronavirus 229E	33	478
Enterovirus	25	837
Enterovirus	29	556
Influenza virus A	34	1,832
Influenza virus A	37	195
Influenza virus A	39	278
Influenza virus B	31	325
Influenza virus B	29	615
Influenza virus B	34	268
Human metapneumovirus	30	996
Human metapneumovirus	35	1,051
Human metapneumovirus	40	98
Parainfluenza virus 1	25	1,432
Parainfluenza virus 1	36	77
Parainfluenza virus 2	18	3,451
Parainfluenza virus 2	28	1,870
Parainfluenza virus 2	37	207
Parainfluenza virus 3	22	2,618
Parainfluenza virus 3	23	3,027
Parainfluenza virus 4	39	2,920
Parainfluenza virus 4	39	1,195
Parainfluenza virus 4	40	54
Rhinovirus	30	2,083
Rhinovirus	40	65
Respiratory syncytial virus A	34	875
Respiratory syncytial virus A	35	1,050
Respiratory syncytial virus A	35	578
Respiratory syncytial virus B	28	2,135
Respiratory syncytial virus B	29	1,609
Respiratory syncytial virus B	29	748
Bordetella pertussis	26	2,665
Bordetella pertussis	27	2,404
Chlamydia pneumoniae	27	1,780
Chlamydia pneumoniae	32	539
Chlamydia pneumoniae	34	208
Legionella pneumophila	25	1,552
Legionella pneumophila	31	188
Legionella pneumophila	39	53
Mycoplasma pneumoniae	25	406
Mycoplasma pneumoniae	26	351
Mycoplasma pneumoniae	35	58

Fig. 1

Results of archived samples with Ct-values >25 by monoplex PCR. A “+” above the columns indicates a positive result by multiplex PCR and a “−“ above the columns indicates a negative result. The broken vertical line denotes a Ct-value of 35

Clinical evaluation

In a next step, a total of 300 clinical samples were prospectively collected from 01.12.2009 until 31.03.2010. A total of 272 patients contributed a single sample and 14 patients contributed two samples from independent ARI episodes. The mean and median age of all patients was 31 and 12.5 years, respectively. Adults contributed 146/300 (49 %) samples and paediatric patients contributed 154/300 (51 %) samples. Of the adult patients, 140/146 (96 %) were immunocompetent, whereas 6/146 (4 %) were immunocompromised. Of the paediatric patients, 149/154 (97 %) were immunocompetent and 5/154 (3 %) were immunocompromised. The collection of samples comprised 112 nasopharyngeal aspirates and 188 pharyngeal swabs. For 290 samples, exact information on the time of symptom onset and sampling was available. The median time between the onset of acute respiratory symptoms and sampling was 4 days [95 % confidence interval (CI) 3.9–5.1 days].

Gold standard monoplex PCR

Monoplex PCR detected ≥1 pathogen in 165/300 (55 %) samples. Single infections were observed in 130/300 (43 %) samples. As expected, the positivity rate was significantly higher in paediatric patients [126/154, (82 %)] compared to adults [39/146, (27 %)] (Fisher’s exact test, p < 0.0001). Of note, the time between symptom onset and sampling was not different between monoplex PCR-positive and -negative samples (Mann–Whitney U-test, p > 0.05). The most frequently identifiable pathogen was rhinovirus, with 42/300 (14 %) detections, followed by RSV-B with 36/300 (12 %) and HMPV, with 27/300 (9 %) detections. The least detectable pathogens were enterovirus, with 3/300 (1 %), Legionella pneumophila, with 2/300 (0.6 %), and parainfluenza virus 4 and coronavirus NL63, with 1/300 (0.3 %) detections, respectively. Seasonal influenza viruses A/H1N1 and A/H3N2, influenza virus B and Bordetella pertussis were not detectable within this study group. In 17/300 (5.6 %) samples, a bacterial pathogen was identified, with Mycoplasma pneumoniae being the most frequent [11/300 (3.7 %)] (Table 2).

Table 2

Performance of the multiplex PCR assay for respiratory pathogen detection in comparison to the monoplex PCR assay as the gold standard

	No. of samples (monoplex PCR/multiplex PCR)				Multiplex PCR performance
Pathogen	+/+	+/−	−/+	−/−	Sensitivity (%)	95 % CI	Specificity (%)	95 % CI	PPV (%)	95 % CI	NPV (%)	95 % CI	Kappa
Adenovirus	9	1	1	289	90.0	71.4–100	99.7	99.0–100	90.0	71.4–100	99.7	99.0–100	0.9
Bocavirus	12	9	0	279	57.1	36.0–78.3	100	100–100	100	100–100	96.9	94.9–98.9	0.71
Coronavirus HKU1	3	0	0	297	100	100–100	100	100–100	100	100–100	99.8	100–100	1
Coronavirus NL63	0	1	0	299	na	na	100	na	na	na	99.7	99.0–100	0.0
Coronavirus OC43	1	5	0	294	16.7	0–46.5	100	100–100	100	100–100	98.3	96.9–99.8	0.28
Coronavirus 229E	4	1	2	293	80.0	44.9–100	99.3	98.4–100	66.7	28.9–100	99.7	99.0–100	0.72
Enterovirus	3	0	3	294	100	100–100	99.0	97.9–100	50.0	10.0–90.0	100	100–100	0.66
Human metapneumovirus	19	8	1	272	70.4	53.1–87.6	99.6	98.9–100	95.0	85.4–100	97.1	95.2–99.1	0.70
Influenza virus A, universal	6	1	0	293	85.7	59.8–100	100	100–100	100	100–100	99.7	99.0–100	0.92
Influenza virus A/H1N1	0	0	0	300	na	na	100	na	na	na	100	100–100	na
Influenza virus A/H3N2	0	0	0	300	na	na	100	na	na	na	100	100–100	na
Influenza virus A/H5N1	0	0	0	300	na	na	100	na	na	na	100	100–100	na
Influenza virus B	0	0	0	300	na	na	100	na	na	na	100	100–100	na
Parainfluenza virus 1	3	3	0	294	50.0	10.0–90.0	100	100–100	100	100–100	99.0	97.9–100	0.66
Parainfluenza virus 2	3	1	0	296	75.0	32.6–100	100	100–100	100	100–100	99.7	99.0–100	0.86
Parainfluenza virus 3	4	2	0	294	66.7	28.9–100	100	100–100	100	100–100	99.3	98.4–100	0.80
Parainfluenza virus 4	1	0	0	299	100	100–100	100	100–100	100	100–100	100	100–100	1
Respiratory syncytial virus A	15	1	3	281	93.8	81.9–100	98.9	98.9–97.8	83.3	66.1–100	99.6	99.0–100	0.88
Respiratory syncytial virus B	27	9	1	263	75.0	60.9–89.1	99.6	98.9–100	96.4	89.6–100	96.7	94.6–98.8	0.83
Rhinovirus	30	12	12	246	71.4	57.8–85.1	95.3	92.8–97.9	71.4	57.8–85.1	95.3	92.8–97.9	0.67
Bordetella pertussis	0	0	0	300	na	na	100	na	na	na	100	100–100	na
Chlamydia pneumoniae	1	3	0	296	25.0	0–67.4	100	100–100	100	100–100	99.0	97.9–100	0.40
Legionella pneumophila	0	2	0	298	na	na	100	na	na	na	99.3	98.4–100	0.0
Mycoplasma pneumoniae	7	4	0	289	63.6	35.2–92.1	100	100–100	100	100–100	98.6	97.3–100	0.77

na: not applicable

Multiple detections (≥2 pathogens) occurred in 35/300 (11.6 %) of samples, almost exclusively in paediatric patients (median age 1 year; 95 % CI 0.8–3.6 years), with the exception of one 19-year-old study subject. Two specimens harboured five and six different pathogens, respectively. However, the majority [29/35 (83 %)] of multiple-positive samples were dual infections (Table 4). Rhinovirus was the co-pathogen most frequently detected (16/35), followed by human bocavirus (14/35) and RSV-B (10/35). Parainfluenza virus 4, CoV NL63, Bordetella pertussis and Legionella pneumophila were not involved in multiple infections.

Table 4

Samples with dual detections as assessed by monoplex PCR. Ct-value and the respective pathogen by monoplex PCR as well as the result of multiplex PCR are given. The higher concentrated pathogen was regarded as the “1st pathogen” and the lower concentrated pathogen as the “2nd pathogen”, as assessed by monoplex PCR. Discordant and concordant results by multiplex PCR with respect to monoplex PCR as the gold standard are listed separately. A “+” indicates positive by multiplex PCR, whereas a −“ indicates a negative result

Discordant samples						Concordant samples
1st pathogen			2nd pathogen			1st pathogen			2nd pathogen
Ct-value	Pathogen	Multiplex	Ct-value	Pathogen	Multiplex	Ct-value	Pathogen	Multiplex	Ct-value	Pathogen	Multiplex
30	Rhinovirus	−	39	CoV OC43	−	26	RSV-B	+	31	Rhinovirus	+
33	Rhinovirus	+	38	Human bocavirus	−	33	Human bocavirus	+	36	Rhinovirus	+
35	Rhinovirus	+	39	A/H1N1 (2009)	−	27	RSV-B	+	37	Rhinovirus	+
36	Rhinovirus	+	39	Mycoplasma pneumoniae	−	24	RSV-A	+	37	Rhinovirus	+
32	RSV-A	+	37	Rhinovirus	−	29	PIV 4	+	38	Rhinovirus	+
32	RSV-A	+	39	CoV 229E	−	26	Human bocavirus	+	39	Rhinovirus	+
34	A/H1N1 (2009)	+	39	Human bocavirus	−	36	Enterovirus	+	40	Rhinovirus	+
34	Bocavirus	−	39	Chlamydia pneumoniae	−	30	RSV-B	+	35	PIV 4	+
30	CoV 229E	+	32	RSV-B	−	24	CoV HKU1	+	34	RSV-B	+
35	Human bocavirus	−	37	RSV-B	−	20	Human bocavirus	+	26	RSV-A	+
29	Human bocavirus	+	40	RSV-B	−	20	Adenovirus	+	24	CoV OC43	+
22	RSV-A	−	29	PIV 2	+
19	Adenovirus	+	34	Human bocavirus	−
35	RSV-A	+	39	Human bocavirus	−
32	HMPV	+	39	Chlamydia pneumoniae	−
25	CoV HKU1	+	40	CoV OC43	−
34	Human bocavirus	−	35	RSV-B	−
34	HMPV	+	41	Rhinovirus	−

Finally, the MS2 phage, which served as an internal control, was detected in 300/300 (100 %) samples, indicating the robustness of the assay.

Novel multiplex PCR

Overall, multiplex PCR yielded positive results in 140/300 (46 %) samples. As experienced with monoplex PCR, significantly more samples were positive in paediatric patients [114/154, (74 %)] compared to adults [26/146, (18 %)] (Fisher’s exact test, p < 0.0001). The most frequently detectable pathogen by multiplex PCR was rhinovirus [42/300 (14 %)], followed by RSV-B [28/300 (9 %)], human metapneumovirus [20/300 (6.6 %)] and RSV-A [18/300 (6 %)], respectively. The least detectable pathogens were parainfluenza virus 4, coronavirus OC43 and Chlamydia pneumoniae, with a single detection (0.3 %), respectively. Seasonal influenza viruses A/H1N1 and A/H3N2 and influenza virus B, coronavirus NL63, Bordetella pertussis and Legionella pneumophila were not found. Notably, this version of the multiplex assay was not designed to type pandemic influenza virus A/H1N1 (2009). Instead, the detection of pandemic influenza virus A/H1N1 (2009) was achieved by exclusion criteria (positive result for universal influenza virus A and concomitantly negative for seasonal A/H1N1, A/H3N2 and A/H5N1). Hence, 6/300 (2 %) samples that were positive for universal influenza virus A but negative for seasonal influenza virus A/H1N1 and A/H3N2 indicated the presence of pandemic A/H1N1 (2009). This finding was confirmed upon the use of an A/H1N1 (2009)-specific real-time PCR.

Multiple detections (2–4 detections) were recorded in 26/300 (8.7 %) samples. The three agents most frequently detectable by multiplex PCR in this subgroup were rhinovirus [9/26 (35 %)] and both RSV-B and human bocavirus [6/26 (23 %)], respectively. Seasonal influenza virus A/H1N1 and A/H3N2 and influenza virus B, coronavirus NL63, parainfluenza virus 1 and 4, Chlamydia pneumoniae and Bordetella pertussis were not detectable. The internal control of the multiplex assay yielded a valid signal in 228/300 (76 %) samples. Detection randomly occurred in the first 100 samples and could be attributed to a processing error. After correction (i.e. use of the recommended concentration of the internal control), a valid signal was obtained in all samples.

Comparison of the gold standard monoplex PCR with novel multiplex PCR

For the most common pathogens, the sensitivity ranged from 57.1 to 93.8 % and the specificity ranged from 95.3 to 100 %, respectively (Table 2). Since the prevalence of some viral as well as bacterial pathogens was rather low, the sensitivity and specificity could not be reliably assessed. The positive and negative predictive value (PPV and NPV) ranged from 50 to 100 % and from 95.3 to 100 %, respectively. Cohen’s kappa was calculated and ranged from 0.28 (fair) to 1 (perfect) (Table 2) [18]. However, the most common pathogens all yielded kappa values >0.67 (95 % CI 0.55–0.78), indicating substantial (rhinovirus, HMPV and human bocavirus) to almost perfect agreement (RSV-A, RSV-B). An analysis on the patient sample level yielded concordant results for 230/300 [77 % (95 % CI 72–81 %)] samples. In the 70 samples with discrepant results, 14/70 (20 %) showed a false-positive deviation, 49/70 (70 %) a false-negative deviation and 7/70 (10 %) in both directions. Discrepant results included a broad variety of pathogens. The most discrepant results were observed for rhinovirus, with a total of 24 (12 false-negative, 12 false-positive results), followed by RSV-B with ten (nine false-negative, one false-positive results), HMPV with nine (eight false-negative, one false-positive results) and human bocavirus with nine discrepant results (nine false-negative results), respectively. Interestingly, for false-negative samples, significantly lower target gene concentrations as indicated by higher Ct-values were recorded for bocavirus, HMPV and RSV-B, but not for parainfluenza virus 1, rhinovirus and Mycoplasma pneumoniae (Table 3). To further analyse the rhinovirus discrepant results, we sequenced the VP1 gene of nine of the monoplex-positive/multiplex-negative and eight of the monoplex-negative/multiplex-positive samples. A total of 12/17 (71 %) samples comprised the novel rhinovirus type C, as demonstrated by sequence analysis (data not shown). In the monoplex-positive/multiplex-negative group, 6/9 samples yielded the novel rhinovirus C, whereas in the monoplex-negative/multiplex-positive group, 6/8 samples were positive for rhinovirus C, respectively. Of note, three rhinovirus-positive samples with concordant results (monoplex PCR-positive and multiplex PCR-positive) concomitantly yielded a false-positive enterovirus signal in the multiplex PCR. However, the corresponding enterovirus MFI value in each of the samples was lower than the rhinovirus MFI value, indicating a possible cross-reactivity of the enterovirus assay.

Table 3

Comparison of monoplex PCR Ct-values in multiplex PCR-positive and -negative samples, respectively

Pathogen	Median Ct-value for monoplex PCR
	Multiplex PCR-positive samples (n)	Multiplex PCR-negative samples (n)	p-value*
Adenovirus	30 (9)	38 (1)	na
Bocavirus	25.5 (12)	36 (9)	0.0001
Coronavirus HKU1	25 (3)	na (0)	na
Coronavirus NL63	na (0)	39 (1)	na
Coronavirus OC43	24 (1)	39 (5)	na
Coronavirus 229E	34 (4)	39 (1)	na
Enterovirus	36 (3)	na (0)	na
Human metapneumovirus	33 (19)	38.5 (6)	0.031
Influenza virus A, universal	34 (6)	39 (1)	na
Influenza virus A/H1N1	na (0)	na (0)	na
Influenza virus A/H3N2	na (0)	na (0)	na
Influenza virus B	na (0)	na (0)	na
Parainfluenza virus 1	29 (3)	40 (3)	0.1
Parainfluenza virus 2	29 (3)	36 (1)	na
Parainfluenza virus 3	28 (4)	40 (2)	na
Parainfluenza virus 4	na (0)	32 (1)	na
Respiratory syncytial virus A	28 (15)	22 (1)	na
Respiratory syncytial virus B	27 (27)	36 (9)	0.002
Rhinovirus	36.5 (30)	35.5 (12)	0.82
Bordetella pertussis	na (0)	na (0)	na
Chlamydia pneumoniae	36 (1)	39 (3)	na
Legionella pneumophila	na (0)	39 (2)	na
Mycoplasma pneumoniae	36.5 (7)	37 (4)	0.9

*Mann–Whitney U-test; na: not applicable

Finally, a comparison on the patient sample level of samples with multiple detections showed that only 12/35 (34 %) samples yielded concordant results (11 with dual infection, one with triple infection). To further analyse these samples, the Ct-values of monoplex PCR-positive samples with dual detections were compared with respect to concordant and discordant multiplex results. Pathogens were categorised into 1st (high target gene concentration) and 2nd (low target gene concentration) pathogen based on the Ct-value of monoplex PCR (Table 4). For the 1st pathogen, the median Ct-value of discordant samples was 32, whereas concordant samples yielded a median Ct-value of 26 (Mann–Whitney U-test, p = 0.03). In a similar fashion, for the 2nd pathogen, the median Ct-values were 39 and 36, respectively (Mann–Whitney U-test, p = 0.04).

Discussion

A novel microarray PCR was developed for the detection of 23 pathogens causing acute febrile respiratory tract infection. From a technical standpoint, the incorporation of 19 viral and four bacterial targets into a single multiplex assay was successful, as demonstrated on a panel of previously characterised samples. To fully appreciate the role of viruses in ARI, we decided to include 19 viruses which have currently been associated with ARI. Including such a broad range of pathogens into one assay further provides the opportunity to study the relevance of multiple infections and possible interactions between ARI pathogens. A recent study points to the clinical relevance of this issue [4]. We further decided to include the atypical bacteria Bordetella pertussis, Mycoplasma pneumoniae, Chlamydia pneumoniae and Legionella pneumophila into the panel, since these are non-colonising bacteria, hard to cultivate and, thus, provide a valuable target for molecular assays. Previous studies have already demonstrated the feasibility of NAT to detect atypical bacteria in formalin-fixed, paraffin-embedded specimens [2]. As infections with Mycoplasma pneumoniae were associated with severe illness in a cohort of children, its testing might also be of value for risk assessment [3]. Interestingly, Brunstein et al. reported on the infectivity-enhancing activity of S. pneumoniae and H. influenzae [4]. Since the combination of influenza and concurrent infection with S. pneumoniae can worsen outcome, the inclusion of S. pneumoniae into the panel might be a future option [22]. However, asymptomatic colonisation with S. pneumoniae does occur and may limit the validity of positive PCR results.

In general, molecular detection methods for the rapid identification of ARI pathogens outperformed classical methods in numerous studies [12, 30, 31]. Therefore, we did not include classical methods like direct fluorescence antigen (DFA) detection and virus isolation into our test algorithm. Instead, a panel of published and optimised monoplex PCR assays was used for comparison. Use of individual real-time PCR assays also allowed a reliable estimation of the relative target gene concentration.

Here, direct comparison on the patient sample level using monoplex PCR as the gold standard showed concordant results in 77 % (95 % CI 72–81 %) of samples, with sensitivities varying across targets. Similar findings were reported by Gadsby et al. [13], who compared the xTAG Respiratory Viral Panel (RVP) with in-house real-time PCR. A low number of positive samples certainly limits the validity for some targets, requiring larger studies in order to reliably assess sensitivity and specificity. Similar to Gadsby et al., we found that the most discrepant results were associated with Ct-values >35. This was seen for viral as well as for bacterial targets. Of note, with the exception of rhinovirus, parainfluenza virus 1 and Mycoplasma pneumoniae, this finding is most likely not associated with sequence mismatches between the primers and the target, but, rather, with low target gene concentrations. However, although the microarray assay presented here was technically optimised, the inclusion of additional targets might have a detrimental outcome on the sensitivity.

A limitation of our approach using monoplex PCR as the gold standard is the non-detection of variant virus as encountered, e.g. for rhinovirus. The resolution of a subset of discrepant results for rhinovirus showed that novel rhinovirus C influenced the performance of the reference method, as well as the novel multiplex test. Recently, Faux et al. [9] compared different rhinovirus PCR assays and reported a high degree of variability in performance, supporting the finding presented here.

Likewise, molecular methods for the detection of adenovirus are challenging due to the range of different types. In particular, Gadsby et al. [13] reported on a fairly low sensitivity of the xTAG RVP assay for adenovirus, which was most likely attributable to primer mismatches. In our study, the sensitivity of the adenovirus assay was rather good, with a value of 90 %. It is unlikely that only one type was detected, since specimens were prospectively collected over a 6-month period at different study sites. Still, without further sequencing, it remains unclear as to which adenovirus serotypes were prevalent in our study.

Importantly, adults especially may benefit from highly sensitive molecular methods, since they shed ARI pathogens in lower multiplicity than children. However, for the adult cohort, the observed detection rates in this study are slightly lower than those reported by Pabbaraju et al. [25]. It remains speculative whether this finding might be attributable, in part, to the different sample types used. In light of the lower detection rates in adult patients, application of the novel microarray assay might be restricted to younger patients. Larger studies among the adult population may clarify which method and test algorithm is the most sensitive and feasible approach.

Using a panel of targets may also prove cost-effective, since the turn-around time decreases considerably when compared to a mix of culture, DFA and selected PCR testing of non-cultivatable agents. The novel multiplex PCR generates results within 5.5 h and can be completed in a single working day. Mahony et al. [21] have shown that multiplex PCR was the least costly option in a cohort of children. However, this result is highly dependent on the prevalence of pathogens and, thus, might not apply to adults.

For the bacterial targets, lower sensitivities were observed, similar to the findings of Benson et al. [2]. Critically, the number of samples was not adequate for some of the targets, thus, limiting the validity of our results. Nevertheless, a number of bacterial infections were detected by both methods in this study. These may valuably guide the clinician’s decision to initiate specific treatment. Of note, Legionella pneumophila remained undetectable in this study. It cannot be ruled out that pharyngeal specimens are not the optimal specimen type, and the use of sputum might provide better results. For better standardisation, however, we decided to collect only one specimen type per patient in our study.

One of the strengths of multiplex PCR is the ability to detect multiple infections, in contrast to the conventional testing of specimens, which is performed in a sequential manner, stopping after the first positive result. Here, each positive result above the threshold of the PCR reaction was communicated to the clinicians. A recent study by Rand et al. [28] demonstrated 15.9 % mixed infections in a cohort of 200 patients. We observed multiple infections at a slightly lower prevalence of 11.6 and 8.7 %, respectively. However, the clinical significance of dual or triple infections remains unclear to date. Interestingly, RNA viruses were frequently associated with multiple infections, whereas the viral/bacterial combination has rarely been seen. Of note, real-time PCR was significantly more sensitive in the detection of multiple infections, whereas the multiplex assay frequently missed the second pathogen. In most instances, this was due to low target gene concentrations, as indicated by Ct-values above 35 in the monoplex PCR.

As a future prospect, measuring target gene concentrations, similarly to cytomegalovirus (CMV) or BK virus (BKV) DNA quantification, might be an interesting option, as shown by Franz et al. [11]. In their study, symptomatic patients displayed significantly higher pathogen concentrations than asymptomatic controls. Moreover, it is suggestive that, e.g. the length of therapy could be based on quantitative data. However, the correlation of MFI values with target gene concentrations has not been firmly assessed for multiplex bead-based assays. Another issue of concern is the sampling of respiratory specimens, which is far from being standardised [19]. The optimal specimen type needs to be determined (and might vary between different agents) and more data on the clinical relevance of virus detection in bronchoalveolar lavage (BAL) specimens are desirable.

To conclude, we present a novel multiplex assay for the detection of viral as well as bacterial pathogens associated with febrile ARI. The assay proved its principal and is currently being evaluated in a large prospective multicentre trial on ARI. This was also initiated under the hypothesis that this approach might be instrumental in reducing unjustified antibiotic usage.