Necessity to critically review the automatic results of the Xpert Flu assay.

While using the Xpert Flu assay we became aware of false-negative results. The study aimed to analyze the causes of these false-negative results. One hundred fifty-nine respiratory specimens were tested in the Xpert Flu assay and in multiplex reverse transcription-polymerase chain reactions (RT-PCRs) for respiratory viruses. Discordant specimens were tested in the Influenza A/B r-gene assay. One hundred fifty-two (96%) and 151 (95%) specimens yielded concordant results for influenza A and B, respectively. Fifteen specimens tested negative in the Xpert Flu assay and positive in a multiplex RT-PCR. Positive results were confirmed for 12 of these specimens in the Influenza A/B r-gene assay. Xpert Flu assay amplification curves and endpoints suggested that the false-negative results were mainly due to erroneous automatic result interpretation. We report false-negative results of the Xpert Flu assay due to erroneous automatic result interpretation. Careful analysis of amplification curves and endpoints is needed to avoid reporting of false-negative results.

Introduction

Rapid diagnosis of influenza is useful for early administration of antiviral therapy, infection control and avoidance of unnecessary antibiotics use. Multiplex RT-PCRs for the detection of respiratory pathogens are now commercially available and used in routine diagnostics of respiratory infections (Gharabaghi et al., 2011, Vallieres and Renaud, 2013). They have the advantage of simultaneous detection of a panel of different respiratory pathogens. However, they are time-consuming and usually performed in large series of specimens thereby prolonging the time until a result is available to the physician in charge of the patient. Rapid molecular diagnostic assays have been developed (Nie et al., 2014, Novak-Weekley et al., 2012) that can be performed upon demand for single specimens. They usually yield results in less than two hours. An example is the Xpert Flu test (Cepheid, France).

Objectives

Using the Xpert Flu test (Cepheid, France) for urgent diagnostics of influenza, we became aware of apparently false-negative results. Therefore, we initiated the current study in order to systematically analyze the causes of these false-negative results.

Study design

Clinical specimens

From October 6th, 2014 to April 10th, 2016, 2108 specimens were sent to our laboratory for rapid influenza testing (Xpert Flu assay) and 3278 specimens were sent for multiplex RT-PCR detection of respiratory viruses. Of these, 159 respiratory specimens were tested with both the Xpert Flu assay and a multiplex RT-PCR for respiratory viruses. The specimens were sent from intensive care units (n  = 45), departments of pulmonary medicine (n  =   30), pediatrics (n  =   23), hematology (n  =   20), infectious diseases (n  =   9), geriatrics (n  =   8), emergency medicine (n  =   6) and less frequently from other departments (n  =   18). They were 151 nasopharyngeal swabs, 3 bronchoalveolar lavages, 1 nasopharyngeal aspiration, 2 expectorations, 1 bronchopulmonary secretion and 1 tracheal aspiration. Remel Microtest M4RT Transport (Roche, France) was used as transport medium.

The 159 specimens that were tested in both assays were analyzed in the current study. The Xpert Flu assay and a multiplex RT-PCR for respiratory viruses were prescribed simultaneously by the physician in charge of the patient in 150 of these specimens (cohort A). In case of the other 9 specimens (cohort B), the laboratory physician additionally prescribed a multiplex RT-PCR for respiratory viruses because she/he doubted the Xpert Flu assay result (i.e. because a negative result was indicated by the automatic interpretation but an amplification curve was visible or an endpoint value was elevated).

Detection of influenza virus

Multiplex RT-PCRs for respiratory viruses

Nucleic acids from 200 μL of each specimen were extracted using the Magtration System 12GC using the MagDEA® Viral DNA/RNA 200 (GC) kit (Precision System Science Co., Ltd.; Japan) according to the manufacturer's instructions (elution volume 100 μL). Nucleic acids extracts were stored at −80 °C.

Nucleic acids extracts were submitted to a commercially available multiplex RT-PCR screening for a panel of respiratory viral pathogens including influenza virus A and B, respiratory syncytial virus A and B, human adenovirus (HAdV), human metapneumovirus, coronaviruses 229E, NL63 and OC43, parainfluenza virus 1–4, rhinovirus A/B/C, human enterovirus, and bocavirus 1–4 (Anyplex II RV16 Detection, Seegene, Seoul, Korea) from October 6th, 2014 to February 8th, 2016. The assay was performed according to the manufacturer's instructions. The assay contains an internal control. 50 amplification cycles are performed. The detection limit is indicated by the manufacturer as 50 copies per reaction. The result is indicated as “invalid”, “negative” or “positive” with a semi-quantitative result: + for weakly positive, ++ for intermediate and +++ for strongly positive specimens. From February 9th to April 10th, 2016, Allplex Respiratory Panel 1 (Seegene, Seoul, Korea) was used according to the manufacturer's instructions. This technique is a multiplex real-time one-step RT-PCR for the detection of influenza virus A and B, human respiratory syncytial virus A and B, and subtyping of influenza A virus (human influenza A virus subtypes H1, H3, and H1 2009) containing an internal control. Forty-five amplification cycles are performed. The result is indicated as “invalid”, “negative” or “positive” and in the latter case, the threshold cycle is specified. The detection limit is indicated by the manufacturer as 100 copies per reaction.

Xpert Flu assay

Xpert Flu assay (Cepheid, France) was used on GeneXpert instrument from October 6th, 2014 to December 31st, 2015 and on GeneXpert Infinity instrument thereafter (Cepheid, France) according to the manufacturer's instructions. Briefly, 300 μL of respiratory specimen was added to the cartridge. Lysis of the specimen, purification and amplification of nucleic acids (40 cycles) as well as detection of the target sequence was fully automated on the above-mentioned instruments. The results were interpreted by the GeneXpert instrument and indicated as “positive”, “negative” or “invalid”. In the case of positive results, a threshold cycle is specified. Furthermore, amplification curves and endpoint values are displayed. This technique includes detection of influenza A, influenza A subtype H1N1 2009, and influenza B virus. The sensitivity is indicated by the manufacturer depending on the virus strain as at least 100 tissue culture infective dose (TCID)50/mL for influenza A and 25 TCID50/mL for influenza B.

Influenza A/B r-Gene assay

All available nucleic acids extracts (stored at −80 °C) of specimens that yielded discordant results between the two assays described above (n  =   12) were submitted to the Influenza A/B r-gene assay (Argene, Biomerieux, France). The assay was performed according to the manufacturer's instructions and was run on a 7500 Real-Time PCR System (Applied Biosystems, Thermofisher, France) (45 amplification cycles). The sensitivity is indicated by the manufacturer depending on the virus strain as at least 199.5 TCID50/ml for influenza A and 1.7 TCID50/ml for influenza B.

Statistical analysis

Median and mean of quantitative variables were calculated. Quantitative variables were compared by using Mann–Whitney U test with IBM SPSS statistics version 22 (IBM, USA). P < 0.05 was considered significant.

Results

During the study period, 258 of 2108 (12.2%) specimens yielded positive results for influenza A and 153 of 2108 (7.3%) for influenza B in the Xpert Flu assay; 1697 specimens yielded negative results for both influenza A and B.

Of the 2108 specimens, 159 (7.5%) were also tested in a multiplex RT-PCR for respiratory viruses. These 159 specimens are object of the current study. The Xpert Flu assay and a multiplex RT-PCR for respiratory viruses were prescribed simultaneously by the physician in charge of the patient in 150 of these specimens (cohort A). In case of the other 9 specimens (cohort B), the laboratory physician additionally prescribed a multiplex RT-PCR for respiratory viruses because she/he doubted the Xpert Flu assay result. In cohort A, 7 specimens tested positive for influenza A and 6 specimens tested positive for influenza B, respectively, in both techniques. Five specimens gave a positive result for influenza A and 4 specimens gave a positive result for influenza B in a multiplex RT-PCR for respiratory viruses and a negative result in the Xpert Flu assay. The details split by assay are shown in Table 1 . In cohort B, 3 specimens tested concordantly negative in both assays whereas 2 and 4 specimens tested positive for influenza A and B, respectively, in the Allplex Respiratory Panel 1 and negative in the Flu Xpert assay. The 15 discordant specimens were investigated in more detail (Table 2 ). Nucleic acids of 12 of these specimens were available and were tested in the Influenza A/B r-gene assay. All of these gave a positive result in the Influenza A/B r-gene assay thus confirming the result of the multiplex RT-PCR (Table 2). Only four and three of the discordant specimens were weakly positive in the multiplex RT-PCR for respiratory viruses and in the Influenza A/B r-gene assay, respectively (defined as threshold cycle >38 or +) (Table 2). However, when all Allplex Respiratory Panel 1 positive specimens (concordant and discordant) were analyzed, the median Ct value of discordant specimens was significantly higher than that of concordantly positive specimens (35.6 vs. 24.2, P  = 0.001) (Supplementary Table 1). The semiquantitative results between concordantly positive and discordant specimens for Anyplex II RV16 assay positive specimens were similar (Supplementary Table 1).

Table 1

Results of the multiplex RT-PCRs for respiratory viruses and the Xpert Flu assay (cohort A).

	Result of the multiplex RT-PCRs for respiratory viruses
Test type and results	1. Anyplex II RV16(n = 107)		2. Allplex Respiratory Panel 1(n = 43)
	Positive	Negative	Positive	Negative
Result of the Xpert Flu assayInfluenza A
Positive	5	0	2	0
Negative	2	100	3	38
Influenza B
Positive	0	0	6	0
Negative	1	106	3	34

Table 2

Details of specimens with discordant results.

Specimen	Specimen	Date	Results			Details Xpert Flu assay
number	type		Multiplex RT- PCR for respiratory virusesa(Threshold cycle)	Xpert Flu assay	Influenza A/B r-gene(Threshold cycle)	EndpointInfluenza A	EndpointInfluenza B	Amplification curve
1	NPS	October 2014	Influenza A (++)b	Negative	not done	4	5	No
2	NPS	January 2015	Influenza A (++)b	Negative	not done	6	0	Doubtful
3	Tracheal aspiration	February 2016	Influenza A (31.3)	Negative	not done	5	5	No
4⁎	NPS	March 2016	Influenza A (39.6)	Negative	Influenza A (37.0)	16	5	Typical
5	NPS	March 2016	Influenza Ac	Negative	Influenza A (37.3)	19	1	Typical
6⁎	NPS	March 2016	Influenza A (41.1)	Negative	Influenza A (39.0)	19	2	Typical
7	NPS	March 2016	Influenza A (35.6)	Negative	Influenza A (34.1)	15	6	Typical
8	NPS	February 2016	Influenza B (++)b	Negative	Influenza B (36.0)	3	20	Typical
9	NPS	March 2016	Influenza B (35.6)	Negative	Influenza B (35.6)	1	1	No
10	NPS	March 2016	Influenza B (37.3)	Negative	Influenza B (37.3)	5	8	Doubtful
11⁎	NPS	March 2016	Influenza B (34.3)	Negative	Influenza B (35.0)	3	31	Typical
12⁎	NPS	March 2016	Influenza B (33.7)	Negative	Influenza B (35.9)	4	15	Typical
13	NPS	March 2016	Influenza B (41.7)	Negative	Influenza B (38.8)	0	19	Typical
14⁎	NPS	March 2016	Influenza B (38.7)	Negative	Influenza B (43.0)	2	10	Typical
15⁎	NPS	March 2016	Influenza B (34.1)	Negative	Influenza B (35.5)	4	38	Typical

NPS = nasopharyngeal swab.

These specimens belonged to cohort B, in which a multiplex RT PCR for respiratory viruses was added by the laboratory physician because she/he doubted the Xpert Flu assay result.

Allplex Respiratory Panel 1 unless otherwise indicated.

Anyplex II RV16: No threshold cycle was indicated but a semiquantitative result: + for weakly positive, ++ for intermediate and +++ for strongly positive specimens.

This specimen gave a negative result for influenza A, but a positive result for influenza A subtype H1N1 2009. According to the manufacturer's instructions of the multiplex RT-PCR this result is valid and interpreted as influenza A positive. Consequently, there is no threshold cycle for influenza A, but a threshold cycle of 37.0 for influenza A subtype H1N1 2009.

Interestingly, in 10 of 15 discordant results, a typical amplification curve was visible in the amplification plot of the Xpert Flu assay. An example is shown for specimen 6 (influenza A, Fig. 1A and B), and for specimen 11 (influenza B, Fig. 1C and D). A doubtful curve was visible in 2/15 discordant results, as shown for specimen 2 (influenza A, Supplementary Fig. 1A and B) and specimen 10 (influenza B, Supplementary Fig. 1C and D).

Fig. 1

Typical amplification curves of the Xpert Flu assay.

Amplification curves of specimen 5 in original format (A) and after zooming on the y-axis (B), of specimen 11 in original format (C) and after zooming on the y-axis (D).

Amplification curves of the internal control (SPC) are shown in light green, amplification curves of influenza A are shown in reddish brown and amplification curves of influenza B are shown in brown.

The Xpert Flu assay indicates endpoint values that correspond to the maximum of fluorescence in relative fluorescence units. In concordantly positive specimens, endpoint values ranged from 21 to 148 for influenza A (n  =   7) and from 63 to 723 for influenza B (n  =   6). 4/7 (57%) influenza A discordant specimens had an endpoint value equal or higher than 10 for influenza A in the Xpert Flu assay and all had an endpoint value <10 for influenza B (Table 2). Six of 8 (75.0%) influenza B discordant specimens had an endpoint value equal or higher than 10 for influenza B and all had an endpoint value <10 for influenza A (Table 2). Specimens with endpoint values equal or higher than 10 all showed a typical curve in the amplification plot. When endpoint values of the Xpert Flu assay were compared between concordantly negative specimens (true-negative) and discordant results (false-negative), there was a statistically significant difference (P  < 0.001) (Supplementary Table 2).

In contrast to the false-negative specimens, only 8 of 145 (5.5%) true-negative specimens had endpoint values equal or higher than 10. Three of these had endpoint values equal or higher than 10 both for influenza A and B suggesting a non-specific endpoint elevation (Supplementary Table 3). Of the 5 other specimens with endpoint values equal or higher than 10 for influenza B, 3 had an endpoint value equal or higher than 10 for influenza A subtype H1N1 2009, also suggesting a non-specific endpoint elevation (Supplementary Table 3). Only 2 of the 8 specimens with endpoint values equal or higher than 10 showed a doubtful amplification curve, whereas the other six showed no amplification curve (Supplementary Table 3). Taken together, these data suggest that false-negative and true-negative specimens can be differentiated if both endpoint values and amplification curves are analyzed in detail.

Discussion

The current study was initiated to further investigate discordant results between different techniques used for the diagnosis of influenza. Although the overall percentage of concordant results was high, there were 15 specimens (out of 159; 9.4%) that yielded negative results in the Xpert Flu assay and positive results in a multiplex RT-PCR for respiratory viruses. One of the specimens with discordant results (specimen 3, Table 2) was a tracheal aspiration. The package insert of the Xpert Flu assay indicates that nasal aspirates/washes and nasopharyngeal swab specimens can be used for the assay. Therefore, the discordant result in case of specimen 3 might be because a tracheal aspiration is not an appropriate specimen type for testing in the Xpert Flu assay. The Influenza A/B r-gene assay was performed on all available discordant specimens (12/15, 80%) and confirmed the multiplex RT-PCR assay result in all cases. Therefore, the results of the Xpert Flu assay were considered false-negative. The Xpert Flu assay contains an internal control (SPC; Fig. 1 and Supplementary Fig. 1) that was positive in all discordant specimens excluding PCR inhibition as the reason for false-negative results. A lack of sensitivity of the Xpert Flu assay could be the cause of false-negative results. A finding that supports this hypothesis is that the median Ct value of false-negative specimens was significantly higher than that of concordantly positive specimens (Supplementary Table 1). However, sensitivity of the Xpert Flu assay has previously been reported to be higher than 93% compared to another molecular assay (Dugas et al., 2014, Novak-Weekley et al., 2012). Most of the discordant results in our study were obtained in comparison with the Allplex Respiratory Panel 1 (Table 1). A possible explanation might be that the Allplex Respiratory Panel 1 is more sensitive than the Anyplex RV16 assay and the assay used in these previously published studies. However, according to the package insert, the sensitivity of the Allplex Respiratory Panel 1 is not higher than that of the Anyplex RV16 assay.

Data provided by the manufacturers in the package inserts of the Xpert Flu and the r-gene assays demonstrate that the sensitivity of the assays varies between different influenza virus types and strains. Although the package inserts of the Anyplex RV16 and Allplex Respiratory Panel 1 assays do not mention that the sensitivity of the assays varies between different virus types and strains, it probably is the case. Therefore, differences in sensitivities of the assays to detect certain virus strains may partly explain the discordant results between the Xpert Flu assay and the other assays. Interestingly, in 5 of the discordant specimens there was a difference of 2 Cts or more between the Influenza A/B r-gene assay and the Allplex Respiratory Panel 1. This might also be due to different sensitivities of the two assays for the detection of different influenza virus strains. Failure of an assay to detect a specific virus strain due to mutations in the primer or probe regions has been previously reported (Binnicker et al., 2013). We cannot rule out that this was the reason for the false-negative results of the Xpert Flu assay because primer and probe sequences are not known to us. However, in the current study, false-negative results were detected for both influenza A and B, therefore our findings do not support the hypothesis that the false-negative results were due to primer or probe mismatches. Furthermore, our analysis of the details of the Xpert Flu assay suggests erroneous automatic interpretation as the main cause of false-negative results. In fact, the detailed analysis of endpoint values and amplification curves suggests that target amplification and detection had occurred in most of the specimens with false-negative results, although the automatic result interpretation indicated “negative”. Once we had become aware of this problem, we therefore carefully verified the amplification curves (after zooming) and endpoint values of each specimen and reported the specimens with a visible curve as “weakly positive” or “indeterminate”. This approach allowed us to avoid reporting false-negative results in 9 of the 15 discordant specimens described in the current study. Importantly, since February 2016, this approach was prospectively applied to specimens tested routinely in the Xpert Flu assay only. In total, 16 of 2108 (0.8%) specimens (including the 9 specimens mentioned above) that were interpreted as negative by the automatic result interpretation of the Xpert Flu assay, were reported “weakly positive” (n  =   7) or “indeterminate” (n  =   9) by the laboratory physician. However, we did not retrospectively review the amplification curves after zooming of all Xpert Flu assay results performed during the study period (n  =   2108). Therefore, we might have missed a number of false-negative results. Taken together, our findings suggest that automatic result interpretation is not completely reliable and could represent a concern in case of use as point of care test. We suggest that the automatic result interpretation should be changed to report “indeterminate” instead of “negative” in case of amplification curves of low amplitude or slightly elevated endpoints. This would alert the user who could come up with a personal interpretation after careful review of both amplification curves and endpoint data and then decide whether to report the assay result, to perform another assay or to recommend testing a second specimen.

The study has the following limitations: testing of respiratory specimens by both techniques was done either if prescribed by the physician in charge of the patient (cohort A, 150/159, 94.3%) or if the laboratory physician in charge of result validation had a doubt about the result of the Xpert Flu assay (cohort B, 9/159, 5.7%). A doubt occurred when an amplification curve was visible or an endpoint value was elevated but the automatic result interpretation indicated “negative”. Furthermore, a doubt arose when the first specimen yielded a negative result and a second specimen from the same patient yielded a positive result in the Xpert Flu assay. Nine of 22 (41%) and 6 of 9 (67%) multiplex RT-PCR positive specimens yielded negative results in the Xpert Flu assay in cohort A and cohort B, respectively. There is enrichment in discordant results in cohort B. For this reason, results of cohort A and B were presented separately.

In conclusion, we have analyzed specimens with false-negative results in the Xpert Flu Assay and show that these are mainly due to erroneous automatic interpretation. Users of this technique should be aware of the problem and carefully interpret both amplification curves and endpoint data and not solely rely on the automatic result interpretation.

The following are the supplementary data related to this article.

Supplementary Fig. 1

Doubtful amplification curves of the Xpert Flu assay. Amplification curves of specimen 2 in original format (A) and after zooming on the y-axis (B), of specimen 10 in original format (C) and after zooming on the y-axis (D). Amplification curves of the internal control (SPC) are shown in light green, amplification curves of influenza A are shown in reddish brown and amplification curves of influenza B are shown in brown.

Supplementary Table 1

Influenza A/B virus positive specimens by multiplex RT-PCRs for respiratory viruses according to concordant or discordant result compared to the Xpert Flu assay.

Supplementary Table 2

Comparison of endpoint values.

Supplementary Table 3

Details of true-negative specimens with endpoints equal or higher than 10 in the Xpert Flu assay.

Compliance with ethical standards

Funding

This work was supported by Ministère de l'Education Nationale, de l'Enseignement Supérieur et de la Recherche, Université Lille 2 (Equipe d'accueil 3610) and Centre Hospitalier Régional et Universitaire de Lille. The funding sources had no involvement in the study design, data collection and analysis nor in writing of the report.

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics statement

The study was performed in accordance with the Declaration of Helsinki. It was a retrospective, noninterventional study. For this type of study, formal consent is not required.