First international proficiency study on West Nile virus molecular detection.

BACKGROUND: West Nile virus (WNV) molecular detection is being conducted by a growing number of laboratories, but the degree of proficiency may vary between them. External quality control is needed.
METHODS: We have conducted an international quality assurance study on WNV molecular detection. Participating laboratories tested noninfectious samples inactivated by heat and gamma irradiation. Participants received 7 coded lyophilized samples containing WNV of genetic lineages 1a, 1b, and 2 at 2600 to 18 000 000 RNA copies/mL, 3 samples containing heterologous flaviviruses, and 2 negative samples.
RESULTS: Thirty laboratories participated. The average laboratory achieved 50% detection probability from 7762 copies/mL onward (probit analysis; 95% CI = 1174-24547 copies/mL). Lineages 1a and 1b were detected with equal efficiencies, but the lineage 2 strain (Ug37) was detected at significantly lower rates. Only 27% of participants were able to detect the 6 samples containing > or =1.8 x 10(4) copies/mL. Three laboratories generated false-positive results in negative samples. Six of 30 laboratories reported correct strain identification in 3 samples containing non-WNV flaviviruses. We observed a significant positive correlation between the capability of detecting non-WNV flaviviruses and detecting WNV lineage 2.
CONCLUSIONS: Most participants showed good performance in detecting lineage 1 WNV, the predominant virus in the Northern Hemisphere. The inability of some laboratories to detect even highly concentrated lineage 2 WNV downgraded the overall outcome. The lineage 2 material received through this study will provide laboratories with the necessary template for improving their assays. Such material is otherwise hard to obtain.

West Nile virus (WNV)1 is a member of the Japanese encephalitis virus group of flaviviruses, causing febrile illness and encephalitis in humans. Two genetic lineages exist (1)(2)(3). Lineage 1 has the largest area of distribution, and its recent introduction into North America has already caused more than 16 000 cases of human illness since 1999 (4)(5). WNV-Kunjin, which is enzootic in Australia, is an outlier cluster within lineage 1. Lineage 2 is restricted to Sub-Saharan Africa and Madagascar (2)(5). In Europe and the Middle East, WNV is being introduced continually by migrating birds, but it is probably also persisting in natural reservoirs (2)(6)(7).

Molecular detection of WNV is used for ecological investigation, case management, and prevention of transmission by transfusion and transplantation (3)(8)(9). Reverse transcription (RT)-PCR is the preferred tool. Reports of several molecular diagnostic assays have been published, and the first commercial test products have become available (8)(9). Performance of assays may vary considerably between laboratories, however. Much of the available evaluation data has been generated in pilot studies only. Little information is available about the relative and overall proficiency in different laboratories.

We report the results of the first international external quality assurance (EQA) study on WNV molecular detection.

Materials and Methods

participants

Thirty laboratories from 18 countries participated, including 11 European, 2 Middle Eastern, 4 North or South American, and 1 African; a complete list is given in the Acknowledgements section.

preparation of test samples

Representative WNV strains of genetic lineages 1a, 1b, and 2 and heterologous flaviviruses were obtained from Vero cell cultures 4 days after infection. Strains comprised WNV-NY99 (GenBank accession number DQ211652), WNV-Pan001 (GenBank accession number AY268132), WNV-Kunjin (M.N., personal observations), and WNV-Ug37 (10).

Supernatants were heated to 56 °C for 1 h and gamma-irradiated with 30 kGy. Infectivity was excluded by reinoculation in Vero cell cultures, with 3 subsequent passages. Test samples were generated by diluting inactivated virus stock solutions in human fresh-frozen plasma tested and confirmed to be negative for HIV-1, hepatitis B virus, hepatitis C virus, and WNV. After dilution, enriched plasma was divided into aliquots, frozen, and lyophilized. Samples from each lot were redissolved and quantified in triplicate by real-time RT-PCR (RealArt West-Nile Virus, Qiagen Diagnostics).

Three additional samples containing mixtures of heterologous flaviviruses were also included. Mixing was done to challenge the specificity of WNV assays with as many strains as possible. Sample 1 contained dengue viruses (DenV) types 1–4 (ATCC VR-344, VR-345, VR-1256, and VR-1257). Sample 2 contained yellow fever virus vaccine strain 17D and tick-borne encephalitis virus strain K23 (11). Sample 3 contained St. Louis encephalitis virus (ATCC VR-1265) and Japanese encephalitis virus (ATCC SA14-14-2). These samples were processed and inactivated with the method described for the WNV test samples. Quantification was done by published (12) and unpublished in-house real-time RT-PCR assays. The contents and concentrations of individual samples are listed in Table 1 .

Table 1.

Results by laboratory.

Code	Strain	RNA, copies/mL1	Laboratory
						1	22	3	4	5	62	7	82	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28	29	30	Total
2	WNV-NY99	1.8E7	+3	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	–	29
5	WNV-PaAn001	5.0E6	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	–	+	+	+	+	+	–	+	+	+	+	+	+	–	–	26
9	WNV-NY99	1.8E6	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	–	–	+	–	27
1	WNV-Ug37	1.4E6	+	+	+	+	+	+	+	+	+	–	–	–	–	–	–	+	–	+	–	–	–	–	–	–	–	–	–	+	+	–	13
12	WNV-NY99	1.8E5	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	+	–	–	–	–	–	–	–	–	22
4	WNV-NY99	1.8E4	+	+	+	+	+	+	+	+	–	+	+	+	+	+	+	+	+	–	+	+	–	+	–	–	–	–	–	–	–	–	19
6	WNV-Kunjin	2.6E3	+	–	+	+	+	–	+	+	+	+	–	–	–	–	–	–	–	+	–	+	–	+	–	–	–	–	–	–	–	–	11
10	SLEV/JEV4	2E8/2E7	–	–	S	S	–	–	–	–	–	+	–	–	–	–	–	–	–	–	–	+	–	–	–	–	–	–	–	–	–	–	2
11	YFV/TBEV4	1E6/1E6	–	–	–	Y	–	–	F	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	1
3	DenV 1–45	2E6–2E8	–	–	D	D	–	–	D	D	–	–	–	–	–	–	–	–	–	D	–	–	–	–	D	–	–	–	–	+	+	–	2
7	Negative	0	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	+	–	–	–	–	–	–	–	–	–	–	–	1
8	Negative	0	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	–	0

Confirmed by real-time PCR in triplicate. Samples 2, 9, 12, and 4 constitute a dilution series: their concentrations are provided as expected upon dilution factors, but they have also been confirmed by real-time RT-PCR.

These participants used commercial RT-PCR assays (Revertal, Qiagen).

+, West Nile virus detected. Viruses other than West Nile virus detected: S, St. Louis encephalitis virus; Y, yellow fever virus; D, dengue virus; F, unspecified flavivirus.

SLEV, St. Louis encephalitis virus; JEV, Japanese B encephalitis virus; TBEV, tick-borne encephalitis virus.

RNA copies in detail: DenV-1, 1.5E8/mL; DenV-2, 2.0E8/mL; DenV-3, 2.0E6/mL; DenV-4, 2.2E8/mL.

Results

Lyophilized test samples were shipped at ambient temperature to the participating laboratories. Each participant received a coded panel of 7 samples containing WNV at 2 600 to 18 000 000 RNA copies/mL, 3 samples containing heterologous flaviviruses, and 2 negative samples (Table 1 ). Participants were asked to use their routine setup of molecular assays for specific diagnosis of acute WNV infections to analyze the material but were not asked to detect or type other flaviviruses.

For each individual sample, we first determined what fraction of the 30 participating laboratories detected virus (Fig. 1 ). These fractions were plotted against the respective virus concentrations in each sample and analyzed by probit analysis (n = 30 replicate datum points per concentration). With the exception of 1 outlier sample, the positivity rate of each sample corresponded exactly with the concentration-dependent response rates calculated by a dose–response model (probit analysis, P ≤ 0.0001). When the outlier was eliminated, the hypothetical average laboratory achieved 50% detection probability from 7762 copies of RNA/mL of sample onward (95% CI, 1174–24547 copies/mL). The outlier sample was the only member of the panel belonging to WNV genetic lineage 2 (strain WNV-Ug37). For this sample, the hit rate was 43.3%, less than half the rate predicted by the regression model. In 10 laboratories, the lineage 2 sample could not be detected, although a lineage 1 sample of 10 times less virus (1.8 × 105 copies/mL) tested positive. For WNV-Kunjin, which is an outlier within lineage 1 (designated lineage 1b), a genotype bias was not observed. The WNV-Kunjin detection rate of 36.7% corresponded well with the dose–response model, because it had a low virus concentration of 2600 copies/mL (Fig. 1 ). It should be mentioned, however, that we did not make a direct comparison of detection rates for genotype 1a. In addition, no significant differences were observed between the detection rates of the North American NY99 WNV prototype strain and the WNV PaAn001 strain that was isolated from an animal in France in 2000 (2)(13). Both strains belong to lineage 1a.

Figure 1.

Probit analysis of the fractions of laboratories achieving a positive result (y axis) in relation to the virus RNA concentration in a given positive sample (x axis).

Data points represent individual samples in the proficiency test panel. Samples without legend, WNV-NY99 and WNV-PaAn001 (lineage 1a); 1b, WNV-Kunjin (lineage 1b); 2, WNV-Ug37 (lineage 2). The thick line is the regression line calculated on the basis of a Probit analysis (dose–response model); the thin lines are 95% confidence intervals. Data fit into the model with P ≤ 0.0001. Sample 2 was excluded from Probit analysis as an outlier.

To appraise the performance for each individual laboratory, we defined 2 proficiency criteria. First, the 6 samples containing 1.8 × 104 or more copies of viral RNA/mL had to be correctly detected as positive (Table 1 ). This concentration was chosen because it was well within the detection interval of published and commercial RT-PCR assays for WNV (e.g., (4)(14)(15). Second, no false-positive results were allowed for the 2 negative samples. When we applied these criteria, only 8 of 30 laboratories (27%) passed the minimum requirements for successful participation. All other laboratories failed because of lack of sensitivity, not because of false-positive results (Table 1 ). Six of the 8 successful laboratories also detected the low copy number sample, containing WNV-Kunjin at 2600 copies/mL. Two laboratories missed the proficiency criteria only because the 1.8 × 104 copies/mL lineage 1 sample was not detected. Nine laboratories failed to detect the lineage 2 sample but detected all other samples correctly. One laboratory missed only 1 of the high-titered samples, an omission that may have been caused by a handling error. The remaining laboratories missed more than 1 sample. One laboratory provided no correct result at all.

Three samples in the test panel contained heterologous flaviviruses and 2 contained no virus (Table 1 ). Five of 30 laboratories generated false-positive WNV results for 1 of these samples, with no more than 1 false-positive test result per laboratory. Four of 5 false-positive results occurred in samples containing heterologous flaviviruses. Six of 30 laboratories correctly identified heterologous flaviviruses in the 3 virus-containing samples. A significant positive correlation existed between the capability of laboratories to detect heterologous flaviviruses and their ability to detect the lineage 2 sample (t-test, P <0.022).

In one of our earlier studies, ANOVA analysis identified the use of commercial test products for severe acute respiratory syndrome coronavirus as the only technical factor that had a positive influence on laboratory performance (16). When we applied the same statistical test to the current dataset, we did not identify a significant advantage, possible because of the small number of participants using commercial assays in this study (n = 3). Nevertheless, the users of commercial products were among the best performers, and all of them detected lineage 2 (Table 1 ). No significant influence on laboratory performance was found to be associated with the use of real-time PCR, column-based RNA extraction methods (Roche, Qiagen), automated RNA preparation, or 1 popular RT-PCR protocol (15).

Discussion

Molecular detection of WNV is becoming an increasingly important task. EQA is therefore necessary. Compared with our earlier EQA studies on emerging agents such as Ebola, Lassa, Pox, and severe acute respiratory syndrome viruses (16)(17), in this study the overall diagnostic performance for WNV appears disappointing. From a technical point of view, much lower detection limits, in the range of 100-1000 copies/mL, can be achieved in RT-PCR, and some of our participants will indeed reach this concentration. The average laboratory, however, was not as efficient, and only a small number of participants passed a rather easy set of proficiency criteria, a critical problem because maximum sensitivity is required in clinical cases of encephalitis, in blood screening, and in testing reservoir components such as mosquitoes.

On the other hand, the low success rate in our study was mainly caused by limitations in detecting WNV lineage 2. Without the lineage 2 sample, 60% of participants would have passed the proficiency criteria. Because lineage 1 is far more prevalent in general and in the Northern Hemisphere in particular, it is conceivable that many participants may have designed their assays according to their own geographic location, a situation that would be acceptable in most settings. Several other laboratories may have detected lineage 2 only by broad-range flavivirus assays, as suggested by the observed correlation between non-WNV flavivirus detection and lineage 2 results. In view of recent findings on the presence of lineage 2 WNV and potentially new lineages 3 and 4 in Europe (1)(18)(19), the importance of using broad-range assays is obvious. At least on the reference laboratory level, broad-range flavivirus assays should routinely be applied in parallel with specific WNV detection.

The present study shows that EQA is adequate and necessary for identifying shortcomings in diagnostic proficiency. For rare pathogens such as WNV, EQA furthermore provides critical virus material that is required to improve and adjust diagnostic assays. The samples used in this study are available from the European Network for Diagnostics of “Imported” Viral Diseases for future reference.

Nonstandard abbreviations: WNV, West Nile virus; RT-PCR, reverse transcription-PCR; EQA, external quality assurance.