First external quality assurance of antibody diagnostic for SARS-new coronavirus.

To confirm an infection with the new coronavirus (SARS-CoV) causing the severe acute respiratory syndrome (SARS) diagnostic assays for detection of SARS-CoV specific antibody are necessary. To evaluate the diagnostic performance of laboratories an external quality assurance (EQA) study was performed in 2004. Participating laboratories (9/20) correctly detected anti-SARS antibodies in serum samples without false positive results in an immunofluorescence assay. In contrast, only 4/13 laboratories detected most of the anti-SARS antibody positive samples without false positive results using enzyme immunoassays (EIA) and/or immunoblot. The overall results clearly demonstrate that serological diagnosis of SARS-CoV remains at an early stage of development, with further technical improvements required, particularly with respect to the use of SARS specific EIAs.

Diagnosis of severe acute respiratory syndrome (SARS) caused by a new coronavirus (SARS-CoV) is of major importance for assisting the control of any future SARS epidemic (Drosten et al., 2003, Ksiazek et al., 2003; Peiris et al., 2003a, Peiris et al., 2003b; Kuiken et al., 2003). The World Health Organization (WHO) helps laboratories all over the world to develop laboratory capability and a global reference network which was initiated and coordinated by WHO has assisted in the provision and evaluation of diagnostic tools for detection of SARS-CoV (WHO, 2004a, WHO, 2004b). Since the initial epidemic that started in November 2002 in south–east China and spread globally in 2003, at least three separate laboratory infection incidents have occurred with some secondary cases, demonstrating the need for constant vigilance and reliable diagnostic tools (WHO, 2004c; Heymann et al., 2004).

For the acute phase of SARS, RT-PCR methods pioneered by several laboratories, are the fastest and most sensitive method for detection of the viral genome, and appropriate for use in diagnosis in the first few days after illness onset. Significant variation is found in the virus load in different patients and clinical samples, which may relate to the severity of the disease, timing and quality of sampling (Peiris et al., 2003a, Peiris et al., 2003b). As a consequence, with current knowledge, a negative RT-PCR result cannot be interpreted to exclude SARS-CoV infection. The detection of antibodies against SARS-CoV is therefore currently a gold standard for confirmation of SARS infection (Wu et al., 2004). Serological assays based on fixed infected cells using immunofluorescence assays (IFA) or SARS-CoV virus specific enzyme immunoassays (EIA) have been developed in many different laboratories worldwide.

One way to assess preparedness of the diagnostic laboratories globally is through the conduct of an external quality assurance (EQA) program providing characterized serum specimens containing SARS-CoV specific antibody. In collaboration with the WHO, we distributed a panel of 15 samples consisting of anti-SARS-CoV positive and negative human sera (WHO, 2004a). The serum material obtained from SARS infected patients was provided from various countries where SARS-CoV cases occurred. Sera were checked for SARS-CoV infectivity by inoculation in Vero cell cultures (three passages) and by RT-PCR. The sera were diluted in human freshly frozen plasma negative for HIV-1, HBV, HCV and heated to 56 °C for 1 h before lyophilization, for easy distribution and were tested again by two reference laboratories for specific activity and unspecific binding.

To analyze the sensitivity of different assays, the EQA panel consisted of sequential serum dilutions, resulting in samples with low and high antibody titers against SARS-CoV as well as definite SARS-CoV negative samples. The positive sera were from patients from China, UK and Germany with a clear clinical diagnosis of SARS. All positive sera were conscientiously analyzed by expert laboratories from China and Germany for their anti-SARS reactivity by IFA, EIA and in neutralization tests. The negative samples consist of human plasma collected before the SARS outbreak in Germany and were not found reactive with any of the SARS specific diagnostic assays. For analysis of the specificity, two sera with known nonspecific reactivity against cell and mitochondrial structures were included.

Here we present the results of the first EQA study on SARS-CoV serological diagnosis. The study was intended as a diagnostic proficiency test, including certification and publication of anonymized results. Thirty laboratories from 18 countries (15 European/Middle Eastern, 11 Australian/Asian/Oceanian, 3 American, 1 African) participated (see later). The participating institutes included members of the international WHO SARS reference and verification laboratory network, as well as national and regional SARS reference laboratories and three commercial laboratories (WHO, 2004b).

Each participant received a coded panel of 15 samples distributed by normal mail. Samples were re-suspended in 100 μl distilled water before use. The participants were asked to analyze the material with the diagnostic assays in use for SARS-CoV diagnosis. Information on the type/format of the test methods used (IFA, EIA, immunoblot, neutralization assay) and whether it was an in-house assay or a commercial kit, was also collected.

The following two criteria were chosen as minimum requirements for overall proficiency: (i) Laboratories had to detect four of the five positive samples correctly (80%); (ii) no false positive results were allowed. Indeterminate results in positive or negative samples were not considered for evaluation.

About 52% (13/25) of the laboratories passed the minimum requirements for successful performance (Table 1, Table 2 ). Three laboratories were not considered for evaluation because of failure to process all EQA samples. The failure of the other 48% of the laboratories to achieve minimum proficiency standards was due to lack of sensitivity, and/or to false positive results. From 20 laboratories using IFA, 15 out of 17 (88%) detected the high positive sera (#17, #20) satisfactorily. The performance was worse for low titer sera (#11, #23, #25) where only 9 out of 17 (53%) detected SARS antibodies. Eight of the 20 laboratories had general problems regarding specificity, detecting negative samples as positive in IF assays, possibly due to inadequate internal positive and negative controls. It is notable that 8/11 laboratories with a good performance used a commercial assay, compared with 4/10 with a poor performance.

Table 1

SARS-CoV external quality assurance—results immunofluorescence assay

Lab. number	Sample number											Correct results in %
	#17 pos.	#20 pos.	#11 pos.	#23 pos.	#25 pos.	#8 neg.	#5 neg.	#19 neg.	#2 neg.	#6 neg.a	#12 neg.b
11bc	+	+	+	+	+	−	−	−	−	−	−	100
12c	+	+	+	+	+	−	−	−	−	−	(+/−)	91
3	+	+	+	+	(−)	−	−	−	−	−	−	91
20c	+	+	+	(+/−)	(+/−)	−	−	−	−	−	−	91
14c	+	+	+	+	(−)	−	−	−	−	−	−	91
5b	+	+	+	(−)	+	−	−	−	−	−	−	91
19	+	+	+	+	(−)	−	−	−	−	−	−	91
23c	+	+	(−)	+	+	−	−	−	−	−	−	91
4bc	+	+	(+/−)	(−)	+	−	−	−	−	−	−	86
30bc	+	+	(−)	(−)	+	−	−	−	−	−	−	82
22c	+	+	(−)	+	+	−	−	−	−	−	(+)	82
9bc	+	(−)	(−)	+	(−)	−	−	−	−	−	−	73
27c	+	+	(−)	(−)	(−)	−	−	−	−	−	−	73
28	(−)	+	+	(−)	+	(+)	−	−	−	−	−	73
8	+	+	+	+	(−)	−	−	−	−	(+)	(+)	73
7	+	+	+	+	+	−	−	(+)	−	(+)	(+)	73
25	(+/−)	(+/−)	(+/−)	(+/−)	(−)	−	−	−	−	(+)	−	73
15	(−)	(−)	(−)	(−)	(−)	−	−	−	−	(+)	−	45
24	+	+	+	+	+	(+/−)	(+/−)	(+)	(+/−)	(+/−)	(+)	45
21c	(+/−)	(+/−)	(−)	(+/−)	(+/−)	−	(+/−)	(+)	−	(+/−)	(+)	36
17bc	+	+	n.d.	(+/−)	+	n.d.	n.d.	n.d.	n.d.	n.d.	n.d.
6	n.d.	n.d.	(−)	n.d.	n.d.	−	−	n.d.	−	−	−
1	n.d.	n.d.	(−)	n.d.	n.d.	−	−	n.d.	−	−	−

Pos.: serum with antibodies directed against SARS-CoV; neg.: serum without antibodies against SARS-CoV; +: correct positive result; −: negative result; (+/−): indefinite; (+): false positive result; (−): false negative result; n.d.: not done.

Serum with antibodies reactive with cell core structure.

Serum with antibodies reactive with mitochondria.

Use of commercial assays. Except for 9b and 17b only the IFA from EUROIMMUN was used.

Table 2

SARS-CoV external quality assurance—results EIA

Lab. number	Sample number											Correct results in %
	#17 pos.	#20 pos.	#11 pos.	#23 pos.	#25 pos.	#8 neg.	#5 neg.	#19 neg.	#2 neg.	#6 neg.a	#12 neg.b
11ac	+	+	(+/−)	+	+	−	−	−	−	−	−	95
4ac	+	+	+	(−)	+	−	−	−	−	−	−	91
18d	+	+	+	(−)	(+/−)	−	−	−	−	−	−	86
9ac	+	+	(−)	(+/−)	+	−	−	−	−	−	−	86
10c	+	(+/−)	(+/−)	(+/−)	(+/−)	−	−	−	−	−	−	82
30ac	+	+	+	(−)	+	(+/−)	−	−	(+/−)	−	−	73
29d	+	+	+	+	+	(+)	(+)	(+)	(+/−)	−	(+/−)	73
16c	(−)	+	(−)	(−)	(−)	−	−	−	−	−	−	64
26	+	+	+	+	+	(+)	−	(+/−)	−	(+)	(+)	55
5a	(−)	(−)	(−)	(−)	(−)	−	−	−	−	−	−	55
13	+	+	n.d.	(−)	+	n.d.	n.d.	−	n.d.	n.d.	n.d.
17ac	+	+	n.d.	+	+	n.d.	n.d.	−	n.d.	n.d.	n.d.
2	+	+	+	+	+	n.d.	(+)	(+)	(+)	(+)	(+)

Pos.: serum with antibodies directed against SARS-CoV; neg.: serum without antibodies against SARS-CoV; +: correct positive result; −: negative result; (+/−): indefinite; (+): false positive result; (−): false negative result; n.d.: not done.

Serum with antibodies reactive with cell core structure.

Serum with antibodies reactive with mitochondria.

Use of commercial assays.

Use of in-house immunoblot assay.

Six out of 10 laboratories (60%) performing EIA detected the high positive sera (#17, #20) satisfactorily, whereas only one laboratory (10%) detected the low positive serum (#23), in comparison to IFA where 55% of laboratories detected this sample. One laboratory using an EIA based on SARS specific peptides showed high reactivity with all samples (Table 2). A similar result was obtained with other peptide based EIAs for SARS evaluated with these samples. This could be only avoided with real serum specimens (data not shown).

In general the results of the laboratories using more than one diagnostic assay were better than of the other laboratories. Five of the 23 laboratories using IFA assay for SARS diagnostic also performed an EIA for confirmation of the results. Four of five laboratories with the best performance used commercial assays or more than one in-house assay.

In conclusion, the results of this first EQA study on SARS-CoV antibody diagnosis indicates the need for performance improvement in a significant number of laboratories undertaking serological diagnosis of SARS-CoV. Since only few laboratories have class 3 safety laboratory facilities and the experience for developing and evaluating in-house assays, commercial assays may be more suitable for SARS serology for laboratories with a low volume of testing. These results are also likely to be the result of the fact that limited international work had been performed or published on the technical development of the assays in use during the time this EQA was performed. A follow up EQA in 12–18 months time would be expected to show substantial improvement as laboratories have a chance to develop both the performance of in-house assays and refine testing strategies.

Only a few laboratories were able to develop their own serological assays, partly because of the constraints imposed by the laboratory handling of SARS-CoV virus, in contrast to the ease with which RT-PCR protocols were disseminated and implemented. Also a variety of different assay formats (e.g. IFA, EIA, IB) and antigen sources (e.g. purified virus, recombinant SARS protein, SARS specific peptides), contributes to the variability in performance. In this regard, the strategy of testing with peptide based assays as a single test should be avoided, since it appears from the laboratories using this approach there is a lack of specificity. Diagnostic algorithms for serological diagnosis of SARS require refining.

The approach of sharing protocols and resources for serology testing should be further promoted to guarantee dissemination of high quality assays to ensure the timely serological confirmation of suspected SARS cases. In the meantime few commercial assays, most of them EIAs are distributed or under development (Chang et al., 2004, Guo et al., 2004, He et al., 2004). The evaluation of these assays based on the accessibility of well-characterized SARS-CoV positive sera is still a major problem for all laboratories and companies working on this task.

In the WHO recommended diagnostic strategy, seroconversion is required to diagnose acute infection. For establishing serostatus, while sensitivity is very important for screening, due to the very low seroprevalence worldwide, sequential testing using various different antigens/methodologies is required to get acceptable positive predictive value. The present EQA could not take into account this objective of testing or different testing strategies; therefore it is difficult to assess the proficiency of each participant for acute diagnosis or serostatus determination.

When testing for seroconversion, using paired sera, the sensitivity of the assays might be less important than a high specificity to avoid false positive results. Nevertheless highly sensitive and good specific serology assays will help to improve the diagnostic performance of the laboratories with a clear benefit for the patient. The EQA performed for SARS antibody diagnostic proved a clear advantage for all participants, either by demonstrating an adequate performance quality or demonstrate the need for necessary technical improvements or the training of the laboratory personal.