Detection of Myxoma Virus in the Classical Form of Myxomatosis Using an AGID Assay: Statistical Assessment of the Assay's Diagnostic Performance.

INTRODUCTION: The aim of the study was to estimate the diagnostic sensitivity (DSe) and specificity (DSp) of an agar gel immunodiffusion (AGID) assay for detection of myxoma virus (MYXV) in the classical form of myxomatosis and to compare its diagnostic performance to that of molecular methods (IAC-PCR, OIE PCR, and OIE real-time PCR).
MATERIAL AND METHODS: A panel of MYXV-positive samples of tissue homogenates with low (1 PCR unit - PCRU) and high (3,125 PCRU) virus levels and outbreak samples were used for method comparison studies. The validation parameters of the AGID assay were assessed using statistical methods.
RESULTS: The AGID attained DSe of 0.65 (CI95%: 0.53-0.76), DSp of 1.00 (CI95%: 0.40-1.00), and accuracy of 0.67 (CI95%: 0.55-0.76). The assay confirmed its diagnostic usefulness primarily for testing samples containing ≥3,125 PCRU of MYXV DNA. However, in the assaying of samples containing <3,125 PCRU of the virus there was a higher probability of getting false negative results, and only molecular methods showed a 100% sensitivity for samples with low (1 PCRU) virus concentration. The overall concordance of the results between AGID and IAC-PCR was fair (ĸ = 0.40). Full concordance of the results was observed for OIE PCR and OIE real-time PCR when control reference material was analysed.
CONCLUSIONS: Findings from this study suggest that AGID can be used with some limitations as a screening tool for detection of MYXV infections.

Introduction

Since the 1950s, when the first cases of myxomatosis were reported in wild rabbits in Europe and Australia, myxoma virus (MYXV) has been circulating constantly in wild and farmed populations of lagomorphs, causing outbreaks of the disease (5). Although the traditional antigen-based serological assays used in viral diagnostics have been gradually replaced by molecular methods, they are still often used in laboratories due to their low costs of analysis and simplicity of execution. Among them there is the agar gel immunodiffusion (AGID) assay, still listed in the OIE manual as a recommended method for MYXV detection (14). This assay has been routinely used for virus detection over many years, although its diagnostic performance is not wholly known. The aim of this study was to estimate the diagnostic sensitivity (DSe) and specificity (DSp) of AGID and assess the assay’s diagnostic usefulness in comparison to the PCR-based methods currently used for MYXV detection.

Material and Methods

Reference material, tissue homogenates, and outbreak samples. Twelve MYXV-positive or negative tissue homogenates prepared from rabbit skin collected from animals which died of nodular myxomatosis were used as control reference material. The presence or absence of viral DNA in all samples was confirmed by the end-point PCR and real-time PCR methods in the OIE Reference Laboratory for Myxomatosis, Brescia, Italy, according to the Manual of Diagnostic Tests and Vaccines for Terrestrial Animals (14). A panel of 40 positive samples was used for comparison studies of AGID and PCR with an internal amplification control (IAC-PCR). These positive samples were 10% tissue homogenates of rabbit skin containing the MYXV Lausanne strain (ATCC VR-115) with low (1 PCR unit – PCRU) and high (3,125 PCRU) virus levels. Anti-MYXV positive and negative rabbit sera were also used in the AGID assay. They were obtained from rabbits vaccinated against myxomatosis and subsequently challenged with a virulent MYXV strain (14) or from blood of these animals drawn before vaccination (as the negative control) and belonged to the reference diagnostic material collection of the National Reference Laboratory for Rabbit Myxomatosis at the National Veterinary Research Institute in Puławy, Poland. In order to compare the diagnostic performance of the AGID and IAC-PCR methods, 23 homogenates of skin samples collected during field outbreaks from rabbits showing signs and lesions of classic nodular myxomatosis were examined.

Detection of MYXV DNA and quantification of the virus using IAC-PCR. The virus quantity in PCRU present in the tissue homogenates used for the preparation of the panel of negative and positive samples was determined based on the PCR results of tenfold serial dilutions of MYXV DNA. The lowest dilution of viral DNA giving a positive signal on gel was designated as 1 PCRU content. Viral DNA was extracted from 200 μL of tissue homogenates using a DNeasy Blood & Tissue Kit (Qiagen, Hilden, Germany) according to the manufacturer’s instructions and subsequently detected using a previously described IAC-PCR method (10). For each set of the tested samples a panel of controls was included to monitor the performance of the MYXV DNA isolation and amplification steps. The positive DNA extraction control was a suspension of cell culture containing the MYXV Lausanne strain and the negative extraction control was water used instead of the sample DNA, the reaction mixture control was PCR mixture without a DNA template, and the environmental control was an open tube during the addition of the DNA sample.

AGID. The AGID test was performed according to the Manual of Diagnostic Tests and Vaccines for Terrestrial Animals (14). The analytical sensitivity of the assay was determined by testing serial twofold dilutions of tissue homogenate containing the MYXV Lausanne strain. Briefly, 30 μL of rabbit tissue homogenates and rabbit sera positive and negative for anti-MYXV antibodies were added to each individual well. Homogenates of rabbit skin with myxomas and tissue without lesions comprised the samples for testing and control samples. The appearance on the agar gel of the characteristic precipitation lines indicated the presence of MYXV antigen.

Statistical analysis. Assessment of the diagnostic value of the AGID assay in comparison to that of the IAC-PCR (10) and OIE PCR (2) was carried out. It was based on the determination of its DSe and DSp with confidence intervals related to assay uncertainty. The accuracy (AC) of AGID, i.e. its ability to correctly identify positive and negative samples, was estimated using a panel of MYXV-positive and -negative samples and samples of skin collected from rabbits showing clinical signs and lesions of myxomatosis. AGID DSe and DSp were calculated using the extent of the agreement between the AGID-yielded proportions of positive (PA) and negative (NA) results and the respective total numbers of positive and negative samples with subsequent determination of confidence intervals (CI) by the Clopper–Pearson method. The measures of result agreement for the set of samples when analysed using both the AGID and IAC-PCR methods were expressed as Kappa (ĸ) values, which for AGID were obtained for dichotomous data (presence or absence of the virus) in all tested samples (6, 13) and compared with those obtained for the IAC-PCR method (10). If ĸ is equal to 1 then very good (complete) agreement is observed. Kappa values within the ranges of 0.00–0.20, 0.21–0.40, 0.41–0.60, and 0.61–0.80 signify poor, fair, moderate, and good agreement strength, respectively (11). The concordance of the DSe and DSp parameters of the assessed methods was also compared by finding the chi-squared values (χ2) for the McNemar test at a ≤0.05 significance level (3). The DSe and DSp were estimated from the results obtained for the animal tissue samples with unknown virus concentration using Bayesian inference (7). These parameters were expressed as the mode and confidence intervals shown as the percentile values of a ß-distribution with 0.025 for the lower and 0.975 for the upper limit (6). Additionally, negative predictive values (NPV) for the AGID and IAC-PCR methods were estimated.

Results and Discussion

The diagnosis of the classic nodular form of myxomatosis generally is not difficult and is based on clinical investigation and the presence of the characteristic skin lesions in affected animals. Nevertheless, in some cases, laboratory testing is required to confirm MYXV infection. The methods used in viral diagnostics should be validated to allow confidence of obtaining the most reliable results. Apart from the detection limit, the most significant validation parameters of each method are the DSe and DSp (13). In particular, the DSe value is important for a qualitative method such as AGID because it determines the assay’s ability to detect a pathogen in the sample.

In this study, the results obtained for samples tested by AGID assay and IAC-PCR depended on the virus concentration in the sample. When samples of rabbit skin collected from myxomatosis outbreak victims were analysed, 18 out of 23 samples gave positive results in AGID (78%, CI95%: 56–93). The presence of MYXV DNA was shown in all samples tested by IAC-PCR (Table 1). Full concordance of the results obtained in AGID and PCR-based methods (OIE-PCR and OIE real-time PCR) was observed for control reference materials. In addition, full result concordance (100%, CI95%: 83–100) was also demonstrated for tissue homogenates of rabbit skin containing at least 3,125 PCRU of MYXV DNA. Only samples containing 1 PCRU of MYXV DNA gave false negative results in AGID (0%, CI95%: 0–17). The assay’s DSe and DSp were estimated based on the results obtained for all samples tested in AGID. The AGID DSe was 0.65 (0.53–0.76) and DSp 1.00 (0.40– 1.00); AC of 0.67 (0.55–0.76) was also attained. In the case of the IAC-PCR method, these parameters were 0.98 (0.91–1.00) as the DSe and 0.95 (0.84–0.99) as the DSp (10). Direct comparison of method parameters showed that AGID gave only half the DSe of the molecular methods for samples containing low virus concentration. It is highly probable that the DSp of the OIE-PCR and OIE real-time PCR methods is comparable to that of the AGID assay, however, due to the low number of negative samples it could not be accurately estimated, as indicated by the wide confidence interval (0.40–1.00) for the AGID DSp.

Table 1

Results of samples tested using AGID and molecular methods with confidence intervals (CI) determined by the Clopper–Pearson method

Molecular methods/ type of samples	Number of tested and PCR-positive/negative samples	AGID-confirmed results	Assessed results* (CI 95%)	References
IAC-PCR (3,125 PCRU DNA of MYXV/sample)	20	20	1.00 (0.83–1.00)	10
IAC-PCR (1 PCRU DNA of MYXV/sample)	20	0	0.00 (0.00–0.17)
IAC-PCR (rabbit skin homogenates)	23	18	0.78 (0.56–0.93)	This study
OIE PCR and OIE real-time PCR (control reference material)	8	8	1.00 (0.63–1.00)
OIE PCR and OIE real-time PCR (control reference material without MYXV)	4	4	1.00 (0.40–1.00)
All positive (with MYXV)	71	46	0.65 (0.53–0.76)
All samples in this study (accuracy)	75	50	0.67 (0.55–0.76)

Median values obtained from proportions of confirmed results in AGID to the total number of samples for which test results agreed positively and negatively

Analysing ĸ values for the AGID and IAC-PCR, the highest result concordance between methods was obtained for samples containing at least 3,125 PCRU of the virus (ĸ = 1.00) (10). Only IAC-PCR showed 100% sensitivity for samples with low (1 PCRU) virus concentration. None of the tested samples from this pool was positive in AGID. However, when the results of samples containing low and high virus levels were considered together, the concordance between the methods was fair (ĸ = 0.40). The reliability of the results was confirmed by a P-value (χ2 < 0.0001) indicating that the number of samples used for comparison of the methods was sufficient. The observed difference in the sensitivity of the AGID and IAC-PCR methods was only 3,125 PCRU. This finding could explain why AGID is still included in the panel of OIE methods used for disease detection. It is noteworthy that AGID is also widely employed in the diagnosis of other viral diseases of animals mainly as a screening tool (1, 8, 9). It appeared to be a sensitive assay in detection of Marek’s disease virus (MDV) in clinically affected and dead birds from disease outbreak flocks, in the testing of which no significant difference between PCR and AGID in the detection of MDV was observed (8).

The probability of getting the correct results with the particular assay used is indicated by NPVs, which are related to the method’s DSe. When AGID NPVs are analysed, it could be assumed that the number of false negative results obtained will be higher for samples with different virus concentrations (low and high viral load). With the increase in the number of true positive samples tested in AGID, especially those with levels of MYXV lower than 3,125 PCRU, the percentage of false negative results will also increase. For example, assuming that 40% of tested samples will contain the virus (the probability of infection), the probability of getting a false negative result using AGID will be 20% (Fig. 1). Using the IAC-PCR method to test these samples, no false negative results will be obtained, and all virus positive samples will be correctly identified. Due to a lack of samples originating from rabbits with the atypical (amyxomatous) form of the disease, it was not possible to assess the diagnostic value of AGID for rabbit tissues other than skin samples. As in the typical nodular form of the disease, in the atypical form the virus was also frequently recovered from the eyelids besides the lungs and liver (12). The skin lesions characteristic for myxomatosis were missing; however, oedematous swelling of the eyelids, scrotum, and anorectal area was observed (4). Because the highest concentration of the virus mainly occurs in rabbit skin with myxomas, testing of other tissues using an assay with a lower sensitivity than molecular methods may give false negative results.

Fig. 1

Comparison of NPV results obtained for AGID and IAC-PCR. The dashed red line indicates predicted NPV for AGID; the blue line indicates estimated NPV for IAC-PCR

In conclusion, the data from this study suggest that AGID can be used as a screening tool for detection of MYXV infections. Its simplified procedure could be particularly useful in poorly equipped laboratories. However, due to the low DSe value compared to molecular methods, its application for virus detection in samples collected from animals with mildly manifested symptoms of the disease may produce false negative results.