Serological diagnosis of avian influenza in poultry: is the haemagglutination inhibition test really the 'gold standard'?

BACKGROUND: The serological diagnosis of avian influenza (AI) can be performed using different methods, yet the haemagglutination inhibition (HI) test is considered the 'gold standard' for AI antibody subtyping. Although alternative diagnostic assays have been developed, in most cases, their accuracy has been evaluated in comparison with HI test results, whose performance for poultry has not been properly evaluated.
OBJECTIVE: The objective of this study was to estimate the diagnostic sensitivity (Se) and specificity (Sp) of the HI test and six other diagnostic assays for the detection of AI antibodies without assuming a gold standard.
METHODS: We applied a Bayesian version of latent class analysis to compare the results of multiple tests from different study settings reported in the literature.
RESULTS: The results showed that the HI test has nearly perfect accuracy (i.e. 98·8% sensitivity and 99·5% specificity). It performed well in both chickens and turkeys and yet was less accurate in experimentally infected poultry, compared to naturally infected. Blocking ELISA and the indirect immunofluorescence assay also performed very well.
CONCLUSIONS: Given its very high Se and Sp, the HI test may be effectively considered a gold standard. In the framework of LPAI surveillance, where large numbers of samples have to be processed, the blocking ELISA could be a valid alternative to the HI test, in that it is almost as sensitive and specific as the HI test yet quicker and easier to automate.

Introduction

The majority of avian influenza (AI) infections are caused by low pathogenicity avian influenza viruses (LPAIV) and appear as mild respiratory diseases. However, LPAIV subtypes H5 and H7 can mutate into highly pathogenic avian influenza viruses (HPAIV), outbreaks of which can threaten human health and cause huge economic losses, given the high mortality in poultry and the cost of control measures.

The most effective means of identifying and controlling AI viruses in poultry is a constant and global surveillance. The surveillance of LPAIV infection aims to detect the causative agent (i.e. the replicating virus or viral RNA) or antibodies against AI viral proteins. Generally speaking, LPAIV or its genome can be detected in an individual bird for only a few days, depending on the virus strain, the bird species, the infectious dose and the method of detection, whereas LPAIV antibodies are often present for the entire production life of the infected poultry. Moreover, LPAIV infection is often clinically asymptomatic.

It follows that, in areas at high risk of AI exposure, adequate surveillance based on the serological detection of LPAIV antibodies is of vital importance for the early detection of LPAIV and, consequently, the prevention of mutation into HPAIV. Although serological LPAIV surveillance can be performed using various diagnostic assays, the haemagglutination inhibition (HI) test, which detects antibodies to the hemagglutinin (HA) antigen, is considered to be the ‘gold standard’ for AI antibody subtyping, and it is recommended by both the European Union (EU) and the World Organization for Animal Health. However, the HI test is quite laborious because it needs manual reading of the results. For this reason, alternative assays have been developed. The accuracy of these assays has in most cases been evaluated in reference to the HI test. However, to the best of our knowledge, only one evaluation of the HI test in poultry has been published, and this evaluation was based on a small dataset, so that there were wide ranges for the estimated sensitivity (Se) and specificity (Sp) (95·7−100% for Se and 59·3−99·6% for Sp).

The Se and Sp of a test are usually estimated by comparison with a reference test (i.e. the ‘gold standard’), which is supposed to determine the true disease state of the animals unambiguously. However, the true disease state is rarely known in practice. An alternative way of evaluating diagnostic tests when the infection status is unknown is latent class analysis. This approach is based on the analysis of multiple populations with different disease prevalences, to obtain estimates of Se and Sp of two (or more) tests without requiring a gold standard.

The objective of this study was to estimate the diagnostic Se and Sp of various serological assays for detecting AIV antibodies – with emphasis on the HI test – without assuming a gold standard. To do so, we applied a Bayesian version of latent class analysis.

Materials and methods

Data collection

To determine which assays would be included in the analysis, we considered studies published in peer‐reviewed journals in which serological assays for detecting AIV antibodies in poultry were evaluated. We initially identified a number of such studies by interviewing experts, performing a PubMed search and looking at the reference lists of previously identified published papers. To be included in our analysis, the studies had to have included the HI test and to have clearly reported test results in 2 × 2 tables; we included only studies based on serum samples (i.e. no egg yolk) and with results for individual bird species (i.e. no mixed results from multiple species).

We initially identified 36 studies (see Appendix S1). In eight of these studies, the results of the HI test were compared with those of another AI serological test. However, two of these studies were discarded because it was not possible to distinguish the results for individual species or the results based on serum samples. The main features of the six selected studies are summarized in Table 1. Four studies were conducted on chickens alone, one on turkeys alone, and one on chickens, turkeys and ducks, yet for the latter study, we did not consider the duck population, for the sake of homogeneity among studies.

Table 1

Main features of the studies included in the comparative analysis

Study ID	Country	Avian species	Developed test	Target protein	Type of test	Source of infection	Tested AIV subtypes	HI antigen versus tested antigen	Reference
A	Italy	Turkey	iIFA	NA	N1‐specific	Natural	H7N1	Not indicated	15
B	Taiwan	Chicken	bELISA*	HA	H5‐specific	Natural	H5N3	Heterologous	16
C	Taiwan	Chicken	bELISA*	HA	H6‐specific	Natural	H6N1	Homologous	17
D	Ohio	Chicken, turkey, duck**	cELISA*	HA	H5‐specific	Experimental	H5N1, H5N2, H5N3, H5N9	Heterologous	14
E	China	Chicken	NP‐ELISA	NP	Type A‐specific	Experimental	H5N1, H9N2	Not indicated	18
F	Japan	Chicken	AGP	NP	Type A‐specific	Natural	H5N2	Homologous	9

NA, neuraminidase; HA, hemagglutinin; NP, nucleoprotein.

*Competitive and blocking ELISA tests under investigation used monoclonal antibodies.

**For the sake of comparison, we considered only chicken and turkey populations.

In all of the six studies included in our analysis, the HI test has been performed according to the protocol prescribed by the OIE Diagnostic Manual (i.e. using four hemagglutinin units of virus antigen and 1% chicken erythrocytes, diluted in PBS), and titres ≥ 1:16 were considered positive. The diagnostic assays that were compared to the HI test in the six studies were indirect immunofluorescence assay (iIFA), blocking enzyme‐linked immunosorbent assay (bELISA), competitive ELISA (cELISA), nucleoprotein‐based specific indirect ELISA (NP‐ELISA) and agar gel precipitation (AGP). Given that the authors of the study on cELISA pointed out that this assay performs quite differently in chickens and turkeys, we considered it two separate tests (herein referred to as ‘cELISA_C’ for chickens and ‘cELISA_T’ for turkeys). Thus, seven different tests were considered in the analysis (Table 2).

Table 2

Cross‐tabulated test results included in the analysis. Joint tests outcome (y) is coded as 1 = positive, 0 = negative. cELISA is assumed to perform differently for chickens and turkeys

			Serological tests		Combination of test results
k	Study ID	n k			y11	y10	y01	y00
1	A	247	T1 = HI	T2 = iIFA	105	2	6	134
2	B	478	T1 = HI	T3 = bELISA	232	4	10	232
3	C	400	T1 = HI	T3 = bELISA	184	0	6	210
4	D	172	T1 = HI	T4 = cELISA_C*	95	39	14	24
5	D	94	T1 = HI	T5 = cELISA_T**	80	2	10	2
6	E	150	T1 = HI	T6 = NP‐ELISA	99	8	19	24
7	F	114	T1 = HI	T7 = AGP	64	32	0	18

*cELISA_C = cELISA used for chickens.

**cELISA_T = cELISA used for turkeys.

Brief description of the studies included in the analysis

Study A aimed to develop and validate a N1‐specific iIFA test to be used in differentiating vaccinated and naturally infected birds. The gene coding for the NA protein was extracted from A/Turkey/Italy/4426/2000 H7N1 LPAI virus. To validate the iIFA test and its discriminatory ability, a number of turkey field sera were tested, referring the HI test as to the gold standard. Sera came from H7N1 naturally infected turkeys, H7‐negative animals and H7N3‐vaccinated birds. Given that the purpose of our study was to evaluate the Se and Sp of the iIFA test in detecting N1‐antibodies (and not its discriminatory ability towards N3‐antibodies), sera from H7N3‐vaccinated birds were not included in our analysis, because they would have generated bias (being positive to HI test but mainly negative to iIFA‐N1 test).

Authors of study B developed and validated a bELISA to detect H5 antibodies in chickens. The recombinant HA protein, derived from A/duck/Yunlin/04 H5N2 LPAI virus, was used to produce monoclonal antibodies and to coat the ELISA plates. Test validation was performed using field sera from chicken flocks naturally infected with A/chicken/Taiwan/1209/03 H5N2. The HI test, performed with the A/duck/Yunlin/04 virus as antigen, was used to discriminate positive and negative sera (i.e. gold standard).

Study C aimed at developing a bELISA for detecting H6 antibodies in chickens. The recombinant HA protein, derived from A/chicken/Taiwan/2838V/00 H6N1 virus, was used to coat the ELISA plates and to produce six purified monoclonal antibodies; the clone showing the best binding activity with A/chicken/Taiwan/2838V/00 was used as tracer in the bELISA. For test validation, field sera from two H6N1 infected broiler breeder flocks were tested, referring to the HI test (performed with the A/chicken/Taiwan/2838V/00 virus as antigen) as to the gold standard.

Study D aimed at developing a H5‐specific cELISA based on recombinant antigen and to evaluating its Se and Sp in different species. The gene coding for the HA protein was extracted from A/duck/NC/674964/07 H5N2 virus, and the recombinant protein was used as antigen. To obtain a possible broader applicable test, the monoclonal antibodies were generated using the recombinant HA protein derived from A/Vietnam/1203/04 H5N1 HPAI (upon deletion of the cleavage site). The performance of the cELISA in various species was assessed by testing turkey, chicken and duck sera. The latter ones were not included in our analysis. The HI test (using A/duck/NC/674964/07 H5N2 virus as antigen) was used as the gold standard. Tested sera came from birds experimentally infected with A/mute swan/MI/451072‐2/06 (H5N1), A/PS/CA/406032/04 (H5N2), A/Emu/NY/12716/94 (H5N2), A/Avian/NY/31588/00 (H5N2), A/TK/MN/10734‐2/95 (H5N2), A/PK/MD/4457/93 (H5N2), A/CK/IA/13609/93 (H5N2), A/CK/TX/167280‐4/02 (H5N3), A/Mallard/WI/42/75 (H5N3) and A/TK/WI/68 (H5N9).

Study E aimed at developing and evaluating a NP‐based type‐specific indirect ELISA to be used in sero‐epidemiological investigations of avian influenza infections in Asia. The recombinant NP protein derived from A/chicken/Hubei/04 H5N1HPAI virus. The performance of the NP‐ELISA was compared to the HI test and to a commercial ELISA by testing different chicken sera with the three assays. Negative sera came from SPF chickens, whereas positive sera came from experimentally infected and vaccinated birds. Sera were collected at 3 weeks after challenge from H9N2‐infected birds and before death from H5N1‐infected birds. Sera from vaccinated chickens (both with H5N1 and H0N2) were collected 4 weeks after the second immunization. We considered in our analysis only the 2 × 2 table presenting the results of NP‐ELISA versus HI test.

Authors of study F evaluated the Se and Sp of AGP and HI test by means of latent class analysis. Chicken field sera tested with both assays came from six farms located near to first detected farm during the Japanese H5N2 outbreak in 2006. The AGP test was performed according to the OIE Diagnostic Manual, using A/budgerigal/Aichi/77 H3N8 virus to prepare the AGP antigen. The antigen for the HI test was A/chicken/Ibaraki/1/05 H5N2 virus, which was isolated from the first detected farm.

Data analysis

We applied a Bayesian version of latent class analysis to compare the results of tests in different populations, as proposed by Branscum et al. The model was implemented in WinBUGS, which uses a Markov Chain Monte Carlo (MCMC) sampling algorithm to obtain a Monte Carlo (MC) sample from the posterior distribution. For this analysis, there was no reliable prior information on prevalences, and prior knowledge of the Se and Sp of the tests was scarce. We thus chose to use uninformative priors to avoid the potential distortion of posterior estimates because of misleading prior information. Prior information was modelled using the Beta(1,1) distribution, which is uniform for the interval between zero and one (i.e. uninformative priors). For the analysis, the first 10 000 MC samples were discarded as a burn‐in, and the successive 150 000 iterations were used for posterior inference. Potential autocorrelation was removed by storing one MC sample every 50 iterations. As suggested by Toft et al. convergence of the MCMC chains was assessed both by visual inspection of the time‐series plots and by computing the Gelman‐Rubin convergence diagnostic plots using three MCMC chains with different starting values.

Posterior inference was performed by calculating the median and the 95% posterior credible intervals (PCI) of the Se and Sp of the seven tests. To compare the different parameters in a manner more similar to traditional frequentist statistical methods, Bayesian posterior probabilities (POPR) were calculated and used to decide in favour of or against several hypothesis (e.g. H : SecELISA_C < SecELISA_T). The POPR used to test H was calculated as the proportion of MC samples for which H 0 was true.

Sensitivity analysis

To investigate whether the available prior knowledge would have affected the posterior estimates of the parameters, we repeated the analysis, including informative priors on the Se and Sp of the iIFA and NP‐ELISA tests (only for these tests were other studies with comparable settings available in the literature). Prior distributions of the Se and Sp of the two tests were modelled as Beta(α,β) distributions, whose specific parameters α and β were derived based on the most likely value (mode) and the 5th percentile of the Se and Sp reported in the literature , (Table 3). The two models (i.e. with and without informative priors) were further compared by means of the deviance information criterion (DIC, smaller is better).

Table 3

Available information and corresponding prior distributions for the sensitivity (Se) and specificity (Sp) of two of the diagnostic tests evaluated

Diagnostic test	Parameter	Mode	5th percentile	Prior distribution	Reference
iIFA	Se	95·0	89·0	Beta (75·959, 4·945)	van der Goot et al. 22
	Sp	92·0	87·0	Beta (119·426, 11·298)	van der Goot et al. 22
NP‐ELISA	Se	99·9	85·0	Beta (18·634, 1·018)	Upadhyay et al. 23
	Sp	97·0	65·0	Beta (7·771,1·210)	Upadhyay et al. 23

Given that the bELISA had been used in two studies, these studies could have had a disproportional impact on the estimated performance of the HI test. To test this, we repeated the analysis, including only the datasets of the two studies with the HI and bELISA. ,

As a last step, we investigated the stability of the HI test by looking at possible external factors capable of influencing its performance, such as the bird species and how the birds were infected. To assess the influence of the species, we estimated the species‐specific Se and Sp of the HI test in chickens and turkeys. The same approach was adopted for investigating the performance of HI in naturally and experimentally infected birds.

Results

The posterior estimates of the Se and Sp of the seven tests are given in Table 4. The HI test had an Se of 98·8% and an Sp of 99·5%. The Se and Sp of the bELISA and iIFA were not significantly different from that of the HI test. The NP‐ELISA had a high Se (92·1%) yet a low Sp (57·5%). By contrast, the AGP had a high Sp (96·3%) but a low Se (66·2%). The Se and Sp of cELISA were lower than those of the HI; they also differed significantly among species. In particular, the Se of cELISA was 70·8% in chickens (SecELISA_C) and 96·8% in turkeys (SecELISA_T). Assuming H : Se  < Se , the Bayesian POPR < 0·0001 (which can be interpreted as statistical significance in a one‐sided test) indicated that SecELISA_C was significantly lower than SecELISA_T. By contrast, the Sp of the cELISA was significantly higher in chickens than in turkeys (POPR = 0·0086).

Table 4

Posterior median and 95% posterior credible intervals (PCI) of the sensitivity (Se) and specificity (Sp) of the serological tests evaluated, according to the specified prior information

	Uninformative priors				Informative priors on Se and Sp of iIFA and NP‐ELISA
	[DIC = 126·2]				[DIC = 127·3]
Test	Se	95%PCI	Sp	95%PCI	Se	95%PCI	Sp	95%PCI
HI	98·8	[96·0; 100]	99·5	[98·4; 100]	99·0	[96·4; 100]	99·6	[98·5; 100]
iIFA	98·1	[94·0; 99·9]	96·3	[91·8; 99·4]	96·6	[93·3; 98·7]	94·0	[90·6; 96·5]
bELISA	99·3	[98·0; 100]	97·6	[95·2; 99·8]	99·3	[97·9; 100]	97·4	[95·0; 99·7]
cELISA_C*	70·8	[62·8; 78·0]	64·6	[48·0; 80·0]	70·7	[62·7; 78·0]	64·3	[48·0; 79·4]
cELISA_T**	96·8	[91·6; 99·3]	22·0	[5·2; 52·4]	96·8	[91·6; 99·3]	21·6	[5·2; 51·0]
NP‐ELISA	92·1	[86·0; 96·3]	57·5	[42·2; 72·6]	93·2	[87·9; 96·8]	63·0	[48·9; 76·6]
AGP	66·2	[56·4; 75·1]	96·3	[81·9; 99·9]	66·2	[56·5; 75·1]	96·3	[81·9; 99·9]

*cELISA_C = cELISA used for chickens.

**cELISA_T = cELISA used for turkeys.

The use of informative priors on the Se and Sp of the iIFA and the NP‐ELISA did not affect the posterior estimates of any of the parameters (Table 4). Furthermore, the DIC slightly favoured the model with uninformative priors (DIC = 126·2 for the model with uninformative priors, compared to 127·3 for the model with informative priors).

The estimated Se and Sp of the HI test derived from the two studies in which the HI test and the bELISA were evaluated , were very close to the estimates obtained when including all of the studies (Table 5).

Table 5

Posterior median and 95% posterior credible intervals (PCI) of the sensitivity (Se) and specificity (Sp) of the HI test and bELISA, considering using only the two studies in which these tests were evaluated ,

Test	Se	95%PCI	Sp	95%PCI
HI	97·9	[95·1; 99·9]	99·5	[98·2; 100]
bELISA	99·4	[98·0; 99·9]	98·2	[95·5; 99·9]

The results of the stability analysis of the HI test are reported in Table 6. The HI test appeared to perform the same for the two species considered: the estimated Se and Sp of HIchicken did not differ significantly from the estimates of HIturkey (POPR = 0·3539 for Se and POPR = 0·3597 for Sp). However, the HI test was more accurate in naturally infected birds than in experimentally infected birds: POPR = 0·022 for Se of HInatural > Se of HIexperimental and POPR = 0·0002 for Sp of HInatural > Sp of HIexperimental.

Table 6

Posterior median and 95% posterior credible intervals (PCI) of the sensitivity (Se) and specificity (Sp) of the HI test, assuming different performance by species (HIchicken and HIturkey) and source of infection (HInatural and HIexperimental)

	Test evaluated				Null hypothesis (H0)	POPR*
Species	HIchicken		HIturkey
Se	98·9	[96·1; 100]	98·1	[92·6; 99·9]	Se HIchicken < Se HIturkey	0·3539
Sp	99·5	[98·1; 100]	99·1	[96·0; 100]	Sp HIchicken < Sp HIturkey	0·3597
Source of infection	HInatural		HIexperimental
Se	99·3	[97·0; 100]	91·1	[85·1; 98·9]	Se HInatural < Se HIexperimental	0·0222
Sp	99·6	[98·5; 100]	81·2	[59·5; 98·6]	Sp HInatural < Sp HIexperimental	0·0002

*POPR, Bayesian posterior probability.

Discussion

Using latent class analysis and published data, we estimated the accuracy of the HI test and six other diagnostic assays in detecting AIV antibodies, without making reference to a gold standard. Because the HI test is commonly considered the gold standard for type‐specific AIV antibody detection, its performance has rarely been questioned. Compared to the only previous study in which the accuracy of the HI test was estimated for poultry, we found a similar Se and a much higher Sp, as well as much narrower credible intervals. This comparison might seem unfair, because we included the data of the previous study in our model. However, according to a sensitivity analysis (data not shown), the estimated Se and Sp of the HI test remained basically unvaried when excluding the study from the analysis, suggesting that the data from such study are consistent with those of the other studies.

This study is the first attempt to estimate the Se and Sp of the HI test based on data collected in different study settings. This allowed us to investigate possible sources of variation in the performance of the HI test. Our results confirmed that the HI test is very accurate and that it performs well in both chickens and turkeys. However, the HI test was less accurate in experimentally infected birds compared with naturally infected birds. Although this result was rather unexpected, it could be explained by differences in the laboratory settings of the studies considered in our analysis. In fact, for two of the three experimentally infected populations, the virus antigen of the HI test was different from the virus strains used to inoculate the birds. Because the performance of the HI test is strongly influenced by the homology between the reference viral antigens and the virus isolates to be tested, this may have biased the estimates of the HI test accuracy in experimentally infected birds. However, the HI test reference antigen differed from the field AI strains also in one of the four naturally infected populations, though this apparently did not lower the estimated accuracy of the HI test in naturally infected birds, which was very high. To this regard, It would have been interesting to further investigate the role antigenic relatedness in the performance of HI test, unfortunately it was not possible because two of the six studies considered in our analysis , did not report any detailed information about the HI antigen and/or the tested antigen. This implies that such studies would have been excluded from the analyses, reducing the number of data points and the degrees of freedom and making the model unidentifiable.

The bELISA seems to be a good alternative to the HI test: apart from being quicker and easier to automate, it has a very high Se and a good Sp. These results are consistent with those of other studies in which the accuracy of the bELISA was calculated based on HI test results on field sera or on a combination of different test results in experimentally infected birds. The iIFA test showed satisfactory accuracy, which was slightly higher than that estimated by van der Goot et al. in the absence of a gold standard yet a little lower than the accuracy reported by Cattoli et al., who used a gold standard defined by a combination of virological and serological test results on field sera. The NP‐ELISA seems to be quite sensitive, yet it lacks specificity, which differs from the results of the study by Upadhyay et al., who estimated both high Se and Sp. However, that study used a commercial ELISA as the gold standard. On the basis of our analysis, the cELISA has the lowest accuracy, and it differs significantly among species. When applied to chickens, the cELISA showed decent Se and moderate Sp, whereas in turkeys, it was more sensitive yet less specific. These differences were also reported by the authors of the original study, whereas other authors did not find any important differences by poultry species in the performance of the cELISA, using the HI test as the gold standard.

The estimates obtained with and without informative priors were very similar, confirming that the data themselves were quite robust and rather insensitive to the choice of the prior information. Furthermore, the DIC slightly favoured the model with uninformative priors.

A key assumption in latent class analysis is that the results of the diagnostic tests are independent given the disease status. The disease status often refers to the presence of the pathogen. However, because we evaluated serological tests, the definition of the disease status for AIV infection must be considered a measure of the presence of AIV antibodies. Conditional to such disease status, the seven tests may be assumed as conditionally independent. Furthermore, a model that assumes that all of the tests are conditionally dependent given the disease status is unidentifiable and should as such not be evaluated without specifying informative priors for all (or at least a large proportion) of the parameters.

This is the first study in which all of these tests were compared in a single analysis. However, some limitations should be acknowledged. Although six studies met the inclusion criteria, apart from the HI test, only the bELISA was evaluated in more than one study. For this reason, the two studies that included the bELISA could have influenced the estimates of the HI accuracy more than the other studies. Another potential limitation is that we did not take into account factors such as the potential differences in accuracy related to the virus subtype (i.e. H5, H6, H7 or H9) or the timing of testing. However, given that the HI test is considered the gold standard for type‐specific AIV antibody detection, its accuracy would have been difficult to estimate using the traditional method for test evaluation. Thus, using latent class analysis, we were able to evaluate for the first time the performance of the HI test under different conditions.

Conclusions

The HI test has a near perfect accuracy and may be considered a gold standard, provided that the reference viral antigen is close enough to the virus isolate to be tested or a panel of different antigens are used. In the framework of LPAI surveillance, in which large amounts of samples have to be processed and the virus subtype for which there is a risk of exposure may not be known, the use of subtype‐specific bELISA as a screening test could be a valid alternative to HI, given that it is almost equally accurate yet quicker and easier to automate.

Appendix S1 List of peer‐reviewed publications initially deemed as potentially relevant and further reviewed.

Supporting info item

Click here for additional data file.