Agreement between allergen-specific IgE assays and ensuing immunotherapy recommendations from four commercial laboratories in the USA.

BACKGROUND: Canine allergen-specific IgE assays in the USA are not subjected to an independent laboratory reliability monitoring programme. HYPOTHESIS/
OBJECTIVES: The aim of this study was to evaluate the agreement of diagnostic results and treatment recommendations of four serum IgE assays commercially available in the USA.
METHODS: Replicate serum samples from 10 atopic dogs were submitted to each of four laboratories for allergen-specific IgE assays (ACTT(®) , VARL Liquid Gold, ALLERCEPT(®) and Greer(®) Aller-g-complete(®) ). The interlaboratory agreement of standard, regional panels and ensuing treatment recommendations were analysed with the kappa statistic (κ) to account for agreement that might occur merely by chance. Six comparisons of pairs of laboratories and overall agreement among laboratories were analysed for ungrouped allergens (as tested) and also with allergens grouped according to reported cross-reactivity and taxonomy.
RESULTS: The overall chance-corrected agreement of the positive/negative test results for ungrouped and grouped allergens was slight (κ = 0.14 and 0.13, respectively). Subset analysis of the laboratory pair with the highest level of diagnostic agreement (κ = 0.36) found slight agreement (κ = 0.13) for ungrouped plants and fungi, but substantial agreement (κ = 0.71) for ungrouped mites. The overall agreement of the treatment recommendations was slight (κ = 0.11). Altogether, 85.1% of ungrouped allergen treatment recommendations were unique to one laboratory or another. CONCLUSIONS AND CLINICAL IMPORTANCE: Our study indicated that the choice of IgE assay may have a major influence on the positive/negative results and ensuing treatment recommendations.

Introduction

Allergen-specific immunotherapy (ASIT) is frequently prescribed to aid in the management of canine atopic dermatitis (AD).1 The formulation of the allergenic extract is customized for each dog based on allergen test results, history and aerobiology.2 An intradermal test (IDT), serum allergen test (SAT) or both are performed to demonstrate allergen-specific immunoglobulin type E (IgE) directed against a panel of plant, fungi, mite, insect and epidermal antigens deemed to be important in the geographical region.3 An IDT is usually performed by a clinician using a panel of antigens that they have customized for their location. Within the same geographical region, these panels vary substantially, as do testing methodologies employed by different clinicians.4 In vitro IgE enzyme-linked immunosorbent assays (ELISAs) are offered by at least six commercial laboratories in the USA, operating without independent oversight of their quality control.5 Each laboratory divides the USA into aerobiological zones and offers allergen-specific IgE tests for a standard panel of antigens for each zone. These laboratories differ in the number and identity of the antigens tested on standardized panels for a given geographical region. An antigen may be included as a mixture of phylogenetically related species by one laboratory, as individual species by a second laboratory, or not at all by a third.

Allergen-specific IgE levels are interpreted as elevated (‘positive’) if the optical density measured in the assay is above a cut-off level established by the laboratory.6 ‘Positive’ allergens are candidates for inclusion in ASIT extract mixtures. Some of the antigens to which elevated allergen-specific IgE levels are reported may be excluded from the ASIT prescription if they are judged to have caused a false-positive reaction, are cross-reactive with other antigens selected for inclusion, exceed the desired maximal number or are otherwise deemed to be clinically irrelevant based on the patient's history.7 Commercial SAT laboratories offer technical assistance to support veterinarians in formulating ASIT prescriptions based on these factors. Overall, the ASIT prescription for an atopic dog is influenced by the following factors: (i) the selection of allergens to be tested; (ii) the intralaboratory and interlaboratory reliability of the testing; and (iii) post-test interpretation and formulation. Each of these variables may affect the reproducibility of ASIT prescriptions for a given dog.

Despite the potential for variability, the factors that may influence the composition of an ASIT prescription have received relatively little attention, and a variety of allergen-selection methods may be evaluated together in a single ASIT study.4,8 A rigorous evaluation of the reliability of IDT has not been published. Unpublished studies have reported fair agreement of three investigators interpreting IDT reactions.9 There is often poor agreement between results of IDT and SAT performed concurrently on the same dog.6,7,10,11

Early studies demonstrated poor reliability of SAT; however, the testing methodologies may have changed since those studies were carried out.5,7,12,13 Subsequently, Thom et al.14 evaluated the reliability of three independent European allergen-specific IgE testing laboratories each using an Fcε receptor ELISA methodology.14 The authors reported a 3% intralaboratory discrepancy rate and a 9% interlaboratory discrepancy rate for the three laboratories, in terms of all positive and negative reactions. In another report, SAT results were compared between two laboratories that employ the same monoclonal antibody cocktail (mac) ELISA methodology and between one reference laboratory using macELISA methodology and another using the aforementioned Fcε receptor ELISA methodology.15 Serum was pooled from samples of known macELISA reactivity, and duplicate aliquots were submitted for each of the comparisons. A follow-up to this study reported on the performance characteristics of six laboratories using the macELISA.16 Agreement of positive and negative reactions was presented as the percentage concordance, which exceeded 90% for interlaboratory comparisons.14–16 An analysis of the degree to which these concordance rates differed from the agreement that would be expected by chance alone was not presented in these reports. The chance-corrected agreement is important, because tests that are usually negative can have a high intra- or interlaboratory unadjusted agreement rate but a low chance-corrected agreement rate.

The objective of the present study was to evaluate the agreement (unadjusted and chance corrected) of four commercial laboratories offering SAT in terms of the following factors: (i) diagnosis of positive or negative allergen-specific IgE reactivity; and (ii) ASIT treatment recommendations for dogs with atopic dermatitis.

Materials and methods

Animals

Client-owned dogs diagnosed with atopic dermatitis were eligible for enrolment in the prospective study. The diagnosis was established by confirming that each dog fulfilled at least five of eight clinical features of canine AD, as described by Favrot et al.17 Differential diagnoses were ruled out as described by DeBoer and Hillier.18 Dogs weighing <10 kg were excluded. A sample size of 10 dogs was set as the enrolment goal.

Sample submission

Approximately 20 mL of blood was collected from each dog with owner consent. Serum from each dog was divided into four aliquots within 30 min, with each aliquot submitted to one of four commercial laboratories for allergen-specific IgE assays [ACTT® Bio-medical Services, Inc., Austin, TX, USA (Lab A); VARL Liquid Gold Veterinary Allergy Reference Laboratory, Pasadena, CA, USA (Lab B); ALLERCEPT® Heska®, Inc., Loveland, CO, USA (Lab C); and Greer® Aller-g-complete® IDEXX Laboratories, Inc., Westbrook, ME, USA (Lab D)]. Although each of the four assays is based on ELISA techniques, they differ from one another.15,19–21 To bind canine IgE in their assays, Labs A, B, C and D use polyclonal goat anti-IgE antibodies, a monoclonal rabbit anti-dog αIgE antibody cocktail, a biotinylated recombinant extracellular domain of human FcεRIα receptor and a monoclonal anti-dog IgE cocktail, respectively. The signal molecules used in the assays by Labs A, B, C and D are enzyme conjugate (type unpublished), horseradish peroxidase, streptavidin–horseradish peroxidase and streptavidin–alkaline phosphatase, respectively.

Handling and storage of all samples was identical for each dog. Laboratory-supplied patient history forms were filled out in detail and submitted with the samples. These provided information about the dogs’ seasonality of signs and their living conditions that might be taken into account by laboratory consultants when formulating ASIT. Serum samples were refrigerated for no longer than 72 h, and then shipped directly to the laboratories using laboratory-provided vials and mailers. The laboratories were left unaware of the occurrence of the study in order to simulate the day-to-day level of quality control that could be expected by primary care veterinarians. Standard, canine regional allergen-specific IgE panels were requested based upon the dogs’ geographical location (southern Texas, USA). Regional panels included 61–63, 40, 48 and 48 allergens or allergen mixtures for Labs A, B, C and D, respectively. A total of 110 individual allergens or allergen mixtures were represented in the four panels (Tables 1 and S1 in Supplementary material). In some cases, there was variation in the spectrum of antigens evaluated by a single laboratory during the course of the study. The antigens evaluated by each laboratory are shown in Table S1 in Supplementary material.

Table 1

Complete spectrum of antigens evaluated on standard, canine, southern Texas regional panels by four laboratories with allergen-specific IgE assays

Trees

Alder mixa, Arizona ashb, white ashb, ash mixb, bald cypressc, bayberry, American beechd, birch mixa, box eldere, box elder/maple mixe, California pepper tree, red cedarc, cedar/juniper mixc, mountain cedarc, cottonwood/poplar mixf, American elmg, elm mixg, eucalyptus, hazelnuta, maplee, maple mixe, Melaleuca sp., mesquite, red mulberryh, white mulberryh, Virginia live oakd, oak mixd, oliveb, pecan/hickory mixi, yellow pinej, pine mixj, common privetb, privet/olive mixb, queen palm, sweet gum, American sycamorek, sycamore mixk, western sycamorek, black walnuti, black willowf

Grasses

Bahia, Bermuda, bluel, bromel, Johnson, orchardl, quackl, red topl, red/sweet vernal mixl, fescuel, ryel, rye/fescue mixl, saltl, timothyl

Weeds

Baccharis sp., carelessweed, clover, cocklebur, daisy, dandelion, yellow dockm, dock mix/sheep sorrel mixm, dog fennel, English plantain, goldenrod, greasewood, Kochi sp., lambsquarter, marsh elder, mugwortn, mustard, nettle, rough pigweedo, pigweed mixo, short ragweedn, tall/giant ragweedn, false ragweedn, western ragweedn, ragweed mixn, Russian thistle, sagebrush mix, shadescalep, spearscalep

Moulds

Alternaria tenuis, Aspergillus mix, Candida sp., Cephalosporium sp., Cladosporium sp., Curvularia/Dreschlaria spp., Fusarium sp., grass smut mix, Helminthosporium sp., Mucor mix, Penicillium mix, Pullularia pullulans, Rhizopus nigricans, Stemphylium sp.

Mites and insects

Ascarus siroq, Bloma/Lepidoglyphus mixq, black ant, cockroach, Dermatophagoides pteronyssinusq, Dermatophagoides farinaeq, dust mite mixq, flea, fire ant, Tyrophagus putrescentiaeq

Epidermals

Cat dander, human dander, sheep wool

*ACTT® (Biomedical Services), VARL Liquid Gold (Veterinary Allergy Reference Laboratory), ALLERCEPT® (Heska), Geer® Aller-g-complete (IDEXX)

Antigens that are grouped by cross-reactivity for the agreement analysis are denoted with a shared superscript letter.

Interpretation of semi-quantitative scores

Semi-quantitative allergen-specific IgE levels reported by the laboratories were interpreted according to each laboratory's ELISA absorbance unit (EAU) guidelines, when available. Detection of potentially significant allergen-specific IgE levels (interpreted as ‘positive’) was defined as greater than or equal to 175, 150 and 80 EAU for Labs A, C and D, respectively, in accordance with guidelines provided with each laboratory report. The results from Lab B were interpreted as positive for values greater than or equal to two on a proprietary, six-point ordinal scale (0–5), as previously reported.13 Levels of allergen-specific IgE below these cut-off values were defined as ‘negative’ for the antigen.

Interassay agreement analysis

The agreement of positive and negative findings was analysed in two ways. First, antigens were evaluated separately (ungrouped), not attempting to correct for potentially insignificant differences between assays (e.g. testing a dust mite antigen mixture versus two individual mite species). In order to examine the extent to which subtle differences in regional panels affected our results, a second analysis was performed after grouping individual allergens that are of the same genus or are reported to be cross-reactive or co-sensitizing in dogs or humans (Tables 1 and S1 in Supplementary material).7,22,23 In this analysis, a positive reaction to any of the grouped components was interpreted as the group being positive; for example, a positive reaction to either one of two tested dust mites by one laboratory and to a mixture of the two mites by another laboratory were treated as displaying agreement in the grouped analysis.

For each dog, six pairwise laboratory comparisons (A:B, A:C, A:D, B:C, B:D and C:D) were made between the positive/negative test results in both the ungrouped antigen and grouped antigen analyses. The treatment recommendations from the laboratories were analysed in the same manner, comparing the agreement of recommended and nonrecommended allergens. Cross-tabulation agreement tables were produced for each pairwise comparison of laboratories. The agreement of the two laboratories in each table was expressed as a percentage agreement and chance-corrected agreement (Cohen's kappa, κ).24 While this report does present both percentage agreement and chance-corrected kappa agreement, the kappa agreement statistic is most relevant for assessing interlaboratory consistency in this study. Light's kappa (the average of all pairwise kappas, also denoted by κ in this text) was calculated for the overall pairwise agreement between all four laboratories for each dog.25 For kappa calculations, only antigens or antigen groups common to each laboratory in the comparison could be included in the analyses. Two subsets of ungrouped antigens (plants and fungi in one subset and mites in the other) were also analysed separately for diagnostic agreement.

All statistical analyses were carried out in the statistical software R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria) version 2.15.2. Non-negative κ values were interpreted as follows, according to the categories defined by Landis and Koch: 0–0.20 as slight; 0.21–0.40 as fair; 0.41–0.60 as moderate; 0.61–0.80 as substantial; and 0.81–1 as almost perfect agreement.26

Results

Study population

Ten dogs were enrolled in the study between June 2012 and March 2013. Three Labrador retriever dogs and one each of the following breeds were represented: Boston terrier, English bulldog, French bulldog, German shepherd, German shorthaired pointer, Lhasa apso and mixed breed. One dog (dog 2) had received systemic glucocorticoid (prednisolone) treatment within 4 weeks of enrolment; the remainder had not. Results from Lab A allowed blinded determination of positive and negative allergens for all 10 dogs; however, Lab A declined to provide a prescription recommendation for dog 4 after becoming aware of the study from an inadvertent communication from one of the investigators. For this reason, the data from dog 4 are not included in the ASIT recommendation agreement analysis. Serum samples from dogs 5–10 were subsequently collected and submitted to Labs A–D by a different investigator, whose participation remained concealed.

Interassay diagnostic agreement

The overall diagnostic agreement across all dogs and all laboratories was 70% for ungrouped antigens (Figure1), slightly higher than expected by chance (66%). When corrected for agreement by chance alone with the kappa statistic, the overall diagnostic agreement was slight (κ = 0.14; Figure1). The diagnostic agreement by dog (Figure2) and by laboratory pair (Figure1) can also generally be characterized as slight for the majority of the agreement comparisons. Similar percentage agreement (58–82%) and kappa values (0.01–0.28) were obtained when the overall, by-laboratory pair and by-dog analyses were repeated with antigens grouped by taxonomy, cross-reactivity and co-sensitization (Figures S1 and S2 in Supplementary material).

Figure 1

Overall and by laboratory pair, ungrouped allergen diagnostic percentage agreement and chance-corrected agreement (kappa statistic) of elevated allergen-specific IgE in 10 dogs with atopic dermatitis. Light's kappa is presented for overall agreement. Cohen's kappa is presented for laboratory comparisons. Laboratories are as follows: Lab A, Biomedical Services; Lab B, Veterinary Allergy Reference Laboratory; Lab C, Heska; and Lab D, IDEXX/Greer.

Figure 2

By dog, ungrouped allergen diagnostic percentage agreement and chance-corrected agreement (Light's kappa statistic) of elevated allergen-specific IgE in 10 dogs with atopic dermatitis.

The best chance-corrected agreement was found between Labs C and D (κ = 0.36, fair). Nine of the 12 positive agreements between Labs C and D across the 10 dogs were for mites. Subset analysis of Labs C and D found 86% agreement (κ = 0.13, slight) for ungrouped plants and fungi, and 87% agreement (κ = 0.71, substantial) for ungrouped mites (Figure3). The κ values can be characterized as slight or fair for all other between-laboratory comparisons of diagnostics on ungrouped antigens limited to either plants and fungi or mites (Figure3).

Figure 3

Overall and by laboratory pair diagnostic agreement for positive versus negative allergen-specific IgE detection of ungrouped plant/fungal and mite antigens in dogs with atopic dermatitis. Light's kappa is presented for overall agreement. Cohen's kappa is presented for laboratory comparisons. Laboratories are as in Figure 1. *Could not be calculated because no identical mite antigens were evaluated by Lab A and Lab B.

Interlaboratory ASIT recommendation agreement

The overall ASIT recommendation percentage agreement across nine dogs and all laboratories was 72% for ungrouped allergens (Figure4). This is only slightly higher percentage agreement than would be expected by chance (69%). The overall ASIT recommendation agreement as analysed with the kappa statistic was only slightly better than expected by chance (κ = 0.11; Figure4). Agreement percentage by dog (Figure5) and by laboratory pair (Figure4) ranged from 62 to 87% for ungrouped antigens. The chance-corrected agreement of ASIT recommendations by dog and by laboratory pair was generally slight. Similar ranges of percentage agreement (59–83%) and chance-corrected agreement (κ = 0.01−0.27) were found when the ASIT recommendations were analysed by antigen groups (Figures S3 and S4 in Supplementary material). The overall and by laboratory pair κ values were very similar (±0.02) when the analysis was repeated for all 10 dogs with the 10 positive reactions for dog 4 in Lab A's assay defined as ‘recommended’ (data not shown). The highest level of chance-corrected ASIT recommendation agreement was found between Labs C and D (Figure4; κ = 0.27, fair agreement).

Figure 4

Overall and by laboratory pair, ungrouped allergen treatment recommendation percentage agreement and chance-corrected agreement (kappa statistic) in nine dogs with atopic dermatitis. Light's kappa is presented for overall agreement. Cohen's kappa is presented for laboratory comparisons. Laboratories are as in Figure 1.

Figure 5

By dog, ungrouped allergen treatment recommendation percentage agreement and chance-corrected agreement (Light's kappa statistic) in nine dogs with atopic dermatitis.

Descriptive data

Among the six pairs of laboratory comparisons, the number of shared antigens available for interassay comparison of a pair ranged from 233 to 316 for ungrouped antigens, and from 239 to 335 for grouped antigens (ranging across all six pairs of laboratories and pooling data across the 10 dogs.)

The prevalence of positive reactions across all dogs for each laboratory ranged from 4.2 to 34% of antigens tested (Table 2). The number of positive reactions per laboratory for each dog varied widely. Four of the 10 dogs (dogs 1, 5, 7 and 9) did not have any positive results reported by Lab C. None of these had received glucocorticoid therapy for at least 4 weeks before serum sampling. The same four dogs had a mean of 11 positive results per dog reported by the other three laboratories.

Table 2

The number of antigens resulting in positive allergen-specific IgE findings by four laboratories evaluating replicate serum samples from dogs with atopic dermatitis

Dog no.	Lab A	Lab B	Lab C	Lab D
No. of antigens tested	61–63	40	48	48
1	25	17	0	1
2	14	14	3	15
3	21	13	4	19
4	10	14	4	10
5	11	14	0	10
6	9	15	3	13
7	9	14	0	1
8	15	6	4	4
9	12	15	0	3
10	17	15	2	4
Mean (% of tested)*	14.3 (23%)	13.7 (34%)	2.0 (4.2%)	8.1 (17%)

The mean was calculated by the averaging the positive allergen-specific IgE findings across dogs. The percentage of tested was calculated as the ratio of the number of the positive allergen-specific IgE findings pooled across all 10 dogs divided by the number of all tests for all 10 dogs × 100%. Laboratories are as follows: Lab A, Biomedical Services; Lab B, Veterinary Allergy Reference Laboratory; Lab C, Heska; and Lab D, IDEXX/Greer.

Forty-four allergens were assigned an ordinal score of ‘1’ by Lab B for the 10 dogs (median, 4.5 per dog). Of these, three were recommended for immunotherapy by Lab B despite falling below the positive/negative cut-off value we defined as positive, in accordance with methodology previously reported by this laboratory.13 All three of these antigen recommendations were made for a single dog (dog 8), which had the lowest number of positive reactions for Lab B amongst the 10 dogs (Table 2).

The number of diagnostic and ASIT recommendation disagreements was very large compared with the number of positive agreements (Figure6). For the nine dogs for which all four laboratories provided treatment recommendations, 261 antigens were recommended for ASIT by the laboratories collectively. Of these antigens, 85.1% were recommended by one laboratory or another (without recommendation by a comparison laboratory), 11.5% by two laboratories, 3.1% by three laboratories and 0.4% by all four laboratories (the sum does not equal 100.0% due to rounding).

Figure 6

Total number of pairwise agreements and disagreements between four serum allergen testing laboratories for assays on 10 dogs. Abbreviations: DX, positive/negative diagnostic agreement; G, grouped antigens with phylogenetically related, cross-reactive and cross-sensitizing antigens considered equivalent; Neg, negative; Pos, positive; TX, include/exclude treatment recommendation agreement; U, ungrouped antigens evaluated as tested.

Discussion

Our results show that the agreement of four allergen-specific IgE assays, while considerable as measured by percentage agreement, is only slightly better than could be expected by chance. This counterintuitive result is possible when a high percentage of results are negative, as was the case in our study. The number of negative pairwise agreements and the number of disagreements greatly exceeded the number of positive agreements (Figure6).

Previous studies have measured the interlaboratory and interassay agreement of the allergen-specific IgE assays used by Labs C and D.14–16 The concordance (percentage agreement) of positive/negative results exceeded 90%, leading the authors to conclude that the assay results were comparable, but the statistical methodology did not account for agreement that might have occurred by chance alone. In addition to the difference in statistical methodology, another possible explanation as to why our findings differed from those of Lee et al.15 is a difference in the nature of the serum samples used in the respective studies.15 Discordant allergen-specific IgE results are more likely to occur at levels close to those defined as the cut-off values for each assay.15 In our study, aliquots of 10 individual blood samples from clinical patients diagnosed with atopic dermatitis were submitted to each laboratory. In an interassay comparison,15 18 serum pools were prepared from individual samples with known macELISA allergen reactivity in order to generate a wide spectrum of reactivity, from negative to highly positive.15 This may have reduced the prevalence of borderline allergen-specific IgE levels in comparison to those found in our individual serum samples.

Laboratory comparisons for indications with very high or very low prevalence can result in a high percentage agreement rate, merely by chance, while the chance-corrected agreement may be low. As a hypothetical example with features similar to those encountered in this study, suppose that Lab X and Lab Y analyse 100 allergens, agree negative on 80 allergens, agree positive on two allergens, have nine allergens positive for Lab X and negative for Lab Y, and nine allergens positive for Lab Y and negative for Lab X. For these data, the prevalence is a low 11%, the agreement rate is a high 82%, while the chance-corrected kappa agreement is a very low 0.08. Among the 20 allergens in this example where either or both laboratories yielded a positive result, there were only two allergens where the two laboratories agreed positive, which is clearly poor agreement. If Labs X and Y randomly and independently assigned allergens to positive or negative status in a manner to retain the 11% positive prevalence for each laboratory, the expected agreement would be 80%, which is not very different from the observed 82% agreement rate. The percentage agreement rate includes potentially random agreement, whereas kappa excludes it. It seems relevant to measure agreement as a correspondence between laboratories that goes beyond chance agreement, and that is the role of kappa. Thus, we report the chance-corrected agreement with the kappa statistic along with percentage agreement for four serum IgE assays and ensuing treatment recommendations.

The highest level of chance-corrected agreement occurred between Labs C and D. Labs C and D displayed substantial chance-corrected agreement (kappa) on diagnostic results for mite antigens, but only slight chance-corrected agreement for plant and fungal antigens. The reproducibility of SAT has previously been shown to differ amongst allergen groups.5,16 The interlaboratory coefficient of correlation was notably lower for fungal than for mite, grass, tree or weed allergens in a study comparing six European laboratories using the macELISA.16

In order for two or more assays of allergen-specific IgE to display a high level of interassay agreement, each must have a high degree of intra-assay reproducibility. Whether the poor interassay agreement observed for the 10 dogs in this study was attributable to intra-assay or interassay variability was not determined. Laboratories offering SAT face significant technical challenges that may contribute to intra-assay variability. Potential pitfalls include nonspecific anti-IgE antibody binding, allergenic extract variability, nonspecific signal molecule activation and establishing clinically relevant cut-off values.6 The assays evaluated in the present study use a variety of methodologies, employing different types of IgE detection methods and enzyme conjugates, each presenting unique technical challenges. As natural products, allergens may differ by source and by batch. This remains the greatest challenge to harmonizing different allergen-specific IgE assays in human medicine.27 It was beyond the scope of the present study to investigate the numerous potential causes of the observed laboratory disagreement. Our findings emphasize the need for an independent, interlaboratory reliability monitoring programme for canine allergen-specific IgE assays, as has been previously proposed.14,15

Allergen-specific serum IgE assays are considered to be a valid means of identifying allergens for possible inclusion in ASIT extracts, based on the observation that ASIT produces a similar clinical benefit for canine AD whether based on SAT or IDT.3,7,28–30 Although the laboratories evaluated in this report are located in the USA, Labs A and B process samples from outside the USA, and the assays used by Labs C and D are used in European laboratories.14–16 Despite the widespread use of the assays and presumably the ASIT recommendations provided by companies marketing them, the reproducibility of ASIT prescription recommendations has not been evaluated. We found that the overall agreement of the laboratories’ ASIT prescription recommendations is only slightly better than could be expected by chance. Our findings emphasize the importance of considering clinical factors beyond elevated allergen-specific IgE levels (e.g. likely allergen exposure) in formulating ASIT.2

Lab C did not make ASIT recommendations for four of 10 dogs due to low levels of reactivity on the ALLERCEPT® assay (Table 2). These same dogs often had 10 or more positive reactions according to the other three assays. Overall, the VARL Liquid Gold assay resulted in the highest prevalence (34%) and the ALLERCEPT® assay the lowest prevalence (4.2%) of positive reactions across the 10 dogs. Whether this stark difference is due to allergen selection, false-positive reactions or false-negative results is uncertain, but it would clearly influence ASIT recommendations and, perhaps, whether the dogs would be diagnosed with ‘atopic-like dermatitis’, without a demonstrable IgE response, rather than ‘atopic dermatitis’. 31

The clinical implications of the poor prescription recommendation agreement on ASIT efficacy are uncertain, but our results suggest that the choice of serum IgE assay is not a trivial matter. It is possible that one laboratory's recommended prescription is consistently more efficacious than another's, but this is not necessarily the case. We did not evaluate the efficacy of the recommended ASIT prescriptions, nor intralaboratory reliability, which could influence the consistency of the recommendations by a single laboratory. Furthermore, while it is implicitly assumed in the term ‘allergen-specific’ that an optimal allergen formulation for each atopic dog exists, this may not be true. Evidence to the contrary is as follows: (i) some beneficial effects of immunotherapy are nonspecific; (ii) ASIT based on varying testing methodologies results in similar efficacy; and (iii) standardized immunotherapy mixtures and those customized based on allergen test results may be of similar efficacy.3,7,28–30,32–34 Robust studies are needed to compare the efficacy of ASIT based on different allergenic extract formulation methods.

There were substantial differences between the four laboratories in the allergens included in regional assays from the same geographical region, which is likely to have influenced the number of unique allergens recommended for immunotherapy. Our agreement analysis, however, was limited to the individual allergens (ungrouped analysis) or allergen groups (grouped analysis) common to both assays in each pairwise comparison. Cross-reactivity or co-sensitivity within allergen groups has been demonstrated in atopic dogs.22,35 In our study, the pairwise agreement results of phylogenetically related, cross-reactive or co-sensitizing grouped antigens were generally similar to those found for the ungrouped antigens; therefore, the poor agreement between laboratories did not appear to stem from the standard panels of the four laboratories incorporating different, but similar antigens. However, the allergen groupings we used in our analysis were based largely on data from humans, which may differ from their behaviour in dogs.

We evaluated the agreement of the results in binary terms (positive or negative reactivity), which may differ from the agreement of continuous EAU scales, when available. The cut-off levels established by each laboratory were used in our study to reflect the manner in which practising veterinarians most often differentiate positive from negative results.14 Using different cut-off levels could have led to different levels of agreement.15

A limitation of our study is that we relied on the treatment recommendations obtained from the laboratories without question, while in practice, the prescribing veterinarian is free to ignore or modify these recommendations. It is possible that veterinarians in practice do not rely heavily on the laboratories’ treatment recommendations, but this is unlikely. Prescription recommendations are either provided routinely or upon request by all four of the laboratories evaluated.

The small number of dogs included in this study is another limitation. With a properly designed study including a larger number of dogs, one should be able to establish a more statistically rigorous inference for the general population of dogs and laboratories, including 95% confidence intervals for the κ and agreement estimates. Although our study was limited to 10 dogs, the chance-corrected agreement was undesirably low for each of them. Agreement by dog was generally characterized only as slight for both grouped antigens and ungrouped antigens. This was true of both diagnostic and ASIT treatment recommendation agreement. The low level of between-laboratory agreement on 261 allergen treatment recommendations in nine dogs (compared with what would be expected merely by chance) is also striking. If the findings from the 10 dogs in our study are representative of a wider population of atopic dogs, it draws into question the comparability of these four allergen-specific IgE assays.

In summary, the chance-corrected agreement of four commercially available allergen-specific IgE assays was evaluated for 10 dogs with atopic dermatitis. The results show that the overall interassay agreement was only slightly better than expected by chance. No two laboratories displayed even moderate chance-corrected agreement (κ > 0.40) with each other. The ASIT recommendations provided by the laboratories for nine dogs also displayed slight overall agreement. Eighty-five per cent of the 261 ASIT antigen recommendations for these nine dogs were unique to one laboratory or another. Further studies are required to determine the degree to which the choice of allergen testing laboratory impacts on the efficacy of ASIT.