Development and statistical validation of a guinea pig model for vaccine potency testing against Infectious Bovine Rhinothracheitis (IBR) virus.

Infectious Bovine Rhinothracheitis (IBR) caused by bovine herpesvirus 1 (BoHV-1) infection is distributed worldwide. BoHV-1 either alone or in association with other respiratory cattle pathogens causes significant economic losses to the livestock industry. The aim of this work was to validate a guinea pig model as an alternative method to the current BoHV-1 vaccine potency testing in calves. Guinea pigs were immunized with two doses of vaccine, 21 days apart and sampled at 30 days post vaccination (dpv). BoHV-1 antibody (Ab) response to vaccination in guinea pigs, measured by ELISA and virus neutralization (VN), was statistically compared to the Ab response in cattle. The guinea pig model showed a dose-response relationship to the BoVH-1 antigen concentration in the vaccine and it was able to discriminate among vaccines containing 1log(10) difference in its BoHV-1 concentration with very good repeatability and reproducibility (CV < or = 20%). A regression analysis of the Ab titers obtained in guinea pigs and bovines at 30 and 60dpv, respectively, allowed us to classify vaccines in three potency categories: "very satisfactory", "satisfactory" and "unsatisfactory". Bovines immunized with vaccines corresponding to each of these three categories were experimentally challenged with BoVH-1 virus, the level of protection, as measured by reduction of virus shedding and disease severity, correlated well with the vaccine category used. Data generated by 85 experiments, which included vaccination of calves and guinea pigs with 18 reference vaccines of known potency, 8 placebos and 18 commercial vaccines, was subjected to statistical analysis. Concordance analysis indicated almost perfect agreement between the model and the target species for Ab titers measured by ELISA and almost perfect to substantial agreement when Ab titers were measured by VN. Taken together these results indicate that the developed guinea pig model represents a novel and reliable tool to estimate batch-to-batch vaccine potency and to predict efficacy of killed BoHV-1 veterinary vaccines. Copyright 2010 Elsevier Ltd. All rights reserved.

Introduction

Infectious bovine rhinothracheitis (IBR) and pustular vulvovaginitis (IPV) are respiratory and reproductive diseases of domestic and wild cattle caused by bovine herpesvirus 1 (BoHV-1) [1], [2], [3]. The disease presents a respiratory form, including coughing, nasal discharge and conjunctivitis. Signs can range from mild to severe, depending on the presence of secondary bacterial pneumonia, with the development of dyspnea. In the absence of bacterial pneumonia, recovery generally occurs 4–5 days after the onset of signs. Because virus latency is a normal sequel to BoHV-1 infection, and antibody (Ab) response after infection seems to be life-lasting, any seropositive animal should be considered as a potential carrier and intermittent shedder of the virus, with the exception of young calves with passive maternal Ab and non-infected cattle vaccinated with killed vaccines [1].

BoHV-1 infection is distributed worldwide affecting domestic and wild ruminants [2], [3], [4], [5], [6], [7]. BoHV-1 either alone or in association with other respiratory cattle pathogens is the cause of significant economic losses in the livestock industry. However, after the implementation of strict control programs, the disease has been eradicated from Nordic European countries (Norway, Finland and Sweden), Austria, Denmark, and part of Italy. Currently, other European countries are under compulsory or voluntary eradication programs, all involving the application of inactivated or live “marker” vaccines, based on the deletion of BoHV-1 gE or gD viral glyproteins. In the rest of the world, classical attenuated and killed BoHV-1 vaccines are commonly applied [1], [8].

In South American countries like Argentina, Brazil and Uruguay BoHV-1 infection is endemic [7], [9], [10] and vaccination is not mandatory. In Argentina, due to regulatory restrictions, only killed vaccines can be used to prevent viral diseases in cattle. Several conventional combined inactivated vaccines containing BoHV-1 together with other viruses (bovine viral diarrhea virus, bovine parainfluenza type 3 virus and bovine respiratory syncytial virus) and bacterial pathogens (Pasteurella multocida, Mannheimia haemolytica, Histophilus sommi, Moraxella bovis, Moraxella ovis, Campylobacter fetus-fetus, Campylobacter fetus-venerealis and Leptospira sp.) have been used for many decades in routine vaccination protocols to prevent bovine respiratory and reproductive diseases in cattle. These multivalent vaccines were designed to control a sanitary problem of complex etiology. However, the potency and efficacy against each antigen contained in some of these combined formulations is unclear and further studies need to be carried out to properly address this issue.

Specifically for BoHV-1 vaccine approval, international regulatory policies recommend to evaluate vaccine quality in a potency test conducted in seronegative calves [1], [11]. A BoHV-1 vaccine must prevent the development of severe clinical signs and markedly reduce virus shedding after experimental challenge. Bovine trials are cumbersome, expensive and time consuming, particularly, in countries like Argentina, where BoHV-1, as well as other viral infections are endemic [10], [12], [13]. The difficulty in finding seronegative bovines, from BoHV-1 free herds, to be used in vaccine potency tests, pose the need for the developing standardized and harmonized tests in laboratory animals. The availability of a laboratory animal model would enable the regulatory authority and vaccine manufacturers to carry out batch-to-batch release tests on a routine basis in a less time consuming and less expensive way.

Although some vaccine manufactures have reported the use of guinea pigs as internal quality test to evaluate their vaccines [14] a validated method for vaccine potency testing in laboratory animals possessing a demonstrated concordance with the target species, is not yet available [15], [16], [17]. Such a properly validated vaccine potency test especially designed for combined vaccines including inactivated viruses is also required in the US and the European Union and could be globally used to control viral vaccines applied in cattle.

Although several ELISA tests were developed to determine BoHV-1 Ab and probed to be more sensitive and specific than the viral neutralization (VN) test [1], [18], the latter is still considered the gold standard technique used for vaccine potency testing [1], [11].

The general aim of this project is the development and statistical validation of a guinea pig model to be used for veterinary vaccine potency testing. The model has been specifically designed to evaluate the immunogenicity against the viruses currently included in combined vaccines for cattle (BoHV-1, bovine parainfluenza type 3, bovine viral diarrhea virus, bovine sincitial virus, bovine rotavirus and bovine coronavirus).

In the present paper we report the statistical in-house validation of a guinea pig model as a method for potency testing of inactivated IBR vaccines. The validation involved the study of the kinetic of the Ab response in the animal model and the target species, a regression analysis applied to the dose–response curve to define categories for vaccines qualification, a concordance analysis between the laboratory animal model and the natural host confirmed with a BoHV-1 experimental challenge in the latter. Results obtained indicted that the Ab titers of immunized guinea pig constitute a useful predictive tool of vaccine efficacy in cattle.

Material and methods

Bovine: vaccination and sampling

A total of 553 male and female beef calves (Aberdeen Angus, Hereford, and their crossbreeds), 6–12-month-old, were included in the study. Vaccination trials were conducted in 12 beef farms located in Buenos Aires, Argentina. Herds without previous history of vaccination against BoHV-1 were selected. As BoHV-1 infection is endemic in Argentina, vaccines were evaluated in BoHV-1 seronegative animals from BoHV-1 free herds, and also in seronegative and seropositive calves from BoHV-1 endemic herds, in order to consider the variability of the real target population [1], [19]. In the trials conducted in BoHV-1 endemic farms, the number of positive and negative animals was randomly distributed in each treatment group so as to initiate the study with statistically similar pre-vaccination mean Ab titers and variances among groups. In every trial, bovines were vaccinated with two doses of vaccine, 30 days apart, as recommended in the label of each of the 22 commercial vaccines tested. Vaccines were administered by the subcutaneal route with doses of 5 or 3 ml according to manufacturerś recommendations. Vaccine formulated for model validation and named as “reference vaccines” were applied following same time intervals and dose volumes as commercial vaccines. Blood samples for serum extraction were collected by puncture of the jugular or coccygeal vein. Animals were sampled at 0, 30 and 60 days post-vaccination (dpv). Control groups included placebo and non-vaccinated animals. Groups of calves that received two doses of placebo formulated with culture media (without virus) emulsified in oil-adjuvant were assayed in eight occasions including the dose–response trials in which the animals were sampled until 90 dpv. In addition, in order to have vaccinated and non-vaccinated groups exposed to similar natural conditions, a group of non-vaccinated calves was included in every bovine trial conducted in each of the 12 farms that participated in this study. A bovine trial was only considered valid and included in the statistical analysis if no seroconversion was detected in the non-vaccinated and placebo control groups. Most experiments were carried out blinded (veterinarian, laboratory technician). Seroconversion was defined as a 4-fold increase in antibody titer for both ELISA and VN.

Guinea pigs: vaccination and sampling

The laboratory animal model for BoHV-1 vaccine potency testing was developed using a total of 497 guinea pigs (Cavia porcellus), SiS Al, around 350–400 g weight. After various preliminary assays, using different vaccination routes, dose volume and intervals between doses (data not shown), the immunization protocol was standardized as follows: a minimum of 5 guinea pigs were vaccinated with two doses of vaccine, 21 days apart, by the intramuscular route, in the hind-leg. The volume of the dose administered to the guinea pigs, corresponded to 1/5 of the volume of the dose given to bovines [20]. For the dose–response study serum samples were obtained at 0, 30 and 60 dpv in order to obtain the kinetic of the Ab response in the lab animal model. For concordance studies guinea pigs were sampled at 0 and 30 dpv. Blood extraction was conducted by cardiac puncture under anesthesia, following ECVAM recommendations for animal welfare [21]. The protocol was approved by the CICV y A, INTA Ethical Committee (CICUAE). In each immunization assay, at least three negative control animals (non-vaccinated or vaccinated with placebo) were included. Guinea pigs experimental groups were coded and serologic analyses were carried out blinded.

Serologic assay methods

Virus neutralization (gold standard technique)

The virus neutralization assay used in this study, considered as the “Gold standard” or “Traditional test”, was performed as previously described [22], [23] and adjusted to meet the recommendations given by the OIE and CFR [1], [11]. Briefly, bovine and guinea pig serum samples were heat inactivated at 56 °C for 30 min. Four replicates of serial 4-fold dilutions of each sample (1:4 to 1:1024) prepared in 96 well plates were mixed with an equal volume of the BoHV-1 reference strain Los Angeles, containing tissue culture infectious doses 50% (TCID50), and leading to a final neutralization stage 1:8 to 1:2048 [9]. An extra sample replicate without virus was used to evaluate the potential toxicity of the serum on the cells. Positive and negative control sera of known Ab titer to BoHV-1 were also included in each assay as internal reference controls. Bovine reference samples were derived from naturally infected, vaccinated and experimentally challenged calves, as previously described [22], [23]. Guinea pig reference samples were obtained from naturally seronegative and vaccined animals, as detailed elsewhere [24]. Serum–virus mixtures were incubated during 1 h at 37 °C, and then, 100 μl of Madin Darby bovine kidney (MDBK) cell suspension containing between 200,000 ± 50,000 cells was added. After 3 days incubation at 37 °C, plates were read microscopically for cyto-pathogenic effect (CPE). The test was considered valid if checking back titration of virus gave an infectious titer of 100 (TCID50) with an acceptance range of 50–200 TCID50. The positive control should give its expected titer ± 1SD, the negative serum should give no neutralization (monolayer with CPE) and the monolayer of control cells (cells plus culture medium, without serum and without virus) should be intact. Virus neutralizing Ab titers were calculated by Reed and Muench method [25]. Negative serum samples received the arbitrary value of 0.30 for calculation purposes.

Enzyme linked immunosorbent assay (ELISA) for the quantification of total antibody to BoHV-1 in bovine and guinea pigs

Total antibodies against BoHV-1 in bovine serum were determined using an indirect ELISA previously described [22], [23]. Briefly, ELISA 96-well plates were coated with 50 μl of positive (concentrated and semi-purified BoHV-1 containing 109  TCID50/ml) or mock infected MDBK cells and blocked with PBS-tween20/1% ovoalbumin buffer. Serial 4-fold dilutions of the serum samples were added and the assay was developed with a peroxidase labeled goat anti bovine IgG affinity purified antibody (Kirkegaard & Perry Laboratories, KPL). The ELISA assay was adapted to detect antibodies to BoHV-1 in guinea pig sera following the same procedure cited above but using a goat affinity purified anti guinea pig IgG (H + L), peroxidase labeled antibody (Kirkegaard & Perry Laboratories, KPL), in a 1:4000 dilution as conjugate. The antibody titer of each sample was expressed as the log10 of the reciprocal of the highest serum dilution with a corrected optical density OD405c (OD405 in the positive coated wells minus OD405 in the negative coated wells) greater than the cut off values of the assay (40% PP for both species). ELISA results were normalized by expressing the corrected OD values as the percentage of a single high-positive control serum (PP%) included in each plate, in every ELISA run. The validation of the assays fulfilling ISO/IEC 17025 requirements [1], [19] is detailed elsewhere [24].

Experimental design, vaccines and statistical analysis

The validation of the Method for Vaccine Potency testing in guinea pigs was conducted following the recommendation of the EMEA Committee for Veterinary Medical Products and European Committee of Alternative Method Validation (ECVAM) [16], [17], [26], [27], [28].

Dose–response experiment in guinea pigs and bovines

Three sets of vaccines consisting of combined water-in-oil vaccines containing increasing concentration of BoHV-1 covering a range from 105 to 107 TCID50/dose, and two vaccines with 108 TCID50/dose, together with fixed concentrations of parainfluenza type 3 bovine virus and bovine viral diarrhea virus were formulated. Viral antigens were inactivated using binary ethylenimine (BEI) [29] and vaccines were formulated using a pilot scale equipment. The adjuvant used in the formulation consisted in a mix of 0.67% Polisorbate 80, 2.1% Sorbitan 80 Monooleate and 57.9% mineral oil, in a 60:40 oil:water proportion. Each vaccine set was formulated with a different dilution of the same bulk of antigen that was previously diluted or concentrated in order to obtain the desired Ag concentration and the vaccine was formulated thereafter. These vaccines of known antigen concentration are referred to as “reference vaccines” and were used to estimate the dose–response calibration curve in bovine calves and guinea pigs. Each set of reference vaccines was tested in two independent experiments in guinea pigs and two independent trials in bovine calves (Table 1 ). Antibody responses were determined by VN, and the validated ELISAs. In the initial stage of development, samples were taken at different time points and the kinetic of the antibody response (mean ELISA and VN antibody titers) induced by these first set of reference vaccines (calibration vaccines) were evaluated until 60 dpv in guinea pigs and 90 dpv in calves. In all cases vaccines were administered in parallel, in groups of a minimum of 5 guinea pigs and 5 bovines. The minimum number of repetitions (animals per group; n  = 5) was calculated to achieve a statistical power of at least 83% to discriminate between vaccines containing BoHV-1 concentrations differing in 1 log10 [30], [31].

Table 1

Vaccines tested in parallel in bovines and guinea pigs included in the concordance analysis.

Type of vaccine	Syndrome	Vaccine composition viral antigens	BoHV-1 concentration (TCID50/dose)	Number of vaccines tested [1]	Number of vaccinated bovines	Number of vaccinated guinea pigs	Number of comparative assaysc
Calibration vaccines for the dose–response curve a (11 vaccines; 20 comparative assays) d	Respiratory	IBR-BVDV-PI-3	1 × 108	2	20	20	4
			1 × 107	3	30	30	6
			1 × 106	3	30	36	6
			1 × 105	3	30	30	4
Reference vaccines for concordance analysisb (n = 7 vaccines; 12 assays)d	Reproductive	IBR-BVDV	107	3	70	51	8
		DIVA gE-	107	1	5	5	1
	Respiratory	IBR-BVDV-PI-3	108	1	5	5	1
			107.5	1	5	5	1
			107	1	5	5	1
Commercial vaccinesc (n = 18 vaccines; 22 assays)d	Respiratory	IBR-BVDV-PI-3-BRSV	Unknown	3	30	30	5
		IBR-BVDV-PI-3		5	29	40	6
	Reproductive	IBR-BVDV		4	35	19	4
	Conjunctivitis	IBR		2	15	11	2
	Multi-purpose	IBR-BVDV-PI-3-RV		4	35	35	5
Placebo				8	58	44	8
Total vaccinated				44	402	366	62
Non-vaccinated					151	131	23
Total				44	553	497	85

Three sets of water-in-oil vaccines with decreasing concentrations of BEI-inactivated BoHV-1 emulsified oil adjuvant were prepared, except for 1 × 108 TCID50/dose of which only two vaccine were tested. Each vaccine set was evaluated in two independent experiments in guinea pigs and two independent field trials in bovines. Each group included 5 guinea pigs and 5 bovines, except for vaccine with 1 × 106, where 6 animals were added to increase the number of replicates in the concentration considered, under these conditions, the detection limit of the model. Groups of 3–5 animals, vaccinated with placebo and non-vaccinated, were included in each assay as negative controls.

Reference vaccines: Second set of vaccines of known antigen concentration (potency) prepared under industrial conditions used for concordance analysis included water-in-oil emulsions.

Oil commercial vaccines included water-in-oil and double water-oil-water emulsions.

Some vaccines were tested in more than one occasion to evaluate stability through time, generating a higher number of comparison lines.

Kinetic of the antibody response: selection of the sampling time point and detection limit for comparison between the model and the target species

The curves of the kinetic of the antibody response to BoHV-1 after vaccination in both species were analyzed using a mixed model for repeated measures and Bonferroni method for multiple comparisons [31]. This analysis allowed determining the earliest time point, for animal sampling, corresponding to the peak or plateau Ab titer that also showed acceptable discriminatory capacity. In a second stage of validation, at the time points selected in the previous stage, the ability of the laboratory animal model and bovine calves to significantly discriminate among vaccines formulated with antigen concentrations of 1 log10 difference was evaluated. At this stage the minimal antigen concentration inducing a detectable Ab response by ELISA and VN was determined (detection limit or minimal immunogenicity dose). Each individual assay/trial using a group of 5 animals was analyzed by one-way analysis of variance (ANOVA) followed by DGC multiple comparison test [32]. Kruskal Wallis non-parametric rank sum test [33] was used when the assumptions of normality and/or homoscedasticity were not met. The control groups (non-vaccinated and placebo groups) were not considered in the analysis when all the animals of these groups remained seronegatives throughout the experience (variance equal zero). In all cases significance level was established at 5%.

Repeatability and reproducibility

Statistical analysis for the vaccines tested for each antigen concentration, considering all the experiments and trials involved in the dose–response study was performed as a nested analysis of variance (nested ANOVA). The applied model allowed quantifying the relative contribution of the different sources of variation associated to the repeatability and reproducibility of the host species and the guinea pig model to evaluate vaccine potency. The repeatability was expressed as the coefficient of variation (CV) or the overall relative variation between individual bovine calves and guinea pigs within a group and the reproducibility was expressed as the CV or overall relative variation between different groups vaccinated with vaccine containing the same BoHV-1 concentration [30], [34].

Regression analysis to establish the cut-off for vaccine classification

The VN and ELISA mean Ab titers of the groups of bovine calves and guinea pigs immunized with the calibration vaccines formulated for the dose–response assay, at the previously selected time points, were analyzed by regression analysis. The lower limits of the 90% prediction intervals were used as classification criterion to establish a range of vaccine quality acceptance in terms of immunogenicity (Potency), in both species. The calculated split points were further used for vaccine classification in three categories: “unsatisfactory”, “satisfactory” and “very satisfactory”. The protection rate of vaccines representing these categories was evaluated by BoHV-1 experimental challenge in the natural host.

Concordance analysis

To evaluate the performance of the selected split point for vaccine classification (described in Section 2.4.1.3) and the agreement between the guinea pig model and the target species two concordance analyses were carried out. In a first analysis a total of 63 pairs of groups of vaccinated calves and guinea pigs, were tested. This first analysis used data generated from 20 groups of animals vaccinated with the calibration vaccines used to estimate the dose–response curve (described in Section 2.4.1) and another 12 assays including a second set of 7 combined water-in-oil immunogens elaborated following the industrial outline of production (≥107  TCID50/dose). These vaccines of known antigen concentration, consequently of known potency, were also assigned as reference vaccines and included three inactivated IBR-BVDV oil vaccines used for prevention of bovine reproductive syndrome, a gE-DIVA vaccine developed in Argentina [35] and three multivalent respiratory vaccines all emulsified oil adjuvant (0.67% Polisorbate80, 2.1% Sorbitan80 Monooleate and 57.9% mineral oil; 60:40 oil:water proportion). Eight placebo groups and 23 non-vaccinated groups, were considered the negative reference groups.

To avoid bias, in a second analysis the calibration vaccines used to set the split points (n  = 20) and most of the non-vaccinated groups were subtracted (n  = 16) and commercial vaccines were added instead. Thus, 41 pairs of groups were included: the 12 assays with reference vaccines, 22 commercial vaccines and their corresponding negative control groups (n  = 7). The commercial vaccines included in this analysis are products marketed for the prevention of viral conjunctivitis, respiratory and reproductive disease of cattle, from various manufacturers and contain unknown titers of inactivated BoHV-1 (Table 1).

In all cases vaccines were administered in parallel, to 5–10 guinea pigs and bovines calves per group (Table 1). Concordance between the vaccine quality predicted by the guinea pig model and the one obtained in the natural host was estimated by the weighted kappa coefficient. The weighted kappa gives different weights to disagreements according to the magnitude of the discrepancy avoiding the weakness of the kappa statistic that takes no account of the degree of disagreement. Values of weighted kappa from 0.41 to 0.60 indicate moderate agreement; values from 0.61 to 0.80 substantial agreement and values from 0.81 to 0.99 almost perfect agreement [36].

BoHV-1 challenge

Challenge experiments were conducted at CICV y A NBS2 facilities, as previously described [23]. Briefly, seronegative calves vaccinated with two vaccines classified by the guinea pig model and bovine calves as having “very satisfactory” or “satisfactory” potency were evaluated. Animals receiving placebo were used as controls and represented an “unsatisfactory” vaccine. Groups of 6 calves were vaccinated with two doses of each vaccine, 30 days apart. At 90 dpv, animals were challenged by the intranasal route with BoHV-1 virus LA at a concentration of 107.5  TCID50/ml. Calves were monitored for infection and disease development. Virus infection was measured by the duration and peak titer of virus shed in nasal swabs [23] and also represented by the “area under the curve of infection” (AUCi) obtained by plotting the infectious titer of virus shed for 14 days after challenge. Disease was measured by the duration and the severity of clinical signs of IBR and also represented by the “area under the curve of clinical signs” (AUCs) obtained by plotting the presence of clinical signs for 14 days after challenge [37]. Both AUC were calculated using MedCalc® version 11.1.1.0 statistical software.

A second challenge experiment – that was not initially planned – was carried out in calves from field trial 3 receiving set 2 calibration vaccines because of the unexpected low Ab response developed by these animals.

Laboratory 9001–2000 ISO standards certification

As part of the validation of the present method for vaccine testing, the laboratory section of the Virology Institute INTA certified a management system to fulfil the 9001–2000 ISO requirements (resolution no. AR QS 1970, 8/2006), and has initiated a program for ISO/IEC 17025 accreditation.

Results

Dose–response experiments: controlled assays in guinea pigs and field trials in calves

The three sets of reference vaccines formulated with increasing concentrations of BoHV-1 per dose (Table 1) were used to estimate the calibration curve and were tested in two independent assays in guinea pigs and two independent trials in bovine calves giving a total of 6 points for each BoHV-1 concentration. Only two sets of vaccines containing the highest concentration (1 × 108 TICD50/dose) were tested, giving a total of four replicates.

The kinetics of the Ab response followed up to 60 dpv is depicted in Fig. 1a. The mean ELISA and VN Ab titers of the immunized guinea pigs obtained in each experiment at 30 dpv and the statistical analysis (ANOVA) is detailed in Supplementary data Table 1. All the experiments carried out in guinea pigs were considered valid and were included in the analysis giving a total of 22 groups to estimate the regression curve.

Fig. 1

Kinetic of the BoHV-1 antibody responses determined by VN and ELISA after immunization in: (a) guinea pigs; (b) seronegative bovines from BoHV-1 free herds; bars and lines represent mean VN and ELISA antibody titers induced by the dose–response vaccines tested, respectively. Arrows indicate vaccination time. Bars/lines, at 30 dpv, with different upper/lower case letters indicate statistical differences among mean VN/ELISA antibody titers induced by the vaccine tested (mixed model for repeated measures and Bonferroni method for multiple comparison, p < 0.05) [32].

The dose–response experiments in bovine calves were conducted in six different herds, using a total of five calves per treatment group. A first set of vaccines was tested in two independent trials involving seronegative calves from two BoHV-1 free farms, without history of previous vaccination or epidemiological evidence of BoHV-1 infection (Fig. 1b). A 2nd set of vaccines was also evaluated in two bovine trials. The first experiment was carried out using selected seronegative calves from an endemic herd and the second in a non-vaccinated herd but with history of natural infection with BoHV-1 and included seronegative and seropositive calves (pre-vaccination Ab titer VN: 0.3–1.5; ELISA: 0.3–2.8). Finally, a 3rd set of vaccines was tested in another two BoHV-1 free farms. The mean ELISA and VN antibody titers of each independent trial, at 0 and 60 dpv and the statistical analysis are detailed in Supplementary data Table 2.

The trials conducted in bovine calves were only considered valid when the negative control groups (calves receiving placebo and non-vaccinated calves) showed no seroconversion at the end of the experiment: calves from free herds should remain seronegative and calves from endemic herds should maintain their initial Ab titers. A huge variation in the Ab response to BoHV-1 vaccination was observed in the different bovine trials. The ELISA Ab response of the calves from the endemic herd (Trial 4) was the highest, as expected for primed animals; the Ab response of negative calves from BoHV-1 free farms (Trials 1, 2, 5 and 6) were intermediate to high; in contrast, the Ab response in the seronegative calves selected from the other endemic herd (Trial 3) was remarkably lower than that registered in the calves derived from BoHV-1 free herds. After this unexpected result, trial 3 was excluded for the regression analysis. However in order to evaluate the protection conferred by a low Ab response, in comparison to the one obtained with an optimal Ab response, animals from trial 3 were challenged with BoHV-1 under controlled conditions (Supplementary Fig. 1 and Table 3). The exclusion of bovine trial 3 meant that only 20 groups were finally used to estimate the bovineś calibration curves.

Selection of a sampling time point for the guinea pig model and the target species

Preliminary studies showed that the laboratory animal model and the host species could not discriminate among vaccines formulated with an antigen concentration within the same order of magnitude. For example, vaccines containing 5 × 107 and 1 × 107 TCID50 of BoHV-1 per dose induced similar Ab responses, vaccines with 5 × 106 and 1 × 106 TCID50 induced a detectable and statistically similar Ab responses, while vaccine formulated with 5 × 105 and 1 × 105 TCID50 induced responses in some guinea pigs but did not induce detectable response in bovines (data not shown). Based on these results it was decided to estimate the calibration curve using vaccines differing in 1 log10 antigen concentration, which is also the recommended dilution rate used for virus titration assays [38].

At 30 dpv, after receiving two doses of vaccine, the guinea pigs developed a strong Ab response to BoHV-1. A clear dose-dependent behavior according to the Ag concentration of the vaccine was observed. The antibody titers remained high and constant until 60 dpv. For the laboratory animal model 30 dpv, corresponded to the peak Ab titer and the earliest time point of maximum discrimination among vaccines differing in 1 log10 in their antigen concentration (Fig. 1a).

After vaccination, all groups of bovines developed Ab response to BoHV-1 and 60 dpv was the time point corresponding to the peak of BoHV-1 Ab titers. At 90 dpv, the response remained in a plateau or started to decrease (Fig. 1b).

The shape of the kinetic curves of the Ab response for both species was similar. The Ab titers detected in guinea pigs at 30 dpv were comparable to those detected in bovines at 60 dpv. Thus, those time points were selected as the optimal moments for sampling and were used as the unique bleeding point for further concordance studies and for future routine use of the guinea pig model as a method for vaccine potency testing.

Detection limit and discriminatory ability of the guinea pig model and the target species to differentiate among vaccines formulated with BoHV-1 concentrations differing in 1 log10

At 30 dpv, the guinea pig model was able to discriminate between vaccines formulated with 107 TCID50/dose of BoHV-1 or higher from vaccines containing 106 TCID50/dose and 105 TCID50/dose, by ELISA. In all the experiments, vaccines with 107 TCID50/dose of BoHV-1 induced mean Ab titers higher than 3.00, while vaccines containing 106 TCID50/dose of BoHV-1 induced mean Ab titers lower than 3.00. Vaccines formulated with 105 TCID50/dose did not induce Ab response, indicating that, under the conditions used; this concentration was below the detection limit of model and the serology test used. The global analysis considering the total number of guinea pigs vaccinated with each Ag dose showed that the guinea pig model was able to significantly differentiate among vaccines formulated with BoHV-1 concentrations differing in 1 log10 (Fig. 1a).

The groups of calves vaccinated with decreasing doses of BoHV-1 also showed a dose–response effect. In all cases vaccines formulated with 106 TCID50/dose or higher induced Ab response, while vaccines formulated with 105 TCID50/dose did not induce a detectable Ab response during the entire period of the trial (Fig. 1b, Supplementary data, Table 2 ). Notably, animals from trials 1 and 2 were followed up to 90 dpv and no seroconversion was detected either (Fig. 1b).

Table 2

Repeatability and reproducibility of the guinea pig model vs. the target specie expressed as CV.

BoHV-1 concentration (TCID50/dose)	Bovine		Guinea pig
	Repeatability	Reproducibility	Repeatability	Reproducibility
ELIS
1 × 108	14.3%	14.6%	8.9%	1.6%
1 × 107	13.8%	18.4%	20.6%	→0a%
1 × 106	15.3%	21.5%	18.4%	9.9%
1 × 105	na	na
VN
1 × 108	18.0%	11.7%	8.8%	7.0%
1 × 107	18.8%	16.9%	17.9%	9.8%
1 × 106	25.4%	29.3%	14.3%	11.9%
1 × 105	na	na

The CV tended to zero, indicating that the variation among assays was irrelevant.

The Ab response measured by VN, the traditional technique, consistently yielded lower Ab titers than those obtained by ELISA for both species. No neutralizing Ab responses were detected in the groups of animals vaccinated with 105 TCID50 of BoHV-1/dose indicating that the limit of detection was similar for both techniques (Fig. 3; Supplementary data Tables 1 and 2).

Fig. 3

Regression analysis and estimation of “split points” for vaccine classification in guinea pigs and bovines by VN. = Mean VN Ab titer to BoHV-1 of 5 guinea pigs vaccinated with 2 doses of vaccine, 21 days apart, corresponding to 1/5 of the bovine dose. Sample collection: 30 dpv. = mean VN Ab titer to BoHV-1 of 5 bovines vaccinated with 2 doses of vaccine, 30 days apart. Sample collection: 60 dpv.

Repeatability and reproducibility

The guinea pig model gave highly consistent results of Ab titers by both, ELISA and VN, among different assays for the same Ag concentrations in the range from 106 to 108  TCID50/dose, giving acceptable repeatability and reproducibility. The CV associated with the reproducibility of the test was less than 10% for all the antigen concentrations tested and tended to zero (the variation among different assays was irrelevant) for vaccines formulated with 107  TCID50/dose. The target species also gave consistent results for testing the potency of vaccines containing 107 and 108  TCID50/dose (CV associated to the reproducibility < 20%) for both techniques (Table 2).

Dose–response curve

Since all the dose–response experiments conducted in guinea pigs and only five out of six trials in calves were considered valid, a total of 22 groups for the laboratory animal model and 20 groups for the host species were included in the regression analysis. As depicted in Fig. 2, Fig. 3 , the mean antibody titers, determined by ELISA and VN were directly related to the BoHV-1 concentration in the vaccines. For both species and techniques the mathematical model that best fit to the data was a polynomic linear model of second order of magnitude. The mathematical model was able to predict ELISA Ab titers with R 2  = 96% for guinea pigs and R 2  = 90% for bovines (Fig. 2). In the case of VN, the model was able to predict VN Ab titers for guinea pigs with R 2  = 90% and R 2  = 93% for bovines (Fig. 3).

Fig. 2

Regression analysis and estimation of “split points” for vaccine classification in guinea pigs and bovines by ELISA. = mean ELISA Ab titer to BoHV-1 of 5 guinea pigs vaccinated with 2 doses of vaccine, 21 days apart, corresponding to 1/5 of the bovine dose. Sample collection: 30 dpv. = mean ELISA Ab titer to BoHV-1 of 5 bovines vaccinated with 2 doses of vaccine, 30 days apart. Sample collection: 60 dpv.

The 90% prediction intervals were calculated, using the lower limit of Ab titer induced for vaccines formulated with 106 and 107  TCID50/dose of BoHV-1 as quality split points. These limits represent the minimum mean Ab titer that must be induced in a group of a minimum of 5 guinea pigs and 5 calves after vaccination with water-in-oil vaccines containing those Ag concentrations with 95% coverage. As detailed in Fig. 2, Fig. 3, the estimated split points allowed the classification of the vaccines in the guinea pig model and the target species by the two techniques as having “unsatisfactory”, “satisfactory” or “very satisfactory” potency.

Concordance between the vaccine quality predicted by the guinea pig model and the one obtained in the natural host by ELISA and VN is shown in Table 3, Table 4 . The initial analysis including 63 groups of animals immunized with vaccines of known antigen concentration, placebo or non-vaccinated control groups assigned as “reference groups” gave an almost perfect agreement in vaccine classification between the model and the target species by both techniques (Table 3a and b).

Table 3

Concordance between the guinea pig model and bovine, analyzed by ELISA (a) and VN (b).a.

Guinea pig/Bovine	Unsatisfactory x¯<1.69	Satisfactory 1.69≤x¯<2.72	Very satisfactory 2.72≤x¯
(a) ELISA; n = 63
Unsatisfactory x¯<1.93	36	2	0
Satisfactory 1.93≤x¯<3.02	0	5	3
Very satisfactory 3.02≤x¯	0	1	16

ELISA Weighted Kappa: 0.894; ASE = 0.041; 95%CI 0.813–0.974; p < 0.0001; almost perfect agreement.

VN Weighted Kappa: 0.876; ASE = 0.050; 95%CI 0.777–0.971; p < 0.0001; almost perfect agreement.

The analysis included the 20 assays corresponding to the calibration vaccines used to estimate the doses response curve; 8 placebos, 23 non-vaccinated groups and 12 assays including vaccines of known potency or reference vaccines.

Table 4

Concordance between the guinea pig model and bovine for the analysis of vaccine of known potency and commercial vaccines of unknown potency.a.

Guinea pig/Bovine	Unsatisfactory x¯ < 1.69	Satisfactory 1.69 ≤ x¯ < 2.72	Very satisfactory 2.72 ≤ x¯
(a) ELISA; n = 41
Unsatisfactory x¯<1.93	12	0	0
Satisfactory 1.93≤x¯<3.02	0	3	3
Very satisfactory 3.02≤x¯	0	2	21

ELISA Weighted Kappa: 0.865; ASE = 0.060; 95%CI 0.748–0.982; p < 0.0001; almost perfect agreement.

VN Weighted Kappa: 0.761; ASE = 0.082; 95%CI 0.601–0.921; p < 0.0001; substantial agreement.

The analysis included 12 assays including vaccines of known antigen concentration (known potency) referred and “reference vaccines” and 22 assays including commercial vaccines of unknown potency and their corresponding negative control groups (n = 7).

When excluding the pairs of groups used in the model estimation (20 vaccines used to estimate to calibration curve, 8 placebos and 18 out of 23 non-vaccinated control groups) and adding instead 22 commercial vaccines of unknown potency, the weighted kappa values were lower (Table 4). Nevertheless, the agreement remained almost perfect for ELISA and substantial for VN, according to the criteria established by Viera et al. [36]. Finally, when only the 22 commercial vaccines are analyzed, the concordance between the guinea pig model and the host species was almost perfect for both techniques (Supplementary data Table 4).

Protection against BoHV-1 challenge

In a first challenge experiment (Table 5 ), all placebo animals became infected, shed high titers of virus in the nasal secretions (mean peak BoHV-1 titer: 1 × 106.9  TCID50/ml) for 7 days after virus challenge. These animals presented an AUCi of 34.7 and developed typical signs of IBR, characterized by severe bilateral rhinitis (AUCs: 33.8). In contrast, all vaccinated animals challenged at 90 dpv were protected as they shed significantly less virus and developed a significantly less severe disease compared to the placebo group. Animals vaccinated with vaccine 1 showed significantly higher protection against IBR infection and disease than those vaccinated with vaccine 2, based on all the variables measured (Table 5). Significantly different protection rates observed in vaccine 1 and 2 animals was associated to the significantly different mean ELISA Ab titers of each group at 60 dpv.

Table 5

Protection against BoHV-1 challenge of calves vaccinated with Gold standard vaccines of different quality.

Vaccinea	n	ELISA Ab titer at 60 dpv	Bovines						Guinea pig
			Virus shedding			Clinical signs			ELISA ab titer at 30 dpv
			Peak of infectious virus titer (TCID50/ml)	Duration of virus shedding (days)	AUCib	Rhinitys severity	Duration (days)	AUCsc
1	6	3.76Ad	2.4C	2.3B	6.2C	1.5B	11A	15.7A	3.1A
2	6	2.6B	5.0B	6.7A	22.2B	1.9AB	12A	20. 6A	2.7A
Placebo	9	0.3C	6.9A	7.1A	34.7A	2.5A	14A	33.8B	0.3B

Vaccines previously classified by the guinea pig model and the host species as “very satisfactory” (1) and “satisfactory” (2).

AUCi area under the curve obtained by plotting the virus titer shed during 14 days after challenge.

AUCs area under the curve obtained by plotting the severity of the disease registered during 14 days after challenge.

Mean in the same column with different uppercase letters, indicates significant differences as determined by one-way ANOVA, p < 0.05.

In a second challenge experiment a significant difference between vaccinated and placebo groups, in terms of infection and clinical signs, was observed. No statistical differences were detected among vaccinated groups. However, only the group of calves with a mean ELISA Ab titer (2.8) higher than the established split point (2.72) showed a reduction of three days in the duration of virus shedding (Supplementary data, Fig. 1 and Table 3).

Discussion

Potency is defined as the relative activity of a biological product as determined by a test method conducted on the final product as described and approved by the regulatory agencies [16], [17], [26], [27], [28]. Potency testing of vaccines batches is an important component of the control tests conducted on the final product to confirm consistency of manufacturing and to ensure batch-to-batch quality [39].

Specifically for BoHV-1 killed vaccines, international regulatory policies [1], [11] recommend to evaluate vaccine quality in a potency test conducted in seronegative bovine calves. Briefly, the first batch of virus inactivated BoHV-1 vaccine should be tested in five calves susceptible to IBR infection (negative for neutralizing antibodies to BoHV-1). Calves are vaccinated following the manufacturer's recommendations (typically 2 doses, 30 days apart), while three animals should be left as non-vaccinated controls. Fourteen days after the second dose of vaccine at least 4 out of 5 vaccinated calves should develop virus neutralizing Ab titers of 1:8 or higher. Vaccines not reaching this potency requirement should undergo a challenge test by intranasal route. Two out of three control calves should develop hyperthermia and clinical signs of IBR, while only one of the vaccinated calves may develop clinical signs, otherwise the vaccine batch is considered of poor quality [11]. After experimental challenge, the OIE also requests the vaccine to reduce nasal virus shedding at least 100 times and virus excretion period in 3 days compared to control calves [1]. Thus, for the release to the market, a BoHV-1 vaccine must prevent the development of severe clinical signs and markedly reduce virus shedding after experimental challenge. Tests conducted in calves are expensive and time consuming to be used as a routine test for vaccine batch release, more so in countries like Argentina and other South American countries like Brazil and Uruguay where BoHV-1 and 5 infections are endemic [4], [6], [7], [10], [12]. In such regions finding seronegative bovines from BoHV-1 free herds, is becoming increasingly difficult. The use of a serologic test in guinea pigs, naturally seronegative for the BoHV-1 and 5, represents a convenient alternative method to evaluate batch-to-batch vaccine potency, which is in alignment with the 3R initiative of refining, reducing and replacing animal experimentation [15], [16].

The BoHV-1 Ab response in guinea pigs and bovines showed a dose-dependent pattern to the antigen dose. Under the conditions use, the minimum dose of antigen capable to induce detectable Ab response (detection limit or minimum immunogenicity dose) was in the order of 106  TCID50/dose of inactivated BoHV-1 in water-in-oil formulation, in both species as measured by VN and ELISA. The quadratic term of the mathematical equation obtained by linear regression analysis reflected the kinetic of antibody response, indicating that above certain Ab titer the Ab response can not be improved by the addition of higher doses of antigen [40]. The guinea pig model was able to significantly discriminate between vaccines formulated with Ag concentrations differing in 1 log10 and showed very good repeatability and reproducibility as recommended by the international guidelines [19]. It is important to highlight that the choice to test vaccines formulated with BoHV-1 antigen concentrations differing in 10-fold scale was based on preliminary data which indicated that no discrimination was observed between vaccine formulations having antigen concentrations within the same order of magnitude. This observation might be related to the fact that the infectious titer of the virus antigens used in the vaccine formulation as well as the virus used in the VN assay are conducted using 10-fold dilutions [38].

The experimental design of the trials conducted in bovine calves attempted to include all the possible sources of variation that can be found in the field (seronegative animals from BoHV-1 free or infected herds and seropositives animals from endemic herds). Despite this high variation, the precision of the target species for testing vaccines containing 107 and 108  TCID50/dose showed CV lower than 20%, indicating that repeatability and reproducibility of vaccine classification in the natural host by both techniques, VN and ELISA, was quite acceptable.

The sampling time points for comparison between the guinea pig model and the target species was 30 and 60 dpv, respectively, representing the peak or plateau of BoHV-1 Ab response to vaccination with a BoHV-1 inactivated vaccine and the earliest time point of high discrimination capacity.

The determination of split points to generate classification criteria is one of the steps in the development of a laboratory animal model, as well as, the verification of its predictive ability on the target species. The agreement in the classification of vaccine potency between the two animal species (the laboratory animal model and the natural host) based on these split points was excellent for ELISA and very good by VN [36] indicating that the developed guinea pig model represents a very good tool to predict vaccine immunogenicity in bovines.

In the concordance analysis conducted to evaluate the agreement between the laboratory animal model and the host species for vaccine classification there are different degrees of misclassifications affecting the vaccine manufacturers or farmers. In this regard, the use of the two techniques (ELISA and VN) for vaccine classification is recommended as an optimal method to reduce classification discrepancies between the model and the target species. For example, in the first concordance analysis, the classification of two vaccines by ELISA as “unsatisfactory” by the guinea pig model while they were classified as “satisfactory” by the natural host, may constitute an economical harm for the vaccine manufacturer if only one technique is considered. One of these cases corresponded to a calibration vaccine containing 106  TCID50/dose of BoHV-1, misclassified by guinea pig by ELISA and properly classified by the model and the host species by VN. The other one was a non-vaccinated control group with pre-existent ELISA antibodies. In this case the guinea pig properly classified the group as “unsatisfactory” by both techniques, while bovine misclassified it by ELISA.

On the other hand, when analyzing the agreement between species by VN, there was one vaccine that was classified as “unsatisfactory” by guinea pig but “very satisfactory” by bovine calves. In this case a failure in the guinea pig immunization was verified, since the same vaccine re-tested twelve months later was properly classified as “satisfactory” by both species and both techniques. On the basis of this incident it was decided to include a reference vaccine in every immunization test as a positive control in every guinea pig immunization for full validation and transference of the model. The classification of a vaccine as “satisfactory” or “very satisfactory” when it is actually unsatisfactory and fails to protect cattle represents a risk for the livestock producers. This kind of misclassification occurred in two cases only by VN and it was due to a technical error (a laboratory mistake in a given VN run) Again, for these special cases the use of both techniques (ELISA and VN) allows to clarify the source of discrepancy.

Other kind of misclassification does not mean an important harm for the industry nor the vaccine producer since the quality of the vaccines classified within both categories, “satisfactory” and “very satisfactory”, are accepted by the regulatory agencies [1].

Regarding the second concordance analysis, including commercial vaccines, only five out of 22 were classified as “unsatisfactory” by both species and both techniques, suggesting that some vaccine manufacturers should improve the quality of their products in order to reach the proposed standards.

Vaccines assigned to different immunogenicity categories, by the proposed statistical model, in guinea pigs and calves, also showed significant differences in protection against infection and disease after challenge in the natural host. Despite their differential immunogenicity and efficacy both, the “very satisfactory” and “satisfactory” vaccines, were able to pass the OIE requirements for approval and release [1]. Regarding the evaluation of vaccine efficacy, it is important to highlight that among the different outputs measured after experimental challenge; the AUCi was the best indicator to evaluate protection against challenge and also agreed with vaccine categories in terms of immunogenicity or potency.

The second challenge experiment conducted on calves from Set 2. Trial 3 indicated that protection data was in agreement with the unexpectedly low Ab titers developed by these animals. Calves with ELISA Ab titers lower to the selected cut off point have a poor protection against challenge. This bovine trial disqualified vaccines of good quality. Although bovines selected for the trials showed good clinical status and body condition, the presence of subclinical pathologies that might interferes with the immune response can not be discarded and may affect the outcome of a vaccine efficacy experiment in the natural host. This fact is commonly observed during herd vaccination.

Although significant progress has been made in using in vitro tests to evaluate antigen quality parameters [41], [42]. Models to measure veterinary vaccine potency are still based on immunization/challenge assays in the natural host or laboratory animals. The use of in vivo models for vaccine potency testing and lot-release are irreplaceable for the moment [15]. From the obtained results we concluded that the developed guinea pig model is a reliable tool to estimate batch-to-batch vaccine potency, avoiding the problem of finding BoHV-1 seronegative bovines in endemic countries, reducing the variability that can be found in bovines and significantly reducing the cost and the duration of the test. The model is also ethically compliant with the refinement, reduction and replacement principle in the use of animals, that can also take part in the new consistency approach [15], [16], [43].

The serology results in guinea pigs were statistically validated as a reliable indicator to predict vaccine immunogenicity and protection against challenge in the natural host. Further inter-laboratory studies are under progress to finish a full validation and transference of the model to the Argentinean National Health Authorities (SENASA). A laboratory animal model for veterinary viral vaccine potency testing such as the one proposed here would be of use to other regional and international regulatory agencies.

In particular, the proposed model is innovative since it was designed to evaluated killed combined vaccines [17] and after full validation it will allow to determine in one immunization assay, using 5 guinea pigs, the immunogenicity for all the viral antigens included in the vaccines currently imarketed for cattle. In this regard, the Argentinean National Authorities started to test the use of the developed model for IBR vaccine potency testing to control the vaccines present in the local market. In addition, the validation of the model for bovine parainfluenza type 3 and rotavirus are completed and the validation for BVDV and BRSV is under progress.