Recommendations of the VAC2VAC workshop on the design of multi-centre validation studies.

Within the Innovative Medicines Initiative 2 (IMI 2) project VAC2VAC (Vaccine batch to vaccine batch comparison by consistency testing), a workshop has been organised to discuss ways of improving the design of multi-centre validation studies and use the data generated for product-specific validation purposes. Moreover, aspects of validation within the consistency approach context were addressed. This report summarises the discussions and outlines the conclusions and recommendations agreed on by the workshop participants.

Introduction to VAC2VAC - Vaccine batch to vaccine batch comparison by consistency testing

VAC2VAC is a wide-ranging collaborative research project funded by Innovative Medicines Initiative 2 (IMI 2) programme and brings together 21 partners in a public-private consortium involving experts from veterinary and human vaccine industry in a partnership with Official Medicines Control Laboratories (OMCLs), regulatory authorities, academia, translational research organisations, and vaccinology alliances. The ultimate goal of this project is to develop tests and approaches that will support acceptance of the “Consistency Approach” for established vaccines by the regulatory agencies and thereby significantly reducing the use of animals for batch testing in routine vaccine production in the future. To achieve their goal, the consortium is divided into seven interacting work packages (WP). The project partners in the first four WPs are developing, optimising and evaluating physico-chemical (WP1), immunochemical (WP2), cell-based (WP3) and bioinformatics-based assays (WP4) for routine batch quality, safety and efficacy testing of vaccines. WP5 is assigned to defining validation criteria, transferability and inter-laboratory validation of consistency approach methods. This is done in collaboration and consultation with regulatory agencies (WP6). The consortium management is allocated to WP7. More information on VAC2VAC is available on: http://www.vac2vac.eu/.

Workshop background and structure

The main objective of WP5 is to define the various aspects of validation covering establishment of criteria to evaluate the readiness of methods to enter the validation process and organization of small-scale validation studies to assess transferability and reproducibility of selected methods. It is further planned to draft proposals for the validation of the most promising methods in large-scale multi-centre studies to be run under the umbrella of the Biological Standardisation Programme (BSP) of European Directorate for the Quality of Medicines & HealthCare (EDQM; Council of Europe).

In order to qualify a new method or test procedure for potential inclusion into a pharmacopoeial monograph or a guideline, it has to be shown that the method is repeatable within laboratories, reproducible between laboratories and has sufficient discriminative power to detect vaccine batches which are of insufficient quality. In addition, any new method introduced for the quality control of a vaccine needs to undergo product-specific validation at manufacturer's level. Both multi-centre validation studies and product-specific validation are demanding with regard to cost, time and resources.

In light of this, WP5 organised a workshop to discuss ways of optimising the design of multi-centre validation studies and to consider whether data generated in the latter might also be used for product-specific validation purposes. The second part of the workshop focused on aspects of validation within the consistency approach context.

The VAC2VAC WP5 workshop was held from 30 January to 1 February 2017 at the European Commission Joint Research Centre in Ispra, Italy and chaired by Dr Catrina Stirling (Zoetis). It involved 30 experts from VAC2VAC partners, EDQM, members of European Pharmacopoeia (Ph.Eur.) expert groups for human and veterinary vaccines and sera as well as from working groups of the European Medicines Agency (EMA).

In preparation of the workshop, designated experts were asked to give presentations covering the following topics: current design of multi-centre validation studies under the umbrella of BSP, new documents related to the use of in vitro methods (e.g. Ph. Eur. general chapter 5.2.14 Substitution of in vivo methods by in vitro methods for the quality control of vaccines [1]) and use of data generated in multi-centre validation studies (e.g. draft EMA Guidance for individual laboratories for transfer of quality control methods validated in collaborative trials with a view to implementing 3Rs EMA guidance [2]) as well as statistical considerations of study design, data analysis and acceptance criteria. Furthermore, manufacturers and OMCLs presented their views on implementation of new methods for quality control. Specific aspects of (BSP) multi-centre validation studies and ways towards regulatory acceptance of new methods were in depth addressed in four break-out groups, i.e. design, critical issues, and presentation of the outcome of (BSP) multi-centre validation studies as well as strategies towards regulatory acceptance.

Summary of main keynote presentations

EDQM presented the design of BSP studies1 addressing the validation of new Replacement, Reduction, Refinement (3Rs) methods, which generally encompasses three phases. Phase 1 (or feasibility study) may last up to three years depending on the amount of work to be carried out, e.g. in-house validation, followed by a transferability study of the new method to achieve proof of concept, drafting of Standard Operating Procedures (SOPs), procurement as well as production and preparation of reagents and reference materials, identification of samples, drafting and negotiation of Material Transfer Agreements (MTA) with donators and collection of study materials followed by preparation and prequalification of sample panels and candidate references. Phase 2 is dedicated to the comparison of current routine in vivo test results with the outcome of the new 3Rs method, and this phase may include in vivo testing if needed. In Phase 3, the 3Rs method is evaluated in a large-scale collaborative trial, usually involving quality control laboratories of 10–20 participants. Participation of non-European laboratories is sought in order to support future international harmonisation. Phases 2 and 3 usually last from 1.5 to 3 years.

Currently, the BSP studies are aimed at validation of methods and establishment of reference preparations and can serve as basis for incorporation of 3R methods into Ph. Eur. monographs. Achieving this is a constant challenge as it requires extensive efforts and large support from all stakeholders (experts, OMCLs, manufacturers, etc.). EDQM is interested in improvement proposals, however, underlines that BSP studies are already complex and costly with regard to time and resources. Covering “product-specific validation” would require more resources, add an additional level of complexity and appears in most cases not to be achievable in the frame of a BSP study, as validation should be achieved for several products/manufacturers.

The draft EMA guidance for individual laboratories for transfer of quality control methods validated in collaborative trials with a view to implementing 3Rs [2] was presented. The aims of the guidance are to give some insight in how to introduce new 3R methods and to facilitate transfer of the quality control methods already validated in collaborative trials to a specific product/laboratory. By implementing methods validated through collaborative studies (e.g. BSP studies), various scenarios can be possible and the extent of validation needed for each circumstance is reported in the guidance.

The presentation on Statistical considerations for study design, data analysis and acceptance criteria underlined that statistical expertise may provide useful information to support study design, data analysis and setting specifications. It will be used at several stages in the process including: assay validation, assay transfer, multi-centre collaborative studies and method transfer and product-specific validation. In the current existing guidelines (e.g. ICH Q2 (R1) [3], VICH GL2 [4], EMA draft guidance on method transfer [2]) there is little information on the statistical details of method validation. Additional sources describing statistical methods that may be more relevant to the assays under consideration could be US Pharmacopoeia (USP) 〈1033〉 [5] and BSP study reports.2 For biological assay validation, statistical analysis is usually carried out to assess intermediate precision, relative accuracy, range and/or reproducibility. The statistical approach to study design and analysis should be considered in advance of the study; i.e. reasons for its choice depend on what information a new test method is generating and on the aim of the study. In some cases, it may be possible for the multi-centre collaborative study report to provide useful information in respect of method transfer and product-specific validation.

The Ph. Eur. general chapter 5.2.14 Substitution of in vivo methods by in vitro methods for the quality control of vaccines [1] provides general guidance “to facilitate the implementation of in vitro methods as substitutes for existing in vivo methods, in cases where a typical one-to-one assay comparison is not appropriate for reasons unrelated to the suitability of one or more in vitro methods.” The document includes general considerations associated with in vivo methods and their inherent variability making one-to-one comparison challenging or even impossible. It further outlines approaches on replacement of in vivo potency and safety tests and describes the most important criteria in vitro methods would need to fulfil, e.g. demonstrated scientific relevance for control of the relevant quality attributes and validation according to ICH/VICH guidelines (ICH Q2 (R1) [3], VICH GL2 [4]). Examples on substitution approaches are given for potency tests and safety tests (e.g. neurovirulence, specific toxicity, extraneous agents testing). As mentioned above, in the Ph. Eur. in vivo methods are typically replaced by in vitro methods following a multicentre validation study; nevertheless, general chapter 5.2.14 emphasises that such studies are not prerequisites to initiate replacement of an in vivo method for individual products.

Summary of discussions, conclusions and recommendations

Based on the presentations, comments and discussions the workshop participants agreed on the following conclusions and recommendations.

Implementation of new non-animal methods

It was emphasised that, in Europe, there are no legal barriers to substituting in vivo methods by non-animal methods. The General Notices of the Ph. Eur. provide instructions how alternative methods (to the official methods) of analysis for control purposes may be used [6]. Moreover, manufacturers are not required to use the methods listed in the Ph. Eur. monographs. Ph. Eur. encourages use of non-animal methods as noted in the Introduction [7], in the General Notices [6], General Chapters and e.g. general monographs for vaccines for human use [8] and vaccines for veterinary use [9]. This principle is underlined further with the General Chapter 5.2.14 Substitution of in vivo methods by in vitro methods for the quality control of vaccines coming into force from 01/01/2018 [1].

Ph.Eur. users should carefully read these texts since they include important general information not included in specific monographs which may be useful for implementation of non-animal based approaches.

With respect to practical implementation of new non-animal methods, early interaction between OMCLs and manufacturers is recommended in the context of the Official Control Authorities Batch Release (OCABR) and detailed SOPs should be shared.

Support to BSP/multi-centre validation studies

At present, most of the multi-centre validation studies are carried out under the umbrella of BSP; however, this is not mandatory and studies could be organised within other frameworks.

Currently, BSP studies aim at validation of methods and establishment of reference preparations. They can serve as basis for inclusion of a method into a Ph. Eur. monograph.

It was noted that most of the new methods submitted to BSP have been developed by OMCLs. Therefore, manufacturers are encouraged to submit new methods to BSP and become project leader/champion (or validate methods in consortia with support of BSP).

Early engagement of manufacturers could be enhanced by opening up a broad and early discussion on the needs and opportunities concerning 3Rs methods. Information from previous surveys, e.g. carried out by the European Partnership for Alternative Approaches to Animal Testing (EPAA) for the former Vaccine Consistency Project [10], may be updated and used.

Laboratories may profit from participation in BSP/multi-centre validation studies in several ways: they get access to new methods (incl. SOPs) and to adapted test panels, reagents and references that are not commercially available; they can provide feed-back on study results and have an impact on future candidate Ph. Eur. methods.

Availability of critical reagents, candidate reference materials, and appropriate samples including altered samples is crucial. Therefore, when a new method is proposed to be validated through BSP, ideally critical reagents (covering the project duration and time needed for product-specific validation) and suitable reference materials should be made available. Moreover, sustainable availability of reagents for later use in routine testing should be envisaged.

Donation of samples and a representative panel of intermediate or final products are crucial for the success of BSP/multi-centre validation studies. Moreover, manufacturers will be able to use the data generated (see 4.3). Therefore, manufacturers are encouraged to donate samples. Moreover, they should consider donation of failed batches and/or modified batches. EDQM would acknowledge receipt of such samples for study purposes in order to allow manufacturers to document the fate of these batches.

When available, the panel of samples should include at least one publically available reference standard (e.g. a World Health Organization International Standard or a Ph. Eur. Biological Reference Preparation [BRP]) that may serve as a link across the study and with potential complementary studies.

There was agreement that method development and validation studies are costly with regard to time and resources for all stakeholders. In the light of the increasing pressure to move away from animal testing, financial and human resources should be made available for the development and validation of non-animal methods, e.g. from public sources to EDQM BSP and OMCLs.

Optimisation of the outcome and use of data generated within BSP/multi-centre validation studies

To date, BSP study reports are mainly published in Pharmeuropa Bio & Scientific Notes (accessible online upon registration and free of charge)3 or in some cases in peer-reviewed journals. It was discussed whether validation study reports could be made even more informative to support future users of the method in defining a strategy for setting criteria for product-specific validation.

When a BSP study on 3Rs method validation is finalised, EDQM (often in collaboration with other stakeholders) typically organises seminars to discuss the results. These seminars are often co-sponsored by other public stakeholders and should be promoted by all partners (e.g. manufacturers' associations, EMA, national 3Rs platforms etc.) to make sure that the right audience participates.

Donors of samples can get access to all data generated for their donated samples, e.g. if the product of a manufacturer was part of a BSP study, the manufacturer could use these data for product-specific purposes. Moreover, manufacturers have the opportunity to run in parallel additional samples to establish product-specific standards.

In addition, the upcoming EMA Guidance for individual laboratories for transfer of quality control methods validated in collaborative trials with a view to implementing 3Rs [2] should be consulted. It lists possible scenarios and the extent of validation needed.

Product-specific validation of new methods

In most cases, the data from a BSP study cannot be used to set product specifications with the new method. Examples presented by manufacturers underline the importance of setting (new) specifications for existing products properly to avoid undue batch rejection when using the new method. Manufacturers emphasised that they would appreciate more guidance from regulators on how to set specifications/validity/acceptance criteria for new methods.

It was mentioned that some Ph. Eur. general chapters (e.g. 2.7.20 In vivo assay for inactivated polio vaccine [11]) include guidance on how to validate the respective in vitro method. However, including this also into other monographs was not considered to be an option. It was discussed whether the BSP study report might be an appropriate place to discuss on a strategy on how to set the specifications for a specific test, such as number of lots/batches to be tested and statistical analysis approach to be used.

It was further recommended that guidance on definition of acceptance criteria for new methods applied for already licensed products should be developed and discussed, e.g. with the relevant Ph. Eur. expert groups.

How to achieve regulatory acceptance?

There was agreement that procedures to get advice from regulatory authorities on the possible use of new methods as well as procedures for filing variations are in place for licensed products. The recently published EMA Guideline on the Principles of Regulatory Acceptance of 3Rs (Replacement, Reduction, Refinement) Testing Approaches may be consulted for this purpose [12].

Recommendations for the VAC2VAC project and possible follow-up activities

At present, the VAC2VAC project is run by European Partners. However, its impact is likely to be global. Various non-EU stakeholders show interest in the project. In order to promote consensus on consistency approach and the use of non-animal methods globally, it is planned to reach out to authorities outside EU, for example, US authorities, World Health Organization (WHO), World Organization for Animal Health (OIE), Asian authorities, African Vaccine Regulatory Forum (AVAREF), Pan American Health Association (PAHO) as well as manufacturers associations globally, for example, via Health for Animals and International Federation of Pharmaceutical Manufacturers & Associations (IFPMA). VAC2VAC partners disseminate information on the project in general; however, only when sound data or case studies become available competent authorities in and outside of Europe can be approached.

The VAC2VAC consortium will consult with competent authorities on best ways forward to set specifications for new methods to be used for quality control of already licensed products.

In order to foster the regulatory acceptance of new methods/approaches, provision of training is crucial and therefore, VAC2VAC partners will explore possibilities to provide training to stakeholders.

The VAC2VAC consortium and in particular the private partners, i.e. manufacturers, will discuss the creation of a platform for sharing methods.

It is foreseen in the VAC2VAC project to launch validation studies and to involve experts from EDQM at an early stage (e.g. transferability studies or small-scale collaborative studies). In order to save time, resources, and to allow to complete BSP studies faster, VAC2VAC methods proposed for validation under BSP should come with the following package: Reagent(s) for BSP study, and for the period necessary for in house validation after BSP as well as candidate reference material batch (es). In addition, the private and public partners of the VAC2VAC consortium generated a MTA template and are willing to share it which may reduce time-consuming negotiations in the future.

Conclusions

With the workshop in WP5 of the VAC2VAC consortium an open discussion has been started with all stakeholders - vaccine manufacturers of major human and animal health companies, competent authorities, OMCLs, EDQM, etc. - signalling a common commitment by all parties to the 3Rs principles. The general recommendations from this meeting can be considered guidance with regard to multi-centre validation studies and the replacement of animal testing for licensed products.

Disclaimer

The contents of this article are the views of the author and do not necessarily represent an official position of the European Commission.

Funding

The VAC2VAC project receives funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement N- 115924. This Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation programme and EFPIA.

Conflicts of interest

Jean-François Dierick and Frédérique Delannois are employees of the GSK group of companies.