Against vaccine assay secrecy.

Increasing the transparency of the evidence base behind health interventions such as pharmaceuticals, biologics, and medical devices, has become a major point of critique, conflict, and policy focus in recent years. Yet the lack of publicly available information regarding the immunogenicity assays upon which many important, widely used vaccines are based has received no attention to date. In this paper we draw attention to this critical public health problem by reporting on our efforts to secure vaccine assay information in respect of 10 vaccines through Canada's access to information law. We argue, under Canadian law, that the public health interest in having access to the methods for these laboratory procedures should override claims by vaccine manufacturers and regulators that this information is proprietary; and, we call upon several actors to take steps to ensure greater transparency with respect to vaccine assays, including regulators, private firms, researchers, research institutions, research funders, and journal editors.

Why Transparency of Vaccine Assays is Important

Today, members of the scientific community, publicly funded research institutions, industry, and regulatory agencies face higher expectations for transparency and accountability. Registration of clinical trials is now legally required in several jurisdictions. Due to poor compliance, many are pushing for disclosure of full “clinical study reports” and anonymized patient-level data. Whether government regulators such as the European Medicines Agency (EMA) or the United States’ Food and Drug Administration (FDA) will follow suit remains to be seen. In the meantime, we draw attention to an issue that has yet to enter the policy conversation: open access to the scientific methods used to determine the immunogenicity of vaccines.

Vaccine efficacy and effectiveness can be quantified directly by evaluating protection from infection or clinical illness, or assessed indirectly by measuring immune responses (immunogenicity) to a given vaccine. Depending on the particular pathogen, vaccine, and other factors, immune responses can correlate, more or less accurately, with clinical protection. Although direct measurement of clinical protection is preferred, so-called “correlates of protection” are substituted when an efficacy trial would be inordinately large, expensive, not feasible because of the rarity of outcomes, or not ethically permissible when comparable vaccines are already available.

Quantitative assessment of serum antibodies before and after vaccination is the most commonly used correlate of protection, and often the only outcome accepted by regulatory authorities. Serologic measures are diverse, and the evidence in support of their validity as indirect measures of efficacy varies. Accurate and consistent findings in repeated studies across laboratories using such assays are thus critical to the successful application of correlates of protection.

However, efforts to standardize methods across settings cannot occur without access to the specific testing methodology, and vaccine manufacturers frequently treat the methods used to support the regulatory approval of vaccines as “proprietary.” In addition they may use reagents that are developed in house, are not available commercially, and are not described in sufficient detail to permit others to replicate the assay or findings.

This proprietary practice precipitates several problems. First, the lack of access to the reagents or standard operating procedures (SOPs) precludes independent validation of reported results. Second, the lack of availability to assays that are, in some circumstances, considered reference assays, precludes the ability of diagnostic laboratories to determine the performance characteristics of commercially available assays to measure antibody levels. Third, if summaries of the validation data that were generated when the assays were approved are not publicly available, there is no way to ensure that methods used in diagnostic laboratories are comparable. Fourth, treating the assays as proprietary precludes application of the assay to new populations that may not have been evaluated in the original studies (e.g. the immune compromised, aboriginal peoples).

This proprietary practice is not universally observed. A few manufacturers have taken an intermediate step of enabling technological transfer of assay methodology for their proprietary vaccines to select groups (e.g., Novartis’ meningococcal B vaccine) in order to ensure quality assurance by training specific individuals to perform the assay. However, details about the assays tied to several widely licensed vaccines (see ) remain entirely secret. We argue that the public interest in having the laboratory methodology used to determine the correlates of protection and summaries of validation data open to independent scrutiny is a threat to public health and clearly outweighs manufacturers’ competing business interests.

Table 1.

Ten vaccines licensed by Health Canada encompassed in our access to information request

Vaccine		Manufacturer	Date of Most Recent Canadian Licensing	Drug Identification Number
Zostavax II	Zoster vaccine live,attenuated [Oka/Merck]-	Merck Canada Inc.	31 October 2011	02375516
Zostavax		Merck Canada Inc.	22 August 2008	02315939
Varivax III	live, attenuated, univalent varicella virus vaccine, (Oka/Merck)	Merck Frosst Canada Inc.	19 June 2002	02246081
Varivax II		Merck Frosst Canada Inc.	26 August 1999	02240776
Varivax		Merck Frosst Canada Ltd.	2 December 1998	02239199
Pentacel	Diphtheria(D)-tetanus(T)-acellular pertussis(aP)-inactivated poliovirus(P) and Haemophilus influenza(Hib)	Sanofi Pasteur Ltd.	12 May 1997	02231343
Quadracel	DTaP-IPV	Sanofi Pasteur Ltd.	20 March 1997	02230946
Tripacel	DTP	Sanofi Pasteur Ltd.	10 July 1996	02230273
Cervarix	Human Papilloma Virus(HPV), bivalent	Glaxosmithkline Inc.	3 February 2010	02342227
Gardasil	HPV, quadrivalent	Merck Frosst Canada Ltd.	10 July 2006	02246081

Challenging Proprietary Practices in Vaccine Research and Regulation

We sought to challenge the status quo using Canada's Access to Information Act (ATI Act). In a letter dated March 12, 2012, we requested from Health Canada information that we believe to be contained in the SOP accompanying 10 vaccines that have been licensed for sale and use in Canada. Specifically, we requested 2 types of information from each vaccine's SOP: (a) the detailed assay protocols (i.e. methods) for the serological tests that were used to measure the immunogenicity of the vaccine in support of its market authorization; and, (b) the assay validation data that was submitted to Health Canada by the manufacturer. (See )

Table 2.

Vaccine assay information encompassed by the access to information request.

Laboratory Assay Standard Operating Procedures
1. Sample collection
2. Sample preparation
3. Reference standard for comparison
4. Assay methods
5. Reagents
6. Equipment
7. Instrumentation
8. Quality control
9. Verification of results
Assay Validation Data
1. Accuracy: closeness of the mean test result to the true value
2. Precision: closeness of individual measures when repeated multiple times on multiple aliquots of a single specimen
3. Analytical Specificity: ability to differentiate and measure the target in the presence of other components in the sample
4. Analytical Sensitivity: The lowest measurement at which the assay can quantify the target with accuracy and precision.
5. Reproducibility: the precision between 2 laboratories or between the same laboratory over time.
6. Clinical Sensitivity: ability to identify specimens that are true positives based on the reference method or “gold standard”(i.e., immune)
7. Clinical Specificity: ability to correctly identify those that are true negatives based on the reference method (i.e. non-immune)
8. Stability: the stability of the measured target in various conditions in which it will be stored or assayed.
References
Food and Drug Administration: Guidance for Industry, Bioanalytical Method Validation. US Department of Health and Human Services, May 2001.

To facilitate processing of our request, we indicated that assay protocols and validation data connected to several vaccines were likely to cross-reference one another and we did not require multiple copies of the same information. Second, we excluded information about the manufacturing processes as they have been characterized as a potential “trade secret” in previous court decisions. Third, we anticipated that Health Canada would likely regard the information as “third party information” (i.e., a) trade secrets, b) financial, commercial, scientific or technical information, c) information that could result in financial or competitive harm, or d) information that could interfere with contractual negotiations by a third party) and thus exempt from disclosure under the ATI Act. Therefore, we invoked a provision in the ATI Act that allows the government institution to disclose third party information (save for trade secrets) if it is deemed clearly in the public interest.

After considerable delay and complaints to the Information Commissioner of Canada about Health Canada's non-response to our request, on August 19, 2013 we finally received a response to our ATI request. The letter from Health Canada, which enclosed a 272-page PDF document, noted that “some records, or portions of records, are withheld from disclosure pursuant to” sections 19(1) (personal information), 20(1)(b) (financial, commercial, scientific or technical information), and 20(1)(c) (information that could result in financial loss or gain, or prejudice the competition position of a third party) of the ATI Act.

Our review revealed that the disclosure provides no information regarding 8 of the 10 vaccines encompassed in our ATI request. Two HPV vaccines included in our request, Cervarix (manufacturer: GSK) and Gardasil (manufacturer: Merck) were included; however, most of the 272 pages are blank apart from a notation indicating that the information in question was withheld on the basis of the above exemptions. (See Supplementary Information for access to the complete disclosure) Other pages in the disclosure contain table of contents, names of figures and tables, and reference to validation data but all of the substantive information is redacted from those pages. At most, these pages suggest that one of the manufacturers (GSK) did in fact submit appropriate validation data to Health Canada during the regulatory review process. But it is not possible to evaluate or independently replicate those findings without further information about the procedures and correlates of protection that were used.

Health Canada's response was not surprising. Since it began publishing access to information data in 1991, the regulator has relied upon third party information exemptions more often than any other type of exemption under the ATI Act to deny, in whole or in part, access to information (See ). Nevertheless, we contend that treating vaccine assays as proprietary is clearly contrary to the public interest.

Figure 1.

Number of access to information requests processed and partially or fully denied by Health Canada, and number of exemptions relied upon to partially or fully deny such requests, 1991-2012. *Figure 1 was compiled using information contained in Health Canada's annual reports regarding access to information requests, which are publicly available. Data for fiscal years 1992-1993, 1996-1997, 1997-1998, and 1998-1999 is not publicly available, and only some data is available for fiscal years 2007-2008, 2009-2010, and 2011-2012).

A Legal Argument for Greater Openness

It is important to note at the outset that any personal information captured by our request was inadvertent. We do not contest Health Canada's decision to withhold the names of the individuals who carried out and reviewed the assay protocols and validation data in question. Our argument concerns Health Canada's decision to exempt all substantive information regarding the assay protocols and validation data from disclosure because it believed disclosure could harm a manufacturer's competitive position or deemed the information to fall within the scope of confidential information.

The business harm exemption should not apply to the information we requested. Conceivably, a manufacturer may assert that disclosure of the information may preclude its ability to commercialize an assay; thus disclosure of the assay protocol and validation data could result in financial loss, competitive harm, or compromise business negotiations. However, courts have repeatedly demanded more than mere assertions of harm (to competitive position) by companies, and there are no signs that the manufacturers implicated by our request are attempting to commercialize the assays accompanying their vaccines. As noted below, the assay associated with Merck's zoster virus vaccine provides a telling example to the contrary.

Conversely, the statutory exemption pertaining to confidential information appears to be the most cogent reason for denying disclosure of assay protocols and validation data. Health Canada referenced this exemption throughout the (largely blank) 272-page PDF it provided. Under Canadian law, to qualify for this exemption the information must be “(i) financial, commercial, scientific or technical information; (ii) confidential and consistently treated in a confidential manner by the third party; and (iii) supplied to a government institution by a third party.” Each of these criteria appears to be met in this case.

This is why, in our original request to Health Canada, we invoked the “public interest override” contained in the ATI Act. The override allows for the disclosure of information notwithstanding its confidential nature if it “would be in the public interest as it relates to public health” and it “clearly outweighs” any financial loss or gain to a third party, or impact on its competitive position or any contractual or other negotiations.

Although Canadian courts have had little occasion to offer guidance around the scope of this kind of public interest override there are several reasons why it should apply in the case of vaccine assays.

First, the 10 vaccines encompassed in our request have tremendous public health importance. Hundreds of thousands of Canadians are vaccinated on the assumption that the studies submitted to the regulator accurately estimate the degree of clinical protection against the various pathogens targeted by the vaccines in our request. If those assays are somehow suboptimal, the measurements of antibody response may lead to inaccurate interpretations about vaccine efficacy, and thus inaccurate estimates of protection against serious infectious diseases. Open access to vaccine assays can, over time, help ensure decisions by health care payers are properly informed of a given vaccine's strengths and limitations as well as enable health care providers to have a better understanding of whether individuals and populations are actually clinically protected against a pathogen.

Second, regulators do not replicate the assay protocols or verify the validation data submitted by vaccine manufacturers. Rather, regulators simply review the protocols and data as submitted. Given the inherent challenges of utilizing serological measures as correlates of protection and demonstrated problems of vaccine assay standardization, opening up assays to independent scrutiny is essential to ensuring that the chosen correlates of protection do, in fact, provide adequate measures of vaccine safety and efficacy. Independent analysis will become increasingly important in the future as more vaccines are licensed on the strength of correlates of protection.

Third, on a related note, manufacturers have been known to manipulate the evidence base behind other health products such as pharmaceuticals in order to secure regulatory approval. GlaxoSmithKline's selective data reporting of “study 329,” which masked risks of increased suicidal ideation among adolescents taking a course of paroxetine (Paxil/seroxat), is perhaps the most infamous example. There are limits to regulators’ ability to detect this thus at least one major regulator, the EMA, is contemplating open access to anonymized patient-level clinical trial data, in part, to improve regulatory decision-making. There is a clear public health interest in improving access to clinical trial data. Precisely the same logic should apply to vaccine assays.

Fourth, disclosure of these diagnostic assays also carries benefits beyond enhancing decision-making of health care payers, providers, patients, and regulators. Specifically, it has benefits for research and research participants. At present, specific sub-populations (e.g. aboriginal populations, immune-compromised individuals) that were not included in the trials that supported licensure of these vaccines will be denied the benefit of these vaccines (assuming the benefits are confirmed) unless and until access to the correlates of protection is provided. That is, researchers planning to study the safety and efficacy of these vaccines in those sub-populations cannot do so absent access to the assays. Denying access to the assay information undermines equitable access to important public health interventions for particular populations. It also precludes efforts to assess the duration of protection over time.

Fifth, and finally, although it is possible that disclosure of the assay information may preclude manufacturers’ ability to commercialize an assay, there are no signs that the manufacturers implicated by our request are attempting to do so. We do not think the speculative possibility of commercializing a vaccine assay—a task that manufacturers have not undertaken despite years of assay use—should preclude disclosure, especially given the public health importance of having access to this information.

An informative example of this final point is the assay associated with Merck's varicella-zoster vaccine. Measuring immunity against varicella zoster virus (VZV) is challenging. While the fluorescent-antibody-to-membrane-antigen (FAMA) assay has been considered the gold standard for measuring immunity to VZV, this assay is cumbersome and is not amenable to automation or high-throughput testing. Merck developed a gpELISA, an enzyme-linked immunosorbent assay based on highly purified glycoproteins (gp) from VZV, which is more sensitive than FAMA. This gpELISA has been the standard method Merck has used to measure immunity in all of its subsequent VZV vaccine trials. While the methods were reasonably detailed in scientific publications, the validation of the assay was limited. In 2006, Merck published a paper on a second-generation assay which optimized the assay further and improved its performance characteristics. In this paper, Merck detailed the extensive validation undertaken; although the methods described were standard ELISA methodology, the antigen used and the standard operating procedures for completing the assay were considered proprietary, i.e. not publicly available. While Merck has granted limited access to the reagents and its standard operating procedures to reference laboratories in the past, Canada no longer has access to this method. As a result of Merck's decision to treat the gpELISA as proprietary, the majority of commercially available assays have not been validated against the gpELISA and their performance in determining VZV immunity is quite variable. Meanwhile, the Canadian Immunization Guide has recently recommended serologic testing for all health care workers who do not have previous VZV infection documented by a health care provider. As the cohort that has received VZV vaccine as part of the primary vaccination series enters the health care workforce, the shortcomings of the commercially available enzyme immunoassays in terms of detecting vaccine-induced immunity will translate into many more vaccinations that would be necessary because diagnostic laboratories do not have access to the gpELISA or its antigen to use for screening or as a reference method for the validation of other enzyme immunoassays. Nevertheless, despite the regulator's endorsement of Merck's gpELISA and a ready-made market of health care workers required to be protected against VZV, in a 10-year period, Merck has neither sought to commercialize its assay nor shown any signs of doing so.

From Argument to Action

The issue of transparency in the development and regulation of health products such as drugs and vaccines presently commands a great deal of attention. Although currently focused on clinical trial registration requirements and access to clinical trial data, we believe that access to vaccine assay protocols and validation data is critical from public health and ethical perspectives. Without access to that information, it is not possible to independently replicate and evaluate the assays used to establish the safety and efficacy of widely utilized vaccines.

We assert that Canadian law should not bar access to vaccine assay protocols and validation data despite manufacturers’ decision to treat this information as proprietary. On the contrary, the public interest override contained in Canada's access to information law should allow the regulator to disclose the information. To the extent they are available, similar flexibilities in other jurisdictions’ laws should be invoked to support disclosure by other regulators.

The inevitable delays in the access to information process and the public health importance of the vaccines included in our request and the fact that future vaccines are increasingly likely to be granted regulatory approval on the strength of serological correlates of protection requires more upfront transparency. Researchers, research institutions, research funders, vaccine manufacturers, and national regulators alike should proactively disclose assay protocols and validation data. As a start, we think that entities such as the International Committee of Medical Journal Editors and research funding organizations could integrate a requirement to make assay protocols and summary validation data into their policies regarding authorship and open access. With the support of their institutions, researchers involved in vaccine trials could stipulate a commitment to make protocols and validation data open to independent scrutiny, and regulators should be legally empowered to enforce such transparency as part of the regulatory process. Transparency may come in different forms. Ultimately, to ensure the putative public health benefits of a vaccine are achieved, meaningful independent scrutiny of vaccine assay protocols and validation data must occur.

Disclosure of Potential Conflicts of Interest

The lead author (MH) is a member of the Health Policy Translation Group of the Canadian Centres for Vaccinology (CCfV), which has carried out a number of clinical trials in partnership with vaccine manufacturers. MH has not been involved, in any way, with the conduct of these trials. The remaining authors have, however, received contract funding from numerous vaccine manufacturers to undertake vaccine clinical trials. Specifically, TH is currently a co-investigator on an investigator-driven collaborative agreement with GlaxoSmithKline and has had similar agreements with Immunovaccine and Microbix in the past 5-10 years. SH is an investigator in multiple clinical trials related to multiple vaccines funded by vaccine manufacturers including Sanofi Pasteur, GlaxoSmithKline, Merck, Pfizer, Novartis, PREVENT, and Folia. JL's institution has received funding for the conduct of clinical trials from Sanofi Pasteur, Merck, GlaxoSmithKline, Pfizer, Novartis, Immunovaccine, the Canadian Institutes of Health Research and for advisory boards (Sanofi Pasteur, Novartis, Merck, GlaxoSmithKline). JL serves on committees advisory to provincial and federal governments in Canada for vaccine policy. JL currently holds the Canadian Institutes of Health – GlaxoSmithKline Chair in Pediatric Vaccinology (2013–2018). All funding for this Chair is paid to Dalhousie University. As well, SH has served on ad hoc scientific advisory boards for GlaxoSmithKline, Novartis, Sanofi Pasteur, Pfizer and Merck and is a member of the Scientific Advisory Board for ImmunoVaccine. He has also served on advisory committees and working groups of the Advisory Committee on Immunization Practice of the US Centers for Disease Control, the Canadian National Advisory Committee on Immunization, the Nova Scotia Department of Health and Wellness, and the World Health Organization SAGE. He has served on the Data Monitoring Committee for Medicago.