Methodological frontiers in vaccine safety: qualifying available evidence for rare events, use of distributed data networks to monitor vaccine safety issues, and monitoring the safety of pregnancy interventions.

While vaccines are rigorously tested for safety and efficacy in clinical trials, these trials do not include enough subjects to detect rare adverse events, and they generally exclude special populations such as pregnant women. It is therefore necessary to conduct postmarketing vaccine safety assessments using observational data sources. The study of rare events has been enabled in through large linked databases and distributed data networks, in combination with development of case-centred methods. Distributed data networks necessitate common protocols, definitions, data models and analytics and the processes of developing and employing these tools are rapidly evolving. Assessment of vaccine safety in pregnancy is complicated by physiological changes, the challenges of mother-child linkage and the need for long-term infant follow-up. Potential sources of bias including differential access to and utilisation of antenatal care, immortal time bias, seasonal timing of pregnancy and unmeasured determinants of pregnancy outcomes have yet to be fully explored. Available tools for assessment of evidence generated in postmarketing studies may downgrade evidence from observational data and prioritise evidence from randomised controlled trials. However, real-world evidence based on real-world data is increasingly being used for safety assessments, and new tools for evaluating real-world evidence have been developed. The future of vaccine safety surveillance, particularly for rare events and in special populations, comprises the use of big data in single countries as well as in collaborative networks. This move towards the use of real-world data requires continued development of methodologies to generate and assess real world evidence. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Assessment of vaccine safety has been enabled through availability of observational data sources and development of new methods, but gaps still exist for the study of rare events and assessment in special populations.

Collaborative studies in distributed data networks increase power but require the use of common protocols, data models, analytics and definitions as well as methods for data privacy preservation.

Inclusion of low-income and middle-income country data sources and efficient reuse of tools and expertise generated in collaborative studies requires sustainable funding for capacity building and readiness.

As more vaccines are recommended in pregnancy, increased attention to the quality and completeness of vaccine safety data surrounding pregnancy is necessary, as is additional research into sources of bias relevant to the study of exposure during pregnancy.

While randomised controlled trials represent a gold standard, the inability of trials to answer all relevant questions must be accepted and criteria for assessing quality of real-world evidence incorporated into decision making on the safety of vaccines.

Introduction

The awareness that medicines can cause harm has been known since ancient times. However, it was the thalidomide incident of the late 1950s and early 1960s that stimulated the monumental shift to proactively regulate drugs internationally. While new mandates around drug registration were implemented, there were also new regulations for postmarketing surveillance. Developments in ways to code adverse events emerged through the 60s and the 70s and continue today, and drug adverse event reporting systems have been established. In 1978, a feasibility study to evaluate standardising spontaneous reports funded by the US Food and Drug Administration (FDA) evolved to what is now known as the International Pharmacovigilance Monitoring Centre based in Uppsala, Sweden. However, while modern pharmacovigilance incorporates spontaneous reporting data and experience from over 128 countries exists to detect signals, the data are not suitable to estimate risk.1

At the time of their licensure, vaccines have been assessed for efficacy, quality and safety in randomised controlled trials (RCTs). While randomised studies are traditionally viewed as the gold standard for vaccine outcomes, they are usually limited by their sample size when it comes to detecting very rare events or long-term outcomes.2 Additionally, during public health emergencies such as the 2009 H1N1 influenza pandemic, recent Ebola epidemics and the 2020 SARS-CoV-2 pandemic, vaccines may by necessity undergo expedited clinical trials and licensing. This is where postmarketing surveillance, based on observational studies, is required.

Pharmacoepidemiology is the study of the use and effects of drugs and other medicinal products in populations. It uses epidemiological methods to study effects in large numbers of people and can contribute important information about the effectiveness and safety of a vaccine that is not available from a prelicensure clinical evaluation. The importance of observational studies in complementing clinical trials of drugs is a well-established cornerstone of pharmacoepidemiology. Observational studies in vaccine safety include a range of methodologies that essentially compare the occurrence of outcomes of interest among people exposed and unexposed to a vaccine. These include the cohort, case-referent (ie, case-control) studies, self-controlled studies, analyses of secular trends (ecological studies), case series and case reports, in descending order of levels of evidence.3 Vaccine safety studies rely on the first three approaches to assess risk, while the others serve to generate hypotheses. In recent years, sequential analyses have also been used to prospectively monitor suspected vaccine safety concerns.4

Over the past three decades, the ability to conduct observational studies has evolved to a point that extremely large, heterogeneous populations can be used to assess vaccine safety outcomes. Traditional methods have used cohort and case-control designs but more recently newer designs such as the self-controlled case series have found favour in overcoming potential confounder and selection bias (table 1).5

Table 1

Postmarketing evidence generation in vaccine safety76

Level of evidence	Designs	Data sources	Outputs
Signal detection/Hypothesis generation	Observed vs expected analyses	Spontaneous reports, observational databases	Binary signal/non-signal based on predefined thresholds and observed values
	Scan statistics	Observational databases	Binary signal/non-signal based on event clustering
	Self-controlled case series	Spontaneous reports, observational databases	Incidence rate ratio
	Time-to-onset analyses	Spontaneous reports	Binary signal/non-signal based on predefined thresholds and p values
	Sequential methods	Observational databases	Binary risk/non-risk based on predefined stopping rule
Hypothesis strengthening	Ecological methods including interrupted time series (ITS)	Observational databases, surveillance data	Incidence rate ratio, p values for slope or level change (in ITS)
Hypothesis testing	Cohort-based studies	Prospective cohorts, observational databases	Absolute risk, hazard and survival functions
	Case-referent studies	Observational databases	Odds ratio, Hazard ratio
	Self-controlled methods	Observational databases	Incidence rate ratio

*Observational databases include population-based health database such as administrative billing and electronic health record databases.

While monitoring the safety of vaccines to the highest standard achievable using the tools available is fundamentally ‘right’ and scientifically important, there is also a need to address perceptions about vaccine safety. In part due to the well-coordinated international antivaccine movement, vaccine hesitancy has risen to become one of the top 10 threats to global public health.6 Achieving and maintaining public confidence in vaccines needs rapid and credible responses to vaccine safety concerns, real or perceived. It is concerns about vaccine safety that contribute the most to vaccine hesitancy and being able to offer reassurance and effective communications supported by high-quality data is essential.7

Another reason for conducting large observational postlicensure vaccine safety studies is to examine potential associations with rare events, differences between vaccine brands and formulations and possible risks in special subpopulations. Rare events such as Guillain-Barre syndrome (GBS) pose challenges with respect to study power. For example, to detect a twofold increased relative risk with a background incidence of 1/100 000 and assuming a 1:1 ratio of vaccinated to non-vaccinated subjects, a study population of over 4.7 million is required.3 Such risks cannot be assessed in clinical trials, nor among smaller populations. Only through large administrative data collections for populations in the millions can a vaccine-associated risk for rare events be assessed. In addition, the globalisation of vaccine manufacturing increases the need to be able to compare brands, as illustrated by measles vaccines and infant mortality,8 mumps vaccines and aseptic meningitis9 and influenza vaccines in association with febrile convulsions10 or narcolepsy.11 12 Finally, there is also the need to assess the safety of vaccines in subpopulations such as pregnant women and their infants,13 people with certain health conditions such as immune suppression14 and among different ethnic groups15 and indigenous peoples.16 These subpopulations are often excluded from RCTs. To these ends, linkable data for large and diverse populations is required.

In recent years, there have been major advances in the tools and methodologies required to undertake the types of large, robust observational studies needed to address these diverse challenges. In 1990, the US Centers for Disease Control and Prevention in collaboration with large managed care organisations created the Vaccine Safety Datalink (VSD).2 This project linked medical events and demographic information with vaccine exposure information. Since then the system has evolved to include more populations, changes in the way data are collected, the development of near real-time data files and the ability to monitor new vaccines or changes in vaccine use in near real-time.17 Since the early 1990s, the number of publications internationally referring to multiple observational databases has exploded thanks to increased existence of electronic databases, the appreciation of the importance of data-linkage and information technology for analysing multiple databases.18 Recent collaborations such as Vaccine Adverse Events Monitoring and Communication (VAESCO), Accelerated Development of Vaccine Benefit-risk Collaboration in Europe (ADVANCE/VAC4EU), Systematic Observational Method for Narcolepsy and Influenza Immunisation Assessment (SOMNIA) and the WHO Global Vaccine Safety-Multi Country Collaboration (GVS-MCC) have successfully incorporated data from multiple countries to produce estimates of vaccine safety and effectiveness.12 19–22 The purpose of this paper is to appraise current methodologies in postmarketing vaccine safety in terms of how these can be used to assess rare events and special populations, and how evidence derived from these studies can be evaluated.

Studying rare events

Self-controlled case-only designs and other methods

Over the past 20 years, there have been an increasing number of epidemiological studies assessing vaccine safety for rare events which cannot be easily assessed in trials. This has been made possible through the availability of large linked databases and innovative tools for analysis. While traditional case-control and cohort studies continue to be used, novel designs such as case-only, case-coverage and case-centred methods have been developed and evaluated for use with these databases.23–27 Improved understanding of the benefits and limitations of the methods and data sources in terms of potential bias and confounding helps determine which are most suitable for addressing a specific adverse event and vaccine.

Case-only designs, which only require an unbiased set of cases, are particularly useful for assessing rare events due to their efficiency in only needing cases and their elimination of time-invariant measured and un-measured confounding factors. Such factors may be multifactorial and difficult to measure and could include frailty, socioeconomic status and levels of healthcare usage. The most commonly used case-only method is self-controlled case-series (SCCS) and its derivatives (eg, self-controlled risk-interval).23 25 SCCS requires definition of a follow-up period in which cases and their vaccine history are obtained, along with risk intervals postvaccination and baseline intervals for comparison. Another method, developed prior to SCCS and applied in the GVS-MCC study of measles and mumps containing vaccines, is the case crossover method in which vaccine exposure in risk windows just prior to the event is compared with earlier control windows within cases.21 24 Case-coverage and referent designs use data on cases and their vaccination status and compare this with vaccination uptake in a referent population or cases which may be from a different data source.26 27 These novel tools have proved highly useful in recent decades for assessing rare events and with more availability of linked databases should continue to be used and developed.

Detection, power, specificity, bias and delayed onset

To study an adverse event, it is first necessary to identify it as a possible adverse reaction. Events likely to be missed and therefore requiring methods for identification are those that are rare, new, unexpected and with delayed onset after vaccination. Even when there is a signal, or other reason to study a rare event, it is often the case that this cannot be done rapidly due to the rarity of the event. For new vaccines, this may mean that the full safety profile will remain unknown for a long time and this limitation should be documented and solutions developed. Large-linked databases and collaborative studies will enhance power, but many countries, including many low-income and middle-income countries (LMIC), do not have such databases. This means that, for vaccines used only in these countries, enhanced safety assessment with epidemiological studies is limited. Case-only methods offer a good solution due to their efficiency but still require the ability to identify an unbiased set of cases with linked vaccination records. While some adverse events are well diagnosed and coded in records, others may have non-specific symptoms with unclear onset meaning coded events have a low positive predictive value. Validation, while desirable, may not be possible or expensive. It is therefore important to understand methodologically the implications of assessing events where specificity may be lacking. Further work on bias analysis would be beneficial in this area as well as the merits of validation on subsets of cases. Finally dealing with confounding, particularly in studies where case-only methods are not appropriate remains a challenge, particularly with events with onset many months or years after vaccination.

Collaborative studies

Although most epidemiological studies of rare events to date have been done within countries, this requires the availability of data from large study populations with high vaccine exposure or over long periods of time. Collaborative studies in which data or results are combined between countries or regions within a country offer the opportunity to assess rare and very rare adverse events where numbers within one site are insufficient to address a question. They may also enable assessment within subpopulations such as pregnant women or using data from several smaller populations such as those obtained through sentinel hospitals. Collaborative studies of vaccine safety are complex due to the variety of potential signals, available data sources and possible study designs. A solution has been successfully employed within the VSD (see ‘Distributed data networks’ section), where a common methodology and analysis is possible when using a network of healthcare organisations.28

There have also been several international collaborative studies in the past decade. This was first done in Europe through the VAESCO consortium for Pandemic influenza and GBS (7 countries) and then internationally through the Global H1N1 GBS consortium (10 countries), the SOMNIA pandemic vaccine narcolepsy study (6 countries) and the proof-of-concept WHO Global vaccine Safety Multi Country Collaboration in LMICs assessing measles-containing vaccines association with thrombocytopenic purpura and aseptic meningitis (16 countries).12 19 29–31 There have also been bipartite collaborations such as Denmark and Sweden for human papillomavirus (HPV) vaccine and autoimmune conditions.32 33 Most recently, the ADVANCE project created a system for monitoring vaccine benefits and risks and is now continuing as the VAC4EU.20 34–39 Most of these studies have involved data sharing to a hub and while some have involved a single methodology, others have allowed different study designs. Key issues to have arisen are the cost and timeliness of such studies and in LMIC having sentinel hospitals that code discharges and link to vaccination data.

Distributed data networks

Common data models, protocols, analytics, definitions

Currently, distributed data networks can be described along two axes: first, whether they employ an ad hoc or dedicated network and second, the extent to which they use common standards. Along the ad hoc to dedicated network axis, collaborative studies can be implemented either through a group of data sources assembled to address a specific study question or through a sustained database network via which studies can be deployed rapidly. Many examples of distributed analyses conducted within ad hoc networks exist (see ‘Collaborative studies’ section) while the only examples of dedicated networks specific to the study of vaccines are the VSD2 and the ADVANCE/VAC4EU in Europe. Other dedicated networks of observational data sources include the FDA Sentinel System and the Observational Health Data Sciences and Informatics community, of which only Sentinel has been employed in the assessment of vaccine safety.40–42

These data networks also vary in the extent to which they employ common standards such as common protocols, definitions, data models and analytics. Common study protocols are written and agreed on by the network and implemented either centrally by distributing analysis scripts which can be run locally against data in a common data model (CDM) or implemented locally by each database. Common definitions used to harmonise data from heterogeneous sources include those for adverse events following immunisation (AEFI) developed by the Brighton Collaboration which require medical record review, or harmonisation of codes from various coding systems both for events and vaccine exposures in retrospective database analyses.43–45 CDMs harmonise the structure and content of the participating data sources to enable the use of common analytics developed centrally to run against all data sources. Finally, common analytics include data management, data quality assessment and analysis scripts which can be deployed against data in a CDM and, in a dedicated database network, can be modularised for reuse.

Models for analysis

Several models exist to analyse data within a distributed data network, the choice of which is dependent on both the privacy restrictions at each contributing data site and the extent to which the network has used common standards. The models vary in how much individual-level information is shared, how analysis results are combined and the autonomy of each site.46 At the extreme of local autonomy and privacy preservation, a common protocol is implemented locally against data in its original format and estimates from all sites are meta-analysed. An alternative which reduces local autonomy is one in which data are converted to a CDM and analysis is conducted locally using a programme written centrally, after which results are meta-analysed. This model can be adapted to allow for pooling of individual-level data by deploying centrally written scripts which generate aggregated or de-identified data sets which can be pooled centrally and analysed. At the extreme of centralisation and data sharing, all individual-level data are pooled and centrally analysed.

Data privacy and implications for one-stage versus two-stage meta-analysis

Typically, studies are conducted in individual sites and synthesis of evidence is conducted through systematic reviews or meta-analysis. In distributed data networks, depending on the model for analysis chosen, data may be retained locally at each site or shared at the aggregate or individual level. This choice depends primarily on privacy concerns and the desire for local control of the data. The approach of conducting analysis locally using only local data has the drawbacks that some data sources fall out of the analysis due to paucity of cases and resulting non-convergence, and that naturally occurring heterogeneity in vaccine formulations and schedules cannot be exploited. In the SOMNIA study, a hybrid approach was used, allowing for pooling of individual-level data from sites for which sharing was possible followed by meta-analytical pooling with results from sites for which sharing of individual-level data was not possible.12

Approaches to facilitate one-stage pooling have been discussed and include sharing of aggregate data sets or sharing of individual-level data sets. Currently, a lot of research is being conducted on approaches to privacy protecting distributed analysis methods. This includes development of DataSHIELD, which exploits a common underlying database structure to produce coefficients and estimates from data processed in parallel identical to those from data pooled at the individual level.47 More recently, simulation as well as empirical studies have shown that privacy-protecting methods, such as sharing of aggregated data sets, perform similarly to analysis in individual-level pooled data.48–50

Inclusion of LMIC and sustainability

To date, distributed data networks for assessment of vaccine safety have struggled to include data sources in LMIC due to unavailability of electronic databases, inability to link vaccination data to outcomes and lack of funding for capacity building in these areas. While studies such as the GVS-MCC have successfully been conducted using data from LMIC, further development of capacity in countries where new vaccines will be developed and deployed requires sustained funding and infrastructure. Additionally, sustained funding is required to enable efficient reuse, dissemination and development of tools and expertise developed in funded projects with a limited time frame such as VAESCO, ADVANCE and SOMNIA.

Studying exposure in pregnancy

Currently, vaccines are increasingly indicated for use in pregnancy, and additional vaccines are undergoing pregnancy trials. Vaccine safety assessment in pregnancy presents unique challenges as pregnancy is a physiologically dynamic state and safety considerations involve both the mother and the fetus. Levels of sex hormones estradiol and progesterone increase during the course of pregnancy and changes in sex hormones are associated with changes in the immune system. For example, as the pregnancy progresses, there is an increase in type 2 helper T-cell (Th2) responses and attenuation of type 1 helper T-cell (Th1); hence a Th1-to-Th2 shift in pregnancy.51 However, the increases in sex hormones and associated immunological changes are not perfectly linear. Moreover, these changes occur relatively quickly. Therefore, exposure assessment in pregnancy should account for the gestational age at vaccination. Often the gestational age at vaccination is grouped by trimester for statistical analyses. However, given the pace of change in pregnancy, trimester-based classification of gestational age might be too crude. If there is sufficient power, week of gestation is a better measure while assessing exposure time in pregnancy.

Considerations of toxic and teratogenic effects (ie, permanent anatomical, functional or developmental disruptions) on the embryo or fetus are dependent on the stage of development. Prior to implantation, adverse events can be a result of injury to a large proportion cells resulting in spontaneous abortion. If there is injury to a small number of cells, then often there is survival without abnormalities. In the embryonic period, that is, 2–9 weeks of embryonic gestational age, there is substantial organogenesis. Therefore, for exposures in this period, outcomes of concern include malformations and altered function. After 9 weeks of gestation through term, there is a period of fetal growth, differentiation and maturation; hence, outcomes of concern include preterm and small for gestational age birth, fetal death, minor malformations and altered function.

Exposure in pregnancy also warrants infant follow-up. While the duration of follow-up depends on the vaccine and the outcome of interest, there is increasing consensus on following the infant for at least 1 year after birth—particularly in maternal vaccine trials. The postpartum period also represents an opportunity to study the effects of antibodies in breast milk due to vaccine exposure in pregnancy or post partum on infant outcomes.

Because conduct of trials in pregnant women has implications for the health and safety of both mother and infant, their conduct is limited, increasing the need for observational studies.52 53 However, only a limited number of observational studies of vaccine exposure in pregnancy have been conducted and include assessments of influenza, tetanus, meningococcal, measles, mumps, oral polio, pertussis and yellow fever vaccines among others.54 55 This limitation to the number of studies conducted in large observational databases may be related both to the difficulty of conducting mother-child linkage and the range of adverse pregnancy and neonatal outcomes, which should be assessed in association with vaccine exposure. Additionally, observational studies of vaccination in pregnancy are prone to sources of biases including differential access to and utilisation of antenatal care, immortal time bias related especially to late-term exposures, seasonal timing of pregnancy and covariates related to adverse pregnancy outcomes such as nutrition, smoking and alcohol use which are typically not captured in observational data sources.56 Adverse pregnancy and birth outcomes must be assessed objectively using standardised definitions and, as is the case for all AEFI, availability of background rates is vital. Towards these ends, the GAIA Consortium has developed and tested case definitions for a set of maternal and neonatal AEFI.57 The US Sentinel system has developed and included a dedicated mother-child linkage table in its CDM.58 59

Options for evaluating existing evidence

When public health policy decisions are made regarding recommendations for vaccine usage, a potential safety issue or regarding the cost-effectiveness of a vaccine, decision makers often use criteria to evaluate the quality of the evidence that is available. The most common assessment tool is the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria.60 GRADE classifies studies into four levels of evidence—very low, low, moderate and high—where evidence from RCTs is rated as high quality and evidence from observational data is rated as low quality.

Despite widespread acceptance by recommending bodies including the WHO Strategic Advisory Group of Experts (WHO SAGE) committee, the assumption that observational data are routinely of low quality and randomised trials are always the best evidence source for recommendations is itself based on low-quality evidence and is increasingly being challenged.61 62 Frieden pointed out that while RCT data have good internal validity and assess efficacy in perfect situations, they can have very low external validity, which is now generally accepted.62 63 RCTs have inclusion and exclusion criteria that select certain populations and follow-up that does not reflect reality. In fact, the FDA, European Medicines Agency (EMA) and other regulators and health technology organisations are now embracing real-world data and real-world evidence (RWE) as the state of the art for safety assessments, because of the limited size and follow-up for RCTs. The 21st century Act places additional focus on the use of RWE for regulatory decision making.62 The EMA has developed the OPTIMAL framework and is increasingly using RWE for decision making.64

While treatment assignment is not randomised in observational studies (leaving potential for confounding) and data is not always source verified (leaving space for bias), they still can offer a robust ‘real-world’ picture of the effectiveness and safety of a vaccine in the target populations that it is being used, including people who might not have been eligible to participate in a clinical trial. Bias in observational studies is a problem to be addressed, as illustrated by conflicting results from studies of the purported association between hepatitis B vaccine and multiple sclerosis, some of which were subject to information and selection biases.65 However, study designs and methods have been developed to deal with bias and confounding such as the self-controlled case series, and the use and matching on propensity scores emulates a clinical trial. Assessment of the correctness of the outcome can be validated and exposure data can be bench marked.66 The availability of electronic health records in many countries has provided the opportunity to assess vaccine coverage, safety, effectiveness and impact on large populations in a retrospective fashion. The VSD in the USA (8 million source population) and the ADVANCE/VAC4EU infrastructure in Europe (40 million source population) are able to rapidly assess the safety, coverage, effectiveness and benefit-risk of vaccines postmarketing and the same infrastructures can be used to evaluate vaccine impact.28 67 68

To illustrate the poor predictive value of some RCT data, when results of a model based on real-world data for effectiveness for pneumococcal conjugate vaccine were compared with the prelicensure randomised trial for PCV7, it was evident that the prelicensure RCT may have underestimated vaccine impact for pneumonia and otitis media and thus the data from the RCT found the vaccine cost US$50 000 per quality-adjusted life year when in fact it was likely to be cost saving.69 This disparity was likely due to the inability to assess herd effects in the clinical trial setting. In a study assessing potentially reduced efficacy of herpes zoster vaccine when administered concomitantly with 23-valent pneumococcal vaccine, no increased risk of herpes zoster was detected in subjects co-administered both vaccines. This evidence contradicts findings of reduced efficacy from clinical trials based on measured antibody levels as a measure of efficacy.70 Similarly, for rotavirus vaccines, while prelicensure RCT trials with >100 000 individuals found no evidence of an increased risk of intussusception following two different RCTs, large studies using RWE postlicensure found such a risk.71

We should point out that our goal here is not to set up a ‘we versus them’ conflict with GRADE but rather to point out that in the case of vaccine safety, while GRADE has its strengths, observational studies do as well. Hence, blind adherence to the GRADE criteria in evaluating vaccine safety is inappropriate. Rather, we would argue that both randomised trial data and observational data should each be considered in light of their strengths and weaknesses.

Real-world evidence

The use of real-world data to generate RWE presents opportunities unavailable through randomised trials. For example, observational data in a diverse distributed network present opportunities to study questions such as the impact of differing childhood vaccination schedules including co-administration.38 72 Observational data sources also allow for application of case definitions and case-finding algorithms with differing balances of sensitivity and specificity as well as customisation on a per-data source basis.36 New sources of real-world data such as patient-generated data including that from apps present new opportunities as well as new regulatory challenges for use and interpretation of this type of evidence. Finally, the volume of real-world data allows for as-yet unexplored applications of machine learning and data mining beyond the data mining currently applied to data generated through passive reporting.

Conclusion

Current methodologies for postmarketing vaccine safety are sufficient, in general, to assess rare events and safety concerns in general populations. There needs to be further effort in identifying and applying these methods to special populations such as pregnant women, persons with comorbidities and diverse ethnicities. Moreover, there is a need to look at combination schedules rather than separate vaccines.

While the study of events too rare to assess within a single population is made possible through collaborative studies at larger scale, these studies also require tools for harmonisation: common protocols, data models, definitions and analytics. The future of vaccine safety surveillance comprises the use of big data in single countries as well as in collaborative networks and requires that networks have sustainable funding rather than being assembled ad hoc. Progress towards this goal is evident in the VAC4EU consortium, which aims to develop a sustained vaccine-benefit risk network for Europe, and the Global Vaccine Data Network currently in development, which aims to maintain a global network of vaccine experts and data owners capable of quickly deploying vaccine safety studies in populations beyond North America and Europe.73

While each of these approaches will lead to increases in the availability of RWE on vaccine safety, each requires access to large volumes of harmonised data, which in turn requires a sustained data network, alongside global capacity building in this area.

Due to barriers to participation in trials for pregnant and lactating women, observational studies are of particular importance in the assessment of vaccine safety in pregnancy. This requires sustainable networks employing CDMs that prioritise mother-child linkage and recording of variables which allow for accurate ascertainment of the timing of exposure during pregnancy. Additionally, there is much room for methodological development in addressing bias and confounding in studies of vaccine exposure during pregnancy.

Our ability to use RWE is growing exponentially. Methods to deal with confounding and bias have matured and guidance documents on the transparency of reporting of such studies and new tools such as Risk Of Bias In Non-Randomised Studies-of Interventions have been developed to evaluate the potential for bias in observational studies.74 75

It may be time to re-evaluate the GRADE criteria themselves and provide more balance in our evaluation of available evidence. It would seem more reasonable to argue, as Frieden has done, that there is no one perfect gold standard for evidence. Rather, RCTs and observational studies each have their own strengths and weaknesses and can complement one another as we make decisions regarding the vaccines that we use.