Case-control vaccine effectiveness studies: Preparation, design, and enrollment of cases and controls.

Case-control studies are commonly used to evaluate effectiveness of licensed vaccines after deployment in public health programs. Such studies can provide policy-relevant data on vaccine performance under 'real world' conditions, contributing to the evidence base to support and sustain introduction of new vaccines. However, case-control studies do not measure the impact of vaccine introduction on disease at a population level, and are subject to bias and confounding, which may lead to inaccurate results that can misinform policy decisions. In 2012, a group of experts met to review recent experience with case-control studies evaluating the effectiveness of several vaccines; here we summarize the recommendations of that group regarding best practices for planning, design and enrollment of cases and controls. Rigorous planning and preparation should focus on understanding the study context including healthcare-seeking and vaccination practices. Case-control vaccine effectiveness studies are best carried out soon after vaccine introduction because high coverage creates strong potential for confounding. Endpoints specific to the vaccine target are preferable to non-specific clinical syndromes since the proportion of non-specific outcomes preventable through vaccination may vary over time and place, leading to potentially confusing results. Controls should be representative of the source population from which cases arise, and are generally recruited from the community or health facilities where cases are enrolled. Matching of controls to cases for potential confounding factors is commonly used, although should be reserved for a limited number of key variables believed to be linked to both vaccination and disease. Case-control vaccine effectiveness studies can provide information useful to guide policy decisions and vaccine development, however rigorous preparation and design is essential. Published by Elsevier Ltd.

Introduction

Many new vaccines have been introduced into public health programs over the past decade and others are under development. Vaccines are generally licensed based on safety and efficacy as measured in randomized controlled trials. Once vaccines are introduced into public health programs, their performance under “real world” conditions also needs assessment [1], including among populations with subgroups that may have been excluded from pre-licensure trials (e.g., malnourished or HIV-infected), with more variable dosing schedules (e.g. age at administration, interval between doses, number of doses), against outcomes not included in randomized clinical trials (e.g. strain-specific protection or mortality), and over more extended periods of time.

Furthermore, some vaccines are licensed based on immunologic correlates of protection [2], and post-licensure evaluations provide important information about protection against disease endpoints. After vaccines have been introduced, conducting placebo-controlled trials is generally not considered ethical [3]. Observational post-licensure evaluations are important to underpin policy decisions on vaccine introduction, to optimize the vaccine program implementation, and to provide evidence for sustaining vaccine use and investment from governments and donors.

Efficacy, effectiveness and impact

‘Efficacy’, ‘effectiveness’ and ‘impact’ are sometimes used interchangeably in everyday language, but in the context of vaccine studies the terms have come to be used with distinctly different meanings (although not entirely consistently) [4-7]. Their usage in this document is defined below:

is the percentage by which the rate of the target disease among those who are vaccinated according to the recommended schedule is reduced compared to the rate in similar unvaccinated persons. This is generally measured in the context of a placebo-controlled randomized trial as the “per protocol” efficacy (i.e. excluding persons who did not receive the recommended schedule), because the intention is to establish the biologic performance capacity of the product under optimal conditions.

measures the same percent reduction in the rate of disease as efficacy, but in the context of routine, real-world use of the vaccine. Vaccine effectiveness may be similar to the efficacy as measured in clinical trials. However, it often differs in magnitude because in routine use the population vaccinated includes some who may have a less robust immune response, and program implementation (e.g. cold-chain maintenance, dosing schedules) is more variable than in clinical trial settings.

quantifies the reduction in disease at a population level following introduction of the vaccine [7]. Impact can be expressed as a percentage decline or as an absolute change in the rate of disease. It is determined by a combination of vaccine effectiveness, vaccine coverage in the population, and any herd effect (i.e. vaccination of part of the population leading to reduced transmission of the infection in the community, and thus lowered risk of disease in both vaccinated and unvaccinated persons) [8].

Studies of vaccine efficacy, effectiveness, and impact may use non-disease outcomes such as colonization as endpoints; however disease endpoints are more commonly used.

Observational methods to assess vaccine effectiveness and impact

Several observational epidemiologic methods are used to assess the impact of vaccination programs and the effectiveness of vaccines in routine use [4,5,9]. Examination of trends in disease incidence before and after vaccine introduction measures vaccination program impact. However this approach requires a stable, unchanged disease surveillance system before and after the introduction of vaccine. Interpretation of such studies can be challenging because of changes in measured disease incidence or the true disease incidence unrelated to vaccination. For example, changes in healthcare seeking behaviors can increase or decrease measured disease incidence, concomitant implementation of non-vaccine interventions can reduce disease risk, and natural temporal variation in disease incidence unrelated to vaccination can also occur.

Vaccine effectiveness is generally measured through either cohort or case-control approaches. Cohort studies estimate effectiveness by comparing the incidence of disease among vaccinated and unvaccinated persons. Cohort studies require large samples, may be costly, and accurate data on the vaccination status and potential confounding variables for an entire population are often not available, especially in resource-poor settings. The cohort design may not be practical for diseases with low incidence. Case-control studies assess effectiveness by comparing the odds of antecedent vaccination among individuals who develop the target disease (cases) and among a group of individuals without the disease (controls) who are representative of the population from which the cases arise [10,11]. Because efforts are focused on accurately ascertaining disease status and vaccination history for a relatively small number of cases and controls (compared to cohort studies), the method can be resource-efficient and particularly useful for diseases or outcomes that are relatively uncommon. The screening method, in which the vaccination status of cases is compared to population-level vaccine coverage, is another approach for assessing vaccine effectiveness [12]; however accurate data on the proportion of the population vaccinated is often not available in resource-poor settings.

In recent years, case-control studies have been conducted to evaluate the effectiveness of Haemophilus influenzae (Hib) [13-24], pneumococcal [25-32], influenza [33], rotavirus [34-47], and cholera [48-50] vaccines. Despite being widely used to evaluate vaccine performance, the case-control methodology is susceptible to bias and confounding [51,52]. Because vaccine effectiveness estimates often impact policy decisions and donor support for vaccines their validity is important. In November 2012, a group of experts met to review recent experience with case-control studies evaluating effectiveness of several vaccines. We summarize the recommendations from that group regarding best practices for the preparation and design of such studies, as well as the enrollment of cases and controls. (Data collection, vaccine status ascertainment, analysis and reporting of results are discussed in a separate paper [insert reference for paired manuscript].) While discussions of case-control methodology in general can be found elsewhere [52,53], here we focus on the application of these methods to evaluate vaccine effectiveness in resource-constrained settings.

Methodological aspects of case-control vaccine effectiveness studies

Preparation for case-control vaccine effectiveness studies

Although data collection for case-control vaccine effectiveness studies begins after vaccine implementation, rigorous study planning and preparation, focused on understanding the local study context, should begin well before cases and controls are recruited, ideally a year or more beforehand. In the preparatory period, it is important to assess factors that may affect case ascertainment, such as healthcare-seeking behavior, barriers to care, determinants of hospitalization and diagnostic capacities. Different potential sources of control groups should be considered to identify the group least likely to lead to bias; for example, if cases are identified from a source population that includes large slum areas and controls are recruited only from more wealthy areas, the controls may be very different from cases in ways that could bias effectiveness estimation. Preparation should also include assessing vaccine coverage, factors associated with non-vaccination, and the ability to obtain valid, complete data on vaccination status among the intended study population. Identifying key potential confounders and the most accurate ways to measure them are also essential components of study preparation.

Prior studies of the outcome of interest in the local study context may inform case definitions and strategies for recruitment. For example, a “vaccine-probe” study in South Africa found that a widely used case definition for likely bacterial pneumonia, based on standardized interpretations of pediatric chest radiographs, underestimated the burden of pneumonia that could be prevented with the pneumococcal conjugate vaccine [54]; therefore a subsequent case-control vaccine effectiveness study used a modified case definition aimed at better capturing probable pneumococcal pneumonia cases in that setting [32]. Health care utilization studies provide important information on where cases might be identified for a case-control vaccine effectiveness study, as well as cases that may be missed by health facility-based studies [55-57]. Vaccine coverage surveys or analysis of immunization data from Demographic and Health Surveys or Multiple Indicator Cluster Surveys can offer insight on the completeness and timeliness of routine immunization in the intended study population, the availability of documented vaccine histories, and factors associated with non-vaccination that may be important confounders for a vaccine effectiveness study [58]. In the context of a Hib vaccine study in the Ukraine it was noted that providers considered underlying immunocompromising conditions to be a contraindication for receiving the vaccine; thus children at increased risk for Hib disease were less likely to receive the vaccine, potentially leading to an overestimation of the actual effectiveness in the full population [23]. Identifying important factors that influence likelihood of vaccination during the planning phase can help avoid bias during study implementation.

Sample size and feasibility

During the preparatory phase, the feasibility of achieving adequate enrollment during the planned study timeline must be assessed. The desired study size is determined by the expected effectiveness (with lower effectiveness requiring larger sample sizes), anticipated vaccine coverage in the study population, and the number of controls enrolled per case [9]. Study size may be based on statistical “power” (i.e. testing the hypothesis that the vaccine is significantly protective) or precision-based (i.e. targeting a certain width of confidence interval) [59]. Sample size calculations should allow for missing data, adjustment for confounding, and the expected prevalence of incomplete vaccination (e.g. one or two doses of a three-dose schedule). Once the desired sample size is determined, an assessment of the ability to enroll that target number must take into account the potential for declining incidence of disease over time following vaccine rollout, refusals, age-eligibility for vaccination, and ability to collect vaccination histories. Thus, simple sample size calculations should be considered as the minimum necessary number needed to assess the primary outcome, but enrollment beyond that minimum is likely required for a robust analysis and the ability to address secondary objectives (e.g. effectiveness in subgroups, effectiveness of incomplete schedule, and strain-specific effectiveness).

Several planned case-control studies of Hib vaccine effectiveness were not completed because of lower than anticipated enrollment attributable to rapid declines in invasive Hib disease burden following vaccine introduction (R. Hajjeh, personal communication, November 16, 2012). Case-control studies may have limited power if the number of available cases is small, which can occur following introduction of highly efficacious vaccines, in settings that achieve rapid, high coverage and significant herd effects.

Timing of study and vaccine coverage

Case-control studies are most likely to be useful when the vaccine coverage is between 20 and 80% [9]. At either very low or very high coverage, unvaccinated persons are likely to differ from the general population in ways that may be associated with increased or decreased risk of disease, independent of vaccination. These differences may be more pronounced where coverage levels are driven by individual factors (e.g. lack of access to care, mistrust of medical system) rather than programmatic factors (e.g. vaccine stock-outs). Results of several rotavirus case-control studies were difficult to interpret due to high coverage (>90%) soon after vaccine introduction [43,60]. Settings with high vaccine coverage (e.g. greater than 85–90%) are not suitable for case-control vaccine effectiveness studies because of the strong potential for confounding. High coverage also increases sample size requirements because more observations are required to detect a significant difference in vaccination between cases and controls. Furthermore, high coverage can lead to a rapid decline in cases of the disease of interest if vaccine efficacy is high. Thus, in contexts where the coverage is expected to increase quickly following vaccine introduction, it may be preferable to conduct a study in a short time period after introduction rather than a prolonged study with a slower rate of enrolment.

Study endpoints

Endpoints for case-control vaccine effectiveness studies range from highly specific for the vaccine target (e.g. invasive pneumococcal disease caused by a serotype included in the vaccine or rotavirus diarrhea) to non-specific (e.g. clinical syndromes such as pneumonia or acute gastroenteritis). Pathogen-specific endpoints have precise case definitions that are generally not open to interpretation or variability in the field application. Non-specific outcomes, however, may be of greater interest from a policy perspective because of the larger associated burden of disease, albeit the fraction of that disease preventable by the vaccine may be low. Yet effectiveness estimates from case-control studies of non-specific outcomes can be confusing or misleading. For example, a systematic review of Hib vaccine effectiveness noted that case-control studies using radiologically confirmed pneumonia endpoints may have overestimated effectiveness (compared to clinical trial estimates of efficacy against that same endpoint), although the reason for the high point estimates is unclear [61].

Protection against a non-specific endpoint depends on the proportion of the endpoint that is attributable to the pathogen targeted by the vaccine; this may vary over time or seasonally, be higher or lower in certain sub-groups (e.g. young infants, malnourished children) or be affected by outbreaks of other pathogens with overlapping clinical symptoms. Such variability can result in inconsistent estimates of effectiveness against non-specific endpoints between studies. For vaccines that lead to herd effect (e.g. Hib or pneumococcal conjugate vaccine), the proportion of a non-specific endpoint (e.g. pneumonia) attributable to the vaccine-preventable pathogen decreases among both vaccinated and non-vaccinated populations; thus as herd effects increase, effectiveness estimates for non-specific endpoints will decline. The risk for developing non-specific clinical syndromes such as all-cause pneumonia or diarrhea may also be strongly affected by individual-level non-vaccine risk factors (e.g. poverty, maternal education, crowding); such factors are difficult to measure well and may be associated with vaccination status. Furthermore, non-specific endpoints require enrolling larger numbers of participants, since effectiveness against non-specific endpoints is lower than that against specific endpoints [10]. Because of variability in the vaccine-preventable portion and the strong potential for bias, case-control vaccine effectiveness studies using non-specific endpoints must be interpreted with care, and are best conducted only when accompanied by analyses of disease trends over time or by a nested or parallel evaluation of a more specific endpoint in the same study setting.

Identification and enrollment of cases

Once the study endpoint is decided, the endpoint case definition must be clearly defined to avoid variable inclusion of cases during study implementation. It is not necessary to enroll all individuals who develop the disease in a given area or time period for a case-control study [10]. However, studies should report the proportion of eligible cases enrolled, since low enrollment may result in selection bias. Some vaccine effectiveness studies focus on a specific subset of cases because the effectiveness of the vaccine against the outcome is of particular public health interest (e.g. hospitalized or severe cases). The generalizability of the vaccine effectiveness will be limited to the types of cases included, and such restrictions must be taken into account in the interpretation of study findings [51]. Whenever possible and culturally acceptable, cases among children who have died should be included in case-control vaccine effectiveness studies, since they represent the most severe spectrum of disease and failing to include them could bias the effectiveness estimate if their likelihood of vaccination differs than that of cases who survive.

Sources of controls

In all case-control studies, controls should be representative of the source population from which the cases are selected [51,62-64]. A way of exploring this is to ask “If this control had developed the disease of interest, would he or she have been identified and included in this study as a case?” If the answer is no, then the control selection method is probably not appropriate. This question should be asked at the study design phase, when the source of potential controls is being determined.

Community controls

In many contexts, it is good practice to seek controls in the community in which the case resides, since those living in the community are most reflective of those who would be identified as cases if they were to fall ill [62]. The community from which the cases are derived from can be defined in various ways, depending on the study context and the available options for identifying controls. Population-based lists, such as birth registries or population-based databases in which the cases are included, can be used to randomly select potential controls [65]. For example, in studies of the pneumococcal conjugate vaccine in the US [17] and Brazil [66], birth registries were used to select potential controls, and in a study in Canada, controls were selected from a health insurance registry that included all residents in a province [27]. Such lists should be comprehensive and inclusive, since selecting children from an incomplete list may limit generalizability [62]. The basis for the list must not be associated with receipt of vaccines (e.g. immunization registries that include only vaccinated children). Lists with the appropriate characteristics often do not exist or are incomplete in many resource-poor settings, obviating this method for control selection. If such a list is used to identify controls, then cases not appearing in the list should be excluded.

Alternatively, community controls may be sought geographically, for example, around a case’s place of residence. Children from the same geographic area often tend to be comparable with respect to underlying risk of disease and access to vaccination, and it is possible to match on, or adjust for, distance to healthcare facilities if there is concern about differential access to care. Matching by neighborhood can also help control for a variety of potential confounding factors that may be difficult to measure, such as socio-economic status or other barriers to vaccination [65]. Geomapping of the population in an area can provide a sample frame from which to select geographically-matched controls, as was done for a Hib vaccine case-control study in Bangladesh [67]. A less sophisticated, but more commonly employed, strategy is to identify the household of the case, and then walk in a random direction (e.g. by spinning a bottle) from that residence, seeking a suitable control from the nearest neighboring house. This method is based on the approach developed for vaccine coverage surveys [15,21,58]. Having standardized procedures for visiting potential control households is essential for reducing selection bias [62]. Procedures should include the requirement to visit non-responsive households multiple times and at different times of day before excluding their residents as potential controls, since children whose parents are not at home might be more or less likely to be vaccinated than children whose parents are at home. Enrolling community controls can be logistically challenging and resource-intensive, particularly when tight age-matching criteria are used. Security concerns can also interfere with recruitment of community controls; investigators of Hib vaccine effectiveness in Colombia and Pakistan had to alter control recruitment strategies due to the safety risks associated with seeking neighborhood controls [24,68]. Conducting control recruitment in locations that are safer or more convenient can induce substantial biases in the vaccine effectiveness measure if residence in those areas is associated with higher likelihood of vaccination.

Hospital or clinic controls

Another common source of controls for case-control vaccine effectiveness studies are children hospitalized with illnesses other than the outcome of interest [62]. The specific inclusion criteria for hospitalized children to serve as controls must be carefully considered in the design phase of each study since the local conditions influence the risk of bias. Children who are hospitalized, particularly those with frequent or prolonged hospitalization, may differ in important ways, including vaccination history, from the general population; recruiting from among recently admitted children may avoid overenrolling children with severe prolonged illness as controls. Children hospitalized with vaccine-preventable diseases should be excluded as controls, as they are probably less likely to be vaccinated in general, including with the vaccine under study [51,62]. Where vaccines that protect against the most common childhood illnesses (e.g. gastroenteritis, pneumonia) are in routine use, and therefore children hospitalized with these illnesses cannot serve as controls, it may be challenging to identify enough eligible hospital controls [31]. In settings where access to health care is limited or hospitalization is largely restricted to certain subsets of children (e.g. children with malnutrition), then hospital controls may have the advantage of being relatively comparable to hospitalized cases with regards to access to care [62]. However, results of a study in such a context are only generalizable to children who would be hospitalized when ill.

Controls can also be identified in out-patient clinics that cases would attend if ill [65], an approach used for Hib vaccine effectiveness studies in Colombia [68] and Ukraine [23]. However, if immunizations are delivered at the clinic, then controls attending the clinic would be more likely to be vaccinated than the general population, as was found in a study of tuberculosis vaccination in Brazil [69]. Thus, if outpatient clinics are to be used as a source of controls, they should be clinics where immunizations are not routinely provided.

Controls with same clinical syndrome who are ‘test-negative’ for the pathogen of interest

Another potential source of controls is children who become ill with the same clinical syndrome as those with the outcome of interest, but whose illness is shown to have an etiologic pathogen not targeted by the vaccine under evaluation [70-72]. Examples of this approach include: rotavirus-negative gastroenteritis as controls for cases of rotavirus [36,41,43], influenza-negative respiratory infection for cases of influenza [71], pneumococcal or non-purulent/Hib-negative meningitis as controls for Hib meningitis cases [18,20], and non-vaccine serotype invasive pneumococcal disease for cases of vaccine-type invasive disease (also known as the ‘indirect cohort’ or ‘Broome’ method) [73-77]. This approach requires accurate diagnostic testing and sample collection at an appropriate time to diagnose the pathogen of interest in order to avoid misclassification. Imperfect test sensitivity and specificity leads to an underestimation of effectiveness using test-negative controls [78]. Some tests, such as culture of blood or cerebrospinal fluid, are too insensitive to reliably identify test-negative controls; however when such tests detect an etiology that is not preventable by vaccines included in the national schedule (e.g. pneumococcal meningitis for evaluation of Hib vaccine [before introduction of pneumococcal vaccine], or non-vaccine-type pneumococcal bacteremia for evaluation of pneumococcal conjugate vaccine), such individuals can serve as controls. The validity of using test-negative controls has been demonstrated by re-analyses of data from randomized clinical trials of influenza [70] and rotavirus [79] vaccines that yielded effectiveness estimates very similar to the efficacy measured by the original trials.

One major advantage of the test-negative approach is a high degree of comparability between cases and controls, since controls would have been enrolled as cases if they had the vaccine-preventable outcome of interest. It also offers logistical and cost advantages, since cases and controls can be recruited from within a single surveillance system. Also, since test results are often not available at the time of recruitment, bias in ascertainment of vaccination through knowledge of case-control status is less likely. A limitation to this method is that it assumes the vaccine being evaluated has no effect on the incidence of test-negative cases who will serve as controls. For pneumococcal conjugate vaccines, this assumption may not be valid, since their widespread use has been associated with increases in non-vaccine type pneumococcal carriage and disease incidence [80]. However, modeling work conducted in conjunction with indirect cohort analyses indicates that while increases in non-vaccine type disease (and carriage) among vaccinated individuals compared with the non-vaccinated can lead to overestimates of VE, the magnitude of the overestimation is relatively small, particularly if conducted before vaccine coverage is very high [74,76]. For influenza vaccine, models have similarly shown that even if influenza infection is presumed to provide transient non-specific immunity to all respiratory infections (and thus individuals vaccinated against influenza, who would not benefit from this immunity, would be over-represented among non-influenza respiratory infection cases) the impact on effectiveness estimates derived from case-control studies using the test-negative design is minor [72,81].

Multiple control groups

In some case-control vaccine effectiveness studies two or more types of control group are enrolled [17,19,40,42]. However, when the estimates of effectiveness differ by the control group used, the disparate results are difficult to interpret and communicate [63]. Multiple control groups may be useful for evaluation of study methods and identifying bias in different groups. In general, however, it is preferable to understand the study context well, to consider carefully the best control group before conducting the study, and then to use one source of controls [62].

Matching

Matching of cases and controls is often used in vaccine effectiveness studies to increase the statistical efficiency of the analysis and to attempt to control for unmeasured confounders [64,65,82]. However, overmatching, which occurs when the matching variable is strongly associated with vaccination but not (or only weakly) with the illness, results in a loss of statistical power [83]. Matching also greatly increases the operational complexity of enrolling controls. Matching in case-control vaccine effectiveness studies is most commonly done at an individual level, where each enrolled control is matched to a specific case based on certain criteria (e.g. date of birth or geographic region). An alternative approach is ‘frequency’ or ‘stratum’ matching, in which the group of controls is enrolled based on the frequency of certain characteristics among all cases (e.g. if 20% of cases are from a certain neighborhood, then controls are enrolled so that 20% are from that same neighborhood) [10]. Matching, if used, should be reserved for a limited number of important variables believed to be linked to both vaccination and disease (i.e. confounding), since unnecessary matching can lead to reduced efficiency in the analysis and substantially increases the complexity of study implementation [65,82,84].

Control to case ratio

The preferred ratio of controls to cases depends upon the relative ease (and cost) of enrolling cases and controls. The statistically most efficient approach is equal numbers of cases and controls, if they are equally easy to enroll. If the number of cases is limited, increasing the number of controls per case will increase statistical power, but generally little additional power is gained by enrolling more than four controls per case [62]. However, for studies using individual matching in contexts where vaccine coverage is very high or very low, more than four controls per case should be considered, since case-control sets in which all cases and controls have the same vaccination status will not contribute to the estimates of effectiveness. In contexts where controls are easy to enroll, for example from a population-based registry [29], then a higher control to case ratio may be used.

Timing of control enrollment

For individually matched case-control vaccine effectiveness studies, controls should be enrolled concurrently (i.e. for each incident case enrolled, one or more new matched controls are enrolled from the population at-risk). Rapid enrollment of matched controls can reduce recall bias, minimize difficulty obtaining compatible vaccination histories for cases and controls, and help ensure comparability between cases and controls with respect to unmeasured temporal factors that may affect the risk of developing the outcome of interest (e.g. outbreaks of viral respiratory infections increasing the risk for pneumococcal pneumonia). However, rapid enrollment of matched controls is not always feasible and risks regarding vaccine history can be mitigated if there is written documentation, with dates of administration, of vaccination status so that only doses received before the corresponding case became ill are considered.

Conclusions

Evidence of the protection afforded by new vaccines in the context of real-world immunization programs is important for accelerating and sustaining their uptake globally [85,86]. Case-control vaccine effectiveness studies, if carefully conducted, can provide such evidence, complementing data from randomized controlled trials as well as findings from other observational approaches, such as analyses of trends in disease incidence over time or cohort studies. Relative to other observational methods for vaccine evaluation, case-control studies have some advantages. They do not require a stable baseline of disease surveillance data prior to vaccine introduction and are often considerably less expensive to perform than cohort studies. Case-control vaccine effectiveness studies do not measure the actual impact of vaccine introduction on disease at a population level. However, when combined with data on prevaccine burden of disease and vaccine coverage, they can be used to provide insight into the public health impact of vaccines.

The belief that case-control studies are quick and easy to carry out is misplaced. Case-control vaccine effectiveness studies are complex and require rigorous planning and implementation. They are susceptible to various types of bias and, if not conducted rigorously and with careful planning, can produce invalid and potentially misleading results. It is imperative that investigators understand the study context well to minimize bias and correctly interpret results. Case-control vaccine effectiveness studies are most likely to provide reliable information when assessing outcomes specific to the vaccine being evaluated (e.g. Hib meningitis rather than all clinical meningitis). Studies using nonspecific outcomes are particularly challenging and prone to misleading results; such studies should not be undertaken unless the investigators ensure a high level of rigor and complementary data assessing other more specific outcomes are available from the same or comparable population. Selection of an appropriate control group and close attention to potential sources of bias during control enrollment are crucial. While case-control studies can provide useful information to guide vaccine policy decisions and vaccine development, they must be thoughtfully planned and rigorously conducted.