Response to Shami et al. 'Evaluating the cost-effectiveness of a sequential pneumococcal vaccination compared to single dose vaccination strategy for adults in Hong Kong' (Hum Vacc Immunother 2020).

The recently published article by Shami et al. describes a cost-effectiveness analysis of sequential pneumococcal vaccination with 13-valent pneumococcal conjugate vaccine (PCV13) followed by 23-valent pneumococcal polysaccharide vaccine (PPSV23), compared with a single PPSV23 dose, in adults in Hong Kong. Sequential vaccination was cost-saving versus PPSV23 alone. The model assumed vaccine effectiveness (VE) of 0% for PPSV23 against all-cause non-bacteremic pneumonia; this was based on studies with flawed methodologies and studies that did not evaluate non-bacteremic pneumococcal disease. In recent studies and meta-analyses, PPSV23 VE pneumococcal pneumonia, including against non-bacteremic pneumococcal pneumonia, ranged from 27% to 64%. In other cost-effectiveness analyses, assumptions for PPSV23 VE against non-bacteremic pneumococcal pneumonia, had dramatic effects on cost-effectiveness estimates. Future analyses must carefully consider PPSV23 VE assumptions to ensure accuracy.

We read with interest the article by Shami et al.[1] published online in Human Vaccines and Immunotherapy on January 24, 2020, which estimated the cost-effectiveness of sequential pneumococcal vaccination with 13-valent pneumococcal conjugate vaccine (PCV13) followed by 23-valent pneumococcal polysaccharide vaccine (PPSV23), compared with a PPSV23 single-dose vaccination strategy, in adults in Hong Kong. The analysis found that incremental cost-effectiveness ratios were dominant for sequential vaccination in both adults aged ≥65 years and adults aged 20–64 years with immunocompromising and chronic conditions, indicating that sequential vaccination may be cost-saving compared with PPSV23 alone.

We disagree with many of the assumptions used in this cost-effectiveness analysis. The model assumed a vaccine effectiveness (VE) of 0% for PPSV23 against “all-cause non-bacteremic pneumonia”. Unfortunately, it is not clear whether the authors intended to refer to all-cause non-bacteremic pneumonia or non-bacteremic pneumococcal pneumonia (NBPP) regardless of Streptococcus pneumoniae serotype. The authors then state that the assumption of effectiveness was based on data from previous randomized clinical trials and meta-analyses. However, of the four publications cited, only one describes a randomized clinical trial. Although this article,[2] published by Ortqvist et al. in 1998, did not detect any benefit of PPSV23 compared with placebo against all-cause or pneumococcal pneumonia in adults aged 50–85 years who had previously been hospitalized for community-acquired pneumonia, the study has been noted as having a high risk of bias.[3] Diagnosis of pneumococcal pneumonia was made by the detection of antibodies against immune complexes of pneumolysin, a cholesterol-dependent cytotoxin produced by almost all strains of S. pneumoniae. This methodology has not been used for the diagnosis of pneumococcal pneumonia in other published studies, nor is it used in clinical practice.[3] Subsequent validation studies concluded that the assay lacks specificity, making it insufficient for the performance of vaccine efficacy studies, as it biases the observed VE toward the null.[3-5] Moreover, the assay is unable to distinguish between infection and colonization with S. pneumoniae.[5] As such, a recent independent meta-analysis excluded it from their analyses of VE against pneumococcal pneumonia,[3] in accordance with well-recognized standards for assessing study bias when performing meta-analyses.[6]

The second article cited by Shami et al. in support of their assumption of 0% VE for PPSV23 against “all-cause non-bacteremic pneumonia” describes a prospective case–control study of adults hospitalized for invasive pneumococcal disease (IPD).[7] In this study published by Shapiro et al. in 1991, patients were eligible only if S. pneumoniae had been isolated from blood, cerebrospinal fluid, or pleural fluid. As such, this study is inappropriate for supporting assumptions relating to NBPP, a noninvasive condition.

The remaining two publications used to support the assumption of a VE of 0% for PPSV23 were, contrary to the authors’ statement, not meta-analyses but other cost-effectiveness studies. The first, published by Evers et al. in 2007, used the Shapiro et al. article as a basis for their assumptions on PPSV23 VE; however, this study only assessed cost-effectiveness against IPD; thus, non-bacteremic pneumonia was not included.[8] The second, published by Smith et al. in 2008, used a modified Delphi panel approach that relies on expert opinion to estimate VE against IPD.[9] Although the authors state that VE against noninvasive pneumococcal disease for PPSV23 was assumed to be 0%, this was not based on evidence, rather it was a limitation of the model, presumably included for the purpose of simplification as the analysis focused on IPD only.

PPSV23 induces protective immune responses against the 23 pneumococcal serotypes included in the vaccine and the indication is against vaccine-type disease;[10] the same is true for PCV13.[11] Hence, the appropriate VE estimate to use in cost-effectiveness modeling is VE against vaccine type NBPP, rather than all pneumococcal pneumonia or all-cause pneumonia. PPSV23 was first licensed in the United States in 1983, and many early studies evaluating VE against NBPP did not have the diagnostic tools to distinguish by serotype. However, a recent multicenter prospective study by Suzuki et al. assessing effectiveness of PPSV23 in more than 2000 adults aged ≥65 years in Japan found a VE for PPSV23 against vaccine-type pneumococcal pneumonia of 33.5% (95% confidence interval [CI] 5.6–53.1) and VE against any pneumococcal pneumonia of 27.4% (95% CI 3.2–45.6).[12] While this study included bacteremic pneumococcal pneumonia, the proportion of patients diagnosed with bacteremic pneumococcal pneumonia was less than 1% of those in whom blood cultures were performed (64% of the overall population). Many other studies were only able to evaluate all pneumococcal pneumonia. An independent meta-analysis published by Falkenhorst et al. in 2017 reported a pooled VE for PPSV23 against pneumococcal pneumonia (any serotype) of 64% (95% CI 35–80%) based on data from two clinical trials and 53% (95% CI 33–68%) based on a case–control study.[3] This meta-analysis was commissioned and performed by the Robert Koch Institute in Germany and was the basis of the 2016 Standing Committee on Vaccination (STIKO) guidelines, which recommend PPSV23 for all adults aged ≥60 years and all patients aged ≥16 years with at least one chronic disease not associated with immune suppression in Germany.[13,14]

Shami et al. calculated the VE of PCV13 against vaccine-type NBPP by applying the VE against vaccine-type NBPP from the CAPITA study to sero-epidemiologic data in Hong Kong. We note that in the CAPITA study, PCV13 was only effective against vaccine-type NBPP; VE against NBPP caused by any S. pneumoniae serotype and VE against community-acquired pneumonia were not statistically significantly different from placebo.[15] Despite this, Shami et al. based their VE assumptions for PCV13 on the vaccine-type specific outcome, while using a non-vaccine-type specific outcome for PPSV23. For a fair and balanced analysis, cost-effectiveness models should employ the same outcomes for both vaccines.

It is a standard procedure to include a sensitivity component for variables with uncertainty in a cost-effectiveness analysis.[16] In line with this, some recent cost-effectiveness analyses have performed sensitivity analyses to acknowledge the variation in VE estimates for PPSV23 against NBPP outcomes.[17,18] A study published by Stoecker et al. in 2020 assessed the cost-effectiveness of continuing to recommend PCV13 in series with PPSV23 in adults aged ≥65 years, compared with PPSV23 alone.[17] Stoecker et al. included a sensitivity analysis in which VE was assumed to be 45% against vaccine-type NBPP. The cost per quality-adjusted life year (QALY) of continuing to recommend PCV13 increased from USD 0.56 million in the base case analysis to USD 2.3 million in the sensitivity analysis, demonstrating the dramatic effect that changes in VE assumptions for PPSV23 can have on cost-effectiveness estimates.

Cost-effectiveness analyses of pneumococcal vaccines are used to inform policy decisions that have consequences for funding, access recommendations, and ultimately the health of adult populations at risk of pneumococcal disease. As such, it is crucial that these analyses are as accurate, comprehensive, and informative as possible. For future analyses, researchers must include the most robust and relevant data on VE of PPSV23 against NBPP to ensure that pneumococcal vaccine policies are driven by evidence-based decisions.