Microneutralization assay titer correlates analysis in two phase 3 trials of the CYD-TDV tetravalent dengue vaccine in Asia and Latin America.

We previously showed that Month 13 50% plaque reduction neutralization test (PRNT50) neutralizing antibody (nAb) titers against dengue virus (DENV) correlated with vaccine efficacy (VE) of CYD-TDV against symptomatic, virologically-confirmed dengue (VCD) in the CYD14 and CYD15 Phase 3 trials. While PRNT is the gold standard nAb assay, it is time-consuming and costly. We developed a next-generation high-throughput microneutralization (MN) assay and assessed its suitability for immune-correlates analyses and immuno-bridging applications. We analyzed MN and PRNT50 titers measured at baseline and Month 13 in a randomly sampled immunogenicity subset, and at Month 13 in nearly all VCD cases through Month 25. For each serotype, MN and PRNT50 titers showed high correlations, at both baseline and Month 13, with MN yielding a higher frequency of baseline-seronegatives. For both assays, Month 13 titer correlated inversely with VCD risk. Like PRNT50, high Month 13 MN titers were associated with high VE, and estimated VE increased with average Month 13 MN titer. We also studied each assay as a valid surrogate endpoint based on the Prentice criteria, which supported each assay as a valid surrogate for DENV-1 but only partially valid for DENV-2, -3, and -4. In addition, we applied Super-Learner to assess how well demographic, Month 13 MN, and/or Month 13 PRNT50 titers could predict Month 13-25 VCD outcome status; prediction was best when using demographic, MN, and PRNT50 information. We conclude that Month 13 MN titer performs comparably to Month 13 PRNT50 titer as a correlate of risk, correlate of vaccine efficacy, and surrogate endpoint. The MN assay could potentially be used to assess nAb titers in immunogenicity studies, immune-correlates studies, and immuno-bridging applications. Additional research would be needed for assessing the utility of MN titer in correlates analyses of other DENV endpoints and over longer follow-up periods.

Introduction

Approximately 40% of the world is at risk of infection with the four serotypes of dengue virus (DENV-1, -2, -3, and -4) [1]. Symptomatic DENV infection can range in severity up to dengue hemorrhagic fever and dengue shock syndrome [2]. The global health and economic burdens of DENV are significant, with about 400 million (including 500,000 hospitalized) infections annually worldwide [1, 3] and an estimated annual $8.9 billion cost of dengue disease [4].

The CYD-TDV dengue vaccine (Dengvaxia®, Sanofi Pasteur) contains four recombinant, live attenuated chimeric viruses, each harboring the dengue premembrane/envelope genes of one serotype [5]. In two Phase 3 trials, CYD14 in 2–14-year-olds in Asia [6] and CYD15 in 9–16-year-olds in Latin America [7], CYD-TDV (or placebo) was administered at Months 0, 6, and 12. After the first injection, participants were followed-up with active surveillance for symptomatic, virologically confirmed dengue of any severity (VCD) until Month 25. Estimated vaccine efficacy (VE) of CYD-TDV against VCD caused by any serotype (DENV-Any) between Months 13 and 25 was 56.5% in CYD14 and 60.8% in CYD15 [6, 7], supporting licensing of CYD-TDV for individuals ≥9 years old in multiple dengue-endemic countries [8]. Subsequent analyses of VE by baseline dengue serostatus showed high estimated VE against hospitalized VCD and against severe VCD over 60 months in baseline-seropositive individuals; however, estimated VE against these two endpoints was negative in baseline-seronegative individuals (i.e., vaccinated baseline-seronegative individuals were at higher risk of these two endopints compared to unvaccinated baseline-seronegative individuals) [9]. The World Health Organization (WHO)’s Strategic Advisory Group of Experts on Immunization has concluded: “a ‘pre-vaccination screening strategy’ would be the preferred option, in which only dengue-seropositive persons are vaccinated” [10].

We recently conducted a case-cohort correlates analysis of 50% plaque reduction neutralization test (PRNT50) neutralizing antibody (nAb) titers in CYD14 and CYD15 and showed that 1) high Month 13 PRNT50 titers were associated with a lower rate of VCD between Months 13 and 25; and 2) estimated VE against VCD between Months 13 and 25 increased with Month 13 PRNT50 titer [11]. However, estimated VE was positive for 9–16-year-old vaccine recipients with no or low Month 13 seroresponse and a few vaccine recipients with high seroresponse experienced breakthrough VCD, making PRNT50 titers a “relative” correlate of protection [12].

The WHO has stated “Only the PRNT measures the biological parameter of in vitro virus neutralization…Newer tests measuring virus neutralization are being developed, but PRNT remains the laboratory standard against which these tests will need to be validated” [13]. However, the PRNT assay is low-throughput and difficult to automate. Alternative assays have been proposed [14-16], but whether and how nAb titers obtained by PRNT-alternative assays correlate with protection against VCD remain unknown. We developed and validated an enzyme-linked immunosorbent assay-based microneutralization (MN) assay. Compared to the PRNT assay, the MN assay requires less serum, is higher throughput, and uses an objective spectrophotometric readout. Here we: 1) assessed the correlation/concordance of MN and PRNT50 assay readouts, at baseline and at Month 13; 2) assessed Month 13 MN nAb titers as correlates of risk (CoRs) of VCD in CYD14 and CYD15; 3) assessed Month 13 MN nAb titers as correlates of VE (CoVEs) against VCD in CYD14 and CYD15; and 4) built models using baseline demographics, Month 13 PRNT50 titers, and/or Month 13 MN titers to classify participants by VCD outcome status. Our approach for (2) and (3) mirrored that used for our previous correlates analysis of PRNT50 titers in CYD14 and CYD15 [11].

Materials and methods

CYD14 and CYD15

In harmonized designs, healthy children and adolescents aged 2–14 (CYD14; ClinicalTrials.gov ID NCT01373281 [6]) or 9–16 (CYD15; ClinicalTrials.gov ID NCT01374516 [7]) were randomized (2:1) to vaccine or placebo, with randomization stratified by age group and site. Vaccinations were administered at Months 0, 6, and 12. Active surveillance for symptomatic VCD occurred from the day of the first injection to Month 25. [6, 7]. As in [11], correlates analyses were based on the primary study endpoint of symptomatic, virologically confirmed dengue of any serotype (DENV-Any) and on the serotype-specific DENV-1, -2, -3, and -4 VCD endpoints.

Ethics statement

The trial protocols were approved by all relevant ethics review boards, and parents or guardians provided written informed consent and older children provided written informed assent before participation, in accordance with local regulations. All patient data were anonymized.

The ethics review boards for CYD14 were the following: The Committee of Medical Research Ethics, Faculty of Medicine, University of Indonesia, Jakarta, Indonesia; The Research and Development Unit Medical Faculty University of Udayana, Sanglah General Hospital, Denpasar, Indonesia; Health Research Ethics Committee, Faculty of Medicine University of Padjadjadrain, Dr Hasan Sadikin Hospital, Bandung, Indonesia; Medical Research and Ethics Committee, Ministry of Health, Malaysia, Kuala Lumpur, Malaysia; Research Institute for Tropical Medicine IRB, Alabang, Muntinlupa City, Philippines; Vicente Sotto Memorial Medical Center EC, Cebu City, Philippines; Chong Hua Hospital Institutional Review Board, Cebu City, Philippines; Walter Reed Army Institute of Research International Review Board (WRAIR IRB), MD, USA; The Ethical Review Committee for Research in Human Subjects, Ministry of Public Health, Thailand; Ethics Committee of the Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand; Pasteur Institute EC, Ho Chi Minh City, Vietnam.

The ethics review boards for CYD15 were the following: Comitê de Ética em Pesquisa do Centro Ciências da Saúde (CCS) da Universidade Federal do Espírito Santo (UFES) (CEP/CCS/UFES); Comitê de Ética em Pesquisa em Seres Humanos do Hospital Universitário Onofre Lopes / RN; Comitê de Ética em Pesquisa em Seres Humanos do Hospital das Clínicas da Universidade Federal de Goiás; Comitê de Ética em Pesquisa em Seres Humanos da Universidade Federal de Mato Grosso do Sul—UFMS; Comitê de Ética em Pesquisa em Seres Humanos da Universidade Federal do Ceará; Comissão Nacional de Ética Em Pesquisa—CONEP; Comité de Etica en la Investigación—CAIMED; Comité Corporativo de Ética en Investigación Fundación Santafe de Bogotá; Comité de Ética en Investigación Biomédica—CDI; Comité de Ética en Investigación Biomédica (CEIB) de la Unidad de Investigación Científica de la UNAH; Instituto Nacional de Pediatría Comité de Ética en Investigación; Instituto Nacional de Pediatría; Comité Ética y de Investigación—UV Universidad Veracruzana; Saluz Comité de Investigación y Bioética; Copernicus Group IRB—CGIRB.

WHO Universal Trial Numbers: U1111-1116-4957; U1111-1116-4986.

Case-cohort sampling design and cohort definitions

Approximately 10% (CYD14) or 20% (CYD15) of all participants enrolled in the first 2 to 4 months of each trial were randomly assigned to the immunogenicity subset (IS) [described in [6, 7]]. Serum samples for assessing nAb responses were collected from participants in the IS on Months 0, 7, 13, and 25. The PRNT50 assay was run on the stored Month 13 serum samples, after which samples were refrozen and thawed approximately 5 years later, at which time the MN assay was run. The sampling design for measurement of Month 13 MN titers is given in S1 Text. For all analyses that used MN (PRNT50) data, participants in the IS who completed the active phase (Day 0 to Month 25) without experiencing the primary DENV-Any endpoint and for whom Month 13 MN (PRNT50) data were available are defined as controls. As in [11], cases are defined as participants who experienced the DENV-Any endpoint between Month 13 and Month 25. Analyses were based on cases and controls with Month 13 MN (PRNT50) data. As Month 0 serum samples were collected only for the IS, Month 0 MN titers could only be measured for 20.4% of all cases in CYD14 and 8.7% of all cases in CYD15.

PRNT50 assay

The PRNT50 assay was performed using Vero cells (CCL-81 from the American Type Culture Collection, Manassas, VA; master and working banks of Vero cells were prepared in-house) as in [17]. PRNT50 titer represents the highest dilution of serum at which ≥ 50% of dengue challenge virus in plaque counts was neutralized compared to the challenge virus-alone control wells, as determined by linear regression.

MN assay

General comparisons of the PRNT and MN assays are provided in [18, 19]. In contrast to the PRNT50 assay, the MN assay was performed in 96-well (vs 24-well) plates, the virus-serum inoculum was not removed after virus adsorption, liquid (vs semisolid) medium was applied after adsorption, and reduction in virus infectivity due to neutralization by antibodies present in serum samples was detected by successive addition and incubations of dengue serotype-specific monoclonal antibodies, anti-mouse Ig HRP conjugate, and a chromogenic substrate. The same serotype-specific anti-dengue monoclonal antibodies and virus strains were used in both assays.

Briefly, 2-fold serial dilutions of serum samples (starting at 1:5 dilution) were incubated with an equivalent volume of a constant challenge dose of virus (200 TCID50 per well for each serotype) and incubated for 90 minutes at 37°C. A separate virus titration plate was prepared to determine the 50% tissue-culture infective dose (TCID50). After neutralization, the serum-virus mixture was added to pre-seeded Vero cell monolayers in 96-well plates and an additional 100 μl of cell culture medium was added without removal of the virus inoculum after adsorption. The plates were incubated at 37°C for either 4–5 days (depending on the virus serotype). The target virus challenge dose and days of incubation post-infection were determined for each serotype in order to provide an optimal signal-to-noise ratio during the ELISA steps. After the incubation period was complete, the cell culture medium was removed from the plates. The cells were then fixed with 80% acetone and incubated at room temperature for 10–15 min, followed by blocking with 5% non-fat dry milk in PBS-Tween-20 wash buffer. Dengue serotype-specific monoclonal antibodies were added, followed by anti-mouse IgG HRP congugate and TMB substrate. The reaction was stopped with 2N sulfuric acid and the optical density (OD) of each well at 450 nm (650 nm as the reference wavelength) was measured using a SpectraMax 384 microplate reader with SoftMax Pro software version 6.5.1.

The 50% neutralization titer of the test serum sample against each serotype was defined as the reciprocal of the test serum dilution for which the virus infectivity was reduced by 50% relative to the challenge virus dose (without any antibodies) introduced into the assay and was calculated using the formula: [(Average OD of Virus Control—Average OD of Cell Control)/2 + Average OD of Cell Control]. The MN titer for each test sample was interpolated by calculating the slope and intercept using the last dilution with an OD below the 50% neutralization point and the first dilution with an OD above the 50% neutralization point to determine the MN titer using the formula: [MN Titer = (50% neutralization point—intercept)/slope]. Neutralization titers are presented as continuous values. For both assays, the lower limit of quantitation was 10; values below this were set to 5. The average titer is the average of each participant’s four serotype-specific log10 titers.

nAb assay correlation and concordance

Rank correlations between MN and PRNT50 titers were adjusted for age and country; correlations were calculated as in [20] using the PResiduals R package [21]. For analyzing concordance of the two assays with respect to baseline serostatus determination, Cohen’s kappa was calculated.

Immune correlates analyses

Month 13 MN titers were assessed as CoRs and CoVEs as in [11]. In brief, the CoR analyses were performed in each of the vaccine and placebo groups separately, relating VCD risk to a given Month 13 MN titer variable with a logistic regression model that accounted for the case-cohort sampling [22] and adjusted for age, sex, and country. Results are reported as odds ratios of DENV-Any, DENV-1, DENV-2, DENV-3, and DENV-4 VCD per log10 increase in Month 13 nAb titer. P values for testing DENV-1, DENV-2, DENV-3, and DENV-4 nAb titer as a CoR were adjusted across the 4 serotypes using family-wise error rate (Holm-Bonferroni [23]) and false-discovery rate (Q values [24]) adjustment, separately for each treatment group and each trial. All P values and Q values are 2-sided.

The CoVE analyses were performed using the VE curve-effect modification framework [25-27]. This framework assesses how VE changes over subgroups of vaccine recipients, where subgroups are defined by Month 13 nAb titers. The analyses used the Juraska et al. method [28], employed with hinge logistic regression models [29] when there was sufficient data, if not, linear logistic regression models were used. Advantages of the hinge models are summarized in reference [29]. VE curves were estimated with pointwise and simultaneous bootstrap-based Wald 95% confidence intervals (CIs).

Super-Learner classification of DENV-Any outcome status

The Super-Learner algorithm, implemented with the SuperLearner R package [30], was used to construct best models of the conditional probability of DENV-Any occurrence by the Month 25 visit based on demographic features (age, sex, country-specific individual serotype rates) and Month 13 PRNT50 and MN titer variables (DENV-1, -2, -3, and -4 titer readouts and the average, minimum, and maximum titers for each participant). Baseline demographics were included in all models. Only participants with complete Month 13 PRNT50 and MN nAb titer data were included in the analysis (n = 2273; 212 cases in the vaccine group, 284 cases in the placebo group). Inverse probability of censored weighting [31] was employed to adjust for the case-cohort sampling and the restriction to 9–16-year-olds. Models of the conditional probability of DENV-Any occurrence by the Month 25 visit were built separately for the vaccine and placebo groups using four input variable sets, aiming to maximize the cross-validated area under the receiver operating characteristic curve (CV-AUC) [32].

Results

Correlation/concordance of MN and PRNT50 titers

High correlation between MN and PRNT50 titers.

Fig 1 shows scatterplots of PRNT50 and MN titers from CYD14 (2–14-year-old) or CYD15 (9–16-year-old) participants at baseline (Month 0) and at Month 13 (1 month post-final dose). For serotype-specific nAb titer pairs, Spearman correlation coefficients were high (0.83–0.95), across all serotypes, trials, and time-points. Within each trial, serotype-specific correlations tended to be lowest for DENV-4, at both time-points. For each trial and each time-point, correlations were highest between average MN and average PRNT50 titer.

Fig 1

Correlation of PRNT and MN nAb titers.

Pairwise plots of PRNT and MN nAb titers in each trial are shown for each serotype and for the average (geometric mean) across all four serotypes. Top row, baseline titers; bottom row, Month 13 titers. Spearman correlation coefficients are shown in red in the upper left of each panel. Plots display values from all participants in CYD14 (2–14-year-olds) and CYD15 (9–16-year-olds) for whom both MN and PRNT50 nAb titers were available. The blue line in each plot is a loess fit to the points.

For 9–16-year-olds in CYD14 and CYD15, covariate-adjusted Spearman rank correlations between PRNT50 and MN titers were consistently high at both baseline and Month 13, and across the vaccine and placebo groups at Month 13 (S1 Table). Correlations were highest for the average titer at both time-points (baseline 0.96, 95% CI 0.95–0.97; Month 13 (placebo) 0.95, 95% CI 0.93–0.96; Month 13 (vaccine) 0.94, 95% CI 0.93–0.95), with high correlations for the DENV-1, -2, and -3 readouts (0.88–0.94 across both time-points). Correlations were slightly lower for DENV-4 (baseline 0.84, 95% CI 0.81–0.88; Month 13 (placebo) 0.87, 95% CI 0.84–0.90; Month 13 correlation (vaccine) 0.81, 95% CI 0.78–0.83).

Substantial agreement in baseline serostatus classification between the two assays, but a higher proportion of individuals test baseline dengue-seronegative by the MN assay.

It is of interest to present vaccine safety and vaccine efficacy results stratified by baseline dengue serostatus [10, 33]. We examined the concordance of the two assays with respect to baseline serostatus classification and found good agreement (Cohen’s κ = 0.81 across both trials, S2 Table, part A). Similarly agreement was found in each study (Cohen’s κ = 0.80, CYD14; Cohen’s κ = 0.82, CYD15; S2 Table, parts B,C) and across age groups (all Cohen’s κ>0.8; S2 Table, parts D-F). The disagreement between the two assays was consistent in that the proportion of MN-/PRNT50+ participants always exceeded the proportion of MN+/PRNT50- participants (S2 Table). Potential explanations are increased specificity of the MN assay compared to the PRNT50 assay, decreased sensitivity of the MN assay compared to the PRNT50 assay, or both.

For 9–16-year-olds across both trials, the percentage of participants testing baseline-seronegative by the MN assay was higher for average titer and for each of the 4 serotype-specific titers than that testing baseline-seronegative by the PRNT50 assay (all P<0.01, McNemar’s test; S1 Fig, panel A). This difference was greatest for DENV-2 (43% vs 27%). This pattern continued for the placebo group when Month 13 titers were assayed, i.e. higher percentages of participants tested Month 13-seronegative by the MN assay compared to the PRNT50 assay, across all five titer measurements (all P<0.01; S1 Fig, panel B). Similar results were obtained in the vaccine group for Month 13 DENV-1 and DENV-2 titers in that significantly greater percentages of participants tested Month 13-seronegative for DENV-1 and for DENV-2 by the MN assay compared to the PRNT50 assay (P<0.01 for both; S1 Fig, panel C). However, no significant difference in Month 13-seronegativity rates between the two assays was seen in the vaccine group for average titer, DENV-3, or DENV-4 (P>0.05 for all), with seronegativity rates ≤ 2% across the two assays (S1 Fig, panel C).

Correlates analysis using MN nAb measurements

Case-cohort sampling scheme

Using the approach described in [11], we analyzed Month 13 MN titer as a CoR of VCD and as a CoVE against VCD in CYD14 and CYD15. The case-cohort sampling design is shown in Fig 2; further details are given in S1 Text. The cohort for inference consisted of all participants who had not experienced VCD due to any DENV serotype (DENV-Any) by Month 13, who were randomly sampled into the IS, and who had an available MN titer measurement.

Fig 2

Case-cohort sampling scheme.

Controls are defined as participants who were randomly sampled into the immunogenicity subset (IS), had an available titer measurement, and completed the active phase (25 months post-first vaccination) without experiencing the DENV-Any endpoint. For CYD14, the IS consisted of a random sample of participants enrolled in the first 2 months of the trial (randomized 2:1 for inclusion), corresponding to ~20% of the total CYD14 participants; for CYD15, the IS consisted of a random sample of participants enrolled in the first 2 to 4 months of CYD15 (randomized 1:1 for inclusion), corresponding to ~10% of the total CYD15 participants. The sampling design for measurement of Month 13 MN titers in CYD14 and CYD15 participants is detailed in S1 Text. As in Moodie et al. [11], cases are defined as participants who experienced the DENV-Any endpoint between Month 13 and Month 25. The table on the bottom shows the numbers of participants with nAb titer data available at Month 0 and at Month 13 in each trial, by case-control status.

Month 13 MN titer and Month 13 PRNT50 titer perform comparably as CoRs of VCD

We previously reported that Month 13 PRNT50 titers were inverse CoRs of VCD in each trial and in each treatment group, as assessed using Cox proportional-hazards and logistic-regression models [11]. Here we used logistic regression models to determine the estimated odds ratios (ORs) of matched-serotype VCD in each trial per log10 increase in Month 13 nAb titer, adjusting for sex and country, for both assays. In both CYD14 and CYD15, average MN titer was a significant CoR for VCD of any serotype (both P<0.001). Serotype-specific MN titers were also significant CoRs for the serotype-matched VCD endpoints across both trials, except for DENV-3 in CYD14 (P = 0.068) and DENV-4 in CYD14 (P = 0.125) (Table 1). The performance of MN vs PRNT50 titer as a CoR differed somewhat between the two trials, with MN titer a consistently stronger CoR than PRNT50 titer across serotypes in CYD15, but with PRNT50 titer outperforming MN titer for DENV-3 and DENV-4 in CYD14. The Month 13 MN and PRNT50 nAb titer distributions of participants in the placebo and vaccine groups of CYD14 and CYD15, stratified by case (matched-serotype/non-matched serotype) control status, are shown in Figs 3 and 4, respectively.

Table 1

Univariate logistic regression odds ratios of DENV-Any, DENV-1, DENV-2, DENV-3, and DENV-4 VCD in the vaccine groups of the CYD14 and CYD15 studies per log10 increase in Month 13 nAb titer.

	CYD14 (n = 1390 Vaccine Recipients)				CYD15 (n = 1458 Vaccine Recipients)
Titer variable, DENV endpoint
Assay	Odds Ratio (95% CI)	P-value	Holm Adj. P-value	Q-value	Odds Ratio (95% CI)	P-value	Holm Adj. P-value	Q-value
Average titera, DENV-Any
PRNT50	0.24 (0.16, 0.37)	<0.001	<0.001	<0.001	0.15 (0.11, 0.21)	<0.001	<0.001	<0.001
MN	0.21 (0.13, 0.33)	<0.001	<0.001	<0.001	0.14 (0.10, 0.20)	<0.001	<0.001	<0.001
DENV-1 titer, DENV-1
PRNT50	0.39 (0.26, 0.58)	<0.001	<0.001	<0.001	0.31 (0.23, 0.42)	<0.001	<0.001	<0.001
MN	0.08 (0.03, 0.21)	<0.001	<0.001	<0.001	0.23 (0.15, 0.33)	<0.001	<0.001	<0.001
DENV-2 titer, DENV-2
PRNT50	0.42 (0.23, 0.76)	0.004	0.006	0.004	0.18 (0.12, 0.27)	<0.001	<0.001	<0.001
MN	0.23 (0.13, 0.43)	<0.001	<0.001	<0.001	0.14 (0.08, 0.26)	<0.001	<0.001	<0.001
DENV-3 titer, DENV-3
PRNT50	0.38 (0.22, 0.63)	<0.001	<0.001	<0.001	0.41 (0.27, 0.64)	<0.001	<0.001	<0.001
MN	0.55 (0.29, 1.04)	0.068	0.136	0.091	0.23 (0.13, 0.41)	<0.001	<0.001	<0.001
DENV-4 titer, DENV-4
PRNT50	0.31 (0.14, 0.67)	0.003	0.006	0.004	0.22 (0.09, 0.51)	<0.001	<0.001	<0.001
MN	0.42 (0.14, 1.27)	0.125	0.136	0.125	0.08 (0.03, 0.19)	<0.001	<0.001	<0.001

Models were adjusted for age, sex, and country.

aGeometric mean of the antibody titers against DENV-1, DENV-2, DENV-3, and DENV-4.

Fig 3

Distributions of log10 Month 13 MN (Panels A, C, E, G) and PRNT50 (Panels B, D, F, H) titers in CYD14 participants (all ages = 2 to 14 years old), stratified by treatment group and case status (serotype-matched vs serotype-mismatched cases).

The gray horizontal shaded band denotes the middle third of nAb responses (log10 MN titer = 1.23 − 2.03; log10 PRNT50 titer 1.76 − 2.42). A “matched-serotype case” is one where the VCD-causing virus was of the same serotype as the virus used in the nAb assay; a “nonmatched-serotype case” is one where the VCD-causing virus was of a different serotype than the virus used in the nAb assay.

Fig 4

Distributions of log10 Month 13 MN (Panels A, C, E, G) and PRNT50 (Panels B, D, F, H) titers in CYD15 participants (all ages = 9 to 16 years old), stratified by treatment group and case status (serotype-matched vs serotype-mismatched cases).

The gray horizontal shaded band denotes the middle third of nAb responses (log10 MN titer = 1.7 − 2.36; log10 PRNT50 titer = 2.13 − 2.8). A “matched-serotype case” is one where the VCD-causing virus was of the same serotype as the virus used in the nAb assay; a “nonmatched-serotype case” is one where the VCD-causing virus was of a different serotype than the virus used in the nAb assay.

Month 13 nAb titer association with VCD is weak or absent for low nAb titers

We previously reported that while high PRNT50 titers were associated with high VE against VCD, for all serotypes, age groups, and across both trials, estimated VE against DENV-Any VCD was about 35% in 9–16-year-old vaccinees with no Month 13 seroresponse [11]. This finding suggested that other immune responses may be important for protection when nAb titers are low and that the association between VCD risk and nAb titer may be weaker (or absent) at low titers. We used generalized additive models with smoothing splines to assess how the estimated log odds of DENV-Any varied in 9–16-year-olds according to Month 13 MN average nAb titer, separately in the vaccine and placebo groups pooled across both trials. The results showed that DENV-Any risk was not logit linear with MN titer (P<0.001 in the vaccine and placebo groups, unpublished data), suggesting that titers below a certain threshold were not (or only weakly) associated with VCD risk. As hinge models fit the data more adequately than logit linear models for most analyses, hinge models were considered for estimating VE when there was sufficient data, if not, logit linear models were employed.

Similar DENV-Any VE curves by Month 13 MN titer and by Month 13 PRNT50 titer, except possibly at mid-range titers in both trials and at very low titers in CYD15

Using the Juraska et al. method [28], vaccine recipients in CYD14 had an apparently stable level of VE against DENV-Any of approximately 20% for Month 13 average MN titers ranging from below the lower limit of detection to the hinge point at 21, after which VE increased as Month 13 average MN titer increased, reaching near 95% for the highest titers (Fig 5A). This pattern was similar to that previously observed for VE by Month 13 average PRNT50 titer [Fig 5A in [11]]. Fig 5B plots the difference in VE against DENV-Any by MN titer minus VE against DENV-Any by PRNT50 titer in CYD14, where the only difference is observed for titers around 50–80. As the MN Month 13 average titer distribution is shifted leftward compared to the PRNT50 Month 13 average titer distribution, we conjecture that in this mid-range a lower value of MN captures the same information about VE as a higher value of PRNT50. In CYD15, potentially negative to zero VE against DENV-Any was observed at Month 13 average MN titers ranging from below the lower limit of detection to around 10, after which VE leveled off to around 25% until the hinge point at 24. For Month 13 average MN titers greater than 24, VE increased as Month 13 average MN titer increased, reaching near 100% for the highest titers (Fig 5C). The VE-by-Month 13 average PRNT50 curve from our previous analysis [Fig 5B in [11]] looked similar for titers greater than the hinge point, but in contrast to the MN VE curve, VE appeared stable around 25% for Month 13 PRNT50 titers ranging from below the lower limit of detection to the hinge point at 61. Fig 5D plots the difference in VE against DENV-Any by MN titer minus VE against DENV-Any by PRNT50 titer in CYD15. While the same difference observed in CYD14 is seen for titers around 50–80, in CYD15 an additional and larger difference in VE is seen for titers below the limit of detection. The magnitude of this difference decreases as the titers approach the limit of detection, after which the two curves appear similar. It is possible that the MN assay may be better at detecting a lack of VE in baseline seronegative individuals than the PRNT50 assay. Alternatively, in CYD15, the VE curve estimation for vaccine recipients with low to no seroresponse could be unstable due to sparse data.

Fig 5

Estimated vaccine efficacy (VE) against DENV-Any by Month 13 MN titer in the vaccine group in A) CYD14 and C) CYD15.

95% pointwise and 95% simultaneous confidence intervals are also shown. Panels B (CYD14) and D (CYD15) show the estimated VE against DENV-Any by Month 13 MN titer minus estimated VE against DENV-Any at the same value of Month 13 PRNT50 titer. [The VE curves by Month 13 PRNT50 titer by themselves are shown in Moodie et al. [11].] DENV-Any = symptomatic, virologically confirmed dengue of any serotype, occurring between Month 13 and Month 25.

nAbs measured by the MN assay may be better at mediating VE against DENV-Any than nAbs measured by the PRNT50 assay

We assessed VE against DENV-Any and against each of the serotype-specific endpoints for vaccine recipients with no Month 13 seroresponse (defined as titer<10 for all serotypes for DENV-Any; DENV-1 titer<10 for DENV-1 VCD, etc.), separately in each trial. For CYD14, all point and interval estimates of VE (except potentially those for VE against DENV-4 VCD) are consistent with full mediation with nAbs measured by the PRNT50 assay (Table 2, part A). However, the VE point estimates against DENV-3 and DENV-4 VCD for individuals without MN-measured DENV-3 or DENV-4 seroresponse, respectively, were higher than their PRNT50 counterparts—suggesting that, for these two serotypes, nAbs measured by the MN assay may not fully mediate VE against their respective endpoints. For CYD15, the lower bound of the 95% CI for VE>0% was above 0 for DENV-3 (MN) and for DENV-3 and DENV-4 (PRNT50) (Table 2, part B). The DENV-4 result suggests that the MN assay may be identifying the lack of VE better than the PRNT50 assay. For 9–16-year-olds pooled across both trials, nAbs measured by the MN assay may be better at mediating VE against DENV-Any than those measured by the PRNT50 assay (Table 2, part C). However, the evidence for positive VE against DENV-3 and DENV-4 VCD in MN-DENV-3 and MN-DENV-4 Month 13 seronegative vaccinees again suggests that nAbs measured by the MN assay do not fully mediate VE against the DENV-3 and DENV-4 endpoints.

Table 2

Comparison of point and interval estimates of VE as assessed by the MN vs. PRNT50 assay in vaccine recipients with no Month 13 seroresponse.

A. CYD14
	MN		PRNT50
Endpoint	V E^ (no Month 13 seroresponse*) (%)	95% CI	V E^ (no Month 13 seroresponse*) (%)	95% CI
DENV-Any	17	(-38, 49)	2	(-49, 36)
DENV-1	-122	(-743, 42)	5	(-69, 46)
DENV-2	-6	(-164, 57)	-14	(-159, 50)
DENV-3	66	(-7, 89)	-39	(-235, 42)
DENV-4	54	(-82, 88)	35	(-88, 77)
B. CYD15
	MN				PRNT50
Endpoint	V E^ (no Month 13 seroresponse*) (%)	95% CI	V E^ (no Month 13 seroresponse*) (%)	95% CI
DENV-Any	-43	(-311, 50)	23	(-5, 43)
DENV-1	12	(-37, 44)	27	(-12, 52)
DENV-2	-52	(-149, 7)	-84	(-216, -7)
DENV-3	59	(31, 76)	64	(35, 81)
DENV-4	34	(-146, 82)	74	(46, 87)
C. CYD14 and CYD15 9–16-year-olds
	MN		PRNT50
Endpoint	V E^ (no Month 13 seroresponse*) (%)	95% CI	V E^ (no Month 13 seroresponse*) (%)	95% CI
DENV-Any	19	(-23, 47)	35	(7, 54)
DENV-1	15	(-27, 43)	23	(-9, 45)
DENV-2	-31	(-110, 18)	-47	(-147, 13)
DENV-3	62	(30, 79)	56	(28, 73)
DENV-4	62	(0, 86)	76	(59, 85)

* No Month 13 seroresponse = Month 13 titer below the lower limit of quantitation, set to 5.

We also applied the Prentice criteria [34] to evaluate whether (or how closely) each Month 13 serotype-specific nAb response satisfied the Prentice definition of a valid surrogate endpoint for the matched-serotype VCD outcome, in CYD14 and CYD15 together. Two Prentice criteria are readily supported across the nAb titer markers (serotype-specific VE > 0% and the marker correlates with VCD in each treatment group; S3 Table columns 2 and 3). The key third Prentice criterion is that treatment group does not predict VCD after accounting for the marker and adjusting for baseline variables that predict both the marker and VCD. Fig 6 shows the logistic regression estimates of cumulative endpoint rates for serotype-specific VCD and sampling weighted distributions of serotype-specific log10 nAb titers, in CYD14 and CYD15 together, separately by Month 13 serotype-specific PRNT50 titer and by Month 13 serotype-specific MN titer. The modeling results were consistent across both assays for all 4 serotypes, with results supporting (1) DENV-1 titer adheres remarkably well to the Prentice criteria (e.g., overlapped vaccine and placebo curves in panels A and B in Fig 6), (2) DENV-3 titer has a similar inverse association with VCD in each treatment group but departs from the third criterion with titer and treatment jointly predicting VCD; and (3) the DENV-2 and DENV-4 CoRs were significantly modified by treatment group, indicating departure from the third criterion. Regarding point (3), the cumulative endpoint rates of DENV-2 VCD by Month 13 DENV-2 PRNT50 titer (Panel C of Fig 6) suggest that CYD-TDV vaccination could have increased DENV-2 VCD risk at lowest Month 13 DENV-2 PRNT50 titers. Moodie et al. [11] previously addressed this issue, noting that simultaneous 95% confidence bands for DENV-2 VE include 0%, and an inference of vaccine-increased risk is based on a very small number of vaccine-recipient DENV-2 cases (0 DENV-2 cases among the 87 vaccine-recipients with no Month 13 PRNT50 DENV-2 seroresponse in CYD14 and 5 DENV-2 cases among the 264 vaccine-recipients with no Month 13 PRNT50 DENV-2 seroresponse in CYD15, with “no Month 13 PRNT50 DENV-2 seroresponse” defined as Month 13 PRNT50 DENV-2 titer < 10). Panel D of Fig 6 shows less concern for potential vaccine-increased DENV-2 VCD risk for individuals with lowest DENV-2 titers based on the MN assay, given that the vaccine and placebo curves are more similar in the left-tail of the plot. For DENV-4, cumulative endpoint rates decreased with increasing Month 13 serotype-matched titers in both treatment groups, with low cumulative DENV-4 rates at low titers (Panels G and H of Fig 6), and a borderline significant result that the rate was lower in the vaccine group at low titers. Together, these results show that Month 13 PRNT50 titer and Month 13 MN titer are consistent with the Prentice criteria for DENV-1 but not for the other serotypes. The other evaluation statistics in S3 Table, and a comparison of the curves in Fig 6, support imperfect but substantial partial surrogate value for DENV-3 and DENV-4, and less so for DENV-2.

Fig 6

Logistic regression estimates of cumulative endpoint rates for serotype-specific VCD and sampling weighted distributions of serotype-specific log10 nAb titers, in CYD14 and CYD15 together.

(A, C, E, G): Month 13 serotype-specific PRNT50 titer, (B, D, F, H): Month 13 serotype-specific MN titer.

VE against DENV-Any is positive and increases with average Month 13 MN titer, in both baseline-seropositive and baseline-seronegative subgroups

We previously showed that estimated VE against DENV-Any was approximately 25% for vaccine recipients with no seroresponse (defined as PRNT50 titer less than the assay lower limit of quantification, 10, for all four serotypes) at Month 13 and increased similarly with average Month 13 PRNT50 titer in baseline seronegative vs. baseline seropositive subgroups in CYD14 and CYD15 9–16-year-old vaccine recipients (baseline serostatus determined by the PRNT50 assay) (Fig 7A and 7B; reproduced with modification from [11]). Using the same method [35], we found that estimated VE against DENV-Any was approximately 35% for vaccine recipients with no seroresponse (measured by the MN assay) at Month 13 and that it likewise increased similarly with average Month 13 MN titer in baseline-seronegative vs. baseline-seropositive subgroups in CYD14 and CYD15 9–16-year-old vaccine recipients (baseline serostatus determined by the PRNT50 assay) (Fig 7C and 7D). Among CYD14 and CYD15 9–16-year-old vaccine recipients, VE estimates against DENV-Any at the median Month 13 average PRNT50 titer of 392 were 77% for the baseline-seropositive subgroup and 68% for the baseline-seronegative subgroup; at a Month 13 average MN titer of the same value (392), VE estimates were 82% for the baseline-seropositive subgroup and 70% for the baseline-seronegative subgroup (Fig 7). Thus, stratification by baseline serostatus of VE estimates by Month 13 titer in CYD14 and CYD15 9–16-year-old vaccine recipients yields relatively similar results for baseline-seropositive vs. baseline-seronegative subgroups, regardless of which assay is used to measure Month 13 titer.

Fig 7

Estimated vaccine efficacy against DENV-Any by average log10 PRNT50 (Panels A and B) or MN (Panels C and D) titer at Month 13 in baseline seropositive (Panels A and C) and baseline seronegative (Panels B and D) subgroups of CYD14 and CYD15 9–16-year-old vaccine recipients. 95% pointwise and simultaneous confidence intervals are also shown. Plots were generated using the Zhuang et al. method [35]. Baseline seropositive individuals were defined as being seropositive (PRNT50 > 10) to at least one serotype and baseline seronegative individuals were defined as being seronegative (PRNT50 ≤10) to all four serotypes. Panels A and B are reproduced with modification from Fig S15 in Moodie et al. [11] and are shown for comparison.

Super-Learner prediction of individual DENV-Any outcomes

Using Super-Learner, an approach that selects the best weighted combination of prediction algorithms from multiple candidates [36], we next assessed whether/how MN and PRNT50 titers helped predict individual-level VCD risk. Each included algorithm classified 9–16-year-old vaccine and placebo recipients in CYD14 and CYD15 as to whether they experienced DENV-Any VCD between Months 13 and 25. We identified best models for 4 covariate groups: 1) baseline demographic information, 2) demographic information+Month 13 MN titer, 3) demographic information+Month 13 PRNT50 titer, and 4) demographic information+Month 13 MN titer+Month 13 PRNT50 titer. S4 Table provides a complete list of the input variables (e.g. demographic variables, MN titer variables, PRNT50 titer variables) used in each covariate group for the various supervised learning analyses, in addition to further information on the statistical learning algorithms in the Super-Learner library of estimators of the conditional probability of DENV-Any.

Classification accuracy using the best model identified by Super-Learner was overall better for the vaccine group, with CV-AUCs ranging from 0.61–0.84 (vaccine) vs. 0.54–0.74 (placebo) (Table 3). In both treatment groups the addition of Month 13 nAb titer (either assay) improved classification accuracy over demographic characteristics only, with CV-AUC increases ranging from 0.19–0.21. In the placebo group, the addition of Month 13 PRNT50 titer data did not improve classification accuracy over that achieved with demographic+Month 13 MN titer data, nor did the addition of Month 13 MN titer data improve classification accuracy over that achieved with demographic+Month 13 PRNT50 titer data (Table 3). In contrast, in the vaccine group slight improvement in classification accuracy was achieved by including both nAb titers with demographic information vs. including only one nAb titer and demographic information, particularly by additionally including Month 13 MN data when demographic+Month 13 PRNT50 data were first considered. Specifically, in the vaccine group the CV-AUC for demographic+Month 13 PRNT50 data alone was 0.79 (0.76–0.82), whereas it was 0.84 (0.82–0.87) for demographic+MN+PRNT50 data (Table 3). Panels A and B of S2 Fig compare the CV-AUCs for all algorithms, including Super-Learner along with individual statistical algorithms such as standard logistic regression, applied using input variables demographics+MN+PRNT50 data. The results show slight gains in classification accuracy for Super-Learner compared to most of the individual algorithms, and large gains in classification accuracy for Super-Learner compared to the polymars, mean, and nnet algorithms. These results held true for both the placebo and vaccine groups. (Part B of S4 Table provides more information on the learning algorithms in the Super-Learner library of estimators of the conditional probability of DENV-Any).

Table 3

DENV-Any classification accuracy for the best vaccine and placebo models for each input variable data set.

Variable Set Name	Treatment	Best Model	CV-AUC (95% CI)a
1. Demo	Vaccine	SL.gamb	0.61 (0.56–0.65)
	Placebo	SL.gam	0.54 (0.50–0.59)
2. Demo + MN	Vaccine	SuperLearner	0.82 (0.79–0.85)
	Placebo	SuperLearner	0.73 (0.69–0.76)
3. Demo + PRNT50	Vaccine	SuperLearner	0.79 (0.76–0.82)
	Placebo	SL.gam	0.74 (0.70–0.77)
4. Demo + MN + PRNT50	Vaccine	SuperLearner	0.84 (0.82–0.87)
	Placebo	SL.gam	0.73 (0.70–0.77)

aCV-AUC is cross-validated area under the receiver operating characteristic curve, with 95% CI for the CV-AUC estimated by the method of Hubbard, Kherad-Pajouh, and van der Laan [37].

bSL.gam is a generalized additive model with smoothing splines for the neutralization titer variables.

Fig 8 shows the CV-ROC curves for the best-performing vaccine group models fit on each of the four input variable sets (Table 3). Panel B displays a magnified version of the CV-ROC curves with cross-validated false positive rate under 0.014 (the overall rate in placebo recipients) for the best vaccine group models. For very low false positive rates, the true positive rates are higher when demographic+MN+PRNT50 information is included in the classification, intermediate performance is achieved by including nAb titer information from one assay (with potentially slightly better classification achieved by the addition of MN vs PRNT50 titer), and worst performance is achieved when only demographic data are used.

Fig 8

Cross-validated ROC curves for the best vaccine group models fit on each of the four input variable sets.

D1 = dataset 1 (demographics alone); D2 = dataset 2 (demographic + Month 13 MN titer data); D3 = dataset 3 (demographic + Month 13 PRNT50 titer data); D4 = dataset 4 (demographic + Month 13 MN + Month 13 PRNT50 titer data). For comparison, the SuperLearner results for D1 are also plotted. Panel B shows a magnification of the lower left region of Panel A.

S2 Fig, panel C shows the cross-validated estimated probabilities of DENV-Any by case-control status for the best-performing models for each covariate group, for the vaccine group. Cases are assigned higher predicted values of DENV-Any than controls when data from either nAb assay are included; this difference is greatest when MN+PRNT50 information are both included, with potentially better prediction achieved by the addition of MN vs PRNT50 titer. S5 Table, part A shows further information on one of the best-fitting and most easily interpretable models for the vaccine group based on demographic+MN+PRNT50 information. The logistic regression model with 5 variables shows that greater age, Month 13 DENV-1 PRNT50 titer, and Month 13 DENV-2 PRNT50 titer are all inversely associated with risk of DENV-Any, whereas countries with higher rates of DENV-2 and DENV-3 have higher risk. S5 Table, part B shows analogous information for prediction based on demographic+MN information.

Discussion

We conclude that the MN assay has equal or potentially even better utility than the PRNT50 assay for defining CoRs, CoVEs, and individual-level predictors of DENV risk, particularly for individuals aged ≥ 9 years. In both trials, average Month 13 MN titer was significantly inversely correlated with risk of VCD of any serotype and was even a slightly stronger correlate than average Month 13 PRNT50 titer. As in our previous analysis of Month 13 PRNT50 titers [11], high Month 13 MN titers were associated with high VE regardless of the subgroup (serostatus, age, study) and estimated VE increased with average Month 13 MN titer; moreover, like Month 13 PRNT50 titer, no absolute threshold Month 13 MN titer was observed that was associated with 100% VE, classifying MN titers as a “relative correlate” [12]. We also found that for vaccine recipients with the lowest MN titers, VE may be closer to zero compared to vaccine recipients with the lowest PRNT50 titers. This difference was most noticeable for VE against DENV-Any in 9–16-year-olds pooled across the two trials and for VE against DENV-4 in CYD15, suggesting that low MN titers may mark absent/low VE better than low PRNT50 titers and that nAbs measured by the MN assay may mediate more of the VE in this age group. While there was insufficient precision to conclude greater mediation by nAbs measured by the MN assay, the data support that the MN assay is not worse. Finally, using Super-Learner, the best prediction of individual-level risk of VCD was achieved when both Month 13 PRNT50 and MN titer data were included in the models, with potentially better performance achieved by the addition of MN vs PRNT50 titer data. Considering these findings and the operational advantages of the MN assay, MN may be a suitable alternative assay to PRNT50 in analyses of large-scale vaccine trials. A limitation of our analysis was that it was restricted to VCD risk and protection against VCD from Month 13 to Month 25. Future analyses would be needed to examine the utility of MN and PRNT titers in correlates analyses of other DENV endpoints and over longer follow-up periods.

While in CYD15 all titer readouts (serotype-specific and average) were significant CoRs for their corresponding VCD endpoints (matched-serotype VCD and DENV-Any VCD, respectively), it is unclear why some of the serotype-specific nAb titer readouts were stronger CoRs for their respective matched-serotype VCD endpoints than other serotype-specific nAb titer readouts were for their respective matched-serotype VCD endpoints. For example, for both assays, DENV-1 and DENV-3 titers tended to be less strong CoRs (albeit still significant CoRs) for serotype-matched VCD than DENV-2 and DENV-4 titers [PRNT50 –DENV-1: 0.31 (0.23, 0.42); DENV-3: 0.41 (0.27, 0.64) vs DENV-2: 0.18 (0.12, 0.27); 0.22 (0.09, 0.51). MN–DENV-1: 0.23 (0.15, 0.33); DENV-3: 0.23 (0.13, 0.41) vs DENV-2: 0.14 (0.08, 0.26); DENV-4: 0.08 (0.03, 0.19)] (Table 1). Differences between in vitro systems for assessing antibody-mediated DENV neutralization, which use cultured cell lines and laboratory DENV strains, versus neutralization of circulating DENV viral variants in the human body, may be relevant. For instance, the PRNT50 and MN assays assess neutralization of only one DENV strain per serotype. If participants are exposed to circulating viral variants that are neutralized less well (or better) than the assayed strain, the obtained titer for that serotype will be less representative of how well nAbs in that participant’s serum neutralize exposing viral variants. While it is generally assumed that nAb binding epitopes are conserved within serotypes, there is evidence supporting significant variation in neutralization across genotypes of a given serotype, particularly for DENV-1 and DENV-3 [38-40]. We speculate that a scenario in which contemporaneously circulating DENV-1 and DENV-3 strains are neutralized less well (or better) than the DENV-1 PUO-359 (isolated in 1980 in Thailand) and DENV-3 PaH881/88 (isolated in 1988 in Thailand) strains used in the PRNT50 and MN assays (i.e. the parental DENVs of the respective recombinant vaccine viruses) could explain why DENV-1 and DENV-3 titers tended to be weaker (yet still significant) CoRs versus DENV-2 and DENV-4 titers.

The proportion of mismatched amino acid residues between the vaccine DENV inserts and the DENV sequences isolated from placebo group cases provides an assessment of the degree of match between circulating viral variants at the time of the trial and the vaccine strains, and may also be relevant to explain potential differences across serotypes in the strength of serotype-specific CoRs of matched-serotype VCD. We have previously analyzed these proportions by serotype and shown that, in CYD15, DENV-1 circulating strains were farthest from the DENV-1 vaccine insert, followed by DENV-2 circulating strains to the DENV-2 insert, DENV-3 circulating strains to the DENV-3 insert, and then DENV-4 circulating strains found to be closest to the DENV-4 vaccine insert [41]. The latter finding is consistent with DENV-4 titer being the strongest serotype-specific CoR of matched-serotype VCD in CYD15 (Table 1); however, these findings do not fully explain why DENV-2 titer tended to be a stronger CoR of DENV-2 VCD than DENV-1 titer was of DENV-1 VCD in CYD15.

Our analysis is distinguished from previous MN-based assay comparisons to the PRNT assay [15, 16, 42] by its much larger sample sizes. We found that MN and PRNT50 titers had excellent correlation for all serotypes, with average titer Spearman correlation coefficients from pairwise plots equalling or exceeding 0.96, across both trials and time-points. Correlations were somewhat lower for DENV-4, perhaps due to the larger size of DENV-4 plaques, which may introduce more subjectivity into the counting, especially when plaques cluster together. Nonetheless, the DENV-4 correlations were higher than the R2 value (0.672) reported in [42] and comparable to the Pearson correlation coefficient (0.84) reported in [15].

The two assays showed good agreement with respect to baseline serostatus classification, although the proportion of MN-/PRNT50+ participants always slightly exceeded the proportion of MN+/PRNT50- participants, across age groups and trials. Technical differences between the two assays must account for this observed difference in baseline seronegativity determination. Of note, the MN assay uses a higher virus input and lower serum volume in the neutralization reaction, thus having a much higher molar ratio of neutralizing epitopes to bind compared to the PRNT assay. Moreover, the MN assay does not use an overlay and thus the antibodies must continuously neutralize the virus, while the PRNT assay measures a one-hit neutralization event. The ability of antibodies to neutralize the virus is also affected by the replication rate of the individual viruses in the MN assay, while it is controlled in the PRNT assay.

Both PRNT50 and MN are complex assays performed in specialized laboratories, typically for research/investigational purposes rather than patient care decisions. Thus, neither assay would be used for determining eligibility for CYD-TDV vaccination (e.g. in “pre-vaccination screening”) [10]. However, the MN assay could be considered as an alternative test to determine eligibility of participation in future clinical trials of the CYD-TDV vaccine, with vaccination restricted to those testing positive. In this context, the MN assay may be theoretically somewhat advantageous to the PRNT50 assay for excluding true dengue-seronegative individuals from vaccination, given that our findings suggest possible higher specificity of the MN assay to determine dengue seropositivity. Moreover, due to the possible lower sensitivity of the MN assay, other vaccine candidates that base assessments of serostatus on MN may misclassify more true seropositives as seronegatives, potentially resulting in some degree of bias in seronegative estimates with this or similar assays.

We next consider why individual-level classification accuracy, using the best model identified by Super-Learner for each of four different covariate groups, was relatively limited. Most participants in vaccine efficacy trials will not experience the VCD endpoint over the study follow-up timeframe, irrespective of vaccination, based solely on the epidemiology of exposure, infection, and symptomatic disease frequency. The relative rareness of the VCD endpoint presents a significant challenge in improving classification of individuals who will vs. will not experience the VCD endpoint. CYD-TDV vaccination may also impact risk differently depending on genetic/antigenic features of different variants within the same serotype or genotype [41], which could also explain the limited classification accuracy.

Overall, we conclude that Month 13 MN titer performs comparably to Month 13 PRNT50 titer as a CoR and as a CoVE, supporting that the MN assay could be an alternative to the PRNT assay for assessing neutralizing antibody titers in immunogenicity studies, immune correlates studies, and immuno-bridging applications (e.g. validating a new vaccine lot).

Correlations between MN and PRNT50 titers.

(DOCX)

Click here for additional data file.

Concordance with respect to baseline serostatus classification in the CYD14 and CYD15 trials for the MN and PRNT50 assays.

(DOCX)

Click here for additional data file.

Logistic regression estimated odds ratios (ORs) (95% CIs) of matched-serotype VCD per log10 increase in Month 13 neutralizing antibody titer as measured by each assay, pooling data from the CYD14 and CYD15 studies together.

For the PRNT50 assay, the CoR analyses were performed on all CYD14 and CYD15 data, as reported in Moodie et al. [11], and for the MN assay, the CoR analyses were restricted to 9–16-year-olds from CYD14 and 15.

(DOCX)

Click here for additional data file.

Distinct input variable sets used for learning algorithms and learning algorithms in the Super-Learner library of estimators of the conditional probability of DENV-Any.

(DOCX)

Click here for additional data file.

Model terms for the best interpretable model for the Vaccine group, for different data sets.

(DOCX)

Click here for additional data file.

Comparison of classification of CYD14 and CYD15 9–16-year-old immunogenicity subset (A, B) cases and controls or (B, C, E, F) controls as (A, B, C) dengue seronegative vs. (D, E, F) dengue seropositive at (A, D) baseline and at (B, C, E, F) Month 13 according to the PRNT50 (blue) or MN (red) assay. Seropositivity was defined as a titer ≥ 10 for each individual serotype and as a titer ≥ 10 of at least one serotype for the Average readout. Seronegativity was defined as a titer < 10 for each individual serotype and as a titer < 10 for all individual serotypes for the Average readout.

(TIF)

Click here for additional data file.

Classification accuracy (A,B) of different algorithms using demographic + MN + PRNT50 data and cross-validated estimated probabilities of DENV-Any by case-control status (C). (A, B): CV-AUC values for classification accuracy of different algorithms using demographic + MN + PRNT50 data as to whether each participant experienced DENV-Any VCD between Months 13 and 25 are shown for (A) the vaccine group and (B) the placebo group for the combined CYD14 and CYD15 9-16-year-old cohort. (C) Cross-validated estimated probabilities of DENV-Any in the vaccine group by case-control status for the best-performing models for each covariate group for the combined CYD14 and CYD15 9-16-year-old cohort.

(TIF)

Click here for additional data file.

Case-cohort sampling design for measurement of Month 13 MN titers in CYD14 and CYD15 participants.

(DOCX)

Click here for additional data file.

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

9 Mar 2020

PONE-D-19-30338

Microneutralization Assay Titer Correlates Analysis in Two Phase 3 Trials of the CYD-TDV Tetravalent Dengue Vaccine in Asia and Latin America

PLOS ONE

Dear Prof Gilbert,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the reviewers' points below raised during the review process.

We would appreciate receiving your revised manuscript by Apr 23 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Ray Borrow, Ph.D., FRCPath

Academic Editor

PLOS ONE

Journal Requirements:

1.  When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Competing Interests section:

"I have read the journal's policy and the authors of this manuscript have the following competing interests: MB, LZ, LC, RS, SS, and CDG are employees of Sanofi Pasteur. LNC, YF, ZM, MJ, YH, BP, YZ, JS, and PBG received a contract from Sanofi Pasteur to conduct the statistical analysis work and submit the results for publication. Sanofi Pasteur is the manufacturer of the CYD-TDV vaccine (Dengvaxia).".

i) Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

ii) Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this manuscript, the authors have developed a high throughput micro-neutralization assay to assess neut. titers of dengue viruses post immunization with Sanofi Pasteur’s tetravalent dengue vaccine. While the PRNT assay is considered the gold standard assay, it is laborious and there are challenges associated with performing the assay in laboratories. Using samples from month 13 and at baseline from the CYD 14 and 15 clinical trials, the authors compare titers using the MN and PRNT50 assays. They find strong correlations between high titers at M13 and vaccine efficacy regardless of serotype. Give the high throughput nature of the MN assay, it certainly would be valuable for vaccine manufacturers to assess a main correlate of protection.

The MN is similar to the PRNT assays and does not solve challenges such as the use of lab strains of virus, cultured cell lines for testing. Other MN assays have been published before and the advantages of this assay are the relatively high throughput nature of the assay and the number of serum samples tested which lend further validity to the assay. The authors have carefully layed out the differences in PRNT50 and MN assays including higher virus input, lower serum amount and antibodies being continuously able to neutralize virus in the reaction.

Overall, the data provided, the large number of samples tested in the MN assay and the stringent comparisons made to the traditional PRNT50 assays lend confidence that the assay should be used to test additional end points.

Reviewer #2: The manuscript by Carpp et al reported the development of new microneutralization (MN) assay and compared the MN titers with a validated 50% plaque reduction neutralization test (PRNT50) in analysis of the correlates in tow Phase 3 clinical trials of the CYDTDV Tetravalent Dengue Vaccine in Asia and Latin America. The results demonstrated that the neutralizing titers assessed by MN and PRNT50 assays had good correlates. The high-throughput MN assay can be useful to for assessing neutralizing antibody responses induced by Dengue vaccines and infections to evaluate the correlates of risk and vaccine efficacy. The manuscript and data are well-organized.

Reviewer #3: The manuscript develops and validates the MN assay in light of the PRNT assay, with respect to dengue vaccine development. The authors consider a case-cohort design. The paper is nicely written, provides significant details, and the statistical analyses is rigorous and appropriate. I have some additional commens/clarifications:

1. In Page 6 (Middle), the authors state "....first 2 to 4 months of each trial". Is this trial registered with ClinicalTrials.gov and has a NCT number? If not, why is it then called a clinical trial? Any prior trial? Details needed for a smoother reading experience.

2. Any justification behind the use of the SuperLearner package? Has it been established that this ensemble machine-learning technique almost always produces better prediction and discriminatory performances in this specific field of research (dengue vaccime development)? Maybe a comparison with a standard statistical model, and further illustration of the actual gain would be worthwhile. I see a comparison with SL.gam; can the authors specify what is that? I assume it is some generalized additive model, and certainly not a standard regression.

3. It was hard to find the complete list of covariates fed into the SuperLearner. It would be better to provide the list somewhere during model fitting.

4. Inverse probability of weighting (line 154, page 8) require a citation.

5. Line 155, page 8: "Models were built...". What models? Write clearly.

6. Line 156, page 8: expand CV-AUC, with a reference.

Reviewer #4: Carpp and Fong et al. compare how well a high-throughput microneutralization assay compares to the gold-standard PRNT50 assay as a correlate of risk and correlate of vaccine efficacy against virologically confirmed dengue in the CYD-TDV Phase 3 vaccine trials. Overall, this manuscript provides valuable information on comparison of the two assays in the context of a vaccine trial, as well as additional novel scientific investigations into dengue vaccine efficacy.

Major comments:

Methods:

- There is very little information provided about the MN. Key useful details that should be included for the MN include how much virus is added to wells, how long virus is allowed to replicate in cells before the assay is terminated, how the assay is terminated, etc. The authors mention some differences between the MN and the PRNT in the discussion. However, actual information on the two assays is not detailed in the manuscript.

- The methods section on the immune correlates analyses only includes a reference to a previous article. There is no limit on words for this manuscript. It would be helpful to the reader to provide a brief description of the immune correlates analyses in the methods section.

Results:

- It is unusual that there are entire paragraphs on results that are only shown in supplemental tables and figures. Why not just include these data as additional manuscript figures and tables? Especially Table S3 and Fig. S2.

- Line 351-352: “the DENV-2 and DENV-4 correlates of risk were significantly modified by treatment group, indicating departure from the third criterion”. While this is true of DENV2 and DENV4, the effects go in opposite directions and are different in important ways. The treatment contributes to a higher cumulative endpoint rate for DENV2 at low titers but a lower cumulative endpoint rate for DENV4 across titers. Also, there is a significant elevated risk of the treatment group in Model 4 for DENV2 in Table S4. These important findings should be mentioned in the text.

- Line 352-354: "Together, these results show that Month 13 PRNT50 titer and Month 13 MN titer are consistent with the Prentice criteria for DENV-1 but not for the other serotypes." This is quite interesting. Why is this as supplemental figure (Fig. S2)?

- Line 360-361: "We previously showed that estimated VE against DENV-Any was approximately 25% for vaccine recipients with no seroresponse at Month 13…” Presumably, no seroresponse means titers <10? The model estimates for the undetectable titers (<10) are not shown in the figure, nor in the original figure S15 of Moodie et al. 2017. However, the <10 value is shown on the x-axis. The figures appear to only show titers from a value of 10, which is detectable. Is this just a plotting issue? On the microneut panel in Fig. 6, the x-axis only goes to 10, even though there are individuals with MN values <10.

- Line 368-370: "At an average MN (PRNT50) Month 13 titer of 1000, VE estimates were 88% (85%) for baseline-seropositive vaccine recipients and 78% (76%) for baseline-seronegative vaccine recipients (Figure 6)." Why state the VE at this high a titer value? Based on the titer distributions, it was rare for individuals to have that high of titers in the trial even among controls. Perhaps it would be better to report VE based on the median titer observed.

- Part B Table S6: the model shows some very strong significant OR >>>1. (e.g. 52, 79, 830). What does this mean? Some of the effects seem to be for interaction terms? It is very difficult to interpret what this means without a description of what the terms are.

Discussion:

- Paragraph, 470: “It is unclear why nAb titer readouts did not perform equally well across serotypes as CoRs for their matched-serotype VCD endpoints.” Is this paragraph referring to Table 1? I thought the non-significant effects were DENV3 and DENV4? This paragraph mentions DENV1 and DENV3?

- Line 517: "We next consider why individual-level classification accuracy using the different variable input." Is this referring to the super-learner algorithm?

Minor comments:

- Line 60: "Estimated VE against these two endpoints was negative in baseline-seronegative individuals”. By negative, you mean had a negative vaccine efficacy, meaning worse off? This is unclear as written.

- Line 156: Please write out CV-AUC (cross validated area under the curve) the first time it appears.

- "Lack of Month 13 sero-response at Month 13 approached zero for both assays (≤ 2%) for DENV-3 and DENV-4 (P>0.05 for both), but significant discordance remained between the MN and PRNT50 results for DENV-1 and DENV-2 (P<0.01 for both) (S1 Fig, panel C)." This sentence is confusing as written.

- Line 232-234: "As one log10 increase in PRNT50 titer approximately equaled one log10 increase in MN titer (Figure 1), we used OR for comparing the two nAb readouts as CoRs.” Why not directly quantify this relationship? Why say approximately one log10 increase here?

- Fig. S2: the cumulative endpoint rate for Fig S2 C and D are quite different (DENV2): 0.05 for the PRNT vs. 0.014 for MN assay? Other y-axes of the PRNT50 vs. MN panels in this figure are more consistent.

- Fig. 7: no color is shown in the legend for for D2 model panel A. I assume it should be blue?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Anuja Mathew

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

12 Apr 2020

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response: We have ensured that all PLOS ONE style requirements are met in the revision.

2. Thank you for stating the following in the Competing Interests section:

"I have read the journal's policy and the authors of this manuscript have the following competing interests: MB, LZ, LC, RS, SS, and CDG are employees of Sanofi Pasteur. LNC, YF, ZM, MJ, YH, BP, YZ, JS, and PBG received a contract from Sanofi Pasteur to conduct the statistical analysis work and submit the results for publication. Sanofi Pasteur is the manufacturer of the CYD-TDV vaccine (Dengvaxia).".

i) Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

ii) Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Response: We have included the statement above in our revised Competing Interests section.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

-----------------------------------------------------------------------------------------------------------------------------

Reviewer #1: In this manuscript, the authors have developed a high throughput micro-neutralization assay to assess neut. titers of dengue viruses post immunization with Sanofi Pasteur’s tetravalent dengue vaccine. While the PRNT assay is considered the gold standard assay, it is laborious and there are challenges associated with performing the assay in laboratories. Using samples from month 13 and at baseline from the CYD 14 and 15 clinical trials, the authors compare titers using the MN and PRNT50 assays. They find strong correlations between high titers at M13 and vaccine efficacy regardless of serotype. Give the high throughput nature of the MN assay, it certainly would be valuable for vaccine manufacturers to assess a main correlate of protection.

The MN is similar to the PRNT assays and does not solve challenges such as the use of lab strains of virus, cultured cell lines for testing. Other MN assays have been published before and the advantages of this assay are the relatively high throughput nature of the assay and the number of serum samples tested which lend further validity to the assay. The authors have carefully layed out the differences in PRNT50 and MN assays including higher virus input, lower serum amount and antibodies being continuously able to neutralize virus in the reaction.

Overall, the data provided, the large number of samples tested in the MN assay and the stringent comparisons made to the traditional PRNT50 assays lend confidence that the assay should be used to test additional end points.

Reviewer #2: The manuscript by Carpp et al reported the development of new microneutralization (MN) assay and compared the MN titers with a validated 50% plaque reduction neutralization test (PRNT50) in analysis of the correlates in tow Phase 3 clinical trials of the CYDTDV Tetravalent Dengue Vaccine in Asia and Latin America. The results demonstrated that the neutralizing titers assessed by MN and PRNT50 assays had good correlates. The high-throughput MN assay can be useful to for assessing neutralizing antibody responses induced by Dengue vaccines and infections to evaluate the correlates of risk and vaccine efficacy. The manuscript and data are well-organized.

Reviewer #3: The manuscript develops and validates the MN assay in light of the PRNT assay, with respect to dengue vaccine development. The authors consider a case-cohort design. The paper is nicely written, provides significant details, and the statistical analyses is rigorous and appropriate. I have some additional commens/clarifications:

Response: Thank you for the positive comments and feedback.

1. In Page 6 (Middle), the authors state "....first 2 to 4 months of each trial". Is this trial registered with ClinicalTrials.gov and has a NCT number? If not, why is it then called a clinical trial? Any prior trial? Details needed for a smoother reading experience.

Response: The CYD14 and CYD15 trials were both registered with ClinicalTrials.gov and each has an NCT number. This information is included in the “CYD14 and CYD15” subsection of “Materials and methods”. For additional clarity, we have added “ClinicalTrials.gov ID” in front of each NCT number (new text in this revision is underlined):

“In harmonized designs, healthy children and adolescents aged 2-14 (CYD14; ClinicalTrials.gov ID NCT01373281 [6]) or 9-16 (CYD15; ClinicalTrials.gov ID NCT01374516 [7]) were randomized (2:1) to vaccine or placebo, with randomization stratified by age group and site.” (lines 97-98)

2. Any justification behind the use of the SuperLearner package? Has it been established that this ensemble machine-learning technique almost always produces better prediction and discriminatory performances in this specific field of research (dengue vaccine development)? Maybe a comparison with a standard statistical model, and further illustration of the actual gain would be worthwhile. I see a comparison with SL.gam; can the authors specify what is that? I assume it is some generalized additive model, and certainly not a standard regression.

Response:

Any justification behind the use of the SuperLearner package?

Super-Learner possesses an oracle property, in that it selects a learner as good asymptotically as any individual learner in the specified ensemble of learners. The fact its model selection is based on double-nested cross-validation and is implemented in press-button fully pre-specified fashion makes it objective. Moreover, its ability to easily incorporate a large library of learners allows it to have good performance in accuracy and precision of models, as found in many simulation studies and data applications (1, 2). Our previous experience using the Super-Learner R package includes assessment of varicella zoster virus-specific glycoprotein-based enzyme-linked immunosorbent assay (gpELISA) antibody titer as a predictor of herpes zoster in the Zostavax Efficacy and Safety Trial (3), assessment of dozens of antibody and T cell markers as predictors of HIV infection in the HVTN 505 HIV vaccine efficacy trial (4), and assessment of gp160 amino acid sequence features as predictors of whether HIV-1 Envelope pseudoviruses are resistant to neutralization by the monoclonal antibody VRC01 (2). In another application of Super-Learner to the CYD14 and CYD15 trials, we found that both logistic regression and Super-Learner were highly predictive of pre-vaccination dengue PRNT50 serostatus (5).

Has it been established that this ensemble machine-learning technique almost always produces better prediction and discriminatory performances in this specific field of research (dengue vaccine development)?

For application to the dengue vaccine efficacy trials CYD14 and CYD15 in particular, Super-Learner is always either the best predictive model or close to the best predictive model, and because the classification accuracy is estimated by double-nested cross-validation, these comparisons are fair. In a statistical methods paper (6), we studied the Super-Learner package by applying it to the CYD14 and CYD15 trials as well as studying its operating characteristics in simulation studies, which verified that its general properties seemed to carry over to the dengue vaccine application setting. However, Super-Learner does not always provide best-predictive performance on a specific data set, as can be seen from the results in Table 3, where Super-Learner provided best performance for 4 of the 8 data analyses, whereas a generalized additive model (SL.gam) provided best performance for the other 4 of 8 data analyses. The Super-Learner model was always close to the best performing model as judged by cross-validated area under the ROC curve (CV-AUC).

Maybe a comparison with a standard statistical model, and further illustration of the actual gain would be worthwhile.

We agree that such a comparison is worthwhile to the reader. S2 Fig provides such a comparison. It compares the classification accuracy (as to whether a participant experienced DENV-Any VCD between Months 13 and 25, using demographic + MN + PRNT50 data) of different algorithms, including Super-Learner and individual statistical algorithms such as logistic regression using all variables (SL.glm) and logistic regression with best model selected by step-wise model-selection (SL.step).

We have added the following text (underlined) to the revision:

Panels A and B of S2 Fig compare the CV-AUCs for all algorithms, including Super-Learner along with individual statistical algorithms such as standard logistic regression, applied using input variables demographics+MN+PRNT50 data. The results show slight gains in classification accuracy for Super-Learner compared to most of the individual algorithms, and large gains in classification accuracy for Super-Learner compared to the polymars, mean, and nnet algorithms. These results held true for both the placebo and vaccine groups. (Part B of S4 Table provides more information on the learning algorithms in the Super-Learner library of estimators of the conditional probability of DENV-Any). (lines 502-509)

I see a comparison with SL.gam; can the authors specify what is that? I assume it is some generalized additive model, and certainly not a standard regression.

As stated in part B of S4 Table, “Learning Algorithms in the Super-Learner Library of Estimators of the Conditional Probability of DENV-Any”, all algorithm type names are built-in algorithms in the Super-Learner R package available at CRAN, and have documentation within the package. However, your comment makes us realize that the tables in the manuscript need more complete annotation of the meaning of the algorithm types. We have added the following information to the tables as appropriate:

To S4 Table:

“b SL.mean is a base reference model using no input variables; SL.glm is a logistic regression model fit to all input variables; SL.glm.interaction is a logistic regression model including all input variables together with all pairwise-interaction variables; SL.step is a logistic regression model with step-wise model selection with best model selected by the AIC criterion; SL.glmnet is the lasso that includes variables with non-zero estimated coefficients in the default implementation of SL.glmnet that optimizes the tuning parameter via cross-validation; SL.bayesglm is Bayesian logistic regression; SL.gam is a generalized additive model with smoothing splines for the neutralization titer variables; SL.nnet is a neutral network; SL.polymars is multivariate adaptive polynomial spline regression.”

To Table 3:

“SL.gam is a generalized additive model with smoothing splines for the neutralization titer variables.”

3. It was hard to find the complete list of covariates fed into the SuperLearner. It would be better to provide the list somewhere during model fitting.

Response: Thank you for raising this point. The complete list of the input variables (e.g. demographic variables, MN titer variables, PRNT50 titer variables) used in the various supervised learning analyses is given in Part A of S4 Table. We have expanded the description of S4 Table in the text as follows:

“S4 Table provides a complete list of the input variables (e.g. demographic variables, MN titer variables, PRNT50 titer variables) used in each covariate group for the various supervised learning analyses, in addition to further information on the statistical learning algorithms in the Super-Learner library of estimators of the conditional probability of DENV-Any.” (lines 483-487)

4. Inverse probability of weighting (line 154, page 8) require a citation.

Response: We have added one (“Inverse probability of censored weighting [31] was employed….”) (line 199).

5. Line 155, page 8: "Models were built...". What models? Write clearly.

Response: We have revised the text as follows (added text is underlined):

“Models of the conditional probability of DENV-Any occurrence by the Month 25 visit were built separately for the vaccine and placebo groups using four input variable sets…” (lines 200-203)

6. Line 156, page 8: expand CV-AUC, with a reference.

Response: We have expanded CV-AUC and added a reference.

“…aiming to maximize the cross-validated area under the receiver operating characteristic curve (CV-AUC) [32].” (lines 202-203)

Reviewer #4: Carpp and Fong et al. compare how well a high-throughput microneutralization assay compares to the gold-standard PRNT50 assay as a correlate of risk and correlate of vaccine efficacy against virologically confirmed dengue in the CYD-TDV Phase 3 vaccine trials. Overall, this manuscript provides valuable information on comparison of the two assays in the context of a vaccine trial, as well as additional novel scientific investigations into dengue vaccine efficacy.

Response: Thank you for the positive feedback.

Major comments:

Methods:

- There is very little information provided about the MN. Key useful details that should be included for the MN include how much virus is added to wells, how long virus is allowed to replicate in cells before the assay is terminated, how the assay is terminated, etc. The authors mention some differences between the MN and the PRNT in the discussion. However, actual information on the two assays is not detailed in the manuscript.

Response: Please see the text added in “MN assay” in Methods (lines 138-164):

“Briefly, 2-fold serial dilutions of serum samples (starting at 1:5 dilution) were incubated with an equivalent volume of a constant challenge dose of virus (200 TCID50 per well for each serotype) and incubated for 90 minutes at 37°C. A separate virus titration plate was prepared to determine the 50% tissue-culture infective dose (TCID50). After neutralization, the serum-virus mixture was added to pre-seeded Vero cell monolayers in 96-well plates and an additional 100 �l of cell culture medium was added without removal of the virus inoculum after adsorption. The plates were incubated at 37°C for either 4-5 days (depending on the virus serotype). The target virus challenge dose and days of incubation post-infection were determined for each serotype in order to provide an optimal signal-to-noise ratio during the ELISA steps. After the incubation period was complete, the cell culture medium was removed from the plates. The cells were then fixed with 80% acetone and incubated at room temperature for 10-15 min, followed by blocking with 5% non-fat dry milk in PBS-Tween-20 wash buffer. Dengue serotype-specific monoclonal antibodies were added, followed by anti-mouse IgG HRP congugate and TMB substrate. The reaction was stopped with 2N sulfuric acid and the optical density (OD) of each well at 450 nm (650 nm as the reference wavelength) was measured using a SpectraMax 384 microplate reader with SoftMax Pro software version 6.5.1.

The 50% neutralization titer of the test serum sample against each serotype was defined as the reciprocal of the test serum dilution for which the virus infectivity was reduced by 50% relative to the challenge virus dose (without any antibodies) introduced into the assay and was calculated using the formula: [(Average OD of Virus Control - Average OD of Cell Control)/2 + Average OD of Cell Control]. The MN titer for each test sample was interpolated by calculating the slope and intercept using the last dilution with an OD below the 50% neutralization point and the first dilution with an OD above the 50% neutralization point to determine the MN titer using the formula: [MN Titer = (50% neutralization point - intercept)/slope]. Neutralization titers are presented as continuous values. For both assays, the lower limit of quantitation was 10; values below this were set to 5. The average titer is the average of each participant’s four serotype-specific log10 titers.”

- The methods section on the immune correlates analyses only includes a reference to a previous article. There is no limit on words for this manuscript. It would be helpful to the reader to provide a brief description of the immune correlates analyses in the methods section.

Response: We have added additional detail (underlined text) to the “Immune correlates” subsection in the “Materials and methods” section, as shown below:

“Month 13 MN titers were assessed as CoRs and CoVEs as in [11]. In brief, the CoR analyses were performed in each of the vaccine and placebo groups separately, relating VCD risk to a given Month 13 MN titer variable with a logistic regression model that accounted for the case-cohort sampling design [22] and adjusted for age, sex, and country. Results are reported as odds ratios of DENV-Any, DENV-1, DENV-2, DENV-3, and DENV-4 VCD per log10 increase in Month 13 nAb titer. P values for testing DENV-1, DENV-2, DENV-3, and DENV-4 nAb titer as a CoR were adjusted across the 4 serotypes using family-wise error rate (Holm-Bonferroni [23]) and false-discovery rate (Q values [24]) adjustment, separately for each treatment group and each trial. All P values and Q values are 2-sided. The CoVE analyses were performed using the VE curve-effect modification framework [25-27]. This framework assesses how VE changes over subgroups of vaccine recipients, where subgroups are defined by Month 13 nAb titers. The analyses used the Juraska et al. method [28], employed with hinge point logit linear models [29] when there was sufficient data, if not, linear logistic regression models were used. Advantages of the hinge point models are summarized in reference [29]. VE curves were estimated with pointwise and simultaneous bootstrap-based Wald 95% confidence intervals (CIs).” (lines 173-188)

Results:

- It is unusual that there are entire paragraphs on results that are only shown in supplemental tables and figures. Why not just include these data as additional manuscript figures and tables? Especially Table S3 and Fig. S2.

Response: Thank you for the helpful suggestion. We have moved Table S3 to the main text (Table 2 in the revised manuscript) and Fig. S2 to the main text (Fig 6 in the revised manuscript).

- Line 351-352: “the DENV-2 and DENV-4 correlates of risk were significantly modified by treatment group, indicating departure from the third criterion”. While this is true of DENV2 and DENV4, the effects go in opposite directions and are different in important ways. The treatment contributes to a higher cumulative endpoint rate for DENV2 at low titers but a lower cumulative endpoint rate for DENV4 across titers. Also, there is a significant elevated risk of the treatment group in Model 4 for DENV2 in Table S4. These important findings should be mentioned in the text.

Response: This is an interesting point. Care must be taken in making comments about the vaccine effect in subgroups defined by DENV-2 titer measured 13 months after randomization, as the models employed in Fig. S2 (Fig 6 in the revised manuscript) and Table S3 (Table 2 in the revised manuscript) do not measure a causal vaccine effect. When studying the vaccine effect in such post-vaccination subgroups there are concerns for post-randomization selection bias (7). This is why we instead focused on reporting correlate of VE curves (as reported in Fig 5), an approach designed to avoid potential post-randomization selection bias by estimating a causal vaccine effect across subgroups defined by Month 13 DENV-2 titer if assigned vaccine (which is a counterfactual variable for individuals assigned to the placebo group). In our previous work by Moodie et al. (8), which assessed correlates of risk and correlates of vaccine efficacy restricting to the PRNT50 assay, we studied whether VE against DENV-2 might be below zero at low Month 13 DENV-2 PRNT50 titers of vaccine recipients. We quote from our Moodie et al. work: “The DENV-2 VE curves across the complete age range in each trial (Supplementary Figures S8 and S9) suggest possible negative VE for vaccine recipients without anti-DENV-2 titers (i.e., PRNT50 below the LLOQ). However, the simultaneous 95% confidence bands for DENV-2 VE include 0%, and an inference of negative VE is based on a very small number of vaccine-recipient DENV-2 cases (0 of 87 in CYD14 and 5 of 264 in CYD15).” For convenience, the DENV-2 panels in Figures S8 and S9 from Moodie et al. are included below.

Panel B (DENV-2 panel) from Figure S8 in Moodie et al. 2018 JID: Estimated vaccine efficacies in CYD14 for all age groups against serotype-specific dengue endpoints between Months 13 and 25 by homologous titers at Month 13 with 95% pointwise and simultaneous confidence intervals and histograms of the homologous titers in the vaccine group for the Month 13 at-risk cohort.

Panel B (DENV-2 panel) from Figure S9 in Moodie et al. 2018 JID: Estimated vaccine efficacies in CYD15 for all age groups against serotype-specific dengue endpoints between Months 13 and 25 by homologous titers at Month 13 with 95% pointwise and simultaneous confidence intervals and histograms of the homologous titers in the vaccine group for the Month 13 at-risk cohort.

Examining Fig 6, there is less evidence for potential negative VE based on DENV-2 MN titer than based on DENV-2 PRNT50 titer, such that the concern for vaccine-enhancement at low titers was already addressed in Moodie et al. We added the following text (lines 417-428):

“Regarding point (3), the cumulative endpoint rates of DENV-2 VCD by Month 13 DENV-2 PRNT50 titer (Panel C of Fig 6) suggest that CYD-TDV vaccination could have increased DENV-2 VCD risk at lowest Month 13 DENV-2 PRNT50 titers. Moodie et al. [11] previously addressed this issue, noting that simultaneous 95% confidence bands for DENV-2 VE include 0%, and an inference of vaccine-increased risk is based on a very small number of vaccine-recipient DENV-2 cases (0 DENV-2 cases among the 87 vaccine-recipients with no Month 13 PRNT50 DENV-2 seroresponse in CYD14 and 5 DENV-2 cases among the 264 vaccine-recipients with no Month 13 PRNT50 DENV-2 seroresponse in CYD15, with “no Month 13 PRNT50 DENV-2 seroresponse” defined as Month 13 PRNT50 DENV-2 titer < 10). Panel D of Fig 6 shows less concern for potential vaccine-increased DENV-2 VCD risk for individuals with lowest DENV-2 titers based on the MN assay, given that the vaccine and placebo curves are more similar in the left-tail of the plot.”

Addressing the reviewer’s related comment about DENV-4, we have also added the following text (lines 429-432):

“For DENV-4, cumulative endpoint rates decreased with increasing Month 13 serotype-matched titers in both treatment groups, with low cumulative DENV-4 rates at low titers (Panels G and H of Fig 6), and a borderline significant result that the rate was lower in the vaccine group at low titers.”

- Line 352-354: "Together, these results show that Month 13 PRNT50 titer and Month 13 MN titer are consistent with the Prentice criteria for DENV-1 but not for the other serotypes." This is quite interesting. Why is this as supplemental figure (Fig. S2)?

Response: As mentioned above, we have moved Fig. S2 to the main text (Fig 6 in the revised manuscript). Moreover, we think this result is interesting enough that it is appropriate to include it in the abstract. We have added the following sentence to the abstract:

“We also studied each assay as a valid surrogate endpoint based on the Prentice criteria, which supported each assay as a valid surrogate for DENV-1 but only partially valid for DENV-2, -3, and -4.” (lines 35-37)

- Line 360-361: "We previously showed that estimated VE against DENV-Any was approximately 25% for vaccine recipients with no seroresponse at Month 13…” Presumably, no seroresponse means titers <10? The model estimates for the undetectable titers (<10) are not shown in the figure, nor in the original figure S15 of Moodie et al. 2017. However, the <10 value is shown on the x-axis. The figures appear to only show titers from a value of 10, which is detectable. Is this just a plotting issue? On the microneut panel in Fig. 6, the x-axis only goes to 10, even though there are individuals with MN values <10.

Response: Thank you for the suggestion to clarify the definition of “no seroresponse” and to harmonize x-axis labels in Fig 6 (Fig 7 in the revised manuscript). Below we respond to the different elements of your comment:

We previously showed that estimated VE against DENV-Any was approximately 25% for vaccine recipients with no seroresponse at Month 13…” Presumably, no seroresponse means titers <10?

Response: Yes. We have added the following text to the revised manuscript (lines 446-451):

“We previously showed that estimated VE against DENV-Any was approximately 25% for vaccine recipients with no seroresponse (defined as PRNT50 titer less than the assay lower limit of quantification, 10, for all four serotypes) at Month 13 and increased similarly with average Month 13 PRNT50 titer in baseline seronegative vs. baseline seropositive subgroups in CYD14 and CYD15 9–16-year-olds (baseline serostatus determined by the PRNT50 assay) (Fig 7A, 7B; reproduced with modification from [11]).”

The model estimates for the undetectable titers (<10) are not shown in the figure, nor in the original figure S15 of Moodie et al. 2017. However, the <10 value is shown on the x-axis. The figures appear to only show titers from a value of 10, which is detectable. Is this just a plotting issue? On the microneut panel in Fig. 6, the x-axis only goes to 10, even though there are individuals with MN values <10.

Response: Thank you for the suggestion to harmonize x-axis labels in Fig 6 (Fig 7 in the revised manuscript) across the VE curves by Month 13 average PRNT50 titer (Panels A, B) and the VE curves by Month 13 average MN titer (Panels C, D), all of which were generated using the Zhuang et al. method. The Zhuang et al. method was developed assuming an immune assay was measured with a lower limit of detection S=max(S*,c) (with c=10 for both the PRNT50 and MN assays) (i.e. the observed immune response is left-truncated at value 10), and on a model of disease risk conditional on the observed S. Thus, each curve shown in Fig 7 starts at x= 10, since that is the smallest value S could take.

Thus, even though panels A and B in Fig 6 of the originally submitted manuscript had a hash mark for “<10” on the leftmost part of the x-axis, there is not actually any data plotted with such a titer value. We see now that it could potentially be confusing to have this hashmark present, as there is no data there, and have removed the “<10” hash mark from panels A and B. As to the statement that “On the microneut panel in Fig. 6, the x-axis only goes to 10, even though there are individuals with MN values <10”, the method description above explains why each VE curve starts with 10, such that no VE value is reported for any MN value <10. If one adds a straight vertical line at a titer value of “10”, it is clear that there are no data points to the left of the line. We considered adding such a line to the curves but thought it would be visually distracting.

- Line 368-370: "At an average MN (PRNT50) Month 13 titer of 1000, VE estimates were 88% (85%) for baseline-seropositive vaccine recipients and 78% (76%) for baseline-seronegative vaccine recipients (Figure 6)." Why state the VE at this high a titer value? Based on the titer distributions, it was rare for individuals to have that high of titers in the trial even among controls. Perhaps it would be better to report VE based on the median titer observed.

Response: This is a good point. We now report VE in baseline-seropositive and baseline-seronegative vaccine recipients based on the median Month 13 PRNT50 titer. The added text is shown below:

“Among CYD14 and CYD15 9–16-year-old vaccine recipients, VE estimates against DENV-Any at the median Month 13 average PRNT50 titer of 392 were 77% for the baseline-seropositive subgroup and 68% for the baseline-seronegative subgroup; at a Month 13 average MN titer of the same value (392), VE estimates were 82% for the baseline-seropositive subgroup and 70% for the baseline-seronegative subgroup (Fig 7). Thus, stratification by baseline serostatus of VE estimates by Month 13 titer in CYD14 and CYD15 9–16-year-old vaccine recipients yields relatively similar results for baseline-seropositive vs. baseline-seronegative subgroups, regardless of which assay is used to measure Month 13 titer.” (lines 456-463)

- Part B Table S6: the model shows some very strong significant OR >>>1. (e.g. 52, 79, 830). What does this mean? Some of the effects seem to be for interaction terms? It is very difficult to interpret what this means without a description of what the terms are.

Response: Thank you for the suggestion to provide descriptions of what the terms are; we agree such descriptions would be helpful to the reader. We have provided the following footnotes to S6 Table (S5 Table in the revision):

aEstimates for interaction terms (indicated by notation X:Y) in the Odds Ratio column are ratios of odds ratios.

bAge.12.16 is the indicator of 12-16 years old compared to the reference category 2-5 years old.

cM13.PRNT.S1 is Month 13 PRNT50 DENV-1 titer, with similar notation for each neutralization assay and serotype.

dSero2.rate is the fraction of placebo group VCD endpoints that are of serotype 2, with similar notation for the other serotypes.

eM13.MN.Ave is Month 13 MN average titer to the 4 serotypes.

Discussion:

- Paragraph, 470: “It is unclear why nAb titer readouts did not perform equally well across serotypes as CoRs for their matched-serotype VCD endpoints.” Is this paragraph referring to Table 1? I thought the non-significant effects were DENV3 and DENV4? This paragraph mentions DENV1 and DENV3?

Response: We have substantially revised this paragraph, as detailed below (underlined text):

“While in CYD15 all titer readouts (serotype-specific and average) were significant CoRs for their corresponding VCD endpoints (matched-serotype VCD and DENV-Any VCD, respectively), it is unclear why some of the serotype-specific nAb titer readouts were stronger CoRs for their respective matched-serotype VCD endpoints than other serotype-specific nAb titer readouts were for their respective matched-serotype VCD endpoints. For example, for both assays, DENV-1 and DENV-3 titers tended to be less strong CoRs (albeit still significant CoRs) for serotype-matched VCD than DENV-2 and DENV-4 titers [PRNT50 – DENV-1: OR per log10 increment (95% CI) 0.31 (0.23, 0.42); DENV-3: 0.41 (0.27, 0.64) vs DENV-2: 0.18 (0.12, 0.27); 0.22 (0.09, 0.51). MN – DENV-1: 0.23 (0.15, 0.33); DENV-3: 0.23 (0.13, 0.41) vs DENV-2: 0.14 (0.08, 0.26); DENV-4: 0.08 (0.03, 0.19)] (Table 1). Differences between in vitro systems for assessing antibody-mediated DENV neutralization, which use cultured cell lines and laboratory DENV strains, versus neutralization of circulating DENV viral variants in the human body, may be relevant. For instance, the PRNT50 and MN assays assess neutralization of only one DENV strain per serotype. If participants are exposed to circulating viral variants that are neutralized less well (or better) than the assayed strain, the obtained titer for that serotype will be less representative of how well nAbs in that participant’s serum neutralize exposing viral variants. While it is generally assumed that nAb binding epitopes are conserved within serotypes, there is evidence supporting significant variation in neutralization across genotypes of a given serotype, particularly for DENV-1 and DENV-3 [38-40]. We speculate that a scenario in which contemporaneously circulating DENV-1 and DENV-3 strains are neutralized less well (or better) than the DENV-1 PUO-359 (isolated in 1980 in Thailand) and DENV-3 PaH881/88 (isolated in 1988 in Thailand) strains used in the PRNT50 and MN assays (i.e. the parental DENVs of the respective recombinant vaccine viruses) could explain why DENV-1 and DENV-3 titers tended to be weaker (yet still significant) CoRs versus DENV-2 and DENV-4 titers.

The proportion of mismatched amino acid residues between the vaccine DENV inserts and the DENV sequences isolated from placebo group cases provides an assessment of the degree of match between circulating viral variants at the time of the trial and the vaccine strains, and may also be relevant to explain potential differences across serotypes in the strength of serotype-specific CoRs of matched-serotype VCD. We have previously analyzed these proportions by serotype and shown that, in CYD15, DENV-1 circulating strains were farthest from the DENV-1 vaccine insert, followed by DENV-2 circulating strains to the DENV-2 insert, DENV-3 circulating strains to the DENV-3 insert, and then DENV-4 circulating strains found to be closest to the DENV-4 vaccine insert [41]. The latter finding is consistent with DENV-4 titer being the strongest serotype-specific CoR of matched-serotype VCD in CYD15 (Table 1); however, these findings do not fully explain why DENV-2 titer tended to be a stronger CoR of DENV-2 VCD than DENV-1 titer was of DENV-1 VCD in CYD15. ” (lines 572-608)

- Line 517: "We next consider why individual-level classification accuracy using the different variable input." Is this referring to the super-learner algorithm?

Response: Yes. We have edited the sentence as follows (added text is underlined):

“We next consider why individual-level classification accuracy, using the best model identified by Super-Learner for each of four different covariate groups, was relatively limited.” (lines 643-644)

Minor comments:

- Line 60: "Estimated VE against these two endpoints was negative in baseline-seronegative individuals”. By negative, you mean had a negative vaccine efficacy, meaning worse off? This is unclear as written.

Response: Yes, by “negative estimated VE”, we mean “negative vaccine efficacy”. We have added the following text to the revision (underlined):

“Subsequent analyses of VE by baseline dengue serostatus showed high estimated VE against hospitalized VCD and against severe VCD over 60 months in baseline-seropositive individuals; however, estimated VE against these two endpoints was negative in baseline-seronegative individuals (i.e., vaccinated baseline-seronegative individuals were at higher risk of these two endpoints compared to unvaccinated baseline-seronegative individuals) [9].” (lines 66-68)

- Line 156: Please write out CV-AUC (cross validated area under the curve) the first time it appears.

Response: We have done this.

- "Lack of Month 13 sero-response at Month 13 approached zero for both assays (≤ 2%) for DENV-3 and DENV-4 (P>0.05 for both), but significant discordance remained between the MN and PRNT50 results for DENV-1 and DENV-2 (P<0.01 for both) (S1 Fig, panel C)." This sentence is confusing as written.

Response: We agree this sentence could be written more clearly and have modified the text as follows:

“This pattern continued for the placebo group when Month 13 titers were assayed, i.e. higher percentages of participants tested Month 13-seronegative by the MN assay compared to the PRNT50 assay, across all five titer measurements (all P<0.01; S1 Fig, panel B). Similar results were obtained in the vaccine group for Month 13 DENV-1 and DENV-2 titers in that significantly greater percentages of participants tested Month 13-seronegative for DENV-1 and for DENV-2 by the MN assay compared to the PRNT50 assay (P<0.01 for both; S1 Fig, panel C). However, no significant difference in Month 13-seronegativity rates between the two assays was seen in the vaccine group for average titer, DENV-3, or DENV-4 (P>0.05 for all), with seronegativity rates ≤ 2% across the two assays (S1 Fig, panel C).” (lines 248-257)

- Line 232-234: "As one log10 increase in PRNT50 titer approximately equaled one log10 increase in MN titer (Figure 1), we used OR for comparing the two nAb readouts as CoRs.” Why not directly quantify this relationship? Why say approximately one log10 increase here?

Response: Thank you for pointing out that this sentence is potentially confusing as written. As the information in the sentence is not vital to understanding or interpreting the results, we have opted to delete this sentence from the revised manuscript.

- Fig. S2: the cumulative endpoint rate for Fig S2 C and D are quite different (DENV2): 0.05 for the PRNT vs. 0.014 for MN assay? Other y-axes of the PRNT50 vs. MN panels in this figure are more consistent.

Response: We first note that Fig S2 has been moved to the main text and is now Fig 6 in the revised manuscript. As we mentioned in the reviewer’s second comment under “Results” (i.e. beginning with “Line 351-352: “the DENV-2 and DENV-4 correlates of risk…”, there were very small numbers of DENV-2 cases among vaccine-recipients with no Month 13 DENV-2 PRNT50 seroresponse (0 DENV-2 cases among the 87 vaccine-recipients with no Month 13 PRNT50 DENV-2 seroresponse in CYD14 and 5 DENV-2 cases among the 264 vaccine-recipients with no Month 13 PRNT50 DENV-2 seroresponse in CYD15, with “no Month 13 PRNT50 DENV-2 seroresponse” defined as Month 13 PRNT50 DENV-2 titer < 10). Likewise, there were very small numbers of DENV-2 cases among vaccine-recipients with no Month 13 DENV-2 MN seroresponse (3 DENV-2 cases among the 739 vaccine-recipients with no Month 13 MN DENV-2 seroresponse in CYD14 and 34 DENV-2 cases among the 2774 vaccine-recipients with no Month 13 MN DENV-2 seroresponse in CYD15, with “no Month 13 MN DENV-2 seroresponse” defined as Month 13 MN DENV-2 titer < 10). These small numbers of DENV-2 cases imply that the estimated risks in the left tail of the curves shown in Fig 6, panels C and D, have wide confidence intervals.

- Fig. 7: no color is shown in the legend for for D2 model panel A. I assume it should be blue?

Response: Thank you for pointing out this inadvertent oversight. Yes, the color should be blue. This has been added to the legend in panel A of Fig 7 (Fig 8 in the revised version of the manuscript).

References

1. Rossenkhan R, Rolland M, Labuschagne JPL, et al.; Combining Viral Genetics and Statistical Modeling to Improve HIV-1 Time-of-infection Estimation towards Enhanced Vaccine Efficacy Assessment. Viruses 2019;11(7). doi: 10.3390/v11070607.

2. Magaret CA, Benkeser DC, Williamson BD, et al.; Prediction of VRC01 neutralization sensitivity by HIV-1 gp160 sequence features. PLoS Comput Biol 2019;15(4):e1006952. doi: 10.1371/journal.pcbi.1006952.

3. Gilbert PB, Luedtke AR; Statistical Learning Methods to Determine Immune Correlates of Herpes Zoster in Vaccine Efficacy Trials. J Infect Dis 2018;218(suppl_2):S99-S101. doi: 10.1093/infdis/jiy421.

4. Neidich SD, Fong Y, Li SS, et al.; Antibody Fc effector functions and IgG3 associate with decreased HIV-1 risk. J Clin Invest 2019;129(11):4838-4849. doi: 10.1172/JCI126391.

5. Sridhar S, Luedtke A, Langevin E, et al.; Effect of Dengue Serostatus on Dengue Vaccine Safety and Efficacy. N Engl J Med 2018;379(4):327-340. doi: 10.1056/NEJMoa1800820.

6. Price BL, Gilbert PB, van der Laan MJ; Estimation of the optimal surrogate based on a randomized trial. Biometrics 2018;74(4):1271-1281. doi: 10.1111/biom.12879.

7. Frangakis CE, Rubin DB; Principal stratification in causal inference. Biometrics 2002;58(1):21-9. doi: 10.1111/j.0006-341x.2002.00021.x.

8. Moodie Z, Juraska M, Huang Y, et al.; Neutralizing Antibody Correlates Analysis of Tetravalent Dengue Vaccine Efficacy Trials in Asia and Latin America. J Infect Dis 2018;217(5):742-753. doi: 10.1093/infdis/jix609.

Submitted filename: MN Correlates Response to Reviewers 11-Apr-2020.docx

Click here for additional data file.

22 May 2020

Microneutralization Assay Titer Correlates Analysis in Two Phase 3 Trials of the CYD-TDV Tetravalent Dengue Vaccine in Asia and Latin America

PONE-D-19-30338R1

Dear Dr. Gilbert,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Ray Borrow, Ph.D., FRCPath

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: No

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have responded to the questions raised by reviewers in a thoughtful and methodical manner. I have no further concerns.

Reviewer #3: The authors addressed my previous set of statistical questions and clarifications with satisfaction; I have no further comments.

Reviewer #4: The authors have been comprehensive in responding to my review of their manuscript. I have no further comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Anuja Mathew

Reviewer #3: No

Reviewer #4: No

2 Jun 2020

PONE-D-19-30338R1

Microneutralization Assay Titer Correlates Analysis in Two Phase 3 Trials of the CYD-TDV Tetravalent Dengue Vaccine in Asia and Latin America

Dear Dr. Gilbert:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Ray Borrow

Academic Editor

PLOS ONE