Creation, characterization, and assignment of opsonic values for a new pneumococcal OPA calibration serum panel (Ewha QC sera panel A) for 13 serotypes.

Pneumococcal conjugate vaccines (PCVs) have been very effective in reducing the disease burden caused by Streptococcus pneumoniae serotypes covered by the current vaccine formulations. However, the incidence of disease caused by serotypes not covered by the vaccine is increasing. Consequently, there are active efforts to develop new PCVs with additional serotypes in order to provide protection against the emergent serotypes. Due to costs and ethical issues associated with performing true vaccine efficacy studies, new PCVs are being licensed based on their immunogenicity, which may be assessed with 2 in vitro assays: enzyme-linked immunosorbent assay (ELISA) for quantitating antibody level and opsonophagocytic assay (OPA) for assessing protective function. While a standardized ELISA has been developed, OPA results from different laboratories can be quite disparate, even among laboratories utilizing the same platform. In order to harmonize OPA data, a recent international collaboration assigned opsonic indices to the US Food and Drug Administration (US FDA) reference serum, 007sp, as well as a panel of US FDA calibration sera. However, due to a low number of aliquots, the availability of these calibration sera is extremely limited. Because calibration sera are critical to establish the performance characteristics of an OPA, a second calibration serum panel was created, comprised of 20 sera collected from adults immunized with the 23-valent polysaccharide vaccine, with 150 to 500 aliquots prepared for each serum. In order to establish consensus OPA values of the 20 sera for the 13 serotypes in 13-valent PCV, the sera were tested by 4 laboratories in an international collaborative OPA study. The 007sp results of 1 laboratory deviated significantly from those obtained by the other laboratories, as well as from previously assigned values. Due to these discrepancies, the consensus values for the calibration sera were determined based on the data from the remaining laboratories. Thus, we were able to create a panel of sera with consensus opsonic values that could be used by outside laboratories to calibrate pneumococcal OPAs. Our results also confirmed findings of a previous study that normalization of OPA results significantly reduces interlaboratory variation, with normalization based on 007sp reducing variation by 43% to 74%, depending on serotype.

Introduction

Streptococcus pneumoniae is a gram positive, commensal bacterial species capable of causing serious diseases in humans, especially those younger than 2 and older than 65 years of age. Pneumococcal conjugate vaccines (PCVs), prepared by conjugating capsular polysaccharide (PS) to carrier proteins, have been quite effective in reducing the incidence of invasive pneumococcal diseases (IPDs) caused by the vaccine serotypes in both children and old adults.[ With the use of PCVs, however, the incidence of IPDs caused by nonvaccine serotypes has significantly increased,[ spurring the development of new PCVs with additional serotypes.

Due to the overall low incidence of IPD, efficacy trials for new PCVs would require impractically large clinical trials, and in many cases would not be ethically possible due to current PCV usage. Thus, efficacy of the newly formulated PCVs is estimated by quantitating antibodies against capsular PS using 2 in vitro immunoassays: enzyme-linked immunosorbent assay (ELISA) and opsonophagocytic assay (OPA). ELISA has been used extensively to study immune responses to PCVs, especially among pediatric populations.[ However, old adults tend to have high antibody concentrations prior to vaccination,[ and ELISA results failed to predict clinical protection for some serotypes.[ Since the OPA mimics the in vivo mechanism of protection against pneumococcal infections, OPA results are better surrogates of immune protection. Therefore, OPAs are needed for vaccine evaluations.

Early OPAs were labor intensive and therefore not suitable for evaluating large numbers of sera from clinical trials. However, many laboratories, both academic and industrial, have now developed high throughput OPAs.[ Furthermore, although OPA results from different laboratories vary significantly and are therefore difficult to compare,[ a recent study suggested that normalization of OPA results from different laboratories with a reference serum (“007sp”) significantly reduced the interlaboratory variation.[ While the reference serum, 007sp, is readily available, the calibration sera in the panel characterized in that study are limited in quantity and are not generally available. The goals of the current study were to produce a new set of calibration sera with consensus values for a general use and to confirm the benefit of normalization previously reported.

Methods

Laboratories

The laboratories participating in this study are listed alphabetically in Table 1. This order does not reflect the anonymized laboratory letter designations used throughout this report.

Table 1

Participating laboratories and opsonophagocytic assay formats.

Sera

The preparation of pneumococcal reference serum 007sp has been described previously.[

To create the Korea OPA Calibration Serum Panel A, 63 individuals were evaluated at the Ewha Center for Vaccine Evaluation and Study, Ewha Womans University College of Medicine after written informed consent. Twenty healthy male and nonpregnant female volunteers between 20 and 50 years of age met the eligibility requirements for this study. Eligibility was determined by a physical assessment and a questionnaire concerning medical history and risk factors associated with exposure to, or clinical evidence of, a relevant transfusion-transmitted infection. Participants were negative for hepatitis B, hepatitis C, and HIV. The study was approved by the Ewha Womans University Mokdong Hospital institutional review board (EUMC 2015-01-062-001). Fifteen volunteers were vaccinated once with 23-valent pneumococcal polysaccharide vaccine (PPSV23) (Prodiax23, Merck & Co. Inc., Whitehouse Station, NJ) by intramuscular injection, and donated a unit of blood 14 to 27 days following immunization and a second unit of blood 8 to 12 weeks after the first donation (see Table 2). Five volunteers who were vaccinated previously (46–50 months prior) with PPSV23 (Prodiax23) donated a unit of blood, a second unit of blood 8 to 12 weeks later, and in some instances a third unit of blood 8 to 12 weeks after the second donation (Table 2). Blood was allowed to clot and the serum was collected and stored at −80°C at the Ewha Center for Vaccine Evaluation and Study. For each donor, the sera from the 2 blood donations were thawed, pooled, and 1-mL aliquots were prepared (153–534 vials were prepared for each of the 20 sera). The aliquots were lyophilized by LG Life Sciences R&D (Daejeon, Republic of Korea) and are stored at ≤−70°C.

Table 2

Donor information.

Study design

Each participating laboratory tested the 20 calibration sera in 3 to 5 separate runs with 007sp included at least once in each run. Each participating laboratory used its own batches/lots of reagents.

OPAs

All participating laboratories utilized the multiplexed OPA format.[ Briefly, target bacteria were thawed, washed with opsonization buffer B (Hanks’ balanced salt solution with 0.1% gelatin and 5% fetal bovine serum), and diluted (to ∼5 × 104 CFU/mL). Ten microliters of diluted bacteria were added to 20 μL of serially diluted sera and assay plates were incubated for 30 min at room temperature with shaking. Baby rabbit complement (final 12.5%) and dimethylformamide-differentiated HL60 cells (4 × 105 cells/well) were added for a total assay volume of 80 μL. Plates were incubated for 45 min at 37°C/5% CO2 with shaking. After incubation, plates were placed on ice for at least 10 to 20 min. A 10-μL aliquot of the final reaction mixture was spotted onto agar plates (Todd–Hewitt broth with 0.5% yeast extract and 1.5% agar). An equal volume of overlay agar (Todd–Hewitt broth with 0.5% yeast extract and 0.75% agar) containing 25 mg/L and the selective antibiotic was added, and the plates were incubated overnight at 37°C/5% CO2. A detailed procedure can be found at https://www.vaccine.uab.edu/UAB-MOPA.pdf (Lab D made minor modifications to the protocol including shaking assay plates at 220 rpm, rather than the speed specified in the protocol, 700 rpm).

After overnight incubation, the number of surviving colonies was determined. Each laboratory converted raw colony counts to opsonic indexes (OIs) using the same MS Excel-based template (“Opsotiter”).

Reference serum 007sp was included once in each run.

Statistical analyses

Statistical analyses were performed as described previously.[ Briefly, calibration sera OIs were normalized using the following formula:

Consensus OIs (unadjusted and normalized) for the calibration sera were estimated for serotype and sample by fitting the log transformed OIs using a mixed-effect analysis of variance model consisting of the random terms Lab and Run (Lab). Consensus OIs and the corresponding 95% confidence intervals (CIs) were obtained by back-transforming the model intercept and its corresponding CI.

For individual serotypes (except serotypes 1 and 23F), the percent reduction in interlaboratory variability due to normalization was calculated as:

with , and defined as the interlaboratory, run-within-laboratory, sample-by-laboratory, and sample-by-run-within-laboratory variance component estimates for the unadjusted OIs, respectively; and , and defined as the corresponding variance components for the normalized OIs.

For serotypes 1 and 23F, the variability actually increased slightly after normalization. For these serotypes, the percent “reduction” was calculated as:

with the same term definitions as above.

Results

OIs obtained for 007sp

The 007sp geometric mean opsonic index (GMOI) obtained by each laboratory is shown in Table 3 and Fig. 1. Generally, the results obtained by Labs A, B, and C were comparable to each other as well as to the assigned values, with the Lab C results trending slightly higher than those of Labs A and B. However, most of the results obtained by Lab D were significantly lower than the other laboratories as well as the assigned values, with the results of multiple serotypes differing by more than 10-fold. The exceptions were serotypes 1 and 5, for which the results from Lab D were within 3-fold and 2-fold, respectively, of the assigned values.

Table 3

007sp geometric mean opsonic indices.

Figure 1

007sp opsonic indices. The 007sp GMOIs obtained by each laboratory (color symbol) and the assigned OIs for 007sp (black horizontal line) are shown for each target serotype. The dashed vertical lines indicate 3-fold deviations from the assigned OI (see Section 4). GMOI = geometric mean opsonic index, OI = opsonic index, Pn = pneumococcal serotype.

Effect of standardization on calibration sera results

For Labs A, B, and C, the unadjusted OIs for the calibration sera agreed reasonably well (the GMOIs for each laboratory are shown in Supplementary Table 1). The coefficients of variation (CVs) for the unadjusted values were <60% for all serotypes except 4 (81%), 6A (96%), 9V (111%), and 23F (64%) with the exclusion of the data from Lab D (Table 4). By contrast, when the data from Lab D was included, the CVs for the unadjusted values ranged from 74% (serotype 1) to 1048% (serotype 4) with the CVs for most serotypes >200% (Table 5).

Table 4

Model-based assessment of the effect of normalization, without Lab D data.

Table 5

Model-based assessment of the effect of normalization with Lab D data.

Without the data from Lab D, normalization resulted in a decrease in variability for all serotypes except 1 and 23F, although in most cases the reductions were fairly minimal (Table 4). Although there was an increase in variability (indicated as a negative reduction in variability) for serotypes 1 (14% increase) and 23F (1% increase), normalization had a minimal impact on the CVs, increasing from 41% to 46%, and 64% to 65% for serotypes 1 and 23F, respectively. The absolute CVs of the normalized results were <60% for all serotypes except serotypes 4 (79%), 6A (81%), 9V (83%), and 23F (65%). With the inclusion of data from Lab D (Table 5), normalization resulted in significant (>30%) decreases in variability for all serotypes except 1 (3%) and 5 (22%), with the absolute CVs for the normalized data ranging from 64% (serotype 5) to 180% (serotype 4).

The effect of standardization is shown graphically in Fig. 2. For each calibration serum, the unadjusted (y-axis, left panels) and adjusted (y-axis, right panels) laboratory-specific GMOI as a function of the consensus OI (x-axis) are presented. To improve visualization, the results from different laboratories for each serum are connected by a vertical black line. The effect of normalization can be seen by comparing the length of the vertical lines, with and without normalization. For multiple serotypes, the benefit of normalization was most obvious with high OI sera.

Figure 2

Effect of normalization. The GMOIs obtained by each laboratory (GMOIs, y-axis) as a function of the consensus OI (x-axis) is shown for each of the 20 sera. For each serotype, the left panel displays the unadjusted data and the right panel shows the normalized data. Each plot also has a line of identity (dashed line). The consensus OI includes data from Lab D. GMOI = geometric mean opsonic index, OI = opsonic index, Pn = pneumococcal serotype.

Determination of consensus OIs for calibration sera

The unadjusted consensus values for the calibration sera for each of the 13 serotypes are shown in Table 6, and the normalized consensus values are shown in Table 7. Due to the disparity of the Lab D results, the consensus values in both tables were estimated based only on the results from Labs A, B, and C. The red, bold text in both tables indicates that at least 1 laboratory reported an irregular result for that sample for that serotype in at least 1 run.

Table 6

Unadjusted calibration sera consensus OIs (without Lab D).

Table 6 (Continued)

Unadjusted calibration sera consensus OIs (without Lab D).

Discussion

A critical component for a laboratory to establish an OPA is the ability to determine the performance of the assay with a readily available set of calibration sera. While a panel of calibration sera with consensus opsonic values already exists (FDA Calibration Sera[), the number of available vials is extremely limited and the sera are not routinely available. Thus, the first goal of this study was to create a new OPA calibration serum panel that was available to all laboratories. These 20 calibration sera (“Korean OPA Panel A”) can be obtained by contacting Dr. Kyung-Hyo Kim at Ewha Womans University (kaykim@ewha.ac.kr) or Dr. Si Hyung Yoo at Biologics Research Division, Ministry of Food and Drug, Republic of Korea (yoosh1130@korea.kr).

In addition, we report consensus OIs for the 13 serotypes included in 13-valent PCV derived from an international collaboration. Table 6 shows unadjusted consensus OIs and Table 7 shows normalized results. Due to the disparate data from Lab D, only data from Labs A, B, and C were used to estimate the consensus values shown in each table (the data from all 4 laboratories can be found in Supplemental Table 1). Although the removal of data from Lab D reduced the number of participating laboratories to 3, some previous 007sp assignments for ELISA were also based on studies involving 3 laboratories.[ An analogous calibration serum panel with assigned values already exists for ELISA use, and rules for determining the comparability of a laboratory's ELISA have been developed (https://www.vaccine.uab.edu/qc3.pdf). The limited amount of data in this study precludes the establishment of such criteria for OPAs. Thus, data from additional laboratories will be needed to construct these parameters and further refine the standardization procedure in the future.

At the moment, the basis for the aberrant results from Lab D is not known. However, as noted in Section 2, Lab D utilized a slower shaking speed than that indicated in the protocol. Although we do not know what effect this change on the results, as noted above, this situation does highlight the need to develop rules for normalizing OPA results based on 007sp, including developing an absolute range of 007sp values that can be used for normalization.

Many reference sera developed for other assays (e.g., ELISA) contain preservatives, such as azide, and/or consist of plasma converted to serum, making them not desirable for OPAs. For instance, some anticoagulants chelate calcium, interfering with phagocytic function. The sera in the Korean OPA Panel A were collected with no preservatives, antibiotics, or anticoagulants and the sera were lyophilized for ease in distribution. In an attempt to obtain samples with low OIs, sera were obtained from 5 adults who were vaccinated 46 to 50 months prior. However, the OIs for these sera (QC01–QC05 in Table 6) were not much different than the OIs of the sera collected 1 month after vaccination (QC06–QC20 in Table 6). The consensus OIs for samples with at least 1 laboratory-reported irregular result are indicated in red, bold font in Tables 6 and 7. Due to the variability associated with such irregular curves, the identified samples should not be used to calibrate the indicated serotype (s), but may be used for other serotypes.

The second goal of this study was to confirm the benefit of normalizing pneumococcal OPA results using reference serum 007sp.[ Indeed, normalization of the results from Labs A, B, and C reduced the variability for 10 of the 13 target serotypes, but the reductions were modest for many serotypes (Table 4) largely because the unadjusted results agreed well among the 3 laboratories even before the normalization. When the data from Lab D was included (Table 5), normalization significantly reduced the deviation of Lab D's results from the consensus values similar to the previous study.[ Taken together, our results confirm that normalization would significantly reduce interlaboratory variability.

In the previous study,[ the absolute 007sp results obtained by the 6 individual laboratories were relatively comparable, with most values differing from the assigned values by <3-fold. In the current study, 007sp results for Labs A, B, and C were also within 3-fold of the assigned values, but the 007sp results from Lab D differed by more than 3-fold for most of the serotypes tested. In fact, the 007sp results for all serotypes were lower than the assigned values, indicating the OPA for Lab D is less sensitive than that of the other laboratories. Although normalization reduced the variability between the 4 laboratories (see Table 5), the absolute variability remained high for many serotypes after normalization, with most CVs >100% (Table 5). Based on these results and the results of the previous study, we tentatively propose that a laboratory's absolute 007sp result for a serotype must be within 3-fold of the assigned value (indicating an assay sensitivity comparable to others) to be used for normalizing OPA data.

It is worth noting that no criteria for OPA sensitivity could be developed until 007sp with assigned values became available. However, to reap the full benefits of OPA standardization, additional operation rules for implementation of standardization still need to be developed. For example, the optimum number of 007sp results used to normalize a sample must be determined. In this study as well as the previous study, normalization was based on a single 007sp result within a run. If the 007sp result was incorrect due to random or technical errors, results of the entire run would be affected. Thus, we believe that 007sp should be, in the future, analyzed more than once, perhaps 3 times per run, and the average of the 3 results should be used to normalize the data from the entire run. Also, as mentioned above, parameters to better define a “calibrated” assay must be developed.

In summary, we have created and characterized a panel of sera that can be used to determine the comparability of a laboratory's OPA results to the results from other laboratories. Now, with this newly available calibration serum panel, individual laboratories can better characterize and standardize their OPAs, making the assay an even more powerful tool in vaccine evaluation.

Acknowledgments

The authors would like to thank Soo Young Lim and Je Eun Cha (Center for Vaccine Evaluation and Study, Medical Research Institute, Ewha Womans University College of Medicine) for laboratory support.

Author contributions

Conceptualization: Robert L Burton, Moon H Nahm, Kyung-Hyo Kim.

Data curation: Robert L Burton, Han Wool Kim, Soyoung Lee, Jee-Hyun Seok, Anne Balloch, Paul Licciardi, Rachel Marimla, Sejong Bae, Kyung-Hyo Kim.

Formal analysis: Robert L Burton, Sejong Bae, Kyung-Hyo Kim.

Funding acquisition: Moon H Nahm, Kyung-Hyo Kim.

Investigation: Robert L Burton, Han Wool Kim, Soyoung Lee, Jee-Hyun Seok, Sang Heon Lee, Anne Balloch, Paul Licciardi, Rachel Marimla, Sejong Bae.

Methodology: Robert L Burton, Sejong Bae, Moon H Nahm, Kyung-Hyo Kim.

Project administration: Robert L Burton, Kyung-Hyo Kim.

Resources: Robert L Burton, Han Wool Kim, Soyoung Lee, Hun Kim, Sang Heon Lee, Anne Balloch, Moon H Nahm, Kyung-Hyo Kim.

Software: Robert L Burton, Sejong Bae, Moon H Nahm.

Supervision: Robert L Burton, Moon H Nahm, Kyung-Hyo Kim.

Validation: Robert L Burton, Sejong Bae, Moon H Nahm.

Visualization: Robert L Burton, Sejong Bae.

Writing – original draft: Robert L Burton, Sejong Bae, Moon H Nahm, Kyung-Hyo Kim.

Writing – review & editing: Robert L Burton, Han Wool Kim, Soyoung Lee, Hun Kim, Jee-Hyun Seok, Sang Heon Lee, Anne Balloch, Paul Licciardi, Rachel Marimla, Sejong Bae, Moon H Nahm, Kyung-Hyo Kim.

Table 7

Normalized calibration sera consensus OIs (without Lab D).

Table 7 (Continued)

Normalized calibration sera consensus OIs (without Lab D).