Assessment of Quadrivalent Human Papillomavirus Vaccine Safety Using the Self-Controlled Tree-Temporal Scan Statistic Signal-Detection Method in the Sentinel System.

The self-controlled tree-temporal scan statistic-a new signal-detection method-can evaluate whether any of a wide variety of health outcomes are temporally associated with receipt of a specific vaccine, while adjusting for multiple testing. Neither health outcomes nor postvaccination potential periods of increased risk need be prespecified. Using US medical claims data in the Food and Drug Administration's Sentinel system, we employed the method to evaluate adverse events occurring after receipt of quadrivalent human papillomavirus vaccine (4vHPV). Incident outcomes recorded in emergency department or inpatient settings within 56 days after first doses of 4vHPV received by 9- through 26.9-year-olds in 2006-2014 were identified using International Classification of Diseases, Ninth Revision, diagnosis codes and analyzed by pairing the new method with a standard hierarchical classification of diagnoses. On scanning diagnoses of 1.9 million 4vHPV recipients, 2 statistically significant categories of adverse events were found: cellulitis on days 2-3 after vaccination and "other complications of surgical and medical procedures" on days 1-3 after vaccination. Cellulitis is a known adverse event. Clinically informed investigation of electronic claims records of the patients with "other complications" did not suggest any previously unknown vaccine safety problem. Considering that thousands of potential short-term adverse events and hundreds of potential risk intervals were evaluated, these findings add significantly to the growing safety record of 4vHPV.

Despite the cancer-preventing promise of human papillomavirus (HPV) vaccines and national recommendations for routine HPV vaccination of females and males at age 11–12 years, HPV vaccine coverage in the United States lags behind that of other adolescent vaccines (1) more than a decade after the first HPV vaccine was licensed. One reason for this is persistent concern about the safety of HPV vaccines on the part of parents and the public (2–4). The scientific literature generally does not bear out these worries. A substantial body of published evidence has accumulated regarding the safety of quadrivalent Gardasil (Merck & Company, Inc., Whitehouse Station, New Jersey) (5–15), which constituted 93% of all HPV vaccine doses distributed in the United States through September 2015 (16), with no confirmed safety problems identified to date, other than syncope and skin infections (5). However, most published studies addressed prespecified outcomes about which theoretical or empirically based concerns had been raised, such as autoimmune diseases, venous thromboembolism, and neurological disease (6–15). More open-ended studies addressing HPV vaccine safety more generally, without prespecifying outcomes of concern, have been fewer and have been somewhat limited in sample size (5) or have tended to rely on spontaneous reports (17, 18), the interpretability of which is hampered by lack of control groups and denominators, underreporting, and reporting biases (19–21).

We studied the safety of Gardasil, hereafter called quadrivalent human papillomavirus vaccine (4vHPV), applying a self-controlled tree-temporal scan statistic data-mining method (22–24) to health insurance claims data for signal detection. The method allows a wide variety of unsuspected but potential adverse reactions and a range of potential postvaccination periods of increased risk (“risk windows”) to be simultaneously evaluated, adjusting for the multiple testing involved.

METHODS

Study population, enrollment criteria, and exposure

We conducted the study within the Sentinel system, which was launched by the US Food and Drug Administration in 2009 to conduct postlicensure assessments of the safety of medical products, using a distributed-data-processing approach (25–27). We used claims data from 5 Sentinel Data Partners, including 4 large national health insurance companies. People receiving their first dose of 4vHPV at 9–26.9 years of age within the period June 1, 2006–December 31, 2014 were eligible for inclusion in the study population. To be included, a health plan participant had to be enrolled from 183 days prior to the first 4vHPV dose through 56 days after the first 4vHPV dose. Any apparent enrollment gaps of 45 days or less were treated as continuously enrolled time. 4vHPV vaccination was identified using Current Procedural Terminology code 90649.

Hierarchical diagnosis tree

Outcomes were identified using International Classification of Diseases, Ninth Revision (ICD-9) codes and a classification of ICD-9 codes into a hierarchical tree structure defined by the Multi-Level Clinical Classifications Software (MLCCS). The MLCCS is a product of the Agency for Healthcare Research and Quality’s Healthcare Cost and Utilization Project (28). The tree has 5 diagnosis levels, although some branches extend only to the second or third level. The first and broadest level identifies 18 broad categories of diagnoses, while the subsequent levels become more and more specific, ending with ICD-9 codes at the finest, “leaf” level. Table 1 presents the classification of “convulsions” as an example.

Table 1.

Example of the Hierarchical Classification Scheme of the Agency for Healthcare Research and Quality’s Multi-Level Clinical Classifications Softwarea

Code	Text Description
06	Diseases of the nervous system and sense organs
06.04	Epilepsy; convulsions
06.04.02	Convulsionsb
780.3	Convulsions
780.31	Febrile convulsions
780.32	Complex febrile convulsions
780.33	Posttraumatic seizures
780.39	Other convulsions

a Reprinted from the final project report to the Food and Drug Administration (39), with permission from Harvard Pilgrim Health Care, Inc. (©2016).

b “Convulsions” is a third-level classification without a fourth level.

ICD-9 codes representing the following conditions were excluded from the tree and therefore from analysis: outcomes very unlikely to be caused by vaccination (e.g., well-care visits, delivery of a baby, vitamin deficiencies, fracture of a lower limb); some conditions unlikely to manifest themselves within the 56-day follow-up period (e.g., cancer); most infectious diseases with an identified organism (e.g., typhoid fever, tuberculosis, shigellosis); congenital conditions (e.g., sickle cell disease, congenital heart disease); and outcomes that are common and of an unspecific or less serious nature (e.g., fever, croup, acute pharyngitis). The resulting “pruned” tree contained 6,551 ICD-9 codes.

Incident diagnoses of interest

The study focused on “incident” diagnoses observed in the inpatient or emergency department setting during the 56-day follow-up period. In defining incidence, we sought to exclude repeated diagnoses due to follow-up visits for an earlier episode of illness; in so doing, however, we would not have captured closely spaced exacerbations of preexisting conditions, which are sometimes of interest as potential vaccine reactions. Incidence was defined and determined on the basis of there being no other diagnosis for the patient in the same third-level branch of the MLCCS diagnosis tree in any setting during the prior 183 days. This means that, even if an event was coded with a never-before-seen ICD-9 code, it was not counted if a different ICD-9 code belonging to the same third-level branch was observed for the individual during the prior 183 days. The third level was chosen for determining incidence in order to avoid double-counting and overestimation of incidence, which could otherwise occur if physicians classified the same episode of illness in 2 slightly different ways (e.g., “convulsions” and “febrile convulsions”) in separate patient visits. We allowed each patient to contribute multiple incident diagnoses during his/her follow-up period, as long as they were not part of the same third-level branch of the MLCCS tree.

Risk and comparison windows

Only health outcomes occurring on days 1–56 after the first apparent dose were included in the analysis. We considered this follow-up period optimal in that it would have included potential adverse reactions occurring (i.e., being diagnosed and coded) up to several weeks after vaccination, while minimizing any time-varying confounding and the likelihood of a second dose of HPV vaccine being received during follow-up. The day of vaccination (day 0) was not included, since 1) a preventive-care visit at which vaccines were given could have generated diagnosis codes for outcomes unrelated to vaccination, such as a health issue noted during the physical examination, and 2) 4vHPV may have been given during a health-care visit that happened due to an illness or other health concern. We evaluated all 665 temporal risk windows that were between 2 and 28 days long, started 1–28 days after vaccination, and ended 2–42 days after vaccination. The comparison period used in the evaluation of each of these potential risk windows consisted of the days within the 56-day follow-up period that were not in the risk window. For example, in evaluating the risk window of days 10–14, the control period was days 1–9 plus days 15–56.

The conditional tree-temporal scan statistic

With the tree-temporal scan statistic, one performs multiple temporal scans, one for each of the many clinical outcomes and groups of related clinical outcomes (i.e., leaves and branches of the tree). At the same time, one evaluates multiple potential risk windows, comparing the number of events within the risk window with what would be expected by chance if they were randomly distributed over time. Under the null hypothesis, there is no unusual clustering of events within any branch or time interval. Under the alternative hypothesis, there is at least 1 branch of the tree for which there is a temporal cluster of events during some time interval. In a conditional analysis, used for the current study, we condition not only on the number of events observed in each node of the tree during the whole follow-up period but also on the total number of events occurring on the first day after vaccination, on the second day after vaccination, etc. This adjusts for the type of temporal confounding that would occur if there were temporal differences in general health-care-seeking behavior shortly after the vaccination date as compared with longer after the vaccination date.

The method adjusts for the multiple testing entailed in evaluating the many branches and time intervals. Each time interval is evaluated on each of the branches, so with our approximately 7,300 nodes (i.e., outcome categories, whether first, second, third, fourth, or fifth level—which include, for example, the codes listed in Table 1) on the tree and our 665 potential time intervals, there were more than 4.8 million potential clusters to evaluate and for which we needed to adjust for multiple testing. With scan statistics, the penalty for adjusting for all of this multiple testing is relatively modest, since many of the potential clusters are highly overlapping with each other. Furthermore, no power is lost (i.e., no α is spent) in scanning nodes where the observed number of events in the follow-up period is less than 2.

To implement the conditional tree-temporal scan statistic, we calculate a Poisson generalized log likelihood ratio test statistic for each tree node and time interval. Let n be the number of events in the node, let c be the number of those node events that are also in the time interval, let z be the number of events in the time interval summed over the whole tree, and let C be the total number of events in the tree. The number of events in the cluster, c, is then contrasted with the expected number of events in the cluster under the null hypothesis, which is u = nz/C. When u > 0, the test statistic is calculated as where I ( ) is the indication function. I (c > u) = 1 when there are more events than expected in the cluster and 0 otherwise, and it is included to ensure that we are looking for an excess risk of having the outcome rather than a protective decreased risk. The node-interval combination with the maximum test statistic is the most likely cluster of events—that is, the cluster that is least likely to have occurred by chance.

The distribution of the test statistic is not known analytically, so there is no simple mathematical formula that can be used to obtain a P value for the detected cluster. To adjust for the multiple testing inherent in the many node-interval combinations considered and evaluate whether the most likely cluster is statistically significant, Monte Carlo hypothesis testing is used. We do this by generating 99,999 random replicates of the data. In each random data set, each node has exactly the same number of events as the real data set, and each day after vaccination has the same number of events when summed over all nodes. The only thing that varies is the pairing of the nodes and times, which is randomized using a permutation approach. The likelihood ratio test statistic from the most likely cluster in the real data set is compared with the likelihood ratio test statistics from the most likely clusters in each of the 99,999 random data sets, and we note its rank. For example, if it has the fifth-highest test statistic, its rank, R, is 5. Note that the most likely cluster will be a different node-interval combination in each of the different data sets, so we are comparing the maxima of the likelihood ratios obtained over all possible node-interval combinations. Since the random data sets were all generated under the null hypothesis, if the null hypothesis is true in the real data set, then the test statistics come from exactly the same probability distribution. This means that, if the null hypothesis is true, the rank test statistic from the real data set will range uniformly from 1 to 100,000, and the probability of having a rank in the top 5% is exactly 5%. If the test statistic from the real data set is in the top 5%, we will reject the null hypothesis at the α = 0.05 level. If the null hypothesis is true, we have a 5% probability of falsely rejecting the null and a 95% probability of not having any alert anywhere on the tree.

The analysis was conducted using the free software TreeScan, available at www.treescan.org (29).

Calculation of excess risk

To calculate excess risks (or attributable risks), we obtained D, the total number of eligible 4vHPV first doses. The attributable risk per 100,000 doses was calculated as (c − u) × 100,000/D.

Statistical alert follow-up

We use the term “alert” to refer to a cluster with P ≤ 0.05. Data related to alerts were frozen after the analysis was conducted. From the frozen data for 1 alert, a “claims profile” was generated for each patient with an incident diagnosis that contributed to the alert, listing all of the procedures, drug dispensings, and diagnoses captured in the claims data during the period from 56 days before 4vHPV vaccination through 84 days after 4vHPV vaccination. The procedures for generating these reports are explained in detail in the publicly available report Infrastructure for Evaluation of Statistical Alerts Arising From Vaccine Safety Data Mining Activities in Mini-Sentinel (30). Two members of the work group, including an internal medicine physician (M.A.B.), reviewed these claims profiles to see the specific diagnosis codes used, concomitant vaccinations, and other clinical information useful in interpreting the alert.

RESULTS

A total of 1,903,697 first doses of 4vHPV vaccine were included in analysis. The analysis results are presented in Table 2. All diagnoses with P < 0.05 are shown, along with some others useful for context. There were no diagnoses with P just slightly greater than 0.05—the lowest P value for diagnoses not included in the table was 0.25. There were 2 sets of alerts, described below.

Table 2.

Details of Statistical Alerts From a Tree-Temporal Scan Statistical Analysis of Quadrivalent Human Papillomavirus Vaccination Among Persons Aged 9–26.9 Years, United States, 2006–2014a

Row	Node Code	Node Text	Risk Windowb	Observed No. of Eventsc	Attributable Riskd	P Value
1	12	Diseases of the skin and subcutaneous tissue	2–4	214	3.8	0.002
2	12.01	Skin and subcutaneous tissue infections	2–4	111	2.3	0.042
3	12.01.01	Cellulitis and abscesse	2–4	93	2.0	0.204
4	12.01.01.03	Cellulitis and abscess of arm (contains code 682.3 only)f	2–3	31	1.3	0.00001
5	682.3	Cellulitis and abscess of upper arm and forearm	2–3	31	1.3	0.00001
6	12.02	Other inflammatory condition of skine,g
7	695.9	Unspecified erythematous conditione	2–3	13	0.5	0.246
8	16	Injury and poisoning	1–3	48	2.2	0.00001
9	16.10	Complications	1–3	36	1.8	0.00001
10	16.10.02	Complications of surgical procedures or medical care	1–3	36	1.8	0.00001
11	16.10.02.07	Other complications of surgical and medical proceduresf	1–3	36	1.8	0.00001
12	780.63	Postvaccination fevere	1–2	4	0.2	0.306
13	999.0	Generalized vacciniae	1–3	3	0.2	>0.99
14	999.4	Anaphylactic reaction due to serume,g
15	999.42	Anaphylactic reaction due to vaccinatione,g
16	999.5	Other serum reaction not elsewhere classified	1–3	7	0.4	0.011
17	999.52	Other serum reaction due to vaccination	1–2	11	0.6	0.00001
18	999.59	Other serum reactione,g
19	999.9	Other and unspecified complications of medical care	1–6	12	0.6	0.002

a Adapted from the final project report to the Food and Drug Administration (39), with permission from Harvard Pilgrim Health Care, Inc. (©2016).

b Specific days after vaccination (e.g., “2–4” means days 2–4 after vaccination).

c Number of events observed in risk window.

d Number of excess cases per 100,000 first vaccine doses.

e Some related diagnoses for which there were no alerts are included for context.

f All International Classification of Diseases, Ninth Revision, codes in node 12.01.01.03 (just 1 code) and node 16.10.02.07 for which there were any events during days 1–56 are listed.

g Blank cells indicate that P = 1 for the respective diagnosis.

Cellulitis and abscess of arm (node 12.01.01.03)

Within “diseases of the skin and subcutaneous tissue,” there were alerts at 4 levels (Table 2, rows 1, 2, 4, and 5). The highest statistical significance was seen at the fourth and fifth levels (rows 4 and 5), for “cellulitis and abscess of arm,” with a risk window of days 2–3 postvaccination, 31 cases, an attributable risk of 1.3 per 100,000 first doses administered, and a P value of 0.00001. (ICD-9 code 682.3 is the only one stemming from node 12.01.01.03, so the results are identical for rows 4 and 5.) Considering especially the statistical significance, these 31 cases appear to be driving the broader “diseases of the skin and subcutaneous tissue” alert (row 1). The 13 cases with ICD-9 code 695.9, “unspecified erythematous condition” (in row 7, within node 12.02, “other inflammatory condition of skin”), on days 2–3, contributed to the broader “diseases of the skin and subcutaneous tissue” alert, too, although there was no alert for the 695.9 code.

Because cellulitis is a known adverse reaction to 4vHPV vaccination and is listed as such in the package insert (31), no further investigation was conducted.

Other complications of surgical and medical procedures (node 16.10.02.07)

There were alerts at 5 levels within “injury and poisoning,” with risk windows all within 6 days after vaccination (Table 2, rows 8–11, 16, 17, and 19). At the first level (row 8), there was a risk window of days 1–3 postvaccination, 48 cases, an attributable risk of 2.2 per 100,000 first doses, and a P value of 0.00001. The second through fourth levels had the same risk window (days 1–3), 36 cases, an attributable risk of 1.8 per 100,000 first doses, and a P value of 0.00001. (The results for rows 9–11 are identical, because there is no branching between nodes 16.10 and 16.10.02.07, due to branches such as “complication of device, implant, or graft” and “postoperative infection” in the full MLCCS tree having been excluded from the “pruned” tree that we used.) These 36 cases in “other complications of surgical and medical procedures” appear to be driving the broader “injury and poisoning” alert, as there are no alerts in other included “injury and poisoning” second-level branches, namely “poisoning” or “other injuries and conditions due to external causes” (not shown in table). There were 3 alerts for specific ICD-9 codes within “other complications of surgical and medical procedures” (rows 16, 17, and 19).

Fifty-eight patients had incident diagnoses at the “other complications of surgical and medical procedures” node (Table 3)—36 with their diagnoses in the days 1–3 risk window (as also shown in Table 2, row 11) and 22 with their diagnoses during the days 4–56 control window (Table 3). Thirty-one (86%) of the 36 cases in the risk window and 11 (50%) of the 22 cases in the control window had a code indicating receipt of at least 1 other vaccine on the same day as 4vHPV. Concomitant vaccines included tetanus-diphtheria-acellular pertussis, meningococcal conjugate, varicella, pneumococcal conjugate, hepatitis A, hepatitis B, inactivated influenza, live attenuated influenza, rabies, typhoid, poliomyelitis, and meningococcal polysaccharide vaccines. Nine (25%) of the 36 cases in the risk window and 6 (27%) of the 22 cases outside of the risk window had a claim for a subsequent dose of 4vHPV vaccine within 84 days of the first.

Table 3.

Distribution of the 58 Adverse Event Cases in the Category “Other Complications of Surgical and Medical Procedures” Among Persons Aged 9–26.9 Years, by Diagnosis Code and Timing After Quadrivalent Human Papillomavirus Vaccination, United States, 2006–2014a

ICD-9 Code	ICD-9 Description	Time Windowb
		Days 1–3	Days 4–56	Total
780.63	Postvaccination fever	4	1	5
999.0	Generalized vaccinia	3	0	3
999.4	Anaphylactic shock due to serum	0	1	1
999.42	Anaphylactic reaction due to vaccination	2	3	5
999.5	Serum reaction not elsewhere classified	7	3	10
999.52	Other serum reaction due to vaccination	11	2	13
999.59	Other serum reaction	2	3	5
999.9	Other and unspecified complications of medical care, not elsewhere classified	7	9	16
Total		36	22	58

Abbreviation: ICD-9, International Classification of Diseases, Ninth Revision.

a Reprinted from the final project report to the Food and Drug Administration (39), with permission from Harvard Pilgrim Health Care, Inc. (©2016).

b Specific days after vaccination (e.g., “1–3” means days 1–3 after vaccination).

Focusing on the 36 cases whose incident diagnosis code fell into the days 1–3 risk window and which therefore contributed to the alert, there were 4, 3, and 2 cases of the specific incident diagnoses of postvaccination fever, generalized vaccinia, and anaphylaxis, respectively (Table 3). Although these 9 cases contributed to the “other complications of surgical and medical procedures” alert, none of these specific diagnoses was associated with an alert of its own (Table 2, rows 12–15). All 4 cases of postvaccination fever (ICD-9 code 780.63) had claims for 1 or more additional vaccines on day 0. Of the 3 cases with the generalized vaccinia ICD-9 code 999.0 as the incident diagnosis, 2 had codes for pain in or swelling of the limb and 1 had codes for allergic urticaria and unspecified urticaria; no additional diagnosis or symptom codes were present for any of the 3 cases. These 3 “vaccinia” patients had claims for at least 2 additional vaccines on day 0, including varicella in all 3 cases (relevant because generalized varicella-like rash, which could plausibly be incorrectly coded as “generalized vaccinia,” has been documented on days 0–23 after varicella dose 2 in adolescents and adults (32)). Regarding the 2 cases of anaphylaxis (ICD-9 code 999.42), it is unclear whether either case was truly anaphylaxis related to 4vHPV vaccination—neither case had claims for epinephrine, and 1 of the patients had received meningococcal conjugate vaccine on the same day as 4vHPV.

There were 27 cases in the risk window with a nonspecific incident diagnosis code (ICD-9 codes 999.5, 999.52, 999.59, and 999.9; Table 3). From the patients’ claims profiles, we determined that, of these 27 cases, 8 had codes for pain in and/or swelling of the limb; 5 had codes for local skin reactions and/or unspecific allergic reactions; 1 had a code for cellulitis; and 6 had codes for somewhat diffuse conditions of nausea and/or vomiting, fever, viral exanthem, dizziness and giddiness, headache, and/or unspecified myalgia and myositis, with few or no subsequent medical visits apparent in the claims profile. An additional 3 cases had unspecified symptoms, and in all 3 cases, the next coded visit did not take place until at least 60 days later. The remaining 4 cases had claims for a variety of medical conditions, with no apparent similarity among them.

DISCUSSION

In our TreeScan analyses of more than 1.9 million recipients of 4vHPV dose 1, we found 2 categories of adverse events within 42 days of vaccination. One was “cellulitis and abscess of the arm.” Cellulitis is listed as an adverse event in the 4vHPV package insert (31) and was not investigated further. The other adverse event category was “other complications of surgical and medical procedures.” Based on the claims data, the clinical characteristics of 29 (81%) of the 36 cases contributing to that alert (the 9 cases with specific incident diagnoses and 20 of the cases with nonspecific incident diagnoses) appeared to conform to what was already known about 4vHPV adverse events (27) (although 4vHPV was not necessarily the cause—most of the 36 patients received 1 or more other vaccines along with 4vHPV). Three cases (8%) with nonspecific incident diagnosis codes and no symptoms specified in the claims profiles had no subsequent coded visits for medical care until at least 60 days later, suggesting that the respective conditions did not require medical follow-up. The other 4 cases (11%) had claims for diverse symptoms, different in each case and therefore not suggestive of a vaccine safety issue.

Limited sample size is often an obstacle to the investigation of possible associations between vaccination and rare adverse events. This has been acknowledged in the case of 4vHPV and certain autoimmune disorders, for instance (8, 13). The current data-mining study, with its 1.9 million doses, had good statistical power—detecting, for example, an attributable risk of 6 excess cases per million first 4vHPV doses for “other serum reaction due to vaccination.” In view of the statistical power, the fact that only 2 categories of adverse events were found from more than 7,000 leaves and branches of the hierarchical tree, neither one of which was unexpected, provides reassurance about both the vaccine and the TreeScan conditional temporal-tree scan method.

Three adverse events that have drawn the attention of public health authorities in relation to 4vHPV are complex regional pain syndrome (CRPS), postural orthostatic tachycardia syndrome (POTS), and Guillain-Barré syndrome (GBS). CRPS is a syndrome that affects 1 or more extremities and is characterized by persistent pain and swelling disproportionate to any known inciting event and at least 1 sign of autonomic dysfunction in the affected limb(s). In 2013, the Japanese Ministry of Health, Labour and Welfare suspended its recommendation of routine immunization with HPV vaccine for females after some postvaccination reports of serious chronic pain emerged (33). POTS is a heterogeneous and potentially debilitating autonomic disorder whose symptoms can include dizziness, nausea, fatigue, palpitations, weakness, sweating, and sleeping disorders. A case series of POTS occurring after 4vHPV vaccination in Denmark was described, raising concern about 4vHPV vaccine safety (34). In 2015, the European Medicines Agency’s Pharmacovigilance Risk Assessment Committee completed a detailed scientific review and concluded that the evidence did not support a causal link between HPV vaccines and CRPS or POTS (35), yet doubts and concerns persist (18, 36, 37). GBS is a disorder of the peripheral nervous system characterized by symmetrical ascending weakness and abnormal sensations over the course of hours, days, or weeks that can progress to temporary paralysis. In a recent large cohort study conducted in France, Miranda et al. (15) found an association between HPV (mostly 4vHPV) vaccination and GBS, with an adjusted hazard ratio of 3.78 (95% confidence interval: 1.79, 7.98) and an attributable risk of 1–2 excess cases per 100,000 girls vaccinated. They concluded that further studies were needed to confirm the finding.

Our method, as applied, may not have been optimal for detecting increased risks of adverse events such as the above 3. One reason is that we considered only risk windows that began between 1 and 28 days postvaccination and ended between 2 and 42 days postvaccination. (While the tree-temporal scan statistic can, in principle, be used for longer follow-up periods, it is as yet untested for such applications.) Thus, we could detect only adverse reactions that manifested themselves within 6 weeks of vaccination, that is, outcomes of relatively acute onset. In the case of CRPS, the interval between the precipitating event and symptom onset can vary widely, with onset typically occurring within 6 months of the injury (38). Regarding GBS, the median time from HPV vaccination to GBS onset was 4.6 months in the Miranda et al. study (15).

The second, potentially important limitation regarding syndromes such as CRPS and POTS is that the analysis was done using 1 particular hierarchical tree that included over 6,000 ICD-9 codes organized largely by functional “system” of the body (e.g., endocrine, nervous, circulatory, respiratory, digestive, and musculoskeletal). The ICD-9 and International Classification of Diseases, Tenth Revision, coding systems themselves are organized similarly, although not identically, to the MLCCS tree. However, conditions such as CRPS and POTS have symptoms that manifest in more than 1 functional system, potentially decreasing their detectability when TreeScan is paired with hierarchical structures organized in this way.

Nonetheless, from a theoretical as well as an empirical standpoint, the method as we applied it here appears to be appropriate for assessing the many potential adverse reactions of acute onset whose symptoms are concentrated in 1 section of the MLCCS (and International Classification of Diseases) hierarchy.

In conclusion, when tree-temporal scan statistics were applied to 1.9 million recipients of 4vHPV dose 1 in the Sentinel system, only 2 categories of adverse events within 42 days of vaccination emerged, and both were consistent with the known safety profile of 4vHPV. Considering the thousands of potential adverse events and hundreds of potential risk intervals evaluated and the good statistical power of this signal detection study, this finding represents a substantive and novel addition to the safety record of 4vHPV vaccine that is accumulating in the scientific literature.