Quantifying the effect of Wakefield et al. (1998) on skepticism about MMR vaccine safety in the U.S.

BACKGROUND: Efforts to trace the rise of childhood vaccine safety concerns in the US often suggest Andrew Wakefield and colleagues' retracted 1998 Lancet study (AW98)-which alleged that the MMR vaccine can cause children to develop autism-as a primary cause of US vaccine skepticism. However, a lack of public opinion data on MMR safety collected before/after AW98's publication obscures whether anecdotal accounts are indicative of a potentially-causal effect.
METHODS: We address this problem using a regression discontinuity framework to study change in monthly MMR injury claims (N = 74,850; from 1990-2019) from the Vaccine Adverse Events Reporting System (VAERS) to proxy concern about vaccine safety. Additionally, we suggest a potential mechanism for the effect of AW98 on vaccine skepticism, via automated sentiment analyses of MMR-related news stories (N = 674; from 1996-2000) in major television and newspaper outlets.
RESULTS: AW98 led to an immediate increase of about 70 MMR injury claims cases per month, averaging across six estimation strategies (meta-analytic effect = 70.44 [52.19, 88.75], p < 0.01). Preliminary evidence suggests that the volume of negative media attention to MMR increased in the weeks following AW98's publication, across four estimation strategies (meta-analytic effect = 9.59% [3.66, 15.51], p < 0.01).
CONCLUSIONS: Vaccine skepticism increased following the publication of AW98, which was potentially made possible by increased negative media coverage of MMR. SIGNIFICANCE: Childhood vaccine skepticism presents an important challenge to widespread vaccine uptake, and undermines support for pro-vaccine health policies. In addition to advancing our understanding of the previously-obscured origins of US vaccine skepticism, our work cautions that high-profile media attention to inaccurate scientific studies can undermine public confidence in vaccines. We conclude by offering several recommendations that researchers and health communicators might consider to detect and address future threats to vaccine confidence.

Introduction

In late February 1998, a research team led by Andrew Wakefield published an article in The Lancet suggesting a link between the Measles, Mumps, and Rubella vaccine (MMR) and the development of autism in children. An investigation into the study (hereafter, AW98), conducted more than ten years after its original publication date, concluded that the article’s data did not support this claim, and documented credible evidence of research malfeasance [1].

Although the piece was eventually retracted in Spring 2010 [1], both scholarly [2-4] and journalistic [5] efforts to identify the origins of contemporary vaccine skepticism in the U.S. suggest that the publication of AW98— and early media attention to it—was a pivotal moment (and perhaps the pivotal moment) in the mainstream acceptance of skepticism about MMR vaccine safety. Today, approximately one in three Americans believes that childhood vaccines can cause children to develop autism [6].

Studies attributing the rise in childhood vaccine skepticism to AW98, however, rely primarily on anecdotal evidence [2, 4–5]. This may be due in part to the scarcity of polling data about MMR safety in the immediate aftermath of AW98, and the virtual absence of such data prior to 1998 [7]. One way that scholars have circumvented this concern is by investigating the effect of AW98 on childhood vaccine skepticism is to assess whether or not MMR refusals (presumably resulting from safety concerns) increased in the aftermath [3]. Using this approach, AW98—and media coverage of it—appears to have had a short-lived and limited influence on vaccine refusal rates.

However, as outright MMR refusal is an extreme form of action—complicated by the fact that taking parents may be required to seek medical or philosophical exemptions to immunization statutes [8]—it may lack the sensitivity necessary to detect broader changes in public opinion toward MMR. In this study, make use of a comparatively more-sensitive indicator of MMR skepticism; parent reports of negative reactions to the MMR vaccine from the Vaccine Adverse Events Reporting System (VAERS; Department of Health & Human Services). These reports enable parents to express skepticism about MMR safety by reporting potential side effects their children may have experienced; meaning that parents need not take the more-extreme step of refusing vaccines outright to register concerns about its safety.

If VAERS reports are sufficiently-sensitive indicators of MMR skepticism, and if conventional wisdom about AW98 is correct, we should expect to see a sharp increase in reports following the publication of—and media attention to—the debunked study. While VAERS reports, unlike public opinion surveys, cannot tell us the “base rate” of MMR skeptical views in the mass public, they can at least offer insights into whether or not the levels of skepticism necessary to attribute MMR to various potential side effects following AW98.

Empirically demonstrating whether or not AW98 “moved the needle” on vaccine skepticism is an important empirical task, as it can help scholars better understand how communication surrounding debunked or misleading science might go on to shape behaviors relevant to public health. Additionally, given the link between MMR skepticism and opposition to policies that encourage vaccination [7, 9], this research can help us understand the potential long-run impacts of attention to fraudulent scientific claims on support for public health policies.

Materials & methods

Study design, population, and setting

Data from this study come from two sources. First, we test whether or not concern about MMR safety increased following the publication of AW98 using publicly available data of reports of adverse reactions to the MMR vaccine, via the Department of Health and Human Services (DHHS) Vaccine Adverse Event Reporting System (VAERS) [10]. Doctors and other vaccine providers report potential adverse events to VAERS in consultation with the parents of children administered the MMR vaccine.

Next, we provide supplemental tests of whether or not media coverage of AW98 presents a plausible mechanism for the effects documented in Fig 1. We obtained data from Lexis Nexis Academic by searching for news stories referencing “measles” or “MMR” in national television broadcasts (ABC, CBS, PBS, and NBC News) and high-circulation newspapers (USA Today, Washington Post, and the New York Times, Associated Press). We measured story sentiment using Linguistic Inquiry and Word Count (LIWC) software [11], which computes the ratio of negatively to positively valenced words in each story, standardized on a scale ranging from 0 (most negative) to 100 (most positive).

Fig 1

MMR VAERS reports pre/post AW98.

Panel a presents yearly MMR VAERS reports before/after the publication of AW98 (dashed vertical line). Panel b presents RD estimates of the effect (B) and two-tailed significance (p) of AW98 on weekly report counts before/after AW98 (dashed vertical line), across several RD estimation strategies (see: Materials & Methods). All RD estimates are calculated as linear effects; which is appropriate given the high degree of consonance between the locally weighted polynomial trend line (dashed lines) and linear trend line (solid line) fit to the monthly data.

Outcome measures

Our measure of MMR safety concerns is a count of all monthly MMR event reports filed to VAERS from 1990 (when the program was created) to 2019. Note that while several varieties of MMR are in use today (e.g., MMRV, which includes varicella), we focus on just MMR; both because it was the subject of AW98, and because it is consistently available throughout the series.

Additionally, for our supplemental tests, negative news volume about the MMR vaccine is the average (mean) negativity score for all MMR-related stories produced each week from March 1996 (two years before AW98’s publication) to March 2000 (two years post). We weighted weekly averages by the total number of stories featured in that week.

Statistical analysis

To test whether or not AW98 had a causal effect on increased MMR safety concerns, we first compare the number of MMR reports filed to VAERS pre/post-AW98 using a regression discontinuity (RD) setup. For robustness, we present several versions of the RD results that vary (1) whether the RD is “sharp” (estimated before/after the date AW98 was published) or “fuzzy” (within a month of the paper’s publication); (2) report aggregation level (monthly, quarterly, yearly, or using an automatic “bin” selection method via a coverage error rate [CER] optimal bandwidth estimator); (3) population adjusted vs. non-adjusted effect size estimators (to ensure that a potential increase in reports is not the result of population growth over time), and (4) the use of bias-correction standard error estimates. Results of all models are presented in Fig 1. All data and code necessary to replicate these analyses, as well as the placebo tests referenced above, can be found at: https://osf.io/n7z2v/.

To detect whether or not media coverage of AW98 might be responsible for increased MMR concern, we again use an RD setup; this time aggregating both the volume and tone of MMR stories from two years before and after the publication of AW98 (1996–2000). These tests, visualizations, and the time demarcations are constructed identically to those reported in Fig 1 —modeling change in sentiment over time—with the exception that: (1) we restrict data aggregation to be at the weekly level (as monthly aggregations would leave the model under-powered; daily aggregations would have ’sparse’ days with no coverage); (2) all models include story counts as a covariate (as we are interested in account for the volume of negative coverage); and, (3) both the locally weighted and linear trend lines adjust for weekly story volume.

In both cases, we summarize the results of these estimation procedures with a formal meta-analysis, using Cohen’s standardized mean difference procedure. Note that we report these quantities at only the monthly level for the adverse events data. This serves as a conservative estimate of the effect of AW98 on vaccine skepticism, as reporting the larger aggregation periods (included as robustness checks) would potentially inflate the average effect size.

Results

Fig 1 presents an initial test of AW98’s effect on MMR safety concerns. For reference, panel a presents the total yearly count of reported adverse experiences with MMR, from 1990 to 2019. Panel b visualizes the results of several regression discontinuity (RD) estimates of both the substantive effect of AW98 on monthly VAERS reports and statistical significance. If AW98 did indeed influence public perceptions of MMR vaccine safety, would we expect to see a substantively large and statistically significant increase in the period following the paper’s publication, compared to the period before it.

The descriptive results presented in Fig 1A document that, prior to the publication of AW98, parents typically reported under 2,000 adverse MMR events per year. Following AW98, that quantity steeply grew to over 4,000 yearly reports by 2004. Formal RD tests presented in Fig 2B document a large and statistically significant increase in events immediately following the publication of AW98. Meta-analyzing the six monthly tests presented in the figure suggests an increase of 70.44 [52.19, 88.75] reports per week.

Fig 2

MMR news sentiment pre/post AW98 (weighted by volume).

Figure presents weekly sentiment ratings of vaccine-related stories from major news outlets (gray shaded circles), weighted by the total amount of stories published that week (larger when there is more coverage in a given week). The figure also reports the effect (B) and two-tailed significance (p) of AW98 on weekly report counts before/after AW98 (dashed vertical line), across several RD estimation strategies (see: Materials & Methods). All RD estimates are calculated as linear effects; which is appropriate given the high degree of consonance between the locally weighted polynomial trend line (dashed lines) and linear trend line (solid line) fit to the weekly data; both of which adjust for total volume. N = 674 stories.

These results hold across tests that vary both the report aggregation period, standard error estimator, and whether AW98 is treated as a “sharp” or “fuzzy” discontinuity. The results also hold when accounting for population growth over time (which might result in more reports being filed).

Additionally, to ensure that growth MMR concern is not confounded by (1) changes in the procedures by which parents report adverse effects to VAERS over time (e.g., the ability to submit reports online as personal home computing and internet access grew over time), (2) changes in how health care providers diagnose autism, and/or (3) time itself–i.e., the possibility that attitudes toward all vaccines grew more negative over time for reasons unrelated to AW98 –we offer a placebo test using reports of adverse reactions to HIBV (Haemophilus Influenzae Type B Vaccine)–which has been routinely administered to children since the early 1990s, but was not identified as a health risk in AW98 –in the online materials.

If the effects of AW98 were to coincide with unobserved changes in reporting mechanisms and/or more general changes in public vaccine sentiment, we should observe comparatively more VAERS reports in the post (vs. pre) AW98 period for the HIBV placebo. This would be indicative of a spurious effect of AW98. However, the analyses presented in the online materials suggest that AW98 had no statistically or substantively discernable effect on HIBV adverse event reports (see: S1 Fig).

Fig 2 presents evidence of a potential mechanism for this effect. In the weeks following AW98’s publication, the volume of negative news stories about the MMR vaccine from major news outlets increased substantially. Meta-analyzing the RD analyses presented in Fig 2 suggests that negativity increased by 9.59% [3.66, 15.51] following publication, approaching conventional two-tailed significance in three out of four estimation strategies. This provides preliminary evidence that AW98 influenced how the media talked about MMR, which in turn drew public attention to concerns about vaccine safety.

Conclusions

This research demonstrates that AW98, and media attention to it, may have changed how some Americans viewed MMR safety. Whereas anecdotal accounts suggest that AW98 led to an increase in vaccine skepticism in the US, early studies of pre/post AW98 vaccine compliance rates cast doubt on the study’s impact on public opinion. By constructing more sensitive indicators of vaccine skepticism, we detect a large, robust, and statistically significant uptick in public concern about MMR safety following AW98.

Without public opinion data, of course, we cannot determine precisely how many Americans came to hold negative views toward MMR post-AW98. However, our approach allows us to document change in vaccine sentiment over time attributable to AW98; enabling us to shed new light on a previously-muddled area of vaccine history.

Discussion

Many Americans hold skeptical views about the safety of childhood vaccines [7, 9, 12–15]. Vaccine skepticism has important public health consequences, as people who believe that vaccines are unsafe tend to be less likely to intend to vaccinate themselves and their children against vaccine-preventable illnesses [14-16], and more likely to oppose pro-vaccine health policies [7, 17]. Understanding the origins of vaccine skepticism in the U.S. can help researchers better preempt how to mitigate the effect that fraudulent claims might have on public vaccine opinion in the future [18].

Consequently, this research has several important public health consequences. First, it suggests that attention to false or misleading vaccine research can impact public confidence in vaccines. Media sources should therefore work in consultation with researchers to stringently vet vaccine-related stories before sharing their results with the public. This will be particularly important in a post COVID-19 pandemic public health environment, where scholars have the opportunity to produce research on the (potential) long-term side effects of vaccines currently approved for public use.

Additionally, our work underscores the importance of conducting regular public opinion polling about vaccine safety; even before the possibility of controversy. While VAERS is a useful way to detect change in vaccine attitudes in the absence of public opinion data, surveys enable us to more-precisely document change in vaccine skepticism, over time. Constant surveillance of vaccine opinion can help researchers better understand how changes in the media environment might influence public vaccine confidence, and (potentially) thereby influence related vaccine policy attitudes and health behaviors.

Finally, our work suggests several opportunities for future research. For example, our study cannot disentangle the precise mechanism by which AW98 media coverage might influence public vaccine attitudes. Media attention could, for example, directly influence how Americans feel about vaccination by drawing parents’ attentions to the possibility that their children experience adverse side effects from vaccination. Alternatively, the vaccine media environment could indirectly influence vaccine attitudes by changing health care professionals attentiveness to potential side effects (e.g., increased monitoring for autism symptoms [19]), and thereby engender concern among parents resulting in increased VAERS reports. Efforts to disentangle direct from more-complex causal accounts are a worthwhile endeavor in previous research.

Additionally, and perhaps most importantly, our research offers an opportunity to learn more about the effects of COVID-19 vaccine media coverage on attitudes toward vaccination. As more Americans became eligible to vaccinate against the virus, vaccine skeptics and prominent media figures who held skeptical views toward vaccination sometimes made use of VAERS reports to justify their (often, misinterpreted) concerns about COVID-19 vaccine safety [20-22]. Consequently, future work should make an effort to determine whether or not media coverage of these issues in turn influenced reports to VAERS. This information can help scholars to not only better understand the effects of media coverage on vaccine attitudes, in application to diseases not studied in our work, but how the quality of adverse effect data reported to VAERS might be influenced by exogenous media, political, and other social events.

Moreover, although public opinion data about COVID-19 vaccine attitudes and behaviors is comparatively more plentiful than MMR opinion data, few surveys ask uptake questions at the daily level. As we demonstrate in this research, VAERS data–which can be readily aggregated at the daily level –could function as a useful supplement for public opinion data to detect short-term effects of changes in the vaccine communication environment. For example, researchers could use VAERS data to document the effects of media attention the federal government’s decision to temporarily pause administration of Johnson & Johnson’s COVID-19 vaccine. Because this pause lasted just ten days [23], VAERS data may offer an opportunity to assess these effects with additional granularity.

Placebo Test: HIBV adverse effect reports.

Placebo test replicates the analyses and tests presented in Fig 1 in the main text (although note that these analyses also account for exponential decline in VAERS reports following the publication of AW98 in the bias correction RD models by specifying a quadratic regression estimator). Please refer to the main text for additional methodological details. HIBV refers to the Haemophilus Influenzae Type B Vaccine, which–while typically administered in childhood since the early 1990s, like MMR—was not mentioned as a potential cause of autism in the AW98 paper. Consequently, if the effects documented in Fig 1 are merely the result of unobserved confounds, or time itself, we would expect to see a corresponding spike in adverse event reports following the publication of AW98. Consistent with the idea that the effects of AW98 are not confounded, the results document little effect of AW98 on VAERS reports for HIBV. Visually, although adverse event reports rise slightly following the publication of AW98, reports (1) were already increasing prior to its publication, and (2) had been in a period of exponential decline before 1998. Statistically, the results fail to document a significant increase in VAERS reports after correcting for the (pronounced) inverted parabolic nature of the pre-AW98 data, using quadratic regression. Note that because the analyses both substantively and statistically document no evidence of a discontinuity in adverse reports following AW98 in higher-powered (weekly) analyses, we do not re-estimate at the yearly or quarterly level.

(TIF)

Click here for additional data file.

21 Jul 2021

PONE-D-21-02863

Quantifying The Effect of Wakefield et al. (1998) on Skepticism about MMR Vaccine Safety in the U.S.

PLOS ONE

Dear Dr. Motta,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

First of all, apologies for the excessive length of this round of reviews; it has been especially difficult to locate available reviewers to asses such a specific paper. Once it has been reviewer by three experts, I would like to ask you for a set of revisions (most of them relatively minor), even though major issues such as the potential role of confounding variables and a complementary analysis suggested by Reviewer # 2 should be also responded by you. Once received your resubmission, and as the reviewers are already available and willing to view your revised paper, I will try to expedite the processing times as much as possible.

Please submit your revised manuscript by Sep 04 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sergio A. Useche, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a manuscript examining media coverage of MMR vaccine in the time frame around the 1998 publication of the infamous Wakefield Lancet paper and examining MMR-related VAERS claims before and after the publication of that paper. Overall this is interesting and contributes to our understanding of vaccine hesitancy, and is particularly timely given the situation with COVID vaccination in 2021.

There is one thing in the manuscript that needs to be clarified. The authors use PPV (pneumococcal polysaccharide vaccine) as a control for time as a variable, showing the data in Figure S1. This figure and the underlying analysis is problematic. PPV is not routinely administered to children, but pneumococcal conjugate vaccine (PCV) is. PPV is typically given to older adults at risk of pneumococcal pneumonia, not children at risk for pneumococcal otitis and sinusitis. Prior to the availability of PCV, PPV was given to children at high risk of pneumococcal disease (eg, children with sickle cell disease, children who lacked splenic function), but the number of immunized children was small. PCV was licensed in the US in February 2000 and was not widely available initially. In addition, there was a slight delay in uptake of the vaccine as insurers waited for efficacy data before agreeing to pay for a new, somewhat expensive vaccine. The rise in claims starting with the increase in 2000 and the later sharper rise around 2004 (as shown in Figure S1) is consistent with the uptake of PCV. Since these data are almost certainly a concatenation of PPV and PCV claims, this needs to be clarified in the manuscript and figure legend. That being said, analysis of Haemophilus influenzae type b conjugate vaccine data would not have this temporal confounding since that vaccine was introduced in the early 1990s. I would not require that this additional analysis be done, but do note that the data would be "cleaner".

Reviewer #2: Considering the slow vaccination rate, in the Discussion the authors should extend their comments regarding the implications of their work on the Corona vaccine skepticism. They should point out to Corona related misinformation similar to the MMR misinformation

Reviewer #3: The authors underscored an actual subject, unfound concerns on vaccine safety, by recalling a landmark event like the spurious publication by Wakefield et. al. On this task, the authors traced back claims on VAERS along three decades (1990-2019). The use of another vaccine as a control provided an interesting approach to assess the data. Nonetheless, three decades is a significant amount of time and during the period, several changes have occurred including changes in reporting systems and awareness of autism diagnosis. Regarding the first aspect, the use of the "control" vaccine partly answered this issue. But, changes in the way to access the reporting system should be considered also because they can affect the overall reporting. It is also advisable to  put the number of claims in the perspective of the number of vaccinees per year as denominators to understand the data. On the autism diagnosis aspect, the changes along the study period are outstanding (see https://www.ncbi.nlm.nih.gov/books/NBK332896/).  Autism was largely unknown in 1990's and awareness of healthcare professionals and the public might play a role in the apparent increase in the diagnosis (see BMJ, 312 (1996), pp. 327-328). The media coverage on the Wakefield's false claim might also raise awareness of autism diagnosis during the period (see Ment Retard Dev Disabil Res Rev 2002;8(3):151-61). In summary, the idea proposed by the authors has relevance and an interesting approach, but confounding factors should be better addressed to build stronger analyses of the data.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Michael Davidson MD

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

22 Jul 2021

** Please see attached Word Document reply. (Comments re-printed below):

Dear Dr. Useche & Reviewers at PLOS ONE,

First, I’d like to sincerely thank you for taking the time to review our manuscript entitled Quantifying The Effect of Wakefield et al. (1998) on Skepticism about MMR Vaccine Safety in the U.S.. Dom and I truly appreciate your willingness to provide feedback on our work, especially during these challenging and unprecedented times.

We were delighted to hear that the piece received a revision decision, and found the reviewers’ comments to be very helpful in revising our manuscript. Although Dr. Useche noted that many of the revisions would require only minor changes to the manuscript, we have nevertheless made several important changes to the updated piece.

Before offering a point-by-point reply, we wanted to briefly summarize some of the more-major changes we made to the updated manuscript. Chiefly, and pursuant with the Reviewers’ recommendations, we have:

1. Provided updated placebo analyses that swap reports to VAERS regarding PPV for a “cleaner” test (HIBV). [As suggested by Reviewer 1]

2. Added a discussion of how our work relates to the ongoing COVID-19 vaccination effort. [As suggested by Reviewers 1 and 2]

3. Described how it is that our placebo tests can guard against confounds not originally referenced in the main text. [As suggested by Reviewer 3]

We have also made a series of more minor edits throughout the manuscript, pursuant with both Dr. Useche’s comments and those of Reviewers 1-3.

Below, we reprint each comment raised by Reviewers 1-3 (in bold) and list our replies below each one. Where relevant, we also include block text quotes from the manuscript, to more-easily document the changes we made.

We thank you all again for your time and effort, and look forward to continuing to work with you throughout the review process.

Sincerely,

Matt Motta & Dom Stecula

REVIEWER 1 (R1)

This is a manuscript examining media coverage of MMR vaccine in the time frame around the 1998 publication of the infamous Wakefield Lancet paper and examining MMR-related VAERS claims before and after the publication of that paper. Overall this is interesting and contributes to our understanding of vaccine hesitancy, and is particularly timely given the situation with COVID vaccination in 2021.

We thank R1 for their kind comments about our manuscript, and noting its application to COVID-19 vaccine uptake in 2021. We also wanted to briefly note here that we now expand on relevance of our work to COVID-19 vaccination in response to comments offered by R2. That discussion can be found in the Discussion section of the main text, and is printed in full in our reply to R2.

There is one thing in the manuscript that needs to be clarified. The authors use PPV (pneumococcal polysaccharide vaccine) as a control for time as a variable, showing the data in Figure S1. This figure and the underlying analysis is problematic. PPV is not routinely administered to children, but pneumococcal conjugate vaccine (PCV) is. PPV is typically given to older adults at risk of pneumococcal pneumonia, not children at risk for pneumococcal otitis and sinusitis. Prior to the availability of PCV, PPV was given to children at high risk of pneumococcal disease (eg, children with sickle cell disease, children who lacked splenic function), but the number of immunized children was small. PCV was licensed in the US in February 2000 and was not widely available initially. In addition, there was a slight delay in uptake of the vaccine as insurers waited for efficacy data before agreeing to pay for a new, somewhat expensive vaccine. The rise in claims starting with the increase in 2000 and the later sharper rise around 2004 (as shown in Figure S1) is consistent with the uptake of PCV. Since these data are almost certainly a concatenation of PPV and PCV claims, this needs to be clarified in the manuscript and figure legend. That being said, analysis of Haemophilus influenzae type b conjugate vaccine data would not have this temporal confounding since that vaccine was introduced in the early 1990s. I would not require that this additional analysis be done, but do note that the data would be "cleaner".

R1 goes on to note that the use of the PPV vaccine as a placebo in our analyses, while not necessarily inappropriate, is nevertheless complicated by (1) its selective administration throughout the 1990s, and (2) the introduction of PCV in 2000. We think that this is a very important point, and thank R1 for bringing their subject matter expertise to bear on this question.

Pursuant with R1’s recommendations, we have swapped the PPV placebo test for the reviewer’s recommended alternative: HIBV. We completely agree with R1 that this offers a “cleaner” test of our expectations. Indeed, as we now report in Figure S1, our placebo test results again document no clear statistical or substantive effect of AW98 on reports to VAERS for the HIB vaccine.

We also wanted to provide a brief and somewhat technical note about these updated placebo analyses. Due to relatively high levels of VAERS reports logged at the start of the time series (i.e., when HIBV was first introduced), we observe an inverted parabolic trend in VAERS reports prior to 1998 (high at the start of the series, falling, and then increasing years prior to AW98). To pose the fairest possible test for spuriousness, we therefore account for the nonlinear nature of the pre-AW98 data by using quadratic (as opposed to linear) regression when estimating our bias correction RD models. These analyses again document no increase in VAERS reports following AW98 (in fact, once we account for the quadratic nature of the data, we observe a very slight decrease in reports).

-

REVIEWER #2 (R2)

Reviewer #2: Considering the slow vaccination rate, in the Discussion the authors should extend their comments regarding the implications of their work on the Corona vaccine skepticism. They should point out to Corona related misinformation similar to the MMR misinformation

R2’s primary concern with the manuscript is that we did not make a sufficient effort to engage with COVID-19 vaccine uptake in the piece’s discussion section. We completely agree with R2’s comments, on this score, and note that R1 mentioned this point in passing as well.

Correspondingly, we have added two paragraphs of text to the piece’s discussion section wherein we discuss the potential applications of our research for understanding vaccine uptake throughout the global pandemic. We also point to applications for future research, and (briefly) sketch out VAERS-related studies that researchers might consider taking up.

Specifically, we write:

“Additionally, and perhaps most importantly, our research offers an opportunity to learn more about the effects of COVID-19 vaccine media coverage on attitudes toward vaccination. As more Americans became eligible to vaccinate against the virus, vaccine skeptics and prominent media figures who held skeptical views toward vaccination sometimes made use of VAERS reports to justify their (often, misinterpreted) concerns about COVID-19 vaccine safety [20-22]. Consequently, future work should make an effort to determine whether or not media coverage of these issues in turn influenced reports to VAERS. This information can help scholars to not only better understand the effects of media coverage on vaccine attitudes, in application to diseases not studied in our work, but how the quality of adverse effect data reported to VAERS might be influenced by exogenous media, political, and other social events.

Moreover, although public opinion data about COVID-19 vaccine attitudes and behaviors is comparatively more plentiful than MMR opinion data, few surveys ask uptake questions at the daily level. As we demonstrate in this research, VAERS data – which can be readily aggregated at the daily level – could function as a useful supplement for public opinion data to detect short-term effects of changes in the vaccine communication environment. For example, researchers could use VAERS data to document the effects of media attention the federal government’s decision to temporarily pause administration of Johnson & Johnson’s COVID-19 vaccine. Because this pause lasted just ten days [23], VAERS data may offer an opportunity to assess these effects with additional granularity.”

We hope that these remarks help broaden the scope of the relevance of our work to current events, and facilitate additional research using these under-studied data.

REVIEWER #3 (R3)

The authors underscored an actual subject, unfound concerns on vaccine safety, by recalling a landmark event like the spurious publication by Wakefield et. al. On this task, the authors traced back claims on VAERS along three decades (1990-2019). The use of another vaccine as a control provided an interesting approach to assess the data. Nonetheless, three decades is a significant amount of time and during the period, several changes have occurred including changes in reporting systems and awareness of autism diagnosis. Regarding the first aspect, the use of the "control" vaccine partly answered this issue. But, changes in the way to access the reporting system should be considered also because they can affect the overall reporting.

R3 begins by noting that, while our study’s placebo vaccine analyses can help buttress concerns that differences in how health care providers diagnose autism might influence the effects of AW98 observed in the main text, it is also important that we consider the possibility that changes in the mechanics of submitting reports to VAERS might have a potentially confounding effect.

We completely agree with R3 on this score, and have made several changes to the main text in response. As we now note directly in the main text (amended language re-printed below), we believe that the placebo tests can also guard against this concern. If it was the case that changes in the reporting system were to (say) co-occur with AW98 and therefore be responsible for increased reporting, we would also expect to see similar increases in reports on other vaccines (as reporting changes, like the opportunity to report claims online, are VAERS-level phenomena, and not necessarily specific to one vaccine).

However, we recognize that we did not make this point clearly in the initial manuscript. Consequently, we now list several potential ways in which our results could be confounded by events aside from AW98, and include language suggesting that the placebo tests are well suited to address these concerns. Specifically, we write (updated text in bold):

“Additionally, to ensure that growth MMR concern is not confounded by (1) changes in the procedures by which parents report adverse effects to VAERS over time (e.g., the ability to submit reports online as personal home computing and internet access grew over time), (2) changes in how health care providers diagnose autism, and/or (3) time itself – i.e., the possibility that attitudes toward all vaccines grew more negative over time for reasons unrelated to AW98 – we offer a placebo test using reports of adverse reactions to HIBV (Haemophilus Influenzae Type B Vaccine) – which has been routinely administered to children since the early 1990s, but was not identified as a health risk in AW98 – in the online materials.

If the effects of AW98 were to coincide with unobserved changes in reporting mechanisms and/or more general changes in public vaccine sentiment, we should observe comparatively more VAERS reports in the post (vs. pre) AW98 period for the HIBV placebo. This would be indicative of a spurious effect of AW98. However, the analyses presented in the online materials suggest that AW98 had no statistically or substantively discernable effect on HIBV adverse event reports (see: S1 Figure).”

It is also advisable to put the number of claims in the perspective of the number of vaccinees per year as denominators to understand the data.

R3 goes on to note that population-adjusted vaccine estimates (i.e., the percentage VAERS reports relative to all MMR doses administered) might help some readers to better engage with the analyses presented in the main text. We think that this is a thoughtful point, and one that we took seriously when revising this piece. We have two brief replies.

First, we wanted to underscore that – although we present the raw number of reports to VAERS in Figure 1 – we do display a model in the figure that adjusts for changes in US population growth over time. Our results are robust to this alternative specification.

Second, we nevertheless recognize that the quantities R3 suggests may be more tractable for some readers. Unfortunately, available time serial MMR surveillance data from the federal government are typically expressed in percentages, and not raw counts, throughout the period we study. While we could theoretically multiply the size of the vaccine-eligible population by these percentages, doing so would introduce additional error into our analyses – as percentages are necessarily rounded (so multiplying one by the other would imply a loss of precision).

(Note: we are of course open to the possibility that these data are readily available, and would be happy to consider re-estimating the models if we are able to access the data).

Still, we note that MMR vaccine uptake rates have held relatively constant throughout the series that we study in this piece. This likely means that the results we observe when we correct for population change in our RD models would produce a similar result to scaling VAERS reports by the total number of vaccines administered (as the population totals would be multiplied by a more-or-less constant factor).

Consequently, though we have not amended our approach in the main text to reflect this concern, we nevertheless think that it is a thoughtful point, and wanted to ensure that we gave it significant consideration in our reply memo.

On the autism diagnosis aspect, the changes along the study period are outstanding (see https://www.ncbi.nlm.nih.gov/books/NBK332896/). Autism was largely unknown in 1990's and awareness of healthcare professionals and the public might play a role in the apparent increase in the diagnosis (see BMJ, 312 (1996), pp. 327-328). The media coverage on the Wakefield's false claim might also raise awareness of autism diagnosis during the period (see Ment Retard Dev Disabil Res Rev 2002;8(3):151-61).

Finally, R3 suggests that changes in the diagnosis of autism throughout the study period might complicate our account of the potential mechanism for the effects we observe in this piece. Perhaps media attention to AW98 raised awareness about autism, which in turn altered the way in which health care providers make diagnoses?

We think that this is an important point, and one we now discuss directly in the manuscript’s main text. This alternative account, in our view, adds a level of conceptual nuance to our proposed mechanism (i.e., negative media coverage of the MMR vaccine). In other words, it suggests that media coverage has downstream effects on how doctors and patients interact, which in turn influence VAERS reports. Media coverage remains the “starting point” of the causal chain, under this view, but takes more-nuanced “turns” before ultimately influencing reports.

As we now discuss in the main text (remarks printed below; note also the inclusion of the helpful Wing & Potter piece that R3 recommended we consult), we think that this is a fascinating direction for future research. Although we characterized our discussion of the hypothesized mechanism for the effects observed in this piece as preliminary in our initial manuscript, we encourage scholars in the future to explore these more-nuanced causal accounts. In fact, we are actively trying to do this ourselves, in working research.

Specifically, we write:

“Finally, our work suggests several opportunities for future research. For example, our study cannot disentangle the precise mechanism by which AW98 media coverage might influence public vaccine attitudes. Media attention could, for example, directly influence how Americans feel about vaccination by drawing parents’ attentions to the possibility that their children experience adverse side effects from vaccination. Alternatively, the vaccine media environment could indirectly influence vaccine attitudes indirectly by changing health care professionals attentiveness to potential side effects (e.g., increased monitoring for autism symptoms), and thereby engender concern among parents resulting in increased VAERS reports. Efforts to disentangle direct from more-complex causal accounts are a worthwhile endeavor in previous research.”

In summary, the idea proposed by the authors has relevance and an interesting approach, but confounding factors should be better addressed to build stronger analyses of the data.

We thank R3 for their kind remarks. We hope that these changes show that we have taken seriously the ways in which this manuscript accounts for potential confounding effects, and made an effort to clarify our efforts to address this issue.

Submitted filename: Response Memo.docx

Click here for additional data file.

6 Aug 2021

Quantifying The Effect of Wakefield et al. (1998) on Skepticism about MMR Vaccine Safety in the U.S.

PONE-D-21-02863R1

Dear Dr. Motta,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sergio A. Useche, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional): Thanks for your revisions and clarifications (and apologies for the delay).

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have addressed my concerns, and I am particularly pleased that they took the suggestion to look at an alternative vaccine comparator.

I have a few comments for copyediting, but none are problematic.

Line 97: The work “taking” seems out of place.

Line 99: I believe the sentence starting, “In this study,” is missing a subject. It seems like it should read, “In this study, we make use…”.

Line 218: I believe the sentence is missing a preposition in, “to ensure that growth MMR concern”.

Reviewer #2: As requested the authors commented in the discussion section on the relevance of their work to Corona hesitancy All remarks have been addressed. No additional revision are necessary.

Reviewer #3: After the review, the manuscript is more clear and properly addressed potential cofounders in the text. Switching the "control" vaccine to HiBV helped to validate results in better than in the previous version. Besides the discussion on the cofounders cannot be resolved in retrospective way, the authors acknowledged this limitation.

One last comment is to suggest using "control test" instead of "placebo test" referring to the HiBV data. The purpose of this test is to offer a comparison and not to pretend to be the evaluated vaccine.

Congratulations to the authors for the interesting manuscript.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Michael Anthony Moody

Reviewer #2: Yes: Michael Davidson

Reviewer #3: No

10 Aug 2021

PONE-D-21-02863R1

Quantifying The Effect of Wakefield et al. (1998) on Skepticism about MMR Vaccine Safety in the U.S.

Dear Dr. Motta:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sergio A. Useche

Academic Editor

PLOS ONE