The Weber effect and the United States Food and Drug Administration's Adverse Event Reporting System (FAERS): analysis of sixty-two drugs approved from 2006 to 2010.

BACKGROUND: The United States Food and Drug Administration's (FDA) Adverse Event Reporting System (FAERS) consists of adverse event (AE) reports linked to approved drugs. The database is widely used to support post-marketing safety surveillance programs. Sometimes cited as a limitation to the usefulness of FAERS, however, is the 'Weber effect,' which is often summarized by stating that AE reporting peaks at the end of the second year after a regulatory authority approves a drug. Weber described this effect in 1984 based upon a single class of medications prescribed in the United Kingdom. Since that time, the FDA has made a concerted effort to improve both reporting and the database itself. Both volume and quality of AE reporting has dramatically improved since Weber's report, with an estimated 800,000 yearly reports now being logged into FAERS.
OBJECTIVE: The aim of this study was to determine if current FAERS reporting follows the trend described by Weber.
METHODS: Sixty-two drugs approved by the FDA between 2006 and 2010 were included in this analysis. Publicly available FAERS data were used to assess the 'primary suspect' AE reporting pattern for up to a 4-year period following each drug's approval date.
RESULTS: A total of 334,984 AE reports were logged into FAERS for the 62 drugs analyzed here. While a few of the drugs demonstrated what could be considered 'Weber effect' curves, a majority of the drugs showed little evidence for the effect. In fact, the general AE reporting pattern observed in this study appears to consist simply of increasing case counts over the first three quarters after approval followed by relatively constant counts thereafter.
CONCLUSIONS: Our results suggest that most of the modern adverse event reporting into FAERS does not follow the pattern described by Weber. Factors that may have contributed to this finding include large increases in the volume of AE reports since the Weber effect was described, as well as a concerted effort by the FDA to increase awareness regarding the utility of post-marketing AE reporting.

Key Points

Introduction

By necessity, pre-approval clinical testing is conducted in relatively homogenous subjects and, accordingly, cannot delineate the complete adverse event (AE) profile of a drug. Limitations common to pre-approval testing include small test populations, brief drug exposure durations, and exclusion criteria that often eliminate testing of pregnant subjects, the elderly, the young, subjects with existing comorbidities, and patients who take multiple medications [1]. Therefore, unexpected actions often occur once a drug is approved and introduced into the broad and heterogeneous population of consumers. In fact, many serious and life-threatening adverse events are commonly encountered only after a drug obtains US Food and Drug Administration (FDA) approval [2-5]. Additionally, only half of all serious AEs are listed in the main source of AE information for many prescribers (the Physician’s Desk Reference) within 7 years after they win FDA approval [5]. In short, careful post-approval AE monitoring is vital to the ongoing drug evaluation process and patient safety.

The FDA’s Adverse Event Reporting System (FAERS) is a centralized, computerized, information database that is broadly used by the FDA and other pharmacovigilance experts for post-marketing drug safety surveillance [1, 6–21]. The FDA uses FAERS analyses to make post-marketing regulatory decisions such as the issuance of warnings, label changes, and/or market removal [22]. International government and related organizations (Australia’s Therapeutic Goods Administration, Canada’s Vigilance Adverse Reaction Online Database, Europe’s EudraVigilance, Japan’s Pharmaceuticals and Medical Devices Agency, The United Kingdom’s Yellow Card Scheme, and The World Health Organization’s VigiBase) also use spontaneous AE databases to identify post-approval drug safety concerns.

Two commonly assumed limitations of spontaneous AE databases is that they suffer from ‘stimulated reporting’ [23-27] and the ‘Weber effect’ [28]. With regard to the Weber effect, in 1984, JCP Weber published a paper in which he detailed adverse event reporting trends in the United Kingdom regarding oral drugs in the non-steroidal anti-inflammatory drug (NSAID) class [28]. Of note, the reporting trend he described occurred during a time period when the UK’s ‘Black Triangle’ reporting guidelines [29] directed prescribers to pay special attention to AEs during the first 2 years after a drug’s approval.

In many modern publications, the Weber effect is assumed to operate in various global AE databases and is often too simply summarized as ‘after regulatory approval of a drug, AE reporting increases over the first 2 years, peaks near the end of year 2, and then reliably, and rapidly, diminishes with further time on the market.’ An important point that Weber made about the pattern, however, is often overlooked by such a generalization. Weber stated “this decline is due to a reduction in the reporting of clinically mild or trivial reactions. The more serious ADR, such as haematemesis, perforation of peptic ulcers, blood dycrasias, etc. are reported from year to year in a quite constant manner” [28].

Since Weber’s original paper was published, multiple publications have either approximated or replicated the Weber effect [30, 31] while other publications that did not directly test for the effect cite it as an accepted principle [32-34]. Some of the same authors, however, who found a Weber effect with the NSAID class of drugs [31] reported only a partial Weber effect with the serotonin-reuptake class of medications [35]. Other authors discovered that some NSAIDs had a Weber-like pattern, while others did not [36]. A recent report that correlated usage data with reported AEs also did not observe Weber-like patterns [37]. An analysis of new drugs approved in 2006 also showed no evidence to support the contention that the modern generalization of the Weber effect accurately describes current AE reporting trends [38]. Most of the studies, however, that analyzed reporting trends have been conducted in a limited drug class or with a narrow time window. Additionally, the nature of AE reporting has dramatically changed since 1984, with improvements both in the quality of reports and the volume of data [39, 40]. In fact, an estimated 800,000 new case reports are currently logged into FAERS every year [41].

Accordingly, in order to determine general AE reporting patterns for recent FAERS data, we analyzed 62 drugs approved from 2006 to 2010.

Methods

Drugs Included

We included prescription drugs approved from 2006 to 2010 that had a total of at least 1,000 ‘primary suspect’ case reports in FAERS. (‘Primary suspect’ is a designation by the person who submitted a given case report, and is their estimate of which drug, if the subject was taking more than one, was likely responsible for the observed AE). Over-the-counter and recreational drugs, vaccines, and broad, undefined compounds listed in FAERS were not analyzed. Duplicate case reports were removed.

Inclusion Criteria for FAERS Case Reports

Case reports that were missing or contained malformed key identification fields (Individual Safety Report number [ISR], patient number, drug sequence identification, or MedDRA® AE term) were discarded. As long as the aforementioned key identification fields were contained in a given case report, allowable missing fields included age, gender, weight, outcome, and condition. Cases were discarded if the drug name was found to be indeterminate or if the name was determined to not represent an FDA-approved drug (e.g., dietary supplements, foods, etc). In an effort to exclude pre-approval AE case reports mistakenly logged into FAERS, the date of receipt for a given case report must have occurred after the drug’s FDA approval date.

Drug Name Mapping

Drug-name text mapping was accomplished as previously described by Hoffman et al. [42]. Drug names were normalized to RxNorm reference codes [43] using string searching and manual curation. National Drug File Reference Terminology [44] was used to provide ancillary information on class and mechanism of action.

Established Pharmacologic Classes (EPC)

EPC is a designation by the FDA that indicates the established pharmacologic class(es) of a drug or other therapeutic compound [44]. EPC data were obtained from the FDA National Drug Code file and mapped by researchers to the appropriate FAERS drug-name data by way of brand or generic compound name.

Case Counting

Quarterly primary suspect case reports submitted by ‘all reporters’ were counted from the first full quarter after a drug’s FDA approval through quarter 4 of 2012.

Normalization

For each drug, the highest case count total reached, in any of the 16 quarters analyzed, was normalized to 100. All remaining 15 quarters were normalized accordingly.

Results

Sixty-two drugs and 334,984 primary suspect reports were included in this analysis. Our results show that the Weber effect may no longer be a valid concern in modern FAERS reporting. AE reporting volume increased, as would be expected, over the first few quarters after a drug was introduced to the market. While no clear pattern could be ascribed to the reporting trends, most drugs reached a near maximum amount of reports by the third full quarter after approval. In general, after that point, case counts per quarter stayed relatively constant.

Tables 1, 2, 3, 4, and 5 list the drugs analyzed by year, with EPC, approval date, total number of primary suspect case reports, and the percentage of ‘healthcare professionals’ versus ‘other’ reporters. The brand name was noted for generic compounds marketed in the US under multiple commercial names.

Table 1

New molecular entity drugs approved in 2006

Generic name	EPC	Date of approval	Total primary reports (n)	Healthcare reporter (%)	Other reporter (%)
Sunitinib	Kinase inhibitor	1/26/2006	12,947	55.50	44.50
Drospirenone; ethinyl estradiola	Estrogenb, progestinb	3/16/2006	21,137	10.48	89.52
Methylphenidatec	Central nervous system stimulant	4/6/2006	3,711	24.28	75.72
Varenicline	Partial cholinergic nicotinic agonist	5/10/2006	56,683	21.39	78.61
Darunavir	Protease inhibitor	6/23/2006	1,066	90.15	9.85
Dasatinib	Kinase inhibitor	6/28/2006	2,880	41.60	58.40
Ranibizumab	Vascular endothelial growth factor-directed antibody	6/30/2006	7,420	45.04	54.96
Etonogestreld	Progestin	7/17/2006	2,464	50.45	49.55
Budesonide; formoterol	β2-Adrenergic agonist, corticosteroid	7/21/2006	10,370	16.84	83.16
Panitumumab	Epidermal growth factor receptor antagonist	9/27/2006	2,283	82.83	17.17
Sitagliptin	Dipeptidyl peptidase 4 inhibitor	10/16/2006	10,708	64.53	35.47
Carvedilole	α-Adrenergic blocker, β-adrenergic blocker	10/20/2006	1,268	5.44	94.56
Paliperidone	Atypical antipsychotic	12/19/2006	6,405	79.98	20.02

aYaz®

bManually mapped established pharmaceutical class (EPC)

cDaytrana®

dImplanon®

eCoreg CR®

Table 2

New molecular entity drugs approved in 2007

Generic name	EPC	Date of approval	Total primary reports (n)	Healthcare reporter (%)	Other reporter (%)
Diclofenaca	Nonsteroidal anti-inflammatory drug	1/31/2007	1,189	19.76	80.24
Lisdexamfetamine	Central nervous system stimulant	2/23/2007	3,076	35.73	64.27
Aliskiren	Renin inhibitor	3/5/2007	4,538	60.45	39.55
Lapatinib	Kinase inhibitor	3/13/2007	6,607	37.26	62.74
Eculizumab	Complement inhibitor	3/16/2007	5,178	38.95	61.05
Metformin; sitagliptin	Biguanide, dipeptidyl peptidase 4 inhibitor	3/30/2007	1,080	64.07	35.93
Zoledronic acidb	Bisphosphonate	4/16/2007	12,880	36.76	63.24
Fluticasone furoate	Corticosteroid	4/27/2007	1,205	16.85	83.15
Rotigotine	Nonergot dopamine agonistc	5/9/2007	1,261	32.36	67.64
Quetiapined	Atypical antipsychotic	5/17/2007	6,053	29.29	70.71
Temsirolimus	Kinase inhibitorc	5/30/2007	1,968	81.61	18.39
Ambrisentan	Endothelin receptor antagonist	6/15/2007	9,083	28.40	71.60
Armodafinil	Central nervous system stimulantc	6/15/2007	2,653	41.42	58.58
Amlodipine; valsartan	Angiotensin 2 receptor blocker, dihydropyridine calcium channel blocker	6/20/2007	2,438	22.07	77.93
Raltegravir	Human immunodeficiency virus integrase strand transfer inhibitor	10/12/2007	1,617	70.32	29.68
Ixabepilone	Microtubule inhibitorc	10/16/2007	1,119	43.07	56.93
Nilotinib	Kinase inhibitor	10/29/2007	3,956	57.25	42.75
Methoxy polyethylene glycol-epoetin β	Erythropoiesis-stimulating agentc	11/14/2007	1,266	59.32	40.68
Nebivolol	β-Adrenergic blockerc	12/17/2007	1,350	58.44	41.56

aFlector®

bReclast®

cManually mapped established pharmaceutical class (EPC)

dSeroquel XR®

Table 3

New molecular entity drugs approved in 2008

Generic name	EPC	Date of approval	Total primary reports (n)	Healthcare reporter (%)	Other reporter (%)
Niacin; simvastatin	HMG-CoA reductase inhibitor, nicotinic acid	2/15/2008	9,663	3.61	96.39
Desvenlafaxine	Serotonin and norepinephrine reuptake inhibitor	2/29/2008	9,278	40.61	59.39
Bendamustine	Alkylating drug	3/20/2008	2,585	83.79	16.21
Certolizumab	Tumor necrosis factor blockera	4/22/2008	11,318	61.34	38.66
Tetrabenazine	Vesicular monoamine transporter 2 (VMAT) inhibitora	8/15/2008	1,244	70.42	29.58
Romiplostim	Thrombopoiesis stimulating agent	8/22/2008	5,144	87.56	12.44
Lacosamide	Anti-epileptic agent	10/28/2008	2,080	64.66	35.34
Fesoterodine	Muscarinic antagonista	10/31/2008	2,223	31.98	68.02
Eltrombopag	Thrombopoiesis stimulating agent	11/20/2008	1,642	54.51	45.49
Choline fenofibrate	Peroxisome proliferator receptor α agonist	12/15/2008	1,851	60.72	39.28
Bimatoprostb	Prostaglandin analog	12/24/2008	3,224	29.62	70.38

aManually mapped established pharmaceutical class (EPC)

bLatisse®

Table 4

New molecular entity drugs approved in 2009

Generic name	EPC	Date of approval	Total primary reports (n)	Healthcare reporter (%)	Other reporter (%)
Milnacipran	Serotonin and norepinephrine reuptake inhibitor	1/14/2009	1,830	56.39	43.61
Everolimusa	Kinase inhibitor	3/30/2009	4,119	58.68	41.32
Golimumab	Tumor necrosis factor blocker	4/24/2009	2,602	68.95	31.05
Dronedarone	Antiarrhythmic	7/1/2009	2,934	69.90	30.10
Prasugrel	P2Y12 Platelet inhibitor	7/10/2009	1,916	61.59	38.41
Treprostinilb	Prostacycline vasodilator	7/30/2009	1,429	47.94	52.06
Saxagliptin	Dipeptidyl peptidase 4 inhibitor	7/31/2009	1,888	45.13	54.87
Asenapine	Atypical antipsychotic	8/13/2009	3,699	80.43	19.57
Interferon β-1Bc	Recombinant human interferon βd	8/14/2009	1,009	21.90	78.10
Ustekinumab	Interleukin-12 and -23 antagonistd	9/25/2009	2,462	76.20	23.80
Pazopanib	Kinase inhibitor	10/19/2009	2,346	37.30	62.70

aAfinitor®

bTyvaso®

cExtavia®

dManually mapped established pharmaceutical class (EPC)

Table 5

New molecular entity drugs approved in 2010

Generic name	EPC	Date of approval	Total primary reports (n)	Healthcare reporter (%)	Other reporter (%)
Tocilizumab	Interleukin-6 (IL-6) receptor inhibitora	1/8/2010	3,839	63.04	36.96
Dalfampridine	Potassium channel blockera	1/22/2010	1,626	32.84	67.16
Liraglutide	Glucagon-like peptide (GLP)-1 receptor agonist	1/25/2010	9,768	40.47	59.53
Everolimusb	Kinase inhibitor	4/20/2010	1,259	68.55	31.45
Denosumabc	RANK ligand inhibitor	6/1/2010	5,880	52.23	47.77
Fingolimod	Sphingosine 1-phosphate receptor modulator	9/21/2010	6,327	42.01	57.99
Dabigatran	Direct thrombin inhibitora, anti-coagulanta	10/19/2010	20,980	57.52	42.48
Denosumabd	RANK ligand inhibitor	11/19/2010	1,202	67.30	32.70

aManually mapped established pharmaceutical class (EPC)

bZortress®

cProlia®

dXgeva®

Figures 1, 2, 3, 4, and 5 show primary suspect case reports submitted by all reporters from the first full quarter after a drug’s approval through quarter 4 of 2012. We also charted year-by-year results for healthcare professional reporters (data included in Electronic Supplementary Material as tables 1–5), but there were no significant differences between the reporter groups (see Figs. 6 and 7 for grand averages for both groups of reporters). The black line represents the average of all drugs for that year. In general, case counts increase until approximately 3–7 quarters after approval and then remain relatively constant.

Fig. 1

Drugs approved in 2006

Fig. 2

Drugs approved in 2007

Fig. 3

Drugs approved in 2008

Fig. 4

Drugs approved in 2009

Fig. 5

Drugs approved in 2010

Fig. 6

The average of all ‘all reporters’ for the 62 drugs

Fig. 7

The average of all ‘healthcare professional reporters’ for the 62 drugs

Figures 6 and 7 show average primary case counts (± standard error) for all drugs, across all years, for up to 16 full quarters after approval. Figure 6 was derived by using the case counts logged into FAERS for all reporters, while Fig. 7 used healthcare professional reports only. As can be seen from Figs. 6 and 7, the average number of case reports filed for new drugs increases steadily until it reaches a plateau within quarter 3–7.

Table 6 (included in the Electronic Supplementary Material) and Fig. 8 show within-drug comparisons of normalized case counts compared with a hypothetical Weber-like decrease in case counts from quarter 8 after approval to quarter 16. For each drug, we modeled the quarterly report counts as a linear function of time and evaluated the fit above a constant (null) model using an ANOVA. Table 6 lists the p values for each drug regarding line fit and intercept to the hypothetical Weber-like decrease in reporting rates after quarter 8. For this analysis we used 42 of the 61 drugs in this study because they had case count data for quarters 8–16. Of the 20 drugs with significant estimates (p < 0.05) for the period coefficient, six drugs (carvedilol, certolizumab, eltrombopag, romiplostim, sitagliptin, and varenicline) had negative slopes (decreasing reporting) and thus fit the expected behavior of our experimental Weber-effect model (df = 7, F test).

Fig. 8

Within-drug comparison of normalized case counts compared with a hypothetical linear Weber-like decrease in case counts from quarter 8 after approval to quarter 16

Discussion

In modern publications, the Weber effect [28] is typically generalized as an increase in AE reporting over the first 2 years after a drug’s approval, followed by a rapid decline in reporting rates. This generalization of what Weber described is often cited as a limitation to the interpretation and usefulness of post-marketing AE databases such as FAERS. We found little evidence of such a reporting trend regarding FAERS data for 62 drugs approved from 2006 to 2010. If a generalizable pattern to the data can be discerned, it is simply that case counts in FAERS tend to increase for approximately the first three quarters after a drug’s approval date and then stay relatively constant for at least the next 13 quarters thereafter. Our findings comport with both an internal FDA study that did not replicate the Weber effect [37] and a recent study that mapped 5-year AE reporting trends for drugs approved in 2006 [38].

We postulate that modern-day FAERS reporting may no longer exhibit the Weber effect due to increased focus on the importance and utility of post-approval AE reporting by both the FDA and key healthcare players over the last decades. With specific reference to potential AE reporting differences between when Weber published his findings and the results presented here, the following events may have contributed to improved AE reporting, and therefore a suppression of Weber-like reporting trends. In 1997, the FDA formed the FAERS (then referred to as AERS) database to facilitate the reporting of post-approval AEs. FAERS now has a total of over seven million AE case reports. In 2005, the FDA formed a permanent Drug Safety Oversight Board with the specific task of monitoring post-marketing AEs [45]. The FDA issued new guidance in 2006 for how AE data should be presented in drug labels [46]. In 2007, in order to enhance post-approval drug safety analysis even further, the FDA Amendments Act was implemented to improve FAERS and provide more resources (including regulatory enforcement) to the FDA [47, 48]. Also in 2007, the FDA began to require drug companies to enact Risk Evaluation and Mitigation Strategies (REMS) to better manage, track, and report AEs [49]. Finally, in 2008, Title IX of the FDA Amendments Act [50] was enacted, which required direct-to-consumer advertisements to contain, in prominent text, “you are encouraged to report negative side effects of prescription drugs to the FDA” with phone numbers and a website listed to facilitate such reporting.

Limitations

A number of constraints must be considered when analyzing these data. FAERS relies heavily upon voluntary reporting, and therefore, the frequency of actual AEs is very likely to be underestimated. We, however, see no obvious reason why overall underreporting would affect the type of trend analyses reported here. Even though we view primary suspect FAERS cases to be the most accurate for quantitative purposes, the designation is subjective and the influence of other drugs or factors cannot be ruled out from a given case report. Since we did not have usage data for these drugs, we could not analyze AE case counts as a ‘reporting rate.’ Our analysis was therefore limited to total case counts per drug. A related study, however, that normalized case counts with usage data did not find evidence of the Weber effect [37].

Conclusions

Our primary objective was to determine if modern FAERS case reports parallel the AE reporting trend first described by Weber for NSAIDs marketed in the UK in 1984. Based on the first 4 years of reporting after drug approval, we did not observe a Weber reporting pattern for drugs approved from 2006 to 2010 with over 1,000 primary suspect case reports. Instead, the general reporting pattern detected was a steep rise in case counts over the first three quarters after approval. After an initial plateau was reached by the third full quarter, relatively constant case count totals continued for the remaining 13 quarters. While a few of the drugs in the group of 62 analyzed here did show Weber-like trending, the vast majority did not. We do not know the exact reasons why our analysis did not replicate Weber’s findings, but we estimate that modern-day AE reporting is quite different than it was in 1984. Indeed, post-marketing AE reporting has dramatically increased in volume since Weber’s analysis. Additionally, the systems for capturing AE reports, as well as the value of analyzing such data, have greatly improved over the last decades. The Weber effect should not be assumed when analyzing modern-day FAERS reporting.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 43 kb)

Adverse event (AE) archives such as the US FDA’s Adverse Event Reporting database (FAERS) are sometimes assumed to suffer from the ‘Weber effect,’ often generalized as ‘after regulatory approval, AE reporting peaks by the end of year 2 and then rapidly diminishes with time.’

We did not find evidence for such a general trend in 62 drugs, therefore assertions that modern FAERS data are unreliable due to the ‘Weber effect’ appear unfounded.