Effect of database profile variation on drug safety assessment: an analysis of spontaneous adverse event reports of Japanese cases.

BACKGROUND: The use of a statistical approach to analyze cumulative adverse event (AE) reports has been encouraged by regulatory authorities. However, data variations affect statistical analyses (eg, signal detection). Further, differences in regulations, social issues, and health care systems can cause variations in AE data. The present study examined similarities and differences between two publicly available databases, ie, the Japanese Adverse Drug Event Report (JADER) database and the US Food and Drug Administration Adverse Event Reporting System (FAERS), and how they affect signal detection.
METHODS: Two AE data sources from 2010 were examined, ie, JADER cases (JP) and Japanese cases extracted from the FAERS (FAERS-JP). Three methods for signals of disproportionate reporting, ie, the reporting odds ratio, Bayesian confidence propagation neural network, and Gamma Poisson Shrinker (GPS), were used on drug-event combinations for three substances frequently recorded in both systems.
RESULTS: The two databases showed similar elements of AE reports, but no option was provided for a shareable case identifier. The average number of AEs per case was 1.6±1.3 (maximum 37) in the JP and 3.3±3.5 (maximum 62) in the FAERS-JP. Between 5% and 57% of all AEs were signaled by three quantitative methods for etanercept, infliximab, and paroxetine. Signals identified by GPS for the JP and FAERS-JP, as referenced by Japanese labeling, showed higher positive sensitivity than was expected.
CONCLUSION: The FAERS-JP was different from the JADER. Signals derived from both datasets identified different results, but shared certain signals. Discrepancies in type of AEs, drugs reported, and average number of AEs per case were potential contributing factors. This study will help those concerned with pharmacovigilance better understand the use and pitfalls of using spontaneous AE data.

Introduction

Spontaneous report systems are utilized for pharmacovigilance. Over 100 countries provide adverse event (AE) reports to the World Health Organization Uppsala Monitoring Center through the World Health Organization Programme for International Drug Monitoring, allowing the Uppsala Monitoring Center to maintain VigiBase™, a database with approximately 8,000,000 cases.1 Another well-known pharmacovigilance database is the US Food and Drug Administration Adverse Event Reporting System (FAERS),2 a database with over 4,000,000 worldwide spontaneous AE reports. These reports have been partially available for free public access online since 1997. In Japan, the Pharmaceutical and Medical Device Agency, established by the Ministry of Health, Labour and Welfare in 2004, began offering free access to part of its database, ie, the Japanese Adverse Drug Event Report (JADER), in April 2012.3 The JADER database contains approximately 230,000 AE cases reported after April 2004.3 Each database complies with the globally accepted standard for electronic submission of individual case safety reports4 in pharmacovigilance, and uses the Medical Dictionary for Regulatory Activities (MedDRA) terminology.5 The JADER contains AE cases occurring specifically in Japan, while VigiBase and the FAERS contain worldwide cases. They are intended for descriptive or statistical analysis, not narrative assessment. It must be emphasized here that the spontaneous reporting system of each country is underpinned by the common purpose of drug safety, and reporting systems are affected by differences in regulations, and social and historical backgrounds across countries. Further, knowledge about how AE data in spontaneous report systems differ is scarce, because most authorities do not make their spontaneous reports system data publicly available.

While they cannot replace qualitative assessment, quantitative methods are expected to enhance pharmacovigilance activity. Point estimates of reporting disproportionality, such as the reporting odds ratio (ROR)6 and proportional reporting ratio,7 have been proposed as quantitative approaches. They are simple methods for disproportionality analyses of frequency statistics. On the other hand, more complex methods for detecting signals from large databases have been discussed and developed. Bayesian methods have been used recently to solve potential false positive errors associated with the spontaneous reporting system database. For example, the Bayesian confidence propagation neural network (BCPNN) was developed by the Uppsala Monitoring Center in 1998 and applied to VigiBase.8 Another method, developed by the US Food and Drug Administration (FDA) in 1998, is the Gamma Poisson Shrinker (GPS).9 Statistical signal generation based on disproportionality among reported drug-event combinations is commonly used to prioritize case reports for further review.4 However, only a decade or so has passed since regulatory authorities adopted such quantitative approaches to explore AE case databases.

This study aimed to address whether different databases developed for the same purpose indicated quantitative signals similarly. The targeted comparative databases were the JADER and FAERS. From a pharmacovigilance perspective, the results will help pharmaceutical companies and researchers avoid the pitfalls of spontaneous reporting system databases.

Materials and methods

Database comparison

This study targeted the JADER and Japanese cases in the FAERS, and was approved by the ethics committee at the Keio University, Faculty of Pharmacy. Due to local regulations, the form of reporting and methods of data provision differed among sources (Table 1).10–12 The sources in both databases, ie pharmaceutical companies’ suspected AE reports, are compliant with the International Conference for Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) guidelines.13–16 Thus, the databases basically adhere to ICH-standardized AE information guidelines and provide the main items for each AE case, such as age, sex, medicinal product/substance name, nature of AE, and case outcome. Publicly available items are rather limited when compared with the full items of AE case reports. Most importantly, ICH-defined case identifiers are not available in the FAERS or the JADER. The MedDRA Preferred Term was provided for medical terminology in both databases. Product names in the FAERS have to be converted to substance names; the Japanese agency provides only substance names. The JADER has vaccine and biologic products and provides medicinal product substance names for all cases; some cases in the FAERS include product name only. The FAERS covers both serious and non-serious cases. Companies report AE cases occurring in the USA and non-USA to the FAERS. Companies also report AE cases that occurred in Japan and outside of Japan to the Japanese authority, which only opens Japanese cases.

Table 1

Overview of the Japanese Adverse Drug Event Report and FDA Adverse Event Reporting System

	Japanese Adverse Drug Event Report	FDA Adverse Event Reporting System
Cases available since	April 2004	January 1997
Number of cases	Over 230,000 (from 2004 to 2012)	Over 4,000,000 (from 2004 to 2012)
Data structure of case reports submitted by pharmaceutical companies	ICH compliant	ICH compliant
Form of cases reported by physicians and patients	Pharmaceutical safety information report (health care professionals)Patient adverse drug reaction report	MedWatch (health care professionals, consumers/patients)
Format of shared data	CSV, latest full set	ASCII and SGML, quarterly periodical set
Frequency of update	Quarterly per year	Quarterly per year
Medical terminology	MedDRA	MedDRA
Substance name availability	Regulated text-based substance name	Text-based substance name as reported, text-based product name to be converted to substance name
Targeted products for reporting	Medicinal products, over-the-counter drugs, combination products with medical devices, vaccines, biologics	Medicinal products, over-the-counter drugs, combination products with medical devices
Reporter	Physician, pharmacist, other health care professionals, patients	Physician, pharmacist, other health care professional, patient, lawyer
Case seriousness for company reporting	Serious cases (concomitant non-serious adverse events can be included)	Serious and non-serious cases
Expedited/periodical reports from companies	Expedited reports	Expedited and periodical reports
Original report country	Japan	Worldwide

Abbreviations: CSV, comma-separated value; ICH, International Conference for Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use; MedDRA, Medical Dictionary for Regulatory Activities; SGML, Standard Generalized Markup Language.

Therefore, Japanese cases were considered for data comparison. To evaluate similarities and discrepancies between the datasets and to avoid country-specific AE reporting customs, this study used the FAERS Japanese cases reported in 2010 (FAERS-JP) and all cases reported to the JADER in 2010 (JP). The studied dataset was downloaded in May 2012 from the Pharmaceuticals and Medical Devices Agency and the January to December 2010 FAERS datasets in August 2012. Year was identified as the date on which the FDA received the report and the quarter of the fiscal year in which the Japanese agency received the report provided in the JADER. In addition, as a reference, US cases in the FAERS (FAERS-US) were selected. With technical support from the Japan Pharmaceutical Information Support Center, the FAERS-JP, FAERS-US, and JP were obtained from the respective spontaneous reporting system databases. Although the FDA introduced a Standard Generalized Markup Language dataset in 2013, the old dataset type was investigated to apply experienced data management techniques. MedDRA version 15.0 was used to code Japanese medical terms verbatim and their English translations for the JADER. The center’s coding technique17 helped to translate drug/substance names in English in the FAERS into Japanese for comparison with the Japanese substance names found in the JADER. “Paroxetine hydrochloride” and “paroxetine mesylate” are marketed in the USA, and the ambiguous “paroxetine” was reported to the FAERS as well, but only “paroxetine hydrochloride” is available in Japan. The analysis distinctively used base-level substance names. Note that, due to patient privacy and unknown product names, it is difficult to judge whether a JADER case report matches to one or more case reports in the FAERS.

Detecting signals of disproportionate reporting

In order to compare the number of signals of disproportionate reporting between the FAERS and JADER, the top three common substances frequently reported as primary suspect drugs in 2010 were selected, ie, etanercept, infliximab, and paroxetine hydrochloride. In Japan, etanercept has been on the market since January 2005, infliximab since June 2002, and paroxetine hydrochloride since November 2000; in the USA, these substances have been available since November 1998, August 1998, and December 1992, respectively.

Three methods, ie, ROR, BCPNN, and GPS, were selected for signal detection at the drug-event combination level, with reference to the report on data mining applied to the safety evaluation process at the Japanese agency.18 Table 2 presents the contingency table used in disproportionate reporting signals based on drug-event combinations. The FDA remodeled the GPS to the Multi-Item GPS to stratify data. However, the Multi-Item GPS has an intrinsic risk of system overload, and has not been tested by the Japanese agency. Others have documented the formulas and implications of these methods.19,20 In this study, ROR/exp (1.96 se [standard error]) >1, Information Component −2 SD [standard deviation] >0, and the lower 5th percentile in the posterior distribution (EB05) ≥2 were the thresholds for ROR, BCPNN, and GPS, respectively, where se, SD and EB05 were the standard error of ROR, the standard deviation of the information component and the lower 5th percentile in the posterior distribution. These thresholds were applied and the number of detected signals was compared between the three datasets for each technique. We referred to Japanese label information as of October 2013 to identify any commonly signaled drug-event combinations.

Table 2

Contingency table used in disproportionality analysis

	Specific event Ej	All other events	Total
Specific drug Di	nij (A)	ni+ nij (B)	ni+ (A + B)
All other drugs	n+j − nij (C)	n++ − ni+ − n+j + nij (D)	n++ − nj+ (C + D)
Total	n+j (A + C)	n++ −n+j (B + D)	n++ (A + B + C + D)

Notes: Reporting frequency for the drug-event pairs (D, with D indicating the specific drug and E the specific event) is expressed using the notation as n; n indicates the reporting frequency for the drug–event pairs (D–E; D indicating the specific drug, and E indicating the specific event); n+ indicates the number of reports of a drug D; n+ indicates the number of reports of an event E; n++ indicates the total reporting frequency in the dataset; the methodologies of disproportionate reporting ratio are based on this table, for example, reporting odds ratio is calculated by AD/BC.

Since the patient reporting system started as a trial at the end of March, 2012 in Japan,12 few patient reports were expected in the JADER. Thus, signal detection was also conducted with medically confirmed reports only to reduce reporter bias and compare with the entire original reports.

Results

Descriptive comparison between datasets

Table 3 shows an overview of AE reports in 2010 for comparison. Although the number of reported cases and primary suspected substances in the FAERS-JP was smaller than that in the JP, the number of reported AEs was larger, resulting in 1.6 and 3.3 AEs per case on average in the JP and FAERS-JP, respectively. The datasets provided roughly the same number of drug-event combinations at the preferred term level, while certain combinations showed different trends. AEs looked more severe in the JP for pyrexia, the most frequent AE in the FAERS-JP. Frequency of AEs at the MedDRA system organ class level differed between the JP and the FAERS-JP, for example, in “Blood and lymphatic system disorders”, “General disorders and administration site conditions”, and “Injury, poisoning and procedural complications” (Figure 1). Most of the Japanese cases (94.1% in the JP, 85.6% in the FAERS-JP) were medically confirmed, while consumers or lawyers initially reported one half of the FAERS-US cases. Although the JADER provides expedited reports, 6.2% of the FAERS-JP were periodic reports.

Table 3

Characteristics of spontaneous adverse events reports in 2010 in the Japanese Adverse Drug Event Report and FDA Adverse Event Reporting System

	Japanese Adverse Drug Event Report	Japanese cases among FDA Adverse Event Reporting System	US cases among FDA Adverse Event Reporting System
Number of cases	31,747	18,182	348,295
Sex, male, female, unknown, n (%)	15,902 (50.1%), 15,298 (48.2%), 547 (1.7%)	8,221 (45.2%), 8,272 (45.3%), 1,689 (9.3%)	108,430 (31.1%), 195,092 (56.0%), 44,773 (12.9%)
Number of primary suspected substances	1,150	590	1,697
Top three substances	Sorafenib tosylate, methotrexate, bevacizumab	Levofloxacin, sorafenib tosylate, oseltamivir phosphate	Varenicline tartrate, etanercept, natalizumab
Number of MedDRA-PT	2,893	4,484	9,403
Top three adverse events (MedDRA-PT)	Interstitial lung disease, thrombocytopenia, hepatic function abnormal	Pyrexia, hepatic function abnormal, drug exposure during pregnancy	Nausea, drug ineffective, headache
Number of DEC (number of drugs per PT)	22,752 (7.86)	32,529 (7.25)	210,326 (22.36)
Top three DEC (cases)	Sunitinib malate: reduced blood platelet count (226)	Oseltamivir phosphate: drug exposure during pregnancy (452)	Varenicline tartrate: nausea (8,135)
	Capecitabine: palmar-plantar erythrodysesthesia syndrome (130)	Sorafenib tosylate: palmar-plantar erythrodysesthesia syndrome (259)	Rosiglitazone maleate: myocardial infarction (5,002)
	Ibritumomab tiuxetan: reduced blood platelet count (128)	Sorafenib tosylate: hepatic function abnormal (187)	Nicotinic acid: flushing (4,460)
Average number of reported events per case	1.6 (SD 1.3, maximum 37)	3.3 (SD 3.5, maximum 62)	3.0 (SD 3.68, maximum 127)
Reporter type (medically confirmed cases,* consumers and/or lawyers, unknown [n, %])	29,885 (94.1%), 439 (1.4%), 1,423 (4.5%)	15,572 (85.6%), 979 (5.4%), 1,631 (9.0%)	130,981 (37.6%), 175,700 (50.5%), 41,584 (11.9%)
Type of report from company	Expedited only	Expedited (93.8%) and periodic (6.2%)	Expedited (35.2%) and periodic (56.6%)

Notes:

One of either physician, pharmacist, or other health care professional was involved with report.

Abbreviations: MedDRA, Medical Dictionary for Regulatory Activities; PT, Preferred Term; DEC, drug and event combination; SD, standard deviation.

Figure 1

Proportion of suspected adverse events included in the reports for each MedDRA System Organ Class for 2010 in the USA and Japan.

Abbreviations: JP, Japanese cases reported to the Japanese authority; F-JP, Japanese cases reported to the US authority; F-US, US cases reported to the US authority.

The signals identified for etanercept, infliximab, and paroxetine hydrochloride are shown in Table 4. The JP had the smallest number of cases for each substance. The proportion of signals to drug-event combinations was higher for the JP than for FAERS-JP and FAERS-US; the number of signals for the three substances accounted for between 9.5% and 57.4% of the total number of drug-event combinations in the JP, between 4.3% and 42.1% of the FAERS-JP, and between 4.7% and 40.7% of the FAERS-US. Regarding the two Bayesian methods, the GPS detected a marginally smaller number of signals than did the BCPNN. As “Reporter type” in Table 3 shows that the JP has fewer consumer and/or lawyer reports than do the other datasets, when limited to medically confirmed reports, the detection ratios of the three substances were similar or increased in the JP, similar with a slight increase and decrease in the FAERS-JP, and rose and fell in the FAERS-US.

Table 4

Results of signal detection by BCPNN, GPS, and ROR for etanercept, infliximab, and paroxetine hydrochloride in the Japanese Adverse Drug Event Report database and US Food and Drug Administration Adverse Event Reporting System for 2010

Substance	Dataset (cases)	Method	Whole dataset			Medically confirmed only*
			DEC	Signal	%	DEC	Signal	%
Etanercept	JP (322)	ROR	136	78	57.35	130	79	60.77
		BCPNN	136	19	13.97	130	19	14.62
		GPS	136	16	11.76	130	16	12.31
	F-JP (799)	ROR	397	167	42.07	384	165	42.97
		BCPNN	397	37	9.32	384	36	9.38
		GPS	397	17	4.28	384	16	4.17
	F-US (54,419)	ROR	2,658	550	20.69	2,168	438	20.20
		BCPNN	2,658	287	10.80	2,168	225	10.38
		GPS	2,658	124	4.67	2,168	111	5.12
Infliximab	JP (426)	ROR	179	91	50.84	179	93	51.96
		BCPNN	179	18	10.06	179	18	10.06
		GPS	179	17	9.50	179	17	9.50
	F-JP (844)	ROR	376	140	37.23	376	146	38.83
		BCPNN	376	31	8.24	376	31	8.24
		GPS	376	16	4.26	376	15	3.99
	F-US (6,502)	ROR	1,404	554	39.46	1,052	401	38.12
		BCPNN	1,404	157	11.18	1,052	108	10.27
		GPS	1,404	74	5.27	1,052	54	5.13
Paroxetine hydrochloride	JP (301)	ROR	148	84	56.76	127	79	62.20
		BCPNN	148	27	18.24	127	19	14.96
		GPS	148	20	13.51	127	19	14.96
	F-JP (936)	ROR	382	154	40.31	348	143	41.09
		BCPNN	382	45	11.78	348	39	11.21
		GPS	382	22	5.76	348	20	5.75
	F-US (5,111)	ROR	1,024	417	40.72	725	324	44.69
		BCPNN	1,024	168	16.41	725	122	16.83
		GPS	1,024	153	14.94	725	115	15.86

Notes:

One of either physician, pharmacist, or other health care professional was involved with the report.

Abbreviations: DEC, drug and event combination; JP, Japanese cases reported to the Japanese authority; F-JP, Japanese cases reported to the US authority; F-US, US cases reported to the US authority; ROR, reporting odds ratio; BCPNN, Bayesian Confidence Propagation Neural Network; GPS, Gamma Poisson Shrinker; FDA, US Food and Drug Administration.

Figure 2 shows signal homogeneity. Across substances and signal detection methods, the FAERS-JP had approximately 1.5 to 2 times more signals identified than the JP. Consequently, the intersection ratio of signals between the JP and FAERS-JP was lower than it was in the FAERS-JP. With the Japanese cases in 2010, GPS detected the smallest number of signals. While the JP found the highest intersection rate in BCPNN, GPS identified more common signals than did ROR and BCPNN for each substance in the FAERS-JP. GPS found seven common signals in etanercept, nine in infliximab, and 12 in paroxetine.

Figure 2

Number of signals for etanercept, infliximab, and paroxetine hydrochloride for 2010 in Japan and the USA.

Abbreviations: JP, Japanese cases reported to the Japanese authority; F-JP, Japanese cases reported to the US authority; DEC, drug and event combination; ROR, reporting odds ratio; BCPNN, Bayesian Confidence Propagation Neural Network; GPS, Gamma Poisson Shrinker.

Among the signals commonly identified between the JP and FAERS-JP, only one drug-event combination, ie, etanercept and organizing pneumonia, was unlisted in the Japanese label information. In comparison, for etanercept, infliximab, and paroxetine hydrochloride the FAERS-US provided only five signals of 28 drug-event combinations commonly detected by the JP and FAERS-JP (Table 5); the FAERS-US, however, had the most drug-event combinations and signals.

Table 5

Drug-event combinations commonly signaled by GPS for etanercept, infliximab and paroxetine hydrochloride in the Japanese Adverse Drug Event Report database and US Food and Drug Administration Adverse Event Reporting System for 2010

	Common DECs between JP and F-JP					Common signaled drug-event combinations (n)
	MedDRA preferred term (code)	Listed	EB05			JPF-JP	JPF-US	F-JPF-US
			JP	F-JP	F-US
Etanercept	Extranodal marginal zone B-cell lymphoma (MALT type) (10061850)	W	23.09	15.02	N/A	7	1	2
	Gastric cancer (10017758)	W	5.06	2.16	N/A
	Herpes zoster (10019974)	ADR	2.28	2.24	2.37
	Interstitial lung disease (10022611)	ADR	2.79	2.43	N/A
	Organizing pneumonia (10067472)	U	4.06	4.83	N/A
	Pneumocystis jiroveci pneumonia (10064108)	ADR	2.25	2.38	N/A
	Pneumonia pneumococcal (10035728)	ADR	2.08	2.50	N/A
Infliximab	Disseminated tuberculosis (10013453)	ADR	4.59	20.51	N/A	9	6	5
	Herpes zoster (10019974)	ADR	2.35	2.17	N/A
	Infusion-related reaction (10051792)	ADR	25.56	34.47	42.87
	Lupus-like syndrome (10050551)	ADR	5.46	11.30	42.87
	Peritoneal tuberculosis (10053583)	W	43.44	24.08	N/A
	Pneumocystis jiroveci pneumonia (10064108)	ADR	16.05	15.67	N/A
	Pneumonia bacterial (10060946)	ADR	7.67	2.16	N/A
	Pulmonary tuberculosis (10037440)	W	19.44	28.01	4.16
	Tuberculous pleurisy (10045104)	ADR	20.64	28.05	N/A
Paroxetine hydrochloride	Activation syndrome (10066817)	ADR	52.82	17.95	N/A	12	2	1
	Aggression (10001488)	W	34.11	15.53	N/A
	Anxiety (10002855)	ADR	2.39	7.17	N/A
	Completed suicide (10010144)	W	8.44	2.67	N/A
	Drug withdrawal syndrome (10013754)	W	4.69	27.97	N/A
	Drug withdrawal syndrome neonatal (10013756)	W	2.92	17.45	N/A
	Inappropriate ADH secretion (10053198)	ADR	4.68	2.09	N/A
	Irritability (10022998)	W	2.91	13.56	N/A
	Mania (10026749)	ADR	3.37	2.95	N/A
	Parkinsonism (10034010)	I	8.33	3.08	N/A
	Parkinson’s disease (10061536)	I	16.66	9.76	N/A
	Serotonin syndrome (10040108)	ADR	13.18	11.44	2.45

Abbreviations: ADH, antidiuretic hormone; ADR, listed in the section “adverse drug reaction”; I, interaction; U, unlisted; W, listed in the section “warning”, including contraindications and special populations; JP, Japanese cases reported to the Japanese authority; F-JP, Japanese cases reported to the FDA; F-US, US cases reported to the FDA; FDA, US Food and Drug Administration; N/A, not available; MALT, mucosa-associated lymphoid tissue.

Discussion

Discrepancies between the FAERS and JADER

This comparison study revealed distinct discrepancies in reported drugs, reported AEs, reporter type, seriousness, and average number of reported events per case, between the JADER and FAERS. It is not surprising that the two databases showed different features, because differences can arise as a function of discrepancies in reporting rules and customs, such as reporters and reported AE terms, which are deeply rooted in regulations. For example, the FAERS includes “drug exposure during pregnancy” and “no adverse events” based on regulations; these are not mandatory in Japan. Significant differences in the frequency of features such as “general disorders and administration site conditions” and “investigations” in Figure 1 can be explained by different requirements in case seriousness; US companies need to submit case reports with non-serious AEs, but reporting of known non-serious AEs is not mandatory for Japanese companies.

Social factors and health care systems also have a considerable impact. Regarding descriptive research on claims and spontaneous reporting system databases in Asia, report discrepancies could be caused by the history of each national effort to raise the reporting rate.21 For example, in the JADER, very few reports from non-professionals were found because Japan initiated a reporting system trial derived directly from patients in 2012.12 As another example, interstitial lung disease is the most frequently reported AE in Japan. Japan has experienced serious social concerns with interstitial lung disease related to several drugs; in addition, X-ray imaging available in Japanese clinics and hospitals allows interstitial lung disease to be found more frequently. There might also be coding bias. The use of terminologies and coding systems requires practices and knowledge for users to minimize heterogeneity in reporting and enhance internal and external validity.

Not all Japanese-approved medicinal products are launched in the USA, so fewer Japanese AE cases were reported to the FDA than to the Japanese authority in 2010. Another striking difference was the average number of events reported per case, presumably due to non-serious issues as discussed at the beginning of the section. If companies reported the same case to the two regulatory databases in the same manner, these values should be identical. However, differences in reporting regulations have been infrequently discussed when it comes to medicines used globally. For example, pharmaceutical companies are encouraged to expedite reports for serious AEs accessible through the JADER, whereas unknown non-serious AEs, required for periodical reports, are not typically available there. On the other hand, the FDA stores both expedited and periodic reports in the FAERS, and these country-specific regulations would lead to different qualitative or quantitative assessment results between regulatory agencies. Additionally, if a pharmacovigilance agreement between corresponding US and Japanese companies was not prepared or well written, there would be a risk of underreporting to the FAERS and vice versa.

Thus, different regulatory orientations would lead to different data content with the same data structure. Regardless of globally accepted standards for reporting, differences in contents of case reports are inevitable across countries, causing deviations in spontaneous reporting system databases.

Both databases would benefit from technical support for handling substance names that are recorded in different formats between the FAERS and JADER. Substance name is core information, and it must be identical; however, differences across the regulatory spontaneous reporting system databases remain an unresolved issue. How to handle ambiguous substance names in text or to retrieve names from product information, the substance level/category to be used for analysis, and AE names and levels are important factors for quantitative analysis using drug-event combinations.

Unfortunately, it would be difficult for users of the two databases to avoid duplicate reports, given that multiple companies may report the same case while identifying different medicines as the primary suspected drug. ICH-standardized reporting theoretically can identify “the same case” by using the regulatory authority’s case report number, other sender’s case report number, or other case identifiers in previous transmissions. However, none of these numbers, except for the other sender’s case report number, were provided by either agency. Many studies on electronic health records discuss how duplicates can be identified using patient sex, age, administered drugs, date of administration, and other factors. However, no consensus has been reached on a standardized or recommended method. It also remains uncertain as to how many reports were common between the FAERS and JADER; similar to duplication related issues, it would be most helpful to identify the same case between the FAERS and JADER. In the current situation without identifiers, anonymized patient information and vague product information make identifying cases difficult. As a result, based on their publicity policy, database users cannot currently eliminate duplicates or overlap cases between databases.

Signals of disproportionate reporting

Between the JP and FAERS-JP, the numbers of signals identified through the three methods were different. Bias exists due to subjective decisions in the selection, data mining procedure algorithm, and output selection, system deployment and interpretation, as discussed by Hauben et al,22 this study reconfirmed that the generalizability of results of statistical analyses on spontaneous reporting system data is affected by spontaneous reporting system data. Database profiles, such as reported substances and AEs, could be responsible for this difference. Therefore, this study considered adjustment options resulting in greater similarity in signal detection between the JP and FAERS-JP by conducting sensitivity analysis. Due to limited options of variables, only reporter-restricted signal detection was applied to two datasets for stratification; however, it did not have a significant effect on either dataset. This study also attempted to examine the use of case seriousness and AE seriousness at the individual level, but no such variable was available.

When using the GPS method, the total number of signals in the JP and FAERS-JP was similar for etanercept, infliximab, and paroxetine hydrochloride. Common signals were limited, which is not surprising when using different databases; however, GPS showed the best homogeneity between the two datasets. The results did not fully support the notion that country-specific cases in the FAERS and cases in a national spontaneous reporting system database such as the JADER would be mutually referenced for signal detection, and this certainly deserves increased attention. Nonetheless, the ratios of common GPS signals ranged between 40% and 60% among the JP and FAERS-JP; these GPS signals, as referenced by Japanese labeling, showed high positive sensitivity. This may be illuminating for global companies to plan worldwide risk management, eg, updating product information.

Limitations

This study investigated datasets of the spontaneous reporting system databases of just two countries. This study did not include the Vigibase® because it does not have a sufficient number of Japanese cases.1 This may be an option for comparisons between other countries. There are limitations in demonstrating the association between variances in regulations and databases. Spontaneous reporting system databases mostly depend on the compliance of pharmaceutical companies’ reporting with regulatory requirements. However, each company has its own operational rules for AE reports, which makes it impossible for researchers to validate contents of spontaneous reporting system databases. Therefore, this paper is just one example for reference. Further, bias likely exists relative to the targeted year, which was 2010. Different results may emerge when exploring across other years. With regard to drugs examined by signal detection, selection bias would exist. For example, a gap between the two datasets was observed among the ratio of signals to drug-event combinations. This rough trend is similar to the description in the report by the Japanese regulatory body, providing the ratios of signals to the total drug-event combination numbers in the JADER as of September 2005, which was 6.4% for the GPS.23 However, the number of common signals identified by the GPS (545 of 1,547 drug-event combinations, or 35.2%)18 differs from the results of this study. This may be also due to substance selection and the limited targeted year of reporting.

While signal detection methodology is an important issue, it was not discussed in detail in this paper. Studies on its development and application have been encouraged elsewhere. ROR as used in this study is a rather simple formula used by regulatory authorities.18,21 Rothman et al suggested that ROR could be more appropriate than the proportional reporting ratio for spontaneous reporting data enabling early detection.24 As described by others,18,19 ROR identifies more signals for each substance than do the Bayesian methods, and it was consistent among the datasets. The two Bayesian methods, GPS and BCPNN, are considered useful because each detects unique signals even when there are few AE reports on a certain drug. This study likewise demonstrated that BCPNN and GPS identified fewer signals than did ROR for the three selected substances. Yet, signal detection findings were altered by analysis methods and by database characteristics. A proposal has been made elsewhere to interrelate ROR and other methods with a new parameter,25 and there certainly remains room for close examination with various measures to elaborate upon application of signals of disproportionate reporting.

There are limitations and biases in this study, and we showed that impact assessment of spontaneous reporting system databases for signals of disproportionate reporting remains unsatisfactory. As a result, global companies need to consider spontaneous reporting system databases for country-specific issues. That said, this study will help individuals concerned with pharmacovigilance to better understand the use and pitfalls of using spontaneous reporting system databases and their signal generation. It is important to bear these in mind for pharmacovigilance and risk assessment for drug safety.

Conclusion

The FAERS-JP differed in its detected signals from the JP. Signals derived from both datasets identified different results, but shared certain signals. Discrepancies in AE types, drugs reported, and average number of AEs per case were potential contributing factors. This study will help individuals concerned with pharmacovigilance to better understand the use and pitfalls of using spontaneous AE data.