The Quality of Clinical Information in Adverse Drug Reaction Reports by Patients and Healthcare Professionals: A Retrospective Comparative Analysis.

INTRODUCTION: Clinical information is needed to assess the causal relationship between a drug and an adverse drug reaction (ADR) in a reliable way. Little is known about the level of relevant clinical information about the ADRs reported by patients.
OBJECTIVE: The aim was to determine to what extent patients report relevant clinical information about an ADR compared with their healthcare professional.
METHODS: A retrospective analysis of all ADR reports on the same case, i.e., cases with a report from both the patient and the patient's healthcare professional, selected from the database of the Dutch Pharmacovigilance Center Lareb, was conducted. The extent to which relevant clinical information was reported was assessed by trained pharmacovigilance assessors, using a structured tool. The following four domains were assessed: ADR, chronology, suspected drug, and patient characteristics. For each domain, the proportion of reported information in relation to information deemed relevant was calculated. An average score of all relevant domains was determined and categorized as poorly (≤45%), moderately (from 46 to 74%) or well (≥75%) reported. Data were analyzed using a paired sample t test and Wilcoxon signed rank test.
RESULTS: A total of 197 cases were included. In 107 cases (54.3%), patients and healthcare professionals reported a similar level of clinical information. Statistical analysis demonstrated no overall differences between the groups (p = 0.126).
CONCLUSIONS: In a unique study of cases of ADRs reported by patients and healthcare professionals, we found that patients report clinical information at a similar level as their healthcare professional. For an optimal pharmacovigilance, both healthcare professionals and patient should be encouraged to report.

Key Points

Introduction

Pharmacovigilance is the science about ‘the detection, assessment, understanding and prevention of adverse effects or any other drug related problems’ [1]. Due to the design of pre-marketing clinical trials, i.e., small and homogeneous populations monitored for short periods of time, not all possible adverse drug reactions (ADRs) are detected. Additional ADRs, some of them serious, may be identified once a drug is used more widely and under more diverse conditions, e.g., concurrent use with other drugs or problems in using drugs by patients [2].

Pharmacovigilance centers maintain the national spontaneous reporting systems. Spontaneous reports of possible ADRs are a valuable source of information, e.g., in the USA, spontaneous reports were the primary evidence source of drug safety issues resulting in drug safety communication from 2007 to 2009 [3]. Traditionally, reporting of possible ADRs was reserved for healthcare professionals. Only a few countries allowed patients to report their ADRs directly, for example, Australia since 1964 and the USA since 1969 [4]. Over the years, patient participation has increasingly been recognized as an important addition to pharmacovigilance [5, 6]. Studies demonstrated that they contributed to identifying new ADRs as well as new information about known ADRs [7-9]. More and more countries have started to accept ADR reports directly from patients, for example, the Netherlands in 2003, the UK in 2005 and Sweden in 2008 [4]. Since 2012, changes in the European pharmacovigilance legislation made it possible for patients of all European member states to report drug concerns directly to the national pharmacovigilance centers [5].

A recent review showed that patient reporting adds new information and perspectives about ADRs in a way otherwise unavailable, for example, information about the impact of ADRs on the patient’s daily life. It also identified gaps in knowledge that should be addressed to improve our understanding of the full potential and drawbacks of patient reporting [10]. One of these aspects is the quality of clinical information. To assess the causal relationship between exposure to a drug and an ADR in a reliable way, clinical information is needed [11]. Studies that have compared information reported by patients and healthcare professionals have so far focused on the completeness of information [12-24]. When it comes to causality assessment, an additional often ignored point of attention is the relevance of the clinical information provided. When a report lacks essential clinical information, it is difficult to assess the reported data. In contrast, a brief report can still provide sufficient clinical information if all relevant information has been reported for that specific case.

As far as we are aware, it has not been studied to what extent patients report relevant clinical information compared with health professionals, in particular, clinically relevant information needed to make causal assessments. The study aims to determine to what extent patients report relevant clinical information about an ADR compared with their healthcare professional.

Method

Study Setting and Design

We used the database of the Dutch Pharmacovigilance Center Lareb. Both patients and healthcare professionals are able to report possible drug concerns directly to Lareb by means of an electronic or paper reporting form. These forms contain standardized questions, of which some are mandatory in the electronic form. Also, reporters can give additional information in a free-text field. Both reporting forms obtain the same information, with the exception of a question about medical history, which is only present on the healthcare professionals reporting form. Reports from patients and healthcare professionals are handled in the same way for the case-by-case analysis, follow-up actions and signal detection.

The number of reports to the Dutch pharmacovigilance center continues to grow. In 2015, Lareb received about 8000 reports directly from patients and 6600 from healthcare professionals [25]. In the majority of cases, the ADR is either reported by the patient or the healthcare professional. Rarely, the patient and the patient’s healthcare professional send reports independently on the same case. For this study, we conducted a retrospective analysis of all reports on the same case, i.e., reported by the patient and the patient’s healthcare professional. This provided us the unique situation to directly compare the differences in clinical information reported by both groups.

Cases were identified as follows: all incoming reports were assessed case by case by a trained pharmacovigilance assessor. During this assessment, the reports were automatically screened for other reports on the same case by checking the reported ADR (based on the Medical Dictionary for Regulatory Activities MedDRA Higher Level Term coding) [26], suspected drug, and patient’s date of birth and gender, using a time frame maximum of 1 year between both reporting dates. Using these data, the pharmacovigilance assessor determined if the reports were on the same case and labeled them accordingly in the database. This procedure is part of routine screening of all reports in the pharmacovigilance center Lareb.

Study Population

All cases of reports that were made on the same case in the period April 1, 2003 until October 1, 2015 were selected from the Lareb database. When a case had more than two reporters, e.g., one patient report and two healthcare professional reports, the case was included twice: patient versus healthcare professional 1 and patient versus healthcare professional 2. Exclusion criteria were as follows: all cases not including a patient report or a healthcare professional report; any cases received through pharmaceutical companies, since these were not directly sent to Lareb (e.g., other reporting forms may be used).

Outcomes

Our primary outcome was a comparison of the level of reporting of clinical information between patients and healthcare professionals. This was determined using a clinical documentation tool (ClinDoc) [27]. This tool was recently developed and tested by Lareb as part of the WEB-RADR project, work package 4 [28]. It provides a structured approach to assess the level at which relevant clinical data has been reported. Four domains were assessed: (1) description of the ADR, (2) chronology of the ADR, (3) suspected drug, and (4) patient characteristics. Each domain consisted of several subdomains (Table 1). To use this tool, first, the assessor indicated which subdomains were relevant in order to assess the report. Subsequently, the assessor indicated if this relevant information was present or absent. A score was calculated for each domain by dividing the number of subdomains with information present by the number of subdomains deemed relevant. The final score was the sum of the domain scores of all domains deemed relevant. The final score was categorized into one of three reporting level categories: well (≥75%), moderately (46–74%) or poorly (≤45%).

Table 1

The clinical documentation tool

1	Adverse drug reaction (ADR)	Relevant?Yes, no	Present?Yes, no
a	Proper description of the ADR
b	Specification reaction ‘localization’ and ‘characterization’
	To strengthen the diagnosis (subdomain c or d or e applicable):
c	Treatment; or
d	Visual material (photo, video); or
e	Lab values, test

As a secondary outcome, we explored whether the proportions of information present in relation to the information deemed relevant was different for the individual (sub)domains. Because differences in the level of reporting for serious versus non-serious cases may be expected, we did a sub-analysis for (non)serious cases. Seriousness was assessed according to Council for International Organizations of Medical Sciences (CIOMs) criteria, which include ADRs leading to (prolongation of) hospitalization, life-threatening events, reactions leading to death, disabling events or congenital abnormalities or other events considered serious by medical judgment [29].

All included reports were scored by two pharmacovigilance assessors independently. All reports were reformatted so that the assessors were kept blind as to whether reports originated from a patient or a healthcare professional. In total, six experienced pharmacovigilance assessors were involved. Reports about the same case, i.e., the report of the patient and the one of the healthcare professional, were scored by the same assessors but were presented to them at random. Differences between scores for each domain were discussed until consensus was reached. Prior to scoring, all assessors were trained how to use the ClinDoc tool by means of scoring and discussing 15 reports.

Statistical Analysis

General characteristics of the included cases were explored using descriptive statistics. We used a paired sample t test for normally distributed data and a Wilcoxon signed rank test for non-parametric testing. Data normality was tested graphically using a histogram and numerically using Shapiro–Wilk test and a test for skewness. Statistical significance was based on p < 0.05. Data were analyzed using the statistical software program SPSS Statistics, version 22.0 (SPSS, Chicago, IL, USA).

Results

General Information Sample Characteristics

We included 197 cases with a report from the patient and their healthcare professional. There was one case reported by the patient and two healthcare professionals. All the other cases contained one patient and one healthcare professional report. A report may contain several ADRs. In total, 227 ADRs were reported by both reporters, with most ADRs belonging to the System Organ Classes ‘Nervous system disorders,’ ‘Psychiatric disorders,’ ‘Gastrointestinal disorders’ and ‘Skin and subcutaneous tissue disorders.’ Of the reported cases, 66 (33.5%) were classified as serious, according to CIOMs criteria [29]. Two examples of the description of information given by patients and healthcare professionals are provided in Table 2.

Table 2

Summaries of two examples to demonstrate the differences and similarities in reporting

Example	Patient	Healthcare professional
1	Male aged 40 years with rhabdomyolysis, CK >10,000 two weeks after start of paroxetine 20 mg, twice a day. The patient was hospitalized. The drug paroxetine was withdrawn; the patient has not recovered. Concomitant medication was reported, including start dates. Furthermore, it was reported that the patient was severely ill, could barely walk, and had pain everywhere	Male aged 40 years with rhabdomyolysis 6 weeks after start of paroxetine for depression. The patient was hospitalized. The drug paroxetine was withdrawn, and the patient was treated with an unknown infusion. The rhabdomyolysis recovered. The patient is of Moroccan origin. Kidney function was normal. Furthermore, no other laboratory abnormalities
2	Female aged 71 years with a definitive loss of taste and smell 1 month after start of lisinopril 5 mg for high blood pressure. The drug lisinopril was withdrawn; the patient has not recovered. The loss of taste and smell suddenly started from 1 day to the other. The patient was examined by a neurologist, but he could not help her. When she ate, she felt like she was chewing on paper. Due to this, she lost body weight. Concomitant medication was reported, including the comment that she had used this drug for years without any problems. Furthermore, it was reported that these complaints are a very serious handicap, especially for an elderly patient	Female aged 71 years with anosmia and loss of taste 1 month after start of lisinopril for hypertension. The drug lisinopril was withdrawn. The patient only slightly recovered. There were no other possible causes for the anosmia and loss of taste. Concomitant medication and patient’s medical history were not reported

CK creatine kinase

For all reports, the two assessors agreed on the level of clinical information for a mean of eight reports (range 6–11). For cases that were scored differently, the level of clinical information mostly differed by one category. Only two assessors had one report for which the score differed by two categories. Differences between scores for each domain were discussed until consensus was reached.

Overall Reporting of Clinical Information

Of all cases, for 107 (54.3%), the patient and the healthcare professional reported the clinical information at the same level. If the level was different, in most cases (87.8%), reports differed by only one category (well vs. moderate or moderate vs. poor) and rarely (12.2%) by two categories (well vs. poor). For 34 cases (17.3%), the patient scored one category higher compared with their healthcare professional. For four cases (2.0%), the patient scored two categories higher. For 45 cases (22.8%), the healthcare professional scored one category higher compared with the patient, and for seven cases (3.6%), the healthcare professional scored two categories higher (Table 3a). Wilcoxon signed rank test demonstrated no statistically significant difference in category between groups (p = 0.126). Similar results were obtained when analyzing serious and non-serious cases separately (respectively, p = 0.196 and p = 0.356). For serious reports, 29 (43.9%) from patients and healthcare professionals on the same case were classified in the same category. For non-serious reports, this number was 78 (59.5%) (Table 3b, c).

Table 3

Level of reporting of clinical information patients vs. healthcare professionals, paired analysis

	Healthcare professional
	Well	Moderate	Poor	Total
(a) All reports (serious and non-serious)
Patient
Well	72	31	4	107
Moderate	45	33	3	81
Poor	7	0	2	9
Total	124	64	9	197
(b) Serious reports
Patient
Well	20	12	1	33
Moderate	19	9	2	30
Poor	3	0	0	3
Total	42	21	3	66
(c) Non-serious reports
Patient
Well	52	19	3	74
Moderate	26	24	1	51
Poor	4	0	2	6
Total	82	43	6	131

Differences in Domains Scores

For the domains ‘ADR,’ ‘chronology’ and ‘suspected drug,’ patients and healthcare professionals scored in about 40% of cases similarly (i.e., scores differed less than 10%) (Fig. 1). Healthcare professionals had higher scores for the domain ‘patient characteristics’ and probably therefore also had more often higher final scores. It has to be noted that the domain ‘drug’ was found to be relevant in only 13 cases (6.6%).

Fig. 1

Number of reports with similar and deviating scores, per domain for patients and healthcare professionals

Paired sample t test and Wilcoxon signed rank test showed that healthcare professionals had a statistically significantly higher score for the domains ‘patient characteristics’ and again probably therefore a higher final score. The mean difference of the percentage score for these domains was, however, found to be small, 65.7 versus 57.1% (p = 0.003) for ‘patient characteristics’ and 77.9 versus 74.7% (p = 0.04) for final score.

When the same analysis was performed using only the serious cases, healthcare professionals had a statistically significant higher score for the domains ‘ADR’ and ‘patient characteristics.’ The mean difference for the domain ‘ADR’ was small, 84.2 versus 75.6% (p = 0.02). For the domain ‘patient characteristics,’ the mean difference was 66.1 versus 55.5% (p = 0.04). When the analysis was performed using only the non-serious cases, healthcare professionals had a statistically significant higher score for the domain ‘patient characteristics.’ The mean difference was, however, small, 58.1 versus 65.3% (p = 0.03).

Differences in Subdomain Scores

The subdomain ‘concomitant medication’ (a subdomain of the domain ‘suspected drug’) was statistically significantly more often reported by healthcare professionals than patients (75 vs. 63.5%, p = 0.017). For the other subdomains, no statistically significant differences were found.

Remarkable findings were that ‘visual material,’ ‘lab values, tests’ and ‘patient’s life style and other risk factors’ were infrequently documented by both groups. In cases where these subdomains were considered to be relevant, respectively, 19, 25 and 20% of the patient reports and 20, 39 and 25% of the healthcare professional reports contained this information.

Discussion

Healthcare professionals and patients reported clinical information about the ADR on a comparable level for over half of the cases. For only one third of all cases, the patient had a lower score compared to their healthcare professional. Vice versa, patients had higher scores for almost one fifth of the reports. Rarely, we found large differences in the level of reporting of relevant information. Items included in the clinical documentation tool reflect items that are important for causality assessment. The results found in this study indicate that reports from patients are comparable to those of healthcare professionals when it comes to making a proper causality analysis.

Healthcare professionals more often reported information concerning ‘patient characteristics,’ but given the mean difference of 8.6%, we considered this finding negligible for daily pharmacovigilance practice. We saw the same pattern when analyzing serious reports separately. However, for these cases, healthcare professionals scored the domain ‘patient characteristic’ significantly higher compared with patients, with a mean difference of 10.6%. Healthcare professionals might see more need to provide this type of information. Furthermore, in cases of hospitalization or death, healthcare professionals may include the hospital discharge letter with their report. This letter provides information about patient characteristics. For patients, this hospital discharge letter is mostly not available.

Previous research about patient versus healthcare professional reporting demonstrated that overall, healthcare professionals reported more information related to the suspected drug, e.g., drug dosage and route of administration [21]. In the present study, information concerning the suspected drug was only relevant in a limited number of cases, such as a ‘brand name in case of an ADR after drug substitution.’ For these cases, mostly one subdomain was relevant for assessment of the report. Therefore, when this subdomain was present in the healthcare professional report (score of 100%) but lacking in the patient report (score of 0%), this resulted in a difference of 100%. Consequently, the mean difference (30.8%) seems to be large but has no practical relevance.

As far as we are aware, this is the first study to use reports from the patient and the patient’s healthcare professional on the same case. Due to this unique approach, we were able to directly compare the differences in clinical information reported by both groups. There may have been some selection bias, as a report had to be ‘interesting’ enough for both patients and healthcare professionals to report it independently. The motivation or reason for reporting has to be considered when exploring to what extent our results are generalizable to reports of the Lareb database as well as to other pharmacovigilance centers. Healthcare professionals as well as patients report because of the severity of the reaction and wanting to contribute to medical knowledge [30]. Patients also report because they felt their complaints were not taken seriously elsewhere or because they already reported the ADR to a healthcare professional with no result [30]. Unfortunately, we have no data on motives for reporting in the Lareb database. Regarding the generalizability, the overall characteristics, male–female ratio and reported ADR (based on System Organ Class classification) of the included reports are in line with previous studies [12, 15, 16, 19, 22, 30–35]. Not surprisingly, our study set concerned 33.5% serious reports, which is a higher percentage than the average percentage of serious reports present in the Lareb database (average of 20% serious healthcare professional reports and 18% patient reports, from 2013 to 2015) [36]. Finally, we do not know to what extent the healthcare professional and patient discussed the case and whether this had an influence on the level of information reporting. Due to these biases, results should be generalized with caution.

Some methodological issues have to be addressed. In order to analyze the level of reporting of clinical information, we used the ClinDoc tool [27]. This tool determines which information is relevant for a case and then assesses whether relevant information has been reported completely. Even though we used a standardized method of assessment, the level of clinical information remains a somewhat subjective measure, but using a structured approach was better than subjectively comparing reports of patients and healthcare professionals. For the present study, we tried to minimize variations between assessors by training assessors how to use the tool. Furthermore, each report was scored by two assessors individually and differences between domain scores were discussed until agreement was reached. In order to keep assessors ‘blind’ about the type of reporter (patient or healthcare professional), we had to remove some identifying information.

Reports by patients and healthcare professionals reflect their own experiences and perceptions of the ADR. The present study specifically compared the level of reporting of clinical information. We did not capture all possible information that can be reported in our study. Others, for example, showed that patients report more about the impact of the ADR on their daily life compared with healthcare professionals [19, 20, 37, 38]. This information is also valuable for pharmacovigilance practice. In our view, reports of both patients and healthcare professionals can contribute to optimal pharmacovigilance.

Conclusion

In a unique study of cases of ADRs reported by patients and healthcare professionals, we found that patients report clinical information at a similar level as their healthcare professional. For an optimal pharmacovigilance, both healthcare professionals and patient should be encouraged to report.

Studies have demonstrated that patients are well capable of reporting possible ADRs. Little is known to what extent patients report relevant clinical information compared to their healthcare professional when reporting ADRs to a pharmacovigilance center.

Analyzing cases of ADRs, reported by patients and their healthcare professionals, we found that the level of reporting relevant clinical information was similar between both groups.