Changes in suspected adverse drug reaction reporting via the yellow card scheme in Wales following the introduction of a National Reporting Indicator.

AIMS: This study aimed to assess the impact of a National Reporting Indicator (NRI) on rates of reporting of suspected adverse drug reactions using the Yellow Card scheme following the introduction of the NRI in Wales (UK) in April 2014.
METHODS: Yellow Card reporting data for general practitioners and other reporting groups in Wales and England for the financial years 2014-15 (study period 1) and 2015-16 (study period 2) were obtained from the Medicines and Healthcare Products Regulatory Agency and compared with those for 2013-14 (pre-NRI control period).
RESULTS: The numbers of Yellow Cards submitted by general practitioners in Wales were 271, 665 and 870 in the control period, study period 1 and study period 2, respectively. This is equivalent to an increase of 145% in study period 1 and 221% in study period 2 compared with the 12-month control period (2013-14). Corresponding increases in England were 17% and 37%, respectively (P < .001 chi-squared test). The numbers of Yellow Cards submitted by other groups in Wales were 906, 795 and 947 in each of the study periods.
CONCLUSIONS: Introduction of the NRI corresponded with a significant increase in the number of Yellow Cards submitted by general practitioners in Wales. General practitioner reporting rates continued to increase year on year through to 2018-19 with the NRI still in place. No concomitant change was found in reporting rates by other groups in the health boards in Wales.

What is already known about this subject

In the UK, spontaneous reporting of suspected adverse drug reactions to the Yellow Card scheme is an important tool to identify medicines safety issues, but under‐reporting remains a limiting issue.

General practitioners prescribe more medicines than any other professional group, but suspected adverse drug reaction reporting rates by general practitioners remain relatively modest.

A previous study from Wales showed that an educational intervention linked with incentives was effective in increasing reporting in general practice, but the effect was short‐lived after the intervention was withdrawn.

What this study adds

A quality indicator measuring reporting of suspected adverse drug reactions with specified targets (National Reporting Indicator [NRI]) aimed at general practitioners can be associated with increased reporting.

Incentivising reporting, in conjunction with the introduction of an NRI, may result in even greater and sustained suspected adverse drug reaction reporting rates by general practitioners.

The marked increase in reporting by general practitioners in Wales was not offset by a concomitant reduction in Yellow Card reporting rates by other groups.

INTRODUCTION

Adverse drug reactions in the community are an important cause of hospitalisation and, in some cases, death. They were shown to contribute to 6.5% of hospital admissions in adults in one large prospective study in the UK. The Yellow Card scheme is the spontaneous suspected adverse drug reaction reporting scheme introduced in the UK in the wake of the thalidomide tragedy. It is managed by the Medicines and Healthcare Products Regulatory Agency (MHRA) and represents an important post‐marketing mechanism for healthcare professionals, patients, parents and caregivers to report suspected adverse reactions to medicines and medical devices. Reports can be submitted either electronically via the Yellow Card website or smartphone application, or via a paper Yellow Card. A number of adverse drug reactions are only identified from spontaneous reporting after a medicine has been prescribed widely once marketing approval has been granted.

Across the UK, the MHRA is supported by five Yellow Card centres (located in Birmingham, Cardiff, Edinburgh, Liverpool and Newcastle upon Tyne), which are commissioned to increase awareness and education about the importance of reporting suspected adverse drug reactions to the Yellow Card scheme within their regions. Yellow Card Centre Wales was established in 1983 (when it was called the Welsh Adverse Drug Reactions Scheme) at a time that Yellow Card reporting was particularly low in Wales (13.6 per 100 000 population compared with 19.1 in the UK overall). Clinical pharmacologists in Cardiff University School of Medicine and pharmacists in the Welsh Medicines Information Centre worked together through Yellow Card Centre Wales to address this issue, and by 2004 Yellow Card reporting rate had risen to 14.2 per 100 000 population in Wales compared with 23.4 in the UK as a whole. However, it was noted that the number of suspected adverse drug reactions from a vital reporting group, general practitioners in Wales, was declining in the first decade of the 21st century. General practitioners are key reporters and are associated with one of seven health boards in Wales. In 2014–15, general practitioners in Wales prescribed 79 million prescription items, rising to 82.1 million items in 2019–20.

In the European Union, case reports, like Yellow Card reports, were the single most common contributory evidence used to inform regulatory decisions leading to withdrawal of a medicinal product. The MHRA has recently published a list of 25 case studies over the last 7 years where Yellow Card reports directly contributed to the assessment of or helped to identify important patient safety issues. However, under‐reporting of suspected adverse drug reactions has been high and a median under‐reporting rate of 94% (interquartile range 82–98%) was reported across 37 studies included in a comprehensive systematic review. In Wales in 2014–15, there were 365 700 emergency admissions to hospital. Based on the study of Pirmohamed et al it might be anticipated that approximately 5% of these would be due to adverse drug reactions (which by virtue of requiring admission to hospital would be classified as serious). This equates to an estimated 18 200 events that could have been reported using a Yellow Card (notwithstanding any nonserious adverse drug reactions associated with new medicines), compared with only 1460 Yellow Cards received.

Many studies over the last 20 years have shown that a range of educational initiatives are associated with improvements in reporting rates by healthcare professionals, at least in the short to medium term. However, these effects may diminish rapidly when the intervention is discontinued. Similarly, increased reporting in association with the introduction of financial or other incentives has been short‐lived after discontinuation of the incentive. , Khalili and colleagues have summarised the evidence for the effects of awards and financial incentives in nine studies. In just one of the studies was an incentive (which included recognition and extra leave days to individuals) the single intervention, and it was associated with a 4.8‐fold increase in reports over a 12‐month period compared with the previous year. Details of the effect seen by Ali and colleagues are only given for the one 12‐month period, but in a previous study from Ireland the effect of a small financial award had disappeared within 6 weeks of this incentive being discontinued. In the other eight studies described by Khalili and co‐workers, incentives or awards were combined with other interventions such as education and feedback. These authors, like Paudyal and colleagues subsequently, conclude that multifaceted interventions may have more effect than single‐component interventions on the rate of spontaneous reporting of adverse drug reactions by healthcare professionals. Most studies had short follow‐up periods, making it difficult to determine whether any associated improvements were sustained over a longer period. This gap in the literature was something that our study aimed to investigate.

The All‐Wales Medicines Strategy Group is an advisory committee established in 2002 by the Welsh Government to promote safe, effective and cost‐effective prescribing. One of its strategies has been to endorse national prescribing indicators to encourage NHS Wales to focus educational interventions on strategic areas of prescribing. Prescribing indicators (such as those developed by the World Health Organization ) are quantitative measures of prescribing which facilitate comparison and allow benchmarking. They are thought to encourage peer pressure and thus influence prescribing behaviour. National Prescribing Indicators in Wales have focussed on the use of medicines associated with significant adverse effects such as benzodiazepines and tramadol, as well as areas such as antimicrobial stewardship. Yellow Card submissions to the MHRA were included as an analogous National Reporting Indicator (NRI) in Wales for the financial year 2014–2015, with the aim of encouraging a reversal of the decline in suspected adverse drug reaction reporting in general practice. The NRI included two performance measures, one specifically targeting reporting by GP practices and the other targeting overall reporting from Welsh health boards. The NRI remained in place for the financial year 2015–16.

The aim of this study was to assess whether the number of Yellow Cards submitted in Wales changed in the 12 months from April 2014 to March 2015 (study period 1) following the introduction of the NRI in April 2014 compared with the previous 12‐month period, and to assess any further changes in the subsequent 12 months from April 2015 to March 2016 (study period 2). Two of the health boards introduced financial incentive schemes (for the general practice rather than individual practitioners) to encourage reporting, so we also compared the changes in reporting rates in these two health boards with those in the five health boards that did not introduce such schemes.

METHODS

The intervention

The NRI, like the other prescribing indicators, was proposed by a Task and Finish Group after wide consultation within NHS Wales, including with health board chief pharmacists, their medicines management teams and assistant medical directors. National Prescribing Indicators were then distributed for wider consultation with relevant stakeholders in Wales prior to endorsement by the All‐Wales Medicines Strategy Group. The agreed Yellow Card NRI target (A) for general practices was to submit at least one Yellow Card per 2000 practice population per financial year. Target (B) for each health board was to submit Yellow Cards in excess of one per 2000 health board population in a 12‐month period from 1 April. Any Yellow Cards submitted after the end of one financial year (31 March) would have been included in the data for the subsequent year. Details of the background to and evidence for adopting the NRI, together with information on useful educational resources and relevant websites were sent to health boards and general practices in January 2014. Feedback was given on a quarterly basis throughout the 24‐month period to facilitate benchmarking across health boards (April 2014–March 2016).

In one of the seven health boards (HB1), the Yellow Card reporting target (with supporting information) was incorporated into its general practice prescribing management scheme. In addition to the support provided to all health boards by Yellow Card Centre Wales, information on medicines safety and reporting (including regular MHRA updates) was provided to general practice prescribing leads and at annual prescribing visits, and email updates were also sent to practices. General practices in HB1 (not individual general practitioners) received payments for achieving the targets in both study periods. In another health board (HB2), Yellow Card reporting was added in 2015–16 (study period 2) to the “quality” element of their already existing voluntary medicines management incentive scheme. Each independent prescriber in the community (medical and nonmedical) was expected to complete at least one Yellow Card report each year. This target remained in the incentive scheme up to and including the financial year 2018–19.

Data

The number of Yellow Cards submitted by general practitioners in Wales in the NHS financial year 1 April 2013 to 31 March 2014 (prior to the introduction of the NRI) and the succeeding NHS financial years 2014–15 and 2015–16 (after the introduction of the NRI) were obtained from Yellow Card Centre Wales. Data on the number of Yellow Cards submitted by general practices in England, along with the total number of cards submitted in England over the same periods were obtained from the MHRA. The total number of Yellow Card submissions received from non‐general practitioner reporters in the Welsh health boards over the same periods was also obtained from Yellow Card Centre Wales. This group was made up predominantly of hospital and community pharmacists, patients/parents/carers and hospital nurses. Patients/parents/carers were allocated to health boards by the postcode of the reporter.

The data were examined both in total numbers and when corrected for the populations of Wales and England, since England is a more populated country. The population estimates used were mid‐year estimates for 2013, 2014 and 2015, and were 53 865 800, 54 316 600 and 54 786 300, respectively, for England and 3 082 400, 3 092 000 and 3 099 100, respectively, for Wales.

The MHRA Yellow Card reporting guidelines state that for established medicines and vaccines all serious suspected adverse drug reactions should be reported, even if the effect is well recognised. Serious suspected adverse drug reactions are those which are fatal, life‐threatening disabling, incapacitating, have resulted in or prolonged hospitalisation, are medically significant or have resulted in congenital abnormalities. Data on the number of reports classified as serious were obtained from the MHRA. The Commission on Human Medicines and the MHRA also encourage the reporting of all suspected adverse drug reactions to newer medicines and vaccines, which are under additional monitoring (denoted by an inverted black triangle symbol, ▼) to aid early identification of possible new safety issues. These data were not available separately for general practitioners and other reporters, so the data were presented in aggregate form.

Statistical analysis

The number of Yellow Cards submitted in Wales and England prior to the introduction of the NRI in 2013–14 was compared with the number submitted in 2014–15 and in 2015–16. Both the number of reports from general practitioners, and the total number of reports from all reporters were compared. Yellow Card reports from Wales were further analysed according to whether they were classified as being “serious” or “non‐serious” according to MHRA criteria, and whether or not the HB from which the report originated had introduced an incentive scheme to reward general practices for reporting suspected adverse drug reactions. Comparisons of categorical data used chi‐squared analysis conducted in GraphPad Prism version 5.00 for Windows (GraphPad Software, La Jolla, CA, USA). The minimum level of statistical significance was taken to be P < .05. A post hoc analysis investigating the number of Yellow Cards submitted in the 4 years after study period 2 (up to and including the financial year 2019–20) was conducted to assess whether any change in reporting was maintained over an extended period.

Ethics statement

This study did not include any patient identifiable information, therefore ethical approval was not required. This study was part of a wider project that was approved by the researchers' LHB research and development department (reference: 14/CLC/5882).

RESULTS

General practitioner submissions

The numbers of Yellow Cards submitted by general practitioners in Wales and England during the control period (12 months from 1 April 2013 to 31 March 2014) and the subsequent two 12‐month study periods are shown in Table 1. In 2014–15 and 2015–16, 665 and 870 Yellow Cards were submitted by general practitioners in Wales. This represents increases of 145% and 221% compared with control in the two 12‐month study periods, respectively. The corresponding increases in reporting in England were 17% and 37%, respectively. Of the submissions from general practitioners in Wales, 111 (41%), 268 (40%) and 286 (33%) were considered serious in 2013–14, 2014–15 and 2015–16, respectively.

TABLE 1

Yellow Card reports submitted by general practitioners in England or Wales and reports classified as “serious”

	Total number of suspected ADRs reported		Chi‐squared analysis vs control period	Reports classed as “serious”	Reports classed as “non‐serious”	Chi‐squared analysis vs control period
	GPs in England	GPs in Wales		GPs in Wales	GPs in Wales
Control period (2013–14), n (%)	3629 (93)	271 (7)		111 (41)	160 (59)
Reports/100000 population	6.74	8.79		3.60	5.19
Study period 1 (2014–15), n (%)	4231 (86)	665 (14)	Χ 2 = 100.5 P < .0001	268 (40)	397 (60)	Χ 2 = 0.03 NS (P = .85)
Percentage increase c/w control period	+16.6	+145.4		+141.4	+148.1
Reports/100,00 population	7.79	21.51		8.67	12.84
Study period 2 (2015–16), n (%)	4980 (85)	870 (15)	Χ 2 = 142.2 P < .0001	286 (33)	584 (67)	Χ 2 = 5.96 P = .015
Percentage increase c/w control period	+37.2	+221.0		+157.7	+265
Reports/100000 Population	9.09	28.07		9.23	18.84

Abbreviations: ADR, adverse drug reaction; c/w, compared with; GP, general practitioner; NS, non‐significant.

Two health boards introduced Yellow Card reporting incentive schemes for general practitioners in association with the introduction of the NPI (see Table 2). One health board (HB1) had a scheme in the first study period (2014–15), which continued into the second study period (2015–16), and one (HB2) had an incentive scheme in place only in 2015–16. In HB1, there was a 387.5% increase in general practitioner reports in the first study period and in HB2 the corresponding change in the first year (when no incentive scheme was in place) was −38%. The increases in reports in the second study period (when an incentive scheme was in place in both HB1 and HB2) compared to the control period were 352% and 381% in HB1 and HB2, respectively. Yellow Cards submitted by general practitioners in health boards with an incentive scheme at any point in the study as a percentage of the total Yellow Cards submitted were 33%, 39% and 48% in the control period, study period 1 and study period 2, respectively. The mean increases in reports from the remaining five health boards without an incentive scheme in either study period were 123.8% in study period 1 and 149.2% in study period 2, both being significantly greater than the increase in rates in England in the corresponding time periods.

TABLE 2

Yellow Cards submitted by general practitioners in Wales in health boards with or without incentive schemes

	By GPs in two HBs where NRI also incentivised at some point in the study			GPs in Wales HBs with no incentive scheme at any point in study	Chi squared analysis vs control period
	HB1 (with incentive scheme in place in both study periods)	HB2 (with incentive scheme in place in study period 2 only)	HB1 and HB2 (both with incentive schemes in study period 2)	Wales HBs 3–7
12‐month control period (2013–14), n (%)	48	42	90 (33)	181 (67)
First 12‐month study period (2014–15), n (%)	234 (54)	26	*NA	405	Χ 2 = 0.33 NS (P = .56)
Percentage increase c/w control period	+387.5	−38.1		+123.8
Second 12‐month study period (2015–16), n (%)	217	202	419 (48)	451 (52)	Χ 2 = 18.7 P < .0001
Percentage increase c/w control period	+352.1	+381	+365.6	+149.2

Note: Grey shaded cells indicate when an incentive scheme was in place.

Abbreviations: c/w; HB, health board; NA, not applicable (no incentive scheme in place in HB2); NRI, National Reporting Indicator; NS, not significant.

In relation to the NRI target, 18.6% of general practices achieved target A (one Yellow Card submitted per 2000 practice population) in study period 1 and 23.0% in study period 2 (Figure 1). The reporting rates continued to rise for the following three financial years after the study, but fell in 2019–20 (see Figure 2).

FIGURE 1

Percentage of general practitioner practices in each health board achieving NRI target A (one Yellow Card submitted per 2000 practice population) in the control period and study periods. HB1 and HB2 had reporting incentives in either or both study period 1 and study period 2

FIGURE 2

Number of Yellow Cards submitted by general practitioners and other reporters in Wales (2013–14 to 2019–20)

Health board submissions from other reporting groups in Wales

In the control period, study period 1 and study period 2 there were 906, 795 and 947 Yellow Cards, respectively, submitted by reporters other than general practitioners in Wales. Of those reports, 64%, 54% and 50% of the submissions were classified as serious in the control period, study period 1 and study period 2, respectively.

Since general practitioner reports rose markedly during the study, the proportion of Yellow Cards submitted by other reporters fell from 77% in the control period to 54.5% in study period 1 and 52.1% in study period 2. The reporting rates by non‐general practitioner reporters in the health boards remained relatively unchanged for the three financial years after the study, whereas general practitioner reporting continued to increase (see Figure 2). As a result, this group of reports only constituted 33.3% of the total reports submitted in 2018–19.

Medicines subject to additional monitoring

Medicines subject to additional monitoring (denoted by an inverted black triangle symbol, ▼) were a suspected medicine on 128 occasions in the control period, 239 in study period 1 and 436 in study period 2 from all reporters. Expressed as a proportion of all reports, they appeared in 10.9%, 16.4% and 24.0% of total reports, respectively, in the three consecutive periods of the study.

DISCUSSION

The number of Yellow Cards submitted by general practitioners increased significantly in both study periods after the NRI was introduced compared with the control period. It was also significantly greater than the corresponding changes in England, where an NRI was not introduced during the study periods. Whilst the timing of the increase in reporting coincided with the introduction of the NRI, causation cannot be directly imputed. However, general practitioner reporting rates (expressed per million population), which were already 30.4% higher in Wales than in England in the control period, became 176.1% and 208.8% higher than in England in study periods 1 and 2, respectively. The increase in general practitioner reporting rate in Wales (149.2% in study period 2) compared with England (37.2%) was still significantly greater in Wales if data from the two health boards which had an incentive scheme in place were excluded. This indicates that the NRI itself (in the absence of an incentive) may have been an important factor associated with the increased reporting rates. We are not aware of any previous studies using an NRI approach to influence suspected adverse drug reaction reporting.

Despite the reduction in the proportion of general practitioner reports classed as serious during study period 2 (from 40.6% to 33.6%), the increase in reporting rate meant that the total number of reports classified as serious submitted by general practitioners increased by 141% in study period 2 compared with the control period. While serious suspected adverse drug reaction reports to all medicines or vaccines are encouraged by the MHRA, other valid reports (ie, meeting the MHRA guidelines) are also solicited. Therefore, it is encouraging to note that from all reporters, the proportion of reports for medicines subject to additional monitoring denoted by an inverted black triangle symbol (▼) more than doubled in study period 2 compared with the control period (from 10.9% to 24.0%). If this were mirrored in general practitioner reports, it would provide a further indication that the increased overall number of Yellow Cards in the study was associated with an increased number of reports deemed valid by the MHRA. However, due to the nature of the aggregate data used in this study it was not possible to investigate this nor other specific aspects of the reported suspected adverse drug reactions further. This was a limitation of the study.

The findings indicate that incentive schemes are not the only effective intervention, and the educational value of the focus on the NRI as a quality indicator with associated regular feedback (to allow benchmarking) may also have been associated with increased reporting. However, incentive schemes require targets and both incentivised health boards used the targets associated with the NRI, so incentives do not stand in isolation. It is interesting to note that in HB2, which introduced an incentive scheme only in study period 2, increases in general practitioner reporting did not occur until that second period. In contrast, in HB4, without an incentive scheme, reporting rate increases by general practitioners were of the same magnitude as in the two health boards with such a scheme in place at some point in the study. This suggests that the NRI initiative itself may have encouraged a focus on Yellow Card reporting rates and how to improve them. The approximately three‐fold increase in overall reporting rates in Wales through to 2018–19 was driven almost completely by the increased number of general practitioner reports, and reports from other groups remained relatively unchanged. These other groups consist of pharmacists, nurses and other healthcare professionals (HCPs) working in primary and secondary care, and patients. We are exploring how to encourage these groups to report as appropriate, and Yellow Card Centre (YCC) Wales has begun to appoint Yellow Card Champions in primary care to seek to emulate the impact of those previously appointed in the secondary care sector in Wales.

In a previous study of general practitioner reporting in Wales, a distance‐learning educational programme in pharmacovigilance linked to educational credits was associated with improved reporting via the Yellow Card scheme. However, the effect had attenuated in the year following the 12‐month study when the intervention was withdrawn. This time‐limited effect was consistent with the findings from a study examining incentives in Ireland. In the present study it was encouraging to note that when financial incentives had been discontinued in one health board, but the NRI, accompanied by quarterly feedback, continued to be provided, overall reporting rates continued to increase year on year for a further 3 years (see Figure 2). It is possible that the regular feedback showing comparative performance of general practices may have continued to stimulate peer pressure to drive quality improvement. As a result of these findings, the NRIs were retained and continue until the present time. Whilst the general practitioner reporting rate increased for a further 3 years after study period 2, the increase in reporting from 2017–18 to 2018–19 was smaller than that seen in previous years, and it fell slightly in 2019–20 (Figure 2). The last quarter of 2019–20 coincided with the COVID‐19 pandemic, and the resulting changes to healthcare provision may have accounted for this reduction. However, it also coincided with the discontinuation of financial incentives in the remaining health board, perhaps reflecting the challenge of maintaining the effect of the intervention.

In conclusion, we have observed that the introduction of an NRI highlighting Yellow Card reporting as a quality indicator in primary care coincided with an increase in reporting rate. The NRI was continued and improvements in general practitioner reporting rates were seen for at least a further 3 years after the prospective study had been completed. The NRI in Wales may have helped to build a “reporting culture” so that healthcare professionals consider reporting suspected adverse drug reaction (ADRs) as an integral component of their professional responsibilities, despite the barriers to ADR reporting associated with their busy and increasingly demanding workplace roles. This approach (an NRI linked to incentive schemes) may be of value in other health systems elsewhere in the UK and internationally to improve suspected adverse effect reporting rates.

COMPETING INTERESTS

None of the authors has any conflict of interest to declare in relation to this work.

CONTRIBUTORS

P.N.D., R.B., K.H. and P.A.R. designed the study, P.N.D., K.J., A.T. and A.A. obtained the data, P.N.D., E.C., J.W. and P.A.R. analysed the data. P.N.D. and P.A.R. wrote the first draft and all authors contributed by reviewing and editing. All authors agreed to the final version of the manuscript prior to submission.