Effectiveness of medicines authentication technology to detect counterfeit, recalled and expired medicines: a two-stage quantitative secondary care study.

OBJECTIVES: To identify the authentication and detection rate of serialised medicines using medicines authentication technology. DESIGN AND INTERVENTION: 4192 serialised medicines were entered into a hospital dispensary over two separate 8-week stages in 2015. Medicines were authenticated using secure external database cross-checking, triggered by the scanning of a two-dimensional data matrix with a unit specific 12-digit serial code. 4% of medicines included were preprogrammed with a message to identify the product as either expired, pack recalled, product recalled or counterfeit.
SETTING: A site within a large UK National Health Service teaching hospital trust. PARTICIPANTS: Accredited checking staff, pharmacists and dispensers in a pharmacy department. PRIMARY OUTCOME MEASURES: Authentication and detection rate of counterfeit expired and recalled medicines.
RESULTS: The operational detection rate of counterfeit, recalled and expired medicines scanned as a combined group was 81.4% (stage 1 (S1)) and 87% (stage 2 (S2)). The technology's technical detection rate (TDR) was 100%; however, not all medicines were scanned and of those that were scanned not all that generated a warning message were quarantined. Owing to an operational authentication rate (OAR) of 66.3% (over both stages), only 31.8% of counterfeit medicines, 58% of recalled drugs and 64% of expired medicines were detected as a proportion of those entered into the study. Response times (RTs) of 152 ms (S1) and 165 ms (S2) were recorded, meeting the falsified medicines directive-mandated 300 ms limit.
CONCLUSIONS: TDRs and RTs were not a limiting factor in this study. The suboptimal OAR poses significant quality and safety issues with this detection approach. Authentication at the checking stage, however, demonstrated higher OARs. There is a need for further qualitative research to establish the reasons for less than absolute authentication and detection rates in the hospital environment to improve this technology in preparation for the incumbent European Union regulative deadline. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

This is the first study to academically assess the effectiveness of medicines authentication technology in the secondary care setting, demonstrating the current strengths and weaknesses of this technology, technical and operational, for consideration by healthcare providers and policymakers.

This study is based on the introduction of 4192 two-dimensional (2D) data matrices into a live hospital dispensary.

Owing to the lack of widespread serialisation, this study required the manual adherence of 2D labels to each product, which made it possible to assess only one National Health Service (NHS) teaching hospital at the outset.

This pilot introduced 2D data matrices into an NHS hospital, which were preprogrammed with falsified or substandard medicine alerts. This study did not introduce any falsified or substandard medicine alerts into the supply chain, as to do so would be unethical.

Introduction

The terms counterfeit and falsified are often used interchangeably. According to the Food and Drug Administration (FDA), a counterfeit medicine is fake medicine. It may be contaminated or contain the wrong or no active ingredient. They could have the right active ingredient but at the wrong dose.1 According to the European Medicines Agency, falsified medicines are fake medicines that pass themselves off as real, authorised medicines.2 The pharmaceutical security institute report that between 2011 and 2015 the global incidence of drug counterfeiting has increased by 51%, with 2015 seeing the highest levels of counterfeiting to date, a 38% increase when compared with 2014.3 This upward trend can also be seen in the UK supply chain, where 11 cases of falsified medicines were detected over an 11-year period (2001–2011)4 The direct result of medicine counterfeiting includes deterioration of medicine quality and therefore patient health, unnecessary drug side effects and death in some of the most vulnerable patient groups.5–12 The indirect effects of drug counterfeiting include a loss in government tax revenue and the funding of illegal activity which may include terrorist organisations.13 High profile cases of counterfeit medicines include anticancer agents such as Avastin (Bevacizumab; USA),6 Herceptin (Trastuzumab; UK, Finland and Germany)8 and epidemic cases such as those seen in Bangladesh, where unsafe levels of ethylene glycol found in paracetamol elixir, which were responsible for the renal failure and death of over 50 patients (mostly children),10 and represents an international medicines safety issue.

The current methods for detecting counterfeit medicine are varied in nature and span from laboratory-based methods through to SMS texting. The detection of counterfeit medicines by customs officials usually occurs as a result of intelligence or random checks, suspect medicines are then sent away for laboratory-based analysis. Advancing technology has made a variety of techniques available which include spectroscopy, chromatography, SMS, handheld or portable laboratories, radiofrequency identification and serialisation.14 Serialisation is the process of identifying a medicine with a unique code printed onto the medicines pack and verification is the process for identifying and checking that code. In terms of the falsified medicines directive (FMD), the term ‘authentication’ relates to the final scanning of a medicine and the subsequent decommissioning of a product at the point of supply to the patient to ensure authenticity. The 2011 FMD15–18 and the 2013 Drug Quality and Security Act (DQSA)19 have adopted the serialisation and verification approach for counterfeit medicine detection. This is a low-cost, non-destructive and quick method for detecting counterfeit medicines. The FMD requires the systematic authentication of medicines at the point of supply to the patient while the DQSA requires verification at every point of sale and exchange throughout the drug distribution cycle, currently without authentication at the point of sale or administration to the patient. Although practices similar to those proposed by the FMD exist within the Italian, Greek and Belgian primary care markets, principally as a reimbursement method, FMD-legislated serialisation and authentication technologies are alien to many countries and have not been academically assessed and may prove difficult to implement, especially in the heterogeneous secondary care environment.20

Objectives

Primary objectives

Primary objectives were to identify the:

Operational authentication rate (OAR): The percentage of medicines scanned as a proportion of those entered into the study.

Technical detection rate (TDR): The ability of the authentication technology to read the two-dimensional (2D) data matrix of a counterfeit drug and generate a message to identify it as such.

Operational detection rate (ODR): The number of medicines quarantined as a percentage of those identified as recalled, expired or potentially counterfeit by the technology.

Secondary objectives

Secondary objectives were to identify the:

Optimum point in the dispensing process to authenticate medicines based on OAR and ODR.

Response time (RT) of the technology: The time it takes to scan a medicine, send the information to an external database for cross-checking and return an accurate result.

Methods

Study site

The district general hospital involved in this study is one of four hospitals in a large UK National Health Service (NHS) foundation trust. This site was selected due to the presence of specialist and general medical and surgical services provided. The variety of clinical services available ensured a diversity of medical treatments in hospital circulation and provided a balanced portfolio of medicines available for serialisation during this study.

Sample selection

Medicines were selected using a set of inclusion and exclusion criteria (figure 1). These criteria ensured that the medicines selected for inclusion reflected the categories of medicines governed by the FMD and the most commonly counterfeited drug groups, which included the top 50 medicines by turnover and the top 50 medicines by cost. Medicines not covered by FMD legislation were then excluded. This process returned a list of 87 products. The top 15 by usage and top 15 by value were then included in the study; a reduced number of study products was implemented for practical administrative reasons.

Figure 1

Inclusion and exclusion criteria for study medications.

The approach taken to identify a study drug sample resulted in a diversity of medicines representing major clinical indications and formulations (see online supplementary appendix 1). This ensured that a variety of products of differing clinical indication, formulation and cost were included in this study to represent the variety of medicines used in the secondary care environment and to avoid the inclusion of medicines which are not governed by FMD legislation. An exception was made for a number of high-volume pharmacy supervised sale (P) and general sales list (GSL) medicines in an effort to maintain high dispensing throughput.

Materials

Unique global standards one (GS1) 2D data matrix labels were produced and cut to size to limit the product area obscured by the label. Corresponding 2D data matrix codes were loaded and stored in an excel spreadsheet. The authentication technology had previously been integrated into the hospital patient medication record (PMR). The aforementioned software was operated by an existing computer terminal. The medicine codes were presented as a 2D data matrix and scanned using a handheld, terminal powered, barcode scanner, which identified the product as either ‘Authenticated elsewhere’ (counterfeit), ‘Item Expired’, ‘Item Recalled (product or batch)’, ‘Authenticated’ or ‘Already Authenticated here’ (figures 2 and 3).

Figure 2

Pop-up messages triggered on authentication of medicines that are safe for administration.

Figure 3

Pop-up messages triggered on authentication of medicines requiring quarantine.

Labelling procedure

Each 2D barcode was detailed in an excel database. Drug details such as product name, form, strength, pack size and the date on which the product was labelled were recorded in the database when the adhesive code was adhered to each study product, providing a complete record of study medicines serialised and the date of inclusion into the study. The 2D data matrix was attached to each study product according to a hierarchy described in the study protocol to ensure that the obscuring of important clinical data such as product name, strength, form, batch number or expiry date were not excessive during the study period.

2D data matrices were attached to all study medicines each Monday and Wednesday between the hours of 7:00 and 14:00 weekly, which maintained the serialisation of product lines throughout the study. Ninety-six per cent of medicines labelled, once authenticated by the operator, would provide a symbol to indicate the product as safe for use and ‘Authenticated’. If a product authenticated within the organisation were to be reauthenticated, the system would display an ‘Already Authenticated here’ message (figure 2). This was useful when dealing with multiple authentications of split pack medicines. ‘Authenticated’ and ‘Already Authenticated here’ messaging did not require quarantine (figure 2). A 1% subgroup of medicines were labelled with a 2D data matrix which prompted a response of ‘Authenticated elsewhere’ (figure 3) indicating that this drug may have been counterfeited or falsified (copied) and introduced or reintroduced into the legal supply chain. A further three subgroups were introduced into the study, classified as recalled pack, recalled product and expired product at a frequency of 1% per subgroup (figure 3). All study products which were labelled with a 2D data matrix generating a warning pop-up message had the expiry and batch number recorded in the excel database upon inclusion in the study to facilitate follow-up, should any of the study products require subsequent investigation. The 1% figure was based on WHO estimate that ∼1% of the world’s medicines are counterfeit.21 To ensure equity amongst subgroups, the expired medicines and recalled medicines groups were also allocated a 1% distribution.

Study design

A 2-week pilot stage was conducted initially to ensure the technology and proposed study process were practical and without external database communication issues. The study was then separated into two stages. Stage 1 (S1) involved the authentication of medicines at the checking stage (by pharmacists and accredited checking technicians) and stage 2 (S2) at the dispensing stage (by dispensers and some accuracy checking technicians).

All staff were subjected to the same basic training (presentation and demonstration) and were instructed to authenticate according to the authentication protocol. Operators authenticated medicines at the point of supply to the patient or ward for named patients. Ward stock authentication was not included in this study.

Data cleansing and analysis was conducted for authentication and detection data using a cleansing and analysis form. This form was independently verified by a separate researcher to confirm results.

Statistical analysis

Drug sample size studies were conducted to ensure the total sample of study drugs was large enough to obtain reasonable CIs and margins of error using two independent sample size calculators.22 23 The total population was based on 2015 average 8-week dispensing figure of 9605 products and the sample sizes were 2115 (S1) and 2077 (S2). Z-tests by proportion for independent groups24 were employed to identify if there was statistically significant differences between results in S1 and S2. Percentages were employed to demonstrate differences between groups, which accounted for the slightly different numbers of study drugs in each stage.

Operator groups

S1 contained a selection of pharmacists and accredited checking technicians. S2 contained a selection of dispensers and accredited technicians. Dispensers could not be involved in S1 by law and pharmacists would not routinely be involved in S2 due to departmental policy; dispensing is not a role conducted by pharmacists during normal working hours. Accuracy checking technicians are largely responsible for involvement in S1 and there are likely to be instances where they would also be involved in S2. Staff are not, however, permitted to be involved in both stages for the same prescription according to hospital policy.

Blinding and disclaimers

Operators

Although the 2D labels contained some adjacent print, which if analysed carefully over numerous scans could reveal a trend between expired and recalled medicine labels, to do so would be very time-consuming, unlikely to have occurred and was not mentioned in operator feedback. The operators were blinded as to which drugs were ‘suspicious/counterfeit’, expired or recalled.

Researcher

Researcher was not blinded at the point of labelling.

As authentication was performed towards the later stages of prescription preparation process, authentication had no part to play in stock control during this study. The study did not relate to or use any patient data.

Patient involvement

Patients, carers or lay persons did not participate in this research. The design of this study, the research questions and the outcome measures were informed by clinical, technical, research and industry leaders and did not include patient involvement. Clinical, technical, research and industry leaders were involved in the recruitment to and conduct of this study. Results will be disseminated to study participants initially via internal presentation and via access to the research manuscript once available. Participants have been acknowledged in this publication.

Results

A total of 4192 drugs were entered into this study (2115=S1; 2077=S2), 180 of which contained a preprogrammed message pop-up which described the product as counterfeit, expired or recalled and requiring quarantine (92=S1, 88=S2) (figure 4). The S1 group authenticated 1447 medicines of which 59 required quarantine. The S2 group authenticated 1344 medicines, of which 54 required quarantine. Not all medicines that were identified as requiring quarantine were quarantined. Only 48 of the 59 medicines in S1 and 47 of the 54 medicines in S2 were quarantined (ODR).

Figure 4

Data tree which identified the total number of medicines serialised for each stage of the study (medicines included), medicines detected by the authentication technology, stored on the secure database (database detected) and finally, the total number of medicines in each stage quarantined for researcher investigation (operator detection (quarantine)).

The OAR relates to the number of medicines authenticated in a particular stage as a percentage of the total number of medicines entered into said stage. For this study, the OAR was 66.3% overall; 68.4% (95% CI) S1 and 64.7% (95% CI) S2.

The TDR relates to the ability of the technology alone to detect counterfeit, expired or recalled medicines, that is, read the 2D data matrix of a counterfeit drug and generate a message to identify it as such, and to store the relevant information. Multisite testing in this study has generated a 100% TDR. ODR demonstrates the relationship between scanned medicines identified as counterfeit, recalled or expired by the technology and those correctly quarantined by the staff. The ODR across scanned medicines was 84% across all groups; 81.4% S1 and 87% S2; a 5.6% difference between the groups. The group with the lowest ODR was the ‘Authenticated elsewhere (counterfeit)’ group which demonstrated a rate of 58.3%.

There were five groups of drugs, with five corresponding pop-up messages entered into this study, counterfeit drugs (authenticated elsewhere), recalled products (product recalled), recalled batch (batch recalled), expired medicines (item expired) and safe-to-use medicines (authenticated). Across both stages, 31.8% of counterfeit medicines, 58% of recalled drugs (product and batch) and 64% of expired medicines were detected as a percentage of those entered into the study.

Z-tests by proportion for independent groups identified if the differences between ODR in each subgroup were of statistical significance (Yes/No outcomes were generated using table 1 data). The only statistical difference lied between the counterfeit group and all other subgroups, individually and as an entire group.22

Table 1

Breakdown of data from figure 4 by stage and authentication technology alerts category to demonstrate detection at each step of the study

	Authentication technology alert categories
Stages	Authenticated elsewhere (counterfeit)	Product recalled	Batch recalled	Item expired
	No. of medicines included
Stage 1	22	24	24	22
Stage 2	22	22	22	22
	Operator authenticated
Stage 1	13	12	18	16
Stage 2	11	17	12	14
	Operator-detected (quarantine)
Stage 1	7	12	13	16
Stage 2	7	16	12	12

The difference between alerts used for potentially counterfeit drugs (authenticated elsewhere) and the alert for medicines which have already been authenticated on site appears similar in this study (figure 7).

Figure 7

Pop-up message warnings which are generated when a counterfeit medicine (left) and a medicine which has already been authenticated on site (right) are scanned.

The medicines authentication technology RT is the total time taken for the information scanned from the 2D data matrix to make a round trip from the scanning terminal to the authentication database and back. The mean RT over each 8-week period was 152 ms in S1 and 165 ms in S2 (figure 5). The FMD-mandated response rate (RR) is <300 ms.15

Figure 5

Total response times for each stage.

Discussion

Medicines were entered into an active secondary care dispensary system. The data generated (figures 4 and 6) identified a gap between serialised medicines entered into the study and those authenticated by the operators, the OAR. There also appears to be a disparity between medicines identified by the technology and those separated for quarantine (ODR) (figure 4). The OAR which represents user compliance across both stages was 66.3%. When compared with the expected standard of 100%, this figure appears to be relatively low which may be due to operator compliance issues. The OAR demonstrated a statistically significant difference of 3.7% (Z-test) (95% CI)24 between S1 and S2, which consisted of two largely different operator groups. A 3.7% difference in authentication rates could lend itself to the argument that the pharmacists and accuracy checking technicians at the checking stage are better suited in terms of manual medicines authentication at the point of checking than their dispenser counterparts at the point of dispensing. This difference could be due to the professional registration obligations of the operators in S1 and professional good practice which protects the staff involved in S1 from interruption during the medicines checking process, or may have been due to a number of organisational behaviour, human and organisational factors associated with the point of authentication, in the medicine supply process. However, further investigation would be required to support this argument further.

Figure 6

Graphic and numerical representation of OAR, TDR and ODR percentages. OAR, operational authentication rate; ODR, operational detection rate; TDR, technical detection rate.

There were no concerns raised during this study regarding the TDR (figure 6); this technology has been subsequently integrated and tested in a further two hospital trust sites demonstrating the same 100% detection rate.

It was observed that even when the technology identified a drug to be counterfeit, recalled or expired, the staff across both stages did not always quarantine that medicine. ODR rates (which represent the number of medicines quarantined by the operator as a percentage of those identified by the technology) demonstrated a 5.6% difference between stages; however, there was not a statistical difference between the groups, and therefore one group could not be described as ‘better’ than another in this study for this parameter (Z-test) (95% CI).24 Despite the lack of statistical significance between groups, there is a clinical and statistical significant difference (Z-test) (95% CI)24 between the overall group in terms of ODR compared with the expected legislative detection rate of 100% (tables 1 and 2). Detection rates appear to be influenced by two main factors, the compliance of staff in the authentication of medicines (OAR) and increased awareness to messaging which identifies a medicine as counterfeit, recalled or expired (ODR).

Table 2

Z-test outcomes for operational detection rate in each subgroup

Subgroup	Counterfeit (authenticated elsewhere)	Pack recalled	Expired	Product recalled
Counterfeit (authenticated elsewhere)		Yes	Yes	Yes
Pack recalled	Yes		No	No
Expired	Yes	No		No
Product recalled	Yes	No	No

As a total group, there is a difference between the ODR rates for the ‘Authenticated elsewhere’ subgroup (58.3%) and those of other subgroups (expired, recalled pack and recalled batch, average ODR 91%) (Z-test) (95 CI)24 (table 2). This is likely due to confusion between the ‘Authenticated elsewhere’ and ‘Already Authenticated here’ messages which are similar in terms of message content and colour (orange) (figure 7), with the former requiring quarantine and the latter requiring no action. This issue could be alleviated by changing the colour of the ‘Authenticated elsewhere’ message to red which would match other pop-ups requiring medicine quarantine.

The RR is the time taken to send information to an external database, cross-check and retrieve a reply which states the status of the drug was 152 ms (S1) and 165 ms (S2) (figure 6) demonstrable in this study over 2791 scans, which is appropriate for systematic verification and or authentication of medicines when compared with the accepted FMD regulatory limit of 300 ms.15 16 These data are, however, based on a relatively small sample and may not necessarily be repeated in the presence of a larger throughput. This RR would require regular assessment once this technology is implemented nationally and internationally.

Study positives and negatives

There was some participant group crossover in this study; however, this is standard practice in UK NHS hospital dispensaries, reflecting normal working patterns in the medicine supply process. This study was carried out in a single hospital and, therefore, similar studies in a number of other UK hospital sites could adopt the present study design and replicate the work to identify whether the results of this study are indicative of the entire NHS environment. Owing to the emerging nature of this technology, there have been no other studies in this field to compare results. In addition, this study included a large sample of study drugs which generated results large enough to demonstrate statistically significant outcomes.

Context and impact

Government organisations such as the Federal Bureau of Investigation (US), the Internal Revenue Service (US) and the NHS (UK) are no strangers to information technology project failures.25 The NHS in the UK has experienced a recent struggle with the national programme for information technology (NPfIT), which required the implementation of the electronic patient record by 2005 (a target set in 1998). By the spring of 2002, only 2% of trusts had reached this target.26 27 The government then ring-fenced the information technology budget and pledged £2.3 billion to NPfIT with the aim of implementing electronic patient records by 2007. An accomplishment which to this day is yet to be complete across all NHS trusts. It is important to understand the role that context plays in healthcare innovation. Each hospital will have different contexts which will affect innovative implementation and it is important to understand internal and external contexts and how they facilitate or negate the successful implementation of this healthcare technology.25 28 It is important for policy and key decision makers to be cognisant of study results and past projects and build on what is known when planning the implementation of this detection tool; to put in place effective strategies for education and training as well as safeguards which may facilitate the authentication and, therefore, detection of counterfeit, recalled and expired medicines.

This study involved the presentation and the dissemination of an authentication protocol to the participants. Carthy et al29 raise concerns regarding the growing number of protocols and guidelines which require attention by NHS staff, which in this case may also have a part to play in non-compliance, perhaps a more innovative and interactive approach to education and training would facilitate a higher compliance rate. Other ways to improve compliance may include incentives. The FMD allows nations to use a reimbursement code as part of the 2D matrix which would result in payment by authentication. This would increase the OAR and help to reduce fraud within the NHS. It is therefore important to be realistic about the introduction of technologies into a heterogeneous environment20 and to involve staff members in the implementation of projects to identify areas for improvement before legal compliance.

Further research

Medicines authentication technology is an approach which aims to safeguard European Union (EU) and US citizens against the poor-quality medicines. It is important to identify the shortfalls of this technology and make improvements before the EU (2019) and US (2023) regulative deadlines. Further qualitative research is required to understand expert opinion on medicines authentication to identify contextual reasons for less than optimum authentication rates and less than absolute detection rates. It would also be important to understand the technological, process and educational adjustments required to improve the authentication and detection rates demonstrated in this study which in turn would improve patient safety. As research in this field moves closer to patient participation, it will be important to include patients, carers and lay persons in the design of future studies.

Conclusions

Medicines authentication technology is capable of meeting the FMD-mandated response speed of <300 ms and demonstrates a 100% TDR. The OAR requires improvement which may be facilitated by innovative and interactive education and training or through the introduction of incentives such as ‘payment by authentication’. The operator detection rate was also <100% and further qualitative research is required to identify technical solutions to facilitate the correct quarantine of medicines identified as recalled, expired or potentially counterfeit.