Examining the Quality of Medicines at Kenyan Healthcare Facilities: A Validation of an Alternative Post-Market Surveillance Model That Uses Standardized Patients.

BACKGROUND: Promoting access to medicines requires concurrent efforts to strengthen quality assurance for sustained impact. Although problems of substandard and falsified medicines have been documented in low- and middle-income countries, reliable information on quality is rarely available.
OBJECTIVE: The aim of this study was to validate an alternative post-market surveillance model to complement existing models.
METHODS: The study used standardized patients or mystery clients (people recruited from the local community and trained to pose as real patients) to collect medicine samples after presenting a pre-specified condition. The patients presented four standardized conditions to 42 blinded facilities in Nairobi, Kenya, resulting in 166 patient-clinician interactions and dispensing of 300 medicines at facilities or nearby retail pharmacies. The medicine samples obtained thus resemble those that would be given to real patients.
RESULTS: Sixty samples were selected from the 300, and sent for analysis at the Kenya National Quality Control Laboratory. Of these, ten (17%) did not comply with monograph specifications (three ibuprofen, two cetirizine, two amoxicillin/clavulanic acid combinations, and one each for prednisone, salbutamol and zinc). Five of the ten samples that failed had been inappropriately prescribed to patients who had presented symptoms of unstable angina. There was no association between medicine quality and ownership, size or location of the facilities.
CONCLUSION: The study shows that the standardized patient model can provide insights into multiple dimensions of care, thus helping to link primary care encounters with medicine quality. Furthermore, it makes it possible to obtain medicines from blinded sellers, thus minimizing the risk of obtaining biased samples.

Key Points

Introduction

There is widespread recognition that promoting access to medicines is not sufficient on its own, and that mechanisms must be put in place to guarantee compliance to acceptable quality standards. However, such mechanisms can only exist in markets where information on the quality of medicines exists, with proper mechanisms for enforcing compliance to standards [1, 2]. This information is not as widely available as it should be in low-income countries, where complaints of substandard and falsified medicines are common, and where regulatory enforcement may be weak [3, 4]. More recent evidence suggests that the problem may be worsening in middle- and higher-income countries as well [5]. Worse still, there has been little effort to describe the causes and broader impact of poor quality medicines [6].

The risk of poor quality medicines in the market is higher in lower-income countries [7, 8]. A pilot study by Bate et al., for instance, found that whereas substandard medicines exist across the globe, unscrupulous manufacturers have a particular preference for the African market, where regulatory efforts are absent or minimal [9]. Low-quality medicines have been linked to adverse public health outcomes; for instance, a 2013 analysis linked 122,350 child deaths in 39 African countries to consumption of poor-quality antimalarials [10]. These statistics underscore the value of understanding and strengthening the quality of medicines in the region.

Past studies in Kenya have established the presence of poor quality medicines, although the severity has varied across studies, regions, and disease components [4, 11]. Post-market surveillance studies have mainly focused on malaria, HIV, and TB.

These studies have relied on medicine samples collected openly from pharmacies for purposes of analysis. While this has proved to be a powerful tool to determine the quality of specific batches of medicines, it is less informative about the quality of medicines that patients receive for given symptoms or diagnoses. The divergence comes from two sources. First, patients often receive medicines in the clinic itself as opposed to pharmacies, a practice that is especially common in public facilities. And second, it is possible for a pharmacy provider to offer samples for analysis that they deem to be of superior quality, and conceal those acquired through dubious means, or those that are beyond their expiry date.

The World Bank Group has supported various reforms aimed at strengthening patient safety and quality of healthcare in Kenya. As part of this mandate, a validation study was conducted using standardized patients (SPs) or mystery clients to describe the quality of medicines at Kenyan health facilities.

SPs are people recruited from the local community and extensively trained to present the same set of symptoms to multiple providers. They are increasingly used to assess the quality of healthcare and are widely regarded as the gold standard in such assessments. On the other hand, mystery clients or shoppers have been used to understand pharmacy/drug seller dispensing practices, usually in a retail setting [12-15]. Although conceptually both methods use blinded patients to assess provider practices, mystery shoppers typically ask for a specific medicine/treatment, while SPs present a symptom and are given prescriptions and dispensed drugs in accordance with the diagnosis and treatment choice selected by the healthcare provider. Blending the two approaches, as we attempt in this study, can provide valuable information as SPs are treated just like any other patient and the quality of the drugs they receive can be closely tied to the illness they present with.

This study was part of a pilot for the larger Kenya Patient Safety Impact Evaluation (or KePSIE), which sought to evaluate the effectiveness of different ways of enforcing regulations on patient safety. This component sought to identity medicines that fail quality tests, describe the proportion of SPs who received medicines that failed the tests, and describe patterns of association between the presence of medicines of questionable quality and selected facility and patient characteristics.

As this is a validation study, our primary objective was to assess whether the methodology could be used successfully in larger samples, and whether doing so would yield new insights on the quality of medicines in the market. The study specifically sought to contribute to the knowledge on how drug quality studies can move beyond assessing the pharmaceutical quality of the product, to answering the broader public health questions such as the proportion of patients with certain disease conditions that get the right products prescribed, and of those, how many go on to receive a product of acceptable quality. Table 1 provides the working definition of key terms used in the article.

Table 1

Definition of key terms used in the report

Falsified medicine	Fake medicines that pass themselves off as real, authorized medicines (European Medicines Agency definition)
Substandard medicines	Genuine medicines produced by authorized manufacturers that do not meet quality specifications set by national standards (World Health Organization)
Excipient	Components of a finished drug product other than the active pharmaceutical ingredient (API), and are added during formulation for a specific purpose (US Pharmacopoeia, 2007)
Impurity	Any component of the drug substance/product that is not the chemical entity that is defined as the drug substance or product, or an excipient in the drug product (US Pharmacopoeia, 2013)

Methodology

The study employed a cross-sectional survey design using SPs to collect samples from blinded facilities and pharmacies. The collection of samples for the analysis was anchored on the broader pilot of the KePSIE study, which used SPs to investigate processes of care, including diagnosis and management of pre-set symptoms specific to four diseases, namely, diarrhea, asthma, tuberculosis, and unstable angina. The four were selected following wide consultations with clinicians in Kenya. A more detailed description of the KePSIE methodology is provided elsewhere [16]. This component of the study focuses specifically on the medicines given to SPs at the end of their respective interactions with healthcare providers. The samples analyzed were medicines dispensed to SPs posing as actual patients.

SPs either received the drugs in the facilities they visited (most often in public facilities) or received prescriptions from providers in sampled facilities, and then purchased the medicines from the nearest pharmacy, all within Nairobi. Health facilities were approached in a convenience sample designed to include low-, middle-, and high-income neighborhoods in various parts of Nairobi. Care was taken to ensure a fair representation of relatively poorer and wealthier neighborhoods in the selection of facilities. The SP visits were conducted in six sub-counties (divisions) in Nairobi: Dagoretti, Kamkunji, Kasarani, Langata, Starehe, and Westlands.

Of 49 health facilities approached, 46 agreed to participate. SP interactions were completed in 42 facilities, with four randomly held as reserves in case a sampled facility was closed or otherwise inaccessible. Of 168 potential interactions in these 42 facilities, we completed 166 (98.5%). Of those facilities, 14 were public and 28 were privately owned and operated. Among the private sector, five were operated by faith-based organizations (FBOs), four by social franchise operations (SFOs) and one was a community clinic. In analysis, facilities are classified only as ‘public’ or ‘private’ (which includes the not-for-profit facilities) due to the small sample size. A total of 300 medicines were prescribed in the 166 interactions. The SPs presented the prescriptions to the facility pharmacy or a nearby retail pharmacy. Of the 300 medicines, 19 were dispensed in unlabeled packaging (dosage written, but name of medicine missing). This typically happens in cases where medicines are purchased in bulk containers; for instance, loose tablets bought in tins of 1000s.

Table 2 shows the medicines given for the four conditions presented by the SPs, and the frequencies with which each medicine was given for each condition. The sources of the medicines varied. Of the 91 medicines offered in 37 public facility interactions, 61 (67%) were obtained from the facility pharmacy and 30 (33%) were purchased from an outside chemist. Of the 209 medications offered in 82 private facility interactions, 179 (86%) were obtained from the facility pharmacy and 30 (14%) were purchased from an outside chemist. SPs presenting with the same set of symptoms could receive different treatments depending on the clinician’s diagnosis and treatment preference. These variations underscore the importance of doing a study of this nature, where questions answered are not just about the pharmaceutical quality of the product, but the proportion of patients who received the correct treatment, and received a product of the right quality.

Table 2

Medicines by condition presented by standardized patients (based on symptoms presented, not diagnosis made). TB tuberculosis

Drug	Asthma	Angina	Child diarrhea	TB	Total
Acetylsalicylic acid	0	1	0	2	3
Albendazole	0	0	2	0	2
Aluminium/magnesium trisilicate	0	1	0	0	1
Amoxicillin	7	6	1	9	23
Amoxicillin and clavulanic acid	2	3	0	1	6
Ampicillin cloxacillin	4	2	0	1	7
Artemether lumefantrine	0	1	2	0	3
Azithromycin	0	2	0	1	3
Benzodiazepine	0	1	0	0	1
Cefalexin	2	2	0	2	6
Cefuroxime	1	0	0	1	2
Celestamine	1	0	0	0	1
Cetirizine	4	1	0	3	8
Chloramphenicol	0	0	2	0	2
Chlorpheniramine	6	3	1	4	14
Ciprofloxacin	1	2	0	0	3
Co-trimoxazole	4	9	4	6	23
Desloratadine	1	0	0	0	1
Diclofenac	1	11	0	0	12
Diphenhydramine	2	0	0	2	4
Erythromycin	0	1	1	2	4
Esomeprazole	0	1	0	0	1
Ferrous fumarate	0	1	0	0	1
Fluconazole	0	1	0	0	1
Guaifenesin	2	0	0	0	2
Hydrochlorothiazide	1	0	0	0	1
Hyoscine	0	2	0	0	2
Ibuprofen	2	7	1	3	13
Levofloxacin	0	0	0	1	1
Mebendazole	0	1	1	0	2
Mefenamic acid	2	0	0	0	2
Metronidazole	1	1	8	1	11
Multivitamin	0	0	1	1	2
Oral rehydration salts	0	0	15	0	15
Paracetamol	9	10	4	9	32
Prednisolone	11	4	0	1	16
Promethazine	0	0	3	0	3
Pseudoephedrine	0	0	0	1	1
Pyridoxine	0	1	0	0	1
Salbutamol	24	1	0	4	29
Unlabelled	3	6	4	6	19
Vitamin B complex and C with zinc sulfate	0	1	1	1	3
Zinc sulfate	0	0	14	0	14
Total	90	83	65	62	300

A chain of custody protocol was developed to guide the handling and transportation of medicine samples from the dispensing point to the analysis laboratory. The SPs were instructed to leave medicines in their original packaging and not handle medicines directly or tamper with the labeling. The SPs were debriefed by supervisors following each interaction, and facility information entered into a questionnaire. Medicines for each interaction were put in a sealable container bearing a unique facility and patient number, before being given to the study managers, who submitted them to the Kenya National Quality Control Laboratory (NQCL). A formal contract was signed between the NQCL and the World Bank for analysis of selected samples, and the remainder of the medicines were destroyed according to the NQCL protocols.

One major difference between this validation study and previous analysis is that drug quality testing typically entails batch sampling, which refers to selection of samples based on batch characteristics. A ‘sample’ will typically refer to an item of a given dosage form and strength, collected from the same packaging (same batch number). Even where medicines bear the same active pharmaceutical ingredients (APIs), dosage form, strength, manufacturer, and batch number, they would still be considered as different samples if they were collected from different sites. However, the SP study design precludes the application of this approach. In this case, samples were collected by the SPs, who could therefore not insist on getting medicines from the same batch, or even ask for the original packaging with batch details. For purposes of this study, and recognizing the peculiarity of using SPs, we defined a sample as “a molecule of a specific active pharmaceutical ingredient (API), dosage form and strength, given to a SP at a facility, regardless of batch details”.

Analysis was limited to describing the pharmaceutical quality, rather than making a statement about the performance of different brands. While this varies from the traditional analysis, the approach addressed the overall objective of describing the proportion of SPs who received medicines of questionable quality.

Actual sampling of dispensed medicines sought to balance scientific rigor with budgetary constraints. Three sets of factors guided the sampling: the need to include representative medicines from major pharmacological classes; the need to apply some kind of proportionate sampling; and the need to keep the analysis costs at below US$18,000. Sampling was done in two stages. First, medicines were put into 11 pharmacological classes (with all unlabeled medicines being grouped together into a twelfth class). Secondly, medicines were sampled in each of the 11 classes, with the number selected varying depending on the overall number of prescriptions carrying the medicine. A total of 60 unique medicine samples were selected from 11 pharmacological classes. From each class, a simple algorithm was used to decide the number of samples to be analyzed for each medicine, based on the popularity of the medicine (Table 3).

Table 3

Criteria used for deciding the number of prescriptions per medicine

No of times the medicine was prescribed	Number of prescriptions sampled for analysis
≤5	2
6–10	3
11–20	4
>20	5

Figure 1 gives an overview of the selection process.

Fig. 1

Flowchart showing the sampling process

Two pharmacological classes (cardiovascular and central nervous system medicines) were excluded from analysis because they each only had one medicine prescribed. Unlabeled medicines were also excluded from the analysis. Table 4 shows the final sample included in the analysis, including the tests performed, and costs per medicine. The tests are defined briefly in Table 5. These included a range of qualitative and assay tests defined by various monographs, including the International Pharmacopoeia, the British Pharmacopoeia (BP) and the US Pharmacopoeia (USP). The decisions on which tests to perform for the different medicines were reached following discussions with the analysis experts at the NQCL. The decisions were guided by three key factors: first, the fact that the quantities available for analysis were relatively small compared with the traditional batch sampling approach, where sufficient samples are collected with the knowledge of the seller, not by standardized clients; secondly, the relatively low budget available for this study component; and thirdly, the availability of reference standards for the different tests (the NQCL observed that not all tests can be done for all samples, based on local availability of reference standards). The lab analysts were blinded to the identity of facilities from which samples were obtained.

Table 4

Final sample included in the analysis, with corresponding cost estimates

Medicine name (molecule)	Asthma	Angina	Child diarrhea	TB	Total	Tests to be done	Cost estimate per sample (US$)
Amoxicillin	1	1	0	1	3	Uniformity of dosage units, identification, assay (USP 37 NF 32)	225
Amoxicillin and clavulanic acid	2	3	0	0	5	Uniformity of dosage units, dissolution, identification, assay (USP 37 NF 32)	270
Cetirizine	1	1	0	1	3	Identification, assay (USP 37 NF 32)	145
Chlorpheniramine	1	1	0	1	3	Identification, uniformity of dosage units, assay (AIM)	225
Co-trimoxazole	2	2	1	1	6	Identification, assay, acidity/alkalinity, dissolution, uniformity of weight, microbial load (USP 37 NF 32)	270
Diclofenac	0	4	0	0	4	Identification, assay, uniformity of dosage unit (USP 37 NF 32)	225
Erythromycin	0	1	0	1	2	Uniformity of weight (BP 2012 Vol. V), microbial assay (AIM)	300
Ibuprofen	1	3	0	0	4	Identification, assay (BP 2012 Vol. III), Dissolution (USP 37 NF 32)	350
Metronidazole	1	1	3	0	5	Identification, dissolution, assay, acidity/alkalinity (USP 37 NF 32)	350
Oral rehydration salts	0	0	4	0	4	Assay (BP 2012 Vol. IV)	150
Paracetamol	2	0	1	2	5	Identification, dissolution, assay, uniformity of dosage unit, microbial load, acidity/alkalinity (USP 37 NF 32)	235
Prednisolone	4	0	0	0	4	Identification, dissolution, assay (BP 2012 Vol. III)	350
Salbutamol	5	0	0	0	5	Identification, uniformity of dosage, assay, dissolution (USP 37 NF 32)	175
Cough preparations	0	0	0	3	3	Microbial load, acidity/alkalinity (BP 2012 Vol. V)	235
Zinc sulfate	0	0	4	0	4	Assay, uniformity of weight, disintegration, fineness of dispersion (BP 2012 Vol. V)	235
Total	20	17	13	10	60

AIM adopted in-house method, BP British Pharmacopoeia, NF National Formulary, TB tuberculosis, USP US Pharmacopoeia

Table 5

Tests conducted by the National Quality Control Laboratory (NQCL)

Identification	Basic qualitative tests to identity the presence of the active pharmaceutical ingredients
Disintegration	Test to assess whether solid dosage forms will break up under standard conditions
Dissolution	Test to determine the amount of active ingredient that is available for absorption following disintegration
Uniformity of weight	Test weighing individual tablets (and getting the average weight) to establish whether formulation is correct in terms of specified weight of active ingredients and additives
Acidity/alkalinity test	Test to quantify the acidity/alkalinity of a product. Acidity refers to total amount of hydrogen ions in a solution. Alkalinity refers to the total amount of hydroxyl ions in a solution
Assay	Performed to assess the concentration of the active ingredient, and expressed as a percentage of the label claim
Microbial load	Test to determine the level of microbial contamination in the liquid dosage forms (cough preparations)

Each analysis was reported as ‘pass’ or ‘out of specification (OOS, or failed)’. Typically, analysts would repeat the test for failed samples to confirm. However, this was not possible here, given the way the samples were collected. The NQCL experts nonetheless revealed that from past experiences, over ninety percent of medicines that fail the first test go on to fail subsequent tests.

The proportion of samples that passed the tests were reported. Pearson’s Chi-square tests were also done to check for any association between samples passing the test and three key facility characteristics: facility size, facility ownership, and facility location.

The study received ethics clearance by the African Medical and Research Foundation’s Ethics and Scientific Review Committee (AMREF-ESRC, approval P94/2013).

Findings

A total of 60 samples were analyzed (Table 6). Of these, ten did not comply with monograph specifications (17%), with four samples (all salbutamol samples) complying, but showing an unidentified peak in the high-performance liquid chromatography (HPLC) output. In the absence of more detailed tests, these were not characterized as ‘failed’ samples. The peaks could have been anything, ranging from impurities to excipients such as carmoisine or erythrosine (used for color), which are not typically included in monographs.

Table 6

Summary of the results of the analyses

Medicine category	Medicine name (molecule)	Asthma	Angina	Child diarrhea	TB	Total	No of samples that complied (%)	Conclusion	Failed samples
									Asthma	Angina	Child diarrhea	TB	Total
Analgestics	Diclofenac	0	4	0	0	4	4 (100)	All samples complied
	Ibuprofen	0	4	0	0	4	1 (25)	Three failed dissolution test		3			3
	Paracetamol	1	1	1	2	5	5 (100)	All samples complied
Anti-asthmatics (bronchodilators)	Salbutamol	5	0	0	0	5	4 (80)	Four samples complied, but an unidentified peak in the HPLC. One sample failed the assay	1				1
Antibiotics	Amoxicillin	1	1	0	1	3	3 (100)	All samples complied
	Amoxicillin and clavulanic acid	2	3	0	0	5	3 (60)	Two samples failed assay for clavulanic acid. One sample also failed uniformity of dosage unit test for amoxicillin	1	1			2
	Co-trimoxazole	2	2	1	1	6	6 (100)	All samples complied
	Erythromycin	0	1	0	1	2	2 (100)	All samples complied
	Metronidazole	1	1	3	0	5	5 (100)	All samples complied
Antihistamines	Cetirizine	2	1	0	0	3	1 (33)	Two samples failed assay	1	1			2
	Chlorpheniramine	1	1	0	1	3	3 (100)	All samples complied
Gastrointestinal medicines	Oral rehydration salts	0	0	4	0	4	4 (100)	All samples complied
Steroids	Prednisolone	3	1	0	0	4	3 (75)	One sample failed assay	1				1
Cough syrups	Various ingredients	0	0	0	3	3	3 (100)	All samples complied
Vitamins and minerals	Zinc sulfate	0	0	4	0	4	3 (75)	All samples complied			1		1
Total		18	20	13	9	60	50 (83)		4	5	1	0	10

HPLC high-performance liquid chromatography, TB tuberculosis

Of the ten samples that did not comply, three were ibuprofen. Two samples each of cetirizine and the amoxicillin/clavulanic acid combination failed, with prednisone, salbutamol, and zinc tablets having one sample failing each. Five of the 10 samples that did not comply were inappropriately prescribed to SPs presenting symptoms of unstable angina (three ibuprofen, one cetirizine, and one amoxicillin/clavulanic acid, see Table 6). Similarly, four medicines prescribed for symptoms of asthma failed, yet only one prescription was correct, based on the treatment protocols (salbutamol).

Pearson’s Chi-square tests were done to describe associations between compendial compliance and facility characteristics. Apart from ownership and location, we also classified facilities by type/size where dispensaries were the smallest type of health facility, usually manned by two or three providers and offering basic curative and preventive services only, while health centers were larger, offering a slightly broader set of services, including basic laboratory testing. The Chi-square test reported p values that had no statistical significance at the 5% level (p values considerably greater than 0.05 in all cases, Table 7), suggesting that any observed differences in compendial compliance across facilities of different ownership, type/size, and location were likely to have been a result of chance.

Table 7

Association between compliance with compendial tests and key facility characteristics (n = 60)

Facility characteristics	Total number of facilities	n (%) compliant with compendial tests	p value
Ownership
Public	17	15 (88)	0.522
Private	43	35 (81)
Facility type (size)
Dispensary	39	33 (84)	0.717
Health centre	21	17 (81)
Facility location
Dagoretti	18	16 (89)	0.149
Kasarani	1	1 (100)
Kamkunji	6	5 (83)
Langata	17	14 (82)
Starehe	5	3 (60)
Westlands	13	12 (92)

Discussion

Recent years have seen a shift in emphasis, from promoting access to commodities, to promoting access to high-quality and effective healthcare services and commodities. In Kenya, issues of quality have become increasingly salient in the policy agenda following devolution of health services delivery to counties. Under the new Constitutional dispensation, the national government retains the stewardship role, which includes coordinating policy development, defining standards, and enforcing regulation, all aimed at ensuring that every Kenyan has access to high-quality services and commodities [17].

While issues of medicine quality are gaining prominence, there is a general dearth of knowledge on the quality of medicines, and especially, how the problems with drug quality impact broadly on health [6]. The problem of inadequate information is particularly important in Kenya, where over 10,000 registered medicines are in circulation. The Pharmacy and Poisons Board’s pharmacovigilance mechanisms are relatively new, and are yet to achieve a meaningful impact, particularly for commodities that are not funded by the Global Fund and other development agencies. Independent assessments such as this add to the existing knowledge stock.

Standard pharmacovigilance entails collecting samples directly from sellers identified through convenience sampling, random sampling, sentinel-site sampling, or lot quality assurance sampling [6]. Some of these approaches, however, may carry inherent biases, as sampled providers may conceal expired batches and those they deem to be of inferior quality. The behavior of medicine sellers concealing certain medicines deemed inappropriate has been documented previously in a study in Tanzania [18]. In addition, using the typical pharmacy-based sampling would have missed the majority of clients who received their medicines directly from the healthcare facilities. Approximately 67 and 86% of medicines given to SPs came directly from public and private health facilities, respectively.

Overall, selecting facilities using random sampling is not directly compatible with the model used here, as it would require that (i) the researcher predict the diagnoses to be made and treatments to be prescribed to the patients, (ii) that the researcher know whether the prescribed medicine is available at the facility or whether patients would have to go to a nearby pharmacy, and (iii) that the patient only go to the sampled pharmacy, regardless of whether or not it was the nearest or most preferred pharmacy in the area. This would have made it impossible to meet the study’s primary purpose of assessing the entire continuum of care by following a typical patient’s journey from diagnosis to treatment, and checking the pharmaceutical quality of medicines given.

Using SPs is therefore a novel approach, as it minimizes the bias and assesses medicines ‘as they are given’ to real patients. In addition, it allows the analysis of the quality of the medicines to be linked to the quality and appropriateness of treatment given to patients with various ailments. The model complements the traditional sampling-based models, as it links the pharmaceutical quality of medicines to the broader impact on health outcomes. Globally, research and innovation are increasingly targeted at availing novel and cost-effective detection technologies that can allow field testing of medicines in low-resource settings, as part of pharmacovigilance efforts [5]. However, these new technologies will still require innovative ways of deployment for maximum impact.

In this study, the SPs were able to successfully obtain both the medicines that were dispensed in the facility as well as those that were prescribed but not dispensed at the respective facilities. For the latter, they purchased the medicines from the nearest pharmacy to the sampled facility. Further, we were able to maintain a sterile chain of custody and successfully sample key drug samples from those obtained by the SPs. Finally, the samples were successfully analyzed in all 60 cases.

This analysis found that over four-fifths of all medicines complied with compendial standards, with ten products failing to meet the specifications (three ibuprofen, two amoxicillin/clavulanic acid, two cetirizine, and one each for salbutamol, zinc, and prednisone). While a single test is not considered conclusive, the findings raise important concerns over the quality of these medicines. It is also telling that some of the samples that did not meet the compendial specifications had been given inappropriately. In our analysis, for instance, we found that five SPs presenting symptoms of unstable angina were inappropriately treated, and still went on to receive medicines that did not meet compendial specifications. Similarly, three SPs with asthma symptoms got the wrong diagnosis, had the wrong treatment prescribed, and received medicines that failed the compendial tests. These findings underscore the value of understanding the quality management challenges in full, rather than examining aspects of poor care at specific points only. The findings also point at a worrying problem of low-quality medicines in the Kenyan market.

Different studies have reported variations in failure rates across medicines in Kenya. Three rounds of post-market surveillance of antimalarials, for instance, revealed failure rates of between 3.0 and 8.0% (8.0, 3.0, and 5.5% of samples failing for surveillance rounds one, two, and three, respectively) [19-21]. All three surveillance rounds, conducted between 2011 and 2013, had samples averaging 500 per round. A separate WHO-supported surveillance study reported a failure of 4% for antimalarials [4]. Higher failure rates were reported in earlier studies. Bate et al., for instance, reported a failure rate of 38% across six types of antimalarials obtained from pharmacies in the Kenyan market in 2007 [22].

Studies conducted in other countries have shown varied results. Nayyar et al. reviewed seven studies on medicine quality, and found that failure rates varied from 9 to 41% [5]. In Cambodia, 31% of 291 artemisinin derivatives failed quality tests, while in Afghanistan, one quarter of antimalarial medicines collected from 60 randomly selected facilities failed the disintegration test [23, 24].

In this study, no associations were reported between the medicine quality and health facility characteristics, although this is likely due to the small sample size. Different studies have looked at patterns of association between medicine quality and facility characteristics, with varying results. The 2013 antimalarial post-market surveillance in Kenya, for instance, reported failure rates of 2 and 8% for public and private sector-obtained samples, respectively, concluding that private sector providers were more likely to have substandard products in stock [21]. Elsewhere, a WHO study found higher failure rates from private sector samples in Uganda, but did not find any associations in Madagascar [4]. Similarly, no association was found between medicine quality and facility location in Cambodia [24].

The registration status of a health facility and prior drug approval by a stringent regulatory agency have also been associated with the quality of medicines. A past review, for instance, found medicines from unlicensed facilities to have a higher likelihood of failing the quality tests [25]. Another study found that medicines that had approval from a stringent regulatory agency had lower failure rates than those that had not been subjected to similar approval processes [26].

This analysis gave a snapshot of the quality of medicines dispensed around Nairobi. The use of SPs helped to minimize bias and allow assessment of the entire process of care, from clinical diagnosis to the quality of medicines given.

The pilot shows that the alternative model is viable, but requires modifications based on certain limitations. The main limitation of the study was insufficient samples for re-analyses. Medicines filled as prescriptions are rarely sufficient for compendial analysis and re-analysis when required. Although the bulk of medicines that fail the initial compendial tests go on to fail subsequent confirmatory tests, one cannot discount the possibility of false negatives. Ideally, repeat testing is recommended for samples that fail. Achieving sufficient sample quantities is difficult when using SPs; one would have to have prior information on prescribing frequencies and medicine choices for the different disease scenarios, and estimated failure rates for the different medicines prescribed.

Another limitation is lack of geographic representativeness. All dispensing facilities were located within Nairobi, meaning that the findings are unlikely to be representative. Nairobi hosts nearly all pharmaceutical manufacturers and distributors. Rural facilities have a higher risk of selling degraded products (long transport times) and/or falsified products (relatively porous borders, particularly in the Northern parts). On the other hand, facilities in more urban locations face much higher competition, which may result in their resorting to inappropriate behavior. Regardless of the directionality of the effect, one would expect variations in quality for medicines collected further away from Nairobi.

Finally, analysis checking for association between poverty levels in the neighborhood and compendial compliance were not done, as poverty data were not available beyond the sub-county (location) level. It is fairly common to find slums and relatively wealthier suburbs located within one sub-county in Nairobi.

Conclusion

The study shows that in addition to providing useful data for post-market surveillance, the standardized patient method can provide insights into other dimensions of care, thus helping to link primary care encounters with medicine quality. Furthermore, SPs make it possible to obtain medicines from blinded sellers, thus minimizing the risk of biased samples if providers know that medicines are being used for quality assurance testing. However, more effort should go towards defining more objective sampling methods that work for studies that use SPs. More thinking should also go towards understanding how medicine samples that are large enough for reanalysis can be collected without revealing the true identities of SPs.

The study found that ten medicines (17%) given to standardized patients did not comply with monograph specifications. Of the ten, five were given inappropriately to clients presenting classical symptoms of unstable angina.

By having standardized patients go through the full processes of care at the health facilities, and collect medicine samples at the end, it is possible to link the pharmaceutical quality of the medicines given to the other dimensions of care, including correctness of diagnosis and proper selection of treatment. Medicine providers are also less likely to give biased samples for analysis.