Quality assurance of medicines supplied to low-income and middle-income countries: poor products in shiny boxes?

OBJECTIVE: In today's context of globalisation of pharmaceutical production and distribution, international and national procurement agencies play a de facto key role in defining the quality of medicines available in sub-Saharan Africa. We evaluated the compliance of a sample of pharmaceutical distributors active in sub-Saharan Africa with the standards of the WHO guideline 'Model Quality Assurance System (WHO MQAS) for procurement agencies', and we investigated factors favouring or hindering the adequate implementation of the guideline.
METHODS: We used mixed-methods methodology to analyse quantitative and qualitative data. The quantitative study consisted of a retrospective secondary analysis of data collected by QUAMED (Quality Medicines for all), a partnership that pleads for universal access to quality-assured medicines. The qualitative survey consisted of formal and informal interviews with key informants. We adopted an embedded multiple-case study design.
FINDINGS: Our analysis suggests that international distributors based in Europe perform, on average, better than sub-Saharan African distributors. However, some weaknesses are ubiquitous and concern critical processes, such as the initial selection of the products and the ongoing reassessment of their quality. This is due to several different factors: weak regulatory oversight, insufficient human/financial resources, weak negotiating power, limited judicial autonomy and/or lack of institutional commitment to quality.
CONCLUSIONS: Our findings suggest that pharmaceutical distributors active in sub-Saharan Africa generally do not apply stringent criteria for selecting products and suppliers. Therefore, product quality is not consistently assured but depends on the requirements of purchasers. While long-term solutions are awaited, the WHO MQAS guideline should be used as an evaluation and training tool to upgrade current standards.

Poor quality medicines are prevalent in low-income and middle-income countries, where national regulatory authorities generally lack sufficient resources for effective regulatory supervision.

The WHO's Model Quality Assurance System (MQAS) guideline for procurement agencies sets adequate quality standards for pharmaceutical distributors.

We assessed the compliance of a sample of pharmaceutical distributors active in sub-Saharan Africa with the WHO MQAS.

Our findings suggest that pharmaceutical distributors active in sub-Saharan Africa generally do not apply stringent criteria for selecting products and suppliers; pharmaceutical quality is thus not consistently assured.

National regulatory authorities in the sub-Saharan African region should be strengthened.

Collaboration and transparent exchange of information between regulators from the North and the South should be pursued.

The WHO MQAS guideline should be used by all concerned stakeholders as an evaluation and training tool to upgrade the current standards of practice of pharmaceutical distributors active in sub-Saharan Africa.

Introduction

Globalisation of trade can be a powerful source of efficiency gains, but it brings new, complex challenges. In the pharmaceutical sector, the globalisation of production and distribution has not been accompanied by a global harmonisation of regulatory supervision, which makes the monitoring of medicines quality more difficult worldwide. Poor quality medicines are a growing issue in low-income and middle-income countries (LMICs)1–7 and especially in sub-Saharan Africa, where it is estimated that they represent 34% of the market. This implies major threats for individual and public health.8

Two major factors contribute to a high prevalence of poor quality medicines in sub-Saharan Africa: on the one hand, medicines are increasingly imported from countries with ‘varying’ regulatory oversight;9 10 on the other hand, many African regulators lack the resources needed to control the quality of medicines imported, manufactured and distributed in their territory.11 The main functions of National Medicines Regulatory Authorities (NMRAs) include control of pharmaceutical products by registration and postmarketing surveillance, as well as control of activities by licensing and inspection of manufacturers, importers, exporters, wholesalers, distributors, pharmacies and retail outlets, control of clinical trials and control of promotion of pharmaceuticals.12 Unfortunately, the WHO estimated in 2010 that 90% of NMRAs in sub-Saharan Africa were not enforcing basic regulatory functions. Their task is complicated by the complexity of the international supply chains, where intermediate traders and brokers operate along transnational routes outside stringent regulatory supervision (figure 1),13 and by the fact that the major donors of medical programmes do not always apply consistent quality criteria for the selection of medicines they fund (Perrin C. Quality assurance policy for the selection of medicines in place at three major international donors: The Global Fund, The World Bank & ECHO. Their impact on the quality of medicines procured with their funds in limited resource settings. Thesis submitted to obtain the degree of Master in Public Health-Disease Control Antwerp: Institute of Tropical Medicine; 2011).

Figure 1

Supply chain of medicines for LMICs. LMICs, low-income and middle-income countries.

In this context, international and national procurement agencies play a de facto important role in defining the quality of medicines available at country level. This is particularly true for national Central Medical Stores (CMSs), which have the government as their main client, and for (profit or not-for-profit) International Humanitarian Distributors (IHDs), which are specialised in supplying medicines to LMICs (more detailed definitions are given in box 1).

Procurement agency: any organisation purchasing pharmaceutical products, vaccines or other health products or otherwise involved in their prequalification, purchasing, storage and distribution (WHO-Model Quality Assurance System (MQAS), 2014).14

Central Medical Store (CMS): a national pharmaceutical procurement agency that manages the central national stock, with the objective of ensuring a continuous supply of essential medical products to the public health facilities. It may be autonomous from the government or not, but it always has government oversight.15 16 It procures medical products from national or international manufacturers/distributors, in line with the standards of the National Medicines Regulatory Authority (NMRA), or of the external donors, or autonomously (definition adapted from USAID, 2013).

International Humanitarian Distributor (IHD): a profit or not-for-profit distributor, located in Europe or in low-income and middle-income countries (LMICs), and specialised in the supply of medical products to LMICs. Their clients include non-governmental organisations, charities, United Nation's (UN) agencies and local organisations (ad hoc definition for this work).

Prequalification: the activities undertaken to define the adequate quality requirements for a pharmaceutical product, and to check whether a given product offered by a given supplier complies with such specifications. Only products complying with such specification may be purchased (definition adapted from the WHO-Quality Assurance System Terminology list, 2011).

Quality assurance: the totality of the arrangements made to ensure that pharmaceutical products are of the quality required for their intended use (WHO-Quality Assurance System Terminology list, 2011).17

For assuring the safety, efficacy and quality of the products they supply, distributors must operate under a ‘quality assurance (QA) system’ complying with the national regulation or, if unavailable, with the framework proposed by the WHO: the WHO Model Quality Assurance System (MQAS) for procurement agencies.18 The WHO MQAS guideline, initially designed for United Nations agencies, may be used by any organisation that wishes to implement the quality system of procurement agencies in all the phases of their work (figure 2). Likewise, the guideline can be used to assess the implementation of a QA system by CMSs and IHDs. Compliance with MQAS standards implies confidence that the procurement agency supplies quality-assured medicines, while non-compliance implies that it supplies medicines whose quality is not (fully) assured.

Figure 2

The procurement cycle in the six modules of the WHO-Model Quality Assurance System (WHO) and in the five components of the QUAMED rating system (QUAMED). QUAMED, Quality Medicines for all.

To the best of our knowledge, there are not yet formal studies documenting the evaluation of the quality systems of CMSs and IHDs using the MQAS guidelines. Therefore, we carried out a study to assess the MQAS compliance of a sample of African CMSs and European or African IHDs. Our initial hypothesis was that several factors contribute, positively or negatively, to the compliance with MQAS standards. These factors are situated at different levels of analysis: they may be attributes of the audited CMS/IHD, of the MQAS guideline, and of the national and international context. We also hypothesised that MQAS compliance is influenced by how the CMSs/IHDs are organised and by the understanding of the MQAS standards by the staff of CMSs/IHDs.

Methods

What are the specific factors that determine the level of compliance with the MQAS guideline? To answer this question, we adopted a mixed-methods methodology. Quantitative methods were used for a retrospective secondary data analysis. We adopted an embedded multiple-case study design. We defined our case as the ‘compliance with MQAS’ and our unit of analysis as ‘the procurement agencies’ (ie, CMSs/IHDs), embedded in the country and region. Qualitative data were obtained through formal and informal interviews.

The study was conducted within the framework of a thesis of the Master's Course in Public Health of the Institute of Tropical Medicine, Antwerp.19

Retrospective quantitative study

Quantitative data were drawn from the database of QUAMED (Quality Medicines for all), a partnership that brings together medical non-governmental organisations active in sub-Saharan Africa and African procurement agencies, and pleads for universal access to quality-assured medicines20 (http://www.quamed.org). To help partner organisations purchasing quality-assured medicines, QUAMED evaluates the performance of various pharmaceutical suppliers, including CMSs and IHDs (box 1). The suppliers are selected for audits if they (1) are current or potential suppliers of one or more QUAMED partners and (2) voluntarily undergo the audit, under a confidentiality agreement. QUAMED auditors, whose skills and qualification are defined by a standard procedure (see online supplementary annex 1, QUAMED-SOP-001), carry out the evaluations. Audit reports are made available to QUAMED partner organisations in a password-protected database. The CMSs and IHDs are audited according to the MQAS criteria (see online supplementary annex 2, QUAMED SOP-005), whereas smaller local distributors undergo a lighter audit. The results of audits of CMSs and IHDs are summarised in a rating system based on 15 MQAS key activities (table 1).

Table 1

QUAMED rating system and basic requirements for compliance with the WHO-MQAS

15 Key activities of the WHO-MQAS	Strong compliance (rate 4)	Non-compliance (rate 0)
GR
1. QA autonomy	The QA unit has sufficient autonomy to ensure that all key personnel implement good practices. Personnel responsible for implementing QA policies and personnel responsible for purchasing are independent from one another.	The concept of QA autonomy is not known and QA autonomy is not in place.
2. Documentation system	All activities are performed according to written approved procedures and documented in a standardised manner. The main elements of a documentation system are available and regularly updated (ie, the quality manual, standard operating procedures, list of qualified products and suppliers).	The activities and QA policy are not documented in a comprehensive documentation system; thus, the proper execution of operations is not ensured.
3. Human resources	There is a sufficient number of appropriately trained, educated and experienced personnel to perform key activities. Responsibilities and reporting lines are described in organisational charts and job descriptions.	No pharmacist works in the structure and there are no adequately qualified personnel for QA-related activities.
4. Autoevaluation	A policy for internal and external auditing is available. Internal audits are regularly performed, records of external audits are kept and corrective actions are implemented.	There is no evidence of internal or external audits.
5. QA follow-up	The quality of products is continuously monitored. The distributor is informed in time of any changes to the products that may affect their safety, efficacy or quality. Policies for dealing with complaints exist and recalls are handled according to written procedures.	A system for monitoring the quality of products, for managing complaints and for batch recall does not exist or is not documented.
PQ
6. Product qualification	Product dossiers are evaluated according to standardised procedures which conform to the WHO recommendations. The quality of the products is evaluated according to a written procedure.	No qualification/assessment of products is in place
7. Manufacturing site assessment	The sites where products are manufactured comply with the WHO-GMP, providing assurance that the facility can manufacture the product as specified in the product dossier.	There is no procedure or standards in place for assessing the sites where products are manufactured.
8. Qualification decision	The decision to qualify products for purchase is taken according to a written procedure. Roles and responsibilities are clearly identified. PQ and purchase are clearly separated.	The decision to qualify products for purchase is not described in a written procedure. Roles and responsibilities are not defined.
GDP
9. GDP	The management of the warehouse follows written procedures. Pharmaceutical products are received and stored in compliance with the WHO Good Storage Practices for pharmaceuticals. Products' quality and integrity are preserved and batch traceability is maintained.	No standards for products’ reception and storage are present/implemented. The management of the warehouse is not organised. Activities and responsibilities are not defined in written procedures.
10. Control at reception	Incoming products are checked for correspondence with the purchase order, integrity and conformity to quality requirements. Products are released for distribution, or quarantined for investigation, according to written procedures. Roles and responsibilities are clearly defined.	There are no written procedures for reception and release of products. Responsibilities are not clearly defined.
11. Distribution	Products’ integrity and quality are maintained throughout the distribution process. Orders’ preparation and transport conditions are described in written procedures, temperatures are monitored and products are traceable from the point of supply to the point of delivery.	Products’ integrity and quality are not assured throughout the distribution process. Order preparation and transport conditions are not described in written procedures. There is no traceability along the product supply chain.
QC
12. QC	A QC plan is described in a written procedure, and a subset of samples are analysed for compliance with the product specifications by an independent accredited laboratory.	There is no QC and/or no records of QC are available.
RE
13. Qualified sources monitoring	Dossiers of qualified products are regularly reviewed and suppliers are reinspected, according to a written procedure, to ensure that the products continue to meet adequate norms and standards over time.	There is no system for products’ re-evaluation and suppliers’ reinspection. Quality of products is not continuously monitored.

GDP, good distribution practices; GMP, Good Manufacturing Standards; GR, general requirements; MQAS, Model Quality Assurance System; PQ, prequalification; QA, quality assurance; QC, quality control; QUAMED, Quality Medicines for all; RE, reassessment.

Quantitative data for the present study were drawn from a subset of QUAMED audit reports, selected according to the following criteria:

Type of audited organisation: CMSs and IHDs;

Geographical location: Europe and sub-Saharan Africa;

Period: 1 January 2011 to 31 December 2014;

Audit type: conducted according to the MQAS;

Audit results: availability of the rating based on the MQAS.

Out of 140 reports, 18 met the inclusion criteria: 7 concerned CMSs in sub-Saharan Africa, 8 concerned IHDs based in Europe and 3 concerned IHDs based in sub-Saharan Africa (figure 3). Out of 122 excluded reports, 118 concerned small distributors (non-CMSs, non-IHDs), not audited according to the MQAS and 4 concerned distributors located out of Europe or sub-Saharan Africa.

Figure 3

Sampling design. IHD, International Humanitarian Distributor; MQAS, Model Quality Assurance System.

First, we analysed data about 13 activities out of the 15 included in the ‘rating’ of QUAMED (the activities ‘procurement’ and ‘company status’ were excluded as non-relevant to our research question). Then we grouped the 13 ‘activities’ into 5 ‘components’. Each ‘component’ represents one of the key phases of a QA system: general requirements (GR), prequalification of sources (PQ), good distribution practices (GDP), quality control (QC) and reassessment (RE). For each ‘component’, the MQAS compliance was calculated as the average of the scores of the ‘activities’ that it includes. The level of compliance assigned to each ‘activity’ and to each ‘component’ ranges from 0.0 (non-compliance) to 4.0 (strong compliance). Intermediate scores are: 0.1–1.0 (weak compliance), 1.1–2.9 (partial compliance), 3.0–3.9 (sufficient compliance). In the same way, a global score was also calculated as the average of the score of the five ‘components’. The scores were analysed by geographic region (distributors based in Europe vs sub-Saharan Africa) and by type of distributor (CMSs vs IHDs). To protect confidentiality, the identity of CMSs/IHDs will not be disclosed.

Qualitative survey

To interpret the quantitative results, we collected qualitative data by means of formal and informal interviews. Key interviewees were recruited through a convenience sampling strategy.

Interviewees for the formal interviews were chosen among the staff of CMSs/IHDs included in the quantitative analysis, provided that they had a role in the QA system of the facility. Six persons were initially contacted; four out of these agreed to participate and were individually interviewed via Skype or by phone. The interviews were conducted using a structured questionnaire of 19 open questions: 5 questions concerned the results of the MQAS audit, and 14 investigated personal views about the implementation of the QA system (see online supplementary annex 3, Interviews question guide).

Informants for the informal interviews were selected among pharmacists or auditors with long-standing international experience in pharmaceutical QA. Three key informants were contacted, agreed to participate and were individually interviewed via Skype or by phone. These informal interviews were conducted after the formal interviews and were based on their findings.

All interviewees gave their consent to participation by email. To ensure confidentiality, the identity of interviewees is not disclosed (see online supplementary annex 4, Information sheet and consent for interviews).

All interviews were audio-recorded, transcribed and manually coded. We analysed the transcripts using Lewin's force field model.21 22 As a tool for managing change, the model allows identifying and framing determinants of problems in terms of ‘hindering’ factors that support the status quo (in our case, ‘non-compliant to MQAS’), and ‘helping’ factors that support change in the desired direction (in our case, ‘compliant with MQAS’). The model also allows to organise the identified factors by level, according to the initial hypothesis.

Results

Quantitative data

The IHDs performed, on average, better than CMSs (mean global score 2.8 vs 1.3; figure 4) and the European IHDs performed, on average, better than African IHDs/CMSs (mean global score 2.9 vs 1.6) (figure 5), but all CMSs/IHDs showed some level of non-compliance with the MQAS.

Figure 4

Average level of compliance with the WHO-MQAS standards by type of supplier. MQAS, Model Quality Assurance System.

Figure 5

Average level of compliance with the WHO-MQAS standards by region. MQAS, Model Quality Assurance System.

The MQAS components with the highest and lowest global scores tended to be the same in all subgroups. The highest scores were obtained for the components GDP and GR: European distributors were compliant (GR=3.3; GDP=3.7) and sub-Saharan African ones partially compliant (GR=2.2; GDP=2.4). The lowest scores were obtained for the components PQ, QC and RE. Unfortunately, these are probably the most critical activities to assure the quality of medicines. Compliance was poor for African distributors (PQ=1.2; QC=1.8; RE=0.5): only two perform some PQ activities, and two do not have any QC activity at all. Compliance was partial for European distributors (PQ=2.5; QC=2.5; RE=2.5): 7/8 IHDs were only sufficiently compliant for PQ and 3/8 for QC, while 2/8 do not implement QC at all. Furthermore, no CMSs were strongly compliant for QC.

Qualitative data

The results of our qualitative analysis suggest that the lack of skilled human resources hinders MQAS compliance in sub-Saharan Africa (table 2 ). An IHD employee said: “…in many of our countries, qualified human resources aren’t available or are insufficient. The tasks allocation and conflicts of interests are difficult to control.”

Table 2

Factors hindering versus helping the implementation of MQAS standards

Hindering factors	Region	Helping factors
Poor knowledge of the MQAS Limited dissemination of the MQAS	Both regions	Perception of the MQAS as a regulatory standard Stringent regulatory system at national level International regulatory harmonisation Financial and judicial autonomy of the procurement agency
Lack of organisational commitment to quality Weak European Union regulatory oversight on quality of export-only products	Europe	Commercial incentives for quality (ie, the purchasers require quality assurance)
Lack of human resources Lack of resources of regulatory authorities (NMRAs) Weakness of national QC laboratories Poor transparency of the information provided by suppliers	Sub-Saharan Africa	Capacity building, for example, by international aid agencies MQAS audits by international agencies Promotion of the MQAS Awareness of the MQAS

MQAS, Model Quality Assurance System; NMRAs, National Medicines Regulatory Authorities; QC, quality control.

Prequalification of suppliers and products is critical to assure the quality of the medicines, and it is complex: “…is one of the most technical and multidimensional tasks…” (sub-Saharan Africa IHD). It is hindered by several factors, such as the weak registration process by NMRAs and the poor transparency of many suppliers: “Even if the supplier is qualified by the WHO…when we need information [the suppliers] don’t care to send complete [dossiers] because they don’t have an [important] business with us…” (sub-Saharan African IHD). Thus, prequalification becomes an administrative process, sometimes only based on the analysis of incomplete documentation.

Likewise, the RE of the sources is a critical but complex activity. It may be hindered by the weak capacities of national QC laboratories: “As their analytical methods or instruments may be inappropriate to perform adequate quality control analysis, we send some products to a WHO-certified laboratory” (CMS employee). Not all distributors, though, can bear the cost of testing products abroad, and nor have they qualified workers able to interpret analytical results: “The price is one of the challenge of QC…” (sub-Saharan Africa IHD).

Among factors helping prequalification and QC, some African interviewees mentioned capacity building and international audits conducted by organisations such as the European Commission's Humanitarian aid and Civil Protection department (ECHO), the US Agency for International Development (USAID) and QUAMED: “…finally it’s [international aid agencies] who push us to use WHO-qualified laboratories” (CMS). Financial and judicial autonomy of the CMSs are seen as elements favouring the implementation of QA systems: timely management of financial transactions has positive effects on the entire procurement cycle and the absence of external influences allows committed structures to invest in quality.

The findings concerning European IHD were sometimes similar. Asked about the most difficult MQAS component, an IHD employee said: “Pre-qualification, because it is too complicate and too expensive.” The disappointing QC performance seems unrelated to insufficient resources: “…it’s not a matter of financial resources but of commitment.” A senior auditor noted that European regulators do not require a QC test for export-only products, so its implementation relies solely on the goodwill of distributors and on clients’ requirements. It was also noted that the current market mechanisms do not reward distributors who invest in QA.

Some observations are common to both regions. Several interviewees reiterated that pharmaceutical regulations should be harmonised across countries and regions, that MQAS compliance should become a regulatory requirement and/or that the implementation of MQAS standards may compensate for gaps in poor regulation. However, according to senior auditors, the lack of knowledge and promotion of the MQAS guideline limits its implementation: “…is not promoted across all actors, some know it and some never heard about it…the passage from knowing to enacting is even more difficult” and “… it hasn’t been yet formally endorsed by regulatory agencies of LMICs for the inspections. So far, it is not yet a reference nor is it legally binding.”

Discussion

Our study confirms that the MQAS is a useful tool to assess the QA system of pharmaceutical distributors. Yet our findings also suggest that pharmaceutical distributors active in sub-Saharan Africa are not compliant (or not fully compliant) with the MQAS standards. In particular, most distributors were only partially, weakly or non-compliant with the most critical MQAS components: prequalification of suppliers and products, RE and QC. For African CMSs, this seems due to a mix of insufficient financial and human resources, poor regulatory oversight, weakness and scarcity of national QC laboratories, and a weak negotiating position in the market. For IHDs, the partial compliance seems due to the lack of regulatory oversight on export-only medicines and to poor institutional commitment.

Most distributors tended to have better compliance with the MQAS components related to good storage and distribution practices. This is not surprising: Europe-based distributors must apply the stringent GDP standards of the European Union23 and in sub-Saharan Africa investments in storage facilities have historically been prioritised over QA.24

Overall, institutional commitment to invest in QA, availability of adequate technical competences and decision-making autonomy appear to be essential pre-requisites for MQAS compliance, irrespective of the type of distributor and of its geographical location.

Our sample is small and all the distributors voluntarily accepted the audit; our findings could thus be biased by a positive attitude towards quality improvement. Therefore, our results need to be confirmed by further research. In particular, the same distributors should be reassessed for verifying if the MQAS audit has triggered actual improvements. Also, a formal MQAS assessment should be conducted for local private distributors, who play a key role in the pharmaceutical market in sub-Saharan Africa.25

Conclusion

To the best of our knowledge, this is the first study investigating the compliance of pharmaceutical distributors active in the public health sector in sub-Saharan Africa with the MQAS standards. Findings suggest insufficient MQAS compliance, particularly for the initial verification and the monitoring of products' quality, which are essential steps for ensuring the quality of medicines.

In the absence of stringent national and international regulatory oversight and while waiting (in the long term) for effective harmonisation of pharmaceutical regulation, ensuring medicines quality becomes a choice rather than a duty and, regrettably, there are little incentives to make such choice. There is thus a significant risk that poor quality medicines enter the supply chain, with harmful consequences for individuals and public health.26 Investing in well-managed warehouses to store medicines of uncertain quality is like providing poor products in shiny boxes. Based on our results, three recommendations may be formulated:

The WHO MQAS should be adopted by NMRAs for the regulatory assessment of pharmaceutical distributors in sub-Saharan African countries.

The NMRAs of exporting and importing countries should create or strengthen collaborations, and transparently share technical information on manufacturers and distributors.

Market incentives should be put in place to promote better QA practices. In particular, major buyers and funding agencies should require MQAS compliance from distributors as a pre-requisite to buy from them.

The WHO MQAS guideline should be consistently used as an evaluation and training tool to upgrade the current standards of QA of pharmaceutical distributors active in sub-Saharan Africa.

The experience of QUAMED shows that the WHO MQAS can help to assess the QA systems of pharmaceutical distributors, and hopefully to support their improvement. There is still a long way to go towards effective harmonisation of pharmaceutical regulation worldwide, but we are not without solutions in front of the global challenge of substandard medicines.27