Smart Safety Surveillance (3S): Multi-Country Experience of Implementing the 3S Concepts and Principles.

INTRODUCTION: The Smart Safety Surveillance (3S) concept is based on the understanding that, when faced with competing pharmacovigilance priorities, countries will have to invest judiciously, by focusing on new priority products, sharing work and resources with other countries when possible and building national competence for those activities that cannot be delegated.
METHOD: The 3S principles were applied to Armenia, Brazil, Ethiopia, India, Peru and Thailand using three priority products: bedaquiline, rotavirus vaccine and tafenoquine. A baseline assessment of pharmacovigilance preparedness was used to identify gaps and establish a work plan. The impact was measured by comparing pre and post 3S-intervention outcomes, which included the number and quality of reports (completeness scores) in the WHO global database of Individual Case Safety Reports, VigiBase, and number of structural indicators met. The implementation period was 9-18 months, ranging from March 2018 (earliest started) until May 2020 (latest). RESULT: An increase in adverse drug reaction (ADR) reporting was demonstrated in Armenia (bedaquiline), Brazil (TB and malaria medicines), India (rotavirus vaccine) and Ethiopia (TB medicines). Completeness scores were above 0.5 at baseline in all countries, and reports improved in quality for Brazil (TB), Peru (malaria), Thailand (malaria) and India (immunization). The number of structural indicators met increased by more than double for Ethiopia. Ethiopia and India demonstrated an increased capacity for signal detection and signal evaluation. Armenia, Brazil, Peru and Thailand showed increased capacity to assess risk management plans following the implementation of 3S principles.
CONCLUSION: The 3S concept has demonstrated success in different ways across the six countries. Activities focused on three products for a proof of concept of the 3S principles, with the expectation that the project impact will be sustained through strengthened systems, to guide pharmacovigilance activities of other products in the future. It is important to continue monitoring the countries to understand if the gains and successes of the current 3S project are sustainable.

Key Points

Introduction

Access to medicines facilitated through expedited regulatory pathways and early access programmes has been supported by regulatory agencies such as the US Food and Drug Administration (FDA) and the European Medicines agency (EMA) from as early as 1992 [1, 2]. Often these programmes are based on limited clinical data to allow accelerated authorization into the market [3]. For example, the novel drug bedaquiline was granted a conditional license by the US FDA in 2010 for the treatment of multidrug-resistant tuberculosis before phase III clinical trials were complete [4]. Drug repositioning (also known as drug repurposing or drug reprofiling) is a process of developing new indications for existing drugs that have been streamlined for approval [5]. However, other safety issues may emerge after a drug is repurposed with changes in posology, when different patient groups such as the elderly and children may get the treatment; the adverse drug reaction (ADR) profile of the drug could change with more extensive use, when lesser known (or less frequent) adverse reactions may surface more prominently. Remdesivir, an antiviral originally designed for treatment of hepatitis C, was repurposed for the treatment of Ebola in 2018 and repurposed again for the treatment of COVID-19 in 2020 [6-8]. Although such expedited processes for licenses in the EU and FDA are primarily for authorization in these countries and regions, low- and middle-income countries (LMICs) often leverage decisions from these reference agencies; the World Health Organization (WHO) also uses the decisions of such reference agencies to inform the work of its programmes such as the WHO prequalification programme, facilitating access to priority products in LMICs. Additionally, the EMA, in cooperation with the WHO, can provide scientific opinions on high priority human medicines, including vaccines, that are intended for markets outside of the European Union (EU). Expedited authorization (accelerated approval in the USA and conditional marketing authorization in Europe) for novel and often urgent treatments or vaccines might be given on the basis of several conditions, including timely completion of post-marketing studies, and should be able to rely upon robust and effective pharmacovigilance (PV) systems for a more thorough understanding and application of these products [9]. Monitoring requirements may be intensive and could even become a barrier for accessing new medicinal products in LMICs with very rudimentary or sub-optimal pharmacovigilance systems.

Given the finite resources at their disposal and the competing health priorities, LMICs would need to invest judiciously when it comes to pharmacovigilance. Rather than focus on general PV system development, the smarter option would be to focus their PV efforts on two types of medicinal products:

Products that are important to the country, introduced to address a high burden disease of public health priority and/or exclusive to the country (e.g. sleeping sickness).

Products with limited clinical data that will be introduced simultaneously, in high-income countries and in LMICs, with little global experience for LMICs to rely on.

These two scenario-based PV efforts can help prepare LMICs with a smart and agile PV system that is fit-for-purpose, fully ready to receive products that are new and/or exclusive to their own settings. In the absence of a focused, smart safety surveillance development plan, resources will continue to be used in a general way, without a meaningful and lasting return on investment. Above all, it is important that the absence of a functional PV system does not become a barrier to introducing new, life-saving medicinal products.

Additionally, global efforts to strengthen regulatory systems should include pharmacovigilance as an essential function within the regulatory framework [10]. For these reasons, many countries seek to invest in strengthening the PV system, to be recognized as a mature regulatory agency of international standards. However, with limited resources, it is essential to know which aspects of PV to invest in to gain maximum output/efficiency.

Smart Safety Surveillance (3S) is a pharmacovigilance risk prioritization strategy conceptualized by WHO to support Member States to identify which PV priorities to invest limited resources in, so that the system is prepared for the introduction of a new and/or an exclusive product. This paper will describe the challenges and achievements regarding the implementation of a 3S approach in a selected group of countries. The objectives of this paper are to

describe the application of the 3S principles in a selection of countries using baseline assessments and country plans in preparation for the introduction of three priority medicinal products; and

evaluate changes of selected variables and processes in the PV systems of targeted countries.

Methodology: Application of 3S Principles

Smart Safety Surveillance principles were applied in six countries about to introduce or having introduced three new products: bedaquiline (BDQ), tafenoquine (TFQ) and rotavirus vaccine (Rotavac®, Bharat Biotech, India), for the treatment of multidrug-resistant TB, malaria and prevention of rotavirus infections, respectively. A baseline assessment of PV preparedness was carried out in the six countries initially. Following this, national PV centres were supported in building their capacity and competence for identifying, assessing and adequately managing the risks associated with products in their markets (BDQ, rotavirus vaccine). For TFQ, a product yet to be introduced in the markets, national pharmacovigilance systems and competence were strengthened in anticipation of the product launch, using a proxy product with somewhat comparable profiles.

Selection of Countries

The six countries selected for testing the 3S principles on PV systems were Armenia (BDQ), Brazil (BDQ and TFQ), Ethiopia (BDQ, TFQ), India (rotavirus vaccine), Peru (BDQ and TFQ) and Thailand (TFQ). Selection was based on burden of disease in LMICs and projected use of the new products. Advocacy at a national level was required and each of the selected six countries committed to implementation of 3S interventions.

Baseline Assessments

WHO coordinated the development of an up-to-date global set of indicators using existing WHO, Indicator-Based Pharmacovigilance Assessment Tool (IPAT) and Global Benchmarking Tool (GBT) indicators, to assess PV preparedness of a country for the introduction of a new pharmaceutical product [11, 12]. Baseline assessments of the PV infrastructure, competence, capacity and gaps in four of the target countries took place during scoping visits (prior to 3S implementation): Armenia (March 2018), Ethiopia (February 2019), India (July 2018), Thailand (March 2019). For Brazil and Peru, the Global benchmarking tool and survey assessments conducted by Pan American Health Organization (PAHO), served as the baseline. Baseline assessments occurred prior to agreement to participate in the 3S project. In all six countries the national pharmacovigilance centre is part of the national regulatory agency.

Country Plans

Using the baseline assessment, a holistic plan for PV was developed in each country according to the gaps identified in the baseline assessments to meet 3S objectives. The work plans covered (i) infrastructure and framework for a functional PV system; (ii) qualitative and quantitative aspects of PV data including roles and systems for data management; (iii) data analysis, signal generation and validation (hypothesis testing); (iv) emphasis on the use of local PV data and analyses; (v) collaboration between PV centres and public health programmes. The project was implemented sequentially in the countries with a tailored approach over a period of 9–18 months during which selected outcomes were evaluated every 3 months from baseline assessments.

3S Implementation and Data Sources

Technical support in implementing the 3S workplans started in 2018, as a collaborative effort between WHO, and technical partners the Medicines and Healthcare products Regulatory Agency (MHRA, UK) and the Uppsala Monitoring Centre (UMC). The key support has been through training, workshops, training materials, study visits, direct support in implementing new IT technologies, tools and support for quality PV data management systems, infrastructure and capacity for data analysis and decision making. Information was obtained through VigiLyze, a tool used to access information from VigiBase. VigiBase is the WHO global database of Individual Case Safety Reports (ICSRs) maintained by the WHO Collaborating Centre for International Drug Monitoring, the UMC. To date (July 2020) there are over 26 million ICSRs originating from 148 Member States held in VigiBase. ICSRs come from a variety of sources and the probability that a suspected adverse effect is drug-related is not the same in all cases.

Outcome Measures

The following outcome measures were considered. Information on the outcome measures were extracted using VigiLyze and by querying the national PV centres. Outcomes were monitored until May 2020 at the latest.

Changes from baseline in the number of structural indicators assessed using WHO PV preparedness tools before and after 3S intervention.

Changes from baseline in the reporting frequency for ICSRs at a national level of any drug aimed to treat malaria (Ethiopia, Thailand, Peru) or tuberculosis (Armenia, Brazil, Ethiopia), and the following specific medicines: bedaquiline (Armenia, Brazil and Ethiopia) and rotavirus vaccine (India). Since TFQ was yet to be introduced, primaquine was selected as a proxy because it is in the same drug class (8-aminoquinoline) and is indicated for the treatment of plasmodium vivax. Like TFQ, it can cause haemolytic anaemia in G6PD deficiency, which is one of the main risks of TFQ. The quality of ICSRs was evaluated using changes in VigiGrade™ completeness score. Completeness score is a quality measure comprising a number that accords a value to each field of clinically significant information contained in an ICSR. It starts at 1 for reports with information on time-to-onset, age, sex, indication, outcome, report type, dose, country, primary reporter and comments. It is reduced if information is missing and according to the clinical relevance of each missing dimension. An ICSR with a completeness score of > 0.8 is considered to be well documented [13]. In the 3S pilot, a completeness score of 0.5 was set as the minimum standard for quality of reports for selected countries, based on the average completeness score in the entire WHO global database of ICSRs (VigiBase). It should be noted that the minimum components of an ICSR are information on the reporter, country, suspected drug/vaccine and ADR. Having a high completeness score is the ideal needed to conduct a thorough clinical assessment, and ICSRs with a completeness score < 0.5 are still valid.

Changes in activities related to signal detection, risk minimization plans and periodic safety update reports (PSURs).

Results

Baseline Assessment and Application of 3S Principles

Table 1 provides information on the number of structural indicators pre- and post-intervention assessed using the WHO PV preparedness tool. In total, there are 21 structural indicators. Armenia (16/21), India (20/21) and Thailand (19/21) had most structures in place at baseline. Ethiopia met an additional 10 structural indicators following 3S interventions. Structural indicators in Ethiopia include the drafting and submitting of the pharmacovigilance directive. Such a directive will ensure that there is a national policy or legislation for PV and that there are legal provisions for Marketing Authorization Holders (MAHs) to monitor and report the safety and quality of their products. This gives the PV centre authority so that processes can be actioned. Tables 1 and 2 list the activities that contributed to this.

Table 1

Structural indicators at baseline and after 3S interventions for four countries following assessment using the WHO preparedness tool

Country*	Number of structural indicators met prior to 3S intervention	Number of structural indicators met after 3S intervention	Example of activities conducted to meet structural indicators
Armenia	16/21 (76%)	17/21 (82%)	Reformation of Pharmacy Committee with PV experts PV campaign and introduction of reporting technologies
Ethiopia	6/21 (29%)	16/21 (76%)	Revision of terms of reference for the Safety Review Committee Draft and submission of PV directive Regional centres, distribution of work so that there are enough staff members to carry out PV functions Making ADR reporting forms available (through technologies) in all settings Communication plan Coordination procedures of PV activities with different stakeholders Update of PV guidelines SOPs for conducting PV activities Existence of library or other reference sources for drug safety information PV courses organized by National PV Centre
India	20/21	20/21	Most structural indicators have been met, interventions focused on public health and process indicators: integrating public health programme and PV centre, and capacity for signal detection
Thailand	19/21	19/21	Most structural indicators have been met, interventions focused on public health and process indicators: integrating malaria programme with PV and building capacity to review PSURs and RMPs

3S Smart Safety Surveillance, ADR adverse drug reaction, PSURs periodic safety update reports, PV pharmacovigilance, RMPs risk management plans, SOPs standard operating procedures, WHO World Health Organization

*Baseline pharmacovigilance was assessed via the WHO Global Benchmarking tool in Brazil and Peru (see Table 2)

Table 2

3S interventions for Brazil and Peru following assessment using the WHO Global Benchmarking tool

Examples of activities conducted to meet structural indicators in Brazil and Peru
Brazil and Peru
Definition of adverse events of special interest and protocol for active pharmacovigilance of DR-TB patients
Review of the national electronic information system for DR-TB
Identification of the appropriate variables of interest existing in the local system and identification of missing key variables in the national information system
Proposal for modification/update. Support for code modification
Deliver informatic code with modifications to the country
Develop a module to import the information from the local system to a data management system (REDCap)a to enhance a platform for communication and analysis
Deployment of the updated system by the country
Permissions for data transfer
Update of data management system to make E2Bb compatible
Coordination mechanism setup between national PV centre and TB disease programme  Assignment of PV focal points in hospital sentinel sites

3S Smart Safety Surveillance, DR-TB drug-resistant tuberculosis, PV pharmacovigilance, TB tuberculosis, WHO World Health Organization

aREDCap (Research Electronic Data Capture) is a browser-based, metadata-driven EDC software and workflow methodology for designing clinical and translational research databases

bE2B: Data elements for transmission of individual case safety reports as per standards set by International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH)

As a result of activities listed in Tables 1, 2 and 3, over 500 people have been trained in PV in all six countries. The training focus varied between different countries due to differences in baseline PV capacity and gaps identified by the WHO PV preparedness tool. In Ethiopia and Peru, the focus was on reporting, analysis and signal detection while concepts around benefit/risk assessment, risk management plans (RMPs), PSURs and communication were the focus in Armenia, Brazil, India and Thailand.

Table 3

Interventions made using the 3S concept

3S Concept	Interventions made across the six countries
Priority products	Each country focused on one or two products according to disease burden and expected volume of use:
	Armenia (BDQ), Brazil (BDQ and TFQ), Ethiopia (BDQ and TFQ), Peru (BDQ, TFQ), Thailand (TFQ)
Holistic planning	PV workplans for each country were devised
	Strengthening activities at different points of the PV cycle included:
	PV campaign and awareness workshops, events and media (Armenia, Ethiopia)
	Training of signal detection and data analysis (all six countries)
	Formation of regional centres or sentinel sites at hospitals, teaching materials, training the trainer on PV and PV tools (Brazil, Ethiopia, Peru)
	Training of healthcare professionals (all six countries)
Tools and methods	Reporting app launched (Armenia, Ethiopia)
	Training on VigiFlow and VigiLyze (Armenia, Ethiopia, India)
	VigiFlow provided in official country language (Armenia—Russia)
	Active surveillance and cohort event monitoring (Brazil and Peru)
Leveraging available resources	Collaboration with NGOs, academics and public health programmes carrying out PV activities (All six countries, e.g. Armenia and MSF)
Reliance	Regional protocol for active surveillance (Brazil and Peru)
	Study visit to other regulatory agencies (Armenia, Ethiopia, India, Thailand)
	Review of RMPs submitted by MAHs for appropriateness and feasibility (Brazil, Peru and Thailand)
Work sharing	Creation of platforms to share data between public health programmes and the PV centre, by expanding access (restricted) to data management systems such as VigiFlow (Armenia, Ethiopia)
	Upgrade of REDCap* data management system for data collection (Brazil and Peru)
	Joint workshops (public health programme and PV centre) for signal detection and data analysis (Armenia, Ethiopia, India, Thailand)

3S Smart Safety Surveillance, BDQ bedaquiline, MAHs Marketing Authorization Holders, MSF Médecins Sans Frontières, NGOs non-governmental organizations, PV pharmacovigilance, RMPs risk management plans, TFQ tafenoquine

*REDCap (Research Electronic Data Capture) is a browser-based, metadata-driven EDC software and workflow methodology for designing clinical and translational research databases

Technological tools were enhanced in each of the six countries. Two countries introduced e-reporting and a mobile reporting app (Armenia and Ethiopia). Other countries modified existing data management systems so that they met international standards (E2B format in Brazil, Peru and Thailand); expanded access of the pharmacovigilance data management systems and databases to public health programmes (VigiLyze in Armenia and Ethiopia); formed reporting gateways/hubs to aid integration (Brazil), and used mobile technology to set up a coordination mechanism among national regulatory authorities (NRAs) and public health programmes using SMS (Brazil and Peru).

Brazil and Peru selected the cohort event monitoring methodology to monitor the safety of TB medicines. Countries were supported to develop specific guidelines, manuals and protocols, and to revise standard operating procedures that were out of date, missing or incomplete. These included a pharmacovigilance communication strategy, national PV guidelines and terms of reference for a Safety Review committee (Armenia, Ethiopia); guidelines for good practice for vaccine safety (India); and a manual for data management systems for cohort event monitoring (Brazil).

In order to learn and gain new ideas from regulatory peers in the practice of signal detection, conducting expert safety committee meetings and assessing RMPs, staff from the national PV centres in Armenia, Ethiopia, India and Thailand were supported on study visits to other regulatory agencies (EMA, MHRA, TGA).

To strengthen collaborations between the public health programmes, other stakeholders such as the pharmaceutical industry and NGOs and the PV centre, stakeholders were invited to be part of the PV planning meetings and to join training workshops.

Reports of ADRs with the use of priority products

Table 4 and Fig. 1 show the number of ICSRs for BDQ, primaquine (chosen as a proxy for TFQ) and rotavirus vaccine in the selected periods before and after the 3S implementation. Baseline assessment and implementation of 3S principles occurred at different times between 2018 and 2019 in different countries. The information was obtained before and after 3S intervention. (Before 3S: baseline; after 3S: baseline + 3 months, + 6 months, + 9 months, + 12 months, + 15 months, + 18 months). 3S support commenced in March and July 2018 for Armenia and India, respectively, March and February 2019 for Thailand and Ethiopia, respectively, and August 2019 for both Brazil and Peru.

Table 4

Number of Individual Case Safety Reports (ICSRs) for (i) priority products (bedaquiline, primaquine and rotavirus vaccine); (ii) public health programmes (malaria, TB and immunization); (iii) all products

Country		Number of ICSRs for BDQ and TB medicines in VigiBase									Information on exposure to priority product
		3S – 6 months	3S – 3 months	Baselinea	3S + 3 months	3S + 6 months	3S + 9 months	3S + 12 months	3S + 15 months	3S + 18 months
Bedaquiline (BDQ)
Armenia (baseline March 2018)	Bedaquiline	12	12	13	16	18	19	20	38	335	BDQ was initially available in 2013 for compassionate use. Donated through MSF. In 2015, 62 pts were exposed, in 2018, 70 pts. It is estimated 100 pts will be exposed each year
	All TB medicines	234	242	278	300	312	317	328	360	815
	All ICSRs	2445	2499	2576	2667	2726	2760	2816	2850	3341b
Brazil (baseline August 2019)	Bedaquiline	0	0	0	0	0	0				Not yet exposed?
	All TB medicines	127	133	147	172	195	219
	All ICSRs	53270	54326	64846	71601	79913	86217
Ethiopia (baseline February 2019)	Bedaquiline	6	21	26	26	26	26	27	27c		In 2016, 110 pts were taking either BDQ or delamanid
	All TB medicines	61	98	104	108	108	109	115	131
	All ICSRs	1229	1294	1346	1383	1398	1441	1469	1570
Peru (baseline August 2019)	Bedaquiline	31	38	38	40	42	43				In 2018, 152 pts were exposed to BDQ
	All TB medicines	4307	4544	4675	4747	4888	4945
	All ICSRs	71647	77072	80584	82639	86082	88422
Rotavirus vaccine
India (baseline July 2018)	Rotavirus vaccine	86	88	89	93	96	123	128	136	143d	Licensed in India in 2014
	All vaccines	4581	4733	4901	5070	5258	6178	6438	6564	6774
	All ICSRs	289172	302682	315610	327480	344161	369596	389799	405432	415176
Primaquine
Brazil (baseline August 2019)	Primaquine	1	1	3	6	196	670				Tafenoquine not available in any of the countries at the time of 3S interventions
	All malaria medicines	16	17	19	25	217	770
Ethiopia (baseline February 2019)	Primaquine	0	0	0	0	0	0	0
	All malaria medicines	6	6	6	6	6	6	6
Thailand (baselinee January 2019)	Primaquine	1246	1246	1246	1246	1246	1247	1247
	All malaria medicines	3033	3039	3043	3043	3043	3043	3060
Peru (baseline August 2019)	Primaquine	184	184	187	187	187	187
	All malaria medicines	374	376	384	387	389	389

BDQ bedaquiline, ICSRs Individual Case Safety Reports, MSF Médecins Sans Frontières, PTs patients, PV pharmacovigilance, TB tuberculosis

aPrior to start of 3S interventions

bOnly 14% of all reports in the national database were shared in VigiBase, however all ICSRs for BDQ have been shared in VigiBase

cThere are a further 50 ICSRs for BDQ shared with the PV centres, however these have not yet been shared in VigiBase

dOver 1000 ICSRs for rotavirus vaccine have been collected by the national immunization programme, but not yet shared with the National PV centre

eBaseline assessment in Thailand was made in January 2019, however agreement to participate in 3S initiative was established in March 2019

Fig. 1

Number of reports in VigiBase for the two priority products (BDQ, Rotavirus) and primaquine as a proxy for tafenoquine in target countries following 3S intervention (up to 18 months post-baseline). 3S Smart Safety Surveillance, BDQ bedaquiline, ICSRs Individual Case Safety Reports, PMQ primaquine

As 3S intervention commenced at different times in different countries, post-intervention data at 9 months are not yet available for Brazil and Peru, or at 12 months for Ethiopia and Thailand. Please note that agreement to participate in the 3S initiative in Thailand occurred 3 months after baseline assessment.

In Armenia, there is a clear increase in the reporting frequency for BDQ at 18 months following 3S implementation. This is also reflected in the total number of reports for TB medicines; hence, reporting has increased within the disease programme (Fig. 2). It is too early to know if a similar trend will be seen also in Brazil, Ethiopia or Peru for BDQ; however, there is a 50% increase in ICSRs for all TB medicines in Brazil 9 months after baseline (see Fig. 2).

Fig. 2

Number of ICSRs in VigiBase for all antimalaria and TB medicines in the target countries following 3S intervention (up to 18 months post-baseline). 3S Smart Safety Surveillance, ICSRs Individual Case Safety Reports, TB tuberculosis

Tafenoquine was not yet in the market at the time of the study, therefore it was not possible to obtain product-specific data. Instead, information on reporting for primaquine (PMQ) (as a surrogate for TFQ) is presented. There is a clear increase in reporting frequency for PMQ at 3 months following initiation of 3S principles in Brazil. Reporting continues to rise and, at 9 months post-baseline, there is over a 200-fold increase in the number of ICSRs for PMQ. This pattern is mirrored in reports for all antimalarials in Brazil. There is also an increase in the number of reports for antimalarials in Peru.

The number of ICSRs for rotavirus vaccine started to increase 6 months post-intervention. This increase is more marked amongst the number of ICSRs for all vaccines. The total number of ICSRs for all products has increased in India (6 months post-3S with respect to baseline), Thailand (at 9 months) and Brazil (steady increase post-3S) (Fig. 3).

Fig. 3

Number of all ICERs reported by target countries following 3S intervention (up to 18 months post-baseline). 3S Smart Safety Surveillance, ICSRs Individual Case Safety Reports

Reporting Quality

Table 5 provides information on the average completeness score of ICSRs for malaria, TB and vaccines in the target countries.

Table 5

Reporting quality at baseline, 6 and 12 months afterwards for TB, malaria medicines and rotavirus vaccine in 3S pilot countries

Country	Baseline	6 months after baseline	12 months after baseline
TB medicines
Armenia	0.91	0.88	0.89
Brazil	0.57	0.65
Ethiopia	0.8	0.73	0.49
Peru	0.69	0.83
Malaria medicines
Brazil	0.52	0.29
Ethiopia	0.76
Peru	0.62	0.92
Thailand	0.55		0.75
Rotavirus vaccine
India	0.69	0.6	0.72

3S Smart Safety Surveillance, TB tuberculosis

At baseline, all countries had an average completeness score > 0.5 for TB and malaria medicines, and rotavirus vaccine. Armenia and Ethiopia had an average completeness score > 0.8 for TB medicines at baseline. However, the completeness score for TB medicines in Ethiopia dropped 12 months after baseline to 0.49. Completeness scores increased 6 months post-baseline from 0.57 and 0.69 to 0.65 and 0.83 for Brazil and Peru, respectively. For malaria medicines, the average completeness score increased in Peru and Thailand from 0.62 and 0.55 to 0.92 and 0.75, respectively, but decreased in Brazil (0.52–0.29).

Signals and Risk Management Plans

In India, PSURs for rotavirus vaccine have been analysed and it was concluded that the vaccine appears to be safe and well tolerated in healthy infants. Based on this information, the immunization programme in India concluded that the benefit of the vaccination to prevent the majority of rotavirus cases continues to outweigh its risks. A white paper providing information on the results of a self-controlled case series study (SCCS) investigating the risk of rotavirus vaccine and intussusception was published, and concluded that there was no increased risk of intussusception associated with rotavirus vaccine [17].

As tafenoquine was yet to be available, for capacity building purposes participants from Thai FDA reviewed the RMP for tafenoquine submitted by the MAH in other settings (e.g. Australia) after signing confidentiality agreements. The Thai FDA also reviewed safety information in the national pharmacovigilance database for signal detection and investigated three signals related to other antimalaria medicines. The signals were communicated to various stakeholders and resulted in the revision of the Summary of Product Characteristics (SmPC) for primaquine and chloroquine.

In Armenia, the national PV centre is now performing PSUR assessments from 100% of the MAHs. The total number of signals detected pre- and post-3S workplan remained similar. The national PV centre is performing a descriptive analysis of the safety profile of BDQ.

In Ethiopia, qualitative causality assessments were made on all ICSRs for BDQ and both qualitative and quantitative case-by-case signal detection was carried out. Three potential signals were examined and were either not valid or evidence to support the signals were found to be weak.

Discussion

The 3S principles have been applied in different countries according to identified PV needs as assessed using the WHO PV preparedness tool. At baseline, Armenia, India and Thailand met a large proportion of structural indicators (76%, 95% and 90%, respectively). The focus of the 3S strategy for these countries was on strengthening processes such as data analysis, benefit/risk assessments, evaluation of risk management plans and the integration with public health programmes to prevent parallel PV reporting systems. Ethiopia had fewer than half of the structural indicators in place at baseline, and the focus was on building these structures as well as strengthening existing processes. Using active surveillance methods, reporting tools and building a regional network were the approaches taken by Brazil and Peru [14-16]. The different approaches taken in these different countries reflect the differences in PV infrastructure and processes at baseline in the individual countries. An increase in outcome measures was expected, and included number of reports for priority products (BDQ, rotavirus vaccine) and/or other medicines used in TB and malaria, total number of reports, reporting quality, signal detection and analysis of PSURs and RMPs. Implementation of 3S helped to prioritize gaps, leading to a phased PV strengthening approach in line with WHO benchmarking assessments and institutional development plans in each of the different settings to attain longer term outcomes and regulatory maturity.

An increase in the number of ICSRs was observed in VigiBase for BDQ in Armenia, and with other TB medicines, PMQ and antimalarials in Brazil. A slight increase in ICSRs for rotavirus vaccine was observed in VigiBase for India 6 months post-3S initiation and the reporting quality improved as well.

In Armenia, increased collaborations with the national TB programme and other stakeholders such as Medicins Sans Frontieres (MSF) has led to data sharing of over 300 ICSRs for BDQ collected by MSF between 2015 and 2018. These ICSRs were later shared with the WHO global database of ICSRs, VigiBase, causing a peak in the number of ICSRs for BDQ 18 months post-3S initiation. There were two main reasons for the 18-month lag-time for the ICSRs to be shared in VigiBase in Armenia: (i) a backlog of reports due to lack of data sharing between data collected by MSF through the national TB programme and the national PV centre prior to 3S implementation and (ii) manual input of data into the data management system at the national PV centre. Other reasons that can cause delays are follow up of missing information and performing causality assessments before submission to VigiBase. The strengthened collaborations and new technologies introduced such as the mobile reporting app and electronic reporting are expected to reduce this lag time in the future. Monitoring of the impact of these technologies on the number of reports and time taken to share reports would be important, to evaluate the true impact of these tools. Additionally, if stakeholder engagement was facilitated at an earlier stage, the backlog of reports would have been prevented. For example, in the case of Armenia, MSF shared data 4 years after initiating active surveillance. Catching up with these reports was resource intensive.

The coordination mechanism between the National PV centre and the TB programme in Brazil and Peru consisted of using text messages for the follow up of ADRs as part of active surveillance methodologies (cohort event monitoring). Reports of adverse events were transferred to the national database for ICSRs. This resulted in an increase of ICSRs and quality of reports for TB medicines in Brazil. This approach was even more successful in increasing ICSRs for antimalarials, in particular, primaquine. However, the use of text messages for follow up resulted in a decrease in reporting quality in Brazil with ICSRs for primaquine, as there are limitations on how much information can be submitted via text messages.

The workshops on signal detection, and evaluation of PSURs and RMPs, besides enhancing capacity for these functions, were also significant in establishing the practice of joint activities: PV staff of the NRA and national disease specialists (malaria, TB, immunization programmes) worked together to complement each other’s expertise in these joint activities. The pharmacovigilance centres in India and Thailand have started to be more confident when assessing PSURs and RMPs, and with signal detection and follow-up action. The joint reviews and advanced workshops were facilitated by technical partners from mature regulatory agencies. This and follow-on visits to other regulatory agencies (EMA, MHRA, TGA) facilitated peer-learning and the opportunity to observe good vigilance practices in different settings.

There was a slight increase in ICSRs for TB medicines at 15 months post-baseline (baseline: 104; 15 months post-3S: 131) from Ethiopia shared globally in VigiBase. Additionally, the PV centre had 50 ICSRs (at 15 months post-baseline) for BDQ that were provided by the national TB programme in Excel line-listing and had not yet been entered into the national data management system and VigiBase, causing a backlog. This could have been avoided if the use of PV reporting tools and integration of PV into the TB programme occurred earlier. Different channels to promote the use of these tools should also be explored. There is also a difference in the number of reports for TB medicines compared with malaria medicines, even though the same 3S work plan was applied for both programmes. Product characteristics may very well have driven the number of reports of adverse events (with more events observed with BDQ, for example). However, it is also possible that in general there is less integration of PV in malaria than in the TB programmes in the country, contributing to the observed differences in reporting numbers. BDQ approval was subject to the condition that it would be monitored for safety, with a positive impact on number of ICSRs. The active Drug Safety Monitoring (aDSM) database was established by the WHO Global TB (GTB) Programme and the WHO Special Programme for Research in Tropical Disease (TDR). The database was to serve as a repository for safety data on new TB drugs and regimens. The quality and quantity of ICSRs for BDQ in the 3S project may have been impacted by aDSM. While aDSM efforts had a positive knock-on effect on ICSR numbers for BDQ in the 3S project, the non-standardized Excel format used to transfer safety information from the national TB programme to the national PV centre could have contributed to the drop in completeness scores 12 months after baseline. BDQ has been supplied through donations with the support of NGOs that often have the obligation to conduct PV activities and report back to donors. In short, for BDQ a coordinated collaboration has existed between NGOs, national PV, national TB programmes and WHO, raising awareness for PV within the programme at the country level, albeit the parallel aDSM database at the global level. Similar collaboration and synergies between the various stakeholders should be considered for the integration of PV in malaria programmes in countries. In the other countries, in general, there was a mixed impact of 3S on reporting quality. In some countries (India, Thailand and Peru) reporting quality increased, and in others it decreased (Brazil and Ethiopia). As stated earlier, the decrease in reporting quality could be due to loss of information through parallel reporting systems with non-standardized data formats, or incomplete data collection through the use of text messages. It should be noted that various overlapping PV strengthening projects such as the European and Developing Countries Clinical Trials Partnership (EDCTP)-funded PAVIA and PROFORMA initiatives are being implemented in Ethiopia and improvement in performance indicators can be attributed to the combined efforts of these three projects [18].

As tafenoquine has not yet been introduced in target countries, the full effects of 3S intervention can only be assessed after the product becomes available and put into use. It is worth re-assessing outcome measures in the future, if and when tafenoquine becomes available for use, to evaluate the full impact of 3S on PV of tafenoquine in Brazil, Peru and Thailand. The effects of 3S on the impact of tafenoquine is expected to be demonstrated through the introduction of risk minimization measures, active surveillance activities and rapid signal detection of emerging safety concerns. Introduction of impact measures such as regulatory decisions and treatment policy using PV data related to the three products can take years to accomplish and was not feasible within the time course of this project, however, these measures would be important to monitor. On the other hand, changes to the PV structures and processes have been demonstrated, particularly in Ethiopia. It would be important to see whether these and other results from the 3S project can be sustained in the long term, which will be a real measure of the success of the 3S initiative.

Conclusion

The 3S principles have been applied in six different countries with diverse PV capacity and varying requirements for preparedness. The diversity of the different countries limited the extent to which the results could be compared across the six countries. On the other hand, the pilot project has demonstrated that the 3S strategy can indeed be applied to PV systems with different capacities. Various other initiatives with similar objectives of strengthening PV systems and capacity are also being implemented in parallel. The WHO Global Benchmarking approach that integrates PV as one of the key regulatory functions within an institutional development plan should provide the relevant framework to sustain the 3S strategy. The results of these collective efforts should be reviewed in the near future.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 21 kb)

Investments in activities to strengthen countries’ capacity to monitor the safety of novel medicines and vaccines can be optimized through the WHO Smart Safety Surveillance (3S) strategy, by focusing on priority products that are exclusive to the country and/or products with limited global experience.

The 3S concept was applied to six countries to enhance their ability to collect, assess and act on safety data on products of relevance to these countries. Although each country varied in the capacity to monitor safety of medicinal products at the start of the project, the study has demonstrated that the 3S strategy can be applied equally to pharmacovigilance (PV) systems with different capacities.