Proposals for Engaging Patients and Healthcare Professionals in Risk Minimisation from an Analysis of Stakeholder Input to the EU Valproate Assessment Using the Novel Analysing Stakeholder Safety Engagement Tool (ASSET).

INTRODUCTION: Understanding the impact of regulatory actions for medicines and enablers/barriers for positive health outcomes is fundamental to effective risk minimisation measures (RMM). Therefore, the Impact Strategy of the European Union (EU) Pharmacovigilance Risk Assessment Committee (PRAC) includes engagement with patient communities and healthcare professional (HCP) bodies regarding RMM. However, there is uncertainty on how best to obtain stakeholder input.
OBJECTIVES: The objectives of this study were to (1) analyse stakeholder input at a public hearing and dedicated meeting for the 2017-18 EU procedure on valproate teratogenicity and (2) draw proposals for enhancing PRAC engagement.
METHODS: For the content analysis, the novel 'Analysing Stakeholder Safety Engagement Tool' (ASSET) was developed with 21 themes in six domains (appropriateness, access, audience, compatibility, integrability, time), based on implementation theories.
RESULTS: Stakeholders provided a wide range of RMM proposals, some beyond the regulatory remit. Patients and most HCPs converged remarkably, but there was some divergence among HCPs on the informed choice objective, the therapeutic place of valproate, the RMM appropriateness, and RMM delivery to HCPs and patients. Ethical aspects emerged as relevant for regulatory decision making, and crucial input gaps were identified from an RMM implementation perspective. Nine pilotable proposals for PRAC were made regarding: (A) Agreeing on appropriate RMM with stakeholders and catalysing healthcare leadership for implementation; (B) Building-up stakeholder input on all elements critical to RMM implementation guided by the ASSET; and (C) Collaborating with all stakeholders for monitoring implementation and evaluating RMM.
CONCLUSIONS: New implementation theory-based approaches are promising for enhancing the valuable dialogue between regulators, patients and HCPs and achieving patient safety. EU PAS REGISTER NUMBER: EUPAS35947.

Key Points

Introduction

Objective of Pharmacovigilance

Pharmacovigilance comprises the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects of medicines or any other medicine-related problem [1]. Legal pharmacovigilance obligations apply to regulatory bodies and marketing authorisation holders (MAHs). Measuring the impact of regulatory decisions as well as identifying enablers and barriers for effective risk management are fundamental for the success of regulatory medicinal product-specific risk minimisation measures (RMM). Such RMM are imposed on MAHs, and patients and healthcare professionals (HCPs) are central stakeholders. Effectiveness evaluations of RMM may trigger adjusted or new RMM and thus serve pharmacovigilance quality management [2]. However, the effects of RMM are often poorly quantified because of limitations in data sources and the choice of methods used for evaluation [3-6]. Considering RMM as planned health interventions calls for formative research [7] into likely RMM effectiveness before regulatory decision making, to find the most adequate RMM. The established product management cycle for ensuring that the benefit-risk balance remains positive can hence be expanded with implementation of RMM in healthcare as part of the cycle, formative research into RMM options and implementability, and feedback from patients and HCPs on experiences with using the medicine and RMM in healthcare for the evaluation step (Fig. 1).

Fig. 1

Life cycle management of medicinal products for improving the benefit-risk balance developed from [17] and expanded with implementation of risk minimisation in healthcare, formative research into risk minimisation options and implementability, and feedback from patients and healthcare professionals on using the medicine and risk minimisation in healthcare for evaluation

European Union Initiative for Pharmacovigilance Impact

In the European Union (EU), routine RMM include the authorised product information, and additional RMM may be imposed as needed, such as controlled access, educational materials or pregnancy prevention programmes (PPPs). Monitoring their effectiveness is a legal requirement for MAHs in the context of risk management plans [8]. Considering options for RMM and monitoring outcomes of regulatory actions are within the legal mandate of the EU regulatory network [9, 10]. The Pharmacovigilance Risk Assessment Committee (PRAC) at the European Medicines Agency (EMA) therefore has a ‘Strategy for Measuring the Impact of Pharmacovigilance Activities’ [11], which aims at further developing proactive pharmacovigilance, including through active engagement and capacity building with patient communities and HCP bodies [12, 13].

Mechanisms of Pharmacovigilance Risk Assessment Committee Engagement with Patients and Healthcare Professionals

Within the existing EMA stakeholder engagement framework [14], the risk assessment process of PRAC contains various options for interacting with patients and HCPs. These consist of written consultations (targeted to organisations and individuals of the EMA’s stakeholder network) and meetings, i.e. dedicated meetings (with invited participants from the EMA’s stakeholder network and expert pool), scientific advisory group (SAG) meetings (with invited HCPs, academics and patients from the EMA’s expert pool) and public hearings (open to all EU citizens and organisations, including MAHs, as well as parties outside the EU). Since PRAC was established in 2012, all these engagement mechanisms have been used. The public hearing [15] is the most recently added tool to obtain input from a wide range of stakeholders.

Current Uncertainties Regarding Engagement

However, uncertainty remains about the optimal approach for engaging with patients and HCPs. Public hearings held by the US Food and Drug Administration have shown that contributions from patients and HCPs may be qualitatively different from what the data regulators usually review, and some argue that regulators need a specific process for soliciting and integrating the views from patients and HCPs for regulatory decision making [16]. International multi-stakeholder experts have also recognised the need for optimising this integration for strengthening benefit-risk assessment, communication/risk minimisation and evaluation (BRACE) [17].

Objectives and Methods

The aim of this study was to further develop active engagement and capacity building with patient communities and HCP bodies regarding effective risk minimisation through the use of a case study with the following objectives:

Analyse the input from patient and HCP representatives regarding RMM during a public hearing and dedicated meeting as major face-to-face interactions, including comparisons between these two engagement mechanisms and between the two stakeholder groups; and

Draw, based on the findings of (1), proposals to be considered by PRAC for enhancing their engagement process with patients and HCPs for effective risk minimisation.

The following methods were applied:

Case Study on the 2017–18 European Union Procedure on the Teratogenic Risk of Valproate

A public hearing was used for the first time by PRAC and followed by a dedicated meeting when assessing the teratogenic risk of valproate in 2017–18. Valproate is an active substance used for treating epilepsy and bipolar disorders as well as, in some EU member states, for preventing migraine headaches. However, it may be teratogenic when taken during pregnancy and cause malformations and lifelong neurodevelopmental harm in the child. The evidence for these effects has been accumulating over time and been subject to assessments at the EU level in 2002–04, 2014 and 2017–18 (see [18, 19] for the references to the evidence assessed by PRAC, i.e. the new committee established in 2012) (see Table 1).

Table 1

Pharmacovigilance Risk Assessment Committee (PRAC) assessments of the teratogenic risk of valproate

EU procedure	Assessment outcome
EU referral review by PRAC in 2014	PRAC agreed to strengthen restrictions for the use of valproate in female patients who are pregnant or can become pregnant, to advise on effective contraception in female individuals for whom valproate is the only option after trying other treatments, and to require prescribing physicians to fully inform female patients about the teratogenic risk and regularly review their treatment. PRAC agreed to update the authorised product information accordingly and to require provision of educational materials (e.g. patient information booklet, checklist for prescribers, checklist for patients, information acknowledgement form) to all HCPs and female patients prescribed valproate in the EU. A dedicated meeting with concerned women and a SAG meeting with experts in neurology, psychiatry, obstetrics and paediatrics, and patient representatives had been part of this EU procedure prior to decision making on the actions [20]. The option of the public hearing had not yet been implemented and available to PRAC in 2014.
EU referral review by PRAC in 2017–18	In March 2017, a new EU procedure was initiated by PRAC to review the effectiveness of the RMM introduced in 2014 and the potential need for further RMM, triggered by concerns and data that the 2014 RMM were not as effective as intended. During this procedure, PRAC used all available options for involving patients and HCPs, i.e. a written consultation launched in March 2017, a public hearing on 26 September, 2017, SAG meetings, and a dedicated meeting with patients and HCPs on 13 October, 2017. In the written consultation, the questions posed by PRAC to patients and HCP representatives related to receipt of information, utility of the educational materials, risk awareness, prescribing, other relevant healthcare behaviours and the implementation of the 2014 RMM overall. The responses consisted of real-world feedback and data, including from surveys that targeted patients and HCP organisations had conducted among their members. At the same time, information was collected by EMA from the regulatory authorities in EU member states. Almost all member states confirmed the dissemination of materials to healthcare settings, but data on the effectiveness of the RMM were limited. The MAHs for valproate-containing medicinal products were obliged to provide all data on RMM effectiveness available to them, and these included data across several member states from healthcare research organisations. The information collected from all stakeholders at the early stage of the procedure indicated that the effectiveness of the 2014 RMM might have been impaired by their delayed, incomplete or inadequate implementation in healthcare. Taking into account all evidence and further stakeholder input during the public hearing and dedicated meeting (for the PRAC questions, see Appendix 1 of the Electronic Supplementary Material [ESM]; for the list of speakers, video recording and written submissions for the public hearing, see [21]), PRAC agreed further product information updates and RMM, i.e. restrictions of use in female patients and a comprehensive PPP, which includes individualised care, communication and disease management. This procedure was concluded on 31 May, 2018 [21].

EU European Union, HCP healthcare professional, MAHs marketing authorisation holders, PPP pregnancy prevention programme, RMM risk minimisation measure(s), SAG scientific advisory group

Recording Input from Patient and Healthcare Professional Representatives

Stakeholder input to PRAC is provided within public hearings as written submissions and in speech (published as submitted/video recording on the EMA website). Input within dedicated meetings is provided in speech only (reflected in the public PRAC assessment reports on the EMA website). For their decision-making, PRAC uses the written submissions and, for speech input, summaries of responses and discussions prepared by EMA. The case study used the written submissions and the video recording as the records to be analysed for the public hearing and verbatim transcripts (specifically made for the study as exhaustive as possible in handwriting or typing by EMA staff members) of the speech input as records to be analysed for the dedicated meeting.

Developing a Content Analysis Tool Based on Implementation Theories

For analysing the input from patients and HCPs, a content analysis tool was developed prior to the public hearing and dedicated meeting, and independent from the sets of PRAC questions used at the hearing/meeting (for the PRAC questions, see Appendix 1 of the ESM). Content analyses are qualitative research and can be used to describe phenomena not yet well understood and to be explored by organising and unifying content items by themes [22, 23]. Such studies apply an interpretative approach to content and are concerned with understanding the meanings that persons attach to actions, decisions, beliefs and values as well as with understanding the mental processes persons use to make sense of the world around them [24]. Qualitative research is still rare in regulatory pharmacovigilance contexts but has previously been undertaken within the EU regulatory network when conducting media monitoring and has proven utility for preparing for vaccine safety communication with journalists, members of parliaments and the public [25].

For this case study, the themes were developed with a deductive approach based on existing implementation theories (see Table 2). Implementation science is the study of methods to promote the adoption and integration of evidence-based practices, interventions and policies into routine healthcare and public health settings [26] as complex systems [27]. This includes studying why implementation succeeds or fails [28]. Implementation hence covers all processes from disseminating knowledge and skills to practicing medical care for positive patient and public health outcomes. Implementation theories were chosen as the basis for the content analysis tool because the 2017–18 EU procedure was triggered by concerns that the 2014 RMM were not sufficiently effective and the study therefore assumed that strengthened RMM implementability would be a target outcome of the procedure. It was also considered that implementation is key to RMM effectiveness for any product and that this approach might be generalisable for future application.

Table 2

Implementation theories underpinning the content analysis tool

Implementation theories
SMART criteria for strategic health communication
Purpose: Objective setting for health communication interventions within a model of stakeholder participation Objective: Reach defined audiences with tailored multi-media approaches and help them to adopt healthy behaviours [29] Theory: 5 criteria for strategic objectives have been coined ‘SMART’: (1) specific in terms of the behaviour called for; (2) measurable for the purpose of evaluation; (3) appropriate for remedying the problem and agreed amongst stakeholders; (4) realistic, taking into account the present systems and potential for change; (5) time-bound, i.e. having a set point in time by which the action should and can be implemented [29, 30]. These criteria have already been adapted for a strategic approach to communication about the risks and safe use of medicines [31] and here have an additional emphasis on the appropriateness of the action for remedying safe use issues
KAP model for evaluating learning
Purpose: Improving learning interventions in complex environments Objective: Evaluate changes in KAP, i.e. knowledge, attitude and practice, and help learners to adopt new behaviours for better performance [32] Theory: Knowledge instruction alone is insufficient to influence behaviour, and knowledge, attitude and behaviour are interconnected. The attitude component acknowledges the crucial role of beliefs, values, emotions and motivation in learning and behaviour change [32]. The KAP model has been applied in the area of health [33, 34], and implementation science refers to this model [27]
Implementation science-based pharmaceutical risk management
Purpose: Advancing risk management of medicines using implementation theories and communication strategies with patient- and stakeholder-centredness Objective: Deliver RMM to HCPs and patients and help them to adopt safe medicine use behaviours Theory: 3 features for optimal RMM implementation: (1) organisation and delivery; (2) process measures; and (3) sustainability. Recommendations relate to: for (1): collaboration and governance, readiness to change, championship, training and assistance; for (2): reach, adoption by healthcare facilities, fidelity to RMM design, available resources/acceptable burden; for (3): sharing of promising practices, continuing of training and assistance. Underpinning principles are: adaptable, multi-faceted, multi-level, using socio-ecological models, aiming at healthcare workflow compatibility [35]

HCP healthcare professional, KAP knowledge, attitudes, practices, RMM risk minimisation measure(s), SMART specific, measurable, appropriate and agreed, realistic, and time-bound

The content analysis tool was finalised with 21 themes in six domains, providing for structure and granularity of the analysis and taking into account that different actors may be responsible for different steps of implementation (for details on the development of the content analysis tool, see Appendix 2 of the ESM). The tool was named the ‘Analysing Stakeholder Safety Engagement Tool’ (ASSET) (see Table 3).

Table 3

Analysing Stakeholder Safety Engagement Tool (ASSET)

	Domain	Theme	Theme code
1	Appropriateness of RMM for harm reduction	1.1 Type of RMM components and their specific capacity for harm reduction when fully implemented	Specific capacity
		1.2 Ethical considerations regarding RMM	Ethics
		1.3 Agreement within HCP communities responsible for patient care that RMM is appropriate	HCP agreement
2	Access-to-treatment-related acceptability of RMM	2.1 Deprivation of the medicine and its benefits	Benefit deprivation
		2.2 Impact of RMM on patient adherence to medication	Adherence to medicine
		2.3 Benefit-risk balance of alternative treatments to which RMM might channel	Alternative benefit-risk balance
3	Audience-tailored approach to RMM	3.1 Current knowledge, attitude and practice of HCPs	Prior HCP KAP
		3.2 Current knowledge, attitude and practice of patients/parents/carers	Prior patient KAP
		3.3 Information and training needs and interests of HCPs	HCP needs
		3.4 Information, training and counselling needs and interests of patients/parents/carers	Patients needs
		3.5 Preferences for tools informing HCPs	HCP tool preference
		3.6 Preferences for tools informing patients/parents/carers	Patient tool preference
4	Compatibility of RMM with healthcare structures and resources	4.1 Compatibility of RMM with the structures of the healthcare system	HC compatibility
		4.2 Compatibility of RMM with clinical guidelines	Guideline compatibility
		4.3 Burden of RMM for the patient	Patient burden
		4.4 Compatibility of RMM with healthcare reimbursement policy	Reimbursement compatibility
5	Integrability of RMM in healthcare processes	5.1 Responsible delivery of RMM to HCPs	Responsible delivery to HCP
		5.2 Responsible delivery of RMM to patients	Responsible delivery to patient
		5.3 Measurability of RMM implementation and evaluation for improvement action	Measurability
6	Time-bound requirements of RMM	6.1 Immediate RMM implementation in healthcare	Immediate effect
		6.2 Sustainability of RMM implementation in healthcare	Sustainable effect

HC healthcare, HCP healthcare professional, KAP knowledge, attitudes, practices, RMM risk minimisation measure(s)

Analysing Input from Patient and Healthcare Professional Representatives

Inclusion of Input

Only RMM-related content of the recorded input was analysed. Given the scope of implementation covering all processes from disseminating knowledge and skills to practicing medical care [26], any input from patient and HCP representatives relating to regulatory or healthcare action that may help to reduce harm was considered RMM-related content, which went beyond the regulatory meaning of RMM.

Exclusion of Input

Content not relating to RMM was not analysed, nor the input from the MAH speaker at the public hearing. Body language was also not part of the analysis (body language includes posture, movement, gestures, facial expression, eye contact, and tone of voice [36] and may hence add content or express agreement or disagreement with statements of others).

Content Analysis of Included Input

The content was summarised and organised into a tabulated format of the ASSET. Regulatory terminology was applied consistently in these summaries, but some verbatim quotes were included for illustration (e.g. ‘contraindication’ as the regulatory term and in addition “ban” [in certain clinical circumstances] as a quote). This terminological consistency facilitated the comparisons between stakeholder groups and engagement mechanisms. Additionally, it was analysed as to which implementation-relevant aspects were provided in response to which question, to gain insight into how far PRAC questions provided for focus (e.g. if all representatives provided input categorised into a certain ASSET theme in response to only one question, the question would be considered specific and focused from an implementation perspective).

The analysis was not conducted quantitatively in terms of how much discussion time was taken for each aspect, or how often certain content was repeated, or how many representatives provided similar content or agreed explicitly with a statement made. This methodological choice was made to treat each statement made by a stakeholder as equally important to be heard and considered for the decision making on RMM, keeping it with the qualitative research approach.

Identification of Input Gaps

As part of the content analysis, ASSET themes for which only limited stakeholder input had been obtained or the discussion did not lead to agreement on RMM and convergence on solutions for overcoming potential barriers to RMM implementation were identified as input gaps.

Results

Content Analysis of the Patient and Healthcare Professional Input for the 2017–18 European Union Procedure on Valproate Teratogenicity

The content analysis yielded the following key findings (for details on the analysis, see Table 4 and Appendix 3 of the ESM).

Table 4

Summary results of the content analysis using the Analysing Stakeholder Safety Engagement Tool (ASSET) with identification of gaps of the patient and healthcare professional (HCP) input to the public hearing on 26 September, 2017 (PH) and dedicated stakeholder meeting on 13 October, 2017 (DM) for the 2017/18 European Union procedure on valproate teratogenicity

	Domain	Theme code	Summary content analysis results and input gaps
1	Appropri-ateness of RMM for harm reduction	1.1 Specific capacity	At PH, there were statements that 2014 RMM contained appropriate information, but that there was a lack of dissemination and further implementation steps, and also statements on insufficiency of 2014 RMM. 23 tool proposals for RMM strengthening were made: restriction of indication; limitation of exposure in women of childbearing potential; limitation of exposure in pregnancy; patient reviews/counselling/pregnancy planning provided by physician; contraception; dosing; plasma level monitoring during pregnancy; co-medication during pregnancy; qualified prescriber; prescribing checklists; prescription validity time; dispensing process; dispensing software alarms; product information warning; pictogram; patient information card; information receipt form; pharmacy checks of patient's information receipt; risk information provision at family planning counselling; financial incentives for HCPs; risk information and awareness campaigns; clinical guidelines; data collection. At DM, 22 proposals or support to PRAC proposals (incl. calls for a PPP) were made: restriction of indication; limitation of exposure in women of childbearing potential/fertile men; limitation of exposure in pregnancy; patient reviews/counselling/pregnancy planning provided by physician; contraception; dosing; plasma level monitoring during pregnancy; co-medication during pregnancy; specialist care; controlled access programme; qualified prescriber; named patient prescription; restriction of prescription volume; coded prescribing; pack size harmonisation; product information warning; pictogram; patient information card; information receipt form; risk information and awareness campaigns; RMM implementation for Internet sale; data collection. RMM proposals were similar at PH and DM, but some PH proposals were not followed up at DM, where a few other proposals were discussed (marked in italics), some of which triggered by the PRAC questions. At DM, additional RMM needs owing to concerns over exposure through semen were raised. There was overall convergence between PTR and HCPR, but among HCPR, there was divergence regarding specialist prescribing, dose monitoring and adjustment during pregnancy, folic acid co-medication and the information receipt form. PTR made a call for “more collaborative action across EU”. Proposals for RMM tools were provided in response to all PRAC questions Input gaps: Evidence gained from previous RMM experiences and evaluations and convergence on if and/or how specific proposed RMM have the capacity to reduce risks and/or improve health outcomes, provided full implementation can be achieved
		1.2 Ethics	Aspects of ethical dimension and related dilemmas for HCPs were discussed and specific questions raised that centred on the complex situation of maternal need for treatment, the adverse effects for the child exposed in-utero and the mother caring for the child, and the risks of under-/untreated epilepsy for the woman and the unborn child. Overall, there was convergence at PH and DM and between PTR and HCPR for an informed choice of women, with divergence of one HCPR. The ethical dimension of the RMM decision was reflected in the responses to most PRAC questions Input gaps: Full convergence on RMM objectives in ethically challenging situations
		1.3 HCP agreement	At PH and DM, lack of full HCPR agreement with proposed RMM manifested in divergence among HCPR arising from differences in views on the specific capacity of some RMM to reduce harm (see 1.1), the therapeutic place of valproate and the underlying evidence (see 2.1), the informed choice objective (see 1.2) and the delivery of RMM to HCPs (see 5.2). There was explicit support from some/across HCPR for RMM proposals such as product information and the patient card. The views were voiced in response to a range of PRAC questions Input gaps: Agreement of HCPR on most appropriate RMM to reduce harm (see 1.1), the therapeutic place of valproate (see 2.1), the informed choice objective (see 1.2) and the responsible delivery of RMM to HCPs (see 5.2)
2	Access-to-treatment-related acceptability of RMM	2.1 Benefit deprivation	This theme was discussed more at PH than DM, mainly by HCPR, and revealed divergence regarding the therapeutic place of valproate and the underlying evidence. There was however convergence among HCPR and PTR that the valproate treatment option needs to be preserved and RMM should not to deprive women in need of valproate benefit Input gaps: Convergence of HCPR on the therapeutic place of valproate
		2.2 Adherence to medicine	There was convergence between PTR and HCPR at PH and DM that inconsistent or inadequate risk information from healthcare or social channels can lead to patient non-adherence to treatment
		2.3 Alternative BR	There were similar concerns voiced by HCPR at DM and PH over risks of alternative medication, albeit at PH there also was a statement on the potentially better benefit-driven BR of alternative medication, which also related to the divergence regarding the therapeutic place of valproate (see 2.1). At DM, this input was provided in relation to PRAC questions on the mandatory checklist and barriers to switching
3	Audience-tailored approach to RMM	3.1 Prior HCP KAP	This theme was discussed more at PH than DM in line with the respective PRAC questions. The discussion showed convergence between PTR and HCPR that current HCP knowledge, attitude and practices, incl. risk communication skills, are not in accordance with 2014 RMM implementation, and there were also HCPR statements that RMM implementation does not happen in healthcare. This was supported by evidence from patient organisation surveys. Input relevant to this theme arose from responses to multiple PRAC questions
		3.2 Prior patient KAP	This theme was discussed more at PH than DM in line with the respective PRAC questions, and the discussion showed convergence between PTR and HCPR with the evidence of patient organisation surveys and published literature findings that current risk knowledge in women is insufficient and that patients with knowledge have an attitude to avoid valproate. The DM discussion added patient non-adherence to epilepsy treatment in general as an issue. Input relevant to this theme arose from responses to multiple PRAC questions
		3.3 HCP needs	There were similar information needs, mainly on the management of risks and adverse effects, identified at PH and DM by PTR and HCPR as well as a training need for communication skills to provide risk information and counselling to female patients. Input relevant to this theme arose from responses to multiple PRAC questions Input gaps: Actors and leadership for developing and providing HCP training for risk management and communication skills
		3.4 Patient needs	There were similar risk information and counselling needs, incl. on the potential life impact of teratogenic effects, pregnancy planning, contraception and alternative parenting, identified at PH and DM by PTR and HCPR, alongside how such information needs to be presented. However, there was one HCPR statement also informing about the possibility of having a healthy child with valproate, which was opposed by PTR and hence constitutes divergence in the information women should be informed about and how this should be framed. Aspects relevant to this theme were raised in response to almost all PRAC questions Input gaps: List of risks and other information items for patients, their presentation and framing
		3.5 HCP tool	There was similar input at PH and DM, more during PH though, in response to multiple PRAC questions and mainly from HCPR themselves, providing preferences and proposals for a range of user-tested print, electronic and conference-based tools for providing RMM information to HCPs. One HCPR committed to dissemination via the scientific journal of his professional organisation Input gaps: Actors and leadership for developing and providing dissemination tools for HCPs, incl. proposals from HCPR to use all their available channels
		3.6 Patient tool	There was similar input at PH and DM in response to multiple PRAC questions, mainly from PTR themselves on preferences and proposals for multiple user-tested tools for providing RMM information to patients, with a visual risk presentation in the form of a pictogram being a major preference
4	Compat-ibility of RMM with healthcare structures and resources	4.1 Healthcare compatibility	There was similar input at PH and DM from PTR and HCPR in response to almost all PRAC questions, highlighting that lack of resources, coordination across healthcare and insufficient availability of specialist care in at least some MS constitute major barriers to RMM implementation Input gaps: Proposals for RMM that are compatible with existing healthcare structures and resource constraints, or proposals on how to free/make needed resources available in the overall care of the given patient population
		4.2 Guideline compatibility	At PH and DM, in response to the PRAC question regarding the GP role, PTR and HCPR considered RMM implementation in clinical guidelines important Input gaps: Leadership and commitment for transposition of RMM in clinical guidelines
		4.3 Patient burden	HCPR at DM, in response to PRAC questions on signed consent forms and pregnancy prevention plans, voiced concerns over patient burden associated with monthly pregnancy testing or with the annual information receipt forms by women without childbearing potential as barriers to RMM implementation Input gaps: Proposals for making RMM bearable to patients
		4.4 Reimbursement compatibility	At DM, PTR and HCPR identified, in response to PRAC questions on pregnancy prevention plans and obstacles to switching, the absence of reimbursement schemes for contraception as a barrier to RMM implementation Input gaps: Proposals for RMM compatible with the existing reimbursement policies or proposals for making needed resources available
5	Integra-bility of RMM in healthcare processes	5.1 Responsible delivery to HCP	This was discussed in a similar manner at PH and DM, but more at PH, with a call to strengthen or mandate RMM delivery to HCPs and divergence in views on who is the responsible organisation for RMM delivery, i.e. only public bodies or MAHs too. Some PTR and some HCPR accepted MAHs as responsible for delivery of RMM information, while some HCPR preferred regulatory bodies as senders and/or voiced concerns over undue marketing by MAHs, and some PTR preferred independent advisors to HCPs Input gaps: Planning of delivery of RMM to HCPs in accordance with roles and responsibilities of MAHs, regulatory bodies, and other stakeholders and methods to convey the mandatory nature of RMM to HCPs incl. how to ensure dissemination, information adoption and leadership for behavioural change of HCPs required for RMM implementation
		5.2 Responsible delivery to patient	There was a strong call from PTR and HCPR at DM and PH for the delivery of RMM to patients in a responsible and coordinated manner by the healthcare system in the framework of annual reviews and counselling. However, there was divergence among HCPR regarding who can take the responsibility of information delivery to patients, i.e. GPs and/or specialists, or pharmacists instead of GPs, or nurses. This theme was discussed in response to almost all PRAC questions Input gaps: Planning of delivery of RMM to patients in accordance with roles and responsibilities of MAHs and regulatory bodies, and integration of RMM tools into healthcare processes with agreement among HCPs and leadership on collaborative roles and responsibilities of HCP types
		5.3 Measurability	This was discussed at PH and DM with similar calls from HCPR and PTR for RMM that are outcome-driven and evaluated. More detailed methodological proposals were provided at DM in response to a specific PRAC question, albeit the theme also emerged in response to two other questions. Quantitative methods, e.g. surveys and health record/prescription data analysis, and qualitative methods as well as RMM implementation in clinical guidelines and compliance audits were proposed by PTR and HCPR. Some divergence consisted in one HCPR questioning the validity of surveys because of potential recall bias, specifically in the given patient population Input gaps: Improvements to survey methodology and identification of a responsible actor for healthcare audits
6	Time-bound require-ments of RMM	6.1 Immediate effect	Similar input from HCPR and PTR at PH and DM, in response to the PRAC question of how to ensure that information is provided to patients at a suitable time, stated that RMM needs to be implemented and information provided to patients urgently Input gaps: Healthcare process and leadership for immediate outreach to patients
		6.2 Sustainable effect	Similar input from HCPR and PTR at PH and DM, in response to the PRAC question how to measure risk awareness, stated that information to HCPs and patients requires regular updating Input gaps: Healthcare processes and leadership for maintenance of RMM implementation

BR benefit-risk balance, DM dedicated stakeholder meeting, GP general practitioner, HCP healthcare professional, HCPR HCP representative(s), incl. including, KAP: knowledge, attitudes, practices, MAH marketing authorisation holder, MS member state(s) of the EU, PH public hearing, PTR patient representative(s), PPP pregnancy prevention programme, PRAC Pharmacovigilance Risk Assessment Committee, RMM risk minimisation measure(s)

Input by Themes and Input Gaps

The input covered all themes relevant to RMM implementation, albeit to a variable extent. Stakeholders detailed severe shortcomings in the implementation of the 2014 RMM with evidence.

A wide range of future RMM was proposed, but without presenting much evidence gained for valproate or other medicinal products from previous RMM experiences and evaluations on the capacity to reduce harm when fully implemented. Overall, the favoured RMM objectives were: to limit exposure to valproate in epileptic women of childbearing potential; to reduce risks of birth defects due to valproate exposure in utero; and to facilitate an informed choice for women regarding pregnancy. Many RMM proposals went beyond the regulation of medicines and related to healthcare (theme 1.1). Ethical questions relevant to applying RMM in healthcare were also raised (theme 1.2).

A clear call was made to not deprive patients from the benefits of valproate where needed (theme 2.1) and to not jeopardise patient adherence to treatment in these cases (theme 2.2).

A number of specific needs for tailoring RMM for patients and HCPs were highlighted to overcome current shortcomings in knowledge, attitudes and practices (themes 3.1 and 3.2) and address patients’ expectations for information and counselling (themes 3.4 and 3.6) and increase HCP skills to communicate with patients and deliver RMM (themes 3.3 and 3.5). Input gaps were left regarding who should develop and provide HCP training on risk management and communication skills (theme 3.3); which information items precisely should be conveyed to patients and how to frame them (theme 3.4); and who should provide leadership for developing and disseminating tailored RMM to various HCP types via all available channels (theme 3.5).

Burden for patients associated with RMM, resource constraints and contraception reimbursement policies were discussed as barriers to implementing RMM. However, the discussion did not try to outline RMM that could be bearable for patients (theme 4.3), or how RMM would be compatible with the existing healthcare structures (theme 4.1) and reimbursement policies (theme 4.4). The need for incorporating RMM into clinical guidelines was highlighted, and some, but not all, HCPs voiced commitment for advocating RMM within their respective professional communities (theme 4.2).

There was some, but no resolving discussion on how proposed RMM could actually be best delivered to HCPs (theme 5.1) and be integrated into healthcare processes, and who will be responsible for healthcare leadership to ensure behavioural change for the benefit of patients. Only the patient card was discussed in the context of the dispensing process (theme 5.2). Patient and HCP representatives called for RMM to be driven by clear objectives and followed up through auditing and effectiveness measurement. They provided good suggestions for evaluative methods, but some uncertainty remained about how to improve survey methods. A responsible actor for the proposed healthcare audits was not identified (theme 5.3).

Timebound requirements were highlighted with a view to an urgent outreach to patients for immediate effectiveness (theme 6.1) as well as to sustainability of RMM effectiveness over time (theme 6.2), but without defining actors and processes to achieve these goals.

Input by Stakeholder Groups

Between patient and HCP representatives, there was a remarkable convergence in views and favoured RMM objectives overall.

Divergence occurred among HCP representatives in relation to capacity of some RMM for harm reduction (theme 1.1); the informed choice objective questioned by one HCP representative (theme 1.2); the therapeutic place of valproate and the underlying evidence (theme 2.1); and the delivery of RMM to HCPs (theme 5.2) [see theme 1.3 focussing on agreement of HCP representatives overall].

Input in Relation to the Pharmacovigilance Risk Assessment Committee Questions

In each EU procedure, questions posed by PRAC to stakeholders are specific to the given medicinal product and safety concern, but at the same time are kept broad to allow stakeholders to elaborate on aspects that may be new to regulators. In the case of valproate, the input did not always match the question asked at the given moment, and most themes were brought up multiple times in response to different questions. This was to be expected during the public hearing where the PRAC questions were high level and posed with the option for stakeholders to address them in an integrated response. The partial disconnect between questions and answers was more unexpected during the dedicated meeting, where the PRAC questions were more specific on RMM implementation to provide structure to the meeting agenda. However, discussing one issue sparked further ideas from stakeholders, and it was the spirit of the dedicated meeting to not interrupt such flows of thought. Furthermore, specific implementation-relevant aspects sometimes matched more than one question. This indicates that the PRAC questions were still rather broad in comparison to the specificity of themes critical to implementation.

Analysis of the Input by Engagement Mechanisms

The dedicated meeting was deliberately held after the public hearing to allow more detailed discussions. When comparing these two engagement mechanisms, input was overall similar in terms of the breadth of items and their content.

The public hearing yielded more (in the qualitative sense) input on: benefit deprivation due to RMM (theme 2.1); prior knowledge, attitudes, and practice of HCPs and patients (themes 3.1 and 3.2); HCP tool preferences (theme 3.5); delivery of RMM to HCPs (theme 5.1); and immediate implementation (theme 6.1). At the end of the public hearing, there was time left to speak for both speakers and participants who had attended the hearing but had not been a speaker. Their remarks re-enforced or expressed agreement with previous statements and did not contain new content.

The dedicated meeting yielded more (in the qualitative sense) input on: patient tool preferences (theme 3.6); patient burden of RMM (theme 4.3); barriers to RMM implementation due to reimbursement issues (theme 4.4); and RMM effectiveness measurement (theme 5.3).

These differences between the public hearing and the dedicated meeting may reflect the differences in the questions posed.

Proposals for Enhancing Patient and Healthcare Professional Engagement for Effective Risk Minimisation

From the key findings, three proposals were drawn for enhancing the PRAC process of engagement with patients and HCPs in relation to RMM and will be presented to PRAC, to be considered for piloting in 2021.

Proposal A—Agreement on Risk Minimisation Measures

Content Analysis Finding

The content analysis identified shortcomings in convergence among HCPs on which RMM are appropriate, i.e. capable to reduce harm when fully implemented and ethically acceptable, as well as on some other themes of RMM implementation.

Proposal: Agreeing on Appropriate Risk Minimisation Measures with Stakeholders and Catalysing Healthcare Leadership for Implementation

While discussing divergent views on RMM is vital and valuable, the lack of achieving convergence is detrimental, given that implementation science has identified agreement between stakeholders as a key success factor for health interventions [29, 30]. The responsibility for agreeing and deciding on RMM as part of the marketing authorisation lies with the regulatory body. Actions A.1, 2 and 3 are proposed to PRAC for achieving agreement across stakeholders on which RMM are appropriate prior to regulatory decision making and catalysing leadership of HCPs for RMM implementation in healthcare. For a start, this might improve dissemination in terms of consistent and synchronised messages on new RMM from both regulators and HCPs, which would correspond to a need expressed by patients. Further, clinical guidelines might be updated more quickly, and risk communication protocols and training could be introduced for strengthening skills and confidence of HCPs to tackle sensitive communication with patients, corresponding to a need expressed by HCPs. Ultimately, full integration of RMM in healthcare processes is necessary for RMM effectiveness. It must be recognised that agreement of all stakeholders may not always be possible, owing to uncertainties about the given risks or RMM implementation, persisting divergency among stakeholders, or differences in healthcare systems and situations between EU member states. Therefore, transparency towards stakeholders and the public about the agreements reached, but also about dilemmas and causes of divergence might increase the understanding of stakeholders for PRAC decisions and increase the readiness for implementation, even if a stakeholder does not fully agree.

Conduct meetings with stakeholders that facilitate creation of common ground, shared problem ownership, and exchange of expertise and result in stakeholder agreement on appropriate RMM

Motivate leadership in HCP organisations and obtain their commitment for taking responsibilities for immediate and sustainable RMM implementation in healthcare, including updates of clinical guidelines and HCP training

Increase transparency on agreements reached with stakeholders or dilemmas faced by PRAC as part of already published rationales of PRAC decisions on RMM

Proposal B—Build-up Towards Implementable Risk Minimisation Measures

The content analysis demonstrated that although the PRAC questions at the dedicated meeting addressed RMM implementation, stakeholder input left gaps regarding some implementation-relevant themes and how to overcome barriers to implementation. Further, responses relating to certain RMM implementation themes were spread over different questions. Hence, the discussion seemed not to have been focused and covered the complete implementation process from the regulatory beginning to the end in healthcare.

Proposal: Building-up Stakeholder Input with a Structured Focus on all Elements Critical to Risk Minimisation Measures Implementation

Build-up of comprehensive stakeholder input on all elements critical to implementation of RMM is proposed for decision making and design of RMM. Actions B.1, 2 and 3 include using and refining the ASSET for the new purpose of soliciting input on the implementability of RMM comprehensively at dedicated meetings. This proposal complements the EMA’s lesson learnt [37] to keep questions simple, focused and limited in number at public hearings.

Use ASSET as a checklist for PRAC questions and as a discussion guide for chairpersons of dedicated meetings, to ensure soliciting stakeholder input on all elements critical to RMM implementation

Identify through stakeholder discussions barriers and enablers of successful RMM implementation in healthcare

Focus discussion outcomes on proactive solutions to overcome foreseeable barriers, integrate RMM in existing healthcare processes and allocate responsibilities for delivery of RMM to HCPs and patients

Proposal C—Collaborative Risk Minimisation Measures Evaluation

Patient and HCP representatives described a lack of implementation of the 2014 RMM and called for an effectiveness evaluation of future RMM. The content analysis also identified a lack of existing research and evidence on the predictable effectiveness in terms of RMM objectives as well as some doubts of stakeholders over the validity of current methods for effectiveness research, in particular surveys.

Proposal: Collaborating with all Stakeholders for Monitoring of Implementation and Evaluation of Risk Minimisation Measures Effectiveness

Actions C.1, 2 and 3 are proposed to strengthen implementation monitoring and high-quality RMM effectiveness research in collaboration with all stakeholders. In addition to EMA monitoring of MAH compliance with RMM requirements and results from RMM effectiveness studies imposed on MAHs, real-time feedback on receipt of RMM and further implementation could be obtained from patients and HCPs. For this purpose, PRAC could intensify their ongoing engagement with patient and HCP members of PRAC, the EMA Patients’ and Consumers’ Working Party, the EMA Healthcare Professionals’ Working Party and the wider EMA stakeholder network. Furthermore, studies undertaken by the EMA or academia to investigate intended and unintended RMM outcomes could be planned in collaboration with patient and HCP organisations. More active roles of patients and HCPs in research, such as defining research topics or collecting data, may be developed by applying participatory research approaches proven beneficial in other areas such as health development [38, 39]. For example, research is needed on the effectiveness of pictograms, as patient representatives called for this at the public hearing, but preliminary evidence has raised doubts over potential risks of misinterpretation of teratogenicity pictograms. The quality of RMM effectiveness studies will be supported with EU good pharmacovigilance practices currently under revision by PRAC. A standard to ensure high-quality reporting of such studies, the RIMES statement, has already become available [40] and its use will be encouraged in the revised EU good pharmacovigilance practices.

Agree with MAH timetables for their RMM delivery to patients and HCPs and RMM effectiveness studies, and publish timetables and updates from compliance monitoring on the EMA website

Investigate together with patient and HCP representatives whether and how RMM has reached them and been implemented in healthcare by means of a methodologically sound survey, focus group, interview and pharmacoepidemiological studies

Promote conduct and high-quality research into RMM effectiveness through calls for research and methods development

Discussion

Novel Tool ‘ASSET’

The 2017–18 EU procedure for valproate was triggered by concerns about insufficient implementation of the 2014 RMM, and therefore the ASSET for analysing stakeholder input was developed based on implementation theories and pharmacovigilance experience. It successfully structured stakeholder input to the public hearing and dedicated meeting, identified input gaps and allowed comparisons. The ASSET might have utility for supporting regulatory decision making on implementable RMM (see below). Further, the ASSET is suggested for piloting with a new purpose, i.e. as a checklist and discussion guide for ensuring stakeholder input on all elements of implementation. Future application may lead to revisions of the tool, allowing wider use for different products, safety concerns, jurisdictions and purposes.

Limitations of the Content Analysis

Only RMM-related stakeholder input was analysed, not other statements such as those on the teratogenic risk itself or the lives of affected children and mothers, and not questions on accountability. This was in line with the study objectives. However, a content analysis of the other input could clarify stakeholder expectations towards governments and inform initiatives for meeting these expectations.

Regarding the analysed input, it cannot be excluded that the real divergence among stakeholders was larger than that found in the analysis. This is because only explicitly voiced or otherwise obvious (e.g. due to other statements) disagreement was recorded; and where no explicit disagreement was voiced, this was interpreted as convergence. Furthermore, body language, which may express agreement or disagreement with the statements of others, was not part of the analysis.

Validity of the Content Analysis

A content analysis needs to be valid to serve as evidence for policy changes. Validity means that study results truthfully reflect the phenomena studied [41] and is mainly achieved through the reliability of the analytical methods. This refers to how well the analytical method accounts for the information inherent in the data and how sensitive it is to distinctions in the data for correctly representing the meanings of the content [42]. For theory-based studies, reliability is established through a coding scheme adherent to its underlying theory and through adherence of the analysis to the coding rules [43]. In this case study, the ASSET adhered to implementation theories (see Table 2 of Appendix 2 of the ESM). During its development, the draft tool was consulted with two co-authors and reviewed by another co-author because peer examination of research design is a further means to ensure the truth value of qualitative research [44]. The draft tool was then tested, which helped finalise the coding rules through clarifying the precise scope of each theme. Specific attention was paid to adhering to these scopes during the case study. Hence, a range of actions was taken to ensure valid results.

Utility of the Content Analysis for the 2017–18 European Union Procedure on Valproate Teratogenicity and Predictive Value in Relation to Ethical Aspects

The content analysis of the stakeholder input was started while the 2017–18 EU procedure for valproate was ongoing, and its preliminary results were shared with PRAC in January 2018 prior to their decision making. The preliminary results predicted that ethical aspects (theme 1.2) would become highly relevant for the final discussions and decision making on a PPP: there was overall stakeholder convergence on risk and family planning counselling to support informed choice for women. HCP representatives raised questions about how to act if a woman without an alternative to valproate is or plans to become pregnant, or does not need contraception because a pregnancy is excluded otherwise, or if there are doubts over adherence to a PPP. Most HCP representatives felt that these women, and their unborn children, cannot be deprived from life-saving treatment because of regulatory requirements and that they need to be supported by the healthcare system regardless of their choices. These HCP representatives experienced a dilemma between the medical needs of their patients and wanting to adhere to safe use requirements and regulation. These preliminary results from the content analysis, in addition to the regular records of the stakeholder input (see Sect. 2.2), were appreciated by the PRAC rapporteurs, as the stakeholder feedback was very extensive and listening by any individual can naturally be selective. They stated that the preliminary results reassured them in their overall understanding of the stakeholders’ views and offered an additional perspective on the stakeholder input that broadened their awareness of potential issues.

Added Value of a Public Hearing for Regulatory Decision Making

The public hearing provided PRAC with multi-perspective insights into problems with real-world healthcare. This was instrumental for PRAC deciding on new RMM, including a new PPP and a visual warning on the teratogenic risk on the outer packaging. A survey and a lessons learnt analysis conducted by EMA reported that the majority of PRAC members said that “they learnt something new” and 79% specifically said that it would “make a difference to the assessment of valproate”. Overall, EMA concluded that the hearing improved the quality of the assessment [37]. In particular, the PRAC rapporteurs were able to confirm that the 2014 RMM were considered by patients as an “excellent toolkit”, but that more needed to be done for the delivery of RMM to patients and HCPs.

Added Value of a Public Hearing for the Texture of Engagement

Different from written consultations, face-to-face interactions provide a higher degree of what has been named ‘texture’ in a conceptualisation of pharmacovigilance engagement developed for the PRAC Impact Strategy. There, texture describes “the interactive dynamics of what engagement feels like, what it means to people, and how this shapes motivations to engage and change behaviour—based on values, emotions, (mis)trust and rationales” [45]. The EMA survey after the public hearing demonstrated the added value of the hearing in terms of texture. One hundred percent of the stakeholder respondents felt that the hearing was a “positive experience” and 88% felt that it “would make a difference to the Committee recommendations” [37]. An individual patient representative sees in public hearings a major sign of regulatory commitment and trustworthiness and describes them as a chance for patients to make impactful presentations that have visibility to the world (François Houÿez, European Organisation for Rare Diseases [EURORDIS], personal communication, 3 March, 2020). As public hearings are broadcast live by the EMA and the recording remains available on the EMA website, the EMA provides transparency of the texture of the interactions, beyond publishing the written submissions to the public hearing.

Although the case study was not a discourse analysis on the use of language nor a sentiment analysis, the following reflections are offered here regarding the texture of the public hearing on the basis of the content analysis and related observations. The face-to-face interaction allowed for the expression of mutual thanks for the invitations and the input in a personal manner. The chairperson could create a common ground among all stakeholders at the beginning of the hearing by highlighting its significance and objectives. Some patient representatives shared personal experiences of the medical, practical, and emotional impact of the valproate teratogenicity for the affected children and families. Furthermore, individual or others’ complaints were conveyed about not having received the 2014 RMM and delays in risk identification and warnings; questions were raised regarding trustworthiness, responsibilities and MAHs’ accountability; references were made to interactions with parliaments and political contexts; and clear demands were directed at the authorities. Patient representatives brought samples of valproate tablets dispensed in plastic bags without package leaflets or in blister packages without the name of the active substance printed on the blister, and also samples of patient cards, which they described as “small and flimsy”. Seeing such physical samples might have been more impactful than photos. Moreover, there was the opportunity for PRAC to ask for details and confirm understanding with the speakers directly. Some speakers chose to present in their own language if this was not English and have their input translated by a trusted person into English as the working language of the hearing. Regulators verbally expressed empathy and appreciation of the time taken, the altruism and the courage of those affected to share personal experiences in the public domain. They also acknowledged the seriousness of the problem to solve and confirmed to apply the regulatory tools in the best way possible “to get this right”. Although body language was not part of the analysis, postures, facial expressions, and tone of voice of regulators corresponding to attention, seriousness, and empathy were observed by the authors and have probably contributed to the engaging texture of the interaction. Likewise, there was verbal and bodily expression of sympathy and commitment to change from some HCP representatives vis-à-vis the patients.

Broad Scope of Risk Minimisation in Healthcare

The content analysis highlighted that stakeholders contributed experiences, expectations and suggestions for risk minimisation beyond the remit of regulatory bodies. These related to healthcare, such as family planning counselling, healthcare software and nationwide risk awareness campaigns. Hence, a view on risk minimisation broader than what can be demanded through regulatory action had to be applied in this case study, and the term RMM had to be used with this broader understanding beyond its regulatory definition. Even for RMM in the regulatory sense, their implementation requires dissemination and changes in knowledge, attitude and practice in healthcare, which are not subject to the regulation of medicines. It is unlikely that patients and HCPs can have the in-depth knowledge of medicine regulation for differentiating between the respective remits of regulatory bodies and healthcare supervision. Most importantly, the interactions between patients, HCPs and regulators are meant to be an open dialogue about RMM and their implementation in the context of other measures for patient care. This broader insight may allow regulators to design RMM that are implementable in real-world healthcare.

Conclusions

Enhancing engagement between regulators, patients and HCPs is considered one of the most important areas of progress for ensuring a positive impact of pharmacovigilance [46]. This case study analysed the stakeholder input at the first ever EMA public hearing and subsequent dedicated meeting. For this analysis, the novel ASSET was developed on the basis of implementation theories and could be used for further purposes. The analysis identified several important gaps in stakeholder input and research relating to critical elements of RMM implementation. From the results, three proposals with practical actions were derived to be considered by PRAC for piloting, to establish how these actions can most efficiently (i.e. in a prioritised risk-proportionate manner) contribute to improving regulatory stakeholder engagement and RMM decision making. Among the three proposals, achieving stakeholder agreement on RMM and catalysing healthcare leadership for RMM implementation may be the most challenging for regulators because this concerns connecting more with patients and HCPs, their real-world experiences and potential for behavioural change within healthcare systems. Inspired by one of the widely acclaimed principles for being effective in life, “Begin with the end in mind” [47], the approach of applying implementation theories to RMM may be coined as ‘Begin with the implementation-end in mind’. This is expected to enhance mutual engagement, trust building and respective leadership of all stakeholders through jointly envisioning the intended positive RMM outcomes upfront, and then designing together RMM that can change medicine use for patient safety and health.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 22 kb)

Supplementary file2 (DOCX 25 kb)

Supplementary file3 (DOCX 64 kb)

European Union regulators highly value the input received from patients and healthcare professionals at their first ever public hearing and dedicated stakeholder meeting on valproate birth defects; however, they have identified crucial gaps in input and research into risk minimisation effectiveness by using a new approach presented here based on implementation science.

Derived proposals for regulators to remedy these gaps aim at catalysing stakeholder agreement and healthcare leadership for the implementation of risk minimisation measures, building up more comprehensive stakeholder input, and collaborating with all stakeholders for monitoring implementation and evaluative research.

Pilotable and practicable actions are presented to enhance the dialogue between regulators, patients, and healthcare professionals and to lead to risk minimisation measures that are implementable in real-world healthcare for patient safety.