Co-creation of patient engagement quality guidance for medicines development: an international multistakeholder initiative.

INTRODUCTION: Meaningful patient engagement (PE) can enhance medicines' development. However, the current PE landscape is fragmentary and lacking comprehensive guidance.
METHODS: We systematically searched for PE initiatives (SYNaPsE database/publications). Multistakeholder groups integrated these with their own PE expertise to co-create draft PE Quality Guidance which was evaluated by public consultation. Projects exemplifying good PE practice were identified and assessed against PE Quality Criteria to create a Book of Good Practices (BOGP).
RESULTS: Seventy-six participants from 51 organisations participated in nine multistakeholder meetings (2016-2018). A shortlist of 20relevant PE initiatives (from 170 screened) were identified. The co-created INVOLVE guidelines provided the main framework for PE Quality Guidance and was enriched with the analysis of the PE initiatives and the PE expertise of stakeholders. Seven key PE Quality Criteria were identified. Public consultation yielded 67 responses from diverse backgrounds. The PE Quality Guidance was agreed to be useful for achieving quality PE in practice, understandable, easy to use, and comprehensive. Overall, eight initiatives from the shortlist and from meeting participants were selected for inclusion in the BOGP based on demonstration of PE Quality Criteria and willingness of initiative owners to collaborate. DISCUSSION: The PE Quality Guidance and BOGP are practical resources which will be continually updated in response to user feedback. They are not prescriptive, but rather based on core principles, which can be applied according to the unique needs of each interaction and initiative. Implementation of the guidance will facilitate improved and systematic PE across the medicines' development lifecycle. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

The value and necessity of engaging patients during the development and life cycle of medicines is increasingly recognised1–3; many discrete programmes have been initiated,4–8 which are often specific to a particular phase of drug development (eg, clinical trials, regulatory or health technology assessment) and to a region.9–13 For example, in the USA, the Food and Drug Administration has conducted patient-focussed drug development (PFDD) meetings to gather patient perspectives on specific conditions and their treatment more systematically.14 In Europe, the Innovative Medicines Initiative has launched PARADIGM (Patients Active in Research and Dialogues for an Improved Generation of Medicines) to develop a framework for patient input in early dialogue, research priority setting and clinical trials design.15 Many more initiatives exist and though complementary, these separate activities have resulted in a fragmented PE landscape, with a lack of continuity, efficiency and coordination. Investment in a holistic, integrated and sustainable initiative is required to cover the entire medicines lifecycle and to connect all stakeholders across geographies. Though there is no widely accepted definition of patients’ engagement in the medicines’ development continuum, in this paper, the term ‘PE’ refers to the active and meaningful involvement of patients and carers as active participants and collaborators in developing medicines. Patient Focused Medicines Development (PFMD; ) is a global multistakeholder collaboration of health stakeholders that aims to synergise PE efforts by working with stakeholders to co-create a meta-framework for PE that spans the entire medicines lifecycle. PFMD is taking a disciplined four-step approach to this co-creation, in order to build on existing work and prevent duplication. The steps are: (1) mapping and connecting the PE landscape to learn from existing efforts and identify needs or gaps; (2) convening multistakeholder workshops to co-create tools that meet identified needs; (3) reiterative refinement of tools to develop and pilot a draft meta-framework for PE and (4) creation of a practical PE Actionable Framework and implementation tools. Here we describe the first two steps of the approach culminating in the development of the PE Quality Guidance which is a core element of the meta-framework. The PE Quality Guidance was co-created as a practical tool to facilitate and improve the quality of PE activities across the medicines’ lifecycle. The principles outlined within the PE Quality Guidance are intended to be applicable for use by any stakeholder involved in the design, execution or evaluation of PE activities (regardless of role, position, experience or geographical location) and relevant for all interactions of stakeholders with patients.

Methods

Landscape review and identification of priority needs in PE

Multistakeholder Working Groups were established through invitation via PFMD and PFMD member networks, existing contributors to meta-framework co-creation activities and owners of PE initiatives. Invitees were required to have some PE experience and expertise in at least one of the medicine development phases. A literature search was conducted to provide Working Group participants with an overview of the PE landscape. Published literature and existing frameworks were identified using keyword searches on PubMed and British Medical Journal (BMJ) Open (timeframe January 2011–December 2016). Keywords were PE, patient and public involvement, clinical trials, patients benefit, PE benefit for pharmaceutical industry, PE benefit for patients, PE benefit for research. Articles relevant to PE in the context of medicines lifecycle that provided the most detail and granularity and were most informative for Working Group interrogation were identified.

SYNaPsE (SYNergising Patient Engagement) was used to augment published information available. SYNaPsE was designed as a platform for capturing PE practices and is a dynamic user-populated repository of objective, well-documented multistakeholder PE initiatives. It categorises and ‘maps’ PE initiatives and frameworks, organisations active in PE, experts and resources. Initiatives in SYNaPsE, follow a standard template for documentation of methods, challenges and outcomes, which gives a structure for understanding each initiative. Owners of relevant initiatives in SYNaPsE were contacted to provide more information on their initiative(s) to capture a deeper level of insight and understanding that then informed the co-creation process. Each Working Group included participants with expertise in the relevant phase(s) of medicines development (figure 1): there was no overlap of individuals in the different Working Groups. Working Groups were tasked with using their PE experience and review of the PE landscape to identify and prioritise PE needs. Co-creation of practical PE Quality Guidance to help stakeholders improve the quality of their PE activities and provision of illustrative examples of good practice were identified as priorities independently across Working Groups.

Figure 1

Process for development of Patient Engagement Quality Guidance (PEQG) tool.

PE Quality Guidance co-creation

Task Forces comprised Working Group participants (or their nominees) who volunteered to contribute to the iterative elaboration of the PE Quality Guidance, sharing their example PE initiatives and testing the PE Quality Guidance in development with these examples. Task Force nominees were invited based on their having the relevant expertise to complete prioritised actions and willingness and capacity to contribute. A Core Team comprising participants of Working Groups or Task Forces with extensive experience in PE was also established to facilitate finalisation of tools. There was intentional overlap with Working Group, Task Force and Core Team members to ensure continuity and incorporate learnings from each stage. A preliminary PE Quality Guidance tool was developed by Task Forces in an iterative process which included validation, feedback and refinement at each step, with input from the Working Groups (figure 1). The guidance introduces PE Quality Criteria to assess PE practices consolidated from published literature and co-developed further by Task Forces. The Quality Criteria were validated through retrospective application to real PE activity with feedback and refinement in an iterative process. Further feedback and validation were secured through public consultation.

Public consultation on draft PE Quality Guidance

A public online, survey-based consultation on the draft PE Quality Guidance was undertaken (from 20 November 2017 to 1 January 2018) to gather wider input from PE stakeholders. This consultation was open to all regardless of experience in PE. The survey had two sections: section one asked general questions about the PE Quality Guidance and its usability; section two asked detailed questions about the PE Quality Criteria specifically (see online supplementary file 1 for Survey Questions). Consultation was invited via the PFMD website and across stakeholder groups via Working Group, Task Force and PFMD networks; and through advertising (21 589 reach), media (40 259 reach) and a social media campaign. In addition, a separate focus group (organised and funded by Parkinson’s UK) with representation from patient organisations, a government health research agency, academic research, healthcare professionals and a regulatory agency was held in October 2017. Feedback was submitted for independent review by an external expert from the Working Group and a patient representative/academic from the Core Team to finalise the PE Quality Guidance.

Development of the Book of Good Practices

Examples of good practice were collected through screening of initiatives in SYNaPsE (entered between SYNaPsE launch in January 2017 and workshop commencement in September 2017) and via Task Force and PFMD networks. All initiatives independently submitted by PE initiative owners were eligible for inclusion in the Book of Good Practices. Preliminary selection criteria were pragmatic and included initiatives: being specific to PE in medicines’ development; involving more than one stakeholder group and having comprehensive description of methodology for PE practice (and outcomes for completed initiatives where available). Final selection for inclusion in the first iteration of the Book of Good Practices was undertaken by the Core Team who reviewed each initiative against the following key criteria: being extensively described by the owner using the PE Quality Guidance; demonstrating PE Quality Criteria as assessed by majority consensus (>half) of the multistakeholder Core Team; focusing on involving patients in decision processes around medicines development or lifecycle management; involving multiple stakeholder groups; owners/organisations agreeing to participate in the Book of Good Practices and being willing to provide additional information to further enrich the examples; being approved for public dissemination. The evaluation process was anonymised to avoid bias.

Results

Summary of PE Quality Guidance co-creation workshops

Nine Working Group, Task Force or Core Team meetings were held (November 2016–June 2018) involving 76 unique participants, representing 51 organisations (including patient charities, academic researchers, funders, pharmaceutical companies). Table 1 summarises objectives and outputs from these meetings.

Table 1

Summary of Working Group/Task Force workshops and key outputs

Date/location	Participants, N	Stakeholders represented*	Meeting objectives	Key outputs‡
November 2016,Brussels	24	PA (n=5); industry (n=13); independent experts (n=3); CRO (n=1); HTA (n=2)	WG1: identification of critical success factors for involvement of patients in discovery and preclinical phases of medicines development†	Identified focus areas, priority projects and key actions for improving PE efforts.Agreed on resources required to undertake prioritised actionsIndividual WG members committed to specific tasks and actions
November 2016, Brussels			WG2: identification of critical success factors for involvement of patients in clinical phases of medicines development (phases I–III)†
February 2017, Brussels	19	PA (n=4); external experts (n=2); university hospital representative (n=1); regulatory/HTA (n=3); industry (n=9)	WG3: identification of critical success factors for involvement of patients in regulatory submission process and post-launch activities†
May 2017, London	16	PA (n=4); academic/research (n=5); HTA (n=1); industry (n=6)	TF Europe: collection of PE initiatives in preclinical research and clinical phases of medicines development (phases I–III)†	Shared PE experiences in research and clinical developmentDeveloped basis of PE Quality Guidance guiding principles (Why, When, How, What)
June 2017,Basel	12	PA (n=5); industry (n=5); academic (n=1); global health organisation (n=1)	TF Europe: collection of PE initiatives in post-launch phase (including clinical development phase IV, adverse event reporting, and patient support activities) using the draft PE Quality Guidance tool	Application of PE Quality Guidance guiding principles to own postlaunch initiativesIdentified gaps in PE Quality Guidance guiding principles and proposed solutions
June 2017, Chicago	13	PA (n=4); industry (n=6); hospital-based PE manager (n=2); external expert (n=1)	TF North America (USA and Canada: review, refinement and validation of the draft PE Quality Guidance tool by retrospective application of the tool to existing PE initiatives covering all phases beyond clinical development phase I	Application of draft PE Quality Guidance tool to own initiativesIdentified gaps and co-created solutions to improve the practical use of the PE Quality Guidance tool based on their own experiences
October 2017,Brussels;February 2018,Brussels;June 2018,Brussels	10	PA (n=2); industry (n=3); academia (n=3); hospital-based PE manager (n=1); external expert (n=1)	Core Team: review and refine PE Quality Guidance tool; co-create Book of Good Practices; assess communications activity required	Finalisation of draft PE Quality Guidance toolEvaluation of PE initiatives for inclusion on Book of Good PracticesInput into communications plans

*Included participants with expertise in the relevant phases of medicines lifecycle.

†Phases of medicine life-cycle (The Drug Development Process. Available at: https://www.fda.gov/ForPatients/Approvals/Drugs/default.htm. (Accessed 21 June 2018); Lifecycle of a medicinal product - European Medicines Agency. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Presentation/2015/01/WC500180644.pdf (Accessed June 21, 2018).

‡Outputs covered Step 1 (mapping and connecting the PE landscape to learn from existing efforts) and Step 2 (multistakeholder co-creation of PE guidance and good practice identification) of the approach for development of a pilot meta-framework for PE.

CRO, clinical research organisations; HTA, health technology assessment; PA, patient associations/organisations; TF, Task Force; WG, Working Group.

Step 1: mapping and connecting the PE landscape to learn from existing efforts

Literature search yielded 239 results: 134 were available through PubMed, of which 12 were relevant or somewhat relevant to the subject and 105 through BMJ Open, of which 13 were relevant or somewhat relevant to the subject. Eight additional resources were identified and assessed to understand the progress of earlier developed approaches and concepts beyond the 2016 cut-off of the search period.1 6 7 16–20 The 33 relevant resources yielded seven conceptual frameworks/models (summarised in table 2). Although the importance of unifying PE approaches was highlighted within several models and frameworks, methodology gaps and variations were identified, for example, within objectives, outputs and applicability to different stages of medicines lifecycle. Working Groups agreed that taken together, the seven frameworks represented comprehensive tools that reflect the complexity of medicines research and development, and capture all stages of medicines lifecycle. They jointly demonstrated the following PE characteristics: diversity and representativeness, joint working and partnership, reciprocity, capabilities development and learning, continuity and sustainability, transparency, meaningfulness, metrics for impact assessment, standardisation and communication plans.

Table 2

Overview of frameworks and resources assessed for PE quality guidance co-creation

Framework	Format, objectives and approaches	Stakeholders	Key learnings and outputs
Clinical Trials Transformation Initiative33	Set of recommendations to improve participation of patient organisations in the work of clinical trial sponsors (the industry and academia), which includes meaningful engagement of patients throughout all stages of the Medicine Development Continuum	Multistakeholder	Early involvement Comprehensiveness Expectations of mutual transparency and confidentiality Diversity and representativeness Creation of standard metrics to assess partnership effectiveness
Patient Focused Drug Development (PFDD) Conceptual Framework or University of Maryland Centre of Excellence in Regulatory Science and Innovation (M-CERSI)34	Forum for patient groups, Food and Drug Administration (FDA), biopharmaceutical industry, payer and other organisations to voice views, challenges, activities and aspirations for PFDD, as well as future direction and opportunity for collaboration. The Framework Supports FDA PFDD concept (Prescription Drug User Fee Act VI and 21st Century Cures Act)	Multistakeholder	PFDD definition Defining engagement level by the gradual criteria: Patient role Continuity Meaningfulness Representativeness Temporality Importance of harmonisation among patient groups and national regulators/health technology assessment bodies Persistent challenge of measuring success
National Health Council/Genetic Alliance Framework35	Recommendations of the multistakeholder working group to reach a consensus on what it entails to meaningfully engage patients and identify key gaps and barriers in patient engagement across drug research, development and approval	Multistakeholder with action plans for:patients/community, industry, academiaregulators	Three categories of barriers: culture, communication and regulation Focus on meaningfulness and its criteria/questions to assess engagement Criteria of best practice examples Complex solutions to overcome the barriers
Patient-Centred Outcomes Research Institute (PCORI) engagement rubric36 37	PCORI engagement principles are developed to illustrate and provide guidance around how input from patients and other stakeholders can be incorporated throughout the entire research process with the particular focus on study planning, conduct and dissemination	Multistakeholder	Reciprocal relationships Co-learning Transparency, honesty and trust Partnerships throughout study planning, conduct and dissemination Supported by real-world examples
Evidence adoption framework38	This case-supported framework examines the factors that may affect the pace of evidence adoption and application into routine clinical practice. Aday, Andersen and Rogers’ framework/model of innovation diffusion and healthcare utilisation were used as baselines	Multistakeholder categorised by sectors:public, private and academic	Key factors to be taken into consideration: Validity, reliability and maturity of the science Communication of the science Applicability Economic drivers Integration into guidelines
FasterCures Value Framework39	This value-based framework was developed to determine gaps in assessing value of treatment options and based on the four earlier established frameworks/ approaches within oncology (American Society of Clinical Oncology, Institute for Clinical and Economic Review, MSK Drug Abacus and National Comprehensive Cancer Network)	Multistakeholder	Five domains of patient value and technical criteria: Patient preferences Patient-centred outcomes Patient and family costs Quality and applicability of evidence Usability and transparency
National Institute for Health Research (UK NIHR) INVOLVE21–23	The set of recommendations agreed by INVOLVE—the national advisory group to bring together expertise, insight and experience in the field of public involvement in research, with the aim of advancing it as an essential part of the process by which research is identified, prioritised, designed, conducted and disseminated. The six national standards for public involvement in research21 and the guidance on co-producing research project22 were based on the INVOLVE’s values and principles framework23	Multistakeholder with the focus on the joint working between researches, health care professionals and public sector	The six national standards for public involvement are: Inclusive opportunities Working together (which incorporates co-production) Support and learning Communication Impact Governance Key elements of co-producing a research project are: Sharing of power Reciprocity Respecting and valuing the knowledge of all those working together on the research Including all perspectives and skills Building and maintaining relationships

Step 2: Multistakeholder co-creation of PE guidance and good practice identification

Task Forces adapted and augmented the characteristics and related indicators from the frameworks into the PE Quality Guidance. The first section of the guidance tool explains how to use it: for planning a new PE project; for gap analysis to compare projects within an organisation and to assess the impact of ongoing or completed PE projects. The INVOLVE recommendations21–23 were agreed by Working Groups to be the most advanced, providing well documented and sufficiently detailed information (particularly on the practical ‘how to’ of PE) and were selected as the basis for development of the first iteration of PE Quality Criteria. The INVOLVE recommendations were developed for use in the UK and for advancing public involvement specifically in research and were therefore adapted and enriched with the analysis of PE initiatives collected in SYNaPsE and from Working Groups and Task Forces to reflect the needs of the broader PE community and stakeholders and to be relevant across all phases of medicines development. Seven PE Quality Criteria were agreed as a manageable number, comprehensive (without being overwhelming), relevant for the majority of PE stakeholders and accessible to all regardless of experience in PE. The PE Quality Criteria describe the core elements that should be integrated for individual PE initiatives or across several (related) initiatives that involve the same partners. They form the backbone of PE methodology and provide a set of basic principles to help standardise the levels of PE, assess the quality of PE in existing projects, and document and share outcomes in a standardised way. Descriptions for each of the seven PE Quality Criteria are summarised in table 3. Tools for assessment of initiatives (either planned, in preparation, ongoing or completed) against the PE Quality Criteria were also developed. Tools contain a definition, a rationale and questions for consideration by the initiative owners used for planning and/or evaluation purposes for each criterion.

Table 3

Patient engagement (PE) quality criteria summary and description

PE Quality Criterion*	Brief description and rationale	Practical illustrations†
1. Shared purpose	Brief description: This refers to the importance of all stakeholders agreeing on the project’s aims and outcomes before starting the project. Rationale: Early involvement is a key factor for quality of the process and includes the consideration of all perspectives in the early phase of planning.40	The organisation worked with the research team and people affected by Parkinson’s to create a survey to consult a large patient population about attitudes to stem cell therapy.1 A subset of survey respondents worked with the stem cell research team to explore and understand survey findings and were then invited to work as part of the research team to further develop the work and apply for funding.1
2. Respect and accessibility	Brief description: This refers to (1) respecting each other, and respectful interactions within the project to be established among partners, and (2) openness to and inclusion of individuals and communities (to the project) without discrimination. Rationale: A key quality aspect is the importance of securing a supportive culture that reflects that all stakeholders acknowledge the patients’ perspective as equally important to that of other professional or authoritative stakeholders.41 Practical steps must be taken to ensure access for all.42	In the workshops we used moderators and verbal and nonverbal communication using for example pictures, as feelings may be better expressed with images.2 We followed up in 1:1 meetings to clarify all individual perspectives and to understand if anything in the group was missed.2 We created a process to provide access for patients who were not comfortable participating in a group setting and those with limited mobility.3
3. Representativeness of stakeholders	Brief description: This refers to the mix of people involved, which should reflect the needs of the project, and the interests of those who may benefit from project outputs. Rationale: Ensuring optimal representativeness is demanding but essential for any PE activity43 and involves careful consideration of the selection of patient representatives. For example, appointed patient representatives in committees may often be particularly resourceful relating to their disease and treatment, and it becomes important to consider how to ensure perspectives of less resourceful patients are considered.	Different age groups, stages of disease, ethnicities and geographies were incorporated into the selection of patient/caregiver partners using a database of ~17 000 patients and caregivers.3 Members represent different socioeconomic backgrounds and diseases. Also, healthy young people are involved to ensure that they cannot have the bias of the disease and ensure that the general feedback is not connected with a specific condition.4
4. Roles and responsibilities	Brief description: This refers to documentation of agreed and ideally co-created roles and responsibilities, indicating that all aspects of project needs will be established upfront and revisited regularly. Rationale: Clarity on roles and responsibilities of all partners is essential for the implementation of equitable working practices that ensure PE opinions and expertise are respected and incorporated where possible into PE projects.	Contracts were mutually agreed on and entered into up front—before the start of the projects—and modified as appropriate along the way.5 Patients and caregivers were clear on their role in patient pathway mapping and how the data would be used. Patients and caregivers were also provided direction on how their input would be used with feedback loops built in throughout.3 Commitment documents were developed with patients and caregivers.3 Meetings and continued teleconferences were used to ensure that each member of the consortium knew what was expected and were accountable to everyone else.6
5. Capacity and capability for engagement	Brief description: This refers to (1) capacity as having relevant and dedicated resources from all stakeholders and (2) capabilities for all stakeholders to enable meaningful engagement. Rationale: It is essential that everyone has sufficient knowledge and skills to contribute effectively. This includes the professionals having sufficient PE knowledge and skills as well as patients having sufficient trials knowledge.	Each patient/caregiver completed an assessment on their engagement preferences and received coaching during this (project) to ensure they were comfortable interacting with various stakeholders.3 Training and tool kits have been developed for internal personnel, and external moderators are selected based on experience working with patients.5 The patients involved had the requisite expertise as established advocates and included people with health communications expertise as well as experience as patients.7
6. Transparency in communication and documentation	Brief description: This refers to the establishment of communications plan and ongoing project documentation that can be shared with stakeholders. Communication among stakeholders must be open, honest and complete. Rationale: Transparent communications throughout the project both internally and externally is essential to ensure credibility of process and findings. Publication of protocols and results of all trials is increasingly recognised as essential for the effective and ethical evaluation of clinical products.44 45	An involvement plan was developed for the project and shared. Other documentation shared between all stakeholders included: pre-read information; an immediate follow-up email detailing next steps; an intermediate follow-up document with interim findings. Further documentation will include long-term follow-up (6–12 months).1 Every month all patients were emailed for questions or updated with information.6
7. Continuity and sustainability	Brief description: This refers to the smooth progression of the project and efforts to maintain relationships with stakeholders beyond a single project. Rationale: Involvement of patients throughout the process as much as feasible, including aspects such as evaluation, dissemination and implementation can be very beneficial for the quality of the process.46 Additionally, ongoing commitment to PE and development of long-term relationships will enhance quality.	Patient representative organisation recommended that further partnership working results in patients becoming part of the research team, co-applicants and co-authors.1 All patients and caregivers that participated were kept abreast of development and continuous improvement of the initiative through electronic updates.3 The plan included launch in two illness areas with the intent, if successful, for expansion beyond those areas and in additional. The team, including patient experts, has been involved in those discussions. A third illness area was launched, and more are planned.7

*Adapted from: National Standards for Public Involvement. Available at: http://www.invo.org.uk/wp-content/uploads/2018/06/Public_Involvement_Standards_v1.pdf (Accessed 21 June 2018). Practical illustrations sourced from the Book of Good Practices (version 1). Available at: https://involvement-mapping.patientfocusedmedicine.org/book-of-good-practices (Accessed 21 January 2019).

†The superscript numbers (1–7) following each practical illustration refers to the example number in the Book of Good Practices: 1, Example 6; 2, Example 7; 3, Example 4; 4, Example 8; 5, Example 1; 6, Example 3; 7, Example 2.

Examples of good practice

Of 170 available initiatives screened, 40 were selected based on preliminary criteria then shortlisted to 20 using key criteria and anonymised for Core Team review. Eight initiatives were selected for inclusion in the first iteration of the Book of Good Practices. Table 4 provides a summary of the PE examples selected for the Book of Good Practices and results of their assessment using the PE Quality Criteria.

Table 4

Summary of the patient engagement (PE) examples selected for the Book of Good Practices and results of their assessment using the PE quality criteria

Case study	Stages of medicine lifecycle	Established by(geography)	Assessment results: number agreed that the case addresses a quality criterion
			Shared purpose	Respect and accessibility	Representativeness	Roles and responsibilities	Capacity and capability	Transparency and communication	Continuity and sustainability
1	Discovery, clinical development phases I–III, registration	Advisory group(Spain)	❖	❖	❖	❖	❖	❖	❖
2	Postregistration/launch	Biopharma company(USA)	❖	❖		❖	❖	❖	❖
3	Discovery	Patient organisation(USA)	❖	❖	❖	❖			❖
4	Clinical development (phase II–III), postregistration/launch including phase IV	Biopharma company(UK)	❖	❖	❖	❖	❖	❖	❖
5	Discovery	Patient organisation(UK)	❖	❖	❖	❖	❖
6	Clinical development (phases II–III)	Patient organisation(UK)	❖		❖	❖	❖	❖
7	Clinical development (phases I–III), postregistration/launch including phase IV	Biopharma company(Switzerland)	❖	❖	❖	❖
8	All	Biopharma company(Switzerland)	❖	❖		❖

Examples were assessed for inclusion in the Book of Good Practices by the Core Team which comprised 10 participants. It was required that the majority (ie, ≥6) had to agree that the example demonstrated each of the PE Quality Criteria and should be included in the Book of Good Practices. All examples included in the Book of Good Practices demonstrated each of the seven PE Quality Criteria (indicated by ·). In addition, greater than or equal to six Core Team members had to agree that a specific example particularly exemplified specific PE Quality criteria (indicated by ❖). All PE examples were initiatives involving patient organisations rather than unaffiliated patients.

PE quality guidance public consultation

The PE Quality Guidance public consultation resulted in 851 website visits and 67 responses from patients, patient advocates/organisations (n=25); pharmaceutical and biotech industry (n=19); research/academia (n=12) and other (n=11). The majority (69%) of responders were from Europe and the remainder from Canada, the USA, Algeria and Australia. Over 80% of respondents indicated that they were ‘advanced’ in terms of their PE experience (‘actively part of PE projects’), while 16% were a ‘beginner’ (‘currently I have no previous experience, but I am planning to do PE projects within the next year’) and, 3% ‘none’ (‘I have no experience in working with patients but would like to do more’).

Of 51 respondents, the vast majority (range 85%–96%) agreed or strongly agreed that: the PE Quality Criteria are useful for achieving quality PE practice; the descriptive language used is comprehensive and easy to understand, and the format is clear. Overall, 56% of respondents felt they did not need help to use the PE Quality Guidance. Specific feedback included needing practical examples and specific tips based on how others have used it. Most respondents (range 65%–84%) agreed or strongly agreed that the PE Quality Guidance could be used to: improve the quality and consistency of PE activities; better plan and develop PE activities; better assess the quality and impact of PE initiatives; identify gaps and opportunities for PE activities; capture and share learnings beyond the project or project team and structure work with partnering organisations (figure 2).

Figure 2

Public consultation feedback on patient engagement (PE) quality guidance.

Of 51 respondents, 86% agreed or strongly agreed that having practical PE guidance was important in their work and 90% indicated they would use such guidance. Almost half (49%) of respondents already use some type of PE guidance in their work. The vast majority (92%) of respondents believed that the PE Quality Guidance should be used by diverse health stakeholders (all those involved in medicines development, patients/patient organisations, pharmaceutical industry, regulators, payers, research and academia). Specific feedback noted that US and EU standards for engaging patients and patient organisations differ and should be recognised; the tool is a ‘shared guidance, understood and applied by all stakeholders’ involved in PE engagement and the tool may be especially useful for less experienced patient representatives. Overall, 21 respondents gave detailed feedback on the PE Quality Criteria including 85 comments on specific criteria and four general comments. Responses were generally evenly distributed across all seven PE Quality Criteria ranging from 67% (shared purpose) to 48% (respect and accessibility, representativeness of stakeholders). Comments ranged from simple agreement (eg, ‘This is crucial’) to detailed suggestions on each criterion and were overall positive. The feedback from all partners, representing varying levels of PE experience, has informed the final versions of the PE Quality Guidance. We have created the Book of Good Practices to provide examples of good quality practical applications of the guidance.

Discussion

The development of the PE Quality Guidance adhered to the core principle of respect for all stakeholders’ knowledge. It also ensured a diversity of viewpoints from: a systematic search of the literature; a wide range of stakeholders co-creating the guidance (table 1) and public consultation, thereby minimising potential bias. This practical guide aims to improve PE quality during planning and development of new medicine development projects or assess the quality and impact of ongoing/completed projects. Examples of good practice of the PE guidance were rated against the quality criteria to provide high-quality exemplars of PE in a range of contexts. This is not intended to be an exhaustive set of examples of how to apply the guidance, rather to provide real-world case studies showing the diversity of approaches and techniques that can be used to generate meaningful PE. Finally, the PE Quality Guidance was assessed for utility and practicality by a wide range of stakeholders. They determined it would be of use, particularly to those new to PE and those wishing to document, and quality appraise their PE processes. In response to feedback, the PE Quality Guidance incorporates good practice examples and tips to aid practical implementation of PE across diverse activities and groups. The PE Quality Guidance is a dynamic tool and is currently being piloted and implemented across several PE initiatives. Learnings and feedback will be used to further refine the tool and we will report on the outcomes of these pilots.

PE is not ‘one size fits all’ and consequently, the PE Quality Guidance is not prescriptive, rather it is based on core principles that should be adapted and applied according to the unique needs of each interaction and project. These principles are relevant and applicable beyond medicines development specifically, for example, also in clinical research settings. The Guidance can be tailored by adding specific modules to deliver consistently high-quality PE activities. This modular approach facilitates a long-term strategy for more systematic and meaningful PE. We hope the detailed specificity of the PE Quality Guidance and supporting Book of Good Practices provides not only sufficient structure to allow genuine engagement but also sufficient flexibility to be adapted to many types of medicines development, cultural considerations, national legislation and relevant specific circumstances. We also believe—and our survey confirmed—that the guidance principles have sufficient generalisability to also be applied to the development of other clinical interventions, such as educational interventions and medical devices.

We acknowledge potential limitations of this work: literature screening may not have identified all relevant existing efforts, and differences in Working Group individuals’ experience, confidence and specific challenges (such as language barriers and health conditions) may influence participation during workshops. However, different processes and checkpoints were established to ensure that all participants were able to contribute fully (eg, anonymously if desired). Though there were limited contributions from participants in developing countries, the distribution of contributors reflects current activity patterns across medicines development. Furthermore, our extensive search of literature for PE guidance effectively expanded the pool of contributors to the PE Quality Guidance tool. As such, we believe our guidance is as comprehensive and diverse as is possible. The resulting first iteration of the PE Quality Guidance intentionally focuses on PE characteristics that are relevant across stakeholder groups to be broadly relevant. The next phase of development involves customisation of the Guidance in identified priority activities. Examples for the Book of Good Practices were primarily identified through SYNaPsE: some initiatives (though worthy) may not have included sufficient information to meet initial inclusion criteria. As the PE Quality Guidance is used by research projects worldwide, it will also serve to collect good practices which will be published on the PFMD website to inspire new initiatives.

The need for PE that informs medicine development at all stages of the development process has been widely recognised. It is known that meaningful PE can improve the relevance of medicines in addressing symptoms of importance to patients and improve health outcomes.24–26 It could help address the observation that current medical research does not optimally address the needs and concerns of patients and the clinicians that treat them.27 28 However, it is important that this PE is genuine and not tokenistic in order to realise the potential benefits.29 30 The field of PE is growing and maturing and use of evidence-based methodologies and documentation is becoming increasingly important.31 32 The PE Quality Guidance provides a much-needed global tool for documenting and improving the quality of the process of PE across the medicines’ lifecycle and subsequently, the quality of desired outputs. Future uses should seek to apply scientific and robust methodologies to further refine the guidance and build the evidence base for the value of systematic PE.

We described the methodological four-step approach towards co-creation of a pilot meta-framework for PE: this requires multistakeholder, long-term investment in a global and collaborative venture. Investment in a foundation of mutual respect and trust through transparent governance, in building and synergising the PE ecosystem, and in driving implementation of co-created solutions to deliver better PE. The co-creation of this meta-framework and implementation toolkit (Steps 3 and 4) is underway and will build on outcomes from piloting the PE Quality Guidance and ongoing feedback to provide a comprehensive and actionable resource for more consistent and meaningful PE.