A review of the sampling methodology used in studies evaluating the effectiveness of risk minimisation measures in Europe.

PURPOSE: This review aims to describe the sampling methodology used in studies assessing effectiveness of risk minimisation measures (RMMs) in the European Union.
METHODS: The European Union electronic Register of Post-Authorization Studies (EU PAS Register) was searched to identify studies that assessed the effectiveness of RMMs and recruited a target population of healthcare professionals (HCPs), sites or patients. Studies with both protocol and report were included and data was extracted from these documents to describe study characteristics and variables involved in the sampling methodology.
RESULTS: Out of 1092 studies finalised between June 2017 and May 2019, 17 studies were eligible for review. Thirteen were surveys, three chart reviews and one combined both methodologies. All the 17 studies recruited HCPs/sites and 8 of them also recruited patients. The most common rationale for country sampling was market uptake (10/17), while for HCP/site sampling, it was representativeness of the prescribing practices (14/17). Only a minority of the studies (4/17) provided supporting evidence to inform this theoretical framework. HCP/site sampling frames were mainly network of physicians (5/17) or HCP databases (5/17), with only one study providing a detailed description of the sampling frame. HCPs were selected mainly using probabilistic sampling (10/17) and patients using non-probabilistic sampling (6/8). Only a few studies compared participating with non-participating HCPs/sites (5/17) and patients (3/8). Eight studies reported that their results were generalisable.
CONCLUSIONS: Overall, the study documents provided insufficient details to understand the rationale behind the sampling decisions. More standardisation and guidance in reporting the sampling strategy and operational considerations applicable to these types of studies would support transparency and facilitate the evaluation of representativeness of the study results.

This is a review of studies using EU PAS register on sampling methodology used in risk minimisation effectiveness.

Overall studies provided a rationale to justify country selection and sampling strategy of HCP/sites as a theoretical framework for sampling methodology.

Supporting evidence used to inform this theoretical framework and description of the sampling frame was missing in the majority of studies, as well as information on how non‐respondents differed from respondents.

More standardisation and guidance in reporting these elements would support transparency and facilitate the evaluation of the representativeness of the final results.

INTRODUCTION

Risk management plans (RMPs) have been introduced to the European Union (EU) since 2005, becoming legally enforceable with the 2012 Pharmacovigilance (PV) Legislation review. One of the main pillars of RMPs is risk minimisation. According to Module XVI of Good Vigilance Practice (GVP), risk minimisation measures (RMMs) are ‘interventions intended to prevent or reduce the occurrence of adverse reactions associated with the exposure to a medicine, or to reduce their severity or impact on the patient should adverse reactions occur’.

Routine RMMs apply to all medicinal products and include instructions provided as part of product reference information (e.g., Summary of Product Characteristics [SmPC] for healthcare professionals [HCPs] and Patient Information Leaflet for patients), prescription requirements, distribution channels and package size. Additional RMMs may be necessary to mitigate the risk of safety concerns when routine measures are not considered sufficient. These may comprise additional communication and education materials (EMs) for patients and HCPs such as brochures, leaflets, guides, checklists, patient cards and Dear HCP Communications, controlled distribution systems or pregnancy prevention programs.

Monitoring the effectiveness of additional RMMs is an explicit requirement of the Directive 2010/84/EU, which states that the marketing authorisation holder (MAH) shall ‘monitor the outcome of RMMs which are contained in the RMP or which are laid down as conditions of the marketing authorisation’. Assessment of the effectiveness of RMMs is an evolving area of regulatory sciences.

Studies conducted with the aim of measuring the effectiveness of the RMMs are per definition within the remit of Post‐Authorisation Safety Studies (PASS) per the PV legislation. Most of the studies assessing effectiveness of RMMs are surveys and drug utilisation studies (DUS). Surveys are mainly intended to assess the knowledge and self‐reported behaviour of the target audience against the RMM content. Most DUS use secondary data from electronic records or from medical chart reviews to estimate the adherence to prescribing behaviour and practices outlined in the RMM content.

The European Medicines Agency (EMA) has released a guideline for studies assessing the effectiveness of RMMs. This guideline emphasises that consideration should be given to the representativeness of the study sample with regards to the generalisation to the RMM target population. For that purpose, the recommendations highlight the need to carefully consider the source population, describe the sampling frame, provide details of sampling methods and document the proportion of non‐responders and their characteristics., ,

The EU electronic Register of Post‐Authorization Studies (EU PAS Register) available through the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) is the repository for PASS protocols and reports. It has been used to assess PASS characteristics and methodological aspects in the assessment of RMMs., , , However little research has investigated the sampling methodologies used by investigators in studies assessing the effectiveness of RMMs. Therefore, this review aims to describe the sampling methodology and supporting evidence reported by investigators to address representativeness in studies assessing the effectiveness of RMMs available in the EU PAS Register.

METHODS

The ENCePP EU PAS register was used as the source of the identified studies. Its entire content was downloaded into an excel file via a web scrapping programme using Python's Selenium library.

Studies finalised between 1 June 2017 and 31 May 2019 were selected by filtering ‘last update field’ between those dates and the field ‘status’ by ‘finalised’. Filtering criteria were applied to keep only records with both protocol and report documents available.

The EU PAS register field ‘Countries in which this study is being conducted’ was checked by one author (MFA) to include studies conducted in at least one EU country. The EU PAS register fields ‘Official title’, ‘Brief description of study’, ‘Primary Scope’ and ‘Main objective’ were screened for the presence of the following keywords: ‘Minimi’, ‘Survey’, ‘Utili’, ‘Behaviour’, ‘Knowledge’, ‘Material’, ‘Physician’, ‘SmPC’ and ‘effectiveness’.

Subsequently, two independent reviewers (M.F.A. and L.S.J.) performed a manual review to analyse the information available in the EU PAS register for each eligible record to retain only studies assessing the effectiveness of RMMs. Among these, studies using existing databases or registries to draw their sample were excluded, to retain only those studies which recruited their sample (HCPs/sites and/or patients) de novo. Discrepancies between the two reviewers were discussed and resolved. Studies that met the eligibility criteria were subsequently moved to the data abstraction step, which was performed by one author (T.P.) and reviewed by another (L.S.J.).

The latest version of the relevant protocols and reports available in the EU PAS Register were systematically reviewed to extract information on study characteristics (including type of RMMs, target population, design, countries included) and the sampling methodology used (including rationale and corresponding evidence for selection of country and target population, sampling frame [i.e. source], type of sampling [probabilistic/non‐probabilistic] and modes of contact of participants). The comparison of respondents to non‐respondents, the response rate and information on the generalisability of the results were also captured.

In cases where there was discrepant information between the protocol and study report, information from the report was retained. The systematisation of the information in different categories resulted from several iterative cycles of document review.

RESULTS

A flow diagram of the screened studies and the final number of studies included for review is presented in Figure 1. A total of 472 studies finalised between 1 June 2017 to 31 May 2019 were identified from the EU PAS Register. Approximately one‐third (31%, 146/472) had both the protocol and report available.

FIGURE 1

Preferred reporting items for systematic reviews and meta‐analyses (PRISMA) figure for study selection. aKeywords indicative of studies assessing effectiveness of RMMs: ‘Minimi’, ‘Survey’, ‘Utili’, ‘Behaviour’, ‘Knowledge’, ‘Material’, ‘Physician’, ‘SmPC’ and ‘effectiveness’ [Colour figure can be viewed at wileyonlinelibrary.com]

After applying the screening and eligibility criteria, a total of 17 studies assessing the effectiveness of RMMs with a full‐text protocol and report were analysed.

Study characteristics

Thirteen of the 17 reviewed studies were surveys, 3 were medical chart reviews and one combined both methods. Nine out of the 13 surveys targeted only HCPs,, , , , , , , , while the remaining 4 surveys targeted both HCPs and patients., , , The one study combining both survey and chart review methods targeted HCPs, sites, patients and caregivers. The three medical chart review studies targeted HCPs/sites who then recruited patients, , (Table 1). In total, 17 studies recruited HCPs/sites,, , , , , , , , , , , , , , , and 8 studies also recruited patients., , , , , , , Five studies assessed the effectiveness of routine RMMs,, , , , and 12 studies evaluated the effectiveness of additional RMMs., , , , , , , , , , ,

TABLE 1

Characteristics of the cross‐sectional and chart review studies

S. No.	Target population	Type of RMM assessed	Study type (Survey/Chart‐review)	Countries of participation	Methodology used to contact target sample	Final sample size (as mentioned in the report)	Response rate of HCPs/Sites (Participants/invited)	Response rate of patients (Participants/eligible)
113	HCP	Additional	Survey	FR, DE, IT, ES, UK	Initial invitation sent by email. Subsequent reminders sent by email and paper copy	HCPs: 131	2.40%	–
214	HCP	Additional	Survey	FR, DE, UK, BE, ES	By E‐mails and up to three telephone calls	HCPs: 1805	96.01%	–
315	HCP	Additional	Survey	UK, DE, FR, ES, IT, BE, AT, LU, NL, SE, FI, DK, NO, IE, BG, SK, EL, PL	By email	HCPs: 1028	7.42%
416	HCP	Additional	Survey	DE, ES, SE, AT, HU, RO, IT, UK	By emails and phone	HCPs: 800	10.32%
517	HCP	Routine	Survey	AT, BE, DK, FR, DE, NO, SE, NL, UK	By Email	HCPs: 323	2.39%	–
618	HCP	Routine	Survey	UK, ES, IT	By telephone, e‐mail, post, or fax	HCPs: 109	0.54%	–
722	HCP + Patient	Additional	Survey	UK, DE, FR, ES, IT	No details provided	HCPs: 428 Patients: 773	5.08%	88.44%
823	HCP + Patient	Additional	Survey	FR, DE, SE	Email or Postal mail	HCPs: 146 Patients: 7	2.38%	0.11%
919	HCP	Routine	Survey	DK, NL, ES, SE, UK	By email and phone	HCPs: 250	Not documented	–
1024	HCP + Patient	Additional	Survey	HCPs: FR, DE, IT, ES, UK Patients: DE, ES, UK	By email or postal mail	HCPs: 157 Patients: 10	9.86%	90.91%
1125	HCP + Patient	Additional	Surveya	UK, DK, DE, SK, NL, ES	Link to the online survey was distributed via professional networks and social networks	HCPs: 51 Patients: 1092	From 21% to 89% depending on the countryb	From 18% to 56% depending on the countryb
1220	HCP	Additional	Survey	AT, CZ, NL, ES, FR	By email and phone	HCPs: 759	6.84%	–
1321	HCP	Additional	Survey	BE, DE, IT, ES, SE, UK	By email and phone	HCPs: 38	2.84%	–
1427	HCP + Patients	Routine	Chart review	UK	No details provided	HCPs: 1196 Patients: 4625	–	95.44%
1528	HCP + Patient	Additional	Chart review	AT, CZ, FR, NL, ES	No details provided	HCPs: 314 Patients: 1513	11.9%	94.05%
1726	HCPsc + Sites + Patient and caregivers	Additional	Survey + Chart review	DE, EL, PT, UK	No details provided	Sites: 52 Patients: 659	–	98.07%

Abbreviations: AT, Austria; BE, Belgium; BG, Bulgaria; CH, Switzerland; CZ, Czech Republic; DE, Germany; DK, Denmark; eCRF, electronic case report form; EE, Estonia; EL, Greece; ES, Spain; FI, Finland; FR, France; HCP, Healthcare professional; HK, Hong Kong; HR, Croatia; HU, Hungary; IE, Ireland; IT, Italy; LT, Lithuania; LU, Luxembourg; LV, Latvia; NL, Netherlands; NO, Norway; PL, Poland; PT, Portugal; RMM, Risk minimisation measure; RO, Romania; SE, Sweden; SI, Slovenia; SK, Slovakia; UK, United Kingdom..

The project involved four workstreams: (1) literature review, (2) internet search to establish sources of information, (3) qualitative interviews, (4) online survey.

The response rate refers to the proportion of people that opened the online survey, in relation to those who opened it and completed it as the survey was advertised.

Number of participating HCPs not reported.

Country selection

The average number of countries included per study was 6 with a minimum of 1 and a maximum of 18. The countries most frequently included in the 17 studies were: Spain (13/17), UK (13/17), Germany (12/17), France (10/17), Italy (7/17) and the Netherlands (7/17).

Thirteen of the 17 studies documented a rationale for country choice. The predominant criteria for country selection were market uptake or drug sales (10/17),, , , , , , , , , followed by geographic or size diversity, or representativeness of the EU healthcare system (7/17)., , , , , , One study selected the participating countries based on a favourable regulatory framework for the conduct of PASS surveys. In another study, countries were selected to complement the countries already involved in a companion study on automated healthcare databases. Only one of the studies that provided a rationale for country selection, documented supportive evidence by presenting drug sales distribution over countries. Four studies did not provide any rationale for country selection., , ,

Sampling methodology for HCPs/sites

Seventeen studies targeted HCPs, , , , , , , , , , , , , , or sites., The rationale for the HCP/sites sampling strategy was based on the proportionate representation of medical specialties involved in patient treatment (14/17),, , , , , , , , , , , , , HCPs/sites' geographical distribution (3/17), , and type of setting (5/17)., , , , Three studies conducted a feasibility or pilot study to support their sampling stratification., , One of these studies also used the results of a syndicated prescription monitoring study to inform on parameters such as prescribing behaviour of several specialties involved in patient care, patient load, work setting, access to drugs. For the two studies that carried out a feasibility assessment at the protocol drafting stage, no detailed information or results of the feasibility were displayed in the protocol or report.,

The sampling frame/source from which the HCP/site sample was drawn consisted of an existing network of physicians (5/17),, , , , HCP databases (5/17),, , , , the list of physicians who were targeted to receive the EMs (4/17), , , or physician lists provided by the MAH without further details (4/17)., , , Two studies mentioned the compilation of multiple sources., One study provided a detailed sampling frame description and corresponding references (Table 2).

TABLE 2

Summary of data variables for all 17 studies

Variable	Summary Results
Country sampling rationale provided (N a = 17)	Geographic or country size diversity or healthcare system and its representativeness: n = 713, 16, 20, 22, 23, 24, 28 Possibility to conduct a patient survey in terms of regulatory access: n = 123 Prevalence of patients or eligible HCPs having experience with the drug contributing to sales of the drug (market uptake or market launch in a country): n = 1013, 14, 15, 17, 18, 19, 20, 21, 22, 23 Other: n = 128 Not specified: n = 425, 26, 27, 29
Evidence provided in the documents to support the country sampling rationale (N = 17)	Drug sales or number of patients: n = 113
Rationale to support the sampling strategy of HCPs/sites (N = 17)	Proportion of treating specialties: n = 1413, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 25, 26, 28 Type of setting (e.g., hospital [teaching or general], private office, size of centre or patient volume): n = 518, 22, 26, 27, 29 Geographical distribution per region: n = 322, 26, 27
Evidence supporting the sampling strategy of HCPs/sites (N = 17)	Feasibility assessment or pilot study: n = 317, 26, 27
Sampling frame of HCPs (N = 17)	Existing network of physicians: n = 517, 19, 25, 27, 28 List of physicians who were targeted to receive the EM provided by the MAH: n = 413, 15, 16, 21 National or commercial HCP databases: n = 514, 15, 17, 23, 24 List provided by MAH without further details: n = 413, 18, 24, 26 Compilation from multiple sources, such as literature, medical directory, peer referrals, hospital books, lists of potential investigators based on previous clinical trials, provided by MAH affiliates and/or CRO: n = 220, 22
Description of the sampling frame (N = 17)	Provided details on the sampling frame and supportive references: n = 117
Was exhaustivity of the sampling frame commented/described? (N = 17)	Yes, complete list: n = 116 Yes, not an exhaustive list: n = 220, 22
Sampling type used for HCPs/sites (N = 17)	Non‐probability sampling (convenience or voluntary)b: n = 513, 15, 25, 27, 28 Probabilistic stratified samplingc: n = 714, 17, 20, 21, 22, 23, 26 Probabilistic samplingd: n = 316, 22, 26 Probabilistic cluster samplinge: n = 129 All HCPs included (no selection): n = 218, 20 Not specified: n = 2 19, 24
Rationale to support the sampling strategy of patients (N = 8)	Inclusion of consecutive patients: n = 722, 23, 24, 25, 26, 27, 28 Random sampling when the number of patients is over the target: n = 122 Patient volume to assure sufficient representation of each practice type/capping of patient enrolment: n = 222, 26 When there are several indications for an hospital‐only drug identification of patients through the hospital pharmacy or through PDMS: n = 129
Sampling frame of Patients (N = 8)	Through selected sites: n = 322, 26, 29 Through HCPs: n = 423, 24, 27, 28 Through university, schools, youth centres: n = 125
Sampling type used for Patients (N = 8)	Non‐probability sampling (convenience sampling)b: n = 622, 23, 24, 26, 27, 28 Random sampling when patient number is over the target (in protocol)d: n = 122 All patients to be included: n = 129
Comparison of participants versus non‐participants amongst a sample of HCPs/sites (N = 17)	Characteristics of participating centres, actives or non‐actives to be compared: n = 126 Comparison of physicians (respondents and non‐respondents): n = 414, 22, 23, 28
Comparison of participants versus non‐participants amongst a sample of patients (N = 8)	Characteristics of participating patients, included and non‐included as well as motives for non‐inclusion to be compared (screening/patient/register log): n = 322, 26, 27
Generalisability (N = 17)	Any specific section in the report commenting on generalisability: Specific section ‘Generalizability’: n = 1313, 14, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27 In section ‘Limitations’: n = 315, 21, 28 None: n = 129 Among studies commenting on generalisability, those concluding that the study is: Generalisable: n = 813, 14, 17, 19, 23, 26, 27, 28 Partially generalisable: n = 216, 18 May not be generalisable: n = 615, 20, 21, 22, 24, 25

Note: These categories were ascertained solely based on the study results and conclusions in the study report.

Abbreviations: CRO, contract research organisation; EM, education materials; HCP, healthcare professional; MAH, marketing authorisation holder; PDMS, patient data management system.

N is the total number studies considered for each variable.

Sampling is not based on random selection of participants but on being conveniently accessible to the researcher.

The population is divided into a homogeneous subpopulation (strata) based on specific characteristics (e.g., speciality) and randomly selected from each strata.

Sampling is based on random selection (i.e., each subject has an equal probability of being selected).

Subgroups (i.e., clusters) of the population are used as the sampling unit (e.g., sites as clusters) and are selected randomly (i.e., each cluster has an equal probability of being selected).

Geographic or country size diversity or healthcare system and its representativeness: n = 7, , , , , ,

Possibility to conduct a patient survey in terms of regulatory access: n = 1

Prevalence of patients or eligible HCPs having experience with the drug contributing to sales of the drug (market uptake or market launch in a country): n = 10, , , , , , , , ,

Other: n = 1

Not specified: n = 4, , ,

Drug sales or number of patients: n = 1

Proportion of treating specialties: n = 14, , , , , , , , , , , , ,

Type of setting (e.g., hospital [teaching or general], private office, size of centre or patient volume): n = 5, , , ,

Geographical distribution per region: n = 3, ,

Feasibility assessment or pilot study: n = 3, ,

Existing network of physicians: n = 5, , , ,

List of physicians who were targeted to receive the EM provided by the MAH: n = 4, , ,

National or commercial HCP databases: n = 5, , , ,

List provided by MAH without further details: n = 4, , ,

Compilation from multiple sources, such as literature, medical directory, peer referrals, hospital books, lists of potential investigators based on previous clinical trials, provided by MAH affiliates and/or CRO: n = 2,

Provided details on the sampling frame and supportive references: n = 1

Yes, complete list: n = 1

Yes, not an exhaustive list: n = 2,

Non‐probability sampling (convenience or voluntary): n = 5, , , ,

Probabilistic stratified sampling: n = 7, , , , , ,

Probabilistic sampling: n = 3, ,

Probabilistic cluster sampling: n = 1

All HCPs included (no selection): n = 2,

Not specified: n = 2 ,

Inclusion of consecutive patients: n = 7, , , , , ,

Random sampling when the number of patients is over the target: n = 1

Patient volume to assure sufficient representation of each practice type/capping of patient enrolment: n = 2,

When there are several indications for an hospital‐only drug identification of patients through the hospital pharmacy or through PDMS: n = 1

Through selected sites: n = 3, ,

Through HCPs: n = 4, , ,

Through university, schools, youth centres: n = 1

Non‐probability sampling (convenience sampling): n = 6, , , , ,

Random sampling when patient number is over the target (in protocol): n = 1

All patients to be included: n = 1

Characteristics of participating centres, actives or non‐actives to be compared: n = 1

Comparison of physicians (respondents and non‐respondents): n = 4, , ,

Characteristics of participating patients, included and non‐included as well as motives for non‐inclusion to be compared (screening/patient/register log): n = 3, ,

Specific section ‘Generalizability’: n = 13, , , , , , , , , , , ,

In section ‘Limitations’: n = 3, ,

None: n = 1

Generalisable: n = 8, , , , , , ,

Partially generalisable: n = 2,

May not be generalisable: n = 6, , , , ,

Only three studies commented on the exhaustivity of the sampling frame. One reported that the ‘complete list’ of physicians targeted to receive the EMs was used for enrolling HCPs, while the other two studies commented that there was ‘no exhaustive list’ of all drug prescribers which could have been used to draw a random sample from, (Table 2). Regarding the type of sampling used for drawing the sample from the sampling frame, majority of the studies (9/17) used probabilistic sampling (i.e., involving random selection), , , , , , , , whereas five studies used non‐probabilistic sampling, , , , (Table 2).

The majority of the studies (11/17) contacted HCPs/sites for participation through emails., , , , , , , , , , Among these, six also reported contacting HCPs/sites by phone, , , , , and four by postal mail., , , The remaining five studies did not provide information on the methodology used for approaching HCPs/sites. , , , ,

Sampling methodology for patients

Eight studies targeted patients. The sampling frame for patients included the selected sites' patient pool (3/8),, , the recruited HCPs' patient pool (4/8), , , or university/schools/youth centres in a study assessing the use of an oral contraceptive (1/8). One study involving a drug available in hospital only mentioned the identification of all patients through a hospital centralised structure. Regarding the type of sampling, six studies applied non‐probabilistic sampling by recruiting patients from the selected HCPs/sites., , , , , Another study mentioned in its protocol, that probabilistic sampling would be applied if the number of patients was over the target. However, no evidence of random sampling was later described in the report.

The majority of the studies recruited patients by consecutive enrolment (7/8)., , , , , , Two studies capped the number of patients enrolled per practice size to promote sufficient representation of each practice (2/8).,

Generalisability section of study reports

Participation rates were reported and calculated as a proportion of invited HCPS/sites and proportion of eligible patients (Table 1) and was generally ≤10% for HCPs and ≥60% for patients, respectively.

Of the 17 studies targeting HCPs/sites, 5 studies had taken steps to address the representativeness of HCPs/sites a posteriori by including a comparison of respondents to non‐respondents on available criteria., , , , These results were subsequently tabulated in the reports of four of the studies, , , (Table 2).

Of the eight studies targeting patients, three studies compared the characteristics of participating and non‐participating patients which they captured with the use of a patient/screening log., ,

Among the 17 studies, 16 studies commented on the generalisability of the results, among which 8 studies considered their results to be generalisable, , , , , , , and 2 studies considered the results partially generalisable in one setting (private and office‐based practices) or in one country.,

DISCUSSION

This review provides an overview of the sampling methodologies used by studies assessing the effectiveness of RMMs conducted and finalised between June 2017 and May 2019, with study documents uploaded in the EU PAS Register. Most studies (76%) collected data directly through a survey, the remaining studies collected data through patient charts or mixed methodologies.

A solid sampling plan is important in observational studies given the heterogeneity of the target population in the real‐world context. In particular, when assessing the success of implementing additional RMMs, it is very important to consider the healthcare context, as this may impact their distribution, acceptance, understanding and adoption. Therefore, targeting a representative sample of subjects for which the RMMs are designed is critical in this context.

The ENCePP checklist requires the description of the sampling methodology in the protocol. Following our review, we considered the details available in the study documents insufficient to fully follow the sampling methodology.

The documents revealed that there is awareness that the selection of a representative sample is important in studies assessing the effectiveness of RMMs, as a rationale for a sampling strategy was often provided in the protocol. These rationales generally specified that the distribution of HCP specialties/sites would be representative of prescribing practices. However, additional details on how the sampling strategy would be operationalised were largely missing (e.g., supportive data from literature, internal or vendor data sources, feasibility results). Considerations on the exhaustivity of the data source were only available in a few of the investigated documents. ENCePP Guide on Methodological Standards specifies that ‘if the objective of the survey is to evaluate whether the RMMs are distributed among the right target population, the lists which are used for the distribution of the RMM materials cannot be used as the source population for sampling’. We found that at least a quarter of the HCPs surveys reported using these lists to recruit their sample, which is slightly lower than reported in a previous study covering a sample of HCPs surveys in the EU PAS register up to 2016. However, our results may underestimate the above given the lack of details reported in the study documents.

In general, the sampling method was more extensively described for HCPs than for patient enrolment (e.g., whether probabilistic sampling, or a systematic criterion such as consecutive enrolment will be used). Almost half of the studies considered their results generalisable, but presented limited information to substantiate this, which impaired the ability to assess the representativeness of the final results. A minority of the studies provided information on how non‐respondents differed from respondents.

Our findings are similar to those of another review which looked into the reporting of key quality criteria in survey research. In this review the authors found that the population of interest from which the sample was drawn was often unclear, with only one third of the studies describing the survey population, and only half of the studies describing the sampling frame. The review further reported that there was no systematic presentation of the sampling methodologies.

Information on sampling methods was found scattered throughout the protocols, mainly in the ‘setting’ and ‘data sources’ sections, with an inconsistent level of detail and standardisation. This poses a challenge for the reader aiming to check the completeness of the different key elements of a protocol. The latest version of the ENCePP Checklist includes an item related to the description of the source population and another item to define how the study population will be sampled from the source population; however the checklist remains high‐level and lacks tailoring to the specific nature of studies assessing the effectiveness of RMMs. The EMA Guidance for the format and content of the final study report of non‐interventional post‐authorisation safety studies, includes a ‘generalisability’ section. We found that three‐quarters of the studies used this section to comment on the generalisability of the study results while one‐fifth used the ‘limitations’ section for this purpose.

The cited ENCePP checklist and EMA PASS templates, are structured to be applicable to all PASS. Given the variety of objectives and designs that these studies may have, some requirements and considerations might be applicable only to some. Thus, more targeted guidance on the design of studies assessing the effectiveness of RMMs is important, as has already been underlined by pharmacoepidemiologic societies.

Our recommendation would be to generate a protocol specific template for this type of studies, that would include a dedicated section for sampling methodology; covering all the steps from country selection to participant sampling. Further this template should provide advice on what evidence needs to be provided.

RMM effectiveness studies encounter many operational challenges. Ethical approval for non‐interventional observational studies varies hugely across Europe., This may lead to delays in study execution or to the exclusion of countries, therefore affecting general validity of the results. Furthermore, due to the voluntary nature of recruitment, the lack of interest from the target populations to participate in these studies is generally reported as a major source of selection bias. Acknowledging that the representativeness of study results may not be achievable due to regulatory and operational constrains makes it essential to understand the generalisability of results using the study documents.

The strengths of this review include the use of the EU PAS Register as a source to retrieve protocols and reports of studies assessing the effectiveness of RMMs. Therefore, this review provides insights into the sampling strategies from documents that have been reviewed and endorsed by regulators.

Although the possibility to identify such studies in the EU PAS Register is the major strength of the review, the amount of studies with missing study documents also constitutes its main limitation and has been previously acknowledged. Additionally, we only had access to study documents that were published on the website. However, additional supportive information may have been presented and discussed between document owners and regulators during review process without being published. The limited number of studies in our sample may also hinder the generalisability of our findings to all studies assessing the effectiveness of RMMs. We highlight the complexity of conducting this review. First studies assessing the effectiveness of RMMs were not flagged as such in the EU PAS register. Further, there was also a lack of harmonisation in the use of categories to characterise studies in the register (e.g., what registrants considered an existing data source differed significantly). These challenges make it difficult to conduct reviews using a large amount of records from EU PAS register, especially given the need to manually validate each record. Therefore, in future revisions of the EU PAS register we would recommend to (a) consider providing instructions on how to interpret each field in the record (field dictionary) and (b) an additional flag to categorise studies as ‘assessing the effectiveness of RMM’, given this is one of the reasons why regulatory PASS are conducted.

CONCLUSIONS

This review provides an overview of the sampling methodologies used by studies assessing the effectiveness of RMMs that were finalised between June 2017 and May 2019 and had both a protocol and report uploaded in the EU PAS Register.

Most studies were surveys targeting HCPs and/or patients. All studies provided a rationale to justify country selection and sampling strategies of HCPs/sites as a theoretical framework for their sampling methodology. However, only a minority of the studies provided evidence to substantiate this rationale. Few studies provided information on how non‐respondents differed from respondents. Overall, the study documents offered insufficient details to understand the rationale behind the sampling decisions. Subsequently it was often unclear if the respective sampling methodologies could generate representative samples. More standardisation and guidance in reporting the sampling frame, strategy and operational considerations applicable to these types of studies would support transparency and facilitate the evaluation of the representativeness of the results.

CONFLICT OF INTEREST

All authors are employees of IQVIA, a CRO that designs, conducts, analyses and reports risk minimisation studies for pharmaceutical companies and health authorities. The authors have no other relevant affiliations or financial involvement with any organisation or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

ETHICS STATEMENT

The authors state that no ethical approval was needed.