Regulatory scientific advice in drug development: does company size make a difference?

PURPOSE: To assess whether the content of Scientific Advice (SA) questions addressed to a national drug regulatory agency is associated with company size. This may help to increase understanding about the knowledge, strategic, and regulatory gaps companies face during drug development.
METHODOLOGY: A cross-sectional analysis was performed of SA provided by the Dutch Medicines Evaluation Board (MEB) in 2006-2008. Definition of company size was based on ranking by total revenues (Scrip's Pharmaceutical Company League Tables 2008). The content of each SA question was scored according to predefined domains (quality, nonclinical, clinical, regulatory, and product information), their subdomains (e.g., efficacy), and a selection of additional content variables (e.g., endpoints, choice of active comparator).
RESULTS: In total, 201 SA documents including 1,087 questions could be identified. Small, medium-sized, and large companies asked for SA 110 (54.7%), 40 (19.9%), and 51 (25.4%) times, respectively. Clinical questions were asked most often (65.9%), mainly including efficacy (33.2%) and safety questions (24.0%). The most frequent topics were overall efficacy and safety strategy. Small companies asked quality and nonclinical questions more often (P < 0.001) and clinical questions less frequently than large companies (P = 0.004). Small companies asked significantly more clinical questions about pharmacokinetics, including bioequivalence, than medium-sized and large companies (P < 0.001).
CONCLUSION: The array of topics addressed in SA provides an interesting outlook on what industry considers to be still unresolved in drug development and worthwhile to discuss with regulators. Company size is associated with the content of SA questions. MEB advice accommodates both innovative and noninnovative drug development.

Introduction

Marketing authorization of a new medicinal product is a critical step in giving the public access to innovative therapies that are needed to fill current pharmaceutical gaps and unmet medical needs. Despite the increasing number of applications for marketing authorization in Europe, the proportion of applications with a negative decision remains relatively high, around 25–30%, and was even 40% for new active substances with a resolved outcome in 2009 [1] [2]. There is increasing concern about the obvious gap between the output of drug development and registration strategies applied by companies, and EU regulatory expectations [3, 4]. Industry response to this development indicates that improved communication with regulatory authorities during drug development is needed [5]. Additional regulatory requirements in recent years have complicated the authorization procedure and have made innovative drug development more costly. Furthermore, with complex biologicals, advanced therapies, and personalized medicines becoming more important, the need for more specific guidance in drug development has increased [6, 7].

Before and during the marketing authorization procedure for a medicinal product, pharmaceutical companies have various opportunities to discuss critical issues in the drug development process with regulators. A continuous and ongoing regulatory dialogue between pharmaceutical industry and regulatory authorities has often been recommended as a strategy to support innovative drug development in an efficient and tailored way [3, 5, 8–10]. A relevant part of scientific regulatory dialogue is so-called Scientific Advice (SA), the opportunity for (early) communication between a company and a regulatory authority on quality, nonclinical, and various clinical aspects (e.g., study design, choice of endpoint, indication) of drug development. In Europe an increasing proportion of market application authorizations are preceded by SA; 47% of all applications in 2007 received SA and in 2008 this percentage was 56% [1].

An applicant for SA can be a pharmaceutical company or scientists developing a product. Applicants are encouraged to seek regulatory SA as many times as necessary, but industry and authorities are not obliged to adhere to the advice received or committed to accept any result of a SA procedure [11]. In Europe, SA can either be sought from the European Medicines Agency (EMA) or from one or more of the national regulatory agencies. National regulatory agencies provide SA either as a response to national SA requests or as an answer to European SA requests, outsourced by the EMA Scientific Advice Working Party (SAWP) to one or two of its member countries according to expertise. A recent study looking at SA provided by EMA demonstrated that the level of industry adherence to SA and company size were both predictors of a positive outcome in a marketing authorization procedure. The study also showed that among companies submitting a marketing application to EMA, large companies requested SA most frequently and were more adherent to the advice than medium-sized and small companies [9].

Considering that adherence to SA is associated with a positive outcome in a marketing authorization procedure and that variability in adherence to SA exists among companies, the question arises whether company size matters when looking at the type of SA that pharmaceutical companies are seeking. Answering this question may help to learn more about the knowledge, strategic and regulatory gaps companies face during drug development and how these differ among the various types of enterprises.

Methods

Study design and scientific advice characteristics

A cross-sectional analysis was performed of national SA provided by the Dutch Medicines Evaluation Board (MEB) in the years 2006–2008. SA documents were retrieved from the MEB SA Database. Requests for SA that were rejected by the MEB for reasons of lack of expertise or previously received EMA advice were excluded. In this study, individual requests for SA were considered, so follow-up SAs for a similar medicinal product were included.

Products for which SA was given in the study period were categorized according to anatomical main group of the ATC classification [12]. In case an ATC classification was missing, the anatomical main group was assessed based on the intended indication of the product. Products were also categorized as new chemical substance (NCS; chemical substance not previously approved), generic (a product with identical qualitative and quantitative composition and similar pharmaceutical form as original product), biologicals (defined as vaccines, blood and blood components, allergenics, somatic cells, gene therapy, tissues, and recombinant proteins), and new application of existing drugs (previously approved chemical substances for which a new indication, dosage form, or other variation was being developed, in such a way that there was a need for additional efficacy and safety studies).

Company size was defined as small, medium-sized, and large, based on ranking by total revenue as reported in Scrip’s Pharmaceutical Company League Tables 2008 [13]. Companies were defined as large if ranked 1–20, medium-sized if ranked 21–150, and small if the company was not on the ranking list. This definition was in line with a previous study on SA [9]. For each SA, we evaluated whether previous advice for the same product had been requested at the MEB, whether parallel advice had been sought at another national regulatory agency, or both.

Data collection: characteristics of questions

Each SA submission consisted of a variable number of questions asked by companies. All questions in 2006, 2007, and 2008 were collected and analyzed in a standardized fashion. Each question, being the unit of analysis, was scored separately according to variables at three different levels: domains, subdomains, and content variables (Fig. 1). At the first level, the question content was analyzed according to the following domains: quality, nonclinical, clinical, regulatory, and product information. Scoring more than one domain was allowed, for example, when a clinical issue and a product information issue were discussed in the same question.

Fig. 1

Scoring method for SA questions

Secondly, subdomains were formulated and scored for questions in the nonclinical and clinical domains. The subdomains of the nonclinical domain were pharmacodynamics, pharmacokinetics, and toxicology. For the clinical domain, subdivisions were pharmacodynamics (including dose finding studies), pharmacokinetics (including bioequivalence studies), efficacy, or safety. Again, scoring more than one subdomain was allowed.

Additionally, at the third and most detailed level, each question was scored by a selection of content variables, e.g., primary endpoint, choice of active comparator, trial duration, and overall efficacy program. The content variables were selected based on general regulatory requirements of the drug development process and existing EMA regulatory guidelines. In this third step, a distinction was made between specific and strategic questions. Strategic questions were defined as questions in which general feedback was asked about, e.g., the overall quality program or the clinical efficacy program. An example of a strategic question was: “Does the MEB agree that the results of the clinical efficacy program will be sufficient to achieve market approval of the product for the specific indication?”. Specific questions were defined as being related to specific topics of the development plan for a particular study. An example of a specific question was: “Does the MEB agree with the chosen primary endpoints for this indication?”.

Data analysis

Associations between the type of SA questions and company size were assessed by Pearson’s chi-squared analysis. P-values were calculated for each variable. Differences in average number of questions were assessed by a one way ANOVA test.

Results

During the study period, the MEB provided SA 214 times. Thirteen advice documents were missing (four and nine documents in 2006 and 2007, respectively). In total, 201 SA documents, including 1,087 questions, could be identified. SA was provided for 187 products, with 117 companies (80 small, 21 medium-sized, and 16 large companies) receiving SA in the study period. The general SA characteristics are given in Table 1.

Table 1

Baseline characteristics of SA

Variables	Advice (n = 201) (% of total no. of advice)
Year
2006	77 (38.3%)
2007	47 (23.4%)
2008	77 (38.3%)
ATC code
A: Alimentary tract and metabolism	12 (6.0%)
B: Blood and blood-forming organs	16 (8.0%)
C: Cardiovascular system	34 (16.9%)
G: Genito-urinary system and sex hormones	15 (7.5%)
J: Anti-infectives	13 (6.5%)
L: Antineoplastic and immunomodulating products	29 (14.4%)
N: Nervous system	50 (24.9%)
Other	32 (15.9%)
Product type (EMEA)
NCE	43 (21.4%)
Generic	59 (29.4%)
New application of existing drug	64 (31.8%)
Biological (including biosimilars)	34 (16.9%)
Other (General advice)	1 (0.5%)
Orphan drugs
Orphan drug	3 (1.5%)
Nonorphan drug	198 (98.5%)
Company size
Small	110 (54.7%)
Medium-sized	40 (19.9%)
Large	51 (25.4%)
Type of registration procedure
Central	48 (23.9%)
Decentral	42 (20.9%)
Mutual recognition procedure	19 (9.5%)
National	11 (5.5%)
To be decided	75 (37.3%)
Missing	6 (3.0%)
Previous/parallel advice
No previous	102 (50.7%)
At MEB	17 (8.5%)
At other agencies	62 (30.8%)
At MEB and other agencies	9 (4.5%)
Missing	11 (5.5%)

SA was most frequently given for nervous system drugs (24.9%), but for a variety of other therapeutic areas, SA was provided as well. More than 60% of SA was given for generics and new applications of existing drugs. Small, medium-sized, and large companies requested SA 110 (54.7%), 40 (19.9%), and 51 (25.4%) times respectively. More than 40% of the companies seeking SA had previously received advice for the same drug at the MEB or another national agency.

On average five questions per SA submission were asked (Table 2). Clinical questions were asked most frequently [716 times (65.9%)]. Within the clinical subdomain, efficacy and safety questions were most frequently asked [361 (33.2%) and 261 (24.0%) times, respectively].

Table 2

Company size in relation to characteristics and content of SA questions

Questions (n = 1,087)	Small Pharma	Medium Pharma	Large Pharma	Total	P-values
Total no. of questions (%)	431 (39.7%)	310 (28.5%)	346 (31.8%)	1,087 (100%)	<0.001
Average ± SD no. of questions (range)	3.9 ± 3.2 (1–18)	7.8 ±  5.7 (1–27)	6.8 ±  3.9 (1–17)	5.4 ±  4.3 (1–27)	<0.001
ATC code
A: Alimentary tract and metabolism	29 (6.7%)	0 (0.0%)	41 (11.8%)	70 (6.4%)	<0.001
B: Blood and blood-forming organs	26 (6.0%)	66 (21.3%)	7 (2.0%)	99 (9.1%)	<0.001
C: Cardiovascular system	39 (9.0%)	63 (20.3%)	69 (19.9%)	171 (15.7%)	<0.001
G: Genito-urinary system and sex hormones	6 (1.4%)	48 (15.5%)	60 (17.3%)	114 (10.5%)	<0.001
J: Anti-infectives	40 (9.3%)	15 (4.8%)	8 (2.3%)	63 (5.8%)	<0.001
L: Antineoplastic and immunomodulating products	58 (13.5%)	30 (9.7%)	75 (21.7%)	163 (15.0%)	<0.001
N: Nervous system	114 (26.5%)	80 (25.9%)	58 (16.8%)	252 (23.2%)	0.003
Other	119 (27.6%)	8 (2.6%)	28 (8.1%)	155 (14.2%)	<0.001
Product type (EMEA)
NCE	47 (10.9%)	89 (28.7%)	173 (50.0%)	309 (28.4%)	<0.001
Generic	125 (29.0%)	17 (5.5%)	31 (9.0%)	173 (15.9%)	<0.001
New application of existing drug	178 (41.2%)	106 (34.2%)	101 (29.2%)	385 (35.4%)	0.002
Biological (incl. biosimilars)	80 (18.6%)	98 (31.6%)	41 (11.8%)	219 (20.1%)	<0.001
Other (general advice)	1 (0.2%)	0	0	1 (0.1%)	n.a.
Previous/parallel advice
No previous	233 (54.1%)	90 (29.0%)	98 (28.3%)	421(38.7%)	<0.001
At MEB	9 (2.1%)	38 (12.3%)	64 (18.5%)	111 (10.2%)	<0.001
At other agencies	161 (37.4%)	154 (49.7%)	134 (38.7%)	449 (41.3%)	0.002
At MEB and other agencies	6 (1.4%)	11 (3.5%)	32 (9.2%)	49 (4.5%)	<0.001
Missing	22 (5.1%)	17 (5.5%)	18 (5.2%)	57 (5.2%)	0.973
Domain
Quality	43 (10.0%)	26 (8.4%)	5 (1.4%)	74 (6.8%)	<0.001
Nonclinical	69 (16.0%)	22 (7.1%)	19 (5.5%)	110 (10.1%)	<0.001
Clinical	272 (63.1%)	192 (61.9%)	252 (72.8%)	716 (65.9%)	0.004
Regulatory	86 (20.0%)	59 (19.0%)	60 (17.3%)	205 (18.9%)	0.649
Product information	11(2.6%)	22 (7.1%)	22 (6.4%)	55 (5.1%)	0.009
Clinical subdomain
Pharmacodynamics (incl. dose finding)	26 (6.0%)	13 (4.2%)	27 (7.8%)	66 (6.1%)	0.154
Pharmacokinetics (incl. bioequivalence)	123 (28.5%)	30 (9.7%)	41 (11.9%)	194 (17.8%)	<0.001
Efficacy	105 (24.4%)	101 (32.6%)	155 (44.8%)	361 (33.2%)	<0.001
Safety	97 (22.5%)	78 (25.2%)	86 (24.9%)	261 (24.0%)	0.639
Type of questions
Strategic questions	93 (21.6%)	64 (20.6%)	65 (18.8%)	222 (20.4%)	0.627

Small companies asked significantly fewer questions per SA compared to medium-sized and large companies (P < 0.001). Large and medium-sized companies asked significantly more SA questions about new chemical entities than small companies did (P < 0.001). Small companies asked 70% of SA questions about drug development of generics and new applications of existing drugs. These small companies were a diverse representation of companies, including generic companies (20%), innovative pharmaceutical or biotech companies (40%), and other companies mainly consisting of medical technology companies, those working on new applications of drugs, and consultants. Medium-sized companies, about 85% of which were innovative pharmaceutical or biotechnology companies, most frequently asked SA questions related to the development of biologicals (P < 0.001).

With regard to domain, the majority of questions asked by companies were about clinical development issues, while for small companies quality and nonclinical questions were more common. One out of five SA questions was on regulatory issues, with no difference among types of companies. Within the clinical domain, small companies asked significantly more often about pharmacokinetics, including bioequivalence, than medium-sized and large companies. These companies posed efficacy questions less often than large companies (24.4 vs. 44.8%, P < 0.001). The proportion of safety and strategic questions was not associated with company size.

In Fig. 2, the overall top ten of most frequently addressed topics on the most detailed third level of variables is given, showing a strong preference for clinical topics. Overall, the most frequently asked questions were about overall efficacy strategy (9.6%) and safety strategy (9.1%). In addition, strategy questions about the clinical pharmacokinetic program were in the top ten. Indication, primary endpoints, dosing, and study population were examples of popular specific topics. More details of the ten most frequently asked topics are given in Table 3.

Fig. 2

Top ten most frequently addressed topics according to size of firm

Table 3

The top ten most frequently addressed topics and example questions

Topic	Definition	Question example
Overall efficacy strategy	Questions related to the overall clinical study program in order to prove efficacy of a drug	Does the MEB think the proposed efficacy program is appropriate for a marketing authorization?
Overall safety strategy	Questions related to the overall clinical study program in order to prove safety of a drug	Does the MEB think the proposed safety program is appropriate for a marketing authorization?
Indication	Questions related to the definition/wording of indication and the appropriateness of the suggested indication	Does the Agency agree that “Treatment of symptoms associated with interstitial cystitis/painful bladder syndrome including bladder pain, urinary urgency, and frequency” is a registrable indication?
Primary efficacy endpoints	Questions related to the appropriateness of the primary endpoint selected to prove efficacy of a drug	Does MEB agree that the primary endpoint of overall survival supported by the secondary endpoints of PFS, tumour response rate, and duration of response is appropriate to support registration of drug X as a first line in advanced non-small cell lung cancer?
Study design	Questions related to the multiple methodological issues of a specific randomized clinical study	The recently initiated Phase II-III clinical trial has the following characteristics: [...] Is this trial design acceptable for definitive confirmation of the clinical benefit and of an acceptable safety profile of drug X?
Dosing	Questions related to the appropriateness of the doses chosen for a clinical study	The scheme for the individual dosing is a 10 mg/kg loading dose followed by a 5 mg/kg maintenance dose. The company is considering increasing the maintenance dose if no adverse effects are seen. Does the MEB agree to the proposed dosing regimen?
Study population	Questions related to the appropriateness of the inclusion and exclusion criteria used for patient selection in a study	Does the Agency concur with the definition of the patient population to be studied in the Phase 3 randomized trial to support regular approval of their respective proposed indications?
Pharmacokinetics strategy	Questions related to the appropriateness of the complete clinical pharmacokinetics study program	Does the MEB agree with the proposed clinical pharmacokinetic program?
Validity of measurement method	Questions related to the application of specific measurement methods (e.g., symptom scores) to assess clinical endpoints	The MD Anderson Symptom Inventory (MDASI) will be used in the randomized Phase 3 study to measure the patient-reported outcomes of symptom severity and interference (SSI). Does the Agency concur with the use of the MDASI instrument?
Special safety issues	Questions related to the investigation of specific safety issues at the organ-system level	Are there any specific aspects of safety you would like us to pay special attention to? Does the agency concur with the company’s proposal to perform only ECGs in the proposed pivotal studies, given the absence of a QTc prolongation effect in a thorough QT study?

Discussion

One of the main findings in this study was that SA provided by a national authority is indeed different, both quantitatively and in terms of kind of questions, when looking at company size. Our content analysis of SA demonstrates that the majority of questions raised by companies, particularly the large ones, were about clinical drug development. Small companies gave more attention to quality and nonclinical issues than large companies.

The array of topics addressed in SA provides an interesting outlook, given all the limitations caused by strategic behavior of companies and selective acceptance of SA by regulators, on what industry considers to be still unresolved in drug development and worthwhile to discuss with regulators.

Regnstrom et al. emphasized the importance of adherence to SA for a successful marketing approval [9]. The question arises whether our findings of companies’ priorities in drug development are in line with the most often occurring major objections or factors for approval failure. A 2002 study with EMA data found that major objections raised by regulators in the marketing authorization procedure were lack of adequate randomized controlled trials to prove clinical efficacy and the occurrence of unresolved safety issues [3]. The EMA reported in 2008 that critical issues related to study design (39%), patient population (35%), endpoint (35%), and the magnitude of an effect (48%) were important drivers of a negative application [1]. In a 2010 study with FDA data on orphan drugs, Heemstra et al. found that failing to achieve primary endpoints and failing to describe the target population were related to nonapproval [10]. Our study also showed that topics such as study design, endpoints, study population, and special safety issues were all among the top ten most frequently addressed issues in SA.

Quality documentation is a particular bottleneck for small and medium-sized enterprises (SMEs). The EMA SME office reported that quality documentation caused 41% of the major objections in application procedures of SMEs in 2008 [14]. Our results demonstrated that SMEs asked significantly more often about quality issues than large companies did, with the latter hardly discussing any quality issues. This implies that SMEs lack knowledge regarding quality documentation or lack capacity to comply with the requirements.

Some limitations of this study need to be discussed. Firstly, the scoring method of the SA, although highly standardized, may be susceptible to some subjectivity. In order to minimize this, we scored the questions according to strict definitions of content variables. These variables were derived from scientific regulatory documents and guidelines. Secondly, our definition of company size differs from the official EU definition of SMEs. According to the official SME definition, only 15 of 201 SA requests would have been classified as an SME request. This would create a group of “large companies” that was too heterogeneous to draw any conclusions about. Therefore, we based the SME definition on ranking by total revenue as reported in Scrip’s Pharmaceutical Company League Tables 2008, which was in line with a previous study on SA with EMA data [9]. Another limitation of our study is that we did not investigate the companies’ reasons for seeking SA. These reasons may range from a real interest in the answers to expected positive effects on the regulatory process (and outcome) by the applicants owing to dialogue and alignment with regulators in general.

It should be noted that company budgets may drive the decision to ask SA. However, during a significant part of the research period, the MEB provided SA free of charge. In addition, the costs that were introduced later were very limited, ranging from 3500 to 8000 euro per SA. Therefore we believe it is unlikely that the costs of SA have influenced our results, and we do not consider this a limitation of our study.

The fact that some SA requests at the MEB were rejected may raise the question whether results of this study are representative for national SA provided by other regulatory agencies. Advice requests were rejected when advice had been obtained from EMA, in cases where the complexity was expected to be better dealt with at the EMA level, or when the product indication was outside the scope of the expertise of the MEB. Therefore the array of clinical areas represented in this study is also a reflection of national MEB expertise. Despite national expertise in, for example, central nervous system and cardiovascular products, the MEB gave SA about drug development in a broad range of therapeutic areas (Table 1). Similar broad ranges are expected for SA at other national regulatory agencies in the EU as well. In addition, when comparing the Dutch national SA with other national SA, the top ten most frequently addressed topics will probably not be influenced by differences in expertise because the topics are related to drug development in general. Therefore, we think our results give a well-balanced overview of issues in drug development.

Regulatory dialogue about challenging issues at the critical edge of drug development is seen as a key success factor for bringing new medicinal products with a positive benefit-risk to the patient. An EMA brainstorm session held with regulators and pharmaceutical industry representatives in 2007 made clear that a special need exists for dialogue about new high-risk advanced therapies and technologies and for new scientific approaches in targeted drug development, such as validation of biomarkers, choice of study endpoints, or better methods to identify treatment responders. In addition, the use of more flexible and adaptive study designs was raised as a key issue to be discussed in a dialogue with regulators [5]. According to the EMA, many SMEs in particular are active in the development of the highly innovative advanced therapy medicinal products (ATMPs) [14]. SA related to such high-risk advanced therapies and technologies are channeled to the EMA. In contrast, other small companies asked MEB advice most frequently about generic applications, bioequivalence, and new application of existing drugs. One may argue that answers to these SA questions could also be found in regulatory guidelines. The need for such advice may be partly attributable to lack of experience in drug development or lack of clarity in existing guidelines.

The role of scientific advice also has bearing on the way companies formulate their questions. For all types of companies about 20% of all questions asked were “strategic.” Further research should assess whether companies benefit more from asking specific or strategic questions. Also, in further research national SA could be compared to European SA to assess whether strategic questions are asked on both levels and to evaluate commonalities and differences in the roles of European and national SA. Moreover, a better understanding of the level of complexity of SA questions would give deeper insight into the issues addressed. This would also enable further research on how complexity drives market authorization holders’ behavior when it comes to SA.

In conclusion, SA as provided by a regulatory authority provides a detailed outlook of unresolved issues in drug development. This picture is a function of industry presence in a certain country, of the expertise at the national regulatory authority, but also of critical issues at the edge of regulatory decision making. Indeed, there is variability in how different companies deal with this. The results of this study show that company size is associated with the content of SA questions and that national SA accommodates both innovative and noninnovative drug development. Clinical pharmacology topics are at the top of issues discussed in SA, a finding that asks for more analysis on how industry, regulatory, and academic clinical pharmacologists can fruitfully interact and align in order to stimulate drug innovation.