White paper: landscape on technical and conceptual requirements and competence framework in drug/disease modeling and simulation.

Pharmaceutical sciences experts and regulators acknowledge that pharmaceutical development as well as drug usage requires more than scientific advancements to cope with current attrition rates/therapeutic failures. Drug disease modeling and simulation (DDM&S) creates a paradigm to enable an integrated and higher-level understanding of drugs, (diseased)systems, and their interactions (systems pharmacology) through mathematical/statistical models (pharmacometrics)(1)-hence facilitating decision making during drug development and therapeutic usage of medicines. To identify gaps and challenges in DDM&S, an inventory of skills and competencies currently available in academia, industry, and clinical practice was obtained through survey. The survey outcomes revealed benefits, weaknesses, and hurdles for the implementation of DDM&S. In addition, the survey indicated that no consensus exists about the knowledge, skills, and attributes required to perform DDM&S activities effectively. Hence, a landscape of technical and conceptual requirements for DDM&S was identified and serves as a basis for developing a framework of competencies to guide future education and training in DDM&S.CPT: Pharmacometrics & Systems Pharmacology (2013) 2, e40; doi:10.1038/psp.2013.16; advance online publication 1 May 2013.

The high attrition rate of new drug entities combined with escalated expenditures led to decreased efficiency of pharmaceutical research and development (R&D). The number of new drugs approved per billion US dollars spent on R&D has halved approximately every 9 years since 1950.[2] This can in part be explained by the increasing complexity of issues that need to be addressed during the drug development process and by the criteria for drug candidate development that is required for progression of that candidate from bench to patients.[3]

Pharmaceutical sciences experts and regulators acknowledge that pharmaceutical R&D and therapeutic usage not only require scientific advancements but also better tools for knowledge integration management for improved, informed decision making.[4,5] Drug disease modeling and simulation (DDM&S) creates a paradigm for enabling an integrated and higher level of understanding of drugs, (diseased)systems characteristics, and their interactions (systems pharmacology) through mathematical/statistical models (pharmacometrics) throughout the entire R&D process and therapeutic usage.[1] DDM&S provides increased quantitative understanding of dynamic complexity in time, space, and population diversity.[6] By advancing the quality and quantity of knowledge integration, DDM&S has already demonstrated an added value on, e.g., predictive (translational) pharmacokinetic/pharmacodynamics models.[7] Specifically, DDM&S proved instrumental in decision making in the pharmaceutical industry[8] on new drug approval and labeling decisions,[9,10,11] and DDM&S is starting to bridge the gap toward improving patient care.[12]

The growth and successes of DDM&S substantially increased the demand for scientists educated and trained to understand the science and underlying framework as well as to develop modeling and simulation concepts for drugs and diseases (pharmacometricians). In addition, to fully exploit the DDM&S paradigm, further competencies are needed. However, the resources for well-trained pharmacometricians are lagging behind. Therefore, if we wish to further improve drug development and regulatory and therapeutic decisions by DDM&S, we need to address the very rapidly growing demand for education and training of pharmacometricians.[13] Barret et al. and Holford and Karlsson already sketched an outline for a pharmacometrics curriculum for undergraduate, graduate, and postgraduate levels, with an emphasis on the following key points: (i) to be able to describe the time course of drug response in individuals, (ii) to identify individual factors predicting differences in response, and (iii) to design clinical trials to elucidate drug properties.[14,15]

Recently, the European Union set up one of the largest ever public–private partnership programs, infusing 2 billion Euro into multidisciplinary, precompetitive research projects, known as Innovative Medicines Initiative.[16] Within this context, in contrast to previous initiatives, this is the first time that knowledge management and its integration and utilization are recognized as a critical component in drug development. Specifically, in 2011, the Drug Disease Model Resources (DDMoRe) consortium was approved as one of the first projects under the knowledge management pillar of Innovative Medicines Initiative with the ultimate objective of developing a drug disease model repository and an open-source interoperability framework. The consortium immediately recognized that an extensive education and training program is a critical factor for delivering innovation, subsequent dissemination, and management of knowledge as implemented in the context of DDM&S. Besides the need for scientific training of pharmacometricians, there is also a need for a thorough understanding on how different professionals and stakeholders currently operate and what their competencies are/should be.[17]

To obtain an accurate and comprehensive inventory of the competencies and skills currently available, the education and training working group of DDMoRe developed an extensive survey with different stakeholders being directly or indirectly involved in DDM&S in academia, pharmaceutical R&D in industry, or therapeutic usage of medicines. The survey should provide insights into the current impact, benefits, weaknesses, and potential hurdles as perceived by the different stakeholders and identify both technical and conceptual requirements for effective decision making and knowledge integration management. The insights gained from these results and landscape derived should ultimately be used as the basis for the development of a framework of competencies and should inform future educational and training efforts in pharmacometrics and systems pharmacology and DDM&S.

Results

Database

In total, 152 responses were collected from 151 responders (1 responded twice (Modeler, Reviewer) in separate responses). From all responders, additional information on organization (Academia, pharmaceutical industries (“PharmInd”), and small–medium enterprises (“SMEs”)), academic level (Lecturer, MD, MSc, PhD, PhD student, Postdoctorial, and Professor), and modeling and simulation level of expertise (junior and senior) was sought. After basic database cleaning of empty questionnaires (n = 14) and double responses (n = 1), the database was locked.

The number of complete responses retained in the database was 137 (n = 74 Academia (55%); n = 59 PharmInd (42%); and n = 4 SMEs (3%)). From those, almost 2 of 3 identified themselves as Modelers (n = 84, 63%), 1 of 4 as Multi (n = 35, 25%), and within the remaining, 10 as Reviewers (7%) and 6 as Appliers (4%). In terms of the modeling and simulation level of expertise, ~2 of 3 were seniors (64%) vs. 36% juniors (i.e., postdoctorials and PhD students). The distribution of the four different populations split for Academia, PharmInd, and SME is represented in Supplementary Figure S1 online.

Here, the results of the main analysis of the five allocated domains of the survey are reported for the two populations: Modelers and Multi. Additional data for the two other populations is discussed in the Supplementary Data online.

Domain I: impact and benefit of DDM&S within organizations

To understand the current impact of DDM&S, the responders were asked to evaluate whether DDM&S influences (i) understanding of drug, system, and disease characteristics; (ii) translational and (iii) line extension modeling; (iv) decision making; (v) new trial design and (vi) approval of new drugs; dose selection; new treatment adjustments and individualized therapy; and (vii) hypotheses generation and methodological aspects. Within both the organizations, Academia (n = 73) and Pharma (PharmInd and SMEs, n = 48), DDM&S is perceived by the responders to have an impact on all levels/fields of the drug development process and on therapeutic usage. For different fields, however, the perceived dominating impact varies from “frequently” to “occasionally” to “never” (): In Academia, >50% perceive a frequent impact (dark blue circles) of DDM&S in the fields of understanding drug, system, and disease characteristics as well as methodological aspects that lead to mechanistic knowledge and methodological progress. By contrast, for Pharma, DDM&S highly impacts direct fields of compound development to foster regulatory submissions: for 65–75%, DDM&S has a frequent impact on the understanding of drug characteristics, rational dose selection, and decision making. For the results of subgroup analysis, see Supplementary Data.

A free-text question on the benefit of modeling & simulation activities in the responders' organization/projects/collaborations reveals that overall DDM&S is recognized by both Academia and Pharma as being of high benefit and a valuable tool for (i) understanding drug development, (ii) using clinical drug data and models, and (iii) supporting drug development and patients' drug dosing (Supplementary Figure S3 online).

Domain II: areas of involvement or engagement of people dealing with DDM&S

To understand which areas concretely use and drive the impact of DDM&S, the responders were asked to specify their actual involvement/engagement in the drug development/usage process and in which therapeutic areas (multiple answers possible). In Academia, besides academic research, ~30% deal with optimization of therapeutic use, and <30% are involved in clinical development. The three top involvement areas in Pharma include clinical development (80%), preclinical development (>45%), and discovery (<30%) ().

The use of DDM&S in different therapeutic areas contained the following categories: diabetes, oncology, Alzheimer's disease, infectious diseases, safety, and others. For Academia (n = 20–30) and Pharma (n = 16–28), there is a strong focus on chronic diseases, i.e. oncology, diabetes, and central nervous system diseases, and on drug safety (). For central nervous system diseases, “Alzheimer's disease” and the responses explicitly dealing with central nervous system diseases specified under “others”, e.g., Parkinson's disease, were regrouped. In contrast to chronic diseases, for which there was no difference between the two groups, for infectious diseases, there is a strong dominance by Academia. This discrepancy could be explained either by a limited number of responders working on infectious diseases or by this field not receiving priority in drug development, for often being nonchronic, low-prevalence diseases with short-term therapeutic intervention including the potential of developing resistance to medicines. However, currently large efforts are undertaken in viral infections such as HCV.[18]

Domain III: concepts, methodologies, and tools utilized for DDM&S

Within the last years, new concepts, scientific methodologies, and tools were invented for DDM&S purposes. An important goal of the survey was to present an inventory by investigating which concepts, methods, and application are used within the community. To achieve this, responders specified (i) DDM&S activities, (ii) software and applications used, (iii) methods and algorithms utilized, and (iv) (Supplementary Data) nature of models developed and of data for DDM&S activities.

Currently, the dominant methodological concept in DDM&S is population analysis (~75% within Academia and within Pharma; ). Moreover, in both, DDM&S is utilized more often in drug development than to build disease models, the latter being more often performed by seniors than juniors (Supplementary Figure S4 online). Clinical trial simulation is more often utilized in Pharma than Academia; conversely, methodology development and evaluation is more often undertaken in Academia. These results are in line with the fields of impact (see Domain I). Of note, systems biology is currently the least often implemented concept with <25% using it occasionally and <10% frequently.

For data-driven modeling with parameter estimation, “nonlinear regression” and the so-called “first-order conditional estimation method” are the most prominent methods used (Supplementary Figure S5 online). A slight tendency toward the exploitation of newer methodologies in Academia relative to Pharma is observable. One reason might be the close collaborations between Academia and Pharma that might imply a faster implementation. However, the dissemination of new methodologies, such as stochastic or Bayesian, seems to be limited despite their (recent) implementation in software applications (see below) but could also be associated with accessibility, e.g., licensing and validation. Another issue is the perception that stochastic approaches might have fewer acceptances by regulatory authorities, despite interest from them.[19]

To accommodate this diversity in data, models, and methodologies, a multitude of applications and tools are used (Supplementary Figure S7 online). By far, the most frequently used software programs are NONMEM and its companion tool PsN, followed by Berkeley Madonna, WinNonlin, Monolix, and general statistical software applications. In particular, the open-source software R is very widely used, as opposed to the commercial software SPlus and Matlab.

Overall, the recognition and impact of concepts, methodology, and software applications of systems biology within the DDM&S community currently seem low.

Domain IV: gaps for and personal challenges with DDM&S

To provide the basis for assessing weaknesses, challenges, and opportunities, the responders were asked to reveal environmental gaps and personal challenges and ways how they are overcome. Significant (major and minor) gaps for DDM&S are identified for all defined categories by Academia and Pharma (Supplementary Figure S8 online). Both indicate as major gaps in their environment the lack of high-performance computing capacities and the lack of time and resources for further education and training, yet the results are more pronounced in Academia compared with Pharma. For the subgroup analysis results, see Supplementary Data. Of note, 70 and 80% of Seniors claim gaps (major and minor) in education and training, respectively ().

When asked for personal challenges with DDM&S (), the majority of both Academia and Pharma responders identified technical aspects (i.e., ODE solvers) and the estimation methods as major challenges. The complexity of the mathematical functions and the assessment/handling of model assumptions were also very often cited. For the subgroup analysis results, see Supplementary Data.

As means to meet personal challenges, self-education such as reading relevant publications, user manuals, postings in user forum, and consulting specialists or colleagues are currently most often stated. Most of Juniors (93%) and the majority (69%) of Seniors ask for support and advice (Supplementary Figure S10 online). Therefore, for communicating results and exchanging knowledge, within the community, personal discussions play a major role. Whereas in Academia, results are more often communicated through scientific publications, in Pharma, technical reports as part of the regulatory submissions and development process are more important.

Overall, a key message is that 2 of 3 (66%) of the Modelers, regardless of their expertise, intend to participate in training. For Seniors, we can speculate that the motivation/need to keep up-to-date with new methodologies/technologies is an incentive; it would be interesting to explore these expectations and possible differences according to expertise in follow-up studies.

Discussion

Most stakeholders, specifically pharmaceutical science experts, regulatory agencies, investors, decision makers in drug development and therapeutic use, in the pharmaceutical arena admit that better decision making is essential for reducing attrition rates in drug development and optimizing the therapeutic usage of medicines.[20] It is anticipated that model-based approaches to address scientific, clinical, and regulatory issues will be of benefit. Underscoring this are a number of examples that have already shown a major impact.[7,8,9,12] Yet, few stakeholders acknowledge that the key advances necessary to achieve the impact of DDM&S relate to personal skills and competencies and not only improvement in technologies and processes. Hence, we have to identify not only the bottlenecks but also the gaps in skills and competencies to be addressed in further and more advanced applications of model-based approaches to improve the drug development process and therapeutic usage.

The DDMoRe consortium[21] believes that apart from a high degree of technical sophistication in DDM&S, it ultimately comes down to skills and competencies in understanding, improvement, and integration of the DDM&S tools and of the DDM&S paradigm for hypothesis generation, optimization, and risk management in guiding decision making for optimizing drug therapy and drug development in both Academia and Pharma. For education and training, the gaps and challenges therein need to be identified. The survey outcomes indicate that no consensus exists about the knowledge, skills, and attributes required for individuals to carry out different types and levels of integrative DDM&S activities effectively.

Survey

A goal of the survey was to include responses not solely from expert individuals in the field but to include regulatory members and “appliers” who are involved less directly in DDM&S activities. Therefore, our survey was conceived and implemented to (i) establish technical and conceptual requirements (ii) to assess their (perceived) performance, and (iii) to define the baseline conditions for implementation of a competence framework. Our choice to stratify the questionnaire into five different domains arose from the need to discriminate the various factors underlying performance in R&D and patient care. In brief, it should be clear (i) in which areas and to which extent quantitative methods are already being successfully used, (ii) which gaps and challenges exist, and (iii) how benefits are perceived across the participating organizations.

In particular, our survey provides extensive quantitative and qualitative results of the inventory of various stakeholders for the first time. It reveals that despite the acknowledged frequent and high impact in several areas, there is (substantial) potential to increase the impact and benefit of DDM&S throughout the entire drug development and therapeutic decision chain. These findings corroborate data reported by Barrett et al.,[15] who suggested a close integration of pharmacometricians along the drug development process, and the conclusions of a survey on the impact of modeling and simulation reviews on new drug approval and labeling decisions between 2000 and 2008 performed by the US Food and Drug Administration demonstrating a substantial increase in pharmacometric analyses with impact.[11]

Potential limitations

Given the scope of the DDMoRe consortium, we have decided to use a targeted approach to reach the stakeholders who participated in this survey. The selection of names was based on existing collaborations or interactions with members of DDMoRe, rather than through the formal identification of a panel of experts, and was mostly limited to Europeans. Although this sampling method may clearly lead to potential bias in the results, we envisaged that “insiders' insight” was critical to evaluate perceived performance and assess current practices and behaviors. In particular, most of the responders were involved in DDM&S activities, spanning across pharmaceutical companies, academic institutions, software development companies, and consultancies, which means their assessment of impact and benefits may be overestimated; the study does not allow comparative performance with other technologies or approaches. On the other hand, it should be noted that the lack of common standards and practices for skills and competencies renders the evaluation of performance rather difficult.

We also recognize that there was a skewed representation from small businesses, clinical organizations (e.g., hospitals and clinical research organizations), and regulatory agencies, in which DDM&S activities are limited to a small number of staff members. Likewise, the low number of systems biologists, statisticians, and other clinical scientists or professionals contributing to decision making may have influenced the assessment of gaps and challenges. To assess how these results evolve, follow-up surveys in a broader audience, including individuals in e.g. decision making and regulatory or “customer” positions, at later times should be performed. The rate of response could also be improved by including a specific section with less technical items.

Landscape

The multifaceted survey and its responses in the five domains resulted in the landscape of conceptual and technical requirements for effective decision making and knowledge integration management utilizing DDM&S (Supplementary Figure S11 online). In addition, these data were deemed critical for an appropriate strengths, weaknesses, opportunities, threats analysis and subsequent planning strategy for a competence framework.

Despite the evolving role of DDM&S in R&D activities and in patient care, our results show that currently a code of best practices is lacking. Such a code is essential to ensure optimal individual performance in DDM&S. A considerable number of responders, even those in senior roles, have indicated the interest in or need for (additional) sophisticated technical and conceptual education and training in various aspects of DDM&S, ranging from face-to-face courses to self-education. It is evident from a large proportion of the participants that needs are not only limited to technical skills but also involve scientific principles, management, and communication skills. Hence, we suggest to structure requirements in the form of a framework of competencies.

Defining a “framework of competencies”

Calls for competence-based approaches to preparing professionals go back 60 years or more.[22,23] A competence framework defines, e.g., skills and attributes needed for people within an organization or community to perform effectively.[24] Some important principles govern the development and use of competence frameworks, which are aimed at the translation of knowledge, skills, attitudes, and their synthesis into performance.[25] Even though the development of a framework of competencies can take considerable effort and has to be implemented with care, it has led to important results in adjacent areas such as medicine.[26] In fact, the availability of a competence framework enabled recognition by a wider community of the roles and contribution of those professionals.

Hence, the results of our survey and the created landscape served as basis to develop a “framework of competencies” to guide future strategic and implementation efforts in education and training in DDM&S (). On the basis of our findings, we propose three main areas for DDM&S activities in R&D and therapeutic usage (outer shell in ): (i) drug development and clinical trials, (ii) regulatory affairs and safety of medicines, and (iii) healthcare and therapeutics. For these three areas, nine competence fields (CF) of DDM&S activities were defined (see ellipses in ). Within each CF, three hierarchical cognitive complexity levels (CCL) can be identified based on Bloom's taxonomy[27] (see inner three layers in ): (i) knowledge, (ii) skills, and (iii) attributes (i.e., the latter being characterized by making critical judgments based on a sound knowledge base). summarizes the characteristics of the three main and six subcategories of CCLs.

Both dimensions—CF as first dimension and CCL as second dimension of the matrix—form the basis for the third dimension, the performance level in DDM&S activities, i.e., “basic”, “competent”, “expert.”. Hence, using the nine CF with the three CCL of this reference matrix, it is possible to allocate a performance level to each of the components of an education and training curriculum. As an example, with respect to the CF “comparative effectiveness of treatments” and the CCL “skills,” the allocated performance level of an education and training component may be “competent.” Furthermore, it can be envisaged that this “framework of competencies” can provide what is desirable for effective performance in each CF and CCL as well as provide a guidance to professionals in any point of their career and ultimately lead to a code of best practices for three main areas for DDM&S activities in R&D and therapeutic usage.

In an era of resource scarcity, when the need for generalized cost containment in R&D expenditure and in healthcare is intensifying, effective professional performance becomes critical. The DDMoRe consortium acknowledges that an extensive education and training program is indispensable for delivering innovation and subsequent dissemination and management of knowledge as implemented in the context of DDM&S. Our survey has shown that despite the increasing contribution of DDM&S activities to drug development and therapeutic usage of medicines, clarity is lacking about its role and which skills are required in this field.

Beyond and above the opportunity for an updated landscape regarding the application, utility, and constraints of DDM&S in drug development and therapeutic usage, the current survey results provided the basis for the recommendation of a “framework of competencies” for people involved in DDM&S activities, which can help stakeholders to identify opportunities, define expectations, and standards for effective performance of professionals in this field. Obviously, such a framework represents a paradigm shift and it will take time for its acceptance, adoption, and refinement. However, shifts occur more quickly when linked to a strong public need or concern (e.g., patient safety) or to regulatory pressure. We anticipate that concerted efforts from industry, academia, and regulators to realize the proposed framework of competencies will have the potential to transform how we prepare the professionals of the next decade.

Methods

Domains or areas of attention. To gather data on perceptions and expert opinion from a variety of perspectives, the survey was divided into five domains or areas of attention: (i) impact and benefit; (ii) area of involvement or engagement; (iii) concepts, methodologies, and tools; (iv) gaps and challenges; and (v) sharing of data/models. The first domain relates to strategic choices in traditional project management settings in which decisions are made based on expected outcome. The second domain is aimed at facilitating the understanding of what processes and functional areas could achieve or may have achieved lasting uptake within the participating organizations. The intention of the third domain was to generate an inventory of concepts, methodologies, and tools currently used and, thus, to present the technical and conceptual framework for achieving impact. The fourth domain provides the basis for assessing weaknesses, challenges, and opportunities for DDM&S. The remaining domain is motivated to identify conditions development of collaborative efforts with regard to the sharing of data and models.

Survey. The questionnaire for the survey was created by a team of experts (the authors of this manuscript, being all involved in DDM&S) using a Delphi-like method for consensus building.[28] The items for the survey were initially discussed in group meetings, leading to a first version of the questionnaire tested by ~20 members of the DDMoRe consortium. The questionnaire was then finalized by taking into account the comments in a consensus meeting. The final version of the questionnaire (Supplementary Data) was then put online on SurveyMonkey.

The questionnaire contained dichotomous, single/multiple choice, scaling questions, and questions with a free-text response. Scaling questions to measure the frequency or magnitude of significance had three prespecified levels (never, occasionally, and frequently or none, minor, and major, respectively). The first five questions were all compulsory. Question 5 was assigned into a logical node to direct the responder into the appropriate subsurvey environment according to which of the four following populations s/he had identified her/himself.

Modelers: responders who develop models/perform DDM&S activities (n = 19 questions)

Appliers: responders who apply/interpret results from DDM&S activities or generate data (n = 17 questions)

Reviewers: responders who review DDM&S results (n = 20 questions)

Multi: responders who are involved with all the above and answered the questionnaire according to their predominant responsibilities (n = 20 questions)

Survey performance. The URL of the questionnaire (SurveyMonkey system) was released between November 2011 and January 2012 by e-mailing the link to one representative of an organization who then was responsible for its further internal circulation. Therefore, the exact number of the recipients and return rate could not be tracked. To ensure a wide participation and a minimal bias, the link was sent to the modeling and simulation community in various universities (Academia), pharmaceutical R&D industries (PharmInd), and SMEs, including all partners of the DDMoRe consortium.

Data analysis and representation. Before data analysis, all answers (except free text) were numerically transformed, and the data set was blinded. Owing to the small number of responses, SMEs were regrouped with PharmInd in the analysis, being named Pharma. The main analysis focused on Modelers and Multi because of the small number of responders from the Appliers and Reviewers populations. Hence, a sub-data set was generated consisting of 121 (88% of total) responses from Modelers and Multi with a 60:40% ratio between Academia and Pharma. Subanalyses were performed by stratifying this data set according to type of organization (i.e., Academia/Pharma) and modeling and simulation level of expertise (i.e., Junior/Senior). In scaling questions, all missing responses were added to the levels “never” or “none” for frequency and magnitude responses, respectively.

Data were tabulated and graphically analyzed using R version 2.15.1[29] and Microsoft Office Excel (version 2002 SP3, 2002; Microsoft, office.microsoft.com). Only descriptive statistics are presented owing to the small number of responses. Questions eliciting only free-text responses were graphically illustrated using word cloud graphs using a text-mining function (“tm” in R) to scale and color the words according to the frequency they appeared.

Author Contributions

C.K., A.K., P.M., E.C., E.C.L., G.V., I.G., and O.D.P. wrote the manuscript; C.K., P.M., E.C., E.C.L., G.V., N.T., and O.D.P. designed the research; C.K., A.K., E.C., E.C.L., G.V., I.G., N.T., and P.M. performed the research; and C.K., A.K., P.M., E.C., E.C.L., G.V., I.G., N.T., and O.D.P. analyzed the data.

Conflict of Interest

GV, OdP are employed by GSK and IG is employed by Lilly, any views or opinions presented in this manuscript represent solely their own ones and do not necessarily represent those of their companies. All authors have declared no conflicts of interest.

Study Highlights

Table 1

Cognitive complexity levels (CCL)