Key Learning Outcomes for Clinical Pharmacology and Therapeutics Education in Europe: A Modified Delphi Study.

Harmonizing clinical pharmacology and therapeutics (CPT) education in Europe is necessary to ensure that the prescribing competency of future doctors is of a uniform high standard. As there are currently no uniform requirements, our aim was to achieve consensus on key learning outcomes for undergraduate CPT education in Europe. We used a modified Delphi method consisting of three questionnaire rounds and a panel meeting. A total of 129 experts from 27 European countries were asked to rate 307 learning outcomes. In all, 92 experts (71%) completed all three questionnaire rounds, and 33 experts (26%) attended the meeting. 232 learning outcomes from the original list, 15 newly suggested and 5 rephrased outcomes were included. These 252 learning outcomes should be included in undergraduate CPT curricula to ensure that European graduates are able to prescribe safely and effectively. We provide a blueprint of a European core curriculum describing when and how the learning outcomes might be acquired.

WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

☑ A recent study showed that the prescribing competencies of final‐year medical students in Europe were poor, resulting in many potentially harmful prescribing errors. Another recent study showed marked variation in the quantity and quality of CPT education within and between European countries. Harmonizing CPT education at a European level might improve the prescribing competencies of future doctors, but to date there is no consensus on the required learning outcomes for European graduates in order to prescribe safely and effectively.

WHAT QUESTION DID THIS STUDY ADDRESS?

☑ In this modified Delphi study, key learning outcomes were identified for teaching and assessing CPT in undergraduate medical curricula in Europe.

WHAT THIS STUDY ADDS TO OUR KNOWLEDGE

☑ A large European expert panel reached consensus on 252 key learning outcomes that should be included in undergraduate CPT curricula to ensure that European graduates are able to prescribe safely and effectively. Additionally, we provide a blueprint of a European core curriculum in CPT describing when and how the learning outcomes might be taught and assessed during the undergraduate medical curriculum.

HOW THIS MIGHT CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE

☑ A European core curriculum in CPT might help to harmonize CPT education and improve the prescribing competencies of future European doctors and thereby patient safety.

Prescribing is a complex and challenging task for every medical graduate. Poor prescribing may result in medication errors and adverse drug reactions, with potential consequences for patient safety and healthcare costs.1, 2 In many European countries, recently graduated doctors write a large proportion of prescriptions in hospitals, often with minimal supervision from senior clinicians. In order to prescribe safely and effectively at the start of their professional careers, medical graduates should have acquired appropriate prescribing competencies (i.e., knowledge, skills, and attitudes). Unfortunately, studies throughout Europe suggest that medical students have not acquired these necessary competencies by the time they graduate. In the UK and the Netherlands, recent graduates were found to be responsible for a large number of prescribing errors and reported having little confidence in their prescribing skills.3, 4, 5, 6 In other European countries, there are similar concerns about a lack of prescribing competencies among medical graduates.7 A recent multicenter study showed a general lack of essential prescribing competencies among 895 final‐year medical students from 17 European medical schools.8 This is even more worrying, since the demands on new prescribers are increasing progressively because of several trends, including a growing number of medicines, increasingly vulnerable patients, and more complicated regimens due to polypharmacy and multimorbidity.9

Poor education in clinical pharmacology and therapeutics (CPT) during the undergraduate medical curriculum may underlie the lack of prescribing competencies among recent graduates. Indeed, most final‐year students of European medical schools felt that their medical curriculum had not adequately prepared them for safe prescribing and thought that too little time had been devoted to CPT.8 Moreover, a recent study showed there was a marked variation in the quantity and quality of CPT education in European medical schools.10 In most schools, both teaching and assessment are mainly based on traditional learning methods (e.g., lectures and written examinations), which seem to be associated with a lower level of prescribing knowledge and skills among final‐year students compared with problem‐based learning methods (e.g., patient simulation and real‐life prescribing).8 Modernizing and harmonizing CPT education at a European level might prevent and reduce prescribing errors in the future, thereby improving patient safety. Furthermore, harmonization is becoming more important, given the increased mobility of medical students and junior doctors across Europe.11

A first step toward harmonization is to establish which learning outcomes should be common to all European undergraduate CPT curricula, as suggested by the British Pharmacological Society (BPS) and the European Association for Clinical Pharmacology and Therapeutics (EACPT) in 2007.7 Defining clear learning outcomes might also help to shape and strengthen the CPT specialty in Europe, since it has lost ground in recent decades and has become increasingly invisible in medical curricula.12 Previous studies of CPT learning outcomes have mostly come from the UK,9, 13, 14, 15 the Netherlands,16 and Sweden.17 To date, there has been no clear and robust definition of what European medical graduates should know about CPT in order to prescribe safely and effectively. Therefore, on behalf of the Education Working Group of the EACPT, we conducted this modified Delphi study to reach consensus on key learning outcomes for teaching and assessing CPT during the undergraduate medical training in Europe.

RESULTS

In total, 129 (129/177; 73%) experts from 27 European countries accepted the invitation and formed the consensus panel. Their characteristics are presented in Table S1. The response rate was 85% (109/129), 74% (95/129), 81% (104/129) for Rounds 1, 2, and 3, respectively. Ninety‐two experts (71%) from 25 European countries completed all three rounds (79 doctors, nine pharmacists, and four scientists). The average educational experience of all panelists was 17.5 years (range 3–38); 78% of the medical doctors were clinical pharmacologists. Thirty‐three panelists (26%) from 17 European countries attended the face‐to‐face meeting (Table S1).

Delphi process

The flow chart of the systematic literature search is shown in Figure 1. The articles identified in the literature search are presented in Table S2. During Round 1, 226 of the 307 outcomes were included; 81 outcomes for which there was no consensus about their relevance, 24 new outcomes, and two adapted outcomes were resubmitted in Round 2. During Round 2, 18 of the 107 resubmitted outcomes were included, 73 were excluded, and 16 were selected for discussion during the panel meeting (75–80% agreement). During the panel meeting, 5 of the 16 outcomes were included, six were excluded, and five were adapted and resubmitted in Round 3. In this round, three of the five adapted outcomes were included, giving a total of 252 outcomes (34 subcategories; 192 knowledge, 47 skills, and 13 attitudes) considered important for CPT education in Europe. The subcategories are shown in Table 1 and the detailed learning outcomes are listed in Tables S3–5. Most panelists indicated that 201 of the 252 included outcomes (80%) should be acquired during the clinical years of the undergraduate medical curriculum, 33 (13%) outcomes should be acquired during the preclinical years, and 18 (7%) during both phases (Table S3–5). Although most of the included outcomes focused on knowledge (76%), outcomes regarding prescribing skills and attitudes were generally considered more important (Figure 2). Sixty‐three outcomes did not meet the 80% agreement cutoff and were excluded (47 knowledge, 14 skills, and 2 attitudes).

Figure 1

Flowchart of the systematic literature search.

Table 1

Subcategories of learning outcomes that were included (n =  34)

Competence	Subcategory (n =  34)	Number of learning outcomes (n =  252)
Knowledge	1. Introduction to clinical pharmacology and therapeutics  1.1 Basic principles  1.2 Drugs in healthcare and society	5
	2. Pharmacodynamics  2.1 Mechanism of action  2.2 Dose‐response relationships	12
	3. Pharmacokinetics  3.1 Drug absorption, distribution, metabolism and excretion  3.2 Concentration–time relationships  3.3 Repeated drug dosing	21
	4. Individual variability in the response to drugs  4.1 Basic principles  4.2 Pharmacokinetic variability  4.3 Pharmacogenetic variability	11
	5. Adherence, compliance, and concordance  5.1 Adherence and compliance  5.2 Concordance	8
	6. Therapeutic drug monitoring  6.1 Basic principles  6.2 Using drug effect and concentration	10
	7. Adverse drug reactions  7.1 Basic principles  7.2 Drug allergy  7.3 Diagnosis, management and prevention  7.4 Pharmacovigilance	17
	8. Drug interactions and contraindications  8.1 Interactions  8.2 Contraindications	11
	9. Medication errors	4
	10. Drug discovery, development and regulation    10.1 Drug discovery and development    10.2 Drug regulation	7
	11. Medicines management    11.1 National and local processes    11.2 Formularies and guidelines	7
	12. Evidence‐based prescribing    12.1 Basic principles    12.2 Critical appraisal of clinical studies    12.3 Find reliable information about drugs	12
	13. Legal and ethical aspects of prescribing    13.1 Legal aspects    13.2 Ethical aspects	10
	14. Prescribing for patient with special requirements    14.1 Elderly patients    14.2 Impaired liver function    14.3 Impaired renal function    14.4 Pregnant women and women of childbearing potential    14.5 Lactation    14.6 Children	29
	15. Rational prescribing    15.1 Rational approach to prescribing    15.2 Dose selection	6
	16. Clinical toxicology	6
	17. Misuse of drugs	2
	18. Complementary and alternative medicine	2
	19. Use of antibiotics and antibiotic resistance	4
	20. Commonly used drugs and high risk medicines	8
Skills	21. Medication history taking	7
	22. Rational prescribing	11
	23. Drug dose calculation	3
	24. Prescription writing	4
	25. Nondrug therapy	1
	26. Communication	5
	27. Reviewing prescriptions	3
	28. Adverse drug reactions	4
	29. Clinical toxicology	2
	30. Obtaining information from guidelines and protocols to support prescribing	3
	31. Monitoring medication	4
Attitudes	32. Risk–benefit analysis	7
	33. Recognizing personal limitations in knowledge	1
	34. Recognition of a balanced approach to the introduction of new drugs	5

Figure 2

Total number of learning outcomes per subcategory (n = 34) for knowledge (blue; n = 192), skills (yellow; n = 47), and attitudes (green; n = 13). The red line demonstrates the mean % “(very) important” per subcategory (calculated by dividing the total % “(very) important” per subcategory by the number of outcomes within that subcategory). TDM, Therapeutic Drug Monitoring.

DISCUSSION

In this study, an international panel of experts identified 252 key learning outcomes for CPT education in European medical schools. These learning outcomes provide a detailed description of competencies that European medical graduates should have acquired in order to prescribe safely and effectively. To our knowledge, this is the first Delphi study to establish CPT learning outcomes at a European level. Previous studies of CPT learning outcomes that used the Delphi technique were mainly focused on the local setting in a specific country.14, 15, 17 Our Delphi study provides additional information that builds on earlier CPT curricula,9, 13, 15, 18 and offers a detailed framework for teaching and assessing CPT in European medical curricula, which could also be useful elsewhere (e.g., USA). Implementing the identified learning outcomes in already overcrowded medical curricula will be challenging but essential in order to improve the prescribing competencies of future European doctors and thereby patient safety.

Perhaps not surprisingly, most of the included learning outcomes (76%) focused on knowledge of basic and clinical pharmacology. This is probably because the initial list of outcomes consisted primarily of knowledge outcomes (76%) and because CPT education in Europe is still mainly focused on teaching theoretical knowledge (e.g., lectures, written exams) instead of skills (e.g., completing drug prescriptions) and attitudes (e.g., communication with patients).10 However, this does not necessarily mean that knowledge of basic and clinical pharmacology is more important than skills and attitudes related to pharmacotherapy. On the contrary, prescribing skills and attitudes were generally considered more important (Figure 2), and these should be taught throughout the medical curriculum because they are highly complex cognitive processes that require repeated training.19 Interestingly, most panelists indicated that skills and attitudes should be acquired during the clinical years (years 4–6), and not the preclinical years of medical training, probably because this reflects the situation in their own curriculum. Since the training of prescribing skills during the early years of medical education has been shown to improve students' ability to prescribe rationally during later years,20, 21 prescribing should be taught as early as possible in the curriculum, preferably using simulated and clinical environments with real responsibility for patient care.13, 22

In contrast to previous studies,9, 13, 15, 18 in this study the panelists considered knowledge about drug marketing, development of drug formularies and guidelines, and complementary and alternative medicines less important, on the basis that these topics are not directly relevant for prescribing in daily clinical practice or are usually covered by other courses in the curriculum. Similarly, they considered preparing and administering drugs (e.g., intravenous, intramuscular) less important because this role has been taken over by nurses and pharmaceutical assistants in many European countries.

The CPT learning outcomes should be considered in association with a relevant list of core drugs (“student formulary”) and their associated therapeutic problems. These lists may help to prioritize learning and avoid overburdening medical students with facts.9 However, only a small proportion of European medical schools (4%) use these lists in their CPT programs.10 This might be because it takes time and effort to develop a valid list, and this might be a problem for medical schools with a limited number of CPT teachers. A potential solution is to develop a European list of core drugs and diseases that European medical students should know about by the time they graduate, as previously suggested by the EACPT.7 In 2002, Orme et al. published a European list of core drugs and diseases, but this probably requires updating.23

There are two prerequisites for the successful implementation of learning outcomes. First, since CPT is often integrated in different courses throughout the medical curriculum, it is important that outcomes are compatible with the learning environment and assessment activities (constructive alignment).24 Coherence between assessment, teaching strategies, and intended learning outcomes is important for successful introduction of outcomes and stimulates students to achieve high grades.25 Currently, CPT learning outcomes in most European medical schools are not adequately aligned with the curriculum content,10 making this requirement even more important. Second, the purpose of introducing learning outcomes is solely to direct teaching and students' learning, and this should be clearly communicated to medical students and teachers. Using outcomes for policy and management activities may weaken their function, reducing their potential value.26

Strengths and limitations

The consensus panel in our study consisted of a large number of experts (n = 129) from various professional backgrounds and with different levels of educational experience, and represented most European countries. Moreover, the overall response rate was good in comparison with that of other studies, highlighting the perceived importance of this topic in Europe. However, there were also several limitations. First, due a limited time frame, not all outcomes without 80% agreement could be discussed during the 2‐h panel meeting. Therefore, only the most critical outcomes (75–80% agreement) were selected for discussion. Second, since we based our list of learning outcomes only on articles published in peer‐reviewed journals, we may have missed valuable information from nonpeer‐reviewed articles such as reports published by educational and pharmacological societies. Third, only 26% of the panelists attended the face‐to‐face meeting. Although this is a relatively small proportion, the group represented most participating European countries and was large enough to provide a reliable consensus.27 Lastly, there was a poor response from junior doctors. This was most likely due to the time involved in participating in the questionnaire rounds.

Conclusion and recommendations

A large European expert panel reached consensus on 252 learning outcomes that should be included in undergraduate CPT curricula to ensure that European medical graduates are able to prescribe safely and effectively. Since CPT is a dynamic specialty that is subject to continual change, the identified learning outcomes should be reviewed and revised periodically by a European group of CPT teachers.

In order to help medical schools to implement the identified learning outcomes, we provide a blueprint of an integrated context‐based learning European core curriculum in CPT describing when and how the outcomes might be taught and assessed during the early and later years of the medical curriculum (Figure 3). This blueprint describes the subcategories of the identified learning outcomes (Table 1) together with useful teaching and assessment methods that have been successful at various medical schools. Context‐based learning refers to learning in the setting that is the same or as similar as possible to the setting of the future profession; for medical students, that is clinical practice. CPT should be integrated longitudinally as a recurrent theme in modules and attachments throughout the medical curriculum, starting as early as possible. We believe that students might acquire a better understanding of CPT if it is frequently repeated in different modules over several study years compared to one or two distinct courses. In the early years of the medical curriculum (i.e., years 1–3 in Europe, and years 1–2 in the USA), the emphasis lies on gaining knowledge of basic and clinical pharmacology (e.g., lectures, E‐learning), while simultaneously learning to apply this knowledge by training prescribing skills in controlled (e.g., case‐based discussions) and simulated (e.g., role‐playing sessions) environments. In the later years (i.e., years 4–6 in Europe, and years 3–4 in the USA), as knowledge increases, more emphasis is given to training prescribing skills in clinical environments (e.g., real‐world prescribing, pre‐prescribing seminars), while the acquisition of knowledge diminishes. During the early and later years, most of the teaching time should be devoted to the teaching and training of prescribing skills and attitudes, since these are highly complex. To ensure that all relevant outcomes have been achieved, there should be a robust and separate CPT assessment structure, with no compensatory mechanism. Computer‐based assessments are useful for testing the knowledge, judgment, and skills of large cohorts of students, since they are relatively quick and easy to mark. Moreover, objective structured clinical examinations are useful to assess the prescribing skills in the early years of the medical curriculum, whereas workplace‐based assessments are preferred in the later years. The development of students' attitudes can be evaluated using a portfolio. At or near the end of the medical curriculum, there should be a valid and reliable summative (national) assessment such as the UK Prescribing Safety Assessment28 or Dutch National Pharmacotherapy Assessment29 to assess whether medical graduates are able to prescribe safely and effectively. One or more individuals, preferably clinical pharmacologists, should be responsible for coordinating all teaching and assessment activities throughout the curriculum. Since the proposed blueprint is resource‐intensive, clinical pharmacologists can and should not work on this alone. Teaching and assessment can be devolved to many other teachers throughout the course, often with organ‐based specialties. Also, “near‐peer” education (e.g., junior doctors, medical students) might be helpful to reduce the workload of the usually small group of CPT teachers. This blueprint describes how a European core curriculum in CPT might look like but can be adapted to suit the local preferences of medical schools, given the differences in institution's culture and resources.

Figure 3

Blueprint of an integrated context‐based European core curriculum in clinical pharmacology and therapeutics describing when and how learning outcomes might be taught and assessed during the early (i.e., years 1–3 in Europe and years 1–2 in the USA) and later years (i.e., years 4–6 in Europe and years 3–4 in the USA) of the medical curriculum. Journal clubs refer to discussing the scientific CPT literature in small groups of students in order to teach critical appraisal skills. Case‐based discussion groups involve discussing written patient cases in small groups of students together with a tutor. Role‐playing sessions refers to conducting therapeutic consultations with simulated patients. Pre‐prescribing seminars involve medical students writing instructions on in‐patient drug charts, which have to be validated by a doctor before drugs are dispensed. Prescribing tutorials involve ward‐based practical tutorials about common therapeutic problems and high‐risk medicines for medical students during clinical attachments. Student‐run clinics are real‐patient clinics designed to teach and train prescribing skills grounded in a real‐life context and to provide students with early clinical experience and responsibility under the supervision of a senior clinician. Workplace‐based assessment entails direct observation of prescribing in clinical practice under the supervision of a senior clinician. OSCE, objective structured clinical examination.

METHODS

A Delphi study is a consensus method to determine the extent to which experts agree about a given issue. This approach is often used when there is a lack of scientific evidence or when there is contradictory evidence on an issue, leading to a diversity of opinions.30, 31 It has proven to be a suitable method for determining the content of a CPT curriculum.14, 15, 17 The modified Delphi process took place between January and July 2017 and comprised the following: a systematic literature search; selection of a European expert panel; and development of a web‐based questionnaire and its modification in two consecutive rounds followed by a face‐to‐face meeting and final round. This modified Delphi process was adopted to minimize time demands on the expert panel. Ethical approval for this study was provided by the Dutch Ethics Review Board of Medical Education (Approved Project no. NVMO‐ERB 860).

Systematic literature search

With the assistance of a medical information specialist, we systematically searched three international databases (PubMed, EMBASE, and ERIC) for articles describing CPT learning outcomes (i.e., knowledge, skills and attitudes) in undergraduate medical curricula and published up to 23 January 2017. Search terms included the following mesh terms as well as a combination of free text words and mesh terms in title or abstract: “Medical Student,” “Medical Graduate,” “Medical Undergraduate,” “Medical Education,” “Curriculum,” “Drug Therapy,” “Pharmacotherapy,” “Pharmacology,” “Prescriptions,” “Prescribing,” “Competence,” “Expertise,” “Knowledge,” “Skills,” and “Attitudes.” Articles were assessed independently for eligibility by two researchers (D.B. and S.d.G.), based on a list of predefined inclusion and exclusion criteria. Reference lists from included publications were also screened to identify additional articles. Reviews, letters to the editor, editorials, conference abstracts, symposium reports, and qualitative studies were excluded. Also, studies were excluded if a specific educational intervention or teaching method was evaluated, the language was different from English, German, or Dutch, or if the abstract or full‐text was not available. Based on articles identified in the literature search (Table S2), specifically the article by Ross and Maxwell,13 the steering committee extracted a list of 307 learning outcomes, divided into 35 subcategories. The steering committee consisted of a clinical pharmacologist, a junior doctor, an internist‐infectious disease specialist, and a senior lecturer in prescribing.

European expert panel

We invited European experts in clinical pharmacology, pharmacy, and medical education, as well as junior doctors and senior clinicians working in primary and secondary care, to participate in a consensus panel. Experts were selected from the EACPT Network of Teachers in Pharmacotherapy (NOTIP), a European platform for CPT teachers that supports the development and sharing of teaching materials and participation in joint research projects. Participants of previous research projects of the EACPT Education Working Group were specifically invited to participate.8, 10 All experts had to be involved in developing or delivering CPT education to students training in a health profession and should have at least 3 years' educational experience. Participants received an e‐mail containing information about the general objectives of the study and instructions about the Delphi process. Participation was voluntary and informed consent was obtained from all participants.

Data collection and analysis

We developed a web‐based questionnaire (using http://www.surveymonkey.com) containing 307 key learning outcomes divided into three categories: knowledge (n = 233), skills (n = 65), and attitudes (n = 9). Minor modifications to the content were made after a pilot study with seven European CPT teachers.

In the first round, panelists were asked to rate each outcome (1 = very unimportant, 2 = unimportant, 3 = neutral, 4 = important, 5 = very important), indicating their agreement that the outcome should be included in the undergraduate CPT curriculum and should be expected of European graduates in order that they can prescribe safely and effectively. If panelists awarded an outcome a score of 4 or 5, they were asked to indicate whether that outcome should be acquired during the preclinical (i.e., bachelor's degree) or clinical (i.e., master's degree, clerkships) years of the curriculum, or both. Additionally, panelists could also change the wording of outcomes and add new outcomes if they felt these were missing. An outcome was included if ≥80% of the experts gave it a score of 4 or 5. This cutoff is in line with a similar study14 and was chosen in order to create a list of “need to know” learning outcomes rather than one with “nice to know” outcomes.

In the second round, panelists were shown which outcomes did not meet the 80% agreement cutoff in the first round, the additional suggested outcomes, and the group score for each outcome. Panelists were asked to reconsider each outcome based on the group opinion using the same 5‐point scale. They had the opportunity to justify their point of view in an open text box. During these two rounds, panelists were anonymous to other panelists and individual scores were confidential.

Panelists were invited to attend a 2‐h face‐to‐face meeting during the 13th congress of the EACPT (http://www.eacpt2017.org). During this meeting, panelists discussed the outcomes with 75–80% agreement after Round 2 and were asked to vote (yes, no, unsure) for acceptance, rejection, or modification. Voting was anonymous via mobile devices. In the third round, panelists were asked to rate outcomes that had been rephrased during the panel meeting using the 5‐point scale. A summary of the arguments for and against changes was provided. Panelists were allowed 3 weeks to complete each questionnaire round; reminders were sent after 1 and 2 weeks. After each round, responses were downloaded in Excel format and analyzed descriptively using IBM SPSS v. 22.0 (Armonk, NY).

CONFLICT OF INTEREST

As an Associate Editor for Clinical Pharmacology & Therapeutics, Ingolf Cascorbi was not involved in the review or decision process for this article. The other authors declare no conflicts of interest.

AUTHORS CONTRIBUTIONS

D.J.B., J.T., L.B.M., M.C.R., and M.A.v.A. wrote the article; D.J.B., J.T., L.B.M., M.C.R., and M.A.v.A. designed the research; D.J.B., J.T., T.C., R.L., R.M., J.C., E.J.S., and S.R.M. performed the research; D.J.B. and L.B.M. analyzed the data; D.J.B. and L.B.M. contributed new reagents/analytical tools.

Supplementary Material

Click here for additional data file.