Harmonizing and improving European education in prescribing: An overview of digital educational resources used in clinical pharmacology and therapeutics.

AIM: Improvement and harmonization of European clinical pharmacology and therapeutics (CPT) education is urgently required. Because digital educational resources can be easily shared, adapted to local situations and re-used widely across a variety of educational systems, they may be ideally suited for this purpose.
METHODS: With a cross-sectional survey among principal CPT teachers in 279 out of 304 European medical schools, an overview and classification of digital resources was compiled.
RESULTS: Teachers from 95 (34%) medical schools in 26 of 28 EU countries responded, 66 (70%) of whom used digital educational resources in their CPT curriculum. A total of 89 of such resources were described in detail, including e-learning (24%), simulators to teach pharmacokinetics and/or pharmacodynamics (10%), virtual patients (8%), and serious games (5%). Together, these resources covered 235 knowledge-based learning objectives, 88 skills, and 13 attitudes. Only one third (27) of the resources were in-part or totally free and only two were licensed open educational resources (free to use, distribute and adapt). A narrative overview of the largest, free and most novel resources is given.
CONCLUSION: Digital educational resources, ranging from e-learning to virtual patients and games, are widely used for CPT education in EU medical schools. Learning objectives are based largely on knowledge rather than skills or attitudes. This may be improved by including more real-life clinical case scenarios. Moreover, the majority of resources are neither free nor open. Therefore, with a view to harmonizing international CPT education, more needs to be learned about why CPT teachers are not currently sharing their educational materials.

What is known about this subject?

Junior doctors have insufficient skills in prescribing and there is large variation in quality within Europe. Improvement and harmonization of European CPT education is urgently required.

Digital educational resources are effective in teaching the knowledge and skills for safe prescribing and may be an ideal way to harmonize international education.

What this study adds

A large variety of digital educational resources are commonly used to teach CPT.

Access to most currently used digital educational resources is restricted to users in specific medical schools. Opening up and sharing digital educational resources for broader use may help to improve and harmonize international CPT education.

INTRODUCTION

An academic degree in medicine (medical qualification) granted by the governing medical body in one country of the European Economic Area or Switzerland is valid for all countries of the European Union (EU). Common requirements for a medical qualification in these countries are at least 5,500 training hours and sufficient knowledge and skills on broadly defined topics (e.g. diagnosis, therapy and human reproduction). However, when it comes to the safe and effective prescribing of medicines, there are large differences in the number of hours of education and teaching methods within the EU. On the basis of the interquartile range of active instruction hours, a recently graduated doctor with less than 36 hours of traditional clinical pharmacology and therapeutics (CPT) lectures and another doctor with more than 100 hours of experience solving authentic case‐based prescribing scenarios may both be at the same bedside in any given EU hospital. A European study has shown that while more education and training in prescribing led to better results in a case‐based examination of prescribing skills, 46% of prescriptions still contained one or more errors. On the basis of these results, the European Association for Clinical Pharmacology and Therapeutics (EACPT) has recognized the need to improve and harmonize European CPT education. Digital educational resources are ideal for this, because they can be easily shared over the Internet, translated and adapted to local standards—they can be reused widely across a variety of educational systems. Moreover, digital educational resources are effective in teaching the complex cognitive skills and attitudes required for safe prescribing. For this reason, the EACPT and its affiliated Network of Teachers in Pharmacotherapy (NOTIP) seek to promote international collaboration via an online platform for sharing and creating digital educational CPT resources. The aims of this study were to provide an overview of the digital educational resources currently used in European CPT education, to assess the learning objectives they address, and to evaluate their suitability for international use.

METHODS

Population

Principal CPT teachers at medical schools in the 28 countries of the EU were sent an invitation to a cross‐sectional online survey. The single medical school in Luxembourg was excluded, because it only offers a bachelor's degree. All invitations included a personal link and were automatically sent via the certified electronic case report program CastorEDC (www.castoredc.com). During the study period (from 27 March 2019 to 13 May 2019), invitees who had not completed the survey received up to two reminders (sent 16 March and 1 May). The process of retrieving contact information is described in detail elsewhere. In brief, one representative in each country was asked to provide the email addresses of his/her colleagues; missing information was retrieved via national pharmacological societies.

Survey

Participants were asked to describe their four most valued (“best practice”) digital resources in detail. Questions about the target population (e.g. medical students, junior doctors or nurse‐prescribers, etc.) and the features listed in Table 2 were mandatory and answered by checking boxes. The estimated time it took to use the resource was scored on a skewed scale (0–20; 20–40; 40–60; 60–90; 90–120 and >120 minutes). Optional open‐text‐questions were available to provide additional information about the resource (e.g. on learning objectives, a web link and possible literature references). Participants could suggest how the resource should be classified according to the definitions in Table 1, but the final classification was made by the researchers. Multiple descriptions about (parts of) the same resource were combined into one and incomplete descriptions were excluded.

TABLE 2

Overview of resource characteristics

Type of resource	Frequency	Median time to complete	Available on‐demand	Blended	Case‐based	Compulsory
E‐learning	21 (23.9%)	90–120 min.	14 (66.7%)	13 (61.9%)	7 (33.3%)	7 (33.3%)
PK/PD simulator	9 (10.2%)	40–60 min.	2 (22.2%)	8 (88.9%)	2 (22.2%)	4 (44.4%)
Video	8 (9.1%)	20–60 min.	5 (62.5%)	6 (75.0%)	2 (25.0%)	0 (0%)
Digital assessment	7 (8.0%)	60–90 min.	1 (14.3%)	1 (14.3%)	3 (42.9%)	4 (57.1%)
Virtual patient	7 (8.0%)	60–90 min.	3 (42.9%)	5 (71.4%)	7 (100%)	2 (28.6%)
CDSS	7 (8.0%)	20–40 min.	4 (57.1%)	3 (42.9%)	0 (0%)	2 (28.6%)
Student formulary	4 (4.5%)	> 120 min.	4 (100%)	3 (75%)	2 (50%)	2 (50%)
Digital book	5 (5.7%)	90–120 min.	3 (60%)	3 (60%)	2 (40%)	0 (0%)
(In‐class) social media	5 (5.7%)	0–20 min.	3 (60%)	3 (60%)	3 (60%)	1 (20%)
Serious game	4 (4.5%)	20–40 min.	2 (50%)	2 (50%)	0 (0%)	0 (0%)
Slide repository	3 (3.4%)	20–40 min.	2 (66.7%)	0 (0%)	1 (33.3%)	1 (33%)
Audio	2 (2.3%)	20–40 min.	1 (50%)	0 (0%)	1 (50%)	0 (0%)
Recorded lectures	2 (2.3%)	> 120 min.	2 (100%)	1 (50%)	1 (50%)	1 (50%)
EPS sandbox	2 (2.3%)	90–120 min.	1 (50%)	0 (0%)	1 (50%)	0 (0%)
Augmented reality	1 (1.1%)	90–120 min.	1 (100%)	1 (100%)	1 (100%)	0 (0%)
Faculty website	1 (1.1%)	40–60 min.	1 (100%)	1 (100%)	0 (0%)	0 (0%)
Total	88 (100%)	40–60 min.	49 (55.7%)	50 (56.8%)	33 (37.5%)	24 (27.3%)

Blended means combined with in‐class teaching on the same subject. PK, pharmacokinetics; PD, pharmacodynamics; CDSS, clinical decision support system; EPS, electronic prescribing system.

TABLE 1

Definitions used for the classification of resources

Resource type	Definition used for this study
E‐learning a	Computer‐based course that offers a variety of texts, multimedia and interactive elements. Often, though not necessarily, online.
PK/PD simulator	Interactive visualization of pharmacokinetics and/or pharmacodynamics modelling.
Video	Educational video (other than recorded lecture).
Digital assessment	Formative or summative examination using computer technology.
Virtual patient	Interactive patient scenario (may be text‐based or high‐fidelity software simulations, not mannequins or real‐life actors).
Knowledge databases and computerized clinical decision support systems	Database with drug information or clinical practice guidelines, drug–drug interaction checker or other computerized clinical decision support.
Student formulary	A digital drug formulary created by students (as personal or collaborative effort), used for educational purposes.
Digital book	Digitized books, primarily text‐based but may contain interactive elements and multimedia.
(In‐class) social media	Software applications that enable students to interact with peers and/or teachers.
Serious game	Game designed to educate its users (may be text‐based or high‐fidelity simulation, not non‐digital games).
Slide repository	A digital archive of slides or images used for educational purposes.
Audio	Educational audio files (other than recorded lecture).
Recorded lectures	Recordings of lectures to be replayed at a later time. May be audio files, screen captures or a combination of both.
EPS sandbox	A copy of the electronic health records and/or prescribing system, created for educational purposes. May contain fictional or anonymized patients.
Augmented reality	Computer‐generated images projected on a real‐world view.

E‐learning is also commonly used as synonym for all types of digital educational resources. Please note that for this study, the more‐strict definition was used in order to distinguish computer‐based courses from other types of digital education.

Resource inspection

A researcher (M.B.) logged in to resources available on the Internet to verify and complete the information supplied by the teacher and to appraise the copyright license. If the resource was in English or Dutch and access was limited with login restrictions, the author was contacted with a request for demo login credentials. If the resource could not be accessed and the information supplied was insufficient, the author was contacted with follow‐up questions via e‐mail. The Key Learning Objectives for Clinical Pharmacology and Therapeutics Education in Europe, as published by Brinkman et al., were used to identify learning objectives. The copyright licenses are described using the Creative Commons (CC) terminology: “BY” means that attribution of the original source is required; “NC” means that non‐commercial use is allowed, but commercial use is prohibited; “ND” indicates that adaptation and re‐distribution is not allowed; and “SA” means that adaptation and re‐distribution is only allowed when the license is left unchanged.

Ethical considerations

The medical ethics board of Amsterdam UMC location VUmc declared that this study did not fall within the scope of the Medical Research Involving Human Subjects Act (WMO). The Dutch Ethics Review Board of Medical Education approved the study (NVMO‐ERB 2018.8.12). All participants provided digital informed consent prior to receiving access to the survey. Consent to re‐analyse the data gathered in previous EACPT education working group surveys was asked separately and was not required to access the survey.

Data analyses

Data were downloaded to Excel and analysed using SPSS (IBM version 24.0). The International Monetary Fund 2018 data for gross domestic product (GDP) per capita were used. Correlations between GDP and number of resources were tested using Pearson coefficients. All other data are shown using descriptive statistics only, mean ± standard deviation or median (25th–75th percentile).

RESULTS

Participating medical schools

Three hundred and ninety‐three CPT teachers from 279 (of 304, 92%) medical schools in the EU were invited to participate. A total of 99 teachers answered a sufficient number of questions (more than just demographics) to allow data analysis (91 teachers answered all the questions). In four universities two teachers answered the survey. Therefore, the total number of distinct medical schools was 95 (34% of all). Medical schools in 26 of 27 eligible EU countries were represented in the analyses (none of the five invitees from Austria answered the survey). The participating teachers were on average 54 ± 0.8 years old and had a mean 20 ± 0.9 years of teaching experience. The median time allocated to teaching was 25% (15–40%) of their full‐time job.

Current use of digital resources

Figure 1 shows the proportion of medical schools per country that used at least one digital resource in their CPT curriculum. Overall, 66 (69.5%) medical schools used digital resources in their CPT curricula. Re‐analysing data from the previous EACPT education working group survey showed a 10.5% increase in the use of digital resources since 2016. Twenty‐one medical schools had started using digital resources since then, whereas seven schools had stopped. The median number of digital resources used per school was 2 (range 1–4). Neither the proportion of schools using digital resources per country, nor the median number of digital resources used per medical school was correlated with GDP per capita (r = 0.25; P = 0.23 and r = −0.17; P = 0.44, respectively).

FIGURE 1

Use of digital educational resources in the European Union

Best practices and resource characteristics

The teachers from the 66 medical schools that used digital resources provided 111 detailed descriptions of 88 distinct resources. Eighteen descriptions were combined into 5 resources used by multiple universities, 13 descriptions about modules of the same resource were combined into 5 resources, and 2 descriptions were excluded because they were not digital resources. A complete overview of all included resources is shown in Supporting Information Table S1. The majority of resources were categorized as e‐learning (23.9%), followed by pharmacokinetics (PK) and pharmacodynamics (PD) simulators (10.2%), videos (9.1%), and digital assessments (8.0%). Most resources (23.9%) took more than 120 minutes to complete, but resources taking 20–40 and 40–60 minutes to complete were common (18.2% each). While we asked about the total time needed to complete the resource, many resources consisted of shorter modules. Slightly more than half of resources (55.7%) were available to students anytime and anywhere, but not all resources were suitable for on‐demand availability (digital assessments, in‐class socials and PK/PD simulators). Fifty‐seven per cent of the digital resources were considered “blended” and could be used before (“flipped classroom”), during or after in‐class teaching on the same subject. Table 2 shows an overview of the different types of digital resources and their characteristics. All resources were primarily used in medical education (88% to teach medical students, 28% junior doctors and 17% consultants). Overall, 33% of the resources were used in more than one type of education (pharmacy, dental and nurse‐prescribing students).

Learning objectives

For 66 resources (75%) there was enough information to score learning objectives. The median number of learning objectives addressed per resource was 3 (range 2–6). The highest number was 24 in the SCRIPT e‐learning resource from the UK. The resources addressed a total of 235 knowledge‐based learning objectives, 88 skills and 13 attitudes. Table 3 shows the number of resources that addressed a learning objective; the learning objectives per resource are shown in Supporting Information Table S1.

TABLE 3

Learning objectives

Learning objectives	No. of resources	Learning objectives	No. of resources
Knowledge		Knowledge (continued)
1. Introduction to clinical pharmacology and therapeutics	5	13. Legal and ethical aspects of prescribing	3
1.1. Basic principles	5	13.1. Legal aspects	3
1.2. Drugs in healthcare and society	4	13.2. Ethical aspects	2
2. Pharmacodynamics	17	14. Prescribing for patients with special requirements	16
2.1. Mechanism of action	15	14.1. Elderly patients	11
2.2. Dose–response relationships	13	14.2. Impaired liver function	10
3. Pharmacokinetics	23	14.3. Impaired renal function	13
3.1. Drug absorption, distribution, metabolism, and excretion	23	14.4. Pregnant women and women of childbearing potential	13
3.2. Concentration–time relationships	20	14.5. Lactation	10
3.3. Repeated drug dosing	19	14.6. Children	11
4. Individual variability in the response to drugs	13	15. Rational prescribing	18
4.1. Basic principles	10	15.1. Rational approach to prescribing	16
4.2. Pharmacokinetic variability	12	15.2. Dose selection	17
4.3. Pharmacogenetic variability	6	16. Clinical toxicology	2
5. Adherence, compliance, and concordance	4	17. Misuse of drugs	3
5.1. Adherence and compliance	4	18. Complementary and alternative medicine	2
5.2. Concordance	3	19. Use of antibiotics and antibiotic resistance	15
6. Therapeutic drug monitoring	2	20. Commonly used and high‐risk medicines	34
6.1. Basic principles	2
6.2. Using drug effect and concentration	2	Skills
7. Adverse drug reactions	21	21. Medication history taking	4
7.1. Basic principles	18	22. Rational prescribing	20
7.2. Drug allergy	18	23. Drug dose calculation	11
7.3. Diagnosis, management, and prevention	17	24. Prescription writing	12
7.4. Pharmacovigilance	15	25. Non‐drug therapy	2
8. Drug interactions and contraindications	18	26. Communication	6
8.1. Interactions	17	27. Reviewing prescriptions	4
8.2. Contraindications	15	28. Adverse drug reactions	7
9. Medication errors	10	29. Clinical toxicology	0
10. Drug discovery, development, and regulation	6	30. Obtaining information from guidelines and protocols to support prescribing	14
10.1. Drug discovery and development	4	31. Monitoring medication	8
10.2. Drug regulation	4
11. Medicines management	5	Attitudes
11.1. National and local processes	4	32. Risk–benefit analysis	8
11.2. Formularies and guidelines	4	33. Recognizing personal limitations and knowledge	1
12. Evidence‐based prescribing	18	34. Balanced approach to the introduction of new drugs	4
12.1. Basic principles	13
12.2. Critical appraisal of clinical studies	12
12.3. Find reliable information about drugs	16

Sub‐objectives are shown in italic. When a resource addressed more than one sub‐objective, the main objective was only counted once. Learning objectives were previously defined in Brinkman et al.

Narrative overview of digital educational resources

E‐learning

E‐learning was the most frequently used type of digital resource. These resources were highly heterogeneous in their learning objectives, duration and characteristics. The resource described most often was the Dutch/Flemish Pscribe, a modular e‐learning program with its own content management system built around the six‐step method of the WHO Guide to Good Prescribing (WHO‐GGP). The focus is on rational prescribing, but modules on adverse drug reactions and pharmacokinetics also exist. Access is restricted to the universities that helped co‐develop the program. SCRIPT e‐learning for effective and appropriate prescribing practice is the most widely used resource in the UK (used by 13 of 33 medical schools). The resource is commercial and accessible via a paid personal or institutional login. It is used to teach almost all (student) prescribers and is a mandatory part of foundation‐year residency training in selected regions. The UK government also funds the E‐learning for healthcare platform, which contains two CPT resources (Safe prescribing and Prescribing simulator ). These appear to be widely used, but are only accessible to UK‐based academic institutes. An example of a European collaboration is the commercial EU2P program on pharmacoepidemiology and pharmacovigilance. The Teaching Resource Centre (TRC) is the only open‐access e‐learning (equivalent to CC‐BY, Internet and mobile application). It focuses on basic pharmacology but offers case‐based WHO‐GGP six‐step modules as well. The IUPHAR education project is a hybrid between e‐learning and an online book. It is freely available (CC By‐NC‐ND) and offers text, videos and hyperlinks to other resources on a range of subjects varying from basic pharmacology to clinical pharmacology and therapeutics. The other 15 included e‐learning programs were local initiatives (see Supporting Information Table S1).

PK/PD simulators

PK/PD simulators were the next most‐used resource. They are used to teach basic pharmacokinetic principles, such as the freely online accessible Interactive pharmacology, which came from New Zealand but was described by two Spanish and one Romanian CPT teacher. These simulators can also be used to model the pharmacodynamic effects of drugs, such as Biosoft Cardiolab, which models the effects of sympathetic agonists and antagonists on blood pressure and heart rate.

Videos

Eight video resources were described, addressing a variety of learning objectives such as how to use the British National Formulary and dose reductions in renal insufficiency. Most of these resources are accessible only via institutional login. Open‐access alternatives include the Medscape continuing medical education (CME) videos on diverse classes of drugs and the Dutch Medicine of the week videos on YouTube (CC By‐NC‐ND). An Italian teacher showed how YouTube videos on physiology and pharmacology can be used as preparation for a flipped classroom experience.

Digital assessments

Seven digital assessments were described, varying from practice questions offered in local Moodle systems to large national or international prescribing safety examinations. The best known is the Prescribing safety assessment, which is used in all UK medical schools and in Ireland and Malta. Students are allowed access to the British National Formulary during the examination, which focuses on essential skills, such as writing a prescription, reviewing it and drug‐dosage calculations. Successful completion is mandatory for all foundation year 1 doctors. The Dutch‐Flemish equivalent is the Assessment for safe prescribing. An increasing number of universities uses it as a summative final pharmacotherapy assessment.

Virtual patients

Virtual patients allow students to diagnose and/or treat a patient without doing harm. The most interesting virtual patients, made by Queen Mary University London, cover anticoagulation and oxygen therapy and include several game‐design principles, such as freedom of choice and achievements, which make these patients highly entertaining and motivating for students to use. The other six included virtual patients are text‐based scenarios, such as the Croatian Advanced Medical Therapeutics course, which offers 90 short clinical cases as well as e‐learning‐like background information. This course was first created by the University of Michigan Medical School and then openly translated and adapted for use in Croatia. Another example of a well‐described virtual patient is the Cypriot resource on Thyroid pathology and pharmacotherapy. In this longitudinal case, one patient is followed through pregnancy and lactation, a thyroid storm, and even an adverse drug reaction. Its complete storyboard was recently published.

Computerized clinical decision support systems and knowledge databases

Three drug formularies (e.g. the Swedish Janusinfo ), two drug–drug interaction checkers (e.g. UpToDate ), a national clinical practice guideline platform (i.e. the Finnish Terveysportti.fi ), and a tool for prescribing gastric ulcer prophylaxis (i.e. the Dutch DrugChoice ) are used in medical education. Learning objectives focus on the skill of finding reliable information in a timely manner.

Student formularies

Rather than using existing drug databases, five teachers described the use of a formulary created by students. These so‐called personal drugs or “p‐drugs” are routinely used medications that students should be familiar with and able to prescribe rationally. They were first introduced by the WHO‐GGP as offline documents, but now several digital variants are available. Examples include collaborative lists such as the Southampton Student Formulary, which has restricted access but is based on eDrug made in Edinburgh (not described, CC‐BY). In the Spanish/English Pdrugs (website and mobile app‐based), students can easily create and access their own personal formularies according to the six steps of the WHO‐GGP. SmartDrug was created by Brighton and Sussex medical school and lets students create a formulary that mirrors their clinical rotations.

Digital books

The Top 100 Drugs is a formulary of essential drugs made by teachers rather than students. Four other digital books were also described. The two volumes of the iBooks Prescribing Skills Handbook are interactive books with videos and quiz questions, available for free on Apple iPhones and iPads, and were created by the University of Manchester to prepare students for the Prescribing Safety Assessment. A Portuguese university uses Access Medicine, which is a digital library containing books, but also a drug database and interactive cases files.

(In‐class) social media

Kahoot!, Socrative, and WooClap are three examples of digital tools that can encourage in‐class interactivity with students. Teachers in France, Slovenia and Spain use these tools to teach a variety of learning objectives. All three resources offer free (limited) as well as paid (full) subscriptions; functionalities for asynchronous use, open questions and gamification vary slightly. Alternatively, Twitter is used for CME with a weekly bilingual (French and English) formative PharmacoQuiz, with questions being posted on Friday and answered on Monday.

Serious games

Four resources could be described as serious games. All are in essence quizzes, with varying game elements such as high scores and competition. The Dutch Battle of the Meds mobile application challenges doctors and students to compete in 1‐versus‐1 quizzes and provides monthly high scores. Learning objectives include commonly used medications, adverse drug events, and PK/PD. The Swedish Läkemedelsquiz (“Medicine quiz”) offers a range of short factual knowledge questions on medication names, indications and dosages of diverse groups of medications. However, due to a lack of competition or scores, it appears to be more of a formative assessment than a serious game. Flippity.net can be used to create a “Jeopardy!”‐like gameshow which can be hosted in‐class, making teams of students compete for most points. Cram.com is a similar resource used to create a “Candy Crush”‐like game called Jewels of wisdom that can be played independently by students.

Slide repositories, recorded lectures and audio

Many universities make their slides available to students, as PowerPoint files, audio recordings, or a combination of both. Seven such resources were described, one of which stands out because it is an image and slide repository on Neuropharmacology made openly available (equivalent to CC‐BY) by the pharmaceutical company Lundbeck.

Electronic prescribing system sandboxes

The Erasmus Medical Center in Rotterdam and the universities of Birmingham and Manchester have created a copy of their electronic patient records and prescribing system (EPS) for use in medical education. Students can experience the complexity of an EPS without doing harm to real patients. Learning objectives for the Rotterdam resource include selecting the right drug dosages and administration routes, dealing with automatic medication safety signals and drug–drug interactions.

Augmented reality

One British teacher mentioned the use of the mobile augmented reality application Blippar to teach prescribing. A camera is used to project an interactive image over the real‐world environment when a specific object is scanned (e.g. the summary of product characteristics when a specific pillbox is scanned). Teachers can use the free Blippbuilder to link information to the objects.

DISCUSSION

Back in 2012, Maxwell and Mucklow predicted that digital resources would be used extensively in the future. Now, 7 years later, almost 70% of European CPT curricula use digital educational resources. The nature of these resources is diverse, ranging from e‐learning programs to serious games and electronic prescribing system sandboxes. The goal of this study was to provide an overview of these resources, to characterize them, and to assess their potential for international use as a way to harmonize CPT education. This article provides a crucial next step in the plans of the EACPT education working group to stimulate European collaboration via an online platform. The underlying aim is to help international CPT teachers to collaboratively create new educational materials and to share existing materials among colleagues.

We have previously published a list of recommendations for the creation of digital educational resources for CPT. From a student perspective, an ideal resource is one that can be accessed on demand (anywhere and anytime), that provides the student with autonomy to choose the order and depth of review, and that is split up in “bite‐size” chunks of information that do not exceed the capacity of the student's working memory. The use of game‐design principles (e.g. competition, collaboration or a time limit) and/or providing authentic patient problems may increase motivation to use these resources. For teachers, the potential to easily share, adapt and distribute educational materials after only a single investment of time and money is undoubtedly the most important aspect of digital educational resources.

Unfortunately, many of the resources mentioned do not exhibit more than a few of the above‐mentioned features, mainly because of copyright and access issues. Currently, only 31% of resources are available for free without the requirement for an institutional login. For successful international sharing, free access is an absolute requirement. However, free access alone may not be sufficient because resources may need to be adapted to address international differences (in guidelines, prescribing etiquette, etc.) and translated. Only two resources (Teaching resource center and Lundbeck's neuropharmacology slides) explicitly state that this kind of adaptation is allowed in their copyright license. These resources can be classified as open educational resources according to the UNESCO definition: teaching, learning and research materials in any medium—digital or otherwise—that reside in the public domain or have been released under an open license that permits no‐cost access, use, adaptation and redistribution by others with no or limited restrictions. , Such open licenses are uncommon for commercial products, but it is not clear why universities, and especially those that are publicly funded, have imposed copyright restrictions on the resources they have developed. Previous research suggests that teachers are hesitant to use and create open educational resources because of a lack of awareness, motivation and training in their use. Other possible reasons why resources are rarely shared include language issues and local differences (in educational systems, guidelines, rates of antimicrobial resistance, etc.). This needs to be investigated among European CPT teachers. Besides the overall increase in use of digital resources over the last years, our data also show that seven universities that used digital educational resources in 2016 have since stopped. In line with our own experience, this may be because it is difficult to find the time and money to maintain and update digital resources in the years after they were first developed. International collaboration could help improve the sustainability of digital resources.

Another important finding was the learning objectives of these resources. The WHO Guide to Good Prescribing is well‐established as the best way to teach CPT, , because it advocates a problem‐based teaching approach whereby students train the actual prescribing process rather than receive factual knowledge only. However, in the current overview, only 37.5% of resources used case‐based problems. Moreover, while all the resources aimed to teach one or more knowledge objectives, only 31 addressed at least one skill and 12 an attitude. The lack of skills teaching is not an issue for digital resources, as it has been previously shown that these resources can be used to teach skills and attitudes as well as knowledge, and there are many examples of resources that focus on skills acquisition, such as the Prescribing safety assessment and Prescribing system sandboxes. Unfortunately, conventional curricula with more traditional teaching than problem‐based teaching methods are still dominant in many European medical schools. Therefore, we recommend that future educational methods (digital and otherwise) should be based on authentic clinical scenarios as much as possible.

Limitations

Because of login restrictions, we were unable to inspect all the resources mentioned. In these cases, we asked the teachers who recommended the resource for additional information and requested a demonstration login for inaccessible Dutch and English resources. As with any voluntary survey, there is a risk of participation bias. In this case, digitally active teachers may have been more likely to participate than their less‐active colleagues, thus overestimating the number of digital resources used. The proportion of participants per country showed an equal geographic distribution across Europe. The overview of resources is less likely to be biased, because teachers were asked to describe maximally four of their most valued or “best‐practice” resources per medical school. Thus, this overview should be seen as providing a list of potentially useful resources rather than a comprehensive overview. It should also be noted that the definitions used for the classification of resources, as provided in Table 1, are ill‐defined in the literature and were restricted to digital resources only, whereas serious games, virtual patients and student formularies are not necessarily digital.

CONCLUSION

Digital educational resources are used to teach CPT in European medical schools and vary from well‐established tools, such as e‐learning and digital assessments, to novel and innovative resources, such as prescribing system sandboxes and virtual patients. Prescribing is a complex cognitive skill, but current resources tend to have knowledge‐based learning objectives rather than skill‐based learning objectives. Therefore, we recommend that a more problem‐based teaching approach be used, which would allow students to practise prescribing on authentic case scenarios. Collaboratively creating new resources and sharing existing ones are important means to harmonize and improve European CPT education. However, copyright restrictions and login requirements currently severely limit the potential to share resources.

Future research should focus on the reasons why these restrictions are in place and what other challenges, such as differences in language, teaching methods or prescribing regulations, teachers perceive about sharing their resources with international colleagues.

CONTRIBUTORS IN THE NETWORK OF TEACHERS IN PHARMACOTHERAPY (NOTIP) (ALPHABETIZED BY COUNTRY)

Atanasova, Ivanka (Sofia University St. Kliment Ohridski, Sofia, Bulgaria); Ganeva, Maria (Trakia University, Stara Zagora, Bulgaria); Gatchev, Emil (Medical University of Sofia, Sofia, Bulgaria); Kostadinova, II (Medical University Plovdiv, Plovdiv, Bulgaria); Mimica Matanovic, S (University of Osijek, Osijek, Croatia); Vitezic, D (University of Rijeka Medical School, Rijeka, Croatia); Wozniak, Greta (University of Cyprus, Nicosia, Cyprus); Kmonickova, E (Charles University, Pilsen, Czech Republic); Urbanek, Karel (Palacky University, Olomouc, Czech Republic); Damkier, P (University of Southern Denmark, Odense, Denmark); Huupponen, RK (University of Turku, Turku, Finland); Auffret, Marine (Hospices civils de Lyon, Lyon, France); Bejan‐Angoulvant, T (Université de Tours, Tours, France); Chouchana, Laurent (Hospital Cochin, Paris, France); Cracowski, Jean‐Luc (University Grenoble Alpes, La Tronche, France); Drici, MD (University of Nice Côte d'Azur, Nice, France); Faillie, JL (CHU Montpellier, Montpellier, France); Geniaux, Hélène (CHU de Limoges, Limoges, France); Molimard, M (Université de Bordeaux, Bordeaux, France); Orlikowski, D (Versailles Saint‐Quentin‐en‐Yvelines University, Versailles, France); Palin, Karine (University of Bordeaux, Bordeaux, France); Pers, Y‐M (CHU Montpellier, Montpellier, France); Picard, Nicolas (CHU de Limoges, Limoges, France); Simon, N (Aix‐Marseille University, Marseille, France); Toussirot, E (CHU de Besancon, Besancon, France); Boger, RH (University Medical Center Hamburg‐Eppendorf, Hamburg, Germany); Cascorbi, I (University of Kiel, Kiel, Germany); Mueller, SC (University Medicine Rostock, Rostock, Germany); Regenthal, R (University of Leipzig, Leipzig, Germany); Schwab, M (Eberhard Karl University of Tübingen, Tübingen, Germany); Schwaninger, MS (University of Luebeck, Luebeck, Germany); Thuermann, PA (University Witten/Herdecke, Witten, Germany); Wojnowski, L (University Medical Center Mainz, Mainz, Germany); Kouvelas, D (Aristotle University of Thessaloniki, Thessaloniki, Greece); Riba, P (Semmelweis University, Budapest, Hungary); Kerins, David M (University College, Cork, Ireland); Williams, David J (Royal College of Surgeons in Ireland, Dublin, Ireland); Cosentino, M (University of Insubria, Varese, Italy); De Ponti, Fabrizio (University of Bologna, Bologna, Italy); Filippelli, Amelia (University of Salerno, Baronissi, Italy); Leone, R (University of Verona, Verona, Italy); Locatelli, Vittorio (University of Milano ‐ Bicocca, Monza, Italy); Jansone, Baiba (University of Latvia, Riga, Latvia); Gulbinovic, Romaldas (Vilnius University, Vilnius, Lithuania); Mifsud, Janet (University of Malta, Msida, Malta); Braszko, Jan J (Medical University of Bialystok, Bialystok, Poland); kocic, I (Medical University of Gdansk, Gdansk, Poland); Breitenfeld, Luiza (Beira Interior University, Covilhã, Portugal); Castelo‐Branco, M (University of Beira Interior, Covilhã, Portugal); Conea, Simona (“Vasile Goldis” Western University of Arad, Arad, Romania); Magyar, Ioan (University of Oradea, Oradea, Romania); Bevc, S (University of Maribor, Maribor, Slovenia); Krzan, Mojca (University of Ljubljana, Ljubljana, Slovenia); Bernal, ML (University of Zaragoza, Zaragoza, Spain); Capellà, D (University of Girona, Girona, Spain); Carcas, A (Universidad Autónoma de Madrid, University of Maribor, Spain); De Abajo, FJ (University of Alcalá, Alcalá de Henares, Spain); Lopez‐Rico, M (University of Salamanca, Salamanca, Spain); Lucena, MI (University of Malaga, Malaga, Spain); Pontes, C (Universitat Autonoma de Barcelona, Sabadell, Spain); Sanz, EJ (Universidad de La Laguna, La Laguna, Spain); Böttiger, Y (Linköping University, Linköping, Sweden); Le Grevès, Madeleine (Uppsala University, Uppsala, Sweden); de Waard‐Siebinga, I (University Medical Center Groningen, Groningen, The Netherlands); Janssen, Ben JA (Maastricht University, Maastricht, The Netherlands); Knol, Wilma (University Medical Center Utrecht, Utrecht, The Netherlands); Pandit, Rahul (University Medical Center Utrecht, Utrecht, The Netherlands); van Rosse, F (Erasmus Medical Center, Rotterdam, The Netherlands); Dent, G (Keele University, Keele, United Kingdom); Ferro, Albert (King's College London, London, United Kingdom); Hitchings, AW (St George's, University of London, London, United Kingdom); Kapil, V (Queen Mary University London, London, United Kingdom); Linton, KD (University of Sheffield, Sheffield, United Kingdom); Loke, YK (University of East Anglia, Norwich, United Kingdom); Okorie, Michael (Brighton and Sussex Medical School, Brighton, United Kingdom); Plumb, Richard David (Queen's University Belfast, Belfast, United Kingdom); Pontefract, Sarah (University of Birmingham, Birmingham, United Kingdom); Ranmuthu, S (Queen Mary University London, London, United Kingdom); Sampson, AP (University of Southampton, Southampton, United Kingdom); Thanacoody, HKR (Newcastle University, Newcastle upon Tyne, United Kingdom); Whitfield, Jonathan P (University of Aberdeen, Aberdeen, United Kingdom); Wilson, Kurt (University of Manchester, Manchester, United Kingdom).

COMPETING INTERESTS

There are no competing interests to declare.

CONTRIBUTORS

All authors were responsible for writing the manuscript and designing the research. M.B., J.T., M.R. and M.A. analysed the data. M.B. also performed the research.

TABLE S1 Overview of all described resources

Click here for additional data file.