Should medical students learn to develop a personal formulary? An international, multicentre, randomised controlled study.

OBJECTIVE: This study was performed to determine whether students who are trained in developing a personal formulary become more competent in rational prescribing than students who have only learned to use existing formularies.
METHODS: This was a multicentre, randomised, controlled study conducted in eight universities in India, Indonesia, the Netherlands, the Russian Federation, Slovakia, South Africa, Spain and Yemen. Five hundred and eighty-three medical students were randomised into three groups: the personal formulary group (PF; 94), the existing formulary group (EF; 98) and the control group (C; 191). The PF group was taught how to develop and use a personal formulary, whereas e the EF group was taught how to review and use an existing formulary. The C group received no additional training and participated only in the tests. Student's prescribing skills were measured by scoring their treatment plans for written patient cases.
RESULTS: The mean PF group score increased by 23% compared with 19% for the EF group (p < 0.05) and 6% for controls (p < 0.05). The positive effect of PF training was only significant in universities that had a mainly classic curriculum.
CONCLUSION: Training in development and use of a personal formulary was particularly effective in universities with a classic curriculum and with traditional pharmacology teaching. In universities with a general problem-based curriculum, pharmacotherapy teaching can be based on either existing or personal formularies.

RCT Entities:   Population Interventions Outcomes

Introduction

Many medical schools in developed as well as developing countries are changing their undergraduate education in pharmacotherapy from classic lecture-based learning to problem-based learning (PBL). They are moving away from the presentation of classic pharmacological knowledge on medicines towards promoting problem-solving and prescribing skills [1, 2]. The World Health Organisation (WHO) Guide to Good Prescribing and several training courses on problem-based pharmacotherapy teaching have contributed to this development [3, 4].

The WHO Guide to Good Prescribing gives medical students a normative model for therapeutic reasoning and prescribing and provides a six-step guide to the process of rational prescribing: (1) define the patient’s problem, (2) specify the therapeutic objective, (3) choose a (drug) treatment, (4) write the prescription and start the treatment, (5) give patient information and warnings, and (6) monitor the treatment. One of the key principles of the WHO approach is the division of step 3 [choose a (drug) treatment] into two steps. Step 3a, consider the suitability of a standard (p-drug) treatment for the disease in general, and step 3b, verify its suitability for the particular patient and alter the (drug) treatment if necessary. Consequently, undergraduate students are taught how to define their own p-drugs and are encouraged during their medical education and clinical activities to develop a personal formulary [5].

The WHO method emphasises that future doctors should master both steps of the drug-selection process. Selecting a p-drug trains students in pharmacological, clinical and epidemiological principles and in collecting and comparing evidence. By considering various treatment alternatives when developing their personal formulary, students are able to choose an alternative drug treatment more rationally when the drug of first choice is not suitable for a particular patient; for example, because of contraindications or side effects. In addition, by considering treatment alternatives, students are able to evaluate the information provided about new drugs during their professional career more critically.

Application of the WHO method has improved the prescribing skills of undergraduate medical students [4]. However, it is not known to what extent the p-drug concept and developing a personal formulary has contributed to this improvement. Some have argued that existing formularies and standard treatment guidelines, developed by experts, would lead to better prescribing than would a personal formulary developed by students. Furthermore, doctors are usually expected to work within the limitations of existing hospital formularies or practice guidelines. Despite the fact that teaching the p-drug concept and developing a personal formulary is a challenging and time-consuming task, we hypothesise that training in developing a personal formulary would lead to increased prescribing skills compared with using an existing formulary.

Therefore, the aim of this study was to determine whether there is a difference in prescribing skills between medical students who have been trained in creating and using a personal formulary and those who have been trained in using existing formularies only.

Materials and methods

Study design

A randomised controlled pre and posttest design with three arms was implemented (Fig. 1). All students participated in a pretest. After the pretest at each university, students were randomly divided into three equally sized groups: a personal formulary (PF) group, an existing formulary (EF) group and a control group. Students in the PF and EF groups received two different training programmes before taking the posttest. The control group received no additional training and was given only the pre- and posttests.

Fig. 1

Study design: PF personal formulary group, EF existing formulary group and C control group. Clinical subjects: A essential hypertension, B osteoarthritis, C acute bronchitis and D gastroenteritis. Complexity levels: 1 straightforward case of a middle-aged adult patient, 2 patient with a contraindication to a generally accepted drug of first choice, 3 severe case and 4 case without effect of previous treatment and side effects

After the posttest, a structured questionnaire with 21 questions was administered to identify differences in the type of teaching students received prior to the study. The questions were about two major teaching issues: curriculum type; and teaching methods in pharmacology, clinical pharmacology and pharmacotherapy. Other questions were designed to verify whether the universities had carried out the study in full accordance with the standard protocol. The study was approved by the educational research committee of the VU University Medical Center (VUMC).

Study population

Six hundred and nine undergraduate medical students from eight countries were selected on a voluntary basis (Vellore in India, Yogyakarta in Indonesia, Maastricht in the Netherlands, Kazan in the Russian Federation, Bratislava in Slovakia, Cape Town in South Africa, La Laguna in Spain and Aden/Sana’a (jointly) in Yemen). All students had passed their pharmacology examinations (basic pharmacology and clinical pharmacology) and were about to start their clinical internships. Participation in the study was in addition to their normal curriculum. All students were informed in advance about the general aim of the study and the method of testing. Cross-contamination of the students in the study groups and the control group was prevented by giving the students in the control group the opportunity to participate in a training programme after the study.

Materials

For the intervention and tests, 16 written patient cases were developed using four clinical subjects (A–D), each with a different level of complexity (1–4) (Fig. 1). All cases were formulated according to a standard design:

General patient information (e.g. age, gender, allergy, occupation and pregnancy)

Summary of previous and current diseases and treatments (comorbidity and comedication)

Extensive description of the present history and a physical examination

Diagnosis

Each case differed in disease severity and complicating factors such as age, comorbidity and comedication, drug allergy, pregnancy and breast feeding. All three study groups received a copy of the British National Formulary (BNF) as a drug reference [6]. In addition, the PF group received the WHO Guide to Good Prescribing, whereas the EF group received an adapted version of the guide without the chapters about the p-drug concept. The EF group also received the Oxford Formulary, containing treatment advice for symptoms and diseases that occur frequently in general practice, including the four clinical subjects used in this study [7].

Intervention

The training course for the PF and EF groups lasted for 5 weeks. The intervention for both groups consisted of an introductory session and four training sessions (Fig. 1)—one per week for about 2 h. The chosen teaching time was based on earlier experiences with teaching programmes using the PF concept [4, 8]. In the introductory session, the WHO step-wise approach to prescribing drugs, was explained. However, the p-drug concept was explained only to the PF group. In each training session, two written patient cases of one clinical subject were discussed. Students completed a treatment plan form, following the six steps of the WHO model. The students prepared for the sessions by studying the treatment of the clinical subject but in different ways. The PF group was asked to make an inventory of available treatments and select a p-drug from this group (on the basis of efficacy, safety, suitability and cost), whereas the EF group was asked to study the relevant treatment guidelines of the Oxford Formulary. One facilitator per university supervised the sessions of both PF and EF groups only by guiding the discussions and clarifying aspects of the case if necessary. Facilitators were specifically instructed not to provide solutions to the patient problems but only to advise students on how to solve these themselves. All facilitators had received specific training on this teaching methodology in a research meeting at the VUMC. The control group received no additional training between the pre- and posttest but proceeded only with their routine curriculum.

Test and scoring

The pre- and posttests consisted of four written patient cases. Each case contained a different clinical problem of a different level of complexity (Fig. 1). The combination between the clinical problem and level of complexity were randomly chosen. For each case, the students had to complete a treatment plan form. They were allowed to use any reference materials but not to consult one another. Directly after the posttest, the answer sheets for both pre- and posttests were blindly scored by the facilitator in a standardised manner using a 5-point scale (0 = no answer, 1 = poor answer, 2 = arguable answer, 3 = acceptable answer and 4 = good answer). Prior consensus about the scoring system had been reached between all facilitators during the research meeting at the VUMC. The scores were sent to the researchers at the VUMC for analysis. Individual student scores were not used by the participating universities for formal student assessments.

Statistical analysis

The results were analysed in SPSS 13.0 for Windows. The scores on the six steps of the WHO model were clustered into four prescribing skills: treatment choice, prescription writing, patient information and treatment monitoring. The differences between groups were analysed by analysis of variance (ANOVA) with the university as a factor, as well as separately. For pair-wise differences between the three groups, a multiple comparison procedure was applied. Significance was accepted at a two-tailed p < 0.05. Only the results of students who finished both pre- and posttest were included in the analysis.

Results

A total of 609 students participated. Five hundred and eighty-three students (96%) completed the pre- and posttests:194/203 (95.6%) students from the PF groups, 198/203 (97.5%) from the EF groups and 191/203 (94.1%) from the C groups. The three groups did not differ in their averages for previous examination results and pretest scores. The increases in the mean scores of the PF group (0.93, 23%) and the EF group (0.78, 19%) were both significantly higher than in the C group (0.23, 6%) (Table 1). In addition, the increase in the PF group was significantly higher than in the EF group (Table 1). Within this total, the differences were significant for three of the four prescribing skills (treatment choice, patient information and monitoring treatment but not prescription writing) and for three of the four complexity levels (straightforward case, cases with a contraindication to first-line drugs and severe cases but not cases with no effect of previous treatment and side effects present) (data not shown).

Table 1

Pre- and posttest scores per university

	Nujmber	Pretest			Posttest
		PF	EF	C	PF	EF	C
Yemen	73	0.41	0.35	0.33	2.98ab	1.56b	0.46
Russia	88	0.83	0.91	0.87	1.83ab	1.67b	1.03
Indonesia	75	1.12	1.20	1.34	2.45ab	1.98b	1.42
Netherlands	64	2.20	2.06	2.25	3.04b	3.06b	2.63c
Slovakia	72	2.27	2.13	2.12	3.20	3.12	2.90c
India	72	2.17	2.39	2.23	2.19	2.51b	2.08
Spain	73	2.38	2.33	2.28	2.61	2.76b	2.36
South Africa	66	2.54	2.48	2.38	3.02	2.97	2.81c
All Increase	583	1.71	1.62	1.69	2.64a,b 0.93 (23%)	2.40b 0.78 (19%)	1.92c 0.23 (6%)

PF personal formulary group, EF existing formulary group, C control group

aPF > EF (p < 0.05), bPF, EF > C (p < 0.05), cC > pretest (p < 0.05)

Results of medical students from the eight universities are listed separately (Table 1). In Yemen, the Russian Federation and Indonesia, pretest scores were lower than in the other five universities. In addition, the increase in their scores was significantly higher in the PF groups than in the EF groups. The differences among the PF and EF groups were also significant for all levels of complexity and all four prescribing skills (Fig. 2). In the remaining five schools, there was no significant difference between the PF and EF groups.

Fig. 2

Pre- and posttest scores for different prescribing skills of students from countries with a traditional curriculum (Yemen, the Russian Federation and Indonesia). PF personal formulary group (–), EF existing formulary group (.....), C control group (- -). Posttest scores are annotated, # = PF > EF

The results of the questionnaire about the type of teaching that students received indicated that Yemen, the Russian Federation and Indonesia had a predominantly classic curriculum. On average, the participating students from these universities had received 90 h (range 52–160 h) of pharmacology teaching by lectures, 23 h (range 3–50 h) of clinical pharmacology teaching also by lectures and no explicit pharmacotherapy teaching. These students were not familiar with PBL but were somewhat accustomed to discussion in small groups. The remaining five universities had a predominantly problem-based teaching curriculum. The participants had received an average of 43 h (range 6–100 h) of pharmacology teaching by lectures, including small group teaching; 24 h (range 6–40 h) of clinical pharmacology teaching, mainly by small group teaching; and no explicit pharmacotherapy teaching. The students from these universities were all familiar with PBL and were all accustomed to discussion in small groups. Some of these students already had some clinical experience (Slovakia and Spain). Except for the presence (or absence) of PBL, no other differences in teaching could be identified between any of the eight universities. No specific differences between universities in the way the study was performed or indications for possible bias were identified.

Discussion

The results of this study show that developing a personal formulary and using an existing formulary both increase the competence of medical students in rational prescribing. However, the overall differences between the PF group and the EF group (23% vs. 19% increase in mean score, p < 0.05) should be interpreted with caution. It is tempting to conclude that teaching students to develop a personal formulary adds to their prescribing skills. However, this effect was only visible in the universities in Yemen, the Russian Federation, and Indonesia. No significant differences between PF and EF scores were found in the universities in the Netherlands, Slovakia, Spain, India or South Africa. A possible reason could be the high pretest scores in the last five universities, which reduced the likelihood of a significant increase after a relatively short intervention period of 5 weeks. A similar effect was seen in the first multicountry study where universities with a problem-based curriculum showed less benefit from a short intervention course than those without a problem-based curriculum [4].

An important question is why these five universities had such high initial scores? The questionnaire was specifically designed for the purpose of this study, and the only explanation that emerged was that students from these five universities had received problem-based teaching and were therefore familiar with the process of solving patient problems. In the universities with lower pretest scores, classic teaching was paramount and students had less experience with solving patient problems.

This finding is in accordance with the fact that successful clinical problem solvers possess comprehensive knowledge, but the way they organise and understand their knowledge is even more critical. Modern teaching is aimed at creating both a knowledge structure and the necessary “search-and-retrieve” strategy for responding to complex diagnostic problems [9, 10]. Students who were already trained in this way via problem-based curricula seemed better able to transfer this ability to therapeutic problem solving and did not show an additional increase in such skills when they participated in our study. However, students from universities with a classic curriculum quickly assimilated the p-drug concept, and their results came very close to those of the other five universities, even after a short intervention of only 5 weeks.

The limitations of our study need to be addressed. The voluntary participation of students may have created a positive bias for the total group, but allocation to the intervention groups was randomised and therefore the comparison was unbiased. In addition, to ensure baseline comparability among the groups, previous results on university examinations were averaged for the three groups. The scoring by the medical schools was blinded, with the pre- and posttest forms of all three groups being mixed during the scoring. Cross-contamination between the study and control groups was prevented by explaining to the students the importance of not exchanging any information and by reassuring the control group that it would receive the training after finishing the study.

Therefore, taking the above-mentioned limitations and considerations into account, it can be concluded that training in developing and using a personal formulary or in the use of an existing formulary may increase the rational prescribing of medical students. However, with regard to universities with a general problem-based curriculum, training in the development of a personal formulary is probably not worth the additional effort. In these circumstances, pharmacotherapy teaching can be based on existing formularies instead of a personal formulary.

In this international study, only the short-term effects on rational prescribing of developing a personal formulary or using an existing formulary have been studied. Despite the fact that the long-term effects are unknown, it can be hypothesised that after developing a personal formulary, students may judge new drugs more effectively and be more capable of deciding whether or not to include these drugs into their (personal) formulary. Furthermore, the results may diminish the concerns ventilated by some clinical pharmacology teachers regarding the changes in the style of undergraduate medical curricula from classic to PBL teaching [11, 12].

In practice, this implies that the decision to organise a teaching programme for medical students on how to develop and use a personal formulary needs to be based on an initial assessment of both the students and the curriculum. The same may be true for postgraduate and continuing education of doctors who have been trained in the classic manner [13, 14]. A screening test, such as the pretest in this study, could perhaps be used to identify universities, students or prescribers that would benefit most from this type of training.