Impact assessment of an educational course on vaccinations in a population of medical students.

BACKGROUND: The inadequate knowledge about vaccinations of healthcare workers, including medical doctors, has certainly contributed to the spread of the vaccine hesitancy. Therefore, it is essential to improve the level of knowledge of future doctors. The aim of the study is to evaluate the impact of a course about vaccinations on the knowledge of medical students.
METHODS: Medical students were asked to complete an anonymous questionnaire before and after a seminar on vaccination that they willingly attended. The two questionnaires contained the same 10 questions about vaccines. Only the students who had attended the lecture were allowed to fulfil the post-lecture questionnaires through the learning management system (LMS) called "Moodle". A descriptive statistical analysis of the data collected through the comparative evaluation of the answers before and after the seminar was performed. Mann-Whitney test for two independent samples was used to compare medians score before and after the interventions.
RESULTS: A total of 100 medical students filled the pre-lecture questionnaire and 81 of them completed the post-lecture questionnaire. Knowledge of the students on the indication of the MMR (Measles-Mumps-Rubella) vaccine strongly improved after the seminar. Moreover, the number of students who would recommend vaccination for pertussis and influenza during pregnancy increased significantly by 37% and 19% respectively after the seminar and those aware of the need for Herpes Zoster vaccination over the age of 65 increased by 22%. DISCUSSION: For future doctors, a thorough knowledge about vaccinations is increasingly required in order to deal with vaccine hesitancy. Extracurricular seminars about vaccines, provided in the second half of the course of study, can have a highly positive impact. ©2019 Pacini Editore SRL, Pisa, Italy.

Introduction

Vaccinations have been very successful in the prevention of infectious diseases but – even though they save millions of human lives every year – nowadays the phenomenon of “vaccine hesitancy” is growing among a relevant part of the world population. “Vaccine hesitancy” comprises a set of negative feelings towards vaccinations, such as insecurity, doubts, and distrust; these attitudes represent a considerable danger to public health [1-3].

In order to deal with this growing phenomenon, which is the main cause of the diminution of vaccination coverage, it is important to detect the reasons for the spread of such a considerable negative perception of vaccinations all over the industrialized countries. The inadequate education and knowledge about the vaccination of healthcare workers, including doctors, has certainly contributed to the growth of the anti-vaccination movements that frequently attack the safety of vaccines by making use of non-evidence-based expositions [4].

It is, therefore, essential to adequately train university students of the Degree Courses in Medicine and Surgery with extracurricular seminars aimed at increasing the knowledge of all aspects concerning vaccination [5]. During these specific lectures, it is also important to give instructions to the students about how to rightly respond to the attacks against the scientific world and how to properly communicate with the population with regard to vaccines [6].

It has been demonstrated that multidisciplinary formative interventions comprise the most powerful strategy to improve knowledge about vaccines of Italian healthcare workers, and this is the key tool to increase the confidence of population regarding vaccination [7].

It is, however, also important to evaluate the effectiveness of these educational interventions. The aim of the current study is to evaluate the impact of the ‘Vaccines and Vaccinations’ chosen course on the knowledge of students enrolled in the IV, V, and VI year of the Degree Courses in Medicine and Surgery at the University of Florence in the academic year 2017-2018.

Methods

During the spring semester of 2018, a total of 100 students from the IV, V, and VI year of the Degree Courses in Medicine and Surgery at the University of Florence decided to voluntary attend an extracurricular lecture about vaccinations.

The educative intervention was held by university personnel belonging to the Section of Hygiene, Preventive Medicine and Public Health of the Health Sciences Department. It was divided into two classes and spread over two consecutive days. Each class lasted for four hours. The topics addressed were:

Italian National Vaccination Plan 2017-2019 (NVP 2017-19);

vaccine coverage trends;

development of vaccines;

epidemiological bases for vaccination strategies;

duration of immunity;

epidemiological impact of vaccines;

real and perceived safety of vaccinations;

risk communication methods in vaccine prevention.

In order to evaluate the impact of the intervention on the knowledge of the students, they were asked to fill in two anonymous questionnaires, the first one before the class and the second one after that. The students could fulfil the questionnaires through the learning management system (LMS) called “Moodle”. Moodle is an open-source platform, where it is possible to deposit and consult educational material, to process questionnaires and tasks, to support exercises, to follow lessons in video-streaming, and to use collaborative work tools. Moodle is useful to organize and manage courses online. For this project, the teaching methodology of the Blended Learning was chosen, which involved alternation of frontal lessons and distance activities.

The students completed the pre-lecture questionnaire before attending the class. Only those who attended the class were allowed to complete the post-lecture questionnaire.

The pre-lecture questionnaire contained 13 questions (Supplementary file 1). We analysed 10 questions for this article: the first question asked students if they recalled which vaccination they had received; the second asked if they had had any kind of negative experience after vaccines; the third asked them about the source of information about vaccination they used. Six multiple choice questions evaluated the knowledge of the students about the Italian immunization schedule for each age group (first, second and sixth year of life, adolescence, elderly) and for pregnant women. Students were asked to indicate which vaccinations were mandatory or strongly recommended according to the Italian NVPP 2017-2019; the seventh question asked the students to self-assess their level of knowledge on vaccination.

The last seven questions were also asked in the post-lecture questionnaire.

Since the survey data do not compromise students’ privacy and the issue under investigation is a public matter, ethical approval for the study was not required.

STATISTICAL ANALYSIS

The percentage of students who claimed to recall which vaccinations they had received in their lifetime and the percentage of students who have had direct or indirect personal experiences of vaccination side effects were calculated.

Sources of information about vaccination used by medical students were also analysed: university, books, family doctor, mass media, institutional web, non-institutional web, and word of mouth.

In order to assess the knowledge of students in terms of immunization schedule, the percentage of correct answers for each disease and for each population target were calculated (first, second and sixth year of life, adolescence, elderly and for pregnant women). In particular, a value of 1, when students correctly answered questions, a value of 0, if students did not answer the question, and a value of – 0.25, if students answered the question incorrectly, were assigned. A single score for each age category and a total score for all categories were generated. Finally, all the scores were rescaled out of 100 to make them comparable with previous literature, which mostly uses percentages of correct answers, while using the raw percentage of correct answers was avoided to discount guessing.

In order to compare the pre-intervention and post-intervention self-reported level of knowledge about vaccines, a score of 0 to “poor”, 1 to “insufficient”, 2 to “sufficient”, 3 to “good”, and 4 to “excellent” was assigned.

To verify the efficacy of the intervention, since the scores were not normally distributed, medians score before and after the interventions were compared using the Wilcoxon rank-sum (Mann-Whitney) test for two independent samples.

Results

Of the students who asked to participate in the course, 100 completed the pre-lecture questionnaire and 81 attended the class. Only those who attended the class were allowed to complete the post-lecture questionnaire; therefore, 81 students filled in the post-lecture questionnaire.

Overall, 7% of the students did not remember which vaccinations they had received in their lifetime; all the rest specified the vaccines they had undergone. Furthermore, 12% of the students declared that they have had direct or indirect personal experiences of side effects after a vaccination.

According to the answers of the medical students, the most common sources of information about vaccination were the University training/academic courses (85%), books (49%), family doctor/general practitioner (41%), mass media (39%), and institutional websites (30%). Other sources of information used by the medical students were school (25%), word of mouth (24%), non-institutional websites (17%), and other medical doctors (paediatrician, gynaecologist, etc.) (12%).

Knowledge regarding the principles and recommendation of the Italian National Plan of Vaccine Prevention 2017-19, before and after the lecture, are reported in Table I. Students who would recommend vaccination for pertussis and influenza for pregnant women increased after the seminar by 37% and 19%, respectively. The number of those aware of the recommendation for Herpes Zoster vaccination over the age of 65 increased by 22%. Future doctors who would recommend vaccination against measles, rubella, and mumps for children under one year of age decreased by 67% after the intervention.

Tab. I.

Knowledge on mandatory and recommended vaccination for every age group and for pregnant women according to the Italian National Vaccination Plan 2017-2019 statements, before and after the educational intervention conducted.

Vaccination	1st year		2nd year		6th year		Adolescence		Elderly		Pregnancy
	Pre n(%)	Post n(%)	Pre n(%)	Post n(%)	Pre n(%)	Post n(%)	Pre n(%)	Post n(%)	Pre n(%)	Post n(%)	Pre n(%)	Post n(%)
Diphtheria	86(86)	77(95)	84(84)	63(78)	50(50)	60(74)	46(46)	54(66)	19(19)	37(45)	15(15)	55(67)
Tetanus	98(98)	75(92)	83(83)	66(82)	59(59)	61(75)	56(56)	54(66)	23(23)	40(49)	36(36)	59(72)
Pertussis	83(83)	76(93)	79(79)	64(80)	52(52)	63(77)	44(44)	54(66)	15(15)	38(46)	40(40)	63(77)
Poliomyelitis	78(78)	73(90)	86(86)	68(84)	46(46)	61(75)	39(39)	53(65)	92(92)	64(80)	59(59)	74(91)
H. influenzae B	28(28)	66(81)	82(82)	68(84)	86(86)	69(86)	95(95)	77(95)	71(71)	69(86)	93(93)	72(89)
Hepatitis B	94(94)	71(87)	80(80)	67(83)	80(80)	70(87)	92(92)	73(90)	90(90)	79(98)	88(88)	73(97)
Hepatitis A	81(81)	80(99)	90(90)	74(92)	89(89)	77(95)	92(92)	75(93)	91(91)	77(95)	81(81)	80(99)
Measles	16(16)	71(88)	67(67)	71(87)	53(53)	63(77)	89(89)	74(92)	95(95)	79(98)	82(82)	67(83)
Rubella	17(17)	72(89)	69(69)	72(88)	51(51)	63(77)	86(86)	73(90)	95(95)	79(98)	64(64)	59(73)
Mumps	29(29)	71(88)	58(58)	66(81)	42(42)	63(77)	88(88)	76(94)	96(96)	78(96)	84(84)	66(82)
Varicella	63(63)	69(87)	60(60)	69(85)	52(52)	61(75)	92(92)	75(93)	96(96)	77(95)	80(80)	70(86)
Meningococcal B vaccination	53(53)	61(75)	31(31)	40(49)	78(78)	73(91)	61(61)	61(76)	87(87)	78(96)	91(91)	79(98)
Meningococcal C vaccination	19(19)	78(97)	49(49)	52(64)	78(78)	73(91)	62(62)	59(72)	88(88)	75(93)	91(91)	79(98)
Pneumococcal vaccination	10(10)	57(70)	91(91)	67(83)	90(90)	72(89)	88(88)	75(93)	63(63)	61(75)	92(92)	71(88)
HPV	77(77)	80(99)	98(98)	81(100)	98(98)	80(99)	76(76)	74(91)	99(99)	81(100)	80(80)	75(93)
Influenza	86(86)	75(93)	91(91)	76(94)	92(92)	77(95)	96(96)	76(94)	86(86)	70(86)	35(35)	44(54)
Tuberculosis	91(91)	80(99)	96(96)	80(99)	100(100)	79(98)	97(97)	80(99)	98(98)	78(97)	98(98)	80(99)
Rotavirus	2(2)	56(69)	94(94)	76(94)	97(97)	77(95)	99(99)	81(100)	95(95)	70(97)	95(95)	79(98)
Herpes Zoster	99(99)	80(99)	97(97)	81(100)	98(98)	81(100)	94(94)	77(95)	44(44)	54(66)	87(87)	79(96)

The self-assessment of the level of knowledge on vaccinations changed significantly with educational intervention, as shown by the reduction (from 46% to 5%) of the answers, indicating a low level (poor/insufficient), and the increase (from 54% to 95%) of the answers, indicating a high level (sufficient-good-excellent).

Median scores obtained by students before the interventions were between 73.6/100 for the vaccines recommended for adolescents to 77.6/100 for pregnant women (Tab. II). After the intervention, median scores improved for all age categories. The highest score obtained was in the adolescent vaccination group (93.7/100), whereas the lowest score reached was in the elderly vaccination group (84.2/100).

Tab. II.

Median of calculated scores (before and after the intervention) and p-value of the Wilcoxon rank-sum (Mann-Whitney) test for two independent samples.

		Median		P*
		Before(n = 100)	After(n = 81)
Schedule knowledge	Childhood	74.6/100	92.1/100	< 0.001
	Adolescence	73.6/100	93.7/100	< 0.001
	Pregnancy	77.6/100	88.2/100	< 0.001
	Elderly	73.7/100	84.2/100	< 0.001
	Total score	74.2/100	88.8/100	< 0.001
Self-reported knowledge		2/4	3/4	< 0.001

*: two-sample Wilcoxon rank-sum (Mann-Whitney) test.

Differences between scores before and after the interventions were all statistically significant (p < 0.001).

Discussion

In recent years, many Italian Universities modified the traditional educational system based on standard courses, developing it with the inclusion of extracurricular interventions. At the faculty of Medicine and Surgery of the University of Florence, every student, during the six years of curriculum, must attend some extracurricular courses; they can freely choose from many different seminars offered to them, concerning any scientific topics.

Thanks to this educational pattern, the students, apart from the regular courses and exams, can also deepen their knowledge in some subjects according to their own personal interests. The aim of our study is to evaluate the efficacy of an extracurricular seminar on vaccinations and its impact on the knowledge and the attitudes of medical students, through pre-post questionnaires.

The answers to the questions concerning the attitudes and the perception of the students toward vaccinations suggest that people who have a scientific background are not hostile toward vaccination. In fact, 100% of our sample, before the intervention, were already in favour of vaccinations, and 94% of them declared that they agree with the introduction of compulsory vaccines for the school attendance.

The same concept is demonstrated by the results regarding the sources of information on vaccines used by the students: the most common ones are those that have a scientific framework. In other studies, we can observe that the general population is mostly used to get information from other sources, like word of mouth or the web, and that the general population is more likely to agree with the anti-vaccination statements compared to medical students [8, 9]. This shows that our sample might be different from the general population since the intervention was tailored to medical students, who most frequently have a good scientific background.

Before the lecture, students were not well-prepared about the schedules and organization of vaccination service in the Italian territory, according to the innovation of the National Vaccination Plan 2017-2019. In particular, our study demonstrates a gap in students’ knowledge on the recommended vaccines for the first year of age: only 2% and 10% of students were aware about the recommendation for the vaccination respectively for Rotavirus and S. Pneumoniae before our intervention, and only 28% of students knew, before the lecture, that the protection against Haemophilus influenzae type B is contained in the hexavalent vaccine. Was also observed that 80% of students, before the intervention, would have recommended the vaccination against Measles-Mumps-Rubella (MMR) for children under one year of age, although it is actually not indicated before the 13th month of life.

Scores obtained from the answers about the recommendations of Human papillomavirus (HPV) vaccine are quite high and satisfying, but it would be interesting to distinguish the answers given by male students from those given by female students. In fact, according to other surveys, we should expect higher scores from female students [10].

Moreover, the students demonstrated that they are not adequately informed about some specific aspects of vaccinations, like maternal-foetal immunization; the proportion of students aware of the recommended vaccines during pregnancy sharply increased after the intervention. The diphtheria-tetanus-pertussis and influenza vaccines are extremely important for not only the new-born but also the mother herself [11].

In addition, regarding the vaccination against influenza, students proved that they adequately know the other indications for this vaccination, in contrast to the vaccine coverage for influenza among Italian Medical Residents (MRs) that was shown by previous studies. Indeed, in the past years, many studies have been conducted on MRs in order to assess the vaccine coverages of this specific group of population, particularly for influenza and hepatitis B, and also to identify the determinants of the vaccine uptake for healthcare workers (HCWs). Looking at the overall vaccination rate observed in a previous study, Italian MRs seem to have a very low compliance with influenza vaccination: acceptance of seasonal and pandemic A (H1N1) influenza vaccination ranges from 12% to 20% [12-14].

Our project shows that a formative intervention is a powerful strategy to improve the knowledge about vaccines among future Italian healthcare workers. Improving their awareness on these topics is the key tool to increase the confidence of the population regarding vaccination, so it is also necessary to understand when such interventions can have a stronger impact on the sensibility of the students. Previous studies indicate that the intention to get vaccinated is greater during the clinical phase of the university career, suggesting that this is a good time to introduce promotion strategies to strengthen this attitude [15].

Wicker et al. found differences between pre-clinical and clinical students regarding the uptake of influenza and hepatitis B vaccines, the chances of being occupationally infected with influenza or hepatitis B, and the likelihood of suffering from severe side-effects following immunization. Medical students of the clinical semesters were more likely to have been vaccinated against influenza (58.1% vs 15.3%, p < 0.001) and HBV (96.6% vs 78.3%, p < 0.001) than students of the pre-clinical semesters [16].

Previous studies indicated that one of the most important determinants of getting vaccinated against flu is to have been previously vaccinated or having participated to vaccination campaigns [17]. Thus, the achievement of considerable awareness and knowledge about vaccination among medical students would contribute to the accomplishment of high vaccine coverage among healthcare workers in the future [18]. Moreover, future healthcare providers should be equipped with not just the knowledge but also the skills to counsel patients regarding the importance of vaccination [18, 19].

Previous surveys show that MRs recognize the importance of the lack of a vaccine prevention culture and acknowledge that there is a need for more information and awareness on the topic. In a national survey, nearly one-third of the students in their last year of medical school in France felt inadequately prepared to deal with vaccination-related questions. These data are in tune with the data pertaining to the self-assessment of the level of knowledge given by our sample before the lecture [20, 21].

Apart from the data obtained from the responses on the vaccination calendar, the positive impact of our intervention is also demonstrated by the answers given by the students on the assessment of their level of knowledge on vaccination. After the lecture, 95% of the students considered their level of knowledge as sufficient, good, or excellent. This is key tool to increase the mastery of and the accuracy on these topics, which will be useful instruments to counter the anti-vaxxers attacks.

Our survey has some limitations. Firstly, it was not possible to match the pre-intervention answers to the post-intervention ones for each student because the questionnaires were anonymous. Therefore, the increasing percentage of correct answers after the class could be partly justified because the less prepared and less interested students had left before the intervention. On the other hand, the anonymity favoured the achievement of more frank answers, since nominal questionnaires would have been perceived as an examination by the students.

Secondly, the survey has been conducted during an optional and non-curricular intervention, which is why our sample size is relatively small compared to the numbers of medical students in the University of Florence.

Another limit to consider is that, since fewer vaccines are recommended for pregnant women, students who did not answer were more likely to get higher scores, since a missing answer was not distinguishable from a correct answer.

Conclusions

For future doctors, a thorough knowledge of vaccination will be increasingly required to deal with vaccine hesitancy. An extracurricular seminar about vaccines, provided in the second half of the course of study, can have a highly positive impact on integrating knowledge and attitudes regarding vaccinations. Medical students are one of the most important target for educational campaigns as they are still in their training period and are open to changing their habits [19]. An appropriate development of technical and cultural skills for next generation of medical doctors is of paramount importance to spread positive vaccination attitudes, also among general population. The best initiatives currently devoted to vaccinology education should join forces and the University should developed a structured platform for future training of scientists in vaccinology, especially in academic courses with a forthcoming healthcare sector employment [22].