A pilot study: research poster presentations as an educational tool for undergraduate epidemiology students.

Students in a fourth year epidemiology course were surveyed after participating in a formal Science Research Day in which they presented original research, in poster form, to be judged by scientists from the community. Of 276 participating students, 80 (29%) responded to the study survey. As a result, 19% of respondents were more likely to pursue a career in science, and 27.5% were more likely to pursue a career in epidemiology. Only one respondent reported being less likely to pursue a science career, while seven were less likely to pursue epidemiology. A majority of respondents felt that the poster experience was on par with, or superior to, a comparable research paper, in terms of both educational appeal and enjoyment. Mandatory, formal poster presentations are an innovative format for teaching advanced health sciences, and may more accurately reflect the realities of a science career than do more traditional educational formats.

Introduction

As observed by White,1 “content laden lectures delivered to large enrollment classes typify science courses at most universities and many colleges.” And, as was recently noted by the National Academies of Science, there is a crisis in the teaching of creativity and innovative thinking in the Health Sciences.2 These two observations point to a failure of university-level science teaching to satisfy the needs both of students and society. Given the variety of student learning modalities, reliance upon traditional lecture formats may disadvantage those who require more interactive pedagogical approaches.

In population health sciences, content is ever-changing in response to new information. Thus, more dynamism is required for the full potential of education to be realized, perhaps based more on social constructivism, which seeks to allow students to assist each other in the learning experience, through shared context and culture. Studies indicate that students learn best when linkages between teaching and research are expressed in a practical manner.3,4 The extent to which traditional education tools – lectures, essays, and written exams – truly expose students to the realities of a science career is questionable; real science is collaborative, social, and sometimes confrontational, but rarely anonymous. Therefore, traditional lectures may do society a disservice by failing to encourage creative and socially-minded students into the professional scientific ranks. This is especially true in epidemiology, which is growing in societal profile and importance, and which needs a diversity of personalities, perspectives, and priorities in order to flourish.

The “research–teaching nexus” has been described by several authors5–8 as a critical space in which to create synergies between traditional teaching and professional scholarship. Investment in the nexus has been proposed as a cost-effective way to increase participation in higher education.6 Four models of student engagement described by Griffiths6 are that teaching can, and should be, either research-led, research-oriented, research-based, or research-informed. With this study, we sought to embrace two of those models: research-led and research-oriented education.

Our study describes an endeavor to implement a creative undergraduate-level teaching and evaluation format for epidemiology students, so that they are more exposed to the realities of professional scientific life, are better able to make informed decisions about their career paths, and have the opportunity to apply and enhance typically untapped skills (oral presentation, dialogue, and visual presentation) to their advanced science education.

Our format was a simulated research symposium, judged by working scientists, in which students must orally defend their original research, presented in the form of a science poster. Our approach was informed by literature findings that student perceptions of professional research are often defined by early exposure to actual research.5 Thus, our overall learning objective was for students to acquire and demonstrate an understanding of modern practices in the synthesis of existing scientific knowledge and the communication of scientific findings to a peer audience.

Methods

The bilingual undergraduate program at the Interdisciplinary School of Health Sciences at the University of Ottawa features a mandatory Introduction to Epidemiology class in fourth year. All students were pursuing an undergraduate degree in Health Sciences, which embraces a variety of health-related disciplines into a single 4-year program of study. The class has three sections taught by three different professors: two in English and one in French. Each section varies in size from 40 to 120 students, for a total of 276 students. The final assignment, worth 25% of the final mark, involves the preparation of a research poster, based upon original research, and conducted in self-selected groups of 2 to 4 students. Topics were selected by the students and were based upon literature reviews that followed a systematic search protocol. Students were required to identify a research question, based upon their reading of current peer-reviewed epidemiological literature, devise a literature search strategy, evaluate the quality of the identified publications, provide a qualitative/quantitative synthesis of research findings, and discuss their results and conclusions. Importantly, students were left to decide for themselves upon the work distribution within each team.

The precise wording of the assignment was as follows: Purpose: The purpose of this assignment is for the students to identify and frame a research question, conduct a literature search, collect and collate literature and synthesize the results. The outcome of this exercise is to be presented as a poster on the research presentation day. Projects must focus on one or more branches of epidemiology.

Format: On the April research day, students will, in teams, present research posters on their selected epidemiological topics. Pairs of judges (professors, scientists and graduate students) will peruse the posters. Each student team will orally present the content of their posters, then answer questions from each judge. The final mark will be based on a combination of judges’ impressions and on the professor’s separate evaluation of the posters’ content.

Learning objectives: It is expected that the students through this exercise will: learn how to identify and frame a research question; search and identify appropriate scientific literature that is relevant to the research question; read, understand and synthesize scientific information; compile and collate information and present it as a poster of a caliber suitable for presentation at an international research conference; present the poster to an invited audience and respond to questions and criticisms.

The posters were presented at a research event organized by the local chapter of the Canadian Society for Epidemiology and Biostatistics. Each poster was judged by two volunteer judges, via an oral examination of each group of students. Students were given a total of 10 minutes per team to orally present their projects, with an additional 2–5 minutes granted for judges’ questions. The judges included scientists from industry, government, or academia. All judges used the same evaluation criteria assessing the research title, abstract, background, methods, results, discussion, recommendations, and conclusions, and the overall quality of the presentation, including the students’ ability to answer questions. The marking template used by judges is provided in Table 1. Final marks were based upon a combination of professors’ visual assessment of the poster and judges’ rankings.

Table 1

Poster evaluation criteria employed by judges

Title (2 points)

• Appropriate and fitting the study presented

Abstract (3 points)

• Concise and provides a good summary of the study

Background (5 points)

• Rationale, purpose, and objectives clearly outlined

• Research question or hypothesis defined and articulated

• For empirical studies, review of existing literature on the topic (existing knowledge)

Methods (10 points)

• Research plan described

• Data collection plan identified and described appropriately

• Methods and approaches are identified and described appropriately

• Data abstraction plan outlined

Results (10 points)

• Results follow from the methods

• Results are clearly presented

• Use of figures and tables where appropriate

Discussion (5 points)

• Interpretation of results is provided

• Limitations of study are identified

• Direction for future research identified

Recommendations and conclusion (5 points)

• Recommendations and conclusion are appropriate and properly described

Quality of poster presentation (10 points)

• Timeliness (under 10 minutes total)

• Answered questions appropriately

• Engaged the audience and presented in a confident and professional manner

• Quality of communication using visual aids

• Organization and logical flow

• Overall attractiveness of poster

Total: /50

The event was meant to resemble as closely as possible a professional science symposium. Sponsors of the event included a science journal, graduate programs, and one professional body. Students were instructed to dress “professionally” and prizes were awarded for the presentations judged the best.

The total budget for the research day was $2,200, put together from small donations from a variety of sources, including relevant university departments, an alumni association, professional associations, and professorial research funds. Expenses were incurred through catering ($1,200), audiovisual equipment ($200), student prizes ($600), and parking reimbursements ($60), with the remaining funds used for miscellaneous expenses and office supplies, such as name tags and “thank you” cards.

In 2010, student participants were sent a link to an online questionnaire hosted by www.surveymonkey.com. The survey consisted of 31 multiple choice questions, which could be completed in less than 10 minutes. While the respondents were either Anglophone or Francophone, due to resource limitations, the questionnaire was only available in English. (All of the Francophone students are known to be either fluent or highly functional in English). Four minutes in length, the survey used closed-ended questions to measure students’ perceptions on their knowledge of the subject area before and after the event, and the usefulness of the assignment for learning epidemiology.

The survey also evaluated perceptions of whether specific aspects of the event had an impact on students’ ability to appreciate the experience, their comfort-level in executing aspects of the assignment, the degree to which specific skills were enhanced, and their preferences in terms of the format of the assignment. For most questions, a three-level scale was used to measure student attitudes: for example, when determining the usefulness of an experience in achieving a learning objective, respondents chose between “not useful,” “moderately useful,” and “very useful.”

Additional questions measured how much money was spent on the posters, and the impact of the event on students’ interest in a career in science. Data analysis was performed using SPSS 12.0 statistical software (IBM Corporation, Armonk, NY, US).

Ethics approval for this study was granted by the University of Ottawa’s research ethics office.

Results

The topics presented by students were quite diverse, and included various types of cancer, human immunodeficiency virus, tuberculosis, drug adherence and compliance issues, psychiatric comorbidities, the effects of micronutrient supplementation, attention deficit hyperactivity disorder, obesity, post-disaster medical response, economic impacts of various interventions, and infertility.

Of 276 participating students, 87 (31.5%) responded to the survey and 80 (29%) provided complete answers. A majority (74%) were female. According to chi-square and Fisher’s exact tests, there were no statistically significant differences between the French-speaking and English-speaking groups, nor between males and females, with respect to responses to any of the survey questions.

All results are summarized in Table 2. Highlights include that over 90% of respondents found that presenting a poster was either useful or very useful in learning the fundamentals of epidemiology, while 96% felt that the overall research day experience was useful in learning epidemiology. Twenty-two students (27.5%) were more likely to consider a career in epidemiology as a result of the experience, though this statistic tells us nothing of how many were already predisposed to such a career path. Compared to writing a major research paper, 48% thought the poster required more hours of work, but 67% thought it was more useful overall for their careers. Indeed, 54% of respondents felt that they learned more from the poster presentation experience than they would have from writing a paper.

Table 2

Selected results of survey of students participating in the epidemiology research day poster presentations

Response	Percentage (n)
• Demographics and cost
Francophone (versus Anglophone)	17% (15)
Female	74% (59)
Had previously participated in a poster presentation	11% (9)
Money spent on poster: $0	2.5% (2)
<$20	19% (15)
$20–$60	46% (37)
>$60	32.5% (26)
• Educational value
Prior to poster preparation, knowledge of content was “good” or “very good”	14% (11)
Post poster preparation, knowledge of content was “good” or “very good”	96% (77)
Poster preparation was moderately or very useful in learning fundamentals of epidemiology	96% (77)
Orally presenting the poster was moderately or very useful in learning fundamentals of epidemiology	92.5% (74)
Overall research day experience was useful for learning epidemiology	96% (77)
More likely now to consider a career in science	19% (15)
Less likely now to consider a career in science	2.5% (2)
More likely now to consider a career in epidemiology	27.5% (22)
Less likely now to consider a career in epidemiology	9% (7)
How many of other students’ posters did you examine: 0	1% (1)
1–10	69% (55)
>10	30% (24)
• Comparisons to other formats
Compared to writing a paper, the poster required more hours of work	48% (38)
Compared to writing a paper, the poster required fewer hours of work	10% (8)
Compared to writing a paper, the poster required more research rigor	21% (17)
Compared to writing a paper, the poster required less research rigor	5% (4)
Compared to writing a paper, the poster was more useful for my career	67% (53)
Compared to writing a paper, the poster was less useful for my career	4% (3)
Compared to writing a paper, I learned more from doing the poster	54% (43)
Compared to writing a paper, I learned less from doing the poster	5% (4)
Would prefer a poster presentation without the oral component	12.5% (10)
Would prefer an oral presentation without the poster component	2.5% (2)
Prefer the poster presentation with oral component (ie, “don’t change a thing”)	79% (63)
• Skills learned
The experience enhanced reading skills moderately or a great deal	67.5% (53)
The experience enhanced writing skills moderately or a great deal	69% (54)
The experience enhanced collaboration skills moderately or a great deal	86% (68)
The experience enhanced presentation skills moderately or a great deal	92.5% (74)
The experience enhanced scientific research skills moderately or a great deal	96% (77)
The experience enhanced knowledge synthesis and dissemination skills moderately or a great deal	99% (79)

In terms of skills gleaned, a slim majority thought that the poster experience enhanced their reading (67.5%) and writing (69%) skills, while a significant majority thought that their collaboration (86%), scientific research (96%), and presentation (92.5%) skills were either moderately or greatly enhanced by the assignment.

Students reported paying from $0 to $100 for the preparation and printing of their posters, with a plurality (46%) having spent $20–$60.

Discussion

Our 29% response rate does not render our survey statistically representative of the student body. It is possible that those most moved by the poster experience were also the most motivated to complete the survey, thus biasing our results. The low response rate is possibly due to the survey being sent out after completion of the class, when many students had already begun their summer plans, and when most had already graduated from the program. It is unclear how this would bias our results. However, given that comparable studies are based upon far fewer cases,9 much wisdom can still be extracted from these results.

Most inspiring is the finding that 19% of students are more likely to pursue science as a career, and 27.5% are more likely to pursue epidemiology as a career, as a result of the poster experience. The measurable benefits to the poster process, then, are a better informed career path, increased knowledge of both the subject of the poster and the general science of epidemiology, and an improvement in the key skills of reading, writing, collaboration, and presentation.

While our students represent a range of career trajectories, a fair number are nevertheless expected to pursue medicine as a career. For this reason, it may be appropriate to consider this exercise as relevant to the core competencies required of physicians, as described by the Accreditation Council for Graduate Medical Education (ACGME), specifically for interpersonal skills and communication.10 Several descriptors of that competency are recapitulated in our exercise, including working effectively in a group and communicating effectively with other professionals.10

The qualitative costs were few. The poster experience was not at the expense of a comparable research paper experience, since most students reported learning as much, if not more, from the former. Unlike a research paper, the poster experience was social and interactive, required real-time assessment of students’ content in response to judges’ probing, and also afforded students the opportunity to further self-educate by exposing them to their colleagues’ posters, something all but one respondent reported doing.

The judging template was an effective measurement of poster quality, inasmuch as it reflected the essence of the assignment. In addition, it provided for a degree of consistency across judges, though scores for all students were overwhelmingly above 40/50, suggesting a need for a more sensitive grading scale to allow for a wider range of marks. In the future, as well, a measure of inter-rater reliability would be useful to determine if there are significant differences between judges’ assessments.

The financial costs were substantial for an undergraduate classroom, which is not atypical.9 Seeking sponsors was essential for event logistics, including costs related to booking the venue, rentals of poster boards, audio-visual, catering, and student awards. Judging and staffing time was donated, though the former received gifts and parking reimbursement. For this format to be sustainable, continued investment by the university is pivotal. While fundraising was challenging, we believe that our approach is more cost-effective than a similarly intense and comprehensive paper-writing exercise would be, since the latter would ideally include significant and time-intensive coaching, workshop, and grading expenses. This does not include costs borne by the students for constructing and printing their posters.

Conclusion

A poster presentation for students is not unique. Our innovations were: its incorporation into the mandatory requirements of a class; the insistence upon the formality expected at a professional science symposium; and the role of external judges, both in adding to the atmosphere of formality, and in contributing to the marking of the presentations. Our results suggest that the combined experience was valuable and interesting for students, educational in terms of both the research content and epidemiology as a whole, and instructive in guiding participants with respect to their career choices. Thus, we would recommend the implementation of a similar approach in other research-based courses, if costs can be contained, and if the evaluation instrument applied can be shown to provide for consistent judging.