Course-based research experience of undergraduate medical students through project-based learning.

PURPOSE: This study aimed to design and implement an introductory-level course-based research experience (CRE) through project-based learning (PBL) for undergraduate medical students and investigate their learning experience and research skill development, with the ultimate objective of exploring whether CRE can be effectively utilized for undergraduate research experience.
METHODS: This study included 40 second-year premedical students enrolled in "Scientific thinking and medical research," which was developed as an introductory-level CRE through PBL. It was a three-credit course and the students met twice a week for one semester. The students' learning experiences were examined with a 15-item survey including three open-ended questions, while their research skills were assessed through the research reports using a research skill rubric at the end of the course.
RESULTS: The findings showed that the students perceived the course structure as appropriate and helpful. They also considered the group work experience to be positive and productive. Learner satisfaction items also earned positive responses in general. Regarding the research skill assessment, the mean value of the research skill scores of each group was 19.11 out of 27 and the mean value of each research skill score was 2.12 out of 3.
CONCLUSION: Overall, the students were satisfied with their research experience, and their research skills were developed, suggesting that this introductory CRE through PBL can be effectively and efficiently utilized for undergraduate research experience.

Introduction

All physicians need to be familiar with research processes even if they are not actively engaged in research [1,2]. They need to keep current with advancements in their fields of expertise, and therefore it is essential that physicians know how to critically appraise scientific papers [3]. To undertake such critical appraisal, they should understand how the information is derived [4]. For this reason, training medical students to acquire research skills is a critical part of medical education [3]. Furthermore, the Korean Institute of Medical Education and Evaluation’s (KIMEE) latest accreditation standards, “ASK 2019” (Accreditation Standards of KIMEE 2019), stipulate that medical schools include medical research in their curriculum, and emphasize the importance of developing medicals students’ research skills. Thus, education about medical research needs to start at the level of the undergraduate program.

Probably, the most common way to develop research skills is for students to undertake an independent research project during their undergraduate program. Research opportunities for undergraduates are often offered as elective components, as it is difficult to effectively integrate them into the core curriculum [2,5]. Such undergraduate research experience is typically offered for one or more semesters in labs, with oneon- one mentoring from a faculty member [6]. However, this absorbs a lot of time, money, and effort. According to Langhammer et al. [7], common barriers for a medical student research program are the lack of preparation (students take too long to figure out the basics of the research process), poor faculty interest (faculty cannot set aside time to train and engage temporary research assistants), and the demand on students’ time (it is difficult for students to meaningfully contribute to research projects without protected time in the curriculum).

One way to solve this hurdle and provide research experience to all students, rather than just a few selected or interested students, is to offer introductory-level course-based research experiences (CREs) as the first point of entry into research [8]. In this course, students could learn research processes, involve themselves in authentic research during protected time in the curriculum, and earn course credits. The professors are relatively free from producing a publishable unit and are responsible for teaching and facilitating students’ research processes [8]. Such a course can nurture not only students’ skills and knowledge, but also their confidence about medical research [6].

CREs make it possible for all students to be exposed to research, with development of research skills within the core curriculum, but then a question about how to design such a course arises. Murdoch-Eaton et al. [4] investigated undergraduate research exposure and associated research skill development in four main areas, namely research methods, information gathering, clinical analysis and review, and data processing in the UK medical school system. They reported that 52% of 905 projects provided opportunities for students to develop just one or two research skills. In addition, only 13% offered development in all areas. This indicates the necessity for careful consideration about how to design CREs and project-based learning (PBL) can be one of the options.

PBL is a teaching and learning method, which engages students in complex, real-world tasks that result in a product [9,10]. PBL is a student-driven, teacherfacilitated approach to learning. PBL has two essential components: a driving question, which serves to organize and drive activities, and a final product, which addresses the driving question [11]. In PBL’s learning process, the projects involve students in constructive investigation, including defining a problem, discussing ideas, designing their own inquiries, organizing their research, making decisions, collecting and analyzing data, discovering answers, and sharing findings with their peers [12]. Students usually choose a question that arouse their natural curiosity and work autonomously, collaboratively, and purposefully toward completion of the project [10]. The features of PBL, such as a driving question, constructive investigation, an end product, and students’ autonomy and collaboration, are all essential features of research processes. PBL has been widely used in various subjects, including science, mathematics, and social science, and is known to increase students’ motivation to learn, enhance their interest in content, and improve their metacognitive skills [10]. However, it has not been widely implemented in the medical education field.

Previous studies on undergraduate research experience have focused on students’ perception about their research experience or research productivity later on. Their results showed students’ high satisfaction with their research experience, students’ increased likelihood of pursuing graduate degrees and the improvement of their research productivity later on [6,13-16]. However, the issue of how to design an undergraduate research experience program has not drawn much attention. Recently, many medical schools have tried to integrate some research training into their core curriculum, partly because of the requirements of the accreditation standards of medical education. Introductory-level CREs through PBL can serve as an ideal vehicle to develop undergraduate students’ research skills, but few studies have investigated this issue.

Therefore, this study aimed to design and implement an introductory CRE through PBL for undergraduate medical students and investigate their responses about their learning experience and research skill development, with the ultimate goal of exploring whether such a course can effectively and efficiently be utilized for undergraduate research experiences. In the near absence of a discussion on effective teaching and learning models for undergraduate research experience, the results of this study can provide actionable insights for the designing of effective and efficient undergraduate research experiences.

Methods

1. Participants and procedure

Forty-two second-year premedical students enrolled in “Scientific thinking and medical research” at a university were the subjects of this study. Among them, two students who did not participate in the survey were excluded, leaving 40 students. Their average±standard deviation age was 22.08±1.07 years, with 13 (32.5%) being female and 27 (67.5%) male students. For the project, they were divided into nine groups of four to five students each, with one professor assigned to each group. The majors of nine professors included medical humanities, anatomy, physiology, biochemistry, microbiology, preventive medicine, parasitology, biomedical science, and microbiology. Before the course started, they participated in an orientation session and they were introduced about the course concept, facilitation, and course evaluations. Each group submitted a final research report at the end of the course, which was used to assess the learners’ research skills using a research skill rubric. The students’ learning experiences were examined with a survey at the end of the course. This study was approved by the Dong-A University Institutional Review Board (IRB approval no., 2-1040709-AB-N-01- 201907-BR-010-02).

2. Designing an introductory-level coursebased research experience through projectbased learning

1) Course outline

“Scientific thinking and medical research” was developed as the first point of entry into research for undergraduate medical students. Its purposes were to expose them to research, provide them with authentic research experiences, and develop their basic research skills. The course was a three-credit course and the students met twice a week during a 15-week semester. Each class meeting was 120 minutes long.

2) Course design

The course was designed based on the five unique design features of PBL, namely centrality, constructive investigation, driving question, authenticity, and autonomy [17]. First, the project of this course is “conducting research and producing a final research report collaboratively through constructive investigation.” This project is central to this course. It was the curriculum and the central teaching strategy, rather than being a mere enrichment program outside of the curriculum. The course largely comprised two parts as seen in Table 1. For the first 5 weeks, the classes delivered lectures on how to conduct research. The second part of the course was designed for students to conduct their own research in a group under the guidance of the assigned professor. Second, the project should involve students in constructive investigation, starting with defining a research question (driving question) to developing answers to the research questions. Constructive investigation is a goaldirected process that includes inquiry, knowledge building, and resolution [17]. As seen in Table 1, the students were asked to be involved into constructive investigation processes, starting with defining a research question (driving question) to developing answers to the research questions (final research reports) during the second part of the course. Their research proposals and final research reports were presented to the whole class at the 10th and 15th week and the students received feedback from the professors and the classmates in class. Third, the students were asked to choose a research question or driving question which they were interested in. Such driving questions serves to organize and drive research activities and these activities end up in a final research report which address the driving questions [11]. The students consulted with the professor working with them and chose a research question which needed constructive investigation and, at the same time, was interesting in a real-world context (Table 2). Fourth, projects should be realistic. PBL should incorporate real-life, not schoollike, challenges where the focus is on authentic problems and this characteristic give the students a feeling of authenticity [17]. Lastly, the projects were also studentdriven to a significant degree. The students coordinated research processes with professors and received feedback from them, but the professors played a facilitator role. It was the students who were in charge of the actual research, including planning, conducting, and presenting the research.

Table 1.

Class Activities

Week	Course	Activities
1st-5th	Lectures	Lectures on how to conduct research, including research ethics, literature search, experimental method, and report writing
6th-9th	Conducting research as a group	Choose a research topic
		Design and organize their research
		Write a research proposal as a group
10th		Present their research plan to the whole class
11th-14th		Conduct research according to the research plan
15th		Present their research in front of the whole class
		Develop a final research report

Table 2.

Research Topics of Each Group

Group	Research topics
Group 1	Working environment of health care provider and patient safety
Group 2	Effect of fine dust on campus students’ health and investigation of their perception about it
Group 3	Sphingolipid metabolism in cancer and cancer therapy
Group 4	Effect of fine dust on cardiovascular system
Group 5	Inhibitory effect of lipase inhibitor on Demodex folliculorum proliferation
Group 6	Medical students’ perception about fine dust and their corresponding behavior patterns
Group 7	Botox injection sites
Group 8	Exploration of Norovirus and its epidemiological investigation cases
Group 9	Investigation of existence of acanthamoeba at campus students’ learning environments

3) Course evaluation

The learning goals of this course are to develop basic research skills, to conduct research collaboratively, to develop collaboration skills, and to develop presentation skills. The course grade was designed according to the learning goals of this course. The course grade consisted of both individual and group components. The individual scores included attendance (10%), individual contribution (10%), self-assessment (5%), and peer-assessment (5%). The group scores included presentation skills (10%: 5% each for presentation of research plan and actual research), research proposal (25%), and research report (35%).

3. Experimental tools

1) Learning experience survey

The students’ learning experience about this CRE through PBL was investigated through a survey comprising 15 items composing of both 5-point Likert scale and open-ended questions as seen in Table 3. As this course was newly developed and they conducted research as a group, their responses to the course structure, their perception about group work, and learner satisfaction were the main concerns of this study. The survey included questions about course structure (four items), group work (five items), learner’s satisfaction (three items), and open-ended questions (three items). The items of the course structure (four items) were developed to explore students’ responses to the course structure including contents, number of students in a group, meeting frequency, and hours for class preparation. Their Cronbach’s α (four 5-likert items) items was 0.89. The items of the group work (five items) and the learner satisfaction (three items) were developed based on the learner perception survey utilized by Si [18] and the learner satisfaction survey utilized by Shin and Chan [19]. The items of these surveys were modified and adapted for the purpose of this study. Their Cronbach’s α were 0.93 and 0.86, respectively. The three openended questions were presented to complement the survey results.

Table 3.

Descriptive Statistics of Students’ Responses about Their Learning Experience

Course	Items	Students’ responses	Mean±standard deviation
Course structure	1	The contents of the first 5 weeks were helpful for conducting research.	4.17±0.78
	2	The number of students in one group was adequate.	4.62±0.62
	2-1	If the number of students was not adequate, how many students do you think are appropriate for one group?	4.29±0.70
	3	Your group met the professor twice a week and 2 hours per session. Do you think it was adequate?	4.05±0.99
	3-1	If not, how many times do you think would have been adequate? (time)	3.89±2.77
	4	I was always well prepared for the class work.	4.22±0.68
	4-1	How many hours did you spend a week for the class preparation? (hr)	4.50±3.77
Group work	5	I actively participated in the class work.	4.33±0.68
	6	Communication between the professors and students went smoothly.	4.40±0.85
	7	The group work went well.	4.45±0.88
	8	Interactions with my colleagues have greatly contributed to my learning.	4.45±0.82
	9	Through the group work, my understanding deepened in the related area.	4.48±0.66
Learner satisfaction	10	I am generally satisfied with this course.	4.19±1.03
	11	Taking this course was a valuable experience for me.	4.04±1.03
	12	I feel a sense of accomplishment while studying at the class.	4.14±0.92
Open-ended questions	13	What were the most beneficial aspects of this course?	-
	14	What were the most difficult aspects of this course?	-
	15	Is there something that needs improvement?	-

2) Research skill rubric

The students’ research reports were assessed using a research skill rubric. Students’ acquisition of research skills is difficult to measure through standardized tests. PBL emphasizes authenticity, and therefore the assessment also needs to be authentic. A rubric is a tool that articulates the expectation for an assignment by listing criteria and describing the level of quality [20]. It is often utilized for performance assessment. Stark et al. [21] and Murdoch-Eaton et al. [4] have classified research skills into four main areas, namely research methods, information gathering, critical analysis and review, and data processing, and have developed 10 learning outcomes corresponding to these skill areas. These learning outcomes were modified to suit this introductory CRE, based on which the research skill rubric was developed as seen in Table 4. The rubric includes nine research skill criteria, with the level of quality of each criterion ranging from 1 to 3.

Table 4.

Mean and SD of Each Research Skill Score of All the Groups

Research skill criteria	Level of quality			Mean±SD
	1	2	3
It is expected that the students will be able to
1. Undertake literature search on topics				2.56±0.53
2. Critically appraise the relevant literature				2.11±0.33
3. Discuss rationale for this study				2.00±0.50
4. Formulate research questions				2.11±0.78
5. Select and execute an appropriate method of investigation				2.33±0.71
6. Effectively present data				1.89±0.60
7. Draw conclusions based on research results				2.00±0.87
8. Acknowledge all sources of information				2.11±0.78
9. Communicate the results well in a written report (Is it well written?)				2.00±0.50
Total	Sum (/27)			2.12±0.20

Scoring guide: 1=little evidence of the criterion being met; 2=some evidence of the criterion being met; 3=acceptable evidence of the criterion being met.

SD: Standard deviation.

4. Analysis

The students’ responses about their learning experiences were analyzed with descriptive statistics. The answers of the open-ended questions were coded using open coding techniques. Using the rubric, the students’ research skills were scored on the basis of the research report that each group submitted. Two professors scored the nine reports separately and then set a final score based on consensus reached through discussions in the case of any disagreement. Their score was also analyzed with descriptive statistics. All statistical analyses were conducted using IBM SPSS ver. 24.0 (IBM Corp., Armonk, USA).

Results

1. Students’ learning experience

The findings of the statistical analysis of the students’ responses about their learning experience in this course are seen in Table 3. All of the items in the three areas showed a mean score of over 4.0 out of 5. The findings showed that students perceived the course structure as appropriate and helpful. They considered the group work experience to be positive and productive. Learner satisfaction items also earned positive responses in general. Overall, they were satisfied with this learning experience.

With respect to the open-ended questions, many students indicated that the development of research skills was the most beneficial aspect of the course. They said they learnt how to read scientific journals, find scientific journals, write a journal article, conduct statistical analyses, and prepare presentation materials. As another advantage, students cited the development of collaboration skills with comments like “it was interesting and helpful to discuss and collaborate with other students,” and “I do understand how to communicate with others.” They also mentioned the benefits of becoming an active learner, saying “I came to know how to learn actively,” and of gaining new knowledge and new skills with comments like “my understanding in the relevant field deepened through the research project” and “I got a lot of experience that can be used in the future.” They also cited increased motivation, saying “it was good to be able to learn what I was interested in,” and a sense of fulfillment with comments like “I felt a sense of accomplishment.”

On the contrary, the students unsurprisingly indicated that conducting research was a difficult aspect of the course, saying “I did not know what to do at the very beginning as this was the first time.” They also cited difficulties in understanding scientific journals, selecting research topics, finding relevant journals, understanding journals in English, conducting an experiment, and writing a report. In addition, some students pointed out the discomfort of being an active learner during the whole project unlike in other courses, saying “I have always been a passive learner, everything was new and it was not easy for me.” Some students indicated excessive preparation for the classes and free-riding in collaborative work as problems.

As for suggestions to improve the learning experience, they called for clearer guidelines about the research process, options for research topics, and more class time, saying “clearer guidelines about the research process are necessary,” “depending on the professors, there are differences in what is to be done,” “I like to choose one from several research topics that the professors may suggest,” and “more class time is necessary considering what we do.”

2. The assessment of students’ research skills

Students’ research skills were assessed on the basis of the research skill rubric. Tables 4 and 5 show the results. Table 5 shows the mean value and standard deviation of the research skill scores of each group. Table 4 shows the mean value and standard deviation of the individual research skills score for all the groups. The mean of the research skill scores of each group was 19.11, with the scores ranging from 13 to 24. The mean of the individual research skills score was 2.12, with the skill to “undertake literature search on topics” earning the highest score and the skill to “effectively present data” earning the lowest score.

Table 5.

Mean and SD of the Research Skill Scores of Each Group

	Group									Mean±SD
	1	2	3	4	5	6	7	8	9
Score (/27)	19	18	13	22	19	21	18	18	24	19.11±3.10

SD: Standard deviation.

Discussion

In recent times, the promotion of undergraduate medical students’ research capacity is being regarded as one of the important educational goals of medical schools. This study developed and implemented an introductory-level CRE through PBL for undergraduate medical students and explored their responses about their learning experience and research skill development. The findings showed that overall, the students were satisfied with the course structure and group work. Although some students expressed difficulties in conducting research, the results, in general, indicated that they were satisfied with the learning experience from this CRE through PBL. The high rating given by students to the undergraduate research experience results is comparable to findings from previous studies that investigated students’ perception about undergraduate research electives [14-16,22].

Regarding research skills development, there was some evidence of the criteria being met for all individual research skills, except the skill to “effectively present data.” This result is supported by the answers to the open-ended questions, where many students cited the development of research skills as an advantage of taking this course. Thus, it is fair to say that they developed research skills from undertaking literature search on topics to communicating the results in a written form through this research experience. The students particularly experienced difficulties in “effectively presenting data,” which might be because they were unfamiliar with the research culture. Thus, interventions to develop this research skill need to be incorporated when designing undergraduate research experiences. This result is similar to a finding by Gavin [23], who reported that PBL is effective in developing problem-solving skills. Although he did not specifically explore the development of research skills through PBL, problemsolving skills incorporate any kind of inquiry process. Thus, both results are thought to show similar results. The students also cited as advantages the development of collaboration skills, becoming an active learner, gaining of new understanding, and increased motivation. These results reinforced findings from a previous study, which reported that PBL increased the motivation to learn, interest in content, teamwork development, and communication skills [23,24].

The students cited conducting research as a major challenge in this course and called for clearer guidelines for research processes to address this challenge. The students said they felt perplexed at the beginning as they could not figure out what to do. According to the study by Murdoch-Eaton et al. [4] on how students perceive research, there is a clear inconsistency in the definition of research and its components among students and researchers. They argue that although students are broadly aware of research, they are not necessarily clear about what it actually entails. In this study, the students were briefly taught the research processes during the first five weeks, but they still felt challenged in figuring out what to do. One way to mitigate this problem can be to provide a rubric before they begin their research. The research skill rubric in this study was utilized to assess their research skills, but it was not disclosed to the students at the beginning of the class. The rubric is an assessment tool, but, at the same time, it can improve the students’ ability to identify the specific research skills they might acquire and figure out the degree to which each research skill is properly employed. Providing students with a rubric with explicit descriptions of research skills as a research guide may help them become more comfortable with the notion of research, including conducting and directing it. Further research to address this issue is necessary.

Demand on student time is a common barrier for research programs [7]. Some students indicated excessive preparation as a challenge. However, the score of the related survey item, “I am always well prepared for the course” was 4.22, with students spending 4.5 hours per week on average on class preparation. Thus, it appears that the learning burden was not too high in general and was manageable. In addition, regarding the group size, the survey results showed that the number of students per group was appropriate. A group of 2-5 students, in general, is recommended for PBL [11]. If the number exceeds 5, it would become difficult to ensure that all of the students contribute their share of the work and participate equally in team discussions [10]. Considering that the students were mostly new to meaningful research experience and given the difficulties they faced while conducting research, it seems that a group of 4-5 students is appropriate for successful collaboration to occur.

Some students suggested that they want to choose a topic from a range of options that their professors suggest. It is ideal for students to choose research topics they are interested in, but if they feel frustrated with selecting one, providing them with some options to choose and letting them choose the most intriguing one among them could be a possible solution as they are still allowed to pursue what they are interested in.

This study has a limitation in that there was no pre-post comparison regarding the development of research skills. Although the students did not mostly have meaningful previous research experience, the audience needs to consider this factor to interpret the results. A couple of future research directions are suggested. In the medical education field, there is little discussion on how undergraduate research experience programs should be designed. Additional research about learning and teaching models to develop undergraduate medical research skills is definitely necessary. Explicit and measurable research-related outcomes should be articulated to determine whether a program is successful. However, there is little research to address this issue, particularly in Korea. Multi-institutional research on the outcome of the undergraduate research-related curriculum is necessary, especially since many medical schools are now trying to integrate some research training into their core curriculum and to provide opportunities for students to be involved in authentic research experience.

In conclusion, it seems that this introductory-level CRE through PBL can be effectively and efficiently utilized for undergraduate research experience. Research skills are required to ensure that students are competent both as future practitioners and as clinical or basic science researchers. This introductory-level CRE through PBL can help students understand the basic skills of research, serving as a gateway to a cumulative undergraduate research experience, wherein the level of specialization, complexity, and rigorousness of research will progressively increase on the research continuum [5].