Evaluation of critical thinking course for premedical students using literature and film.

PURPOSE: Premedical education is one stage in which various educational approaches are used to promote critical thinking. Given that critical thinking ability could be regarded as one of the intended outcomes of social science and humanities education, this study explored the effectiveness of a course to promote critical thinking in a premedical curriculum using both literature and film.
METHODS: Fifty-one 2nd year premedical students enrolled in a 'Critical Thinking for Premeds' course. Students were required to read or watch a selected material, submit group discussion agendas, attend five group discussion sessions, and write critical essays. Five tutors facilitated the group discussions, observed and assessed the students' performance and critical essay. Students' critical thinking disposition and opinion on assigned reading materials were examined before and after the course. A program evaluation survey was conducted to investigate the students' reaction after the course.
RESULTS: On average, students appreciated 78.6% of the total assigned materials. The students indicated that group discussions and the narrative comments of facilitators contributed the most to develop critical thinking. After the course, the students' tendency preferring cheaper price books, as well as medicine-related books decreased significantly. Students who had critical essay scores greater than or equal to the median demonstrated a significant improvement in critical thinking disposition scores.
CONCLUSION: The course was well-accepted by premedical students and had several positive outcomes. A more effective use of the course could be anticipated with a clearer explanation of the purpose, the consideration of previous reading experience, and use of complementary learning activities.

Introduction

The importance of cultivating critical thinking from current medical students who will practice in a constantly changing environment cannot be overemphasized. Critical thinking ability is reported to be related to trainees’ various behaviors concerning clinical practice, such as data gathering, information synthesis, and decision-making [1]. Moreover, many educators consider it as an ability that could be ‘learned’ though appropriate instruction. Although the terminology used is different, various competency frameworks and accreditation standards contains critical thinking or similar concepts [1]. In South Korea, in line with these trends, the Korean Institute of Medical Education and Evaluation (KIMEE), the only medical education accreditation agency recognized by Ministry of Education, recently revised its accreditation standards based on those of the World Federation for Medical Education [2] and recommended medical schools teach ‘critical thinking’ throughout their curriculum [3].

Likewise, the general consensus is that critical thinking is important to medical students as well as practitioners, and premedical education is one typical stage of a medical education continuum in which critical thinking is underlined. For example, undergraduate-entry programs include a premedical phase within their curriculum [4], during which time medical schools provide various opportunities for students to develop critical thinking as one of their core competencies. On the other hand, some medical schools that have a graduate-entry program often assess critical thinking ability when they select new students [5], based on the expectation that successful applicants should have reached a certain level by the time of admission.

Meanwhile, premedical education has consistently pointed out that it has emphasized hard sciences, such as physics and chemistry, at the expense of a social science and humanities (SSH) [6]. Furthermore, in recent years, time and money spent for literary reading by people has declined and is deemed to be a social problem [7,8]. Even millennial students who are more familiar with digital media are no exception. Likewise, they are more accustomed to read fragmented parts that are more appealing than the original text [9].

Taken together, it seems necessary that premedical students need to develop both critical thinking and certain competencies related to SSH before starting their undergraduate medical education. Teaching them together in an integrated course, therefore, can be an effective and reasonable option, and films and literature is worthy to be considered as learning resources [10,11]. Above all, it is because literature suggests that the aim of ‘humanities’ education should not only be a mere transmission of knowledge but fostering the ability to use it in real-life practice as a tool that can complement ‘science’ in our complex healthcare system [12]. Similarly, critical thinking could be more effectively taught by engaging learners in immediately relevant episodes or narratives rather than delivering it as an abstract concept [13]. Indeed, it has been reported that students learn critical thinking in a course that uses literature, regardless of the director’s intended learning objectives [14].

To the best of our knowledge, however, critical thinking is not a common goal in so called ‘literature and medicine’ or ‘cinemeducation’ courses [10,11,15], and only a few studies explicitly state it as an objective [11]. In addition, according to a systematic review, these courses are mostly used in medical courses rather than in premedical courses, and studies conducted in Asian countries are rare [16].

In this study, we introduced a critical thinking course to promote critical thinking skills and encourage more reading. Based on this purpose, we investigated the students’ satisfaction of the course and their changes in behavior and perception related to the selected reading materials to evaluate its outcome. In addition, we tried to explore changes in critical thinking disposition (CTD) of students.

Methods

1. Setting

The Eulji University School of Medicine (EUSOM) is a 6-year medical school, whose curriculum is divided into three 2-year phases—which are premedical, preclinical, and finally the clinical phase. In 2018, EUSOM carried out a minor premedical curriculum revision and introduced a course named ‘Critical Thinking for Premeds (CTP)’ to address the social and educational needs such as revised KIMEE accreditation standards and feedback collected from our students. The course was conducted for a total of 15 weeks. At the beginning of the course, a list of five selected reading/watching materials were announced. These materials were selected with consensus of five professors who participated in the course to cover a broad area including the social sciences and humanities. The list of materials chosen is as follows: “Sapiens” by Yuval Noah Harari, “Apeumi-giri-doeryeomyeon” by Seungsup Kim, “Hidden figures (film),” “When breath becomes air” by Paul Kalanithi, and “Sesangeul-bakkun- jilmundeul” by Kyeongmin Kim.

The students attended five group discussions every 3 weeks. During the group discussions, students were encouraged to present their own point of view related to assigned material or certain agendas to compare diverse perspectives of members. For this purpose, before attending each session, students were asked to read or watch selected material, submit two or three agendas for a group discussion, and then write a critical essay based on it. In order to write a critical essay, students were expected to properly understand and interpret the given materials.

2. Participants

The entire 2nd year premedical students enrolled in CTP, a required course. For group activities, students were randomly divided into five groups consisting of 10 or 11 students. These groups remained the same throughout the course. Five professors, each of whom majored in either anatomy, physiology, biochemistry, preventive medicine, and medical education, participated as tutors. They facilitated group discussions, observed and assessed the students’ performance and submitted critical essays. The five groups were scheduled to meet each of the five professors once. This was done to minimize the stringency influence of the tutors on their performance assessment.

3. Data acquisition

1) Student assessment

The CTP included three components for student assessment. First, each tutor, as a facilitator, assessed the quality of the agenda submitted by students on a 3-point scale (0: not submitted, 1: acceptable, 2: excellent) and their performance in discussion (1: very poor–4: excellent). Second, the tutors evaluated the critical essays according to whether a student used various critical thinking skills effectively or not (1: very poor–4: excellent), then provided a written narrative comment on each essay. Third, as a peer-assessment on performance, we asked students to nominate three peers or less who contributed the most to group discussions.

2) Course evaluation

After completing the course, a survey was conducted to investigate the students’ reactions to various procedural elements, which corresponds to the first level of Kirkpatrick’s four-level evaluation model. Although higher satisfaction in education does not necessarily guarantee better outcomes, it has been argued that, in case of a newly adopted program, satisfaction could be more worthwhile than performance in curriculum evaluation [17]. Thus, we constructed the questionnaire in two parts: affective reactions and utility judgments. In part one, students were asked about subjective satisfaction to each course element, as well as to the whole course. In part two, students were asked about how they perceived the usefulness of the whole course and each learning activity with regard to critical thinking. Furthermore, we gathered students’ basic demographic information and their pre-class and in-class reading behavior.

3) Changes in CTD of students

All students submitted the Yoon’s Critical Thinking Disposition Instrument (YCTDI) in the first and the last week of the course. The YCTDI consists of 27 items in seven factors, with 5-point Likert scale (1: strongly disagree–5: strongly agree). Although the instrument was originally developed and used in the field of nursing, validity and reliability were confirmed from various studies [18].

4) Student opinions on assigned reading material selection

The students were asked to answer an online questionnaire about opinions on book selection criteria. In the questionnaire, students were asked to rate five criteria (e.g., inexpensive books, recently-published books, best-selling books, thin books, and medicine-related books) on a 4-point scale (1: not important–4: very important) considering their significance when selecting books as learning materials in the CTP course. The same questionnaire was conducted anonymously during the 2nd and the 15th week of the course.

4. Data analysis

In this study, we collected quantitative as well as qualitative data. Statistical analysis was performed for all quantitative data by using IBM SPSS for Windows software ver. 22.0 (IBM Corp., Armonk, USA). When comparing the results before and after the course, paired sample t-test or independent sample t-test was used depending on the anonymity of data. Course evaluation data was analyzed by one-way analysis of variance.

5. Ethics approval

This study was approved by the Institutional Review Board of EUSOM (EUIRB 2018-101). The Institutional Review Board waived the need for explicit consent from participants.

Results

1. Demographics

The total number of enrolled students was 51, and 74.5% were male (Table 1). Their mean age was 21.4 years (standard deviation=1.52). Regarding the number of books read during year 1, 48% of the students answered that they did read less than two books in the last year.

Table 1.

Descriptive Statistics of the Students Enrolled in Critical Thinking Course (N=51)

Variable	Value
Gender
Male	74.5 (38)
Female	25.5 (13)
Age (yr)	21.4±1.52 (19–26)
The average no. of books read during the 1st year[a)]	3.04±1.58[b)]
None	14.0 (7)
1–2 Books	34.0 (17)
3–4 Books	18.0 (9)
5–6 Books	14.0 (7)
7–8 Books	8.0 (4)
≥9 Books	12.0 (6)
No. of books read/film watched during this course[c)]	3.93±0.75 (2–5)

Data are presented as % (number of students), mean±standard deviation (range), or mean±standard deviation.

These items were included in the course evaluation survey.

The average score when the response to the item was translated into 6-point Likert scale (1: none–6: more than nine books).

Students responded from 0 to 5. For example, 0.5 means that a student read/watched only half of the one of the selected materials. Five means that a student read/watched all five selected materials.

2. Affective reaction and utility judgement

With respect to affective reaction, the overall satisfaction of the course was 4.20 out of 6 (Table 2). The rating for selected materials (four literature and a film) was 3.59 out of 6 points, which was significantly lower than the other course elements such as facilitators, group members and assessment criteria (p<0.001). When we asked to what extent did each teaching and learning activity contribute to develop critical thinking, the students rated the group discussions and the narrative comments of facilitators as the highest (Table 2).

Table 2.

Students’ Affective Reaction and Utility Judgment to the Course

Elements	Mean±standard deviation	F-value[a)]	p-value	Post-hoc test[b)]
Affective reaction[c)]		19.022	<0.001	(2), (3), (4)>(1)
(1) Selected materials (books and film)	3.59±1.04
(2) Facilitators	5.02±0.88
(3) Group members	4.68±1.20
(4) Criteria included for assessment	4.45±1.33
Overall satisfaction	4.20±0.94
Utility judgment[d)]		3.416	0.006	(4), (6)>(3)
(1) Orientation to the course	3.94±1.32
(2) Selected book reading	3.96±1.11
(3) Agenda generation prior to the group session	3.80±1.06
(4) Group discussion	4.43±0.96
(5) Critical essay writing	4.24±1.27
(6) Narrative comments from facilitators	4.43±0.92
Overall contribution	4.37±0.82

By Welch’s F-test.

A Games-Howell test was used to analyze data with unequal variance.

1: Not at all satisfied–6: Extremely satisfied.

1: Did not contribute–6: Contributed greatly

One notable finding is that these two types of reactions were at odds with their correlation regarding pre-class and in-class reading behavior. While utility judgments showed a significant relationship only with pre-class reading behavior (i.e., the number of books read during year 1), affective reactions showed a significant relationship only with in-class reading behavior (i.e., the percentage of reading/watching selected materials) (Table 3).

Table 3.

Correlation between Students’ Reaction and Reading Behavior

Variable	Pre-class reading behavior	In-class reading behavior	Reaction to the course
			Affective reaction	Utility judgment
Pre-class reading behavior[a)]
Correlation[b)]	-	0.325	0.227	0.315
p-value		0.021	0.113	0.026
In-class reading behavior[c)]
Correlation[b)]		-	0.301	0.225
p-value			0.032	0.113
Reaction to the course
Affective reaction
Correlation[c)]			-	0.654
p-value[c)]				<0.001
Utility judgment
Correlation				-
p-value

The average number of books read during the freshmen.

Pearson correlation.

Students’ answer for ‘the number of books read/film watched during this course’ in Table 1.

3. Behavior and perception about reading materials

As intended, there was an actual increase in amount of reading. Contrast to the fact that almost half of the students read only as few as two books during the premed year 1, 92.2% of the students had appreciated three or more of the five assigned materials, with an average of 78.6% (3.93 out of 5) of total materials (Table 1). In addition, after completing the course, students’ tendency to prefer low price books and medicine-related books was significantly decreased (Table 4).

Table 4.

Changes in Student’s Perception to the Major Considerations of Book Selection

Major considerations[a)]	Pre-course	Post-course	p-value[b)]
Price of the book (prefers low price)	2.74±0.77	2.40±0.76	0.037
Publication date (prefers recent publication)	2.17±0.91	2.22±0.86	0.774
Sales volume (prefers best-sellers)	2.43±0.86	2.38±0.95	0.799
Pages (prefers less voluminous books)	2.90±0.88	2.86±0.88	0.808
Medicine-related (prefers medical category)	2.79±0.65	2.46±0.86	0.041

Data are presented as mean±standard deviation.

1: Not important–4: Very important.

By independent sample t-test.

4. Critical thinking disposition

In this study, Cronbach’s α was 0.844. When we analyzed YCTDI scores for all students, we did not find significant statistical differences between pre- and post-course overall YCTDI scores. The exception was the ‘intellectual fairness’ factor (p=0.037) (Table 5). However, when the students were divided into two groups according to their critical essay scores, we found YCTDI scores for only those who had critical essay scores greater than or equal to the median increased significantly after the course.

Table 5.

Pre- and Post-Evaluation of the Critical Thinking Disposition of the Students

Yoon’s critical thinking disposition	Pre-course	Post-course	p-value[a)]
Factors
Intellectual eagerness/curiosity	3.54±0.67	3.53±0.59	0.913
Prudence	3.5±0.73	3.54±0.64	0.652
Self-confidence	3.58±0.55	3.71±0.47	0.073
Systematicity	3.38±0.51	3.49±0.53	0.093
Intellectual fairness	3.94±0.58	4.09±0.46	0.037
Health skepticism	3.56±0.53	3.54±0.63	0.806
Objectivity	4.09±0.44	4.16±0.44	0.326
Total score
Total students (N=51)	25.6±2.39	26.06±2.54	0.083
≥Median critical essay scores (n=28)[b)]	25.9±2.51[c)]	26.7±2.53[d)]	0.047
<Median critical essay scores (n=23)[b)]	25.2±2.24[c)]	25.2±2.34[d)]	0.927

Data are presented as mean±standard deviation.

YCTDI: Yoon’s Critical Thinking Disposition Instrument.

By paired sample t-test.

The median score for the critical essay was 12.

The difference between the mean YCTDI scores of two groups was not significant from a statistical standpoint (p=0.292).

The difference between the mean YCTDI scores of two groups was significant from a statistical standpoint (p=0.033).

Discussion

In this study, we newly implemented a course using literature and film in a premedical curriculum. The results indicated that the course was well-accepted by students and achieved several positive outcomes including their behavior and perception related to reading and change in CTD. Provided that those outcomes are not independent in a broader context of the course, more implications need to be drawn by understanding them integratively.

Clearly, the course was meaningful to the extent that it actually led students to read more. The course also seems to have influenced students’ preference of book selection in that it mitigated their resistance to the cost involved in book purchase or to using non-medicine related books as course materials. Especially, it is worth mentioning that this was achieved even though the satisfaction level to the selected material was moderate (3.59 out of 6), relatively lower than other elements of the course, such as facilitators or group members. Medical students’ personality traits could be one possible explanation. It is well known that high academic achievement is required to pass the extremely competitive admission process of medical school, and those high achievers tend to have personalities characterized by phrases such as ‘responsibility’ or ‘achievement via conformance’ [19]. In other words, once any requirement is given, medical students are likely to comply with it when they feel they ‘have to,’ even if they do not ‘want to’ [20], and so they did for assigned reading in the CTP.

Nevertheless, it seems evident that assigned reading alone might be insufficient to achieve the intended purpose of a course. This indicates the necessity of additional requirements for the development of critical thinking. According to our findings, the first requirement was the accumulated, voluntary reading experience in daily life. It was supported by the finding that the correlation between utility judgment and one’s pre-class reading behavior, which should be largely active, was statistically significant. On the other hand, the students’ compliance with assigned reading, which tends to be passive, demonstrates significant correlation only with positive affective reaction to the course, but not with utility judgment. In other words, if one does not enjoy reading on his/her own, forced reading in the short term alone may not provide ‘educational’ utility beyond the affective satisfaction of reading itself. This is congruent with the self-determination theory that claims the autonomous motivation is positively correlated with deep approach, reflection, and higher achievement in learning [21].

Another requirement that links the assigned reading to the intended learning outcome is the subsequent use of learning activities that promotes reflection and discussion. Previously, it had been argued that change in CTD required extended and regular interaction with discourse and feedback, and feedback from peers and supervisors can be a central ‘source of information’ that triggers critical reflection [22]. Indeed, in the course evaluation, students responded that activities based on interactions and feedback (i.e., group discussion and narrative comments) were more advantageous for critical thinking development than those done individually in isolation (i.e., book reading, agenda generation, and essay writing). As for discussion, these ideas were also expressed in several students’ comments, for example,

“Because chances to exchange or discuss thoughts with other colleagues have not been given frequently in premedical curriculum, the sessions for group discussions were not only enjoyable but also practically helpful.”

“Listening to the thoughts of other group members gave an opportunity to rethink about things that I have overlooked when I read the same book alone.”

With respect to feedback from teachers in courses such as ‘literature and medicine,’ previous studies called attention to extensive formative feedback to supplement summative assessment [23]. In line with this suggestion, our assessment involves both qualitative components of formative purpose and quantitative numerical rating scales. Narrative comments are important because they provide more nuanced and rich information about student’s critical thinking skills that cannot be interpreted with mere numbers [24].

In spite of several achievements, one drawback was that a significant increase in the YCTDI scores was limited only to a subset of students who achieved relatively higher critical essay scores. The three following reasons can be considered to account for this: First, in terms of curriculum, this might be partly due to the fact that CTP was the only course in the semester which explicitly emphasized critical thinking. Second, cultural context could be another reason, provided that this study was conducted targeting Asian students who are accustomed to prioritize memorizing and understanding rather than questioning and evaluating [25]. Comparably, a study conducted in Japan which shares a similar cultural background with South Korea has reported that high school graduates experience difficulty in developing critical thinking [26]. Lastly, the result may attributable to the nature of CTD, often claimed to be relatively stable so that it is unlikely to expect rapid advancement, particularly within the short term in the curriculum [1].

The findings of this study provides several suggestions to medical educators who plan to develop and implement critical thinking courses using literature or films. First, in the introductory session, the purpose and basic concepts of the course need to be explained clearly and sufficiently. At the most fundamental level, the sharing of course expectations serves as a means of primary prevention for possible learning problems [27]. In critical thinking courses, likewise, the definition and attributes underlying the cognitive base of critical thinking should be taught as early as possible. Adjunctively, introducing any assessment would be beneficial for the purpose to confirm the proper acquisition for the basic concepts of critical thinking. Second, as the course progresses, one can presumably expect the increase in reading volume and satisfaction of reading itself, especially by using materials preferred by students. However, given that the primary intended outcome is the development of critical thinking, not just a mere increase in reading volume, our findings indicate that assigned readings should be complemented by associated learning activities to make the case of practical help. In this aspect, demonstration, guided practice, and interaction with others through discussion and feedback will induce more desired learning behavior to develop critical thinking [28]. Third, it would be worthwhile to give more attention to underachieving students during the course. Our study is consistent with a study targeting 2nd year premedical students by Chun and Lee [29], whereby no significant statistical differences in CTD depending on achievement level at the beginning of the course were found. However, in contrast with the YCTDI scores for high achievement group, which showed a significant increase, those scores of the low achievement group showed relatively no increase in spite of completing the same course.

This study is not without limitations. First, generalization of the study is limited due to the lack of a proper control group, because the research was carried out in a single medical school and CTP was the required course in which all students enrolled. Second, the achievements of this course may have been influenced one way or another by the broader educational environment or other courses simultaneously in progress. Thus, further study is needed in other institutions or cultures to verify the effectiveness and outcomes of such courses. Third, the YCTDI used in this study is an instrument that was initially developed and validated for nursing students. However, considering that premedical students are learners who are still in a relatively undifferentiated stage within the health professions education and that most of the items in YCDTI are not describing discipline-specific contents, the faculty members who participated in the course agree with the appropriateness of the tool.

In conclusion, our study demonstrated that a course in a premedical curriculum to promote critical thinking using both literature and film could be well-accepted, contribute to an increase in reading volume, influence students’ CTD as well as their perception related to reading. Based on our findings, we recommend medical educators to consider students’ previous reading experience and use complementary learning activities for more effective use of literature and films.