Promoting student case creation to enhance instruction of clinical reasoning skills: a pilot feasibility study.

BACKGROUND: It is a common educational practice for medical students to engage in case-based learning (CBL) exercises by working through clinical cases that have been developed by faculty. While such faculty-developed exercises have educational strengths, there are at least two major drawbacks to learning by this method: the number and diversity of cases is often limited; and students decrease their engagement with CBL cases as they grow accustomed to the teaching method. We sought to explore whether student case creation can address both of these limitations. We also compared student case creation to traditional clinical reasoning sessions in regard to tutorial group effectiveness, perceived gains in clinical reasoning, and quality of student-faculty interaction.
METHODS: Ten first-year medical students participated in a feasibility study wherein they worked in small groups to develop their own patient case around a preassigned diagnosis. Faculty provided feedback on case quality afterwards. Students completed pre- and post-self-assessment surveys. Students and faculty also participated in separate focus groups to compare their case creation experience to traditional CBL sessions.
RESULTS: Students reported high levels of team engagement and peer learning, as well as increased ownership over case content and understanding of clinical reasoning nuances. However, students also reported decreases in student-faculty interaction and the use of visual aids (P < 0.05).
CONCLUSION: The results of our feasibility study suggest that student-generated cases can be a valuable adjunct to traditional clinical reasoning instruction by increasing content ownership, encouraging student-directed learning, and providing opportunities to explore clinical nuances. However, these gains may reduce student-faculty interaction. Future studies may be able to identify an improved model of faculty participation, the ideal timing for incorporation of this method in a medical curriculum, and a more rigorous assessment of the impact of student case creation on the development of clinical reasoning skills.

Plain language summary

This study sought to explore the impact of student “case creation”, or student generation of clinical cases, on students’ perceived clinical reasoning skills, quality of student–faculty interaction, and interaction with fellow teammates. Ten students at Stanford University School of Medicine participated in a pilot study in spring 2014, wherein they worked in groups of five to generate a clinical case around a specified diagnosis using “case creation” as a teaching/learning tool. Students took part in pre- and post-study surveys, as well as student focus groups, designed to look at each of the three designated outcomes. Two faculty members helped facilitate the pilot study and also participated in a faculty focus group. Both qualitative and quantitative analyses were performed. Results suggest that student “case creation” can be an effective means of cultivating teamwork and providing an active, student-initiated means of clinical reasoning instruction. However, these gains may come at the cost of decreased levels of student–faculty interaction. Overall, this study suggests that student “case creation” could be an additional teaching/learning tool for instructors to rely upon when engaging students in clinical reasoning instruction.

Background

Clinical reasoning can be a challenging skill to teach early medical students because it requires not only the ability to gather data but also the ability to synthesize and interpret the information. One method of clinical reasoning instruction that has been widely applied is the use of patient-based cases, wherein clinical instructors select specific clinical scenarios designed to allow medical students to practice clinical reasoning in a structured environment.1

Many medical schools, including our own, traditionally use a case-based learning (CBL) format to provide clinical reasoning instruction. CBL is described as a “guided inquiry approach”, wherein small groups of students focus on creative problem solving with some advance preparation, such as reading an assigned textbook chapter.2 Starting in the spring of their first year, Stanford medical students work in groups of six to ten students, with one to two faculty members guiding students through one or more clinical cases over the course of three to four hours.

Previous studies suggest that learning can be enhanced further when learners themselves are encouraged to generate content. In 2004, Ryan and Marlow described a method called “build-a-case”, wherein multiple members of the medical team, including nurses, residents, and faculty, but not including medical students, came together to create a case. The authors suggested that this exercise not only engaged learners in a more reflective dialogue but also addressed one of the greatest challenges in problem-based learning – having only a limited number of cases for learners to solve.3 Palmer and Devitt further built on this concept by engaging medical students in the task of creating multiple-choice questions. They found that students were capable of generating high-quality questions that had potential for inclusion in both formative and summative assessments. However, the students found the activity unfamiliar and, ultimately, the studied group of students did not perform better than the control group.4 Jobs et al also studied the idea of student-generated multiple-choice questions and found that the task favored low- more than high-performing students and did not deem it an appropriate method to include in curricula.5

The notion of engaging students in case creation has been examined previously in advanced medical students. A study conducted at Indiana University School of Medicine engaged four Master of Science in Medical Science (MSMS) students in creating cases directed towards first-year MSMS students as well as medical students through a 15-week case-writing course.6 A 2001 study, also at Indiana University, took this one step further, engaging third- and fourth-year medical students in case creation as part of a 4-week course.7 However, in both of these case-writing courses, participants engaged in case creation with the primary goal of furthering other students’ learning and not their own understanding of case material.

To our knowledge, student case creation has not been tested previously as a modality of clinical reasoning instruction in a pre-clerkship medical curriculum. More specifically, case creation has not been previously tested with first-year medical students. We conducted this study to examine three outcome measures: 1) tutorial group effectiveness, 2) student–teacher interaction, and 3) perceived clinical reasoning gains.

Methods

Participants

Ten first-year medical students and two clinical faculty members volunteered to participate in this pilot curriculum. All participants were affiliated with Stanford University School of Medicine. All participants signed a written informed consent form prior to participating in the study. Student participants were compensated with dinner during the study, as well as a $20 gift from the Stanford bookstore.

Structure and content of case creation session

Students participated in one 3-hour pilot study in May 2014. At this point in their curriculum, participants had taken part in five CBL sessions and, as such, were familiar with the structure and content of clinical teaching cases. The ten students were split into two groups of five students each, with one facilitator for each group. During the first 2 hours, students developed patient cases around the preassigned diagnosis of “dilated cardiomyopathy”, with intentionally limited faculty input. Faculty answered a few student questions during the session but otherwise remained as bystanders, allowing for a student-driven process. At the end of the 2-hour period, all students participated in a 1-hour feedback session with the faculty facilitator on case quality, discussing issues such as choice of etiology and providing clues to narrow down the differential diagnosis.

Student survey feedback

The Stanford Faculty Development Program Clinical Teaching Instrument (SFDP-26)8 was used to assess the level of student–teacher interaction the students experienced during their prior traditional CBL curriculum (the “before” survey, based on recall from prior sessions in the curriculum) and their experiences during the case creation session (the “after” survey). This validated instrument included twenty-six items in seven domains, including learning climate, control of session, communication of goals, promoting understanding and retention, evaluation, and feedback.

The Tutorial Group Effectiveness Instrument (TGEI)9 was used immediately after the case creation session to assess the group effectiveness as a result of this exercise and the teamwork the students experienced during the session. This validated survey consisted of twenty statements for students to agree or disagree with, in three different domains: cognitive aspects (eg, “In the tutorial, group explanations of the subject content were given in own words”), motivational aspects (eg, “The tutorial group stimulated my self-study activities”), and demotivational aspects (eg, “During the course of the tutorial, some group members contributed less to the tutorial group discussion”).

Survey responses from both questionnaires were recorded on a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree). Quantitative data from the surveys were analyzed with descriptive statistics, using a Student’s paired t-test to compare pre- and post-study results of the SFDP-26. No statistical adjustment was made for multiple comparisons. In addition, we calculated overall means for students’ responses on both the SFDP-26 and the TGEI (Microsoft Excel for Mac 2011, Version 14.6.2.).

Student and faculty focus groups

Student and faculty participants also took part in separate focus groups at the end of the session (questions in the Supplementary material). Student focus groups were run immediately after the case creation session and were led by students who were involved in the study design. The faculty-specific focus group was also conducted after the session by the primary author (HC), providing faculty with the opportunity to reflect on their facilitation experiences. One student was unable to attend the student focus group and was interviewed separately by the primary author (HC) with the same focus group questions. All participants verbally agreed to be audiotaped during these sessions for the purposes of this study. Student and faculty focus group transcripts were transcribed and de-identified by the primary author (HC). Three researchers instructed in qualitative data analysis inductively developed the codebook over five passes and came to agreement for all code definitions. We then applied the finalized codebook, consisting of eighteen codes, to excerpts across all three transcripts. We tested for inter-rater reliability of all coders and code applications, ultimately reaching high inter-rater reliability, with k of 0.78. Finally, the research team (HC, NG, JB, SBM) thematically analyzed the coded excerpts to identify emergent themes about student and faculty experiences and evaluate the applicability of the case creation method.10 Dedoose was used to manage the data and facilitate the coding and thematic analysis process (Dedoose, Version 5.2.1.; SocioCultural Research Consultants, LLC, Los Angeles, CA, USA).

Human subjects research

This study was approved by the Stanford Institutional Review Board under the exempt review category for research on the effectiveness of instructional techniques, curricula, or classroom management methods.

Results

Student feedback

A total of ten students volunteered to participate in this feasibility study. All ten (100%) students completed the pre- and post-pilot study questionnaires regarding tutorial group effectiveness and quality of student–faculty interaction. Nine (90%) students participated in the focus group, while one student was interviewed separately using the same set of questions.

The comparative results of the SFDP-26 are shown in Table 1. Overall, compared to their traditional CBL sessions, students reported significant increases in the amount of negative (corrective) feedback they received in the case creation session. Students reported decreases in the level to which facilitators listened to learners, used blackboard or other visual aids, explicitly encouraged further learning, and encouraged learners to do outside reading in their case creation session.

Table 1

Quality of student–teacher interaction in traditional CBL session versus case creation session

Category, items in evaluation of clinical teaching	Traditional CBL session	Case creation session	Paired t-test
Learning climate
Listened to learners	4.90 ± 0.32	4.50 ± 0.53	0.04*
Encouraged learners to participate actively in the discussion	4.30 ± 0.82	4.30 ± 0.82	1.00
Expressed respect for learners	4.70 ± 0.67	4.50 ± 0.85	0.34
Encouraged learners to bring up problems	4.60 ± 0.97	4.20 ± 0.92	0.22
Control of session
Called attention to time	3.80 ± 1.40	3.70 ± 0.67	0.85
Avoided digressions	3.60 ± 1.17	4.00 ± 1.15	0.37
Discouraged external interruptions	3.90 ± 1.10	3.70 ± 1.34	0.66
Communication of goals
Stated goals clearly and concisely	3.80 ± 0.92	3.70 ± 0.82	0.59
Stated relevance of goals to learners	3.80 ± 0.79	3.60 ± 0.97	0.51
Prioritized goals	3.80 ± 0.82	3.40 ± 0.97	0.28
Repeated goals periodically	3.30 ± 0.82	3.40 ± 0.97	0.68
Promotion of understanding and retention
Presented well-organized material	3.90 ± 1.20	3.60 ± 0.97	0.34
Explained relationships in materials	4.30 ± 0.48	3.90 ± 0.88	0.17
Used blackboard or other visual aids	3.50 ± 1.43	1.60 ± 0.70	<0.01*
Evaluation
Evaluated learners’ knowledge of factual medical information	4.00 ± 0.94	4.00 ± 0.82	1.00
Evaluated learners’ ability to analyze or synthesize medical knowledge	4.10 ± 0.74	4.50 ± 0.52	0.17
Evaluated learners’ ability to apply medical knowledge to specific patients	4.10 ± 0.74	4.30 ± 0.82	0.56
Evaluated learners’ medical skills as they apply to specific patients	3.70 ± 1.06	4.30 ± 0.82	0.19
Feedback
Gave negative (corrective) feedback to learners	3.60 ± 0.52	4.50 ± 0.71	0.02*
Explained to learners why he/she was correct or incorrect	4.00 ± 0.67	4.40 ± 0.52	0.10
Offered learners suggestions for improvement	3.80 ± 0.79	4.50 ± 0.71	0.066
Gave feedback frequently	3.80 ± 1.23	3.70 ± 1.16	0.84
Promotion of self-directed learning
Explicitly encouraged further learning	4.60 ± 0.70	3.80 ± 0.63	0.02*
Motivated learners to learn on their own	4.50 ± 0.53	4.00 ± 1.05	0.21
Encouraged learners to do outside reading	4.10 ± 0.74	3.00 ± 0.82	<0.01*
Overall teaching effectiveness	4.60 ± 0.70	4.60 ± 0.52	1.00

Notes: We used a 5-point Likert scale for each statement from the Stanford Faculty Development Program Clinical Teaching Instrument,8 with a range: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Data are presented as mean and standard deviation.

These values are significant at P ≤ 0.05.

Abbreviation: CBL, case-based learning.

Table 2 illustrates students’ perceptions of tutorial group effectiveness during the pilot study on the TGEI. Overall, students reported high levels of teamwork and collaborative learning, with an average response of 4.6 (on a 5-point scale) for overall group productivity.

Table 2

Post-study survey results on tutorial group effectiveness

	Mean ± SD
Factor 1: Cognitive aspects
During the tutorial, many explanations of the subject content were given by individual students	4.40 ± 0.70
In the tutorial, group explanations of the subject content were given in own words	4.00 ± 0.67
Students posed adequate questions to each other to obtain a deeper understanding of the subject matter	4.40 ± 0.52
Students asked critical questions to check the explanations of content given by other students	4.40 ± 0.70
In the tutorial group, I learned much from the contributions of the other group members	4.50 ± 0.53
In the tutorial group, misconceptions about the subject matter were corrected by other group members	4.00 ± 0.82
Group members built on each other’s arguments	4.30 ± 0.48
Factor 2: Motivational aspects
I felt myself as a member of the group responsible for the progress of the group	4.10 ± 0.99
If I did not prepare well for the tutorial group meeting, I felt uncomfortable in the group	2.70 ± 0.95
I became more perceptive and sensitive to the needs of the other students within my group during group work	3.50 ± 0.71
The tutorial group stimulated my self-study activities	4.00 ± 0.94
The tutorial group had a positive effect on my academic commitments/efforts	4.40 ± 0.52
My interest in the subject matter increased due to the discussions in the tutorial group	4.80 ± 0.42
The tutorial group discussion stimulated my group mates to exert maximum effort	3.80 ± 0.92
Factor 3: Demotivational aspects
During the course of the tutorial, some group members contributed less to the tutorial group discussion	2.50 ± 0.97
Some group members intentionally withheld information they had acquired during self-study	1.20 ± 0.42
I did not contribute as much to the tutorial group discussion as I could have done	2.70 ± 1.25
Some group members had a negative effect on the contributions of other group members	1.50 ± 0.97
Some group members let others do the work	2.10 ± 1.20
Give a qualification for the overall group productivity (overall score)	4.60 ± 0.52

Notes: We used a 5-point Likert scale for each statement from the Tutorial Group Effectiveness Instrument,9 with a range: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Data are presented as mean and standard deviation. Note that for the items under Factor 3, a lower score is more positive, that is, indicates a decreased level of demotivational factors.

Abbreviation: SD, standard deviation.

Finally, Table 3 describes themes that emerged during the student focus group session (Supplementary material), which explored students’ perceptions on case creation compared to their traditional CBL sessions. General themes were divided into two domains: “comparison to traditional clinical reasoning” and “session design considerations”. Both of these domains contained two to three qualitative themes. The most common positive features of the student focus group were increased ownership over case content, engagement in clinical reasoning, and understanding of clinical nuances. Students reported that they would have liked more interspersed and directed feedback from faculty members, as well as some additional guidance regarding the etiology of the assigned diagnosis. This latter opinion stemmed from the fact that both case creation groups spent a significant portion of their two hours discussing the most likely etiology for the assigned diagnosis of dilated cardiomyopathy, allowing limited time to focus on the remainder of the case.

Table 3

Student and faculty perceptions of case creation versus traditional case-based learning sessions

Theme	Representative quote
Domain 1: Comparison to traditional clinical reasoning
Both students and faculty reported higher levels of student engagement and ownership over content, following the case creation session.	Student: “I think that this experience and having to create the case ourselves […] forced me to be more invested in learning the particular condition that we needed to create the case around.”
	Faculty: “I think [this exercise] is interesting. I really liked seeing the level of engagement and teamwork. I thought that was great. And that was almost to the point that it was more than I see in our small groups now.”
Students and faculty both felt that students learned the nuances of the specific diagnosis well.	Student: “I think making the case forced us to think about the differences between diseases and how we would differentiate between them because we’re trying to think of how we’re going to give enough clues to someone else so they can actually differentiate between different pathologies.”
	Faculty: “I thought that I could feel the [students] learning about a specific diagnosis - in this case, dilated cardiomyopathy. They learned, I think, a lot about that [diagnosis] - perhaps more than they would have learned in a traditional [teaching] format.”
Domain 2: Session design considerations
Both students and faculty felt that interspersed, directive feedback would be more effective.	Student: “I think we could have benefited from a little more real-time feedback from the facilitator, given that he was – in fact – sitting here anyways. But I’m not sure how we would structure that.”
	Faculty: “I think that if we’d had the opportunity to do evaluation and feedback as they were going through, even through questions, that it would have been more gentle and probably more productive for [the students].”
Students reported wanting more guidance regarding case etiology and team structure.	Student 1: “One thing that we could do is already have the etiology decided beforehand and then we’d just work starting from there so then we wouldn’t have used up time deciding what etiology each group wanted.”
	Student 2: “[We would have benefited from] a structure for choosing someone to be the facilitator and […] and order [about how] we should go about thinking about how to do the case.”
Faculty felt that the case creation method as a learning tool would be more effective after students gained more clinical experience.	Faculty: “I think that [first] year might be a little early, because of what was pointed out about the problem lists and differential [diagnoses]. I would want to make sure that [the students] have a really firm, solid grasp of those kinds of basic principles and how [they] change over time.”

Faculty feedback

In addition to describing student focus group themes, Table 3 also illustrates themes that emerged during the faculty focus group. The most common positive features of the faculty focus group were increased levels of engagement and teamwork. Faculty members did, however, express reservations regarding these first-year students’ clinical experience and how that limited their ability to create a high-quality patient case that was on par with a faculty-generated case. In addition, faculty members echoed the student opinion that the opportunity for interspersed faculty feedback could have been more productive.

Discussion

Student case creation is feasible and may be a creative adjunct to standard methods of clinical reasoning instruction. Students and faculty alike were notably impressed by the level of engagement and teamwork present during this activity, as reflected by findings from both the TGEI survey responses and focus groups.9 This led students to feel that they had developed an appreciation for the nuances required to create a patient case, which in turn may contribute to a deeper comprehension of clinical reasoning skills. These results are consistent with findings from a study by White et al that suggested that medical students are less likely to “check out” of classroom studies when they are engaged in active learning.11 In particular, this study recommended shaping a curriculum to allow medical students to progress from solitary learners to collaborative learners in a way that is engaging. Case creation, by eliciting active student participation, is a key component of early preclinical instruction that prepares students for a team-centered workplace.

For both students and faculty, gains appear to be part of a trade-off in the area of student–faculty interactions, as reflected by focus group responses. Both student and faculty participants in the study reported that additional opportunities to interact during the session would have been valuable. Adult learning theory suggests that clinical reasoning can be enhanced by repeated, deliberate exposure to real cases. Moreover, educational theory suggests that the participation of an instructor can augment the value of an educational experience by immediately pointing out and discussing any errors in information, judgment, and reasoning.1 Thus, the design of our pilot study may have been enhanced by embedding additional opportunities for feedback throughout the session.

In their responses to the SFDP-26, students expressed additional reservations, such as limited use of visual aids and decrease in the ability to engage in learning outside of the classroom.8 However, this study presented the case creation method to students as a one-time activity, separate from the other CBL activities in their curriculum. We propose this method as an adjunct to, rather than a replacement for, current methods. We hypothesize that with more consistent use of this case creation method, in combination with traditional CBL and similar curricular activities, the use of visual aids and outside class learning may increase due to opportunities to make connections between topics in the case creation session and topics in the main curriculum.

Faculty members acknowledged that these first-year students had limited clinical experience, which in turn restricted the extent to which they could develop a high-quality patient case. Indeed, previous studies suggest that ideal learning cases are those that are based on real patients, as opposed to patients that are invented.1 However, we suggest that an early introduction to clinical reasoning and case generation still has educational value. Moreover, we would expect that with repeated engagement in the case creation process, students will be better trained to generate exemplary cases not only for their own understanding of content but also for the use of fellow students.

Limitations

Our small sample of ten first-year medical students from a single, private university may not accurately reflect the broader pre-clerkship student population – either at our own or at other institutions. Moreover, our two participating faculty members may not represent the wide range of clinical teaching styles present in various clinical instruction sessions. Thus, our study cohort may have biased both our quantitative and qualitative findings. However, the goal of our study was to determine feasibility of this method first, and then pursue additional opportunities to continue studying this learning modality. Another limitation is that we only evaluated clinical reasoning gains through focus group responses. Script concordance testing (SCT) is a well-known method that has been used to assess clinical reasoning skills.12 It has been shown to have good internal consistency reliability, although some studies have questioned its validity.13,14 Past studies have shown that SCT is useful for evaluating a learner’s clinical reasoning progress over time.15 We did not apply SCT in our study because our primary aim was to determine feasibility and student response to case creation as a curricular activity. Indeed, an important next step will be to examine changes in students’ SCT scores over each case creation session during a longitudinal analysis of the case creation method.

Conclusion

The results of our feasibility study suggest that student- initiated case creation can be an engaging technique for clinical reasoning instruction, may increase student ownership over content, encourage student-directed learning, and provide opportunities to explore nuances that distinguish various entities in a differential diagnosis. These gains appear to be accompanied by a trade-off, however, particularly in the area of student–faculty interaction, because students and faculty missed additional opportunities to interact during the session, which might have been valuable. The case creation method may have greater benefit later in the curriculum – for example in the second or third year of medical school rather than the first – when students have had more clinical experiences and knowledge to draw upon. Further research may focus on a more rigorous evaluation of this method and how it can be incorporated into clinical reasoning sessions in the pre-clerkship curriculum.

Table S1

Focus group questions

Faculty focus group questions

1. How did this case creation teaching experience compare to POM clinical reasoning sessions?

2. How prepared did you think the students felt for this case creation experience?

3. What challenges, if any, did you experience when leading this case creation exercise?

4. In what ways could this case creation approach be strengthened?

5. How did your experience with students in this case creation session compare to your interaction with students in POM clinical reasoning sessions?

6. At what point in your preclinical education do you think students would derive the most benefit by creating a case, rather than solving one?

7. Do you think there is more benefit to students if they create a case on their own rather than in a group setting?

8. Is there anything else you’d like to add?

9. Did you notice the students assign team roles? How did they do this?

10. Would inclusion of near-peer mentoring enhance this experience?

Student focus group questions

1. How did this clinical reasoning experience compare to your POM clinical reasoning sessions?

2. How prepared did you feel for this case creation experience?

3. What challenges, if any, did you experience during this case creation exercise?

4. In what ways could this case creation approach be strengthened?

5. How did your experience with faculty in this case creation session compare to your interaction with faculty in POM clinical reasoning sessions?

Abbreviation: POM, practice of medicine.