The Bloody Board Game: A Game-Based Approach for Learning High-Value Care Principles in the Setting of Anemia Diagnosis.

Introduction: With growing health care costs, high-value care is an increasingly important subject for medical training. Many resident and medical student curricula have incorporated lectures-based material about this topic. However, practical-type experiences are needed to refine critical-thinking skills essential for high-value care.
Methods: To provide such practical experiences, we developed an instructional game for resident-level education that incorporated cost-constraint-based approaches in the workup of anemia. To play the game, teams of learners were given patients with anemia of unknown cause. To pay for their diagnostic tests of choice, teams earned money by correctly answering internal medicine resident-level anemia questions. The first team to successfully work up and diagnose three patients won.
Results: Resident learners had very positive reviews of our game. As a team, groups of residents across all levels were able to develop cost-effective strategies for diagnosis. Our game also served as a resource for anemia education. Residents on average felt the game enhanced their ability to apply medical knowledge and clinical reasoning (M = 4.7 out of 5, where 5 = strongly agree), as well as high-value care (4.6), and should remain in the program for the high-value care curriculum (4.9). Discussion: Game-based learning provides a fun, orthogonal approach to learning critical-thinking skills used during anemia diagnostic patient workups. Although we did not quantify change in diagnostic test ordering, according to resident-learners, our high-value care game improved their ability to integrate cost-effective strategies into their practice of medicine.

Educational Objectives

By the end of the activity, participants will be able to:

Develop, create, and improve a stepwise process for decision-making while fostering teamwork/dynamics and communication skills.

Request laboratory exams that are needed and are relevant for the case, avoiding unnecessary expenses that could therefore be applied to real scenarios in health care.

Interpret clinical and laboratory clues in the appropriate clinic context to make the correct diagnosis.

Strengthen differential diagnosis skills related to anemias.

Introduction

In the setting of finite resources with rising health care costs,[1] education programs acknowledge a need for residents and medical students to learn high-value care,[2-4] the practice of medicine aimed at maximizing patient outcomes while balancing associated medical costs and harms.[5] Some residency programs have begun to incorporate high-value care in their curricula.[6] A major part of resident education utilizes a case-based lecture style,[7] focusing on the workup and hospital course of a patient. Game-based learning, the use of games as a form of training to engage learners while providing scenarios comparable to real-life situations such that skills and cognitive reasoning can be developed, is starting to be applied to health systems science.[8] In MedEdPORTAL, we found no formal game-based exercises aimed at strengthening high-value health care principles for residents or medical students. Towards the effort of teaching high-value health care principles while giving learners opportunities to practice high-value health care, we developed a game using case-based anemias. A game-based approach has been shown to be preferred by individuals in the typical resident age group.[9]

We designed our game around the differential diagnosis of anemias, since they are often diagnostically worked up using a shotgun-test approach through ordering many (sometimes expensive and unnecessary) tests. Knowledge related to anemias is also generally important to residents. A European study found that more than 50% of hospital admissions to an internal medical department had an associated diagnosis of anemia.[10] Hematological conditions make up more than 5% of the American Board of Internal Medicine's national certification exam.[11] A World Health Organization estimate in 2008 suggested that almost one-quarter of the world's population has a diagnosis of anemia.[12] A 2016 Centers for Disease Control and Prevention report found that the number of ambulatory visits with a primary diagnosis of anemia is 2.8 million[13] and the number of emergency department visits with a primary diagnosis of anemia is 526,000.[14]

Our game requires only a computer and a screen to project a slideshow. The game was developed with resident instruction in mind; however, medical students with some hematology knowledge could still benefit from it. Search results for high-value care on MedEdPORTAL returned 13 hits, none of which were game based. While search results for anemia returned 44 hits, several of which were team-based learning modules, none were game based. We feel that our game takes a novel approach to teaching high-value care at the resident level.

Methods

We designed a slideshow-based game (Appendix A) that could be incorporated into the biweekly didactic schedule for internal medicine residents, intending it to last between 60 and 90 minutes; the first 10 minutes of each session were dedicated to breaking into teams and providing instructions related to the game. During four separate didactic sessions (December 2019-January 2020), each led by a chief resident, around 20 residents broke into three teams. To encourage vertical teaching within teams, we intentionally created teams consisting of first-, second-, and third-year internal medicine residents. Each resident participated in only one session. Faculty did not participate in the lesson. We created the game with internal medicine residents of any level of clinical experience in mind, but residents with moderate clinical experience are likely to benefit the most.

Several days before the game was played, we printed out patient and instruction cards (Appendix B) on 8.5 × 11-inch cardstock, cut them into separate cards, and laminated them (not required). We also printed an Excel sheet containing the hypothetical patients (Appendix C) on 8.5 × 11-inch paper to serve as a resource for the chief resident leading the game. In a medium-sized conference room, about 15 minutes before the game began, we used a computer to project the slideshow (Appendix A) onto a monitor/screen. After the game, we administered an eight-question online evaluation form (Appendix D) to residents (71% response rate; 53 of 75 learners). We designed the postgame evaluation to gauge learner agreement with statements concerning clarity of instructions, organization, learning and applying high-value care, appropriateness to level of training, and keeping the game in the curriculum for future learners. In this survey, we collected postgraduate year so that we could stratify results according to resident level. There was also a free-text comment section at the end of the survey. Finally, we created a high-yield learning points handout (Appendix E, comparable to that of Silverio and Chen[15]).

Gameplay and Rules

The game was best played in a medium-sized conference room with chairs for each participant. We found breaking into three teams of six to seven residents worked well. We feel that the upper limit of this game would be four teams, with a maximum of seven residents per team. Each team congregated in a different part of the room while still maintaining visibility with the game leader (chief resident) and the projector screen.

Each team initially picked a card that had a number corresponding to a hypothetical patient with anemia of unknown cause. The game was turn based, with each 3-minute turn consisting of (1) the question phase, (2) the diagnostic workup phase (optional), and (3) the diagnostic establishment phase (optional). Teams earned money by correctly answering anemia questions in order to pay for diagnostic tests required for patient workup. The game was won by the first team to successfully work up and diagnose three patients.

The question phase

In a Jeopardy-style format, a team selected a question based on a category and value. Unlike other Jeopardy formats, only the team on turn could answer; that team was given 1 minute to discuss and provide an answer. If the team answered the question correctly, it earned money ($200–$600 per question) enabling it to purchase diagnostic information (cost of each diagnostic clue, e.g., patient history, complete blood count, iron studies, etc., was listed on clue cards contained in Appendix B). No money was lost for a wrong answer to the question (note that this was distinct from incorrectly diagnosing patients, for which money was lost; see below). To further encourage high-value care strategies, teams kept any unspent money after each turn and after each successful anemia diagnosis.

The diagnostic workup phase

After providing an answer, during the remainder of the 3 minutes, the team had the option to perform diagnostic tests using money already earned. Diagnostic tests could be done in parallel (test 1, test 2, test 3, result 1, result 2, result 3) or serially (test 1, result 1, test 2, result 2, test 3, result 3). Money not spent remained in the team's bank account for the following turn.

The diagnostic establishment phase

At the end of the 3-minute turn, teams could optionally choose to provide their anemic patient with a presumptive diagnosis. If the correct diagnosis was established, the team was given a new patient at the start of its next turn. However, if the medical team incorrectly selected a diagnosis, the anemic patient sued the medical team for $200 (i.e., the team lost $200; team totals could be negative) and sought medical care elsewhere (the team was given a new anemic patient at the start of the next turn). Unless a patient's diagnosis was correct, the game leader did not reveal any patient diagnosis until the end of the game. These rules were also listed on slide 2 of Appendix A, and rule cards to be given to each team were part of Appendix B.

Results

Seventy-five internal medicine residents, PGY 1-PGY3, participated once in our game across four sessions. Fifty-three responded to our evaluation (71% response rate). Residents were asked to rate the strength of their agreement with eight statements using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree).

The majority of residents strongly agreed with statements regarding the instructions being clear and the game being organized and interactive (M = 4.9, SD = 0.4). Residents strongly agreed with the statement that during the session, they learned how to apply high-value care (M = 4.6, SD = 0.7), as well as feeling that the game would help them apply high-value care principles in the future (M = 4.6, SD = 0.8). Importantly, only one resident had any disagreement with either high-value care statements (rating them as a 2 on the Likert scale). The majority of residents strongly agreed that the level of instruction matched their level (M = 4.8, SD = 0.6), indicating an appropriate level of complexity to our game; interestingly, PGY 2s and PGY 3s had stronger agreement with this statement, possibly suggesting high-value care surrounding anemia workup was more relevant to more senior internal medicine residents. (For more details, see the Figure.) Learners who completed the postgame evaluation remarked on how fun, informative, and interactive and what a great learning opportunity the game was, including the following comments:

Figure.

Postevaluation learner agreement with statements related to the learning session (n = 53). Top panel: agreement ratings independent of postgraduate year (PGY). Bottom panel: agreement ratings by PGY. Note that one anonymous responder did not indicate PGY and was dropped from the year calculations. Diamonds mark averages. In the boxplots, the center line represents the median; box limits, the upper and lower quartiles; and whiskers, 1.5 times the interquartile range.

“It was one of the most informative as well as most fun didactic sessions in my 3 years of residency.”

“Great way to deliver high-yield medical knowledge in an extremely interactive and fun way! Definitely would love to participate in more didactic sessions like this!”

“It was super interactive and we felt that we were involved in the learning process.”

“Excellent!! Really well thought out and planned, very interactive and educational.”

“Excellent way to educate residents on cost effective care. Also a great learning opportunity.”

“Excellent and fun game, great way to encourage learning and discussion!”

“Very innovative and novel way of learning. Most entertaining way to gain knowledge.”

“Fantastic initiative. I learned, I had fun with my peers, I incorporated new concepts. This game should be patented.”

“Hope this stays in the curriculum and is expanded to different subjects.”

Suggestions for improvement were mostly related to enhancing some of the Jeopardy questions, adding more chief complaints (more patients with undiagnosed conditions), and decreasing the team size.

Discussion

There is an estimated 20% waste in the medical field.[5] Educational programs have taken steps to combat physician-related health care costs through curricula incorporating high-value health care,[2-4] yet much teaching occurs through lecture formats rather than hands-on reinforcement. Our game has been designed with these high-value care principles in mind. It was well received by our residents. The game uniquely allows residents both to appreciate high-value care principles and to obtain hands-on experience of the utility of ordering tests only when testing results will directly affect their thought process. Through the incorporation of testing costs into game-based learning, we further reinforce medical decision-making when facing finite resources.

Our game demonstrates high-value care principles in the context of anemia, disorders commonly encountered by internal medicine residents.[10,16,17] By making teams earn credit through academic questions, our game also provides knowledge about anemia to learners. Our high-value, game-based approach could be adapted to other specialties by changing the question content, clinical disorder vignettes, and tests/costs associated with those disorders. Not only does this adaptability allow for game access by other residency specialty programs, it also permits use at the medical student level. Considerations are already being made at our program to adapt the game to other topics within the residency program.

Anecdotally, this was one of the most interactive sessions of our program. Learners were excited and engaged about the concept of constraining the testing done to identify a patient's disorder. Resident cohesion occurred across the levels of training. We observed stepwise clinical reasoning when listening to small-group discussions. Perhaps most exciting, we witnessed evidence of high-value care critical thinking. Inevitably, teams would begin by getting tests in parallel, exhausting their in-game cash. During opposing team turns, residents actively reformulated their strategies towards cost-effective diagnosis. We observed most teams eventually switching to a serial testing method, allowing for one test's result to inform the choice of the next test to order, a hallmark of high-value care.[5,18] Suggestions for improvement typically involved adding more patients of undiagnosed conditions and decreasing the team size. Thus, in future sessions, we plan to decrease team sizes from six to seven learners to four to five learners per team. We did not receive any negative feedback related to the structure of the game.

We have identified two other future changes related to our game. The first is to use real hospital costs for each diagnostic test available in the game. We initially chose diagnostic clue prices to reflect diagnostic information quality (e.g., hemoglobin electrophoresis was made expensive since it can provide a diagnosis in the correct circumstance). We feel this price structure encourages teams to think critically about which information is still required to confidently make a diagnosis. Our goal was to prevent residents from reflexively picking unnecessarily elaborate tests when not warranted (e.g., antinuclear antibody). However, diagnostic tests more reflective of real pricing likely provide added learning and appreciation for learners not aware of the costs associated with a test. The second change is to perform both pre- and posttest evaluations to better understand changes in participants' (1) anemia knowledge, (2) understanding of costs of anemia diagnostic tests, and (3) value placed on high-value care. When designing this game to teach high-value care in the context of anemia, our target audience was PGY 1, PGY 2, and PGY 3 internal medicine residents; however, despite us not testing it directly with medical students, we feel that the knowledge required is reasonable for medical students with some instruction on the topic. Given that medical students have less clinical experience, gains are likely to be related to anemia knowledge rather than appreciation for high-value care.

We acknowledge several limitations to our game. First, the number of learners able to be accommodated per session is comparatively lower than the capacity of a lecture. Also, in the first session of the game, we noticed that 60 minutes, the time allowed for most didactic periods, was insufficient. In subsequent periods, we allocated instead 90 minutes, which allowed for comfortable rule learning and game completion. In order to account for any learning opportunities missed by learners in the 60-minute session, we generated a high-yield learning points handout (Appendix E). Next, for our game to be successful, enthusiasm from learners is required. We feel the structure of the game encourages healthy competitiveness amongst teams, and in our case, lack of enthusiasm was not a problem in any of the four sessions we held. For maximum gains, the game requires prior anemia knowledge, and therefore, it could go nicely with anemia and high-value care lectures before or after. Finally, another limitation related to high-value care in general is difficulty generating objective measures to track changes in resident high-value care behavior.

In summary, we created an anemia game incorporatinig high-value care principles in an entertaining and resident-engaging manner. To give appreciation for the cost-value trade-off of various tests, each test in the game has associated costs, forcing participants to think critically about the highest yield test given circumstances around fictional patients. We include all materials necessary, such that only a printer and a computer are required to play. Residents overwhelmingly agreed that the game was engaging and conveyed high-value care principles. Future study is needed to determine game-influenced changes in high-value care practices habits of participants.

Anemia PowerPoint.ppt

Game Cards.pdf

Facilitators Sheet.xlsx

Postevaluation Form.pdf

Learning Points.pdf

All appendices are peer reviewed as integral parts of the Original Publication.