Septris: a novel, mobile, online, simulation game that improves sepsis recognition and management.

PROBLEM: Annually affecting over 18 million people worldwide, sepsis is common, deadly, and costly. Despite significant effort by the Surviving Sepsis Campaign and other initiatives, sepsis remains underrecognized and undertreated. APPROACH: Research indicates that educating providers may improve sepsis diagnosis and treatment; thus, the Stanford School of Medicine has developed a mobile-accessible, case-based, online game entitled Septris (http://med.stanford.edu/septris/). Septris, launched online worldwide in December 2011, takes an innovative approach to teaching early sepsis identification and evidence-based management. The free gaming platform leverages the massive expansion over the past decade of smartphones and the popularity of noneducational gaming.The authors sought to assess the game's dissemination and its impact on learners' sepsis-related knowledge, skills, and attitudes. In 2012, the authors trained Stanford pregraduate (clerkship) and postgraduate (resident) medical learners (n = 156) in sepsis diagnosis and evidence-based practices via 20 minutes of self-directed game play with Septris. The authors administered pre- and posttests. OUTCOMES: By October 2014, Septris garnered over 61,000 visits worldwide. After playing Septris, both pre- and postgraduate groups improved their knowledge on written testing in recognizing and managing sepsis (P < .001). Retrospective self-reporting on their ability to identify and manage sepsis also improved (P < .001). Over 85% of learners reported that they would or would maybe recommend Septris. NEXT STEPS: Future evaluation of Septris should assess its effectiveness among different providers, resource settings, and cultures; generate information about how different learners make clinical decisions; and evaluate the correlation of game scores with sepsis knowledge.

Problem

Sepsis is a global epidemic, as well as a fertile area for educational initiatives and systems-based improvements. Over the past decade, interventions focused on early recognition and management have demonstrated improved outcomes in severe sepsis and septic shock.[1] Despite these advances, the worldwide incidence of sepsis continues to rise. Approximately 1.8 million cases of diagnosed sepsis are reported worldwide each year, but research indicates that this number is low because of poor reporting from many geographic areas.[2] If data from the U.S. incidence of sepsis are generalized to the worldwide scale, there are approximately 18 million new cases per year.[2] Mortality rates remain high (25%–50%).[2] Sepsis-focused educational curricula directed at physicians have shown promising results in improving patient care.[3]

Approach

Creating Septris (sepsis + Tetris)

The name Septris is a portmanteau, created by combining the word sepsis with the word Tetris, the name of a popular video game from the 1980s. Septris (http://med.stanford.edu/septris/) represents a novel approach to medical education; the Internet-based game leverages the near-ubiquitous presence of mobile devices (e.g., smartphones, tablets) and the widespread popularity of gaming by placing a virtual patient simulator at everyone’s fingertips. This simulation-based modality—in contrast to traditional medical education immersive modalities (e.g., procedural simulation training)—uses a platform that is free, accessible, and easily disseminated, including throughout the developing world.

User experience.

Septris provides a case-based, interactive learning environment for medical trainees. The Septris learning objectives comprise the following: (1) classify the epidemiology of sepsis syndrome; (2) differentiate forms of sepsis syndromes (simple, severe, and septic shock); (3) integrate evidence-based practices, clinical expertise, and diagnostic test results for early identification and optimal management of septic states; (4) describe priority actions for triage; and (5) implement early goal-directed therapies and source control for septic patients along the continuum of care. Septris players can access these learning objectives, as well as tips for treating sepsis and source material from the medical literature, through the Septris platform before beginning the game. Key learning points include the following: the time-dependent nature of sepsis management, the importance of using early supportive care (e.g., IV fluids), the importance of early administration of antibiotics, and the need for aggressive source control (which in a lower-resource setting may involve referral/transfer of the patient). All learning points are applicable to nations with either highly developed or still-developing health care systems.

In Septris, players (learners) manage two simulated patients concurrently (see Figure 1). The height of the patient icon along the vertical axis represents each patient’s health status: The bottom of the screen corresponds to death, and the top of the screen corresponds to complete recovery. The learner or gamer is presented with each patient’s brief medical history, vital signs, and physical examination results. The patient icons fall towards the bottom of the screen at a rate reflective of the severity of their sepsis physiology.

Figure 1

A black and white version of a screen shot presenting a patient’s (Matt’s) case history and physical exam results in the game Septris. Septris is a mobile-accessible, case-based, online game (http://med.stanford.edu/septris/) launched worldwide in 2011 that takes an innovative approach to teaching early sepsis identification and evidence-based management. Septris allows learners to order labs, radiology images, various treatments, consults, and cultures to cure a patient’s sepsis. If players select appropriate interventions and tests for a patient, the avatar will rise to the top of the screen. If players fail to select the appropriate treatments and test, the patient’s avatar falls to the bottom of the screen. A simulated patient who reaches the very bottom dies; a simulated patient who reaches the very top is cured, and the player earns bonus points.

Learners may choose from a variety of diagnostic tests and treatments. If the learner makes correct treatment choices based on established guidelines, the patient’s icon rises as the patient’s health improves, and the learner receives bonus points. If the patient icon reaches the top of the screen, the patient is considered “cured,” and a new patient is presented. If the learner makes inappropriate treatment choices, the patient’s icon drops as the patient’s condition worsens. If the patient drops to the bottom of the screen, he or she “dies” and is replaced by another patient.

In addition to experiential learning through simulated patient care, Septris provides learners with real-time feedback. Throughout the game the “attending,” Dr. Sepsis, provides feedback based on the testing and treatment choices players or learners make. Patient care within Septris is based on current best practices as outlined by the Surviving Sepsis Campaign[1] and other professional initiatives and organizations. Patients cannot be completely cured until the players have ordered all the appropriate resuscitative procedures, including labs, imaging studies, antibiotics, IV fluids, consults, and procedural interventions as necessary. The game calculates and records a cumulative score for each player, allowing for learners to compare their progress with other players.

Design rationale.

A multidisciplinary group at Stanford School of Medicine designed and pilot-tested Septris from 2011 to 2012. This group included physicians from internal medicine, the emergency department, surgery, and the intensive care unit, as well as nurses and experts in medical education technology. The interprofessional nature of the group promoted the development of a medically diverse set of simulated patients.

To increase the educational value of Septris, the group focused a significant amount of attention on design elements that would allow patient care in Septris to mimic patient care in actual practice as much as possible. In particular, learners playing Septris must handle complex patient care scenarios concurrently; make decisions regarding triage, testing, and treatment; properly sequence testing and treatment; and observe and react to the consequences of both correct and incorrect decisions. Further, the game’s designers assigned a relative strength to treatments in Septris in a way that resembles the real-life treatment of sepsis (see Table 1 and Figure 1).

Table 1

Comparison of Real-World Patient Care and Septrisa Simulated Patient Care

We believe the Septris platform has several strengths as an educational tool. Septris relies on a combination of learning techniques such as simulation, multiple-choice questions (interspersed throughout the patient simulation), real-time individualized feedback (offered throughout the game and given as answers to “pop up” questions), and the didactic presentation of evidence-based information (provided as “tips” prior to the activity). Because Septris applies best practice guidelines, eligible health care providers can receive continuing medical education credit on completion of the game. The game designers created the platform with expandability in mind, allowing for continual evolution and addition of educational content in the form of new cases, new treatments/diagnostics, more directed feedback, and access to new guidelines/resources.

Harnessing emerging trends and innovations

Gamification.

Septris uses the principles of gamification, or the process of applying gaming principles (competition, rewards, enjoyment) to a nongame endeavor.[4] Gamification principles have been used in a variety of educational contexts to promote knowledge retention and learner pleasure.[5] Previous investigators have addressed the effect of video games (including “serious games,” or games designed for a particular educational purpose) on learning, though the resulting publications have typically been either descriptive or speculative; very few randomized or controlled trials have evaluated the effectiveness of gaming on actual learning.[6] The literature includes a number of medical educational gaming modalities (ranging from playing cards to video games) that target a wide variety of medical conditions; however, the utility of these games has not yet been determined in large randomized controlled trials.[7]

Technology.

The designers of Septris applied emerging technologies, including mobile technologies (which have become a platform for both mobile patient care and mobile education[8]), to increase the availability and accessibility of the game (via, for example, smartphones and tablets). Specifically, the group created Septris using JavaScript, HTML5, CSS3, and XML Web technologies to ensure cross-platform compatibility and to reach as wide an audience as possible. Septris runs best on iPad or Android tablets. On desktop, Septris requires the latest version of Firefox, Google Chrome, or Apple Safari with pop-ups allowed. Applications developed for patient care, patient education, communication, research, and just-in-time learning have become an increasingly important component of medical practice, but the usefulness of many of these resources has yet to be examined rigorously.[8]

Evaluating Septris

Given the lack of research on learning modalities that incorporate both mobile functionality and gamification, we wanted to examine not only the dissemination of Septris, but also how such a platform might improve learner knowledge, skills, and attitudes around sepsis recognition and management. This study serves as a first step in exploring these questions.

In early January 2012, we piloted Septris with learners at two different levels from Stanford School of Medicine. The learners provided feedback for improving both Septris (e.g., format, time to complete, tips) and the tools we developed to assess its effectiveness (e.g., question clarity, content relevance). Five clerkship students completed quizzes immediately before and after playing Septris for 20 minutes. The quizzes, which contained multiple-choice and matching questions, tested learners’ knowledge related to sepsis identification and management. We then piloted the assessment tools (in addition to the game itself) with 17 internal medicine residents using an 18-item pretest and a 7-item posttest. On the basis of learner feedback, Septris was modified and the assessments were refined to a 10-question pretest and 15-question posttest. The posttest comprises all 10 pretest questions, 4 retrospective pre–post questions, and 1 question about user satisfaction (see below).

Evaluation methods

We measured the impact of Septris in four areas: (1) dissemination, (2) learner’s knowledge related to sepsis, (3) learners’ self-reported knowledge, skills, and attitudes related to sepsis, and (4) learner satisfaction with the game, which align with the first two levels of Kirkpatrick’s evaluation model (Learner Satisfaction and Learner Attitudes and Knowledge).[9]

Measuring worldwide game dissemination.

To track engagement and audience reach, we have integrated various features of Google Analytics (GA), such as page tracking and event tracking throughout the Septris platform. By using GA, we are able to track the following: the number (as measured by cookies) of visits to Septris, starts of the game, and completed games; data on users’ actions within Septris in aggregate, on learners’ hardware (e.g., browser type, screen resolution, operating system), and on learners’ demographics (e.g., location, Internet service provider, language); and other important information such as referrers, bounce rates, and average visit durations.

Measuring impact on learner knowledge, skills and attitudes related to sepsis, and satisfaction.

We conducted an exploratory study of Septris at Stanford School of Medicine in June and July 2012 with pregraduate learners (61 second-year medical students transitioning to third-year clerkships) and 95 postgraduate learners (46 internal medicine interns; 12 emergency medicine interns; 37 surgery residents). We assessed the impact of Septris on learners’ knowledge related to sepsis through online pre- and postgame quizzes that the learners took immediately before and after playing the game for 20 minutes. The multiple-choice pregame quiz consisted of 10 items (each with four or five answer choices) that mapped onto the five game objectives. The postgame assessment consisted of 15 items, including 10 quiz questions from the prequiz, as well as 4 items that measured on a scale of 1 (poor) to 5 (excellent) the game’s impact on learners’ self-reported, retrospective knowledge, skills, and attitudes related to sepsis. We included the 4 retrospective pre- and postgame self-assessment items because research has shown that these can provide more sensitive and valid measures of the effect of medical education training on attitudes than traditional pre–post intervention self-assessments.[10] We evaluated participant satisfaction with the game via their reported likelihood of recommending the game after playing it (on a scale of 1 [yes] to 3 [no]).

We analyzed differences using mean Student one-tailed t test for the following pairings: pre and post ratings, and retrospective pre and post ratings. We performed all statistical analysis using Excel (2010 Version, Microsoft, Redmond, Washington). We collected online data through Qualtrics (Provo, Utah). Our study was reviewed and determined exempt from human subjects research by the Stanford School of Medicine institutional review board.

Outcomes

Septris had a positive effect in all four areas we examined: (1) dissemination, (2) learner’s knowledge related to sepsis, (3) learners’ self-reported knowledge, skills, and attitudes related to sepsis, and (4) learner satisfaction with the game.

Since launching Septris in December 2011, the site has been visited over 61,000 times as of October 2014. GA has registered over 35,500 starts of the activity and 4,500 (13%) completions. Approximately 60% of traffic has come from the United States, followed by about 6% from both Australia and the United Kingdom, and approximately 4% from Brazil and Canada (see Table 2). Fifty-five percent of visits to Septris have been direct/organic, whereas 45% of visits have come from referrals (e.g., social media, blogs, traditional Web media). The most popular site for referrals has been Facebook.com, which has been responsible for over 25% of all referral traffic.

Table 2

Number of Visits to Septrisa by Country

Significant improvements (P < .001) in sepsis knowledge were demonstrated through changes in pre- and postquiz scores for all learners: Overall average quiz scores increased from 5.85 (standard deviation [SD] = 2.31) to 6.94 (SD = 2.88) out of 10 possible points. The average quiz score of pregraduate learners increased from 5.16 (SD = 2.34) to 6.49 (SD = 3.72), and postgraduate learners’ average quiz scores increased from 6.29 (SD = 1.81) to 7.23 (SD = 2.16).

Changes in retrospective, self-reported pre- and postgame ratings provide evidence of the positive impact of Septris. Data showed a significant increase (P < .001) for all learners in both (1) their self-reported knowledge, skills, and attitudes related to sepsis (average ratings increased from 2.13 [SD = 0.80] to 3.02 [SD = 0.77]) and (2) their self-assessed ability to identify and manage sepsis (average ratings increased from 2.05 [SD = 0.95] to 2.83 [SD = 0.72]). Pregraduate learners’ self-assessed ability to identify and to manage sepsis increased from overall average scores of, respectively, 1.57 (SD = 0.38) to 2.59 (SD = 0.65) and 1.48 (SD = 0.45) to 2.41 (SD = 0.45). Postgraduate learners’ ability to identify and to manage sepsis increased from overall average scores of, respectively, 2.47 (SD = 0.75) to 3.29 (SD = 0.67) and 2.41 (SD = 0.93) to 3.09 (SD = 0.71).

Learners’ satisfaction with the game, as measured by the likelihood that they will recommend Septris after playing it, was high; 86.5% answered yes or maybe when asked if they would recommend it to a colleague.

Next Steps

To our knowledge, Septris is the first medical education tool of its kind. The mobile, online game—created in response to the global sepsis epidemic—was successfully disseminated worldwide, and it improved local learners’ knowledge, skills, and attitudes related to sepsis identification and management.

The assessment tools used were designed to measure the first two steps of Kirkpatrick’s learning evaluation theory9: (1) satisfaction or reaction, as reported through likelihood to recommend; and (2) learning through increase in knowledge score after simulation activity. Application of results to behavior and patient outcomes, the third and fourth step in Kirkpatrick’s learning evaluation theory, entail evaluating changes in users’ speed and accuracy when managing septic patients (either simulated or actual) and determining if there is a change in patient outcomes. We will continue to use the Kirkpatrick framework to measure outcomes in higher levels of impact.

The results of the evaluation may not be generalizable. Our evaluation of Septris was carried out with medical students and residents across three departments at a single, private institution in the United States. Although utilization and personal communications suggest that Septris is well received among different disciplines, learning levels, populations, and nations, it is not clear that Septris would be as effective in these cohorts. In the future, evaluation of Septris as a teaching tool should assess its effectiveness among different resource settings, cultures, and providers. We are in the process of expanding Septris to include more nurse-centric sepsis education, and we are now offering it more widely through the Surviving Sepsis Campaign Collaborative E-Community Web site.

Other efforts to build off of our findings are already under way. These include expansion of the mobile gaming platform to other medical conditions and translating the game into other languages. A Stanford group has developed SICKO (Surgical Improvement of Clinical Knowledge Operations) using the Septris platform to teach and assess surgical decision making. Current collaborators in England and Germany will further distribute Septris through translation, and we hope to collaborate with other universities around the world to further disseminate the game and study its effectiveness.

The quiz was not validated for reliability. Future assessments should occur after validation of the knowledge questions. Knowledge retention was not tested, and it would be beneficial to determine both the frequency with which learners returned to play Septris and their knowledge retention over time, as measured by game score(s) and/or quiz. We have now incorporated the ability to track learners who return to the game by capturing their internet protocol or IP address when they submit their score, and our future Septris platform will allow us to measure the order of choices that players make, which will give us better insight into learner decision making. To explore questions regarding the effectiveness of high- versus low-fidelity simulation, Septris could be studied in a randomized control design.

Qualitative studies may provide insights into the aspects of the game that learners find most helpful or enjoyable and why some who begin the game do not finish it. Future studies may provide information about how different learners make clinical decisions and how well game scores correlate with sepsis knowledge.

As this study demonstrates, gamification has significant potential for augmenting medical education. Further study is required to investigate how mobile games may contribute to other methods of medical instruction such as didactic lectures, directed reading, and teaching rounds. We believe that the advantages of these mobile games (e.g., interactive, simulation based, immediate feedback) will make them very effective adjunctive tools.

Acknowledgments: The authors wish to express their gratitude to the following for their contributions to the development of Septris and the pilot studies: Eileen Pummer, RN, Brian Tobin, MEd, Aviva Regev, Janet Leung, MD, James Laird, Jonathan Tatum, Glenn Zephier, Pauline Brutlag, and Andy Estrin. The authors also gratefully acknowledge the support of a Pfizer continuing medical education grant and the Stanford University Quality Improvement Department.