Virtual Patient Simulation in Continuing Education: Improving the Use of Guideline-Directed Care in Venous Thromboembolism Treatment.

Results of a CME-certified activity completed by a total of 986 cardiologists and 783 haematologists-oncologists (haem-oncs) from around the world were examined to determine whether virtual patient simulation could improve decision-making and performance within the simulation related to patient evaluation, tailoring anticoagulant therapy, and patient management to improve adherence using patient-centred care strategies. Results showed a significant overall impact of education from pre- to post-clinical guidance (CG) on correct decisions made in both cases for cardiologists, with a relative improvement of 22% for Case 1 (45% pre- to 55% post-CG, n = 475, t(474) = 14.12, P<.001, Cohen's d =.46) and 19% for Case 2 (62% pre- to 74% post-CG, n = 245, t(244) = 11.95, P<.001, Cohen's d =.59). Impact also was seen for haem-oncs, with a relative improvement of 27% for Case 1 (45% pre- to 57% post-CG, n = 280, t(279) = 11.91, P <.001, Cohen's d =.60) and 19% for Case 2 (63% pre- to 75% post-CG, n = 147, t(146) = 9.52, P <.001, Cohen's d =.58). Virtual patient simulation improved cardiologists' and haem-oncs management of patients with pulmonary embolism in a simulated environment.

Introduction

Venous thromboembolism (VTE), which consists principally of deep vein thrombosis (DVT) and pulmonary embolism (PE), is a common cause of morbidity and mortality. Healthcare providers encounter DVT or PE in a variety of settings and clinical scenarios (e.g., acute vs chronic, provoked vs unprovoked, and cancer related), all of which have different treatment algorithms [1]. While there is strong evidence that the use of non-vitamin K antagonist oral anticoagulants (NOACs) have benefits for treatment of VTE, continuing medical education (CME) is needed to address persistent clinical practice gaps that result in suboptimal use of these agents.

Among these gaps, physicians who care for patients with VTE lack confidence in selecting appropriate anticoagulation therapy based on a patient’s risk for PE and incorporating NOACs into treatment strategies to optimise patient outcomes [2-5]. Second, physicians have misperceptions regarding the need to monitor adherence in patients who are using NOACs [3,6]. And third, physicians who manage anticoagulant therapy have limited knowledge of the latest real-world data on the use of NOACs for prevention and treatment of VTE and emerging clinical applications for NOACs, including cancer-associated thrombosis [7,8].

CME activities are needed to educate cardiologists and haematologists-oncologists (haem-oncs) and to address clinical practice gaps to improve patient outcomes. A Medscape virtual patient simulation (VPS) platform, known as MedSims, was chosen as the educational modality because it was expected that it would address the identified clinical practice gaps. Cardiologists and haem-oncs who treat patients with VTE are challenged to stratify patient risk for PE to support confident selection of appropriate anticoagulation therapy to optimise outcomes (gap 1); monitor patients to gauge adherence to therapy (gap 2); and apply the latest data on the use of NOACs in patients with cancer (gap 3). We sought to determine whether VPS could improve decision-making and performance within the simulation, particularly as they relate to patient evaluation (risk stratification), tailoring anticoagulant therapy (applied use of data based on patient characteristics), and patient adherence (monitoring). Because simulation is associated with real-world behaviour – by one definition, it is “a bridge between classroom learning and real-life clinical experience” [9] – it was expected that if we saw improvements in the simulation, they would translate to real-world practice.

At its essence, simulation in education follows a theoretical framework of having the freedom to explore, make mistakes, and learn from them. Simulation in healthcare is defined as an educational modality that replaces or amplifies real patient experiences with scenarios that replicate real health encounters [10].

VPS simulates a patient-healthcare provider visit and gives learners the perspective of the provider who is listening to a patient and then making decisions about patient assessment and care via a simulated electronic health record (EHR). Direct process and outcome feedback on learners’ decisions and actions concerning each individual patient case, combined with opportunities to review decisions and the consequences of those decisions, allow learners to make real-time clinical course corrections based on formative feedback and knowledge transfer (e.g., a review of literature in the case of the current study’s intervention). This process of review and reflection in response to feedback supports behaviour change by building a repertoire of patient encounters and clinical problem solving, which learners can transfer from virtual to real clinical settings [11,12].

As part of a trend towards the use of simulation, modalities such as online clinical case simulations and VPS have emerged as appealing educational resources. In 2007, only 24% of medical schools in the USA and Canada were using virtual patients in their curricula [13]. By 2016, an Aquifer (then MedU) collection of virtual patients was in use at approximately 75% (130) of those medical schools [14]. Furthermore, VPS is increasingly endorsed as an evidence-based, interactive learning format in continuing education for healthcare professionals [15].

Effective VPS is characterised by immersive, credible, case-based, situated learning activities that address the complexity of specific clinical domains and support transfer of knowledge and skills beyond the learning situation. Learners encounter virtual patients in an adaptive, controlled, and safe environment that is designed to provide an authentic approximation of clinical practice. This environment allows learners to assume the role of an active protagonist who is treating a real patient – with real concerns and issues – and supports experiential learning by giving health professionals control over their pace of learning. A well-designed VPS requires true-to-practice, open-ended treatment decisions that allow learners to think critically about their actions and to apply their knowledge in point-of-care situations. Treatment decisions that replicate deliberate practice in a clinical environment allow us to gain objective and in-depth insight into clinician practice behaviour and treatment preferences. In the process, data regarding decisions are collected, allowing for robust evaluation of the educational programme.

A large body of research consistently associates simulation-based healthcare education, such as VPS, with better outcomes for knowledge, skill, behaviour change, speed of learning, and better long-term retention [10, 16–22]. For example, Burgon and colleagues [22] found that in a quasi-controlled experiment, in which VPS was part of the intervention with an accountable care organisation in the USA, there was a 27% improvement in evidence-based quality scores and a 55% reduction in unneeded testing in the patient simulations. Those improvements correlated with improvements in real-world quality measures and were greater than those of the quasi-control group [22].

In summary, VPS provides a customised learner experience, in which targeted formative and summative feedback are provided to promote effective learning. This, coupled with substantive detailed data collection of all actions made or omitted during the course of a simulation, creates a highly stylised learning intervention.

Materials and Methods

Ethical Considerations

This study was exempt from institutional review board approval as it is research involving normal education practices; it is, therefore, exempt under 45 CRF 46.104(d)(1) according to the US Department of Health and Human Services [23].

Setting

“Complex Cases in Thromboembolic Disease”, a CME-certified activity, was offered online to all Medscape.com members starting 16 December 2017, and was valid for credit for 1 year [https://www.medscape.org/viewarticle/885454]. Credit was available for a maximum of 1.25 AMA PRA Category 1 Credits™.

Intervention and Its Participants

The goal of this activity was to build cardiologists’ and haem-oncs’ confidence in applying strategies for individualised treatment of patients with PE and to ensure that best practices are applied across a variety of clinical presentations. The learning objectives were as follows:

Upon completion of this activity, participants will demonstrate improved performance associated with:

Performing appropriate workup for the risk stratification of patients with PE (aligning with gap 1)

Tailoring anticoagulant therapy in patients with PE (aligning with gap 3)

Selecting appropriate patient-centred care strategies to improve adherence to long-term anticoagulant therapy (aligning with gap 2)

The MedSims VPS is an education format that replicates a physician learner’s experience of treating patients and making point-of-care decisions. The platform captures all learner decisions and their rationales. Two cases were in this intervention: Case 1 focused on a patient with newly diagnosed PE; Case 2 focused on a patient with PE and cancer, who was on anticoagulant therapy (not a NOAC). Each case would take approximately 30 to 45 minutes to complete (Figure 1). There were 18 decision points in Case 1 and 7 in Case 2 (see Table 1 for case descriptions). The intervention was open to any registered Medscape member. Upon starting the activity, participants could select which case they wanted to complete, with the option of completing both cases. The case started with a video of a patient discussing their condition and history in a clinical setting.

Figure 1.

Patient Simulation

Table 1.

Case descriptions

Case	Description
#1 Doreen	History of Present IllnessDoreen is an 85-year-old woman who has been seen in this practice for >10 years. She came in early this morning accompanied by her daughter with no appointment complaining of right-sided back pain. She is concerned about the pain, which she rates as 3 to 5 out of 10, but she does not wish to go to the hospital. She says the pain began about 3 days ago and gets worse with inspiration. She does note a dry cough occasionally with deep inspiration. She denies dyspnoea, fever, or syncope. She has no history of heart disease, cancer, or asthma. She does note that her left leg has been moderately painful and swollen for the past week. No recent falls or injuries. Past medical history: hypertension (controlled with losartan) and chronic kidney disease Surgical history: cataract removal (both eyes) 18 months ago Current medications: losartan 50 mg once daily Review of systems: Negative except as above
#2 Clem	Clem is a 65-year-old man who was diagnosed with colonic adenocarcinoma 18 months ago after he noticed bloody stools. Colonoscopy showed a nonobstructing mass in the descending colon. About 13 months ago, he underwent partial colectomy. Preoperative computed tomography (CT) scans of the chest/abdomen/pelvis showed no evidence of metastatic disease. Pathology showed invasion into the subserosa with no evidence of cancer in the resected regional lymph nodes (T3, N0, M0). Margins were negative and no high-risk features were identified. Clem was followed thereafter with periodic imaging and labs, and 7 months ago, CT scans showed metastatic disease to the liver and lungs. He has been receiving systemic chemotherapy (FOLFOX with bevacizumab) since that time and undergoing CT imaging every 3 months while on chemotherapy for assessment of treatment response.Six months ago, Clem developed chest pain and dyspnoea and went to the emergency department. There, CT angiography was performed that showed bilateral lower lobe segmental pulmonary emboli. Since then he has been receiving anticoagulant therapy with low molecular weight heparin (LMWH). His dyspnoea improved within a few weeks of beginning treatment and he has no new symptoms today. Clem is having difficulty with his anticoagulation therapy: he does not like needles, hates injecting himself, and is tired of the bruising on his abdomen at the injection sites. He would like to stop therapy today if that is possible. He has some neuropathy from FOLFOX but it is quite minimal, disappears during his week off, and does not interfere with functioning. Clem’s most recent CT, in addition to showing stable or smaller metastatic lesions, showed near-resolution of the previously visualised pulmonary emboli and no new large pulmonary embolus. Past medical history: colorectal cancer and hypertension Surgical history: partial colectomy 13 months ago Current medications: enalapril 5 mg twice daily, FOLFOX (leucovorin calcium, 5-fluorouracil, oxaliplatin) with bevacizumab every 2 weeks, enoxaparin 80 mg subcutaneously twice daily Review of systems: negative except as above

The participants then interacted with the MedSims VPS platform, which simulates an EHR system. Participants make orders and receive and review test results. Clinical guidance (CG), which provides evidence-based information after selection(s) are made, serves as direct education to the participant. Guidance may include results from clinical trials, descriptions of tests, or the opinion of a leader in the clinical area in terms of their own experience and preference in the field. Upon reaching the end of a case, the participant may review all decisions made as well as important decisions omitted and change them if desired. There is also a literature review available to consult prior to finalising decisions. In this instance, the educational design is oriented to both formative and summative feedback; it can serve as best practice reinforcement or as a vehicle for behaviour change.

Data were collected from launch (16 December 2017) to programme expiration (16 December 2018). A total of 986 Ex-US cardiologists and 783 haem-oncs were learners in this online activity. Learners view the content but may not complete or make decisions in the activity. The completion rate for the first case was 54% for cardiologists and 40% for haem-oncs and 67% for cardiologists and 47% for haem-oncs for the second case. The analysis in this study includes all cardiologists (n = 475 in Case 1 and n = 245 in Case 2) and all haem-oncs (n = 280 Case 1 and n = 147 Case 2) who completed all decision points and content. Within the analytic sample, 40% were practising in Europe, 19% in Asia, 18% in North Africa/Western Asia, 14% in Latin America, 4% in Australia and New Zealand, and 5% in other countries.

Outcomes Measures

Two to 13 decision points indicated each learning objective. The pre-educational guidance (pre-CG) and the post-educational guidance (post-CG) decisions were documented for each participant. Decision points assessed with their corresponding learning objectives are in Tables 2 and 3.

Table 2.

Case 1 results by learning objective and decision points

Speciality	Learning Objective (LO)Decision Point	% Correct Pre-CG	% Correct Post-CG	Test Statistica (df)	P Value
Cardiologists	LO1 – Performing appropriate workup for the risk stratification of patients with PE	48%	56%	10.62 (474)	<.001
Cardiologists	B-type Natriuretic Peptide (BNP)	45%	53%	37.10 (474)	<.001
Cardiologists	Chemistry Screen	51%	60%	38.10 (474)	<.001
Cardiologists	Coagulation Studies	51%	61%	45.08 (474)	<.001
Cardiologists	Complete Blood Count (CBC) – Basic	54%	61%	31.11 (474)	<.001
Cardiologists	Compression Ultrasound Study of Leg Veins	67%	71%	13.24 (474)	<.001
Cardiologists	CrCl (Estimated Creatinine Clearance)	35%	49%	66.06 (474)	<.001
Cardiologists	Electrocardiogram (ECG)	54%	61%	27.13 (474)	<.001
Cardiologists	Estimated Glomerular Filtration Rate (eGFR)	30%	47%	76.05 (474)	<.001
Cardiologists	Oxygen Saturation	49%	56%	30.12 (474)	<.001
Cardiologists	Pulmonary Multidetector CT Angiography (MDCTA)	59%	62%	12.25 (474)	<.001
Cardiologists	Transthoracic Echocardiography (TTE)	44%	53%	38.10 (474)	<.001
Cardiologists	Troponin	41%	49%	35.10 (474)	<.001
Cardiologists	LO2 – Tailoring anticoagulant therapy in patients with PE	39%	51%	16.47 (474)	<.001
Cardiologists	Oral anticoagulation therapy	31%	63%	87.04 (474)	<.001
Cardiologists	warfarin	2%	5%	9.31 (474)	<.01
Cardiologists	LO3 – Selecting appropriate patient-centred care strategies to improve adherence to long-term anticoagulant therapy	38%	52%	9.69 (474)	<.001
Cardiologists	First Follow-up Visit	36%	50%	63.06 (474)	<.001
Cardiologists	Patient Education and Counselling	40%	53%	61.06 (474)	<.001
Haem-oncs	LO1 – Performing appropriate workup for the risk stratification of patients with PE	48%	59%	9.55 (279)	<.001
Haem-oncs	B-type Natriuretic Peptide (BNP)	38%	53%	39.09 (279)	<.001
Haem-oncs	Chemistry Screen	52%	64%	30.12 (279)	<.001
Haem-oncs	Coagulation Studies	66%	75%	22.15 (279)	<.001
Haem-oncs	Complete Blood Count (CBC) – Basic	64%	73%	21.16 (279)	<.001
Haem-oncs	Compression Ultrasound Study of Leg Veins	74%	79%	9.31 (279)	<.01
Haem-oncs	CrCl (Estimated Creatinine Clearance)	34%	53%	50.07 (279)	<.001
Haem-oncs	Electrocardiogram (ECG)	51%	60%	24.14 (279)	<.001
Haem-oncs	Estimated Glomerular Filtration Rate (eGFR)	30%	50%	54.07 (279)	<.001
Haem-oncs	Oxygen Saturation	53%	60%	19.17 (279)	<.001
Haem-oncs	Pulmonary Multidetector CT Angiography (MDCTA)	60%	65%	11.75 (279)	<.001
Haem-oncs	Transthoracic Echocardiography (TTE)	42%	53%	1.00 (279)	.317
Haem-oncs	Troponin	25%	40%	39.09 (279)	<.001
Haem-oncs	LO2 – Tailoring anticoagulant therapy in patients with PE	34%	50%	42.09 (279)	<.001
Haem-oncs	Oral anticoagulation therapy	36%	48%	12.71 (279)	<.001
Haem-oncs	warfarin	21%	51%	57.07 (279)	<.001
Haem-oncs	LO3 – Selecting appropriate patient-centred care strategies to improve adherence to long-term anticoagulant therapy	5%	11%	13.24 (279)	<.001
Haem-oncs	First Follow-up Visit	41%	54%	7.11 (279)	<.001
Haem-oncs	Patient Education and Counselling	39%	53%	36.10 (279)	<.001
Haem-oncs	B-type Natriuretic Peptide (BNP)	43%	55%	30.12 (279)	<.001

aTest statistic is reported as t-statistic for Learning Objectives (LOs), and chi-square statistic is reported for McNemar’s test for the decision points under LOs.

Table 3.

Case 2 results by learning objective and decision points

Speciality	Learning Objective or Decision Point	% Correct Pre-CG	% Correct Post-CG	Test Statistica (df)	P Value
Cardiologists	LO1 – Performing appropriate workup for the risk stratification of patients with PE	82%	86%	4.65 (244)	<.001
Cardiologists	Chemistry Screen	81%	87%	11.27 (244)	<.001
Cardiologists	Complete Blood Count (CBC) – Basic	85%	87%	2.67 (244)	.102
Cardiologists	Estimated Glomerular Filtration Rate (eGFR)	79%	85%	11.27 (244)	<.001
Cardiologists	LO2 – Tailoring anticoagulant therapy in patients with PE	49%	66%	10.27 (244)	<.001
Cardiologists	Discontinue: enoxaparin	56%	59%	4.50 (244)	<.05
Cardiologists	Start: NOAC therapy	41%	72%	59.06 (244)	<.001
Cardiologists	LO3 – Selecting appropriate patient-centred care strategies to improve adherence to long-term anticoagulant therapy	45%	64%	8.40 (244)	<.001
Cardiologists	Follow-Up Appointment with Provider	44%	64%	46.08 (244)	<.001
Cardiologists	Patient Education and Counselling	46%	64%	41.09 (244)	<.001
Haem-oncs	LO1 – Performing appropriate workup for the risk stratification of patients with PE	83%	86%	2.61 (146)	<.05
Haem-oncs	Chemistry Screen	80%	85%	4.50 (146)	<.05
Haem-oncs	Complete Blood Count (CBC) – Basic	85%	88%	1.80 (146)	.18
Haem-oncs	Estimated Glomerular Filtration Rate (eGFR)	82%	84%	1.00 (146)	.317
Haem-oncs	LO2 – Tailoring anticoagulant therapy in patients with PE	47%	66%	9.04 (146)	<.001
Haem-oncs	Discontinue: enoxaparin	54%	58%	3.57 (146)	.059
Haem-oncs	Start: NOAC therapy	41%	74%	35.10 (146)	<.001
Haem-oncs	LO3 – Selecting appropriate patient-centred care strategies to improve adherence to long-term anticoagulant therapy	49%	68%	6.38 (146)	<.001
Haem-oncs	Follow-Up Appointment with Provider	48%	67%	25.14 (146)	<.001
Haem-oncs	Patient Education and Counselling	49%	69%	26.13 (146)	<.001

aTest statistic is reported as t-statistic for Learning Objectives (LOs), and chi-square statistic is reported for McNemar’s test for the decision points under LOs.

Statistical Analysis

The statistical software that was used to conduct this analysis was the RServe Analytics extension within the Tableau server environment. Paired samples t-tests were conducted to examine change from pre- to post-CG at the participant level. McNemar’s tests were used for individual decision points for each participant. P values <.05 are considered statistically significant.

Source of Funding

This CME activity was supported by Bayer AG in 2017.

Results

Results showed a significant overall impact of education on correct decisions made in both cases for cardiologists, with a relative improvement of 22% for Case 1 (45% pre- to 55% post-CG, n = 475, t(474) = 14.12, P< .001, Cohen’s d = .46) and 19% for Case 2 (62% pre- to 74% post-CG, n = 245, t(244) = 11.95, P< .001, Cohen’s d = .59). Impact also was seen for haem-oncs, with a relative improvement of 27% for Case 1 (45% pre- to 57% post-CG, n = 280, t(279) = 11.91, P < .001, Cohen’s d = .60) and 19% for Case 2 (63% pre- to 75% post-CG, n = 147, t(146) = 9.52, P < .001, Cohen’s d = .58). See Tables 2 and 3 for results by speciality, learning objective, and select decision points.

In particular, the largest improvements for cardiologists and haem-oncs under the first learning objective, performing appropriate workup for the risk stratification, were seen in ordering measurements of renal function: estimated glomerular filtration rate (eGFR), relative increase of 55% (cardiologists) and 67% (haem-oncs), and ordering estimated creatinine clearance (CrCL), relative increase of 42% (cardiologists) and 56% (haem-oncs). Overall, about half of cardiologists and haem-oncs performed the appropriate workup for risk stratification so after education.

Under the second learning objective, tailoring anticoagulant treatment, in Case 1, there was a 103% (cardiologists) and 143% (haem-oncs) relative increase in ordering appropriate oral anticoagulation therapy, with 63% of cardiologists and 51% of haem-oncs doing so after education. Of note, only 5% and 11%, respectively, ordered warfarin – a decision that was clinically appropriate for this patient after education. In Case 2, there was a 76% (cardiologists) and 80% (haem-oncs) relative increase in appropriately starting NOAC therapy, with 72% and 74%, respectively, doing so after education. There was little impact for both cardiologists and haem-oncs on appropriately discontinuing enoxaparin. Under learning objective 3, selecting appropriate patient-centred strategies to improve adherence to long-term anticoagulant therapy, orders for a follow-up appointment and patient education and counselling had a relative increase of 45% and 39%, respectively, for cardiologists and 40% and 41%, respectively, for haem-oncs.

Discussion

Overall, case-based VPS education significantly improved the percentage of appropriate decisions made for patients with PE. Other studies that used simulation with clinical guidance have also reported positive results [24,25]. In our analysis, cardiologists and haem-oncs showed significant improvements in all areas identified as gaps: Gap 1: risk stratification – a greater proportion are now ordering appropriate tests for risk stratification; Gap 2: monitoring patients to gauge adherence to therapy – a greater proportion are selecting patient-centred care strategies to improve adherence; Gap 3: applying the latest data on NOACs in patients with cancer – a greater proportion are now tailoring anticoagulant therapies based on patient presentation.

However, despite the significant performance improvement for nearly every decision point measured using VPS, this activity also identified opportunities for improvement:

37% of cardiologists and 49% of haem-oncs in Case 1 and 28% of cardiologists and 26% of haem-oncs in Case 2 showed persistent practice gaps as they did not initiate oral anticoagulation therapy or NOAC therapy, respectively, as recommended by current clinical guidelines.

36% of cardiologists and 43% of haem-oncs did not demonstrate patient-centred care strategies by ordering a follow-up appointment or patient education counselling after education.

Significance

VPS improved cardiologists’ and haem-oncs management of patients with PE in a simulated environment. The greatest improvement observed was related to treatment decisions: appropriately initiation of oral anticoagulation therapy nearly doubled for cardiologists and more than doubled for haem-oncs, and starting NOAC therapy increased by over 70% for both cardiologists and haem-oncs. In addition, the method of collecting outcomes in simulation by means of embedded assessments using open-ended platforms is quite novel in CME, compared with the use of multiple-choice questions. Future research will examine the correlation and impact of the MedSims VPS platform on real-world practice.

Limitations

All cardiologists or haem-oncs who viewed the activity content did not make all of the decision points; therefore, they are not included in the results. The completion rate for Case 1 was 54% for cardiologists and 40% for haem-oncs and for Case 2 was 67% for cardiologists and 47% for haem-oncs. This may bias the results towards cardiologists who were motivated to participate fully in the activity. Second, although the MedSims VPS platform provides a consequence-free environment to make clinical decisions, therapy selection may be limited in the real world, depending on location or institution. Therefore, selection of therapies in the intervention may not fully reflect what a clinician would choose in actual practice.

These results are indicative of performance in a simulated environment. However, research shows simulations are predictive of real-world practice [22]. To understand whether practice changes translated to real-world performance, future studies should examine the simulation behaviours and their correlation with real-world behaviours. Future work should also look at the impact of education on those who participated in both cases and assess whether the magnitude of effect is greater for those participants compared with those who completed only 1 case.

Conclusions

The results of this study are relevant to educators in that they demonstrate that a VPS, which immerses and engages clinicians in an authentic and practical learning experience, can significantly improve decision making of physicians with simulated patients, with potential implications for clinical care. This is also relevant to physicians who wish to practice decision-making in a consequence-free environment while receiving evidence-based guidance and supplemental information. The authors have plans to conduct follow-up surveys with participants in future simulations to understand both the correlation and impact of the simulation with real-world behaviour. The MedSims VPS platform has been used successfully in other therapeutic areas to improve decision-making [26-28]. Given that physicians prefer online education that is case-based and 15 to 30 minutes in length [29]. It may be worthwhile to explore simulations that are shorter in duration to examine their impact. Such cases could be presented in a series, where diagnosis is one portion of the activity and treatment and future management is the second.