Impact of Novel Multiinstitutional Curriculum on Critical Care Fellow Ventilator Knowledge.

Background: Management of mechanical ventilation (MV) is a curricular milestone for trainees in pulmonary critical care medicine (PCCM) and critical care medicine (CCM) fellowships. Though recognition of ventilator waveform abnormalities that could result in patient complications is an important part of management, it is unclear how well fellows recognize these abnormalities. Objective: To study proficiency of ventilator waveform analysis among first-year fellows enrolled in a MV course compared with that of traditionally trained fellows.
Methods: The study took place from July 2016 to January 2019, with 93 fellows from 10 fellowship programs completing the waveform examination. Seventy-three fellows participated in a course during their first year of fellowship, with part I occurring at the beginning of fellowship in July and part II occurring after 6 months of clinical work. These fellows were given a five-question ventilator waveform examination at multiple time points throughout the two-part course. Twenty fellows from three other fellowship programs who were in their first, second, or third year of fellowship and who did not participate in this course served as the control group. These fellows took the waveform examination a single time, at a median of 23 months into their training.
Results: Before the course, scores were low but improved after 3 days of education at the beginning of the fellowship (18.0 ± 1.6 vs. 45.6 ± 3.0; P < 0.0001). Scores decreased after 6 months of clinical rotations but increased to their highest levels after part II of the course (33.7 ± 3.1 for part II pretest vs. 77.4 ± 2.4 for part II posttest; P < 0.0001). After completing part I at the beginning of fellowship, fellows participating in the course outperformed control fellows, who received a median of 23 months of traditional fellowship training at the time of testing (45.6 ± 3.0 vs. 25.3 ± 2.7; P < 0.0001). There was no difference in scores between PCCM and CCM fellows. In anonymous surveys, the fellows also rated the mechanical ventilator lectures highly.
Conclusion: PCCM and CCM fellows do not recognize common waveform abnormalities at the beginning of fellowship but can be trained to do so. Traditional fellowship training may be insufficient to master ventilator waveform analysis, and a more intentional, structured course for MV may help fellowship programs meet the curricular milestones for MV.

Prior studies have demonstrated that medical trainees lack comfort in their ability to manage patients requiring mechanical ventilation (MV) (1, 2). Because patients with respiratory failure requiring MV are principally managed in the intensive care unit (ICU) (3), management of MV is a core competency in critical care training. This is reflected in the fact that management of MV is an American College of Graduate Medical Education curricular milestone for critical care medicine (CCM) and combined pulmonary critical care medicine (PCCM) fellowship programs (4).

Despite this, there is no universally accepted curriculum related to MV training, so the educational approach varies widely across programs (5). In fact, only 50% of surveyed fellows were satisfied with their MV education (6). Fellow satisfaction with MV education correlated with the presence of formal educational activities (introductory course, hands-on sessions, and longitudinal programs) as well as their own confidence in understanding respiratory physiology, ventilator waveform analysis, and management of patient–ventilator asynchronies (6).

Patient–ventilator asynchronies can occur at each phase of the respiratory cycle, including trigger (ineffective triggering or autotriggering), inspiratory flow (flow starvation or flow excess), cycling (premature or delayed cycling), or expiration (auto–positive end-expiratory pressure [auto-PEEP]) (7). There is a growing appreciation that many of these asynchronies can result in diaphragmatic myotrauma due to ventilator overassistance, underassistance, or eccentric contraction (8), with significant adverse clinical consequences, including patient discomfort, diaphragmatic weakness, prolonged MV, and decreased survival (9–12). For example, ineffective triggering, typically due to auto-PEEP, has been associated with prolonged MV and increased mortality (9–11). Double cycling, which can result from premature cycling or reverse triggering, may significantly increase or even double the intended tidal volume, which would be harmful in setting of low tidal volume strategy frequently used in patients with acute respiratory distress syndrome (ARDS) (12). Though there are little data assessing the ability of critical care trainees to manage the ventilator, physicians-in-training and practicing intensivists perform poorly at identifying common patient–ventilator asynchronies (13).

Thus, there is a clear need for critical care fellowship programs to address the gap between the programmatic expectations for fellows and the actual comfort and skill level among intensivists in managing MV. Furthermore, there is also a need to identify optimal educational approaches to teaching MV, as well as appropriate formative and summative assessments of the effectiveness of these approaches on fellows’ skills as they progress through training.

Therefore, we prospectively studied the ability of critical care fellows from multiple fellowship programs who participated in an interactive longitudinal MV course to interpret clinically important common ventilator waveform abnormalities compared with that of traditionally trained fellows from three other fellowship programs who did not participate in the course.

Methods

Subjects

The current study took place from July 2016 to January 2019, with a total of 93 fellows from 10 fellowship programs completing the MV waveform examination (Figure 1). Of these fellows, 20 fellows, who trained at three fellowship programs and did not participate in our course but instead received traditional didactic and bedside training as part of their fellowship programs, served as the control group. Control subjects were fellows in their first, second, or third year of fellowship, with a median of 23 months of training at the time of the ventilator examination. In each of the control fellowship programs, the first year of fellowship consisted primarily of clinical rotations, of which 5–6 months were spent in ICUs, whereas the final year consisted of 3–5 months of clinical time, of which 2–3 months were spent in ICUs. The study subjects consisted of 73 fellows from seven fellowship programs that participated in the ventilator course. All of the study subjects were first-year fellows. Institutional review board exemption for this study was obtained from all of the participating institutions.

Figure 1.

Comparison of timing of waveform testing for ventilator course participants and traditionally trained fellows. The top part of the figure shows timing of testing of fellows who took the ventilator course. These fellows took the waveform examination in the first month of fellowship, before and after completing the 3 days of education in part I of the course and then during their seventh month of fellowship, before and after completing the 2 days of education in part II. The bottom of the figure shows testing for the traditionally trained control group, who were either first-, second-, or third-year fellows who took the waveform examination a single time. aOne fellow who took the waveform test in the traditionally trained group did not report their year of training in fellowship, so 19 of 20 are included in the breakdown of scores by fellowship year.

Curriculum

For the 73 study subjects, MV education was provided as part of a multiinstitutional collaboration in which fellows from several critical care fellowships were taught various key critical care topics (14). The MV course consisted of two parts. Part I of the course was a 3-day introductory course in July of the first year of fellowship before any clinical rotations. Part II of the course was a 2-day advanced course that took place 6 months later, in January of the first year of fellowship. The advanced course reviewed concepts taught during the first part of the course and built on that foundation through the introduction of more advanced concepts. The course covered a wide spectrum of MV topics, with only a portion of the course specifically covering patient–ventilator asynchronies and waveform analysis (Table 1). Fellows completed precourse reading assignments and then participated in highly interactive lectures, followed by simulations and small group problem solving activities that reinforced the topics discussed in the lectures. For the simulations and small group sessions, the student:faculty ratio was 6:1.

Table 1.

Curriculum description for MV course taught at summer and winter education blocks

Course Unit	Lecture Topics
Part I: summer education block*
Unit 1	Pretest
Pulmonary physiology and physics of applied	Ohm’s law, equation of motion and alveolar pressure
MV	Natural decay equation, time constant and auto-PEEP
	Test Lung Praxis I: PIP, Ppl, τ, and auto-PEEP
	MV clinical simulation I
Unit 2	Overview of the basic modes and breath types
Modes of MV and architecture of delivered breaths	Normal ventilator waveforms
	Test lung praxis II: basic modes and breath types
	MV clinical simulation II
	Noninvasive ventilation
	Pressure control, volume control and volume targeted breaths
	Test lung praxis III: normal waveforms
	MV clinical simulation III
	High-flow oxygenation
Unit 3	Ventilator-associated lung injury
Goals of MV	Goals of ventilation
	Goals of oxygenation
Unit 4	Patient–ventilator synchrony
Assessment of the mechanically ventilated patient	Test lung praxis IV: waveform analysis
	MV clinical simulation IV
	Bedside assessment of the mechanically ventilated patient
	Posttest
Part II: winter education block†
Review of units 1–4 from summer education block	Pretest
	Ventilator physiology
	Normal wave forms
	Review MV clinical simulation I
	Review: test lung praxis I
	Patient–ventilator asynchrony
	Review MV clinical simulation II
	Review: test lung praxis II
Unit 5	Initiating and terminating the inspiratory breath
Advanced concepts in MV	Work of breathing, atrophy and fatigue of the diaphragm
	Airway pressure release ventilation
	Esophageal manometry
	Proportional assist ventilation
	Neurally adjusted ventilatory assistance
	Bedside assessment of the mechanically ventilated patient
	MV clinical simulations III
	Test lung praxis III
	Posttest

Definition of abbreviations: MV = mechanical ventilation; PEEP = positive end-expiratory pressure; PIP = peak inspiratory pressure; Ppl = plateau pressure.

Summer course (part I) was held for 3 days in July at the beginning of fellowship.

Winter course (part II) was held for 2 days in January, midway through first year of fellowship.

For the 20 control subjects, the fellows received 3–5 hours of structured teaching on MV each year in addition to bedside teaching during their ICU rotations. Those hours of dedicated instruction in the control group consisted of lectures related to modes of ventilation, troubleshooting ventilator emergencies, ventilator management in ARDS, as well as various simulation sessions with actual ventilators and test lungs related to modes, hypoxemia, and airway pressure abnormalities.

Waveform Test

Because there is no gold standard for proficiency in MV (5), one of the authors (B.W.L.), an expert in MV with more than 20 years of experience teaching the topic, developed a practical assessment tool. The decision was made to use a pictorial waveform examination of common asynchronies that may cause significant morbidity or mortality if not recognized and managed. Early versions of the examination were given to trainees in past ventilator courses and were modified on the basis of learner feedback and through consensus with a coauthor with expertise in MV (N.S.). Finally, this version was pilot tested with learners and then used in the current study. The waveform examination consisted of five different pictures of both pressure-time and flow-time scalars taken from the ventilator demonstrating common waveform abnormalities that may result in adverse clinical sequelae, as classified in the literature (7), such as asynchronies of trigger (ineffective triggering or autotriggering), flow (flow starvation or excess), cycle (premature or delayed cycling), or auto-PEEP (see data supplement for examination). For each of the pictured waveforms, fellows were asked to select from a list of waveform abnormalities and potential consequences, such as diaphragmatic fatigue and diaphragmatic atrophy. Learners were asked to select all possible correct answers for a given image. The test took between 5 and 15 minutes for fellows to complete. Each of the five waveform abnormalities were scored on a 20-point scale and weighted evenly, resulting in a maximum test score of 100 points. Two of the authors (N.S. and B.W.L.) scored all of the waveform examinations. Interrater reliability was verified by each scoring 40 tests independently and then ensuring that there were no differences in the scores.

The study subjects who completed the two-part MV course took the test at the following four different time points: 1) a pretest at the start of part I of the course in July at the beginning of fellowship, 2) a posttest 3 days later at the end of part I, 3) a pretest 6 months after the initial course before starting part II of the course in January, and 4) a posttest 2 days later at the end of part II. Though the content of this examination did not change, the ordering of the waveforms was changed periodically. The control subjects were first-, second-, or third-year fellows who did not participate in the MV course and took the test once at the end of the academic year (Figure 1).

Survey

At the conclusion of part I and part II of the course, the participating fellows were asked to complete an anonymous web-based survey (SurveyMonkey Inc.). The fellows rated the individual lectures, small group exercises, and simulations on a four-point scale in which 1 corresponded with the description, “substantial change is needed,” 2 corresponded with “adequate but some changes are needed,” 3 corresponded with “very helpful, no major change needed,” and 4 corresponded with “outstanding.”

Statistical Analysis

The primary outcome of interest was the difference in the waveform examination scores between the study subjects who completed the two-part MV course relative to the traditionally trained control subjects who did not participate in the course. Other outcomes studied include fellows’ evaluation of the course as well as progression of waveform examination scores for the study subjects during the first year of fellowship while participating in the course. A prespecified subgroup analysis was performed comparing test scores of fellows trained in PCCM programs to those in standalone CCM programs. Linear mixed models were used to compare scores on waveform examinations to account for repeated measurements of individual subjects and the clustering of fellows in their individual training programs. Standard residual diagnostics were used to check model assumptions. SAS version 9.4 was used for all analyses. All reported P values are two sided.

Results

All 73 fellows who participated in the course were in the first year of fellowship training; 56% were in a PCCM fellowship program, whereas 44% were in a CCM program. We also administered the waveform examination to 20 traditionally trained fellows who did not participate in the MV course. At the time of the examination, 40%, 20% and 35% of the control fellows were in Years 1, 2, and 3 of fellowship training, respectively (Table 2). There was no significant difference in the type of fellowship training (PCCM or CCM) in the course participant group compared with the traditionally trained group.

Table 2.

Type and year of fellowship training

	Traditionally Trained Fellows in the Control Group	Fellows Participating in the MV Course
Number of fellows	20	73
Fellowship year, n (%)
1	8 (40)	73 (100)*
2	4 (20)	0 (0)
3	7 (35)	0 (0)
Missing	1 (5)†	0 (0)
Type of fellowship, n (%)
Pulmonary critical care	16 (80)	42 (57.5)‡
Critical care only	4 (20)	31 (42.5)

Definition of abbreviation: MV = mechanical ventilation.

Significantly more first-year fellows in the MV course group compared with traditionally trained group (P < 0.0001).

One fellow who took the waveform test in the traditionally trained group did not report their year of training in fellowship.

No significant difference in type of fellowship in MV course group compared with traditionally trained group (P = 0.07).

The mean pretest score of the study subjects at the beginning of fellowship was not significantly different from scores of the traditionally trained control subjects who took the test once (mean ± standard error: 18.0 ± 1.6 vs. 25.3 ± 2.7; P = 0.10), despite the control fellows having had a median of 23 months of fellowship training. In the control group, senior fellow test scores were similar to those of first-year fellows, suggesting that lower scores in the overall control group were not due to decay of senior fellows knowledge because of fewer clinical rotations (Figure 2). After 3 days of education during part I of the MV course, the mean posttest score for the study subjects significantly improved (45.6 ± 3.0 vs. 18.0 ± 1.6; P < 0.0001). This posttest score was also significantly higher than the mean score for the traditionally trained fellows who took the test once (P < 0.0001) (Figure 2)

Figure 2.

Control group of 20 fellows from one critical care medicine and two pulmonary critical care medicine fellowships took the waveform examination a single time after a median of 21 months of fellowship training, with a mean score of 25.3 ± 2.7. At the beginning of fellowship, in July, part I of the ventilator course took place. Mean ± standard error waveform examination score was 18.0 ± 1.6 before training in part I and increased significantly to mean 45.6 ± 3.0 after the 3 days of training was completed for 73 fellows (P < 0.0001). Between the July training and part II of the course in January, waveform examination scores decreased to mean 33.7 ± 3.1 (P = 0.0004). By the end of the January course, scores increased to highest level to date (mean 77.4 ± 2.4; P < 0.0001). FY = Fellow Year; Vent = Ventilator.

The study subjects’ pretest scores before part II of the course decreased by 12 points compared with their posttest scores after completing part I of the course 6 months before (33.7 ± 3.1 for part II pretest vs. 45.6 ± 3.0 for part I posttest; P = 0.0004). However, there was a 43.7 point increase in posttest score after 2 days of further education during part II of the MV course (33.7 ± 3.1 for part II pretest of vs. 77.4 ± 2.4 for part II posttest; P < 0.0001).

We performed a prespecified subgroup analysis comparing the scores of 32 fellows in standalone CCM with those of 41 fellows in PCCM fellowship programs. A two-way ANOVA without interaction found no significant difference in test scores between the two fellowship types (P = 0.45) (Figure 3).

Figure 3.

Comparison of waveform examination scores for fellows in standalone critical care versus pulmonary critical care fellowships at each of the four time points when the examination was administered. There was no significant difference in examination scores for either pretests or posttests based on type of fellowship during testing for part I in July or part II in January. Jan = January.

In the anonymous course evaluation, survey response rates were 62.5% and 47.1% for part I and part II of the course, respectively, averaged over the 3-year study period. The mean ratings for the ventilator course lectures, small group exercises, and simulations were 3.56 and 3.50 (of 4) for part I and part II of the course, respectively, averaged over the 3-year study period (Table 3). The mean scores were higher for the ventilator course compared with the other portions of the critical care education block (P < 0.0001 for both part I and part II).

Table 3.

Fellows’ rating* of the ventilator and nonventilator lectures, small group exercises, and simulations

	Fellows Responding (N)	Ventilator Lectures, Small Group Activities, and Simulations		Nonventilator Lectures, Small Group Activities, and Simulations
		Activities Rated (N)	Mean Rating (SD)	Activities Rated (N)	Mean Rating (SD)
Course part I July 2016–2018	60	62	3.56 (0.27)†	145	3.21 (0.33)†
Course part II January 2017–2019	41	45	3.50 (0.17)†	59	3.16 (0.32)†

Definition of abbreviation: SD = standard deviation.

Fellows rated the activities on a four-point scale as follows: 1 = needs substantial change, 2 = adequate but needs some improvement, 3 = very helpful and no major change needed, and 4 = outstanding.

P < 0.0001 for comparison of ventilator lectures, small group activities, and simulations with nonventilator lectures, small group activities, and simulations.

Discussion

Given the potential adverse clinical consequences of patient–ventilator asynchrony, including patient discomfort, prolonged MV, and increased mortality, it is important for critical care fellows to be able to interpret ventilator waveforms. We found that after completing a longitudinal, two-part interactive simulation and small group–based MV course, first-year critical care fellows from multiple fellowships performed significantly better at recognizing common asynchronies compared with traditionally trained fellows who had completed a median of 23 months of clinical training at the time of testing. In addition, our longitudinal course was favorably evaluated by the participating fellows.

There were several other interesting findings in our study. After completing residency training that involved caring for patients receiving MV, incoming fellows were not able to recognize most ventilator waveform abnormalities. Because waveform recognition is not an emphasized skill in most residency training programs, this finding is not surprising but shows the need for this training in CCM or PCCM fellowship. Three days of MV education at the start of fellowship significantly improved recognition of these abnormalities. We expected that after completing part I of the course in the summer followed by 6 months of clinical training, the first-year fellows’ ability to interpret waveforms would continue to improve. However, fellows performed worse on the waveform-based examination at the beginning of part II of the course in winter than after completion of the summer program. This suggests that either fellows did not incorporate the knowledge gained from part I of the course in their care of mechanically ventilated patients, or that standard bedside MV education did not prevent decay of learning. However, 2 more days of spaced learning in part II of the course in the winter improved waveform recognition skills in first-year critical care fellows by the greatest percentage and led to the highest scores of all the time points measured. These findings were consistent in fellows who trained in either PCCM or standalone CCM fellowship programs.

This study has several strengths. We studied the effect of the curriculum in a large number of fellows from multiple academic institutions and fellowship classes over several years. The cohort of fellows represents a mix of trainees who pursue both clinical and research careers. We were also able to study the curriculum in fellows trained in both pulmonary/critical care as well as standalone critical care fellowships. Because of this, we believe this cohort is a representative sample of critical care training programs. We also used a standardized curriculum to teach management of MV, with foundational topics introduced at the beginning of fellowship and more advanced concepts taught after the fellows had clinical experience in the ICU. This allowed us to measure decay of learning between blocks of teaching and measure progression of knowledge over time.

The study also has several limitations. Although the curriculum covered a broad range of topics related to the risks and potential harm of MV and the importance of proper ventilator management for patient safety, the assessment only measured fellows’ ability to recognize major asynchronies by interpreting ventilator waveforms. The assessment did not capture other important aspects of management of MV, such as implementing lung protective ventilator strategies in ARDS. This approach to assessment was practical and intentional. The assessment tool was feasible to implement in a large group of learners over multiple time points and allowed us to measure a specific skill in MV. However, we believe it is important to develop a valid clinical assessment of MV management with broad face validity that can be easily implemented to determine the effectiveness of future educational interventions related to MV beyond patient–ventilator asynchrony. Another study limitation is that there were only three control programs studied. Because of this, it would be inappropriate to extrapolate the abilities of these fellows to interpret waveform asynchronies to all traditionally trained critical care fellows. However, our study findings for the control group are consistent with available literature that suggests that experienced intensivists perform poorly at identifying common patient–ventilator asynchronies (13).

Another potential limitation is that it is possible that repeated testing introduced test–retest bias, improving test performance without improving asynchrony knowledge. However, there are several reasons that make this unlikely. We did not use the test images during the course and did not provide the answers to the test questions. We also changed the order of the waveform pictures during repeated testing. Furthermore, the questions consisted solely of pictures of waveforms that would be difficult to remember, so it is unlikely that fellows could use simple recall during retesting. Though unlikely, it is possible that because posttests are only 2–3 days after the pretests, fellows could research questions on the basis of their recall of individual test images. However, if learners went to such great lengths to improve their test scores, it would require study by the learner to master waveform analysis, which is our desired educational goal. Another limitation of the study is that we do not report any test results after part II of the course in January, so we cannot determine further decay of learning in fellows who completed the course versus those who are traditionally trained. This would be important to determine because more dedicated MV education may be necessary later in fellowship to maintain knowledge.

Our findings are consistent with prior work showing that dedicated training in MV management improves knowledge and skills in both fellows and residents. An MV curriculum with online modules and virtual simulation followed by in-person simulation-based education and exposure to clinical scenarios improved fellow knowledge and satisfaction (15). Fellows completing hands-on training in MV had better absolute improvement in knowledge scores and satisfaction compared with self-directed learning (16). The use of simulation-based education throughout the curriculum is consistent with best practices from prior studies evaluating resident education in MV. Simulation-based education in MV before ICU rotations improved residents’ bedside clinical skills assessment and confidence compared with traditional training (17). First-year residents trained with simulation outperformed traditionally trained third-year residents in a bedside clinical skills assessment at the end of ICU rotation (18). Furthermore, we used manikin-based simulation, which is more effective than computer-based simulation (19), within the multiday curriculum, which is effective in improving knowledge, performance of critical actions, and learner confidence in management of MV (20).

Several important questions remain unanswered. It is unknown whether improved ability to recognize patient–ventilator asynchrony will translate into improved clinical decision-making at the bedside and better clinical outcomes, such as shorter duration of MV or improved survival. Future work should attempt to assess this through clinically relevant simulation-based assessment or at the bedside. Though this manuscript explores the impact of dedicated ventilator teaching on fellows in pulmonary critical care and critical care, it would be important to determine ventilator waveform analysis proficiency of practicing intensivists as well as respiratory therapists and to determine whether similar training would benefit these groups. Though the total length of our course was 5 days, consisting of a 3-day part I and 2-day part II, the optimal duration of an MV course is unclear. Furthermore, future studies of MV education should evaluate the optimal approach to preventing decay of knowledge, predominantly identifying whether short bursts of structured instruction versus multihour workshops at repeated intervals improve learner understanding and retention. Finally, further research is needed in identifying specific characteristics of learners that impact their ability to learn complex concepts and apply this knowledge in a clinical context.

We found that despite lectures and bedside teaching during standard ICU rotations, the traditional approach to teaching MV may be insufficient to master ventilator waveform analysis. Our findings are consistent with the available literature indicating that some practicing intensivists do not recognize common patient–ventilator asynchronies (13), suggesting that graduating fellows may not have mastered these skills during training. This is worrisome because management of MV is an essential skill for the practicing intensivist and failure to recognize and treat common asynchronies may adversely impact patient outcomes (8–12). However, we found that an interactive, longitudinal, two-part, small group– and simulation-based curriculum is effective in training critical care fellows to interpret ventilator waveforms and may help fellowship programs to better meet the curricular milestones for MV.