Assessing Mechanical Ventilation Management Skills: More Tools for the Toolbox.

Mechanical ventilation (MV) is a commonly required, high-risk, lifesaving intervention for patients admitted to an intensive care unit (ICU). Despite its complexity, management of MV has been shown to have profound effects on patient mortality, length of stay, and hospital costs when done successfully (1–3). The coronavirus disease (COVID-19) pandemic has further highlighted the need for improved baseline MV management skills in an even wider provider population, because critically ill patients requiring respiratory support have multiplied (4). However, despite its importance, treating patients receiving MV is a challenging skill to master, and current MV education models are unsatisfactory and insufficient (5–7). Currently, strategies for teaching and assessing MV management skills are highly variable, with no unanimous “best way” to go about teaching this topic (8). Although simulation is being used more frequently in medical education, there remains a need for competency-based MV curricula with standardized assessment tools.

In this issue of ATS Scholar, Hayashi and colleagues developed an objective structured clinical examination using a simulation-based assessment (SBA) of baseline MV management skills among residents rotating in the ICU (9). They used a modified Delphi technique to create six unique clinical cases and a 32-item checklist to assess MV management skills. The cases and checklist targeted high-yield skills such as identifying ventilator synchrony and waveforms, implementing lung-protective ventilation in acute respiratory distress syndrome, managing ventilators for patients with obstructive lung disease, and patient liberation from MV. Once validated, 80 internal medicine residents completed the examination before their ICU rotation, and a standardized score between 1 and 10 was determined for each learner. The authors reported that a portion of the participants also completed a pre- and post-ICU rotation multiple-choice examination (MCE). They used the Cronbach α coefficient to assess reliability of the SBA and Spearman correlation to estimate the correlation between the SBA and the MCE.

The authors found that the reliability of the SBA was high, with a Cronbach α coefficient of 0.72 (95% confidence interval, 0.64–0.81). The mean standardized score of residents on the SBA was 6.2 ± 1.3. There was significant variability in the percentage correct for different checklist items. For example, the majority of learners successfully used lung-protective tidal volume ventilation in patients with acute respiratory distress syndrome. However, only 12% of learners correctly identified the more nuanced entity of ventilator dyssynchrony. The mean standardized score of residents on the prerotation MCE was 7.6 ± 2.4 and improved to 8.2 ± 2.3 on the postrotation MCE (P < 0.001). Using the Spearman test, the SBA score and MCE pretest score showed a statistically significant, moderate-intensity positive correlation (rho = 0.41; P = 0.002).

This study has many strengths. The authors created multiple simulation cases to teach and assess clinically relevant aspects of MV management to resident learners. The obstructive lung disease case that was provided in the supplemental materials was concise and assessed clinically relevant aspects of MV management, such as necessary ventilator adjustments to address arterial blood gas derangements and the identification of auto-positive end expiratory pressure. This example case could easily be used or adapted as a teaching script for educators in the ICU. Furthermore, the authors outlined a rigorous validation process of the SBA. A modified Delphi approach was applied to checklist creation, pilot testing was performed to fine-tune the case scenarios and questions, and the SBA was administered by respiratory therapists who standardized their grading approach. A standardized script and grading rubric facilitated administration of the SBA by other members of the interprofessional team rather than relying only on physicians. This is important because interprofessional collaboration positively impacts patient care, and physician time and resource demands are often discussed as limitations to successful implementation of simulation in medical education. Importantly, the authors demonstrated the feasibility of implementing an SBA during an ICU rotation, and, as expected, knowledge and skill gaps were present. Higher scores on the MCE than on the SBA may suggest gaps in clinical skills that are not adequately identified in a knowledge-based assessment alone.

This study is not without limitations. Importantly, the authors only used the SBA as a pretest, and it was not paired with a specific educational intervention and sequential posttest. Therefore, the participants’ learned knowledge and skill during their ICU rotation were not assessed by the SBA. Use of a pretest paired with deliberate practice and feedback before a posttest has been shown to successfully get learners to a mastery standard for MV management and other common ICU procedures (10, 11). The learners in this study were not provided feedback on their SBA, nor were they allotted the opportunity for deliberate practice. In addition, the MCE was not included as part of the study a priori and did not address the exact same skills as the SBA. This makes the calculated internal consistency between the MCE and SBA less impactful. Last, as acknowledged by the authors, there was no assessment of skill retention or patient care outcomes.

Within critical care medicine, there has been a call for more dedicated MV education (12); yet, very few validated assessment tools currently exist. Simply relying on experiential learning could lead to patient harm if learners are called upon to provide MV management despite a paucity in background knowledge and training. Prior studies of MV education have shown simulation-based practical examinations to be superior to traditional assessments (10, 13, 14). In this study, the authors make an important contribution by developing an MV curriculum in an objective structured clinical examination format that incorporates core competencies required to care for critically ill patients receiving MV (15). In addition, the authors created succinct cases addressing common MV scenarios and cases to address more high-risk, less commonly encountered clinical scenarios. Overall, these authors put more tools in the toolbox for educators who teach and assess MV management skills. The curriculum in this study can easily be modified and applied to a variety of learners going forward, and we hope that it can help improve the downstream care provided to mechanically ventilated patients.