Effectiveness of online versus in-person structured training program on arterial blood gas, electrolytes, and ventilatory management of critically ill patients.

BACKGROUND: Due to the risk of viral transmission during in-person training, a shift toward online platforms is imperative in the current pandemic. Therefore, we compared the effectiveness of an in-person interactive course with a structurally similar online course designed to improve cognitive skills among clinical health professionals in arterial blood gas analysis, management of electrolyte imbalances, and approaches to mechanical ventilation in critically ill patients.
METHODS: In an observational, outcome assessor-blinded, cohort trial, group A included participants enrolled prospectively in an online course, while group B included those who took part in an in-person course (retrospective arm). The primary objective was comparison of cognitive skills through a pre and post-test questionnaire. Statistical analysis was performed using Student t-test.
RESULTS: In total, 435 participants were analyzed in group A, while 99 participants were evaluated in group B. The mean pre-test score was 9.48±2.75 and 10.76±2.42, while the mean post-test score was 11.94±1.90 (passing rate, 64.6%) and 12.53±1.63 (passing rate, 73.3%) in groups A and B, respectively. Group B scored significantly higher in both pre-test (P=0.001) and post-test evaluations (P=0.004). The improvement in post-test score was significantly greater (P=0.001) in group A (2.46±2.22) compared to group B (1.77±1.76). The medical specialties fared better in group B, while surgical specialties scored higher in group A. The pre-test vs. post-test scores exhibited a moderate correlation in both groups (P<0.001). The feedback survey showed a Likert score >3.5 for most points in both groups.
CONCLUSIONS: The online teaching module exhibited a significant benefit in terms of participant sensitization and knowledge sharing.

INTRODUCTION

The coronavirus disease 2019 (COVID-19) pandemic is a challenging global health crisis that has significantly impacted the medical system, and those least prepared have been the worst affected. The huge load of critically ill patients has overwhelmed the working capacity of frontline physicians [1,2]. Critical care medicine (CCM) is an ever-demanding medical specialty not only in terms of life-threatening situations, but also in complexity and need for rapid decision-making. Therefore, inexperienced non-CCM health care professionals (HCPs) are finding it difficult to address this load without adequate training in acid-base analysis, correction of electrolyte disorders, and ventilatory management in critically ill patients [3]. As a result, the onus lies on intensivists to treat such patients and train other clinicians. Traditionally, such education is provided through in-person training. However, with the high risk of viral contamination through in-person training, teaching has shifted toward online platforms. This also addresses the lack of availability of educators for physical sessions due to their prolonged working hours [4]. The need for social distancing has precluded the possibility of physically gathering for lectures, skill stations, or group discussions. It has not only disrupted routine medical education, but also has forced educators to develop other methods of continuing education.

There is no standardized alternative method of trainee education, and clinical practice patterns are highly variable [5]. To strengthen the available health care workforce, we designed an online interactive course (structurally similar to previously conducted in-person courses) aimed to improve cognitive skills among non-CCM HCPs in arterial blood gas (ABG) analysis, correction of electrolyte imbalance, and approaches to mechanical ventilation in critically ill adult patients. Even though online modules can serve as an effective teaching modality for training of clinical interventions, the current evidence is insufficient to infer its definitive role [6]. We hypothesized that the cognitive skills acquired through this online training program would be at least equivalent to those achieved from offline training and would help in development of state of the art online teaching courses fine-tuned to the needs of HCPs. We compared the effectiveness of such an online course with an in-person course (retrospective arm) that was conducted before initiation of the online version.

MATERIALS AND METHODS

After Institutional Ethical Board approval of AIIMS Rishikesh (AIIMS/IEC/20/287) and obtaining written informed consent from patients, we included all clinical HCP (medical and surgical specialties) faculty members and junior and senior residents of our institute in this observational, outcome assessorblinded, cohort trial. We excluded clinicians who could not attend due to ongoing commitments or illness during the trial time period. However, they were subsequently trained in the next available course. For each online training course, we prepared the participant list based on a homogenous sample of medical and surgical specialties. We also kept the respective heads of the departments in closed-loop communication, ensuring the attendance of participants in the course within the stipulated time and the replacement of those who could not attend due to various reasons. The 1-day course included a maximum of 20 participants at a time.

The online arm (group A) was organized thrice weekly and included participants from clinical specialties who underwent an online course conducted through an online “Google Meet” platform link. The link was e-mailed to all participants the evening before the class, and a reminder was sent 10 minutes before the scheduled time. The in-person arm (group B) comprise participants who had undergone an in-person training course before initiation of the online version (retrospective arm).

The course content, learning objectives, skill stations, opportunities for participant interaction, and assessments were the same for the two groups. The course content included three parts: management of electrolyte imbalance, ABG analysis, and approaches to mechanical ventilation. The first part included a summary of common electrolyte (sodium, potassium, calcium, and magnesium) disorders, including their causes, signs, and symptoms, and a stepwise approach to their diagnosis and management. The second part included the indications, contraindications, and sequence of steps for obtaining arterial blood samples and the structured approach for ABG interpretation. The third part included identification of signs and symptoms of respiratory compromise, the sequence of immediate resuscitative steps, approaches to diagnosis of an underlying respiratory disorder, and overview of mechanical ventilation including indications, initial settings, ventilation modes, troubleshooting during maintenance, and weaning. The course material was compiled from real scenarios covering various abnormalities to ascertain comprehensive coverage of the content. The desired attributes included clinical competence and understanding of the topic and clear concepts on the steps of the related procedure. The validity of the course content and questionnaire was independently reviewed by two intensivists with at least 5 years of experience. We allocated the course questionnaire to a pilot group of 30 participants who were instructed to complete the tool on two separate occasions, 2 weeks apart. We believed that the 2-week gap would be enough time for the participants to have forgotten their responses and to experience real change in knowledge. The responses were used to estimate test-retest reliability and internal consistency of the questionnaire.

The teaching program was divided into four sessions in a sequence of pre-test, didactic lectures, online skill stations, and post-course feedback survey. The course assessment was performed through identical pre- and post-test questionnaires sent through “Google Forms” before and after the teaching program. The pre-course test contained 15 multiple-choice questions (1 point each) including case scenarios, video clips/still images of the ABG strips, and ventilatory settings. The lectures were composed of three sessions (1 hour each), with a live audio-visual relay, PowerPoint slides, speaker notes, case scenarios, and video clips. We ensured a real-time interaction of participants with instructors for active participation during the lectures. The skill station also included three sessions (1 hour each). The mechanical ventilation station included a virtual simulator (open pediatrics online ventilator simulator) and a mechanical ventilator physically connected to an intubated mannequin simulator (CAE human patient simulator) for active interactions in case scenarios. In group B, the participant performed ventilatory adjustments according to the scenario question at the skill stations when prompted. In group A, the virtual simulator was screen shared, while the mannequin and ventilator display were relayed in real-time on Google Meet. Ventilatory adjustments were performed by the instructor on request of a participant. The ABG and electrolyte stations included ABG strips and case scenarios in a PowerPoint presentation. The participants were encouraged to submit queries at the end of each session. After each scenario, we conducted a debriefing session, followed by an analytical phase in which the trainee summarized the merits and demerits of the process. All the instructors maintained adequate physical distancing during the online course. The post-test assessed the knowledge and skills acquired by the participants during the course. For successful course completion, it was compulsory to attend all the sessions and secure at least 80% (12/15) in the post-test assessment. The participants were required to complete a post-course feedback survey to assess the self-efficacy of the course. The survey included 15 assertions, and the participants had to rate each on a 5-point Likert scale.

The primary objective was comparison of cognitive skills in an online versus in-person training course as assessed by pre- and post-test questionnaires. The secondary objective included evaluation of the self-efficacy of online versus in-person formats of the course assessed through a survey form and to assess the course components the participants found difficult or on which significant improvements were observed after completion of the course. The raw data from the responses from each participant were coded numerically. The sample size was based on convenience sampling and included all applicable participants in the institution. All participants trained from May 2020 to July 2020 were included. Statistical analysis was performed with IBM SPSS ver. 23.0 (IBM Corp., Armonk, NY, USA). The soft data (pre/post questionnaires and survey) received through Google Forms were subjected to analysis by an investigator blinded to study protocols. The results were presented as descriptive statistics and summarized as mean and standard deviation and number (percentage). Data were analyzed by Student t-test, bivariate correlation, and chi-square test. The mean Likert score was averaged to the number of items to assess the survey form. Cronbach alpha and Spearman correlation analyses were used to estimate the internal consistency and reliability of the questionnaire, respectively. A P-value ≤ 0.05 was considered statistically significant.

RESULTS

In total, 435 participants completed the training in group A (online course), while 99 members participated in group B (offline course). The Cronbach alpha score assessing the internal consistency of the pre- and post-test questionnaire was 0.82 (n = 30). The reliability analysis demonstrated a correlation coefficient of 0.695 (P<0.001), indicating a moderate correlation between the values obtained 2 weeks apart. Group A included participants from all 36 specialties, while group B included only members from 14 specialties (Figure 1). Most participants in group A belonged to medicine, anesthesiology, surgery, dermatology, orthopedics, pediatrics, or obstetrics and gynecology departments, while group B had preponderance of participants from anesthesiology, pediatrics, and obstetrics and gynecology specialties.

Figure 1.

Distribution of participants among departments.

The mean pre-test score was 9.48 ± 2.75 in group A and 10.76 ± 2.42 in group B, with a significant difference between the groups (P=0.001). The mean post-test score was 11.94 ± 1.90 in group A and 12.53 ± 1.63 in group B, with significantly better performance in group B (P=0.004). The overall improvement in post-test scores was significantly higher (P=0.001) in group A (2.46 ± 2.22) compared to group B (1.77 ± 1.76). On subgroup comparison, medical specialties fared better in group B for both pre- and post-test, although surgical specialties performed better in group A. The overall improvement in post-test scores was higher in group A for both medical and surgical specialties, with greater improvement in participants of surgical specialties (Table 1). Upon assessing the bivariate correlation between pre- and post-test scores, we observed a moderate correlation between groups (correlation coefficient: 0.60 and 0.69 for groups A and B, respectively) and high statistical significance (P≤0.001) (Figure 2).

Table 1.

Comparison of test scores among the selected groups

No.	Variable		Number of participants (group A:B)	Group A	Group B	P-value
1	Pre-test	Overall	435:99	9.48±2.75	10.76±2.42	0.001
2		Medical Sp	218:74	10.01±2.72	11.45±2.22	0.043
3		Surgical Sp	217:25	8.95±2.68	8.76±1.85	0.035
4	Post-test	Overall	435:99	11.94±1.90	12.53±1.63	0.004
5		Medical Sp	218:74	12.13±1.95	12.89±1.52	0.014
6		Surgical Sp	217:25	11.75±1.83	11.48±1.53	0.288
7	Post-Pre	Overall	435:99	2.46±2.22	1.77±1.76	0.001
8		Medical Sp	218:74	2.12±2.08	1.45±1.64	0.018
9		Surgical Sp	217:25	2.80±2.31	2.72±1.81	0.162

Values are presented as mean±standard deviation. The total score in pre-test/post-test assessments was 15, and the passing rate was 80% (i.e., 12/15).

A P≤0.05 was considered significant.

Sp, specialty.

Figure 2.

Bivariate correlation between the pre- and post-test scores.

Comparing the passing percentage among the groups, 24.6% of participants cleared the pre-test in group A vs. 46.5% in group B (P≤0.001), while 64.6% of participants cleared the post-test in group A vs. 73.3% in group B (P=0.083). On subgroup comparison, medical specialties performed better in group B for both pre-/post-test, although surgical specialties performed better in group A. Analyzing those with post-test score ≥ 12 with at least 20% improvement in post-test score, group A (30.8%) fared better than group B (16.2%) and also for both medical and surgical specialties individually (Table 2).

Table 2.

Comparison of passing and significant improvement percentages among the selected groups

No.	Variable		Number of participants (group A:B)	Group A	Group B	P-value
1	Pre-test	Overall	435:99	107 (24.60)	46 (46.46)	≤0.001
2		Medical Sp	218:74	71 (32.57)	43 (58.10)	≤0.001
3		Surgical Sp	217:25	36 (16.59)	3 (12)	0.554
4	Post-test	Overall	435:99	281 (64.60)	73 (73.73)	0.083
5		Medical Sp	218:74	154 (70.64)	61 (82.43)	0.046
6		Surgical Sp	217:25	127 (58.52)	12 (48)	0.313
7	Significant improvement	Overall	435:99	134 (30.80)	16 (16.16)	0.003
8		Medical Sp	218:74	62 (28.44)	10 (13.51)	0.010
9		Surgical Sp	217:25	72 (33.18)	6 (24)	0.352

Values are presented as number (%). The total score in pre-test/post-test assessments was 15, and the passing rate was 80% (i.e., 12/15). A significant improvement was defined as post-test score ≥12 with at least 20% improvement. A P≤0.05 was considered significant.

Sp, specialty.

On the post-test feedback survey, we observed significantly higher scores in group B compared to group A (p≤0.001) (Table 3). The mean scores in the groups were greater than 3.5 of 5 for most of the parameters, suggesting positive feedback for both offline and online teaching programs. The participants in group A were most confident in identifying an acid-base disorder, while anxiety persisted in monitoring the dynamic parameters during mechanical ventilation or identifying a triple acid-base disorder. In group B, the parameter with the lowest score was “setting alarm limits of the ventilator.” The major participant suggestions included possible addition of education on the high-flow-nasal cannula, non-invasive ventilation, and more elaborate future sessions.

Table 3.

Comparison of mean Likert score of the post-test feedback survey among the groups

No.	Feedback survey response	Group A	Group B	P-value
1	How confident are you in determining the need for mechanical ventilation?	3.64±0.85	4.02±0.76	≤0.001
2	How confident are you in choosing between invasive and non-invasive ventilation?	3.81±0.83	4.28±0.71	≤0.001
3	How confident are you in initiating mechanical ventilation settings?	3.46±0.98	4.18±0.80	≤0.001
4	How familiar are you with the basic modes of mechanical ventilation?	3.69±0.88	4.18±0.75	≤0.001
5	How confident are you in setting up alarm limits?	3.86±0.84	3.39±0.95	≤0.001
6	How confident are you in monitoring dynamic changes in ventilatory parameters?	3.38±0.92	3.92±0.83	≤0.001
7	How familiar are you with weaning criteria and step-down ventilation strategies?	3.54±0.91	3.95±0.91	≤0.001
8	How confident are you in identifying acid-base disorders by blood gas analysis?	4.06±0.81	4.39±0.65	≤0.001
9	How confident are you in applying equations for estimating gas compensation?	3.67±0.88	4.23±0.74	≤0.001
10	How confident are you in calculating anion-gap estimations?	3.83±0.91	4.36±0.75	≤0.001
11	How confident are you in identifying triple disorders by blood gas analysis?	3.45±0.95	4.08±0.78	≤0.001
12	How familiar are you with electrolyte distributions in body compartments?	3.72±0.89	4.17±0.73	≤0.001
13	How confident are you in identifying electrolyte imbalances?	3.79±0.83	4.09±0.62	≤0.001
14	How confident are you in understanding the effects of rapid correction of electrolytes?	3.98±0.83	4.28±0.69	0.001
15	How confident are you in managing electrolyte imbalances?	3.79±0.82	4.08±0.63	≤0.001

Values are presented as mean±standard deviation.

A P≤0.05 was considered significant.

DISCUSSION

In a pandemic situation, mechanical ventilation instructors are facing time constraints in educating trainees to offset their burden in supervising management of critically ill patients [1]. A meticulous strategy is required to increase the HCP workforce in a stipulated time-frame and to safeguard this task force. Utilizing non-CCM HCPs for assistance after online training is a risky alternative but was required due to the demands of the situation [7,8]. Here, we compared the efficacy of an online course with an in-person course organized before the pandemic situation. The offline group fared better in the post-test score, while the online group had a nearly 66% passing rate, with greater post-test improvement. The online version was a valuable alternative both for crisis situations and to sensitize the participants.

The human brain can intake only a few aspects at a time [9]. Therefore, our teaching module contained only the salient aspects of CCM management, comprising three learning topics. Simulation is an efficient technique to replace real patient scenarios and provide a controlled environment of learning and teaching skills [5,10-12]. However, as evident by the feedback survey with higher confidence among participants after an in-person training session, personal interactions during face to face training seem to better impart knowledge in a controlled environment. The higher pre-existing knowledge of participants (pre-test score and passing %) and the presence of a major portion of medical specialties in group B could explain their better post-test scores. However, the much larger sample size, better homogeneity between medical and surgical specialties, and a post-test passing rate of 64.6% are significant factors for group A. Greater post-test change in group A is apparent considering its lower pre-test score, though this highlights the knowledge gained through the online course. The efficacy of online training is supported by the nearly 33% of participants with post-test score ≥ 12 with at least 20% improvement in post-test scores compared to pretest values. The most common specialties in this category were dermatology, ophthalmology, pediatrics, orthopedics, obstetrics and gynecology, and surgery, which usually do not include mechanical ventilation. Therefore, online training provided a notable sensitization to residents to acquaint them with the basics of mechanical ventilation, ABG analysis, and management of electrolyte disorders over a brief duration of time.

Even though it might not be realistic to expect trainees to work independently in a critical care environment, their assistance under supervision of intensivists can help in translating knowledge gained through an online training program into clinical settings [13]. Furthermore, if an online version could be utilized as a prerequisite for in-person courses, participant preconditioning could save time that would have been devoted to lecture presentation during in-person courses. This allows full dedication to skill stations and better learning during in-person courses. The efficacy of such a course, however, requires testing in future trials.

We evaluated the internal consistency and reliability of pre- and post-test questionnaires through the Cronbach alpha score and Spearman correlation, respectively. The results indicated good reliability and internal consistency of the questionnaire. A moderate correlation between the pre- and post-test scores showed homogenous improvement in acquired knowledge in both groups. The correlation highlights that the modules were carried out in a standardized and consistent manner. The feedback survey showed that group B participants were more confident in most course components compared to group A (Table 3). This was an expected outcome, taking into account the persistent anxiety in participants with no previous exposure to a ventilator and their inability to obtain physical experience through an online course. However, an average Likert score > 3.5 in group A indicates that the online version could be useful in gaining acceptability and interest of participants for initiating work under supervision of CCM experts in a crisis. Lack of confidence in monitoring of dynamic ventilatory parameters or identifying triple acid-base disorders is likely after a short online course. Higher confidence in group B could be attributed to their better pre-existing knowledge and history of physical experience in ventilation skill stations.

This study had several limitations. We compared the online group with the retrospective arm of offline training as the current pandemic did not allow an in-person training course. Even though we maintained similar methodologies for the two courses in all possible perspectives, the difference could be due to the attentiveness of participants in the online version. The instructors attempted to engage actively with all participants during the online course. We could not derive a suitable sample size due to lack of previous study and the fixed sample in the offline group. Due to the emergent need for consistent teaching and the unequal numbers of residents and faculty in each department, we could not enroll equal sample numbers among department. Moreover, we had to reorganize the question sequence after each course to avoid copying by the participants. Therefore, we could not analyze the results for each theme separately.

In conclusion, this study highlights the utility of simulator-based online teaching for cognitive knowledge on management of mechanical ventilation, ABG analysis, and management of electrolyte disorders, creating an effective workforce for pandemic situations.

▪ Use of an online module for training on arterial blood gas analysis, management of electrolyte imbalance, and approaches to mechanical ventilation demonstrated a significant benefit in terms of participant sensitization and knowledge sharing.

▪ Even though the offline group fared better in both pre- and post-test scores, the online group had a 66% passing rate with high overall post-test improvement.

▪ This highlights the utility of simulator-based online teaching for achieving cognitive knowledge in the abovementioned skills among inexperienced health care professionals and creates an effective workforce for pandemic situations.