Effectiveness of education in point-of-care ultrasound-assisted physical examinations in an emergency department: A before-and-after study.

Implementation of point-of-care ultrasonography (POCUS)-assisted physical examination (PE) in emergency departments (EDs) was conducted in the ED of an urban tertiary teaching hospital. This study examines the effect of POCUS implementation in emergency medicine departments by using a systematic education program on image acquisition to analyze decision making.Educating staff on POCUS involved a technique related to image acquisition and then accurately diagnosing subsequent POCUS results. The quasi-experimental, uncontrolled before-and-after study was performed to evaluate the education effect. POCUS orders for eligible patients, length of stay (LOS) in ED, and return visits (RVs) to ED between the "before" period (March 1, 2015 to February 28, 2016) and the "after" period (March 1, 2016 to February 28, 2017) were compared. Piecewise regression was used to assess trend differences of LOS and RVs between the periods.A total of 16,942 and 16,287 patients were included in the before and after periods of education, respectively. During the study periods, 966 (6%) and 2801 (18%) POCUS were ordered, respectively (rate difference  =  12%; P < .001). Before the education, the median LOS was 6.55 (interquartile rage [IQR]: 6.2-6.75) and the trend slope of LOS was -0.01. After the education, the median LOS was 5.25 (IQR: 4.85-5.45) and the trend slope (the change of which was considered significant, at a P value of .012) was -0.15. Before the education, the median RV rate was 6.4% (IQR: 6.15-6.65) and the trend slope of RVs was -0.01. After the education, the median RVs was 5.25% (IQR: 4.95-5.35) and the trend slope of RVs was also significant, at -0.11.The education of POCUS-PE in ED successfully increased use of POCUS, and reduced the LOS and RV rate in ED.

Introduction

Point-of-care ultrasonography (POCUS) is a safe and rapidly evolving diagnostic modality that has seen emerging interest in its routine use to potentially expedite and provide cost-efficient care.[ Technological advances have improved portability and miniaturization of equipment, allowing ultrasound imaging “at the bedside” to make timely diagnoses and guide procedures.

POCUS can help to narrow a differential diagnosis with clinical information revealed by the history of the patient and immediate physical examination (PE), as well as to refine decision making for further treatment.[ Despite the undisputed advantages of utilizing POCUS, there are several barriers to consider due to challenges related to equipment and technology, experience and skills of the operator(s), availability of documentation templates, electronic storage capability for image archiving, and policies and procedures for quality assurance and billing.[

One component of a plan to increase the use of POCUS at Seoul National University's Bundang Hospital was to develop and implement an education program on POCUS-assisted PE, image acquisition, and decision making in an emergency department (ED). This study assesses the effectiveness of the education element with respect to the rate of POCUS usage and the probable effects on patient management and safety parameters, such as the length of stay (LOS) in ED and return visits (RVs).

Materials and methods

Study design

A quasi-experimental, uncontrolled before-and-after study design was used to evaluate the effects of this education program in ED patients at the selected urban tertiary teaching hospital, which has 1100 beds and receives approximately 80,000 ED visits annually. This study was approved by the hospital institutional review board, and the requirement for informed consent was waived.

Study population and setting

The education program was initiated on January 1, 2016, which served as the start of the “before” period (to December 31, 2015); the period from January 1, 2016 to December 31, 2016, served as the “after” period. We used electronic medical records to identify eligible patients and reviewed the records to determine inclusion or exclusion, with specific exclusion criteria as follows. All patients aged 18 years or older who arrived at the ED between 9:00 am and 5:00 pm during weekdays with the chief complaint of new-onset symptoms such as chest pain, abdominal pain, unidentified shock, syncope, and shortness of breath were included. We also excluded patients who underwent other imaging modality such as a standard X-ray, computed tomography, and magnetic resonance imaging before POCUS. We also excluded patients who visited the ED between 5:00 pm and 9:00 am, and who visited the ED during weekends because duty schedules at night and on weekends were not consistent. Patients who visited the ED due to trauma were also excluded, because extended focused assessment with sonography for trauma had already been widely used in the selected ED.

Development of education program for POCUS-assisted PE in the ED

The committee of emergency physicians at our institution developed an education program for POCUS-assisted PE to target patients with specific new-onset symptoms in the ED, based on published literature regarding POCUS. All members of this committee were board-certificated emergency physicians with more than 5 years of experience using POCUS. In-person committee, meetings were held to design the education program following a structured document search and literature review. The final program comprised indications, differential diagnosis, image acquisition, and effectiveness of POCUS-assisted PE (Table 1).[

Table 1

Educational program for point-of-care ultrasound-assisted physical examination.

Education performance and monthly feedback for physicians

We conducted the program through monthly sessions (1 h/d, 3 days/mo), during which time the committee of emergency physicians analyzed the charts of all patients subjected to POCUS. Education program had been conducted as a workshop every month and 90 physicians had been involved. The total numbers and trends of POCUS, LOS, and RVs were reported monthly to all emergency physicians at our institution. There was no incentive to use POCUS.

Methods of measurement

Patients’ medical records from January 1, 2015 to December 31, 2016 were collected by searching the clinical data warehouse, which enables access to all medical records within a center. We searched patients’ medical charts using a standardized data collection query that included demographics, presenting signs and symptoms, PE findings, utilized radiographic studies and their findings, timing of RVs, mortality, morbidity, and LOS. Our institution achieved a Stage 7 on the Electronic Medical Record Adoption Model scale (developed in 2010 by the Healthcare Information and Management Systems Society Analytics). At a Stage 7 achievement level, care coordination throughout the hospital is improved by data warehousing, which enables the capture and analysis of care data for performance improvement and clinical decision advancement.

Data analysis

Data were analyzed using Stata statistical software, version 14.2 (Stata Corp LP, College Station, TX). Student t test was used for comparisons of continuous variables involving independent samples with normal distributions. For continuous data that did not follow a normal distribution, a nonparametric analysis was performed. The 95% confidence intervals were also calculated, and Fisher exact test was used for categorical data distributions. All negative values were 2-tailed, and a P value of <.05 was considered statistically significant for all tests.

Piecewise regression was performed to evaluate dynamic changes in LOS and RV rate following implementation of this education campaign in the ED.[ A regression model was used to determine the following parameters of LOS and RV rate: slope of the “before” period, slope of after education, difference in slopes between the before and after periods, and net effect of the education, estimated as the difference between the fitted and expected rates at the beginning of the before period if the slope was uninterrupted by the education.

Results

Patient characteristics and performing POCUS in the ED

A total of 16,942 patients were included during the 12-month before education period, and 16,287 patients were included during the 12-month after education period (Fig. 1). The ratio of male gender and mean age were significantly higher during the “after” period (51% vs. 55%, P < .001 and 57.32% vs. 60.96%, P < .001) (Table 2). The total number and rate of POCUS was significantly increased after exposure to the educational program (1066 [6%] vs. 8201 [50%], P < .001). For example, 65 times more POCUS was performed on patients with “undifferentiated shock” (15 [1%] before vs. 1035 [65%] after, P < .001).

Figure 1

Patients’ enrollment before and after education program for point-of-care ultrasound (POCUS).

Table 2

Patients’ characteristics before and after exposure to the education program for point-of-care ultrasound-assisted physical examination.

Differences in LOS X-ray before and after participation in the program

LOS decreased significantly after exposure to the education program (Table 2). Piecewise regression was performed to compare the slopes of LOS change between the before and after periods (Fig. 2). The slope of LOS exhibited a negative trend before the education and changed to more significantly negative trend after the education (−0.19 vs. −2.86, P  =  .004) (Table 3).

Figure 2

Piecewise regression of the length of stay (LOS) before and after education program for point-of-care ultrasound.

Table 3

Piecewise regression of the length of stay and rate of return visits before and after exposure to the education program.

Differences in the rate of RVs before and after the education

RVs to the ED were compared as representative indicators of patient care. The RV rate was significantly reduced after the education (Table 2). Piecewise regression was performed to compare the slopes of RV rate change before and after exposure to the education program (Fig. 3). The slope of RV trend after education was changed to significantly more negative (−0.23 vs. −2.17, P  =  .035) (Table 3).

Figure 3

Piecewise regression of the return visits before and after education program for point-of-care ultrasound.

Discussion

This study illustrates the effectiveness of an education program for POCUS-assisted PE in the ED. We achieved a significant increase in the total POCUS number and rate, and noted a downward shift in the slope of LOS and RVs relative to the before education period.

Training of operators is necessary for POCUS to be utilized correctly on patients, with barriers to POCUS adoption including insufficient faculty training, high cost of ultrasonography machines, and time required to train physicians.[ The relatively high level of operator dependency compared with other diagnostic testing is reasonably expected, given the multiple skills required to perform a POCUS examination. First, a POCUS examination begins with a question about clinical findings and a decision-making process about whether to utilize POCUS to answer this question. Second, acquisition of images requires knowledge of sonographic windows, ultrasound physics, and hand-eye-brain coordination to obtain optimized image quality. Third, interpretation of POCUS images requires operators’ recognition about artifacts that may be encountered during image acquisition and interpretation. Finally, POCUS findings must be interpreted and integrated with other clinical data to effectively guide clinical decision making. Failure during any step of this process may undermine the true value of using POCUS.[

The skills needed to perform POCUS examinations have not been uniformly taught in undergraduate or graduate medical education and there has been no consensus on the training required to reach adequate POCUS competency levels.[ The skills and knowledge required by each hospital or institution are unique, but a lack of basic guidelines has been a burden in creating educational programs such as the one used in this study. Previous studies regarding barriers to POCUS implementation have also described factors such as a lack of time and/or training, and cost of training and equipment.[ Overall, a lack of robust evidence on the effectiveness of POCUS might be an important barrier to encourage use of the technology.

Research about POCUS reveals shifts in focus, from diagnostic accuracy to demonstration of improved health outcomes. Use of POCUS to guide bedside procedures has been reported to reduce procedure-related complications, including arterial punctures during central venous catheterization, post-thoracentesis pneumothorax, and postparacentesis bleeding complications, along with the costs and LOS associated with these complications.[ Few studies have shown the effectiveness of POCUS in shortening decision-making processes and improving safety parameters such as RV rate. This study showed reduced LOS and RV rate significantly, and our results might support the routine use of POCUS for diagnostic evaluations, especially in EDs.

This study had several limitations. First, this was a before-and-after study conducted at a single center with historical controls; therefore, our results cannot be generalized to other institutions. Second, the statistical model might not have captured the ceiling effect or attenuation of the increasing preintervention trajectory; accordingly, the modeled net effect might be greater than would be clinically observed. Third, the time period of only 1 year after the education might limit the significance and validity of the piecewise regression analysis. Fourth, we did not perform implementation of documentation of POCUS due to a lack of knowledge about how to increase proper documentation of findings. Finally, a very important limitation to this is that the ED physicians were not blinded to the goals of this study and were given constant feedback on LOS and RV performance (in addition to the utilization of POCUS). While one can certainly conclude that the POCUS program did not result in increased LOS or RVs, due to the Hawthorne effect, one really cannot conclude the POCUS program or increased POCUS utilization resulted in decreased LOS or RVs.

In conclusion, the implementation of an education program related to POCUS increased its use in the target ED and resulted in successfully improving patient safety and management processes, as determined by LOS and RV rate in this ED.

Acknowledgment

The authors thank the Medical Research Collaborating Center at Seoul National University Bundang Hospital for statistical analyses.