A comparison of handheld ultrasound versus traditional ultrasound for acquisition of RUSH views in healthy volunteers.

Few studies evaluate the use of handheld ultrasound devices for point-of-care ultrasonography in the emergency department. We hypothesized that image acquisition time and image quality are similar between a handheld device and a traditional device. We compared these 2 types of devices in healthy, non-pregnant adults with using a crossover non-inferiority design while acquiring Rapid Ultrasound for Shock and Hypotension (RUSH) view. We excluded those with a history of surgical intervention or known abnormality to the lungs, abdomen, or pelvis. Images were compiled into a de-identified video clip reviewed for image quality by 2 blinded reviewers. Cohen's Kappa was used to determine interrater agreement. Disagreements were adjudicated by an independent physician. Imaging time was compared using a paired Student's t test. Of 59 screened participants, 9 were excluded. Most subjects (N = 30, 60%) were female with a mean age of 39 (Range: 19-67) years. The median time to complete the RUSH exam did not differ (handheld 249.4, interquartile range 33.5 seconds); traditional 251.4, interquartile range 66.3 seconds); [P = 0.81]). Agreement between ultrasound reviewers was good (agreement 83%; k = 0.69; 95% CI, 0.49-0.88). Images were determined to be of adequate quality for interpretation in 41/50 (82%) and 43/50 (86%) in the handheld and traditional devices, respectively (P = 0.786). Neither time to image acquisition nor image quality differed between the handheld and traditional devices. The handheld device may be an alternative for use in RUSH exams.

INTRODUCTION

Background

The use of bedside ultrasound in the practice of emergency medicine has become ubiquitous, in part because of the miniaturization of equipment and the expansion of residency training programs to include training on ultrasound. , Point‐of‐care ultrasonography (PoCUS) has multiple potential applications in the emergency department (ED), including the assessment of volume status, assessment of cardiovascular anatomy and function, and in trauma patients using the Focused Assessment with Sonography in Trauma (FAST) exam, along with the similar assessment of shock etiology in the undifferentiated medical patient, known as the Rapid Ultrasound in Shock and Hypotension (RUSH) Exam. , , Both the ultrasound technique and devices continue to evolve. Recently several economical handheld devices have entered the market promising diagnostic quality images that connect to existing smartphones or use a proprietary tablet for image acquisition. One such device is the Butterfly iQ. This particular device advertises a probe configuration that allows for multiple configurations of the single probe, facilitating the RUSH exam, obtaining vascular access, and as guidance for other bedside procedures.

Importance

Despite the rapid early adoption of the device by some emergency physicians, there is little published research examining the quality of the images provided by these new devices. To our knowledge, there are no comparative studies in the literature. Although anecdotal evidence, published in the form of tutorials and videos online, suggests that the device is practical and provides reasonable image quality, no head‐to‐head study of any kind has compared image quality obtained with traditional cart‐based devices with those from handheld devices.

Goals of this investigation

This project aims to determine if a new type of commercial handheld ultrasound device is capable both in terms of usability and imaging quality when acquiring the core views needed for a RUSH exam in the emergency department. We accomplished this using a crossover non‐inferiority randomized control study.

METHODS

This study was approved by the Guthrie Clinic Institutional Review Board. All subjects provided written informed consent prior to any imaging procedures.

Study design and materials

We used a crossover non‐inferiority randomized control study design to compare the handheld to the traditional device. Subjects were randomized to undergo either handheld or traditional image acquisition first. RUSH views acquired included a parasternal long‐axis, apical 4‐chamber, inferior vena cava (IVC), hepatorenal interface with diaphragm view, splenorenal interface with diaphragm view, bladder, aorta to the bifurcation of the iliac arteries, and bilateral lung apices in both B mode and M mode. Images were acquired in the RUSH sequence with whichever device that was randomized to be first followed by the same sequence of images acquired using the other device (See Figure 1).

FIGURE 1

Participant flow diagram RUSH, Rapid Ultrasound for Shock and Hypotension

The Bottom Line

A big question on the minds of current and potential point‐of‐care ultrasonography practitioners is how good are the images from handheld ultrasound devices when compared to the cart‐based ultrasound machines we are used to using? This study attempted to compare a popular handheld ultrasound device to a common cart‐based system in the performance of a Rapid Ultrasound for Shock and Hypotension (RUSH) protocol (an ultrasound examination designed for evaluating hypotensive patients). Image quality for interpretability was compared between the 2 devices and no significant difference was found, suggesting a handheld ultrasound may be good enough for RUSH examinations.

The ultrasound devices used included the Butterfly iQ device (Butterfly Network, Guilford, CT, USA) connected to an iPad Pro (Apple Corporation, Cupertino, CA, USA), and the Phillips Sparq Ultrasound System (Phillips, Amsterdam, Netherlands). These devices were used with factory standard settings. Factory presets were used to acquire imaging series. For both devices, all images, except for the lung images, were acquired using the FAST exam preset. The lung views were acquired in the respective lung image preset for each device with an acquisition of a B Mode and M Mode clip. Images were acquired from both devices using room temperature Aquasonic 100 Ultrasound Transmission Gel (Parker Laboratories, Fairfield, NJ, USA).

Selection and participants

Healthy subjects were recruited from the local hospital system via email and word of mouth advertisement. Our inclusion criteria included adults age >18 years without current pregnancy. Exclusion criteria were a history of previous surgical intervention or known abnormality to an area imaged during the RUSH protocol (lungs, abdomen, or pelvis).

Sample size

Although there are no prior data comparing these 2 devices, prior literature has demonstrated an interpretability of 89% in resident ultrasonographers. The variance was not reported. Assuming a variance of 10% in this cohort, a sample size of 50 would be able to detect a 6% difference in interpretability between handheld and traditional groups with power of 0.8 and alpha of 0.05. (Stata 14.2, Stata Corp., College Station, TX, USA; 2 sample means test).

Measurements

Our primary outcome was the portion of overall scans determined to be adequate for interpretation. Our secondary outcome was time to acquire the complete series of images and videos for the RUSH exam. The scans were all acquired by one ultrasound‐trained emergency medicine resident who had previously completed a 4‐week ultrasound training rotation and had completed 200 scans before beginning this study. The ultrasonographer was able to adjust depth, gain, and positioning of the probe and patient during individual views to optimize acquisition, such as they would during real‐life application of the technology. All images on the handheld device were obtained using the single probe. Lung images on the traditional device were obtained using the LI2‐4 MHz transducer for B and M Mode. All other images on the traditional device were obtained using the C6‐2 MHz transducer.

Image quality and interpretation by reviewers

After image acquisition, the images were compiled into a complete video with all imaging information removed to blind reviewers (see web appendix videos 1 and 2 for representative video clips of both a handheld and traditional scan). The order of image review by the reviewers was also randomized. The clips were sent to 2 ultrasound‐credentialed emergency physicians for review to determine if the images acquired were of appropriate quality to answer clinical questions. Each physician was credentialled by the institution to perform, interpret, and teach emergency ultrasound, with a combined 40 years of experience in performing emergency ultrasound. Disagreement between the reviewers was then adjudicated by a blinded third ultrasound‐credentialed emergency physician with a similar level of experience, who was also credentialed to perform, interpret, and teach bedside emergency ultrasound.

The reviewers were given a simple scoring sheet on which they would indicate if a study and its components were of adequate quality for interpretation (see Appendix I); this was our measure of image quality. Adequacy for interpretation was defined as a clip with the requisite resolution and quality to answer the clinical questions posed by that specific element in the RUSH exam; left ventricular function when viewing a parasternal long axis, for example. Their direction was to score the individual views of the study and then provide an overall assessment of adequacy. No specific cutoff was prespecified to the reviewers that they would use to determine if an overall study was adequate. They were directed to conclude the study was adequate if it contained enough interpretable images to be used, in their clinical opinion, in practice. The third‐party adjudicator was given more strict review conditions, so as to ensure minimization of bias, and would grade the study as adequate only if every view was adequate for interpretation (as defined previously).

Analysis

Descriptive demographic statistics were collected and reported for all participants. Mean time to complete the entire sequence of views was collected and compared between both devices using a paired Student'st test. Proportions were compared with Fisher's Exact Test. Interrater agreement was calculated with a simple agreement percentage and with Cohen's Kappa. All data points collected and analyses performed were selected a priori. Statistical significance was determined to be less than P = 0.05. All statistical analysis was completed in R 3.5.1 (R Foundation for Statistical Computing).

RESULTS

A total of 59 participants were screened for enrollment. Nine potential participants were ineligible to participate because of preexisting surgical history or known abnormality to an area imaged during the RUSH protocol. A total of 50 subjects were enrolled. Of those, 30 were female (60%). The mean age of participants was 39 (range: 19–67) years. Randomization of which device first was balanced, with a total of 50% of subjects undergoing imaging with either device first. The median time to complete the entire RUSH series of views was 249.4 seconds (interquartile range [IQR] 33.5 seconds) for the handheld device and 251.4 seconds (IQR 66.3 seconds) for the traditional device. There was no significant difference between the time taken to acquire the image series on the 2 ultrasound devices on paired t test (P = 0.81). There was a non‐significant trend toward decreased time to acquisition of the image series for both the handheld and traditional devices respectively (r2 = 0.199 P = 0.1659 for handheld, r2 = 0.0681 P = 0.6384 for traditional) (see Figure 2). Agreement between ultrasound reviewers was good, with agreement prior to adjudication of 83% (k = 0.69; 95% CI, 0.49–0.88). The 17 studies with a discrepancy between the 2 ultrasound reviewers underwent independent and blinded third‐party adjudication. After adjudication, there was no difference in perceived image adequacy between the 2 devices, as images collected were determined to be of adequate quality for interpretation in 41/50 (82%) and 43/50 (86%) in the handheld and traditional devices respectively (P = 0.786).

FIGURE 2

Time required to complete scan for both devices types over time, from beginning of study to end of data collection, with linear trendlines. R2 = 0.199 for handheld device, and R2 = 0.0681 for traditional device

LIMITATIONS

As with all small studies with healthy subjects, several limitations bear mention. First, our study did not have separate traditional and ultraportable groups. The same subjects were used, and despite having been randomly assigned to one device first, we cannot account for the fact that individual variation in the subjects that was not measured confounds our results. Second, our study had only one ultrasonographer. This was a practical limitation, as our study was a small pilot study initiated by an investigator within a residency program. Although this study design reduces the potential variability between sonographers, it is vulnerable to the Hawthorne effect. Therefore, further research could broaden the design to include multiple ultrasonographers with varying levels of training to validate our findings.

Furthermore, the overall interpretability was 82% and 86% for the handheld and traditional devices, respectively. Although there is no universal threshold for an acceptable rate of interpretability, an ultrasound protocol such as the RUSH exam in the ED, previous work in the ED ultrasound field demonstrates interpretability with a wide range from 82% to 96%. , , It should also be noted that images were marked as interpretable only when a majority of the clips were deemed to be so. This high criterion for interpretability may have lowered the overall number of clinically interpretable studies. Future research could better define a scale or measure for image adequacy to better allow for comparative ultrasound research.

Additionally, our project evaluated only one handheld device against one traditional device. The findings within our study may not be representative of other traditional or ultraportable devices. Finally, our study was on healthy volunteers under artificially stable conditions. The dynamic, sometimes chaotic, nature of resuscitation of critically ill patients is not such an environment; therefore, these findings should be replicated in a clinical population before the findings can be generalized.

DISCUSSION

In our study, time to acquire RUSH views and the adequacy of these images did not differ between traditional and ultraportable ultrasound devices. Time to acquire images was, on average, under 5 minutes for both devices. This is faster than the average time found in a previous study on trainee performance on FAST exams. This could be a result of several factors, but we suspect this is because of the high volume of scans completed by a single sonographer over a relatively short period of time. This is reflected in the downward trend in time to completion for both devices over the span of the study (Figure 2). Reviewers agreed that images were interpretable in the majority of cases. This is critical, given limited data image quality or diagnostic accuracy of handheld devices used at the bedside compared to traditional devices. Although anecdotally the devices appear to be usable in clinical practice and have seen widespread adoption by some clinicians, handheld devices, such as the one used here, have little clinical evidence to back up their use in everyday practice. PoCUS is a key component in the general practice of emergency medicine, covering most organ systems and assisting in the diagnosis and treatment of both stable and critically ill patients alike. , Barriers to broader adoption of this technique include bulk of traditional devices and costs associated with the acquisition of multiple devices. The evolution of handheld devices into convenient, economical, and portable packages is significant interest to emergency medicine and other medical disciplines.

We hope that this initial investigation sparks interest in further comparative research in the use of handheld ultrasounds. As the cost of these devices decrease, broad availability will facilitate PoCUS in emergency medicine.

In conclusion, a handheld ultrasound device yielded similar quality images and time to image acquisition when compared to a traditional cart‐based ultrasound device. Further research should attempt to validate and generalize these findings with other devices, using a broader base of PoCUS ultrasonographers, in a clinical population.

CONFLICTS OF INTEREST

The authors have no conflict of interest to declare.

AUTHOR CONTRIBUTIONS

ZED and JCR conceived of the project. ZED, JW, HH, and AA collected the data. ZED, JCR, LO, and GC reviewed the data, assessed and scored it for the study. ZED and JCR designed the analysis. ZED performed the analysis and interpretation. ZED was responsible for drafting the article. ZED, JW, HH, AA, LO, GC, and JCR were responsible for critical revisions of the article. ZED and JCR were responsible for final approval of the article version to be published.

Supporting Information

Click here for additional data file.