Abdominal aorta measurements by a handheld ultrasound device compared with a conventional cart-based ultrasound machine.

BACKGROUND: Ultraportable or pocket handheld ultrasound devices (HUD) may be useful for large-scale abdominal aortic aneurysm screening. However, the reproducibility of measurements has not been compared with conventional cart-based ultrasound machines.
OBJECTIVES: Investigate the intra- and inter-operator reproducibility of a HUD compared with a conventional ultrasound machine for aortic screening.
DESIGN: Analytical, cross-sectional.
SETTING: Ultrasound department at a large tertiary care hospital in Riyadh. PATIENTS AND METHODS: Eligible male participants aged ≥60 years were invited to participate upon arriving for a non-vascular ultrasound appointment. Three repeated anteroposterior measurements of the transverse aorta were made at the proximal and distal locations for each machine before repeating the measurements on a subset of participants by a second blinded operator. Intraclass correlation coefficients (ICC) and the Bland-Altman method were used to analyze reproducibility. MAIN OUTCOME MEASURE: Inter-system and intra- and inter-operator ICCs. SAMPLE SIZE: 114 males with repeated measurements by second operator on a subset of 35 participants.
RESULTS: The median age (interquartile range) of participants was 68 years (62-74 years). The intra- and inter-operator ICCs were all >0.800 showing almost perfect agreement except for the inter-operator reproducibility at the proximal location using a conventional machine (ICC= 0.583, P=.007) and the Butterfly device (ICC=0.467, P=.037). The inter-system ICCs (95% CI) were 0.818 (0.736-0.874) and 0.879 (0.799-0.924) at the proximal and distal locations, respectively. The mean difference in aortic measurement between the ultrasound systems was 0.3 mm (1.7%) in the proximal location and 0.6 mm (3.6%) in the distal location. In total, >91% of the difference in measurements between the machines was <3 mm. The mean scanning time was 4:16 minutes for the conventional system and 3:53 minutes for the HUD (P=.34).
CONCLUSIONS: Abdominal aortic screening using a HUD was feasible and reliable compared with a conventional ultrasound machine. A pocket HUD should be considered for large-scale screening. LIMITATIONS: No cases of abdominal aortic aneurysm in the sample and lack of blinding. CONFLICT OF INTEREST: None.

INTRODUCTION

Abdominal aortic aneurysm (AAA) is an asymptomatic but serious condition defined as aortic enlargement with a diameter of 3 cm or larger. It is associated with fatal complications such as aortic rupture with a mortality estimated as high as 81%.[1] The prevalence has declined in European countries from 3.9%–7.2% in the last century compared with recent estimates from 1.2% to 3.3% in persons 60 years or older.[2,3] However, the prevalence in developing countries is less frequently reported. Studies from Saudi Arabia reported a range of AAA prevalence from 0.3% to 7.0%.[4,5]

Guidelines recommend ultrasonography as the imaging technique for AAA screening due to its safety, patient acceptance, and accuracy compared with physical examination.[6] However, this screening recommendation is hindered by several factors such as infrequent systematic screening campaigns and the burden on primary care physicians.[7] Moreover, the deployment of a large-scale screening program is conditional on the availability of sonographers and competent ultrasound operators.[8] The financial costs of the scans are another important barrier.[9] The expense and bulkiness of standard ultrasound machines is one of the main obstacles to applying AAA screening guidelines.[7] Studies suggest that large systematic AAA screening programs can be successfully undertaken by using handheld ultrasound devices (HUD).[10-12] compared with traditional ultrasound machines, the advantages of HUDs include price, size, and user-friendly interfaces for novice operators. HUD can also enable primary care physicians to perform bedside screening, which can significantly improve cost-effectiveness and save time.[12] However, this may be hindered by lack of motivation to perform the examination and physician reimbursement.

While numerous HUDs are available, two systematic reviews on diagnostic performance and reproducibility have been limited to Vscan (GE) and Lumify (Phillips) devices, which are more expensive handheld ultrasound devices.[13,14] Newer HUD devices, such as the Butterfly iQ+, use modern microchip technology to generate and detect ultrasound waves. The cost is reduced by more than 50% by substituting traditional piezoelectric elements with capacitive micromachined ultrasound transducers. This technology allows a single probe to operate at different ultrasound frequencies and beam profiles, negating the requirement for several probes. Previous studies have demonstrated its diagnostic performance in inflammatory arthritis,[15] giant cell arthritis,[16] and lung scanning in COVID-19.[17] The performance of this device in AAA screening compared with conventional machines has not been investigated. We hypothesized that a HUD can generate a reproducible abdominal aortic measurement equivalent to a conventional ultrasound system. The objectives of this study were to investigate the intra- and inter-operator reproducibility of a HUD as well as the inter-system reproducibility compared with a conventional ultrasound machine.

PATIENTS AND METHODS

This analytical cross-sectional study was conducted between October 2020 and March 2021 at the ultrasound department at Prince Sultan Military Medical City in Riyadh Saudi Arabia. Participants were selected consecutively by invitation from patients attending for routine scans. The study was explained to them after the clinical scan and participants provided written informed consent. Participants were eligible to participate if they were male, aged 60 years or older, and were able to provide consent by themselves or an authorized guardian. They were excluded if they had a history of AAA or any aortic surgery. Fasting was not a condition of participation. Collected data included age, sex, weight, height, history of smoking, diabetes, hypertension, hypercholesterolemia, and previous vascular surgeries. The study was approved by the research ethics committee at Prince Sattam bin Abdulaziz University (RECHSD-001-2020) and the institutional review board at PSMMC (IRB approval no: 1437).

Ultrasound systems and protocol

All participants were scanned using a Butterfly iQ (Butterfly Network Inc., Guilford, Connecticut, USA) and an EPIQ 7 (Philips Medical Systems, Bothell, Washington, USA), a conventional machine that uses a 5–1 MHz curved array transducer. The Butterfly iQ HUD employs capacitive micromachined ultrasonic transducers to emit and receive ultrasonic waves using one standard probe with a frequency range of 1–10 MHz. The HUD was connected to a 12.9” tablet (iPad Pro iOS 14.6, Apple, California, USA) to control and acquire the images.

Ultrasound scanning was conducted by two accredited sonographers (AIA and SOA). The first (AIA) had 10 years of experience in medical sonography and the second (SOA) had 4 years of experience. They were both experienced in aortic scanning and reviewed the study's protocol prior to data collection. The second sonographer only scanned a subset of the participants to evaluate inter-observer reproducibility.

The whole abdominal aorta was screened for AAA. However, the study protocol collected images at two locations: proximal (distal to the diaphragm near the celiac artery) and distal (distal to the renal arteries and above the iliac bifurcation). The measurements were acquired using the transverse plane. The measurement calipers were placed at the anterior outer edge to the posterior outer edge of the aorta.[18] Three measurements were acquired for each location to evaluate for intra-operator reproducibility. The probe was lifted and replaced again for the repeated measurements. The first measurements were acquired by the HUD followed by the conventional system. The two operators were blinded to each others' measurements. Based on the literature,[19-21] an a priori cut-off of ≤4 mm for the difference between the observers represented good agreement. Finally, the total time for each scan was calculated as the time difference between the first and last images.

Sample size

The sample size determination was analyzed using the PASS 2021 (Power Analysis and Sample Size Software. NCSS, LLC. Utah, USA). A sample size of 110 subjects with two observations per subject was required to achieve 90% power to detect an intraclass correlation of 0.45 under the alternative hypothesis where the intraclass correlation under the null hypothesis is 0.20 using an F-test with a significance level of 0.05.

Statistical analysis

Descriptive and inferential statistics were done using IBM SPSS version 27 (Armonk, NY: IBM Corp). The data was first evaluated for normality using histograms and the Shapiro-Wilk test. Quantitative variables are presented as the mean and standard deviation (SD) with 95% confidence intervals (95% CI). The intra- and inter-reader reproducibility were analyzed using the Bland-Altman plot and intraclass correlation coefficients (ICC). For ICC, the results were interpreted as follows: .00–.20, ‘poor agreement’; .21–.40, ‘fair agreement’; .41–.60, ‘moderate agreement’; .61–.80, ‘substantial agreement’; and >.80, ‘almost perfect agreement’.[22] The limits of agreement were calculated as the mean difference plus and minus 1.96 times the standard deviation of the differences.[23] Bland-Altman plots were drawn to compare the agreement between the two systems.

RESULTS

The 114 male subjects had a median age of 68 years (range, 60 to 107 years, interquartile range of 62–74 years). The mean (SD) of the body mass index was 27.3 (3.7 kg/m2). There were 42 participants (36.8%) with hypertension of which 35 (30.7%) were on anti-hyper-tensive medications, 46 (40.4%) had type II diabetes mellitus, 12 (10.5%) had coronary artery disease, and 12 (10.5%) were present (n=3) or past (n=9) smokers. The abdominal aorta was successfully measured in all participants using the conventional system and the HUD (, ). No participant had an AAA ≥30 mm. However, five participants had aortic ectasia (≥25 mm). Examples of the images are shown in . There was no statistically significant association between the AAA measurements for either system and demographic and clinical variables including age, body mass index, smoking, diabetes, hypertension, hypercholesterolemia, and previous vascular surgeries (P≥.2).

Figure 1.

Median (IQR) aortic measurement at proximal (left) and distal (right) locations using the conventional system and the hand-held ultrasound device.

Table 1.

Mean and limits of agreement for the aortic measurements at each location using both devices.

	Operator A (n=114)	Operator B (n=35)	Mean difference	Percentage of difference	Limits of agreement
Proximal location, conventional system	19.2 (2.3) [18.7–19.6]	20.1 (1.8) [19.5–20.8]	−0.2	−1.0%	−4.2, 3.9
Distal location, conventional system	17.4 (2.4) [16.9–17.8]	18.3 (1.9) [17.6–19.0]	−0.1	−0.5%	−3.1, 2.9
Proximal location, handheld device	17.9 (2.5) [18.5–19.4]	19.8 (1.8) [19.2–20.4]	0.3	1.5%	−4.3, 4.8
Distal location, handheld device	16.8 (2.5) [16.3–17.3]	18.0 (2.1) [17.2–18.7]	−0.2	−1.1%	−3.3, 2.9

The data are presented as mean (SD) and 95% CI in mm.

Figure 2.

Aortic ultrasound from the proximal and distal locations on 61-year-old (a,b) and 81-year-old (c,d) patients using the conventional system (a,c) and the handheld device (b,d).

The mean difference in aortic measurement between the ultrasound systems was 0.3 mm (1.7%) in the proximal location and 0.6 mm (3.6%) in the distal location with limits of agreement of -3.4, 3.9 and -2.3, 3.5 respectively. The Bland-Altman plots for the agreement between the systems are shown in . The repeated measurements by the second operator were done on a subset of 35 participants. All reproducibility metrics showed an almost perfect agreement (ICC >.80), except for inter-operator reproducibility at the proximal location where the agreement was fair for both systems. The ICC reliability coefficients are listed in . More than 93% of the difference in measurements were below the 4 mm cutoff, representing good agreement between the two systems. presents these results with more stringent cutoffs. The total time to acquire three repeated readings of the proximal and distal abdominal aorta was on average 4:16 minutes for the conventional system and 3:53 minutes for the HUD (P=.34).

Figure 3.

Bland-Altman plots for the agreement between the two ultrasound systems in the proximal (left) and distal (right) locations. Horizontal lines are drawn at the mean difference (black), and at the limits of agreement (red dashed).

Table 2.

The intraclass correlation coefficients for the reproducibility within and between the ultrasound systems.

Reliability metric	ICC	95% Cl	P value
Intra-operator reproducibility (Proximal location, high-end system)	.938	.916–.956	<.001
Intra-operator reproducibility (Distal location, high-end system)	.954	.938 – .967	<.001
Intra-operator reproducibility (Proximal location, handheld device)	.938	.915–.955	<.001
Intra-operator reproducibility (Distal location, handheld device)	.949	.930 – .963	<.001
Inter-operator reproducibility (Proximal location, high-end system)	.583	.166–.790	.007
Inter-operator reproducibility (Distal location, high-end system)	.816	.634 – .907	<.001
Inter-operator reproducibility (Proximal location, handheld device)	.467	.062 – .732	.037
Inter-operator reproducibility (Distal location, handheld device)	.834	.671 – .916	<.001
Inter-system reproducibility (Proximal location)	.818	.736–.874	<.001
Inter-system reproducibility (Distal location)	.879	.799 – .924	<.001

All ICCs are for average measures.

Table 3.

Rates of pairs of measurements between the two systems with ≤4 mm, ≤3 mm, ≤2 mm, and ≤1 mm difference.

Location	Pairs of measurements with a difference
	≤4 mm	≤3 mm	≤2 mm	≤1 mm
Proximal	93.9%	91.2%	79.8%	49.1%
Distal	98.2%	94.7%	76.3%	47.4%

DISCUSSION

To our knowledge, this is the first study to compare the performance of the Butterfly IQ HUD to a conventional ultrasound system. The results demonstrated that this HUD can be used to accurately measure the abdominal aorta. The measurements of inter-operator reproducibility were excellent except at the proximal location. This could be attributed to the depth and overlying organs in the proximal location.

Few studies have compared HUD to conventional systems in the abdominal aorta. Dijos et al[24] investigated the V-scan (GE Healthcare, Wauwatosa, WI, USA) and demonstrated a mean bias of 0.33 mm, which is in excellent agreement with our 0.30 mm detected difference. Using correlation for inter-system reproducibility,[25] they reported a high correlation coefficient (r=0.98) demonstrating a strong association between the two devices. However, it should be noted that a high correlation does not necessarily indicate a good agreement. Later, Bonnafy et al[10] used the same system to investigate the vScan reliability in the hands of novice operators. They demonstrated a ≤4 mm inter-operator variability between expert operators using a conventional system compared with novice operators using the HUD in 92.0% of cases. Previous research in 2007 found a <5 mm absolute difference of aortic diameter in 88% of patients using a dated and bulky HUD (SonoSite Titan, SonoSite Inc., Bothell, Washington).[26] However, the limits of agreement were wide (-6.6, 9.4 mm), exceeding acceptable limits of clinical acceptability of approximately ± 4–5 mm. In another study, which compared the aortic measurement acquired by two expert cardiologists using a HUD and a standard device on 110 patients, a high agreement with a kappa coefficient of 0.88 was reported.[27] In addition, a study by cardiologists reported a perfect measurement (agreement of 1) using the Vscan device.[28] Such results highlight the technology breakthroughs and recent advancements in HUD, which improved the spatial resolution of such scans to accurately depict smaller acoustic impedance mismatches in tissue.

Our results support the use of HUD as a cheaper and non-inferior alternative to conventional machines for AAA screening. This might be especially useful in primary or home care settings with limited funding. However, further research is needed in these settings to verify clinical feasibility and cost-effectiveness. HUDs are known for their smaller footprints and user-friendly interfaces. They can be performed at the bedside to complement the clinical physical examination.[24] Several previous studies demonstrated the feasibility and diagnostic accuracy of HUD in the hands of inexperienced operators, medical residents, and primary health care physicians.[10,11] This is extremely valuable considering the inferior diagnostic accuracy of AAA physical examination.[29] Our results showed that the HUD exam had a relatively similar mean scanning time compared with a standard ultrasound system. This suggests that the operators did not spend extra time interpreting the images due to significantly lower spatial resolution or struggle with image acquisition. Moreover, in a previous study in opportunistic AAA screening as the cost was lower and quality-adjusted life years were increased compared with standard care.[12]

Our study was not originally designed or powered to determine the prevalence of AAA in Saudi Arabia. Nevertheless, we observed no AAA in our sample. This finding may be because 38.5% of the participants were between 60–64 years old. This age range is slightly younger than the age group recommended for screening (≥ 65 years) by the United States Preventive Services Task Force[30] and the National Health Service abdominal aortic aneurysm screening program (NAAASP) in England.[31] To date, Saudi Arabia has no national AAA screening program. In 1996, Alzahrani et al[4] reported a 1.8% overall prevalence and 7.0% prevalence in patients with peripheral vascular disease. More recently, Alhaizaey et al[32] reported a higher prevalence at 2.9% in a sample of 701 patients. In contrast, a retrospective review of 2032 abdominal computer tomography scans of Saudi male patients at age 65–75 years showed a significantly lower prevalence at 0.3%.[5] Qatar, a neighboring country with similar demographics, has a significantly higher AAA prevalence of 5% in patients >80 years old.[33] It is yet unclear whether AAA in Saudi Arabia is underdiagnosed, or the prevalence is considerably smaller than international numbers. More research on this important topic is warranted.

Our study has multiple limitations. We did not record the number who declined to participate due to the load of clinical exams. Despite having few aortic ectasia cases, our sample did not include large AAA where the margin of difference between the two machines can be slightly higher due to the larger scale of the measurements. However, considering the small percentage we found of measurements with >3 mm difference, a large AAA at risk of rupture should not be missed by the HUD. The tablet connected to the HUD had a screen size larger than common tablets or smart-phones, which may have improved the discernibility of the aorta. We did not investigate the performance of the HUD in the hands of novice operators or clinicians. Moreover, it was not possible to blind the operator to the aortic measurements when repeating the scan on the HUD. However, this should not present a significant bias as the measurements appear only after the calipers are placed by the operator. The proximal measurements acquired by the device could have been affected by overlying bowel and gastric gasses. Moreover, we did not evaluate the iliac arteries in our protocol, where potential AAA might extend. Finally, dual measurements by two operators were not completed on all participants due to the heavy workload of clinical scans. Future research should investigate the reproducibility between different HUD models. Positive results can support their interchangeability.

In conclusion, the HUD showed almost perfect reproducibility within and between operators and compared with a conventional ultrasound system in the assessment of abdominal aortic diameter of normal cases. Further work is needed to determine its performance in patient populations. Nevertheless, the results support its use as an inexpensive solution for large-scale AAA screening due to its feasibility, accessibility, and most importantly, accuracy when compared with a conventional ultrasound system. Further studies are needed to determine the clinical outcomes and cost-effectiveness of AAA screening in countries such as Saudi Arabia.