Systematic approach towards reliable estimation of abdominal aortic aneurysm size by ultrasound imaging and CT.

BACKGROUND: The management of abdominal aortic aneurysm (AAA) is fully dictated by AAA size, but there are no uniform measurement guidelines, and systematic differences exist between ultrasound- and CT-based size estimation. The aim of this study was to devise a uniform ultrasound acquisition and measurement protocol, and to test whether harmonization of ultrasound and CT readings is feasible.
METHODS: A literature review was undertaken to evaluate evidence for ultrasound-based measurement of AAA. A protocol for measuring AAA was then developed, and intraobserver and interobserver reproducibility was tested. Finally, agreement between ultrasound readings and CT-based AAA diameters was evaluated. This was an observational study of patients with a small AAA who participated in two pharmaceutical intervention trials.
RESULTS: Based on a literature review, an ultrasound acquisition and reading protocol was devised. Evaluation of the protocol showed an intraobserver repeatability of 1.6 mm (2s.d.) and an interobserver intraclass correlation coefficient (ICC) of 0.97. Comparison of protocolled ultrasound readings and local CT readings indicated a good correlation (r = 0.81), but a systematic +4.1-mm difference for CT. Harmonized size readings for ultrasound imaging and CT increased the correlation (r = 0.91) and reduced the systematic difference to +1.8 mm by CT. Interobserver reproducibility of protocolized CT measurements showed an ICC of 0.94 for the inner-to-inner method and 0.96 for the outer-to-outer method.
CONCLUSION: The absence of harmonized size acquisition and reading guidelines results in overtreatment and undertreatment of patients with AAA. This can be avoided by the implementation of standardized ultrasound acquisition and a harmonized reading protocol for ultrasound- and CT-based readings.

Introduction

An abdominal aortic aneurysm (AAA) is a symptomless yet potentially dangerous disease. It is estimated that rupture of an AAA is responsible for approximately 200 000 deaths per year worldwide. AAA management is dictated fully by the 55-mm consensus intervention threshold in men, and possibly at a slightly smaller diameter in women. Multiple trials have shown no survival benefit of earlier repair, and it has been estimated that the costs of ‘premature’ repair (repair at diameters below the consensus threshold) are close to 1 million US dollars per prevented rupture-related death .

Remarkably, registry and medical intervention trial data indicate that elective AAA repair occurs regularly in patients with an AAA diameter well below the consensus intervention threshold. For instance, the median intervention diameter for patients who had elective endovascular aneurysm repair (EVAR) in the Vascular Quality Initiative was 54 mm. Early AAA repair may result from shared medical decision-making and patient preferences, but it may also involve iatrogenic aspects such as the absence of uniform, harmonized guidelines for AAA size estimation. In fact, there appears to be substantial variability in the measurement and reading methodology in clinical practice. When reviewing patient records for eligibility for the pharmaceutical small aneurysm trials, the Pharmaceutical Aneurysm Stabilization Trial (PHAST) and TElmisartan in the management of abDominal aortic aneurYsm (TEDY) trial, remarkable interhospital and intrahospital, as well as interoperator, variations and inconsistencies were noticed in ultrasound-based AAA size estimation (such as inconsistent measurement plane and direction, and caliper position).

Interpretation of AAA size estimates is further complicated by reported consistent (positive) differences between ultrasound- and CT-based size estimates. Considering the fact that the 55-mm consensus intervention threshold is ultrasound-based, consistently larger estimates for CT-based readings may result in ‘premature’ decisions for AAA repair.

The aim of this study was to develop and evaluate a harmonized ultrasound protocol for the assessment of AAA diameter in the context of longitudinal preintervention follow-up and medical decision-making.

Methods

A literature review was performed to evaluate the available evidence with regard to aspects of ultrasound-based methodology, measurement reproducibility, and variability in AAA diameter estimation. Then, based on the acquired best evidence, a protocol for the measurement and reporting of AAA diameter for ultrasonography was developed, and its intraobserver repeatability and interobserver reproducibility were tested. Finally, based on this measurement protocol, the agreement between protocolized ultrasound readings and CT-based AAA diameters was evaluated. The ultrasound protocol was developed in preparation for the PHAST. The comparison of ultrasound and CT readings was performed as a post hoc analysis of data acquired in the PHAST and TEDY trials.

Literature review

Publications that reported ultrasound measurement methodology and reproducibility of AAA diameter were identified using a search of MEDLINE and PubMed for reports published between 1966 and 30 April 2020. The Medical Subject Headings (MeSH) search terms included abdominal aortic aneurysm, abdominal aortic aneurysm/ultrasonography, ultrasonography, screening methods, reproducibility of results, and observer variation. Text keywords included abdominal aortic aneurysm, AAA, ultrasound, sonography, repeatability, reliability, and reproducibility. In addition, studies were identified through a manual search of references of initially identified articles, reviews and commentaries.

Inclusion criteria for the literature review were as follows: peer-reviewed publication; comparison of two or more aspects of the ultrasound measurement methodology; evaluation of small AAAs (maximum diameter 30–55 mm); research on humans; at least two observers.

Two independent researchers identified and evaluated the studies for eligibility. Based on the literature recommendations, an ultrasound measurement and reading protocol was established and then tested for interobserver reproducibility and intraobserver repeatability.

Ultrasound methodology and protocol

The present study included patients involved in two RCTs. Study outlines for these trials have been published previously,. In the PHAST, abdominal aortic ultrasound images were acquired and subsequently measured on a mobile ultrasound device (Siemens Acuson P50™, 1.67–4.0-MHz phased-array transducer; Siemens Healthcare Nederland, The Hague, Netherlands). In the TEDY trial, abdominal aortic ultrasound images were acquired and measured using the Siemens Acuson P300™ mobile US system, 1.0–8.0-MHz transducer; Siemens Healthcare Nederland). A standard ultrasound protocol was developed and used to estimate intraobserver repeatability and interobserver reproducibility. All ultrasound images were acquired and subsequently measured on the same ultrasound machine, at the bedside with the patient in supine position. Results were recorded and analysed digitally afterwards.

Ultrasound interobserver reproducibility

To test the interobserver reproducibility of the ultrasound measurement protocol, patients under surveillance for a small AAA (35–50 mm) and participating in the PHAST were invited consecutively for participation in the radiology department. Four observers, two radiologists and two radiology residents, all with at least 3 years’ experience in abdominal and vascular ultrasound imaging, measured all patients on the same day using the standard ultrasound protocol. The observers were fully blinded to one another’s image acquisition and measurements.

Ultrasound intraobserver repeatability

For this study arm, a random sample was taken from the study population of patients with a small AAA (35–50 mm) in the PHAST. AAA measurements were performed twice on the same day, using the ultrasound protocol, under standardized conditions. The study physician was blinded to the results of both measurements (placing of the calipers on the aortic wall without reading the results). All measurements were obtained and read by one investigator, a trained physician who performed all baseline and follow-up measurements in the PHAST.

Evaluation of agreement between protocolized ultrasound- and CT-based AAA diameters

To test CT reproducibility, baseline ultrasound and CT images of participants in the TEDY trial were used. In this study, CT scans were acquired in four hospitals in the Netherlands as part of the TEDY trial protocol, using a variety of scanner brands and types. All scans had a minimum in-plane resolution of 1 mm. Local CT readings reflected the AAA diameters reported by local observers in the patient records, measured as maximal infrarenal diameter in any plane. Next to the local CT readings, centralized CT readings were performed on dedicated picture archiving and communication systems workstations (Sectra, Linköping, Sweden) using multiplanar reconstruction. Images were stored electronically. Post hoc central CT readings (largest anterior–posterior (AP) diameter in inner-to-inner wall and outer-to-outer wall) were performed by two independent observers (two radiology residents with several years’ experience). Observers were blinded to each other’s reading and to the local reading. Matching AAA ultrasound readings were all performed by a single trained investigator.

This resulted in a data set containing six paired AAA size readings: central ultrasound reading, local CT reading, and four central CT readings (2 observers using 2 methods).

Statistical analysis

AAA measurements are expressed as mean(s.d.) millimetres.

In the literature review, reported interobserver reproducibility and intraobserver repeatability outcomes were converted to standard deviations of the difference and limits of agreement.

Ultrasound intraobserver repeatability was defined as two times the standard deviation of the differences of measurements of the same observer. The interobserver reproducibility was calculated using the intraclass correlation coefficient (ICC). An ICC of 0.75 or above was considered to indicate good agreement. Comparison of mean differences in AAA diameter from protocolized ultrasound and CT measurements was done using the methods of Bland and Altman. Linear regression analysis was performed to study the relation between protocolized ultrasound- and CT-based AAA diameters. Calculations and statistical analysis were performed using SPSS® 26.0 (IBM, Armonk, NY, USA).

Results

The literature review identified nine studies that addressed different aspects of ultrasound-based measurement of maximal AAA diameter, as shown in the PRISMA diagram (). Most studies considered the image acquisition process and the size estimation on the acquired (frozen) image as an integral process. Two studies, addressed acquisition and reading as distinct processes and reported these aspects separately.

PRISMA diagram for the literature review

AAA, abdominal aortic aneurysm.

The identified studies addressed four potential sources of variability: the timing of the acquisition within the cardiac cycle; the plane in which the images were frozen; the direction of the measurement; and the positioning of the caliper on the vascular wall ().

Most agreement between the studies was found in the AP direction of the measurement; two, of the four studies that compared AP with transverse diameter measurements recommended the AP direction, and one study reported no difference between AP and transverse diameters. This preference for AP readings was incorporated in the four more recent studies that focused on the influence of timing in the cardiac cycle, and caliper position, ().

Reproducibility and repeatability of the ultrasound protocol

The recommendations from the literature review with respect to aspects of ultrasound-based image acquisition and AAA size estimation were incorporated into a practical protocol, detailed in and . The performance characteristics of the protcol were evaluated.

Position head of transducer parallel to longitudinal axis of the aorta to avoid parallax error ()

Fig. 2

Illustrated measurement guide to include three-dimensional information on the point of maximum dilatation

Maximum diameter should be corroborated in longitudinal and axial planes to avoid both overestimation and underestimation due to parallax error. See for an explanation of panels a–e.

Corroborate longitudinal AP measurement with axial AP measurement (use of split screen is recommended here) ()

The intraobserver repeatability of the protocol, tested in 29 participants from the PHAST with a mean AAA diameter 43 (range 35–51) mm, was 1.6 mm. The interobserver reproducibility of the ultrasound protocol was evaluated in nine patients with a mean AAA diameter of 43 (range 36–52) mm by four observers. The ICC for average (mean) measures was 0.97 (95 per cent c.i. 0.92 to 0.99), showing excellent agreement between observers.

The TEDY study protocol included paired ultrasound- and CT-based AAA size estimates at baseline. These readings (n = 35) were used to evaluate the agreement between protocolized ultrasound-based diameters and local CT-based diameters, as reported in the patients’ local records. The mean(s.d.) AAA diameter for ultrasound was 42.8(4.1) (range 35.7–48.8) mm, and that for local CT readings was 46.9(4.8) (34.0–55.6) mm. Linear regression analysis shows a robust correlation between CT- and US-based size readings (r = 0.81). However, the analysis revealed a systematic 4.1-mm difference between the protocolled ultrasound readings and local CT readings (P < 0.001) (). Exploration of this systematic positive difference identified several methodological aspects, summarized in , that all resulted in larger readings on CT.

Scatterplot of protocolled ultrasound central abdominal aortic aneurysm readings and local CT readings

n = 35; mean difference 4.1 mm (P < 0.001, linear regression analysis).

To improve the agreement of CT- and US-based readings, the impact of harmonizing the reading protocol (for instance, largest AP diameter perpendicular to the central luminal line, inner-to-inner diameter) was evaluated. This increased Pearson’s correlation between ultrasound and CT findings for the inner-to-inner method to 0.91, and decreased the systematic overestimation to 1.8 mm (P < 0.001) ().

Scatterplot of ultrasound central anterior–posterior reading and CT central anterior–posterior reading, inner-to-inner wall

n = 35; mean difference 1.8 mm (P < 0.001, linear regression analysis).

Evaluation of the interobserver reproducibility for the harmonized CT reading protocol indicated an ICC for average (mean) measures of 0.94 (95 per cent c.i. 0.88 to 0.97) for CT inner-to-inner readings and 0.96 (0.91 to 0.98) for CT outer-to-outer readings.

Discussion

To overcome the extreme variation in ultrasound-based AAA size estimates in clinical practice, a uniform ultrasound protocol was developed and validated. The proposed ultrasound protocol is simple, requires limited technical skills, and provides an opportunity to apply direct visual feedback during image acquisition. Evaluation of the protocol in daily practice confirmed its excellent performance characteristics.

Although there is consensus on the 55-mm intervention threshold for men with an AAA, there is little consensus on how the diameter is established. During the process of evaluating patient eligibility for participation in clinical trials for aneurysm growth reduction, a wide variety of diameter reading techniques and an indiscriminate use of ultrasound- and CT-derived diameters was noticed, and the systematic reading differences are well known,,.

Ultrasound-based size estimation underlies all screening, follow-up and primary intervention guidelines for AAA. However, ultrasound assessment of AAA diameter has some limitations, and is prone to operator-dependent errors, such as failing to image the largest diameter and/or an inappropriate angle (non-perpendicular) during image acquisition. Yet, the active acquisition process also provides quality-control opportunities, such as direct feedback on perpendicularity, that are absent in CT-based acquisitions. A second concern, not fully exclusive for ultrasonography, is the absence of a uniform size reading protocol. Inconsistent use of the inner-to-inner reading method versus the outer-to-outer method can result in a difference of several millimetres in the estimated AAA size. These size reading inconsistencies have consequences for clinical interpretations as ‘(fast) growing’ or ‘stable (not growing)’ AAAs, and may result in overtreatment or undertreatment of patients with an AAA.

AAA size reading (caliper positioning) on ultrasound images remains a matter of debate. A review of the literature on recommendations for maximum size estimation found that the AP diameter was the preferred measurement direction, that measurement should be taken at systole during the cardiac cycle, and that the measurement plane should be perpendicular to the blood flow. Lack of consensus was observed for the measurement landmarks in the aneurysm vessel wall, as illustrated by the use of different landmarks in the Multicentre Aneurysm Screening Study (MASS) (inner-to-inner) and the UK Small Aneurysm Trial (outer-to-outer). Borgbjerg and colleagues showed superior reproducibility for the leading-edge-to-leading-edge method and inner-to-inner compared with outer-to-outer. Yet, in contrast to the inner wall, the leading edge of an aneurysm is generally not visible on CT. In the light of clinical practice, in which ultrasound and CT size estimates are used indiscriminately, it was reasoned in this study that the use of inner-to-inner wall measurements would be preferable. Consequently, inner-to-inner was used for performance testing of the ultrasound protocol and in the efforts to harmonize ultrasound- and CT-based size diameter assessment.

A systematic difference exists between ultrasound- and CT-based size readings. It has been argued that ultrasonography underestimates AAA diameter, and that CT provides a more accurate reflection of the actual AAA diameter, an aspect that is of limited clinical value as the 55-mm threshold is actually based on ultrasound readings. In this context, it should be noted that various aspects, such as the inability of CT to adapt to situational factors during the scanning and measuring process and the use of singular plane measurements (for example, largest diameter in any plane), all result in maximalization of AAA diameter in CT readings. In contrast, adherence to a measurement plane perpendicular to the blood flow, image acquisition during diastole, use of AP single direction, and inner-to-inner wall measurements in an ultrasound-based reading protocol all result in minimalization of AAA diameter. Hence, the larger part of the systematic difference between CT- and ultrasound-based AAA size estimates might relate to differences in standardization of maximum diameter evaluation. In this context, the systematic differences between ultrasound readings (maximum diameter in one direction perpendicular to the blood flow) will not be affected by improvements in CT technology, such as reduced slice thickness and improved three-dimensional (3D) reconstructions.

Notwithstanding the systematic differences, this study has shown that AAA size estimates by ultrasound imaging and CT are equally reliable under comparable circumstances. Without harmonization, and with interchangeable use of these two modalities in clinical practice, the field is confronted with a persistent systematic reading difference of 5 mm. It is likely that this 5-mm difference in AAA size underlies the discrepancies between the 55-mm guideline intervention threshold and real-life clinical practice. As ultrasound imaging is the basis for surveillance guidelines, and the evidence-based intervention threshold is based on ultrasound readings, (relative) overestimation of AAA diameter by CT will result in premature repair and excess health costs. To deal appropriately with this difference, either both measurement techniques need to be harmonized by using a corresponding measurement protocol, as used in the present study, or the intervention threshold needs be set at a larger diameter for CT (such as 60 mm).

The data also show that the larger proportion of ultrasound and CT measurement variability relates to a lack of harmonization (such as inconsistent measurement planes and caliper positions). Strict harmonization of ultrasound and CT measurement will reduce iatrogenic misdiagnoses such as ‘fast growing’ aneurysms, and should be incorporated in the emerging artificial intelligence applications aimed at estimating AAA size. Moreover, it will greatly reduce measurement variability and increase the power of future trials aimed at reducing AAA progression.

This study had some limitations. The CT scans used in the CT reproducibility study were derived from various hospitals, using various scanners and scanning protocols. Even though all CT scans had the same minimum resolution, the variability could result in small differences in scan quality. However, this variability reflects real-life situations, which makes the reproducibility results more translatable to current clinical practice. The study focused on the follow-up of small AAAs (smaller than 55 mm), and therefore the results could not be directly translated to an AAA screening setting or large AAAs. Finally, 3D ultrasound imaging is a relatively new technique with great potential for visualizing and assessing the suitability of aneurysms for EVAR, without subjecting a patient to radiation and contrast fluids. Ghulam and co-workers reported good reproducibility and agreement between two-dimensional (2D) and 3D ultrasound readings of small AAAs. As further research is needed to validate and translate these results, the ultrasound protocol proposed in this study can be applied only for 2D ultrasonography.

Ultrasound images and data from both trials used in this study are dated, which challenges the robustness of the results. In the TEDY trial, all ultrasound images were made with new ultrasound equipment. Reproducibility of AAA diameters was comparable between the TEDY trial and the PHAST (which used comparable, but older, ultrasound equipment), suggesting that this would not significantly alter the findings of the present study.

Disclosure. The authors declare no conflict of interest.

Supplementary material

Supplementary material is available at BJS Open online.

Click here for additional data file.

Table 1

Standardized protocol for ultrasound measurement of abdominal aortic aneurysm

1	Patient in supine position  Or lateral supine position in obese patients (to shorten distance from transducer to aorta)
2	After centering the abdominal aorta on the screen, scan axially for largest diameter  Correct depth, double focus at near and far wall
3	Switch to longitudinal view, determine maximum diameter (Fig. 2, panel a)
4	Acquire image in systole  Position head of transducer parallel to longitudinal axis of the aorta to avoid parallax error (Fig. 2, panel b)  Avoid excessive compression that might decrease the AP diameter  Maximum diameter in centre of view (to optimize resolution of the divergent ultrasound beam)
5	AP measurement with caliper position on inner-to-inner vessel wall
6	Rotate transducer 90° to axial view at maximum diameter  Corroborate longitudinal AP measurement with axial AP measurement (use of split screen is recommended here) (Fig. 2, panels c–e)  If diameters are discrepant, start again at 3
7	Repeat measurements twice
8	Record mean of three measurements

AP, anterior–posterior.

Table 2

Causes of discrepancy in abdominal aortic aneurysm size estimates between ultrasound imaging and CT

Cardiac cycle (diastole/systole pulse wave in aorta)

Positioning/respiratory movements of patient

Angulation of the aorta (measuring perpendicular to blood flow)

Caliper placement (anterior–posterior on ultrasound imaging versus maximum diameter in any plane on CT)

Presence of thrombus in the aneurysm

Differences in slice thickness on CT