Mitigation of Variability among 3D Echocardiography-Derived Regional Strain Values Acquired by Multiple Ultrasound Systems by Vendor Independent Analysis.

INTRODUCTION: This study compared the variability of 3D echo derived circumferential and longitudinal strain values computed from vendor-specific and vendor-independent analyses of images acquired using ultrasound systems from different vendors.
METHODS: Ten freshly harvested porcine hearts were studied. Each heart was mounted on a custom designed phantom and driven to simulate normal cardiac motion. Cardiac rotation was digitally controlled and held constant at 5°, while pumped stroke volume (SV) ranged from 30-70ml. Full-volume image data was acquired using three different ultrasound systems from different vendors. The image data was analyzed for longitudinal and circumferential strains (LS, CS) using both vendor-specific and vendor-independent analysis packages.
RESULTS: Good linear relationships were observed for each vendor-specific analysis package for both CS and LS at the mid-anterior segment, with correlation coefficients ranging from 0.82-0.91 (CS) and 0.86-0.89 (LS). Comparable linear regressions were observed for results determined by a vendor independent program (CS: R = 0.82-0.89; LS: R = 0.86-0.89). Variability between analysis packages was examined via a series of ANOVA tests. A statistical difference was found between vendor-specific analysis packages (p<0.001), while no such difference was observed between ultrasound systems when using the vendor-independent program (p>0.05).
CONCLUSIONS: Circumferential and longitudinal regional strain values differ when quantified by vendor-specific analysis packages; however, this variability is mitigated by use of a vendor-independent quantification method. These results suggest that echocardiograms acquired using different ultrasound systems could be meaningfully compared using vendor-independent software.

Introduction

Strain imaging is an increasingly prevalent method of quantifying myocardial mechanics. This technology is used to determine global and regional myocardial deformation throughout a cardiac cycle and is affiliated with cardiac magnetic resonance imaging (cMRI), cardiac computed tomography (CT) and echocardiography (echo)[1, 2]. Strain-based imaging techniques are predominantly used to monitor cardiac function and direct therapy under a variety of conditions such as myocardial infarction [3-6], mechanical dyssynchrony [7-10] and cardiomyopathy [11-18]. These strain-based methods provide a unique and sensitive approach to quantitatively express the function of the myocardium, which represents cardiac health [1–3, 5, 6, 9].

Speckle-tracking echocardiography (STE) derived strain has been validated both experimentally and clinically [19, 20]. Two-dimensional (2D) STE tracks the motion of natural acoustic markers between acquisition frames to detect the direction and magnitude of segmental wall motion. 2D STE is angle independent; however, it has been shown to be limited by through-plane motion and the complex architecture of the heart [21, 22].

Three-dimensional echocardiography (3DE) enables the measurement of myocardial deformation within a volumetric acquisition and is not subject to many of the limitations of 2D STE. The temporal resolution of 3DE suffers relative to that of 2D STE and there are concerns regarding the accuracy of 3DE-derived strain values [23, 24]. However, experimental and clinical studies have validated the strain determining capabilities of multiple vendor-specific analysis packages based on 3DE acquisitions [25-27].

It has been shown that, while comparable in accuracy to 2DE-based methods, 3DE-derived strain values acquired from different vendors’ ultrasound systems and analysis packages lack reproducibility [28, 29]. The variability in myocardial strain measurements of different analysis programs has not been extensively studied. Vendor-independent analysis packages, such as TomTec Image Arena (TomTec Imaging Systems, Unterschleissheim, Germany), provide a means by which 3D volumes from multiple ultrasound systems can be analyzed [30]. The strain values determined by TomTec Image Arena have been shown to accurately and reliably quantify myocardial deformation [31, 32].

Koopman et al showed that vendor independent software was capable of determining global strain values from B-mode images acquired using multiple ultrasound systems [33] while Risum et al confirmed this finding using 2D STE [34]. However, strain quantification from 3D volumes across ultrasound systems has not been adequately explored using vendor-independent software. This study aimed to establish the variability of circumferential and longitudinal strain values acquired from different vendors and demonstrate a reduced variability due to analysis using a vendor-independent analysis package.

Materials and Methods

A closed-circuit pulsatile harvested pig heart model was designed for this study. Pig hearts were purchased from Carlton Farms (Carlton, Oregon), which processes pork for commercial sale and is licensed to provide organs for in vitro research. OHSU does not require IACUC approval for in vitro projects.

Ten freshly harvested porcine hearts had atria and major vessels removed. The aortic leaflets were sutured together. A latex balloon was fixed within the left ventricle (LV) at the mitral annulus and connected to a pulsatile pump. Each phantom model was mounted on a custom mechanized base plate with the apex fixed (Fig 1A). The base plate was driven by a rotary actuator motor which was interfaced with the pulsatile pump and set to rotate 5° and compact 10mm in phase with the simulated cardiac cycle according to a simulated QRS Complex.

Fig 1

A) Experimental apparatus: Each heart is passively pumped in phase with a rotary actuator baseplate with images acquired from a fixed apex. B) Sonomicrometry crystal placement: Relative displacement between crystals 1 and 2 is used to determine reference longitudinal strain while displacement between crystals 2 and 3 represents reference circumferential strain.

Three sonomicrometry crystals were sutured into the myocardial wall of the LV within the region adjacent to the left anterior descending coronary artery (LAD) (depicted in Fig 1B). The LAD was used as a landmark for the orientation of crystals numbered 1 and 2 which represented longitudinal strain (LS) and were perpendicular to crystal 3 which, coupled with crystal 2, provided circumferential strain (CS) reference values.

Each phantom was pumped at a stroke rate of 60 beats per minute (bpm) with a stroke volume (SV) ranging from 30-70ml. SV was increased in increments of 10ml with the appropriate volume added to the system to maintain a static end systolic volume (ESV). Full-volume 3DE data was acquired on the following ultrasound systems at each SV; GE Vivid E9 with a 3V-D transducer (GE Healthcare; Horten, Norway); Toshiba Aplio Artida with a PST-25SX transducer (Toshiba Medical System Corporation, Tochigi, Japan); and ACUSON SC2000 with a 4Z1c transducer (Siemens Medical Solutions USA Inc., Mountain View, CA). Each transducer was positioned in the same apical acquisition window and other parameters were conserved among ultrasound systems. The relative displacement between sonomicrometry crystals was recorded at each SV to provide reference strain values.

Data was analyzed offline by vendor-specific analysis packages: GE EchoPAC PC; Toshiba 3D-WMT; and Siemens VMM (Fig 2). Three isolated observers, each experienced with one of these analysis packages, obtained peak negative circumferential and longitudinal strain (CS and LS) values at the mid anterior (MA) segment of the LV for each SV of each phantom. The volumes used for each vendor-specific analysis were then stripped of identifying information and exported into TomTec Image Arena. Each observer analyzed a complete dataset using TomTec Image Arena to determine CS and LS at the MA for each volume (Fig 3). The sonomicrometry-derived displacement values were analyzed using SonoVIEW (SonoMetrics Corporation, Ontario, Canada) to generate reference CS and LS values at the MA segment.

Fig 2

A) GE EchoPAC PC: Peak strain analysis using EchoPAC PC. B) Toshiba 3D-WMT: Peak strain analysis using Toshiba 3D-WMT. C) Siemens VMM: Peak strain analysis using Siemens VMM.

Fig 3

TomTec Image Arena: Peak strain analysis using TomTec Image Arena.

Inter-observer variability was determined from a sample of 20% of the volumes analyzed by a different user blinded to the previously determined values. Intra-observer reproducibility was examined according to a sample of 20% of the volumes analyzed by the blinded initial operator six weeks after initial analysis. The intraclass correlation coefficients (ICCs) were calculated and presented.

Results

Ten hearts were imaged by three distinct ultrasound systems at five different stroke volumes, resulting in a total of 150 full-volume acquisitions analyzed. Negative peak CS and LS were determined from each of these volumes using both vendor-specific and vendor-independent analysis programs. Including reference sonomicrometry results, over 750 data points were generated and used to determine the results presented in this study. The following data was excluded from all analysis due to inconsistent border tracking: Siemens Heart 4 CS SV 30-70ml and EchoPAC Heart 2 LS SV 40ml.

Circumferential and longitudinal strain data were analyzed separately. Linear regression analyses were performed on vendor specific echo-derived CS and LS values against sono-derived strain. The echo-determined strain values for GE EchoPAC PC, Toshiba 3D-WMT and Siemens VMM were plotted on the same xy coordinate plane against sonomicrometry determined strain (Fig 4A and 4B). Good linear agreement was observed for all analysis packages (CS: R = 0.82–0.91; LS: R = 0.86–0.89). Bias was gauged by a series of Bland-Altman analyses (CS: 1.8–6.1%; LS: 2.9–6.7%). Inter-vendor variability was determined significant for both CS and LS by one-way analysis of variance (ANOVA) tests (Table 1) with ICC values ranging from 0.52–0.66 (CS) and 0.59–0.77 (LS).

Fig 4

A) Circumferential strain derived from vendor specific analysis. B) Longitudinal strain derived from vendor specific analysis. C) Circumferential strain derived from vendor independent analysis. D) Longitudinal strain derived from vendor independent analysis. All echocardiography-derived strain values (y-axis) are plotted against sonomicrometry derived strain values (x-axis).

Table 1

One-way ANOVA results: Significant difference in vendor dependent CS and LS values among ultrasound systems and no significant difference in vendor independent results among ultrasound systems.

	ANOVA
		P-value	F	F-crit	F > F-crit
Vendor-Dependent Analysis Package	Circumferential Strain	5.04E-12	31.41741	3.059831	Yes
	Longitudinal Strain	1.8E-15	43.22723	3.05805	Yes
Vendor-Independent Analysis Package	Circumferential Strain	0.060119	2.867829	3.059831	No
	Longitudinal Strain	0.361086	1.025779	3.05805	No

CS and LS values were determined by a vendor independent analysis package (TomTec Image Arena) and plotted in distinct datasets, based on the ultrasound system upon which the image loops were acquired, on the same xy coordinate plane against sonomicrometry determined strain (Fig 4C and 4D). Linear regression analyses were performed for each dataset and a good linear correlation was observed for all ultrasound systems (CS: R = 0.82–0.89; LS: R = 0.86–0.89). A series of Bland-Altman analyses revealed bias values for each dataset of 9.1–10.9% (CS) and 9.3–10.2% (LS). Inter-vendor variability was determined not to be significant by one-way analysis of variance (ANOVA) tests (Table 1) with ICC values ranging from 0.71–0.87 (CS) and 0.82–0.90 (LS). The variability of strain measurements computed from vendor specific and vendor independent analyses was determined by intra-class correlation (ICC). The ICC values using 1000 bootstrap samples are 0.5457392 (95% bootstrap CIs: [0.4445, 0.6390]) for vendor specific, and 0.793742(95% bootstrap CIs: [0.7182, 0.8465]) for vendor independent strain measurements. The lower bound of the 95% CI for ICC with vendor independent is higher than the upper bound of the 95% CI for ICC with Vendor specific. Higher intra-class correlation (ICC) indicates smaller inter-variability.

For standardization across SVs, CS and LS values were normalized by dividing the echo-derived strain by the reference strain. Normalized strain values were averaged across all hearts and SVs for each vendor-specific and -independent analysis package. These averages were plotted against each other with the error being represented by the Standard Error of the Mean (SEM). These normalized strain values were used to compare the vendor-specific derived CS and LS (Fig 5A and 5B) as well as the vendor-independent CS and LS (Fig 5C and 5D).

Fig 5

A) Normalized circumferential strain values from vendor specific analysis. B) Normalized longitudinal strain values from vendor specific analysis. C) Normalized circumferential strain values from vendor independent analysis. D) Normalized longitudinal strain values from vendor independent analysis. All error bars are represented as the standard error of the mean (SEM).

Analysis of Variance (ANOVA) tests were used to compare the difference of the strain values acquired from each vendor dependent and independent analysis package (Table 1). A statistical difference was found between vendor-specific analysis packages (F>Fcrit; p<0.001), while no such difference was observed between ultrasound systems when using the vendor-independent program (F0.05) for both CS and LS.

Inter-observer variability for the vendor-specific software was relatively good with ICC values ranging from 0.83 to 0.96 (CS) and 0.90 to 0.98 (LS) while the intra-observer reproducibility was found to range from 0.97 to 0.99 (CS) and 0.96–0.99 (LS). For the vendor-independent program, inter-observer variability ranged from ICC = 0.93 to 0.99 (CS) and 0.98 to 0.99 (LS) while intra-observer reproducibility ranged from 0.98 to 0.99 (CS) and 0.99 (LS) (Fig 6).

Fig 6

Intra-Class Correlation plots to assess the variability of strain measurements derived from vendor specific and vendor independent analyses.

The plots show no overlapping between the 95% CI for ICC between the two approaches: vendor specific (blue) vs. vendor independent (red).

Discussion

A fundamental obstacle to inter-institutional, and in some cases intra-institutional, collaboration on echocardiography-based studies is the lack of reproducibility between vendors [28, 29]. Badano et al concluded that longitudinal 3D-echocardiography studies must be conducted using the same imaging platform to assure accuracy [29]. Previous inter-vendor reproducibility studies have shown a statistically significant difference in 3DE-derived CS and LS values between ultrasound systems [28, 29, 35]. These findings are in agreement with the results of this study, which verify that CS and LS values determined by vendor-dependent software exhibit low levels of agreement. Recognizing the critical need for standardization in strain imaging, the European Association of Cardiovascular Imaging, EACVI) and the American Society of Echocardiography (ASE) has also published a technical document intended to create a common standard [28]. The observed lack of reliability between vendors may result from differences in tracking algorithms, interpolation techniques and strain calculations [29, 36].

The ability to convert full-volume acquisitions from multiple ultrasound systems to digital imaging and communications in medicine (DICOM) data allows the use of a single vendor-independent analysis utility [33]. In this study, the use of vendor-independent software to determine CS and LS values was shown to reduce the variability between ultrasound systems such that there was no significant difference. The ability of a vendor-independent analysis package to reduce inter-vendor variability has been shown in previous studies [28, 37].

Clinical Significance

The use of this vendor-independent quantification technology in a clinical setting has a wide variety of beneficial applications. Previous studies have indicated that the same ultrasound system and vendor-dependent analysis package must be used for a longitudinal study to assure accuracy [30, 36, 37]. However, the results of this study suggest that the use of a vendor-independent software package obviates the variability in both CS and LS values between ultrasound systems. This technology has the potential to reduce health care costs by precluding the reacquisition of images previously captured by a different ultrasound system than the one currently in use at a particular facility. Additionally, the quality of patient care may be improved by the ability to compare previous 3DE studies to present values and subsequently determine the progression of cardiac function.

This technology could also promote an increase in inter-institutional collaboration resulting from the ability to analyze data acquired on multiple ultrasound systems. This would allow studies to be completed in shorter periods of time with greater subject diversity. Due to the increased potential patient population associated with collaborative studies, uncommon or rare disorders could be much more comprehensively studied. Additionally, the application of this vendor-independent analysis package may allow unprecedented longitudinal study of disease progression.

Study Limitations

Temporal resolution was optimized, but not explicitly examined as a source of variability in this study. Previous studies have determined that temporal resolution values consistent with those used in this study are not a source of variability [30, 38].

The nature of this in vitro model prevents examination of the effects of image quality as all acquisitions occurred under optimal conditions. The proposed model of the CS in this study, however, is different with respect to the in vivo situation, as it does not keep in consideration electrical conduction, which causes a delay in contraction of different sections of the heart. Additionally, this phantom study used extrinsic pump to generate pulsatile cardiac motion, which is quite different than intrinsic myocardial contraction. Based on the results of this study and the known limitations of strain quantification reproducibility, especially associated with CS, it is important to continue this research using in vivo acquisitions.

This study utilized regional strain values at the mid-anterior segment due to its presence in the near field to optimize resolution. These values were used as surrogates for global strain; however, they may not accurately represent global strain values due to position relative to the transducer. This limitation in study design was necessary due to the practical constraints of using sonomicrometry crystals to determine stain in a passively driven cardiac model. Variability, therefore, could have been reduced by optimized image quality allowed in the phantom setting and/or amplified by regional evaluation.

Conclusions

Circumferential and longitudinal regional strain values differ when quantified by vendor specific analysis packages; however, this variability is mitigated by use of a vendor independent quantification method. The results of this study suggest the plausibility of inter-institutional collaborative studies, regardless of different ultrasound systems used in acquisition.