What is the minimum change in left ventricular ejection fraction, which can be measured with contrast echocardiography?

BACKGROUND: There are limited data on what is the minimum change that can be detected in cancer patients undergoing treatment with cardiotoxic drugs and are referred for monitoring left ventricular (LV) function.
OBJECTIVE: To assess the variability in the measurement of LV volumes and ejection fraction (EF) in contrast echocardiography and to determine the minimum detectable difference (MDD) between two EF measurements that can be deemed significant.
METHODS: A total of 150 patients were divided into three groups according to EF (EF <53, 53-60, and >60%). Each group consisted of 50 randomly selected cancer patients who underwent contrast echocardiography between July 2010 and May 2014. Repeated measurements of LV volumes and EF were performed offline by a sonographer and a cardiologist. Inter-observer variability was assessed by analysis of variance. Measurement error was estimated by standard error of measurement and MDD.
RESULTS: The 95% confidence interval with a single measurement of EF was 2 percentage points in the groups of patients with EF <53% and EF >60%, and 2.5 percentage points for patients with EF 53-60%. The MDD for EF, end-diastolic volume and end-systolic volume that could be recognized with 95% confidence interval were 4 percentage points, 7 mL and 4 mL, respectively.
CONCLUSION: Contrast echocardiography is a reliable tool for serial measurements of EF to monitor cardiotoxicity due to chemotherapy. In a high-volume echocardiography laboratory with experienced staff, the MDD for EF of 4 percentage points on a good-quality recording demonstrates the high reproducibility of the Simpson's method using contrast echocardiography.

Introduction

Serial measurements of left ventricular (LV) ejection fraction (EF) are commonly performed in cancer patients to monitor cardiotoxic effects of chemotherapy. Similarly, patients with other cardiac conditions, including valvular heart disease (1), heart failure (2, 3) and myocardial infarction (4) require regular monitoring of EF to follow the clinical course of disease. When two-dimensional (2D) echocardiography is used for serial assessments of EF, one of the most important questions is ‘Are changes in EF real or due to reader quantification variability?’ Although contrast 2D echocardiography has been recommended for EF measurements when clinical management depends on accurate assessment of EF such as monitoring cardiotoxicity of chemotherapy (5), there is still inter-observer variability of EF among readers (6). Therefore, it is difficult for clinicians who monitor changes in EF over time to determine whether differences between the two studies represent a clinically significant change or they are due to measurement error. The aim of this study was to perform repeated measurements of LV volumes and EF using high-quality 2D contrast echocardiography in order to determine the minimum difference in EF that could be used as a cut-off for defining a true change in LV function.

Methods

Patients

The study included 150 cancer patients who had been referred from the Cross Cancer Institute for echocardiographic evaluation before or during chemotherapy. They were divided into three groups according to EF (EF <53, 53–60, and >60%). Each group consisted of 50 randomly selected patients who underwent contrast echocardiography between July 2010 and May 2014. A cut-off of 53% for EF groups was used according to the American Society of Echocardiography (ASE) and the European Association of Cardiovascular Imaging (EACVI) 2014 expert consensus for multimodality imaging evaluation of patients during and after cancer therapy (6). The ASE/EACVI defined cardiotoxicity as a drop in the EF of >10%, to a value <53%. We also defined an upper range in order to have three EF groups: patients with impaired LV function (<53%), a low-normal group (53–60%) and patients with normal LV function (EF >60%). The reason for including a low-normal EF group was that the variability of the EF in this group can have major impact on the decision whether to stop or continue chemotherapy.

All echocardiograms were clinically indicated either to assess EF prior to chemotherapy initiation or to evaluate cardiotoxicity of anticancer drugs. The patients were asked to give written consent to use their anonymized recordings as part of a quality assurance program. Almost 80% of patients studied had breast cancer and were monitored because of treatment with trastuzumab.

2D contrast echocardiography

Comprehensive echocardiographic examinations were performed in the left lateral decubitus position with an ultrasonographic system (IE 33 or EPIQ 7C, Philips Medical Systems, N.A.) equipped with S5-1 or X5-1 transducers. The local protocol for patients of the cardio-oncology clinic includes injection of contrast unless there is a contraindication for the contrast agent. At the end of the scanning, an ultrasound contrast bolus of 0.1–0.3 mL of Definity (Lantheus Medical Imaging, North Billerica, MA, USA) was administered intravenously to all patients regardless of non-contrast image quality, in order to minimize variability of EF measurements (5, 7, 8).

Intravenous (IV) cannulation and contrast agent injection were performed by the sonographer. We have published the benefits of the training sonographers in administering contrast agents in a large-volume echocardiography laboratory (9). By training sonographers to perform IV access and contrast injection, the time to complete contrast echocardiograms has been significantly reduced compared with having a physician or nurse performing the IV catheter insertion and contrast administration. The additional time for insertion of the IV cannula, preparation and administration of the contrast agent was less than 10 min.

In 58% of the 150 studies, two or more adjacent endocardial segments were not well visualized. However, we followed the recommendation of the EACVI 2017 on the clinical practice of contrast echocardiography (5): contrast 2D echocardiography should be considered irrespective of image quality when clinical management depends on accurate measurements of EF such as monitoring of patients treated with cardiotoxic drugs and when implantation of intracardiac defibrillator (ICD) or cardiac resynchronization therapy (CRT) devices are considered (Class IIa, Level B).

A very low mechanical index (MI) contrast imaging modality was used according to the ASE sonographer guideline (10). Focus, power and gain were adjusted to optimize LV opacification and to facilitate the LV volumes and EF measurements. At least three cardiac cycles of apical four- and two-chamber views were digitally stored for offline analysis.

The end-diastolic and end-systolic LV volumes (EDV, ESV) and EF were calculated by using biplane Simpson’s method from the contrast recordings in Philips Xcelera system. Non-contrast recordings were not used for measurement of LV volumes and EF because contouring the LV endocardial border was much easier on the contrast-enhanced images. Wall motion score index and global longitudinal strain were assessed in every patient; however, these were not included in the present study. The ethics approval for the study protocol was granted by the University of Alberta Hospital Research Ethics Committee (Pro 00015530). Informed written consent was obtained from all patients.

Reproducibility

To assess inter-observer variability, contrast LV volumes and EF were independently measured offline by a sonographer and a cardiologist (H B and J C). The sonographer performed the measurements first – immediately after the recording and therefore was blinded to the results of the cardiologist. Then, the cardiologist performed the measurements that were put in the final report by choosing frames independently for end diastole and end systole measurements from at least two loops per apical view. As part of the quality assurance program, the cardiologist could review the sonographer’s tracings. Thus, the cardiologist was not fully blinded.

Examples of tracings of LV borders at end diastole and end systole in both apical four-chamber and two-chamber views by the two readers, as well as the LV volumes calculated from the tracings are shown in Fig. 1.

Figure 1

Tracings of LV borders by two readers and the volumes calculated from the tracings, the EF was reported as 48% by reader 1 and 49% by reader 2. (A) Four-chamber view, (B) two-chamber view. EDV, end-diastolic volume; ESV, end-systolic volume.

Intra-observer variability of contrast EF measurements was determined by the sonographer repeating measurements in 20 randomly selected patients.

Statistical analysis

Data were analyzed by SPSS, version 21.0 (SPSS). Continuous data are expressed as mean ± s.d. or where skewed as median and interquartile range (IQR). Categorical data are expressed as absolute values and frequency percentage. The normality of distribution was determined using normality plots and the Kolmogorov–Smirnov test. Inter-observer variability of EF and volume measurements was calculated in three EF groups (EF <53, 53–60% and >60%).

Inter-observer variability in the assessment of LV volumes and EF was determined as the intra-class correlation coefficient (ICC) and the relative mean error (RME). The RME was defined as the absolute difference between two measurements relative to their mean and expressed in percent (11). Measurement error was estimated by the standard error of measurement (SEM) and the minimum detectable difference (MDD) (12, 13). SEM is a standard deviation of the distribution of repeated measurements, while MDD represents the minimum difference between two measurements that must be overcome to ascertain a true change. One-way analysis of variance (ANOVA) was performed to obtain the mean square error. The square root of the mean square error provided the SEM. The SEM was used to construct confidence interval (CI) around the index measurement by multiplying SEM by 1.96 for 95% CI (12). The MDD was calculated as 1.96 × SEM × √2 and represented the 95% confidence that a change in the measurement exceeding this threshold is true and not due to measurement error (14, 15). The SEM and MDD have the same units as the measurement of interest. Intra-observer variability of EF was assessed using RME as described earlier.

For the estimation of sample size, we assumed that within the subject the distribution of observations was normal. We wanted to have CIs that were within 20% of the population within-subject standard deviation. For two measurements per subject, we would require 48 subjects in each EF group to estimate SEM (16).

Results

The patient characteristics are shown in Table 1. The inter-observer variability and measurement errors expressed as RME, ICC, SEM and MDD for EF measurements are summarized in Table 2. The SEM and MDD for LV volume measurements are demonstrated in Table 3.

Table 1

Baseline characteristics of study participants.

Patient variable	Total (n = 150)
Age (years)	59 ± 12
Female	120 (80%)
Body surface area (m2)	1.8 ± 0.2
Cancer diagnosis
Breast	115 (77%)
Hematologic	27 (18%)
Other	8 (5%)
Echocardiographic parameters
LVEDV (mL)	109.6 (93.6–131.8)
LVESV (mL)	46.7 (37.2–62.9)
Index LVEDV (mL/m2)	62.8 (55.0–72.6)
Index LVESV (mL/m2)	26.7 (21.3–33.9)
LVEF (%)	58.0 (50.6–62.8)

Data are expressed as mean ± s.d., as median (interquartile range) or as number (%).

EDV, end-diastolic volume; EF, ejection fraction; ESV, end-systolic volume; LV, left ventricular.

Table 2

Inter-observer variability and minimum detectable difference for EF measurements in each EF group.

EF (%)	ICC	RME	SEM	MDD
<53	0.99	2.3 (0.3–3.8)	1.1	3.0
53–60	0.81	1.5 (0.0–3.4)	1.3	3.6
>60	0.96	1.0 (0.0–2.4)	1.1	3.0

Relative mean error (RME) is calculated as a percent and expressed as median (interquartile range). The standard error of measurement (SEM) and the minimum detectable difference (MDD) have the same units as the measurement of interest.

EF, ejection fraction; ICC, intra-class correlation coefficient.

Table 3

Standard error of measurement and minimum detectable difference for EDV and ESV measurements in each EF group.

EF (%)	EDV SEM (mL)	ESV SEM (mL)	EDV MDD (mL)	ESV MDD (mL)
<53	2.5	1.4	7.0	4.0
53–60	2.6	1.1	7.3	3.1
>60	2.0	1.2	5.6	3.3

The SEM and MDD have the same units as the measurement of interest.

EDV, end-diastolic volume; EF, ejection fraction; ESV, end-systolic volume; MDD, minimum detectable difference; SEM, standard error of measurement.

For both LV volumes and EF measurements, the distributions of the inter-observer RMEs were positively skewed. Therefore, the median and interquartile range of the RME were calculated in order to provide useful descriptive statistics of the errors of each measure. There was good inter-observer agreement for measurements of EF between the two readers.

Based on the inter-observer variability of LV volumes and EF, we can construct CI around EF from the SEM and know a true change between two EF measurements from the MDD. For example, when EF is measured as 50% with the SEM of 1 percentage point, this means that with 95% confidence the true EF is between 48% and 52%. Furthermore, when an absolute difference of 3 percentage points between EF measured at baseline and at follow-up is found, the clinician can be up to 95% confidence that a true change in EF has occurred in a patient. The same reliability estimates can be applied to LV volume measurements.

The numbers of patients with serial echocardiography and patients with real changes in EF by using 4 percentage points as a cut-off, as well as the mean EF changes during follow-up for each EF group are presented in Table 4. The range of follow-up time was 3–4 months from cancer therapy initiation. The MDD for EF measurements is lower than the mean changes in EF for all three EF groups.

Table 4

Patients with real changes in EF (≥4%) in those patients who had follow-up studies.

EF (%)	Number of patients with follow-up studies (%)	EF changes during follow-up (percentage points)	Number of patients with ‘real change’ (EF changes ≥4%)
<53	28 (56)	10 ± 6	18 (36)
53–60	31 (62)	8 ± 5	21 (42)
>60	28 (56)	8 ± 5	20 (40)

Data are expressed as mean ± s.d. or as number (%).

EF, ejection fraction.

The intra-observer variability for EF measurements was low (RME 2.1%). Test–retest variability could not be assessed in the study because we could use only recordings of clinically indicated echocardiograms.

Discussion

To our knowledge, this is the first report calculating the SEM and defining the minimum difference for EF measurements in different EF groups by 2D contrast echocardiography. Not only does our study emphasize good reproducibility of contrast echocardiography, but we also define a cut-off limit that represent a clinically significant change in EF measurements by contrast echocardiography. According to our results a minimum difference of 3.6 percentage points appears to be detectable in clinical practice; however, we should not report EF in percent with decimals. Therefore, we suggest using 4 percentage points as a cut-off. This means that variation due to chance is unlikely if an observed change between two studies is >4 percentage points for contrast 2D EF measurements.

As for any cardiac imaging modality, traditional assessment of EF by 2D echocardiography measurement is subject to variability because of several factors such as LV geometric assumption, image quality (especially endocardial border definition), LV foreshortening, LV volumes and loading conditions. These factors can affect EF measurements, leading to a minimal detectable change in EF measured by non-contrast 2D echocardiography of up to 8.5% (17). Furthermore, non-contrast 2D echocardiography often fails to detect small changes in LV contractility in patients undergoing chemotherapy (6). Thavendiranathan et al. reported in 2013 that 2D echocardiography was capable of recognizing changes in serial measurements of EF of approximately 10% (range 9.1–11.8%) (13). Because this is the same percentage points used to adjudicate cardiotoxicity of chemotherapy, the sensitivity of non-contrast 2D echocardiography has been questioned (6).

The reproducibility of EF measurements of contrast 2D echocardiography is shown to be better than that of non-contrast 2D echocardiography, even in patients in whom image quality was visually judged as adequate (8, 18). The use of ultrasound contrast agents optimizes endocardial border definition and overcomes the difficulties of inadequate visualization of the true LV apex often associated with non-contrast 2D echocardiography. Thus, minor changes in EF could be detected by contrast 2D echocardiography (19).

Our data firmly support the use of 2D contrast echocardiography for serial evaluation of LV function to monitor cardiotoxic effects of chemotherapy. Contrast 2D echocardiography can be reliable in the detection of a 10% change in EF that defines cardiotoxicity of anticancer treatments (6). 95% CI around EF measurements by contrast echocardiography was about 3%, while the minimal detectable change in EF was 4%. However, a previous study performed by Thavendiranathan et al. appears to contradict our findings (13). They reported the higher inter-observer variability for 2D contrast echocardiography with minimum detectable change in EF of >10%. The low variability in our study is probably related to that contrast echocardiography is used in all patients and only performed by the most experienced sonographers. We agree with Thavendiranathan et al. that blooming and attenuation artifacts may hinder delineation of the mitral annular plane, leading to variability in contouring of the LV. However, in our high-volume laboratory, considerable efforts are made to avoid blooming and attenuation artifacts such as giving small incremental doses of the contrast agent or waiting for the attenuation artifact to diminish before image acquisition. As a result, the inter-observer variability of EF measurements of contrast 2D echocardiography in our study is better than that of non-contrast 3D echocardiography with minimum detectable change in EF of 6% (13). Likewise, for EDV and ESV measurements, the inter-observer variability in our study showed smaller variability compared with non-contrast 3D echocardiography with minimum detectable change in EDV and ESV of as high as 25 mL and 12 mL, respectively (13).

Some limitations of our study need to be mentioned. First, these results probably are valid only for high-volume echocardiography laboratories with experienced sonographers; in other laboratories, the minimum difference in EF may be larger. In principle, every echocardiography laboratory should establish their own range. Secondly, ideally a test–retest design should have been used for the study. However, this was not feasible because we could use only the recordings of clinically indicated echocardiograms. Although the two readers analyzed the same study, they could choose frames independently for end diastole and end systole measurements from at least two loops per apical view. Finally, most of our patients had normal or near-normal EF and LV volumes representing the typical cardio-oncology patients, who usually have normal EF at the first echocardiogram. Therefore, we do not have the evidence that the minimal difference is the same in patients with severely reduced LV function.

Conclusion

2D contrast echocardiography is a reliable tool for serial measurements of EF to monitor cardiotoxic effects of chemotherapy. In a high-volume echocardiography laboratory with experienced staff, the MDD for EF of 4 percentage points on a good-quality recording demonstrates the high reproducibility of the Simpson’s method using contrast echocardiography.

Declaration of interest

Harald Becher: Bracco Imaging – research grants, honorarium for presentations at Bracco sponsored meetings. Jonathan Choy: Philips Healthcare – honorarium for conducting the 3D echocardiography course at the Mazankowski Alberta Heart Institute.

Funding

This research did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector.