Clinical utility of semi-automated estimation of ejection fraction at the point-of-care.

INTRODUCTION: To compare estimation of ejection fraction at the bedside by AutoEF compared with conventional methods and to assess feasibility and time consumption.
METHODS: A total of 102 relatively hemodynanically stable mixed medical and surgical patients were included. All patients underwent ultrasonography of the heart at the bedside performed by a novice examiner. Three assessments of ejection fraction were made: 1) Expert eyeballing by a single specialist in cardiology and expert in echocardiography; 2) Manual planimetry by an experienced examiner; 3) AutoEF by a novice examiner with limited experience in echocardiography.
RESULTS: Expert eyeballing of ejection fraction was performed in 100% of cases. Manual planimetry was possible in 89% of cases and AutoEF was possible in 83% of cases. The correlation between expert eyeballing and AutoEF was r=0.82, p < 0.001, for manual planimetry and for AutoEF it was r=0.82, p < 0.001; for expert eyeballing and manual planimetry it was r=0.80, p < 0.001. The mean time consumption for manual planimetry was 98 ( 90-106 ) seconds; correspondingly the mean time spent for AutoEF was 41 ( 36-46 ) seconds, which was significantly less (p < 0.001).
CONCLUSIONS: AutoEF seems to be a valid supplement to the clinical assessment of ejection fraction in the hands of less experienced examiners, yielding result similar to manual planimetry with less time consumption and less intra-observer variability. However, manual editing may be required and training is thus recommended before AutoEF is applicable for use by novices.

Introduction

Evaluation of the circulatory system is essential during examination of any hospitalized patient. As a supplement to the initial clinical evaluation, point-of-care (POC) echocardiography is performed with increasing frequency, in particular among physicians in intensive care, emergency medicine and anesthesia settings [1,2,3]. The POC examination facilitates better triage, diagnostics and treatment. Physicians are able to evaluate the patients at the bedside and commence appropriate initiatives without further delay.

However, while some pathologic findings are dichotomous and most often very apparent (e.g. pericardial exudates or gross chamber enlargement) others can be a challenge (e.g. moderately impaired systolic function or discrete valvular disease). In particular, quantification of systolic function is relevant in most situations; although ejection fraction (EF) is probably not the most accurate and descriptive parameter of left ventricular systolic function it is widely used clinically and it remains a key criterion for different pharmacological and invasive treatment strategies [4,5,6,7].

Three-dimensional echocardiography, cardiac magnetic resonance and cardiac computed tomography provide non-user dependent and reproducible estimates of EF [8, 9]. However, none of the methods are feasible as POC examinations. In consequence, assessment of EF at the bedside has to rely on imprecise surrogate markers, visual estimates or manual tracing of the endocardial border.

The above mentioned methods all require substantial expertise to obtain an accurate result. Previous attempts have been made to circumvent the issues with user-dependent assessment of EF by means of semi-automated endocardial tracking. However, difficulties with gain-dependence and suboptimal tracing throughout the cardiac cycle have limited the applicability [10,11,12].

A relatively new semi-automated method for estimation of EF has recently been introduced. The method is named AutoEF (GE Healthcare, Horten, Norway) and is based on angle-independent speckle tracking for detection of the myocardium throughout the cardiac cycle [13, 14].

However, no studies have so far assessed the feasibility and diagnostic performance of AutoEF measurements at the bedside.

The aim of this study was to compare estimation of EF at the bedside by AutoEF compared with conventional methods and to assess feasibility and time consumption.

Methods

Study population. The study was performed in accordance with the Helsinki Declaration and informed consent was obtained from all patients. The study was reviewed by the Central Denmark Region Committees on Biomedical Research Ethics and due to the design of the study it was exempt from further ethical approval.

Patients undergoing a standard echocardiographic examination at the Department of Cardiology, Aarhus University Hospital were eligible for inclusion. The selection process was performed by an independent nurse and physician affiliated with the study. During data collection, patients were screened and included consecutively to avoid selection bias. All eligible patients were assessed on all study days; details about the selection process are described elsewhere [15]. The clinical presentation of enrolled patients was characterized by hemodynamic stability and no severe distress symptoms. All patients were admitted at the Department of Cardiology or the Department of Cardiothoracic and Vascular Surgery, Aarhus University Hospital.

Equipment and data acquisition. A Vivid S6 (GE Healthcare, Horten, Norway) ultrasound system equipped with a M4S phased array transducer (1.5-4.5 MHz) with second harmonic imaging was used to obtain data.

All patients underwent POC echocardiography at the bedside performed by a novice examiner (limited experience in echocardiography and certification equal to level I) [16, 17]. The examination was initially performed with patients placed in the supine position; if the condition of the patient allowed it, image acquisition was also performed in the left lateral position. The POC echocardiography included the following views: Subcostal 4-chamber view, apical 4-chamber view, parasternal long- and short-axis views. Raw data were digitally stored in cineloop format defined by the R-wave in the corresponding electrocardiogram for off-line analyses.

Data analyses. The interpretations and estimates were done as post-examination analyses using EchoPac software (GE Healthcare, Horten, Norway).

Three assessments of EF were made:

Expert eyeballing: A single specialist in cardiology and expert (experience equivalent to level III) [18, 19] in echocardiography visually estimated all the POC echocardiography recordings blinded to any previous assessments. All the obtained images were taken into account when the assessments were performed and the EF was reported as a percentage from 5%-75% and when in doubt as a range of 5%.

Manual planimetry: When feasible in relation to the image quality, all apical 4-chamber views were analysed by an experienced examiner blinded to previous assessments (experience equivalent to level II). Manual endocardial border tracing from end-diastole to end-systole was applied and EF was calculated according to the Simpson’s method of discs [20].

AutoEF: When feasible in relation to the image quality, all apical 4-chamber views were analysed by a novice examiner blinded to previous assessments and with limited experience in echocardiography and certification equal to level I. This examiner was not experienced enough to perform eyeballing estimates of EF.

The AutoEF software relies on 2D speckle tracking, utilizing natural acoustic markers in the tissue (Figure 1).

Example of an AutoEF assessment. The endocardial border is traced throughout the cardiac cycle by means of speckle tracking and the software automatically locates end-systolic and end-diastolic frames.

The software requires the examiner to define three regions of interest in the left ventricle, the endocardial border is then traced throughout the cardiac cycle and the software automatically locates the end-systolic and end-diastolic frames. End-diastolic and end-systolic volumes are calculated based on the tracings and serves as the basis for the EF calculation.

During manual planimetry and AutoEF assessments, the time spent from the beginning of the procedure to the determination of the EF was recorded. In addition, all quantifications of EF by manual planimetry and AutoEF were performed twice in order to assess intra-observer variability. The examiners were blinded to previous results during the entire analysis.

Statistical analyses. Data are presented as mean values and standard deviations in parenthesis. A p-value <0.05 was considered significant. Normality was assessed by inspection of histograms and quantile plots. Intergroup data were compared using a paired 2-tailed Student t-test. Correlations were estimated using the Pearson method. Comparisons of methods were performed as proposed by Bland and Altman [21] and by means of the intraclass correlation coefficients. Sample size was based on a previous study using a similar methodology [22]. Tests and calculations were performed using Stata 11.0 software (StataCorp LP, Texas, USA).

Results

A total of 102 mixed medical and surgical patients were included. The mean age of patients was 63.2 years (±16.4) and 31% were females. Information on discharge diagnosis is shown in Table 1.

Discharge diagnosis of all included patients categorized according to the primary clinical problem(s). Atrial fibrillation is separated from other arrhythmias.

Expert eyeballing of EF was performed in 100% of cases. Manual planimetry was possible in 89% and AutoEF was possible in 83% of cases. The remaining cases were excluded from analyses due to endocardial or myocardial dropout.

The mean EF in the expert eyeballing group was 45% (±17), in the manual planimetry group 49% (±16) and in the AutoEF group 45% (±14). There difference between the mean EF in the expert eyeballing group and the AutoEF group was 0.08% (-2.0-2.2) and not statistically significant (p=0.94). However, there was a significant difference in the mean EF between the manual planimetry group and the AutoEF group of 3.8% (1.8-5.8) (p=0.0003) as well as between the expert eyeballing group and the manual planimetry group of 3.6% (1.4-5.6) (p=0.0016). The relationship between expert eyeballing and AutoEF is shown in Figure 2 (r=0.82, p < 0.001), corresponding Bland-Altman analysis revealed 95% limits of agreement ranging from -19% to 19%, bias was 0%.

EF by expert eyeballing versus AutoEF. Scatter plots with line of identity on the left side

and Bland-Altman plots illustrating agreement in EF measurements on the right side. Central horizontal line in the Bland-Altman plot represents mean bias or systematic difference, upper and lower dashed horizontal lines represent 95% confidence intervals of differences (limits of agreement).

EF = ejection fraction.

The bias between expert eyeballing and manual planimetry was -3.6% and 95% limits of agreement ranged from -24% to 17%. The correlation for this comparison is shown in Figure 3 (r=0.80, p < 0.001).

EF by expert eyeballing versus manual planimetry. Scatter plots with line of identity on the left side and Bland-Altman plots illustrating agreement in EF measurements on the right side. Central horizontal line in the Bland-Altman plot represents mean bias or systematic difference; upper and lower dashed horizontal lines represent 95% confidence intervals of differences (limits of agreement).

EF = ejection fraction.

Figure 4 shows the relationship between manual planimetry and AutoEF  (r=0.82, p < 0.001). In this case the intraclass correlation coefficient was 0.79.

EF by manual planimetry versus AutoEF. Scatter plots with line of identity. EF = ejection fraction.

Figure 5 shows intra-observer data from manual planimetry assessments (r=0.87, p < 0.001) and AutoEF assessments (0.94, p < 0.001).

Intra-observer variability of manual planimetry measurements on the top and AutoEF on the bottom. Scatter plots with line of identity on the left side and Bland-Altman plots illustrating agreement in EF measurements on the right side. Central horizontal line in the Bland-Altman plot represents mean bias or systematic difference, upper and lower dashed horizontal lines represent 95% confidence intervals of differences (limits of agreement).

EF = ejection fraction.

The 95% limits of agreement for manual planimetry are -12% to 15% and in the case of AutoEF -8% to 11%.

When excluding patients with atrial fibrillation from the analysis, the correlation between expert eyeballing and AutoEF was 0.81; between expert eyeballing and manual planimetry it was 0.80 and between AutoEF and manual planimetry 0.82.

The mean time spent for manual planimetry was 98 ( 90-106 ) seconds; the corresponding mean time spent for AutoEF was 41 ( 36-46 ) seconds, which was significantly less (p < 0.001).

Discussion

This is the first study to evaluate semi-automated estimation of EF based on speckle tracking technology. In contrast to previous automated border-detection methods suffering from gain dependency, AutoEF analyses standard 2D gray scales images in a rapid and semi-automated manner. However, AutoEF depends on the same parameters as speckle tracking strain analyses, thus frame rates need to be high ( 60-80 ) and images free from myocardial dropouts [14]. These issues explain why AutoEF was only possible in 83% of cases, when manual planimetry was possible in 89% of cases.

The agreement between our chosen reference (expert eyeballing) and AutoEF was moderate to good and in fact very similar to the agreement between expert eyeballing and manual planimetry. This is in spite of the fact that the manual planimetryexaminer had experience level II and the AutoEF examiner had level I experience. In addition, intra-observer variability and time consumption was considerably less for AutoEF compared to manual planimetry.

Two previous studies have evaluated an automated method for assessment of EF, based on artificial intelligence [22, 23]. However, one of the studies showed poor correlation (r=0.64) between visual reading and automated reading and the other showed good correlation (r=0.98). This difference can probably be attributed to differences in allowing manual editing. Similarly, result from our study was affected by some degree of manual editing. However, the AutoEF assessments were performed by a level I examiner, and it is possible that the agreement would improve with a more experienced examiner.

Atrial fibrillation is a challenge when estimating EF and guidelines suggest multiple measurements because of the cycle variability. However, only 5.9% of the patients included in this study had atrial fibrillation and they were thus part of the analyses to show robustness of the method. Supplemental subgroup analyses without patients with atrial fibrillation showed no significant discrepancy of results. However, we still recommend multiple measurements of EF for clinical practice as described in the chamber quantification guidelines [20].

Clinical implications. Although EF is probably not the best measure of systolic function in the left ventricle it still plays a major role in the clinical management of many patients. In everyday clinical practice, visual assessment of EF is the most common method. However, visual assessments are highly dependent on the examiner’s experience; this is probably why many less experienced examiners rely on manual planimetry.

For the sake of less time consumption and intra-observer variability, AutoEF seems to be a valid alternative to manual planimetry. In particular, emergency situations where time is of the essence, AutoEF seems a promising method. However, caution is advised for novice examiners, because speckle tracking based methods are dependent on frame rates and myocardial presentation and in some cases requiring manual editing.

Limitations. No true gold standard method like magnetic resonance imaging or 3-dimensional echocardiography was used for reference purposes in the current study. However, as expert eyeballing is the everyday clinical gold standard in many clinical settings it seems to be a valid choice for the current study. Manual planimetry and advanced methods also report diastolic and systolic volumes which have prognostic value and these parameters should be included in future studies of the AutoEF method.

Since 34.3% of the research cohort had ischemic heart disease or acute coronary syndrome, some of these patients may have regional wall motion abnormalities. In these patients the lack of information about regional dyskinesia obviously represents a minor limitation to the use of AutoEF.

In accordance with guideline recommendations, manual planimetry needs to be a biplane assessment. However, due to the specification in the POC protocol used to acquire images for this study, no 2-chamber view was available. This has possibly contributed to the (somewhat) less accurate agreement of AutoEF compared to previous studies. However, in spite of this limitation, we still obtained better agreement than the method based on artificial intelligence without manual editing.

As no hemodynamically unstable patients were included is this study, the results need to be further validated in the acute setting with patients in supine- or semirecumbent position as well as intubated/ventilated patients.

Conclusion

AutoEF seems to be a valid supplement to the clinical assessment of EF in the hands of less experienced examiners, yielding results similar to manual planimetry when compared to expert eyeballing with less time consumption and less intra-observer variability.

However, manual editing may be required and this is why some training is recommended before AutoEF is applicable for use by novices.