Intra-observer and interobserver variability of biventricular function, volumes and mass in patients with congenital heart disease measured by CMR imaging.

Cardiovascular magnetic resonance (CMR) imaging provides highly accurate measurements of biventricular volumes and mass and is frequently used in the follow-up of patients with acquired and congenital heart disease (CHD). Data on reproducibility are limited in patients with CHD, while measurements should be reproducible, since CMR imaging has a main contribution to decision making and timing of (re)interventions. The aim of this study was to assess intra-observer and interobserver variability of biventricular function, volumes and mass in a heterogeneous group of patients with CHD using CMR imaging. Thirty-five patients with CHD (7-62 years) were included in this study. A short axis set was acquired using a steady-state free precession pulse sequence. Intra-observer and interobserver variability was assessed for left ventricular (LV) and right ventricular (RV) volumes, function and mass by calculating the coefficient of variability. Intra-observer variability was between 2.9 and 6.8% and interobserver variability was between 3.9 and 10.2%. Overall, variations were smallest for biventricular end-diastolic volume and highest for biventricular end-systolic volume. Intra-observer and interobserver variability of biventricular parameters assessed by CMR imaging is good for a heterogeneous group of patients with CHD. CMR imaging is an accurate and reproducible method and should allow adequate assessment of changes in ventricular size and global ventricular function.

Introduction

Assessment of ventricular function is important in the follow-up of patients with congenital and acquired heart disease. Cardiovascular magnetic resonance (CMR) imaging is frequently used for the assessment of both left ventricular (LV) and right ventricular (RV) size and function, because it is an accurate and non-invasive method, which has been validated extensively [1].

Reproducibility of CMR measurements plays an important role in establishing the feasibility of CMR imaging in clinical practice. Whether differences in measurements are caused by progression of disease or could be explained by intra- or interobserver variability is of crucial importance, because CMR imaging has a main contribution to decision making and timing of (re)interventions.

There are several reports on the reproducibility of CMR measurements, but most have been done in healthy patients or patients with acquired heart disease [2-11]. Only a few studies measured reproducibility in patients with CHD [12-15]. These studies were performed using gradient echo imaging pulse sequences [12] or only examined a selected group of patients [tetralogy of Fallot (TOF), atrial septal defect (ASD) or systemic RV] [13-15]. Intra-observer and interobserver variability has never been studied in a heterogeneous group of patients with CHD, representative for the total spectrum in a clinical program.

In patients with CHD, the RV is often involved in the disease process. The geometric shape of the RV can be altered by abnormal volume- and/or pressure loading conditions, e.g. caused by pulmonary regurgitation in patients with TOF, or due to extensive trabeculation with hypertrophy, as in patients with intra-atrial correction of transposition of the great arteries (TGA). In theory, the complex geometry of the RV in patients with CHD can potentially lead to higher intra- and interobserver variability, compared to measurements in healthy volunteers.

The objective of this study was to assess intra-observer and interobserver variability of biventricular function, volumes and mass in a heterogeneous group of patients with CHD using CMR imaging.

Methods

Subjects

Thirty-five patients with CHD (26 males, 9 females; mean age 22 ± 13 years, range 7–62 years) were included in this study. The subjects were selected from the total group of patients with CHD in whom CMR imaging was requested in daily clinical practice in 2007. The characteristics of the study population are displayed in Table 1. The distribution of diagnoses in our study group was representative of the distribution of diagnoses in the total group of subjects with CHD undergoing CMR imaging in 2007.

Table 1

Characteristics of the study population

Characteristic	Value
Gender (male/female)	26/9
Age (years)	22.2 ± 13.2 (6.8 – 61.6)
Heart rate (beats per minute)	76 ± 11 (62 – 101)
Diagnosis	n = 35
Aortic stenosis (repaired/unrepaired)	n = 5 (4/1)
ASD (unrepaired)	n = 1
ccTGA, PA, VSD (repaired)	n = 1
DORV, VSD, coarctation (repaired)	n = 1
Fontan circulation (dominant RV/dominant LV)	n = 10 (3/7)
Intra-atrial correction of TGA	n = 3
PA, VSD (repaired)	n = 3
Pulmonary stenosis (repaired)	n = 1
Tetralogy of Fallot (repaired)	n = 8
VSD (unrepaired)	n = 2

Reported data are expressed as mean ± SD (range)

ASD atrial septal defect, ccTGA congenitally corrected transposition of the great arteries, PA pulmonary atresia, VSD ventricular septal defect, DORV double outlet right ventricle, RV right ventricle, LV left ventricle, TGA transposition of the great arteries

The study was approved by the institutional review board.

CMR image acquisition

CMR imaging was performed using a Signa 1.5 Tesla whole-body MR imaging system (General Electric, Milwaukee, WI, USA). An 8-channel phased-array cardiac surface coil was placed on top and beneath the chest. All patients were monitored by vector cardiogram gating and respiratory monitoring. Studies were performed by experienced MR-technicians, supervised by one of the four physicians (SEL, DR-V, AM, WAH), or by the physicians themselves. Standard scout images were made to obtain a four chamber view of the heart. A short axis set, using steady-state free precession (SSFP) cine imaging, was acquired from base to apex. An average of 13 contiguous slices were planned on the four chamber image, parallel to the atrioventricular valve plane of the LV in end-diastole. Typical imaging parameters were: repetition time 3.4 ms, echo time 1.5 ms, flip angle 45º, receiver bandwidth 125 kHz, slice thickness 7–10 mm, inter-slice gap 0–1 mm, field of view 380 × 380 mm, phase field of view 0.75 and matrix 164 × 128 mm. All images were obtained during breath-hold in end-expiration.

CMR analysis

The CMR studies were analyzed on a commercially available Advanced Windows workstation (General Electric Medical Systems, Milwaukee, WI, USA), equipped with Q-mass (version 5.2, Medis Medical Imaging Systems, Leiden, the Netherlands).

The ventricular volumetric data set was quantitatively analyzed using manual outlining of endocardial and epicardial borders in end-systole and end-diastole. The following parameters were calculated: biventricular end-diastolic volume (EDV), end-systolic volume (ESV), stroke volume (SV), ejection fraction (EF) and mass. Criteria for border detection were used as described by Robbers-Visser et al. [11]. Specifically: end-diastole and end-systole were visually defined on multiple midventricular slices. In the basal slices, the following criteria were used: (1) when the cavity was only partially surrounded by ventricular myocardium, only the part up to the junction with atrial tissue was included in the ventricular volume; (2) when the pulmonary or aortic valve was visible in the basal slice, contours were drawn up to the junction with the semilunar valves [16]. The interventricular septum was included in the left ventricular mass. Major papillary muscles and trabeculations were excluded from the ventricular volumes and included in the ventricular mass [17].

Ventricular volume was calculated as the sum of the ventricular cavity areas multiplied by the slice thickness. Ventricular mass was calculated as the difference between the epicardial and endocardial contours multiplied by the slice thickness and a specific gravity of the myocardium of 1.05 g/ml [18].

All data sets were analyzed by one observer (SEL). For intra-observer variability, studies were reanalyzed after an average period of 6 months. For interobserver variability, a second observer (DR-V) analyzed all studies and measured the aforementioned parameters independently and blinded to previous results.

Statistical analysis

Data are expressed as frequencies, or mean ± standard deviation. Intra- and interobserver variability was assessed using the method of Bland–Altman [19]. The coefficient of variability, i.e. the standard deviation of the difference of the two measurements divided by the mean of the two measurements, and multiplied by 100%, was calculated to study the percentage of variability of the measurements. A P value < 0.05 was considered statistically significant.

Results

The intra-observer and interobserver variability data are displayed in Table 2. Intra-observer variability was between 2.9 and 6.8%, with the smallest variation in measurements of LV and RV EDV (2.9 and 3.0% respectively). The highest variation was found in LV ESV (6.8%) and RV mass (5.7%). Interobserver agreement demonstrated more variation for all variables and was between 3.9 and 10.2%. The smallest variation was found in LV EF (3.9%) and RV and LV EDV (4.0 and 4.3% respectively). The highest variation was found in measurements of LV and RV ESV (10.2 and 7.7% respectively) and LV and RV mass (6.0 and 6.2% respectively).

Table 2

Intra-observer and interobserver variability analysis

	LV-EDV	LV-ESV	LV-SV	LV-EF	LV-mass	RV-EDV	RV-ESV	RV-SV	RV-EF	RV-mass
Intra-observer
Mean difference ± SD	1.6 ± 3.9	1.9 ± 3.4	–0.4 ± 3.4	–0.9 ± 2.3	0.6 ± 4.1	0.9 ± 5.3	3.4 ± 3.4	–2.5 ± 4.4	–1.6 ± 1.9	0.1 ± 4.7
Limits of agreement	−6.3 to 9.4	−4.9 to 8.8	−7.2 to 6.5	−5.6 to 3.7	−7.6 to 8.9	−9.7 to 11.5	−3.3 to 10.2	−11.4 to 6.4	−5.4 to 2.1	−9.3 to 9.5
Mean value ± SD	134.9 ± 42.5	50.3 ± 19.2	84.7 ± 27.8	62.9 ± 6.4	125.2 ± 47.3	179.3 ± 84.8	86.9 ± 51.9	92.3 ± 37.2	53.8 ± 9.5	81.8 ± 44.6
Coefficient of variability (%)	2.9	6.8	4.1	3.7	3.3	3.0	3.9	4.8	3.5	5.7
Inter-observer
Mean difference ± SD	0.3 ± 5.8	1.6 ± 5.1	–1.1 ± 4.4	–0.9 ± 2.5	–0.3 ± 7.5	4.0 ± 7.1	6.8 ± 6.4	–2.6 ± 5.2	–2.7 ± 3.0	2.4 ± 5.0
Limits of agreement	−11.2 to 11.9	−8.6 to 11.8	−9.9 to 7.6	−5.9 to 4.1	−15.3 to 14.8	−10.2 to 18.1	−6.1 to 19.6	−13.0 to 7.8	−8.7 to 3.4	−7.5 to 12.4
Mean value ± SD	135.0 ± 41.6	49.7 ± 18.9	85.4 ± 28.0	63.5 ± 6.9	125.3 ± 47.6	177.6 ± 83.7	83.8 ± 50.9	93.9 ± 37.4	55.2 ± 9.8	80.6 ± 45.6
Coefficient of variability (%)	4.3	10.2	5.1	3.9	6.0	4.0	7.7	5.5	5.5	6.2

Reported data are expressed as mean ± SD. Coefficient of variability: expressed as a percentage of the SD of the difference divided by the mean of the two measurements. Volumetric variables in milliliters, mass variables in grams

LV left ventricle, EDV end-diastolic volume, ESV end-systolic volume, SV stroke volume, EF ejection fraction, RV right ventricle

Discussion

In our study, intra-observer and interobserver variability for all variables was good. Overall, the variations were smallest for biventricular EDV and highest for biventricular ESV. Although we expected higher intra- and interobserver variability in the RV, because of its complex shape and heavy trabeculations, we did not find a difference in results for both ventricles. Our results are comparable to those of other studies who reported on reproducibility with SSFP CMR imaging [3, 4, 6–8, 10, 11, 13, 15] (Tables 3 and 4).

Table 3

Reports assessing intra-observer variability of biventricular parameters by SSFP CMR imaging

Study	Population/diagnosis	M/F	Age (yrs)	Statistics	LV					RV
					EDV (%)	ESV (%)	SV (%)	EF (%)	Mass (%)	EDV (%)	ESV (%)	SV (%)	EF (%)	Mass (%)
This study	n = 35 CHD	26/9	22 ± 13 (7–62)	CoV n = 35	2.9	6.8	4.1	3.7	3.3	3.0	3.9	4.8	3.5	5.7
Catalano [3]	n = 45 Acquired/Healthy	33/12	49 ± 20 (8–83)	CoV n = 45						7	9		10	12
Clay [4]	n = 20 Acquired/Healthy	14/6	30	CoV n = 20	5.2	6.5		2.9	4.7
Hudsmith [6]	n = 108 Healthy	63/45	38 ± 12 (21–68)	CoV* n = 12	5.6			2.3	6.1	9.0			5.3
Maceira [8]	n = 120 Healthy	60/60	(20–80)	CoV n = 10						3.6	6.5	5.9	4.0	5.7
Plein [10]	n = 41 Acquired/Healthy	23/18	53 (32–77)	CoV n = 10	2.8	10.1	3.4	5.2	3.9
Robbers-Visser [11]	n = 60 Healthy	30/30	(8–17)	CoV n = 15	2.1	5.8	2.6	0.01	3.9	2.1	6.0	2.5	0.08	6.2
Winter [15]	n = 29 Systemic RV	20/9	35 ± 12	CoV n = 20						6.1	6.7	9.4	5.8

Reported data are expressed as mean ± SD (range). * CoV, indexed for body surface area (BSA)

CHD congenital heart disease, Acquired acquired heart disease, Healthy healthy controls, M male, F female, CoV coefficient of variability, LV left ventricle, RV right ventricle, EDV end-diastolic volume, ESV end-systolic volume, SV stroke volume, EF ejection fraction

Table 4

Reports assessing interobserver variability of biventricular parameters by SSFP CMR imaging

Study	Population/diagnosis	M/F	Age (yrs)	Statistics	LV					RV
					EDV (%)	ESV (%)	SV (%)	EF (%)	Mass (%)	EDV (%)	ESV (%)	SV (%)	EF (%)	Mass (%)
This study	n = 35 CHD	26/9	22 ± 13 (7–62)	CoV n = 35	4.3	10.2	5.1	3.9	6.0	4.0	7.7	5.5	5.5	6.2
Catalano [3]	n = 45 Acquired/Healthy	33/12	49 ± 20 (8–83)	CoV n = 45						8	11		12	13
Clay [4]	n = 20 Acquired/Healthy	14/6	30	CoV n = 20	3.3	6.8		4.6	3.7
Hudsmith [6]	n = 108 Healthy	63/45	38 ± 12 (21–68)	CoV* n = 12	2.7			3.3	5.2	9.6			10.7
Karamitsos [7]	n = 10 Healthy	5/5	34 ± 14	CoV n = 10	2.6	6.9	3.4	2.9	5.8
Maceira [8]	n = 120 Healthy	60/60	(20–80)	CoV n = 10						6.3	8.6	7.0	4.4	7.8
Mooij [13]	n = 60 ASD/TOF/Healthy	34/26	21 ± 13	CoV* n = 60	3.6	10.5	6.6	5.8	5.3	6.4	13.0	11.8	8.0	11.3
Plein [10]	n = 41 Acquired/Healthy	23/18	53 (32–77)	CoV n = 41	4.3	9.4	9.5	6.0	6.0
Robbers-Visser [11]	n = 60 Healthy	30/30	(8–17)	CoV n = 15	5.3	13.9	3.3	2.3	6.1	3.4	7.6	4.5	2.0	8.9
Winter [15]	n = 29 Systemic RV	20/9	35 ± 12	CoV n = 20						10.0	12.7	12.2	7.5

Reported data are expressed as mean ± SD (range). * CoV, indexed for BSA

ASD atrial septal defect, TOF tetralogy of Fallot. Other abbreviations as in Table 3

Recently, Mooij et al. [13] reported on reproducibility in patients with RV dilation [unrepaired ASD (n = 20); TOF (n = 20)] and in a normal RV group (n = 20). Variability for all patients ranged from 3.6 to 13.0%. Mooij et al. examined a selected group of patients, whereas our study population consisted of a more heterogeneous group of patients with CHD, representative for the total spectrum in a clinical program. Valsangiacomo–Buechel et al. [14] reported that observer variability, in ten children with TOF, ranged from <1 to 5% for intra-observer analysis and from <1 to 13% for interobserver analysis. Although we cannot compare all our reported results to theirs, this seems comparable to our results.

Similar to our results, other authors found the largest amount of intra-observer or interobserver variation in biventricular ESV [4, 8, 10, 13]. One of the possible explanations is the smaller absolute value of ESV. Similar absolute measurement errors will therefore lead to higher observer variation in ESV, compared to for example EDV. Another source of error is the endocardial border detection, which is more difficult in end systole due to more densely packed trabeculations and papillary muscles [15].

Image analysis

A critical review of contours traced revealed that the interobserver variation for both ventricles was mainly caused by different interpretations in the basal slice and in the apical slices. Guidelines for image analysis might be helpful, but are still a subject of debate. Most authors agree on criteria on how to draw contours in the basal slice [4, 6, 9, 16, 20]. However, there is less consensus about inclusion or exclusion of papillary muscles and trabeculations [3, 6, 15, 16]. It is important that the used criteria for border detection are described in reports, because inclusion or exclusion of papillary muscles and trabeculations cause differences in measurements of biventricular dimensions and function [15]. To reduce observer variability, it is important to have clear guidelines for methods of delineation in routine clinical practice as well as in research projects, since different observers may develop slightly different habits.

The basal slice will remain an area in which there may be discussion if image acquisition in the short axis plane is used, particularly for the RV. Alternative imaging orientations have been studied, as well as methods to improve image analysis and assessment of volumes and mass. For example, Alfakih et al. [2] found that observer variability for RV measurements in the axial orientation was slightly lower compared to results of the short axis orientation. Strugnell et al. [21] reported on a modified RV short axis orientation, which is aligned to the outflow of the RV. This method demonstrated a closer agreement between the RV and LV stroke volumes compared to the current method. However, observer variability analysis was not performed and should be assessed to establish the real advantage of this new method. Both the axial orientation as well as the modified RV short axis orientation make detection of the atrioventricular valve border easier. However, the major advantage of the use of the short axis orientation is that only one data set is required for both LV and RV measurements. Furthermore, in the axial orientation, the partial volume effect of blood and myocardium on the inferior wall of the RV can make it difficult to identify the blood/myocardial boundary [2].

Kirschbaum et al. [22] have reported that identification of the mitral valve plane and apex on long-axis images in addition to short axis contours reduces the interstudy variability for all parameters in LV functional assessment, when compared with using short axis images alone. This method might be applicable for the RV as well.

Van der Geest et al. [20] suggested that semiautomated contour detection is less hampered by random variabilities. At present, semiautomatic contour detection algorithms are only available for the LV and still require manual correction in a significant number of slices [10, 20]. Further analysis and improvement of these algorithms is needed to demonstrate a reduction in observer variation.

Catalano et al. [23] and Corsi et al. [24] reported on a technique for volumetric surface detection (VoSD) and quantification of biventricular volumes without tracing and geometric approximations. The VoSD method showed lower observer variation for all parameters compared to the short axis method. Although limitations clearly exist, this technique might improve reproducibility of biventricular assessments [23, 24].

Limitations

The size and variation of our population prevented subgroup analysis. The amount and size of trabeculations and papillary muscles might be a cause of differences in variation between subgroups. In theory, the extensive trabeculations and large papillary muscles in patients with intra-atrial correction of TGA can potentially lead to higher observer variability compared to patients, in whom the shape of the RV is less altered by abnormal loading conditions. In patients after Fontan operation, the interpretation of the basal slice might be more difficult due to the abnormal anatomy, which can potentially lead to higher observer variability too.

Another issue that should be taken into consideration when evaluating follow-up data, is that variation in CMR measurements could also be caused by interoperator variation, introduced during CMR planning, as reported by Danilouchkine et al. [25]. In research protocols it is favourable to have all studies carried out by the same operator, but in routine clinical practice, this is more difficult to achieve. In our center, image acquisition is performed according to a standard protocol and studies are carried out by experienced technicians, under direct supervision of an experienced CMR cardiologist/radiologist, to reduce operator variability.

Conclusions

Variations within and between observers and operators will remain an important issue to be taken into consideration when evaluating follow-up data of MRI measurements, especially in patients in whom CMR imaging contributes to decision-making and timing of (re)interventions as in patients with CHD. Our results show that intra-observer and interobserver variability of biventricular parameters assessed by CMR imaging, using SSFP, is good in a heterogeneous group of patients with CHD. CMR imaging is an accurate and reliable method for follow-up of biventricular function and mass and should allow adequate assessment of changes in ventricular size and global ventricular function.