A Pilot Study Comparing Aortic Valve Area Estimates Derived from Fick Cardiac Output with Estimates Based on Cheetah-NICOM Cardiac Output.

Cardiac output during cardiac catheterization is often estimated using the modified Fick method (COFick). In this proof-of-concept, prospective non-randomized study carried out in a single academic healthcare centre, we examined whether replacing COFick in the Gorlin formula with Cheetah-NICOM monitor cardiac output (COCheetah) could produce an accurate and precise estimate of aortic valve area in patients with severe aortic stenosis. In twenty-six subjects, COFick and COCheetah were obtained concurrently. A spot and 3-minute running average of COCheetah was used. Bland and Altman analysis was used to derive bias, 95% limits of agreement (LOA) and confidence intervals (CI). The mean difference (bias) between AVACheetah (average) and AVAFick was 0.11 cm2 and the 95% LOA were ±0.42 cm2. The 95% CI of the bias was 0.02-0.2 cm2. The bias and 95% LOA of AVACheetah (spot value) were 0.14 ± 0.42cm2, with a 95% CI of 0.06-0.23 cm2. No proportional bias was present. AVACheetah thus appears to be a reasonably accurate measure of AVA in patients with severe aortic stenosis compared to AVAFick measured using a modified Fick CO. However, the limits of agreement were not narrow enough to consider AVACheetah and AVAFick interchangeable.

Introduction

Left and right heart catheterization is standard of care in the pre-operative evaluation of the patient with aortic stenosis (AS). Catheterization is performed to evaluate the coronaries and not simply to confirm the presence and degree of severe aortic stenosis. The decision to pursue intervention depends on many factors including the patient’s symptomatology, and is not based solely on the transvalvular gradient or the calculated stenotic orifice area.

Left heart catheterization assesses peak and mean pressure gradients across the aortic valve, the state of the patient’s coronary vasculature, and provides an angiographic assessment of left ventricular performance. Right heart catheterization is performed in order to measure the various pressures from the superior vena cava to pulmonary artery, to obtain mixed venous blood sampling and estimate cardiac output (CO) using the Fick method (COFick). In clinical practice, a modified Fick method is commonly used, which assumes oxygen consumption to be 125 millilitres x BSA (body surface area). Aortic Valve Area (AVAFick) is then calculated using the Gorlin formula[1,2].

We hypothesized that substituting CO derived from a Cheetah monitor (Cheetah Medical Inc., Newton Centre, Massachusetts, USA; COCheetah) in the Gorlin formula could produce accurate and precise estimates of AVA relative to AVAFick. If this were the case, it could be possible to avoid right heart catheterization in patients in whom this procedure is performed solely for the purposes of obtaining COFick via the modified Fick method. Using a non-invasive CO monitor in this fashion would not eliminate the assumptions of the Gorlin formula itself nor any potential errors deriving thereof.

Results

After the subjects provided informed consent, we enrolled 26 subjects in the study in a consecutive, non-randomized fashion. Subjects were screened through the cardiac catheterization laboratory schedule. All patients in the final analysis had at least moderate AS. Only one subject had atrial fibrillation, but was permanently paced with an implanted pacemaker. All subjects had a tricuspid AV. There was one missing AVAEcho value, which was not calculated during the subject’s TTE. In two patients the AV could not be crossed, and they were therefore unevaluable for the purposes of the study. In one subject, the aortic valve was found not to be stenotic, and that patient was also excluded from the Bland and Altman analysis. The final sample thus consisted of 23 subjects. No unexpected outliers or artefacts were encountered, and the patients were at steady state when the Fick and Cheetah measurements were recorded. This was verified via review of the Cheetah data export files and the catheterization procedure record. None of the subjects were noted to have cardiomegaly or dilatation of the aorta on their TTE reports or during aortography/ventriculography.

The baseline physiologic characteristics of the subjects are summarized in Table 1.

Table 1

Baseline physiologic characteristics of the subjects in the study sample.

	Age	Gender	Height (cm)	Weight (kg)	BSA (m2)	BMI (kg/m2)	Peak to peak AV gradient (mmHg)	Mean AV gradient (mmHg)
Mean	78.7	Male = 13 (57%)	166.2	87.8	1.9	30.4	41	43
Range	61–88		147–188	49–142	1.4–2.6	21–44	5–119	18–82

cm, centimetres; kg, kilograms; m2, meters squared; mmHg, millimetres mercury; BSA, body surface area; BMI, body mass index; AV, aortic valve.

Average and spot COCheetah were substituted in the Gorlin formula (Eq. 2), deriving the corresponding average AVACheetah and spot AVACheetah. The biases (mean differences), standard deviations, 95% limits of agreement, mean error percent and 95% confidence intervals are summarized in Table 2.

Table 2

Bias (mean difference), limits of agreement, confidence intervals and mean error percent (percent limits of agreement) for Cheetah CO and AVA, as well as echocardiogram-derived AVA, compared to the cardiac catheterization (Fick) reference values.

Variable	Mean Bias	Range of bias	SD	95% LOA	Mean error % (%LOA)	95%CI of bias	95%CI of LLOA	95%CI of ULOA
average COCheetah − COFick (L/min)	0.70	−1.89–3.35	1.28	±2.65	53%	0.15–1.25	−2.90– −0.99	2.39–4.30
spot COCheetah − COFick (L/min)	0.88	−2.06–2.95	1.24	±2.57	51%	0.35–1.42	−2.62– −0.76	2.52–4.38
average AVACheetah − AVAFick (cm2)	0.11	−0.17–0.57	0.20	±0.42	51%	0.02–0.2	−0.46– −0.16	0.38–0.69
spot AVACheetah − AVAFick (cm2)	0.14	−0.49–0.50	0.20	±0.42	51%	0.06–0.23	−0.42– −0.12	0.41–0.71
AVAEcho − AVAFick (cm2)	−0.05	−1.25–0.91	0.37	±0.76	92%	−0.2–0.11	−1.08– −0.53	0.44–0.99

CO = cardiac output; SD = standard deviation; LOA = limits of agreement; Mean error % = %LOA = 95% limits of agreement expressed as percentage of the mean AVAFick or COFick, respectively; 95% CI = 95% confidence interval; LLOA = lower limit of agreement; ULOA = upper limit of agreement; average COCheetah = COCheetah averaged over the 3-minute period preceding the time of blood sampling for COFick; spot COCheetah = COCheetah obtained at the time of blood sampling for COFick as a spot value; average AVACheetah = AVA calculated with average COCheetah in the Gorlin formula; spot AVACheetah = AVA calculated with spot COCheetah in the Gorlin formula.

The Bland and Altman plots for average COCheetah, spot COCheetah, average AVACheetah, spot AVACheetah, and AVAEcho are presented in Figs. 1 and 2, along with regression lines and 95% CIs. COFick and AVAFick were used on the x-axis per the methodology explained by Jan Krouwer[3], as these were the measurements of reference.

Figure 1

Bland and Altman plots showing the bias (mean difference) of average COCheetah − COFick (panel A) and spot COCheetah − COFick (panel B). The dashed lines show the 95% LOA defined as ± tα, n−1 *SD, where tα is the t-value corresponding to n−1 degrees of freedom at an α error of 0.05, n is the sample size and SD = standard deviation. The sloped line represents a regression line quantifying the presence of proportional bias (if any). The lack of significant difference of the slope from 0 suggests lack of proportional bias. The Shapiro-Wilk test for normality was passed for regressions in the average (P = 0.64) and the spot data (P = 0.75). Shaded areas represent the 95% confidence intervals around the limits of agreement and the bias.

Figure 2

Bland and Altman plots showing the bias (mean difference) of average AVACheetah (panel A), spot AVACheetah (panel B), and AVAEcho (panel C). The dashed lines show the 95% LOA defined as ± tα, n−1 *SD, where tα is the t-value corresponding to n−1 degrees of freedom at an α error of 0.05, n is the sample size and SD = standard deviation. The sloped lines in panels A and B represent regression lines quantifying the presence of proportional bias (if any). The Shapiro-Wilk test for normality was passed for regressions in the average Cheetah (P = 0.19) and the spot Cheetah data (P = 0.45), but not in the AVAEcho (P = 0.01). Therefore, a Spearman correlation line was plotted in panel C instead. The lack of significant difference of the slope from 0 suggests lack of proportional bias for the Cheetah or echo-derived AVA. Shaded areas represent the 95% confidence intervals around the limits of agreement and the bias.

The biases (mean differences) for CO and AVA were checked for normality using normal quantile plots as recommended by Montenij et al.[4]. The plots are shown in Figs. 3 and 4. The plots indicated normal distribution of the data.

Figure 3

Normal quantile plots of the bias (mean difference) of average COCheetah − COFick (panel A), spot COCheetah − COFick (panel B), and COFick (panel C). The plots indicate normal distribution of the data.

Figure 4

Normal quantile plots of the bias (mean difference) of average AVACheetah (panel A), spot AVACheetah (panel B) and AVAEcho (panel C) from the reference method (AVAFick). Panel D is a normal quantile plot for AVAFick. The plots indicate normal distribution of the data.

Figure 5 is a plot of the CO biases against BSA with a regression line used to demonstrate the absence of proportional bias. The lack of significant difference of the regression slope from 0 (p = 0.44 and p = 0.42 for average and spot COCheetah, respectively) indicates that there was no such bias related to BSA.

Figure 5

Average COCheetah − COFick (panel A) and spot COCheetah − COFick (panel B) plotted against BSA. The regression lines do not significantly differ from 0, indicating absence of bias proportional to BSA.

We also plotted the biases of AVACheetah and AVAEcho against cardiac index obtained from COFick and performed a linear regression analysis (Fig. 6). All regressions passed a normality test. We found a moderate correlation of the biases with CI (p < 0.05). This finding was expected, as both the Gorlin formula and the continuity equation are known to be flow-dependent[5]. The significant correlation between the biases and CI further indicates that there are different response characteristics between the Cheetah and Fick estimations of AVA with increases of CO. The constant between the two indicates a predictability between the two measurements with changes in CO, while not being interchangeable. Because CI also depends on BSA, we assessed if the difference between the AVACheetah and AVAFick measurements was associated with BSA. We performed linear regression between BSA and the AVACheetah − AVAFick measurements. We found that the difference between average AVACheetah (R = 0.06, p = 0.78) or spot AVACheetah (R = 0.09, p = 0.69) and AVAFick was not significantly associated with BSA.

Figure 6

Linear regression analysis of average AVACheetah −AVAFick (panel A), spot AVACheetah − AVAFick (panel B), and AVAEcho − AVAFick (panel C) vs. CI obtained from COFick. A normality test was passed by all the mean differences (biases).

Discussion

The concept of using a non-invasive CO monitor to estimate AVA in patients undergoing cardiac catheterization for pre-operative evaluation of AS has, to our knowledge, not been studied in the past. Our main objective was to compare AVA calculated by substituting COCheetah for COFick in the Gorlin formula. If replacing the modified Fick CO with COCheetah produced accurate and precise results over an acceptable range of AVA, the more invasive right heart catheterization could theoretically be eliminated in patients who do not have right sided pathology (e.g. shunts) that requires invasive evaluation. This could potentially increase overall patient safety and decrease risk, procedure time, complications and perhaps also cost. The use of the Cheetah-NICOM monitor would seem suitable during cardiac catheterization as it can be used in spontaneously-breathing patients and is completely non-invasive. For the estimation of AVA, a single reliable CO measurement would be required (perhaps averaged over several minutes). The use of COCheetah would not eliminate the known shortcomings of the Gorlin formula. These include its assumption of a constant flow through a fixed orifice, its lack of accounting for the inertia to leaflet opening introduced by the diseased valve, the lack of accounting for valvular load and pulsatile arterial load, and the assumed quadratic relationship between blood flow and the pressure gradient[6,7]. Whether a continuous CO monitor used in this way accurately measures trends in CO over time, for which a variety of assessment methods have been proposed[4,8], is immaterial in this scenario and was not tested.

We found the bias of AVACheetah vs. AVAFick to be acceptable (0.11 and 0.14 cm2 for average and spot AVACheetah, respectively). No proportional bias was present, and the biases were normally distributed. However, the 95% LOA were wider than our desired clinical equivalence cut-off. In comparison to the Cheetah data, AVAEcho had a smaller bias (−0.05 cm2), but even wider limits of agreement vs. AVAFick. This discrepancy is hardly surprising given that AVAEcho is a measurement relying on Doppler ultrasound and the continuity equation, whereas AVACheetah relies on CO and the empirical relationships and assumptions inherent in the Gorlin formula. In addition, most of the patients in our sample had received small amounts of fentanyl and/or midazolam for comfort, whereas none had been sedated during their echocardiograms. This can be considered a limitation of our study. Nevertheless, it can be concluded that neither AVACheetah nor AVAEcho are clinically equivalent to AVAFick in this setting.

COCheetah vs. COFick merits some additional consideration. Non-invasive CO monitors have variable biases and limits of agreement relative to a method of reference such as the Fick or thermodilution CO (TDCO)[9-14]. Prior studies comparing Cheetah-NICOM to a reference method have shown smaller biases and LOA for COCheetah than the ones we found in our study. Squara et al. compared the performance of the Cheetah-NICOM monitor to a continuous TDCO system in 110 intensive care patients, and found COCheetah to have a bias of +0.16 L/min with LOA of ±1.04 L/min[15]. Another multi-institution study in intensive care patients (n = 70) demonstrated a bias of −0.09 L/min, with LOA ± 2.4 L/min, when the Cheetah monitor was compared to TDCO[16]. A study by Rich et al. compared COCheetah to both COFick and TDCO during right heart catheterization in 24 subjects with pulmonary hypertension[17]. In this study, bias and LOA of COCheetah vs. COFick were 0.21 ± 2.3 L/min. Bias and LOA of COCheetah vs. TDCO were −0.37 ± 2.6 L/min. For comparison, bias and LOA of COFick vs. TDCO were −0.91 ± 2.1 L/min.

We calculated the mean error % of COCheetah and AVACheetah as recommended by Montenij9, a value which has also been referred to as %LOA[18,19]. In order to determine whether a new method of CO measurement can be considered equivalent to a reference method, one has to take into account the precision of the reference method. An imprecise reference technique will lead to wide LOA and high mean error of measurement (ME) independent of the precision of the new method (in this case, COCheetah), because:

For[4,18] the methods to be interchangeable, the experimental precision should not exceed the reference precision.

Few reports exist in the literature providing data on the precision of COFick as compared to another reference method such as TDCO. From the data in the aforementioned report by Rich et al.[17], we were able to calculate the precision of COFick as 44%. A study by Dhingra et al. in critically ill patients revealed an overall precision of COFick of 64% across a wide range of CO values[9]. Conservatively accepting the lower of these values as the lowest acceptable precision of an experimental method and substituting this value (0.44) in Eq. 1, we obtain a ME of 63%. In other words, the mean error percent of the test method should not exceed this value in order to be considered of equivalent (interchangeable) precision as COFick. In our study, the mean error of COCheetah (3-minute average) was 53%, so it would appear to fit this criterion. However, our absolute bias of COCheetah was higher compared to that found in the aformentioned studies[9,15-17]. We wish to once again stress that our study should not be perceived as a true CO monitor validation study. For this to have been the case, we would have measured O2 consumption in the calculation of COFick and not used the estimated O2 consumption. Alternatively, in a CO validation study, TDCO could have been used as the reference method. In this study, our main goal was to compare AVACheetah to AVAFick, with the latter value being calculated using estimated O2 consumption. The rationale was to examine the possibility of eliminating right heart catheterization where it is not otherwise indicated, and replace it with a completely non-invasive CO monitor.

A limitation of this study was the relatively small sample size. We assessed the agreement between the Cheetah and the Fick measurements using Bland-Altman analysis. Bland-Altman analysis is not a statistical test and performing power analysis is thus challenging[4]. Before the study commenced, we defined the systematic bias (0.1 cm2) and limits of agreement (±0.2 cm2) that we deemed acceptable for the primary outcome AVA measurement to conclude that the Cheetah and Fick AVA measurement would be interchangeable. We found that the systematic bias was close to the predefined threshold (0.11 and 0.14 cm2, for average and spot Cheetah values), whereas limits of agreement were considerably wider than the predefined threshold (±0.42 cm2 and ±0.42 cm2, Table 2 and Fig. 2). This let us to conclude that the two methods are not interchangeable, since one predefined criterion was not met. To assess the influence of the sample size on the findings we calculated 95% confidence intervals around the lower and upper limits of agreement[20]. We found that upper and lower boundaries of these respective limits were still beyond the predefined threshold (±0.27 and ±0.27, Table 2 last two columns). These results indicate that it is unlikely that including more subjects would have led to a different conclusion for our primary outcome (i.e. that we would have concluded that the Cheetah AVA estimation is interchangeable with Fick AVA estimation at the predefined threshold for limits of agreement).

In conclusion, based on this pilot study, we cannot recommend that right heart catheterization and the modified Fick CO be replaced by Cheetah-NICOM CO in the evaluation of the stenotic aortic orifice area. The bias of AVACheetah was 0.11 and 0.14 cm2 (average and spot values, respectively), which is a slight overestimation of orifice area and would likely not lead to the patient being turned down for a necessary intervention if their aortic valve were sufficiently stenotic. However, the 95% LOA were ±0.42 cm2. We consider this too wide to claim that AVACheetah is interchangeable with AVAFick. The value this pilot study brings to science is that it highlights the potential role a non-invasive CO monitor could play in the evaluation of AVA. More studies could further refine this concept.

Methods

Cheetah non-invasive cardiac output monitor

The Cheetah-NICOM monitor uses bioreactance to measure cardiac output in a non-invasive fashion[15,16,21]. Bioreactance is based on the phase shifts of an alternating current applied to the thorax, produced by the pulsatile flow of blood in the large thoracic arteries. These phase shifts between the applied alternating current and the measured thoracic voltage are tightly correlated with the stroke volume (SV). By accurately and continuously measuring phase shifts, the monitor determines SV using 4 electrodes applied to the chest wall, two above and two below the diaphragm on each side of the thorax[12]. The NICOM signal effectively measures the blood volume change in the thorax between systole and diastole. The measurement can be performed both in spontaneously breathing and mechanically ventilated patients, and in patients with arrhythmias. Unlike bioimpedance-based CO monitors, the bioreactance Cheetah method has the advantage of being independent from the distance between the electrodes[11]. Because the Cheetah monitor electrodes are paired (one set on either side of the body), two separate CO signals are obtained and then averaged to produce the final CO measurement. The Cheetah CO values were exported as 1-minute running averages from the Cheetah monitor for the duration of the procedure.

Inclusion and exclusion criteria

Subjects were enrolled if they met the following criteria: (i) were at least 18years of age, (ii) were able to provide signed written informed consent to participate in the study, (iii) had a diagnosis of at least moderate aortic stenosis (AS) on transthoracic echo (TTE), and (iv) the subjects’ height and weight could be accurately obtained prior to entering the study.

Subjects were excluded if the following conditions were present: (a) aortic or tricuspid valve regurgitation; (b) atrial fibrillation with irregular rhythm (i.e., not paced); (c) intra-cardiac shunt; (d) the use of intra-aortic balloon pump; (e) intubated or unconscious patients; (g) patient known to be pregnant; (h) emergency heart catheterization; (i) uncompensated congestive heart failure; (j) suspected significant hypovolemia; and (k) subject currently participating in an investigational drug or device study that interferes with the study endpoints.

Catheterization laboratory procedures

The subjects underwent right and left heart catheterization. All time-stamped catheterization data, including COFick, peak and mean transvalvular gradients, and the systolic ejection period (SEP) were obtained using the McKesson Cardiology Station Release 13.0 (McKesson Corporation, San Francisco, CA). The study subjects received an average of 41 micrograms of fentanyl (range: 0–100; two did not receive any fentanyl) and 0.96 milligrams of midazolam (range: 0–2; five subjects did not receive any midazolam). COFick was obtained from mixed venous and aortic blood samples. COCheetah was obtained by applying the Cheetah monitor electrodes to the chest wall for the duration of the catheterization. Two sets of COCheetah values were derived: the spot value from the Cheetah monitor at the time of COFick (spot COCheetah), and a 3-minute average (average COCheetah). These were derived from the data export file of each case provided by the Cheetah monitor. The monitor clock was synchronized to that of the McKesson system. The Cheetah export file lists CO as minute-by-minute values. The spot value was the one listed at the time of blood sampling for COFick, and the average COCheetah was the average of the two preceding minute values and the spot value (a 3 minute consecutive average). COFick and AVAFick were considered the methods of reference for the purposes of this study. AVAEcho was obtained from the subjects’ referral echocardiograms. TTE-derived AVA (AVAEcho) was calculated using the continuity equation from 2D images. None of the patients had 3D TTE, or any form of transoesophageal echocardiograms. No subject had been given any sedative during their echocardiography exam.

Calculationswhere SEP = Systolic Ejection Period in seconds per minute obtained at the time of the crossing of the aortic valve; the empiric constant in the formula for a tricuspid aortic valve is 44.3, and for a bicuspid valve, 37.7; CO = cardiac output in liters/minute; AVA = aortic valve area.where CO = cardiac output, VO2 = O2 consumption, CaO2 = arterial oxygen content, CvO2 = venous oxygen content. VO2 was estimated per cardiac catheterization laboratory standard operating procedure as 125 millilitres O2 × Body Surface Area (BSA), CaO2 = 1.36 × Hgb [mg/dl] × SaO2, CvO2 = 1.36 × Hgb [mg/dl] × SvO2. SaO2 = arterial oxygen saturation, SvO2 = mixed venous oxygen saturation (from proximal pulmonary artery), Hgb = haemoglobin.

Statistical methods

Accuracy of AVACheetah and COCheetah was assessed using Bland and Altman analysis for bias and 95% LOA[4,20]. No power calculation was performed because the study was exploratory, the Bland and Altman method is descriptive and is not a statistical test, and most importantly, there was no historical data for AVA measured with the aid of a Cheetah monitor. To set the bias and LOA of AVACheetah in perspective, these parameters were also calculated for AVAEcho. Limits of agreement are presented as ±tα, n−1*SD, where tα is the t-value corresponding to n−1 degrees of freedom at an α error of 0.05, n is the sample size and SD = standard deviation. Regression was used to check for the presence of proportional bias. This was done not only for the mean differences (biases) between AVACheetah, AVAEcho, COCheetah and their respective Fick-derived comparators, but also for the biases of COCheetah against BSA, because BSA is the sole other variable in the formula for estimated VO2. Normal quantile plots of the bias (mean difference) of CO and AVA were used to visually verify that the biases were normally distributed. The mean error percent (percent LOA) was derived as the LOA divided by the mean AVAFick or mean COFick, respectively. Finally, 95% confidence intervals (CI) were calculated using the formulas proposed in the original Bland and Altman paper:where tα is the t-value corresponding to n−1 degrees of freedom at an α error of 0.05, n is the sample size and SD = standard deviation.

Equivalence between the methods of calculating aortic valve area would be present if the mean bias (difference) was on the order of 0.1 cm2, since such a difference would unlikely have an undesirable diagnostic implication (i.e., underestimating the degree of stenosis and deferring intervention). For the same reason, the limits of agreement and their corresponding 95% confidence intervals would have to be narrow. No consensus as to how narrow exists, since replacing right heart catheterization with a non-invasive cardiac output measurement has not been attempted in a clinical setting before. We found it reasonable to accept absolute limits of agreement of no more than 0.2 cm2 if the methods were to be accepted as clinically equivalent.

The Institutional Review Board (IRB) of our hospital approved the study (IRB #16–004EX). All procedures performed in this study were in accordance with the ethical standards of the local IRB and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.