The fallacy of indexed effective orifice area charts to predict prosthesis-patient mismatch after prosthesis implantation.

AIMS: Indexed effective orifice area (EOAi) charts are used to determine the likelihood of prosthesis-patient mismatch (PPM) after aortic valve replacement (AVR). The aim of this study is to validate whether these EOAi charts, based on echocardiographic normal reference values, can accurately predict PPM. METHODS AND
RESULTS: In the PERIcardial SurGical AOrtic Valve ReplacemeNt (PERIGON) Pivotal Trial, 986 patients with aortic valve stenosis/regurgitation underwent AVR with an Avalus valve. Patients were randomly split (50:50) into training and test sets. The mean measured EOAs for each valve size from the training set were used to create an Avalus EOAi chart. This chart was subsequently used to predict PPM in the test set and measures of diagnostic accuracy (sensitivity, specificity, and negative and positive predictive value) were assessed. PPM was defined by an EOAi ≤0.85 cm2/m2, and severe PPM was defined as EOAi ≤0.65 cm2/m2. The reference values obtained from the training set ranged from 1.27 cm2 for size 19 mm up to 1.81 cm2 for size 27 mm. The test set had an incidence of 66% of PPM and 24% of severe PPM. The EOAi chart inaccurately predicted PPM in 30% of patients and severe PPM in 22% of patients. For the prediction of PPM, the sensitivity was 87% and the specificity 37%. For the prediction of severe PPM, the sensitivity was 13% and the specificity 98%.
CONCLUSION: The use of echocardiographic normal reference values for EOAi charts to predict PPM is unreliable due to the large proportion of misclassifications.

Introduction

Prosthesis–patient mismatch (PPM) occurs when the effective orifice area (EOA) of a prosthetic heart valve is too small in relation to a patient’s body size, thus resulting in high-residual post-operative pressure gradients across the prosthesis., To classify patients with PPM, a cut-off value of indexed effective orifice area (EOAi), formulated by EOA divided by body surface area (BSA), is used. Although the association between PPM and mortality was not always found in previous studies,, the consensus is that PPM negatively affects survival after surgical and transcatheter aortic valve replacement (SAVR and TAVR). Therefore, prevention of PPM is important for surgeons and cardiologists.

EOAi charts have been developed to aid decisions on choice of specific valve type and size. These charts represent the projected EOAi for specific patient BSAs, for each valve size and type, based on either in vitro or in vivo reference EOAs. While tables on these echocardiographic normal reference values of different surgical and transcatheter prosthetic valves are reported across the literature and are reported in clinical guidelines, there is little evidence to support the use of these normal reference values in EOAi charts to predict PPM. Therefore, the hypothesis was tested whether projected PPM, derived from EOAi charts, accurately predicts measured PPM.

Methods

Study and patients

The PERIcardial SurGical AOrtic Valve ReplacemeNt (PERIGON) Pivotal Trial is a prospective, non-randomized, international, multicentre trial designed to evaluate the safety and effectiveness of the novel bovine stented Avalus aortic valve bioprosthesis (ClinicalTrials.gov: NCT02088554). The methods and primary outcomes were previously published., In short, patients with symptomatic moderate or severe aortic stenosis or severe aortic regurgitation were enrolled to undergo SAVR with the Avalus bioprosthesis (Medtronic, MN, USA). The institutional review board of each centre approved the protocol and written informed consent was obtained from all patients. Selection of the valve size was performed with the use of a sizer probe; the size that corresponded with the largest fitting replica was implanted. Transthoracic echocardiography (TTE) was performed at baseline, discharge, between 3 and 6 months and annually for 5 years. All echocardiographic images were analysed at a single core lab (Cardiovascular Core Laboratories, MedStar Health Research Institute, Hyattsville, MD, USA). For this analysis, only patients who underwent TTE at the first follow-up visit were included, this visit took place between 3 and 6 months after implantation. As only a limited number of patients received a size 17 mm or 29 mm prosthesis, these sizes were excluded from further analysis.

Echocardiographic measurements

Measured EOA was derived with the continuity equation; the stroke volume, measured at the level of the left ventricular (LV) outflow tract, was divided by the velocity time integral across the prosthetic valve. Measured EOAi was determined as measured EOA divided by BSA, which was derived according to the Dubois formula at the time of the TTE: BSA (m2) = weight (kg)0.425 × height (cm)0.725 × 0.007184. PPM was defined by an EOAi ≤0.85 cm2/m2; moderate PPM by an EOAi of 0.66–0.85 cm2/m2, and severe PPM by an EOAi ≤0.65 cm2/m2.,

Training and test sets

For each valve size, patients were randomly split (50:50) into a training set and a test set. To reflect the distribution of valve sizes observed in the PERIGON trial, stratified sampling techniques were used in the data splitting. To assess the variability in measured EOA values in the training dataset, the distribution of measured EOA was plotted for each size. For both groups, the mean measured EOA and the incidence of PPM for each valve size were determined. In accordance with the current method to determine normal reference EOA values, the mean measured EOA for each valve size from the training dataset was used to construct an EOAi chart. This EOAi chart was subsequently used to calculate the projected EOAi for each patient in the test set, by dividing the reference EOA of the implanted valve size with the BSA of the patient. If this projected EOAi was below 0.85 cm2/m2, the patient was classified as having projected/expected PPM.

Endpoints

The measured EOAi was plotted against the projected EOAi for each patient in the test set. The inaccuracy, sensitivity, specificity, negative predictive value, and positive predictive value of the EOAi chart to predict PPM and severe PPM were calculated.

Statistical analysis

Categorical variables are summarized as number and percentage, and continuous variables are summarized as mean ± standard deviation. The χ2 test was used to compare categorical variables between groups, and the independent samples t-test was used to compare continuous variables between groups. Sensitivity, specificity, negative predictive value, and positive predictive value of the EOAi chart to predict PPM and severe PPM were expressed in percentages. The correlation between measured and projected EOAi was calculated using Pearson’s sample correlation coefficient. To address potential bias in our single split of the data, two-fold Monte Carlo cross-validation was performed. In 1000 iterations, the sensitivity, specificity, negative predictive value, and positive predictive value of the EOAi chart in the prediction of PPM and severe PPM were calculated. The combined results of all iterations were expressed as mean ± standard deviation.

To exclude low-flow status as an explanation of the misclassifications of PPM, a separate analysis was performed with only the patients with a good LV function [left ventricular ejection fraction (LVEF) > 50%] in the test dataset of the single split. All statistical analyses were performed using R version 3.4.4 (R Development Core Team, Vienna, Austria, 2018).

Results

Of the 1115 patients included in the PERIGON trial, 996 patients underwent a TTE between 3 months and 6 months after implantation. There were no missing data for BSA or EOA for all included patients, but LVEF was missing in 175 (18%) patients. Exclusion of patients with a size 17 mm valve (n = 1) and size 29 mm valve (n = 9) resulted in a cohort of 986 patients. After randomization, the training and test datasets consisted of 492 and 494 patients, respectively.

Training set

Table  displays the patient characteristics stratified by training and test datasets, confirming that the two samples were similar. In the overall training set, 67% of the patients had PPM, and 21% had severe PPM. Size 21 mm had the largest incidence of PPM (77%) and severe PPM (33%). The mean measured EOA ranged from 1.27 cm2 for size 19 mm up to 1.81 cm2 for size 27 mm (Table ). Figure  illustrates the distribution of measured EOA values for all valve sizes and stratified for each size. The difference between the minimum and maximum measured EOA was 0.91 cm2 for size 19 mm, 1.22 cm2 for size 21 mm, 1.48 cm2 for size 23 mm, 1.63 cm2 for size 25 mm, and 1.61 cm2 for size 27 mm (Table ). The mean measured EOA of each valve size in the training dataset was used to construct a new EOAi chart for the Avalus valve.

Distribution of measured EOA values for all valve sizes and stratified by size. The red dashed line represents the mean measured EOA. For each valve size, this value would be used as reference EOA in the EOAi chart.

Test set

There were some differences between the reference EOAs and the mean measured EOAs in the test set (Table ). The incidence of PPM and severe PPM was 66% and 24%, respectively. The accuracy of the EOAi chart to predict PPM is illustrated in Figure . The sensitivity was 87% to predict PPM, and the specificity was 37%. In addition, the positive predictive value was 73%, and the negative predictive value was 60%. The prediction of PPM was incorrect in 148 (30%) of the patients. In 42 (9%) patients, the projected EOAi was larger than the measured EOAi, resulting in the incorrect prediction that these patients had no PPM. The opposite occurred in 106 (21%) patients; the projected EOAi was smaller than the measured EOAi, resulting in the incorrect prediction that these patients had PPM. The prediction of PPM and no PPM was correct in 284 (57%) patients and 62 (13%) patients, respectively. The lowest accuracy was reported for the size 27 mm, as the prediction was incorrect in 42% of the patients. The EOAi chart was more accurate for the valve size 19 mm, as PPM was incorrectly predicted for only 5% of the patients (Table ). In 1000 iterations, the sensitivity to predict PPM was 85 ± 2% and the specificity was 44 ± 4%. The negative predictive value was 60 ± 4%, and the positive predictive value was 75 ± 2%. The prediction of PPM was inaccurate in 29 ± 2% of the patients.

Use of EOAi chart to predict PPM. Accuracy of EOAi chart to predict PPM (EOAi ≤0.85 cm2/m2). True PPM (light blue) = 284 patients (57%); true no PPM (dark blue) = 62 patients (13%); false PPM (red) = 106 patients (21%); false no PPM (orange) = 42 patients (9%). The red dashed lines represent the cut-off value for measured and projected PPM (EOAi ≤0.85 cm2/m2). There was a moderate correlation between projected and measured EOAi (r =0.50).

The accuracy of the EOAi chart to predict severe PPM is illustrated in Figure . The sensitivity was 13% to predict severe PPM, and the specificity was 98%. In addition, the positive predictive value was 71%, and the negative predictive value was 78%. The prediction of severe PPM was incorrect in 111 (22%) of the patients. In 105 (21%) patients, the projected EOAi was larger than the measured EOAi, resulting in the incorrect prediction that these patients had less than severe PPM. The opposite occurred in 6 (1%) patients; the projected EOAi was smaller than the measured EOAi, resulting in the incorrect prediction that these patients had severe PPM. The prediction of severe PPM and no severe PPM was correct in 15 (3%) patients and 368 (74%) patients, respectively. The lowest accuracy was reported for the size 19 mm as the prediction was incorrect in 45% of the patients. The EOAi chart was more accurate for the valve size 27 mm as severe PPM was incorrectly predicted for 9% of the patients (Table ). In 1000 iterations, the prediction of severe PPM had a sensitivity of 14 ± 3% and a specificity of 98 ± 1%. The negative and positive predictive values were respectively 79 ± 1% and 65 ± 9%. The prediction of severe PPM was inaccurate in 21 ± 1% of the patients.

Use of EOAi chart to predict severe PPM. Accuracy of EOAi chart to predict severe PPM (EOAi ≤0.65 cm2/m2). True severe PPM (light blue) = 15 patients (3%); true no severe PPM (dark blue) = 368 patients (74%); false severe PPM (red) = 6 patients (1%); false no severe PPM (orange) = 105 patients (21%). The red dashed lines represent the cut-off value for measured and projected severe PPM (EOAi ≤0.65 cm2/m2). There was a moderate correlation between projected and measured EOAi (r =0.50).

Impact of LV function

Of the 811 patients with reported LVEF, 734 (91%) patients had a good LV function (LVEF ≥ 50%). The patients with good LV function were distributed evenly in the training and test sets (50% vs. 50%). The incidence of PPM was 66% and the incidence of severe PPM 23% among patients with good LV function. The Pearson correlation between projected EOAi and measured EOAi was r = 0.53. The sensitivity, specificity, positive predictive value, and negative predictive value for the prediction of PPM were 85%, 56%, 79%, and 66%, respectively. Of the 375 patients, 56 (15%) were misclassified as having PPM and 37 (10%) were misclassified as having no PPM, resulting in a total inaccuracy of 25%. The sensitivity, specificity, positive predictive value, and negative predictive value for the prediction of severe PPM were 11%, 98%, 56%, and 79%, respectively. Nine (2%) patients were misclassified as having severe PPM and 76 (20%) were misclassified as having no severe PPM, resulting in a total inaccuracy of 22%.

Discussion

In this study, the use of an EOAi chart led to the incorrect prediction of PPM in 30% of the patients and severe PPM in 22% of the patients. Because of the weak correlation between the normal reference EOA and the actual measured EOA, projected PPM derived from an EOAi charts does not accurately reflect a prosthetic valve too small for a certain body size. For this reason, studies that analyse the effect of PPM on survival should be based only on individually measured EOA values. Even more important, our findings suggest that EOAi charts should not be used by surgeons and cardiologists to predict PPM.

To select the optimal prosthesis size during SAVR, the conventional method is to use a sizer probe with a barrel that can be passed through the annulus on one end, and a valve replica on the other. This approach should lead to the implantation of a valve with the largest opening that is allowed by the native annulus of the patient. However, this method is considered inappropriate to prevent PPM in all patients, as some patients with a relatively small annulus may need a different type of valve with a larger opening or a larger valve implanted after annular enlargement. To make this clinical judgement, EOAi charts were introduced to predict PPM pre- or peri-operatively., These charts are based on reference EOAs, obtained from either in vivo or in vitro studies. In this way, for each patient, a projected EOAi can be calculated.

Our study, however, shows that the correlation between these projected EOAi and actually measured EOAi values, was weak (r = 0.50), with a sensitivity of 87% and specificity of 37% to predict PPM (Figure ). This poor specificity calls into question the legitimacy of the use of EOAi charts. Although echocardiographic normal reference values of surgical and transcatheter prosthetic valves are important to test the overall haemodynamic performance of prostheses, the projected EOAi does not correctly indicate that an annular enlargement or different prosthesis is required. In the current study, 63% of patients with no PPM were classified as projected PPM and were at risk of receiving an unnecessary annular enlargement. The opposite occurred for patients with a false-negative prediction. For these patients, the measured EOAi was underestimated by the projected EOAi, resulting in the incorrect prediction of a lesser degree of PPM, which was observed in 13% of patients with PPM and in 87% of patients with severe PPM. Thus, the EOAi chart would provide a false belief that an adequate prosthesis size is being implanted to prevent (severe) PPM.

Other studies also discussed the accuracy of EOAi charts to predict PPM. While these studies have identified limitations of EOAi charts for the prediction of PPM,, none of these studies have used a large single uniform cohort of patients, implanted with a single type of prosthesis and an echocardiogram evaluated by an independent core lab. As part of the error in predicting PPM is likely related to measurement error of EOA in normal reference value charts, the use of a single assessor reduces this error and increases the accuracy of the correlation between projected and measured EOA. Indeed, Bleiziffer et al. demonstrated that the correlation between projected EOAi and measured EOAi varied strongly between EOAi charts based on institutional data (r = 0.62), manufacturers data (r = 0.43), geometric orifice area (r = 0.27), and literature data (r = 0.53). Based on these studies, it was suggested to construct EOAi charts with in vivo data from the prosthesis’ pre-market approval trials , such as the PERIGON Pivotal Trial. Nevertheless, we demonstrate that despite using in vivo data from a single echocardiographic core lab, EOAi charts fail to accurately predict PPM. This finding has important clinical implications, as the current guidelines favour transcatheter aortic valve replacement (TAVR) over SAVR in case of projected/expected PPM. While we do not debate whether transcatheter valves have superior haemodynamic performance compared to surgical valves, the substantial percentage of misclassifications, in the present study, suggest that projected PPM should not be a primary consideration in the decision between SAVR and TAVR.

Our study has furthermore identified that the proportion of misclassifications of PPM and severe PPM varied between valve sizes (Table ). For increasing size, the risk of misclassification of PPM also rose; 42% of patients with size 27 mm were incorrectly classified, compared to 5% of patients with size 19 mm. This is due to the fact that the mean measured EOAi of size 27 mm (0.93 cm2/m2) is closer to the cut-off of PPM (≤0.85 cm2/m2), compared to the mean EOAi of size 19 mm (0.67 cm2/m2). Thus, a smaller difference between measured and projected EOAi is needed to cause misclassification of PPM for size 27 mm, compared to size 19 mm. The opposite occurs with the prediction of severe PPM, as the mean EOAi of size 19 mm is closer to the cut-off of severe PPM (≤0.65 cm2/m2); for size 19 mm, 45% of patients were misclassified as having severe PPM, compared to 9% of patients with size 27 mm. This demonstrates the limited applicability of the use of EOAi charts.

While this study considered measured EOAi as the gold standard, other authors have argued that projected EOAi should be used instead of measured EOAi for the classification of PPM. However, the correlation between projected EOAi and post-operative gradient is less than the correlation between measured EOAi and post-operative gradient, which ultimately forms the theoretical framework to use EOAi as a surrogate for relative valve size. In addition, when our analysis was limited to patients with good LV function, there was an almost identical rate of false-positive and -negative predictions of PPM and severe PPM. This reduces the likelihood that misclassification was due to insufficient flow to facilitate complete opening of the prosthesis.

The fact that so many patients are misclassified as having PPM, based on reference EOA values, is also relevant for the interpretation of studies that examined the effect of PPM on survival. The majority of studies use reference EOAs derived from the literature to calculate projected EOAi values and determine the presence of PPM., In 30% of the patients in our cohort, the difference between the reference and measured EOA was enough to result in misclassification of PPM. Therefore, the impact on long-term outcomes of PPM based on projected EOA, or the lack of an impact, may be confounded by misclassification bias. This bias is not present when using measured EOA to define PPM. Indeed, two meta-analyses found a higher impact of PPM on perioperative and long-term mortality when measured EOA was used instead of reference in vivo EOA., For this reason, we recommend using measured EOA values from each individual patient to study the effect of PPM on clinical outcomes. As a side note,

he use of measured PPM as a surrogate marker for a too small prosthesis is also questionable, however, this is outside the scope of the current study. Future studies should consider whether the relation between EOAi and transprosthetic gradients is sufficient to support the hypothesis that measured PPM reflects a pathologic degree ofhaemodynamic obstruction . Furthermore, the use of cut-off values to split EOAi, a continuous variable, in different groups of PPM, is debatable.

This study was conducted among patients who underwent SAVR, but the results are also applicable to patients who undergo TAVR. Although no EOAi charts are yet available for the, supposedly, appropriate selection of transcatheter valve and size, normal reference value tables that would allow this have been established. As these tables with transcatheter valves reported wide distributions of EOA similar to those observed in the current study with surgical valves, we argue against the publication and use of EOAi charts to select the prosthesis size in TAVR.

Limitations

A limitation of this trial is the small number of very large or small valves that were implanted. For this reason, valve sizes 17 mm and 29 mm were excluded from this analysis. Furthermore, the measures of diagnostic accuracy found for the sizes 19 mm and 27 mm have a limited precision. Another limitation of this trial is the missing values of LVEF in 18% of the patients. While this is a substantial proportion, we do not think that these missing data are relevant to the outcomes of the present study, as a similar rate of misclassifications was found in patients with a reported LVEF >50% in comparison to the overall cohort. Moreover, patients with missing LVEF values showed non-significant differences in other haemodynamic parameters at the same visit.

Conclusions

The use of EOAi charts to predict PPM is unreliable, as it inaccurately predicts PPM in 30% of the patients. We recommended the use of measured instead of projected PPM to study the impact of PPM on outcomes.

Funding

The PERIGON Pivotal Trial was funded by Medtronic.

Conflict of interest: M.D.V. reports grants from Medtronic, during the conduct of the study. R.A.F.D.L.V.W. has no conflict of interest to declare. A.-P.K., S.J.H., and G.L.H. are employees of Medtronic. V.R. reports personal fees from Medtronic, during the conduct of the study; personal fees from Abbott Labs, outside the submitted work. N.J.W. reports grants from Medtronic, during the conduct of the study; grants from Abbott Vascular, Boston Scientific, Edwards Lifesciences, LivaNova, and St. Jude, outside the submitted work. M.G.M. reports grants from Medtronic, Edwards, and Abbott, outside the submitted work. M.J.R. reports other from Medtronic (fees for educational services, paid to his department), during the conduct of the study; other from Boston Scientific (fees for educational services, paid to his department), outside the submitted work. J.F.S. reports personal fees from Medtronic; grants from Edwards and Abbott, outside the submitted work. R.J.M.K. reports other from Medtronic, LivaNova, and Edwards, outside the submitted work.

Declaration of Helsinki

The authors state that this study complies with the Declaration of Helsinki, that the locally appointed ethics committees have approved the research protocol and that informed consent has been obtained from the subjects (or their legally authorized representative).

Table 1

Patient characteristics of training vs. test dataset

Patient characteristics	Training dataset (n = 492)	Test dataset (n = 494)	P-value
Baseline
Age (years)	70 ± 9	70 ± 9	0.79
Male	364 (74%)	378 (77%)	0.40
Body surface area (m2)	1.99 ± 0.2	1.99 ± 0.2	0.84
BMI (kg/m²)	29 ± 5	29 ± 5	0.58
STS risk of mortality (%)	1.98 ± 1.4	1.87 ± 1.3	0.17
Diabetes	133 (27%)	126 (26%)	0.64
Hypertension	375 (76%)	367 (74%)	0.53
Peripheral vascular disease	29 (6%)	37 (7%)	0.38
Chronic obstructive lung disease	67 (14%)	49 (10%)	0.09
Left ventricular hypertrophy	204 (41%)	201 (41%)	0.86
Left ventricular ejection fraction (%)	59 ± 9	60 ± 9	0.35
Cardiac output (L/min)	5.2 ± 1.4	5.2 ± 1.4	0.72
Mean aortic gradient (mmHg)	42 ± 18	42 ± 17	0.75
Indexed AVA (cm2/m2)	0.45 ± 0.2	0.46 ± 0.3	0.64
Previous aortic valve implanted	5 (1%)	3 (1%)	0.72
Procedure
Annulus diameter (mm)	23.7 ± 2.1	23.7 ± 2.1	0.79
Isolated AVR	244 (50%)	243 (49%)	0.95
Ascending aorta replacement	32 (7%)	37 (7%)	0.63
Pledget-reinforced sutures	289 (59%)	280 (57%)	0.56
Post-implant mean gradient by TOE (mmHg)	9 ± 5	9 ± 5	0.13

Categorical variables expressed as count (%). Continuous variables expressed as mean ± standard deviation.

AVA, aortic valve area; AVR, aortic valve replacement; BMI, body mass index; STS, Society of Thoracic Surgeons; TOE, transoesophagheal echo.

Table 2

EOA and PPM for size 19–27 mm in the training dataset (n = 492)

Valve size	n	Mean measured EOA ± SD (cm2)	Min EOA (cm2)	Max EOA (cm2)	Mean measured EOAi ± SD (cm2/m2)	Incidence of true PPM (%)	Incidence of true severe PPM (%)
19	20	1.27 ± 0.3	0.86	1.77	0.76 ± 0.2	70	35
21	90	1.34 ± 0.3	0.82	2.04	0.74 ± 0.2	77	33
23	178	1.51 ± 0.3	0.92	2.40	0.78 ± 0.2	69	23
25	159	1.66 ± 0.3	0.93	2.56	0.81 ± 0.2	63	14
27	45	1.81 ± 0.3	1.08	2.69	0.86 ± 0.2	51	9

EOA, effective orifice area; EOAi, indexed effective orifice area; max, maximum; min, minimum; PPM, prosthesis–patient mismatch.

Table 3

EOA and PPM for size 19–27 mm in the test dataset (n = 494)

Valve size	n	Reference EOA (cm2)	Mean reference EOAi ± SD (cm2/m2)	Mean measured EOA ± SD (cm2)	Mean measured EOAi ± SD (cm2/m2)	Incidence of true PPM (%)	Incidence of true severe PPM (%)	Inaccuracy of projected PPM (%)	Inaccuracy of projected severe PPM (%)
19	20	1.27	0.73 ± 0.1	1.16 ± 0.3	0.67 ± 0.2	95	40	5	45
21	91	1.34	0.74 ± 0.1	1.26 ± 0.2	0.70 ± 0.2	84	42	8	34
23	178	1.51	0.77 ± 0.1	1.56 ± 0.3	0.80 ± 0.2	65	22	33	21
25	160	1.66	0.81 ± 0.1	1.66 ± 0.3	0.82 ± 0.2	60	19	34	19
27	45	1.81	0.86 ± 0.1	1.96 ± 0.5	0.93 ± 0.2	42	9	42	9

For each size, the projected EOA in the test set is the mean measured EOA from the training set.

EOA, effective orifice area; EOAi, indexed effective orifice area; PPM, prosthesis–patient mismatch.